1.1.481 - a lot of speed optimization, the ability to use the presets in the chains and scheduler

Discussion in 'News' started by Support, Apr 7, 2016.

  1. Support

    Support Administrator
    Staff Member A-Parser Enterprise

    Joined:
    Mar 16, 2012
    Messages:
    4,545
    Likes Received:
    2,163
    [​IMG]

    In this version was implemented a number of qualitative improvements, designed to increase parsing speed:
    • Fast processing of UTF-8, a speed boost in some tasks up to 1.5x
    • Implement a mechanism of analysis the variables used in the results, which allows to optimize the speed, extracting only the required data. The first optimized parser has become SE::Google SE::Google: two-time increase of limit speed of parsing, when collecting only links (7000-10000 requests per minute, 100 results per query, in other words, now you can get 1 million links for 1 - 1.5 minutes)
    • Now information about queries $response is generated only on demand
    • Optimized work of template engine when using multiple parsers in the same task
    Other enhancements:
    • Now when you use the option "Run on complete" is used the preset name instead of the task number from queue
    • In the Task Scheduler is also used preset name, that allows you to clean queue without the risk of losing the saved tasks for the scheduler
    Corrections due to changes in the output of services:
    Fixes:
    • In the parser Rank::Archive Rank::Archive fixed a logic of the processing of error 403 Forbidden
    • Fixed crash when changing the number of threads in the task
    • Fixed TLS support for some sites
    • Fixed freezing of task when crashing worker process
    • Now the task can be stopped if it is waiting slot
    • Fixed option "Save window size"
    • Fixed error in HTML::LinkExtractor HTML::LinkExtractor when working Parse to level option in cases, when the site gave a redirect to another page
     

Share This Page