Complete description of Tasks queue

Dec 25, 2015
  • In A-Parser exists task queue, allowing to perform sequentially a set of in advance created tasks, is recommended will examine with concept of work Task queue.

    Queue control(top)

    [​IMG]
    • Number of the tasks displayed on one page is regulated. This parameter is specified in Global settings (Settings -> Global settings -> Tasks per page)
    • Opportunity to displace tasks, move tasks up or down, move tasks directly to the beginning or to the end of queue
    • Tasks can be edited after a stop or a pause. It is possibly to edit quantity of threads, setup of results and additional options
    • Also task can be deleted, or to duplicate

    Complete overview of Card task(top)

    [​IMG]
    Settings - Settings contain summary of the task.
    • Parsers - It is displayed what parsers are used in task
    • Add time - The date and time of adding of task is specified
    • Queries - Information on that from where parser takes requests, from the file or from a field of requests
    • Config preset - Selected preset of threads for the current task
    • Results file - File name for saving result. By default set up on a format of date %b-%d_%H-%M-%S, when formatting gives out the current time and date in look May-08_20-08-38
    State - Statistics in which displayed:
    • Changed time - Lately updates of information of the task
    • Queries done/all - Amount of requests processed/Quantity requests only. Progress is displayed in a type of percent of the executed requests
    • Failed queries - Amount of requests which the parser didn't process, are considered as unsuccessful requests
    • Results unique/all - If the unique is turned, parser will display statistics of unique results and total quantity of results
    • Status - A current status of task, description of all statuses are tabulated below
    • Active threads - Displays quantity of the threads participating at present when parsing. Initially corresponds to number of the threads specified in case of compilation of the task. Maybe will decrease in case of shortage live proxies, or at the end of work task, when the amount of the remained requests is less than quantity of threads specified in case of setup in task
    • Wait proxy threads - Displays quantity of idle threads waiting for live proxies. If in task there are expecting threads, it is necessary to add working proxy servers
    • Speed cur/avg - Speed of work of task, expresses in number of the executed requests in a minute. The current speed displays amount of the requests executed in the last minute, and the average speed displays mean value for the entire period of work of task
    • Current query - Current query which parser processes

    • To each task is displayed
      additional information on the requests, used proxies, etc.
    • In case of switching on option Do log in Task editor, View log for the specific task is possible. Log on each thread is carried separately

    Task status(top)

    Each task has the status, here the list of possible statuses:
    StatusDescription
    waitSlotWaiting of the free slot for execution of task, the new task always gets to queue with this status
    workTask is performed
    pausedTask is paused, thus it retains the working slot, without allowing to be launched by the following tasks
    stoppedTask is stopped, thus the working slot is released
    completedTask is finished - all requests are processed, the task passes into queue of complete task, the working slot is released


    Failed queries(top)


    • Parser considers request unsuccessful (failed) when the number of attempts for parsing is settled. Parser passes this request writing it as not successful. The number of attempts for parsing of request is specified in Settings of parser in the field Request retries.
    • To prevent unsuccessful requests, it is necessary to increase number of attempts. The more number of attempts, the will be possible more slowly parsing. Nevertheless, that is guaranteed to receive results of parsing for each request, it is possible to set a large number of attempts, for example 1000.