1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
  2. Join our Telegram chat: https://t.me/a_parser_en
    Dismiss Notice

Tasks and queue

Sep 2, 2015
  • Work of A-Parser based on an tasks queue - you are adding a tasks and they are executed one after another, allowing to load parser on long time and do not watch when it finish the job.

    Basic advantages of tasks queue in A-Parser:
    • Support for multiple tasks at the same time
    • Pause or full stop of the task processing
    • Saving intermediate results of work, that allows you to continue the work from moment when the parser off (or turn off the computer\server)
    • Moving individual tasks in a queue
    • The ability to edit the completing tasks, and also copying tasks for the repeated use
    • The completed tasks pass to the completed tasks queue, where you can watch the results of each task and download the result
    Algorithm of work


    Quantity of at the same time working tasks is regulated by working slots, this parameter is specified in Global settings (Settings -> Global settings -> Max active tasks). Standartly set value 1, i.e. one at the same time operating task, others will expect the queue

    Each task has the status, here the list of possible statuses:
    waitSlotWaiting of free slot for execution of task, the new task always gets to queue with this status
    workTask is performed
    pausedTask is paused, thus it retains the working slot, without allowing to be launched by the following tasks
    stoppedTask is stopped, thus the working slot is released
    completedTask is complete - all requests are processed, the task passes into queue of complete tasks, the working slot is released
    In case of change value of the Maximum active tasks parameter over one should be considered that the quantity of threads will increase, as well as the general consuming of resources a parser. For achievement of maximum speed of operation always use only one task, with the fitted optimum number of threads for the specific computer\server..
samcram, Dellius and high_skill like this.