Skip to main content

Debugging Tasks

Testing Templates​

Learn more about Templates Testing

Task Testing​

The task tester allows testing the entire preset completely, including multiple scrapers and Constructors. The entire work process can be monitored thanks to the display of logs for each thread and viewing the results. All data is updated in real-time.

Test Task in the Task Editor

The following information is available in the task tester interface:

  • Requests completed/total
    • Information about requests, the number of completed requests, the total number of requests
    • Maximum number of threads - 5
    • Maximum number of requests - 10
  • Current request
  • Number of failed requests
  • Choice of request limit
  • Log of each of the 5 threads, where you can find complete information about requests and errors
  • Parsing result field
    • If multiple result files are used - the contents of each file are displayed
Task Tester

Test Parsing​

In the Test Parsing tab, there is an opportunity for test parsing to create and debug your own presets. Unlike the Task Editor, here it is not possible to use constructors for queries and results, to use multiple scrapers, macros are not supported, there is no multi-level parsing and other features that are supported in the task editor.

This section is created on the principle of a unit test, in which you can test individual scrapers. The entire work process is displayed in the form of a log, which allows obtaining detailed information about the scraper's (preset's) work, starting from sending a request to receiving results:

Test parsing results

In "Test Parsing" there is a quick settings area: disabling auto-scroll and enabling debug mode

By default, "auto-scroll" is enabled - this means that as information is output to the log, we will automatically move down


When "auto-scroll" is turned off, the icon changes its appearance - this means that as information is output to the log, we will stay in place, there will be no automatic movement down


By default, debug mode is disabled - this means that the log will display general information about scraping.


When we activate debug mode, the icon changes its appearance and the log will display detailed information about scraping.

In the log window with debug mode enabled, we see detailed information about scraping:

Test scraping in debug mode

In the request part, we can see:

  • The name of the request
  • The request string that is sent to the server
  • The list of parameters that are sent along with the request (headers, encoding, filters, etc.)

In the response part, which we receive from the server, we see the following information:

  • See in debug window - a link to the window where the page obtained as a result of the request is displayed

    See in debug window

  • See full data - a link to the window with the source code obtained as a result of the request

    See full data

  • Go to Regex Builder - a link to the Regex Builder with the source code obtained as a result of the request

If the "auto-scroll" mode is created more for the convenience of viewing, then the "debug" mode is an indispensable tool for debugging your own scrapers and presets. By analyzing the information obtained from the log in "debug" mode, we can see at what stage and what happens during the execution of the task, how the request is formed and what the scraper receives in response.

Last updated