Debugging tasks
Testing templates
Task testing
Task tester allows you to test the entire preset, including multiple scrapers and Constructors. The entire process can be monitored thanks to the display of logs for each thread and viewing the results. All data is updated in real time.
The following information is available in the task tester interface:
- Requests completed/total
- Information about requests, number of completed, total number of requests
- Maximum number of threads - 5
- Maximum number of requests - 10
- Current request
- Number of unsuccessful requests
- Selection of request limit
- Log of each of the 5 threads, in which you can find complete information about requests and errors
- Parsing result field
- If multiple result files are used, the contents of each file are displayed
Test parsing
In A-Parser on the Test parsing tab, there is an opportunity for test parsing to create and debug the operation of your own presets.
Unlike the Task editor, here it is not possible to use query and result constructors, use multiple scrapers, macros are not supported, there is no multi-level parsing and other features that are supported in the task editor.
The section is created on the principle of a modular test, in which you can test individual scrapers. The entire process of work is displayed in the form of a log, which allows you to get detailed information about the scraper (preset) operation, starting from sending the request and ending with receiving the results:
Creating a test preset
- Scraper selection
- Query field
- Scraper settings
by default, the "default" preset is selected, it is possible to select a previously saved preset from the list
additional settings - a list of additional settings identical to those placed in the
Task editor. Allow you to use filters, uniqueness, override preset settings, etc.
Quick settings area
In "Test parsing" there is a quick settings area: disabling autoscroll and enabling debug mode
- By default, "autoscroll" is enabled - this means that as information is output to the log, we will automatically move down
- When "autoscroll" is turned off, the icon changes its appearance - this means that as information is output to the log, we will remain in place, there will be no automatic scrolling down
- By default, debug mode is disabled - this means that general information about parsing is output to the log.
- When we activate debug mode, the icon changes its appearance and detailed information about parsing will be output to the log.
In the log window, we see detailed information about parsing:
In the request part, we can see:
- Request name
- The query string that is sent to the server
- A list of parameters that are sent along with the request (headers, encoding, filters, etc.)
In the response part, which we receive from the server, we see the following information:
- "See in debug window" - a link that, when opened, displays a window showing the page received as a result of the request
- "See full data" - a link that, when opened, displays a window with the source code received as a result of the request
- Test parsing results, in the format specified by the Result format setting
If the "autoscroll" mode is created more for viewing convenience, then the "debug" mode is an indispensable tool for debugging your own scrapers and presets. Analyzing the information obtained from the log in "debug" mode, we can see at what stage and what happens during the task execution, how the request is formed and what the scraper receives in response.