Query Processing Order
In A-Parser A-Parser there are many functions and features, this diagram shows the order of processing a request from reading it from a file (or text) to saving the final result to a file
Scheme of request processing order
Notes
- When filtering and making results unique, the request and its results are canceled entirely if simple result, is used as a comparison; if an array is used in the comparison, elements are removed from that array
- Many steps in the diagram are optional and depend on the settings specified in the Job Editor.
- Additional requests may appear when using the options Parse all result and Parse to level. All additional requests have the next level relative to the request from which the additional requests were created; level counting starts from zero, i.e., the initial requests from a file or text always have level 0. Requests after applying substitutions also have level 0
Failed requests
A request is considered failed and is skipped if it could not be executed within the specified number of attempts.
How to determine why a request failed? Enable logging or run a Job Test. All errors are logged. By studying the log, you can understand what went wrong.
Example of a failed request. Logs suggest that the request failed due to a captcha, and the attempts ran out. In this case, connecting a captcha solving service or increasing the number of attempts might help (only if you are scraping with a proxy, otherwise increasing attempts is useless).
How to increase the number of attempts? You need to override the option Request retries and set a higher value.