Proxy Checkers
This section displays the performance statistics of all proxy checkers. Each proxy checker is a continuously running module (if enabled) that checks proxies and thus maintains an up-to-date list of alive proxies.
You can add an unlimited number of proxy checkers and select one or more of them for each task or even each parser within a task. This allows using one set of proxies for parsing Google and completely different ones for Yandex within a single task.
The total number of alive proxies and the number of launched (running) proxy checkers are displayed at the top. The button to add a new proxy checker is located at the top right. More details on the procedure for adding proxy checkers are described in the Proxy settings section.
Below is a list of all existing proxy checkers in the form of cards with information about each one. Each card displays the following information:
- Working path - folder with files of the proxy checker in
aparser/files/proxy - Update time - the time of the last check of the loaded proxy list
- Number of proxies in the check queue and the total number of loaded proxies
- Number of alive proxies
- Load state or next loading date from proxy sources
- Number of sources from which proxies were last successfully loaded and the total number of sources in this proxy checker
- Current proxy check status
The Enabled checkbox next to the proxy checker control buttons allows you to enable/disable the proxy checker.
The default proxy checker always comes first in the list. It serves as a template for new proxy checkers and cannot be edited or deleted.
File structure
The working files of the proxy checker are located in the folder files/proxy/<proxy checker name>:
proxy.txt- proxies are loaded from this file; you should place your proxy list heresites.txt- you should place the list of proxy sources in this file (links to proxies, one link per line)alive.txt- alive proxies are saved to this file every 5 seconds if the corresponding option is enabledregex.txt- this file contains a list of regular expressions for parsing proxies from external sources (one regular expression per line, $1 should be the IP address, $2 - the port)
If you have links to proxy sources - specify them in the sites.txt file, the proxy.txt file should be left empty
For the "default" proxy checker, the files are located in the root of the files/proxy/ directory
Adding and configuring a proxy checker
Go to the "Proxy Checker" menu and click "Add checker" or select "Edit" from the dropdown menu of an existing proxy checker. You will be taken to the proxy checker settings page.
If necessary, set the required number of threads for checking proxies (Check threads), select the proxy type (Proxy type), and change other settings. Default parameter values are suitable for most tasks. Save the settings as a new proxy checker. You cannot change and save the settings of the default proxy checker.
Proxy sources are specified in files inside the folder named after the created proxy checker (files/proxy/.../):
- links in sites.txt
- proxy list in proxy.txt
Proxies with IP access
Proxies with access by IP are configured in a similar way.
Proxy list with the same login and password for all proxies
This method is suitable for cases where the proxy list has the format ip:port and the login/password is the same for the entire proxy list.
In the checker settings, specify:
- login
- password
- Use proxy authorization
Proxy list with different passwords for each proxy
In this case, the proxy list must have the format login:password@ip:port; in the checker settings, it is sufficient to specify Use proxy authorization.
⏩ Video: connecting proxies with authorization
Choosing a proxy checker for a task
These settings are necessary to differentiate task operations with various proxy checkers; you can skip this section if you need to use all available proxies in all tasks.
Go to the Settings -> Threads settings menu, select the desired preset or create a new one (Add new button).
In the Proxy Checkers field, select one or more proxy checkers (proxy checkers must be enabled to be used) and save (Save). You can also select all proxy checkers at once by choosing All (default value).
Now you can use the created Threads Config with the specified proxies in your tasks by selecting it in the Task Editor.
You can also override the proxy checker in each parser using the override function - Proxy Checker.
The Exclude from "All" option in the proxy checker settings allows you to exclude its proxies from general use in A-Parser. This option is useful when you need to make certain proxies available only for specific tasks or only for specific parsers:
- for a task, you must explicitly select the excluded proxy checker
- for a specific parser, you must set the use of the excluded proxy checker in the settings
Changes in logic
Previously, if a specific proxy checker was selected in a task and a different proxy checker was specified in the parser, the parser would wait for proxies. Now, the settings of the specific parser have higher priority:
- "All" - uses all proxies selected for the task
- specific proxy checker - uses it, even if it is not selected in the task
Proxy checker parameters
| Parameter name | Default value | Description |
|---|---|---|
| Loading type | Replace | Determines whether to keep previously loaded proxies or not: Add - always adds new proxies to the general list, Replace - replaces old proxies with new loaded ones |
| Load threads count | 5 | Number of threads for loading proxies from sites |
| Load interval | 30 | Interval between full re-checks of the site list |
| Load timeout | 30 | Timeout for a request to a site with proxies |
| Load max size | 524288 | Maximum size of a page with proxies; if the page is larger, it is trimmed to the specified size |
| Load limit count | 0 | Limit on the number of loaded proxies, 0 to disable |
| No check proxies | ☐ | Allows disabling proxy checking. All loaded proxies are automatically considered alive |
| Proxies type | HTTP, SOCKS5 | Selection of which proxy types to check and in what sequence; if both HTTP and SOCKS are specified, then upon a failed HTTP check, the proxy will be re-checked for the SOCKS protocol |
| Check threads | 15 | Number of threads for checking proxies |
| Check url | http://work.a-poster.info:25000/ | Link to the proxy check script; currently, checking is performed via the parser server, this behavior may change in the future |
| Check interval | 30 | Interval between full re-checks of all proxies |
| Check timeout | 5 | Proxy timeout |
| Check max size | 5120 | Maximum size of the downloaded page during proxy checking |
| Check anonymous | ☐ | Check proxies for anonymity; if selected, External IP must be specified |
| External IP | External IP address of the computer\server; must be specified if the Check anonymous option is enabled | |
| Exclude from "All" | ☐ | By default, "All" is selected as the proxy checker in each parser, meaning all available proxy checkers are used. If the option is enabled, the proxy checker will be excluded from All. |
| Save alive proxies to file | No | Save alive proxies to the file files/proxy/alive.txt |
| Use proxy authorization | ☐ | Use authorization for proxies by login\password |
| Authorization login | Login for authorization | |
| Authorization password | Password for authorization |
Installing the check script on hosting
By default, A-Parser checks proxies through its own check script, without the need to install a script on your hosting
Upload the following PHP script to your hosting or server and specify the link to it in Check url:
<?php
print_r($_SERVER);
print_r($_POST);
?>