-
Join our Telegram chat: https://t.me/a_parser_enDismiss Notice
Proxy setup
-
Contents
Main capabilities(top)
- Simultaneous support of HTTP, SOCKS4 and SOCKS5 proxy
- Multithread check
- Loading proxy from the local file
- Multithreaded loading from external sources
- Check on anonymous
- Support of authorization according to login\password both for HTTP and for SOCKS, and also support of different data for authorization in the format login:[email protected]:port
- Opportunity to set arbitrary regular expressions for the IP address and port of a proxy when parsing from external sources
- Possibility of output checked proxy to the file
- Possibility to use multiple proxy sources in a single task
- Since version 1.2.948, support for domain proxies has been added in the domain: port and login:р[email protected]:рort
File structure(top)
Work directory of a proxy cheker:
- proxy.txt - from this file loading proxy (in the ip:port format), here you need to put the proxy list
- sites.txt - in this file it is necessary to paste the list of sources proxy (links to proxy, in a format one link to a line)
- alive.txt - in this file remain each 5 seconds live proxies if the appropriate option is included
- regex.txt - in this file is the list of the regular expressions for parsing proxy from external sources (in a format one regular expression for the line, in $1 there shall be an IP address, in $2 - port)
Control(top)
Control of a proxy checker is exercised in the tab Proxy checker, where you can add, delete, and enable or disable the proxy checkers. Also in this tab shows statistics of each proxy checker, graph of live proxy and statistics for the processing of sources::
Adding and configuring a proxy checker(top)
- Select "Proxy checker" and press "Add Proxy Checker" or select in drop-down menu "Edit" item. Page appears of configure the proxy checker, select the preset default.
- If necessary, set the desired number of threads for checking proxies (Check threads), select the type of proxy (Proxies type) and change other settings. If you use a proxy from A-Parser (from Members Area) then is enough to disable checking proxy (No check proxies - to put a tick), all the rest left by default.
- Save preset (for existing - Save, for the new - Save as new).
- Go back to the "Proxy checker", check to see if the newly created checker is enabled, if not - enable.
- Open the directory specified in the "Working path".
- Next, you must specify the sources of proxies: links in sites.txt, list of proxy in proxy.txt. If you use a proxy from A-Parser (from Members Area) specify in the file sites.txt the link from Proxy tab in the Members Area, previously in the same place saving IP.
- Go back to the A-Parser into "Check proxy" and make sure that the "Total alive" for the edited proxy checker is greater than 0 - means the proxy configured correctly.
Choosing proxy checker for task (parser)(top)
- Goto "Settings - Configs Presets", select desired preset or create a new (Save as new).
- In the field "Proxy Checkers" select one or more proxy checkers (for use, a proxy checkers must be enabled) and save (Save).
- Now you can use the created Config preset, with specified proxy in their own tasks, by selecting it in the Task editor.
- You can also override the proxy checker in each parser using the function Override - Proxy Checker.
- you need to forcibly select an excluded proxy for a task
- you must set to use the excluded proxychecker for a particular scrapers.
Changes in logic.
Previously, if a particular proxychecker was selected in a task, and another proxychecker was specified in the scraper, the scraper would expect the proxy. Now the settings of a particular scraper are a higher priority:
- "All" - uses all the proxycheckers selected for the task
- a particular proxychecker - uses it even if it is not selected in the job
Use proxy with authorization(top)
1. If login and the password for all proxy identical that in settings of proxy cheker we specify:
Proxy is saved in files/proxy/proxy.txt file or specify links to the sites in files/proxy/sites.txt file
Proxy need to be specified in format ip:port
2. If login and the password for all proxy are different that in settings of proxy cheker we specify:
Proxies need to be specified in format login:[email protected]:port
Briefly about each parameter(top)
Installation of a script of check to the hosting(top)
Upload the following PHP script to your hosting or the server: