-
Join our Telegram chat: https://t.me/a_parser_enDismiss Notice
Net::HTTP - Downloads the specified page, supports multipage parsing
-
Collected data(top)
- Response code from server
- Description of the response of the server
- Titles of the response of the server
- Content
- Proxies used in case of this request
- Array with all pages (it is used by work of the option Use Pages)
Opportunity(top)
- Option Check content - checks if the regular expression didn't work, the page will be loaded again with other proxy
- Option Use Pages - allows to enumerate the specified number of pages with a certain step. $pagenum - variable that contains the current page number when iterating. It should be used for substitution in the right place.
- Option Check next page - RegEx, whether which defines there is the following page or not, and if exists - switches to it, within a specified limit (0 - no limit)
- Option Page as new query - sends a link to the next page as a new query, thus allowing to remove a limit on the number of pages to navigate
Use options(top)
Requests(top)
As requests it is necessary to specify links to pages:
Possible settings(top)
Global settings for all parsers