Additional options for some scrapers
Some scrapers support additional options that can be added through the context menu of settings:
Parse all results
This option is available for scrapers SE::Google,
SE::Yandex,
SE::Bing,
SE::Yahoo and allows you to bypass the limit of 1000 results per query:
- When parsing queries, A-Parser estimates the number of results displayed by Google or Yandex
- Depending on the number of results, the scraper adds many additional queries, thereby allowing you to parse many more results (often all) with one query
Parse to level
This option is present for hint scrapers (for example, SE::Google::Suggest), for the scraper
SE::Yandex::WordStat and for the link scraper
HTML::LinkExtractor:
- For hint and wordstat scrapers, it substitutes the parsed keywords into the queries again, allowing you to parse all nested keywords to the specified level
- For the scraper
HTML::LinkExtractor, it substitutes all internal links of the site into the queries again, allowing you to bypass the entire site by creating its map of links
For this option, it is necessary to include query deduplication, otherwise there is a possibility that the scraper will parse the same keywords or links in a circle
Parse related to level
This option allows you to specify the ability to parse in depth by related keywords. Supported by scrapers SE::Google,
SE::Bing,
SE::Yahoo.
Other options
There are a number of additional options for different scrapers, more information about their purpose can be found in the documentation for each scraper:
- Use Pages, Check Content, Check Next Page - options for scraper
Net::HTTP