Skip to main content

Additional options for some scrapers

Some scrapers support additional options that can be added through the settings context menu.

Additional options for scraper

Parse all results

This option is available for the scrapers SE::GoogleSE::Google, SE::YandexSE::Yandex, SE::BingSE::Bing, SE::YahooSE::Yahoo and allows to bypass the limitation of 1000 results per query:

  • When parsing queries, A-Parser estimates the number of results in the output displayed by Google or Yandex
  • Depending on the number of results, the scraper adds a multitude of additional queries, thus allowing to parse many more results (often - all) for a single query

Parse to level

This option is present for suggestion scrapers (for example SE::Google::SuggestSE::Google::Suggest), for the scraper SE::Yandex::WordStatSE::Yandex::WordStat and for the link scraper HTML::LinkExtractorHTML::LinkExtractor:

  • For suggestion and wordstat scrapers, it reinserts the parsed keywords into the queries, thus allowing to parse all nested keywords up to the specified level
  • For the HTML::LinkExtractorHTML::LinkExtractor scraper, it reinserts all internal site links into the queries, allowing to traverse the entire site, creating its map of links

For this option, it is necessary to enable query deduplication, otherwise, there is a possibility that the scraper will keep parsing the same keywords or links in a loop

This option allows specifying the ability to parse in depth by related keywords. Supported by scrapers SE::GoogleSE::Google, SE::BingSE::Bing, SE::YahooSE::Yahoo.

Other options

There are a number of additional options for different scrapers, you can learn more about their purpose in the documentation for each scraper:

  • Use Pages, Check Content, Check Next Page - options for the scraper Net::HTTPNet::HTTP
  • Parse People also ask - an option for the scraper SE::GoogleSE::Google