Skip to main content

Multiple Scrapers in One Task

Capabilities​

A-Parser supports processing requests with several scrapers at once, which allows you to:

  • Scrape results from multiple search engines simultaneously
  • Analyze domains based on numerous parameters
  • Simultaneously gather keywords from suggestions of various search systems and Yandex.Wordstat
  • Gather Whois and DNS information for domains
  • Numerous other use cases

Adding a Scraper​

To add a new scraper, simply click the Add Scraper button in the Task Editor:

Add Scraper button in Task Editor

Each scraper is numbered, results for each scraper in the results constructor and when formatting results are available via abbreviations:

$p1, $p2... - where the number indicates the scraper number.

In the general result format results for each scraper are output sequentially by default, in the format specified in the settings of each scraper

Using multiple scrapers in one task increases the overall scraping speed because the number of simultaneous requests to a single scraper decreases, thereby reducing the frequency of proxy bans or captcha displays

Each scraper scrapes the original query or its variation when using the query constructor, has its own query format in the settings; however, it is not possible to pass the results of a previous scraper's work as queries to the next scraper (this feature is planned for the future)

Example of Domain Analysis​

Example of Domain Analysis

In this example, we use a list of domains that will soon expire (become available for registration again) as queries; we are interested in selecting domains with good parameters, for this we used six different scrapers:

In the results format, we output the original query (domain) and the results from each scraper, in the order we need:

$query, $p1.totalcount, $p2.globalRank, $p3.totalcount, $p4.trustrank, $p5.exists, $p6.bl\n
Download example

How to import the example into A-Parser

eJy1Vd1v2zYQ/1cMIg/bYKhxuvZBb47X7ANZ7dnOU1wEjHSSWVMkS1J2PCP/++5I
yrLTFOiADXrR8b5+v7vj8cA8dxs3s+DAO5bfH5gJ/yxnY8Xl/m8Y/KIbLpQbXGWX
bMgMtw4smd6zxYc8/1XrWgIqZrwGN5joVvlB62Aws/ppz4YH5vcGMJzegrWiJFNR
ovylBbuvtG24x6Mtly1ZOeEhvwg69vxpeM/mXG3y/FqoOuDxonBoXkLFW+kZWfwP
KFYsQlixCKJPkedL2zo/WUOxOcHxbxOsvTf5mzenRE9zLHgF11bvHNL+b7PEco7X
Fir3XZHvvJAPy9Yqhz/wkKajzzFW2BJufLHmmOHTkKEBRnQ3EUyeKjkcXJhR5rXn
sqDmkHyV1VI/ckmYSH77Qv9z5qnWNqnfZfAknHf0/z57lKuVYsd0C76FpcZ0lQhz
0KFA6SNvCOhFyT2QNot1+uHHzD8RE16WwguN0xUx02z3PO6U+BKIKo22xEWAu7G6
wSMPIUAg2PG9T4RxNFkbfP+KPiyvuHQwZA6h3nAEUr7U4Oxb7rWdGsKD5wem1VjK
W9iC7M1C/OtWyBIv4rhCp9+T4+sm069iPB/pnabCzu8sYjhGCdL19M/eq9S3uu6K
IUUjPMouXDc8pf2wATDHmn0ks0ZbOKZJkVN23DsGFI1Z37Kx6Y/OaJy15fyw0KoS
9TSNbmfZqiUut6ma6MZIIF6qlRLb4mDej8fYpTaQ0AN86TwJKRDW8cowr7V0fywi
VGMFjt87AthgJU+zppAFl/JufnuqIUfntZmqD9a+aF8YDMblVjjdZIVuVoo3Da9l
/C/WAmsT/7EQvNzFf5Ca21KrJDnP7b7mUfrcqoIWaJSkrrGBLukaTphsFJCLg1Ik
lRO1Es6kZG6jpeEqSbu1qCVXZZT2equtXycojDh7qDXeBuwI7p/u9Ti+N4fT7Y3S
6+sbFacLrtsp3WuktK/QA2dmROWfxG00Rzahhm+HrJapSRi+SMM6Ck3D8GGXAxaf
/YRHazIFwl6Gdw+l3W6X1QEi8aLGiwZ0i0HeXw7jgpx3Wa8eju9nPyjIxEQiCFBV
BnOhIy4PyyOROAhgDUWOktRYO7wzUcII+Ll4h0aETfkl38AiHoZ4uNs80RjRzVQb
ZzAe8iWql3HN/QY83pkuXTe6iM125YrWFhCZ8yVIvg/RS9iKAgItt8GR7fweuVJQ
FhK4ag1VhJCEPsWbeuxWPrjo13u3u1PVZty51Gp0l9QL9ozr5rXHPz+g4hsP8le6
s4c0aM8eQDzBo89uFseSNkBKgKskFnb0/A8Hayy4

⏩ Video Example of Creating a Preset from Multiple Scrapers​

This video lesson covers how to create a preset for scraping thematic domains based on a keyword phrase from Google search results, and retrieving various parameters for these domains.

The lesson covers:

  • Scraping Google search results
  • Outputting unique results as main domains
  • Retrieving parameters such as:
    • whether the domain is registered or not
    • registration date and expiration date of the domain
    • website language
    • title and description
    • website quality index
    • global Alexa ranking position
    • country with the most traffic according to Alexa
    • ranking position in that country according to Alexa
Last updated on