SE::Yandex::Position - Checking website positions by keywords in Yandex
Overview of the scraper
Scraper for checking site positions by keywords in Yandex. With the SE::Yandex::Position scraper, you can automatically check positions in Yandex search results using your own domain databases. Using the SE::Yandex::Position scraper makes it easy, accurate, and fast to determine a site's position in Yandex. Checking positions in Yandex is performed in multi-threaded mode, with the option to use captcha solving services (AntiCaptcha or any other supporting their API). The Yandex position scraper is always up-to-date, as it is regularly updated by our specialists.
A-Parser's functionality allows you to save the SE::Yandex::Position scraper parsing settings for future use (presets), setting a parsing schedule, and much more. You can use automatic substitution of subqueries from files.
Saving results is possible in the format and structure you need, thanks to the integrated powerful templater Template Toolkit which allows you to apply additional logic to the results and output data in various formats, including JSON, SQL and CSV.
Scraper use cases
🔗 Overview of presentation options
The article discusses 4 different options for presenting the result: text, CSV, JSON, HTML
🔗 ⏩Positions for multiple regions
Getting site positions for multiple regions simultaneously
Collected data
- The site's position and link to the site page
- List of all site positions and page links
Capabilities
- All capabilities of the
SE::Yandex - Automatically stops scraping when the site is found
- Supports subdomain search
- Ability to compare the searched position by domain, top level domain, and exact URL
- Collects positions for multiple domains simultaneously
Use cases
- Checking the positions of your sites and competitors' sites
- Finding high-traffic site pages
Queries
As queries, you must specify the domain of the site being searched and the search query separated by a space, for example:
lenta.ru news
lenta.ru news online
If you need to check one site against a list of queries, you can specify the domain in the query format (Query format):
lenta.ru $query
Or just use a list of keywords. To use multiple domains in one query, list the domains separated by commas and the keyword separated by a space, for example:
lenta.ru,ria.ru,notfound.com news feed
The results will be written to the $bulkcheck array.
The option Stop when found is also supported; parsing will finish if positions are found for all domains.
Query substitutions
You can use built-in macros for automatic substitution of subqueries from files, for example, if we want to check a site/sites against a key database, we specify a few main queries:
ria.ru
lenta.ru
rbc.ru
yandex.ru
In the query format, we specify the macro for substituting additional words from the file Keywords.txt, this method allows checking a site database against a key database and getting positions as a result:
$query {subs:Keywords}
This macro will create as many additional queries as there are in the file for each original search query, which in total will give [number of original queries (domains)] x [number of queries in the Keywords file] = [total number of queries] as a result of the macro operation.
Output results examples
A-Parser supports flexible result formatting thanks to the built-in templater Template Toolkit, which allows it to output results in an arbitrary form, as well as in a structured form, such as CSV or JSON
Exporting the list of positions
Getting the result in the form:
searched domain - key: position number in search results
Result format:
$domain - $key: $position\n
Example result:
lenta.ru - news: 3
lenta.ru - news online: 13
...
Simultaneous checking of multiple domains (bulk check)
Information on all domains when simultaneously checking multiple domains is contained in the $bulkcheck array.
Result format:
$bulkcheck.format('$domain - $position\n')
Query example:
lenta.ru,ria.ru,notfound.com news feed
Example result:
lenta.ru - 1
ria.ru - 4
notfound.com - 0
Links + anchors + snippets with position output
Links + anchors + snippets in a CSV table
Saving related keywords
Keyword competition
Checking link indexing
Saving in SQL format
Dumping results to JSON
Results processing
A-Parser allows you to process results directly during scraping; in this section, we provide the most popular use cases for the SE::Yandex::Position scraper
Saving domains without zero positions
The example of simultaneous checking of multiple domains (see above in the output options) was taken as a basis, and a filter was added.
Add a filter and select the position output variable from the drop-down list. Select type: >. Next, you need to enter 0 in Number. With this filter, you can remove all results with a zero position.
Download example
How to import the example into A-Parser
eJx1VE1v2zAM/SuGEKAr4AXJ1gKDDwPSYAE2dE3Xj8OQ5KBGdKtFFj1JTpsF/u+j
ZNlOuu4im9Qj+fhEac8ctxt7bcCCsyxb7FkZ/lnGbr9k2U+uBbxk2TVa6STq5H1y
y7eQCCy41DZ5lu4JK5fw5A8YTMoIYykrubFgfMrFm5kIIiDnlXIs3TO3K4FK4haM
kQJoUwqyczQFd0QowNiWq8rDBg+V2qyfYL0ZNoh3J4OGEfEbtCSWS31yyur/Z68s
lAZfdn3mnCsLBxG5VA4M7UcK2YJ1pX2PbS+rNELvmrjPB2RH9I9lgGXMgrasXq3a
jHYW+PueyvEwSt9teqnvsOEBvXtG1hUvghKCO/C7rRCnQ/fiM3AhAjWumgr+IPqq
91r+DuQ0EpZ+jQQ7M1iQy0FI4J27lt2CDYLtG61C7I8mJkqWMktUZ5yIiNc7kmTh
Ds08aED+PUM9UeoStqB6WMh/UUklaGomOQV9jYFvQ+b/5Ki79g5L0Zk/G+LQZQnW
xfx7HyXwEh+pc/FAfStZSEe2nWKlXTy/DUDZaXblNSvQQFcmZo7V6TKVoP2A9Uc2
KXvXURtHx3LsXKPO5eM8Dm2LrPQd3di5nmJRKvB96Uqp1A/zTT8eExuPwRs9wdfB
01DCt95eReYQlf1221AtjaTxO/cEC1LysGpMueZK3d9cHu6wfqTIUKAdH5oqNTJ8
NLqclBXDNRbJshqdfRRhhbB+6P/PxmFtPJ8a6ENYz/uwuD1a+vdkTXfhEWlKSal6
1T1B3eO2f/shyvY1DcEve93AvWIeTD6S3gbEuP4LmbnKEA==
See also: Result filters
Link deduplication
Link deduplication by domain
Extracting domains
Removing tags from anchors and snippets
Filtering links by inclusion
Possible settings
Supports all settings of the SE::Yandex, scraper, as well as additionally:
| Parameter Name | Default Value | Description |
|---|---|---|
| Pages count | 1 | Number of search results pages to scrape (from 1 to 25) |
| Links per page | 20 | Number of links per search results page (10 / 20 / 30 / 50) |
| Result format | $domain - $key: $position\n | Default result output format |
| Stop when found | ☑ | Stop parsing if the domain is found; will not proceed to the next pages |
| Match type | Exact domain | Option to compare the searched position by domain, top level domain, or full link (Exact domain / Top level domain / Exact url) |