Skip to main content

SE::Yandex - Yandex Search Results Scraper


Scraper Overview

The Yandex search results scraper is rightfully among the TOP-5 most in-demand scrapers. One of the main competitive advantages of our scraper is its colossal performance. Thanks to the multi-threaded work of A-Parser, the processing speed can reach 3000-7000 requests per minute, which on average allows obtaining up to 5000000 links per minute, while the resource consumption is minimal, any office or home computer will do, as well as any entry-level VDS. Our scraper supports all Yandex search operators, which significantly expands the scraping possibilities. Stability and uninterrupted scraping of results are ensured by captcha recognition through AntiCaptcha or any other supporting their API (Anti-Captcha, RuCaptcha,, 2captcha, and others).

The flexibility in settings allows specifying the type of results (mobile/desktop), region, language, sorting results by date, and much more. A-Parser's functionality allows you to save scraping settings for further use (presets), set a scraping schedule, and much more. You can use automatic query multiplication, substitution of subqueries from files, permutation of alphanumeric combinations and lists to obtain the maximum possible number of results.

Saving results is possible in the form and structure that you need, thanks to the built-in powerful templating engine Template Toolkit which allows applying additional logic to the results and outputting data in various formats, including JSON, SQL, and CSV.

Scraper Use Cases

Data Collected

Collected data
  • Number of results for the query
  • Links, anchors, and snippets from the search results
    • Also collects information about flags for each result, currently supported flags include: Date, Image Preview, Video, Rich snippet, Featured snippet
  • Time of the first and last caching (desktop search results only)
  • List of related keywords (Related keywords)
  • Determines whether Yandex considered the query a typo or not
  • Scraping the link to the page cache (desktop search results only)
  • List of site icon names and their types, if any (including Turbo)
  • Top, middle, and bottom ad blocks from all pages
    • Link, visible link, anchor, snippet, and domain
    • Additional links and anchors
    • Metro station
  • List of quick answers (Quick answers): questions, answers, links to sources (enabled by a separate option Parse Quick answers)


  • Choice of search results type: mobile/desktop
  • Support for all search operators of Yandex (site:, lang:, etc.)
  • Scraper parses the maximum number of results provided by Yandex - 25 pages with 50 items each
  • Ability to search for related keywords
  • Supports selection of search region, domain
  • Ability to sort search results by date
  • Scraper parses the time of page indexing, with the ability to filter results by this parameter
  • Option for browser emulation (allows for high scraping speed and low captcha consumption)
  • Ability to bypass captcha using the AntiCaptcha service or any other supporting their API (it is recommended to enable the is_russian parameter in the anti-gate settings)
  • Ability to set the number of quick answers (Quick answers) the scraper should collect by clicking through each question in depth


The following scrapers are based on the Yandex scraper:

Use cases

  • Collecting link databases - for A-Poster, XRumer, AllSubmitter, etc.
  • Competition assessment for keywords
  • Searching for backlinks (mentions) of websites
  • Checking website indexing
  • Searching for vulnerable websites
  • Any other scenarios involving scraping Yandex in one way or another


As queries, you need to specify search phrases, just as if you were entering them directly into the Yandex search form, for example:

окна Москва
lang:en windows Moscow

Query substitutions

You can use built-in macros for query diversification, for example, if we want to get a very large database of forums, we will specify several main queries in different languages:


In the query format, we will specify a character permutation from a to zzzz, this method allows for maximum rotation of the search results and obtaining many new unique results:

$query {az:a:zzzz}

This macro will create 475254 additional queries for each original search query, which in total will give 4 x 475254 = 1901016 search queries, an impressive figure, but this is not a problem for A-Parser. At a speed of 2000 queries per minute, such a task will be processed in just 16 hours.

Using operators

You can use search operators in the query format, so it will be automatically added to every query from your list:


Output results examples

A-Parser supports flexible formatting of results thanks to the built-in Template Toolkit, which allows it to output results in any form, as well as in structured formats, such as CSV or JSON

Exporting the list of links

Result format:


Example of result:

Result format:

[% FOREACH item IN serp;    loop.count _ ' - ' _ _ ' - ' _ item.anchor _ ' - ' _ item.snippet _ "\n"; END %]

Example of result:

1 - - <b>forum</b> - 
2 - - <div class=a11y-hidden>Веб-результат с дополнительными ссылками</div><b>Форум</b> – Arizona Role Play - Меню. Главная. <b>Форумы</b>. Новые сообщения. Что нового? Новые сообщения. Пользователи. Текущие посетители. <b>Форумы</b>. Вход. ... Статистика <b>форума</b>. Темы. 1,247,176. Сообщения. 5,225,340. Пользователи. 623,675.
3 - - <b>Форум</b> — Википедия - <b>Фо́рум</b> (лат. <b>forum</b> — арх. преддверие гробницы; площадка в давильне для подлежащего обработке винограда; рыночная площадь, городской рынок; торжище, центральная площадь):
4 - - Что такое <b>форум</b> и всё что с ним связано | Креативная... - Сегодня мы поговорим о том, что такое <b>форум</b> и обо всём, что с ним связано доступными и простыми словами. Каждый из нас стремиться найти такого человека, с кем можно было бы построить общение на общие для Вас интересы, поделиться своим опытом, дать совет. В современном мире, всё это стало возможно делать, не выходя из дома. Как раз одной из форм такого виртуального контакта и является веб-<b>форум</b>.
5 - - VimeWorld - <b>Форум</b> - <b>Форум</b> проекта идеальных игровых серверов Minecraft - VimeWorld...

The built-in utility $tools.CSVLine allows you to create correct table documents, ready for import into Excel or Google Sheets.

General result format:

[%  FOREACH i IN p1.serp;    tools.CSVline(, i.anchor, i.snippet); END  %]

File name:


Initial text:



In the General result format, the Template Toolkit templating engine is used to output the $serp array in a FOREACH loop.

In the file name of the results, you just need to change the file extension to csv.

To make the "Initial text" option available in the Task Editor, you need to activate "More options". In the "Initial text," we write the names of the columns separated by commas and make the second line empty.

Output of advertising blocks

Result format:

$ads.format('$link - $anchor - $snippet\n')

Example of result:;q=%D0%BA%D1%83%D0%BF%D0%B8%D1%82%D1%8C+%D0%B0%D0%B2%D0%B8%D0%B0%D0%B1%D0%B8%D0%BB%D0%B5%D1%82%D1%8B&amp;etext=2202.wBaB7RlytlP_PRaU6jPhHD2nHBNT_4gSF6009OyJEfpWjUPoA5WrSJAqCfap0m9ReXhoaGRlaWhmYmllemF2ZA.f04b1271668949ea17ffcafcb11c72c2ab6454c5 - <b>Купить</b> <b>авиабилет</b> до / - Мы предлагаем: <b>Купить</b> <b>авиабилет</b> до здесь. Суперпредложение! Спешите!;q=%D0%BA%D1%83%D0%BF%D0%B8%D1%82%D1%8C+%D0%B0%D0%B2%D0%B8%D0%B0%D0%B1%D0%B8%D0%BB%D0%B5%D1%82%D1%8B&amp;etext=2202.wBaB7RlytlP_PRaU6jPhHD2nHBNT_4gSF6009OyJEfpWjUPoA5WrSJAqCfap0m9ReXhoaGRlaWhmYmllemF2ZA.f04b1271668949ea17ffcafcb11c72c2ab6454c5 - <b>Купить</b> <b>авиабилет</b> онлайн на Туту.ру! Официальный сайт! - Билеты на самолет по низкой цене! Выгодные рейсы по всему миру! <b>Купить</b> билет онлайн!;q=%D0%BA%D1%83%D0%BF%D0%B8%D1%82%D1%8C+%D0%B0%D0%B2%D0%B8%D0%B0%D0%B1%D0%B8%D0%BB%D0%B5%D1%82%D1%8B&amp;etext=2202.wBaB7RlytlP_PRaU6jPhHD2nHBNT_4gSF6009OyJEfpWjUPoA5WrSJAqCfap0m9ReXhoaGRlaWhmYmllemF2ZA.f04b1271668949ea17ffcafcb11c72c2ab6454c5 - <b>Купить</b> <b>авиабилет</b> до / - Мы предлагаем: <b>Купить</b> <b>авиабилет</b> до здесь. Суперпредложение! Спешите!

Result format:


Example of result:

<b>test</b> tanki online
tanki online
все <b>тесты</b> точка ру
i exam ru тестирование
<b>тест</b> скорости интернета
онлайн <b>тест</b> производительности пк
онлайн тестирование
my <b>test</b> student как узнать ответы

To automatically remove HTML tags in the result, use the Results Builder, select the $related array and apply Remove HTML tags.

Keyword competition

Result format:

$query - $totalcount\n

Example of result:

все тесты точка ру - 25000000
онлайн тестирование - 13000000
tanki online - 7000000
i exam ru тестирование - 27000000 - 238000000
онлайн тест производительности пк - 16000000
speedtest - 2000000
тест скорости интернета - 16000000
test tanki online - 19000000
my test student как узнать ответы - 16000000

Identifying keywords with errors

Result format:

$query - $misspell\n

Example of result:

онлайн тестирование - 0
тест скорости интернета - 0
онлийн тастирование - 1
тест скарости интрнета - 1

Checking link indexing

Query format:


Result format:

$query.orig - $totalcount\n

Example of result: - 2 - 16 - 0 - 1000 - 499

To check link indexing, insert the corresponding operator into the Query format: site:.

The result format is displayed as "original url - number of pages in the index".

As a result, we get the address of the pages and their number in the search engine index.

If the page is missing, then the result will be: 0.

Saving in SQL format

Result format:

[%  FOREACH serp;   "INSERT INTO serp VALUES('" _ query _ "', '";   link _ "', '";  anchor _ "')\n"; END  %]

Example of result:

INSERT INTO serp VALUES('тест', '', 'Популярные онлайн <b>тесты</b>')
INSERT INTO serp VALUES('тест', '', 'Онлайн <b>тесты</b> c точным результатом от психологических...')
INSERT INTO serp VALUES('тест', '', '<b>Тесты</b> онлайн: лучшие, интересные и популярные')
INSERT INTO serp VALUES('тест', '', 'Интересные <b>тесты</b>. Пройти психологические интересные...')
INSERT INTO serp VALUES('тест', '', '<b>Тесты</b> онлайн | Online Test Pad')

Dumping results to JSON

Общий формат результата:

[% IF notFirst;
notFirst = 1;

obj = {};
obj.totalcount = p1.totalcount;
obj.links = [];

FOREACH item IN p1.serp;

obj.json %]

Начальный текст:


Конечный текст:


Example of result:


To make the "Initial text" and "Final text" options available in the Task Editor, you need to activate "More options".

Results processing

A-Parser allows you to process results directly during scraping, in this section we have provided the most popular cases for the Yandex scraper

Add deduplication and select from the dropdown list $serp.$ - Link.

Download example

How to import an example into A-Parser


Add deduplication and select from the dropdown list $serp.$ - Link. Choose the type of deduplication: Domain.

Download example

How to import an example into A-Parser


Extracting domains

Add Results Builder and select the source from the dropdown list: $p1.serp.$ - Link. Choose the type: Extract Domain.

Download example

How to import an example into A-Parser


See also: Results Builder

Removing tags from anchors and snippets

Add Results Constructor and from the dropdown menu select the source: $p1.serp.$i.anchor - Anchor. Choose type: Remove HTML tags.

Add Results Constructor again and from the dropdown menu select the source: $p1.serp.$i.snippet - Snippet. Choose type: Remove HTML tags.

Download example

How to import an example into A-Parser


You can add the Results Constructor as many times as you need.

See also: Results Constructor

Add a filter and from the dropdown menu select: $serp.$ - Link. Choose type: Contains string. Then in String write down the filtering criterion, for example, if we want to save only links that contain .com, then we write this in "String".

Download example

How to import an example into A-Parser


See also: Results Filters

Possible settings

Parameter NameDefault ValueDescription
AntiGate presetdefaultChoice of Util::AntiGateUtil::AntiGate preset, more about the setup here
AntiGate preset for old captchadefaultSimilar to AntiGate preset, but used only for regular (old, single image) captchas. If no preset is selected here, then for such captchas the preset chosen in AntiGate preset will be used.
Auto-Solve ClickCaptchaAutomatic solving of click captcha (without using services)
Experimental img captcha max count1Maximum number of captcha images per attempt
DeviceModern desktop computer (Windows 10, Chrome 84)Choice of the type of search results (Desktop computer / Mobile device)
Pages count5Number of pages to scrape (from 1 to 25)
Sort serp by dateSorting search results by date
Yandex domain for scraping, all domains are supported (.ru, .ua, .by, .kz,, .com). Starting from version 1.1.345, it is selected automatically depending on the chosen region.
Region of serp (lr=)MoscowChoice of search region (parameter lr=)
Custom region IDAbility to specify the ID of a region not present in the selection field. This option takes precedence over Region of serp (lr=). It is mandatory to set the corresponding domain in Yandex domain.
Search sites from (rstr=)Not setChoice of regional binding of sites (parameter rstr=)
LanguageAnyLanguage of search results (Russian, English, Belorussian, French, German, Indonesian, Kazakh, Tatar, Turkish, Ukrainian)
Parse not foundDetermines whether to scrape the search results if zero results are found for the searched query and another query's results are suggested
Not personalizedSearch personalization. More details here
Filter pagesModerate filterFiltering results from unwanted content (Family search / Moderate filter / Do not filter)
Use AccountsWorking with existing accounts in the file files/SE-Yandex/accounts.txt. SE::Yandex::RegisterSE::Yandex::Register - Allows registering accounts in Yandex
Remove bad accountsRemoving invalid accounts