SE::Yandex::Suggest - Yandex Search Suggestions Scraper
Scraper Overview
Scraper for search suggestions by keywords in Yandex. Thanks to the SE::Yandex::Suggest scraper, you can automatically collect keyword databases from Yandex search system suggestions by query. Using the SE::Yandex::Suggest scraper, you can easily and quickly scrape Yandex suggestions by query based on the region.
The Yandex suggestions scraper solves one of the main SEO tasks, namely the fast automated obtaining of an extended semantic core. Google search suggestions allow covering the maximum number of phrases, and in combination with the Yandex Keyword Scraper - 
SE::Yandex::WordStat you will get the most complete semantics, which will help attract more organic traffic.
Thanks to the multi-threaded work of A-Parser, the processing speed can reach 5700 requests per minute, which on average allows obtaining up to 17000-18000 results per minute.
You can use automatic query multiplication, substitution of subqueries from files, iteration of alphanumeric combinations and lists to get the maximum possible number of results. Using result filtering you can immediately clean up the result, removing all unnecessary junk (using negative keywords).
A-Parser functionality allows you to save the scraper's parsing settings for further use (presets), set a parsing schedule, and much more.
Saving results is possible in the form and structure you need, thanks to the built-in powerful template engine Template Toolkit which allows applying additional logic to the results and outputting data in various formats, including JSON, SQL, and CSV.
Collected Data
- Number of results by query
- Suggestions for the query
- When choosing the 2nd query method, additional data and types of suggestions are also collected
Use Cases
- Collecting keyword databases
Queries
As queries, you need to specify search phrases, for example:
форекс
написать реферат
рефераты онлайн
купить машину в Москве
новости
Query Substitutions
You can use built-in macros for automatic substitution of subqueries from files, for example, we want to add some list of other words to each query, let's specify several main queries:
essay
article
thesis
In the query format, we will specify a macro for substituting additional words from the file Keywords.txt, this method allows to increase the variability of queries many times over:
{subs:Keywords} $query
This macro will create as many additional queries as there are in the file for each original search query, which in total will give [number of original queries (domains)] x [number of queries in the Keywords file] = [total number of queries] as a result of the macro's work.
For example, if the file Keywords.txt contains:
buy
cheap
As a result, the substitution macro will turn 3 main queries into 6:
buy essay
cheap essay
buy article
cheap article
buy thesis
cheap thesis
Output options
A-Parser supports flexible formatting of results thanks to the built-in template engine Template Toolkit, which allows it to output results in any form, as well as in structured formats, such as CSV or JSON
Exporting a list of suggestions
Result format:
$results.format('$suggest\n')
Example of result:
рефераты онлайн бесплатно
рефераты онлайн заказ
рефераты онлайн скачать бесплатно
рефераты онлайн скачать
рефераты онлайн сделать
рефераты онлайн делать
рефераты онлайн найти
рефераты онлайн бесплатно сделать
рефераты онлайн заказать
Output of the query, number of found suggestions, suggestions for the query
Result format:
$query - $totalcount:\n$results.format('$suggest\n')
Example of result:
рефераты онлайн - 10:
рефераты онлайн
рефераты онлайн бесплатно
рефераты онлайн заказ
рефераты онлайн скачать бесплатно
рефераты онлайн скачать
рефераты онлайн сделать
рефераты онлайн делать
рефераты онлайн найти
рефераты онлайн бесплатно сделать
рефераты онлайн заказать
Output in a CSV table
The built-in utility $tools.CSVLine allows creating correct table documents, ready for import into Excel or Google Sheets.
Result format:
[% FOREACH i IN results;
tools.CSVline(i.suggest);
END %]
File name:
$datefile.format().csv
Initial text:
Подсказки
In the General result format, the Template Toolkit is used to output suggest elements of the results array in a FOREACH loop.
To change the file name of the results, simply change the file extension to csv.
To make the "Initial text" option available in the Task Editor, you need to activate "More options". In the "Initial text," write the column names separated by commas and make the second line empty.
Keyword competition
Saving in SQL format
Result format:
[% FOREACH results;
"INSERT INTO serp VALUES('" _ query _ "', '"; suggest _ "')\n";
END %]
Example of result:
INSERT INTO serp VALUES('рефераты онлайн', 'рефераты онлайн')
INSERT INTO serp VALUES('рефераты онлайн', 'рефераты онлайн бесплатно')
INSERT INTO serp VALUES('рефераты онлайн', 'рефераты онлайн заказ')
INSERT INTO serp VALUES('рефераты онлайн', 'рефераты онлайн скачать бесплатно')
INSERT INTO serp VALUES('рефераты онлайн', 'рефераты онлайн скачать')
INSERT INTO serp VALUES('рефераты онлайн', 'рефераты онлайн сделать')
INSERT INTO serp VALUES('рефераты онлайн', 'рефераты онлайн делать')
INSERT INTO serp VALUES('рефераты онлайн', 'рефераты онлайн найти')
...
Dumping results to JSON
Общий формат результата:
[% IF notFirst;
",\n";
ELSE;
notFirst = 1;
END;
obj = {};
obj.totalcount = p1.totalcount;
obj.suggests = [];
FOREACH item IN p1.serp;
obj.suggests.push(item.suggest);
END;
obj.json %]
Начальный текст:
[
Конечный текст:
]
Example of result:
[{"suggests":["рефераты онлайн","рефераты онлайн бесплатно","рефераты онлайн заказ","рефераты онлайн скачать бесплатно","рефераты онлайн скачать","рефераты онлайн сделать","рефераты онлайн делать","рефераты онлайн найти","рефераты онлайн бесплатно сделать","рефераты онлайн заказать"],"totalcount":10}]
To make the "Initial text" and "Final text" options available in the Task Editor, you need to activate "More options".
Results processing
A-Parser allows processing results directly during scraping, in this section we have presented the most popular cases for the scraper SE::Yandex::Suggest
Parse to level option
The option instructs the scraper to substitute the obtained results into the query queue to the specified depth level, for example:
- If the 1st level is specified, the scraper will add to the queries all the results obtained from the original query
- If the 2nd level is specified, the scraper will add to the queries all the results obtained from the original query + all the results obtained from the 1st level queries
- and so on.
In simple terms - this is the substitution of the obtained results into the query queue, which in turn allows scraping more results. Since the results will likely contain duplicates, to prevent the scraper from doing extra work (not scraping the same thing), it is recommended to enable Unique queries
Download example
How to import an example into A-Parser
eJx1VFtv2jAU/iuVhdRVQggKfVjeKCrSJtZ0pX2YgAe3Pom8OrZnO6woyn/vsRPi
wLoXy+f2nct37Io4at/sgwELzpJkUxEd7iQhDDJaCkeGRFNjwXjzhqzvkuQXlQze
k2Rd5jlY79H5VsQdNGC02oMxnAEaOUM5U6agDrEbyD0VpXcbNBo7auxfLge2Ad1u
5eUVqXuA2nEl7REv1LSCPYiINhnX/y9AmOh4PZnWu92QtMmXIbevRk9Gbf+dcU33
8KR8B1xAVC9RuqdF6IFRB956bOJq5N49AmWM+6KpaDL4Ccasz5L/CTOQCn3xajjY
pVEFqhwEAK88HKvbkEGQCUKUIfZnE0OSjAoLQ2Kx1CXFQti5hTsw1CmTtkNMKqLk
XIgwwOgW8G9LLhjSPc8w6Fsb+LlL+g9G3bXXT4VU/DVYQ4cSpNv0R4xiaqXy4zAE
L7hD2S5UKT0xY1S+AehuZvferVAGujQtcpsdN1qD9LxHyuY6qk7aOKHlVPmqZMbz
tN2lo2cpn/DZpHKhCi3A9yVLIZAWC49xPea2pcELscDz4EVIcfLgnFLCfl83pWrD
cf1ufIEFTrKftYV8pUI8P676Fh9ondKpvDPmjL6wGGRbjmezsT+nN+E+690bfThn
1+F8uQgqCCcL50t0mn5t9TIwewHW0gPxlTnIFe4szq3edT9J999Un/4nSVXjRvy2
D423H5/3RR3yYJFufOr1Bxh5omw=
Filtering results (using negative keywords)
By using negative keywords, it is possible to immediately remove the results that you do not need. Similarly, by using a filter, you can also keep only those results that contain the necessary words.
Download example
How to import an example into A-Parser
eJx1VN9vGjEM/leqCKmrVCEoIG33RlGRNrHSFfowAQ+BMyxrLrklOVbE7n+f44S7
g3UvUWx/tr/4R47McftqnwxYcJYliyPL6c4SlsKWF9KxW5ZzY8F484LNHpLkO1cp
vCXJrNjtwHpEhT0yd8gBvfUejBEpoFGkKG+1ybjD2CHknsvCw1pBY9vB/uG6ZUPQ
5VJd37CyETB3Qit7ikecJrAHWUfrdsr/E5CmBt51ew3kVkgH3hrZJYt488kiHba6
jbh5cDKgtEPGmx+N1yyLTr/f92fvI51r0mzovvlDQpeEAd07DWi4rwnUu2sY+rVD
z78mFAKzsXK1OpG2Y6qfr2jebcceVsYZ38Nch6dCrR6j9Mgz6kPKHXjrqRE3bffm
I/A0FT4flyGDn4I664sSv+jlSiMWr0aAHRudocoBBfDKw4ndgrVI9uUsyPdb8GHJ
lksLWG+kOuZIJL20CCw+d9pM4yAkR6bVUEoaghpG8e8LIVMc2eEWnT5Hx/ch039i
lNXzmqlwnH4b5FBFIel++rX2SvVE707FkCITDmU70oXyjemg8hUgr2r26GGZNlCl
iZFjdtzKHJSf3bplw7xWnT3jrC3nyo1WW7Gbxn04IQs1x9WfqpHOcgn+XaqQEtti
4bkej6GNbfBCTfDSeUQpzj4Np7W0X2aBam4Ejt/AE8ywks2sMeSGS/nyPGlavKN1
Op+qB2Mu2keDwRpLNGjsXnO56OzTPvXXV2GJ6EzrBY279ynqFXX2CqzlB+aZOdhp
nFmsW7mqfsPqzzy++ycmxxIn4qd9CmhfPo9FHfbB0gJ3y7/kMN7o
See also: Results filters
Possible settings
| Parameter Name | Default Value | Description |
|---|---|---|
| Region | Russia | Selection of the region from which the search is conducted |
| Request method | Method 1 | Request method: Method 1 - old method, the same as before; Method 2 - new method, allows collecting all extended hints (ads, facts, website links) |