SE::Yandex::WordStat::ByDate - Scraper of Yandex WordStat statistics by months or weeks
Scraper Overview
Wordstat is a Yandex service designed to assess user interest in various topics and to select keywords for SEO optimization and contextual advertising. In addition, with the help of Wordstat Yandex, you can assess the seasonality and geographical dependence of search queries.
The Yandex WordStat by date scraper supports automatic query multiplication, you can be sure that you will get the maximum number of results from the output. Also, A-Parser can automatically navigate through related queries to the specified depth.
The functionality of A-Parser allows you to save parsing settings for further use (presets), set a parsing schedule, and much more. You can use automatic query multiplication, substitution of subqueries from files, iteration of alphanumeric combinations and lists to obtain the maximum possible number of results.
Saving results is possible in the form and structure that you need, thanks to the built-in powerful template engine Template Toolkit which allows you to apply additional logic to the results and output data in various formats, including JSON, SQL, and CSV.
Use Cases for the Scraper
🔗 Automation of Wordstat bydate
Used to store up-to-date information in the database, and with each launch, it generates a csv with fresh data
Accounts
For the 
SE::Yandex::WordStat::ByDate scraper to work, Yandex accounts are required. Accounts can be registered using the 
SE::Yandex::Register scraper or simply by adding existing accounts to the file files/SE-Yandex/accounts.txt in the supported format.
Alternatively, you can enable "on-the-fly" account registration.
Collected Data
- Keyword statistics by months or weeks
- Date
- Absolute value
- Relative value
Capabilities
- Supports the selection of search region (with subgroups)
- Ability to select several regions at once for evaluation
- Support for automatic bypass of Smart captcha and the ability to bypass graphic captcha using the AntiCaptcha service or any other supporting their API
- Choice of device type
- Ability to choose the method of authorization
- Ability to register accounts "on the fly"
- Supports working with extended account format and can answer the secret question (if the answer is in
info). It also uses the saved proxy for authorization (if it is ininfo).
Use Cases
- Evaluation of the amount of traffic by keyword
- Identifying seasonal keywords
Queries
As queries, you need to specify keywords, just as if they were entered directly into the Wordstat search form, for example:
test
Output Results Options
A-Parser supports flexible formatting of results thanks to the built-in template engine Template Toolkit, which allows it to output results in any form, as well as in structured formats like CSV or JSON
Default Output
Result format:
Views:\n$views.format('$date $count $relcount\n')
The result displays statistics for keywords for the month and for the week:
Monthly:
2011-09-30 3010832 0.0008903808
2011-10-31 681432 0.0001825883
2011-11-30 628532 0.0001575008
2011-12-31 629072 0.0001495699
2012-01-31 561206 0.0001300651
2012-02-29 572039 0.0001290000
2012-03-31 614897 0.0001225754
2012-04-30 520433 0.0001185340
2012-05-31 521967 0.0001235327
2012-06-30 502568 0.0001299958
...
Weekly:
2012-09-16 118715 0.0001222877
2012-09-23 120799 0.0001211773
2012-09-30 137809 0.0001365837
2012-10-07 133929 0.0001313643
2012-10-14 140373 0.0001293922
2012-10-21 136014 0.0001242209
2012-10-28 148350 0.0001293328
2012-11-04 139556 0.0001232566
2012-11-11 154830 0.0001314057
2012-11-18 136458 0.0001147489
2012-11-25 149463 0.0001261401
2012-12-02 144724 0.0001197564
2012-12-09 149142 0.0001212195
2012-12-16 162864 0.0001298181
Output to a CSV Table
Result format:
[% FOREACH i IN views;
tools.CSVline(query, i.count, i.date);
END %]
Example of result:
"тест",9661734,2012-03-31
"тест",8567243,2012-04-30
"тест",9028986,2012-05-31
"тест",6082099,2012-06-30
"тест",5531950,2012-07-31
"тест",5214663,2012-08-31
"тест",6603865,2012-09-30
"тест",9127457,2012-10-31
"тест",9238652,2012-11-30
Saving in SQL Format
Result format:
[% FOREACH i IN views;
"INSERT INTO views VALUES('" _ query _ "', '"; i.count _ "', '"; i.relcount _ "', '"; i.date _ "')\n";
END %]
Example of result:
INSERT INTO serp VALUES('тест', '9661734', '0.0019259985', '2012-03-31')
INSERT INTO serp VALUES('тест', '8567243', '0.0019512785', '2012-04-30')
INSERT INTO serp VALUES('тест', '9028986', '0.0021368683', '2012-05-31')
INSERT INTO serp VALUES('тест', '6082099', '0.0015732140', '2012-06-30')
INSERT INTO serp VALUES('тест', '5531950', '0.0013160071', '2012-07-31')
INSERT INTO serp VALUES('тест', '5214663', '0.0013327945', '2012-08-31')
INSERT INTO serp VALUES('тест', '6603865', '0.0015936909', '2012-09-30')
INSERT INTO serp VALUES('тест', '9127457', '0.0018740506', '2012-10-31')
INSERT INTO serp VALUES('тест', '9238652', '0.0018308715', '2012-11-30')
Dumping Results to JSON
Общий формат результата:
[% IF notFirst;
",\n";
ELSE;
notFirst = 1;
END;
obj = {};
obj.views = [];
FOREACH item IN p1.views;
obj.views.push({
date = item.date
relcount = item.relcount
count = item.count
});
END;
obj.json %]
Начальный текст:
[
Конечный текст:
]
Example of result:
[{
"views": [
{
"count": "9661734",
"date": "2012-03-31",
"relcount": "0.0019259985"
},
{
"count": "8567243",
"date": "2012-04-30",
"relcount": "0.0019512785"
},
{
"count": "9028986",
"date": "2012-05-31",
"relcount": "0.0021368683"
}
]
}]
See also: Results Filters
Possible Settings
| Parameter | Default Value | Description |
|---|---|---|
| Period | Monthly | Select the period |
| Region | All | Search region |
| AntiGate preset | default | You need to preconfigure the scraper  Util::AntiGate - specify your access key and other parameters, then select the created preset here |
| AntiGate preset for Login | default | AntiGate preset for login. You need to preconfigure the scraper Util::AntiGate with parameters, then select the created preset here |
| Type | All | Select the type of device |
| Accounts | Only from "accounts.txt" | Method of working with accounts: Always auto register - always automatically register accounts "on the fly", you need to select a configured preset in the SE::Yandex::Register preset parameter. Auto register if no more in "accounts.txt" - first use existing accounts from accounts.txt, and if they run out - use automatic registration "on the fly", for which you need to select a configured preset in the SE::Yandex::Register preset parameter. Only from "accounts.txt" - use only existing accounts from accounts.txt, and if they run out - wait a specified time (parameter Wait new accounts in "accounts.txt") for new ones to appear |
| Wait new accounts in "accounts.txt" | 0 | Waiting time for new accounts to appear in accounts.txt |
| Remove bad accounts | Always, except wrong login/password | Automatic removal of "bad" accounts: Always - always remove. Always, except wrong login/password - always remove, except when Yandex reports that the wrong login/password has been entered. The fact is that Yandex can give such a message when the IP is banned for an absolutely working account, so optionally you can leave such accounts for reuse. Never - never remove. Regardless of the selected option, accounts are not deleted in case of proxy/browser errors |
| SE::Yandex::Register preset | default | Select a preset of settings for SE::Yandex::Register |
| Authorization method | HTTP | Authorization method: HTTP - fast, not resource-intensive. Chrome - slow, resource-intensive, theoretically can extend the life of accounts |
| Chrome headless | ☑ | If the option is enabled, the browser will not be displayed |
| Use sessions | ☑ | Use of sessions |
| Do not reset session if authorization passed | ☑ | Do not reset the session upon errors if the scraper has already authorized |