Shop::Wildberries::Suggest - Wildberries search suggestions scraper
Scraper overview
Scraper for search suggestions by keywords in Wildberries. Thanks to the Shop::Wildberries::Suggest scraper, you can automatically collect databases of keys from the search bar suggestions of the Wildberries marketplace by request. Using the Shop::Wildberries::Suggest scraper, you can easily and quickly scrape Wildberries suggestions by request.
The Wildberries suggestions scraper solves one of the main SEO tasks, namely the rapid automated obtaining of an extended semantic core. Wildberries search suggestions allow you to cover the maximum number of phrases, especially when using the Parse to level function.
Thanks to the multithreading work of A-Parser, the processing speed of requests can reach 10000 requests per minute, which on average allows obtaining about 20000-50000 results per minute.
You can use automatic query multiplication, substitution of subqueries from files, iteration of alphanumeric combinations and lists to get the maximum possible number of results. Using results filtering, you can immediately clean up the result, removing all unnecessary garbage (using minus words or filtering by suggestion type).
A-Parser functionality allows you to save the parsing settings of the Shop::Wildberries::Suggest scraper for further use (presets), set a parsing schedule, and much more.
Saving results is possible in the form and structure you need, thanks to the built-in powerful template engine Template Toolkit which allows you to apply additional logic to the results and output data in various formats, including JSON, SQL, and CSV.
Collected data
- Number of results by request
- Suggestions for the request
- Type of suggestion (suggest/category/brand)
Capabilities
- Ability to filter keywords when parsing in depth
- Ability to use the Parse to level function
Use cases
- Collection of keyword databases
Queries
As queries, you need to specify search phrases, for example:
adidas
майки и футболки
смартфон Xiaomi
Query substitutions
You can use built-in macros for automatic substitution of subqueries from files, for example, we want to add some list of other words to each query, let's specify several main queries:
скидка
акция
распродажа
In the query format, we will specify a macro for substituting additional words from the file Keywords.txt, this method allows to increase the variability of queries many times over:
$query {subs:Keywords}
This macro will create as many additional queries as there are in the file for each original search query, which in total will give [number of original queries] x [number of queries in the Keywords file] = [total number of queries] as a result of the macro's work.
For example, if the above substitutions are indicated in the file Keywords.txt, then in the end there will be 9 queries:
adidas скидка
майки и футболки скидка
смартфон Xiaomi скидка
adidas акция
майки и футболки акция
смартфон Xiaomi акция
adidas распродажа
майки и футболки распродажа
смартфон Xiaomi распродажа
Output results examples
A-Parser supports flexible result formatting thanks to the built-in Template Toolkit templating engine, which allows it to output results in any form, as well as in structured formats, such as CSV or JSON
Exporting the list of suggestions
Result format:
$results.format('$suggest\n')
Example of result:
adidas кроссовки
adidas для женщин
adidas кроссовки мужские
adidas terrex
adidas superstar
adidas originals
adidas terrex кроссовки мужские
Кеды и кроссовки
Дезодоранты
adidas
Output of the query, suggestions for the query + type of suggestion
Result format:
$query:\n$results.format('$suggest - $type\n')
Example of result:
adidas кроссовки - suggest
adidas для женщин - suggest
adidas кроссовки мужские - suggest
adidas terrex - suggest
adidas superstar - suggest
adidas originals - suggest
adidas terrex кроссовки мужские - suggest
Кеды и кроссовки - category
Дезодоранты - category
adidas - brand
Output in CSV table
The built-in utility $tools.CSVLine allows creating correct table documents, ready for import into Excel or Google Sheets.
Result format:
[% FOREACH i IN results;
tools.CSVline(i.suggest);
END %]
File name:
$datefile.format().csv
Initial text:
Подсказки
In the Result format, Template Toolkit is used to output elements of the $results array in a FOREACH loop.
In the file name of the results, you just need to change the file extension to csv.
To make the "Initial text" option available in the Task Editor, you need to activate "More options". In the "Initial text," we write the names of the columns separated by commas and make the second line empty.
Saving in SQL format
Result format:
[% FOREACH results;
"INSERT INTO serp VALUES('" _ query _ "', '"; suggest _ "', '"; type _ "')\n";
END %]
Example of result:
INSERT INTO serp VALUES('adidas', 'adidas кроссовки', 'suggest')
INSERT INTO serp VALUES('adidas', 'adidas для женщин', 'suggest')
INSERT INTO serp VALUES('adidas', 'adidas кроссовки мужские', 'suggest')
INSERT INTO serp VALUES('adidas', 'adidas terrex', 'suggest')
INSERT INTO serp VALUES('adidas', 'adidas superstar', 'suggest')
INSERT INTO serp VALUES('adidas', 'adidas originals', 'suggest')
INSERT INTO serp VALUES('adidas', 'adidas terrex кроссовки мужские', 'suggest')
INSERT INTO serp VALUES('adidas', 'Кеды и кроссовки', 'category')
INSERT INTO serp VALUES('adidas', 'Дезодоранты', 'category')
INSERT INTO serp VALUES('adidas', 'adidas', 'brand')
Dumping results to JSON
General result format:
[% result.${query} = {
totalcount = p1.totalcount
suggests = []
};
FOREACH i IN p1.results;
result.${query}.suggests.push(i.suggest);
END %]
Initial text:
[% result = {} %]
Final text:
[% result.json %]
Example of result:
{
"adidas": {
"suggests": [
"adidas кроссовки",
"adidas для женщин",
"adidas кроссовки мужские",
"adidas terrex",
"adidas superstar",
"adidas originals",
"adidas terrex кроссовки мужские",
"Кеды и кроссовки",
"Дезодоранты",
"adidas"
],
"totalcount": 10
},
"майки": {
"suggests": [
"майки женские летние",
"майки мужские летние",
"майки детские",
"майки спортивные",
"майки для мальчика",
"майки женские",
"майки женские бельевые турция",
"Майки"
],
"totalcount": 8
}
}
To make the "Initial text" and "Final text" options available in the Task Editor, you need to activate "More options".
Results processing
A-Parser allows processing results directly during scraping, in this section we have listed the most popular cases for the scraper Shop::Wildberries::Suggest
Parse to level option
The option instructs the scraper to substitute the obtained results into the query queue to the specified depth level, for example:
- If the 1st level is specified, the scraper will add to the queries all the results obtained from the original query
- If the 2nd level is specified, the scraper will add to the queries all the results obtained from the original query + all the results obtained from the queries of the 1st level
- and so on.
Since the results will most likely have duplicates, to prevent the scraper from doing extra work (not scraping the same thing), it is recommended to enable Unique queries
Possible settings
| Parameter Name | Default Value | Description |
|---|---|---|
| Follow suggests | All | Choice of the type of suggestions to be substituted into the queries when using Parse to level (All / Suggest / Category / Brand), multiple selection supported |