SE::Google::Suggest - Google Search Suggestion Scraper
Overview of the scraper
Google search suggests scraper by keywords. Thanks to the SE::Google::Suggest scraper, you can automatically collect keyword bases from Google search engine suggests by query. Using the SE::Google::Suggest scraper, you can easily and quickly scrape Google suggests by query based on the selected country, language, or domain.
The Google suggests scraper solves one of the main SEO tasks, namely the fast automated acquisition of an expanded semantic core. Google search suggests allow covering the maximum number of phrases, and in conjunction with the Google Keyword Scraper - 
SE::Google::KeywordPlanner you will get the most complete semantics that will help attract more organic traffic.
Thanks to A-Parser's multi-threaded operation, query processing speed can reach 6000 queries per minute, which on average allows receiving up to 45000-46000 results per minute.
You can use automatic query multiplication, substitution of subqueries from files, iteration of alphanumeric combinations and lists to get the maximum possible number of results. By using results filtering you can immediately clean the result, removing all unnecessary junk (by using negative keywords).
A-Parser's functionality allows saving parsing settings for the SE::Google::Suggest scraper for future use (presets), setting a parsing schedule, and much more.
Results can be saved in the form and structure you need, thanks to the built-in powerful templater Template Toolkit which allows applying additional logic to the results and outputting data in various formats, including JSON, SQL and CSV.
Scraper application use cases
🔗 JS::Google::FromSnippets
Example of work in JS-scrapers with embedded scrapers
Collected data
- Number of results per query
- Suggests for the query
- Suggest type (0 - human, 1 - synthetic)
Capabilities
- Supports selection of search country, domain, language results
- Ability to select the source, to use for collecting suggests (option Client)
- Ability to remove HTML tags from scraping results (option Remove HTML tags)
Use Cases
- Collecting keyword bases
Queries
As queries, you must specify search phrases, for example:
write essay
Football
Waterfall
Speak in english
Cats and dogs
forex
cheap essay
Query substitutions
You can use built-in macros for automatic substitution of subqueries from files, for example, we want to add a list of other words to each query, we specify several main queries:
essay
article
thesis
In the query format, we specify the substitution macro for additional words from the file Keywords.txt, this method allows increasing the variability of queries many times over:
{subs:Keywords} $query
This macro will create as many additional queries as there are in the file for each initial search query, which will total [number of initial queries (domains)] x [number of queries in the Keywords file] = [total number of queries] as a result of the macro operation.
For example, if the file Keywords.txt contains:
buy
cheap
As a result, the substitution macro will turn 3 main queries into 6:
buy essay
cheap essay
buy article
cheap article
buy thesis
cheap thesis
Output results variants
A-Parser supports flexible results formatting thanks to the built-in templater Template Toolkit, which allows it to output results in an arbitrary form, as well as in a structured form, such as CSV or JSON
Exporting suggests list
Result format:
$results.format('$suggest\n')
Result example:
buy essays online
buy essay cheap
buy essay uk
buy essays online no plagiarism
buy essay papers
buy essay online reviews
buy essays reddit
buy essay friend
Output of query, suggests for the query + suggest type
Result format:
$query:\n$results.format('$suggest - $type\n')
Result example:
cheap essay:
cheap essay writing service - 1
cheap essay writing service uk - 1
cheap essay writing service canada - 1
cheap essay writing 24 - 1
cheap essays online - 1
cheap essay writing service reddit - 1
cheap essay writing service australia - 1
cheap essay writing service review - 1
buy essay:
buy essay online - 1
buy essay cheap - 1
buy essay uk - 1
buy essay papers - 1
buy essay online reviews - 1
buy essays reddit - 1
buy essay friend - 1
buy essay online uk - 1
Output to CSV table
The built-in utility $tools.CSVLine allows you to create correct tabular documents, ready for import into Excel or Google Sheets.
General result format:
[% FOREACH i IN p1.results;
tools.CSVline(i.suggest);
END %]
File name:
$datefile.format().csv
Initial text:
Suggests
In the General result format, the Template Toolkit templater is used to output elements of the $results array in a FOREACH loop.
In the results file name, you just need to change the file extension to csv.
To make the "Initial text" option available in the Task Editor, you need to activate "More options". In "Initial text", we write the column names separated by commas and leave the second line empty.
Keyword competition
Saving in SQL format
Result format:
[% FOREACH results; "INSERT INTO serp VALUES('" _ query _ "', '"; suggest _ "', '"; type _ "')\n"; END %]
Result example:
INSERT INTO serp VALUES('cheap essay', 'cheap essay writing service', '1')
INSERT INTO serp VALUES('cheap essay', 'cheap essay writing service uk', '1')
INSERT INTO serp VALUES('cheap essay', 'cheap essay writing service canada', '1')
INSERT INTO serp VALUES('cheap essay', 'cheap essay writing 24', '1')
INSERT INTO serp VALUES('buy essay', 'buy essay online', '1')
INSERT INTO serp VALUES('buy essay', 'buy essay cheap', '1')
INSERT INTO serp VALUES('buy essay', 'buy essay uk', '1')
INSERT INTO serp VALUES('buy essay', 'buy essay papers', '1')
...
Dump results to JSON
Общий формат результата:
[% IF notFirst;
",\n";
ELSE;
notFirst = 1;
END;
obj = {};
obj.totalcount = p1.totalcount;
obj.suggests = [];
FOREACH item IN p1.results;
obj.suggests.push(item.suggest);
END;
obj.json %]
Начальный текст:
[
Конечный текст:
]
Result example:
[{"suggests":["buy essay online","buy essay cheap","buy essay uk","buy essays online no plagiarism","buy essay papers","buy essay online reviews","buy essays reddit","buy essay friend"],"totalcount":8}]
To make the "Initial text" and "Final text" options available in the Task Editor, you need to activate "More options".
Results processing
A-Parser allows processing results directly during scraping, in this section we have presented the most popular use cases for the SE::Google::Suggest scraper
Parse to level option
The option instructs the scraper to substitute the received results into the query queue in depth up to the specified level, for example:
- If the 1st level is specified, the scraper will add all results obtained from the initial query to the queries
- If the 2nd level is specified, the scraper will add all results obtained from the initial query + all results obtained from level 1 queries
- and so on
In simple words - this is the substitution of the received results into the query queue, which in turn allows scraping more results. Since there will likely be duplicates in the results, to prevent the scraper from doing unnecessary work (not scraping the same thing), it is recommended to enable query uniqueness (Unique queries)
Download example
How to import the example into A-Parser
eJx9VE1v2zAM/SsGkcMGZEF72MW3NFiGDlmd1c0pyEGIaVerLHmSnDQw/N9HKo7t
bEFvpvj4+PFIN+CFe3Nriw69g3jbQBW+IYb0Wxx/N6ZQGMdpXRTofPQlWgvrMPIm
UnhAFeG7KCuFMIWKHZY5trdCCZFhLmpFXw34U4WUwlReGu3IJzMyA8WKeenlIFRN
mPu7doQ/oLUyw0tAZkoh9QCG4/E4K0Li2d6U8EFoMcoBm/Qj6OsYihra3W4KNCXq
xS2NLQVPa1Ldz7rR9c5UHPDFkDOXYUSXGLKeRMlsk0x4ZO8sD0SfPs/8OzOILJM8
G6HOGXiuQ9aNln9CNc5bqQvCk2kluqWlrmPwGEj48XSpcAuTYAPR1CH+1zkGYm9r
nIKjapeCasl6Ry6UI4/0aIU3Nunkihsweq5UkGqABfqHWqqM9mCeU9BjF3gbkvzH
0fYdjlORHEdLNfQswXpIfg5RmVmZghrXhtpWspSebLcwtWZt7ujxDbHqx/bEsNJY
7NN0zF12uocKNWs/qDavhqerNq6UuX7cG53LIun26YKs9QsdXaIXhk+H+9K1UqSK
w+dhQ+auk4GNocB/gxchBd/C5brAG6Pcj/RcamUlbeBXLrCkSY6zdpR7odTmeTX2
wLBRfFQ87gidEydguMfC0B5RM+2uv/v+F9LcvP64aUmm3259RnNPjKU3Go4jDejS
27/f2JZm
Filtering results (using negative keywords)
By using negative keywords, it is possible to immediately remove results that you do not need. Similarly, by using the filter, you can leave only those results that contain the desired words.
Download example
How to import the example into A-Parser
eJx9VE1vGjEQ/SvIQkojpSgcetkbQaFqRUMa4EQ4WOywceO1tx4vFBH+e2e83g9a
xG1n5r2ZNx/ro/AS3/HZAYJHkayOogjfIhHzxyT5am2mIUnmZZYB+t7n3hKhlytT
Ym9vXYriThTSITgmry5xCJHCVpaavo7CHwqg3Lbwyhpmq5TMkGIKO9Dk2UldEmZ4
f+rgd+CcSqEmpDaXyrRgsd/vB1koPNjYXFyhZp0aYjm/Bn3rQsF0oVulPTgK07S4
tWQVv7gnjJ2v7yJuUZEcGOtz6TdvnbwO0lT5D6U1ZFJ/YCGNQgZUMyLENYlb6yhh
FNEm7Ucxgyr+6aYfNb2+mptbcVqva+U4CQjmFMNBXH4TnMsdLGzVL7TuCVlPMg+V
UumBo3Wp24H/wxkkt0UNSF1V4ANpqy6N+h2UonfKZIQn0ynAiaP1JcJDSMLOQ61w
JfrB5rmWgf+z4ojEuxJo7qR2IklL2gS2UiNFFC1Beutm8e6So7BmpHW4uRYW0j+U
Sqd00KMtkb5F4mXI7L8cp6bDbila2t6RhlpoMB5mP1pSaqc2o76Npa61ypUnG8e2
NLyae3K+AxTN1J4YllsHTZUoLxanH7oAwwfSLm1UtK6zLs4Wc+7cWLNV2SweXY0s
zYJejZkZ27zQwG2ZUmtaCsJLeyAjjFtgoxX4L3kcSvA/Xb8Swlur8fu8klo4RQf4
hQXmNMhu1ZhyI7Vevky7EdEeFD8OPO4eIMqDYLiHzNIZUTOndfN+NW/g8eIrlhxP
tKZf+FyhuSfGko+Gg+FHHZ7+As4s2Yc=
See also: Results Filters
Possible settings
| Parameter name | Default value | Description |
|---|---|---|
| Client | Chrome omnibox | Selection of the source to use for collecting suggests (Search page / Chrome omnibox) |
| Follow suggests | Human | Selection of the type of suggests to substitute into queries when using Parser to level (All / Synthetic / Human) |
| Google domain | www.google.com | Domain selection |
| Search from country | Global | Selection of the country from which the search is performed |
| Interface language | English | Interface language selection |
| Remove HTML tags | ☑ | Removing HTML tags |