SE::Google - Google Search Results Scraper
Overview of the scraper
The Google search results scraper is one of the most requested, thanks to which you can get huge databases of links ready for further use. You can use queries in the same way as you enter them into Google, including search operators (inurl, intitle, etc.).
The Google scraper supports automatic query multiplication; you can be sure to get the maximum number of results from the SERP. A-Parser can also automatically navigate through related queries to the specified depth. Thanks to A-Parser's multithreaded operation, query processing speed can reach 3000-7000 queries per minute, which on average allows you to get up to 500000 links per minute
A-Parser's functionality allows you to save scraping settings for future use (presets), set a scraping schedule, and much more. You can use automatic query multiplication, substituting subqueries from files, iterating over alpha-numeric combinations, and lists to get the maximum possible number of results.
Saving results is possible in the form and structure you need, thanks to the built-in powerful templater Template Toolkit which allows you to apply additional logic to the results and output data in various formats, including JSON, SQL and CSV.
Scraper use cases
🔗 Domain scraping
Scraping thematic domains by keyword phrase from Google and retrieving various parameters for the domains
🔗 Google News scraping
This preset scrapes Google news by search query, and collects the dates of these news
🔗 Indexing check
The preset checks the indexing of site pages in Google, going through the list of specified links
🔗 Competition assessment
The preset determines competition in the Google search engine by keywords
🔗 Top 3 SERP scraping
The preset saves the first top three snippets of Google SERP
🔗 Questions and Answers
Scraper collecting questions and answers from the People Also Ask section
Collected data
- Links, anchors, and snippets from the search results, as well as the date from the snippet (if available)
- Information about the flags of each result is also collected; the following flags are currently supported: Date, AMP, Image Preview, Video, Rich snippet, Featured snippet
- Presence and content of ad blocks, as well as their position on the page
- Number of results per query (competition)
- List of related keywords
- Presence of additional blocks on the page: product carousel, video, etc.
- The scraper also collects the following additional data:
- Presence of a typo in the query and the corrected query
- Geo-location determined by Google
- Presence of AMP pages
- List of People also ask: questions, answers, source links, their anchors and media links (enabled by separate option Parse People also ask)
- AI answer (AI overview), its type and list of sources
Capabilities
The Google search engine scraper has many capabilities and settings:
- support for all search operators Google (site:, inurl: etc.)
- specifying the SERP size (10, 20, 30, 50 or 100 results) and specifying the number of pages(from 1 to 10), with maximum settings Google returns from 300 to 500 results per query, A-Parser easily bypasses this limitation due to query multiplication
- ability to automatically navigate through related keywords
- specifying the language and country of the SERP, option to select a local Google domain, as well as the SERP interface language
- option to specify the geo-location, which allows you to get an accurate local SERP for any place on Earth
- choice between desktop or mobile display
- option to select the type of SERP, in addition to the main organic SERP, the Google scraper can collect news, book, or video SERP
- if necessary, you can connect automatic ReCaptcha2 recognition via recognition services or XEvil/CapMonster
- supports specifying the SERP time (all time or specific interval from 24 hours to a year)
- ability to disable the Google filter to hide similar results (filter=)
- ability to specify whether to scrape the SERP if Google reports that nothing was found for the specified query and suggests results for a similar query
- option to set the number of People also ask, that the scraper should collect by clicking deeper into each question
- ability to collect tags
The following scrapers are based on the Google scraper:
SE::Google::Position - determining the positions of any sites in the SERP by a list of queries
SE::Google::Compromised - checks domains for purity from Google's point of view, can detect hacked and phishing sites
SE::Google::TrustCheck - checks for Google Trust in the site
Use Cases
- Collecting link databases - for XRumer, AllSubmitter, GSA Ranker, etc.
- A full SERP dump, including links, anchors, snippets, ad blocks, and other information, allows for deep analysis for SEO specialists and marketers
- Competition assessment for keywords
- Competition assessment in PPC (advertising) SERP
- Finding backlinks and site mentions
- Checking site indexing
- Finding vulnerable sites
- Any other options involving obtaining search results for an unlimited number of queries
Queries
As queries, you must specify search phrases, exactly as if you were entering them directly into the Google search form, for example:
car purchase
windows in moscow
site:https://lenta.ru
inurl:guestbook
Query substitutions
You can use built-in macros for multiplying queries, for example, we want to get a very large database of forums, we will specify a few main queries in different languages:
forum
forum
foro
论坛
In the query format, we will specify the iteration of characters from a to zzzz, this method allows you to maximally rotate the search results and get many new unique results:
$query {az:a:zzzz}
This macro will create 475254 additional queries for each initial search query, which in total will give 4 x 475254 = 1901016 search queries, the number is impressive, but this is not a problem for A-Parser. At a speed of 2000 queries per minute, such a task will be processed in just 16 hours.
Using operators
You can use search operators in the query format, so it will be automatically added to each query in your list:
inurl:$query
Output results examples
A-Parser supports flexible result formatting thanks to the built-in templater Template Toolkit, which allows it to output results in an arbitrary form, as well as structured, such as CSV or JSON
Exporting link list
Result format:
$serp.format('$link\n')
Result example:
https://www.weforum.org/open-forum/
https://www.weforum.org/about/world-economic-forum/
https://www.merriam-webster.com/dictionary/forum
https://en.wikipedia.org/wiki/Forum
https://dictionary.cambridge.org/dictionary/english/forum
https://www.collinsdictionary.com/dictionary/english/forum
https://www.linkedin.com/company/world-economic-forum
https://docs.moodle.org/en/Forum_activity
https://wordpress.org/support/forums/
https://www.facebook.com/worldeconomicforum/
...
Link + anchor + snippet with position output
Result format:
[% FOREACH item IN serp; loop.count _ ' - ' _ item.link _ ' - ' _ item.anchor _ ' - ' _ item.snippet _ "\n"; END %]
Result example:
1 - https://ru.wikipedia.org/wiki/%D0%A4%D0%BE%D1%80%D1%83%D0%BC - Форум — Википедия - <em>Фо́рум</em> (лат. forum — арх. преддверие гробницы; площадка в давильне для подлежащего обработке винограда; рыночная площадь, городской рынок; ...
2 - https://ru.wikipedia.org/wiki/%D0%A4%D0%BE%D1%80%D1%83%D0%BC_(%D0%BC%D0%B5%D1%80%D0%BE%D0%BF%D1%80%D0%B8%D1%8F%D1%82%D0%B8%D0%B5) - Форум (мероприятие) — Википедия - <em>Форум</em> — мероприятие, проводимое для обозначения или решения каких-<wbr>либо в достаточной степени глобальных проблем. Это понятие встречается в ...
3 - https://support.google.com/googleplay/community?hl=ru - Добро пожаловать на справочный форум сообщества ... - Добро пожаловать на справочный <em>форум</em> сообщества Google Play. Избранные записи. Просмотреть все интересные записи · Нужна помощь с игрой?
4 - https://support.google.com/mail/community?hl=en - Gmail Community - Google Support - Welcome to the Gmail Help Community · Featured posts · Categories.
5 - https://www.weforum.org/ - The World Economic Forum - The World Economic Forum is an independent international organization committed to improving the state of the world by engaging business, political, academic ...
6 - https://www.kunena.org/ - Home - Kunena - To Speak! Next Generation Forum ... - Kunena! - To Speak! Next Generation Forum Component for Joomla.
7 - https://forum.adguard.com/index.php - AdGuard Forum - <em>Форум</em> бета тестеров. Пишем сюда отчеты о багах бета-версий. Threads: 355. Messages: 11.6K. Sub-forums: Комментарии к релизам бета-версий ...
8 - https://www.sofiaforum.bg/ - Софийски Форум за Сигурност: Платформа за обсъждане ... - Софийски <em>Форум</em> за Сигурност / Sofia Security Forum.
9 - https://forum.keenetic.net/ - Forums - Keenetic Community - Keenetic fan club. A place to meet software developers, get the latest updates, and share experience.
10 - https://forum.euroaion.com/ - Perfect quality European private server of Aion - EuroAion.com - Perfect quality European private server of Aion!
...
Outputting links, anchors, and snippets to a CSV table
The built-in utility $tools.CSVLine allows you to create correct tabular documents ready for import into Excel or Google Sheets.
Overall result format:
[% FOREACH i IN p1.serp; tools.CSVline(i.link, i.anchor, i.snippet); END %]
File name:
$datefile.format().csv
Initial text:
Link,Anchor,Snippet
Result example:
Link,Anchor,Snippet
https://ru.wikipedia.org/wiki/%D0%A4%D0%BE%D1%80%D1%83%D0%BC,"Форум — Википедия",
https://en.wikipedia.org/wiki/Forum,"Forum - Wikipedia","<em>Forum</em> (plural forums or fora) may refer to: Contents. 1 Common uses; 2 Places. 2.1 Natural features; 2.2 Populated places. 3 Arts and entertainment; 4 Media."
https://www.weforum.org/,"The World Economic Forum","The World Economic <em>Forum</em> is an independent international organization committed to improving the state of the world by engaging business, political, academic ..."
https://support.google.com/webmasters/community?hl=ru,"Добро пожаловать на справочный форум сообщества ...","Добро пожаловать на справочный <em>форум</em> сообщества Центр Google Поиска. Избранные записи. Просмотреть все интересные записи · Ответы на ..."
https://support.google.com/chrome/community?hl=ru,"Добро пожаловать на справочный форум сообщества ...","Добро пожаловать на справочный <em>форум</em> сообщества Google Chrome. Избранные ..."
...
The Overall result format uses the Template Toolkit templater to output the $serp array in a FOREACH loop.
In the results file name, you just need to change the file extension to csv.
For the "Initial text" option to be available in the Task Editor, you must activate "More options". In "Initial text", write the column names separated by commas and leave the second line empty.
Ad block output
Result format:
$ads.format('$link - $anchor - $snippet\n')
Result example:
https://www.rentalcars.com/ - Rent a Car Worldwide - Best Prices Online Guaranteed - Secure Your <em>Car Hire</em> Today. The Best Price Guaranteed. Book at Over 53,000 Locations. Search, Compare and Save Using the World's Biggest Online <em>Car Rental</em> Service.
https://www.kayak.com/United-States-Car-Rentals.253.crc.html - United States from $9/day - Search for Rental Cars on Kayak - Find and Compare Great <em>Car</em> Deals in USA. Book with Confidence on KAYAK®!
https://www.discovercars.com/ - -70% Worldwide Car Rental - Rent Your Car in 5 Minutes - <em>Car rental</em> prices are rising, but if you act fast, you can get a good deal. Don’t stress! We...
https://www.economybookings.com/ - Rent a Car for Summer Holidays - Car Rentals for the Best Price - Theft protection and Third Party liability part of a great deal. Free Mileage included.
...
Saving related keywords
Result format:
$related.format('$key\n')
Result example:
test <b>speed</b>
<b>net speed</b> test
<b>google speed</b> test
<b>fast speed</b> test
<b>ping</b> test
<b>xfinity speed</b> test
<b>speed</b> test <b>mobile</b>
test <b>my</b>
...
To automatically remove HTML tags in the result, you need to use the Results Builder, select the array $related and apply Remove HTML tags.
Keyword competition
Result format:
$query - $totalcount\n
Result example:
speed test mobile - 1080000000
test score - 4020000000
net speed test - 1210000000
fast speed test - 2150000000
speed test - 2500000000
test match - 4160000000
ping test - 425000000
google speed test - 1870000000
Detecting misspelled keywords
Result format:
$query - $misspell\n
Result example:
spead test - 1
test match - 0
speed test - 0
temst match - 1
Checking link indexing
Query format:
site:$query
Result format:
$query.orig - $totalcount\n
Result example:
https://a-parser.com/pages/buy - 2
https://a-parser.com/wiki/parsers - 4
https://a-parser.com/resources - 883
https://trjkjfkdf.bg.ky - none
https://a-parser.com/forum - 371
To check the indexing of links, substitute the corresponding operator in the Query Format: site:.
The result format is output as "original url - number of pages in the index".
As a result, we get the address of the pages and their number in the search engine's index.
If the page is missing, the result will be: none.
Saving in SQL format
Result format:
[% FOREACH serp; "INSERT INTO serp VALUES('" _ query _ "', '"; link _ "', '"; anchor _ "')\n"; END %]
Result example:
INSERT INTO serp VALUES('test', 'https://www.speedtest.net/', 'Speedtest by Ookla - The Global Broadband Speed Test')
INSERT INTO serp VALUES('test', 'https://fast.com/', 'Fast.com: Internet Speed Test')
INSERT INTO serp VALUES('test', 'https://www.business-standard.com/article/sports/ind-vs-aus-live-score-4th-day-5-india-vs-australia-live-cricket-score-online-brisbane-weather-121011900103_1.html', 'IND vs AUS 4th Test highlights: India creates history, wins ...')
INSERT INTO serp VALUES('test', 'https://www.test.com/', 'Find online tests, practice test, and test creation software | Test ...')
INSERT INTO serp VALUES('test', 'https://www.espncricinfo.com/series/india-in-australia-2020-21-1223867/australia-vs-india-4th-test-1223872/match-report-4', 'Recent Match Report - Australia vs India 4th Test 2020 ...')
INSERT INTO serp VALUES('test', 'https://www.icc-cricket.com/world-test-championship/standings', 'World Test Championship (2019-2021) Points Table - Live ...')
INSERT INTO serp VALUES('test', 'https://www.icc-cricket.com/rankings/mens/team-rankings/test', 'ICC Test Match Team Rankings International Cricket Council')
INSERT INTO serp VALUES('test', 'https://projectstream.google.com/speedtest', 'Speedtest - Google')
INSERT INTO serp VALUES('test', 'https://www.google.com/search?hl=en&q=Software+Testing&stick=H4sIAAAAAAAAAONgecQ4g5Fb4OWPe8JSfYyT1py8xtjOyMUVnJFf7ppXkllSKaTCxQZlSXHxSHHo5-obmJul5GkwSHFxwXlKwUbuuy5NO8fmKMgABGJm_g5SmlpCXOyexT75yYk5ggpvuB68mfLeXkuYiyMksSI_Lz-3UtCBgcHhx__39kqcnEBND7aoddhrMTTtW3GIjYWDUYCBZxGrQHB-Wkl5YlGqQkhqcUlmXjoAS5B1P7EAAAA&sa=X&ved=2ahUKEwiW-rnmlajuAhWpAGMBHR-JAv4Q6RMwHXoECDQQBQ', '')
...
Dump results to JSON
Общий формат результата:
[% IF notFirst;
",\n";
ELSE;
notFirst = 1;
END;
obj = {};
obj.totalcount = p1.totalcount;
obj.links = [];
FOREACH item IN p1.serp;
obj.links.push(item.link);
END;
obj.json %]
Начальный текст:
[
Конечный текст:
]
Result example:
[{"totalcount":"6450000000","links":["https://www.speedtest.net/","https://fast.com/","https://projectstream.google.com/speedtest","https://www.test.com/","https://www.speakeasy.net/speedtest/","https://www.att.com/support/speedtest/","https://speedtest.xfinity.com/","https://developers.google.com/speed/pagespeed/insights/","https://www.espncricinfo.com/series/india-in-australia-2020-21-1223867/australia-vs-india-4th-test-1223872/match-report-4","https://nasional.tempo.co/read/1424570/listyo-sigit-temui-ahy-menjelang-fit-and-profer-test-calon-kapolri","https://www.google.com/search?hl=en&q=Test+Assessment&stick=H4sIAAAAAAAAAONgecRYyC3w8sc9YamMSWtOXmNM4uIKzsgvd80rySypFNLiYoOyFLj4pbj10_UNjQyzKsvyzDQYpHi5kAWUNIxkdl2ado5NTJABCMTKAhyUODmBLIVA-wX2WgxN-1YcYmPhYBRg4FnEyh-SWlyi4FhcnFpcnJuaVwIAwEAP9ogAAAA&sa=X&ved=2ahUKEwj17MzXmajuAhW8CWMBHRlzBP4Q6RMwDHoECBEQBQ"]}]
For the "Initial text" and "Final text" options to be available in the Task Editor, you must activate "More options".
Results processing
A-Parser allows you to process results directly during scraping, in this section we have provided the most popular use cases for the Google scraper
Link deduplication
Add deduplication and select $serp.$i.link - Link from the drop-down list.
Download example
How to import an example into A-Parser
eJx9VE1v2zAM/SsFkcMGBEFy2MW3NFiKDVnTNekpyEGNaUOLLGmSnDUw/N9H+ktO
N/Rmko+PfCTlCoLwZ//k0GPwkBwqsM03JLD7miQPxuQK7zZSn/3di5a/S4QpWOE8
OoYfRigKpJiJUgWYVhCuFonEXNA5mXJQpmRbZ96uDoOT6Ml3Eapk2GI+n0P9QZrI
8WRKHWLO4gO44n4tOk4bZcxHKWUvhuRyy8kBSJMlByfDcdoh9i3cU8c6h977oMyr
UJAEV2J9PPYsfm1cIXh4E7uYdZMcgjtxwb2hYCZVrOzXZD2KgqtMUhGQo7OsIfr0
eRbemEGkqQzSaKHaCjz7WLVbTALaEJY+ebprZwpyBWwI2HntuzvApLGjyp9tDiSZ
UB6n4KnVtaBG0vcRGdCJYNzWcj/kr8DopVIbvKCKsIb/vpQqpUNZZpT0rUv8P2T7
D0c9yBuXokX/cdTDwNJY99sfMSs1G5OT8vS1WWYhA9l+1VxPAnNynhHtMLNHnllh
HA5lOuauOr0Ni5qvKq5saaPrRsbNWm6dJ6MzmW+7S+2Rpd7TA9zqlSmsQtalS6Vo
LR6f43ksfbcGNmKD75NXTQmW3r9DCMYo/33XtmqdpPP7wg0WNMlx1Y7yJJR6ed6M
IxBPqjknz7QnutPc0AWRivo4/BGG/0g1/i8kVU1r+eWfWhBrYAj5aBieZs6P+S/t
6pW4
See also: Results deduplication
Link deduplication by domain
Add deduplication and select $serp.$i.link - Link from the drop-down list. Select deduplication type: Domain.
Download example
How to import an example into A-Parser
eJx9VE2P2jAQ/SvI4tBKaAWHXnJj6bJqRZftwp4QB0MmyMXxuLZDF0X89844IQ7b
am+ZmffefDq1CNIf/bMDD8GLbFMLG79FJlYPWfaIeNAwWChz9INXo35XMNidB1+x
lMqIkbDSeXBM3PTwFMihkJUOYlSLcLZAcngC51TOQZWTbR2+nR0Ep8CT7yR1xbDJ
eDwWlw9o8gB7rExInMkHcM2VW3BM6zHGPUoV26IgNc4lZxtBPVlyMFlsRy1i3cDz
a++N91HjTmqRBVfBZbu9qvg5ulLyGId2ctfOtAuu5AnWSMFC6ZTZz8l6kiVnGeYy
AEfviij06fNdeGMFmecqKDRSNxl49ilrsyLiGyQsffJ05w5LcgWIAuw8X6vbiGG0
U5c/G47ICqk9jISnUueSCsnfR1QAJwO6peV6yF8LNFOtF3ACnWBR/75SOqdDmRZE
+tYS/w9Z/qNx6drrp6JF/3FUQ6cSrfvlj8TKcYEHXtkuLrNUgWw/i9eTiTE5jwC2
m9kTz6xEB12aVrnNTq/EguGrSiub2uS6aeNmLbfOPZpCHZbtpV6RlVnTU1yaGZZW
A/dlKq1pLR5e0nlMfbsGNlKB78mzmIJbv75DERC1/75qSrVO0fl94QJLmmQ/ayu5
l1q/viz6EZFOKp6TZ9k93ekB6YKoi8u2+yN0f5S6/1/I6gut5Zd/bkDcA0PIR8Pw
NHN+zH8BRVyZDA==
See also: Results deduplication
Extracting domains
Add Results Builder and select the source from the drop-down list: $p1.serp.$i.link - Link. Select type: Extract Domain.
Download example
How to import an example into A-Parser
eJx9VE1v2zAM/SuFkMMGBIFz2MW3NGuGDVnTNe0p6EGzaUOLLGqSnCUw8t9HKort
dkNv5scj+cgndyJIv/cPDjwEL/JdJ2z8FrnY3uX5F8Raw83dMThZhJvP2EhlxFRY
6Tw4BuxGeRQooZKtDmLaiXCyQGXwAM6pkoOqJNs6PJ4cBKfAk+8gdctp8yzLxPkd
mKyhwNaEATN/J10rs/cWHMNGiOz88jIVxJBm9Ct0jWSmEzufJdp9cCsP8IQUrFQk
dsWQdS8bbjkpZQCOzqpY6MPHWThyBVmWKig0Ul868JqGrs9G/Y6MDVIuffIiVg4b
cgWIBdh5uk63E5NoCyrRRuyPC0bkldQepsLTqCtJg5RvIyqAkwHdxvI85O8EmoXW
aziAHtJi/dtW6ZJuuqgI9DUB/5+y+afGuac3bkU3+eNohr5KtG433wdUiWusiXn5
k3hr1ahAtl/GQ+ciI+cewPY7u+edNeigb5Mqp+4kZAuGBTCcbGEH1ysar84ycnbC
Y+sK6rfLpjtBQrdxOLMXrJ6kOLi8if5JSOfkifwpPWBSoTgTpkBTqXqTRHrt3Jon
en0bs8TGauA9mVZrOrOHx0FuC5/OysZA+C14GVvwKq9PkGZA7b9tL9StUzTSJybc
0GXGXVPJQmr9/LgeR8Qg0ShPz2UL0n2NpEhiweTSz6D/iXTjX0LenenMv/zDJYk5
cAr5aBmebsjv+C86oZM/
See also: Results Builder
Removing tags from anchors and snippets
Add Results Builder and select the source from the drop-down list: $p1.serp.$i.anchor - Anchor. Select type: Remove HTML tags.
Add Results Builder again and select the source from the drop-down list: $p1.serp.$i.snippet - Snippet. Select type: Remove HTML tags.
Download example
How to import an example into A-Parser
eJyVVD1v2zAQ/SsC4aEFBEMeumhzjLpp4cSp7UxGBlY6qawpkiUpN4bg/947mpaU
NAjQjby79+7rkR3z3B3cgwUH3rF83zETzixn2895/kXrWkKygUYfIbnd3a0Sz2uX
VFY3yVwVP7V1CVdlslXCGKJImeHWgSWy/YgDHSVUvJWepR3zJwOYAkmtFSU5RYl3
Y/XzyYK3AojpyGVLYbMsy9j5HRivodCt8gNm9k64FOrgDFiCjRDZ+ekpZdg91uiW
2jacpjAxs2kcSe/c8iPsNDorERq7YvB2zxtKOSm5B/JOq0D04ePUPxMDL0vhhVZc
XjLQmIasj0r8Dh0rjbF4pEEscdZo8hAIyHi6Vrdnk3BnSNEG7PcLhuUVlw5S5rDU
JcdCytce4cFyr+3aUD1o75hWcylXcAQ5hAX+m1bIEnc6rxD0NQLfDln/w3Hu2xun
wp38sVhDzxJuN+u7AVXqla6x8/IH9i1FIzze3SIsOmcZGg8App/ZPc2s0Rb6NJE5
ZkeRG1AkgGFlczOYXrTxYi0jY8ecbm2B+fZZumcodENbDS+BkX6i5mx4Mbe+keS2
lp/QGKM9SSdCSKZvMbrLe/ovyivmjJhCq0rU66j8azut2uFzX6uFbowEGr5qpUTt
ONgMGp67qBW6DFN8DV6EFLSf67vGMrR037aXeRorsKpPaSx8nDVSFlzKx81q7GGD
7oPmHdEW+JhqjTLHLqi5+MP0v1Y3/mfy7oza+eUeLkHUA4WgDYfhUBj0OfwFH/O5
UQ==
The Results Builder can be added as many times as you need.
See also: Results Builder
Filtering links by inclusion
Add a filter and select from the drop-down list: $serp.$i.link - Link. Select type: Contains string. Next, you need to enter the filtering criterion in String.
Download example
How to import an example into A-Parser
eJx9VE1v2kAQ/StoxSGVEIJDL74RVKpWNKSBnBCHDR5bG9Y72901DbL83zuzNrZJ
qtw8H+/Nm491JYL0J//owEPwItlXwsZvkYjttyT5jphrGK2UDuCUyUcvl5EP8UuZ
kVbmJCbCSufBMXo/AFEghUyWOohJJcLFAnHiGZxTKQdVSrZ1+HZxQIzgyXeWuuS0
+Ww2E/UnMJnDEUsTesz8k3TW6S04hg0QswEkix1SkLpnycleUE+WHLHJw6TN2DXp
RzRBKjOQPD1iQSbaoNCQ7cF4UR8OV0a/QldInuvYzqftkLvgVp5hh40O6N00d3iQ
BfOPUxmAo9MsEt19mYY3ZpBpqrim1E0F3kNf9dmoP1GfQcqlT570ypHWRASIBOy8
XNXtxTja3HEZsb8bjEgyqT1MhCepK0lC0vcRRfORAd0mzoD8lUCz0HoNZ9B9WuS/
L5VO6WgWGYF+tMD/p2w+cNRde8NStPS/jjR0LNG63/zqUSmuMafO05e42EIFsv0y
XlIiZuQ8AdhuZg88swIddGVa5rY6PRsLhi+sX9nC9q6bNm7Wcuuka8pUvmmv9ppZ
mh29zY1ZYmE1cF+m1JrW4uGpP4+Fb9fARi/wPXgZS3Dr1zcpAqL2P7eNVOsUnd9X
FljQJIdVW8qj1Pr5aT2MiP6k4jl5pj3SneZIF0Rd1Ifu79D9YqrhPyKpalrLq39s
krgHTiEfDcPHlzSv/wHtZp3U
See also: Results Filters
Available settings
Regional settings
Google domain - the Google domain used, default is google.com
Results language - search for pages in the selected language, in the browser this corresponds to the Advanced Search option -> Advanced Settings -> Search in (url lr parameter). By default, it is not set, which means automatic determination based on IP
Spoiler: Screenshot
Interface language - language of Google products, in the browser this is Languages -> Interface Language (url hl parameter). English is selected by default
Spoiler: Screenshot
Search from country - selection of the search region, in the browser this is Languages -> Search Region (url gl parameter). By default, it is not set, which means automatic determination based on IP
Spoiler: Screenshot
Narrow results by region - search for pages created in a specific country, in the browser this is Advanced Search -> Advanced Settings -> Country (url cr parameter). By default, it is not set, which means disabling this option
Spoiler: Screenshot
Location (city) - exact search location, in the browser it is determined automatically based on the user's location. By default, it is not set, which means automatic determination based on the request IP
Spoiler: Screenshot
All regional settings influence the SERP to some extent.
| Parameter Name | Default Value | Description |
|---|---|---|
| Device | Desktop | Choosing desktop or mobile SERP: Desktop / Mobile |
| Pages count | 5 | Number of pages to scrape (from 1 to 100) |
| Serp type | Default (All) | Specifies whether to scrape from the main page, news, or blogs (Books, News, Videos) |
| Hide omitted results | ☑ | Specifies whether to hide omitted results (filter= parameter) |
| Serp time | Anytime | SERP time (time-dependent search, tbs= parameter, possible values: Past 1 hour, Past 24 hours, Past week, Past month, Past year) |
| Parse not found | ☑ | Specifies whether to scrape the SERP if Google reported that nothing was found for the specified query and suggested results for a different query |
| Disable autocorrect | ☐ | Allows disabling Google's autocorrection and scraping the SERP exactly for the specified query |
| Exact match | ☐ | Corresponds to the search engine option "Exact match". Attention, this option overwrites the value of the Serp time parameter (similar to how these options work in a browser). |
| Safe search | Blur | Option to enable "Safe Search" |
| Google domain | www.google.com | Google domain for scraping, all domains are supported (www.google.ac, www.google.com.af, www.google.co.ck, etc.) |
| Narrow results by region | Any region | Option to narrow the search to a specific country |
| Results language | Auto (Based on IP) | Selecting the results language (lr= parameter) |
| Search from country | Auto (Based on IP) | Selecting the country from which the search is performed (geo-dependent search, gl= parameter) |
| Interface language | English | Option to select the Google interface language for maximum identity of results between the scraper and the browser |
| Location (city) | Search by city, region. Cities can be specified as novosibirsk, russia; a full list of locations can be found in Geotargets (copy - you need to use the value from the Canonical Name column). It is also necessary to set the correct Google domain | |
| Util::ReCaptcha2 preset | default | Specifies whether to use  Util::ReCaptcha2 to bypass reCaptcha |
| Util::AntiGate preset | default | Specifies whether to use  Util::AntiGate to bypass graphic captchas |
| ReCaptcha2 retries | 3 | Number of attempts to send the reCaptcha answer the specified number of times without changing the proxy |
| ReCaptcha2 pass proxy | ☐ | Allows passing the proxy (used in the request to Google) and cookies (received in the response from Google) to the ReCaptcha recognition service |
| Use sessions | ☑ | Saves good sessions, which allows scraping even faster, getting fewer errors. |
| Don't take session | ☐ | Option not to use saved good sessions |
| Additional headers | Allows specifying any custom headers | |
| PAA questions count | 0 | Maximum number of questions-answers (People also ask) per query that the scraper should collect |
| Empty totalcount is error | ☐ | When this parameter is enabled, the query will be considered unsuccessful if the value for $totalcount is missing, and repeat attempts will be made accordingly |
| Redirect browser max pages | 10 | Number of browser pages used to bypass protection that checks for enabled JavaScript |