SE::Google::Position - Checking site positions by keywords in Google
Scraper Overview
The scraper checks the site's positions by keywords in Google. Thanks to the SE::Google::Position scraper, you can automatically check positions in Google search results using your own domain databases. By using the SE::Google::Position scraper, you can easily, accurately, and quickly determine the site's position in Google.
The A-Parser functionality allows you to save the parsing settings of the SE::Google::Position scraper for further use (presets), set up a parsing schedule, and much more. You can use automatic substitution of subqueries from files.
Saving the results is possible in the format and structure you need, thanks to the built-in powerful Template Toolkit that allows you to apply additional logic to the results and output data in various formats, including JSON, SQL, and CSV.
Scraper Use Cases
🔗 Working with API, part 1
An example of working with the A-Parser API using the SE::Google::Position scraper
🔗 Another example of using SE::Google::Position
Determining the site's position by keyword, as in the user's browser
🔗 Review of presentation options
The article considers 4 different presentation options: text, CSV, JSON, HTML
🔗 Finding positions by keywords, how?
Getting to know the SE::Google::Position scraper and checking the position of a keyword in the search results
Collected Data
- Site position and link to the site page
- List of all site positions and links to pages
Capabilities
- All capabilities of the
SE::Google scraper - Automatically stops parsing when the site is found
- Supports subdomain search
- Ability to compare the desired position by domain, by main domain, and by full link
- Collecting positions for multiple domains at once
Use Cases
- Checking positions of your own sites and competitor sites
- Searching for site traffic pages
Queries
As queries, you need to specify the domain of the site being searched and the search query separated by a space, for example:
lenta.ru новости
lenta.ru новости онлайн
If you need to check one site for a list of queries, you can specify the domain in the query format:
lenta.ru $query
Or simply use a list of keywords. To use multiple domains in the query at once, you need to specify a list of domains separated by a comma and a space, followed by the keyword, for example:
lenta.ru,ria.ru,notfound.com новости лента
The results will be recorded in the array $bulkcheck.
The Stop when found option is also supported, parsing will stop if positions are found for all domains.
Query Substitutions
You can use built-in macros to automatically substitute subqueries from files, for example, if we want to check sites/site by a list of keywords, we specify several main queries:
ria.ru
lenta.ru
rbc.ru
yandex.ru
In the query format, we specify a macro for substituting additional words from the file Keywords.txt. This method allows checking the base of sites by a list of keywords and obtaining positions as a result:
$query {subs:Keywords}
This macro will create as many additional queries as there are in the file for each original search query, which will result in [number of original queries (domains)] x [number of queries in the Keywords file] = [total number of queries] as a result of the macro's operation.
Result Output Options
A-Parser supports flexible result formatting thanks to the built-in templating engine Template Toolkit, which allows it to output results in any form, as well as in a structured form, for example, CSV or JSON.
Export of Position List
Obtaining the result in the form of:
desired domain - keyword: position number in the SERP
Result format:
$domain - $key: $position\n
Example of the result:
lenta.ru - новости: 6
lenta.ru - новости онлайн: 7
...
Simultaneous check of multiple domains (bulk check)
Information about all domains during the simultaneous check of multiple domains is contained in the array $bulkcheck.
Result format:
$bulkcheck.format('$domain - $position\n')
Example of the request:
lenta.ru,ria.ru,notfound.com новости лента
Example of the result:
lenta.ru - 1
ria.ru - 3
notfound.com - 0
Links + anchors + snippets with position output
Output of links, anchors, and snippets in a CSV table
Saving related keywords
Competition of keywords
Checking link indexing
Saving in SQL format
Dumping results to JSON
Results Processing
A-Parser allows processing results directly during parsing. In this section, we have provided the most popular use cases for the SE::Google::Position scraper.
Saving domains without zero positions
The example of simultaneous check of multiple domains was taken as a basis (see above in the result output options) and a filter was added.
Add a filter and select the position output variable from the drop-down list. Select the type: >. Then, you need to enter 0 in the Number. With this filter, you can remove all results with a zero position.
Download example
How to import the example into A-Parser
eJx1VNtu2zAM/RVDCNAVyIJ2a4HBDwPSYhk2ZE3Wy1OaB9WiUy2y6Ely2szwv4+S
ZTvtuhdZJA/Jw4tcM8ft1i4NWHCWpaualeHOUnbzJU2/Im4UpOkSrXQSdfI+ueE7
SAQWXGqbPEn3iJVLePIHDCZlhLExK7mxYHzI1ZuRCCIg55VybFwzty+BUuIOjJEC
yCgFyTmagjsiFGBsx1XlYaOHSm2zR8i2kxbx7mjUMiJ+o47E/b0+OmbN/6NXFkqD
z/shcs6VhQOPXCoHhuyRQrpifWpfY1fLehyht63f5wOyJ3THMsBSZkFb1qzXXUQ7
C/x9TeXpJLa+N/pW32LLAwb1jKQrXoROCO7AW7tGHE/cs4/AhQjUuGoz+EEMWe+0
/B3IaSQsXY0EOzNYkMpBCOCV+47dio2C7Autgu/P1ie2bMwsUZ1xIiJeWyS1hTs0
i9AD0tcM9VSpOexADbAQ/6KSStDWTHNy+hYd34Ys/onR9OUdpqKZPxni0EcJ0sXi
x+AlcI4bqlw8UN1KFtKRbC+x0i7ObwtQ9j278j0r0ECfJkaO2ekxlaD9gg0jm5aD
6kUZL8byUpmhzuVmEZe2Q1b6ll7sQl9iUSrwdelKqbFf5uthPaY2jsELA8HXzpch
hS+9e4rMISr7/aalWhpJ63fuCRbUycOsMWTGlbq7nh9a2LBSJCjQjk9MNTYyfDS6
nDorJhkWyX11cvZRhBPC+WG4n52Gs9V8aqEP4Twf3KLZzyijl7BB2lHqU7Puf0D9
r61++zeU1g2twC+7bOG+Xx5MOmq8DYjT5i+HCcki
See also: Results Filters
Link Deduplication
Link Deduplication by Domain
Extracting Domains
Removing Tags from Anchors and Snippets
Filtering Links by Inclusion
Possible Settings
Supports all settings of the SE::Google scraper, as well as additional:
| Parameter Name | Default Value | Description |
|---|---|---|
| Pages count | 1 | Number of search result pages to parse (from 1 to 10) |
| Links per page | 20 | Number of links per page in the search result (10 / 20 / 30 / 50 / 100) |
| Result format | $domain - $key: $position\n | Default output format of the result |
| Stop when found | ☑ | Stop parsing if the domain is found, will not move on to the next pages |
| Match type | Exact domain | Ability to compare the desired position by domain, by top-level domain, and by the full URL (Exact domain / Top level domain / Exact url) |