SE::DuckDuckGo::Position - Checking website positions by keywords in DuckDuckGo
Scraper Overview
A scraper for checking website positions by keywords in DuckDuckGo. With the SE::DuckDuckGo::Position scraper, you can automatically check positions in DuckDuckGo using your own domain databases. Using the SE::DuckDuckGo::Position scraper, you can easily, accurately, and quickly determine the website's position in DuckDuckGo.
The functionality of A-Parser allows you to save the parsing settings of the SE::DuckDuckGo::Position scraper for further use (presets), set up a parsing schedule, and much more. You can use automatic substitution of subqueries from files.
Saving the results is possible in the form and structure you need, thanks to the built-in powerful templating engine Template Toolkit which allows you to apply additional logic to the results and output data in various formats, including JSON, SQL, and CSV.
Collected Data
- Website position and link to the website page
- List of all website positions and links to the pages
Capabilities
- All capabilities of the scraper
SE::DuckDuckGo - Automatically stops parsing when the website is found
- Supports subdomain search
- Ability to compare the desired position by domain, by main domain, and by full link
- Collecting positions for multiple domains
Use Cases
- Checking positions of your websites and competitors' websites
- Finding traffic pages of the website
Queries
As queries, you need to specify the domain of the website being searched and the search query separated by a space, for example:
lenta.ru новости
lenta.ru новости онлайн
ria.ru новости
a-parser.com А-Парсер
If it is necessary to check one website for a list of queries, you can specify the domain in the query format:
lenta.ru $query
Or simply use a list of keywords. To use multiple domains in the query, you need to specify a list of domains separated by a comma and a space, followed by the keyword, for example:
lenta.ru,ria.ru,notfound.com новости лента
The results will be recorded in the array $bulkcheck.
The Stop when found option is also supported, parsing will stop if positions are found for all domains.
Query Substitutions
You can use built-in macros for automatic substitution of subqueries from files, for example, if we want to check websites/website based on the key base, we will specify several main queries:
ria.ru
lenta.ru
rbc.ru
yandex.ru
In the query format, we will specify a macro for substituting additional words from the Keywords.txt file, this method allows you to check the website base based on the key base and get positions as a result:
$query {subs:Keywords}
This macro will create as many additional queries as there are in the file for each original search query, which will result in [number of original queries (domains)] x [number of queries in the Keywords file] = [total number of queries] as a result of the macro's operation.
Output Results Examples
A-Parser supports flexible formatting of results thanks to the built-in template toolkit, which allows it to output results in any form, as well as in a structured form, for example CSV or JSON.
Export of position list
Getting the result in the form of:
desired domain - key: position number in the issue
Result format:
$domain - $key: $position\n
Example result:
lenta.ru - новости онлайн: 13
lenta.ru - новости: 26
ria.ru - новости: 1
a-parser.com - А-Парсер: 1
...
Simultaneous checking of multiple domains (bulk check)
Information about all domains during simultaneous checking of multiple domains is contained in the array $bulkcheck.
Result format:
$bulkcheck.format('$domain - $position\n')
Request example:
lenta.ru,ria.ru,notfound.com новости лента
Example result:
lenta.ru - 2
ria.ru - 6
notfound.com - 0
Links + anchors + snippets with position output
Output of links, anchors, and snippets in a CSV table
Saving related keywords
Keyword competition
Checking link indexing
Saving in SQL format
Dumping results to JSON
Results processing
A-Parser allows processing results directly during parsing, in this section we have provided the most popular use cases for the SE::DuckDuckGo::Position scraper
Saving domains without zero positions
The example of simultaneous checking of multiple domains was taken as a basis (see above in the result output options) and a filter was added.
Add a filter and select the position output variable from the drop-down list. Select type: >. Then you need to enter 0 in Number. With this filter, you can remove all results with a zero position.
Download example
How to import the example into A-Parser
eJx1VNtu2zAM/RVDCNAVyIJ2a4HBDwPSbhk2dE3Wy1OSB8WiWy2y6OmSXoL8+yhZ
ttNufYgi3g55SMpb5rhd25kBC86yfL5ldbyznF1/zfMvvliH3zfM8xla6STq7H12
zTeQCay41DZ7kO4evct49gwGszq5sSGrubFgAuz8TTRyE1Byrxwbbpl7qoFS4waM
kQLIKAXJ3kJt8PGJ5A1XnlxKrizs3o4o0VTcEZUI3EaxwcqrdXEPxXrUeLw7GDQ8
iNWgLX2x0AeHbA+9lMqBIZwEmM9ZBxR4tlyWw+R608R93kt9RHeso1vOLGjLdstl
i2gnsZpQYX08SiPojKHdN9jUAb16QtIlryIvwR0Ea0vrcOQeAwIXIpbGVZMhDKPP
eqvln1icRvKlq5FgJwYrUjmIAEH51FY3Z4MoB6I+xv5qYtJAhsxSqRNOhYjXFklt
4Q7NNPaA9FuGeqzUBWxA9W4R/8xLJWhzxiUFfU+B/3eZ/oOx6+jtp6L9eDBUQ4cS
pbPpzz5K4AXeEXOxIt5KVtKRbM/Ra5fmtwaou55dhp5VaKBLk5BTdnpUNeiwjP3I
xnWvekHjxVheKgvUpbybpgVvPb2+oZc71edY1QoCL+2VGoanctWvx9imMQShL/B1
8HlMEai3T5E5RGV/XDel1kbS+p2GAivq5H7WBFlwpW6vLvYtrF8pEhRox0fGD42M
fxpdSZ0VowKrbOGPTj6KeEI8P/T3k+N4NppPjesqnqd9WDKHGRX0Eu6QdpT6tFt2
H6HuE7d9+1OUb3e0Br/trAkJPQsBpKPm2+hxvPsLSinNVw==
See also: Results filters
Link deduplication
Link deduplication by domain
Extracting domains
Removing tags from anchors and snippets
Filtering links by inclusion
Possible settings
Supports all settings of the SE::DuckDuckGo scraper, as well as additionally:
| Parameter name | Default value | Description |
|---|---|---|
| Result format | $domain - $key: $position\n | Default result output format |
| Stop when found | ☑ | Stop parsing if the domain is found, will not move on to the next pages |
| Match type | Exact domain | Ability to compare the desired position by domain, by main domain, and by full link (Exact domain / Top level domain / Exact url) |