Rank::Ahrefs::KeywordGenerator - Ahrefs Keyword Generator scraper
Scraper Overview
Parsing thousands of keyword variations in seconds. Searching for relevant keywords from the Ahrefs database, which contains more than 8 billion queries.
The scraper collects keyword ideas and questions with keywords. Data is collected from the page https://ahrefs.com/keyword-generator. The scraper allows the selection of a country and a choice of search engines (Google, Bing, Youtube, Amazon).
For the scraper to work, you need to select a preset for captcha solving 
Util::Turnstile.
Collected Data
- Keyword ideas
- Questions with keywords
- Keyword Difficulty (KD) - this value provides an estimate of how difficult it is to rank in the top 10 search results for a keyword on a 100-point scale.
- Frequency - this indicator shows how many times per month on average people in the given country search for your target keyword.
- Date of the last check of search engine results for the keyword.
Capabilities
- Automatic work with proxies
- Selection of the country for which the keyword difficulty will be determined
- Choice of the search engine for which the keyword difficulty will be calculated
Use Cases
- Obtaining aggregate information about the keyword query
- Estimating the frequency of the keyword query
- Assessing the difficulty of the keyword
Queries
As queries, you need to specify keywords, for example:
essay writing
buy notebook
купить билет
If the queries are in Russian or any other language, then choose the corresponding country in the scraper settings. It may be that there will be no results for the country United States, which is set by default.
Output Results Examples
A-Parser supports flexible formatting of results thanks to the built-in template engine Template Toolkit, which allows it to output results in any form, as well as in structured formats, such as CSV or JSON
Output of keyword ideas and questions in the result
Result format:
Ideas:\n$ideas.format('$key\n')\nQuestions:\n$questions.format('$key\n')
Example of result:
Ideas:
best buy notebook
best buy notebook computers
buy notebook
buy notebook computer
essay writing service
essay writing
best essay writing service
essay writing services
essay writing help
купить билет на самолет
купить билет на поезд
ржд купить билет
купить билет ржд
аэрофлот купить билет
...
Questions:
where to buy midori traveler's notebook
where to buy moleskin notebook
where to buy moleskine notebook
where to buy dot grid notebook
where to buy traveler's notebook
a common mistake when writing a reflective essay is to
how to start writing an essay
how to improve essay writing
what is john steinbeck’s purpose in writing the essay "symptoms"?
what is the second step in writing an argumentative essay
за сколько дней можно купить билет на поезд
как купить билет на самолет
как купить билет на электричку
где купить лотерейный билет
как купить билет на поезд
...
Output of "keyword ideas" with parameters (difficulty, frequency, and check date) in a CSV table
The built-in utility $tools.CSVLine allows you to create correct table documents, ready for import into Excel or Google Sheets.
Result format:
[% FOREACH item IN ideas;
tools.CSVline(item.key, item.difficulty, item.volume, item.updated);
END %]
File name:
$datefile.format().csv
Initial text:
Ключ,Сложность,Частота,Дата
Example of result:
Ключ,Сложность,Частота,Дата
"essay writing service",98,7500,2021-03-04T12:43:40Z
"essay writing",90,6600,2021-03-04T11:05:22Z
"best essay writing service",96,3000,2021-03-03T19:37:02Z
"essay writing services",98,900,2021-03-03T02:39:34Z
"essay writing help",91,700,2021-03-05T07:13:18Z
"best essay writing service reddit",49,600,2021-03-03T22:07:33Z
"cheap essay writing service",90,600,2021-02-26T01:25:25Z
"essay writing tips",64,600,2021-02-27T22:15:00Z
"custom essay writing",87,450,2021-02-20T03:51:14Z
...
Download example
How to import the example into A-Parser
eJyFVF1v2jAU/SuRRaVWYtE6aS/ZU5rB1o2RNpS9MIS85IZ6OLZnO7QI8d937YQE
ui9eyP0691zfY++JpWZj7jQYsIZEiz1R/ptEZPRMK8VhFT9qKE0UfYbdk9TFBxCg
qZWaDImi2oB2dQuSUbGJon8kF1DSmlsy3BO7U4Ad5Ba0ZgVgkBVozy3jqwwSqmz+
SN+sWipDsqW8dgUZ5FRh6JWm3yX+7yg5/B2ulLqiFjF81w5kcRGM02wUJx8DZqEK
bqcBFlHz7psI/M9KyU2YzL5yJuDS5YQb2A19dliwsmQ5Ih4dW8nrClqjVgW1UFw1
WKPp++BiSQ7L5ZA0NMzYc0IWA3UddvO1wRndwoN0zBmH3j1Ga0orx33g4F00bIa7
vArts0OgRcEsk4LypoPbSd91LthPP7uQmIufmoEZa1mhy4IHcM7dkd2CDLxNEKL2
tfdNDYlKyg0MiUGqY4pEipcRPAe/8VQ5PujfEyliziewBd6nefybmvECBRSXWHTb
Fv45Jf0N49CNd9oKNfCkkUOH4q2b9EtfVciJXB8Pg7OKWbRNImvhFvManRsA1Z3Z
1KVVUkPXpkVuu+PlUSCc4PqVxap3nY1xtpZzZy5FydZpK+JjZi0e8IamIpHuMrq5
RM05rsVA1ssjNu0anNETfFmc+BZIq7uLxIv906yhqjRD+b11BCs8ydOuLWROOZ9n
k9MI6SXl5WQcbI46XUtUEE5xWHYvRffQ7P/3XkT7A67qh7lrCt1crgx9eEAG90Ci
68MvqRKrIg==
In the Result Format, the Template Toolkit is used to output elements of the $ideas array in a FOREACH loop.
In the file name of the results, you just need to change the file extension to csv.
To make the "Initial text" option available in the Task Editor, you need to activate "More options". In the "Initial text," we write the names of the columns separated by a comma and make the second line empty.
Output of "questions with keywords" with parameters (difficulty, frequency, and check date) in a CSV table
The built-in utility $tools.CSVLine allows you to create correct tabular documents, ready for import into Excel or Google Sheets.
Result format:
[% FOREACH item IN questions;
tools.CSVline(item.key, item.difficulty, item.volume, item.updated);
END %]
File name:
$datefile.format().csv
Initial text:
Вопрос,Сложность,Частота,Дата
Example of result:
Вопрос,Сложность,Частота,Дата
"a common mistake when writing a reflective essay is to",0,150,2021-03-04T12:17:36Z
"how to start writing an essay",58,150,2021-02-24T09:48:50Z
"how to improve essay writing",24,150,2021-02-28T01:16:10Z
"what is john steinbeck’s purpose in writing the essay ""symptoms""?",0,150,2021-03-01T20:33:45Z
"what is the second step in writing an argumentative essay",1,150,2021-02-21T21:36:03Z
"when assembling a writing plan for an essay, which of the following should come first?",0,150,2021-02-16T15:20:21Z
"which step must you complete before writing a media assessment essay?",3,100,2021-03-04T02:48:14Z
"how is theme treated when writing a response to literature essay?",1,100,2021-03-01T13:54:31Z
"which phase of a writing strategy involves deciding on the topic and the purpose of the essay?",0,100,2021-02-19T23:58:36Z
...
Download example
How to import an example into A-Parser
eJyFVN9v2jAQ/lcii0qtxKJ10l6yJ5rB1o1BB2UvFCFDLtTDsV3boY2i/O87OyGB
btV4IXe+77vvftglsdTszZ0GA9aQaFkS5b9JRIYvNFMc1oNHDamJou9QPEudfAEB
mlqp1085GMukMKRPFNUGtGNYkhkV+yh6C4bBCaQ055b0S2ILBZhLHkBrlgAesgTt
hWV8PYOYKrt9pB/Wjag+OVCeO8AMtlTh0TtNNxL/C0qqt+lSqTNqkcNnbUmWF0Ew
ms6Gg/hrwCxkwe0kUNdhW9enBxHUPyslN2E8/8WZgEsXG+6h6HtUmLA0ZVukPjoO
kucZNEauEmohuWrIhpPPwcWKVKtVn9SCzMirQz09TN5W2hzO6QHupauBcejcI7Qm
NHNV9By/Ow3rMi+vQvviGGiSMFcH5XUGN50u60KwJ98FITEWPzUDM9IyQ5cFT+Cc
xVHdkvS8TZAi99ifNYZEKeUG+sSg1BFFIcnrE2yEn/1U1fsSlUSKAedjOADvwjz/
Tc54gqs0SBF02wD/HTL9i6NqyztNhdvwrFFDy+Ktm+mPDpXIsdwdm8FZxizaJpa5
cIN5j849gGp7NnFhmdTQpmmYm+x4oRQIt3rdyAaqc52VcTaWc+dWipTtps06HyNz
cY+3dipi6S6oq0vknONYDMy69RiYZgzO6AS+Bsc+BcpqbyXx2/5tXktVmuH6fXQC
M+zkadaGcks5X8zGpyekWyk0wBhaBK7pTOwexCYvAiEtbKTcEwe3sJO4WFhctWqf
kvZNKv/3oERlhRP8be5qoCvXwdCHfTM4HhJdV38AHCm7pA==
In the Result Format, the Template Toolkit templating engine is used to output the $questions array in a FOREACH loop.
In the file name of the results, you just need to change the file extension to csv.
To make the "Initial text" option available in the Task Editor, you need to activate "More options". In the "Initial text", write the column names separated by commas and make the second line empty.
Dump results to JSON
Общий формат результата:
[% IF notFirst;
",\n";
ELSE;
notFirst = 1;
END;
obj = {};
obj.query = query;
obj.questions = [];
FOREACH item IN p1.questions;
obj.questions.push(item.key);
END;
obj.json %]
Начальный текст:
[
Конечный текст:
]
Example of result:
{"query":"casino online","questions":["online casino no deposit bonus keep what you win","casino ohne anmeldung gratis online spielen where& 39","why online gambling is more dangerous than casino gambling","how to start an online casino","online casino no deposit bonus keep what you win usa","online casino games where you can win real money","how to open online casino","how to start an online casino business","when will the casino open in gta 5 online","how to play online casino games","what is the best online casino","what online casino has gammomat provider for canadians","online casino no deposit bonus keep what you win usa 2017","what online casino has the best payouts","how to start an online casino free","what online casino pays out the most","which online casino pays out the most","how to cheat online casino slot machines","how do online casino slots work","what is the best online casino canada","how to create online casino business","where can i play casino games online for free","what is the best online casino to play at","which online casino is legit","how to win money online casino","how to play online casino","how to win online casino roulette","what is the best online casino game","what is the best online casino for slots","what is the best online casino yahoo answers","how to start a online casino","how to play casino games online","which online casino game has the best odds","how to win online casino slots","how to open online casino business","what online casino is legit","usa online casino no deposit bonus keep what you win","how to buy an online casino","where can i play free casino games online","what is the best online casino for roulette","how to withdraw money from online casino","how to make money from online casino bonuses","what is the best online casino for us players","what is the best bank or credit union to have for online casino deposit","how to start casino mission gta 5 online","how to pay taxes from online casino reddit","how does online casino work","what other online casino is affilated with royal planet casino","where do you enter little casino promo codes online","what is harder play casino or online poker"]}
To make the "Initial text" and "Final text" options available in the Task Editor, you need to activate "More options".
Results processing
A-Parser allows you to process results directly during scraping, in this section we have provided the most popular cases for the scraper Rank::Ahrefs::KeywordGenerator
Deduplication of keyword ideas and keyword questions
It is necessary to add deduplication twice and select from the dropdown lists:
$questions.$i.key - Keyword and $ideas.$i.key - Keyword.
Download example
How to import an example into A-Parser
eJydVN9vGjEM/leQhdRVYmidtJe8UTSqbqy0FJ4AocD5aEYuCUmO9oT43+vcT2jX
TdoLnO3P3+fYTg7gudu6e4sOvQM2O4DJv4HB9xeeGInL3pPF2DH2E7NnbaMbVGi5
13a5W4plqsQOOmC4dWgDwQzGXG0Z+yiLwBHGPJUeOgfwmUGS0nu0VkRIQRGRPfVC
LsfY58avn/jXZVlTB/ZcpiFhjGtuKPTZ8pWm/4zD8WO6WNuEe+LIVWuS2wi5Y/O5
aovw1S1gny7aW8zIe3FJPw8pOi+0KnC7ynqPPdUPTUmDeinJZpArkIfgsOiUiEkB
d94KtYHKeyP1iktg3qb4d9K6nP8gXiwqJjfIz0L4trnq1q0ug498jxMdmihko+4G
ZN3xJKi0I+4xRKumXHb9S2DgUSRCeVwWCmE9GtVpcR4GShOWPq1AN7A6IZfHnCA4
s6q6GbRzuznlQ5EDLObSYQcclTrgVEj0NiJ8sXwjU7SLHUCrnpRD3KNsYDn/dSpk
RLvciynptkz8M2T0juNYH+9Uitbx2VINNUtuXY9+NVmRHupN1QwpEuHJdn2dqjCY
L/mA0dQ9uwuwRFusZUrmUp0utEEVdr8ZWc80rrNjnI3l3LnWKhabUXmfKmSqJvRq
jFRfhwcinEulUtJYHI6b9ei5cgzBaAp8m9zPJais+lkAr7V0Px6LUo0VtH7fQoEJ
dfJUtaRccymn4+FpBJqVIgOd41krNJ2uw1yt0qyltMeV1tu5qr5aZk0hulB1rOV1
i7AQFDxuNO0enf+4qJ+7+tk8/OvRY4cjDfm3uy8SQ0dCGvmotY4mCOzq+ApW0Pkz
See also: Results deduplication
Filtering questions with keywords
Add a filter and select from the dropdown list: $questions.$i.key - Keyword. Choose the type: Contains string. Next, in String, write the filtering feature. For example, filter questions that contain the word what.
Download example
How to import an example into A-Parser
eJyFVFtv2jAU/ivIQuoqMbRO2kveKBrTNlZaCk+AkCEn4OHYxnZoI5T/vnOcG3Sr
9pScy/edu8/Mc3dwjxYceMeixZmZ8M8i9vWVp0bCerC3kLgo+gn5i7bxN1Bgudd2
fVwnQnrLesxw68ASfsGmXB2i6D0QOseQ8Ex61jsznxvASPoE1ooY0ChilOdeyPUU
htz47Z5/Xlcp9diJy4wAU9hyg6aPlm80fnPOivfpEm1T7pEjRG1InjJwXmjlouVS
dY+11C/dP9x0D5Cj5eb2kpsqBqqioosWrEGiFiFs1au8ZiVkq5XnQl1EftlzSkQb
gqEslAOEF6tVzetGIQm0dc1dv6m/Mj7zE8x0mQ206hFKDzylCN2YeyBrXc1t378S
A49jQVG5LCPQzNqocyWOIUOl0Rd/rQA3sjpFlYdAQMq8zm7BukGmmrOAfSoxLEq4
dNBjDlMdcUwkfmsRvtyIiSmbF52ZVgMpx3AC2boF/vtMyBgXbJAg6HsF/LfL5C+O
oinvMhTuyIvFHBqWIN1PfrWoWI/1rm6GFKnwKLuhzhQN5lMYN5imZw/klmoLTZiK
uYqOR2ZA0UK2IxuYVnVVxtVYrpW4T4nYTaolrz0zNcNLnqihpqOlulQmJY7FwbRd
j4GrxkBCm+Bb8DCEwLSaW2Vea+l+PJepGitw/b5Qgil28jJqRbnlUs6n40sLa1cK
BXCO5x1qulC7pdpkeUdpDxutD0tV/3XMFk14Xo2t43UHfRlF8LDTuHtYf7Fq3qDm
KTv/7yWKzgUO+bd7LIHUEYKhDlvrwmXeFX8Al3PStA==
See also: Results filters
Possible settings
| Parameter | Default value | Description |
|---|---|---|
| Util::Turnstile preset | default | Selection of the Util::Turnstile preset for captcha bypass. It is necessary to previously configure the Util::Turnstile scraper - specify your access key and other parameters, after which select the created preset here. |
| Turnstile pass proxy | ☐ | Passing the proxy to the captcha solving service. More details. |
| Search engine | Google | Choice of search engine |
| Country | United states | Choice of search country |
| Do not search for sitekey | ☑ | Experimental option, disables the search for the captcha sitekey, which in turn speeds up the task start. In case of problems such as an invalid sitekey, this option should be disabled. |