Rank::Ahrefs::KeywordDifficulty - Ahrefs Keyword Difficulty Checker Scraper
Scraper Overview
Rank::Ahrefs::KeywordDifficulty – this scraper retrieves the "Keyword Difficulty" value. It also additionally collects data for search results for a query (only for the first page TOP-10).Keyword Difficulty is an indicator that helps understand how difficult it will be to rank for a specific search query.
Data is collected from the page https://ahrefs.com/keyword-difficulty.
To operate, it requires connection to the captcha solving service 
Util::Turnstile.
Collected Data
- Keyword Difficulty
- Approximate number of sites from which backlinks need to be set to get into the TOP10 for the searched keyword (shortage)
- Overview of search results for the query (only the first page TOP-10)
- Links from the search results (TOP-10)
- Anchor (TOP-10)
- AR (only TOP-3)
- DR (only TOP-3)
- UR (only TOP-3)
- Backlinks - the number of backlinks from other sites leading to this search result (only TOP-3)
- Domains - the total number of unique domains linking to this search result (only TOP-3)
- Traffic - estimated monthly organic traffic that this search result receives (only TOP-3)
- Keywords - the number of keywords for which this search result ranks in the top 100 positions (only TOP-3)
Capabilities
- Automatic work with proxies
- Selection of the country for which the keyword difficulty will be determined
Use Cases
- Getting aggregate information about search results for a query from the TOP-10
- Estimating the difficulty of a keyword/query
Queries
As queries, you need to specify keywords, for example:
essay writing
buy notebook
купить билет
магазин шин Спб
If the queries are in Russian or any other language, then choose the corresponding country in the scraper settings. Because there may be no results for the country United States, which is set by default.
Output Results Examples
A-Parser supports flexible formatting of results thanks to the built-in template engine Template Toolkit, which allows it to output results in any form, as well as in structured formats, such as CSV or JSON
Output "query difficulty" in the result
Result format:
$query: $difficulty\n
The default result will display the query and the keyword difficulty value:
essay writing: 84
buy notebook: 80
магазин шин Спб: 55
купить билет: 90
Output additional parameters for the top-10 search results for keywords in a CSV table
The values anchor, ar, dr, ur, bl, domains, traffic, keywords are available only for the TOP-3.
The built-in utility $tools.CSVLine allows creating correct table documents, ready for import into Excel or Google Sheets.
Result format:
[% FOREACH item IN serp;
tools.CSVline(query, difficulty, item.link, item.anchor, item.ar, item.dr, item.ur, item.bl, item.domains, item.traffic, item.keywords);
END %]
File name:
$datefile.format().csv
Starting text:
Запрос,Сложность,Ссылка,Анкор,AR,DR,UR,BL,Домены,Трафик,Ключи
Example of the result:
Запрос,Сложность,Ссылка,Анкор,AR,DR,UR,BL,Домены,Трафик,Ключи
"buy notebook",80,https://www.bestbuy.com/site/laptop-computers/all-laptops/pcmcat138500050001.c?id=pcmcat138500050001,"Laptops and Notebooks: PC Laptop, Notebook - Best Buy",2120,88,34,219,80,139112,2248
"buy notebook",80,https://www.bestbuy.com/site/computers-pcs/laptop-computers/abcat0502000.c?id=abcat0502000,"Laptops & Notebook Computers - Best Buy",2120,88,49,2306,474,37222,3148
"buy notebook",80,https://www.bestbuy.com/site/searchpage.jsp?_dyncharset=UTF-8&browsedCategory=pcmcat138500050001&cp=2&id=pcat17071&iht=n&ks=960&list=y&sc=Global&st=categoryid%24pcmcat138500050001&type=page&usc=All%20Categories,"Laptops and Notebooks: PC Laptop, Notebook - Best Buy",2120,88,14,2,1,243,179
"buy notebook",80,https://www.amazon.com/Notebooks-Laptop-Computers/b?ie=UTF8&node=565108,"Laptops | Amazon.com",,,,,,,
"buy notebook",80,https://www.walmart.com/cp/laptops/1089430,"Laptops - Walmart.com",,,,,,,
"essay writing ",84,https://owl.purdue.edu/owl/general_writing/academic_writing/essay_writing/index.html,"Essay Writing // Purdue Writing Lab - Purdue OWL",1256,90,45,731,316,2702,864
"essay writing ",84,https://grademiners.com/,"Essay Writer & Essay Writing Service | GradeMiners.com",7081,82,87,390507,21505,7328,2836
"essay writing ",84,https://www.scribbr.com/category/academic-essay/,"The Beginner's Guide to Writing an Essay | Steps & Examples",144207,66,40,966,249,12199,3281
"essay writing ",84,https://www.internationalstudent.com/essay_writing/essay_tips/,"General Essay Writing Tips - Essay Writing Center",,,,,,,
"essay writing ",84,https://writingcenter.fas.harvard.edu/pages/essay-structure,"Essay Structure | – Harvard Writing Center",,,,,,,
...
"купить билет",90,https://www.tutu.ru/,"Tutu.ru: Авиа, ЖД, билеты на автобус и туры онлайн ...",39974,74,80,459414,2349,1052237,7272
"купить билет",90,https://www.rzd.ru/,"Пассажирам",11185,80,84,5118775,7165,4163730,32342
"купить билет",90,https://pass.rzd.ru/,"Пассажирам",11185,80,75,475270,1426,249503,9533
"купить билет",90,https://www.aviasales.ru/,"Дешевые авиабилеты онлайн, цены. Поиск билетов на ...",,,,,,,
"купить билет",90,https://www.kupibilet.ru/,"Дешёвые авиабилеты онлайн, поиск билетов на самолёт ...",,,,,,,
"магазин шин Спб",55,https://www.spbkoleso.ru/,"Купить колеса, диски и шины в интернет магазине ...",4527548,28,42,1489,285,2240,855
"магазин шин Спб",55,https://spb.kolesa-darom.ru/,"«Колеса Даром» — интернет-магазин шин, дисков и ...",370176,55,27,136,44,1740,146
"магазин шин Спб",55,https://www.koleso-russia.ru/,"КОЛЕСО - автошины и диски | автошины | авторезина ...",992706,44,40,1295,222,14428,1319
"магазин шин Спб",55,https://kolesa812.ru/,"Шины и диски в Санкт-Петербурге | Купить в интернет ...",,,,,,,
"магазин шин Спб",55,https://ship-ship.ru/,"Интернет магазин шин и дисков - купить недорого ...",,,,,,,
...
Download example
How to import the example into A-Parser
eJyFVNtOGzEQ/ZWVBRJI24gAVdH2KYREpaUJDdCXEEVmPQtuvLaxvYEoyr937L0l
0MuLd25n5sx4vGviqF3YawMWnCXJdE10kElCBq801wLmvScDmf0GqwueZTwthFuR
mGhqLBgPmZIJlYskKeOSBCNflGE70QwyiiKJ18StNGB2tQRjOAN0cob6neNiPoE+
1S59osfzikZMllQUHjCBlGp0fTD0QeF3Rcnm7+lSVUhnVlt4U/wrPlMmpw5rBpYN
aLofDceTQa//JeIO8uhyFOluBxvXn+9lFEVOKWE7/Zufgks4eC7ArOKINZ3HAdVB
56ISqUyflKmVWmC1UNTCg6hdKqdc2kpzhvrclbYoJ20PkcxgdBHtz8hmNotJ2YYd
hp6wiz3k3Myzct7QJdwq3zkX0JqHqI1o7nvfY9SB93bK4Rwcdtyrz0AZ444rSUVZ
wS9BW/VO8ucwO6kw1o+Egx0alaPJQUgQ5lSzm5K9oBNMUQTsjxJDkowKCzGxSHVI
kQh768EpGOqUGWvPB+1romRPiCtYgmjDQv7zgguGG9vLEHRZAf8cMn6XY9O0t10K
d+jFIIcmS9DOx99bFFNX6rEehuA5d6jbvt9OtB6hcQGgm5mNfFiuDDRlqsxVdXyo
GqRf2PbKero17bSxcy27xlTJjD+Oq0dQRxbyFv8GY9lX/uH7vmQhBF6LhUm7Hj1b
XYNXWoJvwf1QAmk1b5+E1/L1pqSqDcf1++gJ5jjJ7apVypQKcTe52vaQdqVQuS+O
Tk/ScB6F82RL/hTOs3CyyH9Oz96ZjrtBycLZJb6mg0eF24gT2cya31zzg1z/92eX
rDd477/sdYn0Q/I4tOG0LV4qSbqb30r05Ys=
In the Result Format, the Template Toolkit templating engine is used to output elements of the $serp array in a FOREACH loop.
To change the file name of the results, simply change the file extension to csv.
To make the "Initial text" option available in the Task Editor, you need to activate "More options". In "Initial text", we write the names of the columns separated by commas and make the second line empty.
Dumping results to JSON
Общий формат результата:
[% IF notFirst;
",\n";
ELSE;
notFirst = 1;
END;
obj = {};
obj.query = query;
obj.links = [];
FOREACH item IN p1.serp;
obj.links.push(item.link);
END;
obj.json %]
Начальный текст:
[
Конечный текст:
]
Example of the result:
[{"query":"магазин","links":["https://www.dns-shop.ru/","https://www.ozon.ru/","https://ru.wikipedia.org/wiki/%D0%9C%D0%B0%D0%B3%D0%B0%D0%B7%D0%B8%D0%BD","https://www.lamoda.ru/","https://www.eldorado.ru/","https://www.mvideo.ru/","https://www.wildberries.ru/","https://www.citilink.ru/","https://www.bonprix.ru/","https://www.housebrand.com/ru/ru/"]}]
To make the "Initial text" and "Final text" options available in the Task Editor, you need to activate "More options".
Results processing
A-Parser allows you to process results directly during scraping, in this section we have provided the most popular cases for the scraper Rank::Ahrefs::KeywordDifficulty
Link deduplication
Add deduplication and from the dropdown menu select $serp.$i.link - Link.
Download example
How to import an example into A-Parser
eJyFVG1v0zAQ/i+nSmNSqToGAuVbVygCyjq69lNbVV5y6Uwd29hOtyjKf+ec9w4Q
Xxzf+Z7nnvOdk4Nj9mjvDFp0FoJNDrrcQwCfnlmiBe4njwZj+w2zjzyOeZgKl+3X
kv9KEYagmbFoPHIDSyaPQVCFBwEBnpSJOhBFRxgz2sIwB5dppCTqhMbwyFPxiOy1
42K/xCnTLnxkb/a1miGcmEg9YIkh03T02rAHRd+MQfFvulCl0pmshzdpPz5t6qA0
XliwASpHk0NweYTdsI5YVeHWGS4P0Hg/C/XABATOpFjsdg2LnSmTMH+FA3018nyj
uPS8uhh42u1WXly2Oe09O+FKUXTMRSfFzsi6ZYlPO4iYQ3/a8FyO3LO/FBZF3HEl
mahS+j50MuomBSAVxdLWcLQzoxJyOSwJvDNr5G5gUNpd2T8qDAQxExaHYEnqjJGQ
6OUJd2iYU2ahvR7y56DkRIg5nlB0YSX/TcpFREMziQn0pQb+PWTxB0fRltdPRW1/
MqShZSmtm8X3DhWpuTo0lyF4wh3ZduoHhLxjch4RdXtntz4sUQbbNDVznZ2ejEbp
Z6xr2UR3rrMyztpy7gyVjPlhUc9tE5nKFb3LhZwq/wR9XTIVgtpicdmNx8TWbfBG
J/AleFqmIFnt8wOnlLBf7yup2nAav3deYEI32c9aU4ZMiPVy3j+BbqTI2Kbjt9dh
uY7L9bq3f1+uH8o1As/m8KBozqjWYtf+Q9qfUP7fP0mQF9TRn/auQvryPY58dI+W
2gXBVfEbj260Iw==
See also: Results deduplication
Link deduplication by domain
Add deduplication and from the dropdown menu select $serp.$i.link - Link. Choose deduplication type: Domain.
Download example
How to import an example into A-Parser
eJyFVFtv0zAU/i9WpTGpVB0DgfLWdRQBY9269qmtKi8+6Uwd29hOtyjKf+fYuXaA
eHF8Lt93rk5BHLUHe2fAgrMkWhdEhzuJyOcXmmoBu8mTgcR+h/yaJwmPM+Hy3Ury
XxnsrlVKuSRDoqmxYDzBmiyoPERRhYoixD0rwzosejNIKF7JsCAu14Cx1BGM4QzQ
yBnKK8fFbgFTql38RN/t6qSG5EhF5gELiKlG01tDHxV+c0rKf9PFKpPO5D28yfr+
WSgHzRjGJxatCZajUSG4PJDtsPZYVu6sKbvSfhHqkQoSOZNBud02LHamTEp9Jwf6
YuT5RknQvDkbeNrNRp6dtzHtAz3CUqF3wkWXip2hdEtTH3bAqANvbXjOR+7FN4Uy
xh1XkooqpJ9Dl0Y1K8RLhb54NRzszKgUVQ4CgVfmTbprMghyV/Z9hSFRQoWFIbGY
6oxiIuy1hTsw1Ckz1z4f1BdEyYkQN3AE0bkF/quMC4ZLM0kQ9LUG/t1l/gdH2ZbX
D4VjfzaYQ8sSpKv5jw7F1I3aN80QPOUOZTv1C4LaMSoPALrt2a13S5WBNkzNXEfH
l6NB+h3rRjbRneqkjJOxnCpjJRO+n9d723hmconPcy6nyr9EX5fMhMCxWFh06zGx
9Ri80CX4GjwNIfz6Ns+POKWE/fZQpaoNx/X74BNMsZP9qDVlTIVYLW76FtKtFAqb
bPz+Mg7nOJyXvfvHcH4KJyOezcFe4Z5hreW2/Ye0/6Liv3+SqChxoj/tXYX05Xsc
6rCPFsdFoovyN1c+trs=
See also: Results deduplication
Extracting domains
Add Results Constructor and from the dropdown menu select the source: $p1.serp.$i.link - Link. Choose type: Extract Domain.
Download example
How to import an example into A-Parser
eJyFVG1v2jAQ/i8npK4SQ3TdtCnfKB3Stq60tHyCCt2SC/VwbM92aKOI/76zCQnt
Nu2L43t5nnt1avDoNu7GkiPvIFnUYOIdEvj8jIWRtBo9WsrdN6ouRZ6LtJS+Ws2V
+FXS6lIXKBT0waB1ZAPBAmaoNkmyRyUJ4560zTose2eUI1+hX4OvDHEsvSVrRUZs
FBnLcy/kakZjND59xHerJqk+bFGWATCjFA2b3lr8oflbIez+TZfqUnlbHeFtCbuH
hz4wL2fiJtoWGIrumbMBV2IGedS8OelJoTbLpTo5hdb7Drd0r9k7F5I69YSlaywC
fS9DT8F64Dkd+OeQP2aZ8EIrlPuQoWVdGvu2Ml5p9uWrFeQmVhes8hQJgrI6pLuA
XpSBKcqIvd1jIMlROuqD41QnyIlkry3Ck0Wv7dSEfFhfg1YjKa9oS7Jzi/wXpZAZ
z3eUM+hLA/y7y/QPjl1b3nEontCT5RxalihdTL93qExf6fWhGVIUwrPsxmGWrB2y
ckNk2p5dB7dCW2rDNMxNdF5yQyqsQzeykelUL8p4MZYjZQ1OlzbleIthfwFhVWJy
agNhnZr941FZTH37PNBarFjfuPuwOxGzY0yqVS7W02ZlD5FLdc8vc6rGOjzC0CdV
SsljdjTr1m3kmrEGoSv4NXgcQ3DQ9uVxDlq6r3f70o0VnNKHUHDBkzmO2lCmKOV8
dnVsgW5FWViWw/fnaTyH8Tw/un+M56d4ZhDYPK017y3XGlrQ/D7a31D9359IUu94
Q366mz0ylB9wrOM+Oh4/JGe736N7s6Y=
See also: Results Constructor
Removing tags and decoding entities for anchors
Add Results Constructor and from the dropdown menu select the source: $p1.serp.$i.anchor - Anchor. Choose type: Remove HTML tags.
Add Results Constructor again and from the dropdown menu select the source: $p1.serp.$i.anchor - Anchor. Choose type: Decode HTML entities.
Download example
How to import an example into A-Parser
eJyNVG1P2zAQ/i+nSgwpQ2Vs2pRvpazaC6OswKeCKi+5gIdje7ZTiKL+9925blPY
pvHFsc/3PPfi59JBEP7enzv0GDzk8w5s3EMOHx9FbRUuRncOK/8V2xNZVbJoVGgX
M6zNEhcnWJgSIQMrnEfHBHOYCX2f52tUnhPuwbiyx5J3iZWgLWQdhNYixSIy52Sk
kiWdr4JUFGQsbCjuxJtFSiqDpVANA2ZYCEtXr534YejbClj9m64wjQ6u3cG7BlY3
NxkQL2XiJ8bVgose2MMDqsQeVNHyam8gdHFn3PW13tuHrf+FWOKlIf9KKuzNEzqd
iZoDDEoRkG83TPsH4ZErEGUpgzRaqHVQblqfyJWWv2KC2pAvbZ1EP3GmJlPASMDG
dpPwHAbxDETRROz3NQbySiiPGXhKdSIokfL5jQzoRDBuajkfsndg9EipU1yi6t0i
/3EjVUkvPKoI9DkB/+4y/YNjtS1vNxS90YOjHLYs8XQ8/dajSnNqbjfNULKWgc5+
zK9J1iEZ7xHttmdn7FYbh9swiTlFJ5lb1CyI/slGtjc9KePJs+wYO/CmcQXFmw+z
ObBY+FWjSoAllTTo4oh8CrXia+dES8bkHVg6CcKyfRljGaftBYwEKYyu5O00zcGm
mEZf0rhP9djwZHPrdaMUKcfjrFfwyCel8KHv4XPwOIaIeaVxpiyM8l8u1t20TlJS
77LUid2oibIQSl3NTndvoFc9Ha6b4dujIq7DuB7t7N/H9UNcS2C2gLeGRoFq5Rak
f9L239b998+UdysS3U9/vkZy+YwjG/XRk6IgP1z9BrMuz2Q=
You can add Results Constructor as many times as you need.
See also: Results Constructor
Filtering links by inclusion
Add a filter and from the dropdown menu select: $serp.$i.link - Link. Choose type: Contains string. Next, you need to write the filtering feature in String.
Download example
How to import an example into A-Parser
eJyFVMFu2zAM/ZVACNAVyIJ1wC6+pdkCbMuaLk1OaRCoNp1qkSVNktMahv+9pOzY
TtthJ4sU+fhIPrlknruDu7XgwDsWbUpmwplF7Nszz4yE3eTRQup+QvFVpKmIc+mL
3UxID1ao/eChGDgfTkINpFAHNmKGWweW4DZsydUhimqMKEKUJ22TDgmjE0g5Htmo
ZL4wgJX1EawVCeClSNBeeyF3S5hy4+NH/nnXUByxI5c5JSwh5gavPlr+oPFbcFb9
Gy7WufK26OXbvB+fhubwGssQsWjDsB2DjtDfdtRErOrwWCvPherBjWOdoamNF1qh
7UA5Vm23J0Q30zbjNOOhuRoT9jgNng8XQypxf68uLtv67o4fYaVrYtC5cQdwwzMq
OEy4B7o94VyO/TMNiCeJIBJc1iVpJx2NtRJ/A2GlMRaPVoCbWSQfMQ8BgJzFie6G
DYNNI8hD7u86h0Uplw5GzCHVGUciyesbgQPjXttFGAr6S6bVRMo5HEF2YQH/Ohcy
QQFNUkz63iS+H7J4g1G17fVLoQSeLHJoUYJ1vfjVZSV6rvenYUiRCY+2m5JY0PsJ
nQcA087shsIybaEt0yA31fFNGVCkt25lE9O5zto4W8u5E+WViv2i0fApMlcrfLgL
NdX0RqkvlUuJa3Gw7OQxcc0ayOgIvk6ehhJIq32KzGst3Y+7mqqxAuX3hQhmOMl+
1QYy5lKul/P+DeskFeTkCDZGne41Kgi7qLbtn6L9/5T//V9EZYW7+uNu60xqjPLQ
hxNy4b1dVS8vyrR/
See also: Results filters
Possible settings
| Parameter | Default value | Description |
|---|---|---|
| Util::Turnstile preset | default | Selection of the Util::Turnstile preset for captcha bypass. It is necessary to preconfigure the scraper Util::Turnstile - specify your access key and other parameters, after which select the created preset here. |
| Turnstile pass proxy | ☐ | Passing the proxy to the captcha solving service. More details. |
| Country | United states | Choice of search country |
| Do not search for sitekey | ☑ | Experimental option, disables the search for captcha sitekey, which in turn speeds up the task start. In case of problems such as an invalid sitekey, this option should be disabled. |