Overview and Application
List of all existing built-in scrapers in A-Parser with descriptions and links to their pages.
Search engine scrapersβ
| Scraper Name | Description |
|---|---|
 SE::Google | Scraping all data from Google search results: links, anchors, snippets, Related keywords, scraping ad blocks. Multithreading, ReCaptcha bypass |
 SE::Yandex | Scraping all data from Yandex search results: links, anchors, snippets, Related keywords, scraping ad blocks. Maximum scraping depth |
 SE::AOL | Scraping all data from AOL search results: links, anchors, snippets |
 SE::Bing | Scraping all data from Bing search results: links, anchors, snippets, Related keywords, Maximum scraping depth |
 SE::Baidu | Scraping all data from Baidu search results: links, anchors, snippets, Related keywords |
| SE::Baidu | Scraping all data from Baidu search results: links, anchors, snippets, Related keywords |
 SE::Dogpile | Scraping all data from Dogpile search results: links, anchors, snippets, Related keywords |
 SE::DuckDuckGo | Scraping all data from DuckDuckGo search results: links, anchors, snippets |
 SE::MailRu | Scraping all data from MailRu search results: links, anchors, snippets |
 SE::Seznam | Czech search engine scraper seznam.cz: links, anchors, snippets, Related keywords |
 SE::Yahoo | Scraping all data from Yahoo search results: links, anchors, snippets, Related keywords, Maximum scraping depth |
 SE::Youtube | Scraping data from Youtube search results: links, title, description, username, thumbnail link, number of views, video length |
 SE::Ask | American Google search results scraper through Ask.com: links, anchors, snippets, Related keywords |
 SE::Rambler | Scraping all data from Rambler search results: links, anchors, snippets |
 SE::Startpage | Scraping all data from Startpage search results: links, anchors, snippets |
Examplesβ
Indexing pages in Google and Yandex
Video Scraping of thematic domains by keyword phrase from Google
Questions and answers from Google search results
Scraping thematic domains by keyword phrase from Google and getting various parameters by domains
Scraping Google news with date and converting it
Scraping only snippets from Google
Spell checking in Google
Scraping links and anchors from Google to CSV
Scraping text from Google search engine using task chain
Yandex cache scraper
Scraping only snippets from Yandex
Getting Yandex search results and result position number
Number of pages of a site in Yandex
Scraping competition by key from Yandex
Finding free niches in the RU segment of the Internet
Search engine position scrapersβ
| Parser Name | Description |
|---|---|
 SE::Google::Position | Checking website positions by keywords in Google: links, list of all positions |
 SE::Yandex::Position | Checking website positions by keywords in Yandex: links, list of all positions |
 SE::Bing::Position | Checking website positions by keywords in Bing: links, list of all positions |
 SE::MailRu::Position | Checking website positions by keywords in MailRu: links, list of all positions |
Examplesβ
Getting website positions simultaneously for multiple regions
Getting website positions simultaneously for multiple regions
Overview of result representation options
Search engine suggestion parsersβ
| Parser Name | Description |
|---|---|
 SE::Google::Suggest | Google suggestion parser: number of results for the query, suggestions for the query, query type |
 SE::Yandex::Suggest | Yandex suggestion parser: number of results for the query, suggestions for the query |
 SE::Bing::Suggest | Bing suggestion parser: number of results for the query, suggestions for the query |
 SE::Aol::Suggest | Aol suggestion parser: number of results for the query, suggestions for the query |
 SE::Yahoo::Suggest | Yahoo suggestion parser: number of results for the query, suggestions for the query |
 SE::Youtube::Suggest | Youtube suggestion parser: suggestions for the query |
Examplesβ
Preset for parsing suggestions with the ability to exclude suggestions containing stop words from the results
Keyword parsersβ
| Parser Name | Description |
|---|---|
 SE::Google::Trends | Google trends keyword parser |
 SE::Yandex::Wordstat | Keyword and impression statistics parser from wordstat.yandex.ru |
 SE::Yandex::Wordstat::ByDate | Yandex WordStat impression statistics parser by months or weeks, keyword statistics |
 SE::Yandex::Direct::Frequency | Number of views |
 Rank::Bukvarix::Domain | Collection of keywords by domain from Bukvarix: keyword, frequency, position |
 Rank::Bukvarix::Keyword | Collection of keywords by keyword from Bukvarix: keyword, frequency, position |
 Rank::Ahrefs::KeywordDifficulty | Keyword difficulty, keywords, number of keywords |
 Rank::Ahrefs::KeywordGenerator | Ahrefs keyword parser: keyword ideas, questions with keywords |
 SE::Google::KeywordPlanner | Collection of suggestions, keywords, competition, min. and max. bids |
 SE::Yandex::Direct | Number of ads for a specific query, title, text and domain, links, anchors, tags |
Examplesβ
Seasonality of keywords
Estimating frequency using WordStat
Deep parsing of Wordstat
Getting a list of Google Trends categories
Formatting Google Trends date
Account registration parsersβ
| Parser Name | Description |
|---|---|
 SE::Yandex::Register | Yandex account registration parser |
Examplesβ
Getting a complete list of Yandex regions
Automating Yandex account registration
Site and domain parameter parsersβ
| Parser Name | Description |
|---|---|
 SE::Google::TrustCheck | Checks website for trust |
 SE::Google::Compromised | Checks for the presence of the message "This site may be hacked" |
 SE::Google::SafeBrowsing | Checks domain in Google blacklist |
 SE::Yandex::SafeBrowsing | Checks domain in Yandex blacklist |
 SE::Yandex::SQI | Checks site quality index in Yandex |
 Net::Whois | Determines if a domain is registered or not, domain creation date, and registration expiration date |
 Net::Dns | Resolves domains to IP addresses |
 Rank::Cms | Determines all popular forums, blogs, CMS, guestbooks, wikis, and many other types of engines |
 Rank::Alexa | Determines global Alexa ranking position |
 Rank::Alexa::Api | Fast Alexa checker via API |
 Rank::Archive | Parses the date of the first and last caching of the site in the web archive |
 Rank::Linkpad | Parses backlinks and statistics from linkpad.ru |
 Rank::MajesticSEO | Parses the number of backlinks from majesticseo.com |
 Rank::Mustat | Evaluates traffic on the site, as well as the cost and rating of the domain |
 Rank::Social::Signal | Parses social signals |
 Rank::Curlie | Checks the presence of the site in the Curlie directory (analog of DMOZ) |
 Rank::Ahrefs | Parses ahrefs.com |
 Rank::KeysSo | Parses keys.so |
 Rank::MOZ | Parses MOZ |
 SecurityTrails::Ip | Collects domains by IP |
 SecurityTrails::Domain | Parses SecurityTrails |
Examplesβ
Parsing thematic domains by keyword phrase from Google and obtaining various parameters by domains
Information about the site
Net::Whois selecting arbitrary data
Get notifications in Telegram about the end of domain registration period
Collecting a database of e-mail addresses
Determining a list of domains on IP
Determining CMS for 1000000 domains in 15 hours
Parsing site domains by a certain CMS
WordPress sites with subscriptions
Parsers of various servicesβ
| Parser Name | Description |
|---|---|
| SE::Yandex::Register | Allows to register accounts on Yandex |
 SE::Bing::Translator | Translator via www.bing.com/translator/ |
 SE::Google::Translate | Google translator |
 SE::Yandex::Translate | Yandex translator |
 SE::Bing::Images | Scraper for images from Bing search engine |
 SE::Google::Images | Scraper for images by keywords from Google |
 SE::Google::ByImage | Search for images by link |
 SE::Yandex::Images | Scraper for images by keywords from Yandex |
 SE::Yandex::ByImage | Scraper for images by link from Yandex |
 SE::DuckDuckGo::Images | Scraper for images from DuckDuckGo |
 SE::Dogpile::Images | Scraper for images from Dogpile |
 Util::AntiGate | Captcha recognition via AntiGate service |
 Util::YandexRecognize | Captcha recognition from Yandex |
 Util::ReCaptcha2 | ReCaptcha recognition from resources Anti-captcha.com, Rucaptcha.com, Capmonster, Π‘apmonster.cloud, Xevil, 2captcha, R.I.P.captcha |
 SEO::Ping | Mass sending of Ping requests to services supporting Weblog API (Google Blog Search, Feed Burner, Ping-o-Matic, etc.) |
 Check::RosKomNadzor | Site check in Roskomnadzor database |
 SE::Yandex::Speller | Checking pages for text errors via Yandex.Speller |
 GooglePlay::Apps | Google Play apps scraper |
 Social::Instagram::Post | Scraper for data on Instagram posts |
 Social::Instagram::Profile | Scraper for data from Instagram profiles |
 Social::Instagram::Tag | Scraper for posts by tags on Instagram |
 Social::Instagram::Geo | Scraper for posts on Instagram in a specified location |
 Telegram::GroupScraper | Scraper for data from public groups in Telegram |
 API::Server::Redis | Working with A-Parser via Redis |
 SE::Google::Cache | Checking for page availability in Google cache |
| Social::Instagram::Geo | Scraper for posts on Instagram in a specified location |
 DeepL::Translator | Text translator via www.deepl.com |
 CoinMarketCap::LastPrice | CoinMarketCap store scraper: token name, current price in USD |
Examplesβ
Automating Yandex account registration
Comparison of translator speeds
Using files as requests in DeepL
Loading files from a directory for translation in DeepL
Downloading images of a specified resolution and size
Creating JS scrapers. Working with CAPTCHA
Creating JS scrapers. Working with ReCaptcha2
Example of working with SEO::Ping
Parsing location links for Social::Instagram::Geo
Parsing information about Instagram users
Parsing users from public groups in Telegram
Store scrapersβ
| Parser Name | Description |
|---|---|
 Shop::Amazon | Scraper for amazon.com search results |
 Shop::Yandex::Market | Yandex.Market scraper |
 Shop::AliExpress | Scraper for AliExpress search results |
 Shop::eBay | Scraper for Ebay search results |
Examplesβ
Parsing products from a website
Map scrapersβ
| Parser Name | Description |
|---|---|
 Maps::Google | Google Maps scraper: organization name, address and coordinates, rating, number of reviews and pricing policy, website, phone, photo |
 Maps::Yandex | Yandex Maps scraper: organization name, address and coordinates, rating, number of reviews and pricing policy, website, phone, photo, social networks |
Examplesβ
Parsing Google Maps to collect all organizations in a certain area
Content Scrapersβ
| Scraper Name | Description |
|---|---|
 Check::BackLink | checks the presence of your link(s) in the link database |
 HTML::LinkExtractor | scraper of external and internal links from the specified site, can follow internal links to the selected level |
 HTML::ArticleExtractor | scraper of articles |
 HTML::TextExtractor | scraper of text blocks |
 HTML::TextExtractor::LangDetect | detection of the page language without using third-party services |
 HTML::EmailExtractor | scraper of email addresses from site pages |
 Net::HTTP | downloads the specified page, supports multi-page scraping |