SE::AOL - Scraper for search results from search.aol.com
AOL Scraper Overview
The AOL scraper is a scraper for AOL search results.. Thanks to the AOL scraper, you can obtain huge databases of links ready for further use. You can use queries in the same format as you would enter them directly into the AOL search bar, including search operators (contains, intitle, language, site etc.). More details on the official AOL Search FAQs.
A-Parser functionality allows you to save the scraping settings for the AOL scraper for future use (presets), ), set a scraping schedule, and much more. You can use automatic query proliferation, sub-query substitution from files, or iteration through alphanumeric combinations and lists to get the maximum possible number of results.
Results can be saved in the necessary form and structure thanks to the powerful built-in templater Template Toolkit which allows applying additional logic to the results and outputting data in various formats, including JSON, SQL and CSV.
Collected Data
- Number of results per query
- Links, anchors, and snippets from search results
Capabilities
The AOL search engine scraper offers the following capabilities and settings:
- Supports all AOL search operators (site:, prefer:, url:, feed: etc.). More details on the official AOL Search FAQs
- Scrapes the maximum number of results provided by AOL - 100 pages with 20 elements per page in the search results
- Supports result periods (time ranges) (Anytime, Day, Week, Month)
Use Cases
- Collecting link bases - for A-Poster, XRumer, AllSubmitter, etc.
- Assessing competition for keywords
- Scraping US Google search results
- Finding backlinks (mentions) of websites
- Checking website indexation
- Finding vulnerable websites
- Any other scenarios involving AOL scraping in one way or another
Queries
Queries should be specified as search phrases, exactly as you would enter them directly into the AOL search form, for example:
test
buy house
site:www.realtor.com
feed:football
a-parser loc:ca
Query Substitutions (Macros)
You can use built-in macros to proliferate queries; for example, if we want to get a very large database of forums, we'll specify a few main queries in different languages:
forum
forum
foro
论坛
In the query format, we will specify the iteration of characters from a to zzzz, This method allows maximum rotation of search results and retrieval of many new unique results:
$query {az:a:zzzz}
This macro will create 475254 additional queries for each initial search query, which totals 4 x 475254 = 1901016 search queries. This number is impressive, but it’s no problem for A-Parser. At a speed of 2000 queries per minute, this task will be processed in just 16 hours.
Using Operators
You can use search operators in the query format, thus it will be automatically added to every query in your list:
site:$query
Output Results Examples
A-Parser supports flexible results formatting thanks to the built-in templater Template Toolkit, which allows it to output results in an arbitrary form, as well as in structured formats, such as CSV or JSON
Exporting a List of Links
Links + Anchors + Snippets with Position Output
Outputting links, anchors, and snippets into a CSV table
Saving Related Keywords
Keyword Competition
Checking Link Indexation
Saving in SQL Format
Dump Results to JSON
Results Processing
A-Parser allows you to process results directly during scraping; in this section, we have provided the most popular use cases for the AOL scraper
Link Deduplication
Link Deduplication by Domain
Extracting Domains
Removing Tags from Anchors and Snippets
Filtering Links by Inclusion
Possible Settings
| Parameter Name | Default Value | Description |
|---|---|---|
| Pages count | 5 | Number of pages to scrape (from 1 to 100) |
| Period | Anytime | Result period (Anytime, Day, Week, Month) |
| AOL domain | search.aol.com | Selection of the search results country (search.aol.com, search.aol.co.uk, recherche.aol.fr, suche.aol.de) |