Skip to main content

SE::AOL - Scraper of search results from search.aol.com

AOL

Overview of the scraper

The AOL scraper is a search results scraper for AOL. With the AOL scraper, you can obtain huge databases of links ready for further use. You can use queries in the same way you enter them in the AOL search bar, including search operators (contains, intitle, language, site, etc.). More details on the official page AOL Search FAQs.

A-Parser functionality allows you to save AOL scraper parsing settings for future use (presets), set a parsing schedule, and much more. You can use automatic query multiplication, substitution of subqueries from files, permutation of alphanumeric combinations and lists to obtain the maximum possible number of results.

Saving results is possible in the form and structure you need, thanks to the built-in powerful templating engine Template Toolkit which allows you to apply additional logic to the results and output data in various formats, including JSON, SQL, and CSV.

Collected data

  • Number of results per query
  • Links, anchors, and snippets from the search results
Collected data

Capabilities

The AOL search engine scraper has the following capabilities and settings:

  • Support for all AOL search operators (site:, prefer:, url:, feed:, etc.). More details on the official page AOL Search FAQs
  • Scrapes the maximum number of results provided by AOL - 100 pages of 20 items in the search results of each page
  • Support for search result periods (Anytime, Day, Week, Month)

Use cases

  • Collecting link databases - for A-Poster, XRumer, AllSubmitter, etc.
  • Competition assessment for keywords
  • Scraping American Google search results
  • Searching for backlinks (mentions) of websites
  • Checking website indexing
  • Searching for vulnerable websites
  • Any other scenarios involving scraping AOL in one way or another

Queries

As queries, you need to specify search phrases, just as if you were entering them directly into the AOL search form, for example:

тест
buy house
site:www.realtor.com
feed:football
a-parser loc:ca

Query substitutions

You can use built-in macros for query multiplication, for example, we want to get a very large database of forums, let's specify several main queries in different languages:

forum
форум
foro
论坛

In the query format, we will specify a character permutation from a to zzzz, this method allows for maximum rotation of the search results and obtaining many new unique results:

$query {az:a:zzzz}

This macro will create 475254 additional queries for each original search query, which in total will give 4 x 475254 = 1901016 search queries, an impressive figure, but this is not a problem for A-Parser at all. At a speed of 2000 queries per minute, such a task will be processed in just 16 hours.

Using operators

You can use search operators in the query format, so it will be automatically added to each query from your list:

site:$query

Output results examples

A-Parser supports flexible formatting of results thanks to the built-in Template Toolkit, which allows it to output results in any form, as well as in structured formats, such as CSV or JSON

Exporting the list of links

Similarly as in SE::Google.

Similarly as in SE::Google.

Similarly as in SE::Google.

Similarly as in SE::Google.

Keyword competition

Similarly as in SE::Google.

Checking link indexing

Similarly as in SE::Google.

Saving in SQL format

Similarly as in SE::Google.

Dumping results to JSON

Similarly as in SE::Google.

Results processing

A-Parser allows processing results directly during scraping, in this section we have provided the most popular cases for the AOL scraper

Similarly as in SE::Google.

Similarly as in SE::Google.

Extracting domains

Similarly as in SE::Google.

Removing tags from anchors and snippets

Similarly as in SE::Google.

Similarly as in SE::Google.

Possible settings

Parameter NameDefault ValueDescription
Pages count5Number of pages to scrape (from 1 to 100)
PeriodAnytimeTimeframe of the search results (Anytime, Day, Week, Month)
AOL domainsearch.aol.comChoice of country for search results (search.aol.com, search.aol.co.uk, recherche.aol.fr, suche.aol.de)