SE::AOL::Suggest - AOL Search Suggestion Scraper
Overview of the scraper
A scraper for search suggestions by keywords in AOL. Thanks to the SE::AOL::Suggest scraper, you can automatically collect keyword databases from AOL search engine suggestions based on a query. Using the SE::AOL::Suggest scraper, you can quickly and easily scrape AOL suggestions for your query.
Thanks to the multi-threaded operation of A-Parser, the speed of query processing can reach 4700 queries per minute, which on average allows you to get up to 13500-14500 results per minute.
You can use automatic query multiplication, substitution of sub-queries from files, iteration of alphanumeric combinations and lists to get the maximum possible number of results. Using results filtering you can immediately clean the result, removing all unnecessary junk (using negative keywords).
A-Parser's functionality allows you to save SE::AOL::Suggest scraper parsing settings for later use (presets), ), set a parsing schedule, and much more.
Results can be saved in the form and structure that you need, thanks to the powerful built-in templating engine Template Toolkit which allows you to apply additional logic to the results and output data in various formats, including JSON, SQL and CSV.
Collected data
- Number of results per query
- Suggestions for the query
Capabilities
- Support for all AOL search operators (site: etc.)
- Scrapes the maximum number of results provided by AOL - 100 pages with 20 items per output
Use cases
- Collecting keyword databases
Queries
Search phrases must be specified as queries, for example:
write essay
Football
Speak in english
forex
cheap essay
Query substitutions
You can use built-in macros for automatic substitution of sub-queries from files. For example, if we want to add a list of other words to each query, we specify several main queries:
essay
article
thesis
In the query format, we specify the macro for substituting additional words from the file Keywords.txt, this method exponentially increases the variability of queries:
{subs:Keywords} $query
This macro will create as many additional queries as there are in the file for each initial search query, which will give a total of [number of initial queries (domains)] x [number of queries in the Keywords file] = [total number of queries] as a result of the macro operation.
For example, if the file Keywords.txt contains:
buy
cheap
As a result, the substitution macro will turn 3 main queries into 6:
buy essay
cheap essay
buy article
cheap article
buy thesis
cheap thesis
Output results examples
A-Parser supports flexible results formatting thanks to the powerful built-in templating engine Template Toolkit, which allows it to output results in an arbitrary form, as well as in a structured format, such as CSV or JSON
Exporting a list of suggestions
Similar to SE::Google::Suggest.
Outputting to a CSV table
Similar to SE::Google::Suggest.
Saving in SQL format
Result format:
[% FOREACH results;
"INSERT INTO serp VALUES('" _ query _ "', '"; suggest _ "')\n";
END %]
Result example:
INSERT INTO serp VALUES('write essay', 'write essay for me')
INSERT INTO serp VALUES('write essay', 'write essay')
INSERT INTO serp VALUES('write essay', 'write essay online')
INSERT INTO serp VALUES('write essay', 'write essay for you')
INSERT INTO serp VALUES('write essay', 'write essay free')
INSERT INTO serp VALUES('write essay', 'write essay conclusion')
INSERT INTO serp VALUES('write essay', 'write essay today')
INSERT INTO serp VALUES('write essay', 'write essays for money')
...
Dump results to JSON
Similar to SE::Google::Suggest.
Results processing
A-Parser allows you to process results directly during scraping; in this section, we have provided the most popular use cases for the SE::AOL::Suggest scraper
Option Parse to level
Similar to SE::Google::Suggest.
Filtering results (using negative keywords)
Similar to SE::Google::Suggest.