SE::Dogpile - Dogpile search results parser
Dogpile parser overview
Dogpile search results parser. With the SE::Dogpile parser, you can get large databases of links ready for further use. You can use queries in the same format as you enter them in the Dogpile search box, including search operators (url, language, site, etc.).
The A-Parser functionality allows you to save the parsing settings of the Dogpile parser for further use (presets), set up a parsing schedule, and much more. You can use automatic query multiplication, substitution of subqueries from files, enumeration of alphanumeric combinations and lists to obtain the maximum possible number of results.
Saving results is possible in the format and structure that you need, thanks to the built-in powerful Template Toolkit template engine, which allows you to apply additional logic to the results and output data in various formats, including JSON, SQL, and CSV.
List of collected data
- Links, anchors, and snippets from search results
- List of related keywords (Related keywords)
- Parses the maximum number of results returned by Dogpile - 50 pages of 10 items in the results
- Total number of results - 500
- Ability to search for related keywords
- Collecting link databases - for A-Poster, XRumer, AllSubmitter, etc.
- Keyword competition assessment
- Finding backlinks (mentions) of sites
- Checking site indexing
- Finding vulnerable sites
- Any other scenarios involving parsing Dogpile in one form or another
- Queries must be specified as search phrases, just as if you were entering them directly into the Dogpile search form, for example:
parser language: ru
site: slideshare.net Java Developer gmail.com resume -sample -samples -example -templates
You can use built-in macros to multiply queries. For example, we want to get a very large database of forums, so we specify several basic queries in different languages:
In the query format, we specify the enumeration of characters from a to zzzz. This method allows you to rotate the search results to the maximum and get many new unique results:
This macro will create
475254 additional queries for each original search query, which will give a total of
4 x 475254 = 1901016 search queries. This is an impressive number, but it is not a problem for A-Parser. At a speed of
2000 requests per minute, this task will be processed in just
You can use search operators in the query format, so it will be automatically added to each query from your list:
Result output options
A-Parser supports flexible result formatting thanks to the built-in Template Toolkit template engine, which allows it to output results in any form, as well as in a structured form, such as CSV or JSON.
Exporting a list of links
Links + anchors + snippets with position output
Output of links, anchors, and snippets in a CSV table
Saving related keywords
Checking link indexing
Saving in SQL format
Dumping results in JSON
A-Parser allows you to process results during parsing, and in this section, we have provided the most popular use cases for the Dogpile parser.
Link uniqueness by domain
Removing tags from anchors and snippets
Filtering links by inclusion
|Parameter name||Default value||Description|
|Pages count||Number of pages to parse (from 1 to 50)|
|Util::ReCaptcha2 preset||Determines whether to use Util::ReCaptcha2 to bypass captchas|
|ReCaptcha2 retries||Number of attempts to send a response for reCAPTCHA, specified number of times without changing the proxy|