Skip to main content

SE::DuckDuckGo - DuckDuckGo Search Results Scraper


Parser Overview

DuckDuckGo search results scraper. With the DuckDuckGo scraper, you can obtain large databases of links ready for further use. You can use queries in the same way as you enter them in the Dogpile search bar, including search operators (intitle, inurl, site, etc.). More details on the official page DuckDuckGo Search Syntax.

A-Parser functionality allows you to save the parsing settings of the DuckDuckGo scraper for further use (presets), set up a parsing schedule, and much more. You can use automatic query multiplication, substitution of subqueries from files, enumeration of alphanumeric combinations and lists to obtain the maximum possible number of results.

Results can be saved in the format and structure you need, thanks to the built-in powerful templating engine Template Toolkit which allows you to apply additional logic to the results and output data in various formats, including JSON, SQL, and CSV.

Collected Data

  • Links, anchors, and snippets from the results
Collected data


  • Support for all DuckDuckGo search operators (intitle:, inurl:, site:, etc.). More details about search operators on the official page DuckDuckGo Search Syntax
  • Scrapes the maximum number of results returned by Bing - 10 pages of 10 items in the results
  • Total number of results - 100
  • Ability to scrape by selected location (option Location)
  • Ability to choose the output language (option Language)

Use Cases

  • Collecting link databases - for A-Poster, XRumer, AllSubmitter, etc.
  • Checking site indexing
  • Backlink (mention) site search
  • Any other use cases involving scraping DuckDuckGo in one way or another


Queries should be specified as search phrases, for example:

IoT filetype:pdf

Query Substitutions

You can use built-in macros to expand queries. For example, if we want to obtain a very large database of forums, we will specify several main queries in different languages:


In the query format, we will specify the enumeration of characters from a to zzzz. This method allows for maximum rotation of search results and obtaining a multitude of new unique results:

$query {az:a:zzzz}

This macro will create 475254 additional queries for each original search query, which will result in 4 x 475254 = 1901016 search queries in total, an impressive figure, but no problem for A-Parser. At a speed of 2000 queries per minute, this task will be completed in just 16 hours.

Using Operators

You can use search operators in the query format, so it will be automatically added to each query in your list:


Output Results Examples

A-Parser supports flexible formatting of results thanks to the built-in templating engine Template Toolkit, allowing it to output results in any form, as well as in a structured format, such as CSV or JSON

Export of the list of links

Similar to SE::Google.

Similar to SE::Google.

Similar to SE::Google.

Similar to SE::Google.

Checking the indexing of links

Similar to SE::Google.

Saving in SQL format

Similar to SE::Google.

Dumping results to JSON

Similar to SE::Google.

Results processing

A-Parser allows processing the results directly during scraping, in this section we have provided the most popular use cases for the DuckDuckGo scraper

Similar to SE::Google.

Similar to SE::Google.

Extracting domains

Similar to SE::Google.

Removing tags from anchors and snippets

Similar to SE::Google.

Similar to SE::Google.

Possible settings

Parameter nameDefault valueDescription
Pages count5Number of pages to scrape (from 1 to 10)
RegionUS (English)Location selection
LanguageEnglish (United States)Language selection
Safe searchModerateAbility to enable "Safe search"
Serp timeAny timeSearch period
Use HTTP/2Determines whether to use HTTP/2 instead of HTTP/1.1
User agentMozilla/5.0 (Windows NT 10.0; Win64; x64; rv:120.0) Gecko/20100101 Firefox/120.0User-Agent header when requesting pages