Skip to main content

SE::DuckDuckGo - DuckDuckGo search results parser

img

DuckDuckGo parser overviewโ€‹

DuckDuckGo search results parser. With the DuckDuckGo parser, you can get large databases of links ready for further use. You can use queries in the same way as you enter them in the Dogpile search bar, including search operators (intitle, inurl, site, etc.). More details on the official DuckDuckGo Search Syntax page.

A-Parser functionality allows you to save DuckDuckGo parser parsing settings for further use (presets), set up a parsing schedule, and much more. You can use automatic query multiplication, substitution of subqueries from files, enumeration of alphanumeric combinations and lists to get the maximum possible number of results.

Saving results is possible in the form and structure that you need, thanks to the built-in powerful Template Toolkit that allows you to apply additional logic to the results and output data in various formats, including JSON, SQL, and CSV.

List of collected dataโ€‹

  • Links, anchors, and snippets from search results

Data collected by the SE::DuckDuckGo parser

Capabilitiesโ€‹

  • Support for all DuckDuckGo search operators (intitle:, inurl:, site:, etc.). More details on the official DuckDuckGo Search Syntax page.
  • Parses the maximum number of results returned by Bing - 10 pages of 10 items in the results
  • Total number of results - 100
  • Ability to parse by selected location (option Location)
  • Ability to select the output language (option Language)

Use casesโ€‹

  • Collecting link databases - for A-Poster, XRumer, AllSubmitter, etc.
  • Checking site indexing
  • Finding backlinks (mentions) of sites
  • Any other options involving parsing DuckDuckGo in one form or another

Query examplesโ€‹

  • Queries should be specified as search phrases, for example:
Football  
ั‚ะตัั‚
site:a-parser.com
ะฟะฐั€ัะตั€ site:a-parser.com
test -site:tests.com
IoT filetype:pdf

Query substitutionsโ€‹

You can use built-in macros to multiply queries, for example, we want to get a very large database of forums, specify several basic queries in different languages:

forum
ั„ะพั€ัƒะผ
foro
่ฎบๅ›

In the query format, we will specify the enumeration of characters from a to zzzz, this method allows us to rotate the search results to the maximum and get many new unique results:

$query {az:a:zzzz}

This macro will create 475254 additional queries for each original search query, which in total will give 4 x 475254 = 1901016 search queries, an impressive figure, but it is not a problem for A-Parser. At a speed of 2000 requests per minute, such a task will be processed in just 16 hours.

Using operatorsโ€‹

You can use search operators in the query format, so it will be automatically added to each query from your list:

site:$query

Result output optionsโ€‹

A-Parser supports flexible formatting of results thanks to the built-in Template Toolkit, which allows it to output results in any form, as well as in a structured form, such as CSV or JSON.

Same as in SE::Google.

Same as in SE::Google.

Same as in SE::Google.

Same as in SE::Google.

Same as in SE::Google.

Saving in SQL formatโ€‹

Same as in SE::Google.

Dumping results to JSONโ€‹

Same as in SE::Google.

Results processingโ€‹

A-Parser allows you to process results during parsing, in this section, we have provided the most popular cases for the DuckDuckGo parser.

Same as in SE::Google.

Similarly to SE::Google.

Extraction of domainsโ€‹

Similarly to SE::Google.

Removal of tags from anchors and snippetsโ€‹

Similarly to SE::Google.

Similarly to SE::Google.

Possible settingsโ€‹

Parameter nameDefault valueDescription
Pages count5Number of pages to parse (from 1 to 10)
RegionUS (English)Location selection
LanguageEnglish (United States)Language selection
Safe searchModerateAbility to enable "Safe search"
Serp timeAny timeSearch period
Use HTTP/2โ˜Determines whether to use HTTP/2 instead of HTTP/1.1