SecurityTrails::Ip - Collecting domain data from SecurityTrails

Overview of the scraper

SecurityTrails is the world's largest repository of historical DNS data, Data for security companies, researchers, and teams who need to detail, find suspicious changes in DNS records, and prevent future fraudulent or criminal activity.

A-Parser's functionality allows you to save parsing settings for future use (presets), set a parsing schedule, and much more. You can use automatic query multiplication, substitution of sub-queries from files, iteration over alphanumeric combinations and lists to get the maximum possible number of results.

The results can be saved in the form and structure you need, thanks to the powerful built-in templating engine Template Toolkit which allows you to apply additional logic to the results and output data in various formats, including JSON, SQL and CSV.

Go to DEMOBuy A-Parser Pro ($299)

Collected data

• Domain
• Hosting provider
• Alexa rank
• Mail provider

Capabilities

• Authorization by login and password
• Automatic pagination traversal
• Output results to CSV table, JSON, SQLite database

Use Cases

• Retrieving subdomains by IP
• Collecting Alexa Rank for subdomains
• Retrieving Hosting Provider
• Retrieving Mail Provider

Queries

Queries should specify ipv4 addresses:

172.217.164.174

Output results examples

A-Parser supports flexible result formatting thanks to the built-in templating engine Template Toolkit, which allows it to output results in arbitrary form, as well as in a structured format, such as CSV or JSON

Default output

Result format:

$list.format('$domain\n')

The result displays subdomains:

aalt158.gaming.youtube.com
o-o.preferred.ivanticsa177a.v20.cache.c.youtube.com
o-o.preferred.alt138-batelco-bah1.v24.lscache2.c.youtube.com
o-o.preferred.sigmarprd.v20.cache.c.youtube.com
thegameofpwns.io
o-o.preferred.orange-pfightmaster.v9.lscache4.c.youtube.com
o-o.preferred.kubectlivanticsa571.v20.cache.c.youtube.com
o-o.preferred.trial-one-my.v20.cache.c.youtube.com
o-o.preferred.activestc-ruh3.v2.lscache1.c.youtube.com
test.palaash.com
shoppe33.vip

Listing all subdomain data

Result format:

$list.format('$domain, $host_provider, $alexa_rank, $mail_provider\n')

Example result:

google.com, Google LLC, 4, Google LLC
developers.google.com, Google LLC, 40, none
earth.google.com, Google LLC, 640, none
uk.youtube.com, Google LLC, 1305, none

Output to CSV table

Result format:

[% FOREACH list; 
    tools.CSVline(domain, host_provider, alexa_rank, mail_provider);
END %]

Example result:

google.com,"Google LLC",4,"Google LLC"
developers.google.com,"Google LLC",40,none
earth.google.com,"Google LLC",640,none
uk.youtube.com,"Google LLC",1305,none

Dump results to JSON

Общий формат результата:

[% IF notFirst;
  ",\n";
ELSE;
  notFirst = 1;
END;

obj = {};
obj.query = query;
obj.subdomains = [];

FOREACH item IN p1.list;
    obj.subdomains.push({
        domain = item.domain
        alexa_rank = item.alexa_rank
        host_provider = item.host_provider
        mail_provider = item.mail_provider
    });
END;

obj.json %]

Начальный текст:

[

Конечный текст:

]

Example result:

[
    {
        "query": "172.217.164.174",
        "subdomains": [
            {
                "domain": "google.com",
                "mail_provider": "Google LLC",
                "host_provider": "Google LLC",
                "alexa": 4
            },
            {
                "domain": "developers.google.com",
                "mail_provider": "none",
                "host_provider": "Google LLC",
                "alexa": 40
            },
            {
                "domain": "earth.google.com",
                "mail_provider": "none",
                "host_provider": "Google LLC",
                "alexa": 640
            },
            {
                "domain": "uk.youtube.com",
                "mail_provider": "none",
                "host_provider": "Google LLC",
                "alexa": 1305
            }
        ]
    }
]

tip

To make the "Start text" and "End text" options available in the Job Editor, you need to activate "More options".

Possible settings

note

General settings for all scrapers

Parameter	Default value	Description
Method	New	Selection of the scraping method (New / Old / Account authorization)
Login		Enter login
Password		Enter password