Queries to other parsers

await this.parser.request(parser, preset, overrideParams, query)

await this.parser.request(parser, preset, overrideParams, query)

Receiving results from another parser (built-in or another JS parser), the following arguments are specified:

• parser - parser name (SE::Google, JS::Custom::Example)
• preset - settings preset of the called parser
• overrideParams - hash with setting overrides for the called parser
• query - query

Partially ignores the preset of the called parser, which is set by the preset parameter. Specifically, the useproxy, proxyChecker, and proxybannedcleanup settings are taken from the parser that calls await this.parser.request.

In overrideParams, you can override the parameters of the called parser; the following flags are also available:

overrideParams.resultArraysWithObjects

resultArraysWithObjects: 0 - determines the format in which to return the result arrays of the called parser:

• if enabled (1) - arrays of objects will be returned

  [{link: 'link1', anchor: 'anchor1'}, {link: 'link2', anchor: 'anchor2'}, ...]
• if disabled (0) - standard arrays of values will be returned

  ['link1', 'anchor1', 'link2', 'anchor2', ...]

Example:

import { BaseParser } from 'a-parser-types';

class JS_DocExample extends BaseParser {
    static defaultConf: typeof BaseParser.defaultConf = {
        results_format: "$links.format('$link\n')",
        results: {
            arrays: {
                links: ['Links', [
                    ['link', 'link']
                ]]
            }
        }
    }

    async parse(set, results) {
        let response = await this.parser.request('SE::Google', 'default', {
            resultArraysWithObjects: 1,
            pagecount: 1
        }, set.query)
        results.success = response.success;
        
        if(response.success) {
            response.serp.forEach(element => {
                results.links.push(element.link);
            });
        }

        return results;
    }
}

Result example:

https://www.speedtest.net/
https://www.investopedia.com/terms/t/t-test.asp
https://www.cdc.gov/coronavirus/2019-ncov/testing/diagnostic-testing.html
https://fast.com/
https://www.thinkwithgoogle.com/feature/testmysite/
https://projectstream.google.com/speedtest
https://www.nhs.uk/conditions/coronavirus-covid-19/testing/
https://www.fda.gov/consumers/consumer-updates/coronavirus-disease-2019-testing-basics
https://zoom.us/test
https://www.gov.uk/get-coronavirus-test
https://en.wikipedia.org/wiki/Test_(assessment)
...

overrideParams.needData

needData: 1 - determines whether to pass (1) or not (0) data/pages[] in the response; can be used for optimization

overrideParams.needResults

needResults: [ ... ] - a list of results that need to be returned.

Example:

let response = await this.parser.request('SE::Bing', 'default', {
    needResults: [
        'totalcount'
    ]
}, set.query)

Result:

{"success":1,"info":{"success":1,"retries":0},"totalcount":"2130000000"}

tip

Works similarly for API::Server::Redis

overrideParams.skipProxySettingsInheritance

skipProxySettingsInheritance: 0 - this option allows disabling the inheritance of the useproxy parameter for the called parser

Requests to parsers supporting bulk mode

Calling parsers operating in bulk mode is also supported. To do this, an array of queries like ['key1', 'key2', ...] must be passed in query.

The result of processing by the called parser will be contained in the bulkResults array; an example of such an array is below:

{
    "bulkResults": [
        {
            "success": 1,
            "someArrayResult": [...],
            "someFlatResult": '...',
            "query": "key1",
            "data": "..."
        },
        {
            "success": 1,
            "someArrayResult": [...],
            "someFlatResult": '...',
            "query": "key2",
            "data": "..."
        },
        {
            "success": 1,
            "someArrayResult": [...],
            "someFlatResult": '...',
            "query": "key3",
            "data": "..."
        }
    ],
    "success": 1,
    "info": {
        "success": 1,
        "retries": 1
    }
}

Example of calling another parser in bulk mode

import { BaseParser } from 'a-parser-types';

export class JS_Example_BulkQueries extends BaseParser {
    static defaultConf: typeof BaseParser.defaultConf = {
        version: '1.0.0',
        results: {
            flat: [
                ['views', 'Views count per month']
            ]
        },
        results_format: "$query: $views\\n",
        SE_Yandex_Direct_Frequency_preset: 'default',
        bulkQueries: 10 // setting the number of queries in a "batch"
    };

    static editableConf: typeof BaseParser.editableConf = [
        ['SE_Yandex_Direct_Frequency_preset', ['combobox', 'SE::Yandex::Direct::Frequency preset']]
    ];

    async parse(set, results) {
        const { success, bulkResults } = await this.parser.request(
            'SE::Yandex::Direct::Frequency',
            this.conf.SE_Yandex_Direct_Frequency_preset,
            { useAccounts: 1 },
            set.bulkQueries.map((el) => el.query) // converting the set.bulkQueries object array into an array like [query1, query2, ... query10]
        );
        if(success) {
            // filling results into results.bulkResults
            for(let query_number = 0; query_number < set.bulkQueries.length; query_number++) {
                results.bulkResults[query_number].views = bulkResults[query_number].views;
                results.bulkResults[query_number].success = bulkResults[query_number].success;
            }
        }

        return results;
    }
}