API Methods

`ping`

Check the server and API operation.

Request example

{
    "password": "pass",
    "action": "ping"
}

Response example

{
    "success": 1,
    "data": "pong"
}

`oneRequest`

Single parsing request, any scraper and preset can be used. As a result, a string will be formed in accordance with the result format specified in the preset, as well as a full log of the scraper's work.

Request example

{
    "password": "pass",
    "action": "oneRequest",
    "data": {
        "query": "test",
        "parser": "SE::Google",
        "configPreset": "default",
        "preset": "Pages Count use Proxy"
    }
}

Description of possible parameters:

parser - which scraper to use for the request
preset - which preset to use
configPreset - which stream config to use
query - the request itself
rawResults - optional parameter, if set, instead of the resulting string resultString will return an array of results that the specified scraper supports
needData - optional parameter indicating whether to pass data and pages in the response, used to save memory, disabled by default
doLog - optional parameter indicating whether to return the work log along with the results, enabled by default
options - an array of additional options applied to the scraper, for example - override - allows you to override values in the preset

Response example

{
    "success": 1,
    "data": {
        "resultString": "test: 6070000000\n",
        "logs": [
            [
                0,
                1614933603,
                "Parser SE::Google::0 parse query test"
            ],
            [
                0,
                1614933603,
                "Parse page 1"
            ],
            ...
            [
                0,
                1614933604,
                "Thread complete work"
            ]
        ]
    }
}

Description of variables in the response:

resultString - result string
logs - array of request execution logs

Overriding scraper parameters

In options, you can override any parameters of the used scraper, they will be used on top of the values specified in the settings preset.

Request example:

{
    "password": "pass",
    "action": "oneRequest",
    "data": {
        "parser": "SE::Google",
        "preset": "default",
        "query": "test",
        "rawResults": 1,
        "doLog": 0,
        "options": [
            {
                "value": 1,
                "type": "override",
                "id": "pagecount"
            },
            {
                "value": 10,
                "type": "override",
                "id": "linksperpage"
            },
            {
                "type": "override",
                "id": "useproxy",
                "value": false
            }
        ]
    }
}

Description of possible variables in data:

threads - number of threads for parsing
queries - array of requests Other variables are completely analogous to oneRequest, except for query

Response:

{
    "success": 1,
    "data": {
        "logs": [],
        "results": [
            {
                "ads": [],
                "success": 1,
                "info": {
                    "success": 1,
                    "retries": 0,
                    "stats": {
                        "success": 1,
                        "retries": 0,
                        "proxiesUsed": 0,
                        "requests": 0,
                        "queries": 1
                    }
                },
                "query": {
                    "first": "test",
                    "threadId": "0",
                    "lvl": 0,
                    "query": "test",
                    "queryUid": "0",
                    "orig": "test",
                    "prefered_proxy": null
                },
                "totalcount": "6130000000",
                "serp": [
                    "https://www.test.de/",
                    "Stiftung Warentest",
                    "Sichern Sie sich mit der <em>test</em>.de-Flatrate freien Zugriff auf alle <em>Tests</em> und Produktfinder. ï»¿Mehr erfahren: <em>test</em>.de-Flatrates. <em>Test</em> CO2&nbsp;...",
                    0,
                    ...
                ],
                "related": [
                    "test <b>speed</b>",
                    ...
                ],
                "detected_geo": "Germany",
                "rich": [
                    "Featured snippet"
                ],
                "misspell": 0,
                "resultsCount": 11
            }
        ]
    }
}

`bulkRequest`

Mass parsing request, any scraper and preset can be used, and you can also specify in how many threads to perform parsing. As a result, a string will be formed in accordance with the result format specified in the preset, as well as a full log of the scraper's work for each thread.

Request example

{
    "password": "pass",
    "action": "bulkRequest",
    "data": {
        "parser": "SE::Google",
        "preset": "Pages Count no Proxy",
        "configPreset": "default",
        "threads": 3,
        "rawResults": 1,
        "queries": [
            "test1",
            "test2",
            "test3",
            "test4",
            "test5"
        ]
    }
}

Response example

{
    "success": 1,
    "data": {
        "logs": {
            "0": {
                "1": [
                    [
                        4,
                        1614935687,
                        "SE::Google::0",
                        "test1"
                    ],
                    ...
                    [
                        0,
                        1614935688,
                        "Thread complete work"
                    ]
                ]
            },
            ...лог по остальным потокам...
        },
        "results": [
            {
                "ads": [],
                "success": 1,
                "info": {
                    "success": 1,
                    "retries": 0,
                    "stats": {
                        "success": 1,
                        "retries": 0,
                        "proxiesUsed": 0,
                        "requests": 0,
                        "queries": 1
                    }
                },
                "query": {
                    "first": "test1",
                    "threadId": "0",
                    "lvl": 0,
                    "query": "test1",
                    "queryUid": "0",
                    "orig": "test1",
                    "prefered_proxy": null
                },
                "totalcount": "35800000",
                "serp": [
                    "https://www.test1solutions.com/",
                    "FoamFlex200 | Oil Spill Absorbent | Oil Disaster solution",
                    "<em>TEST1</em> IS SOLUTION. In addition to the different models&nbsp;...",
                    0,
                    ...
                ],
                "related": [
                    "<b>test2</b>",
                    ...
                ],
                "detected_geo": "Germany",
                "rich": [
                    "Featured snippet",
                    "People also ask"
                ],
                "misspell": 0,
                "resultsCount": 12
            },
            ...результаты по остальным запросам...
        ]
    }
}

`addTask`

Adding a task to the queue, all parameters are similar to those set in the task editor interface

tip

To quickly get the full task configuration, you can use the Task Editor, create a task and get the JSON for the API request

Request example

{
    "password": "pass",
    "action": "addTask",
    "data": {
        "preset": "default",
        "configPreset": "100 Threads",
        "parsers": [
            [
                "SE::Bing",
                "default",
                {
                    "type": "override",
                    "id": "pagecount",
                    "value": 1
                },
                {
                    "type": "options",
                    "id": "parseAll",
                    "value": true
                }
            ]
        ],
        "resultsFormat": "$p1.serp.format('$link;$anchor\\n')",
        "resultsSaveTo": "file",
        "resultsFileName": "$datefile.format().csv",
        "additionalFormats": [],
        "resultsUnique": "string",
        "queriesFrom": "text",
        "queryFormat": [
            "$query"
        ],
        "uniqueQueries": true,
        "saveFailedQueries": false,
        "iteratorOptions": {
            "onAllLevels": false,
            "queryBuildersAfterIterator": false,
            "queryBuildersOnAllLevels": false
        },
        "resultsOptions": {
            "overwrite": false,
            "writeBOM": true
        },
        "doLog": "db",
        "limitLogsCount": "0",
        "keepUnique": "No",
        "moreOptions": true,
        "resultsPrepend": "Ссылка;Анкор\n",
        "resultsAppend": "",
        "queryBuilders": [],
        "resultsBuilders": [],
        "configOverrides": [],
        "runTaskOnComplete": null,
        "useResultsFileAsQueriesFile": false,
        "runTaskOnCompleteConfig": "default",
        "toolsJS": "",
        "prio": 5,
        "removeOnComplete": false,
        "callURLOnComplete": "",
        "queries": [
            "test",
            "bla-bla"
        ]
    }
}

Response example

The response contains the id of the created task.

{
    "success": 1,
    "data": "697403"
}

Task with requests from a file

To make requests in the task come from a file, you need to set "queriesFrom": "file" and the path(s) to the request file(s) "queriesFile": ["queries1.txt", "queries2.txt"]. Otherwise, everything is the same as in the example above.

Request example

{
    "password": "pass",
    "action": "addTask",
    "data": {
        "preset": "default",
        "configPreset": "100 Threads",
        "parsers": [
            [
                "SE::Google",
                "default"
            ]
        ],
        "resultsFormat": "$p1.preset",
        "resultsSaveTo": "file",
        "resultsFileName": "$datefile.format().csv",
        "additionalFormats": [],
        "resultsUnique": "no",
        "queriesFrom": "file",
        "queryFormat": [
            "$query"
        ],
        "uniqueQueries": false,
        "saveFailedQueries": false,
        "iteratorOptions": {
            "onAllLevels": false,
            "queryBuildersAfterIterator": false,
            "queryBuildersOnAllLevels": false
        },
        "resultsOptions": {
            "overwrite": false,
            "writeBOM": false
        },
        "doLog": "no",
        "limitLogsCount": "0",
        "keepUnique": "No",
        "moreOptions": false,
        "resultsPrepend": "",
        "resultsAppend": "",
        "queryBuilders": [],
        "resultsBuilders": [],
        "configOverrides": [],
        "runTaskOnComplete": null,
        "useResultsFileAsQueriesFile": false,
        "runTaskOnCompleteConfig": "default",
        "toolsJS": "",
        "prio": 5,
        "removeOnComplete": false,
        "callURLOnComplete": "",
        "queriesFile": [
            "queries/Text Document.txt"
        ]
    }
}

Response example

{
    "success": 1,
    "data": "5432"
}

Running a previously saved task

You can run a preset previously created through the interface. In this case, it is only necessary to specify the requests. You can also override any of the task parameters, they will be used on top of the values in the preset.

Request example

{
    "password": "pass",
    "action": "addTask",
    "data": {
        "queriesFrom": "text",
        "queries": [
            "google.com",
            "yandex.ru"
        ],
        "configPreset": "default",
        "preset": "Analyze Domains"
    }
}

Response example

{
    "success": 1,
    "data": "5436"
}

Flag `removeOnRestart`

This flag indicates that tasks will be deleted when the scraper is restarted.

Request example

{
    "password": "pass",
    "action": "addTask",
    "data": {
        "queriesFrom": "text",
        "queries": [
            "google.com",
            "yandex.ru"
        ],
        "configPreset": "default",
        "preset": "Analyze Domains",
        "removeOnRestart": 1
    }
}

Response example

{
    "success": 1,
    "data": "5437"
}

Flag `removeOnComplete`

This flag indicates that tasks will be deleted after completion.

Request example

{
    "password": "pass",
    "action": "addTask",
    "data": {
        "queriesFrom": "text",
        "queries": [
            "google.com",
            "yandex.ru"
        ],
        "configPreset": "default",
        "preset": "Analyze Domains",
        "removeOnComplete": 1
    }
}

Response example

{
    "success": 1,
    "data": "5438"
}

`info`

Getting general information about the state of A-Parser and getting a list of all available scrapers.

tip

You can get the number of the current version of A-Parser directly at the link: https://a-parser.com/members/versions

Example: {"lastBetaVersion":"1.2.1484","lastAlphaVersion":"1.2.1484","lastVersion":"1.2.1432"}

Request example

{
    "password": "pass",
    "action": "info"
}

Response example

{
    "tasksInQueue": 0,
    "pid": "13968",
    "activeProxyCheckerThreads": 0,
    "workingTasks": 0,
    "activeThreads": 0,
    "version": "1.2.1151",
    "availableParsers": [
        "API::Server::Redis",
        "Check::BackLink",
        "Check::RosKomNadzor",
        "DeepL::Translator",
        "GooglePlay::Apps",
        "HTML::EmailExtractor",
        ...
        "Util::AntiGate",
        "Util::ReCaptcha2",
        "Util::YandexRecognize"
    ]
}

`getParserPreset`

Getting the settings of the specified scraper and preset.

tip

With this method, you can get a complete list of parameters for use in other API requests.

Request example

{
    "password": "pass",
    "action": "getParserPreset",
    "data": {
        "parser": "SE::Google",
        "preset": "default"
    }
}

Response example

{
    "success": 1,
    "data": {
        "queryformat": "$query",
        "parsenotfound": 1,
        "reCaptchaRetries": 3,
        "pagecount": 5,
        "gl": "",
        "proxyChecker": "*",
        "hl": "en",
        "domain": "www.google.com",
        "timeout": 60,
        "Util_ReCaptcha2_preset": "default",
        "useproxy": 1,
        "nfpr": 0,
        "extraquery": "",
        "serptime": "all",
        "location": "",
        "usesessions": 1,
        "filter": 1,
        "linksperpage": 100,
        "dontTakeSession": 0,
        "addHeaders": "",
        "serp": "",
        "proxyretries": 10,
        "device": "desktop",
        "requestdelay": 0,
        "debug_nonexists_domains": 0,
        "proxybannedcleanup": 600,
        "formatresult": "$serp.format('$link\\n')",
        "reCaptchaPassProxy": 0,
        "lr": ""
    }
}

`getProxies`

Request a list of live proxies. A list of live proxies from all proxy checkers is returned.

Request example

{
    "password": "pass",
    "action": "getProxies"
}

Response example

{
    "success": 1,
    "data": {
        "127.0.0.1:23486": [
            "socks"
        ],
        "127.0.0.1:23140": [
            "socks"
        ],
        "127.0.0.1:21971": [
            "http"
        ]
    }
}

The IP:port of the proxy will be indicated as the name of the array. The first element of the array is the type of proxy, it can take 3 values - http, socks, socks4. If authorization by login/password is specified, the second and third elements will be the login and password.

You can also get a list of proxies only from certain proxy checkers. To do this, you need to additionally pass the checkers array.

Example:

{
    "password": "pass",
    "action": "getProxies",
    "data": {
        "checkers": [
            "Elite proxies",
            "free proxies"
        ]
    }
}

`getTaskState`

Getting the state of a task by its id.

Example of a request

{
    "password": "pass",
    "action": "getTaskState",
    "data": {
        "taskUid": "181"
    }
}

Example of a response

{
    "success": 1,
    "data": {
        "status": "completed",
        "stats": "<b>Overall stats</b><br>Runtime: 0:00:19<br>HTTP requests: 464<br><br><b>1. HTML::LinkExtractor</b><br>Queries done: 254<br>Successful queries: 252<br>Proxies used: 0 (per query)<br>Retries used: 1.07 (per query)<br>HTTP requests: 1.82 (per query)",
        "state": {
            "totalFail": 2,
            "totalWaitProxyThreads": 0,
            "minimized": 0,
            "queriesDoneCount": 254,
            "avgSpeed": 802,
            "activeThreads": 0,
            "startTime": 1507281122,
            "changeTime": 1507281141,
            "queriesCount": 1,
            "logExists": 0,
            "runTime": 19,
            "uniqueResultsCount": 656,
            "requests": "464",
            "addTime": 1507281120,
            "additionalCount": 253,
            "queriesDoneCountAtStart": 0,
            "lastQuery": "https://www.nytimes.com/ref/membercenter/help/infoservdirectory.html",
            "curSpeed": 846,
            "started": 1,
            "resultsCount": 31079
        }
    }
}

The response contains the task status (status) and its statistics (state).

You can also get information about several tasks at once by passing an array of ids.

Example of a request

{
    "password": "pass",
    "action": "getTaskState",
    "data": {
        "taskUid": [
            "22",
            "23",
            "31"
        ]
    }
}

In this case, the response will be an array with data for each task.

Example of a response

{
    "success": 1,
    "data": [
        {
            "status": "completed",
            "stats": "<b>Overall stats</b><br>Runtime: 0:00:01<br>HTTP requests: 0<br><br><b>1. SE::Bing</b><br>Queries done: 0<br>Successful queries: 0<br>Proxies used: 0 (per query)<br>Retries used: 0 (per query)<br>HTTP requests: 0 (per query)",
            "state": {
                "totalFail": 0,
                "totalWaitProxyThreads": 0,
                "minimized": 0,
                "queriesDoneCount": 0,
                "avgSpeed": 0,
                "activeThreads": 0,
                "startTime": 1507023540,
                "changeTime": 1507023541,
                "queriesCount": 1,
                "logExists": 1,
                "runTime": 1,
                "uniqueResultsCount": 0,
                "requests": 0,
                "addTime": 1507023443,
                "additionalCount": 0,
                "queriesDoneCountAtStart": 0,
                "lastQuery": "none",
                "curSpeed": 0,
                "started": 1,
                "resultsCount": 0
            }
        },
        ...данные по остальным заданиям...
    ]
}

`getTaskConf`

Getting the task configuration by its id.

Example of a request

{
    "password": "pass",
    "action": "getTaskConf",
    "data": {
        "taskUid": "181"
    }
}

Example of a response

The response contains the task settings, including the resulting file name.

{
    "success": 1,
    "data": {
        "parsers": [
            [
                "SE::Bing",
                "default",
                {
                    "value": 1,
                    "type": "override",
                    "id": "pagecount"
                }
            ]
        ],
        "resultsFileName": "Mar-05_13-12-23.txt",
        "runTaskOnComplete": null,
        "limitLogsCount": "0",
        "resultsPrepend": "",
        "origResultsFileName": "$datefile.format().txt",
        "queriesFrom": "text",
        "runTaskOnCompleteConfig": "default",
        "doLog": "db",
        "useResultsFileAsQueriesFile": 0,
        "additionalFormats": [],
        "resultsSaveTo": "file",
        "callURLOnComplete": "",
        "resultsFormat": "$p1.related.format('$key\\n')",
        "queryBuilders": [],
        "preset": "default",
        "resultsAppend": "",
        "uniqueQueries": 0,
        "keepUnique": 0,
        "prio": "5",
        "saveFailedQueries": 0,
        "configPreset": "100 Threads",
        "queries": [
            "test"
        ],
        "toolsJS": "",
        "moreOptions": 0,
        "resultsBuilders": [],
        "resultsUnique": "string",
        "iteratorOptions": {
            "onAllLevels": 0,
            "queryBuildersAfterIterator": 0,
            "queryBuildersOnAllLevels": 0
        },
        "removeOnComplete": 0,
        "queryFormat": [
            "$query"
        ],
        "configOverrides": [],
        "resultsOptions": {
            "overwrite": 0,
            "writeBOM": 0
        }
    }
}

`getTaskResultsFile`

Getting a link to download the result by task id. You can download the file only once using the received link, without authorization (a one-time token is used).

info

Works only with a static file name and $datefile.format(). To turn a dynamic result file name into a static one, you can use the isStaticTemplate() template toolkit flag.

Example of a request

{
    "password": "pass",
    "action": "getTaskResultsFile",
    "data": {
        "taskUid": "181"
    }
}

Example of a response

{
    "success": 1,
    "data": "http://127.0.0.1:9091/downloadResults?fileName=Mar-05_13-12-23.txt&token=wbvwlkes"
}

`getTasksList`

Getting a list of active tasks. If you pass an additional parameter completed: 1, you will get a list of completed tasks.

Example of a request

{
    "password": "pass",
    "action": "getTasksList",
    "data": {
        "completed": "1"
    }
}

Example of a response

{
    "success": 1,
    "data": [
        "2291",
        "2324",
        "2331",
        "2384",
        "2398",
        "2434",
        "2445",
        "3482",
        ...
    ]
}

`getParserInfo`

Displays a list of all available results that the specified scraper can return.

Example of a request

{
   "password" : "pass",
   "action" : "getParserInfo",
   "data" : {
      "parser" : "SE::Google"
   }
}

Example of a response

{
    "success": 1,
    "data": {
        "results": {
            "arrays": {
                "ads": [
                    "Ads list",
                    [
                        [
                            "link",
                            "Link"
                        ],
                        ...
                    ]
                ],
                "related": [
                    "Related keywords",
                    [
                        [
                            "key",
                            "Key"
                        ]
                    ]
                ],
                "rich": [
                    "Rich snippets list",
                    [
                        [
                            "name",
                            "Name"
                        ]
                    ]
                ],
                "serp": [
                    "Main serp list",
                    [
                        [
                            "link",
                            "Link"
                        ],
                        ...
                    ]
                ],
                "pages": [
                    "Raw data array",
                    [
                        [
                            "data",
                            "Raw data"
                        ]
                    ]
                ]
            },
            "flat": [
                [
                    "query",
                    "Formatted query"
                ],
                ...
            ]
        }
    }
}

`getAccountsCount`

Getting the number of active Yandex accounts.

Example of a request

{
    "password": "pass",
    "action": "getAccountsCount"
}

Example of a response

{
    "success": 1,
    "data": {
        "SE::Yandex": 18
    }
}

`deleteTaskResultsFile`

Deleting the result file by task id.

Example of a request

{
    "password": "pass",
    "action": "deleteTaskResultsFile",
    "data": {
        "taskUid": "181"
    }
}

Example of a response

{
    "success": 1
}

`changeTaskStatus`

Changing the task status by its id. There are only 4 states that a task can be switched to:

starting - starting the task
pausing - pausing the task
stopping - stopping the task
deleting - deleting the task

Example of a request

{
    "password": "pass",
    "action": "changeTaskStatus",
    "data": {
        "taskUid": "181",
        "toStatus": "deleting"
    }
}

Example of a response

{
    "success": 1
}

`changeProxyCheckerState`

Changing the state of the proxy checker (1 - enabled / 0 - disabled).

Example of a request

{
    "password": "pass",
    "action": "changeProxyCheckerState",
    "data": {
        "checker": "proxychecker name",
        "state": 1
    }
}

Example of a response

{
    "success": 1
}

`moveTask`

Moving the task in the queue by its id. Possible directions of movement:

start - to the beginning of the queue
end - to the end of the queue
up - one position up
down - one position down

Example of a request

{
    "password": "pass",
    "action": "moveTask",
    "data": {
        "taskUid": "181",
        "direction": "start"
    }
}

Example of a response

{
    "success": 1
}

`update`

Updates the scraper executable to the latest available version. After sending the command, A-Parser will be automatically restarted. The API will return a success response after loading and updating the executable, which may take 1-3 minutes.

Example of a request

{
    "password": "pass",
    "action": "update"
}

Example of a response

{
    "success": 1
}

ping​

oneRequest​

Overriding scraper parameters​

bulkRequest​

addTask​

Task with requests from a file​

Running a previously saved task​

Flag removeOnRestart​

Flag removeOnComplete​

info​

getParserPreset​

getProxies​

getTaskState​

getTaskConf​

getTaskResultsFile​

getTasksList​

getParserInfo​

getAccountsCount​

deleteTaskResultsFile​

changeTaskStatus​

changeProxyCheckerState​

moveTask​

update​

`ping`

`oneRequest`

Overriding scraper parameters

`bulkRequest`

`addTask`

Task with requests from a file

Running a previously saved task

Flag `removeOnRestart`

Flag `removeOnComplete`

`info`

`getParserPreset`

`getProxies`

`getTaskState`

`getTaskConf`

`getTaskResultsFile`

`getTasksList`

`getParserInfo`

`getAccountsCount`

`deleteTaskResultsFile`

`changeTaskStatus`

`changeProxyCheckerState`

`moveTask`

`update`