API Methods
ping
Checking the server and API operation.
Request example
{
"password": "pass",
"action": "ping"
}
Response example
{
"success": 1,
"data": "pong"
}
oneRequest
A single scraping request, any scraper and preset can be used. The result will be a string formed in accordance with the result format specified in the preset, as well as the full log of the scraper's work.
Request example
{
"password": "pass",
"action": "oneRequest",
"data": {
"query": "test",
"parser": "SE::Google",
"configPreset": "default",
"preset": "Pages Count use Proxy"
}
}
Description of possible parameters:
parser- which scraper to use for the requestpreset- which preset to useconfigPreset- which stream config to usequery- the request itselfrawResults- an optional parameter, if set, instead of the resulting string resultString it will return an array results with all the results supported by the specified scraperneedData- an optional parameter indicating whether to pass data and pages in the response, used to save memory, disabled by defaultdoLog- an optional parameter indicating whether to return the work log along with the results, enabled by defaultoptions- an array with additional options applied to the scraper, for example - override - allows you to override values in the preset
Response example
{
"success": 1,
"data": {
"resultString": "test: 6070000000\n",
"logs": [
[
0,
1614933603,
"Parser SE::Google::0 parse query test"
],
[
0,
1614933603,
"Parse page 1"
],
...
[
0,
1614933604,
"Thread complete work"
]
]
}
}
Description of variables in the response:
resultString- result stringlogs- an array of logs from the request execution
Overriding scraper settings
In options, you can override any parameters of the used scraper, they will be applied on top of the values specified in the settings preset.
Request example:
{
"password": "pass",
"action": "oneRequest",
"data": {
"parser": "SE::Google",
"preset": "default",
"query": "test",
"rawResults": 1,
"doLog": 0,
"options": [
{
"value": 1,
"type": "override",
"id": "pagecount"
},
{
"value": 10,
"type": "override",
"id": "linksperpage"
},
{
"type": "override",
"id": "useproxy",
"value": false
}
]
}
}
Response:
{
"success": 1,
"data": {
"logs": [],
"results": [
{
"ads": [],
"success": 1,
"info": {
"success": 1,
"retries": 0,
"stats": {
"success": 1,
"retries": 0,
"proxiesUsed": 0,
"requests": 0,
"queries": 1
}
},
"query": {
"first": "test",
"threadId": "0",
"lvl": 0,
"query": "test",
"queryUid": "0",
"orig": "test",
"prefered_proxy": null
},
"totalcount": "6130000000",
"serp": [
"https://www.test.de/",
"Stiftung Warentest",
"Sichern Sie sich mit der <em>test</em>.de-Flatrate freien Zugriff auf alle <em>Tests</em> und Produktfinder. Mehr erfahren: <em>test</em>.de-Flatrates. <em>Test</em> CO2 ...",
0,
...
],
"related": [
"test <b>speed</b>",
...
],
"detected_geo": "Germany",
"rich": [
"Featured snippet"
],
"misspell": 0,
"resultsCount": 11
}
]
}
}
bulkRequest
A bulk scraping request, any scraper and preset can be used, and you can also specify the number of threads for scraping. The result will be a string formed in accordance with the result format specified in the preset, as well as the full log of the scraper's work for each thread.
Request example
{
"password": "pass",
"action": "bulkRequest",
"data": {
"parser": "SE::Google",
"preset": "Pages Count no Proxy",
"configPreset": "default",
"threads": 3,
"rawResults": 1,
"queries": [
"test1",
"test2",
"test3",
"test4",
"test5"
]
}
}
Description of possible variables in data:
threads- the number of threads for scrapingqueries- an array of requests. The rest of the variables are fully analogous to oneRequest, except forquery
Response example
{
"success": 1,
"data": {
"logs": {
"0": {
"1": [
[
4,
1614935687,
"SE::Google::0",
"test1"
],
...
[
0,
1614935688,
"Thread complete work"
]
]
},
...лог по остальным потокам...
},
"results": [
{
"ads": [],
"success": 1,
"info": {
"success": 1,
"retries": 0,
"stats": {
"success": 1,
"retries": 0,
"proxiesUsed": 0,
"requests": 0,
"queries": 1
}
},
"query": {
"first": "test1",
"threadId": "0",
"lvl": 0,
"query": "test1",
"queryUid": "0",
"orig": "test1",
"prefered_proxy": null
},
"totalcount": "35800000",
"serp": [
"https://www.test1solutions.com/",
"FoamFlex200 | Oil Spill Absorbent | Oil Disaster solution",
"<em>TEST1</em> IS SOLUTION. In addition to the different models ...",
0,
...
],
"related": [
"<b>test2</b>",
...
],
"detected_geo": "Germany",
"rich": [
"Featured snippet",
"People also ask"
],
"misspell": 0,
"resultsCount": 12
},
...результаты по остальным запросам...
]
}
}
addTask
Adding a task to the queue, all parameters are analogous to those set in the Task Editor interface
To quickly obtain the full task configuration, you can use the Task Editor, compose a task and get JSON for the API request
Request example
{
"password": "pass",
"action": "addTask",
"data": {
"preset": "default",
"configPreset": "100 Threads",
"parsers": [
[
"SE::Bing",
"default",
{
"type": "override",
"id": "pagecount",
"value": 1
},
{
"type": "options",
"id": "parseAll",
"value": true
}
]
],
"resultsFormat": "$p1.serp.format('$link;$anchor\\n')",
"resultsSaveTo": "file",
"resultsFileName": "$datefile.format().csv",
"additionalFormats": [],
"resultsUnique": "string",
"queriesFrom": "text",
"queryFormat": [
"$query"
],
"uniqueQueries": true,
"saveFailedQueries": false,
"iteratorOptions": {
"onAllLevels": false,
"queryBuildersAfterIterator": false,
"queryBuildersOnAllLevels": false
},
"resultsOptions": {
"overwrite": false,
"writeBOM": true
},
"doLog": "db",
"limitLogsCount": "0",
"keepUnique": "No",
"moreOptions": true,
"resultsPrepend": "Ссылка;Анкор\n",
"resultsAppend": "",
"queryBuilders": [],
"resultsBuilders": [],
"configOverrides": [],
"runTaskOnComplete": null,
"useResultsFileAsQueriesFile": false,
"runTaskOnCompleteConfig": "default",
"toolsJS": "",
"prio": 5,
"removeOnComplete": false,
"callURLOnComplete": "",
"queries": [
"test",
"bla-bla"
]
}
}
Response example
The response includes the id of the created task.
{
"success": 1,
"data": "697403"
}
Task with requests from a file
To have the task take requests from a file, you need to set "queriesFrom": "file" and the path(s) to the request file(s) "queriesFile": ["queries1.txt", "queries2.txt"]. Otherwise, everything is the same as in the example above.
Request example
{
"password": "pass",
"action": "addTask",
"data": {
"preset": "default",
"configPreset": "100 Threads",
"parsers": [
[
"SE::Google",
"default"
]
],
"resultsFormat": "$p1.preset",
"resultsSaveTo": "file",
"resultsFileName": "$datefile.format().csv",
"additionalFormats": [],
"resultsUnique": "no",
"queriesFrom": "file",
"queryFormat": [
"$query"
],
"uniqueQueries": false,
"saveFailedQueries": false,
"iteratorOptions": {
"onAllLevels": false,
"queryBuildersAfterIterator": false,
"queryBuildersOnAllLevels": false
},
"resultsOptions": {
"overwrite": false,
"writeBOM": false
},
"doLog": "no",
"limitLogsCount": "0",
"keepUnique": "No",
"moreOptions": false,
"resultsPrepend": "",
"resultsAppend": "",
"queryBuilders": [],
"resultsBuilders": [],
"configOverrides": [],
"runTaskOnComplete": null,
"useResultsFileAsQueriesFile": false,
"runTaskOnCompleteConfig": "default",
"toolsJS": "",
"prio": 5,
"removeOnComplete": false,
"callURLOnComplete": "",
"queriesFile": [
"queries/Text Document.txt"
]
}
}
Response example
{
"success": 1,
"data": "5432"
}
Starting a previously saved task
You can run a preset previously created through the interface. In this case, you only need to specify the requests. You can also override any of the task parameters, they will be used on top of the values in the preset.
Request example
{
"password": "pass",
"action": "addTask",
"data": {
"queriesFrom": "text",
"queries": [
"google.com",
"yandex.ru"
],
"configPreset": "default",
"preset": "Analyze Domains"
}
}
Response example
{
"success": 1,
"data": "5436"
}
Flag removeOnRestart
This flag indicates that tasks will be removed upon the scraper's restart.
Request example
{
"password": "pass",
"action": "addTask",
"data": {
"queriesFrom": "text",
"queries": [
"google.com",
"yandex.ru"
],
"configPreset": "default",
"preset": "Analyze Domains",
"removeOnRestart": 1
}
}
Response example
{
"success": 1,
"data": "5437"
}
Flag removeOnComplete
This flag indicates that tasks will be removed upon completion.
Request example
{
"password": "pass",
"action": "addTask",
"data": {
"queriesFrom": "text",
"queries": [
"google.com",
"yandex.ru"
],
"configPreset": "default",
"preset": "Analyze Domains",
"removeOnComplete": 1
}
}
Response example
{
"success": 1,
"data": "5438"
}
info
Getting general information about the state of A-Parser and obtaining a list of all available scrapers.
You can get the current version number of A-Parser directly via the link: https://a-parser.com/members/versions
Example:
{"lastBetaVersion":"1.2.1484","lastAlphaVersion":"1.2.1484","lastVersion":"1.2.1432"}
Request example
{
"password": "pass",
"action": "info"
}
Response example
{
"tasksInQueue": 0,
"pid": "13968",
"activeProxyCheckerThreads": 0,
"workingTasks": 0,
"activeThreads": 0,
"version": "1.2.1151",
"availableParsers": [
"API::Server::Redis",
"Check::BackLink",
"Check::RosKomNadzor",
"DeepL::Translator",
"GooglePlay::Apps",
"HTML::EmailExtractor",
...
"Util::AntiGate",
"Util::ReCaptcha2",
"Util::YandexRecognize"
]
}
getParserPreset
Retrieving the settings of the specified scraper and preset.
This method allows you to get a full list of parameters for use in other API requests.
Request example
{
"password": "pass",
"action": "getParserPreset",
"data": {
"parser": "SE::Google",
"preset": "default"
}
}
Response example
{
"success": 1,
"data": {
"queryformat": "$query",
"parsenotfound": 1,
"reCaptchaRetries": 3,
"pagecount": 5,
"gl": "",
"proxyChecker": "*",
"hl": "en",
"domain": "www.google.com",
"timeout": 60,
"Util_ReCaptcha2_preset": "default",
"useproxy": 1,
"nfpr": 0,
"extraquery": "",
"serptime": "all",
"location": "",
"usesessions": 1,
"filter": 1,
"linksperpage": 100,
"dontTakeSession": 0,
"addHeaders": "",
"serp": "",
"proxyretries": 10,
"device": "desktop",
"requestdelay": 0,
"debug_nonexists_domains": 0,
"proxybannedcleanup": 600,
"formatresult": "$serp.format('$link\\n')",
"reCaptchaPassProxy": 0,
"lr": ""
}
}
getProxies
Request for a list of live proxies. Returns a list of live proxies from all proxy checkers.
Request example
{
"password": "pass",
"action": "getProxies"
}
Response example
{
"success": 1,
"data": {
"127.0.0.1:23486": [
"socks"
],
"127.0.0.1:23140": [
"socks"
],
"127.0.0.1:21971": [
"http"
]
}
}
IP:port of the proxy will be indicated as the name of the array. The first element of the array is the type of proxy, which can take 3 values - http, socks, socks4. If authorization by login/password is specified, then the second and third elements will be login and password.
It is also possible to get a list of proxies only from certain proxy checkers. For this, you need to additionally pass an array checkers.
Example:
{
"password": "pass",
"action": "getProxies",
"data": {
"checkers": [
"Elite proxies",
"free proxies"
]
}
}
getTaskState
Getting the status of a task by its id.
Request example
{
"password": "pass",
"action": "getTaskState",
"data": {
"taskUid": "181"
}
}
Response example
{
"success": 1,
"data": {
"status": "completed",
"stats": "<b>Overall stats</b><br>Runtime: 0:00:19<br>HTTP requests: 464<br><br><b>1. HTML::LinkExtractor</b><br>Queries done: 254<br>Successful queries: 252<br>Proxies used: 0 (per query)<br>Retries used: 1.07 (per query)<br>HTTP requests: 1.82 (per query)",
"state": {
"totalFail": 2,
"totalWaitProxyThreads": 0,
"minimized": 0,
"queriesDoneCount": 254,
"avgSpeed": 802,
"activeThreads": 0,
"startTime": 1507281122,
"changeTime": 1507281141,
"queriesCount": 1,
"logExists": 0,
"runTime": 19,
"uniqueResultsCount": 656,
"requests": "464",
"addTime": 1507281120,
"additionalCount": 253,
"queriesDoneCountAtStart": 0,
"lastQuery": "https://www.nytimes.com/ref/membercenter/help/infoservdirectory.html",
"curSpeed": 846,
"started": 1,
"resultsCount": 31079
}
}
}
The response returns the task status (status) and its statistics (state).
You can also get information about several tasks at once by passing an array of ids.
Request example
{
"password": "pass",
"action": "getTaskState",
"data": {
"taskUid": [
"22",
"23",
"31"
]
}
}
In this case, the response will be an array with data for each task.
Response example
{
"success": 1,
"data": [
{
"status": "completed",
"stats": "<b>Overall stats</b><br>Runtime: 0:00:01<br>HTTP requests: 0<br><br><b>1. SE::Bing</b><br>Queries done: 0<br>Successful queries: 0<br>Proxies used: 0 (per query)<br>Retries used: 0 (per query)<br>HTTP requests: 0 (per query)",
"state": {
"totalFail": 0,
"totalWaitProxyThreads": 0,
"minimized": 0,
"queriesDoneCount": 0,
"avgSpeed": 0,
"activeThreads": 0,
"startTime": 1507023540,
"changeTime": 1507023541,
"queriesCount": 1,
"logExists": 1,
"runTime": 1,
"uniqueResultsCount": 0,
"requests": 0,
"addTime": 1507023443,
"additionalCount": 0,
"queriesDoneCountAtStart": 0,
"lastQuery": "none",
"curSpeed": 0,
"started": 1,
"resultsCount": 0
}
},
...данные по остальным заданиям...
]
}
getTaskConf
Getting the configuration of a task by its id.
Request example
{
"password": "pass",
"action": "getTaskConf",
"data": {
"taskUid": "181"
}
}
Response example
The response returns the task settings, including the resulting file name.
{
"success": 1,
"data": {
"parsers": [
[
"SE::Bing",
"default",
{
"value": 1,
"type": "override",
"id": "pagecount"
}
]
],
"resultsFileName": "Mar-05_13-12-23.txt",
"runTaskOnComplete": null,
"limitLogsCount": "0",
"resultsPrepend": "",
"origResultsFileName": "$datefile.format().txt",
"queriesFrom": "text",
"runTaskOnCompleteConfig": "default",
"doLog": "db",
"useResultsFileAsQueriesFile": 0,
"additionalFormats": [],
"resultsSaveTo": "file",
"callURLOnComplete": "",
"resultsFormat": "$p1.related.format('$key\\n')",
"queryBuilders": [],
"preset": "default",
"resultsAppend": "",
"uniqueQueries": 0,
"keepUnique": 0,
"prio": "5",
"saveFailedQueries": 0,
"configPreset": "100 Threads",
"queries": [
"test"
],
"toolsJS": "",
"moreOptions": 0,
"resultsBuilders": [],
"resultsUnique": "string",
"iteratorOptions": {
"onAllLevels": 0,
"queryBuildersAfterIterator": 0,
"queryBuildersOnAllLevels": 0
},
"removeOnComplete": 0,
"queryFormat": [
"$query"
],
"configOverrides": [],
"resultsOptions": {
"overwrite": 0,
"writeBOM": 0
}
}
}
getTaskResultsFile
Getting a link to download the result by the task id. The file can be downloaded only once via the received link, without authorization (a one-time token is used).
Works only with a static file name and $datefile.format(). To convert a dynamic result file name into a static one, you can use the template flag isStaticTemplate()
Request example
{
"password": "pass",
"action": "getTaskResultsFile",
"data": {
"taskUid": "181"
}
}
Response example
{
"success": 1,
"data": "http://127.0.0.1:9091/downloadResults?fileName=Mar-05_13-12-23.txt&token=wbvwlkes"
}
getTasksList
Retrieving a list of active tasks. If you pass an additional parameter completed: 1, then you will get a list of completed tasks.
Request example
{
"password": "pass",
"action": "getTasksList",
"data": {
"completed": "1"
}
}
Response example
{
"success": 1,
"data": [
"2291",
"2324",
"2331",
"2384",
"2398",
"2434",
"2445",
"3482",
...
]
}
getParserInfo
Displays a list of all available results that the specified scraper can return.
Request example
{
"password" : "pass",
"action" : "getParserInfo",
"data" : {
"parser" : "SE::Google"
}
}
Response example
{
"success": 1,
"data": {
"results": {
"arrays": {
"ads": [
"Ads list",
[
[
"link",
"Link"
],
...
]
],
"related": [
"Related keywords",
[
[
"key",
"Key"
]
]
],
"rich": [
"Rich snippets list",
[
[
"name",
"Name"
]
]
],
"serp": [
"Main serp list",
[
[
"link",
"Link"
],
...
]
],
"pages": [
"Raw data array",
[
[
"data",
"Raw data"
]
]
]
},
"flat": [
[
"query",
"Formatted query"
],
...
]
}
}
}
getAccountsCount
Getting the number of active Yandex accounts.
Request example
{
"password": "pass",
"action": "getAccountsCount"
}
Response example
{
"success": 1,
"data": {
"SE::Yandex": 18
}
}
deleteTaskResultsFile
Deleting a result file by the task id.
Request example
{
"password": "pass",
"action": "deleteTaskResultsFile",
"data": {
"taskUid": "181"
}
}
Response example
{
"success": 1
}
changeTaskStatus
Changing the status of a task by its id. There are only 4 states into which a task can be transferred:
- starting - starting the task
- pausing - pausing the task
- stopping - stopping the task
- deleting - deleting the task
Request example
{
"password": "pass",
"action": "changeTaskStatus",
"data": {
"taskUid": "181",
"toStatus": "deleting"
}
}
Response example
{
"success": 1
}
changeProxyCheckerState
Changing the state of the proxy checker (1 - enabled / 0 - disabled).
Request example
{
"password": "pass",
"action": "changeProxyCheckerState",
"data": {
"checker": "proxychecker name",
"state": 1
}
}
Response example
{
"success": 1
}
moveTask
Moving a task in the queue by its id. Possible directions of movement:
- start - to the beginning of the queue
- end - to the end of the queue
- up - one position up
- down - one position down
Request example
{
"password": "pass",
"action": "moveTask",
"data": {
"taskUid": "181",
"direction": "start"
}
}
Response example
{
"success": 1
}
update
Updates the executable file of the scraper to the latest available version. After sending the command, A-Parser will be automatically restarted. The API will return a success response after it downloads and updates the executable file, which can take 1-3 minutes.
Request example
{
"password": "pass",
"action": "update"
}
Response example
{
"success": 1
}