API Methods
ping
β
Check the server and API operation.
Request example
{
"password": "pass",
"action": "ping"
}
Response example
{
"success": 1,
"data": "pong"
}
oneRequest
β
Single parsing request, any scraper and preset can be used. As a result, a string will be formed in accordance with the result format specified in the preset, as well as a full log of the scraper's work.
Request example
{
"password": "pass",
"action": "oneRequest",
"data": {
"query": "test",
"parser": "SE::Google",
"configPreset": "default",
"preset": "Pages Count use Proxy"
}
}
Description of possible parameters:
parser
- which scraper to use for the requestpreset
- which preset to useconfigPreset
- which stream config to usequery
- the request itselfrawResults
- optional parameter, if set, instead of the resulting string resultString will return an array of results that the specified scraper supportsneedData
- optional parameter indicating whether to pass data and pages in the response, used to save memory, disabled by defaultdoLog
- optional parameter indicating whether to return the work log along with the results, enabled by defaultoptions
- an array of additional options applied to the scraper, for example - override - allows you to override values in the preset
Response example
{
"success": 1,
"data": {
"resultString": "test: 6070000000\n",
"logs": [
[
0,
1614933603,
"Parser SE::Google::0 parse query test"
],
[
0,
1614933603,
"Parse page 1"
],
...
[
0,
1614933604,
"Thread complete work"
]
]
}
}
Description of variables in the response:
resultString
- result stringlogs
- array of request execution logs
Overriding scraper parametersβ
In options
, you can override any parameters of the used scraper, they will be used on top of the values specified in the settings preset.
Request example:
{
"password": "pass",
"action": "oneRequest",
"data": {
"parser": "SE::Google",
"preset": "default",
"query": "test",
"rawResults": 1,
"doLog": 0,
"options": [
{
"value": 1,
"type": "override",
"id": "pagecount"
},
{
"value": 10,
"type": "override",
"id": "linksperpage"
},
{
"type": "override",
"id": "useproxy",
"value": false
}
]
}
}
Description of possible variables in data
:
threads
- number of threads for parsingqueries
- array of requests Other variables are completely analogous to oneRequest, except forquery
Response:
{
"success": 1,
"data": {
"logs": [],
"results": [
{
"ads": [],
"success": 1,
"info": {
"success": 1,
"retries": 0,
"stats": {
"success": 1,
"retries": 0,
"proxiesUsed": 0,
"requests": 0,
"queries": 1
}
},
"query": {
"first": "test",
"threadId": "0",
"lvl": 0,
"query": "test",
"queryUid": "0",
"orig": "test",
"prefered_proxy": null
},
"totalcount": "6130000000",
"serp": [
"https://www.test.de/",
"Stiftung Warentest",
"Sichern Sie sich mit der <em>test</em>.de-Flatrate freien Zugriff auf alle <em>Tests</em> und Produktfinder. Mehr erfahren: <em>test</em>.de-Flatrates. <em>Test</em> CO2 ...",
0,
...
],
"related": [
"test <b>speed</b>",
...
],
"detected_geo": "Germany",
"rich": [
"Featured snippet"
],
"misspell": 0,
"resultsCount": 11
}
]
}
}
bulkRequest
β
Mass parsing request, any scraper and preset can be used, and you can also specify in how many threads to perform parsing. As a result, a string will be formed in accordance with the result format specified in the preset, as well as a full log of the scraper's work for each thread.
Request example
{
"password": "pass",
"action": "bulkRequest",
"data": {
"parser": "SE::Google",
"preset": "Pages Count no Proxy",
"configPreset": "default",
"threads": 3,
"rawResults": 1,
"queries": [
"test1",
"test2",
"test3",
"test4",
"test5"
]
}
}
Response example
{
"success": 1,
"data": {
"logs": {
"0": {
"1": [
[
4,
1614935687,
"SE::Google::0",
"test1"
],
...
[
0,
1614935688,
"Thread complete work"
]
]
},
...Π»ΠΎΠ³ ΠΏΠΎ ΠΎΡΡΠ°Π»ΡΠ½ΡΠΌ ΠΏΠΎΡΠΎΠΊΠ°ΠΌ...
},
"results": [
{
"ads": [],
"success": 1,
"info": {
"success": 1,
"retries": 0,
"stats": {
"success": 1,
"retries": 0,
"proxiesUsed": 0,
"requests": 0,
"queries": 1
}
},
"query": {
"first": "test1",
"threadId": "0",
"lvl": 0,
"query": "test1",
"queryUid": "0",
"orig": "test1",
"prefered_proxy": null
},
"totalcount": "35800000",
"serp": [
"https://www.test1solutions.com/",
"FoamFlex200 | Oil Spill Absorbent | Oil Disaster solution",
"<em>TEST1</em> IS SOLUTION. In addition to the different models ...",
0,
...
],
"related": [
"<b>test2</b>",
...
],
"detected_geo": "Germany",
"rich": [
"Featured snippet",
"People also ask"
],
"misspell": 0,
"resultsCount": 12
},
...ΡΠ΅Π·ΡΠ»ΡΡΠ°ΡΡ ΠΏΠΎ ΠΎΡΡΠ°Π»ΡΠ½ΡΠΌ Π·Π°ΠΏΡΠΎΡΠ°ΠΌ...
]
}
}
addTask
β
Adding a task to the queue, all parameters are similar to those set in the task editor interface
To quickly get the full task configuration, you can use the Task Editor, create a task and get the JSON for the API request
Request example
{
"password": "pass",
"action": "addTask",
"data": {
"preset": "default",
"configPreset": "100 Threads",
"parsers": [
[
"SE::Bing",
"default",
{
"type": "override",
"id": "pagecount",
"value": 1
},
{
"type": "options",
"id": "parseAll",
"value": true
}
]
],
"resultsFormat": "$p1.serp.format('$link;$anchor\\n')",
"resultsSaveTo": "file",
"resultsFileName": "$datefile.format().csv",
"additionalFormats": [],
"resultsUnique": "string",
"queriesFrom": "text",
"queryFormat": [
"$query"
],
"uniqueQueries": true,
"saveFailedQueries": false,
"iteratorOptions": {
"onAllLevels": false,
"queryBuildersAfterIterator": false,
"queryBuildersOnAllLevels": false
},
"resultsOptions": {
"overwrite": false,
"writeBOM": true
},
"doLog": "db",
"limitLogsCount": "0",
"keepUnique": "No",
"moreOptions": true,
"resultsPrepend": "Π‘ΡΡΠ»ΠΊΠ°;ΠΠ½ΠΊΠΎΡ\n",
"resultsAppend": "",
"queryBuilders": [],
"resultsBuilders": [],
"configOverrides": [],
"runTaskOnComplete": null,
"useResultsFileAsQueriesFile": false,
"runTaskOnCompleteConfig": "default",
"toolsJS": "",
"prio": 5,
"removeOnComplete": false,
"callURLOnComplete": "",
"queries": [
"test",
"bla-bla"
]
}
}
Response example
The response contains the id of the created task.
{
"success": 1,
"data": "697403"
}
Task with requests from a fileβ
To make requests in the task come from a file, you need to set "queriesFrom": "file"
and the path(s) to the request file(s) "queriesFile": ["queries1.txt", "queries2.txt"]
. Otherwise, everything is the same as in the example above.
Request example
{
"password": "pass",
"action": "addTask",
"data": {
"preset": "default",
"configPreset": "100 Threads",
"parsers": [
[
"SE::Google",
"default"
]
],
"resultsFormat": "$p1.preset",
"resultsSaveTo": "file",
"resultsFileName": "$datefile.format().csv",
"additionalFormats": [],
"resultsUnique": "no",
"queriesFrom": "file",
"queryFormat": [
"$query"
],
"uniqueQueries": false,
"saveFailedQueries": false,
"iteratorOptions": {
"onAllLevels": false,
"queryBuildersAfterIterator": false,
"queryBuildersOnAllLevels": false
},
"resultsOptions": {
"overwrite": false,
"writeBOM": false
},
"doLog": "no",
"limitLogsCount": "0",
"keepUnique": "No",
"moreOptions": false,
"resultsPrepend": "",
"resultsAppend": "",
"queryBuilders": [],
"resultsBuilders": [],
"configOverrides": [],
"runTaskOnComplete": null,
"useResultsFileAsQueriesFile": false,
"runTaskOnCompleteConfig": "default",
"toolsJS": "",
"prio": 5,
"removeOnComplete": false,
"callURLOnComplete": "",
"queriesFile": [
"queries/Text Document.txt"
]
}
}
Response example
{
"success": 1,
"data": "5432"
}
Running a previously saved taskβ
You can run a preset previously created through the interface. In this case, it is only necessary to specify the requests. You can also override any of the task parameters, they will be used on top of the values in the preset.
Request example
{
"password": "pass",
"action": "addTask",
"data": {
"queriesFrom": "text",
"queries": [
"google.com",
"yandex.ru"
],
"configPreset": "default",
"preset": "Analyze Domains"
}
}
Response example
{
"success": 1,
"data": "5436"
}
Flag removeOnRestart
β
This flag indicates that tasks will be deleted when the scraper is restarted.
Request example
{
"password": "pass",
"action": "addTask",
"data": {
"queriesFrom": "text",
"queries": [
"google.com",
"yandex.ru"
],
"configPreset": "default",
"preset": "Analyze Domains",
"removeOnRestart": 1
}
}
Response example
{
"success": 1,
"data": "5437"
}
Flag removeOnComplete
β
This flag indicates that tasks will be deleted after completion.
Request example
{
"password": "pass",
"action": "addTask",
"data": {
"queriesFrom": "text",
"queries": [
"google.com",
"yandex.ru"
],
"configPreset": "default",
"preset": "Analyze Domains",
"removeOnComplete": 1
}
}
Response example
{
"success": 1,
"data": "5438"
}
info
β
Getting general information about the state of A-Parser and getting a list of all available scrapers.
You can get the number of the current version of A-Parser directly at the link: https://a-parser.com/members/versions
Example:
{"lastBetaVersion":"1.2.1484","lastAlphaVersion":"1.2.1484","lastVersion":"1.2.1432"}
Request example
{
"password": "pass",
"action": "info"
}
Response example
{
"tasksInQueue": 0,
"pid": "13968",
"activeProxyCheckerThreads": 0,
"workingTasks": 0,
"activeThreads": 0,
"version": "1.2.1151",
"availableParsers": [
"API::Server::Redis",
"Check::BackLink",
"Check::RosKomNadzor",
"DeepL::Translator",
"GooglePlay::Apps",
"HTML::EmailExtractor",
...
"Util::AntiGate",
"Util::ReCaptcha2",
"Util::YandexRecognize"
]
}
getParserPreset
β
Getting the settings of the specified scraper and preset.
With this method, you can get a complete list of parameters for use in other API requests.
Request example
{
"password": "pass",
"action": "getParserPreset",
"data": {
"parser": "SE::Google",
"preset": "default"
}
}
Response example
{
"success": 1,
"data": {
"queryformat": "$query",
"parsenotfound": 1,
"reCaptchaRetries": 3,
"pagecount": 5,
"gl": "",
"proxyChecker": "*",
"hl": "en",
"domain": "www.google.com",
"timeout": 60,
"Util_ReCaptcha2_preset": "default",
"useproxy": 1,
"nfpr": 0,
"extraquery": "",
"serptime": "all",
"location": "",
"usesessions": 1,
"filter": 1,
"linksperpage": 100,
"dontTakeSession": 0,
"addHeaders": "",
"serp": "",
"proxyretries": 10,
"device": "desktop",
"requestdelay": 0,
"debug_nonexists_domains": 0,
"proxybannedcleanup": 600,
"formatresult": "$serp.format('$link\\n')",
"reCaptchaPassProxy": 0,
"lr": ""
}
}
getProxies
β
Request a list of live proxies. A list of live proxies from all proxy checkers is returned.
Request example
{
"password": "pass",
"action": "getProxies"
}
Response example
{
"success": 1,
"data": {
"127.0.0.1:23486": [
"socks"
],
"127.0.0.1:23140": [
"socks"
],
"127.0.0.1:21971": [
"http"
]
}
}
The IP:port of the proxy will be indicated as the name of the array. The first element of the array is the type of proxy, it can take 3 values - http, socks, socks4. If authorization by login/password is specified, the second and third elements will be the login and password.
You can also get a list of proxies only from certain proxy checkers. To do this, you need to additionally pass the checkers
array.
Example:
{
"password": "pass",
"action": "getProxies",
"data": {
"checkers": [
"Elite proxies",
"free proxies"
]
}
}
getTaskState
β
Getting the state of a task by its id.
Example of a request
{
"password": "pass",
"action": "getTaskState",
"data": {
"taskUid": "181"
}
}
Example of a response
{
"success": 1,
"data": {
"status": "completed",
"stats": "<b>Overall stats</b><br>Runtime: 0:00:19<br>HTTP requests: 464<br><br><b>1. HTML::LinkExtractor</b><br>Queries done: 254<br>Successful queries: 252<br>Proxies used: 0 (per query)<br>Retries used: 1.07 (per query)<br>HTTP requests: 1.82 (per query)",
"state": {
"totalFail": 2,
"totalWaitProxyThreads": 0,
"minimized": 0,
"queriesDoneCount": 254,
"avgSpeed": 802,
"activeThreads": 0,
"startTime": 1507281122,
"changeTime": 1507281141,
"queriesCount": 1,
"logExists": 0,
"runTime": 19,
"uniqueResultsCount": 656,
"requests": "464",
"addTime": 1507281120,
"additionalCount": 253,
"queriesDoneCountAtStart": 0,
"lastQuery": "https://www.nytimes.com/ref/membercenter/help/infoservdirectory.html",
"curSpeed": 846,
"started": 1,
"resultsCount": 31079
}
}
}
The response contains the task status (status
) and its statistics (state
).
You can also get information about several tasks at once by passing an array of ids.
Example of a request
{
"password": "pass",
"action": "getTaskState",
"data": {
"taskUid": [
"22",
"23",
"31"
]
}
}
In this case, the response will be an array with data for each task.
Example of a response
{
"success": 1,
"data": [
{
"status": "completed",
"stats": "<b>Overall stats</b><br>Runtime: 0:00:01<br>HTTP requests: 0<br><br><b>1. SE::Bing</b><br>Queries done: 0<br>Successful queries: 0<br>Proxies used: 0 (per query)<br>Retries used: 0 (per query)<br>HTTP requests: 0 (per query)",
"state": {
"totalFail": 0,
"totalWaitProxyThreads": 0,
"minimized": 0,
"queriesDoneCount": 0,
"avgSpeed": 0,
"activeThreads": 0,
"startTime": 1507023540,
"changeTime": 1507023541,
"queriesCount": 1,
"logExists": 1,
"runTime": 1,
"uniqueResultsCount": 0,
"requests": 0,
"addTime": 1507023443,
"additionalCount": 0,
"queriesDoneCountAtStart": 0,
"lastQuery": "none",
"curSpeed": 0,
"started": 1,
"resultsCount": 0
}
},
...Π΄Π°Π½Π½ΡΠ΅ ΠΏΠΎ ΠΎΡΡΠ°Π»ΡΠ½ΡΠΌ Π·Π°Π΄Π°Π½ΠΈΡΠΌ...
]
}
getTaskConf
β
Getting the task configuration by its id.
Example of a request
{
"password": "pass",
"action": "getTaskConf",
"data": {
"taskUid": "181"
}
}
Example of a response
The response contains the task settings, including the resulting file name.
{
"success": 1,
"data": {
"parsers": [
[
"SE::Bing",
"default",
{
"value": 1,
"type": "override",
"id": "pagecount"
}
]
],
"resultsFileName": "Mar-05_13-12-23.txt",
"runTaskOnComplete": null,
"limitLogsCount": "0",
"resultsPrepend": "",
"origResultsFileName": "$datefile.format().txt",
"queriesFrom": "text",
"runTaskOnCompleteConfig": "default",
"doLog": "db",
"useResultsFileAsQueriesFile": 0,
"additionalFormats": [],
"resultsSaveTo": "file",
"callURLOnComplete": "",
"resultsFormat": "$p1.related.format('$key\\n')",
"queryBuilders": [],
"preset": "default",
"resultsAppend": "",
"uniqueQueries": 0,
"keepUnique": 0,
"prio": "5",
"saveFailedQueries": 0,
"configPreset": "100 Threads",
"queries": [
"test"
],
"toolsJS": "",
"moreOptions": 0,
"resultsBuilders": [],
"resultsUnique": "string",
"iteratorOptions": {
"onAllLevels": 0,
"queryBuildersAfterIterator": 0,
"queryBuildersOnAllLevels": 0
},
"removeOnComplete": 0,
"queryFormat": [
"$query"
],
"configOverrides": [],
"resultsOptions": {
"overwrite": 0,
"writeBOM": 0
}
}
}
getTaskResultsFile
β
Getting a link to download the result by task id. You can download the file only once using the received link, without authorization (a one-time token is used).
Works only with a static file name and $datefile.format(). To turn a dynamic result file name into a static one, you can use the isStaticTemplate() template toolkit flag.
Example of a request
{
"password": "pass",
"action": "getTaskResultsFile",
"data": {
"taskUid": "181"
}
}
Example of a response
{
"success": 1,
"data": "http://127.0.0.1:9091/downloadResults?fileName=Mar-05_13-12-23.txt&token=wbvwlkes"
}
getTasksList
β
Getting a list of active tasks. If you pass an additional parameter completed: 1
, you will get a list of completed tasks.
Example of a request
{
"password": "pass",
"action": "getTasksList",
"data": {
"completed": "1"
}
}
Example of a response
{
"success": 1,
"data": [
"2291",
"2324",
"2331",
"2384",
"2398",
"2434",
"2445",
"3482",
...
]
}
getParserInfo
β
Displays a list of all available results that the specified scraper can return.
Example of a request
{
"password" : "pass",
"action" : "getParserInfo",
"data" : {
"parser" : "SE::Google"
}
}
Example of a response
{
"success": 1,
"data": {
"results": {
"arrays": {
"ads": [
"Ads list",
[
[
"link",
"Link"
],
...
]
],
"related": [
"Related keywords",
[
[
"key",
"Key"
]
]
],
"rich": [
"Rich snippets list",
[
[
"name",
"Name"
]
]
],
"serp": [
"Main serp list",
[
[
"link",
"Link"
],
...
]
],
"pages": [
"Raw data array",
[
[
"data",
"Raw data"
]
]
]
},
"flat": [
[
"query",
"Formatted query"
],
...
]
}
}
}
getAccountsCount
β
Getting the number of active Yandex accounts.
Example of a request
{
"password": "pass",
"action": "getAccountsCount"
}
Example of a response
{
"success": 1,
"data": {
"SE::Yandex": 18
}
}
deleteTaskResultsFile
β
Deleting the result file by task id.
Example of a request
{
"password": "pass",
"action": "deleteTaskResultsFile",
"data": {
"taskUid": "181"
}
}
Example of a response
{
"success": 1
}
changeTaskStatus
β
Changing the task status by its id. There are only 4 states that a task can be switched to:
- starting - starting the task
- pausing - pausing the task
- stopping - stopping the task
- deleting - deleting the task
Example of a request
{
"password": "pass",
"action": "changeTaskStatus",
"data": {
"taskUid": "181",
"toStatus": "deleting"
}
}
Example of a response
{
"success": 1
}
changeProxyCheckerState
β
Changing the state of the proxy checker (1 - enabled / 0 - disabled).
Example of a request
{
"password": "pass",
"action": "changeProxyCheckerState",
"data": {
"checker": "proxychecker name",
"state": 1
}
}
Example of a response
{
"success": 1
}
moveTask
β
Moving the task in the queue by its id. Possible directions of movement:
- start - to the beginning of the queue
- end - to the end of the queue
- up - one position up
- down - one position down
Example of a request
{
"password": "pass",
"action": "moveTask",
"data": {
"taskUid": "181",
"direction": "start"
}
}
Example of a response
{
"success": 1
}
update
β
Updates the scraper executable to the latest available version. After sending the command, A-Parser will be automatically restarted. The API will return a success response after loading and updating the executable, which may take 1-3 minutes.
Example of a request
{
"password": "pass",
"action": "update"
}
Example of a response
{
"success": 1
}