-
Join our Telegram chat: https://t.me/a_parser_enDismiss Notice
Tools in the Templаte toolkit
-
Contents
- Description
- $tools.memory.* - key / value storage in memory
- $tools.sqlite.* - work with SQLite databases
- $tools.ua- changeover of User agent
- $tools.query - adding of requests
- $tools.parseJSON - parsing JSON structures
- $tools.js.* - JavaScript support in tools
- $tools.base64.* - ability to encode and decode base64
- $tools.data - built-in parameter directory
- $tools.CSVline - easy creation of CSV files
- $tools.aparser.version() - Get information about the A-Parser version
- $tools.task() - returns task id and number of threads
Description(top)
In template processing system Template Toolkit the global variable $tools is available, which stores in itself a tool kit available in any template.
$tools.error - contains a description of errors that may occur during executing any of tools.
$tools.memory.* - key / value storage in memory (top)
Simple storage in memory, common to all tasks, api requests, etc., is reset when the parser is restarted
- tools.memory.set(key, value) - sets the value of the key
- tools.memory.get(key) - returns the value corresponding to the key
- tools.memory.delete(key) - deletes the record from memory
$tools.sqlite.* - work with SQLite databases(top)
- $tools.sqlite.get - method that allows you to retrieve information from a database using SELECT
Code:[% res = tools.sqlite.get('results/test.sqlite', 'SELECT COUNT(*) AS count FROM test') %]
- $tools.sqlite.run - method that allows you to perform operations with DB (INSERT, DROP, etc)
Code:[% res = tools.sqlite.run('results/test.sqlite', 'INSERT INTO test VALUES(?)', 'test') %]
- $tools.sqlite.all - method that allows you to print all the data from a table
$tools.ua- changeover of User agent(top)
This tool is intended for substitution of User agent in parsers which use it (for exampleNet::HTTP).
- $tools.ua.list - contains the User agent list
- $tools.ua.random() - a method, brings accidental User agent out of the list
$tools.query - adding of requests(top)
This tool allows to add requests to already existing directly during work of task, creating them on a basis already parsed results. It can be used as analog of the Parse to level function in those parsers where it isn't realized.
- [% tools.query.add (query, maxLevel) %] - adds single request
- [% tools.query.addAll (array, item, maxLevel) %] - adds an array of requests
Result:
$tools.parseJSON - parsing JSON structures(top)
This tool allows you to deserialize the data in JSON format to the variables, available in the template engine.
- [% tools.parseJSON(data) %] - launches the analysis of structure
$tools.js.* - JavaScript support in tools(top)
This tool allows you to add custom JS functions and use them directly in the template engine. Since version 1.2.527, the use of Node.js modules is supported.
Functions are added in the JavaScript Editor:
$tools.base64.* - ability to encode and decode base64(top)
This tool allows you to encode the text in base64 and decode back. Example:
$tools.data - built-in parameter directory(top)
This tool contains a lot of pre-installed information - languages, regions, domains for Google and Yandex search engines. Complete list of elements:
["YandexWordStatRegions","TopDomains","CountryCodes","YahooLocalDomains","GoogleDomains","BingTranslatorLangs","Top1000Words","GoogleLangs","GoogleInterfaceLangs","EnglishMonths","GoogleTrendsCountries"]
each of which is an array or hash data, you can see the contents like this:
Code:[% tools.data.GoogleDomains.json() %]
$tools.CSVline - easy creation of CSV files(top)
This tool automatically converts the values to the CSV format and adds a line break, now in the result format it is enough to list all the variables, the output will be a valid CSV file, ready for import into Google Docs/Excel/etc.
Usage example:
Code:[% tools.CSVline(query, p1.serp.0.link, p2.title) %]
$tools.aparser.version() - Get information about the A-Parser version(top)
This tool allows you to get information about the version of A-Parser and display it in the result.
Example of use:
Code:[% tools.aparser.version() %]
$tools.task() - returns task id and number of threads(top)
This tool allows you to get information about the task id and show the number of threads.
Example of use:
Code:[% tools.task.id %] [% tools.task.threadsCount %]