Tools in the Templаte toolkit

Description(top)

In template processing system Template Toolkit the global variable $tools is available, which stores in itself a tool kit available in any template.
$tools.error - contains a description of errors that may occur during executing any of tools.

$tools.memory.* - key / value storage in memory (top)

Simple storage in memory, common to all tasks, api requests, etc., is reset when the parser is restarted

• tools.memory.set(key, value) - sets the value of the key
  
• tools.memory.get(key) - returns the value corresponding to the key
  
• tools.memory.delete(key) - deletes the record from memory

$tools.sqlite.* - work with SQLite databases(top)

• $tools.sqlite.get - method that allows you to retrieve information from a database using SELECT
  
  Code:

  [% res = tools.sqlite.get('results/test.sqlite', 'SELECT COUNT(*) AS count FROM test') %]

• $tools.sqlite.run - method that allows you to perform operations with DB (INSERT, DROP, etc)
  
  Code:

  [% res = tools.sqlite.run('results/test.sqlite', 'INSERT INTO test VALUES(?)', 'test') %]

• $tools.sqlite.all - method that allows you to print all the data from a table

$tools.ua- changeover of User agent(top)

This tool is intended for substitution of User agent in parsers which use it (for example Net::HTTP).

• $tools.ua.list - contains the User agent list
• $tools.ua.random() - a method, brings accidental User agent out of the list

List of all User agents is stored in file files/tools/user-agents.txtWhen using this tool for User agent parameter in parsers, it is necessary to specify it obviously: [% tools.ua.random() %]

$tools.query - adding of requests(top)

This tool allows to add requests to already existing directly during work of task, creating them on a basis already parsed results. It can be used as analog of the Parse to level function in those parsers where it isn't realized.

• [% tools.query.add (query, maxLevel) %] - adds single request
• [% tools.query.addAll (array, item, maxLevel) %] - adds an array of requests

The maxLevel parameter specifies to what level adding requests, and is optional: if it is lowered, actually the parser will add new requests they is so far. Also it is recommended turn on option Unique requests to avoid cycling and excessive work of parser.

Result:

$tools.parseJSON - parsing JSON structures(top)

This tool allows you to deserialize the data in JSON format to the variables, available in the template engine.

• [% tools.parseJSON(data) %] - launches the analysis of structure

After deserialize to keys from JSON array can be accessed as regular variables and arrays.

$tools.js.* - JavaScript support in tools(top)

This tool allows you to add custom JS functions and use them directly in the template engine.
Functions are added in the JavaScript Editor:

$tools.base64.* - ability to encode and decode base64(top)

This tool allows you to encode the text in base64 and decode back. Example:

$tools.data - built-in parameter directory(top)

This tool contains a lot of pre-installed information - languages, regions, domains for Google and Yandex search engines. Complete list of elements:
["YandexWordStatRegions","TopDomains","CountryCodes","YahooLocalDomains","GoogleDomains","BingTranslatorLangs","Top1000Words","GoogleLangs","GoogleInterfaceLangs","EnglishMonths","GoogleTrendsCountries"]
each of which is an array or hash data, you can see the contents like this: