Skip to main content

Auxiliary methods(utils, tools, sleep)

this.utils.*​

.updateResultsData(results, data)​

await this.utils.updateResultsData(results, data) - a method for automatic filling of $pages.$i.data and $data, it is necessary to call it to add content to the resulting page

.urlFromHTML(url, base)​

await this.utils.urlFromHTML(url, base) - processes a link obtained from HTML code - decodes entities (& etc.), optionally you can pass base - base URL (for example, the URL of the original page), thus a full link can be obtained

.url.extractDomain(url, removeDefaultSubdomain)​

await this.utils.url.extractDomain(url, removeDefaultSubdomain) - the method takes a link as the first parameter and returns the domain from that link. The second optional parameter determines whether to cut the www subdomain from the domain. By default 0 - that is, do not cut.

.url.extractTopDomain(url)​

await this.utils.url.extractTopDomain(url) - the method takes a link as the first parameter and returns the domain from that link, without subdomains.

.url.extractTopDomainByZone(url)​

await this.utils.url.extractTopDomainByZone(url) - the method takes a link as the first parameter and returns the domain from that link, without subdomains as well. Works with all regional zones

.url.extractMaxPath(url)​

await this.utils.url.extractMaxPath(url) - the method takes a string and selects a URL from it

.url.extractWOParams(url)​

await this.utils.url.extractWOParams(url)- the method takes a link and returns the same link trimmed to the parameters string. That is, it will return the URL up to ?

.removeHtml(string)​

await this.utils.removeHtml(string) - the method takes a string and returns it cleaned from HTML tags

.removeNoDigit(string)​

await this.utils.removeNoDigit(string) - the method takes a string, removes everything except digits from it, and returns the result

.removeComma(string)​

await this.utils.removeComma(string) - the method takes a string, removes such characters as .,\r\n and returns the result

.getAllBlocks(html, regexp, opts?)​

await this.utils.getAllBlocks(html, regexp, opts?) - getting all blocks on the page with corresponding closing tags, the method takes an HTML string and a regular expression that indicates the beginning of a block (any blocks that have paired closing tags, for example <div>...</div>), as a result, an array of all found blocks is returned

Options opts:

  • searchStartIndex - indicates the index in the string from which to start the search, by default 0 
const blocks = this.utils.getAllBlocks(html, /<div [^>]*?class="results"/)

.getAllBlocksByAttr(html, tag, attrName, attrRegExp, opts?)​

await this.utils.getAllBlocksByAttr(html, tag, attrName, attrRegExp, opts?) - a method similar to .getAllBlocks, instead of a regular expression for searching the beginning of a block, the name of the tag, the name of the attribute for the search (for example id, class) and the regular expression that will be applied to the value of the specified attribute are indicated

const blocks = this.utils.getAllBlocksByAttr(html, 'div', 'class', /results/)

await tools.*​

The global object tools, allows access to built-in A-Parser functions

Analog of templating tools $tools.*

note

tools.query is not available, you must use this.query

await this.sleep(sec)​

await this.sleep(sec)

Sets a delay in the thread for a number of seconds (sec), can be fractional.

await this.mutex.*​

Mutex for synchronization between threads, allows to lock a section of code for one thread

.lock()​

Waiting for the lock, execution will continue with the first thread that captured the lock, other threads will wait for the lock to be released

.unlock()​

Releasing the lock, the next thread will continue execution if it was waiting for the lock - .lock()

results.<array>.addElement()​

The method results.<array>.addElement() allows more convenient filling of arrays in results. When using it, there is no need to remember the sequence of variables in the array and list them manually.

results.serp.addElement({
    link: 'https://google.com',
    anchor: 'Google',
    snippet: 'Loreps ipsum...',
});

this.isContextAlive()​

This method is necessary for long-lived threads that process requests in a loop, allowing them to correctly finish work when stopping or deleting a task

while (this.isContextAlive()) {
    await this.request(...)
}
Last updated