1.2.455 - puppeteer module support, all methods in Net::HTTP, Cloudflare protection bypass update

Since version 1.2.455, we are launching official support for the Node.JS module of the puppeteer module — automation tools for working with the Chrome (Chromium) browser, due to which full DOM rendering with JS is possible. As an example, we publish the scraper, which takes screenshots of sites in PNG format, there is a choice of the size of the screenshot also. Chromium can be run on both Windows and Linux servers without a GUI.

• Scraper example and more detailed description (russian)
• Documentation for puppeteer

Improvements

• SE::Bing::Images completely rewritten, added additional data scraping, as well as the choice of region and interface language added in the settings
• In Net::HTTP added all the methods to the settings, and also implemented the ability to override via the Method header
• Added the ability to format all digits in the interface
• JS scrapers: in this.parser.request for override added option needData, which determines whether to transmit or not in the response data/pages[], can be used for optimization
• JS scrapers: added support for url and ws modules

Corrections due to changes in the issuance

• Net::Whois - the com.br domain zone was not always correctly processed
• SE::Yandex, SE::Ask, SE::DuckDuckGo, SE::DuckDuckGo::Images, Rank::MajesticSEO

Corrections

• Fixed the work of the function Bypass Cloudflare Net::HTTP
• Fixed a bug due to which the Preppend text was not written under certain settings
• Fixed work of some socks4 proxy
• Fixed error in $tools.base64.encode() when working with Cyrillic characters
• In Shop::Amazon fixed bug where domain was ignored
• JS parsers: fixed a rare error in the work of sockets in Node.js