JS scrapers: examples
Here are just a few examples of using JavaScript scrapers, demonstrating the core functionality. Many more examples are published in the Scraper Catalog
🔗 JS::TitleParser
Parsing HTML title tag on an arbitrary page
🔗 JS::TextFromSE
Combining several built-in scrapers in one
🔗 Downloading files
Example of saving files of any size directly to disk
🔗 Working with CAPTCHA
Example JS scraper for working with captchas
🔗 Working with ReCaptcha2
Example JS scraper for working with recaptchas
🔗 Working with ready-made scrapers
Example of getting results from a ready-made scraper inside a JS scraper
🔗 Working with SQLite DB
Example of creating a table, inserting and selecting data
🔗 Working with MySQL DB
Example of working with the mysql2 module, this scraper collects links from Google search results and writes them to the DB
🔗 Pagination
An example of parsing a site by request with navigation through pages
🔗 Filtering by multiple attributes
Example of filtering pages by a large number of attributes, uses fs to read a file of attributes
🔗 Filtering images by their content
Example of using Google Vision to determine the content of images
🔗 Posting to Wordpress blogs
Posting messages to Wordpress blogs using xmlrpc technology
🔗 Sending email messages
Example of working with nodemailer (sending emails from a JS scraper). Yandex SMTP is used.
🔗 Creating page screenshots
Demonstration of working with puppeteer(Chrome) by creating website screenshots
🔗 Telegram notifications
Receiving notifications in Telegram about the expiration of domain registrations