JS Scrapers: Examples
Here are just a few examples of using JavaScript scrapers, demonstrating the main functionality. Many more examples are published in the Scraper Catalog
🔗 JS::TitleScraper
Parsing the HTML title tag on an arbitrary page
🔗 JS::TextFromSE
Combining multiple built-in scrapers into one
🔗 Downloading files
Example of saving files of any size directly to disk
🔗 Working with CAPTCHA
Example of a JS scraper for working with CAPTCHAs
🔗 Working with ReCaptcha2
Example of a JS scraper for working with ReCAPTCHAs
🔗 Working with ready-made scrapers
Example of getting results from a ready-made scraper inside a JS scraper
🔗 Working with SQLite DB
Example of table creation, insertion, and data selection
🔗 Working with MySQL DB
Example of working with the mysql2 module; this scraper collects links from Google search results and writes them to the DB
🔗 Paging through results
An example of scraping a site by query, with traversal through pages
🔗 Filtering by multiple criteria
Example of filtering pages by a large number of criteria, using fs to read the criteria file
🔗 Filtering images by their content
Example of using Google Vision to determine image content
🔗 Posting to Wordpress blogs
Posting messages to Wordpress blogs using xmlrpc technology
🔗 Sending email messages
Example of working with nodemailer (sending emails from a JS scraper). Yandex SMTP is used.
🔗 Creating page screenshots
Demonstration of working with puppeteer (Chrome) using the example of creating website screenshots
🔗 Telegram notifications
Receiving Telegram notifications about domain registration expiration