Why are updates necessary, and why are they paid?
A-Parser is constantly evolving. With the release of new versions, improvements and fixes are introduced. In this article, we will examine what updates are, how they differ from the license, what role they play, and why you need to pay for them.
License ≠ Updates
By purchasing A-Parser, you get a perpetual license to use it and 3-6 months of free updates depending on the purchased license. After the period of free updates expires, you can update to the latest available stable version and continue using the scraper to its full extent, — as much as the version available at the time the subscription ended allows.
To renew your subscription, you can purchase one of three update packages: for 3 months, one year, and lifetime for $49, $149 and $399 respectively.
You do not need to pay for updates constantly. The period during which there was no update subscription does not need to be paid for.
Why Are Updates Paid?
🐞 Bug Fixes
Websites and various resources evolve quite quickly. Any, even the most minor, changes on the target site can affect scraping. This happens because scrapers are initially tailored to a specific structure, and changes in the markup, protection, or other various internal mechanisms lead to incorrect data in the results, their complete absence, and other errors. Scraping itself negatively affects the servers allocated to websites: requests and, consequently, load increase. Services losing profit are forced to find a way out of the current situation, which leads to the emergence of new types of protection and the development of old ones.
With every such change, modifications must be introduced. Each one involves problem analysis, finding a solution, and its implementation.
🧰 Every day, each built-in scraper undergoes a system of internal tests. If the test requests are successful, the resulting values are checked. A failed test signals errors present in the scraper. Thanks to tests, we quickly identify breakdowns and immediately begin work on fixing them.
Among the most complex, in-demand, and therefore prioritized for us are the – Yandex and Google search engine scrapers. Each consists of many parts that solve a specific task. These include request preparation, header formation, obtaining the page's source code, various result formatings, working with captcha, etc. All of this needs to be maintained in a functional state. The scraper provides for variables containing all the necessary data from the page: search results, advertisements, related keywords, and other values. They are extracted using regular expressions that assume a specific document structure on the page (order of elements, their types, classes, and various other characteristics). If this structure changes critically, the regex that worked for the past version stops extracting the desired fragment, and the scraper is sent for refinement.
✨ Improvements
In addition to maintaining the functionality of built-in scrapers, with each release, new features are added and various improvements, are introduced, affecting both performance, and the amount of data obtained. The build also includes new scrapers, and new methods are implemented in the JavaScript API.
You can view all changes here.
Problems associated with the lack of updates
The absence of timely updates causes incorrect operation of built-in scrapers. The reasons can vary. For example, the page markup might have changed. A scraper that has not received an update tries to collect data using old regular expressions not adapted to the new format. Consequently, failed requests, appear, various errors emerge, and results are missing..
Example using the Google Scraper
A user contacted support with the following issue:
I am gathering Google search results using your proxies. I set 300 attempts for the request. All requests are failing. It was all working yesterday.
At first glance, it seems that the problem is with the proxies, but tests with identical settings and requests on the latest version run successfully. Therefore, the problem lies elsewhere. During the dialogue, it becomes clear that the user is running an outdated version of A-Parser. This is the real reason for the Google scraper's incorrect operation.
Example using the Yandex Scraper
Yandex changed the markup of captcha pages, causing it to stop being solved. A corresponding topic.
The next morning, a fix was released. The task was closed and moved to the Next release. section. All topics regarding fixes and improvements that will be included in the next stable version are located there.
Consequently, in A-Parser instances that did not receive the fresh update, the Yandex captcha was no longer solved.
Conclusion
By purchasing A-Parser, you receive a perpetual usage license for the program and a package of free updates for a specific period.. If necessary, after the subscription period expires, you can renew it by purchasing one of the offered update packages.
Websites are unstable, – and scrapers require constant adjustments and improvements. Maintaining their functional state – is our job. This is a priority task on which we put great effort to release working fixes as quickly as possible. The cost of updates is determined by the labor behind them. Every release – is not just a list of fixes and improvements, – it is months of focused work by the A-Parser team.