Query builder
Query builder - allows you to split the source query into parts and/or transform queries according to certain rules
Capabilities
- Splitting a query into parts using a regular expression or an arbitrary separator
- Replacing a substring in the query or replacing it with a regular expression
- Extracting a domain/main domain from a link
- Adding an exclamation mark before each word in the query (necessary for the
SE::Yandex::WordStat scraper)
- Converting the query to uppercase/lowercase
The resulting new variables can be used in the Query Format when forming a query to the scraper and in the Result Format when forming a result
Basic use cases
- Extracting a domain from a link for subsequent domain checking in
Rank::Alexa and similar scrapers
- Using only a part of the original string in the query, for example, we have a query file:
keyword1;100000
keyword2;200000
We can split the keyword and the existing keyword frequency data, check the keyword for competitiveness and write the result to a file in the format keyword;frequency;competitiveness
- Automatic insertion of an exclamation mark before each word in the query for
SE::Yandex::WordStat
Using proxies to bypass the proxy checker
If you create a variable $proxy, then this proxy will be used in the current query to bypass the Proxy Checker. The proxy must be passed in the format http://ip:port or socks://ip:port
Example of using
Checking the Alexa Rank of a domain and saving additional information from the query Suppose we have a file with queries in the format of a link to a page and a description separated by a space:
http://gofuckbiz.com/showthread.php?t=30454&page=11 Форум GoFuckBiz.com
https://searchengines.guru/ru/forum/720568 Форум SearchEngines.guru
We need to check the Alexa Rank of the domain using the Rank::Alexa scraper and save the original link, domain, Alexa Rank, and the original description in the result. The task will look like this:
As a result, we will get:
https://searchengines.guru/ru/forum/720568 Alexa Rank of searchengines.guru: 109538, Description: Форум SearchEngines.gur
http://gofuckbiz.com/showthread.php?t=30454&page=11 Alexa Rank of gofuckbiz.com: 111396, Description: Форум GoFuckBiz.com
Download example
How to import an example into A-Parser
eJyFVGtP2zAU/SuRhSaYqrSFsk2RECqwTpuAMh6fKJO85Cb16tjGjwKr+t937bhJ
YZMmIVPfe+65r+OsiKVmYa40GLCGZPcrosJvkpECSuq4JT2iqDagvfueXFOxyLIx
h2eKnhazIvZFAUbJJWjNCkAnK/D+6EC/lFLX1DMtKXcetRPMaSFrygRZPzz0CKZF
JjNpoAhRw7RBcSYWSciY+OyJLJPO2VBkwaLR20vOwOSaKctkNEcgmmczQdpMN3QJ
txIzlYxDZ57g7ZLWocqCWvDetGlgdy+1z74NWhTM81PelOtH07VwJ9hj6FJIxPrs
DMxEyxpNFgJBKGnT6n0cB0EKF2K/NzEkKyk30CMGS51QLKR462EWNLVST0O/aF8R
Kcacn8MSeAcL/CeO8QL3OC4x6GsM/Ddk+hfHum1vOxUu+0ljDS1LuJ1ML7qoQp7L
ajMMzmpm8W5OpRN+ywM0LgBUO7NLD6ulhjZNZI7ZUaoKhFdWt7Kx6kyv2giCNtLp
HDZSREhUqoYKngOJ/5+RH7vp++O9BM+9nY35NkCF4xzDpF+VF2MQvsnJw7q3RR89
kR33rGluzxqFN9Gk1Xtb+lalaMylKFk1jU9ooyonbvGRTsWprBUH2xbkDFx3mh2b
qA1/6ab2Nvg0pHj1vK2U3Hy7aeanNMNKD32BNa53O2ukzCnnd9fn2x7S6Rwvc2tV
1u9XsnT54if7neay7pu5fLJzDbRI1Vwd26ODwehw9E7RCo6Gw2TmBqP9kT8PwJ+j
QTgPgiVPvsgJcp00XDPhMxhMYYDqfA6iYgJMWjnt+viHj9XV/Y/7g8MPn/5HfBMY
Pm8xEN+hhUqiVnD+flfx+9d+JVevvoLZao0y/2WuGpQfv8egDfdoUMMkG67/AIrX
3ho=