I have a very powerful Windows server which is running one Google parser task. It has 500 unlimited proxy threads, that I bought from a parser. But the result is not very good, it is doing 200 queries per minute. Can anyone give me some guidance on how to improve the query per minute?
Are you scraping more than the first page? That will slow things down a lot. What are the settings for the 500 threads preset?
I am scraping only the first page. I am not using any captcha solving just the 500 threads that match with my aparser 500 threads proxy purchase. There is only 1 task running. The aparser support told me to increase the "redirect browser max pages" from 10 to 20. I am using the default SE Google parser preset Below is my task preset (please ignore the 200threads config preset name, I upgraded to 500 threads and did not change name) https://i.imgur.com/iv92339.jpeg
What are your settings for the number of threads preset? Sometimes you have to play around with these settings to get better results with your proxies. What works for one proxy provider might not work the best for another. I've also found sometimes just because you can use up to 500 threads doesn't mean better performance. Try 250 threads and just see what happens. Maybe 500 threads is burning through the proxies faster before the next rotation cycle.
You are definitely on the right side. I found a mistake I have in setting fo the proxy. I previously have MAX threads per proxy is 1 It should be 0 like yours. Because when I set it to be 1, I think aparser won't be able to use Google sessions, so every request is a new request and can encounter Recaptcha. I did not even turn on Xevil, right now, the per minute query is around 700 (before it was never close to 200)
