Clever Geek Handbook

How do I make the Mixnode scanner scan slower?

We exceeded 20 million pages per hour and I really appreciate the speed; however I am afraid that I might put too much pressure on the targeted sites, is there a way to decrease the speed at which websites are crawled?

+3


source to share


1 answer


Not sure why you want to decrease the speed, as the documentation clearly states that:



There is a minimum delay of 10 seconds between requests sent to the same site . If a website's robots.txt directives require longer latency, Mixnode will follow the length of the delay specified by the robots.txt directives.

+3


source






More articles:


All Articles