404 Bot attack on my site (DDoS sorts)
Over the past few days, I've noticed that my Wordpress site is pretty slow, so I decided to research. After checking my database, I saw that the table that was responsible for tracking 404 errors was over 1 GB in size. At this point, it was obvious that bots were attacking me.
After checking my access log, I could see that there was a pattern, the bot seemed to land on a legitimate page that listed my categories and then navigated to the category page, and at this point they ask for seemingly random page numbers, many of which are nonexistent pages causing the problem.
Example:
/watch-online/ - Landing Page
/category/evolution/page/7 - 404
/category/evolution/page/1
/category/evolution/page/3
/category/evolution/page/5 - 404
/category/evolution/page/8 - 404
/category/evolution/page/4 - 404
/category/evolution/page/2
/category/evolution/page/6 - 404
/category/evolution/page/9 - 404
/category/evolution/page/10 - 404
This is the actual order of the requests and they all happen within a second, at this point the IP is blocked as too many 404s were picked, but that doesn't seem to be affected, due to the large number of bots doing the same thing.
Also, the category changes with each bot, so they all attack random categories and generate 404 pages.
There are currently 2037 unique ip's that have chosen similar 404s in the last 24 hours.
I also use Cloudflare and have manually blocked a lot of ip from my reach, but this attack is relentless and it seems like they keep generating new ip. Here is a list of some of the offending ip:
77.101.138.202
81.149.196.188
109.255.127.90
75.19.16.214
47.187.231.144
70.190.53.222
62.251.17.234
184.155.42.206
74.138.227.150
98.184.129.57
151.224.41.144
94.29.229.186
64.231.243.218
109.160.110.135
222.127.118.145
92.22.14.143
92.14.176.174
50.48.216.145
58.179.196.182
Apart from auto blocking ip for too many 404 errors, I can't think of another real solution, and this in itself is rather inefficient due to the large amount of ip.
Any suggestions on how to deal with this would be highly appreciated as there seems to be no end to this attack and the performance of my websites is really amazing.
Some user agents include:
Mozilla/5.0 (Windows NT 6.3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.86 Safari/537.36 Mozilla/5.0 (Windows NT 6.2; rv:26.0) Gecko/20100101 Firefox/26.0 Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 7.0; WOW64; Trident/6.0) Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:22.0) Gecko/20100101 Firefox/22.0 Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36
source to share
Ok, so after a lot of searching, experimenting and headbutting, I finally softened the attack.
The solution was to install the apache module 'mod_evasive' to see:
So, for any other poor soul that slammed shut as soon as I looked at it and fine-tuned your thresholds. This is a simple, cheap and very effective way to radically understate any attack like the one I have experienced.
My server is still bombarded by bots, but this really limits their damage.
source to share