Finding bad IPs from bot attacks

[BitRipped]

Hi all,

Not sure if this has been posted before, I couldn't find it when I searched. I'm assuming near enough all faucetbox users without a login are getting hammered by bots at the moment. My CAPTCHA fail rate has been dreadful. These bots don't show in analytics either and kill income on ads and CAPTCHAS.

This is a mini tutorial that should help you find and drop these IPs. I have seen a few good posts about preventing them but they are relentless which makes them easier to find harnessing the power of grep 

You will need cli access eg. SSH as long your not hosting on Windows you should be good with these commands not sure how restricted this is on shared hosting but will work on VPS or dedicated.

Find your access log in this case we will assume /etc/httpd/logs/access_log but may well differ for you depending on what you are hosting on

Code:

cd /etc/httpd/logs

grep -o '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}' access_log > ips.txt

sort ips.txt | uniq > unique.txt

 

download unique.txt or open it with nano or vi.
You should be able to spot the genuine users out easy enough. Can't really explain you can just tell

Next step is use iptables or htaccess to deny access. I can't guarantee this method won't block the odd genuine user. I tend to block at 0/24 0/8 that does the trick nicely

Code:

iptables -A INPUT x.x.x.0/24 -j DROP 

Hopefully it helps a few people out. I will make a link for my iptables blacklist at some point if anyone is interested

[hoop]

 Is this for Ubuntu OS?

[BitRipped]

It will work but your log file will be in a different location. You just need to cd to the right directory then you can go from grep command. From memory give this a try

Code:

cd /var/log/apache2
grep -o '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}' access.log > ips.txt

[Werko]

download unique.txt or open it with nano or vi.
You should be able to spot the genuine users out easy enough. Can't really explain you can just tell

I don't know, what do you mean with that.

Here my unique-file
http://www.filedropper.com/unique_1

What can I do now with the unique file?

[BitRipped]

Your log doesn't look too bad really to be fair
My log looked like a mass of just the same IP ranges with the odd completely unique looking IP every now and then so was easy to spot

I was noticing on Solvemedia my fail rate was stupidly high, I mean a good 2k impressions per hour. At a peak of 112k in a day

I backed up my access log and made a fresh kept a look at ad impressions on a-ads etc google analytics vs number of claims on faucetbox. Fail2Ban is quite good at catching out the spammy ips abusing POST requests eg 0.0.0.1     0.0.0.2 or even 0.0.1.1    0.0.1.2 it gives you something to compare to. Basically the claim button should be disabled if ads are not displayed. This sadly isn't the case as satoshis go down and ad impressions only go up corresponding to analytics. Something is getting bypassed somehow because solvemedia impressions are ultra high in comparison. A fresh log always helps I think as you can factor in time a lot better

It's a pain but

Code:

grep 'POST' access.log > whatever.txt
sort whatever.txt > sortedwhatever.txt

[Werko]

I have several anti bot measures. It seems to work. 

[BitRipped]

I have several anti bot measures. It seems to work. 

How is your overall CAPTCHA fail rate?

[Werko]

8% recaptcha

I block all tor user, the biggest hosts like DigitalOcean or Leaseweb and all user where I can't detect the country.

[BitRipped]

Nice, I was at 99% on SolveMedia so I wanted to up my payout. Then along came the bots with zero income from them. They seem to get through whatever given the time. It's either drastic action or disable it sometimes