It never works right for me.

if you look at https://check.torproject.org/cgi-bin/TorBulkExitList.py can help you,

Also make sure your scripts can only be accessed from other scripts like it sounds like you just allowed anyone to execute it, you need to use .htaccess file to make sure only scripts and your site can execute those files individually

If I understand your site correctly, I can request 0.0025btc instantly without having to wait the 5 minutes. If I wait, I can get 0.005btc instead.
I'd start with a little math:
Captcha solving costs something between $1-$2 per 1000 captches.
1000 successful requests mean 2.5btc when you don't bother to wait.
Assuming ~$5/btc, you make $12.50 while paying $2, resulting in a $10.50 profit.

You can always try to change the captcha, although I think that won't be much of a success since solvers offer a professional service dealing with them.
Probably a good idea is to look around for a good real-time blacklist of proxies and block them.
Or, instead of blocking, accept the request but don't send it out. That costs whoever does that money.

you should have a database on your side that keeps track of the times for every ip that comes on the site, this looks like some kid found out a part of your script that doesn't do what you thought it did, I would suggest you do more testing to make sure, all your links and scripts are being secured from the outside so they can't be runned by any ip but only by other scripts, add tokens to make sure they person is who they are. things like that

I bet the guy was using tor so ip blacklist will not work on that, cause the user can change the ip and there is so many nodes, so look into blocking the whole tor network, I know there a way to do that.

it was security issue in your code, and these things just don't go away you better start auditing it and making sure that is was the issue you were thinking

your bitcoind wasn't secure enough, your site should be the only incoming and out going packets from your bitcoind, everything else should be blocked, and logged by your firewall. Also I would make the bitcoind it's own user and so it can't be run from a normal user or even sudo user. Also connections to your bitcoind shouldn't be on an obvious port either. You should change all passwords right now!