Bitcoin Forum
November 19, 2024, 04:32:30 AM *
News: Check out the artwork 1Dq created to commemorate this forum's 15th anniversary
 
   Home   Help Search Login Register More  
Pages: [1] 2 3 4 »  All
  Print  
Author Topic: BTCGuild and it's relation to DDoS attackers  (Read 10838 times)
slush (OP)
Legendary
*
Offline Offline

Activity: 1386
Merit: 1097



View Profile WWW
October 20, 2011, 02:00:25 PM
Last edit: October 20, 2011, 02:19:51 PM by slush
 #1

Hi,

Quote
I have very strong evidence that btcguild.com is somehow related to those DDoS attacks.

I think that I owe an explanation about my yesterdays "accusation" that btcguild.com is behind last DDoS attacks. Let me clarify that I'm not saying that btcguild.com is an *attacker*, but that there's some relation between this pool and attackers, which is big difference. I'll try to use only hard facts in following text.

Firstly, let me talk about how those attacks worked. Basically it's network of thousands zombie computers, listening it's operator and doing some simple commands like "flood some specific IP address". There's no or very small chance to shut this botnet down. All people asking me to use whitelisting or filtering this traffic on server don't really understand, how massive those attacks are. If you have 100Mbit pipe to server, but attacker's uplink is 1Gbit, there's no chance to filter out this traffic, because your pipe is simply too small. You may say that buying a bigger pipe is a solution, but actually it isn't. Actually last attacks were far over 1Gbit/s and paying dedicated 10Gbit line is simply out of my financial possibilities.

During many of those attacks I learned a lot how they're working. I tried everything; setting up more balancers, using DDoS mitigation proxies etc. But *everytime* when I changed DNS records to new IP or even when I created yet another DNS record (do you remember my post about new DNS api3.bitcoin.cz last week?), attack followed those changes in DNS or posts on forum almost immediately. Thanks to that I know attacker is here between us, following what's going and targeting attack to new places instantly (in matter of minutes).

I don't remember when exactly I had first DDoS attack, but as I was the biggest pool, attacking to me was pretty logical step. Later, when deepbit had higher hashrate than me, attacking two biggest pools was still pretty good way how to harm Bitcoin network. However last two attacks where more pool was affected, attackers picked first (deepbit) and *third* (my) pool.

I know btcguild once had massive many-days DDoS attack, but it was related to banning some botnet out of his pool. I'm not actively following btcguild community, but as other people told me, those attack finished to "peace agreement" between some botnets and btcguild, because btcguild probably understood that it has no sense to fight with them. Personally I understand that; if I would have an options to reject botnet and be ddossed to death or silently accept them and don't receive attacks, I'll probably pick first one, too.

Yesterday I did one thing which I'm proud of, but which was a logical step in closing the circle; what happen when I change my DNS to btcguild servers? At this time, me and deepbit were completely off, but btcguild had even higher hashrate (probably thanks to failover configurations in miners). So I modify DNS records and point everything to btcguild.com. I had 5minutes DNS timeouts, so there was an easy way how to revert this traffic back from btcguild.

As I expected, nothing happen. I leaved DNS to btcguild over a half of hour, which was many times longer than botnet need to switch to new IP address before. And btcguild was still untouched.

You may say that btcguild is using some DDoS protection, so redirecting traffic didn't affect them so much as me. But those IPs are owned by Hetzner Online AG, the same housing company as deepbit is using and which was convicted that they cannot handle DDoS attacks. This is also reason why deepbit isn't using them for facing DDoS attacks and he uses 3rd party company to handle it.

There's only one logical conclusion - an attacker didn't want to shut down btcguild.com for some reason. If I don't want to say that btcguild itself is an attacker, then I can at least say that attacker is probably using btcguild and he don't want to shoot his own leg.

Note: Currently is btcguild under an attack, too. I don't want to speculate more, because I don't have any more facts for current situation. However this attack started after I moved DNS back to my servers and post about attack on forum.

---
There's yet another strange stuff on btcguild. As cosurgi calculated and other guys confirmed by re-calculation. It practically means that 4% of accepted shares of btcguild cannot be used to "win" a bitcoin block. Although btcguild rejected that those 4% are anything more than bad luck, with thousands of mined blocks there is really tiny chance that this variance is "natural". From cosurgi's calculations you can see that other pools fits mathematical expectations. Basically there are three general explanations why btcguild performs so badly:

a) btcguild operator is cheating and those 4% are hidden fees
b) there is some major bug in pool software he's using. This cannot be just a downtime, because those 4% shares were accepted.
c) it's an evidence of an "withdrawal attack" (attack where miner is submitting only shares which don't fit full difficulty)

Until now, all points were hard facts. Let's me say my own opinion:
* Personally I'm inclined to point c), becuse I don't believe that any pool operator is intentionally stealing so much; as you see, it is pretty easily detectable after some time. However point c) fits pretty good to an image of botnet trying to earn money for himself, but otherwise hurting bitcoin network (including withdrawal attack hurting other pool users). 4% from 2THashes are something like 80GHashes, which is doable by medium-sized botnet.

* However if I'm right at least in few points, btcguild is only one entity who can try to track who is an attacker, because *if* one of his mining botnet is one who's attacking to other pools (to lower difficulty and earn more for him) and at least partially protect btcguild itself (because he don't want to hurt his own pool), then btcguild have some IPs and also payout wallet of attacker, which can be small pieces in a puzzle.

I agree that last points are wild speculation, but the first part of this post are hard facts. Please think about it before you'll write that I'm kicking around me. Personally I wish all best to operational pools, because they're very important part of Bitcoin ecosystem; and as I wrote yesterday, I believe in Bitcoin success. However I wrote everything important now and I don't want to join some following flamewar; I didn't write anything personal against other pool operators. Now I'm locking myself to room just with pen and paper and I'm thinking how to make pools more DDoS resilent.

Best,
slush

eleuthria
Legendary
*
Offline Offline

Activity: 1750
Merit: 1007



View Profile
October 20, 2011, 02:25:40 PM
Last edit: October 20, 2011, 02:40:23 PM by eleuthria
 #2

I can say you're half right on the part about striking a deal with the attackers back when BTC Guild first got hit.  The first attack in June, I was contacted in IRC after 2 days of downtime and basically given two options:

 A) Let them back on the pool [I had a ban that completely filtered one particular large botnet due to how they connected]
 B) Lose my servers completely when they ramp up the attack larger and the ISPs drop my account

I took the deal, at the time I was out over $1,000 scaling the pool up to handle our regular load and adding on servers, and BTC value was just climbing.

However, in July I had spread out our servers to multiple locations, including Awknet which is supposed to specialize in DDoS protection.  On July 3rd, I put my ban back in on the botnet, based on how they connected to the pool.  We were then offline from July 3rd til the 7th.  I refused to strike a deal again, and waited out the attacks.  Eventually they stopped, but I did end up having multiple ISPs drop me from their services.  Luckily I still had Awknet, and I also had a new dedicated server being colocated rather than leased with Justin Shattuck (shat in IRC).  He knew I was going to get attacked, but said my server would at least remain available after the attacks were over.

Since then we've moved to many other servers/scaling attempts.  I've kept an active watch on banning users I was positive were not legit, but everytime I kicked one out a new one would take its place within a few hours.


That is the limit of my involvement from June-July, a temporary peace treaty while I tried to find servers that would let me stay online after the attacks ended.  This latest round, my only guess is the people behind the attacks were currently using my pool.  Based on what's been happening to my pool since, I don't know if this is accurate or not.

EDIT: Splitting post into two parts addressing the two separate issues.

RIP BTC Guild, April 2011 - June 2015
eleuthria
Legendary
*
Offline Offline

Activity: 1750
Merit: 1007



View Profile
October 20, 2011, 02:30:44 PM
 #3

As for the second part regarding 4% luck.  I still am not convinced one way or another.  The pool software we used for the first four months was Pushpool.  Same as almost every other pool.  There was a period where I broke one part of the invalid share detection, but I'm fairly certain the effect on luck would've been minimal (maybe 0.1-0.5% over the lifetime of the pool).  Even using the maximum amount, that's -3.9% on luck.

I have no explanation for it.  When Vladmir brought it up the first time, I was frantic, especially because at the time we were having a significantly bad period.  Every bitcoind's block generations were reported in the pool stats.  I debugged the software with some help ArtForz to make sure there wasn't something else wrong.

It's entirely possible there was a withholding attack, but I still have my doubts on it.  A withholding attack causing the pool to have -4% luck would mean the person doing it is reducing their rewards by 4%.  That is a lot when we're talking about 80-100 GH/s worth of hashing power.

RIP BTC Guild, April 2011 - June 2015
zerokwel
Sr. Member
****
Offline Offline

Activity: 464
Merit: 250



View Profile
October 20, 2011, 02:41:24 PM
 #4

Just one thing.. So you pointed your dns to btcguild when you where getting attacked.

so anyone attacking via slushpool.com or whatever your domain is would hit him.. Hmmm thats just wrong.. plus your logic with dns is a little off also

I think this is one of them points where if I have not got anything nice to say... say nothing at all...

Decides to keep quiet



Iyeman
Sr. Member
****
Offline Offline

Activity: 1183
Merit: 251



View Profile
October 20, 2011, 02:49:28 PM
 #5

DNS propogation is not instant, it can take hours in some cases for the new ip's to propagate to all the DNS servers in the world, especially if the server is caching it can take up to 24hrs for the clients to get the new IP, so the DNS test you did doesn't really prove much.

▄████████████▄▄▄▄▄▄▄▄
 ████▀▀▀▀▀▀▀▀▀▀▀████████▄▄
█████                 ▀▀███▄
█████▄                   ▀██▄
███▀███▄                  ███
███  ▀████▄▄▄▄▄▄▄▄▄       ███
███     ▀▀▀▀▀███████████▄▄██▀
███                   ▀▀████
███      ▄▄▄▄▄▄▄▄▄▄██████▀▄█▄
███   ▄███████▀▀▀▀▀▀▀▀▀   ▀██
███ ▄██▀▀                  ██
██████                    ██▀
█████                   ▄██▀
▀████▄       ▄▄▄▄▄▄▄▄████▀
 ▀
█████████████▀▀▀▀▀▀▀▀
.
billionair
A TOKEN POWERED RAFFLE PLATFORM
IN THE BSC ECOSYSTEM
▄█████████████████████▄
███████████████████████
████▀███████▀   ▀▀▀▄███
███▌  ▀▀███▌       ▄███
███▀               ████
███▄              █████
████▄            ██████
█████▄▄        ▄███████
████▄       ▄██████████
███████████████████████
▀█████████████████████▀
▄█████████████████████▄
███████████████████████
████████████████▀▀█████
███████████▀▀▀    █████
██████▀▀▀   ▄▀   ██████
███▄     ▄█▀     ██████
██████▄ █▀      ███████
███████▌▐       ███████
████████ ▄██▄  ████████
██████████████▄████████
▀█████████████████████▀
▄█████████████████████▄
█████▀▀▀▀▀▀▀▀▀▀▀▀██████
███▀ ████████████ ▀████
██  ██████▀▀▀██  █  ███
██  ████▀ ███ ▀███  ███
██  ███▀ █████ ▀██  ███
██  ███▄ █████ ▄██  ███
██  █████ ▀▀▀ ████  ███
██  ██████████████  ███
███▄ ████████████ ▄████
▀████▄▄▄▄▄▄▄▄▄▄▄▄█████▀
▄███████▀▀▀▀████████▀▀▀▀█████████▄
█████▀  ▄▄██▀▀▀▀▀▀▀▀██▄▄  ▀███████
█████  ▀▀▀              ▀▀▀  █████
███                            ███
██                              ██
██        ████      ████        ██
█        ██████    ██████        █
█         ████      ████         █
█                                █
█      ▀▄▄              ▄▄▀      █
██▄      ▀▀█▄▄▄▄▄▄▄▄▄▄█▀▀      ▄██
███▄▄      ██████████      ▄▄█████
███████▄▄▄████████████▄▄▄█████████
▀████████████████████████████████▀
.
JOIN AIRDROP
bitlane
Internet detective
Sr. Member
****
Offline Offline

Activity: 462
Merit: 250


I heart thebaron


View Profile
October 20, 2011, 02:54:06 PM
 #6


Yesterday I did one thing which I'm proud of, but which was a logical step in closing the circle; what happen when I change my DNS to btcguild servers? At this time, me and deepbit were completely off, but btcguild had even higher hashrate (probably thanks to failover configurations in miners). So I modify DNS records and point everything to btcguild.com. I had 5minutes DNS timeouts, so there was an easy way how to revert this traffic back from btcguild.

As I expected, nothing happen. I leaved DNS to btcguild over a half of hour, which was many times longer than botnet need to switch to new IP address before. And btcguild was still untouched.

Best,
slush
WOW....
So what you have done is basically re-wrote the 'book' on DDoS attacks and how to 'DEAL' with them.

So now every time a Pool operator is pissed off and under attack, he can 'SPREAD THE WEALTH' among other pools to have them suffer the same fate ?

LAME. Sour Grapes is all that is.....

anatolikostis
Legendary
*
Offline Offline

Activity: 2026
Merit: 1005



View Profile
October 20, 2011, 02:58:12 PM
 #7

I can say you're half right on the part about striking a deal with the attackers back when BTC Guild first got hit.  The first attack in June, I was contacted in IRC after 2 days of downtime and basically given two options:

 A) Let them back on the pool [I had a ban that completely filtered one particular large botnet due to how they connected]
 B) Lose my servers completely when they ramp up the attack larger and the ISPs drop my account

I took the deal, at the time I was out over $1,000 scaling the pool up to handle our regular load and adding on servers, and BTC value was just climbing.
I still remember such a nice first trojan which pointed cpu`s mining to btcguild...
yeah baby, that`s it!
 Roll Eyes
Jezzz
Full Member
***
Offline Offline

Activity: 120
Merit: 100


View Profile
October 20, 2011, 03:00:43 PM
 #8

Out of curiosity, what prompted you to point your traffic to BTCG?
Tmoney
Newbie
*
Offline Offline

Activity: 40
Merit: 0


View Profile
October 20, 2011, 03:00:57 PM
 #9


Yesterday I did one thing which I'm proud of, but which was a logical step in closing the circle; what happen when I change my DNS to btcguild servers? At this time, me and deepbit were completely off, but btcguild had even higher hashrate (probably thanks to failover configurations in miners). So I modify DNS records and point everything to btcguild.com. I had 5minutes DNS timeouts, so there was an easy way how to revert this traffic back from btcguild.

As I expected, nothing happen. I leaved DNS to btcguild over a half of hour, which was many times longer than botnet need to switch to new IP address before. And btcguild was still untouched.

You may say that btcguild is using some DDoS protection, so redirecting traffic didn't affect them so much as me. But those IPs are owned by Hetzner Online AG, the same housing company as deepbit is using and which was convicted that they cannot handle DDoS attacks. This is also reason why deepbit isn't using them for facing DDoS attacks and he uses 3rd party company to handle it.

There's only one logical conclusion - an attacker didn't want to shut down btcguild.com for some reason. If I don't want to say that btcguild itself is an attacker, then I can at least say that attacker is probably using btcguild and he don't want to shoot his own leg.

Note: Currently is btcguild under an attack, too. I don't want to speculate more, because I don't have any more facts for current situation. However this attack started after I moved DNS back to my servers and post about attack on forum.


The only proven attack in this post was you attempting to attack btcguild.com
cosurgi
Sr. Member
****
Offline Offline

Activity: 298
Merit: 250


View Profile
October 20, 2011, 03:01:25 PM
 #10

LAME. Sour Grapes is all that is.....
You see that slush pool has died, so apparently there was no hope to save it. So the last thing a dying man could do was to try to at least find out why&how? this happens. There were some suspicions around that Eleuthria had deals with attackers. I think this step was - of course - not nice, but logical.

And also - btcguild was not harmed, by doing this! It only revealed that attackers like btcguild and will not attack.

Caesium
Hero Member
*****
Offline Offline

Activity: 546
Merit: 500


View Profile
October 20, 2011, 03:01:37 PM
 #11

DNS propogation is not instant, it can take hours in some cases for the new ip's to propagate to all the DNS servers in the world, especially if the server is caching it can take up to 24hrs for the clients to get the new IP, so the DNS test you did doesn't really prove much.

Sorry, this is nonsense. slush said in his post he has a 5 minute timeout on his zone and this is easily verifiable:

$ dig mining.bitcoin.cz

; <<>> DiG 9.7.3 <<>> mining.bitcoin.cz
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 59770
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 5, ADDITIONAL: 4

;; QUESTION SECTION:
;mining.bitcoin.cz.             IN      A

;; ANSWER SECTION:
mining.bitcoin.cz.      300     IN      A       178.79.183.97


See that 300? 300 seconds, 5 minutes.

No DNS server (unless deliberately misconfigured) will hold onto that value for more than 5 minutes.

It is conceivable that if a client is going through a long chain of DNS servers each with their own cache, that you will see old data for slightly more than 5 minutes, but I would guess this is rare. And it certainly wouldn't be 24 hours.

It is also conceivable that the botnet attacking software could have done one lookup when it started then kept the value until told to do otherwise, but then it would require babying by the operator to keep up with his previous DNS changes when trying to evade them. I doubt this is the case.

Everything slush said about DNS was correct. Yes, I am a sysadmin.

Tired of annoying signature ads? Ad block for signatures
eleuthria
Legendary
*
Offline Offline

Activity: 1750
Merit: 1007



View Profile
October 20, 2011, 03:07:12 PM
 #12

I can say that TTL values on DNS are not always honored.  Even moreso if the program they're using to DDoS works the following:

1) Bot comes online, gets told to DDoS by a DNS name
2) Bot looks up DNS entry and caches it in the DDoS software
3) DDoS software hits that IP and never looks up for changed DNS settings

I have no idea how actual DDoS clients work, but I do know in most beginner networking courses your software looks up an IP address once and caches the IP address, rather than looking it up for each subsequent connection.

BTC Guild has had a lot of fun with DNS.  Our TTLs were normally set between 5 and 30 minutes.  It was not uncommon for somebody to come into IRC a day later asking why they can't connect.  Every time it was a DNS caching issue.

RIP BTC Guild, April 2011 - June 2015
bitlane
Internet detective
Sr. Member
****
Offline Offline

Activity: 462
Merit: 250


I heart thebaron


View Profile
October 20, 2011, 03:07:50 PM
 #13

LAME. Sour Grapes is all that is.....
You see that slush pool has died, so apparently there was no hope to save it. So the last thing a dying man could do was to try to at least find out why&how? this happens. There were some suspicions around that Eleuthria had deals with attackers. I think this step was - of course - not nice, but logical.

And also - btcguild was not harmed, by doing this! It only revealed that attackers like btcguild and will not attack.
Well, this is an unfortunate part of life and the world we live in......not just the Bitcoin world.

If eleuthria made a deal with the 'Devil' to save BTCGuild for the rest of us (miners that is) then I commend him for doin so.
I see NO SHAME in doing so and to attack BTCGuild because of it's relationship with the attackers of other pools is completely RETARDED....and of course, I use the term RELATIONSHIP loosely.

Anyone who thinks that eleuthria did wrong, should NEVER become involved in World Politics.

Anyone who supports Slush's attack on BTCG is wrong.....

eleuthria
Legendary
*
Offline Offline

Activity: 1750
Merit: 1007



View Profile
October 20, 2011, 03:09:55 PM
 #14

LAME. Sour Grapes is all that is.....
You see that slush pool has died, so apparently there was no hope to save it. So the last thing a dying man could do was to try to at least find out why&how? this happens. There were some suspicions around that Eleuthria had deals with attackers. I think this step was - of course - not nice, but logical.

And also - btcguild was not harmed, by doing this! It only revealed that attackers like btcguild and will not attack.

Not harmed is a bit exaggerated, considering the pool has been completely unstable for the past 16 hours or so, and I had two servers get nullrouted (possibly suspended, waiting on hetzner to reply to the explanation ticket).

RIP BTC Guild, April 2011 - June 2015
slush (OP)
Legendary
*
Offline Offline

Activity: 1386
Merit: 1097



View Profile WWW
October 20, 2011, 03:12:11 PM
 #15

DNS propogation is not instant, it can take hours in some cases for the new ip's to propagate to all the DNS servers in the world

Quote from: slush
But *everytime* when I changed DNS records to new IP or even when I created yet another DNS record (do you remember my post about new DNS api3.bitcoin.cz last week?), attack followed those changes in DNS or posts on forum almost immediately. Thanks to that I know attacker is here between us, following what's going and targeting attack to new places instantly (in matter of minutes).

Don't forget that this wasn't first attack to pool and this pattern was here every time. So yes, this *prove* that attacker didn't follow IP changes intentionally.

Caesium
Hero Member
*****
Offline Offline

Activity: 546
Merit: 500


View Profile
October 20, 2011, 03:12:24 PM
 #16

Not harmed is a bit exaggerated, considering I had two servers get nullrouted yesterday.

This is turning into slush's word vs your word now then. slush says nothing happened when the DNS was changed (though he's not really in a position to say that definitively, I'm guessing he just checked to see if btcg was still up) and you say the traffic did hit you.

Which is right?

Tired of annoying signature ads? Ad block for signatures
slush (OP)
Legendary
*
Offline Offline

Activity: 1386
Merit: 1097



View Profile WWW
October 20, 2011, 03:13:40 PM
 #17

Anyone who supports Slush's attack on BTCG is wrong.....

Actually this wasn't an attack, just a test. I didn't want to hurt btcguild and I was ready to switch DNS back asap in case of any troubles.

Btw I don't expect that all people understand what I did with DNS. Because not all people can empathise to the situation when you're facing attack and you want to understand at least something...

slush (OP)
Legendary
*
Offline Offline

Activity: 1386
Merit: 1097



View Profile WWW
October 20, 2011, 03:14:32 PM
 #18

Which is right?

btcguild went down around one hour later after my change of IPs *back* to my servers and was under an attack also today. So I really don't expect that attacker didn't notice that IP changed back agian.

bitlane
Internet detective
Sr. Member
****
Offline Offline

Activity: 462
Merit: 250


I heart thebaron


View Profile
October 20, 2011, 03:16:03 PM
 #19

Anyone who supports Slush's attack on BTCG is wrong.....

Actually this wasn't an attack, just a test. I didn't want to hurt btcguild and I was ready to switch DNS back asap in case of any troubles.

Semantics.....

Someone is firing a machine gun at you and the bullets are piercing and damaging you, so you knowingly turn the firing gun onto to someone else to see if they bleed too ?
....because you THOUGHT they might be bullet proof ? Or the attacker would remove his finger from the trigger once focused on someone else ?
Let's see THAT argument stand up in a court of law.

AnnihilaT
Full Member
***
Offline Offline

Activity: 210
Merit: 100



View Profile
October 20, 2011, 03:21:58 PM
 #20

edit: post outdated
Pages: [1] 2 3 4 »  All
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!