Payouts have been flowing steadily since about 12 hours after the attack.  I modified the code to do full precision payouts so anybody looking to leave and not come back after the event isn't going to start screaming about their 0.003 BTC that I held hostage.

Last night I added back wallet changing to the temporary account page.

New server is coming online later today (next 12 hours roughly).  Was having some difficulties getting out pfsense VM frontend to properly connect to the outside world last night, and I was not going to put up a brand new server configuration before getting some sleep.

Brand new server is being setup in a different facility.  Fatter pipes, they're not afraid of helping someone with a DDoS (whereas many other hosts would prefer a person who attracts DDoS's would GTFO).

DDoS is back on US East.

I've got another server getting setup with Awknet TODAY (website provider) which has been able to keep us online reasonably during the attacks.


Eri:  The botnet was donating 5%.  Considering the volume of CPU miners eating system resources, I think banning them (even though they were the biggest donator) would have ended up making me more due to everybody else working even faster without the idles.

Over a reasonable period of time.  Dynamic IP users won't need to fear the auto ban.

The logging I put in place is for a two criteria auto-blacklist:
  1) Too many IPs on a single worker.
  2) Too many requests with too few shares returned.

Well the secret is out.  Yes, US East is currently online.  The other servers are staying off while I monitor US East's activity.

The shares being sent to US east are valid and being counted, but My Account is currently still in its low-priority DDoS mode where it does not try to communicate with servers (thus no real-time stats getting pulled from US East).

US East is still counting shares.  It still had about 75k of share submission data from last night's uptime as well.  I'll be worknig on the stat polling/caching system tonight to get Worker Stats available again on a small delay.

It is not mining traffic.  If you look at the imgur link in the previous page, you'll see what i mean.

The DDoS is hitting all of our pools and the webserver.  The 100 mbps port on all of our servers was being capped.  A botnet of CPU miners does not generate 300k packets/sec.

Latest word from the DE servers is the servers there are getting hammered by 70-85k packets.  Per Second.

There's no whitelisting.  They're flooding the pipes, and it happens even if I iptable block ALL ips:

https://i.imgur.com/7MBZf.png

DDoS is back in full swing flooding our bandwith to its full capacity.

US East is coming back online as the DNS propagates.  Keeping a close eye on it.  I've completely rewritten the "getwork spam" logging to help identify problem IPs as well as potential botnets (high # of IPs on one worker).


US East is running, your shares are counting PERFECTLY.  The user stats are currently disabled just to stop the servers from constantly talking to each other until I know the DDoS is dying out.  You'll see the shares and rewards pop up on a new block as they always have.

I will be taking this unplanned downtime opportunity to rework the stats system to use cached user stats updated at regular intervals, rather than pulling live stats from all of the servers for a user everytime they load the API or My Account page.  This will allow the My Account and API to load almost instantly, the downside being information may be 1-5 minutes stale.

Manually setting the blocks as 120 confirms, even though a few of them aren't yet.  They're secure enough to allow people to get off the pool with 100% of their rewards if this DDoS has completely scared them away.

Website has been restored.  My Account is in a temporary state since it can't poll the other servers reliably.  The Request Payout has been added to the temporary page, and it will give you a payout in FULL PRECISION (all 8 decimals).

In the end, all this DDoS "cost" users is:
  A) Idle mining time if no failover was setp
  B) One round of shares (Block 1464).


At this time I am unable to access DE2 and US Central due to ISPs nulling the IP addresses from the attacks flooding their servers.  DE1 and US East did not find Block 1464 yet, so technically no actual rewards have been lost, only the shares submitted in the current round (which has not yet completed unless DE1/East found a block during the first few moments of the DDoS before they went offline completely).

Time for website functionality restoration pushed back about 1 hour.  Pages should now be properly redirecting everybody to the ddos explanation page.

Before bringing the "My Account" page back online, I'm adding some extra security checks to the payout code to make sure a payout doesn't get recorded to the database without its matching txid showing that it was successfully procssed by bitcoind.

Only 6.5 BTC unfortunately.  They were cashing out very regularly.  Apparently their high donation % was a subtle bribe.

Working on setting up an IP Whitelist similar to what slush implemented.  Taking a day off work so I can stay up and get it implemented ASAP.  Until its implemented, all the pools have had all traffic completely blocked off.

We are being DDoS'd.  DE1 and 2 have been completely shut down, website is highly unresponsive at random intervals.


UPDATE:  Apparently this guy is extremely pissed off.  We're getting bandwith flooded on all 5 servers, to the point that it's risking hitting our monthly bandwith caps.  Trying to fix the issue but all of the pools HAVE to go offline right now.

Lesson learned.  Filtering out a botnet ends up in a DDoS.  I guess they had to do SOMETHING with all those computers.

Put in some filters to stop the botnet(s) that were pointed at the servers.  IMMEDIATELY saw a performance boost to the servers.  Will monitor the results overnight to see if banning THOUSANDS of CPU miners cures the problems.

If you're having trouble connecting after the filters were put in place, send me a PM.  Botnets need not apply.

Registrations have been re-opened due to the servers showing an incredible recovery after the bans.

The account balance of the botnet has been donated to Bitcoin Faucet.