ssinc
Sr. Member
Offline
Activity: 364
Merit: 250
Hash for Cash!
|
|
January 17, 2014, 05:57:55 AM |
|
Definitely down.
|
|
|
|
dmcdad
|
|
January 17, 2014, 06:05:17 AM |
|
Slush appears to be down as well...
|
|
|
|
ssinc
Sr. Member
Offline
Activity: 364
Merit: 250
Hash for Cash!
|
|
January 17, 2014, 06:06:47 AM |
|
Slush appears to be down as well...
and BTCguild... what's going on
|
|
|
|
dmcdad
|
|
January 17, 2014, 06:08:52 AM |
|
Yeah, slush seems to be limping along for me now. Others are reporting problems though. Definitely smells like something is up (coordinated DDoS?).
|
|
|
|
klondike_bar
Legendary
Offline
Activity: 2128
Merit: 1005
ASIC Wannabe
|
|
January 17, 2014, 06:14:09 AM |
|
Yeah, slush seems to be limping along for me now. Others are reporting problems though. Definitely smells like something is up (coordinated DDoS?).
ghash.io appears fully functional...
|
|
|
|
lightfoot
Legendary
Offline
Activity: 3178
Merit: 2260
I fix broken miners. And make holes in teeth :-)
|
|
January 17, 2014, 06:23:08 AM |
|
On a train to Arisia here. Eclipse is up and fully operational; they're my backup pool. Hm. I'm cruising at 182gh there, not bad actually.
So hit eclipse if you need hash space.
C
|
|
|
|
merv77
|
|
January 17, 2014, 06:42:41 AM |
|
stratum+tcp://stratum.mining.eligius.st:12234 - went down for me
stratum+tcp://stratum.mining.eligius.st:3334 - is working
|
|
|
|
wizkid057
Legendary
Offline
Activity: 1223
Merit: 1006
|
|
January 17, 2014, 07:07:05 AM |
|
Seems someone tried an interesting DoS against Eligius (and possibly other pools). For Eligius, it caused our database to overload and stop accepting shares. (I'll post more details once I patch against this.) Nothing compromised. No security issues. Servers are fine, etc. No worries, I have the pool in full failsafe mode so that shares are accepted and logged. Unfortunately stats pages do not work in this mode. I'm working on correcting the underlying issue so this can't happen again. Since it is ~2AM here I'm not sure I have the energy to actually finish this morning, but I will try. In any case, no accepted shares are lost and all will be reconciled and caught up as soon as I get everything back in working order. No shares are lost! No earnings are lost, we even just found a block 00000000000000010190c8a62986f0fc126fd7082a580422f47bb91af80b2728. Once all is done I will get a manual payout together and catch up the payout queue. Thanks for mining! -wk
|
|
|
|
HellDiverUK
|
|
January 17, 2014, 08:18:25 AM |
|
Thanks for all your efforts wizkid057. I didn't see any of my miners change pool all night, so looks like it didn't effect me (though if BTCGuild was also hit, they were my backup I'm not how that'd have been handled).
|
|
|
|
wizkid057
Legendary
Offline
Activity: 1223
Merit: 1006
|
|
January 17, 2014, 10:17:25 AM Last edit: January 17, 2014, 11:01:24 AM by wizkid057 |
|
Looks like this is taking longer than expected, and I'm working off of only a couple hours of sleep. Sorry folks, I tried. I need at least a short nap before my commute. Unfortunately by my estimates it will take another couple of hours to finish this full recovery, and I have to head out to my normal day job in about 90 minutes. So, we're going to be in fail safe mode for a while, since it will have to wait until I get home from work (~4PM ET, or about 11 hours from now) before I can actually continue to address it. I'll repeat as normal: No shares are lost, no earnings are lost. Please do not panic and just continue to mine as normal. The pool is still finding blocks, they're being paid to the offline wallet for now. I will get stats and payouts caught up after I return home from work, since the majority of everything needed to be done (waiting on databases) should be done by then. I expect everything to be fully operational no later than about 16 hours from the time of this post, allotting time for unexpected delays. Thanks for your patience. As always, if Eligius accepts a share from your miner, and your username is a proper bitcoin address (with or without optional _workername appended), then your share is counted and you have nothing to worry about. -wk
|
|
|
|
lightfoot
Legendary
Offline
Activity: 3178
Merit: 2260
I fix broken miners. And make holes in teeth :-)
|
|
January 17, 2014, 10:29:32 AM |
|
Looks like this is taking longer than expected, and I'm working off of only a couple hours of sleep. Sorry folks, I tried. I need at least a short nap before my commute. Yaknow, the fact that you're up all night working on this says a lot about you. Thank you for working on it, sleep well and our bitcoin money will be there. C (Still on Amtrak train, it's nice traveling this way. They need to take bitcoins at the snack bar though)
|
|
|
|
kodiak1120
|
|
January 17, 2014, 12:55:54 PM |
|
Thanks for the effort, WK.
Kevin
|
|
|
|
|
joeventura
|
|
January 17, 2014, 06:15:28 PM |
|
Not malicious Discus Fish Pool
|
|
|
|
rankodan
Newbie
Offline
Activity: 6
Merit: 0
|
|
January 17, 2014, 07:01:14 PM |
|
Thank you for the hard work WK! Appreciate you took the time to communicate with the miners during this high pressure period. Best wishes!
|
|
|
|
wizkid057
Legendary
Offline
Activity: 1223
Merit: 1006
|
|
January 17, 2014, 10:23:26 PM |
|
Just a quick update.
Overall things are progressing. One item in particular is taking much longer than expected, so, I may overshoot my ETA for full functionality.
In any case, no worries. I'm on it and I'm not going anywhere until everything is back up and running as it should.
-wk
P.S. - As always, if Eligius accepts a share from your miner, and your username is a proper bitcoin address (with or without optional _workername appended), then your share is counted and you have nothing to worry about.
|
|
|
|
wizkid057
Legendary
Offline
Activity: 1223
Merit: 1006
|
|
January 18, 2014, 03:17:14 AM |
|
Another update.... things are progressing slowly. I'm currently migrating the emergency/failsafe share logs to the main database servers.
It looks like its going to take a while unfortunately. I will keep you posted.
-wk
|
|
|
|
wizkid057
Legendary
Offline
Activity: 1223
Merit: 1006
|
|
January 18, 2014, 04:01:26 AM |
|
Still pushing shares to the database. Here is what needs to happen for everything to be back to normal: - Finish getting failsafe shares from the past ~24 hours into the shares database
- Let CPPSRB catch up to realtime using this data
- Let replication to the www/stats server catch up
- Let stats server complete stats scripts on this data once it is replicated
So, not as far along as I would have liked. I expected to be able to begin moving the failsafe shares into the database immediately when I returned home about 7 hours ago, but the database was not ready. So I'm several hours behind my original time table, which put even more shares in the failsafe logs, slowing things down even more. No worries though. It will get done. Thanks for mining! -wk
|
|
|
|
arousedrhino
|
|
January 18, 2014, 05:06:41 AM |
|
Thanks Wiz, I really appreciate the regular updates on the situation!
|
|
|
|
merv77
|
|
January 18, 2014, 05:49:20 AM |
|
thanks wizkid, much appreciated
|
|
|
|
|