Bitcoin Forum

Bitcoin => Bitcoin Discussion => Topic started by: Mt.Gox Support on February 21, 2012, 10:14:34 AM



Title: [ANN] Mt.Gox’s February 21st, Downtime Explanation
Post by: Mt.Gox Support on February 21, 2012, 10:14:34 AM
https://mtgox.com/press_release_20120221.html (https://mtgox.com/press_release_20120221.html)

TOKYO, JAPAN - February 21, 2012 - We would like to communicate that no data was lost, or stolen during this incident.

On February 21 at approximately 3pm JST, Mt.Gox and other websites hosted by Mt.Gox suffered an unplanned outage caused by a Kernel Panic on our main server.

Our system is configured so that a switch-over to our backup server should occur in such an incident, however we were caught off guard as this server is currently having upgrades performed.

A summary of actions taken to resolve the issue is as follows:

Our support deliver and management group requested immediately for the server to be rebooted by our provider in Texas. Once the main server was back online we found that its file system was corrupted on one drive. After a careful inspection of the file system we decided to update the server’s kernel to its latest stable version to avoid any further downtime as we weren't certain what caused the kernel panic.

While we understand how disruptive this downtime has been for our users, we would like to again reassure all our customers and the Bitcoin community that no data has been lost, leaked, or compromised and that the delay in getting things back online is due to having to re-compile a new Kernel and revert to offsite backups.

On behalf of Mt.Gox’s staff and Mark Karpeles we would like to sincerely apologizes for the inconvenience. Please rest assure that we will look at alternative solutions to prevent such incidents in the future.

Regards,
Mt.Gox Co. Ltd Team.

Media Contacts
press@mtgox.com


Title: Re: [ANN] Mt.Gox’s February 21st, Downtime Explanation
Post by: cuz0882 on February 21, 2012, 10:16:53 AM
I guess we can thank your for the latest price drop.


Title: Re: [ANN] Mt.Gox’s February 21st, Downtime Explanation
Post by: vuce on February 21, 2012, 10:21:41 AM
I guess we can thank your for the latest price drop.
what price drop? What the hell are you talking about.


Title: Re: [ANN] Mt.Gox’s February 21st, Downtime Explanation
Post by: lonelyminer (Peter Šurda) on February 21, 2012, 10:27:34 AM
Why not use a distributed system instead of sticking everything onto one server? Based on a google search, bid matching algorithms can be parallelised.


Title: Re: [ANN] Mt.Gox’s February 21st, Downtime Explanation
Post by: finway on February 21, 2012, 10:45:14 AM
Not bad!


Title: Re: [ANN] Mt.Gox’s February 21st, Downtime Explanation
Post by: thirdlight on February 21, 2012, 10:49:15 AM
2nd failover required, even if only static page saying "This is the second failover. We're working on it."


Title: Re: [ANN] Mt.Gox’s February 21st, Downtime Explanation
Post by: cuz0882 on February 21, 2012, 11:01:10 AM
Deposits from before the crash are still not showing up. Hopefully they get that figured out soon.


Title: Re: [ANN] Mt.Gox’s February 21st, Downtime Explanation
Post by: Alphonso Bedoya on February 21, 2012, 03:24:09 PM
For what it is worth, I appreciate this info in a clear and timely manner. Stuff happens, but not knowing makes things Much worse. Good job.


Title: Re: [ANN] Mt.Gox’s February 21st, Downtime Explanation
Post by: rjk on February 21, 2012, 04:44:05 PM
why are bitcointalk and mtgox hosted on the same server anyway?

..which suggests even more trouble:
Gain root on "bitcointalk" = gain root on "MtGox" too. megawallet.dat, achievement unlocked!

All this doesnt sound too professionally thought out.

The forum isn't on the same server as MtGox.
Whoever invented such a story ought to be strung up by their toenails ;D

Sucks when the main server goes down just as the backup is undergoing maintenance. Having a static page saying "we're still here!" is more reassuring though.


Title: Re: [ANN] Mt.Gox’s February 21st, Downtime Explanation
Post by: gusti on February 21, 2012, 04:49:49 PM
Our support deliver and management group requested immediately for the server to be rebooted by our provider in Texas.

Am I the only one worried about the fact that mtgox servers are hosted in the US ?


Title: Re: [ANN] Mt.Gox’s February 21st, Downtime Explanation
Post by: Serge on February 21, 2012, 04:53:46 PM
in unrelated note,

are you guys still using mechanical hdd's or ssd's for your server storage? if yes, consider something with much greater read/write speeds
http://www.fusionio.com/
found out about it through one web host provider offering at http://www.webair.com/webhosting-dedicated-fusion.html

maybe it could even improve mtgoxlive ;)

my understanding financial markets on Wall Street and such have been familiar with and utilizing this tech for quite some time now


Title: Re: [ANN] Mt.Gox’s February 21st, Downtime Explanation
Post by: DmanX on February 21, 2012, 06:21:37 PM
It must have affected Get-Bitcoin. I still haven't received my bitcoins from a deposit to Chase that I made yesterday. I have always received them about 14-16 hours later. Basically after midnight.

Sent emails, but no actual confirmation as to what the problem is. They're only 1 1/2 hour away from the 24 hour mark at the time of this post. 


Title: Re: [ANN] Mt.Gox’s February 21st, Downtime Explanation
Post by: evoorhees on February 21, 2012, 06:58:10 PM
For what it is worth, I appreciate this info in a clear and timely manner. Stuff happens, but not knowing makes things Much worse. Good job.

+1  Thank you Gox for the quick communication.


Title: Re: [ANN] Mt.Gox’s February 21st, Downtime Explanation
Post by: Ente on February 21, 2012, 07:19:07 PM
For what it is worth, I appreciate this info in a clear and timely manner. Stuff happens, but not knowing makes things Much worse. Good job.

Indeed, thank you for the quick and transparent info.
Well, why was bitcointalk offline, then?
(I was afraid they finally blocked it in the company's proxy! lol)

Ente


Title: Re: [ANN] Mt.Gox’s February 21st, Downtime Explanation
Post by: rjk on February 21, 2012, 11:04:02 PM
in unrelated note,

are you guys still using mechanical hdd's or ssd's for your server storage? if yes, consider something with much greater read/write speeds
http://www.fusionio.com/
found out about it through one web host provider offering at http://www.webair.com/webhosting-dedicated-fusion.html

maybe it could even improve mtgoxlive ;)

my understanding financial markets on Wall Street and such have been familiar with and utilizing this tech for quite some time now
http://www.violin-memory.com/

How about some Violins? That would be epic.


Title: Re: [ANN] Mt.Gox’s February 21st, Downtime Explanation
Post by: lonelyminer (Peter Šurda) on February 21, 2012, 11:30:45 PM
http://www.fusionio.com/
http://www.violin-memory.com/
While I love SSD for databases, without having more detailed data about Mt. Gox systems it can't be said in advance that a SSD with make a noticeable improvement.


Title: Re: [ANN] Mt.Gox’s February 21st, Downtime Explanation
Post by: iamapi on February 22, 2012, 01:27:59 AM
Seems little impact on the price.