[ANN] Mt.Gox’s February 21st, Downtime Explanation

[Mt.Gox Support]

https://mtgox.com/press_release_20120221.html

TOKYO, JAPAN - February 21, 2012 - We would like to communicate that no data was lost, or stolen during this incident.

On February 21 at approximately 3pm JST, Mt.Gox and other websites hosted by Mt.Gox suffered an unplanned outage caused by a Kernel Panic on our main server.

Our system is configured so that a switch-over to our backup server should occur in such an incident, however we were caught off guard as this server is currently having upgrades performed.

A summary of actions taken to resolve the issue is as follows:

Our support deliver and management group requested immediately for the server to be rebooted by our provider in Texas. Once the main server was back online we found that its file system was corrupted on one drive. After a careful inspection of the file system we decided to update the server’s kernel to its latest stable version to avoid any further downtime as we weren't certain what caused the kernel panic.

While we understand how disruptive this downtime has been for our users, we would like to again reassure all our customers and the Bitcoin community that no data has been lost, leaked, or compromised and that the delay in getting things back online is due to having to re-compile a new Kernel and revert to offsite backups.

On behalf of Mt.Gox’s staff and Mark Karpeles we would like to sincerely apologizes for the inconvenience. Please rest assure that we will look at alternative solutions to prevent such incidents in the future.

Regards,
Mt.Gox Co. Ltd Team.

Media Contacts
press@mtgox.com

[1714055993]

1714055993

[1714055993]

1714055993

[1714055993]

1714055993

[1714055993]

1714055993

[cuz0882]

I guess we can thank your for the latest price drop.

[vuce]

I guess we can thank your for the latest price drop.

what price drop? What the hell are you talking about.

[lonelyminer (Peter Šurda)]

Why not use a distributed system instead of sticking everything onto one server? Based on a google search, bid matching algorithms can be parallelised.

[finway]

Not bad!

[thirdlight]

2nd failover required, even if only static page saying "This is the second failover. We're working on it."

[cuz0882]

Deposits from before the crash are still not showing up. Hopefully they get that figured out soon.

[Alphonso Bedoya]

For what it is worth, I appreciate this info in a clear and timely manner. Stuff happens, but not knowing makes things Much worse. Good job.

[rjk]

why are bitcointalk and mtgox hosted on the same server anyway?

..which suggests even more trouble:
Gain root on "bitcointalk" = gain root on "MtGox" too. megawallet.dat, achievement unlocked!

All this doesnt sound too professionally thought out.

The forum isn't on the same server as MtGox.

Whoever invented such a story ought to be strung up by their toenails

Sucks when the main server goes down just as the backup is undergoing maintenance. Having a static page saying "we're still here!" is more reassuring though.

[gusti]

Our support deliver and management group requested immediately for the server to be rebooted by our provider in Texas.

Am I the only one worried about the fact that mtgox servers are hosted in the US ?

[Serge]

in unrelated note,

are you guys still using mechanical hdd's or ssd's for your server storage? if yes, consider something with much greater read/write speeds
http://www.fusionio.com/
found out about it through one web host provider offering at http://www.webair.com/webhosting-dedicated-fusion.html

maybe it could even improve mtgoxlive

my understanding financial markets on Wall Street and such have been familiar with and utilizing this tech for quite some time now

[DmanX]

It must have affected Get-Bitcoin. I still haven't received my bitcoins from a deposit to Chase that I made yesterday. I have always received them about 14-16 hours later. Basically after midnight.

Sent emails, but no actual confirmation as to what the problem is. They're only 1 1/2 hour away from the 24 hour mark at the time of this post. 

[evoorhees]

For what it is worth, I appreciate this info in a clear and timely manner. Stuff happens, but not knowing makes things Much worse. Good job.

+1  Thank you Gox for the quick communication.

[Ente]

For what it is worth, I appreciate this info in a clear and timely manner. Stuff happens, but not knowing makes things Much worse. Good job.

Indeed, thank you for the quick and transparent info.
Well, why was bitcointalk offline, then?
(I was afraid they finally blocked it in the company's proxy! lol)

Ente

[rjk]

in unrelated note,

are you guys still using mechanical hdd's or ssd's for your server storage? if yes, consider something with much greater read/write speeds
http://www.fusionio.com/
found out about it through one web host provider offering at http://www.webair.com/webhosting-dedicated-fusion.html

maybe it could even improve mtgoxlive

my understanding financial markets on Wall Street and such have been familiar with and utilizing this tech for quite some time now

http://www.violin-memory.com/

How about some Violins? That would be epic.

[lonelyminer (Peter Šurda)]

http://www.fusionio.com/

http://www.violin-memory.com/

While I love SSD for databases, without having more detailed data about Mt. Gox systems it can't be said in advance that a SSD with make a noticeable improvement.

[iamapi]

Seems little impact on the price.