Re: [ANN] Mt.Gox’s February 21st, Downtime Explanation

[coin_toss]

why are bitcointalk and mtgox hosted on the same server anyway? That is just dumb. We should never be in a situation where gox is down and we can't even come to the forums to find out what the hell is going on.

[1714945812]

1714945812

[Brian DeLoach]

why are bitcointalk and mtgox hosted on the same server anyway?

They are hosting it for free.

[Ente]

why are bitcointalk and mtgox hosted on the same server anyway?

..which suggests even more trouble:
Gain root on "bitcointalk" = gain root on "MtGox" too. megawallet.dat, achievement unlocked!

All this doesnt sound too professionally thought out.

Ente

[Mushoz]

why are bitcointalk and mtgox hosted on the same server anyway?

..which suggests even more trouble:
Gain root on "bitcointalk" = gain root on "MtGox" too. megawallet.dat, achievement unlocked!

All this doesnt sound too professionally thought out.

Ente

90-95% if not more of their funds is in cold storage, meaning a hack won't be able to access those coins. The "hacker" would need psychical access, ie robbing one of the banks where the private key to their wallet is stored. Good luck with that 

[muyuu]

90-95% if not more of their funds is in cold storage, meaning a hack won't be able to access those coins. The "hacker" would need psychical access, ie robbing one of the banks where the private key to their wallet is stored. Good luck with that 

I bet 5-10% of mtGox funds is still a LOT of money.

[Mushoz]

90-95% if not more of their funds is in cold storage, meaning a hack won't be able to access those coins. The "hacker" would need psychical access, ie robbing one of the banks where the private key to their wallet is stored. Good luck with that 

I bet 5-10% of mtGox funds is still a LOT of money.

Sure, but then again, I doubt there are that stupid that gaining root access to the forums would compromise their Mtgox site as well. Just saying
And I just checked the numbers again, it's 90-98% in cold storage, so that's great =]

[muyuu]

Sure, but then again, I doubt there are that stupid that gaining root access to the forums would compromise their Mtgox site as well. Just saying
And I just checked the numbers again, it's 90-98% in cold storage, so that's great =]

If it's true that they run in the actual same server I'm appalled with this decision. Absolutely crazy and absolutely not banking security standards.

[theymos]

why are bitcointalk and mtgox hosted on the same server anyway?

..which suggests even more trouble:
Gain root on "bitcointalk" = gain root on "MtGox" too. megawallet.dat, achievement unlocked!

All this doesnt sound too professionally thought out.

The forum isn't on the same server as MtGox.

[muyuu]

The forum isn't on the same server as MtGox.

Glad to hear that. Where did that rumour start then?

[theymos]

Glad to hear that. Where did that rumour start then?

MtGox does host the forum on the same network as MtGox (I think), but it's definitely on a different server.

[Matthew N. Wright]

Sure, but then again, I doubt there are that stupid that gaining root access to the forums would compromise their Mtgox site as well. Just saying
And I just checked the numbers again, it's 90-98% in cold storage, so that's great =]

If it's true that they run in the actual same server I'm appalled with this decision. Absolutely crazy and absolutely not banking security standards.

This video explains how it all happened. [Dated]

[theymos]

This video explains how it all happened. [Dated]

Please don't post this video without telling people that most of it is false. Someone might get confused.

[Matthew N. Wright]

This video explains how it all happened. [Dated]

Please don't post this video without telling people that most of it is false. Someone might get confused.

Not sarcasm, but what's false? Can we start a new thread about it and clarify?

[theymos]

Not sarcasm, but what's false?

- The vulnerabilty was not "well-known". It was a 0-day vulnerability later patched by SMF. Probably you took the Buttcoin "report" on the vulnerability to be true, but that article was entirely made-up. The vulnerability had nothing to do with JavaScript or smilies.
- The issue wasn't used as an excuse to transfer the forum to MtGox. Sirius and I tried to track down the vulnerability, but we had no experience in this area and we couldn't find it, so Sirius decided to transfer responsibility of managing the server to someone else.
- The transfer to MtGox wasn't a secret. Sirius emailed ~13 of the top Bitcoiners to discuss transfering management of the server to someone else. Mark volunteered: Jeff Garzik, Sirius, and I agreed that he was the best choice. I mentioned this on IRC shortly after the decision was made. The email conversation was later published.
- If that $100,000 offer would have been accepted, all of the money would have been donated to the development group or some other appropriate non-profit organization. (This offer was discussed among the same "core group" as above.)

[Matthew N. Wright]

Not sarcasm, but what's false?

- The vulnerabilty was not "well-known". It was a 0-day vulnerability later patched by SMF. Probably you took the Buttcoin "report" on the vulnerability to be true, but that article was entirely made-up. The vulnerability had nothing to do with JavaScript or smilies.
- The issue wasn't used as an excuse to transfer the forum to MtGox. Sirius and I tried to track down the vulnerability, but we had no experience in this area and we couldn't find it, so Sirius decided to transfer responsibility of managing the server to someone else.
- The transfer to MtGox wasn't a secret. Sirius emailed ~13 of the top Bitcoiners to discuss transfering management of the server to someone else. Mark volunteered: Jeff Garzik, Sirius, and I agreed that he was the best choice. I mentioned this on IRC shortly after the decision was made. The email conversation was later published.
- If that $100,000 offer would have been accepted, all of the money would have been donated to the development group or some other appropriate non-profit organization. (This offer was discussed among the same "core group" as above.)

I'll add this quote to the video page for clarity. Thanks!

[coin_toss]

why are bitcointalk and mtgox hosted on the same server anyway?

..which suggests even more trouble:
Gain root on "bitcointalk" = gain root on "MtGox" too. megawallet.dat, achievement unlocked!

All this doesnt sound too professionally thought out.

The forum isn't on the same server as MtGox.

Thank you for correcting my earlier misstatement. I do not have any special knowledge about the server's MtGox and bitcointalk are hosted on. I was merely basing my comments on the press release made by MtGox following the downtime, which said:

"On February 21 at approximately 3pm JST, Mt.Gox and other websites hosted by Mt.Gox suffered an unplanned outage caused by a Kernel Panic on our main server."

I assumed this main server was hosting both sites. If, as you say, they are hosted on separate servers, that does alleviate some of my concerns. However, I still maintain that we should not ever be in a situation where both sites go down together. These sites are unquestionably the two most important resources in the bitcoin community - surely they can be hosted in a more independent manner. If Mark is going to be responsible for both sites so be it, but he should at least use 2 separate hosting providers.

[DBordello]

My understanding from the Mt. Gox documents and IRC conversations with MagicalTux is that Mt. Gox owns a pair of very beefy servers (64 cores, 96gb of ram, etc).  These servers are used to host virtual machines for various uses (Mt. Gox web servers, backend, etc).  I would guess that bitcointalk is hosted on one of these virtual machines.  When the host server kernel panic'ed, all of the virtual machines went down, including bitcointalk.   

Source: https://mtgox.com/press_release_20120201.html slide 8.