[ANN Mt.Gox] Resuming Operation: Now what?

[Mt.Gox_Alex]

https://mtgox.com/press_release_20130412.html

As announced yesterday, (https://mtgox.com/press_release_20130411.html) we have decided last night (https://support.mtgox.com/entries/21519569-Market-Cooldown-for-12-hours) to shutdown all trades for 12hrs.

This decision was made based on two major factors.

1. Trade Engine Lag
We have seen an astonishing growth in both volume and number of people joining Mt.Gox and trading on our platform. While we were ready and expected to see the number of new customers and trade rising, we did not anticipate such AMAZING growth that has been partially translated in visible and unbearable trade engine lag.

To fix this issue, we decided to make some major changes in our hardware infrastructure. It is now capable of handling 50 times more of the trades than in the past, giving us more time to release our new and improved trade engine, which is scheduled to be launched within a month.

2. DDoS
We have seen our fair share of DDoS attack in the past, but the sheer volume of the DDoS we received in the past couple of day was rather important with a peak at 80Gbps. Despite our service provider's best effort, we had a lot of difficulties to handle both our user growth and the DDoS.

What’s next?

Technically speaking, we are now ready to better handle our exchange's rapid growth and assure normal operation. As far as the DDoS are concerned, we are still fine-tuning our security features and actively working on implementing a solution that will drastically protect us from such huge attack in the future.
But rest assured that we are taking this situation very seriously, and that we are doing everything in our power to protect the Exchange and your trade from such attacks.

Even as we speak, we are still being attacked by a yet unknown group of people that have clearly an intention in hurting Mt.Gox and Bitcoin as a whole. But rest assured that we are not giving up and are dedicated to win this, one way or another!

What will happen to all the pending trades?

This has been discussed at length here at Mt.Gox. We have decided that we will not flush the order book before trading resumes. We have no intention in proactively clearing people’s order regardless of what’s happening, and we will let people be the judge of what’s best with their order.
There is no doubt that we are expecting a violent crash of Bitcoin after trading resumes, but we are confident that this Panic Sell will be temporarily and that Bitcoin will rise again.

Also active immediately, we will waive all trading fees for the next 48hrs (Starting from 11am today JST).

Finally

Again, thank you for your continued support. Despite suffering one of the most impressive DDoS that we ever seen, we are doing everything in our power to overcome this situation and to restore stability and confidence in our exchange.
      
   

Regards
Mt.Gox Co. Ltd Team.

Mt.Gox Contact press@mtgox.com

[aantonop]

Excuses and more excuses. Meanwhile 10 minutes of lag on your new "50x" scaled system (bullshit).

Has your COO been fired yet?

What am I talking about.... dorm room companies don't have COOs.


Maybe you should consider hiring some people who have done scalable architecture before?


Meanwhile everyone else should consider alternative exchanges and diversify. MTGOX has clearly demonstrated they have no clue, no professionalism, no plan and no operational experience. Their software has clearly reached the limits of scaling and now they're just throwing boxes at the problem (a sure sign of clueless reaction).

Some of the people here actually have a clue about operations and they're calling your BULLSHIT.

[Grover]

What part of BTC staying over $75usd didn't you get?  Then $100, $150, $175 and $200.

Either you have zero clue as to what is going on, or you thought BTC was overpriced and would crash causing a slowdown in traffic.

Your site has a huge fiduciary responsibility to the entire BTC network.  So much so that you either need to sell and become BTC sourced, or get taken out by a better company with open source architecture that is scalable and still crowd sourced with BTC.  Too much power in the hands of too few people who admittedly have no clue. 

Your admittance came in the form of an admittance by MagicalTux about your system:

https://bitcointalk.org/index.php?topic=172991.msg1802127#msg1802127

[12:29] <keymone_> MagicalTux tell me why are you not using more mature exchange system?
[12:29] <keymone_> like milleniumit or something
[12:30] <keymone_> the one that is behind london stock exchange
[12:30] <keymone_> why reinventing the wheel especially when you see the incoming exponential usage growth
[12:30] <@MagicalTux> keymone_: does it handle ddos attacks well?
[12:31] <keymone_> yesterday press release said it was not due to ddos
[12:31] <keymone_> it's 2 separate problems, solving 1 is already a huge progress
[12:31] <@MagicalTux> keymone_: we are already solving this
[12:31] <bit14> @MagicalTux: why exactly would a ddos affect the matching and execution engine?
[12:31] <@MagicalTux> bit14: overloading system cpu
[12:31] <@MagicalTux> ddos are not all about cpu
[12:32] <dandate2> um did rebooting mtgox cause already deleted recent bids to come into play?
[12:32] <bit14> @MagicalTux: wouldn't the front end and execution engine be on different machines?
[12:32] <@MagicalTux> dandate2: rebooting?
[12:32] <keymone_> are you trying to battle trading bots? with all these 0.1 buys and sells?
[12:32] <@MagicalTux> bit14: not yet
[12:32] <@MagicalTux> keymone_: unrelated
[12:32] <bit14> @MagicalTux: wow, well, then that explains everything

I don't mean to add even more insult to injury, but as a software engineer myself - I can't even comprehend why someone would create such an abomination.

[bitsalame]

https://mtgox.com/press_release_20130412.html

As announced yesterday, (https://mtgox.com/press_release_20130411.html) we have decided last night (https://support.mtgox.com/entries/21519569-Market-Cooldown-for-12-hours) to shutdown all trades for 12hrs.

This decision was made based on two major factors.

1. Trade Engine Lag
We have seen an astonishing growth in both volume and number of people joining Mt.Gox and trading on our platform. While we were ready and expected to see the number of new customers and trade rising, we did not anticipate such AMAZING growth that has been partially translated in visible and unbearable trade engine lag.

To fix this issue, we decided to make some major changes in our hardware infrastructure. It is now capable of handling 50 times more of the trades than in the past, giving us more time to release our new and improved trade engine, which is scheduled to be launched within a month.

2. DDoS
We have seen our fair share of DDoS attack in the past, but the sheer volume of the DDoS we received in the past couple of day was rather important with a peak at 80Gbps. Despite our service provider's best effort, we had a lot of difficulties to handle both our user growth and the DDoS.

What’s next?

Technically speaking, we are now ready to better handle our exchange's rapid growth and assure normal operation. As far as the DDoS are concerned, we are still fine-tuning our security features and actively working on implementing a solution that will drastically protect us from such huge attack in the future.
But rest assured that we are taking this situation very seriously, and that we are doing everything in our power to protect the Exchange and your trade from such attacks.

Even as we speak, we are still being attacked by a yet unknown group of people that have clearly an intention in hurting Mt.Gox and Bitcoin as a whole. But rest assured that we are not giving up and are dedicated to win this, one way or another!

What will happen to all the pending trades?

This has been discussed at length here at Mt.Gox. We have decided that we will not flush the order book before trading resumes. We have no intention in proactively clearing people’s order regardless of what’s happening, and we will let people be the judge of what’s best with their order.
There is no doubt that we are expecting a violent crash of Bitcoin after trading resumes, but we are confident that this Panic Sell will be temporarily and that Bitcoin will rise again.

Also active immediately, we will waive all trading fees for the next 48hrs (Starting from 11am today JST).

Finally

Again, thank you for your continued support. Despite suffering one of the most impressive DDoS that we ever seen, we are doing everything in our power to overcome this situation and to restore stability and confidence in our exchange.
      
   

Regards
Mt.Gox Co. Ltd Team.

Mt.Gox Contact press@mtgox.com

1st: Don't make promises you can't keep.
2nd: You've already broken your assurance: your website doesn't load. All trading is currently halted.
This is an utter embarrassment.

You should offer a whole month free of commissions... and even then we wouldn't be even.

[bitsalame]

"It is now capable of handling 50 times more of the trades than in the past"

[smoothie]

SO you are waving all trading fees for the next 48 hours yet your site is down. So how does that benefit anyone? You guys have the largest volume of trades per day and make the most money doing it. YOu have no excuse on why you could not scale up your system to account for all the new users and interest. These types of bullshit excuses will only net you less volume.

June 2011 your volume was 80% of total

April 2013 your volume is about 50% of total

My prediction is by August 2013 your total volume will be 33% at best.

Incompetence nets you less profit.

[bitsalame]

[bitcoinik]

Have you considered the possibility of changing your commission structure? They way it is currently calculated by percentages is not typical of trading exchanges, at least not in the United States. To help provide liquidity, you should charge a flat-rate commission fee. Then people don't have to think about percentages, and can factor in a flat fee to their trading strategies.

As an exchange you should benefit from people trading, as opposed to just the price of Bitcoins going up. That's what your commission structure looks like now. You are making it appear that you just have a vested interest in Bitcoins just going up, up, up.

[creativex]

Mt. Gox future commission structure should be very simple. 0% charged on zero volume as their former customers take their business to an exchange that wants the business.

[BTC4Amazon]

Excuses and more excuses. Meanwhile 10 minutes of lag on your new "50x" scaled system (bullshit).

Has your COO been fired yet?

What am I talking about.... dorm room companies don't have COOs.


Maybe you should consider hiring some people who have done scalable architecture before?


Meanwhile everyone else should consider alternative exchanges and diversify. MTGOX has clearly demonstrated they have no clue, no professionalism, no plan and no operational experience. Their software has clearly reached the limits of scaling and now they're just throwing boxes at the problem (a sure sign of clueless reaction).

Some of the people here actually have a clue about operations and they're calling your BULLSHIT.

According to another press release they have like 30+ employees in 1 department....then again IDK why if there going to keep screwing up

[aantonop]

I recently visited a HFT shop I was working with in NY. They're a small custom trading house that handles several hundred institutional clients for specilist HFT/algo trading.

5 people. Hundreds of millions. No downtime in four years.

31 people? Half of them are writing press releases and finding excuses.

[smoothie]

MTGOX IS DOWN AGAIN: NOW WHAT?

[repentance]

When a business is having problems of this magnitude, its principals should be the ones issuing public statements and making promises - not underpaid customer support staff who have no authority to do anything other than regurgitate press releases.

Nobody gives a toss about Alex's opinions on this matter.  What people want to hear is what Mark is going to do to fix the problem and when.  No-one gives a shit how much you've increased your capacity if it's still not enough.  The fact that you can be brought down by legitimate trading makes people worry about the robustness of the rest of your system because right now you've got one hell of a target on your back.

[aantonop]

You are imagining a level of executive competence that is greater than that of their spokesmonkeys. Given their recent performance and past history, I expect their management is about as incompetent as their PR hires.

[Surpbitcoin]

seems like there up again, these guys remind me of a yo-yo.

[creativex]

I wonder if Gox has a share a retard exchange program with BFL. I really think this program could benefit both failed companies.

[Scrat Acorns]

It is now capable of handling 50 times more of the trades than in the past

Quoted for hilarity.

[bowen151]

Has your COO been fired yet?

+ fucking 1  would like to hear the answer to this also, although Mt.Gox_Alex is probs just some kid sitting in his basement taking the complete piss, so I dont really think I would believe him if he told us that their COO is on the chopping block

[creativex]

Has your COO been fired yet?

+ fucking 1  would like to hear the answer to this also, although Mt.Gox_Alex is probs just some kid sitting in his basement taking the complete piss, so I dont really think I would believe him if he told us that their COO is on the chopping block

Gox's entire staff of the gifted with special needs should be looking for work. They've done more than enough harm to Bitcoin adoption. Fire them by boycotting their broken exchange.

[bowen151]

I used gox only once in around about 10 months or so and that was due to having a gox code and no other means/methods of accumulation.

I wont send anything finance related through those amatuers, and I hope that everyone decides that this is something the community as a whole cannot tolerate.

We can all come to a consensus within a few hours regarding hard forks, which chain to continue using, which client NEEDS to be in use... and other technical related bitcoin protocol problems that have the power to do much more damage, and we can fix those in no time at all, why can we not fix this?