Bitcoin Forum

Economy => Service Discussion => Topic started by: aantonop on April 11, 2013, 10:12:31 AM



Title: mtgox under Distributed Denial of Competence attack
Post by: aantonop on April 11, 2013, 10:12:31 AM
A horde of incompetent engineers, mealy-mouthed apologists and clueless morons have inundated MagicTheOnlineGatheringeXchange hiring department and have been inadvertently hired.

As a result, mtgox is currently limited to one trade PER hour. This is to ensure the continued "reliability" of the service.

See, when you're making hundreds of thousands of dollars in commissions, it is important to laugh at your customers and THANK them for making you a "victim of success". Because nothing says "Thank you", like a one hour lag that wipes out 1 BILLION USD from the valuation of your nascent currency.

No, thank YOU mtgox. Thank you for teaching us all to avoid your "service" before bitcoin gets so big that you become the Lehman Brothers of bitcoin.



Title: Re: mtgox under Distributed Denial of Competence attack
Post by: jorov on April 11, 2013, 10:34:13 AM
Bitcoin should have stayed decentralized.

80 % of all Bitcoin trade ? WTF ?


Title: Re: mtgox under Distributed Denial of Competence attack
Post by: bit-joker on April 11, 2013, 11:47:21 AM
If mtgox is so crap as you're saying, you can always create a better company and offer a better service. Instead of blaming mtgox ", you can start using their competitors or create a new company to compete.


Title: Re: mtgox under Distributed Denial of Competence attack
Post by: aantonop on April 11, 2013, 11:52:36 AM
Unless I am willing to start an exchange, I can't complain about the obvious incompetence of mtgox?
I am a customer who just watched a company destroy $1 billion of value because of technical glitches.

Bet you loved Microsoft too and thought no one should criticize them unless they went out and made their own OS. Eventually someone did and I worked on that project too (Linux). Isn't that a rather wasteful way to solve problems?

I'd rather people realized that mtgox were victims of their own incompetence and tried the other exchanges. More volume on the other 4-5 exchanges would remove mtgox as the single point of failure.




Title: Re: mtgox under Distributed Denial of Competence attack
Post by: Lethn on April 11, 2013, 11:59:58 AM
Bitcoin should have stayed decentralized.

80 % of all Bitcoin trade ? WTF ?

The great thing about an unregulated market is these guys will go out of business and the next people will have to be better rather than get a bailout.


Title: Re: mtgox under Distributed Denial of Competence attack
Post by: alexeft on April 11, 2013, 12:00:44 PM
Every piece of machinery has a certain capacity and its managers usually size it to the work it usually has to do. Greatly oversizing it is not a good thing to do.

Yesterday, almost every other exchange had great lag. I can perfectly understand them being the victims of the sudden bitcoin publicity and the new deposit confiscation laws around the world.

Gox is already adding servers and bettering their code, at least as far as they have told us.

Have a little patience and faith people.  


Title: Re: mtgox under Distributed Denial of Competence attack
Post by: aantonop on April 11, 2013, 12:03:11 PM
Every piece of machinery has a certain capacity and its managers usually size it to the work it usually has to do. Greatly oversizing it is not a good thing to do.

Yesterday, almost every other exchange had great lag. I can perfectly understand them being the victims of the sudden bitcoin publicity and the new deposit confiscation laws around the world.

Gox is already adding servers and bettering their code, at least as far as they have told us.

Have a little patience and faith people.  



Oversizing?

Have you ever done capacity planning? Have you ever configured a scale-out architecture?

Don't worry, neither has anyone at mtgox...

You keep patient, your patience will be rewarded by more flash crashes, if you think that mtgox is getting better. They're getting worse because a non-scalable architecture becomes more and more fragile the more hardware you throw at it if you don't know how to architect.


Title: Re: mtgox under Distributed Denial of Competence attack
Post by: rograz on April 11, 2013, 12:09:03 PM
They should just close down account creation until they can sort their shit out, would be good for the bitcoin community as a whole if they helped pushing people towards other exchanges and stabilize their service.


Title: Re: mtgox under Distributed Denial of Competence attack
Post by: jorov on April 11, 2013, 01:04:51 PM
But they want the mils !
They just made a million $ in 1 day ? Would you close your business if it performs so good ?


Title: Re: mtgox under Distributed Denial of Competence attack
Post by: aantonop on April 11, 2013, 01:12:05 PM
Would you close your business if it performs so good ?

Let's close it for them. The exit is that way -->


bitfloor, btc-e, intersango, kraken, crypto-trader... many old and new options.



Title: Re: mtgox under Distributed Denial of Competence attack
Post by: shkiser on April 11, 2013, 01:17:27 PM
BOYCOTT GOX. They control the majority of btc exchange, and with every flash crash they are taking in more of the BTC%; allowing them to continue their pump and dump, and orchestrating these spikes and falls, profiting from each one.. How fucking obvious does it have to be. Their recent anouncement is proof enough. Gox gloating at their tremendous profits they just reaped in a matter of hours. How foolish can ppl be? If we cant divert this power away from mt gox, I dont want to have anything to do with the btc and making these guys billions. This is crazy.


Title: Re: mtgox under Distributed Denial of Competence attack
Post by: alexeft on April 11, 2013, 02:10:17 PM
Every piece of machinery has a certain capacity and its managers usually size it to the work it usually has to do. Greatly oversizing it is not a good thing to do.

Yesterday, almost every other exchange had great lag. I can perfectly understand them being the victims of the sudden bitcoin publicity and the new deposit confiscation laws around the world.

Gox is already adding servers and bettering their code, at least as far as they have told us.

Have a little patience and faith people.  



Oversizing?

Have you ever done capacity planning? Have you ever configured a scale-out architecture?

Don't worry, neither has anyone at mtgox...

You keep patient, your patience will be rewarded by more flash crashes, if you think that mtgox is getting better. They're getting worse because a non-scalable architecture becomes more and more fragile the more hardware you throw at it if you don't know how to architect.


How exactly would you scale out any architecture having to quadruple its capacity within less than a month? Including the relevant human resources, not just machinery.


Title: Re: mtgox under Distributed Denial of Competence attack
Post by: Aztec on April 11, 2013, 02:18:39 PM
They were very unprepared but I don't know how much of that is their fault. Would other organisations be able to deal with the level of DDoS attacks they experienced?

This is a question :)


Title: Re: mtgox under Distributed Denial of Competence attack
Post by: alexeft on April 11, 2013, 02:26:03 PM
They were very unprepared but I don't know how much of that is their fault. Would other organisations be able to deal with the level of DDoS attacks they experienced?

This is a question :)

It was not only gox that were unprepared. It was just about everybody, including this very forum (btw it is slow as we speak).
But that is to expected with that kind of skyrocketing growth that bitcoin and the community around it is experiencing.

At least that is my thought.


Title: Re: mtgox under Distributed Denial of Competence attack
Post by: deadweasel on April 11, 2013, 02:37:39 PM
They were very unprepared but I don't know how much of that is their fault. Would other organisations be able to deal with the level of DDoS attacks they experienced?

This is a question :)

It was not only gox that were unprepared. It was just about everybody, including this very forum (btw it is slow as we speak).
But that is to expected with that kind of skyrocketing growth that bitcoin and the community around it is experiencing.

At least that is my thought.

It is expected of incompetent operators of exchanges, forums, etc.  Everyone knew this influx was coming, especially those operators.  They have not hardened their systems, upgraded servers, or protected themselves with anti-ddos technology.

Forum has some THOUSANDS of BTC for upkeep, none of which has been spent to improve the forum.

MtGox, only really profitable in the last couple of months (due to incompetence), had plenty of time to upgrade and re-organize their systems for the incoming onslaught that was apparent to everyone.  They did not do so.

The only thing that should have been expected is that none of these organizations have acted in the slightest bit professional way.  This is Bitcoin-BETA -- expect BETA service or worse -- it is all you will receive.  


You can all still buy/sell/trade here:  http://bitcoin-otc.com/vieworderbook.php    <-- The only decentralized exchange besides LocalBitcoins.com.


Title: Re: mtgox under Distributed Denial of Competence attack
Post by: Sage on April 11, 2013, 02:41:09 PM
Another exchange doesn't solve the problem that is inherent in the system architecture.

The only thing that solves the Mtgox problem is a decentralized exchange.  And that's gotta be an open-source endevour.



Title: Re: mtgox under Distributed Denial of Competence attack
Post by: alexeft on April 11, 2013, 02:54:55 PM
Many decentralized exchanges would be even better as there is no single point of failure, but this industry in brand new and undeveloped yet.

Demanding that much is like going into a newly discovered forest and getting angry because there is no fast-food!!!


Title: Re: mtgox under Distributed Denial of Competence attack
Post by: alexeft on April 11, 2013, 02:57:15 PM


You can all still buy/sell/trade here:  http://bitcoin-otc.com/vieworderbook.php    <-- The only decentralized exchange besides LocalBitcoins.com.

Thanks but I don't day-trade. In the case of bitcoin, I don't even year-trade.


Title: Re: mtgox under Distributed Denial of Competence attack
Post by: someguy123 on April 11, 2013, 03:00:10 PM
Another exchange doesn't solve the problem that is inherent in the system architecture.

The only thing that solves the Mtgox problem is a decentralized exchange.  And that's gotta be an open-source endevour.


I wonder what's keeping people... I mean there is ripple, sort-of, and there's also bitcoin-otc.


Title: Re: mtgox under Distributed Denial of Competence attack
Post by: Aztec on April 11, 2013, 03:12:19 PM
This is Bitcoin-BETA -- expect BETA service or worse -- it is all you will receive.  

Good way to describe it. In a book I read called the mythical man month (I'd recommend to anyone) the author pretty much says the first thing you develop should be thrown away and rebuilt because it's only when you actually build the thing that you realise it's failings and potential. Feature creep anyone?

At least after all this debacle lessons should have been learned and services should improve. Either that or BTC and it's governing infrastructures will be replaced with something else.


Title: Re: mtgox under Distributed Denial of Competence attack
Post by: dutt on April 11, 2013, 03:58:03 PM


You can all still buy/sell/trade here:  http://bitcoin-otc.com/vieworderbook.php    <-- The only decentralized exchange besides LocalBitcoins.com.

Thanks but I don't day-trade. In the case of bitcoin, I don't even year-trade.

That's good then because OTC is probably the worst way to day trade ;)


Title: Re: mtgox under Distributed Denial of Competence attack
Post by: moni3z on April 11, 2013, 06:59:00 PM
http://i1110.photobucket.com/albums/h451/silvercent/goxed.png



Title: Re: mtgox under Distributed Denial of Competence attack
Post by: aantonop on April 11, 2013, 09:28:11 PM

How exactly would you scale out any architecture having to quadruple its capacity within less than a month? Including the relevant human resources, not just machinery.


Is that a serious question? Because if it is, then it is the question mtgox did NOT ask of any serious professional.

You see, I *CAN* answer the question: How do you plan for 10x, 100x, 1000x scalability. That's what a systems architect for financial trading systems does.

It is a very answerable question. The only question is how much money can you spend and how quickly can you hire experts. They've had 2 years and had not anticipated 4x growth? Seriously? A proper capacity plan for something like this would anticipate and accomodate 100x growth not 4x.

Business do this stuff ALL THE TIME. I know, I do this kind of data center work for large companies.

There is no excuse. It's plain incompetence


Title: Re: Mt. Gox --> Bitfloor
Post by: DataPlumber on April 11, 2013, 09:31:55 PM
FWIW, I contacted Bitfloor and asked if they have their papers in order and if they could handle an exodus from Mt. Gox.

According to the email that I received, they are registered, and—of course they would say this—they believe that they could scale.

As I say, FWIW, but there it is.
The last hour or two has disputed that allegation.

FFS.


Title: Re: mtgox under Distributed Denial of Competence attack
Post by: moni3z on April 11, 2013, 09:34:47 PM
When you have multi-millions in trading fees you'd be surprised what good trading engines you can buy, or people you can hire. Or don't hire anybody and run a 2 man show and outsource the rest to India





Title: Re: mtgox under Distributed Denial of Competence attack
Post by: koobie on April 11, 2013, 09:43:33 PM
I didn't notice anyone complaining when the price was going up.  When the bubble burst, as (almost) everyone knew it would, the blame get's thrown on the exchange.
Yeah, they underestimated capacity.  They did not cause the bubble or its inevitable burst.


Title: Re: mtgox under Distributed Denial of Competence attack
Post by: aantonop on April 11, 2013, 09:47:50 PM
I didn't notice anyone complaining when the price was going up.  When the bubble burst, as (almost) everyone knew it would, the blame get's thrown on the exchange.
Yeah, they underestimated capacity.  They did not cause the bubble or its inevitable burst.



You didn't notice anyone complaining when there was no lag and you noticed people complaining when the lag reached 1.2 HOURS. Your point?

I actually made quite a lot of money yesterday from the crash. Personally, I gained, but did so under tremendous uncertainty and only because I can code more resilient code. My trading app was able to continue to make trades even under tremendous lag and so I made a bit of money.

The point here is that mtgox didn't just fail, they failed spectacularly, in a way that was entirely predictable and seriously damaged the market's ability to rebound and adjust. It fed the panic uneccessarily.

It will only get worse. You can pretend this is just sour grapes, but for me the grapes were sweet and I still SPIT in mtgox's face.


Title: Re: mtgox under Distributed Denial of Competence attack
Post by: koobie on April 11, 2013, 10:07:09 PM
I didn't notice anyone complaining when the price was going up.  When the bubble burst, as (almost) everyone knew it would, the blame get's thrown on the exchange.
Yeah, they underestimated capacity.  They did not cause the bubble or its inevitable burst.



You didn't notice anyone complaining when there was no lag and you noticed people complaining when the lag reached 1.2 HOURS. Your point?

I actually made quite a lot of money yesterday from the crash. Personally, I gained, but did so under tremendous uncertainty and only because I can code more resilient code. My trading app was able to continue to make trades even under tremendous lag and so I made a bit of money.

The point here is that mtgox didn't just fail, they failed spectacularly, in a way that was entirely predictable and seriously damaged the market's ability to rebound and adjust. It fed the panic uneccessarily.

It will only get worse. You can pretend this is just sour grapes, but for me the grapes were sweet and I still SPIT in mtgox's face.

My point is that this is a new level for BTC.  I'm not excusing mtgox for their failure.  I'm hoping this will wake up the other exchanges and services and even the community as a whole to the fact that there are systems out there that will fail under extreme loads.  I don't know if it will get worse or if this will scare off a good number of speculators.  I am confident it will happen again at some point.  Maybe people will be more cautious.


Title: Re: mtgox under Distributed Denial of Competence attack
Post by: darkmule on April 11, 2013, 10:42:30 PM
Maybe I'm in tinfoil hat territory, but I find it hard to believe any entity that claims to have 80% of BTC exchange traffic is not going to leverage its market position for its own personal profits at the expense of its customers.  That is just too much for any entity to have.

I don't have any proof they're personally doing blatant pump-and-dump scams, but the temptation to make self-interested choices, or even to take the exchange down just to protect their own market position, seems as if it will inevitably be irresistible.

I've never dealt with Mt. Gox though I have an account there, and other than being up, coinbase.com frankly did not do much better, having a spread between their buy and sell price of over $40 while Gox was down, a ludicrous spread.  This is why I generally don't keep anything in these untrustworthy exchanges other than when I'm actually buying or selling.  It cuts my nimbleness a bit in responding to sudden swings in the market, but also keeps me from ending up with a lot of money in some belly-up scam operation.


Title: Re: mtgox under Distributed Denial of Competence attack
Post by: repentance on April 11, 2013, 10:48:43 PM
Many decentralized exchanges would be even better as there is no single point of failure, but this industry in brand new and undeveloped yet.

Demanding that much is like going into a newly discovered forest and getting angry because there is no fast-food!!!

How much longer are we going to continue accepting "Bitcoin is new" and "these are start ups" as an excuse?

Services launching today need to launch with a greater capacity and more resources than those which launched two years ago - that's a given.

MtGox has been the dominant exchange for a long time now and this kind of surge was expected.  It's one thing for an exchange which launched last year and which has a 5% market share to get blind-sided by this kind of surge (although I still maintain EVERY service needs the capacity to scale quickly baked in), it's an other entirely when an effective monopoly has no coherent plan for an anticipated situation.

Again and again we've seen Bitcoin enterprises fail because they lack the infrastructure to handle rapid growth.  When these enterprises are custodians of millions of dollars worth of user funds, such short-comings are simply unacceptable.  You don't take the money and put the infrastructure in place when you feel like it or when you "can afford it".  You limit how many customers you accept and the amount of funds for which you assume responsibility until you have in place the infrastructure to manage all of the risks such growth entails.



Title: Re: mtgox under Distributed Denial of Competence attack
Post by: darkmule on April 11, 2013, 11:22:16 PM
MtGox has been the dominant exchange for a long time now and this kind of surge was expected.  It's one thing for an exchange which launched last year and which has a 5% market share to get blind-sided by this kind of surge (although I still maintain EVERY service needs the capacity to scale quickly baked in), it's an other entirely when an effective monopoly has no coherent plan for an anticipated situation.

If they have an effective monopoly, that's our fault.  Stop patronizing entities like this.


Title: Re: mtgox under Distributed Denial of Competence attack
Post by: PuertoLibre on April 11, 2013, 11:57:30 PM
When you have multi-millions in trading fees you'd be surprised what good trading engines you can buy, or people you can hire. Or don't hire anybody and run a 2 man show and outsource the rest to India




Spot on.

I know I have paid them quite a bit in just two or three trades. So multiply that by several tens of thousands accounts...and you get the idea.

They can afford to hire the right people and buy the right hardware. I am not saying it will be cheap. But guess what? I am (as the rest of the community is) funding it through our fees.

They can afford it. They can even afford to get it done [very] quickly.

Edit: If they haven't hired consultants and have incoming orders for the fastest servers you can place in a datacenter, then they are truly being incredibly [willfully] incompetent.

Consider for a moment that one day in the near future, MTGox will likely be trading several billion USD/BTC and other [crypto]currencies. Their infrastructure costs are going to be a tiny penny on every dollar at best.


Title: Re: mtgox under Distributed Denial of Competence attack
Post by: PuertoLibre on April 12, 2013, 12:12:29 AM
They NEED someone to audit their platform and give them concise and rapid plan for implemented changes. They make it sound like the proper hardware and infrastructure is far above them.

Let them show you their balance sheet on a daily basis and I'd bet no one here would agree with them on that.

If they have a DB problem, they can buy specialized hardware and implementations that can go online very quickly to handle massive amounts of transactions and scale efficiently. If they have front end problems, there are lots of companies they can approach to get that under control in less than a day. No matter how large the demand.

The more you know about large scale business implementations, the more doubtful the words coming from MtGox actually sound. The less you know, the more their story sounds legitimate. Get educated about scaling and you'll quickly know that any problem that can be encountered can be readily solved.

That MTGox is still unable to fix it after this many days just goes to show that priority is being put elsewhere. (IMO)



Title: Re: mtgox under Distributed Denial of Competence attack
Post by: darkmule on April 12, 2013, 12:54:56 AM
The more you know about large scale business implementations, the more doubtful the words coming from MtGox actually sound. The less you know, the more their story sounds legitimate. Get educated about scaling and you'll quickly know that any problem that can be encountered can be readily solved.

That MTGox is still unable to fix it after this many days just goes to show that priority is being put elsewhere. (IMO)

Frankly, their apparent inability to fix what should be the core of their business during a high time of volume almost inevitably suggests that they have some motivation for not doing it.  Since the other reason is pure incompetence, neither bodes well for their future or their trustworthiness.


Title: Re: mtgox under Distributed Denial of Competence attack
Post by: repentance on April 12, 2013, 01:58:27 AM

They can afford to hire the right people and buy the right hardware. I am not saying it will be cheap. But guess what? I am (as the rest of the community is) funding it through our fees.

They can afford it. They can even afford to get it done [very] quickly.



I'm not so sure that they can "afford it".  Revenue and profit are two different things and you're talking about a massive investment in infrastructure which has to be funded upfront.  While I agree that they should have been looking at ways to fund this investment for a long time now, I don't think that it can be assumed they have enough unused capital sitting around to fund it themselves without borrowing/outside investment.


Title: Re: mtgox under Distributed Denial of Competence attack
Post by: aantonop on April 12, 2013, 02:06:49 AM
Getting capital funding for hardware, on proven revenues and growth is about the easiest funding pitch in the world.

So what you're saying is that they CEO is as incompetent as their COO?

ok. I agree.


Title: Re: mtgox under Distributed Denial of Competence attack
Post by: shkiser on April 12, 2013, 02:51:40 AM

How exactly would you scale out any architecture having to quadruple its capacity within less than a month? Including the relevant human resources, not just machinery.


Is that a serious question? Because if it is, then it is the question mtgox did NOT ask of any serious professional.

You see, I *CAN* answer the question: How do you plan for 10x, 100x, 1000x scalability. That's what a systems architect for financial trading systems does.

It is a very answerable question. The only question is how much money can you spend and how quickly can you hire experts. They've had 2 years and had not anticipated 4x growth? Seriously? A proper capacity plan for something like this would anticipate and accomodate 100x growth not 4x.

Business do this stuff ALL THE TIME. I know, I do this kind of data center work for large companies.

There is no excuse. It's plain incompetence

Incompetence, or exploitation?


Title: Re: mtgox under Distributed Denial of Competence attack
Post by: aantonop on April 12, 2013, 02:53:12 AM
Incompetence.

MTGOX could be making transaction fees on billions LEGALLY, all they have to do is not fuck up badly. Yet, they keep fucking up badly.

Totally not worth playing the market manipulation game when your reputation and opportunity for a bigger pie is on the line.

It's not manipulation, they're not smart or capable enough for that.