MtGox blames Bitcoin protocol problem for BTC withdrawal issue

[fairglu]

its enough when mtgox does the same as the bitcoind wallet: check for outputs and dont rely on txid alone.

Once the volume is large enough, that's exploitable as well if the exchange doesn't spam the blockchain with unique withdrawal addresses (since you'll get or can manufacture duplicates).

And if TX ID are useless for identifying transaction, improve the protocol by dropping them from the blockchain, that'll help cut down on the bloat, you can always locate a transaction by block height + tx index.

you really want a hardfork because one company did fuck up their custom client?

Seriously? If forking for improvements has become such a scary concept to bitcoiners, then that tech is fast turning into a train wreck waiting to happen.

[DeathAndTaxes]

This.  Other exchanges, merchants, and service providers have dealt with this for years.

Urgh, actually I think most of them don't.

All services that provide you with an unconfirmed transaction ID right at withdrawal times are likely to be vulnerable from the same MtGox issue in one form or another.

IMHO they were just too puny for anyone to have bothered, we'll see sinkers and floaters in the next weeks.

Not really.

The first thing to point out is that if you are competent very few of your txs should ever end up dropped from the network.  MtGox had thousands of them.  They had not one problem but at least 4
a) double spending previously spent coins = invalid tx so nodes dropped it.
b) spending immature coins = invalid tx so nodes dropped it (although these would eventually get through after 120 blocks).
c) paying insufficient fees for relay (not following the fee rules in the reference client so nodes running on those rules would drop the tx instead of relaying them).
d) using non canonical signing two years after the issue was reported and modified in the reference client.

This changed what normally is an incredibly rare event (tx not propagating the network) to one that was widespread and affecting innocent users.  Had this giant cluster fuck of layered incompetency not happened the actions of any scammer would be more obvious.  If you only have a handful of users reporting they didn't get paid it is a little more obvious when 20% to 30% of all your withdraws across all users are failing.

A small exchange doesn't need an automated system to deal with tx mutability.  We don't. That is because 99.9%+ of the tx will never be mutated and there are no false positives.  There is nothing wrong with using the tx id as long as you don't rely on it as your first, last, and only method of accounting.   Mutability doesn't happen by accident.  

IF a user reports they didn't get paid and IF the tx hash is not in the blockchain then you MANUALLY flag the account and look for a tx which has the same inputs and outputs.

If you don't have that laundry list of problems above going on this should be a VERY VERY RARE EVENT and one that likely warrants manual attention anyways.  Sure a super duper automated system would be great and if you have the funds to build, extensively test, and deploy it go ahead but this isn't a routine occurrence for a properly running node.  So if your backend can process 99.9% of tx in an automated fashion and flags 1 in 1000 (and honestly probably more like 1 in 100,000) for manual review well guess what?  That is called doing business.  Most automated systems have some event which are logged for manual review.

The other thing to keep in mind is that most likely, the only reason it happened is because the user who reports he "didn't get paid", modified the tx intentionally so he can try to collect a second payment.  That means putting a man in the loop is a good idea even if you do have a super duper automated handle every edge case back end processor.  

[il--ya]

>Now for 99.999% of your transactions (if you are competent and not doing things like double spending your own coins, paying insufficient fees on masive spammy txs, and spending immature coins) the tx will go through fine.


...and signign with non-canonical signatures, after it's a known issue for two years

[af_newbie]

b) check the blockchain to see if the tx exists by looking up the unique set of inputs and outputs which make the tx.

Even better.  Someone should send them the above line.  

[flower1024]

Once the volume is large enough, that's exploitable as well if the exchange doesn't spam the blockchain with unique withdrawal addresses (since you'll get or can manufacture duplicates).

outputs != withdraw address
its very easy to do and bitcoind does it. just mtgox thought txid is ok: and i expect a major exchange which custom software to follow bitcoin changes closely: so they should have known it before

[fairglu]

Technically you don't even need to do that because the client/wallet already does it but for customer support it likely is easier to have that information available outside of the wallet.

The wallet solves different problems, it's like a piggy bank, it only cares about what goes in and out, with limited attribution capability.

In the grand scheme of things, exchanges should not be using the wallet, just like stock exchanges aren't run with Excel spreadsheets and VB macros.

[DeathAndTaxes]

Solution: do not re-use addresses. Send all change to another address, and track spends on the blockchain using the sending address...  No chance of duplicated tracking data (the sending address basically becomes your tx hash).  voila. No problem here other then MtGox incompetance.  Demanding a change to the core protocol is ridiculous.

You mean spam the solution is to spam the blockchain with transactions to credit the one-time addresses used only for tracking purposes?

Please learn how Bitcoin works.

All txs have change (except in the edge case that the output exactly matches the inputs).  Sending the change back to the same address, or sending it to a new address doesn't have any effect on the size of the blockchain or UXTO.

[flower1024]

Technically you don't even need to do that because the client/wallet already does it but for customer support it likely is easier to have that information available outside of the wallet.

The wallet solves different problems, it's like a piggy bank, it only cares about what goes in and out, with limited attribution capability.

In the grand scheme of things, exchanges should not be using the wallet, just like stock exchanges aren't run with Excel spreadsheets and VB macros.

but you agree that if an exchange builds there own wallet software they are responsible for it?

i repeat: it is easy to check. just dont use txid for comparison instead use input and outputs (and i dont mean addresses).
it is public knowledge: every wallet developer should closely follow bitcoin changes and bugs. if mtgox had done that they could have avoided this desaster

[fairglu]

A small exchange doesn't need an automated system to deal with tx mutability.  We don't.  That is because 99.9%+ of the tx will never be mutated.

We'll see who sinks and floats as other exchanges pick up the slack and grow.

I wish you and them all well, but I would be surprised if there aren't other noteworthy casualties.

And anyway, this is a good opportunity to improve things, assuming the devs act on it. Otherwise, it'll just generate bad press.

[DeathAndTaxes]

And anyway, this is a good opportunity to improve things, assuming the devs act on it. Otherwise, it'll just generate bad press.

MtGox will continue to generate bad press as long as they remain incompetent and then lie with inflammatory language in order to obfuscate that fact.  Either they will become competent, the exchange will get sold to people who are, or they will go out of business.  There is nothing for the developers to change at this point.  The protocol isn't going to be hard forked due to the incompetence of one exchange.  The reference client handles this fine.

If you make a payment with the reference client and the tx ends up in the blockchain but with a mutated tx hash what do you think the reference client does?
a) continue to report the tx as unconfirmed so users will think they need to make a second payment
OR
b) detect the modified version of the tx (when it scans blocks for new txs anyways) and modify the client's records so the tx shows the proper confirmations and the new modified tx hash?

Hint the answer is B

In other words the reference client handles this issue properly.  MtGox hacked together code does not (along with at least 4 other errors in processing txs).  MtGox also tries to spend immature newly mined coins.  That is another obviously wrong thing to do, No other client does that, The protocol doesn't allow that.  Should the protocol also be changed to allow the spending of immature coins so MtGox broken client will be not as broken?   This isn't a protocol issue, it is a client issue.  The client should handle this condition.  Most clients do, the "Gox Custom v0" ended up goxxing the exchange because it doesn't.

If the HTML 5 standard has a tag <x> which requires browsers to do X and they all do X except one browser lets say Internet Explorer does Y what is the solution?
a) update the standard and force all other compliant browsers to do Y when they encounter and <x>.
OR
b) demand the broken browser update their implementation so that when it encounters an <x> tag it does X not Y.

[fairglu]

but you agree that if an exchange builds there own wallet software they are responsible for it?

i repeat: it is easy to check. just dont use txid for comparison instead use input and outputs (and i dont mean addresses).
it is public knowledge: every wallet developer should closely follow bitcoin changes and bugs. if mtgox had done that they could have avoided this desaster

Sure, they are responsible for it, but that doesn't absolve the Bitcoin devs from working out better solution, nor does it absolve the bitcoin community to all pile it on MtGox.

This is an image issue there, look at the outside perception:

• the obvious well defined transaction ID (a basic accounting principle) isn't guaranteed by bitcoin
• bunch of geeks says MtGox was wrong because of obscure incomprehensible reasons for most outsiders
• the biggest exchange obviously got it wrong on those incomprehensible reasons
• bitcoin devs resist change, giving an image of denial
• bitcoin community runs afraid screaming "hard fork" whenever a protocol change is mentioned

So that just gives an image of a fossilized complex tech that even large players get wrong.

Just won't build much mainstream confidence, will it?

[justusranvier]

Sure, they are responsible for it, but that doesn't absolve the Bitcoin devs from working out better solution, nor does it absolve the bitcoin community to all pile it on MtGox.

This is an image issue there, look at the outside perception:

• the obvious well defined transaction ID (a basic accounting principle) isn't guaranteed by bitcoin
• bunch of geeks says MtGox was wrong because of obscure incomprehensible reasons for most outsiders
• the biggest exchange obviously got it wrong on those incomprehensible reasons
• bitcoin devs resist change, giving an image of denial
• bitcoin community runs afraid screaming "hard fork" whenever a protocol change is mentioned

So that just gives an image of a fossilized complex tech that even large players get wrong.

Just won't build much mainstream confidence, will it?

There is no image problem, just altcoin pumpers who grasp at any straw to keep their exchange rates pumped up just a little bit longer. Same shit that's been going on for three years now.

[fairglu]

If the HTML 5 standard has a tag <x> which requires browsers to do X and they all do X except one browser lets say Internet Explorer does Y what is the solution?
a) update the standard and force all other compliant browsers to do Y when they encounter and <x>.
OR
b) demand the broken browser update their implementation so that when it encounters an <x> tag it does X not Y.

If the most widespread implementation does it one way, then it becomes standard. Happened with IE bugs, happens these days with Safari and Chrome bugs.

Public does NOT GIVE A DAMN about geeky correctness. If a page fails on FireFox but works with IE, users will accuse FireFox/Chrome and stick with IE. They did. For years. So FireFox and Chrome supported Quirks mode, and still do.

If MtGox fails because of bitcoin technical reasons, well, bitcoin gets bad press.

Either this is taken as an opportunity, or it will be taken as a blow, that's all.

[af_newbie]

If MtGox fails because of bitcoin technical reasons, well, bitcoin gets bad press.

I think in this case, MtGox got bad press in more ways than one.  Bitcoin is fine.

They tried to blame bitcoin, didn't work.  Move along.

[cr1776]

If the HTML 5 standard has a tag <x> which requires browsers to do X and they all do X except one browser lets say Internet Explorer does Y what is the solution?
a) update the standard and force all other compliant browsers to do Y when they encounter and <x>.
OR
b) demand the broken browser update their implementation so that when it encounters an <x> tag it does X not Y.

If the most widespread implementation does it one way, then it becomes standard. Happened with IE bugs, happens these days with Safari and Chrome bugs.

Public does NOT GIVE A DAMN about geeky correctness. If a page fails on FireFox but works with IE, users will accuse FireFox/Chrome and stick with IE. They did. For years. So FireFox and Chrome supported Quirks mode, and still do.

If MtGox fails because of bitcoin technical reasons, well, bitcoin gets bad press.

Either this is taken as an opportunity, or it will be taken as a blow, that's all.

Thankfully MtGox is not anywhere near the "most widespread implementation" (and is dropping wildly by the second), and so it is not the "standard". So MtGox is wrong. The opportunity here is for Gox to either implode or fix their sh!t.  In all likelihood, given their issues and inability to fix repeated problems for a year now, it will be for former.  

MtGox is on its way to being an extremely minor player in the exchange business. If MtGox comes out of this with anything but a minuscule percentage of the exchange business (if ANY), I'll be surprised.

[DeathAndTaxes]

If the most widespread implementation does it one way, then it becomes standard.

Ok then the standard is that tx hashes are mutable and clients need to account for that.  Glad you agree. 

[Zzzack]

I was wondering something about this issue.. Would a coin with a 30 minute block interval be more susceptible to this problem?

[killerstorm]

b) check the blockchain to see if the tx exists by looking up the unique set of inputs and outputs which make the tx.

It makes sense to find all transactions which spend same coins.

1. No such transaction -> txn was not included into a block
2. One such transaction -> that's what we're looking for, now check its outputs
3. More than one transaction -> bug in software, investigate

Unlike what you described, this check can be done with simple tools like block explorer (or, automatically).

E.g. suppose software says that transaction in question spends b8bdf71e9e3ee414fa57144cd971fa665fe07aacf1b9203cac485859000d3895:1

Then you look it up: https://blockchain.info/tx/b8bdf71e9e3ee414fa57144cd971fa665fe07aacf1b9203cac485859000d3895/1

Red link (Spent) points to a transaction which spends this output, which is the transaction we are looking for.

(I simply want to clarify that it is better to look up inputs one by one than look up "the unique set of inputs" (whatever it means)).

[killerstorm]

And if TX ID are useless for identifying transaction, improve the protocol by dropping them from the blockchain, that'll help cut down on the bloat, you can always locate a transaction by block height + tx index.

ITT: People who have only a very vague idea about how Bitcoin works offer solutions and propose protocol changes.

Seriously, if you haven't read the source code, SHUT THE FUCK UP. You're simply not competent to say anything on this matter.

This mess was caused by people who weren't competent enough, do not make it worse by making strange noises.

(What you wrote above is ridiculous, you have no idea what is "TX ID" and how it is used, how it is included into blockchain etc.)

[DeathAndTaxes]

I was wondering something about this issue.. Would a coin with a 30 minute block interval be more susceptible to this problem?

No.  This has nothing to do with the block interval.