MtGox blames Bitcoin protocol problem for BTC withdrawal issue

[mb300sd]

My proposal for a very simple fix: track change addresses instead of txid.

https://bitcointalk.org/index.php?topic=458668.0

[mightycount]

It is not even clear that the malleability problem is solvable. Besides the problem that there is innumerable ways to transform a transaction to give it a different hash without affecting scriptSig validity, it is simply unknown whether it is possible to algebraically transform an elliptic curve signature without invalidating it. If so, then no matter what you do to cover up the other holes, that gaping one is left open.

Transactions are malleable. Deal with it. If a transaction is observed on the network that has the same input outpoints and the same outputs, it is the same transaction, and mtgox should treat it as such. This is a simple check to do, and trivial to automate.

MtGox's problem is not "malleability", it's the fact that they didn't submit transactions to the network, at the same time making internal hashes public. The answer to Mr. Karpiles from the core development team should be clear and resounding "NO". Bitcoin is fine as is and MtGox should pull their $#!+ together. I could not believe the idiot dared to blame the Bitcoin protocol.

[flower1024]

I could not believe the idiot dared to blame the Bitcoin protocol.

i hope i am wrong.
but i begin to think that they lost so much btc to this issue that they need to buy coins.

[wolongong]

It is easy to check whether a similar mutated transaction got on the chain or not.

Surely, with out even needing to modify the bitcoin client or protocol an easy solution would have been to monitor the inputs of a transaction when a user withdraws.

i repeat: it is easy to check. just dont use txid for comparison instead use input and outputs (and i dont mean addresses).

b) detect the modified version of the tx (when it scans blocks for new txs anyways) and modify the client's records so the tx shows the proper confirmations and the new modified tx hash?

Someone please show me how to do this easy check with a few bitcoind commands?

[flower1024]

Someone please show me how to do this easy check with a few bitcoind commands?

when a new block arrives get it with rpc and look through the json output?

[ZephramC]

If the most widespread implementation does it one way, then it becomes standard. Happened with IE bugs, happens these days with Safari and Chrome bugs.

Public does NOT GIVE A DAMN about geeky correctness. If a page fails on FireFox but works with IE, users will accuse FireFox/Chrome and stick with IE. They did. For years. So FireFox and Chrome supported Quirks mode, and still do.

If MtGox fails because of bitcoin technical reasons, well, bitcoin gets bad press.

Either this is taken as an opportunity, or it will be taken as a blow, that's all.

Well, then the public deserves to loose their money. Any system that would shield them from this is a bad system. Doesn't matter how widespread is the non-complaint solution.

[wolongong]

Someone please show me how to do this easy check with a few bitcoind commands?

when a new block arrives get it with rpc and look through the json output?

Sure, poll getblockcount, then what? Please show how easy it is to spot this.

[flower1024]

Someone please show me how to do this easy check with a few bitcoind commands?

when a new block arrives get it with rpc and look through the json output?

Sure, poll getblockcount, then what? Please show how easy it is to spot this.

Code:

$ bitcoind getblock 000000000000010c5ba86d05c6f43df46921d453d23fdafe85229f6a7d840e16 '{"tx":"obj", "script":"asm"}'
{
    "hash" : "000000000000010c5ba86d05c6f43df46921d453d23fdafe85229f6a7d840e16",
    "confirmations" : 43,
    "size" : 1473,
    "height" : 186154,
    "version" : 1,
    "merkleroot" : "47de80c28529a69bddefb90ea15388f10e2647482560bb50857e7b586d45a8c3",
    "time" : 1340614357,
    "nonce" : 728350342,
    "bits" : "1a09b78a",
    "difficulty" : 1726566.55919348,
    "tx" : [
        {
            "txid" : "11af9ba807942f7db6274f40442651c7b58b02b0a942867847b2b8a3932df7b1",
            "version" : 1,
            "locktime" : 0,
            "size" : 142,
            "vin" : [
                {
                    "coinbase" : "048ab7091a027702062f503253482f",
                    "sequence" : 4294967295
                }
            ],
            "vout" : [
                {
                    "value" : 50.00000000,
                    "scriptPubKey" : "04b034ecbcbe86f327d2edd25ce1fce3f4463710c00ae00d1a25198bab882be95c72d3468dc6931f756b03d99a3b7f3a07543834370ad023b37cb1127c6266d91a OP_CHECKSIG"
                }
            ],
            "confirmations" : 0
        },
        {
            "txid" : "9fb85b2c9085284a6c39407025cbec31b4c88b1075f3d98840395bc4e389616d",
            "version" : 1,
            "locktime" : 0,
            "size" : 259,
            "vin" : [
                {
                    "prevout" : {
                        "hash" : "1f16f657fb093f4618c1ad6504130d08fa71bb1964d0be5574f44011124f7b64",
                        "n" : 1
                    },
                    "scriptSig" : "3046022100dea86a40b8ffcfa446e05886e18c7ea8bca371118da48d4adfde846b87af43190221009652d86f7013afd35311fe8d080c9e592758716c08e971c39283ca08746db78501 04a1ce79a9fec1018129bb79c19dc90dcb4657e8cf725c635909ea402143b64e84fb06bf0b2c96ca6f0db9ac8ae480399f7724a9b288faf52c16d1535f13b90fe6",
                    "sequence" : 4294967295
                }
            ],
            "vout" : [
                {
                    "value" : 1066.13121507,
                    "scriptPubKey" : "OP_DUP OP_HASH160 e51a349be7044a8d84c6e8370ff56ff6860f3597 OP_EQUALVERIFY OP_CHECKSIG"
                },
                {
                    "value" : 1242.21306062,
                    "scriptPubKey" : "OP_DUP OP_HASH160 51fb66b01bcdbcdc67049fa97a8da21a2fa912e0 OP_EQUALVERIFY OP_CHECKSIG"
                }
            ],
            "confirmations" : 0
        },
        {
            "txid" : "060cbd899b2c1fac0cdbcf580a3f90aae131b16d516f049baf1d13c66e410892",
            "version" : 1,
            "locktime" : 0,
            "size" : 226,
            "vin" : [
                {
                    "prevout" : {
                        "hash" : "f2b6efbef50c026efc19dac73ed787b0ccc1697c8640e96637ab861e85afca60",
                        "n" : 0
                    },
                    "scriptSig" : "3045022100a0be436cb85b899740792206a17658ef96e6d786e53448cab8b11f9e51be96a6022063ab6d9337fa71b9d5a75e3ebe0a33c151ad6cdca70f9170c1b0297d655c5b3301 022cb39adb02162270b89b08975256293182bcaa84622d18a30c3bd72751d4ddd5",
                    "sequence" : 4294967295
                }
            ],
            "vout" : [
                {
                    "value" : 1.40018289,
                    "scriptPubKey" : "OP_DUP OP_HASH160 d6211f74851b1cc1d181d0fe591445a18f49c818 OP_EQUALVERIFY OP_CHECKSIG"
                },
                {
                    "value" : 1.00000000,
                    "scriptPubKey" : "OP_DUP OP_HASH160 713c581950bbfb0545f1418bbfe825911e8efa5c OP_EQUALVERIFY OP_CHECKSIG"
                }
            ],
            "confirmations" : 0
        },
        {
            "txid" : "2da6e2a83abe9e870a3a4762d8159e523a3068cce26c5670afdcd2c63486a1ac",
            "version" : 1,
            "locktime" : 0,
            "size" : 765,
            "vin" : [
                {
                    "prevout" : {
                        "hash" : "9aa47d859bda3eba62fd366461adda78f6df3293dd7b1681361f54531220b378",
                        "n" : 2
                    },
                    "scriptSig" : "3046022100f10d0f6fc6ce90ae97a04627a659791b1caed48aae8d70b82c73c5c730758576022100aba76d54166189b50e678faac63a4265d6d2daf48889a23a1fc285e042fb9c8901 04addfdfd09989768bc93678fd4eca1bf8d485e98690b7584a7559ab30a63ac16fd33e8e7db59f72c21b402a78bacbde205d642e60037d8b660a57341a60c4e646",
                    "sequence" : 4294967295
                },
                {
                    "prevout" : {
                        "hash" : "125ea4a6f9e86333de7566af28e6a5558a2c66e5c41b197560bd9a8e6c56866b",
                        "n" : 0
                    },
                    "scriptSig" : "3045022100c9e3d69de5bcbb9252b2b637e6846eaef971959653c887e7bb13f6528f1587f2022059d47b80f98460c0a02a9f7a7977202533fe3566a1cbecc599b122af02a8b8ac01 04a875f1a901e23be1435350f1a94820f84f590e297c0a34e393b873ac5b3182d071ca43890a54c750ad2fd6288416a319e70e48f2c2d2fe828666cfc98689ce0e",
                    "sequence" : 4294967295
                },
                {
                    "prevout" : {
                        "hash" : "e2b9a62cd1ad312f510510bf071ae0079b6b858db8b43e86809b7269d8ad1ff7",
                        "n" : 1
                    },
                    "scriptSig" : "3045022100a198dea14374a5d0584cfedeb8f6a457556b9bdd0488c433b179ad328a007c7102200a3179f1b8a95ae4e02183c25fb3234f1efe0db9129806d38579b2845e77a11801 047b19a9c3e6b0a92e9bdeeb294ebea709dd1d9b5e2ccb7e1fb03ca6bfa43b78dd0a6d72ddd9e64130513d63dc1ea5609906a15f1d3044f50ee6415b2e4eaf59e4",
                    "sequence" : 4294967295
                },
                {
                    "prevout" : {
                        "hash" : "b6548149a182fd10330b9f69024bdd55ce521ce367e9125d1bd8c91a8c5539da",
                        "n" : 0
                    },
                    "scriptSig" : "30450220670628b426d996131f064bb861554e31bd3afcdc581ae9824af1610dcc079080022100d240e2f445670f8e2be82f5fc74dfbc98312f0adb5f2ec163449588bdd8e9f8401 045c7e935730d0a7e45615e4a5df7c50b8bdde7fad7c978c569563b5451a756710d7dc0f1a248bd233467f17dd28e27484a91c0af4123c131b87d959dd37850692",
                    "sequence" : 4294967295
                }
            ],
            "vout" : [
                {
                    "value" : 0.19950871,
                    "scriptPubKey" : "OP_DUP OP_HASH160 c6b92e0b69d3a7aa53a28e2bc5a6de3f32fbff5a OP_EQUALVERIFY OP_CHECKSIG"
                }
            ],
            "confirmations" : 0
        }
    ],
    "previousblockhash" : "00000000000006d68b5d22acef2a43b38dc692782d7c12c0ae43259b9eb46078",
    "nextblockhash" : "000000000000023b6a46f950ef1ccd08519c2e26208d1af157c6e7fa541bd043"
}

just loop through vin and check your already sent transaction for dups.
anyway: i dont have a bitcoin node ready to check and i just copied it from the forum.
for anybody developing custom client software this should be easy, dont you think?

[killerstorm]

Sure, poll getblockcount, then what? Please show how easy it is to spot this.

Making custom Bitcoin clients, in general, isn't easy. It should be avoided unless it is absolutely necessary.

If you're asking a question like this, then you're simply not qualified to do this.

[mightycount]

Here is the link to the thread, where the patch addressing malleability was discussed:
https://bitcointalk.org/index.php?topic=8392.0

The whole MtGox goxing was only possible because:
1) They were issuing transactions with sloppy signature format
2) This was accepted by the network for some time
3) New bitcoin client with tightened rules was released
4) Their sloppy transactions started being rejected
5) Exploiters "fixed" those transactions
6) MtGox sloppy software didn't notice transaction went through
7) Mt-gox sloppy software didn't notice output's were spent (making them unaware that they lose coins). It didn't even lock outputs which are used in pending transactions!!
MtGox incompetent customer support resubmitted transactions manually without looking into issue and alarming developers. Or maybe they did, but developers were too busy/confident/not able to fix the problem.

Overall: MtGox are incompetent bunch of liers

To be fair, the "workaround" implemented by Gox following a core dev recommendation, was horrible. Staging transactions is much worse than the initial "bad sig" problem, IMHO.

[wolongong]

Someone please show me how to do this easy check with a few bitcoind commands?

when a new block arrives get it with rpc and look through the json output?

Sure, poll getblockcount, then what? Please show how easy it is to spot this.

Code:

$ bitcoind getblock 000000000000010c5ba86d05c6f43df46921d453d23fdafe85229f6a7d840e16 '{"tx":"obj", "script":"asm"}'
{
    "hash" : "000000000000010c5ba86d05c6f43df46921d453d23fdafe85229f6a7d840e16",
    "confirmations" : 43,
    "size" : 1473,
    "height" : 186154,
    "version" : 1,
    "merkleroot" : "47de80c28529a69bddefb90ea15388f10e2647482560bb50857e7b586d45a8c3",
    "time" : 1340614357,
    "nonce" : 728350342,
    "bits" : "1a09b78a",
    "difficulty" : 1726566.55919348,
    "tx" : [
        {
            "txid" : "11af9ba807942f7db6274f40442651c7b58b02b0a942867847b2b8a3932df7b1",
            "version" : 1,
            "locktime" : 0,
            "size" : 142,
            "vin" : [
                {
                    "coinbase" : "048ab7091a027702062f503253482f",
                    "sequence" : 4294967295
                }
            ],
            "vout" : [
                {
                    "value" : 50.00000000,
                    "scriptPubKey" : "04b034ecbcbe86f327d2edd25ce1fce3f4463710c00ae00d1a25198bab882be95c72d3468dc6931f756b03d99a3b7f3a07543834370ad023b37cb1127c6266d91a OP_CHECKSIG"
                }
            ],
            "confirmations" : 0
        },
        {
            "txid" : "9fb85b2c9085284a6c39407025cbec31b4c88b1075f3d98840395bc4e389616d",
            "version" : 1,
            "locktime" : 0,
            "size" : 259,
            "vin" : [
                {
                    "prevout" : {
                        "hash" : "1f16f657fb093f4618c1ad6504130d08fa71bb1964d0be5574f44011124f7b64",
                        "n" : 1
                    },
                    "scriptSig" : "3046022100dea86a40b8ffcfa446e05886e18c7ea8bca371118da48d4adfde846b87af43190221009652d86f7013afd35311fe8d080c9e592758716c08e971c39283ca08746db78501 04a1ce79a9fec1018129bb79c19dc90dcb4657e8cf725c635909ea402143b64e84fb06bf0b2c96ca6f0db9ac8ae480399f7724a9b288faf52c16d1535f13b90fe6",
                    "sequence" : 4294967295
                }
            ],
            "vout" : [
                {
                    "value" : 1066.13121507,
                    "scriptPubKey" : "OP_DUP OP_HASH160 e51a349be7044a8d84c6e8370ff56ff6860f3597 OP_EQUALVERIFY OP_CHECKSIG"
                },
                {
                    "value" : 1242.21306062,
                    "scriptPubKey" : "OP_DUP OP_HASH160 51fb66b01bcdbcdc67049fa97a8da21a2fa912e0 OP_EQUALVERIFY OP_CHECKSIG"
                }
            ],
            "confirmations" : 0
        },
        {
            "txid" : "060cbd899b2c1fac0cdbcf580a3f90aae131b16d516f049baf1d13c66e410892",
            "version" : 1,
            "locktime" : 0,
            "size" : 226,
            "vin" : [
                {
                    "prevout" : {
                        "hash" : "f2b6efbef50c026efc19dac73ed787b0ccc1697c8640e96637ab861e85afca60",
                        "n" : 0
                    },
                    "scriptSig" : "3045022100a0be436cb85b899740792206a17658ef96e6d786e53448cab8b11f9e51be96a6022063ab6d9337fa71b9d5a75e3ebe0a33c151ad6cdca70f9170c1b0297d655c5b3301 022cb39adb02162270b89b08975256293182bcaa84622d18a30c3bd72751d4ddd5",
                    "sequence" : 4294967295
                }
            ],
            "vout" : [
                {
                    "value" : 1.40018289,
                    "scriptPubKey" : "OP_DUP OP_HASH160 d6211f74851b1cc1d181d0fe591445a18f49c818 OP_EQUALVERIFY OP_CHECKSIG"
                },
                {
                    "value" : 1.00000000,
                    "scriptPubKey" : "OP_DUP OP_HASH160 713c581950bbfb0545f1418bbfe825911e8efa5c OP_EQUALVERIFY OP_CHECKSIG"
                }
            ],
            "confirmations" : 0
        },
        {
            "txid" : "2da6e2a83abe9e870a3a4762d8159e523a3068cce26c5670afdcd2c63486a1ac",
            "version" : 1,
            "locktime" : 0,
            "size" : 765,
            "vin" : [
                {
                    "prevout" : {
                        "hash" : "9aa47d859bda3eba62fd366461adda78f6df3293dd7b1681361f54531220b378",
                        "n" : 2
                    },
                    "scriptSig" : "3046022100f10d0f6fc6ce90ae97a04627a659791b1caed48aae8d70b82c73c5c730758576022100aba76d54166189b50e678faac63a4265d6d2daf48889a23a1fc285e042fb9c8901 04addfdfd09989768bc93678fd4eca1bf8d485e98690b7584a7559ab30a63ac16fd33e8e7db59f72c21b402a78bacbde205d642e60037d8b660a57341a60c4e646",
                    "sequence" : 4294967295
                },
                {
                    "prevout" : {
                        "hash" : "125ea4a6f9e86333de7566af28e6a5558a2c66e5c41b197560bd9a8e6c56866b",
                        "n" : 0
                    },
                    "scriptSig" : "3045022100c9e3d69de5bcbb9252b2b637e6846eaef971959653c887e7bb13f6528f1587f2022059d47b80f98460c0a02a9f7a7977202533fe3566a1cbecc599b122af02a8b8ac01 04a875f1a901e23be1435350f1a94820f84f590e297c0a34e393b873ac5b3182d071ca43890a54c750ad2fd6288416a319e70e48f2c2d2fe828666cfc98689ce0e",
                    "sequence" : 4294967295
                },
                {
                    "prevout" : {
                        "hash" : "e2b9a62cd1ad312f510510bf071ae0079b6b858db8b43e86809b7269d8ad1ff7",
                        "n" : 1
                    },
                    "scriptSig" : "3045022100a198dea14374a5d0584cfedeb8f6a457556b9bdd0488c433b179ad328a007c7102200a3179f1b8a95ae4e02183c25fb3234f1efe0db9129806d38579b2845e77a11801 047b19a9c3e6b0a92e9bdeeb294ebea709dd1d9b5e2ccb7e1fb03ca6bfa43b78dd0a6d72ddd9e64130513d63dc1ea5609906a15f1d3044f50ee6415b2e4eaf59e4",
                    "sequence" : 4294967295
                },
                {
                    "prevout" : {
                        "hash" : "b6548149a182fd10330b9f69024bdd55ce521ce367e9125d1bd8c91a8c5539da",
                        "n" : 0
                    },
                    "scriptSig" : "30450220670628b426d996131f064bb861554e31bd3afcdc581ae9824af1610dcc079080022100d240e2f445670f8e2be82f5fc74dfbc98312f0adb5f2ec163449588bdd8e9f8401 045c7e935730d0a7e45615e4a5df7c50b8bdde7fad7c978c569563b5451a756710d7dc0f1a248bd233467f17dd28e27484a91c0af4123c131b87d959dd37850692",
                    "sequence" : 4294967295
                }
            ],
            "vout" : [
                {
                    "value" : 0.19950871,
                    "scriptPubKey" : "OP_DUP OP_HASH160 c6b92e0b69d3a7aa53a28e2bc5a6de3f32fbff5a OP_EQUALVERIFY OP_CHECKSIG"
                }
            ],
            "confirmations" : 0
        }
    ],
    "previousblockhash" : "00000000000006d68b5d22acef2a43b38dc692782d7c12c0ae43259b9eb46078",
    "nextblockhash" : "000000000000023b6a46f950ef1ccd08519c2e26208d1af157c6e7fa541bd043"
}

just loop through vin and check your already sent transaction for dups.
anyway: i dont have a bitcoin node ready to check and i just copied it from the forum.
for anybody developing custom client software this should be easy, dont you think?

Parsing whatever bitcoind spits out is easy, yes, custom bitcoin clients are not or github would be full of them. All that's left from my sendtoaddress is a txid, please, someone claiming how easy this is show me how to spot this.

[DeathAndTaxes]

That is the point.  Custom bitcoin clients are NOT easy.  Unless you know what you are doing you probably should NOT be making a custom bitcoin client.  

As much money as MtGox has made over the last four years they could have hired someone who knew what they were doing but install hack together a client which is non compliant with the reference implementation in at least half a dozen areas (who knows how many others as the source code isn't public so we can only observe the fail from the outside).  This is the same MtGox which has no development server and just makes changes to production live so they decided getting the implementation of a mission critical component which involves the transfer of money, "close enough" was an acceptable standard.

A compliant client should not:
a) violate spam rules
b) pay less than the min mandatory fee (well a business client where users expect timely payments shouldn't)
c) send immature (less than 120 confirmations) newly generated coins
d) double spend its own "coins" (use outputs which were already spent in a prior tx)
e) use non-canonical signatures
f) rely only on tx-id as absolute proof on if a payment has been made or not

The "Gox Special" client did all of those, and who know what else.  It would be like building a jet airline by copying a photo of a jet you once saw, and being surprised when someone dies because "well it looks kinda like a jet, the engines are pointing the right way, and it has some wings and stuff".  Who possibly could imagine that slapping together some random parts so it looks like a jet wouldn't actually be a functional, safe jet?

Like cryptography, if you "roll your own" bitcoin node, you have to get it EXACTLY RIGHT.  Not kinda right, not mostly right, not hey it does sometimes push tx the network doesn't reject but absolutely right down to every single detail.   Most entities (companies and individuals) should not attempt that.  They should build off existing proven clients.

[justusranvier]

As much money as MtGox has made over the last four years they could have hired someone who knew what they were doing

There are companies who do know what they are doing offering to send a team of coders to Japan to fix their problems free of charge, just because of how much damage Mt Gox is doing right now.

[wolongong]

That is the point.  Custom bitcoin clients are NOT easy.  Unless you know what you are doing you probably should NOT be making a custom bitcoin client.

There is nothing custom about doing bitcoind sendtoaddress and getting a txid back.

f) rely only on tx-id as absolute proof on if a payment has been made or not
[snip]
They should build off existing proven clients.

So, then how do I easily spot this with a reasonably recent standard client?

[wolongong]

MtGox's problem is not "malleability", it's the fact that they didn't submit transactions to the network, at the same time making internal hashes public. The answer to Mr. Karpiles from the core development team should be clear and resounding "NO". Bitcoin is fine as is and MtGox should pull their $#!+ together. I could not believe the idiot dared to blame the Bitcoin protocol.

Bitcoin doesn't have to change one bit to come up with a standardised way of making these kind of hashes. I prefer we all come up with one that has a fighting chance to get included in the bitcoin client (not chain!) than every cat and dog inventing their own transaction numbers.

[DeathAndTaxes]

So, then how do I easily spot this with a reasonably recent standard client?

Using QT client.  0.8.0+ works fine, probably older clients as well.

Code:

getrawtransaciton <txid>

This will give you the raw tx details in hex.
 

Code:

decoderawtransaction <rawtxhex>

This will give you json representation of a tx.

You can do your own parsing of the raw hex or you can feed the hext into decoderawtx.  Either way you now have the details of your payment.

For example (tx pulled randomly from the blockchain)

getrawtransaction da38a5fa60993f96a855265e1b6986d8a13eba983edf52992dab558656f45270

01000000026b7a832578be08083147c5b5734b359b2c04c552d8dda3c0b3eef007f852d5ac01000 0008e4d4700304402202e3cc369b33adca49763d3bd2f91f090b8a86117f67bcdb32d0607a5212f 065a02207bfefc91193c23d8a9b43120541d8e9a40bd06c277f064d5dd5805b45d867f31014d410 0049763964852d711f5b230601a9cb081dd71c07d46b589305ebfa46d3f4f2e2c45f26819efe4c5 b0fa2de5af2d4c77acec8eab9f1dec1b5e70438c2f8b943de68cffffffff57a11702d068380603e e9c71d97a280d4d8ef34c46443034de3095fbbcef7ee8010000008e4d4700304402202ada2194c1 f68a6ed2e7712693c7fde823c75223319cbe38a9818503e00d613002200a34637ce86dacc835603 088b3519fb9c419c77b79b4084a554b55487f803e8e014d410004b13e7ae7f4a7a9942c3a2998c8 e427fec24cf2e3f3d3501a7c35ab951b07daaeddd621e0115d0e6291615fed8b3ef67a5a2fc7971 70db7028710d0cdb0e18cb5ffffffff02f0076606000000001976a91472e57193b8672cd0875adc 9c0264ca6fa15e3e4688acf7641800000000001976a91404157cfd60177f4fa31c7572d7f5fe14a 8f7930d88ac00000000

decoderawtx 01000000026b7a832578be08083147c5b5734b359b2c04c552d8dda3c0b3eef007f852d5ac01000 0008e4d4700304402202e3cc369b33adca49763d3bd2f91f090b8a86117f67bcdb32d0607a5212f 065a02207bfefc91193c23d8a9b43120541d8e9a40bd06c277f064d5dd5805b45d867f31014d410 0049763964852d711f5b230601a9cb081dd71c07d46b589305ebfa46d3f4f2e2c45f26819efe4c5 b0fa2de5af2d4c77acec8eab9f1dec1b5e70438c2f8b943de68cffffffff57a11702d068380603e e9c71d97a280d4d8ef34c46443034de3095fbbcef7ee8010000008e4d4700304402202ada2194c1 f68a6ed2e7712693c7fde823c75223319cbe38a9818503e00d613002200a34637ce86dacc835603 088b3519fb9c419c77b79b4084a554b55487f803e8e014d410004b13e7ae7f4a7a9942c3a2998c8 e427fec24cf2e3f3d3501a7c35ab951b07daaeddd621e0115d0e6291615fed8b3ef67a5a2fc7971 70db7028710d0cdb0e18cb5ffffffff02f0076606000000001976a91472e57193b8672cd0875adc 9c0264ca6fa15e3e4688acf7641800000000001976a91404157cfd60177f4fa31c7572d7f5fe14a 8f7930d88ac00000000

{
"txid" : "da38a5fa60993f96a855265e1b6986d8a13eba983edf52992dab558656f45270",
"version" : 1,
"locktime" : 0,
"vin" : [
{
"txid" : "acd552f807f0eeb3c0a3ddd852c5042c9b354b73b5c547310808be7825837a6b",
"vout" : 1,
"scriptSig" : {
"asm" : "304402202e3cc369b33adca49763d3bd2f91f090b8a86117f67bcdb32d0607a5212f065a02207bf efc91193c23d8a9b43120541d8e9a40bd06c277f064d5dd5805b45d867f3101 049763964852d711f5b230601a9cb081dd71c07d46b589305ebfa46d3f4f2e2c45f26819efe4c5b 0fa2de5af2d4c77acec8eab9f1dec1b5e70438c2f8b943de68c",
"hex" : "4d4700304402202e3cc369b33adca49763d3bd2f91f090b8a86117f67bcdb32d0607a5212f065a0 2207bfefc91193c23d8a9b43120541d8e9a40bd06c277f064d5dd5805b45d867f31014d41000497 63964852d711f5b230601a9cb081dd71c07d46b589305ebfa46d3f4f2e2c45f26819efe4c5b0fa2 de5af2d4c77acec8eab9f1dec1b5e70438c2f8b943de68c"
},
"sequence" : 4294967295
},
{
"txid" : "e87eefbcfb9530de343044464cf38e4d0d287ad9719cee03063868d00217a157",
"vout" : 1,
"scriptSig" : {
"asm" : "304402202ada2194c1f68a6ed2e7712693c7fde823c75223319cbe38a9818503e00d613002200a3 4637ce86dacc835603088b3519fb9c419c77b79b4084a554b55487f803e8e01 04b13e7ae7f4a7a9942c3a2998c8e427fec24cf2e3f3d3501a7c35ab951b07daaeddd621e0115d0 e6291615fed8b3ef67a5a2fc797170db7028710d0cdb0e18cb5",
"hex" : "4d4700304402202ada2194c1f68a6ed2e7712693c7fde823c75223319cbe38a9818503e00d61300 2200a34637ce86dacc835603088b3519fb9c419c77b79b4084a554b55487f803e8e014d410004b1 3e7ae7f4a7a9942c3a2998c8e427fec24cf2e3f3d3501a7c35ab951b07daaeddd621e0115d0e629 1615fed8b3ef67a5a2fc797170db7028710d0cdb0e18cb5"
},
"sequence" : 4294967295
}
],
"vout" : [
{
"value" : 1.07350000,
"n" : 0,
"scriptPubKey" : {
"asm" : "OP_DUP OP_HASH160 72e57193b8672cd0875adc9c0264ca6fa15e3e46 OP_EQUALVERIFY OP_CHECKSIG",
"hex" : "76a91472e57193b8672cd0875adc9c0264ca6fa15e3e4688ac",
"reqSigs" : 1,
"type" : "pubkeyhash",
"addresses" : [
"1BUWttqJ7gNNhqfYF8rjVpYBPj735PVhRQ"
]
}
},
{
"value" : 0.01598711,
"n" : 1,
"scriptPubKey" : {
"asm" : "OP_DUP OP_HASH160 04157cfd60177f4fa31c7572d7f5fe14a8f7930d OP_EQUALVERIFY OP_CHECKSIG",
"hex" : "76a91404157cfd60177f4fa31c7572d7f5fe14a8f7930d88ac",
"reqSigs" : 1,
"type" : "pubkeyhash",
"addresses" : [
"1NbSoCQiHQCuJc7jPn4SuNXu2xrkyvrWs"
]
}
}
]
}

These RPC call(s) can be made at the time the payment is made and stored.  The bolded part is what we are interested.  You can save the full output but this is the minimum information you need.  Lets imagine tx da38a5fa60993f96a855265e1b6986d8a13eba983edf52992dab558656f45270 was a payment from MtGox to a customer.

The payment to the user was made using two inputs:
"txid" : "acd552f807f0eeb3c0a3ddd852c5042c9b354b73b5c547310808be7825837a6b", "vout" : 1
"txid" : "e87eefbcfb9530de343044464cf38e4d0d287ad9719cee03063868d00217a157", "vout" : 1

Note this is all we care about.  The amount of the inputs are not needed.  If these inputs are in are found in a tx to the user in a block regardless of the tx hash then the user has been paid.

So before MtGox pays the user again they need to verify that these outputs don't exist in a tx in any block since the tx was created.

How do to that.  Well here are two options (there likely are more):

Method 1: Scan all tx in all blocks as blocks are received.
Pro: you get realtime status of mutated txs.
Cons: it is probably more resource intensive then it is worth given the low rate that tx should be mutated in normal operation.

When bitcoind reports a block has been received, you pull the block details

Code:

getblock <blockhash>

You can then parse all the raw tx in the block using getrawtransaction and decoderawtransaction.  If any of the tx have the same inputs from your payment tx then the tx is confirmed, update your records to reflect that the hash is changed, and also flag the tx as having been "modified" after being sent.

Personally I wouldn't even do this because in my experience mutated tx are insanely rare under normal conditions but it is an option.

Method 2: Just flag tx which don't confirm.  Don't send repayment until you validate flagged transactions.

At block time simply update tx based on tx hash.  This should get 99.9% of your tx.  Tx normally are not going to be modified.  Note this will never give false positives.  There is no risk in doing this.

Now some tx hashes may be modified but they should be rare so we can handle them in batches at periodic intervals (say every 24 hours).  Tx which don't confirm in some expected period of time (say 60 blocks) are flagged. 

Once a day do a "missing tx" check.
1) generate a list of blocks since the last time the check was run.
2) pull the tx hashes from the blocks.
3) get the raw tx and decode (either using internal process or using the decoderawtx RPC).
4) For each decoded raw tx compare the inputs (txin_hash and txin_index) of the decoded block tx to your list of "missing tx".

If you get a match the tx is not missing.  "Someone" likely the user mutated the hash.  So update your records to reflect the correct tx_hash, block#, and confirmation status.  Like in method #1 I would also flag the tx that it was mutated. 

Matching based on raw tx is slower than tx hash.  Since most tx are going to have same tx hash this is why I prefer to do this as a periodic job.  You can also run this job on a different server so as to not impact the load on the main tx server. 

The advantage of flagging tx (by eithe rmethod) is it provides you the ability to report on how often this is occurring and with which users.  If you have 10,000 tx in a month and only have 20 "missing" txs and of those 20, 18 involve a single user, you might want to drop that user, especially if he keeps reporting to customer support that he isn't "getting paid".  

[phelix]

I wonder if it would be possible to guesstimate the number of coins stolen from gox.

Probably there is somebody monitoring all TXs floating around through the network (blockchain.info). Maybe this data could be used to check how many Gox TXs have been modified. It would be very interesting to see the size of this issue.

[killerstorm]

The payment to the user was made using two inputs:
"txid" : "acd552f807f0eeb3c0a3ddd852c5042c9b354b73b5c547310808be7825837a6b", "vout" : 1
"txid" : "e87eefbcfb9530de343044464cf38e4d0d287ad9719cee03063868d00217a157", "vout" : 1
Note this is all we care about.  The amount of the inputs are not needed.  If these inputs are in are found in a tx to the user in a block regardless of the tx hash then the user has been paid.

One can simply record this information and wait until he gets complaints.

Normally there is no need to do anything. When a complaint is received, knowing inputs he can check whether these inputs are spent, and thus determine if complaint is valid or not. (If it is a valid complaint things become tricky, at that point consultation with specialists might be necessary as bitcoind won't just forget about wallet transactions.)

If you want an automatic sanity checker, the easiest way is to lookup spends of these coins via some kind of a block explorer API, in this case it might be just like a dozen lines of code. If you don't trust external block explorer, run your own ABE instance.

Processing block data like DeathAndTaxes described is also an option, but requires more code

[DeathAndTaxes]

The payment to the user was made using two inputs:
"txid" : "acd552f807f0eeb3c0a3ddd852c5042c9b354b73b5c547310808be7825837a6b", "vout" : 1
"txid" : "e87eefbcfb9530de343044464cf38e4d0d287ad9719cee03063868d00217a157", "vout" : 1
Note this is all we care about.  The amount of the inputs are not needed.  If these inputs are in are found in a tx to the user in a block regardless of the tx hash then the user has been paid.

One can simply record this information and wait until he gets complaints.

Agreed.  There is no "one solution".  What works best will depend on the business needs and available resource.  The only way you get "robbed" is if you just look to see if a tx confirmed based on the tx id, and if it doesn't show then you pay the user again.  Even beyond mutability there are other (harder) ways this could be exploited.  A tx which doesn't get relayed to all nodes for example could get relayed to the attacker who saves a copy and once MtGox pays them again, directly broadcasts the tx to miners.  If one of them picks it up then the user could get paid twice. 

The simplest method would be as you said, to manually review tx which don't confirm and not assume that no confirmation when looking up by txid means it is safe to pay again.

Another way to prevent this would be for MtGox to pay THEMSELVES before paying the user "again".  If the user in this tx reported they hadn't been paid.  MtGox could create a new tx spending these inputs back to an address they (MtGox) controls.  If the complaint was legit then MtGox would be able to spend those coins back to themselves and then after that "refund" confirms would they pay the user again (or return funds back to the user's account).  If the user was lying and the had mutated the signature then MtGox attempt transfer the coins back to their hotwallet would fail.  Why? because it would be rejected by the network as a double spend of a outputs already spent in a tx in a block.  Although this method provides less "automatic information" it would have prevent a single satoshi from being stolen.

Yet another way would be for MtGox to delete the "missing tx" from their wallet and then check the unspent outputs list.  With the tx gone, in this case those two inputs should return to the unspent output list. If they don't then it is because the attacker has already been paid in a modified transaction  (Note I don't recommend using this method without testing).

Yet another way would be to have a business rule of only using the same inputs when making a duplicate payment attempt.  This would ensure the user couldn't possibly be paid twice as that would mean both copies of a double spend were included in a block.

So there are a lot of ways (with differing degrees of reporting, complexity to code, and advantages).  The only thing you absolutely can't do is to naively "check tx id, yup not confirmed let me make a new payment to the user".  

[phzi]

So, shall we speculate how many bitcoins MtGox lost when someone realized they were not checking their txouts properly?  Could be 1000s of BTC.  But, I bet it was nothing at all... MtGox just screwed up on how they track their txouts, and re-spent txouts that were already spent.  Cross fingers the next announcement isn't MtGox has lost your bitcoins (which would probably be yet another lie)...

Bitcoin wiki: "assuming a txout exists because the client created it previously is unsafe." Obvious MtGox failed to heed this advise...  This is MtGox stupidity plain and simple.  I hope anyone still using MtGox (god knows why) withdrawals everything as soon as they have the opportunity (presuming they ever re-enable withdrawals).

I also think it is high time the Bitcoin foundation removed MtGox's member status and kicked out their failing leader (if you can even call him that).  Badmouthing the bitcoin protocol because you have a faulty inplementation with a lack of checks and balances is ridiculous and should not be acceptable to the foundation or the community.  I would say the foundation's credibility is severly at stake here.

Transaction maliability is not a 'defect'... it is a known and published reality of the bitcoin protocol.  Txid's identify a transaction reliably _after_ it is included in a block.  Using txid as an identifier to track your txouts is beyond stupid.  (Which must be the case if MtGox was trying to re-use txouts that where previously spent but confirmed with different txids).