The pull requests on github are related to removing duplicates from display and preventing the spending of unconfirmed change outputs.  It will prevent FUTURE problems but it isn't going to magically fix old problems. 

How many times have you heard that (never backed up claim)?

Infographic in MIT technology review.

It is a known issue of mutated transactions. 
https://bitcointalk.org/index.php?topic=460944.0

The immediate fix is to delete the broken tx (using pywallet).  Make sure to make a backup first.
Then make sure the wallet is synced and all prior transactions have confirmed.
Recreate the broken transaction and it will go through this time.

In the future it would be best to ensure you do not attempt to spend unconfirmed change.  The next release of the client will have an option to ensure that doesn't happen but for now if you make sure all your prior txs are confirmed before making a new tx this can't happen (even if the prior txs are mutated).

including the OP.

It doesn't it will just keep rebroadcasting periodically.  If you are concerned about that scenario, ensure you have a high number of peers and use a vpn tunnel to get "past" your ISP.  No client does this but you could design a client so that it only broadcasts a tx randomly to some of its peers.  If the tx fully propagates the network the other half should inform you about it.  If they don't that would be a warning sign.  The node could also rotate peers to make isolation even more difficult.

For a commercial entity this can be taken a step further by putting a number of geo diverse listening nodes out that which communicate with your main node via an out of band channel.  We use a setup similar to that to troubleshoot customer reports of "transaction not showing up".  The listening nodes can also be used by 0-confirm merchants to look for things like double spend attempts.  If you imagine a 0-confirm merchant who has 8 listening nodes setup around the world each with an average of 2,000 peers connected, there may be some overlap put lets assume that gives the merchant 10,000 or so unique peers.  If any one of those 10,000 unique peers is relayed the double spend, then it would be relayed to one of the listening nodes who would alert the main node.

It actually isn't that uncommon.  Since the laws on the books are written over the course of centuries the particular status of something can sometimes been undefined.  While no law previously in CA explicitly by name declared Bitcoin illegal, the existing laws of "lawful money" brings up at least the question on if Bitcoin violates those.

A company would much rather have a situation where a law explicitly defines something as lawful then a scenario where it "probably is lawful because nobody has had all their assets seized by the state ... yet".  My guess is some business interest in California lobbied to get this passed.  Putting something in explicit black and white is always a good thing (well as long as you are favor of what is written).

The good news is nobody had their Bitcoins stolen.   The site now long has the means to repay the bitcoin IOUs it issued (because of the alleged and possibly made "hack", but if you don't have the private key for "your coins" guess what?  You never had any bitcoins.

Not sure how many thousand more of these hacks and "hacks" it will take before that concept sinks in.

I am confident that eventually tx ids will be immutable.  Sipa proposal seems to be a good way forward and it can be done gradually (favor "better over "worse", then consider "worse" nonstandard, then eventually "worse" is invalid at the protocol level).  It is a complex issue because there are multiple ways to mutate a tx.  

Long term goal should be immutable tx ids.
Short term goal should be to minimize the disruption caused by tx id mutability.

As a side note alt-coin designers if you actually want to improve on Bitcoin and not make a pump and dump, don't make the signature part of the transaction input have it follow the tx body and sign the entire transaction.

Bitcoin
-----------------------
<some stuff here>
Inputs (including signature inside)
<some extra stuff here>
Output
<some extra stuff here too>

Better
--------------------------
Header
Input(s)
Output(s)
Signature(s)

The tx body is the header, inputs, and outputs.  Hash the body you get the tx id.  Use the same hash and the private key for each input to produce the signatures.  Append the signatures to the tx body and broadcast the whole thing.   You don't even need to provide pubkey in the input as it can be reconstructed from the signature.  I am not really sure if Satoshi was drunk the day he came up with how to sign transactions but it has to be the most overly complex, convoluted system which really provides no benefit and lot of edge cases.  Compared to the elegance of other parts of the design is seems out of place.  I can't think of a cryptographic system where you sign a payload and then put the signature back into the payload.  It is normally payload & signature.

No.  The "solution" involves restricting tx such that there is only one form but it is a non-trivial change.  

See here for some more info:
https://bitcointalk.org/index.php?topic=459678.msg5128555#msg5128555

Disclaimer: it is intended for general information as opposed to a developer implementation guide.  For brevity and simplicity some things have been abstracted.


Two of the three issues relate to the fact that other cryptographic systems are not canonical.  When it is all done Bitcoin will be more restricted than the underlying systems is relies on.

I think either one would be fine and far superior to the wallet without warning creating a tx which it "knows" may be broken in the future.  Personally I prefer the first one only because I am "security paranoid".  The second option is more user friendly but the jump to creating and broadcasting tx automatically shouldn't be taken lightly.  Either way advanced users if they have a specific need could always disable this warning via an option in the menus (i.e. [ ] Do not warn on creating transaction with unconfirmed input).

This limitation is most likely to affect users with a small number of confirmed unspent outputs.  A way to improve the user experience would be for the client to be a little proactive about "output management".

A user who has 100 BTC as a single unspent output spends 1 BTC.  Most clients today would make that In: 100BTC, Out: 1 BTC & 99 BTC.  However if the client seeing this tx will reduce the "unspent output pool" below some safety margin would instead make it Out: 1 BTC, 49 BTC, 50 BTC.   Now once this tx confirms the user could potentially two more txs before, even being presented with the warning.

Just an idea.  It is far less critical than what I put in the OP but sometimes solving a problem means making the problem less likely to occur in the first place.

That is a rather large "all" by itself as the input script is rather "loose".  That isn't all you need to do though.  ECDSA signatures are not immutable - Ouch. You also need to restrict signatures to a single "form" (DER) as OpenSSL (which Bitcoin uses) validates a variety of forms - double ouch.

To produce immutable hashes you need to:
a) Limit signatures to DER.  OpenSSL verifies non canonical forms of signatures.  Initially Bitcoin was no more restrictive than OpenSSL.  Since v0.8 Bitcoin, non-DER signatures are non standard and are not relayed by nodes running v0.8+.  This was MtGox problem their "Goxxed v0" custom client was creating non-DER signatures and thus the tx were being dropped by most nodes.

b) Make ECDSA Signatures immutable.  ECDSA signatures are not immutable, they were never designed to be and they never will be.  ECDSA will produce the same signature with both S and -S for a given payload and priv key.  So same tx data, same signature, different hash.  To resolve this "Bitcoin signatures" will need to be more restrictive than ECDSA signatures.  In other words all (future) valid Bitcoin signatures are valid ECDSA signatures, but not all valid ECDSA signatures will be valid Bitcoin signatures.

c) Allow only one form of input script for a given outcome.  The Bitcoin protocol is rather loose when it comes to op-codes in the input script.  It is possible to make some changes to these and still produce the same output (just like 3 + 2 = 3 + 2 +0).  Fixing this aspect will require tighter rules on what is a valid input script.


That is a lot to do, test, and get right.  Think of the "upgrading a plane in flight" analogy.

It can be done is phases to provide a safer path to the hard fork.
First the improved rules would be implemented in new version of clients (i.e. all clients would always use a +S instead of a + or -S) but clients wouldn't treat the "worse" tx any differently.
Next clients would favor "better" form of the transaction (i.e. if a node receives duplicates it will keep the one with the +S).
Next clients would consider the "worse" forms to be non-standard and would refuse to relay them (i.e. -S = non-standard, and tx is dropped by an upgraded node but still valid if in a block).
Finally clients would tx which break the new "better" rules to be invalid (i.e. after block # XXXXXXXXXX a tx with a -S is invalid, a block containing a tx with a -S is invalid).

It isn't going to happen overnight so in the interim mutable transaction ids are just a reality for Bitcoin.

The issue isn't 0-conf transactions.  It is 0-confirm tx which use as their inputs the unconfirmed outputs of another transaction.

All Bitcoin clients ALREADY prevent the client from spending unconfirmed outputs for lots of very good reasons.   However since the wallet can "trusts itself" they wallet makes an exception for unconfirmed CHANGE.   Obviously no user is going to double spend themselves.   So the prior tx will eventually confirm and that means the change will eventually confirm as well.

Except if an attack mutates the original tx, then the original will never confirm, the duplicate will.  That means the change output is now invalid and the subsequent tx will never confirm.

Simple version:
This isn't about 0-conf tx.  This is about transactions using the as their input 0-conf outputs.

It isn't a security issue it is a usability issue.  Traditional double spends require the SENDER to be malicious.  That is a rather narrow security scope.  For a duplicate tx to break a subsequent tx which uses unconfirmed change output only requires that a single third party on an anonymous global decentralized network is malicious.   That is a rather asininely large security scope.  It becomes "Are there any bad/mean people in the world right now?  If no then you are fine nobody can break your tx. If yes then your tx might break randomly".

Example
I create a tx with a hash of "A".  It send 1 BTC somewhere and 4 BTC back to me as change. A malicious user looking to just create chaos mutates it so that it has a hash of "B". For this tx there is no possibility of the payment being broken.  Either A or B will be confirmed and the receiver will get their 1 BTC. 

The problem is created when I create a second payment (tx "C") before "A" or "B" confirms and my wallet uses the change output from "A".  As far as it knows (due to relay rules) tx "B" doesn't even exist so of course it is ok to use "A" we aren't going to double spend ourselves.  This makes tx "C" dependent on tx "A" being confirmed.  If "B" confirms then "A" never will and as a result "C" never will either.

The user is honest, he is following the rules, creating a valid tx (proper fee, no spam, etc), the merchant is being safe and waiting for a confirmation, essentially both parties are doing the "right thing".  However the payment just breaks due to the actions of a third party.  Can it be resolved?  Sure delete Tx "C", create a new Tx "D" which is identical to C except it uses the change output from "B" not "A" and send again.  Now try to explain why you "randomly" need to do that sometimes to a non-technical user.

To a non-technical user it simply looks like the Bitcoin network "breaks" randomly and then you need to pay again, sometimes.  Not exactly confidence building huh.  Sometimes after you pay it will just break and then you need to pay again.  On top of that imagine you believe you paid a merchant and then they ask you to pay them again because Bitcoin broke.  It seems suspicious right?   I just makes the tx flow counter intuitive.  

Until transaction ids are immutable, allowing users to spend unconfirmed change outputs is just going to cause chaos.

You probably won't be able to so the only solution is to prevent it from happening to begin with.


The client waiting until change is confirmed isn't "ideal".  The ideal solution is to make tx ids immutable but that is a long term fix.  In the interim a user (possibly) having to wait for the change to be confirmed is far superior to transactions randomly breaking and non-technical users and merchants trying to figure out why it seems to just fail randomly.  The later is just going to kill adoption. 

What is a malled tx look like?  There are many ways that a tx can be mutated that makes identifying the "correct" one impossible.  Most nodes and pools already drop the duplicate the issue is the duplicate is getting to the pool before the original.  It wouldn't end up in a block otherwise.

If you have two tx each are equally valid and correct, how do you know which one was mutated after sending?   You can't so you do what you already do.  You keep the one you receive first and you drop the other one.  That means if the mutated tx gets to a miner's memory pool first, it will get included in a block.

I had no doubts that it would get done.  Reading the comments though shows it is a challenge.  Looks like a bug already exists where if you have sufficient confirmed outputs to cover the amount sent but not the amount of the fee, the fee will be paid using unconfirmed output which breaks the "fix".

Hopefully the OP helps "non programmers" to understand the issue and what to expect.

That is the long term fix.  You essentially are talking about transaction immutability.  However it can't simply be a client side "feature" as an attacker would just modify their client.  It will need to be enforced by the protocol.   That means (most likely) a hard fork and there just is no way to do that "fast and easy".   The situation is made more complex by the fact that ECDSA signatures are not "unique" it is possible for more than one set of inputs to produce the same signature. 

Simple version:
Long Term Goal (hopefully share unanimously by core developers):  Make unconfirmed tx ids immutable (or at least immutable to third parties).
Short Term Goal: Client behavior in dealing with mutable tx ids needs to be more consistent and expected until we get to the long term goal.

As a historical note.  Adding P2SH was a much simpler fork to the protocol.  From the time that P2SH code was completed to the time P2SH was supported by the protocol was about 8 months.

The attacker stopped.  Why and for long run remains unknown.  A patched client doesn't prevent the duplicates from existing so if tx were still being duplicated they should show up.

Wallets will need to be "smarter".  A wallet knows how many unique unspent outputs it has.

So in the 100 BTC example it could spend it back to itself as 5x 20 BTC outputs.  Then a user could spend 1, have 19 BTC change confirming and still have 80 BTC available to spend.

Wallets also probably need to use simpler language like:  "Total balance: XXXX (available to spend: XXXX)  Why is this different?" with link/popup to simple user level explanation.

A "smart wallet" can engage in output management to minimize the disruption to a user.  There is no reason a tx must have 1 change output (to the protocol change is simply an output no different than the "intended" output).  If anything multiple change outputs would make blockchain analysis slightlymore difficult.  So when creating an outbound tx the wallet could look at its internal confirmed output pool and if it is "low" make 2 or more change outputs.  How many to make would depend a lot of the amount of the change (in BTC) and the number of unspent outputs (in discrete outputs not value).

Instead of

The wallet could instead do this:


Where * is a random amount such that the sum of the change addresses is 99 BTC.



It is important now with mass mutation attack going on to distinguish between:

0 confirm tx where all inputs are confirmed
vs
0 confirm tx where one or more inputs are also 0 confirm.

The former requires a race attack and malicious intent of the sender for the receiver to lose funds.  So if you trust your customers, or feel only a small % of your customers would attempt an attack you are relatively safe (especially for low value, low risk purchases).   The later however only requires that SOMEONE on the network mutate the prior unconfirmed transaction and that the mutated version ends up confirming.   The risk profile becomes "do you trust the entire world to not mess with your money even though it is painfully easy?"  In most cases that answer should be no.

In the donut shop example up thread, the customer isn't the one doing the mutating. He could be a regular customer and liked by the shopkeeper however his coffee tx gets mutated by a malicious third party.  Now mutated or not the coffee tx will go through, no risk to the shop there.   However if the duplicate is the one which is confirmed, then the change output used for the doughnut becomes invalid.  Now through no fault of either the shopkeeper or the legit customer the donut payment will never confirm.  It gets worse when you consider how confusing this will be to put the shopkeeper and the customer especially if their are "crypto-nerds".

Honestly until tx ids are immutable the use of unconfirmed change outputs is just going to cause mass chaos.  The long term goal should be immutable tx ids but lets be honest that is probably a six to nine month project.  It will most likely require a hard fork of the protocol and there is simply no fast and easy way to do that.    So in the immediate future transactions are mutable (even by third parties) and that means unconfirmed change can be broken by a malicious third party.  For the record the "DDOS attack" against exchanges involves this aspect of mutable transactions. 

I would also point out how nuanced this conversation is.  Now imagine someone who has a "user level" knowledge of Bitcoin.  Many people have no idea that Bitcoin works on the concepts of inputs and outputs.  Their knowledge is no deeper than the "abstraction" of wallet balances.  Now that is fine, the job of good software is to abstract a very complex system into a simple concept.  It isn't the user's fault that they don't know the inner workings, however you can't even start to explain the problem, until you explain how Bitcoin "really works".  That simply isn't a viable scenario for non-enthusiasts.  Case in point my need for a side note here: https://bitcointalk.org/index.php?topic=460944.0

You have to be verified to do a withraw (including gox to gox transfer).  His account isn't verified yet.