Make malleable transactions the norm, not the exception

[Elo]

Am I the only one here who thinks the "malleable transaction" issue is being approached all wrong?

The problem isn't really that transactions are malleable, it's that people are being led to think they can identify a previously unconfirmed transaction, when it's included in the blockchain, by its unconfirmed hash. This is true in most cases, but it's not true in special circumstances, or if you're subject to an attack.

To discourage this misplaced reliance, why not request miners to always modify all transactions before including them in the blockchain? As long as a fair proportion of miners do this, it will be clear to everyone that transaction success should not be monitored by the transaction hash.

I understand that MtGox wants transactions to have some kind of hash they can unambiguously refer to even if they get modified before inclusion in the blockchain. But to achieve this, simply defer referencing the transaction hash until it's actually included in several blocks - wait a day if you need to - and then quote it to the customer. An unconfirmed transaction hash is not admissible as evidence of payment before inclusion in the blockchain, anyway.

MtGox can track transactions internally via the actual transaction script hash that they signed (as opposed to hash of the entire encoded transaction).

Given the complexity of Bitcoin transaction script, coming up with a non-malleable transaction hash to cover all transactions (not only the few standard types) is futile. If we only standardize canonical hashes for standard transactions, the chance remains that something will be overlooked, either by the agreed-to standard or by a particular implementation, and the problem will rear its ugly head again.

[1715332958]

1715332958

[1715332958]

1715332958

[grau]

The attack only works if the one who is attacked is unable to reconcile his accounts, that commonly leads to bankruptcy, even without Bitcoin.

[Elo]

The attack only works if the one who is attacked is unable to reconcile his accounts, that commonly leads to bankruptcy, even without Bitcoin.

Right, but in the interest of making Bitcoin safer for businesses to adopt, and thereby to increase its spread and value, let's help them avoid common pitfalls, such as relying on transaction hashes MtGox-style.

[grau]

I do not think we need to add obfuscation to avoid people using a feature for a purpose it is not sufficient.

Again, a serious business should have an accounting that would spot if one gets robbed without relying on hashes on the block chain.

[TierNolan]

Another way would be for miners to modify the transaction to the standard version.

Pieter Wuille developed a set of rules for what is an unmodifiable transaction.

He suggests adding it as a new transaction version.

Before adding a set of rules to the memory pool, nodes could modify them to the standard format.  This is already allowed?

- Force "s" to be even in the signature
- Re-encode all values so that they are standard DER
- Remove all unused pushes values in the sigScript
- convert the sigScript to a set of (canonical) pushes
-- Compute the stack at the end of the sigScript and then just push that

[Elo]

The problem with trying to have a canonical format is that if you miss anything - anything, a single bit that an attacker can tweak without changing the hash - you're back to square one.

[TierNolan]

The problem with trying to have a canonical format is that if you miss anything - anything, a single bit that an attacker can tweak without changing the hash - you're back to square one.

That isn't a reason not to try. 

The core problem is that the transaction hash is different from the signature hash.

The possibility that the EC signature is malleable can't be defended against though.

Even then, once it is discovered, a canonical version could be picked (unless you can create an large number or it is expensive).

[Elo]

That isn't a reason not to try.

Yes, it is. When your intended solution may likely just create more problems than there were in the first place, it is better not to try.

Currently, at least informed people know that transaction hashes are unreliable. People who understand this are a small minority of Bitcoin users already. Now consider that you implement canonical transaction hashes, which aren't actually fully canonical. Then people rely on them because no one yet knows they aren't. Then someone figures out how to make them malleable. Kaboom.

The core problem is that the transaction hash is different from the signature hash.

The core problem, really, is that there's transaction script in the first place. This was an idiotic design decision in an otherwise brilliant protocol, and has caused nothing but grief and pain. We end up with a hobbled transaction script that can't really be used for any of the flexible purposes it was meant for, and a few hard-coded, "trusted" transaction types which don't require a script language, anyway.

If we could just define transaction hash = signing hash, that would be great. But a transaction can have many signing hashes, depending on the number of previous out points used and their pk script. Which signing hashes do we pick? We could concatenate them all, and then hash that. But then, what if a transaction is non-standard, and uses zero signatures? Then it's always malleable.

What if the signatures are of a type that doesn't sign all the new outputs, so the transaction could be disassembled, and its pieces put together to produce a different transaction that spends some of the same previous outputs? Then you can't rely on the transaction hash again, even if it's canonical.

The possibility that the EC signature is malleable can't be defended against though.

Indeed, that is one risk.

Even then, once it is discovered, a canonical version could be picked (unless you can create an large number or it is expensive).

But the entire deployed base of software will fail to do this, so for all practical purposes, you're back to non-canonical hashes again.

[TierNolan]

Yes, it is. When your intended solution may likely just create more problems than there were in the first place, it is better not to try.

Currently, at least informed people know that transaction hashes are unreliable. People who understand this are a small minority of Bitcoin users already. Now consider that you implement canonical transaction hashes, which aren't actually fully canonical. Then people rely on them because no one yet knows they aren't. Then someone figures out how to make them malleable. Kaboom.

Fair enough.

It also argues against having

The core problem, really, is that there's transaction script in the first place. This was an idiotic design decision in an otherwise brilliant protocol, and has caused nothing but grief and pain. We end up with a hobbled transaction script that can't really be used for any of the flexible purposes it was meant for, and a few hard-coded, "trusted" transaction types which don't require a script language, anyway.

Re-designing the sigScript so that it just pushes would help.

I agree that it is a pity that the scripting system was crippled.

The MAST system would have been a big improvement.  You pay to a MAST hash.  All the complexity is on the spending side.

To prevent people using the MAST hashes as messages, there would need to be a way to encode pruneable data too.

[Elo]

I agree that it is a pity that the scripting system was crippled.

I think it had to be crippled once it was there. Too much complexity. It should have been just the standard transaction types to begin with. (Though I would have liked a "user data" field for extensions)

The MAST system would have been a big improvement.  You pay to a MAST hash.  All the complexity is on the spending side.
To prevent people using the MAST hashes as messages, there would need to be a way to encode pruneable data too.

I haven't put thought into this, but that does sound like it could have been an improvement. Is there a benefit to MAST over the current Pay to Script Hash concept?

[Peter Todd]

The attack only works if the one who is attacked is unable to reconcile his accounts, that commonly leads to bankruptcy, even without Bitcoin.

That's an excellent way of describing it.

[TierNolan]

I think it had to be crippled once it was there. Too much complexity. It should have been just the standard transaction types to begin with. (Though I would have liked a "user data" field for extensions)

There could still be standard scripts. 

The current rule is that you have to justify why you need an opcode unlocked rather than unlocking it and seeing if anyone wants to use it.

I haven't put thought into this, but that does sound like it could have been an improvement. Is there a benefit to MAST over the current Pay to Script Hash concept?

It is potentially smaller.  The scriptPubKey is always the exact same size.

If you have lots of if branches, you only need to send the one that is actually executed.

[Peter Todd]

That isn't a reason not to try.

Yes, it is. When your intended solution may likely just create more problems than there were in the first place, it is better not to try.

Currently, at least informed people know that transaction hashes are unreliable. People who understand this are a small minority of Bitcoin users already. Now consider that you implement canonical transaction hashes, which aren't actually fully canonical. Then people rely on them because no one yet knows they aren't. Then someone figures out how to make them malleable. Kaboom.

Agreed.

There's some specialized multi-party-protocols that absolutely must have tx malleability stopped to work; for everyone else malleability is just a nuisance that requires some care to avoid. I think for those specialized protocols we should introduce a special signature type that hashes only the scriptPubKey and nValue of the output being spent - something that we can guarantee will never be malleable.

For everything else though the safest would be to just handle malleability properly. But I can easily see the "engineering" compromise chosen to be to try to eliminate malleability and accept that if we fail things are going to be ugly.

[kano]

So ... is it possible to modify the script itself?
Does this lead to the ability of modifying transaction scripts such that they now require something that stops the recipients from receiving their coins?

[TierNolan]

So ... is it possible to modify the script itself?
Does this lead to the ability of modifying transaction scripts such that they now require something that stops the recipients from receiving their coins?

You can only make changes that have no effect.  You can't change the inputs or outputs.  It just changes the transaction hash.  

Onward spending would need to use a different txid to refer to the coin.

[kano]

So ... is it possible to modify the script itself?
Does this lead to the ability of modifying transaction scripts such that they now require something that stops the recipients from receiving their coins?

You can only make changes that have no effect.  You can't change the inputs or outputs.  It just changes the transaction hash.  

Onward spending would need to use a different txid to refer to the coin.

No, I mean change the script to use all the same inputs and outputs but use one of the new transaction types ... multi-party-protocols (as stated above ...)