0.9 Malleability Fix - isStandard() changes

[TierNolan]

The changes were to the isStandard() check.  The extra protection only helps if miners don't include non-standard transactions.

Does Eligius include the malleability rules, even if it doesn't include general isStandard() checks?

Looking at the code for changes, the only thing I could find was that sigScripts are verified to be "canonical".  This means that when pushing, the smallest opcode must be used.

This prevents malleability by encoding an identical byte array in a different way.

From Sipa's document, there are a few ways to change the transactions.

1) Inherent ECDSA signature malleability
- not fixed (0.9 produces canonic signatures only)

2) Non-DER encoded ECDSA signatures
- already fixed in 0.8

3) Superfluous scriptSig operations
- not fixed

4) Non-push operations in scriptSig
- already fixed

5) Push operations in scriptSig of non-standard size type
- fixed in 0.9

6) Zero-padded number pushes
- not fixed (only affects multisig)

7) Inputs ignored by scripts  
- doesn't need to be fixed

8) Sighash flags based masking
- doesn't need to be fixed

9) New signatures by the sender
- doesn't need to be fixed

1) could be handled using the (disabled) OP_XOR opcode to verify if the signature was even.

3) could be handled using the (active) OP_DEPTH opcode to verify that the stack is empty.  This assumes the sigScript is only pushes.

It claims on the wiki that OP_ADD etc. all use 32 bit numbers, but script.cpp seems to cast the scripts to BigNums.  

If your scriptPubKey doesn't process any inputs as numbers, then 6) doesn't seem like a big deal.

Opcodes like OP_IF would need to be preceded by a check that that the top of the stack is a single byte array of 0 or 1.

That suggests that except for the ECDSA signature malleability issue, transaction malleability is protected for blocks mined by pools which use isStandard().

[edit]
Added edits based on gmaxwell's post

[1714038841]

1714038841

[1714038841]

1714038841

[1714038841]

1714038841

[gmaxwell]

The changes were to the isStandard() check.  

No, they weren't. The real malleabiltiy fixes in 0.9 were in the wallet code.

(but indeed we did merge the changes from last September to expand the non-canonical forms to include the non-canonical pushes.)

Does Eligius include the malleability rules, even if it doesn't include general isStandard() checks?

Of course. It implements its own IsStandard() rules.

Looking at the code for changes, the only thing I could find

You were looking in the wrong place.

1) Inherent ECDSA signature malleability
- not fixed

0.9 authors transactions which always have low-S, this is the a prerequisite to making non-low-S non-canonical.

3) Superfluous scriptSig operations
- not fixed

Additional superfluous operations get blocked by IsStandard, IIRC.

6) Zero-padded number pushes
- not fixed

Though only relevant for multisig.

1) could be handled using the (disabled) OP_XOR opcode to verify if the signature was even.

You might as well say that it could be fixed using OP_S_EVEN ... "Disabled" OP code are no more existent than purely hypothetical ones and require no less effort to deploy.  What Sipa proposes is that we make one of the version bits on transactions effectively a flag that controls enforcing a tighter set of canonical rules, including this one.

It claims on the wiki that OP_ADD etc. all use 32 bit numbers, but script.cpp seems to cast the scripts to BigNums.  

You've misunderstood the code. Go look at the code that actually implements the cast.

If your scriptPubKey doesn't process any inputs as numbers, then 6) doesn't seem like a big deal.

Correct.

That suggests that except for the ECDSA signature malleability issue, transaction malleability is protected for blocks mined by pools which use isStandard().

"Protected" unless the attacker has the help of a miner. This is an adequate defense against griefing but doesn't make things like refund using protocols safe.

[TierNolan]

The changes were to the isStandard() check.  

1) Inherent ECDSA signature malleability
- not fixed

0.9 authors transactions which always have low-S, this is the a prerequisite to making non-low-S non-canonical.

But is this actually checked by isStandard()?  Will clients still forward/mine transactions with non-canonical?

Is the plan to do it in 2 stages, first make the reference client only produce canonical signatures and then make non-canonical non-standard?

I guess the problem is that old clients would have transactions rejected. 

In theory, miners could switch the signatures to canonical, but I assume that is discouraged.

3) Superfluous scriptSig operations
- not fixed

Additional superfluous operations get blocked by IsStandard, IIRC.

This means including extra pushes.  How can that be checked?

A transaction is non-standard if any of its signatures leave data on the stack?  That can't be checked without the input transaction.

1) could be handled using the (disabled) OP_XOR opcode to verify if the signature was even.

You might as well say that it could be fixed using OP_S_EVEN ... "Disabled" OP code are no more existent than purely hypothetical ones and require no less effort to deploy.

Sadly true. 

That suggests that except for the ECDSA signature malleability issue, transaction malleability is protected for blocks mined by pools which use isStandard().

"Protected" unless the attacker has the help of a miner. This is an adequate defense against griefing but doesn't make things like refund using protocols safe.

I think that would give reasonable safety in practice.

[TierNolan]

[used reply instead of edit]

[gmaxwell]

But is this actually checked by isStandard()?

Can't be yet, a majority of transactions on the network still fail the check.

Is the plan to do it in 2 stages, first make the reference client only produce canonical signatures and then make non-canonical non-standard?

Sort of, the first stage is "get all widely used signers to only produce canonical signatures", which making the reference client do it is just the first step. I believe the latest bitcoinj does this now too.

Thanks for the reminder though, it's time to start the nagging engines.

I guess the problem is that old clients would have transactions rejected. 
In theory, miners could switch the signatures to canonical, but I assume that is discouraged.

Sort of a tough problem considering how poor wallets handle the change.

A transaction is non-standard if any of its signatures leave data on the stack?  That can't be checked without the input transaction.

We can't verify a transaction without the input since they're included in the signatures.
I thought we templaitized the signatures, but apparently we don't (sometimes it's hard to remember whats been implemented vs discussed).

I think that would give reasonable safety in practice.

Safety for what? I agree that it's reasonably fine against trouble making when the attacker gains little and the harm of a loss is small. But there are already substantial pools like ghash.io who have mined maliciously in order to rip people off.

[TierNolan]

I think that would give reasonable safety in practice.

Safety for what? I agree that it's reasonably fine against trouble making when the attacker gains little and the harm of a loss is small. But there are already substantial pools like ghash.io who have mined maliciously in order to rip people off.

I wonder if the cross chain trade scheme would work, especially for small amounts.