Why can't I use a negative value for OP_CHECKLOCKTIMEVERIFY

[CIYAM]

I've been testing a P2SH redeem script that looks like this:

Code:

OP_DUP
OP_SHA256
<secret hash>
OP_EQUAL
OP_IF
OP_DROP
OP_DUP
OP_HASH160
<public key hash 1>
OP_EQUALVERIFY
OP_CHECKSIG
OP_ELSE
<CLTV value>
OP_NOP2
OP_DROP
OP_DUP
OP_HASH160
<public key hash 2>
OP_EQUALVERIFY
OP_CHECKSIG
OP_ENDIF

I got it to work using -regtest (after a lot of mucking around as it appears that "coinbase" txs can't send to P2SH addresses without screwing up the scriptPubKey script) by using a <CLTV value> of 0x7f (block 127) and the CLTV redeem is working perfectly (in that it won't redeem until you have generated 127 blocks).

But if I change the <CLTV value> to 0xff (for block 255) suddenly the script interpreter is complaining about a "Negative locktime".

So how exactly am I supposed to tell it block 255?

(and according to the documentation locktime is an unsigned value so why is the OP_CHECKLOCKTIMEVERIFY reading a signed value?)

[1714816406]

1714816406

[1714816406]

1714816406

[1714816406]

1714816406

[1714816406]

1714816406

[1714816406]

1714816406

[achow101]

Have you tried padding it with 0's?

[CIYAM]

Have you tried padding it with 0's?

Not yet - but I thought that padding with zeros is going to fall foul of the rule that the pushed data must be minimal (as I read when looking into doing this P2SH stuff).

EDIT: I tried changing 0x017f to 0x02ff00 but then when I try and redeem (after 255 blocks) I get the following error:

Code:

error: {"code":-26,"message":"64: non-mandatory-script-verify-flag (No error)"}

Which is a strange looking error (but an error nonetheless).

[achow101]

Have you tried padding it with 0's?

Not yet - but I thought that padding with zeros is going to fall foul of the rule that the pushed data must be minimal (as I read when looking into doing this P2SH stuff).

EDIT: I tried changing 0x017f to 0x02ff00 but then when I try and redeem (after 255 blocks) I get the following error:

Code:

error: {"code":-26,"message":"64: non-mandatory-script-verify-flag (No error)"}

Which is a strange looking error (but an error nonetheless).

Ok, so maybe 0 padding makes the script non-standard. The script verifies fine which is why you have no error, so I guess it just doesn't pass standardness checks, probably because of 0 padding.

[CIYAM]

Ok, so maybe 0 padding makes the script non-standard. The script verifies fine which is why you have no error, so I guess it just doesn't pass standardness checks, probably because of 0 padding.

So - back to square one - how to get OP_CHECKLOCKTIMEVERIFY to understand block 255?

[achow101]

Ok, so maybe 0 padding makes the script non-standard. The script verifies fine which is why you have no error, so I guess it just doesn't pass standardness checks, probably because of 0 padding.

So - back to square one - how to get OP_CHECKLOCKTIMEVERIFY to understand block 255?

I can't really help you but you can probably find it in the code. The locktime in an OP_CLTV script is a CScriptNum so it is signed. Here is the code for CScriptNum: https://github.com/bitcoin/bitcoin/blob/595f93977c5636add1f4f2e64cd2d9b19c65a578/src/script/script.h#L194.

[CIYAM]

I can't really help you but you can probably find it in the code. The locktime in an OP_CLTV script is a CScriptNum so it is signed. Here is the code for CScriptNum: https://github.com/bitcoin/bitcoin/blob/595f93977c5636add1f4f2e64cd2d9b19c65a578/src/script/script.h#L194.

Yup - I checked the source code and that is what is confusing me.

If nLockTime is unsigned (which all the docco and source code I looked at shows that it is) then why on earth is OP_CHECKLOCKTIMEVERIFY expecting a signed value?

[achow101]

I can't really help you but you can probably find it in the code. The locktime in an OP_CLTV script is a CScriptNum so it is signed. Here is the code for CScriptNum: https://github.com/bitcoin/bitcoin/blob/595f93977c5636add1f4f2e64cd2d9b19c65a578/src/script/script.h#L194.

Yup - I checked the source code and that is what is confusing me.

If nLockTime is unsigned (which all the docco and source code I looked at shows that it is) then why on earth is OP_CHECKLOCKTIMEVERIFY expecting a signed value?

Honestly I have no idea. All I could find is that the actual nLockTime in the transcation (not in an OP_CLTV script) is supposed to be unsiged. I don't know why it would be signed for OP_CLTV. Bring it up in an issue on github, you'll get more answers that way.

[CIYAM]

Honestly I have no idea. All I could find is that the actual nLockTime in the transcation (not in an OP_CLTV script) is supposed to be unsiged. I don't know why it would be signed for OP_CLTV. Bring it up in an issue on github, you'll get more answers that way.

Yup - it seems the core devs don't bother with this forum now (can't blame them - it is mostly just ad sig nonsense posts anyway).

https://github.com/bitcoin/bitcoin/issues/7275

EDIT: It appears that I found a bug.

[sniveling]

OP_CHECKLOCKTIMEVERIFY is validated in EvalScript() in src/script/interpreter.cpp

At the bottom of the quoted code it says in a comment that in the rare event that the argument may be < 0 due to some arithmetic being done first, you can always use.

0 MAX CHECKLOCKTIMEVERIFY

I think that stops the error SCRIPT_ERR_NEGATIVE_LOCKTIME shown in red.



https://github.com/bitcoin/bitcoin/blob/master/src/script/interpreter.cpp#L340-L380

            case OP_CHECKLOCKTIMEVERIFY:
            {
                if (!(flags & SCRIPT_VERIFY_CHECKLOCKTIMEVERIFY)) {
                    // not enabled; treat as a NOP2
                    if (flags & SCRIPT_VERIFY_DISCOURAGE_UPGRADABLE_NOPS) {
                        return set_error(serror, SCRIPT_ERR_DISCOURAGE_UPGRADABLE_NOPS);
                    }
                    break;
                }

                if (stack.size() < 1)
                    return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION);

                // Note that elsewhere numeric opcodes are limited to
                // operands in the range -2**31+1 to 2**31-1, however it is
                // legal for opcodes to produce results exceeding that
                // range. This limitation is implemented by CScriptNum's
                // default 4-byte limit.
                //
                // If we kept to that limit we'd have a year 2038 problem,
                // even though the nLockTime field in transactions
                // themselves is uint32 which only becomes meaningless
                // after the year 2106.
                //
                // Thus as a special case we tell CScriptNum to accept up
                // to 5-byte bignums, which are good until 2**39-1, well
                // beyond the 2**32-1 limit of the nLockTime field itself.
                const CScriptNum nLockTime(stacktop(-1), fRequireMinimal, 5);

                // In the rare event that the argument may be < 0 due to
                // some arithmetic being done first, you can always use
                // 0 MAX CHECKLOCKTIMEVERIFY.
                if (nLockTime < 0)
                    return set_error(serror, SCRIPT_ERR_NEGATIVE_LOCKTIME);

                // Actually compare the specified lock time with the transaction.
                if (!checker.CheckLockTime(nLockTime))
                    return set_error(serror, SCRIPT_ERR_UNSATISFIED_LOCKTIME);

                break;
            }

[CIYAM]

0 MAX CHECKLOCKTIMEVERIFY

Yes - I read that in the code - but how exactly do you do that?

[sniveling]

0 MAX CHECKLOCKTIMEVERIFY

Yes - I read that in the code - but how exactly do you do that?

Could you use OP_MAX after pushing zero to the stack, then use OP_NOP2? If OP_MAX returns the larger of the two top items then I assume it would return zero rather than a negative value.

OP_MAX     0xa4     Return the larger of the two top items

I Googled for examples of how to use that code, but found nothing.

[CIYAM]

Could you use OP_MAX after pushing zero to the stack, then use OP_NOP2? If OP_MAX returns the larger of the two top items then I assume it would return zero rather than a negative value.

That doesn't seem right to me - personally I think there is a bug with the way that OP_CHECKLOCKTIMEVERIFY is working.

I have reported it on here: https://github.com/bitcoin/bitcoin/issues/7275 and it seems that the core devs agree that something is not right.

Unfortunately the problem might be a generic one (that unsigned values cannot be used in Bitcoin scripts).

[sniveling]

Unfortunately the problem might be a generic one (that unsigned values cannot be used in Bitcoin scripts).

In this conversation sipa explains how Bitcoin accepts unsigned values, but then passes signatures directly to OpenSSL, and OpenSSL just parses everything as unsigned. Therefore you have to read the byte string and if it's not already zero padded, add a 0x00 byte.

https://download.wpsoftware.net/bitcoin/wizards/2013/03/13-03-11.log

19:00 < HM> how do signed values even work
19:00 < sipa> they're stored as 2's complement
19:00 < sipa> so if the highest bit of the first byte is set, it's a negative vale
19:00 < sipa> but OpenSSL just parses everything as unsigned
19:02 < HM> sure, but how did these transactions get accepted ?
19:02 < sipa> because OpenSSL parses everything as unsigned
19:02 < sipa> and every bitcoin full node has used OpenSSL to parse signatures

19:04 < sipa> imagine you want to store the value 0x9999
19:04 < sipa> so the positive integer 39321
19:04 < sipa> the correct DER encoding is 0x009999
19:05 < sipa> as 0x9999 is interpreted as -26215

19:06 < sipa> the problem is, because OpenSSL knows it expects an unsigned integer, even if you store 0x9999, it will interpret that as 39321 and not as -26215
19:06 < HM> how is sane to encode a perfectly reasonable 2 byte unsigned value in 3 bytes with 1 useless 0x00 byte?
19:06 < sipa> because it is not an unsigned value
19:06 < sipa> DER doesn't have an unsigned integer type

19:09 < sipa> anyway: bottom line: every implementation _must_ accept 0x9999 as 39321, even though a standards-compliant DER parser would interpret that as a negative number, which would cause ECDSA to reject the signature as out of range

19:12 < sipa> well in a way it makes sense: "ok you give me this signature *parse* ok, syntactically correct. wait... this R value is negative? i don't expect a negative number here... let's assume you just missed a 0 byte in front"
19:13 < sipa> the only problem is that bitcoin passes signatures directly to OpenSSL

19:13 < HM> so basically you have to read the byte string and if it's not already zero padded, add a 0x00 byte

Your last comment on Github asks if 5 bytes are allowed.

okay - but does your change allow for 5 bytes?
(if it does then I guess the problem is resolved)
clearly we need it to work for: 0x017f, 0x02ff00 and 0x03ffff00 at the very least

It says in a comment here that CScriptNum is coded to accept up to 5-byte bignums, which are good until 2**39-1, well beyond the 2**32-1 limit of the nLockTime field itself.


https://github.com/bitcoin/bitcoin/blob/master/src/script/interpreter.cpp#L355-L358

               // If we kept to that limit we'd have a year 2038 problem,
                // even though the nLockTime field in transactions
                // themselves is uint32 which only becomes meaningless
                // after the year 2106.
                //
               // Thus as a special case we tell CScriptNum to accept up
                // to 5-byte bignums, which are good until 2**39-1, well
                // beyond the 2**32-1 limit of the nLockTime field itself.
                const CScriptNum nLockTime(stacktop(-1), fRequireMinimal, 5);

[CIYAM]

The problem with adding the zero byte is that there is another check when pushing bytes that you have pushed the "minimum number possible".

So it currently won't accept 0x02ff00 which is how it apparently should accept 255.

The problem won't affect mainnet as its block numbers will appear as non-negative numbers (and it probably also doesn't affect *nix timestamp values).

[hhanh00]

The problem with adding the zero byte is that there is another check when pushing bytes that you have pushed the "minimum number possible".

So it currently won't accept 0x02ff00 which is how it apparently should accept 255.

The problem won't affect mainnet as its block numbers will appear as non-negative numbers (and it probably also doesn't affect *nix timestamp values).

02 FF 00 is the shortest representation of 255 - so I'm guessing the error is somewhere else.

[CIYAM]

02 FF 00 is the shortest representation of 255 - so I'm guessing the error is somewhere else.

Yes - the issue has been tagged as a bug on Github.