Extending script and reserved opcodes

[TierNolan]

(I updated this with suggestions from theymos)

According to the wiki, a script which makes use of a unassigned opcode is invalid.

"Using an unassigned opcode makes the transaction invalid"

The script language could be extended if there was an opcode which enabled new opcodes.

Coin owners could enable new opcodes in the scriptPubKey.

New opcodes

The system would need 2 new opcodes.

OP_OP_ENABLE
OP_OP_EXTEND (placeholder opcode)

This opcode reads from the stack

var_int number_of_new_opcodes
byte_array new_opcodes[number_of_new_opcodes]
byte_array extended script

Opcodes are considered invalid if their last byte is the same as OP_OP_EXTEND.

Multi-byte opcodes are possible and that they can always be extended, since OP_OP_EXTEND must always be left free as the last byte.

Rules

If the script would fail under legacy rules, then the transaction is considered invalid.

If any unknown opcodes are enabled, do not incorporate the transaction into a block.

Blocks with unknown opcodes are considered valid if included in a mined block.

replace the byte array with the extended script with the bytes contained in the byte array.

If the new script fails, the transaction is invalid and blocks containing the transaction are invalid.

Extended scripts

"scriptSig":"<new signature algorithm signature> <current algorithm public key> <current algorithm signature>"

"scriptPubKey":"OP_DUP OP_HASH160 <hash of public key> OP_EQUALVERIFY OP_CHECKSIG <op enable setup byte arrays> OP_OP_ENABLE OP_CHECK_NEW_SIGNATURE"

This would mean that the old check would be performed by legacy clients and would have to pass, or they would reject it.

Adoption

If >50% of miners agree on the definition of an opcode, then it is in the interests of other miners to use the same definition.  Otherwise, the miner is risking the block being refused if he uses the wrong definition.

Including a transaction with an opcode he doesn't understand is risky, as it could invalidate the block.

After a period of time, 99% of miners would agree on what the new opcode means and then it would be safe to stop using the old signatures (though many probably still would).

Bootstrap

There is a boot strap problem here, since script can't be changed.

OP_OP_ENABLE could be implemented using the OP_IF opcode

<op_enable_setup> OP_0 OP_IF OP_ENDIF

This is guaranteed to be ignored by legacy clients.

Agreement on definition

Once a new opcode is accepted, it could be permanently added to the block chain.

A new opcode would be required

<opcode byte array>  <hash type> <hash of definition> OP_VALIDATE_DEFINITION VERIFY

This takes a hash of the definition and an opcode byte array.  The script should return false unless the miner feels that there is consensus that the definition is accepted.  

A definition can be hashed multiple times and that would update the definition.

Incorporating the script into a block would be risky unless there is agreement.

A contract could be created that pays out a large transaction fee for any miner who adds it to the block chain, to cover their risk that it is refused.

It would also be possible to incorporate the actual definition (in English) into the block chain using messages, but there is a limit on acceptable length, so it may have to be split into pieces.

Once the definition is added to the chain, then <some number> of blocks later it would be the official definition for all future blocks.

[theymos]

Every time script is updated, the network will be split until everyone upgrades to the new version. So there's no point in adding "extensible" script ops: adding any extended script op in the future still requires everyone to upgrade if they want to be part of the Bitcoin network.

For example:

This would mean that the old check would be performed by legacy clients and would have to pass.  Once it hit OP_OP_ENABLE, the script would end, but if it had been tagged as invalid, it would still be invalid.

Old clients now have a different view of which blocks are valid. They are on a separate network, isolated from the rest of Bitcoin.

[TierNolan]

Old clients now have a different view of which blocks are valid. They are on a separate network, isolated from the rest of Bitcoin.

I don't think that's true, barring bugs.

There was an invalid block that was accepted when it shouldn't have been.  However, I think even those clients would agree that the current chain is valid, right?

The idea is that clients have strict rules away what script they will forward as transactions and incorporate into blocks.  However, once a block is mined, the clients should agree if it is a valid block.

Newer clients might have stricter rules than the older ones, but if the latest clients say a chain is valid, then so should all legacy clients.

[theymos]

Actually, I think it would work if there was just one OP_OP_ENABLE that takes a list of many opcodes to enable. If the node can't understand any of these opcodes, then it stops here. If it does understand all of the opcodes, it also needs to verify the transaction again as though it doesn't understand them, and this "legacy" verification must also pass.

This will prevent new nodes from allowing any transactions that would be rejected by nodes without an understanding of new opcodes. Then it should be safe as long as clients using new opcodes control the network.

It might be a better method of changing things than the currently-planned method.

[TierNolan]

Actually, I think it would work if there was just one OP_OP_ENABLE that takes a list of many opcodes to enable. If the node can't understand any of these opcodes, then it stops here. If it does understand all of the opcodes, it also needs to verify the transaction again as though it doesn't understand them, and this "legacy" verification must also pass.

Sounds good.

The bootstrap needs to be changed slightly.

A script that uses the new protocol will have the following format

Code:

<legacy script> OP_0 OP_IF <byte array containing extended script> OP_ENDIF

Legacy clients will just process the legacy script.  As far as they are concerned the the rest of the script means the following.

Code:

push false onto the top of the stack

if the top value on the stack is true

 push a byte array onto the stack

endif

New nodes will verify the script as if they were legacy clients and then verify the script as if it was

Code:

<legacy script> <script contained in the byte array>

This will prevent new nodes from allowing any transactions that would be rejected by nodes without an understanding of new opcodes. Then it should be safe as long as clients using new opcodes control the network.

Exactly.  Old client miners might end up creating a few blocks that are invalid, but if 99% of the network refuses to build on them, then they will be discarded.

It might be a better method of changing things than the currently-planned method.

Do you mean the original post, or are their other proposals?

[TierNolan]

I updated the original post slightly.

[theymos]

Do you mean the original post, or are their other proposals?

I mean that your method might be an improvement over incrementing the transaction version and doing a "phased rollout" for every script change.

[Hans0]

But why would you have a new opcode if there is already an existing "legacy" sequence of opcode that you can use to emulate it? The new code would either not be able to do something new or it would cause a split. This also smells like vulnerabilities like getting the two split nets to do different things. And as nobody is using scripts anyway this is currently not needed.

[TierNolan]

But why would you have a new opcode if there is already an existing "legacy" sequence of opcode that you can use to emulate it? The new code would either not be able to do something new or it would cause a split. This also smells like vulnerabilities like getting the two split nets to do different things. And as nobody is using scripts anyway this is currently not needed.

The idea is

Old Clients
Legacy code: Accepts
New code: Accepts, but only if in mined blocks

New Clients
Legacy code: Accepts
New code: Accepts if it is valid under old and new rules

If 99% of the nodes are new clients, then splits aren't likely and will correct automatically.

The old client won't incorporate transactions which contain the new code.  This means  that it will never create a bad block according to the new rules.

If a new client creates a block that meets the new rules, the old client will accept the block.

There is no conflict between the 2 and the chain stays unified.

If an intermediate old client was to try to incorporate a script that fails according to the new code, the all the new miners would refuse to accept it.  This fork would last only a short time.  Once the 99% of the process power has generated 2 more blocks, that will be the main chain again.

The key point is that any script which is valid under the new rules is also valid under the old rules.  However, the new nodes will reject some blocks that would have been valid under the old rules.  This is why it is important that they have > 50% of the processing power before the rule is used in general.