OP_CHECKCOLORVERIFY: soft-fork for native color coin support

[jl2012]

Existing colored coin scheme relies on the order of inputs and outputs: https://bitcointalk.org/index.php?topic=106373.0;topicseen . There are few problems with this scheme:

1. Signature tag like SIGHASH_ANYONECANPAY cannot be used since the order of inputs in the final transaction is not known.

2. SPV nodes cannot verify colored coins

3. The whole transaction chain could not be pruned


A soft-fork could solve these problems:

1. Redefine OP_NOP3 as OP_CHECKCOLORVERIFY

2. When a user wants to mint colored coin, he sends some coin to an output with a script of

Code:

<color> OP_CHECKCOLORVERIFY OP_DROP OP_DUP OP_HASH160 <address> OP_EQUALVERIFY OP_CHECKSIG

where <color> is defined as

Code:

RIPEMD160(SHA256(script of previous output))

For example, if Satoshi wants to create a colored coin with his Genesis Block address 1A1zP1eP5QGefi2DMPTfTL5SLmv7DivfNa, <color> will be

Code:

RIPEMD160(SHA256(76a91462e907b15cbf27d5425399ebf6f0fb50ebb88f1888ac)) = 653756a92059ba4f3086471bd3c9e8442e4da3d3

By hashing the whole script, colored coin can be minted out of any future payment scheme

3. It is necessary for Satoshi to sign the minting output to make the whole transaction valid. The transaction is not valid if he is signing with SIGHASH_NONE, or SIGHASH_SINGLE but not for the minting output

4. Redistributing colored coin is done just like normal transaction, with inclusion of <color> OP_CHECKCOLORVERIFY in appropriate outputs. However, the total value a specific colored coin in outputs must be equal to or less than the total value in inputs. If the value in outputs is less than inputs, some color coins turn back to normal BTC. The only exception is Satoshi decided to mint more colored coin with his address, so the total colored output value could be larger than the input value.

5. Coins of different color could be exchanged in one transaction. An output may bear more than one color using multiple OP_CHECKCOLORVERIFY (e.g. an 1 BTC output with color xxx and color yyy means 1 xxx coin and 1 yyy coin)

6. Using coins from other addresses, one may mint more colored coin than the coin he has. For example, with 10 BTC from address xxx and 1 BTC from address yyy, one may mint 11 yyy coins.

Backward compatibility:

For existing clients, the coloring information is just

Code:

<random string> OP_NOP3 OP_DROP

and will be ignored


Possible extensions
1. To allow SPV verification of unique smart property (e.g. linking a colored coin with a real car), we need to ensure the uniqueness of the colored coin on the blockchain. A flag could be added to the minting and subsequent transactions to declare that only one UTXO may have this color at any time.

2. Sometimes you may want to restrict the total amount of a specific color coin in circulation. A flag could be added to the minting and subsequent transactions to declare that no more minting of the same color is allowed. (I'm not quite sure how this may work. Need more discussion)

These extensions require full nodes to keep an index for all colored coins in UTXO set

EDIT: These are not needed as there are better solutions: https://en.bitcoin.it/wiki/Smart_Property

[jackjack]

With OP_CHECKCOLORVERIFY you can't use OP_DROP. But as you need it for backward compatibility, you need to use OP_CHECKCOLOR instead. (Which puts a bool on the stack)

[jl2012]

With OP_CHECKCOLORVERIFY you can't use OP_DROP. But as you need it for backward compatibility, you need to use OP_CHECKCOLOR instead. (Which puts a bool on the stack)

It's just a naming issue. No matter how it is called, it will either do nothing, or invalidate the whole transaction.

Maybe I would call it a "meta OP code". Normal OP codes validate/invalidate individual transaction input. Meta OP codes validate/invalidate the whole transaction.

There are other discussion on OP_CHECKVALUE and OP_CHECKFEEVERIFY at https://bitcointalk.org/index.php?topic=181734.20 , which are also meta OP code.

[TierNolan]

It's just a naming issue. No matter how it is called, it will either do nothing, or invalidate the whole transaction.

Maybe OP_CHECKCOLOR could be defined

If the top of the stack is TRUE and the output is of the wrong colour type, then replace the TRUE with FALSE.

This means that this code works the same on old and new clients.  Except old clients always verify as true.

OP_TRUE OP_CHECKCOLOR OP_VERIFY

Maybe I would call it a "meta OP code". Normal OP codes validate/invalidate individual transaction input. Meta OP codes validate/invalidate the whole transaction.

They already did it with the P2SH change.

[jl2012]

It's just a naming issue. No matter how it is called, it will either do nothing, or invalidate the whole transaction.

Maybe OP_CHECKCOLOR could be defined

If the top of the stack is TRUE and the output is of the wrong colour type, then replace the TRUE with FALSE.

This means that this code works the same on old and new clients.  Except old clients always verify as true.

OP_TRUE OP_CHECKCOLOR OP_VERIFY

Maybe I would call it a "meta OP code". Normal OP codes validate/invalidate individual transaction input. Meta OP codes validate/invalidate the whole transaction.

They already did it with the P2SH change.

No, P2SH still works within individual input/output. My proposed OP_CHECKCOLOR works across all inputs/outputs in the same transaction.

If the transaction does not follow the color rule, the whole transaction is invalid so there is no reason to further exam the validity of individual output. If the transaction follows the color rule, all outputs must also follow the color rule. Therefore, OP_CHECKCOLOR does not really need to modify anything on the stake. When it comes to validation of individual signature, OP_CHECKCOLOR is nothing more than OP_NOP1

[grau]

Therefore, OP_CHECKCOLOR does not really need to modify anything on the stake. When it comes to validation of individual signature, OP_CHECKCOLOR is nothing more than OP_NOP1

Would this mean that clients that interpret OP_NOP1 as OP_CHECKCOLOR would reject transactions that others accept?

If yes, a block containing such transaction forks between those clients.
If not, then this is just an annotation on the transaction that does not prove that the colored coin can be traced back to its genesis, therefore SPV clients can not rely on the color indicated.

[jl2012]

Therefore, OP_CHECKCOLOR does not really need to modify anything on the stake. When it comes to validation of individual signature, OP_CHECKCOLOR is nothing more than OP_NOP1

Would this mean that clients that interpret OP_NOP1 as OP_CHECKCOLOR would reject transactions that others accept?

Yes. (To avoid confusion, I changed it to OP_NOP3 as some previous BIPs has proposed on OP_NOP1 and OP_NOP2)

If yes, a block containing such transaction forks between those clients.

It requires a soft-fork. As long as a vast majority of miners (e.g. >75%) follow the new rules, there won't be any prolonged fork. At that time, the market force will make the minority to reject invalid OP_CHECKCOLOR transaction or they will get orphaned. See how v2 block was gracefully implemented: https://en.bitcoin.it/wiki/BIP_0034

If not, then this is just an annotation on the transaction that does not prove that the colored coin can be traced back to its genesis, therefore SPV clients can not rely on the color indicated.

OP_CHECKCOLOR compatible clients (both mining and non-mining nodes) will never create, relay, or mine invalid OP_CHECKCOLOR transactions. Before 75% of miners enforce the new rules, they will still accept blocks with invalid OP_CHECKCOLOR transactions to avoid a prolonged fork. However, they will not acknowledge the color in these invalid OP_CHECKCOLOR transactions. And yes, SPV clients won't work in this sub-optimal situation.

[grau]

I like this, and also the SIGHASH_WITHINPUTVALUE  (https://bitcointalk.org/index.php?topic=181734.20).
We should perhaps bump the version of the transaction too.

[jl2012]

I like this, and also the SIGHASH_WITHINPUTVALUE  (https://bitcointalk.org/index.php?topic=181734.20).
We should perhaps bump the version of the transaction too.

The original SIGHASH_WITHINPUTVALUE idea, unfortunately, requires a hard-fork because there is no room to inject extra data when signing. I have a soft-fork proposal for this called OP_CHECKVALUE

[jl2012]

The first OP_CHECKCOLORVERIFY transaction is on testnet now, but I have some difficulty to redistribute it.

https://bitcointalk.org/index.php?topic=254354.0

[TierNolan]

It requires a soft-fork. As long as a vast majority of miners (e.g. >75%) follow the new rules, there won't be any prolonged fork. At that time, the market force will make the minority to reject invalid OP_CHECKCOLOR transaction or they will get orphaned. See how v2 block was gracefully implemented: https://en.bitcoin.it/wiki/BIP_0034

Right, and that change has a "point of no return".  

Your target is to propose an upgraded transaction type right?

That makes voting a little more difficult.

Maybe, propose a block type 3 that accepts transaction type 2.

If more than 75% of the last 1000 blocks are type 3 or higher, then reject blocks containing invalid OP_CHECKCOLOR transactions
If more than 95% of the last 1000 blocks are type 3 or higher, then reject any blocks of type less than 3, forever afterwards.

OP_CHECKCOLOR compatible clients (both mining and non-mining nodes) will never create, relay, or mine invalid OP_CHECKCOLOR transactions. Before 75% of miners enforce the new rules, they will still accept blocks with invalid OP_CHECKCOLOR transactions to avoid a prolonged fork. However, they will not acknowledge the color in these invalid OP_CHECKCOLOR transactions. And yes, SPV clients won't work in this sub-optimal situation.

Right, the point of the change to make miners support it.

Is my understanding of your process correct?

Minting

Input (1 BTC):  from "some-output"
Output (1 BTC): Hash("some output") OP_CHECKCOLOR ....

Transfer

Input (1 BTC): from the previous transaction (Hash("some output") OP_CHECKCOLOR ....)
Output1 (0.5 BTC): Hash("some output") OP_CHECKCOLOR ....
Output2 (0.5 BTC): Hash("some output") OP_CHECKCOLOR ....

So, effectively OP_CHECKCOLOR does both checking for minting and checking for spending.

Transaction Checking

It is nice that the colour is stored in the sig script of the output.  That way it can be done as a local check for the transaction.

Something like:

If normal processing of the transaction where OP_CHECKCOLOR is a NOP fails, then the invalid transaction.

Scan all output scripts for the OP_CHECKCOLOR script
- If OP_CHECKCOLOR occurs twice in any output then invalid transaction
- If any OP_CHECKCOLOR is not preceded by a 20 byte array, then invalid transaction
- All colored outputs are marked as the color of the byte array
- Determine total value of outputs for each color
- Scan input scripts
-- find the corresponding output script
--- If it contains <color> OP_CHECKCOLOR somewhere, then the input is of that color
--- If hash(output) is equal to <color>, then the input is of that color
--- Credit that color with the value of the input
- If the total outputs for a particular color is greater than the inputs, then invalid transaction

[Peter Todd]

You know OP_CHECKCOLORVERIFY can be implemented more usefully by adding op-codes to let scripts analyze the txouts of the transactions spending them. It'd be useful for a lot more than colored coins, and frankly there is no way in hell you are going to get a colored-coin-specific opcode included in Bitcoin.

I'd suggest you think about how a more general solution could work.

[jl2012]

It requires a soft-fork. As long as a vast majority of miners (e.g. >75%) follow the new rules, there won't be any prolonged fork. At that time, the market force will make the minority to reject invalid OP_CHECKCOLOR transaction or they will get orphaned. See how v2 block was gracefully implemented: https://en.bitcoin.it/wiki/BIP_0034

Right, and that change has a "point of no return".  

Your target is to propose an upgraded transaction type right?

That makes voting a little more difficult.

Maybe, propose a block type 3 that accepts transaction type 2.

If more than 75% of the last 1000 blocks are type 3 or higher, then reject blocks containing invalid OP_CHECKCOLOR transactions
If more than 95% of the last 1000 blocks are type 3 or higher, then reject any blocks of type less than 3, forever afterwards.

OP_CHECKCOLOR compatible clients (both mining and non-mining nodes) will never create, relay, or mine invalid OP_CHECKCOLOR transactions. Before 75% of miners enforce the new rules, they will still accept blocks with invalid OP_CHECKCOLOR transactions to avoid a prolonged fork. However, they will not acknowledge the color in these invalid OP_CHECKCOLOR transactions. And yes, SPV clients won't work in this sub-optimal situation.

Right, the point of the change to make miners support it.

+1

Is my understanding of your process correct?

Minting

Input (1 BTC):  from "some-output"
Output (1 BTC): Hash("some output") OP_CHECKCOLOR ....

To be precise, it is "script of some-output" (see transaction a0eecca313623fb39ddf92ed36782ec58003fec049d8395166b459bfbbfa3e7a on testnet as a minting example)

The risk of color collision would be similar to address collision, which could be neglected

Transfer

Input (1 BTC): from the previous transaction (Hash("some output") OP_CHECKCOLOR ....)
Output1 (0.5 BTC): Hash("some output") OP_CHECKCOLOR ....
Output2 (0.5 BTC): Hash("some output") OP_CHECKCOLOR ....

So, effectively OP_CHECKCOLOR does both checking for minting and checking for spending.

Yes. I can't think of a case which minting and spending need different code

Transaction Checking

It is nice that the colour is stored in the sig script of the output.  That way it can be done as a local check for the transaction.

Something like:

If normal processing of the transaction where OP_CHECKCOLOR is a NOP fails, then the invalid transaction.

Scan all output scripts for the OP_CHECKCOLOR script
1. If OP_CHECKCOLOR occurs twice in any output then invalid transaction
2. If any OP_CHECKCOLOR is not preceded by a 20 byte array, then invalid transaction
3. All colored outputs are marked as the color of the byte array
4. Determine total value of outputs for each color
4.1. Scan input scripts,
4.1.1. find the corresponding output script
4.1.1.1. If it contains <color> OP_CHECKCOLOR somewhere, then the input is of that color
4.1.1.2. If hash(output) is equal to <color>, then the input is of that color
4.1.1.3 Credit that color with the value of the input
4.2. If the total outputs for a particular color is greater than the inputs, then invalid transaction

1. We may or may not allow multiple OP_CHECKCOLOR in one output. If it is allowed, we can have mixed colored coin, which could be de-mixed in the next transaction.

2. If any OP_CHECKCOLOR is not preceded by a 20 byte array and followed by OP_DROP, then invalid transaction. It is for backward compatibility

3. Yes

4.1.1.1. Do you mean spending here?

4.1.1.2. Do you mean minting here?

4.1.1.3. Yes if it is spending colored coin. There is no restriction for the value in minting.

For example, I have 1 normal BTC in address xxx, and 1 normal BTC in address yyy. I can mint at most 2 Hash(xxx)-color BTC in one transaction. This will make minting more effective. Otherwise, I have to send 1 BTC from yyy to xxx first, before I could mint 2 Hash(xxx)-color BTC

[jl2012]

You know OP_CHECKCOLORVERIFY can be implemented more usefully by adding op-codes to let scripts analyze the txouts of the transactions spending them. It'd be useful for a lot more than colored coins, and frankly there is no way in hell you are going to get a colored-coin-specific opcode included in Bitcoin.

I'd suggest you think about how a more general solution could work.

Thanks for your comment. I will think about it. At least, does my proposal technically work?

[TierNolan]

You know OP_CHECKCOLORVERIFY can be implemented more usefully by adding op-codes to let scripts analyze the txouts of the transactions spending them. I

That's true.  However, the current script system is greatly restricted by the standard transaction rules (and the fact that some opcodes are disabled).

The problem is that the system requires analysis of all inputs and other outputs.  You need to add the values together.

OP_PUSH_INPUTS:  This would push all the inputs on to the stack

<value> <script> <value> <script> ... <value> <script> <number of inputs>

OP_PUSH_OUTPUTS:  This would push all the outputs on to the stack

<value> <script> <value> <script> ... <value> <script> <number of other outputs> <index of this output>

You need the "splice" opcodes to be enabled to actually manipulate the results.

However, that is a huge amount of overhead.

Another issue is that the system assumes that if <hash> OP_CHECKCOLOR is in an output script, that is proof that the output is a specific color.

[jl2012]

You know OP_CHECKCOLORVERIFY can be implemented more usefully by adding op-codes to let scripts analyze the txouts of the transactions spending them. I

That's true.  However, the current script system is greatly restricted by the standard transaction rules (and the fact that some opcodes are disabled).

The problem is that the system requires analysis of all inputs and other outputs.  You need to add the values together.

OP_PUSH_INPUTS:  This would push all the inputs on to the stack

<value> <script> <value> <script> ... <value> <script> <number of inputs>

OP_PUSH_OUTPUTS:  This would push all the outputs on to the stack

<value> <script> <value> <script> ... <value> <script> <number of other outputs> <index of this output>

You need the "splice" opcodes to be enabled to actually manipulate the results.

However, that is a huge amount of overhead.

Another issue is that the system assumes that if <hash> OP_CHECKCOLOR is in an output script, that is proof that the output is a specific color.

I think we could have a more generic code called OP_META, which is OP_NOP3 for old clients. A typical case would look like:

Code:

<serialized script> OP_META OP_DROP OP_DUP OP_HASH160 <address> OP_EQUALVERIFY OP_CHECKSIG

OP_META will dictate the validity of the whole transaction.

Old clients will ignore <serialized script> OP_META OP_DROP.

New clients will run the serialized script. So we can include whatever new OP codes we want without breaking old clients.

This will also solve the problem of SIGHASH_WITHINPUTVALUE in a soft-fork (https://bitcointalk.org/index.php?topic=181734.20)

I don't know much about zerocoin but I suspect OP_META will be useful for it too

Going to have meal now. Will give more details later

[TierNolan]

I think we could have a more generic code called OP_META, which is OP_NOP3 for old clients. A typical case would look like:

Code:

<serialized script> OP_META OP_DROP OP_DUP OP_HASH160 <address> OP_EQUALVERIFY OP_CHECKSIG

OP_META will dictate the validity of the whole transaction.

What does OP_META do?  Push all inputs and outputs onto the stack and then run <serialize script>?

A loop opcode would be pretty useful for things like this.   Script spam could be keep to a minimum by requiring that the loop length can be determined by looking at the script. 

<loops> OP_LOOP ...... OP_ENDLOOP

If <loops> is fixed, then it is clear how many loops are required.  All opcodes between the OP_LOOP and OP_ENDLOOP would have their weights multipled by <loops>.

[Peter Todd]

You know OP_CHECKCOLORVERIFY can be implemented more usefully by adding op-codes to let scripts analyze the txouts of the transactions spending them. It'd be useful for a lot more than colored coins, and frankly there is no way in hell you are going to get a colored-coin-specific opcode included in Bitcoin.

I'd suggest you think about how a more general solution could work.

Thanks for your comment. I will think about it. At least, does my proposal technically work?

Probably, though I didn't look at it in detail.

The issue is it trades off small scripts for a whole lot of effort creating code for colored coins; this code will need to be duplicated across many different bitcoin implementations. Frankly that's just not worth it.

re: OP_META, have you heard of merkelized abstract syntax trees?

[TierNolan]

The issue is it trades off small scripts for a whole lot of effort creating code for colored coins; this code will need to be duplicated across many different bitcoin implementations. Frankly that's just not worth it.

What about some way to define custom opcodes then.

<opcode-id> OP_META

A special transaction could define an <opcode-id> to <script> mapping.  There could be a limit of one per block (or maybe charge 1BTC each).

100 blocks after it is included, the script can be inserted into any transaction using the id and OP_META.

This gives flexibility but doesn't clutter up the blockchain with repeated code.

Client authors might create dedicated functions to implement popular OP_META functions.  Otherwise, they can just run the script.

[Peter Todd]

TierNolan: learn about merkelized abstract syntax trees.