What is latest good practice to store data in a transaction

[guillaume.bonnot]

Hi everyone !

I need to embed a 256 bits hash + a few bits of metadata in the bitcoin blockchain.

What is the best way to achieve this ? What are the size constraints ?

Thanks for your reply.

[hexafraction]

You can use a data-carrier transaction. Just push that data onto the script stack and use OP_RETURN. Note that a small percentage of nodes is configured not to relay them, and some miners won't confirm them, though they should work. Note that any coins sent this way are not spendable. I recommend either spending a small unspent output or sending change back to an address you control.

[gmaxwell]

Do you actually need to embed the hash or do you just need to commit to it?

If you only need to commit to it you are better of using a pay-to-contract (https://github.com/Blockstream/contracthashtool) or potentially the sign-to-contract (an analog with the contract commitment in the nonce), because the result takes less space (lower transaction fees) and is not easily censorable by miners or tracable to third parties that you don't wish to share the information with.

[guillaume.bonnot]

Thanks for the replies !

So i can make a transaction with 1 input and 2 outputs where the first one is the change and the other one is data with the op_return code. Sounds great !

Is there any consensus on the fees required ? Where can i find this information updated ?

I need all the data to be embedded on the blockchain, and if i understand well what a commitement is, you need to communicate the full data via another channel. It is like embedding a proof of existence right ?

[BrianDeery]

Is what you are describing an implementation of Timo Hanke's 2013 San Jose presentation?

https://www.youtube.com/watch?v=qwyALGlG33Q

[DeathAndTaxes]

Thanks for the replies !

So i can make a transaction with 1 input and 2 outputs where the first one is the change and the other one is data with the op_return code. Sounds great !

Is there any consensus on the fees required ? Where can i find this information updated ?

Yes that is correct 1 in 2 out (OP RETURN+ change) is the most common.  It can also be "combined" with a normal transaction to reduce cost so that would be 1 or more inputs, plus 1 or more "paying" outputs (including change) plus the OP_RETURN.  Fee shouldn't need to be more than any other transaction.  The minimum is 1,000 satoshis per KB.  If you upgrade to v0.10 of the client there is an RPC call which estimates the fee required to be quickly confirmed.  It can be used for any transaction (including those with OP_RETURN outputs).

In the past this was done by dumping the data into a "normal" output which would be unspent.  Please don't do this as the network has no way of knowing that output is "fake" and thus it remains in the UTXO set forever.  OP_RETURN allows you to create an output which is included in a block but is marked as unspendable thus not bloating the UTXO (a more critical resource often ignored by most).

Three important points to consider about OP_RETURN:
1) More than one OP_RETURN output in a transaction is invalid non-standard (and won't be relayed by most nodes).
2) A "payload" of more than 40 bytes in the OP RETURN output is invalid non-standard as well.
3) Any value assigned to the OP_RETURN output is unspendable so the "coins" are effectively destroyed.  Unless you intentionally want to destroy some coins, you should ensure the OP_RETURN outputs always have a value of zero.

Some additional info:
http://bitcoin.stackexchange.com/questions/29554/explanation-of-what-an-op-return-transaction-looks-like

On edit: Thanks Gavin for non-standard vs invalid and routine counting correction.
 

[guillaume.bonnot]

Is what you are describing an implementation of Timo Hanke's 2013 San Jose presentation?

https://www.youtube.com/watch?v=qwyALGlG33Q

This video made my day, amaze !

If you upgrade to v0.10 of the client there is an RPC call which estimates the fee required to be quickly confirmed.

Exactly the answer i was looking for. Great !

[Gavin Andresen]

Three important points to consider about OP_RETURN:
1) A transaction can only have one OP_RETURN output.  Two or more is simply invalid (not non-standard but invalid) and will be rejected by all nodes.
2) The "payload" of the OP RETURN output is limited to 40 bytes.  More than 40 bytes is invalid as well.
4) The OP_RETURN output is unspendable and thus normally should have a value of 0 BTC however be aware you can set it to any valid value and if you do so accidentally then the coins are effectively destroyed.

"Three sir!"

Actually, 1 and 2 aren't correct: the one-output and only-40-bytes checks are "what is a standard transaction" policy rules. If you can get a miner to include it in a block, a transaction with 11 100-byte OP_RETURN outputs is valid.

[dexX7]

Yes that is correct 1 in 2 out (OP RETURN+ change) is the most common.  It can also be "combined" with a normal transaction to reduce cost so that would be 1 or more inputs, plus 1 or more "paying" outputs (including change) plus the OP_RETURN.  Fee shouldn't need to be more than any other transaction.

As posted in the OP_RETURN discussion thread on GitHub, one should be aware that with the standard rules regarding OP_RETURN it might not be as efficient as embedding data in a bare multisig or P2SH transaction.

[DeathAndTaxes]

As posted in the OP_RETURN discussion thread on GitHub, one should be aware that with the standard rules regarding OP_RETURN it might not be as efficient as embedding data in a bare multisig or P2SH transaction.

True but it is generally considered poor practice to embed hundreds of bytes instead of just embedding a hash.  Personally given that one can encode data in "fake" transactions (which are not prunable) I wish there was no limit on OP_RETURN at all but that ship has sailed.  My point was that fees are per KB and that means the OP should be able to "price" the txn the same as any other txn.

[DeathAndTaxes]

the one-output and only-40-bytes checks are "what is a standard transaction" policy rules. If you can get a miner to include it in a block, a transaction with 11 100-byte OP_RETURN outputs is valid.

Corrected.  It hindsight it makes sense.  This is no more effective at spamming than large low value "normal" outputs (which are also non-standard not invalid).  Spam/bloat prevention wouldn't be improved by making one method invalid and the other non-standard. Attacker will simply use the method which is available.

Of course one can only hope non-malicious miners would see the burden of a bloated UTXO set and thus not include those types of transactions.   At least their interests are aligned with general users.

[doge94]

Three important points to consider about OP_RETURN:
1) A transaction can only have one OP_RETURN output.  Two or more is simply invalid (not non-standard but invalid) and will be rejected by all nodes.
2) The "payload" of the OP RETURN output is limited to 40 bytes.  More than 40 bytes is invalid as well.
4) The OP_RETURN output is unspendable and thus normally should have a value of 0 BTC however be aware you can set it to any valid value and if you do so accidentally then the coins are effectively destroyed.

"Three sir!"

Actually, 1 and 2 aren't correct: the one-output and only-40-bytes checks are "what is a standard transaction" policy rules. If you can get a miner to include it in a block, a transaction with 11 100-byte OP_RETURN outputs is valid.

Most nodes will not propagate a tx with 11 OP_RETURN outputs because it is a non standard transaction.
Another way to embed lots of data that will be accepted by more nodes in the network is to embed the data in a multisig output. This will clutter the UTXO though.

Alternatively you could just submit a 40 byte hash of the data in a standard OP_RETURN. This has the benefit of not messing up the UTXO set and being a standard transaction.

[dexX7]

This will clutter the UTXO though.

Not, if you include one public key that you have control over. I build http://redeem.bitwatch.co to prepare a transaction to sweep those.

[Peter Todd]

re: pay-to-contract, note how using it creates very significant and dangerous backup risks. Every pay-to-contract transaction creates a scriptPubKey that isn't derivable from the HD wallet seed, which means you need to immediately create a backup of your wallet metadata least you risk losing user funds. I strongly recommend not implementing it for this reason, without carefully considering this risk. Storing metadata in OP_RETURN outputs - as is done in stealth addresses - is an excellent way of avoiding this risk.