Removing OP_return limits seems like a huge mistake

[Wind_FURY]

What would be the best/most efficient way to allow dick pics and fart sounds/embedding data on-chain, giving a cleaner blockchain state and with minimal blockchain bloat?

Just use commitments. In this way, you can hide any data, behind a given signature. Then, on-chain transaction looks exactly the same, as if there would be no committed data, so they cannot be censored by Knots, or any other client, because there are no on-chain fields, which are different from regular payments in that case.

Example from the mailing list, how it can be done: https://gnusha.org/pi/bitcoindev/ZVcdKupXU+wjawRI@erisian.com.au/

Code:

Sign-to-contract looks like:

 * generate a secret random nonce r0
 * calculate the public version R0 = r0*G
 * calculate a derived nonce r = r0 + SHA256(R0, data), where "data"
   is what you want to commit to
 * generate your signature using public nonce R=r*G as usual

And if spammers will keep pushing data on-chain anyway, then the source code can be changed, for example to make more lightweight nodes, which will store commitments to historical transactions, instead of storing their content. Which means, that commitments can be enforced on all historical transactions, if needed. I think it is only a matter of making enough spam, because when node operators will start running out of resources, then developers will be forced to implement such changes, or stop using Bitcoin at all, if their machines will be unable to keep handling all historical data.

I expect enforcing commitments on consensus level will be needed sooner or later anyway, because it doesn't matter that much, if blocks are filled with huge OP_NOPs, huge OP_RETURNs, or if blocks are full of regular transactions. As long as new nodes are forced to download hundreds of gigabytes, there is a huge risk, that we won't have enough full nodes in the future, which means that the option to synchronize the full chain, without downloading everything, is needed (and will be needed, because of the constant growth of the chain).

OK, then why aren't the developers pushing for these solutions? Or is the real motive is actually to enable "features" on-chain to encourage more on-chain activity, such as dick pics and fart sounds, to "fabricate" more demand for block space and make it more sustainable for miners to be incentivized for securing the network?

  ¯\_(ツ)_/¯

[stwenhao]

OK, then why aren't the developers pushing for these solutions?

Because writing good code is hard, because it takes time, and because people are more interested in improving regular transactions, than they are in developing tools for JPEG users. Bitcoin is about sending money from one address to another. Even if it is technically possible to send money and commit to some JPEG, without making your transaction bigger, then many developers are not interested in developing such things. Also because spammers are going to use the worst ways anyway, and won't switch to better solutions, because their goal is not to handle JPEGs, but to spam the network.

Also, in the Open Source world, people develop things, which they are interested in. You cannot force a developer to make tools they don't want to make. And many of them simply don't care, because as long as people are focused on JPEGs, they don't try much worse attacks, and other changes can be made peacefully and quietly, as long as nobody is looking at it.

Or is the real motive is actually to enable "features" on-chain to encourage more on-chain activity, such as dick pics and fart sounds, to "fabricate" more demand for block space and make it more sustainable for miners to be incentivized for securing the network?

No, it is much simpler than that: each soft-fork takes a lot of time and effort to deploy. When Taproot was introduced, some limits were lifted, to make it possible to deploy some future protocols, without making yet another soft-fork. Many developers assumed, that abusing some limits is hard enough, that nobody will write the code, which will do that.

And quite often, it is the case. If Casey Rodarmor wouldn't share Ordinals, then we wouldn't have so much on-chain spam. If Garlo Nicon wouldn't deploy CPU miners for testnet, the minimal difficulty rule could be left unmodified, if only devs would mine single CPU blocks from time to time. There are many skilled people, who shared some code, and made it possible to abuse the network, because their code changes didn't have any protections from being abused.

And here, it is a similar thing: the maximum size of the block is the only thing, which protects us from seeing how Bitcoin is turned into Bloatcoin. And as long as regular transactions can still be confirmed, and existing nodes can handle all of that, nobody cares enough. Many code changes are driven by anger: if things stop working, then developers are focused on fixing the problem, because then, it is the only way to make the system usable again. But as long as it is not the case, then people can peacefully work on more exciting problems than JPEGs, because the world is not burning yet.

[BayAreaCoins]

If Garlo Nicon wouldn't deploy CPU miners for testnet, the minimal difficulty rule could be left unmodified

I don't think it is Garlo Nicon ftr.

[stwenhao]

I don't think it is Garlo Nicon ftr.

Example modification in Bitcoin Core, that could help mining blocks on CPU:

Code:

$ git diff 86fea43762 5ffd95377b
diff --git a/src/chain.h b/src/chain.h
index bb70dbd8bc..30d5d48d5d 100644
--- a/src/chain.h
+++ b/src/chain.h
@@ -26,7 +26,7 @@
  * Maximum amount of time that a block timestamp is allowed to exceed the
  * current time before the block will be accepted.
  */
-static constexpr int64_t MAX_FUTURE_BLOCK_TIME = 2 * 60 * 60;
+static constexpr int64_t MAX_FUTURE_BLOCK_TIME = 20 * 60 * 60;
 
 /**
  * Timestamp window used as a grace period by code that compares external
diff --git a/src/node/miner.cpp b/src/node/miner.cpp
index 87f40e993f..e83d3bde27 100644
--- a/src/node/miner.cpp
+++ b/src/node/miner.cpp
@@ -31,7 +31,7 @@ namespace node {
 int64_t UpdateTime(CBlockHeader* pblock, const Consensus::Params& consensusParams, const CBlockIndex* pindexPrev)
 {
     int64_t nOldTime = pblock->nTime;
-    int64_t nNewTime{std::max<int64_t>(pindexPrev->GetMedianTimePast() + 1, TicksSinceEpoch<std::chrono::seconds>(NodeClock::now()))};
+    int64_t nNewTime{std::max<int64_t>(pindexPrev->GetMedianTimePast() + 1, pindexPrev->GetBlockTime() + consensusParams.nPowTargetSpacing*2 + 1)};
 
     if (nOldTime < nNewTime) {
         pblock->nTime = nNewTime;
$ 

He shared needed changes on bitcointalk, as you can see above. And he informed earlier, that he can mine testnet blocks on CPU, when other people didn't believe it is possible:

Just for your information: no, you can't...

Sure, but my node, and some block explorers think otherwise:

https://mempool.space/testnet/address/tb1qvy67actlsslmvwpjzk9hnx6jf04g5y26sz5crx
https://mempool.space/testnet4/address/tb1qvy67actlsslmvwpjzk9hnx6jf04g5y26sz5crx

Public key: 0261c2c04c8133b863d9df2ef4082c21074f8849ef86ddf54b13e07d6c828faac6
Legacy address: mpNxT11hJpGH2YhUgQKZm2S1rBnnieWbHr
Segwit address: tb1qvy67actlsslmvwpjzk9hnx6jf04g5y26sz5crx

Code:

verifymessage "mpNxT11hJpGH2YhUgQKZm2S1rBnnieWbHr" "IORhFfwQl/VK8NU6sQrRxQ6pqoGjHTElUirU+H0lrsyPKE+cOYUZ3TNx3JF8WM69TvhfwZ9iafDk2f6WK1tZZMg=" "June 10, 2024, 04:13:45 PM: I am Garlo Nicon, and I confirm in topic 5499150 on bitcointalk, that I can mine new blocks in testnet3 and testnet4, by using address tb1qvy67actlsslmvwpjzk9hnx6jf04g5y26sz5crx. All of those blocks have minimal difficulty, equal to one, and were mined on my own CPU."
true

See? Many users were skeptical, and thought, that you need ASICs to mine testnet blocks. At that time, developers and skilled users mined a lot of test blocks on CPU, while the rest of the network was trying hard to mine ASIC blocks on top of them, which only confirmed developers' blocks, and set them in stone.

Of course, later, he stopped mining testnets, when people with ASICs started copy-pasting his changes, and recompiling Bitcoin Core, but until that moment, many blocks were mined on CPU by him, and his friends (and probably sold later on your exchange). You can try to check historical data, and see, how much percentage of testnet blocks were produced on CPUs by him, and his friends, at that point in time. Also, it was the first moment, when testnet difficulty started to rise abnormally: before June 2024, it was much more accurate, because more blocks were produced by ASICs.

[Wind_FURY]

OK, then why aren't the developers pushing for these solutions?

Because writing good code is hard, because it takes time, and because people are more interested in improving regular transactions, than they are in developing tools for JPEG users. Bitcoin is about sending money from one address to another. Even if it is technically possible to send money and commit to some JPEG, without making your transaction bigger, then many developers are not interested in developing such things.

Why? It's in the interest of the whole network.

There's probably a technical reason?

Also because spammers are going to use the worst ways anyway, and won't switch to better solutions, because their goal is not to handle JPEGs, but to spam the network.

Also, in the Open Source world, people develop things, which they are interested in. You cannot force a developer to make tools they don't want to make. And many of them simply don't care, because as long as people are focused on JPEGs, they don't try much worse attacks, and other changes can be made peacefully and quietly, as long as nobody is looking at it.

Debatable. I don't know how you define "spammers", but dick pics and fart sound lovers would definitely use the method that would give them more fee-savings.

Or is the real motive is actually to enable "features" on-chain to encourage more on-chain activity, such as dick pics and fart sounds, to "fabricate" more demand for block space and make it more sustainable for miners to be incentivized for securing the network?

No, it is much simpler than that: each soft-fork takes a lot of time and effort to deploy. When Taproot was introduced, some limits were lifted, to make it possible to deploy some future protocols, without making yet another soft-fork. Many developers assumed, that abusing some limits is hard enough, that nobody will write the code, which will do that.

And quite often, it is the case. If Casey Rodarmor wouldn't share Ordinals, then we wouldn't have so much on-chain spam. If Garlo Nicon wouldn't deploy CPU miners for testnet, the minimal difficulty rule could be left unmodified, if only devs would mine single CPU blocks from time to time. There are many skilled people, who shared some code, and made it possible to abuse the network, because their code changes didn't have any protections from being abused.

And here, it is a similar thing: the maximum size of the block is the only thing, which protects us from seeing how Bitcoin is turned into Bloatcoin. And as long as regular transactions can still be confirmed, and existing nodes can handle all of that, nobody cares enough. Many code changes are driven by anger: if things stop working, then developers are focused on fixing the problem, because then, it is the only way to make the system usable again. But as long as it is not the case, then people can peacefully work on more exciting problems than JPEGs, because the world is not burning yet.

 🤔

What positive use cases would we have if indeed OP_RETURN limits were removed?

[stwenhao]

Why? It's in the interest of the whole network.

There's probably a technical reason?

No. It is related to the Open Source world in general. People are implementing what they want to implement, and they are often unpaid, and spend their free time to code things they want to code. Why should they waste time on bringing NFT-friendly features, when the whole Script can be disabled during transition to quantum-resistant addresses? And now, one of the criterias, when picking quantum algorithms, is JPEG resistance. In general, it is easier to focus on regular payments first, let NFTs use whatever they want (because spammers are going to spam, as you can see, when they put ASCII hex characters inside OP_RETURN, instead of at least packing things in binary, and making it 50% smaller).

What positive use cases would we have if indeed OP_RETURN limits were removed?

For example it is cheaper, and it takes less bytes, when you use OP_RETURN for some particular messages. And in general, by having just OP_RETURN, you don't need any Segwit or Taproot envelope. I don't remember exactly, but for something up to 150 bytes, OP_RETURN is cheaper, and then, after making a message bigger than that, Taproot becomes cheaper, because of Segwit discount. Which means, that if OP_RETURN limit would be raised, then it would be more aligned with fees (if making non-standard transaction is cheaper, than making a standard one, then often something is wrong, because then, miners have an incentive to accept non-standard transactions on-chain, which reflects badly on future upgrades).

Also, if people use OP_RETURN, instead of Taproot, then the block size limit is up to 1 MB, and not 4 MB.

Debatable. I don't know how you define "spammers", but dick pics and fart sound lovers would definitely use the method that would give them more fee-savings.

Quite often, people don't want to store their data by themselves. They only want to delegate that job to other nodes. And things can be changed, to make it harder, if the responsibility to keep information about UTXOs will be shifted from nodes to users. Then, during spending, more conditions will be required (for example proving, that a given coin is part of the UTXO set), which would allow full nodes to store less information, and share less information during Initial Blockchain Download.

Also, as you can see, Anthony Towns shared technical information about commitments in November 2023. And, as you can see, Casey Rodarmor, or someone else outside of Bitcoin Core developers didn't implement it in wallets yet, even though the concept is well-known in the community, and was well-known long before 2023.

And if you think, that people are not trying to spam, then tell me: why they use text-based protocols like JSON, instead of using more packed formats, and saving just >50% on fees? Or why they are sending 99% of the coin amount as fees? Spammers simply don't care to make it cheap, because otherwise, they wouldn't use dust amounts so heavily, that it triggered discussion about UTXO expiration.

[d5000]

To @stwenhao's excellent post I only wanted to add one aspect which often is overlooked.

OP_RETURN is always "cheaper" for the whole system than the Taproot envelope, i.e. it consumes less ressources. This is because the Taproot envelope always takes more bytes than the same data in an OP_RETURN output because the Taproot envelope needs an additional output from another transaction for the Taproot script hash, which often results in needing a complete additional transaction, but even if you add the hash to a transaction you would have done anyway (e.g. make a payment and add the hash as an additional output, which I believe almost nobody does) then the script hash adds (afaik) 32 bytes which would not have been necessary if you used an OP_RETURN transaction.

This means: For the full nodes, OP_RETURN is always, even if you have a NFT with 4 MB, the least resource-consuming option. The Taproot method is only "cheaper" regarding fees, due to the witness discount. (Of course this applies when all transactions are generated optimally, without additional unnecessary outputs.)

In addition: One of the reasons why the witness discount (data in the witness has 4x less weight) was applied is the idea that witness data can be aggregated in the future, making blocks smaller. This is true for some other kind of witnesses like those containing Schnorr signatures. But Taproot "envelopes" used for data storage impede the aggregation if I'm not missing something. Imagine a block with 2000 transactions, for 1800 of them perhaps the signatures can be aggregated and compressed to <1% of their size, but 200 "Taproot envelopes" would have to stay as they are. This would mean that the Taproot envelopes, even if small (like in BRC-20 or similar formats) would take a completely disproportionate amount of space in the witnesses.

So Taproot envelopes "get in the way" of this specific scaling idea. That's not true for OP_RETURN because there the data resides in outputs.

[Wind_FURY]

To @stwenhao's excellent post I only wanted to add one aspect which often is overlooked.

OP_RETURN is always "cheaper" for the whole system than the Taproot envelope, i.e. it consumes less ressources. This is because the Taproot envelope always takes more bytes than the same data in an OP_RETURN output because the Taproot envelope needs an additional output from another transaction for the Taproot script hash, which often results in needing a complete additional transaction, but even if you add the hash to a transaction you would have done anyway (e.g. make a payment and add the hash as an additional output, which I believe almost nobody does) then the script hash adds (afaik) 32 bytes which would not have been necessary if you used an OP_RETURN transaction.

This means: For the full nodes, OP_RETURN is always, even if you have a NFT with 4 MB, the least resource-consuming option. The Taproot method is only "cheaper" regarding fees, due to the witness discount. (Of course this applies when all transactions are generated optimally, without additional unnecessary outputs.)

In addition: One of the reasons why the witness discount (data in the witness has 4x less weight) was applied is the idea that witness data can be aggregated in the future, making blocks smaller. This is true for some other kind of witnesses like those containing Schnorr signatures. But Taproot "envelopes" used for data storage impede the aggregation if I'm not missing something. Imagine a block with 2000 transactions, for 1800 of them perhaps the signatures can be aggregated and compressed to <1% of their size, but 200 "Taproot envelopes" would have to stay as they are. This would mean that the Taproot envelopes, even if small (like in BRC-20 or similar formats) would take a completely disproportionate amount of space in the witnesses.

So Taproot envelopes "get in the way" of this specific scaling idea. That's not true for OP_RETURN because there the data resides in outputs.

THANK YOU! It currently makes SO MUCH sense now. I was waiting for someone smarter than a mere pleb like me write a VERY clear and honest post about OP_RETURN.

OK, with all things having been discussed and all options having been considered, removing OP_RETURN limits is the best option for the future of dick pics and fart sounds in the Bitcoin blockchain.

[achow101]

In addition: One of the reasons why the witness discount (data in the witness has 4x less weight) was applied is the idea that witness data can be aggregated in the future, making blocks smaller.

If you're talking about signature aggregation, that would require another format change, so usage within the current witness data wouldn't really make a difference.

[d5000]

If you're talking about signature aggregation, that would require another format change, so usage within the current witness data wouldn't really make a difference.

Thanks for that clarification.

The argument I made in the previous post was more directed to the future scenario when that format change had already happened making such aggregation techniques possible. In that situation, if I understand it correctly, the disproportionate weight of (new) arbitrary data in Taproot envelopes would become evident: We could say that the witness discount then for "normal" signatures/public keys would be justified because they then occupy much less space, but the Taproot envelopes are "cheating" in some way as they can't be aggregated.

Using OP_RETURN would not save additional space in that aggregation scenario, but as the data has no witness discount it would also not be "cheating" in this fashion. (And large Taproot envelopes do need some additional bytes due to the push opcodes necessary to "extend" the data field).

[gmaxwell]

The weight limit counts stuff that goes into the UTXO set more than it counts the rest. It doesn't have anything to do with aggregations or discounts or whatever--  it's just the limit on what can be in a block recognizes that UTXO data takes more resources.  Neither signature data nor op-return stuff goes in the utxo.   In all fairness the up-weighing ought to be much higher but there is a tradeoff vs how much bandwidth people can cause if they make unusual transactions with more witness data than typical... and a subset of users are more bandwidth limited than memory/io/etc. limited.

Were it not for compatibility reasons scriptsigs, the txin fields, etc. would also be counted as witness data.