Listing all consensus rules changes

[jl2012]

I want to single list for all consensus rules changes in the history. I am referring only to those affecting block validity. Other protocol rules, e.g. isStandard, is not counted.

Terminology:
Hard-fork: an invalid block under old rules may become valid under new rules
Soft-fork: 1. a valid block under old rules may become invalid under new rules; 2. not a hard-fork

These are the rules changes I am aware of since version 0.1. All of them are soft-fork except BIP-0050

CVE-2010-5137; July 2010; v. 0.3.5; OP_LSHIFT and some other OP codes disabled *
CVE-2010-5141; July 2010; v. 0.3.5; OP_RETURN marks tx as invalid
CVE-2010-5139; Aug 2010; v. 0.3.11; fixing output value overflow
BIP-0016; Apr 2012; P2SH
BIP-0034; Mar 2013; v.2 block. block height in coinbase
BIP-0050; Mar/May 2013; v. 0.8.1; Discovery/fix of Berkeley DB "bug" (the only hard-fork in the history?)
BIP-0042; Jun 2014; v. 0.9.2; zero block reward after block 13,440,000

* Currently, CAT, SUBSTR, LEFT, RIGHT, INVERT, AND, OR, XOR, 2MUL, 2DIV, MUL, DIV, MOD, LSHIFT, RSHIFT are disabled. Were they disabled in the same fix?

Have I listed all consensus rules changes?

[Peter Todd]

You're missing a few early ones:

The max blocksize was reduced from 32MiB (an accidental limit?) to the present 1MB.

Various limits were added/lowered on PUSHDATA length, sigops, etc. (this happened multiple times)


OP_RETURN marks tx as invalid was also associated with changing the scripting system to evaluate scriptSig and scriptPubKey separately, rather than concatenated together with OP_CODESEPARATOR in the middle.

Also the Berkely DB bug is arguably not really a hard-fork as it wasn't deterministically triggerable - only most, not all, 0.7 nodes had issues with it.

[jl2012]

You're missing a few early ones:

The max blocksize was reduced from 32MiB (an accidental limit?) to the present 1MB.

Various limits were added/lowered on PUSHDATA length, sigops, etc. (this happened multiple times)


OP_RETURN marks tx as invalid was also associated with changing the scripting system to evaluate scriptSig and scriptPubKey separately, rather than concatenated together with OP_CODESEPARATOR in the middle.

Also the Berkely DB bug is arguably not really a hard-fork as it wasn't deterministically triggerable - only most, not all, 0.7 nodes had issues with it.

If most pre-0.8 nodes couldn't process our current chain, it's already a hard fork. Actually, no one is running a pre-0.8 node: https://getaddr.bitnodes.io/dashboard/

How about BIP-0030 Duplicate transactions? Is it also a soft-fork?

https://github.com/bitcoin/bips/blob/master/bip-0030.mediawiki

I'd also like to clarify the status of the 2 violations of BIP-0030. Does it mean 100BTC was permanently burnt?

[Peter Todd]

No-one is running a pre-0.8 node right now; a few months ago there were a number of them, including unpatched ones. Like I say, it's non-deterministic behavior rather than deterministic.

BIP-0030 is a soft-fork, and yes, 100BTC was permanently destroyed.

[gmaxwell]

If most pre-0.8 nodes couldn't process our current chain, it's already a hard fork. Actually, no one is running a pre-0.8 node: https://getaddr.bitnodes.io/dashboard/

You can sync up a totally unmodified 0.2.10 node, or initial release if you add a gateway to deal with the checksum on the version handshake. They're just not reliable in the face of reorgs (they'll get stuck on reorgs involving large blocks). If you drop in the one line bdb config change, they're even somewhat reliable (but SLLLLLLLLLLLLLOOOOOOOOOOWWWWWWWW).

Fwiw, as far as I know bitnodes.io filters out pre-0.8. There are actually many still running. Though presumably most of the ones that aren't stuck by now have the bdb tweak in place.

[jl2012]

If most pre-0.8 nodes couldn't process our current chain, it's already a hard fork. Actually, no one is running a pre-0.8 node: https://getaddr.bitnodes.io/dashboard/

You can sync up a totally unmodified 0.2.10 node, or initial release if you add a gateway to deal with the checksum on the version handshake. They're just not reliable in the face of reorgs and can get stuck. If you drop in the one line bdb config change, they're even somewhat reliable (but SLLLLLLLLLLLLLOOOOOOOOOOWWWWWWWW).

I think the "one line bdb config change" is already a hard fork. Anyway, it just depends on the precise definition of "hard fork". We may have one more type of forks: to turn a non-deterministic behavior into a deterministic one.

The worst fact revealed, however, is the existence of non-deterministic behavior in the rules of bitcoin, and we won't be sure if there were still other undiscovered non-deterministic rules.

There were some debates on whether we should have an RFC of bitcoin rules. If we had a RFC, all non-deterministic behaviors (like BIP50) and bugs (like BIP42) could be fixed based on the document and many political debates could be avoided. However, some argued that RFC won't be helpful because eventually only the actual source code matters. Having an RFC is just translating C++ into English, and one could never do it perfectly.

I think having an RFC, agreed by most stakeholders, is still better. The RFC is not an English transaction of C++, but a contract that all bitcoin users agreed to. Otherwise, it is always arguable that Satoshi put the indefinite monetary supply there intentionally and BIP42 is actually violating the "real" spirit of bitcoin.

[jl2012]

Talking about RFC, I think Peter will use the SIGHASH_SINGLE example again ( https://bitcointalk.org/index.php?topic=260595.0 ). Yes, there could be many undiscovered bugs in the code. However, as a start, the RFC may just mention some fundamental rules, such as the block reward schedule and structure of block headers. As we learn more and more about the code, we could then document more sensitive parts of it.

[gmaxwell]

I think the "one line bdb config change" is already a hard fork. Anyway, it just depends on the precise definition of "hard fork". We may have one more type of forks: to turn a non-deterministic behavior into a deterministic one.

I agree that the config change is a hard fork, but the point of my post was to point out that those old versions do synchronize okay, they're just unreliable on reorgs involving large blocks depending on the phase of the moon (really how many BDB pages the transactions being modified span).

The worst fact revealed, however, is the existence of non-deterministic behavior in the rules of bitcoin, and we won't be sure if there were still other undiscovered non-deterministic rules.

That there was non determinism wasn't a surprise. It's not even the first non-determinism, the handling of rewrites on reorgs for duplicate txids was also non-deterministic (behavior depended on which order the node saw a series of forks), which was why it had to be fixed. Absent a formal proof you can't be sure software is bug free— or even with one, considering the bugs that have been found in compcert.

and many political debates could be avoided.

There has never been a single political debate about buggy behavior like that, and I never expect there to be one.  Non-determinism in a consensus system is a fatal error, no one has any impression that it ought to stay. A document also will not reliably resolve non-deterministic behavior, since the behavior that would have been non-deterministic would simply have not been specified in the document.

[jl2012]

and many political debates could be avoided.

There has never been a single political debate about buggy behavior like that, and I never expect there to be one.  Non-determinism in a consensus system is a fatal error, no one has any impression that it ought to stay. A document also will not reliably resolve non-deterministic behavior, since the behavior that would have been non-deterministic would simply have not been specified in the document.

As I remember, in the BIP50 fork, Gavin first thought we should follow the longest chain, and Luke suggested to orphan the longest chain. I am not criticizing the decision, but this decision made the double spend against OKPay possible. If we had a document, I think the most natural decision was to follow the longest fork, as it apparently had nothing wrong.

[gmaxwell]

As I remember, in the BIP50 fork, Gavin first thought we should follow the longest chain, and Luke suggested to orphan the longest chain.

As someone who was there and involved in dealing with it I don't recall any _dispute_, I'm really unhappy about the history revisionism people throw around hwere— it was obvious what needed to be done: there was a chain that was acceptable to all nodes, and a chain that was acceptable to a small minority (which, on quick testing didn't include many major businesses)— because of the consolidation of hash-power around pools the mining deployed software looked very much unlike the systems overall, but even if it had been a slim majority only one was acceptable to all nodes.  Being acceptable to all nodes meant that the issue could be healed back to consensus immediately— as soon as the acceptable-to-all chain was also the longest, the other side would have meant weeks of split consensus as people gradually fixed their software.

[jl2012]

As I remember, in the BIP50 fork, Gavin first thought we should follow the longest chain, and Luke suggested to orphan the longest chain.

As someone who was there and involved in dealing with it I don't recall any _dispute_, I'm really unhappy about the history revisionism people throw around hwere— it was obvious what needed to be done: there was a chain that was acceptable to all nodes, and a chain that was acceptable to a small minority (which, on quick testing didn't include many major businesses)— because of the consolidation of hash-power around pools the mining deployed software looked very much unlike the systems overall, but even if it had been a slim majority only one was acceptable to all nodes.  Being acceptable to all nodes meant that the issue could be healed back to consensus immediately— as soon as the acceptable-to-all chain was also the longest, the other side would have meant weeks of split consensus as people gradually fixed their software.

I am not saying that there were any dispute. At least we may have some SOP: in case there is an accidental fork, we prefer the more restrictive one until a better solution is available. I hope this won't eventually make the rules too restrictive.