It is an inherent feature of a blockchain to be re-writable to some degrees , short-range chain re-writes are not a problem at all it is how consensus works in a distributed p2p network. You have always propagation delays and orphans and extreme scenarios are possible where parts of the network are isolated because of global communication disasters, there is no immediate finality feature affordable in such an environment, hence blockchains are to be re-writable and blocks are subject to orphanization by ordinary, honest competitors and/or adversaries.
actually the main feature of a blockchain based currency that makes it viable is its immutability and the fact that "re-writes" don't happen. Whether an extreme scenario happens such as a communication disaster where such things happened more is considered special cases not a regular occurrence.
also "immediate" in this context is
a couple of blocks (eg. 1 or 2) not large numbers (eg. 100)From where did you get that interpretation? If 100 looks to be too "large", suggest a more reasonable number and it will be the limit both for maturity and re-org depth cap purposes.
A miner should be ready to pay the price when his/her block becomes an orphan. The problem with the current situation is miners' ability to project this risk over innocent users who are not part of the competition and have no obligation to keep this or that chain on the top.
that's only the case for users if the replacing block was malicious otherwise stale blocks have pretty much the same transactions as the ones they are replacing so there is no risk for users.
You are getting it wrong: For reorgs (being either intentional or unintentional) shallower than maturity level (100 for bitcoin) ordinary users are not in danger, correct, but the mere possibility of a deep re-org (deeper than 100 blocks in bitcoin) implies an existential threat to innocent users who are not even the subject of a double-spending attack. Such an existential threat is what the whole 51% attack discussions in the literature is focused on because it would be easy for paranoid users or people engaged in very high-stakes transactions to wait for a limited number of confirmations (100 in bitcoin) but not forever.
Secondly, the infamous overflow bug happened in block #74638 and the new improved chain took over the wrong one in block #74691, it was just about a 53 blocks deep re-org and an exceptional incident which is not going to happen ever again even for new projects because lessons have been learned since then.
my point is that similar to this being 53 (a very large number) of blocks we can't come up with any number that doesn't have negative side effects. if it is placed at a very high number it would be useless and if it is at a low number it could be harmful without solving anything (since 51% attack in bitcoin doesn't happen due to extremely high cost).
i also wouldn't be so sure about it not happening
again.
CVE-2018-17144 that you are mentioning above was a special case and more detailed examinations revealed that even after a malicious transaction was added to the blockchain, nodes would commit to the right chain immediately after a simple reboot (because bitcoin client checks the integrity of the blockchain when it restarts). In practice, 100 blocks is good enough to cover the problem domain and I don't understand why should anybody dispute this solution:
You want finality? Wait for 100 confirmations! Otherwise, wait for as many confirmations as you find useful for your trade and meanwhile be sure about one thing: Unless you are
directly targeted by an adversary with a huge hash power, you are almost safe even with 1 confirmation.
Rather than rehashing false arguments about how useful it is to put such a cap on the depth of re-org attempts or whether it is useful at all , one should focus on the price: what the implications and consequences are?
As of the later question, because of my general approach to blockchain technology, I am more than happy with the most distinguished consequence: putting an end to the extreme individualism built into
bitcoin ideology for years!
I'm mentioning the same individualism that is the main driving force behind the slogans like '
do not trust, verify', according to this extremism, which is mainstream in the bitcoin community BTW, users should boot from the genesis block and verify both the integrity and consistency of the blockchain on one hand and the infamous longest/heaviest proposed chain rule on the other hand for themselves. It is the root of the possibility of medium to long range chain re-write attacks, for the record.
From a pure mathematical point of view, it looks to be an interesting problem: how an individual, e.g. a robot or an alien, came from nowhere could possibly boot from scratch in a wild uncertain environment full of scammers and adversaries? This is supposed to happen without having any clue about who is who in the actual business world, just the bitcoin code and a 32 bytes long hash hard-coded in it: 000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f.
Let's not be distracted by checkpoints for now. A bitcoin
puritan , already has the answer: boot from the genesis and verify the whole history as if you are travelling in the time! S/he never asks anything about the ontology of the original problem: who defined it and why, in the first place? How important or useful it is? Does it worth to pay a huge price like giving up about the finality and immutability of the blockchain?
Not every interesting mathematical question is a valuable problem or at least a practical one. In the real world, bitcoin is a social phenomenon and should be treated as such phenomenon. Extreme mathematical considerations are void and worthless and a source of confusion and impotency. IMHO, it is time to grow up and put the
bitcoin puritanism behind.