What are checkpoints in bitcoin code?

[kjj]

Do you understand that this option is protocol specification change? It effectively forks blockchain.

Heh.  This is the sort of nonsense I was talking about.

This is local control.  You, the node operator, have options.  Hardly a new thing.  Since day 1, all node operators have had the ability to fork for whatever reason and through whatever means they desire.

You have a warped view of "decentralized".  It doesn't mean that no one has any authority, it means that no one can stop you.

Who can stop you from turning checkpoints off?  No one.
Who can stop you from stripping them out of the source entirely and running your own pure node?  No one.

[TierNolan]

I should look closer to your proposal.

It's not my idea, the OP in that thread thought of it.

For now I have one question:
Does "up" link points to future? If so - is it included in block hash or can it be falsified?

No, they always points backwards.

To determine where to point the up link.

- count the number of leading zeros on the previous block's hash (i.e. the block you plan to build on top of)
- starting from the block before that, find the first block with more leading zeros
- point to the block one after that

So, if the blocks had leading zeros

A: 11(up = Genesis)
B: 10 (up = Genesis)
C: 20 (up = B)   (Not both prev and up point at B)
D: 10 (up = Genesis)
E: 10 (up = D)
F: 10 (up = D)
G: 10 (up = D)
H: 15 (up = D)
I: 10 (up = D)
J: 10 (up = I)
K: 10 (up = I)

The for block L, the links would be

prev: K
up: I

The reason is that H has more leading zeros than K.  You don't link to C, even though it has more leading zeros.  Once you find that block, the up link points to the one after that.

So, finding the best block

stating at K, follow up link, until up link points to genesis
- follow up link to get to I (prev now point at H, so found block with 15 leading zeros)
- follow up link to get to D (prev now points at C, so found block with 20 leading zeros)
- up for D link points to genesis, so break from loop

follow D's previous link to get C which is the best block

You can also find all blocks with more than n leading zeros by following the previous link any time you find a block with more than that number of leading zeros.

[Nite69]

I'm quite sure this has been discussed before, but why clients does not make automatic checkpoints? I understand that transactions which are at least 6 blocks old, are considered valid. There should be no reason to re-arrange the blockchain after that, for any reason. If 6 blocks sounds too short, maybe 120 blocks then? Coinbase coins should definitely be stoned when they are spendable.

So I'd like to hear what is the reasoning behind the fact that there are no automatic checkpoints?

Let's *try* to imagine situations where automatic checkpoint (here I mean every node considers it chain's blocks from head-6 and earlier to be 'stoned', non-changeable) would cause harm:

1. Branch which will survive 6 blocks. Isn't this practically an error case? If it can happen, that just means 6 block validation time is too short since it cannot guarantee transactions confirmed.
2. Some node mines isolated from the others. This is the case where it would cause minor problem. However the problem is very local, and very similar problem would be regardless of the automatic checkpoint.

Basicly it would just be a non-centralised agreement between nodes after they find a new head block; we all consider blocks backwards from this-6 nodes to be immutable.

[DeathAndTaxes]

The main issue is that in the event of a flaw which causes the network to fork you have now hard coded that fork.

For example if the db bug issue which caused a fork in the network, miners on the longer fork modified their blockchains to support the shorter fork.  So some intervention was required but for all 100,000+ other nodes they simply saw a re-org once the the shorter fork became the longer one.  If they used ratcheting checkpoints they would have been permanently locked onto a minority fork and attackers could have exploited that.

There may be some merit in using a ratcheting checkpoint system however it would have to be set much much deeper than 6 blocks.  Even 144 blocks (one day) may be insufficient.  Something like 1,024 blocks (~1 week) is more the range that would be conservative enough.

Also understand a ratcheing checkpoint system only helps existing nodes.  Bootstraping nodes would be remain vulnerable as they haven't seen the longest chain yet.

[Nite69]

The main issue is that in the event of a flaw which causes the network to fork you have now hard coded that fork.

For example if the db bug issue which caused a fork in the network, miners on the longer fork modified their blockchains to support the shorter fork.  So some intervention was required but for all 100,000+ other nodes they simply saw a re-org once the the shorter fork became the longer one.  If they used ratcheting checkpoints they would have been permanently locked onto a minority fork and attackers could have exploited that.

There may be some merit in using a ratcheting checkpoint system however it would have to be set much much deeper than 6 blocks.  Even 144 blocks (one day) may be insufficient.  Something like 1,024 blocks (~1 week) is more the range that would be conservative enough.

Also understand a ratcheing checkpoint system only helps existing nodes.  Bootstraping nodes would be remain vulnerable as they haven't seen the longest chain yet.

Do you mean this:
https://bitcoin.org/en/alert/2013-03-11-chain-fork

But it would have been solved simply by bootstrapping the blockchain along with the software downgrade. If it needs software update, you allways have the option to force block reload, maybe from some hardcoded checkpoint.

[DeathAndTaxes]

But it would have been solved simply by bootstrapping the blockchain along with the software downgrade. If it needs software update, you allways have the option to force block reload, maybe from some hardcoded checkpoint.

That is my point and until 100% of nodes upgrade their software they would be vulnerable to an isolation attack.  It creates a more fragile and exploitable system than users wallets simply being able to switch to the longest chain with no action required on the part of the user.

[kjj]

The main issue is that in the event of a flaw which causes the network to fork you have now hard coded that fork.

For example if the db bug issue which caused a fork in the network, miners on the longer fork modified their blockchains to support the shorter fork.  So some intervention was required but for all 100,000+ other nodes they simply saw a re-org once the the shorter fork became the longer one.  If they used ratcheting checkpoints they would have been permanently locked onto a minority fork and attackers could have exploited that.

There may be some merit in using a ratcheting checkpoint system however it would have to be set much much deeper than 6 blocks.  Even 144 blocks (one day) may be insufficient.  Something like 1,024 blocks (~1 week) is more the range that would be conservative enough.

Also understand a ratcheing checkpoint system only helps existing nodes.  Bootstraping nodes would be remain vulnerable as they haven't seen the longest chain yet.

Do you mean this:
https://bitcoin.org/en/alert/2013-03-11-chain-fork

But it would have been solved simply by bootstrapping the blockchain along with the software downgrade. If it needs software update, you allways have the option to force block reload, maybe from some hardcoded checkpoint.

In that case, whoever controls the update system controls everything.

[ning]

who makes the checkpoints?

The bitcoin core devs

so ultimately, the state of the block chain depends entirely on them.

not really. they only put checkpoint for blocks that are buried at least few hundreds deep already - unlikely to be reverted by honest means.
but if you disagree with any checkpoint they add, just don't upgrade your node and you will be fine.

well that's a practical solution, but it violates the notion of 'peer-to-peer' or equality for all nodes in the network.

there is no indication of 'checkpoints' in the satoshi whitepaper.

furthermore, if we rely on such checkpoints for security, why do we even have proof of work at all?  Why don't we just rely on the checkpoints?

Being peer-to-peer also means that everyone in the network can choose to use whatever client or whatever version of a certain client in order to be part of the Bitcoin network. As of now, we don't have too many options for running a full node, but in the future, more independent implementations will likely to emerge.

[somic]

amazing checkpoint,still cant quitely follow

[Nite69]

I'm quite sure this has been discussed before, but why clients does not make automatic checkpoints? I understand that transactions which are at least 6 blocks old, are considered valid. There should be no reason to re-arrange the blockchain after that, for any reason. If 6 blocks sounds too short, maybe 120 blocks then? Coinbase coins should definitely be stoned when they are spendable.

I still consider that since coinbase transactions are agreed by the network to be spendable, that means they are agreed to be existent to foreseeable future and thus should be checkpointed.

How about following rule:
when a block is generated, the block 120 blocks deep the chain is checkpointed, *if*
- current hashing power is more than half of the to-be-checkpointed block

Not sure if this checkpointing even need to be broadcasted; every client should be in full agreement of it.

Then, I also suggest that coinbase transactions *are not spendable* unless they are checkpointed.

Does this guarantee solving problems with forked chains? Only the chain is checkpointed which has majority of the hashing power.
Still thinking good formula to estimate if the hashing power is more than half.. both blocks difficulty and time should be considered. And maybe some safety reserve, so variations on block times does not affect the result.

Does this open any attack vectors?


Edit: when a new soft checkpoint is made, the previous one can be forgotten, ie there need not to be more than one soft checkpoint at a time.

[TierNolan]

How about following rule:
when a block is generated, the block 120 blocks deep the chain is checkpointed, *if*
- current hashing power is more than half of the to-be-checkpointed block

How do you measure current hashing power?

Do you mean difficulty must be at least 50% of what it was 120 blocks ago?  That isn't much protection since difficulty doesn't change very often.

Does this guarantee solving problems with forked chains? Only the chain is checkpointed which has majority of the hashing power.

The problem with checkpointing is what happens if there is a reversal.

For example, if I launch a 51% attack starting from behind the checkpoint, then all new nodes will follow my chain.

Nodes that were offline will also follow my chain.

Nodes that were online would follow the other chain.

This forks the network and it can't heal itself.

I think sending the entire header tree would help here.  A new node that connects would see that there was a 120+ block fork.  It could warn the user that the chain is potentially unstable at the moment.

Nodes would be able to see that there was a fork that would have caused a checkpoint.

Still thinking good formula to estimate if the hashing power is more than half.. both blocks difficulty and time should be considered. And maybe some safety reserve, so variations on block times does not affect the result.

Including block time is risky, since it creates an incentive to mess with the field.  An attacker could set it to whatever he likes (but not to far in the future).

Faking high hash rate means earlier block times, so the future rule doesn't apply.  If I produce a sequence of blocks with timestamps 1 second apart, it will seem like a massive hash rate.

Edit: when a new soft checkpoint is made, the previous one can be forgotten, ie there need not to be more than one soft checkpoint at a time.

If they are both on the same chain, then keeping the old one has no effect one way or another.

What do you mean by soft?  The rule could be that a fork 120 blocks longer can displace a checkpoint.  This would at least allow the network to re-sync.

Clients could enter emergency mode using this rule.

- find longest chain (L)
- find any other chain which
-- forked more than 120 blocks ago
-- where the fork is at least 120 blocks long
-- has a weight of at least (L's weight - 120 blocks)

The hard checkpoints also speed up validation.  They could do that without acting as checkpoints.

Validation would start at the last hard checkpoint that is on the longest chain.

If the entire chain was displaced by a higher POW chain, then nodes would accept the new chain.  They would have to verify it in its entirety.

[kklingo65]

who makes the checkpoints?

The bitcoin core devs

so ultimately, the state of the block chain depends entirely on them.

not really. they only put checkpoint for blocks that are buried at least few hundreds deep already - unlikely to be reverted by honest means.
but if you disagree with any checkpoint they add, just don't upgrade your node and you will be fine.

well that's a practical solution, but it violates the notion of 'peer-to-peer' or equality for all nodes in the network.

there is no indication of 'checkpoints' in the satoshi whitepaper.

furthermore, if we rely on such checkpoints for security, why do we even have proof of work at all?  Why don't we just rely on the checkpoints?

I'm having a hard time figuring out what you mean by "equality of all nodes".  The equality (or inequality) of nodes is not part of the system.  No one cares about nodes.  We care about the chain.  If a node has the correct chain and can give you a copy, it is "equal" to all of the other good nodes.  If it doesn't, or can't, then it equal at all, isn't even a node.

The checkpoints do not provide any security, nor were they ever intended to.  They just keep "unequal" nodes from wasting your time with bogus chains (whether by accident or malice).

Also, you are free to edit and recompile as needed.  You can strip them out entirely, or add your own, or change them.  Whatever you want.  The reference client is written for (and compiled for) working well for a wide audience.

Technically, a longer chain is the best chain, regardless of the checkpoints.  In practice, the checkpoints are far enough in the past that if a new chain showed up that overturned one, we'd almost universally consider it an attack, and roll back.

P.S.  No one particularly likes the checkpoints, the devs least of all.  There has been much discussion about removing them, or switching to a soft system (config file, etc).  So far, no one has come up with a really good solution.

I'm having a hard time figuring out why checkpoints do not provide any security ....

[TierNolan]

I'm having a hard time figuring out why checkpoints do not provide any security ....

They have positives and negatives.

A checkpoint means that miners won't jump to a secret fork.

Miners who didn't checkpoint will still jump and so will some clients if they were offline when the checkpoint triggered.

This means you split the network into 2 pieces if a major fork happens.