Far better to just get rid of them: Headers first makes most reasons obsolete.  The circus above doesn't really help, since it's using the chain itself, which of course checkpoints distort the selection of, so it's just circular.

Old checkpoints distort the selection of the chain, but there's no reason new checkpoints can't be done with network consensus on chain long after a previous checkpoint (thus its more bootstrapping than circular). It's potentially a powerful new way to compress the history and bring new nodes up to speed fast, especially since you can include a utxo patricia tree hash in there too.

From what I understand, headers first doesn't affect the new full node sync time at all. Please correct me if I'm wrong

You do have an existing blockchain. The bitcoin one, up to now. And you can tell time in number of blocks. The genesis block was the first checkpoint. We could have the hashing power vote to checkpoint block 10,000, including the patricia tree hash of the utxo set up to that point. Then anything from before the checkpoint can be ignored, since the checkpoint can be considered part of the PoW consensus mechanism - if you trust the PoW generally to make ledger updates, then (conceivably) you can trust it to checkpoint.


Exactly the point. Node's already trust that blocks are valid because they have PoW on them. The checkpoint will have PoW too, and hence be trusted in the same way, relieving the new client from having to validate anything before the checkpoint. That's the point, it's like a new genesis block, plus a utxo set.

Good point. Parallel downloading is awesome. But the CPU still has to crunch ALL those EC verifies. ::sigh::

The point is, everyone trusts the genesis block, and all updates from there are made via pow. This is essentially a proposal for a consensus process to update the genesis block forward, and to attach a tree of utxos so that the txs between the new gen block and the old never need to be seen again (we can put them in a museum of bitcoin history, if you like, but spare the new full nodes!). I still have not heard a reasonable argument as to why this can't/won't work.

No it doesn't. It can skip verifying very deeply burred signatures (as is the checkpoint behaviour), since with headers first it knows the amount of work on top of them and can perform the tests only probabilistically past a certain point.

Pieter and I-- at least-- have been talking about it for a while as part of the motivation for headers first, even prior to the fraud proofs: past some limit (E.g. max(60days work at the highest difficulty ever observed in the best chain, 2016 blocks) ) a huge reorganization at tip is a guaranteed system failure. (Once you cross 100 blocks a reorg starts forever invalidating an exponentially expanding cone of transactions).   For a while this wasn't as interesting because the total time to replace the chain with the tips hashrate was very low, but it's finally expanding nicely: http://bitcoin.sipa.be/powdays-50k.png

Another potential softer safety mechanism is simply holding the confirmation count in the RPC at zero when there has been a 'system failure grade' reorganization to allow time for a "higher process" to sort out the mess. This also needs a couple other pieces to be completely useful, like the ability to manually invalidate a block over the RPC.

First priority was getting headers first in, tested, and mature.

Obviously fraud proofs have been on my mind for a long time, but they have a lot of protocol surface area. The recent reorganizations of Bitcoin core (e.g. libscript work) will make it easier to work on them with confidence, e.g. be able to build an external process that receives and checks fraud proofs.

It's an arbitrary big number which is hopefully macroscopic relative to whatever external process you'd hope to rescue the system if it failed, but hopefully still small enough to give a good speedup on the checking.

I only used a specific number because thats an example of the kind of numbers that were banded about before... even given pretty healthy hashrate growth far in excess of computer industry historic trends numbers in that space should still be adequate to keep downloads bandwidth bounded instead of cpu bounded. (Also keep in mind that libsecp256k1 is >6x faster than openssl).

Yes, for many months: http://bitcoin.sipa.be/growth.png

Yep.

Complexity by itself isn't really the engineering constraint (passing a message is no big deal, in any case) the concern would be complexity coupled with risk. If there is something complex but its strongly isolated you can reason about what kind of things can happen if it goes wrong. E.g. fraud proofs can initially run over a separate protocol, communicate to an isolated sandboxed process, and even if they've massively gone wrong the threat is likely limited to a nuisance of making all your transactions show as unconfirmed, rather than splitting the consensus or stealing keys or what have you.  See Matt's relaynode network client (https://github.com/TheBlueMatt/RelayNode) for an example of how to develop and try out protocol features (in that case, relaying blocks more efficiently by taking advantage of transaction pre-forwarding) with reduced risk to the production Bitcoin system.

It isn't. See the discussion about fraud proofs and probabilistic testing. Please also note that the tone you've taken here is irritating and is likely to cause experienced people to ignore your messages in the future if you continue with it.

Thats not correct. If you give a bitcoin node a chain and then someone else gives it a mutually exclusive longer one, your chain will simply be unwound and replaced, so giving it that extra data is harmless, once the honest network is observed its like the node never saw the forgery. When you start talking about "check-pointing" based on that chain the situation changes and you get the attack. (and not just that attack, there are several others, e.g. announcing two competing equally valid forks concurrently and leaving the network in a never-resolving perpetual consensus split).

Every node begins its life partitioned, our security model also allows temporary partitioning. Assuming you will never be partitioned requires resolving the sybil problem in a strong sense at a minimum and isn't generally compatible with the reality of computer networks today (they're just sometimes partitioned).

Uh. You're pointing out precisely why they cannot. An invalid mined block is invalid, no less than if it didn't meet the target.

I can mine 80 blocks in a row at height 100,000 trivially in a few seconds myself (there are many thousand in a row mined by me in testnet, for example). I could also mine 80 blocks in a row forking from the current heights at considerable cost given a few months. A bitcoin node doesn't have an absolute synchronous ordered view of the world, it only learns what it learns from its peers... and it can't tell if a simulated blockchain took 6 months of computation or one day, it can't tell if it was created recently or long in the past.  etc.  If we could tell these things we wouldn't need a blockchain.

What you're describing is not a checkpoint then. A checkpoint forces the identity of the selected chain, regardless of it has the most work or not. So that would be the point of departure.

What is the point of the rest of the complexity in what you're discussing then? The "propose", "earmark" milibits? etc. (and no, newly generated coins cannot be spent for 100 blocks). The blockchain itself is already the measurement of its history.  If you're willing to trust the data in the blockchain, you just can no extra information is required. (Though if you're willing to adopt that reduced security model, why are you not going all the way and using SPV (see section 8 of bitcoin.pdf)?  Presumably you're aware that you still need to transfer the data and process it to be able to verify further blocks, right?

[If you're talking about small numbers of blocks like that, what you're proposing would be a considerable reduction in the security model, since the reward for a miner to hop ahead of the network would basically be unbounded, so that the incentives arguments on behaviour are much weaker. Weaker has it's place, but thats what SPV already accomplishes.]

Even with headers first, checkpoints still mean that old txs don't need to be validated (assuming you trust the reference client programmers).

You could leave the checkpoints in, and just say that all txs before a checkpoint are automatically valid.

If there was a conflict, I think going into some kind of emergency mode is better than saying nothing.

Better is detecting large forks.  If there is a fork that is 1000 blocks long within the last 2000 blocks, then flag a warning and tell users that their balances could be wrong.


Pruning is not a big deal really.  As long as at least 1 node keeps everything, then the network can recover from forks.

If 10,000 nodes each hold 1% of the data, then you are highly likely to have everything.

It is likely that at least a few nodes will be "archive" nodes that will store everything.