What are checkpoints in bitcoin code?

[HZKto]

It's an optimization that can be disabled with a command line switch.

Even if it was done for the purposes of optimization - it is still protocol specification change. Clients with disabled "optimization" could accept some branches which would be rejected by client on default settings.

There are other possible approaches to do such kind of optimization, for example MAIN_CHAIN_MIN_POW described above, but it does not compromises decentralization trust as checkpoints do.

[1714024608]

1714024608

[1714024608]

1714024608

[1714024608]

1714024608

[kjj]

You didn't actually read this thread before posting, did you?

I did read this thread, and several old ones: https://bitcointalk.org/index.php?topic=437 , https://bitcointalk.org/index.php?topic=1647
Do you have any particular objection to discuss?

Way back on post #5 gmaxwell explains it pretty well.

Yes, you are correct.  They are there so some quantum computing farm (doesn't exist, but...) can't come out of nowhere and roll back blocks by mining a long chain from far in the past.

Thats not really what they're for— though that have the effect too. Most of their usefulness is that they prevent a dos attacker from filling up bitcoin node's disk space with long runs of low difficulty blocks forked off low in the chain.

e.g. you start off with difficulty 1 blocks at block 0, now mine-able by the millions by a single asic— _MAYBE_ a chain that starts off that way could eventually turn out to be the longest so absent the checkpoints a node would happily follow an endlessly long chain of them.

They also make is so that an attacker who has complete control of your network (and thus can prevent you from hearing the longest chain from the honest bitcoin network) from putting you on a fake (low difficulty) isolated chain unless they can also trick you into running replaced software. With the checkpoints such an attacker hast to have a ton of mining power in order to continue the chain.

[HZKto]

Way back on post #5 gmaxwell explains it pretty well.

Again, it is possible to solve this problem by other approaches.
For example via MAIN_CHAIN_MIN_POW as suggested above - it does not compromises decentralization.

We know when Bitcoin was released, we know current date - we can calculate upper bound for block count in any chain.
When somebody tries to send us old chain - we ask it to sent block headers sorted by difficulty, starting with the most expensive one and continue in descending order. 1% of top block headers should have at least 0.01*MAIN_CHAIN_MIN_POW accumulated difficulty, if it is not - then it is fake, else continue, and so on.

[kjj]

Way back on post #5 gmaxwell explains it pretty well.

Again, it is possible to solve this problem by other approaches.
For example via MAIN_CHAIN_MIN_POW as suggested above - it does not compromises decentralization.

We know when Bitcoin was released, we know current date - we can calculate upper bound for block count in any chain.
When somebody tries to send us old chain - we ask it to sent block headers sorted by difficulty, starting with the most expensive one and continue in descending order. 1% of top block headers should have at least 0.01*MAIN_CHAIN_MIN_POW accumulated difficulty, if it is not - then it is fake, else continue, and so on.

Cool.  When can we expect your patch?

[bluemeanie1]

I am reading bitcoin codes, and I don't quite get what are checkpoints in the bitcoin code:

        ( 11111, uint256("0x0000000069e244f73d78e8fd29ba2fd2ed618bd6fa2ee92559f542fdb26e7c1d"))
        ( 33333, uint256("0x000000002dd5588a74784eaa7ab0507a18ad16a236e7b1ce69f00d7ddfb5d0a6"))
        ( 74000, uint256("0x0000000000573993a3c9e41ce34471c079dcf5f52a0e824a81e7f953b8661a20"))
        (105000, uint256("0x00000000000291ce28027faea320c8d2b054b2e0fe44a773f3eefb151d6bdc97"))
        (134444, uint256("0x00000000000005b12ffd4cd315cd34ffd4a594f430ac814c91184a0d42d2b0fe"))
        (168000, uint256("0x000000000000099e61ea72015e79632f216fe6cb33d7899acb35b75c8303b763"))
        (193000, uint256("0x000000000000059f452a5f7340de6682a977387c17010ff6e6c3bd83ca8b1317"))
        (210000, uint256("0x000000000000048b95347e83192f69cf0366076336c639f9b7228e9ba171342e"))
        (216116, uint256("0x00000000000001b4f4b433e81ee46494af945cf96014816a4e2370f11b23df4e"))
        (225430, uint256("0x00000000000001c108384350f74090433e7fcf79a606b8e797f065b130575932"))

Are they hashes of the blocks at 11111,33333, etc positions? If so how can we precompute them, as going from 1 to block 225430 would take big amount of time of computations...

they're what make Bitcoin not Peer-To-Peer.

[HZKto]

Cool.  When can we expect your patch?

I think this idea should be discussed first in separate topic. Then, as I understand it should go to Bitcoin Improvement Proposals.

[kjj]

Cool.  When can we expect your patch?

I think this idea should be discussed first in separate topic. Then, as I understand it should go to Bitcoin Improvement Proposals.

Not everything goes to BIP.  This is a purely internal function of the reference client.  Most discussion of changes like this actually happen on github, attached to the pull request that implements them.

[TierNolan]

Not everything goes to BIP.  This is a purely internal function of the reference client.  Most discussion of changes like this actually happen on github, attached to the pull request that implements them.

I think the headers first change is a key stepping stone for this.

Downloading 300k blocks is around a 20MB download.

With modern bandwidth, nodes could download that from 8 peers reasonably quickly and then discard any chain that doesn't have sufficient POW.

The more blocks in a blockchain, the higher the total POW.  It would be cool if someone could work out a formula.

Max blocks = f(time since genesis, POW)

[HZKto]

It would be cool if someone could work out a formula.
Max blocks = f(time since genesis, POW)

Yes, this is the key for improvement. It does not have to provide strict upper_bound, but something that >= strict_upper_bound.
Plus, if this approach will be accepted, we should add ability to request block headers sorted by difficulty, starting with the most expensive one and continue in descending order. This would allow to detect flood with even less traffic, like few percents of Max_Blocks.

[HZKto]

I think the headers first change is a key stepping stone for this.

Downloading 300k blocks is around a 20MB download.

With modern bandwidth, nodes could download that from 8 peers reasonably quickly and then discard any chain that doesn't have sufficient POW.

I agree, for now and most likely for future downloading all block headers is not an issue. So don't even need to adjust protocol.

[TierNolan]

we should add ability to request block headers sorted by difficulty, starting with the most expensive one and continue in descending order. This would allow to detect flood with even less traffic, like few percents of Max_Blocks.

There would be no way to prove that they are part of the chain.

In the High hash highway, a second pointer is added.  Headers would have a link to the previous block and also an "up" link.

This allows you follow backwards from the leaf of the chain much faster, but still hit all the high blocks.  However, it is a hard fork.

The 2nd link points to the nearest block with more leading zero bits than the previous block.

In fact, I think optimal would be, point to the block after the nearest block with more leading zero bits than the previous block.

So, the blocks had the following leading zero bits

A) 30
B) 31 - up -> genesis (since 30 is the best so far)
C) 29 - up -> genesis (since 31 is the best so far)
D) 33 - up -> C
E) 31 - up -> genesis (since 33 is the best block)
F) 32 - up -> E
G) 30 - up -> E
H) 32 - up -> G

If you follow the up link backwards, you go to a block that is the successor of a block with at least 1 more leading zero than the previous block.

To find the highest block, these are the steps

- Set N to the index of the leaf block
- Set b to the number of leading zeros for block N-1

Loop
- The up pointer for block N points to block M
- Block M-1 has more leading zeros than block N-1 (by definition)
- It also has more leading zeros than any blocks from N-1 to M, inclusive
- This means you can't have skipped a better block than block M-1
- set N = M
- set b = the number of leading zeros for block M-1

Each time through the loop b is guaranteed to increase and also you are guaranteed not to skip the best block.

Therefore, this search will find the best block.

The PoW of the main chain can be estimated just by looking at the work of the best block.

You can also use the system to find all blocks which have difficulty within a number of bits of the best block.

[Dabs]

I think the checkpoints are about at least a month or two back in time. Anyone who has more than 51% of 6.5 Petahash of mining power that can fork their own chain past the latest check point, is better off mining regularly and getting 51% of 3600 coins daily or about 1800 coins, at least for the next 2000 blocks when the difficulty will double.

[HZKto]

There would be no way to prove that they are part of the chain.

Yes - I considered this. I think it is not a problem, because it is very unlikely that someone can prepare 0.01*MaxBlocks different blocks (unchained, as you noted) which have accumulated difficulty more than 1% or so of total accumulated work spent on main chain for whole time of Bitcoin existence.

In the High hash highway, a second pointer is added.  Headers would have a link to the previous block and also an "up" link.

This allows you follow backwards from the leaf of the chain much faster, but still hit all the high blocks.  However, it is a hard fork.

The 2nd link points to the nearest block with more leading zero bits than the previous block.

I should look closer to your proposal. For now I have one question:
Does "up" link points to future? If so - is it included in block hash or can it be falsified?

[HZKto]

I think the checkpoints are about at least a month or two back in time. Anyone who has more than 51% of 6.5 Petahash of mining power that can fork their own chain past the latest check point, is better off mining regularly and getting 51% of 3600 coins daily or about 1800 coins, at least for the next 2000 blocks when the difficulty will double.

Of course hidden fork crunched at high speed in shadow is unlikely, and maybe checkpoints do not harm due to low probability of such fork.
But they harm in other way - they compromise Bitcoin decentralization, which is in my opinion is the biggest attractive feature of Bitcoin.
Today they add checkpoints (this effectively changes protocol specification), tomorrow they could freeze BTC on your address, by adding small tweaks to validation routine (just like checkpoints validation).

[Dabs]

In a way, bitcoin (and almost all other crypto coins) are still some what centralized. The devs are the centralization. There's not much we can do about that.

The redeeming thing is that almost all of them are open source with code available to be self-compiled.

If it were not check points, there is also the bootstrap file uploaded by Jeff Garzik. That is sort of centralized, but distributed by torrent, for all new users who need an initial sync of the correct block chain.

While the protocol itself is decentralized, it is a requirement that honest nodes form a consensus. Remember the hard fork early this year.

[kjj]

I think the checkpoints are about at least a month or two back in time. Anyone who has more than 51% of 6.5 Petahash of mining power that can fork their own chain past the latest check point, is better off mining regularly and getting 51% of 3600 coins daily or about 1800 coins, at least for the next 2000 blocks when the difficulty will double.

Of course hidden fork crunched at high speed in shadow is unlikely, and maybe checkpoints do not harm due to low probability of such fork.
But they harm in other way - they compromise Bitcoin decentralization, which is in my opinion is the biggest attractive feature of Bitcoin.
Today they add checkpoints (this effectively changes protocol specification), tomorrow they could freeze BTC on your address, by adding small tweaks to validation routine (just like checkpoints validation).

Nonsense, every word.

root@linuxcoin:/home/user# bitcoind --help
Bitcoin version v0.8.6.0-g03a7d67-beta

Usage:
  bitcoind [options]
  bitcoind [options] <command> [params]  Send command to -server or bitcoind
  bitcoind [options] help                List commands
  bitcoind [options] help <command>      Get help for a command

Options:
  -?                     This help message
  -conf=<file>           Specify configuration file (default: bitcoin.conf)
  -pid=<file>            Specify pid file (default: bitcoind.pid)
  -gen                   Generate coins (default: 0)
  -datadir=<dir>         Specify data directory
  -dbcache=<n>           Set database cache size in megabytes (default: 25)
  -timeout=<n>           Specify connection timeout in milliseconds (default: 5000)
  -proxy=<ip:port>       Connect through socks proxy
  -socks=<n>             Select the version of socks proxy to use (4-5, default: 5)
  -tor=<ip:port>         Use proxy to reach tor hidden services (default: same as -proxy)
  -dns                   Allow DNS lookups for -addnode, -seednode and -connect
  -port=<port>           Listen for connections on <port> (default: 8333 or testnet: 18333)
  -maxconnections=<n>    Maintain at most <n> connections to peers (default: 125)
  -addnode=<ip>          Add a node to connect to and attempt to keep the connection open
  -connect=<ip>          Connect only to the specified node(s)
  -seednode=<ip>         Connect to a node to retrieve peer addresses, and disconnect
  -externalip=<ip>       Specify your own public address
  -onlynet=<net>         Only connect to nodes in network <net> (IPv4, IPv6 or Tor)
  -discover              Discover own IP address (default: 1 when listening and no -externalip)
  -checkpoints           Only accept block chain matching built-in checkpoints (default: 1)
  -listen                Accept connections from outside (default: 1 if no -proxy or -connect)
  -bind=<addr>           Bind to given address and always listen on it. Use [host]:port notation for IPv6
  -dnsseed               Find peers using DNS lookup (default: 1 unless -connect)
  -banscore=<n>          Threshold for disconnecting misbehaving peers (default: 100)
  -bantime=<n>           Number of seconds to keep misbehaving peers from reconnecting (default: 86400)
  -maxreceivebuffer=<n>  Maximum per-connection receive buffer, <n>*1000 bytes (default: 5000)
  -maxsendbuffer=<n>     Maximum per-connection send buffer, <n>*1000 bytes (default: 1000)
  -paytxfee=<amt>        Fee per KB to add to transactions you send
  -daemon                Run in the background as a daemon and accept commands
  -testnet               Use the test network
  -debug                 Output extra debugging information. Implies all other -debug* options
  -debugnet              Output extra network debugging information
  -logtimestamps         Prepend debug output with timestamp (default: 1)
  -shrinkdebugfile       Shrink debug.log file on client startup (default: 1 when no -debug)
  -printtoconsole        Send trace/debug info to console instead of debug.log file
  -rpcuser=<user>        Username for JSON-RPC connections
  -rpcpassword=<pw>      Password for JSON-RPC connections
  -rpcport=<port>        Listen for JSON-RPC connections on <port> (default: 8332 or testnet: 18332)
  -rpcallowip=<ip>       Allow JSON-RPC connections from specified IP address
  -rpcconnect=<ip>       Send commands to node running on <ip> (default: 127.0.0.1)
  -rpcthreads=<n>        Set the number of threads to service RPC calls (default: 4)
  -blocknotify=<cmd>     Execute command when the best block changes (%s in cmd is replaced by block hash)
  -walletnotify=<cmd>    Execute command when a wallet transaction changes (%s in cmd is replaced by TxID)
  -alertnotify=<cmd>     Execute command when a relevant alert is received (%s in cmd is replaced by message)
  -upgradewallet         Upgrade wallet to latest format
  -keypool=<n>           Set key pool size to <n> (default: 100)
  -rescan                Rescan the block chain for missing wallet transactions
  -addtag=<tag>           Add a tag to your coinbase
  -salvagewallet         Attempt to recover private keys from a corrupt wallet.dat
  -checkblocks=<n>       How many blocks to check at startup (default: 288, 0 = all)
  -checklevel=<n>        How thorough the block verification is (0-4, default: 3)
  -txindex               Maintain a full transaction index (default: 0)
  -loadblock=<file>      Imports blocks from external blk000??.dat file
  -reindex               Rebuild block chain index from current blk000??.dat files
  -par=<n>               Set the number of script verification threads (up to 16, 0 = auto, <0 = leave that many cores free, default: 0)

[Dabs]

  -addtag=<tag>           Add a tag to your coinbase
  -txindex               Maintain a full transaction index (default: 0)

What do those two do? I'm guessing the tag is meant for miners? Is it useful for regular people who just run a node or wallet?

Since I just run QT like normal, I don't get the full transaction index? Does that take up additional space or isn't everything in the blockchain anyway? Or I am correct to guess that this is better used by large online wallets or something?

[kjj]

  -addtag=<tag>           Add a tag to your coinbase
  -txindex               Maintain a full transaction index (default: 0)

What do those two do? I'm guessing the tag is meant for miners? Is it useful for regular people who just run a node or wallet?

Since I just run QT like normal, I don't get the full transaction index? Does that take up additional space or isn't everything in the blockchain anyway? Or I am correct to guess that this is better used by large online wallets or something?

Heh, oops, -addtag is a custom mod that I maintain.  It does indeed add tags to the coinbase for mining.

-txindex makes the node keep an index of all transactions.  This is very helpful for RPC services, particularly those using raw transactions.

[HZKto]

In a way, bitcoin (and almost all other crypto coins) are still some what centralized. The devs are the centralization. There's not much we can do about that.

Yes, they are centralization. But as long as they do not adjust blockchain on regular basis (select which chain is good and which is not) - they gain trust of users.

The redeeming thing is that almost all of them are open source with code available to be self-compiled.

Yes, but the problem is that the client with checkpoints and one without - are not 100% compatible with each other. Checkpoints are not just cosmetic change.

While the protocol itself is decentralized, it is a requirement that honest nodes form a consensus. Remember the hard fork early this year.

Bitcoin already has mechanism to achieve consensus - voting by computing resources, it should not be substituted by any centralized authority.

[HZKto]

Of course hidden fork crunched at high speed in shadow is unlikely, and maybe checkpoints do not harm due to low probability of such fork.
But they harm in other way - they compromise Bitcoin decentralization, which is in my opinion is the biggest attractive feature of Bitcoin.
Today they add checkpoints (this effectively changes protocol specification), tomorrow they could freeze BTC on your address, by adding small tweaks to validation routine (just like checkpoints validation).

Nonsense, every word.

Unbeatable arguments, thank you.

root@linuxcoin:/home/user# bitcoind --help
Bitcoin version v0.8.6.0-g03a7d67-beta

  -checkpoints           Only accept block chain matching built-in checkpoints (default: 1)

Do you understand that this option is protocol specification change? It effectively forks blockchain.