Bitcoin Forum
April 21, 2019, 11:06:51 PM *
News: Latest Bitcoin Core release: 0.17.1 [Torrent]
 
   Home   Help Search Login Register More  
Pages: « 1 [2] 3 4 5 »  All
  Print  
Author Topic: What are checkpoints in bitcoin code?  (Read 14531 times)
bluemeanie1
Sr. Member
****
Offline Offline

Activity: 280
Merit: 257


bluemeanie


View Profile WWW
June 17, 2013, 05:33:02 PM
 #21

I think you don't understand.
Checkpoints are not there to fight proof of work - they are there to protect your node from DoS attacks.

Do you understand that if there were no checkpoints, anyone with a graphic card could just keep mining blocks that link to the genesis one, and this way fill up your hard disk?


Do you understand that checkpoints completely violate the entire purpose of Bitcoin?

What they do is introduce a form of approval(consensus) about the state of a chain.  This form of authority could certainly be abused, and it runs counter to practically every design principle of Bitcoin.

at the most fundamental level, the client code should not have ANYTHING to say about the block chain.  Whether this is required in order to operate is irrelevant.  The reason people use Bitcoin is *because* it's peer-to-peer.

Just who IS bluemeanie?    On NXTautoDAC and a Million Stolen NXT

feel like your voice isn't being heard? PM me.   |   stole 1M NXT?
1555888011
Hero Member
*
Offline Offline

Posts: 1555888011

View Profile Personal Message (Offline)

Ignore
1555888011
Reply with quote  #2

1555888011
Report to moderator
1555888011
Hero Member
*
Offline Offline

Posts: 1555888011

View Profile Personal Message (Offline)

Ignore
1555888011
Reply with quote  #2

1555888011
Report to moderator
1555888011
Hero Member
*
Offline Offline

Posts: 1555888011

View Profile Personal Message (Offline)

Ignore
1555888011
Reply with quote  #2

1555888011
Report to moderator
PLAY NOW
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1555888011
Hero Member
*
Offline Offline

Posts: 1555888011

View Profile Personal Message (Offline)

Ignore
1555888011
Reply with quote  #2

1555888011
Report to moderator
1555888011
Hero Member
*
Offline Offline

Posts: 1555888011

View Profile Personal Message (Offline)

Ignore
1555888011
Reply with quote  #2

1555888011
Report to moderator
1555888011
Hero Member
*
Offline Offline

Posts: 1555888011

View Profile Personal Message (Offline)

Ignore
1555888011
Reply with quote  #2

1555888011
Report to moderator
piotr_n
Legendary
*
Offline Offline

Activity: 1988
Merit: 1048


aka tonikt


View Profile WWW
June 17, 2013, 05:46:08 PM
 #22

Do you understand that checkpoints completely violate the entire purpose of Bitcoin?
I understand that they violate some purpose of bitcoin.
But I am also a pragmatic person and so I prefer to have a system that violates some very unlikely to happen purposes, as long as it is a way to keep my resources away from spammers.

What they do is introduce a form of approval(consensus) about the state of a chain.  This form of authority could certainly be abused, and it runs counter to practically every design principle of Bitcoin.
But your concerns are 100% theoretical and it isn't really possible to start a fork like 2000 blocks back and get with it to the top.
And even if it was possible, would you really want it? Because I would not.

Quote
at the most fundamental level, the client code should not have ANYTHING to say about the block chain.  Whether this is required in order to operate is irrelevant.  The reason people use Bitcoin is *because* it's peer-to-peer.
Again: the node, and the network, must protect itself from DoS attacks.
What are you even trying to achieve forbidding people doing this?

If you one day manage to create a branch that would beat their chosen checkpoint - then I will agree with you.
But so far, I don't get your problem, though I appreciate your innocent ideas. Smiley

Check out gocoin - my original project of full bitcoin node & cold wallet written in Go.
PGP fingerprint: AB9E A551 E262 A87A 13BB  9059 1BE7 B545 CDF3 FD0E
kjj
Legendary
*
Offline Offline

Activity: 1302
Merit: 1001



View Profile
June 17, 2013, 05:48:57 PM
 #23

P.S.  No one particularly likes the checkpoints, the devs least of all.  There has been much discussion about removing them, or switching to a soft system (config file, etc).  So far, no one has come up with a really good solution.
I can come out with a really good solution, if you want Wink

Just make a rule that the client would not accept any forks deeper than N blocks back from its currently known top.
Set N to whatever constant you want, but I think more than a week time would be exaggerating.

No, that sucks.  Sorry.

17Np17BSrpnHCZ2pgtiMNnhjnsWJ2TMqq8
I routinely ignore posters with paid advertising in their sigs.  You should too.
piotr_n
Legendary
*
Offline Offline

Activity: 1988
Merit: 1048


aka tonikt


View Profile WWW
June 17, 2013, 05:50:07 PM
 #24

P.S.  No one particularly likes the checkpoints, the devs least of all.  There has been much discussion about removing them, or switching to a soft system (config file, etc).  So far, no one has come up with a really good solution.
I can come out with a really good solution, if you want Wink

Just make a rule that the client would not accept any forks deeper than N blocks back from its currently known top.
Set N to whatever constant you want, but I think more than a week time would be exaggerating.

No, that sucks.  Sorry.
Yeah, I'm sure your checkpoints idea is much better.
I'm actually even ashamed mentioning something such stupid Smiley

Check out gocoin - my original project of full bitcoin node & cold wallet written in Go.
PGP fingerprint: AB9E A551 E262 A87A 13BB  9059 1BE7 B545 CDF3 FD0E
kjj
Legendary
*
Offline Offline

Activity: 1302
Merit: 1001



View Profile
June 17, 2013, 06:04:35 PM
 #25

P.S.  No one particularly likes the checkpoints, the devs least of all.  There has been much discussion about removing them, or switching to a soft system (config file, etc).  So far, no one has come up with a really good solution.
I can come out with a really good solution, if you want Wink

Just make a rule that the client would not accept any forks deeper than N blocks back from its currently known top.
Set N to whatever constant you want, but I think more than a week time would be exaggerating.

No, that sucks.  Sorry.
Yeah, I'm sure your checkpoints idea is much better.
I'm actually even ashamed mentioning something such stupid Smiley

I'm glad you managed to dig up and read my proposal for exponential difficulty reorgs, and that you find it superior.  Not surprising, considering that yours is a degenerate case of mine where the exponent is infinite.

But it really isn't a checkpoint system.  The checkpoint system is all about the past, while both your proposal and mine are useless in that domain.

17Np17BSrpnHCZ2pgtiMNnhjnsWJ2TMqq8
I routinely ignore posters with paid advertising in their sigs.  You should too.
piotr_n
Legendary
*
Offline Offline

Activity: 1988
Merit: 1048


aka tonikt


View Profile WWW
June 17, 2013, 06:07:40 PM
 #26

I'm glad you managed to dig up and read my proposal for exponential difficulty reorgs, and that you find it superior.  Not surprising, considering that yours is a degenerate case of mine where the exponent is infinite.

But it really isn't a checkpoint system.  The checkpoint system is all about the past, while both your proposal and mine are useless in that domain.
I wish I did, but as you figured, I didn't. Smiley
I'm a man of simple solutions.
Checkpoints that are there now, in the bitcoin client, are stupid.
My idea - may be simple, but at least it would do what it has to do and close the problem once and for all. And it's surely much simpler.
Your idea - I believe it's impressive, but why to complicate things that are simple and do the job?

Check out gocoin - my original project of full bitcoin node & cold wallet written in Go.
PGP fingerprint: AB9E A551 E262 A87A 13BB  9059 1BE7 B545 CDF3 FD0E
Trongersoll
Hero Member
*****
Offline Offline

Activity: 490
Merit: 500



View Profile
June 17, 2013, 06:22:38 PM
 #27

if you disagree with the use of checkpoints the just fork 'em.
kjj
Legendary
*
Offline Offline

Activity: 1302
Merit: 1001



View Profile
June 17, 2013, 06:35:06 PM
 #28

I'm glad you managed to dig up and read my proposal for exponential difficulty reorgs, and that you find it superior.  Not surprising, considering that yours is a degenerate case of mine where the exponent is infinite.

But it really isn't a checkpoint system.  The checkpoint system is all about the past, while both your proposal and mine are useless in that domain.
I wish I did, but as you figured, I didn't. Smiley
I'm a man of simple solutions.
Checkpoints that are there now, in the bitcoin client, are stupid.
My idea - may be simple, but at least it would do what it has to do and close the problem once and for all. And it's surely much simpler.
Your idea - I believe it's impressive, but why to complicate things that are simple and do the job?

Your solution is indeed simple.  But it doesn't do the same job that checkpoints do.  Checkpoints are not about forks in the present or future, they are about helping clients avoid dead end forks in the past, while they are doing their initial sync.  Checkpoints help new nodes catch up to the network with minimal annoyance for the node operator, nothing more*.

The real replacement for checkpoints is a smarter sync system.  Greg is working on a smarter sync (see here), but I don't think it has been implemented or tested yet, so for now, we are stuck with the dumb downloader and checkpoints as a sanity check.

* They can also be used to rectify a fork in the network, but this is an extraordinary usage that requires a lot of manual intervention by a lot of node operators.  When a signed/unsigned bug was found in the value interpretation code and someone used it to give himself billions of bitcoins, a checkpoint was used to help the corrected chain overtake the errant chain that had grown for several hours before the fix could be applied.  It was also used to unroll the more recent BDB/blocksize fork.  But again, these were not checkpoints already built into the code, we just used the same mechanism as a manually override.

17Np17BSrpnHCZ2pgtiMNnhjnsWJ2TMqq8
I routinely ignore posters with paid advertising in their sigs.  You should too.
piotr_n
Legendary
*
Offline Offline

Activity: 1988
Merit: 1048


aka tonikt


View Profile WWW
June 17, 2013, 06:40:58 PM
 #29

Your solution is indeed simple.  But it doesn't do the same job that checkpoints do.  Checkpoints are not about forks in the present or future, they are about helping clients avoid dead end forks in the past, while they are doing their initial sync.  Checkpoints help new nodes catch up to the network with minimal annoyance for the node operator, nothing more*.
So you are saying that checkpoints don't actually prevent my node from downloading, storing and routing an alternative block that would refer to the genesis?
Or are you saying that they do prevent it, but the important thing here is that I should care more about those who are just downloading the chain and dont know yet which branch to choose? Smiley

Check out gocoin - my original project of full bitcoin node & cold wallet written in Go.
PGP fingerprint: AB9E A551 E262 A87A 13BB  9059 1BE7 B545 CDF3 FD0E
coinedBit
Newbie
*
Offline Offline

Activity: 14
Merit: 0


View Profile
June 17, 2013, 06:43:10 PM
 #30

Also see:
https://en.bitcoin.it/wiki/Vocabulary
http://bitcoin.stackexchange.com/questions/1061/can-a-51-attack-be-detected-and-dealt-with/1065#1065
kjj
Legendary
*
Offline Offline

Activity: 1302
Merit: 1001



View Profile
June 17, 2013, 08:21:50 PM
 #31

Your solution is indeed simple.  But it doesn't do the same job that checkpoints do.  Checkpoints are not about forks in the present or future, they are about helping clients avoid dead end forks in the past, while they are doing their initial sync.  Checkpoints help new nodes catch up to the network with minimal annoyance for the node operator, nothing more*.
So you are saying that checkpoints don't actually prevent my node from downloading, storing and routing an alternative block that would refer to the genesis?
Or are you saying that they do prevent it, but the important thing here is that I should care more about those who are just downloading the chain and dont know yet which branch to choose? Smiley

Well, the genesis hash was coded in from day one.  If you want an alternate chain, you need a new genesis hash, plus you'd have to clear out the checkpoints.  (My apologies if that wasn't what you meant, I'm not entirely sure that I understand you.)

The early blocks had very low difficulty.  If I wanted to, I could generate thousands of low difficulty blocks, with appropriate fake timestamps, and I could set my node to wait for new connections, then feed those bogus blocks to the noob.  They'd figure it out eventually, but I could keep them mired down for hours or days, working on my garbage.  The checkpoints give a handy shortcut to that "eventually".  If my chain doesn't include the checkpoint blocks, the new node won't waste their time on it.

The notion of "the correct chain" gets fuzzy.  If I have a lot of hashing power, like more than the rest of the world, and I start today making a new chain, but I keep my farm isolated from the rest of the network, is it "the correct chain"?  The only really hard rule is that the longer (more work, not necessarily more blocks) chain is correct.  My chain would be (work) longer, so in that sense, it is correct.  But no one knows about it but me, so the world would be working on a chain that they think is correct.

When I release my chain, it will overturn possibly thousands of blocks, possibly millions of transactions.  The hard rule says that it should happen, but almost no one would actually wants that outcome.  A checkpoint could invalidate my whole chain, if a big part of the network upgrades to a client that includes a checkpoint that happens to come after I started.

That situation is unrealistic, intentionally.  But it shows that the simple rule that we can easily write into our software is incomplete.  We humans want the rule to be "the longer chain is correct, except...", and the "except" part is hard to write.  We know that checkpoints aren't the right way to codify the exception.  If you've ever talked to Gavin, or even read his posts on the subject, you'll know that he absolutely hates that he could end up in the position of picking which chain wins.

So, we end up in the strange situation we are in.  The checkpoints are useful in the real world , but we can see that they also hold a power that no one actually wants to wield.  The devs advance them now and then, but always sorta grudgingly, and always months behind.  If someone released a 5000 block attack chain, we'd all be bitching that the checkpoint was too far behind.  Because no one has (and probably never will) hide a fork of that length, we are all bitching that checkpoints are contrary to the bitcoin design.

17Np17BSrpnHCZ2pgtiMNnhjnsWJ2TMqq8
I routinely ignore posters with paid advertising in their sigs.  You should too.
TierNolan
Legendary
*
Offline Offline

Activity: 1232
Merit: 1002


View Profile
August 25, 2013, 11:38:17 PM
 #32

Thats not really what they're for— though that have the effect too. Most of their usefulness is that they prevent a dos attacker from filling up bitcoin node's disk space with long runs of low difficulty blocks forked off low in the chain.

e.g. you start off with difficulty 1 blocks at block 0, now mine-able by the millions by a single asic— _MAYBE_ a chain that starts off that way could eventually turn out to be the longest so absent the checkpoints a node would happily follow an endlessly long chain of them.

Instead of checkpoints, client could hard-code a MAIN_CHAIN_MIN_POW variable.  This would be the PoW of the main chain when that version of the client was released.  All clients wouldn't need to agree on a value.  It is just a spam protection value.

Clients should download headers first and only commit blocks to disk that are on a chain that has PoW higher than the hard coded minimum.

Clients should respond to "inv" messages containing unknown block hashes by sending a "getheaders" message rather than asking for the block.

If the block extends your main chain, you will be sent it.

Peer sends: "inv" containing unknown block
Send to peer: "getheaders" with standard block locator (send at most once per inv message received)
Peer sends: "headers" with new blocks on it's main chain
Add these headers to header tree
If the main chain is extended and the main chain has PoW greater than MAIN_CHAIN_MIN_POW, request those full-blocks

Quote
They also make is so that an attacker who has complete control of your network (and thus can prevent you from hearing the longest chain from the honest bitcoin network) from putting you on a fake (low difficulty) isolated chain unless they can also trick you into running replaced software. With the checkpoints such an attacker hast to have a ton of mining power in order to continue the chain.

No matter what happens, the longest chain will always have more PoW than the longest one when the client was writen, so the best chain is guaranteed to have more PoW than MAIN_CHAIN_MIN_POW.

The client won't consider itself synced unless it is on a main chain with PoW that exceeds that value.

1LxbG5cKXzTwZg9mjL3gaRE835uNQEteWF
bitx64
Newbie
*
Offline Offline

Activity: 28
Merit: 0


View Profile
August 26, 2013, 03:35:15 AM
 #33

Instead of checkpoints, client could hard-code a MAIN_CHAIN_MIN_POW variable.  This would be the PoW of the main chain when that version of the client was released.  All clients wouldn't need to agree on a value.  It is just a spam protection value.

Clients should download headers first and only commit blocks to disk that are on a chain that has PoW higher than the hard coded minimum.

Clients should respond to "inv" messages containing unknown block hashes by sending a "getheaders" message rather than asking for the block.

If the block extends your main chain, you will be sent it.

Peer sends: "inv" containing unknown block
Send to peer: "getheaders" with standard block locator (send at most once per inv message received)
Peer sends: "headers" with new blocks on it's main chain
Add these headers to header tree
If the main chain is extended and the main chain has PoW greater than MAIN_CHAIN_MIN_POW, request those full-blocks
...
No matter what happens, the longest chain will always have more PoW than the longest one when the client was writen, so the best chain is guaranteed to have more PoW than MAIN_CHAIN_MIN_POW.

The client won't consider itself synced unless it is on a main chain with PoW that exceeds that value.

And then what if multiple chains are above this min pow?
gmaxwell
Moderator
Legendary
*
qt
Offline Offline

Activity: 2702
Merit: 2156



View Profile
August 26, 2013, 03:40:20 AM
 #34

And then what if multiple chains are above this min pow?
Then you pick among them like normal, the point there is to close of the denial of service attack without undermining the consensus algorithm with an element of centralization.
justusranvier
Legendary
*
Offline Offline

Activity: 1400
Merit: 1006



View Profile WWW
August 26, 2013, 04:23:42 AM
 #35

Instead of checkpoints, client could hard-code a MAIN_CHAIN_MIN_POW variable.  This would be the PoW of the main chain when that version of the client was released.  All clients wouldn't need to agree on a value.  It is just a spam protection value.
Haven't there been periods where the difficulty went down?
DeathAndTaxes
Donator
Legendary
*
Offline Offline

Activity: 1218
Merit: 1005


Gerald Davis


View Profile
August 26, 2013, 04:43:15 AM
 #36

I think you don't understand.
Checkpoints are not there to fight proof of work - they are there to protect your node from DoS attacks.

Do you understand that if there were no checkpoints, anyone with a graphic card could just keep mining blocks that link to the genesis one, and this way fill up your hard disk?


Do you understand that checkpoints completely violate the entire purpose of Bitcoin?

What they do is introduce a form of approval(consensus) about the state of a chain.  This form of authority could certainly be abused, and it runs counter to practically every design principle of Bitcoin.

at the most fundamental level, the client code should not have ANYTHING to say about the block chain.  Whether this is required in order to operate is irrelevant.  The reason people use Bitcoin is *because* it's peer-to-peer.

You are free to use and distribute a fork of the client without any checkpoints.  Checkpoints aren't required. Assumming an attacker isn't attempting to DDOS your node by filling it with years worth of bogus blocks or try to perform a 51% attack months deep in the chain the lack of checkpoint will have absolutely no effect on your node.

So if you are worried then remove the checkpoints, compile it and use that node.  If you are really worried set up an online mirror and advertise your checkpoint free version.

See peer to peer at work.
gmaxwell
Moderator
Legendary
*
qt
Offline Offline

Activity: 2702
Merit: 2156



View Profile
August 26, 2013, 04:47:26 AM
 #37

Haven't there been periods where the difficulty went down?
Yup, I just mentally corrected TierNolan's proposal to match ones made by many people before— have a minimum total chain work checkpoint (e.g. any valid chain will have at least 2^70 work by height XXX). That allows the same protection from spamming as the current stuff.

You are free to use and distribute a fork of the client without any checkpoints.  Checkpoints aren't required. Assumming an attacker isn't attempting to DDOS your node by filling it with years worth of bogus blocks or try to perform a 51% attack months deep in the chain the lack of checkpoint will have absolutely no effect on your node.
Uhhhhhh. Or just run with the checkpoints=0 command-line/config option.
DeathAndTaxes
Donator
Legendary
*
Offline Offline

Activity: 1218
Merit: 1005


Gerald Davis


View Profile
August 26, 2013, 04:49:05 AM
 #38

Uhhhhhh. Or just run with the checkpoints=0 command-line/config option.

Well he seemed to think the fact that they are included at all = the death of Bitcoin.  I was just providing a way for him to "save" Bitcoin from the ebil developers. 
gmaxwell
Moderator
Legendary
*
qt
Offline Offline

Activity: 2702
Merit: 2156



View Profile
August 26, 2013, 05:25:18 AM
 #39

Uhhhhhh. Or just run with the checkpoints=0 command-line/config option.
Well he seemed to think the fact that they are included at all = the death of Bitcoin.  I was just providing a way for him to "save" Bitcoin from the ebil developers.  
Fair enough— for whatever it's worth, I'm no fan of them either. If nothing else they really create a hard time for people reasoning about the security model of Bitcoin: There are a lot of people who have a hard time getting how the POW consensus works and then they hear about the checkpoints and say "Ah ha! this is how its secure!", after all people are familiar with centralized security so this is just a lot easier to understand.

The big damage from that comes because we're then subjected to a non-stop stream of IDEAS and IMPROVEMENTS from people who's understanding of the security model was broken by the mere existence of checkpoints, even though checkpoints do nothing relative to consensus. These people propose things which depend on a centralized security model and don't have the benefit of the checkpoints being set thousands of block in the past so they are never a practical influence on the consensus, and don't spend the mental cycles trying to fit their ideas into the real, primarily zero trust, Bitcoin security model. As a cautionary sign of where that could be going: There altcoins which user developer controlled checkpoints broadcast to checkpoint blocks in real time, and more altcoins being added that do this... as if this were an acceptable way to construct a decentralized cryptocurrency!

When headers first syncing is merged, just by adding a "must be this tall" minimum sum difficulty check we'll be able to remove checkpoints for all DOS purposes, and we'll also be able to remove them for syncing acceleration (using random sampling for ECDSA in the deeply burried chain).

Checkpoints as a cognitive security blanket against doomsday reorgs will likely stay, but we could update them less frequently, with a more regimented process, and perhaps change how nodes respond to them (e.g. a conflicting chain becoming longer making all nodes go into a safemode where they regard everything new as unconfirmed, process no transactions, and discourage any new blocks that process transactions allowing for a public resolution of the "impossible event", rather than directly coercing the consensus).
justusranvier
Legendary
*
Offline Offline

Activity: 1400
Merit: 1006



View Profile WWW
August 26, 2013, 05:48:05 AM
 #40

Checkpoints as a cognitive security blanket against doomsday reorgs will likely stay, but we could update them less frequently, with a more regimented process, and perhaps change how nodes respond to them (e.g. a conflicting chain becoming longer making all nodes go into a safemode where they regard everything new as unconfirmed, process no transactions, and discourage any new blocks that process transactions allowing for a public resolution of the "impossible event", rather than directly coercing the consensus).
I think it would be cool for checkpoints to be distributed periodically in dead tree format.
Pages: « 1 [2] 3 4 5 »  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!