Bitcoin Forum
May 04, 2024, 01:14:56 PM *
News: Latest Bitcoin Core release: 27.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1]
  Print  
Author Topic: Block headers download  (Read 724 times)
r.willis (OP)
Jr. Member
*
Offline Offline

Activity: 42
Merit: 11


View Profile
April 15, 2013, 09:41:02 PM
 #1

I have trouble implementing parallel blockchain download (headers only). I read wiki back and forth, but do not understand:
1) What exactly is block locator object.
2) How to download headers before certain block (towards genesis block)
Now I can start from genesis and work up blockchain (using 0 as hash_stop). Is it possible to move in opposite direction (from newest blocks to genesis)?
1714828496
Hero Member
*
Offline Offline

Posts: 1714828496

View Profile Personal Message (Offline)

Ignore
1714828496
Reply with quote  #2

1714828496
Report to moderator
1714828496
Hero Member
*
Offline Offline

Posts: 1714828496

View Profile Personal Message (Offline)

Ignore
1714828496
Reply with quote  #2

1714828496
Report to moderator
1714828496
Hero Member
*
Offline Offline

Posts: 1714828496

View Profile Personal Message (Offline)

Ignore
1714828496
Reply with quote  #2

1714828496
Report to moderator
Even if you use Bitcoin through Tor, the way transactions are handled by the network makes anonymity difficult to achieve. Do not expect your transactions to be anonymous unless you really know what you're doing.
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction.
1714828496
Hero Member
*
Offline Offline

Posts: 1714828496

View Profile Personal Message (Offline)

Ignore
1714828496
Reply with quote  #2

1714828496
Report to moderator
1714828496
Hero Member
*
Offline Offline

Posts: 1714828496

View Profile Personal Message (Offline)

Ignore
1714828496
Reply with quote  #2

1714828496
Report to moderator
1714828496
Hero Member
*
Offline Offline

Posts: 1714828496

View Profile Personal Message (Offline)

Ignore
1714828496
Reply with quote  #2

1714828496
Report to moderator
TierNolan
Legendary
*
Offline Offline

Activity: 1232
Merit: 1083


View Profile
April 15, 2013, 10:05:43 PM
Last edit: April 15, 2013, 10:35:59 PM by TierNolan
 #2

I have trouble implementing parallel blockchain download (headers only). I read wiki back and forth, but do not understand:
1) What exactly is block locator object.
2) How to download headers before certain block (towards genesis block)
Now I can start from genesis and work up blockchain (using 0 as hash_stop). Is it possible to move in opposite direction (from newest blocks to genesis)?

You are supposed to start at the start and work forward.  The locator object is just hashes.  You effectively have to give the hash of the start and finish.  You can give extra hashes.  You are sending a list of hashes that you know about.  The other node will scan them, and start at the hash that has the highest height and is on the main chain.

You could use the getblocks message.  They returns the hash of the blocks in an "inv" packet.  However, that is 33 bytes long per entry, so you aren't really saving much.

For parallel download what is needed is a message that returns the hash of a block for a set of heights.

1LxbG5cKXzTwZg9mjL3gaRE835uNQEteWF
Zeilap
Full Member
***
Offline Offline

Activity: 154
Merit: 100


View Profile
April 15, 2013, 10:11:16 PM
Last edit: April 15, 2013, 10:21:23 PM by Zeilap
 #3

I have trouble implementing parallel blockchain download (headers only). I read wiki back and forth, but do not understand:
1) What exactly is block locator object.
2) How to download headers before certain block (towards genesis block)
Now I can start from genesis and work up blockchain (using 0 as hash_stop). Is it possible to move in opposite direction (from newest blocks to genesis)?

Block locator is a list of block hashes that you know about, starting from your head, back towards the genesis block. The idea is that the remote peer can know which blocks/headers to send you. If it determines that your most recent blocks are actually now on a side chain, then it will work backwards through your block locator to find a block which is on the main chain and then send you blocks from there onwards. If it cannot find any blocks in your locator which are on the main chain, it will send you blocks from the genesis block.
Current implementations create a block locator roughly as follows,

Code:
 
var i = current head height
var stepSize = 1

while(i > 0) {
    insert the hash of the block at height i
    i = i - stepSize
    stepSize = stepSize * 2
}


So many more recent blocks are added, and as you work back, the steps get bigger (because of the decreased likelihood of deeper and deeper reorgs).
When downloading the headers before the last checkpoint, you can simply use the hash of the most recent header you have as the locator, i.e. only one hash.

EDIT:
As for working backwards, this is not possible. What is possible however is to hardcode checkpoint hashes, this way you can download the headers between each checkpoint separately enabling your parallel download. I have code to do exactly this, but I haven't bothered to test it to see if the speed improvement was worth it compared to downloading from a single peer. Others may see this as a waste of network resources perhaps.
r.willis (OP)
Jr. Member
*
Offline Offline

Activity: 42
Merit: 11


View Profile
April 15, 2013, 10:19:30 PM
 #4

TierNolan, Zeilap - thanks. Now I understand better.
It seems the only option to implement parallel download is to provide multiple checkpoints - if I want N parallel requests, I must hardcode N somewhat evenly-spaced checkpoints.
Zeilap
Full Member
***
Offline Offline

Activity: 154
Merit: 100


View Profile
April 15, 2013, 10:22:26 PM
 #5

TierNolan, Zeilap - thanks. Now I understand better.
It seems the only option to implement parallel download is to provide multiple checkpoints - if I want N parallel requests, I must hardcode N somewhat evenly-spaced checkpoints.

Yes exactly  Smiley - see edit
r.willis (OP)
Jr. Member
*
Offline Offline

Activity: 42
Merit: 11


View Profile
April 15, 2013, 10:32:56 PM
 #6

Quote
Others may see this as a waste of network resources perhaps.
Why? It's the same total amount, isn't it?
Btw, even is sequential download I select peer to send getheaders randomly - this way I create more even load on peers. After I got headers, I can get blocks using getdata or getblocks with much better control.
Zeilap
Full Member
***
Offline Offline

Activity: 154
Merit: 100


View Profile
April 15, 2013, 10:58:16 PM
 #7

Well it depends how lazy you are Wink. If you share the downloads between the peers without connecting to more peers, then no problem. If you purposely connect to as many peers as there are checkpoints, then it's not the same, especially if you don't bother to disconnect from most of them after you've finished downloading.
Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!