Bitcoin Forum
September 27, 2016, 08:39:11 PM *
News: Latest stable version of Bitcoin Core: 0.13.0 (New!) [Torrent]. Make sure you verify it.
 
   Home   Help Search Donate Login Register  
Pages: [1]
  Print  
Author Topic: Faster initial block download (5x faster)  (Read 4944 times)
satoshi
Founder
Sr. Member
*
qt
Offline Offline

Activity: 364


View Profile
July 23, 2010, 06:24:56 PM
 #1

By making some adjustments to the database settings, I was able to make the initial block download about 5 times faster.  It downloads in about 30 minutes.

The database default had it writing each block to disk synchronously, which is not necessary.  I changed the settings to let it cache the changes in memory and write them out in a batch.  Blocks are still written transactionally, so either the complete change occurs or none of it does, in either case the data is left in a valid state.

I only enabled this change during the initial block download.  When you come within 2000 blocks of the latest block, these changes turn off and it slows down to the old way.

I built a test build if you'd like to start using it:

http://www.bitcoin.org/download/bitcoin-0.3.2.5-win32.zip
http://www.bitcoin.org/download/bitcoin-0.3.2.5-linux.tar.gz

These binaries also include Gavin Andresen's JSON-RPC HTTP authentication feature and the other important security improvements from 0.3.2.

I've been running a test over the last 24 hours that kills and restarts it randomly every 2-60 seconds (poor thing) while it's trying to do an initial block download and it's been fine.

There are no changes to the way it handles wallet.dat.  This change is only for blk*.dat and the non-critical addr.dat.  You can always delete blk*.dat if it gets screwed up and let it re-download.
1475008751
Hero Member
*
Offline Offline

Posts: 1475008751

View Profile Personal Message (Offline)

Ignore
1475008751
Reply with quote  #2

1475008751
Report to moderator
1475008751
Hero Member
*
Offline Offline

Posts: 1475008751

View Profile Personal Message (Offline)

Ignore
1475008751
Reply with quote  #2

1475008751
Report to moderator
1475008751
Hero Member
*
Offline Offline

Posts: 1475008751

View Profile Personal Message (Offline)

Ignore
1475008751
Reply with quote  #2

1475008751
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1475008751
Hero Member
*
Offline Offline

Posts: 1475008751

View Profile Personal Message (Offline)

Ignore
1475008751
Reply with quote  #2

1475008751
Report to moderator
1475008751
Hero Member
*
Offline Offline

Posts: 1475008751

View Profile Personal Message (Offline)

Ignore
1475008751
Reply with quote  #2

1475008751
Report to moderator
1475008751
Hero Member
*
Offline Offline

Posts: 1475008751

View Profile Personal Message (Offline)

Ignore
1475008751
Reply with quote  #2

1475008751
Report to moderator
knightmb
Sr. Member
****
Offline Offline

Activity: 308


Timekoin - Save Electricity, Don't Waste It!


View Profile WWW
July 23, 2010, 06:27:50 PM
 #2

Are these for use with the test network you have setup or the current public network?

NewLibertyStandard
Sr. Member
****
Offline Offline

Activity: 252



View Profile WWW
July 23, 2010, 06:42:25 PM
 #3

Are these for use with the test network you have setup or the current public network?
This is for the regular network. The test network doesn't have enough blocks to test it effectively. From what I can tell, satoshi's builds are always for the live network unless he specifies otherwise.

Treazant: A Fullever Rewarding Bitcoin - Backup Your Wallet TODAY to Double Your Money! - Dual Currency Donation Address: 1Dnvwj3hAGSwFPMnkJZvi3KnaqksRPa74p
knightmb
Sr. Member
****
Offline Offline

Activity: 308


Timekoin - Save Electricity, Don't Waste It!


View Profile WWW
July 23, 2010, 06:50:05 PM
 #4

Awesome, got some extra slow machines to wipe/test with then. Will be curious to see what it does on faster machines as well.  Smiley

knightmb
Sr. Member
****
Offline Offline

Activity: 308


Timekoin - Save Electricity, Don't Waste It!


View Profile WWW
July 23, 2010, 07:01:50 PM
 #5

So far, awesome increase for the Windows client. Doing about 500 blocks / second using a private node. After the timer is done, I'll try it on a wild node out on the Internet to see if the speed is just as good.

knightmb
Sr. Member
****
Offline Offline

Activity: 308


Timekoin - Save Electricity, Don't Waste It!


View Profile WWW
July 23, 2010, 07:32:58 PM
 #6

By making some adjustments to the database settings, I was able to make the initial block download about 5 times faster.  It downloads in about 30 minutes.

The database default had it writing each block to disk synchronously, which is not necessary.  I changed the settings to let it cache the changes in memory and write them out in a batch.  Blocks are still written transactionally, so either the complete change occurs or none of it does, in either case the data is left in a valid state.

I only enabled this change during the initial block download.  When you come within 2000 blocks of the latest block, these changes turn off and it slows down to the old way.
The first part of the 70k downloaded in about 20 minutes, the rest of the remaining 2000 blocks in about 7 minutes, so only 27 minutes from 0 to 100%, very nice! Is there a safety reason to stop within the last 2000 blocks or can it be tweaked to stop at remaining 500 blocks for example?

In case anyone was curious, average download speed was around 3 Mbps for each burst of download.

knightmb
Sr. Member
****
Offline Offline

Activity: 308


Timekoin - Save Electricity, Don't Waste It!


View Profile WWW
July 23, 2010, 08:04:26 PM
 #7

Blocks download just as fast in the wild (about 3 Mbps), so a big thumbs up, this will certainly get new users up and running much quicker.

satoshi
Founder
Sr. Member
*
qt
Offline Offline

Activity: 364


View Profile
July 23, 2010, 08:13:27 PM
 #8

Is there a safety reason to stop within the last 2000 blocks or can it be tweaked to stop at remaining 500 blocks for example?
Not really.  I'll change it to 1000 next time.
knightmb
Sr. Member
****
Offline Offline

Activity: 308


Timekoin - Save Electricity, Don't Waste It!


View Profile WWW
July 23, 2010, 08:29:15 PM
 #9

Is there a safety reason to stop within the last 2000 blocks or can it be tweaked to stop at remaining 500 blocks for example?
Not really.  I'll change it to 1000 next time.
Either way, it's an amazing improvement over the older version! Great work!

I ran two side by side (using the current release, and your new faster one) and in the time it took the newer one to download (twice) after a wipe, the original was still only about a few thousand blocks into it's download.  Grin

lachesis
Full Member
***
Offline Offline

Activity: 210


View Profile
July 24, 2010, 02:57:12 AM
 #10

Congrats! I ran with libeatmydata for the initial download I had to do after I accidentally wiped my blk*.dat and it helped a bit, but this sounds even better!

Bitcoin Calculator | Scallion | GPG Key | WoT Rating | 1QGacAtYA7E8V3BAiM7sgvLg7PZHk5WnYc
Bitcoiner
Member
**
Offline Offline

Activity: 70


View Profile
July 24, 2010, 03:54:55 PM
 #11

Nice; perhaps it's only necessary to immediately flush transactions that are "self-originated"; i.e. if you spent or received some money?

Want to thank me for this post? Donate here! Flip your coins over to: 13Cq8AmdrqewatRxEyU2xNuMvegbaLCvEe  Smiley
jgarzik
Legendary
*
qt
Offline Offline

Activity: 1470


View Profile
July 24, 2010, 04:50:03 PM
 #12

IMO it should be time-based ("1 day"), not an arbitrary number.

Jeff Garzik, bitcoin core dev team and BitPay engineer; opinions are my own, not my employer.
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
knightmb
Sr. Member
****
Offline Offline

Activity: 308


Timekoin - Save Electricity, Don't Waste It!


View Profile WWW
July 24, 2010, 04:59:33 PM
 #13

No one mentioned anything about the Linux client, but I've tested it on mine, runs 500 blocks / second during download, everything else works great, no issues to report so far.

Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!