Bitcoin Forum
December 04, 2016, 10:16:06 AM *
News: To be able to use the next phase of the beta forum software, please ensure that your email address is correct/functional.
 
   Home   Help Search Donate Login Register  
Pages: [1]
  Print  
Author Topic: Blockchain synchronization speed improvements incoming  (Read 3062 times)
gmaxwell
Moderator
Legendary
*
qt
Offline Offline

Activity: 2016



View Profile
January 03, 2012, 08:28:53 PM
 #1

I thought people would be excited to hear that I finally got around to really investigating why the initial chain sync-up was so slow (at least for some users) and that I found an easily fixed problem.  

It's well known that the reference software does a lot of unneeded synchronous disk IO during syncup (in addition to the large amount of needed async IO), so syncup is expected to be slow right now on systems without fast disks... but even on systems with super fast disks and CPUs, the syncup is still quite slow. People frequently make all kinds of wild network related suggestions that they think will help syncup which I've been discounting: https://bitcointalk.org/index.php?topic=53389.msg640824#msg640824, but I hadn't had a chance to sit down and work on any of the actual causes of slowness.

Well, it turns out that when the wallet encryption was introduced some usage of mlock() was added to keep private key data out of swap. This is a good thing, but the mlock is used naively which is bad because it's slow (results in a TLB flush). That wouldn't matter too much, except a side effect of the change was to mlock all the memory used by a very common data type used all through bitcoin. This is pointless because most of the usage doesn't contain private data. (and may have even adversely impacted security of the encrypted wallets on systems with limited mlockable memory)

This was somewhat hard to track down— it doesn't show up in the oprofile cycle counter (it would have shown up in TLB flushes, but thats only obvious in retrospect) and didn't show up in some other performance tools like valgrind/callgrind (which is too simplistic an emulation to know mlock is slow). I eventually caught it in ltrace output.

There is still some ongoing discussion about the best path to fix this, but the performance improvement at least on some nodes is quite significant. YMMV, if you're not on a super fast SSD this improvement will be much smaller (at least until the excessive fsync usage is resolved), and my AMD AMD Phenom(tm) II X6 1055T  with kernel 2.6.35.14-106.fc14.x86_64 might mlock slower than some others.

This should also speed up the client generally, though its especially obvious for the chain syncup.

Enough talk, How about some pictures:

The blue and red lines are variants of this fix, green is the stock client.



(note the log scale)

There is more discussion on the pull request (https://github.com/bitcoin/bitcoin/pull/740)
1480846566
Hero Member
*
Offline Offline

Posts: 1480846566

View Profile Personal Message (Offline)

Ignore
1480846566
Reply with quote  #2

1480846566
Report to moderator
1480846566
Hero Member
*
Offline Offline

Posts: 1480846566

View Profile Personal Message (Offline)

Ignore
1480846566
Reply with quote  #2

1480846566
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1480846566
Hero Member
*
Offline Offline

Posts: 1480846566

View Profile Personal Message (Offline)

Ignore
1480846566
Reply with quote  #2

1480846566
Report to moderator
1480846566
Hero Member
*
Offline Offline

Posts: 1480846566

View Profile Personal Message (Offline)

Ignore
1480846566
Reply with quote  #2

1480846566
Report to moderator
Mushoz
Hero Member
*****
Offline Offline

Activity: 686


Bitbuy


View Profile WWW
January 04, 2012, 12:50:42 AM
 #2

This is amazing, great work!  Shocked Do you happen to have any idea when this is going to get merged into a release of the mainclient? Thanks!

www.bitbuy.nl - Koop eenvoudig, snel en goedkoop bitcoins bij Bitbuy!
Transisto
Donator
Legendary
*
Offline Offline

Activity: 1624



View Profile WWW
January 04, 2012, 02:25:57 AM
 #3

Awesome ! I couldn't understand why syncing the blockchain was taking longer than downloading a 30gb torrent.
gmaxwell
Moderator
Legendary
*
qt
Offline Offline

Activity: 2016



View Profile
January 05, 2012, 03:17:11 PM
 #4

This is amazing, great work!  Shocked Do you happen to have any idea when this is going to get merged into a release of the mainclient? Thank!

It was merged about an hour ago, and will ship in the next release of the client.
finway
Hero Member
*****
Offline Offline

Activity: 714


View Profile
January 06, 2012, 07:04:52 AM
 #5

Thank you for your contribution, gmaxwell.

BTW: Is it safe? I mean, is zero-after-free still there ?

gmaxwell
Moderator
Legendary
*
qt
Offline Offline

Activity: 2016



View Profile
January 06, 2012, 07:12:03 AM
 #6

Thank you for your contribution, gmaxwell.

BTW: Is it safe? I mean, is zero-after-free still there ?

Yes, the zero after free was kept.  And it still mlocks the private key data (which is slow because its still doing it for every allocation but it only adversely impacts key pool filling, this will be fixed later as it provides less benefit and needs a more complicated fix to maintain a pool of mlocked memory for that purpose).
pusle
Member
**
Offline Offline

Activity: 89


View Profile
January 06, 2012, 07:25:11 AM
 #7


Thank you, this is really nice!  I'm sending you some coins for this Smiley
prof7bit
Hero Member
*****
Offline Offline

Activity: 672



View Profile WWW
January 08, 2012, 11:41:04 AM
 #8

Why does it do synchronized disk IO anywhere anyways? Its not used with removable media (its even almost impossible using it on external removable media when it is implemented this way anyways.

Old Bitcoins recycling and disposal: 1N7ZcHoe77JncRYAg27RVqJZbrjqUQ77WE
Blockchain.info -> Multibit converter (and much more) https://github.com/prof7bit/wallet-key-tool
gmaxwell
Moderator
Legendary
*
qt
Offline Offline

Activity: 2016



View Profile
January 09, 2012, 03:02:49 AM
 #9

Why does it do synchronized disk IO anywhere anyways? Its not used with removable media (its even almost impossible using it on external removable media when it is implemented this way anyways.

Because the sync disk IO is important to avoid losing database updates or corrupting the database when the power goes out during a write—  which matters a lot for the wallet but not everything else.  However, it currently uses the same database setup (though different files) for all things.

So an update on this— After having some people test the RC builds on lower performing windows boxes (e.g. random spinning disk systems with only 1GB ram) they're still seeing 18 hour syncs, so at least on those systems this particular doesn't make as big of a difference as it does on fast/linux/ssd system.   I'm still working on finding additional improvements for all systems.

If you'd like to benchmark it on your own system(s), you should add logtimestamps=1  to your bitcoin.conf so your debug.log file will get a timestamp for every entry. Then you can start it up and check later to see how long it took (this is how I generated the graphs used on this page).

There are RC builds with this fix (as well as some of the recent deadlock fixes), made by Matt Corallo <matt@bluematt.me> using the reproducible build process:
http://dl.dropbox.com/u/29653426/bitcoin-0.5.2rc1.tar.bz2
http://dl.dropbox.com/u/29653426/bitcoin-0.5.2rc1-win32.tar.bz2

(though, er, personally I wouldn't run binaries from dropbox urls, except for the fact that I knew Matt posted them)

If you do try out the RC builds please let me know (either here, in #bitcoin-dev on irc, or in PM) as getting a few more people trying them out is the only thing delaying their official release right now. Always make sure to backup your wallet before trying new software (and periodically even if you're on old software!).
Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!