Bitcoin Forum
December 16, 2017, 01:42:12 PM *
News: Latest stable version of Bitcoin Core: 0.15.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: [1]
  Print  
Author Topic: Speed up the import of bootstrap.dat using mining hardware - is it possible?  (Read 1510 times)
ruevs
Newbie
*
Offline Offline

Activity: 2


View Profile
September 16, 2014, 08:21:40 AM
 #1

After installing a fresh Bitcoin Core (0.9.2.1) on a 2GHz Core2Duo laptop and downloading the bootstrap.dat file (~21GB at 6MB/s) I started the client and it it started importing the block chain from disk.

Initially the import speed was about 20 seconds/week (for the blocks from 4 years ago). In about an hour or two the import progressed to 37 weeks behind... and nine hours later it is only at 21 weeks behind and the speed is about 1 hour/week

It seems that the complexity of the import algorithm is worse than linear with the number of transactions per block and/or the block chain size.

   https://blockchain.info/charts/n-transactions-per-block?timespan=all&showDataPoints=false&daysAverageString=1&show_header=true&scale=0&address=
   https://blockchain.info/charts/blocks-size?timespan=all&showDataPoints=false&daysAverageString=1&show_header=true&scale=0&address=

The transactions per block have increased (from about 2012-04 to 2014-09) from ~50 to ~400 - an increase of 8x and the block chain size has increased from ~1000 to ~21000 - and increase of 21x. While (for me) the import speed of bootstrap.dat has decreased from ~20s/wk to ~3600s/wk - a decrease of ~180x

So my question is (and it may be a stupid question - I have not looked at the code or read about the import process) does the import process involve many calculations of SHA-256? If it does it may be possible to speed up the bootstrap.dat import by using mining hardware - is it? 

Just for fun I tried to run p2pool (http://p2pool.in/) and sure enough it said "-10 bitcoin is downloading blocks"... so no luck :-)


P.S. I did browse the "interesting" threads :-)

https://bitcointalk.org/index.php?topic=145386.0
https://bitcointalk.org/index.php?topic=51456.0

I can only imagine how bad the import would have been today without the 0.8+ versions https://bitcointalk.org/index.php?topic=130160.msg1392310#msg1392310
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1513431732
Hero Member
*
Offline Offline

Posts: 1513431732

View Profile Personal Message (Offline)

Ignore
1513431732
Reply with quote  #2

1513431732
Report to moderator
gmaxwell
Moderator
Legendary
*
qt
Offline Offline

Activity: 2366



View Profile
September 16, 2014, 06:44:52 PM
 #2

It seems that the complexity of the import algorithm is worse than linear with the number of transactions per block and/or the block chain size.
[...]
I have not looked at the code or read about the import process) does the import process involve many calculations of SHA-256?
Clearly you haven't: It's very nearly hlinear with the transactions.  There just were very few transactions in the past. There is a lot of sha256, but most of the import time is spent elsewhere. (ECDSA, writing out the database, etc.). There are some changes in the pipe that make it a fair bit faster.

Bitcoin will not be compromised
jl2012
Legendary
*
Offline Offline

Activity: 1750


View Profile
September 17, 2014, 03:27:15 AM
 #3

ASIC are specialized in mining and mining only, i.e. looking for SHA256(SHA256) with many leading 0-bits. They are not used for general SHA256.

Donation address: 374iXxS4BuqFHsEwwxUuH3nvJ69Y7Hqur3 (Bitcoin ONLY)
LRDGENPLYrcTRssGoZrsCT1hngaH3BVkM4 (LTC)
PGP: D3CC 1772 8600 5BB8 FF67 3294 C524 2A1A B393 6517
ruevs
Newbie
*
Offline Offline

Activity: 2


View Profile
September 19, 2014, 11:33:27 AM
 #4

Thank you for the clarification.
Sukrim
Legendary
*
Offline Offline

Activity: 2212


View Profile
September 19, 2014, 01:17:38 PM
 #5

While signature checking might profit from e.g. GPUs, there is a lot of data transfer too, which is something that GPUs are not very good at (better give them something to work at for some time and then collect the results, don't constantly throw small bits of work whenever a core is free).

So yes, it might be possible to speed this up a little bit by using custom or more specialized (not mining!) hardware, there are likely other limitations that come into play and destroy that advantage. It's a one time thing though and once you did it, you can also copy the index to new installations (since you know it comes from a trusted source).

https://bitfinex.com <-- leveraged trading of BTCUSD, LTCUSD and LTCBTC (long and short) - 10% discount on fees for the first 30 days with this refcode: x5K9YtL3Zb
Mail me at Bitmessage: BM-BbiHiVv5qh858ULsyRDtpRrG9WjXN3xf
Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!