|
Title: Speed up the import of bootstrap.dat using mining hardware - is it possible? Post by: ruevs on September 16, 2014, 08:21:40 AM After installing a fresh Bitcoin Core (0.9.2.1) on a 2GHz Core2Duo laptop and downloading the bootstrap.dat file (~21GB at 6MB/s) I started the client and it it started importing the block chain from disk.
Initially the import speed was about 20 seconds/week (for the blocks from 4 years ago). In about an hour or two the import progressed to 37 weeks behind... and nine hours later it is only at 21 weeks behind and the speed is about 1 hour/week It seems that the complexity of the import algorithm is worse than linear with the number of transactions per block and/or the block chain size. https://blockchain.info/charts/n-transactions-per-block?timespan=all&showDataPoints=false&daysAverageString=1&show_header=true&scale=0&address= https://blockchain.info/charts/blocks-size?timespan=all&showDataPoints=false&daysAverageString=1&show_header=true&scale=0&address= The transactions per block have increased (from about 2012-04 to 2014-09) from ~50 to ~400 - an increase of 8x and the block chain size has increased from ~1000 to ~21000 - and increase of 21x. While (for me) the import speed of bootstrap.dat has decreased from ~20s/wk to ~3600s/wk - a decrease of ~180x So my question is (and it may be a stupid question - I have not looked at the code or read about the import process) does the import process involve many calculations of SHA-256? If it does it may be possible to speed up the bootstrap.dat import by using mining hardware - is it? Just for fun I tried to run p2pool (http://p2pool.in/) and sure enough it said "-10 bitcoin is downloading blocks"... so no luck :-) P.S. I did browse the "interesting" threads :-) https://bitcointalk.org/index.php?topic=145386.0 https://bitcointalk.org/index.php?topic=51456.0 I can only imagine how bad the import would have been today without the 0.8+ versions https://bitcointalk.org/index.php?topic=130160.msg1392310#msg1392310 Title: Re: Speed up the import of bootstrap.dat using mining hardware - is it possible? Post by: gmaxwell on September 16, 2014, 06:44:52 PM It seems that the complexity of the import algorithm is worse than linear with the number of transactions per block and/or the block chain size. Clearly you haven't: It's very nearly hlinear with the transactions. There just were very few transactions in the past. There is a lot of sha256, but most of the import time is spent elsewhere. (ECDSA, writing out the database, etc.). There are some changes in the pipe that make it a fair bit faster.[...] I have not looked at the code or read about the import process) does the import process involve many calculations of SHA-256? Title: Re: Speed up the import of bootstrap.dat using mining hardware - is it possible? Post by: jl2012 on September 17, 2014, 03:27:15 AM ASIC are specialized in mining and mining only, i.e. looking for SHA256(SHA256) with many leading 0-bits. They are not used for general SHA256.
Title: Re: Speed up the import of bootstrap.dat using mining hardware - is it possible? Post by: ruevs on September 19, 2014, 11:33:27 AM Thank you for the clarification.
Title: Re: Speed up the import of bootstrap.dat using mining hardware - is it possible? Post by: Sukrim on September 19, 2014, 01:17:38 PM While signature checking might profit from e.g. GPUs, there is a lot of data transfer too, which is something that GPUs are not very good at (better give them something to work at for some time and then collect the results, don't constantly throw small bits of work whenever a core is free).
So yes, it might be possible to speed this up a little bit by using custom or more specialized (not mining!) hardware, there are likely other limitations that come into play and destroy that advantage. It's a one time thing though and once you did it, you can also copy the index to new installations (since you know it comes from a trusted source). |