Bitcoin Forum
December 13, 2017, 03:46:47 AM *
News: Latest stable version of Bitcoin Core: 0.15.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: [1]
  Print  
Author Topic: Downloading pruned blockchain inefficient?  (Read 865 times)
SebastianJu
Legendary
*
Offline Offline

Activity: 1988


Legendary Escrow Service - Tip Jar in Profile


View Profile WWW
July 12, 2017, 03:39:29 AM
 #1

Hello,

I just download the blockchain in pruned mode. I downloaded the unpruned blockchain some days ago. It was way faster and I notice that downloading a pruned blockchain now takes way longer. It looks like takes more cpu and it uses the harddisc more often. It isn't the swap file also and I gave bitcoin-qt 6gb RAM and 16 threads. The same setup. Antivirus is also not running.

So can it be that the pruning is constantly done and writing and reading from the harddisc in an inefficient way? The same way like it works with single blocks when it is already synchronized? If so then maybe it would make sense to make the prune work just every couple of minutes. This might save ressources then.

Greetings!
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
TexasHuck
Full Member
***
Offline Offline

Activity: 126

Ⓚ Kore Projects CTO Ⓚ


View Profile WWW
July 12, 2017, 03:54:36 AM
 #2

This has been my experience as well.  I was thinking there could be some type of issue with the indexing of the values retained from the prune.  I would interested to see a play by play of the prune in action.  Even pseudo code would be helpful... thanks for posting!

Huck

Ⓚ Kore Projects - Privacy and Security - This is our Kung Fu..... Ⓚ
achow101
Moderator
Legendary
*
Offline Offline

Activity: 1246


17kKQppUsngUiByDsce4JXoZEjjpvX9bpR


View Profile WWW
July 12, 2017, 04:22:16 AM
 #3

Pruning will be more taxing on your disk as it has to constantly read and write to the disk far more than without pruning. This is because it deletes things on the fly. That means that, for each block, it will need to additionally compute what blocks should be deleted, look up where the block is, delete it, and remove it from the database. That will use extra CPU, RAM, and disk IO as there are simply more things that Core is doing in order to prune.

tspacepilot
Legendary
*
Offline Offline

Activity: 1400


I may write code in exchange for bitcoins.


View Profile
July 12, 2017, 07:13:59 PM
 #4

I was away from following development on core for some time.  Last time I was up-to-speed pruning was just starting to be implemented.  @achow101: where I can read about the pruning implementation (obviously I can study the source, but I suppose there's a writeup somewhere)?

achow101
Moderator
Legendary
*
Offline Offline

Activity: 1246


17kKQppUsngUiByDsce4JXoZEjjpvX9bpR


View Profile WWW
July 12, 2017, 07:53:47 PM
 #5

@achow101: where I can read about the pruning implementation (obviously I can study the source, but I suppose there's a writeup somewhere)?
AFAIK, there is no write up of how pruning is implemented. You will have to look in the source code for that. Since manual pruning is now available via an RPC command, I would recommend that you start looking there to see what it does to prune: https://github.com/bitcoin/bitcoin/blob/master/src/rpc/blockchain.cpp#L849

-ck
Staff
Legendary
*
Offline Offline

Activity: 2366


Ruu \o/


View Profile WWW
July 12, 2017, 08:20:53 PM
 #6

If you want to run a pruned node but have the storage capacity to download the whole blockchain it is indeed much faster to download the whole blockchain and then restart bitcoind in pruned mode to do the pruning afterwards since it is constantly being pruned during the download process otherwise.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
tspacepilot
Legendary
*
Offline Offline

Activity: 1400


I may write code in exchange for bitcoins.


View Profile
July 12, 2017, 08:43:40 PM
 #7

@achow101: where I can read about the pruning implementation (obviously I can study the source, but I suppose there's a writeup somewhere)?
AFAIK, there is no write up of how pruning is implemented. You will have to look in the source code for that. Since manual pruning is now available via an RPC command, I would recommend that you start looking there to see what it does to prune: https://github.com/bitcoin/bitcoin/blob/master/src/rpc/blockchain.cpp#L849

Thanks!  I also rhappened to re-look over the original whitepaper and I noticed (for the first time) the section on "reclaiming disk space".  I guess I'm trying to figure out what the disadvantages are (if any) over running a pruned node vs non-pruned.

EDIT: I found some further notes about pruning in the release notes for 0.11: https://github.com/bitcoin/bitcoin/blob/v0.11.0/doc/release-notes.md#block-file-pruning

SebastianJu
Legendary
*
Offline Offline

Activity: 1988


Legendary Escrow Service - Tip Jar in Profile


View Profile WWW
July 15, 2017, 02:04:21 AM
 #8

Thanks for the answer. It might be useful for the future to implement something like
if more than 100 blocks behind then download normally 50 blocks then prune on those. Repeat.
gmaxwell
Moderator
Legendary
*
qt
Offline Offline

Activity: 2366



View Profile
July 15, 2017, 02:46:22 AM
 #9

If you want to run a pruned node but have the storage capacity to download the whole blockchain it is indeed much faster to download the whole blockchain and then restart bitcoind in pruned mode to do the pruning afterwards since it is constantly being pruned during the download process otherwise.

Have you actually measured that? If it's true-- it's a bug. the pruning should delete a whole blockfile at a time (128MB) and shouldn't take much time at all.  It's totally plausible to me that there is a bug here, but I've not noticed it myself or heard it reported before.

Bitcoin will not be compromised
-ck
Staff
Legendary
*
Offline Offline

Activity: 2366


Ruu \o/


View Profile WWW
July 16, 2017, 09:12:08 PM
 #10

If you want to run a pruned node but have the storage capacity to download the whole blockchain it is indeed much faster to download the whole blockchain and then restart bitcoind in pruned mode to do the pruning afterwards since it is constantly being pruned during the download process otherwise.

Have you actually measured that? If it's true-- it's a bug. the pruning should delete a whole blockfile at a time (128MB) and shouldn't take much time at all.  It's totally plausible to me that there is a bug here, but I've not noticed it myself or heard it reported before.
Only when pruning was first available did I experience this and I did not do a fair comparison of identical hardware so no I cannot say for certain. I've not downloaded the whole blockchain since.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!