deleting old blocks to save space

[btc_enigma]

Hi,

I have a bitcoind instance running and I have a abe-blockexplorer type daemon that stores blocks/txs from bitcoind in local database.
So to save space I deleted old block files in bitcoind, old files of type blk*.dat, rev*.dat from .bitcoin

Now bitcoind is sometimes just exiting with the following error

2014-12-24 16:15:43 Added time data, samples 200, offset +0 (+0 minutes)
2014-12-24 16:15:48 Unable to open file /home/enigma/.bitcoin/blocks/blk00159.dat
2014-12-24 16:15:48 ERROR: ReadBlockFromDisk : OpenBlockFile failed

If I restart bitcoind it starts running again for a few blocks and is then again exiting . I did not touch the contents of .bitcoin/chainstate
Any thoughts on why this is happening ? My thinking is since I didnt touch chainstate, bitcoind should be still able to verify newer blocks

[dserrano5]

Some new peer joining the network may be asking to download old blocks from your node. When your node goes to fetch them from the disk, it finds that the blocks aren't there so it exits. Currently there is no option but to store the whole blockchain in disk—however in the near future we'll be able to tell Bitcoin Core "hey, don't use more than X Gb of disk space for storing blocks". You can do some searches on "pruning" to learn more about that.

[btc_enigma]

Thanks you are right
I tried getblock on block number 100 and bitcoind exited
bitcoind getblock 000000007bc154e0fa7ea32218a72fe2c1bb9f86cf8c9ebf9a715ed27fdb229a

Why should  bitcoind exit in this case ? Can I  patch bitcoind so it that it just ignores this error and keeps running ?
Is there a penalty associated if the node doesn't relay old blocks ?

[gmaxwell]

It is not quite so simple. You'll abuse your peers if you make them think you can serve them blocks but fail. If you erase too much you'll be unable to reorganize.   We plan on actually supporting this, in a sane organized manner probably in the next major version. (The fact that you were able to try that at all is a result of our work to facilitate that feature, prior to 0.8 there was no data you could just delete without making yourself unable to verify new blocks.)

[btc_enigma]

I am not sure if all nodes should store all the block data to support new nodes,  maybe a few altruistic nodes could do so. Already people are not running nodes by getting scared of the costs. Maybe give an option to only keep current block-k to support reorganization ?

Thank you , I know you devs are working on this,  its just very tempting to delete old blocks ! Looking forward to next major version

[goosoodude]

I am not sure if all nodes should store all the block data to support new nodes,  maybe a few altruistic nodes could do so. Already people are not running nodes by getting scared of the costs. Maybe give an option to only keep current block-k to support reorganization ?

Thank you , I know you devs are working on this,  its just very tempting to delete old blocks ! Looking forward to next major version

How can you be sure that the node is altruistic? The strength of Bitcoin lies in thoudsands of copies of the blockchain throughout the world verifying each transaction.

[btc_enigma]

I am not sure if all nodes should store all the block data to support new nodes,  maybe a few altruistic nodes could do so. Already people are not running nodes by getting scared of the costs. Maybe give an option to only keep current block-k to support reorganization ?

Thank you , I know you devs are working on this,  its just very tempting to delete old blocks ! Looking forward to next major version

How can you be sure that the node is altruistic? The strength of Bitcoin lies in thoudsands of copies of the blockchain throughout the world verifying each transaction.

Not storing older blocks is not in anyway reducing the strength of the network . Nodes could still store enough blocks to support forks / any reorganization needed to verify newer transactions. Blocks older than say 3 months need not be stored.

[gmaxwell]

Making it potentially infeasible or much more costly to introduce new nodes to the network would undermine security enormously, it's critical that this not be undermined. In any case, we've been working on this for _years_ uninformed navel gazing on bct isn't going to move it along.

None of you showed up to test the patch when we made a public call in the hopes of having an initial version in 0.10. Perhaps in the next version.

[btc_enigma]

Just had a look at https://github.com/bitcoin/bitcoin/pull/4701 . Will try to test.

[dserrano5]

I found no issues when I tried the pull request 4701, just for the record.