Title: Automated Bitcoin startup with backup Post by: DrakonX on July 16, 2018, 10:25:42 AM I use the Windows taskmanager to start automaticly backup the blockchain every I logon on with windows from my internal hdd to the external one. Afterwards Bitcoin Core starts automaticly too.
Here is how it goes:
*Note: I inserted the colored step later with an update of this guide. Note: The first execution might take several hours. Title: Re: Automated Bitcoin startup with backup Post by: bob123 on July 16, 2018, 10:38:45 AM I use the Windows taskmanager to start automaticly backup the blockchain every I logon on with windows There isn't a real reason to backup the blockchain daily (except maybe if you are offering an online service which requires an intact blockchain to operate normally). Usually it should be enough if you just backup the chainstate folder. It contains all relevant information (UTXO's, current verification state of the blockchain, .. .) and it is just a few GB big. You can find it in the data directory of core (usually: C:\Users\YourUserName\Appdata\Roaming\Bitcoin for windows and ~/.bitcoin/ for linux). Is there a specific reason you need to backup the blockchain daily ? Title: Re: Automated Bitcoin startup with backup Post by: DrakonX on July 16, 2018, 10:59:58 AM The only reason for me to backup daily is that it tooks extremely long to download the last few 10,000 blocks again when I need to redownload the whole blockchain after an blockchain corruption. It happened at least once before. I even backup the backup every month, which I do manually.
Mayby I should proof the method with the chainstate folder though. Title: Re: Automated Bitcoin startup with backup Post by: HCP on July 20, 2018, 02:58:30 AM That all seems a bit redundant, given that the blockchain is effectively backed up by all the nodes... I'd be more worried about making sure I had backups of wallet.dat than the blocks/chainstate.
But I guess if you don't have storage space issues then I guess it doesn't hurt to have 3 copies of the blockchain!!?! :P Title: Re: Automated Bitcoin startup with backup Post by: DrakonX on July 23, 2018, 10:00:51 PM We're living in a world where storage space is less an issue than time is. ;)
Title: Re: Automated Bitcoin startup with backup Post by: DrakonX on July 30, 2018, 09:33:37 AM Keeping just the chainstate folder seems not to be enough, becaus it seems to build up the whole blockchain from scratch if I delete all the files and folders except the chainstate folder.
Title: Re: Automated Bitcoin startup with backup Post by: bob123 on August 03, 2018, 05:59:45 AM Keeping just the chainstate folder seems not to be enough, becaus it seems to build up the whole blockchain from scratch if I delete all the files and folders except the chainstate folder. I hope you didn't delete your wallet.dat with 'all the files and folders' you have deleted. If you want to save space, you can run core with the prune-command and set a limit on the maximum available space. Core is building the chainstate by downloading and verifying all blocks. You can't just delete all files except the chainstate folder and expect it to still work properly. But you can backup the chainstate directory to have a full functional backup of the current state (balance, UTXOs, ..) the next time it somehow gets corrupted. It is a fast way to make a backup and to return to a functional state if your system somehow doesn't work properly anymore. But the chainstate folder itself does not replace the whole data directory. Title: Re: Automated Bitcoin startup with backup Post by: DrakonX on August 04, 2018, 08:46:01 PM I suppose, it is right what you say about the chainstate folder. Would you please tell me, how long it takes to reconstruct the whole blockchain with just the chainstate folder?
Title: Re: Automated Bitcoin startup with backup Post by: bob123 on August 06, 2018, 07:51:35 AM Would you please tell me, how long it takes to reconstruct the whole blockchain with just the chainstate folder? You can not reconstruct the blockchain out of this data. Your individiual chainstate folder stores data related to your personal transactions / balances. This is being derived from the blockchain data. This basically is your 'backup' of 'synching the blockchain'. To restore if something fails, delete and redownload (or use your backup the blockchain) the corrupted files. In case of something happening which leads to your chainstate to break (would require another sync) you can simply copy your backup of the chainstate folder into the correct directory. Title: Re: Automated Bitcoin startup with backup Post by: DrakonX on August 07, 2018, 02:03:58 PM So I need to backup everything, but in case of a failure I just need to recover the chainstate folder with the backup?
Title: Re: Automated Bitcoin startup with backup Post by: bob123 on August 07, 2018, 02:13:56 PM So I need to backup everything, but in case of a failure I just need to recover the chainstate folder with the backup? This depends on what you define as 'need to backup'. The only thing you really need to backup is the wallet.dat. This file contains all of your private keys (which are needed to access your BTC). Anything further is helpful, but not necessary to be backed up. If you don't want to resync the whole blockchain (which might take some time depending on the specs), you'll need to backup the chainstate folder. In case of a corruption of your DB, you will be able to remove the block files and redownload the blocks again. And instead of needing to completely process the whole blockchain to build the chainstate, you will be able to use your backup instead. Instead of downloading the corrupted blocks, you might also store the whole blockchain as a backup. But this does need quite a lot of space and doesn't bring that much time advantage with it. Downloading a few files isn't the problem. Building the chainstate if it got corrupted takes the most work (CPU, I/O, ..) and therefore time. Title: Re: Automated Bitcoin startup with backup Post by: DrakonX on August 12, 2018, 03:18:49 PM If you don't want to resync the whole blockchain (which might take some time depending on the specs), you'll need to backup the chainstate folder. This is exactly what I intended, but it doesn't seem to work. I made a backup of everything, I deleted the original data directory and afterwards I copied the backup of the chainstate folder (and only this) back to its original place. The result was that I had to redownload (rsync as you name it) the whole blockchain with consumption of all the time a complete download (and rebuild) requires.In case of a corruption of your DB, you will be able to remove the block files and redownload the blocks again. And instead of needing to completely process the whole blockchain to build the chainstate, you will be able to use your backup instead. Title: Re: Automated Bitcoin startup with backup Post by: bob123 on August 13, 2018, 07:02:46 AM I made a backup of everything, I deleted the original data directory and afterwards I copied the backup of the chainstate folder (and only this) back to its original place. If you have deleted the data directory and only copied the backup of the chainstate into it, you did delete your local copy of the blockchain. You need to keep your blocks (*.blk files) if you aren't running core with pruning. The result was that I had to redownload (rsync as you name it) the whole blockchain with consumption of all the time a complete download (and rebuild) requires. What i was revering to with 'syncing' was the process of core reading the block files and building the chainstate (which usually takes way more time than the download process). Title: Re: Automated Bitcoin startup with backup Post by: TheArchaeologist on August 23, 2018, 07:21:08 AM Quote So what if your hard drive crashes? Then you get a new hard drive and sync the entire blockchain again. All full nodes have a copy of the entire chain so there are plenty of backups!Quote Did you know you can store bitcoins blockchain on USB stick or external hard drive as the startup directory? Sure you can do this. But given the size of the entire thing it would be hard to find a USB stick which can hold the entire blockchain! But you can store the data everywhere and then reference it to bitcoind on startop (or at the bitcoin.conf file):Code: bitcoind --datadir='/path/to/alternative/' Title: Re: Automated Bitcoin startup with backup Post by: TheArchaeologist on August 23, 2018, 07:28:31 AM The only reason for me to backup daily is that it tooks extremely long to download the last few 10,000 blocks again when I need to redownload the whole blockchain after an blockchain corruption. Is that because your on a slow connection? Or do you use a slow drive? Did you try to set the -dbcache parameter to a higher value because in my experience this makes a hugh difference when rebuilding the entire blockchain.I can recreate the entire blockchain in a couple of hours even with -txindex when setting -dbache to 16384 and storing the result on an SSD. Backups off the public data are literally everywhere, I would worry most about your personal data: the wallet.dat file. Title: Re: Automated Bitcoin startup with backup Post by: DrakonX on August 30, 2018, 05:08:06 PM Is that because your on a slow connection? Or do you use a slow drive? Presumably both is true.I don't use the wallet.dat from Bitcoin Core, but Bitcoin Armory. Right now I'm not motivated to try ist with the -dbcache parameter. I'll try as soon as I have to rebuild the blockchain. Title: Re: Automated Bitcoin startup with backup Post by: bob123 on August 31, 2018, 07:18:18 AM Quote Did you know you can store bitcoins blockchain on USB stick or external hard drive as the startup directory? Sure you can do this. But given the size of the entire thing it would be hard to find a USB stick which can hold the entire blockchain! Actually, flash memory is extremely cheap right now. You can buy a 256 gb stick (which is enough for the whole blockchain) for 35$. And a 126 gb stick only costs around 20$. With the current overthrow of the storage market, most prices are hilariously low. Especially flash memory (SSDs / USBs). |