Bitcoincore copy blocks for new synchronization. Tell me friends

[Muromskiy]

Hello friends! Instead of downloading the entire blockchain to the bitcoincore wallet, I decided to copy it from a friend so that synchronization would be faster. This is the first time I've done it this way.
Please tell me friends, which folders exactly need to be copied. In the installed wallet there are 2 large folders - Blocks (there is also an index folder inside Blocks) and Chainstate, should I copy all these folders to myself?

[LoyceV]

In the installed wallet there are 2 large folders - Blocks (there is also an index folder inside Blocks) and Chainstate, should I copy all these folders to myself?

Yes.
Make sure Bitcoin Core isn't running while copying.

[ABCbits]

Don't forget to check free space of your storage drive, since current size of those 2 folders is about 655GB or a bit more.

[notocactus]

Hello friends! Instead of downloading the entire blockchain to the bitcoincore wallet, I decided to copy it from a friend so that synchronization would be faster. This is the first time I've done it this way.

You can do it but there is risk.

It's recommendable to do Initial Blockchain Download (IBD) by yourself with your own node. It takes more time but it's safer than copy the blockchain from someone else or any available source.

Don't do what you're doing is safer.

Bitcoin Core pruned blockchain: download it here! (DON'T DO THIS!)

[Muromskiy]

Hello friends! Instead of downloading the entire blockchain to the bitcoincore wallet, I decided to copy it from a friend so that synchronization would be faster. This is the first time I've done it this way.

You can do it but there is risk.

It's recommendable to do Initial Blockchain Download (IBD) by yourself with your own node. It takes more time but it's safer than copy the blockchain from someone else or any available source.

Don't do what you're doing is safer.

Bitcoin Core pruned blockchain: download it here! (DON'T DO THIS!)

If you copy it yourself? Safe?

[LoyceV]

If you copy it yourself? Safe?

If you trust your friend and his computer.

[apogio]

Why not let it download the blockchain though? You sacrifice security for time. But the time needed to download the blockchain isn't so terrific...

By the way, copying 700GB of data, how long does it take? What if the data get corrupted in the process?

[ABCbits]

By the way, copying 700GB of data, how long does it take? What if the data get corrupted in the process?

I would speculate OP's internet is relative slow, where visiting his friend while bringing his external HDD/SSD is faster or more convenient. And i wouldn't worry about data corruption, unless old storage drive is involved.

[LoyceV]

By the way, copying 700GB of data, how long does it take?

I recently copied the data from a 1 TB USB 2.0 disk, which took about 6 hours.

I would speculate OP's internet is relative slow

In that case, letting Bitcoin Core verify the data after copying is still a good option. But: from what I've seen, usually it's the lack of RAM that slows down Bitcoin Core's IBD, and not the internet speed. In that case verification after copying still takes about the same amount of time.

And i wouldn't worry about data corruption

Me neither. A drive could just as well fail later.

[apogio]

But: from what I've seen, usually it's the lack of RAM that slows down Bitcoin Core's IBD, and not the internet speed. In that case verification after copying still takes about the same amount of time.

Yes, I believe it's because the dbcache size isn't enough to exceed the actual memory size of some blocks.

I would speculate OP's internet is relative slow, where visiting his friend while bringing his external HDD/SSD is faster or more convenient. And i wouldn't worry about data corruption, unless old storage drive is involved.

Would you do that though? That's the question I ask myself. I would choose to download and verify myself, but I realise there may not be a huge danger involved if done otherwise.

[LoyceV]

Yes, I believe it's because the dbcache size isn't enough to exceed the actual memory size of some blocks.

Blocks are small (no more than 4 MB). Chainstate is what consumes so much RAM.

Would you do that though? That's the question I ask myself. I would choose to download and verify myself, but I realise there may not be a huge danger involved if done otherwise.

I wouldn't do it, but I can think of scenarios where it's the only option, for instance if your internet has bandwidth quota.

MicroSD-prices are dropping faster than the blockchain is growing. It would be fun to "sell blockchains" online, but I kinda expect countries with limited internet to have an unreliable postal service.

[notocactus]

If you copy it yourself? Safe?

If you do you IBD, complete it, and copy it to another computer of yours, it's safe.

In your case, you copy it from your friend, risk starts from the source of blockchain, it's from your friend.

Don't trust, do IBD by yourself.

Boost Your Bitcoin Node Sync With UTXO Snapshots. Lopp tested it but if you can, do IBD by yourself as if you want to run and use your Bitcoin full node, you plan to use it for a very long time so convenience is not main concern, but safety is.

[ABCbits]

Talking about trust, it reminds me of UTXO snapshot feature on Bitcoin Core[1]. You only download blocks after when the UTXO snapshot created and you don't need to trust the one who create it since Bitcoin Core will verify everything.

I would speculate OP's internet is relative slow, where visiting his friend while bringing his external HDD/SSD is faster or more convenient. And i wouldn't worry about data corruption, unless old storage drive is involved.

Would you do that though? That's the question I ask myself. I would choose to download and verify myself, but I realise there may not be a huge danger involved if done otherwise.

Yes, assuming it's someone who i trust and i know them have good security practice. Although i probably i would perform reindex once just in case.

[1] https://blog.lopp.net/bitcoin-node-sync-with-utxo-snapshots/

[Cricktor]

~~~

From what I read in Lopp's blog post I don't think you're fully correct with your assumption that Bitcoin Core will "only" download blocks from the UTXO snapshot on. Or did I misunderstand you?

From what I understand what Lopp wrote, didn't try it myself so far, it will first sync from UTXO snapshot's blockheight to chaintip and then act like fully synced (sync progress will first jump to progress indicator of snapshot's blockheight and then work its way up to chaintip aka progress=1.000000.

Then in the background the node will start a sync from Genesis block upward to actually also verify that the loaded UTXO snapshot is indeed valid and not tampered. During this background sync the node is fully usable and like synced to chaintip with progress=1.000000. I guess you can ask the node to give you block or transaction data that it doesn't yet have downloaded in the background process. I assume you could load a wallet and spend coins from it or receive coins and showing an accurate balance.

Sounds like a fun test project the next time I have spare time and need to quickly spin up a new Bitcoin Core node. I also like the fact that it should work fine for a pruned node when you're constrained by storage space and still want and need a quickly fully synced pruned node.


Interpreted from

...
When you use AssumeUTXO, the node starts syncing from the snapshot to the chain tip, then the node downloads and verifies the old blockchain history in the background until it eventually verifies its initial UTXO state at the loaded snapshot, allowing it to ultimately obtain the same trustless security as a node that doesn’t use AssumeUTXO.

How to Use UTXO Snapshots

• Download a UTXO Snapshot file. You can find one for block height 840,000 on my site or from this torrent:
  magnet:?xt=urn:btih:596c26cc709e213fdfec997183ff67067241440c&dn=utxo-840000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969
• Download Bitcoin Core v28 or later.
• Run Bitcoin Core: bitcoind -daemon
• Wait for the block headers to finish syncing (should only take 1 or 2 minutes)
• Load the snapshot via: bitcoin-cli loadtxoutset /path/to/utxo-snapshot-height-840000.dat
• Wait for the command to finish (this will take tens of minutes depending upon the speed of your machine)
• Once the command finishes, your node's sync progress will jump forward to the block height of the snapshot and continue syncing normally from there.
• After it reaches chain tip, the node will start performing a sync from genesis in the background, but the node will be usable during that time.

...

[ABCbits]

~~~

From what I read in Lopp's blog post I don't think you're fully correct with your assumption that Bitcoin Core will "only" download blocks from the UTXO snapshot on. Or did I misunderstand you?

From what I understand what Lopp wrote, didn't try it myself so far, it will first sync from UTXO snapshot's blockheight to chaintip and then act like fully synced (sync progress will first jump to progress indicator of snapshot's blockheight and then work its way up to chaintip aka progress=1.000000.
--snip--

Actually i wasn't being explicit enough. I actually meant with UTXO snapshot, you just need to download blocks after when the UTXO snapshot is created in order to have full node and wallet (newly created ones) functionality. And i appreciate you took time to read and understand reference blog i mentioned.

[apogio]

Sounds like a fun test project the next time I have spare time and need to quickly spin up a new Bitcoin Core node. I also like the fact that it should work fine for a pruned node when you're constrained by storage space and still want and need a quickly fully synced pruned node.

It's indeed interesting.

I 've had some questions regarding pruned nodes and how easy it is for the user to change the size of the data that's stored (link for info).

I guess it should be an option if you wanted to load wallets that are created before the pruning period.

The downside is that you 'd need to essentially trust data that you 'd theoretically already downloaded and verified for yourself.

[satscraper]

But: from what I've seen, usually it's the lack of RAM that slows down Bitcoin Core's IBD, and not the internet speed. In that case verification after copying still takes about the same amount of time.

Yes, I believe it's because the dbcache size isn't enough to exceed the actual memory size of some blocks.

From my experience the best choice is to set dbcache in bitcoin.config as dbcache=1/4 RAM (^{in the case of  RAM equals to  16 Gb for instance  dbcache should be set as  dbcache=4096)}. Rest of RAM should be reserved for OS  operations. Also it is useful to add blocksonly=1 entry into  bitcoin.config file ^{as this would speed up a bit IBD procedure}.

 IBD is not so frightful as it is painted.

[apogio]

From my experience the best choice is to set dbcache in bitcoin.config as dbcache=1/4 RAM (^{in the case of  RAM equals to  16 Gb for instance  dbcache should be set as  dbcache=4096)}. Rest of RAM should be reserved for OS  operations. Also it is useful to add blocksonly=1 entry into  bitcoin.config file ^{as this would speed up a bit IBD procedure}.

 IBD is not so frightful as it is painted.

It definitely isn't frightful.
But personally, I 've had best results setting dbcache=5000 on a 8GB total RAM, which corresponds to ~60% of total RAM.
This is because my node is set up on a dedicated device and not on a device that requires too much free RAM for other user interaction.

[LoyceV]

From my experience the best choice is to set dbcache in bitcoin.config as dbcache=1/4 RAM (^{in the case of  RAM equals to  16 Gb for instance  dbcache should be set as  dbcache=4096)}. Rest of RAM should be reserved for OS  operations.

That depends on how you use the rest of your computer. On a dedicated node, dbcache can be much more than 25% of your RAM.

Also it is useful to add blocksonly=1 entry into  bitcoin.config file ^{as this would speed up a bit IBD procedure}.

This is what it does:

Code:

# Send and receive complete blocks only, no loose transactions. Uncommenting can reduce bandwidth by 88%
#blocksonly=1

I think this means it only keeps mempool empty, and I don't think that matters much during IBD.

[satscraper]

This is because my node is set up on a dedicated device and not on a device that requires too much free RAM for other user interaction.

This is true. My machine is not dedicated one.

I think this means it only keeps mempool empty, and I don't think that matters much during IBD.

 I don't agree.

It really matters as it results in the faster IBD because the node doesn't have to fully validate those  transactions which are not in the blocks. As we know some nodes may keep the low-fee-trxs  for the period which exceeds that one required to sync entirely. Thus blocksonly=1 eliminates the need to validate such transactions at IBD. The gain is not big compared with the total time, but it is still a gain.

That depends on how you use the rest of your computer. On a dedicated node, dbcache can be much more than 25% of your RAM.

Agreed.