Final Year of Bitcoin Sync Question

[Vilnoc]

Out of curiosity...

why does it take the final year of the Blockchain sync significantly longer than the previous 7?

I'm on a "gigabit" connection with a stable 999mbps down and I started a fresh sync at midnight central, when I woke up today at 8am it was at 1yr 18w, now it's 12:45pm central at 1yr, 12w...

Slightly confused.

Also confused as to why my block chain corrupts every three weeks, four days, six hours, seven minutes exactly regardless of what drive it's on or computer system it's on between three different platforms.

Very perplexed.

Thanks for any insight!

[achow101]

Starting around 1.5 to 2 years ago, blocks started getting fuller and fuller. This means that they are larger and contain more transactions. So downloading those blocks takes a little longer, and verifying them takes longer too as there is more data to verify. This slows down syncing.

Also confused as to why my block chain corrupts every three weeks, four days, six hours, seven minutes exactly regardless of what drive it's on or computer system it's on between three different platforms.

This is usually indicative of a hardware issue. The blockchain should not corrupt at all during the sync unless there is some underlying problem that is causing corruption. This can be related to hard drive issues, memory issues, or an Antivirus that is deleting part of the blockchain.

[Vilnoc]

Starting around 1.5 to 2 years ago, blocks started getting fuller and fuller. This means that they are larger and contain more transactions. So downloading those blocks takes a little longer, and verifying them takes longer too as there is more data to verify. This slows down syncing.

Also confused as to why my block chain corrupts every three weeks, four days, six hours, seven minutes exactly regardless of what drive it's on or computer system it's on between three different platforms.

This is usually indicative of a hardware issue. The blockchain should not corrupt at all during the sync unless there is some underlying problem that is causing corruption. This can be related to hard drive issues, memory issues, or an Antivirus that is deleting part of the blockchain.

Thank you, that clears it up nicely, Is there anything local network wise about speeding it up other than giving my computer the entire "pipe" of my home network?  I'll catch flak from the attachments, but I'm the technomage around here haha.

I originally assumed it was hardware as well, given the linearity of the corruption. My first step was to move the block chain from One SSD to another, then to externals, then from one machine to another, then to a server. I've even moved it between Linux, MacOS and Windows. in some cases with no AV.

Do you think it could be my triple layer encryption?

[achow101]

Thank you, that clears it up nicely, Is there anything local network wise about speeding it up other than giving my computer the entire "pipe" of my home network?  I'll catch flak from the attachments, but I'm the technomage around here haha.

Besides giving it all the network bandwidth, you could do it on a more powerful computer with more RAM and CPU power. You can also start Bitcoin Core with the -dbcache=<n> option where <n> is a value in Mb of RAM to give to the database. Increasing the dbcache will help it go faster.

I originally assumed it was hardware as well, given the linearity of the corruption. My first step was to move the block chain from One SSD to another, then to externals, then from one machine to another, then to a server. I've even moved it between Linux, MacOS and Windows. in some cases with no AV.

Do you think it could be my triple layer encryption?

It could be the encryption. Anything that modifies the blockchain data is a potential source of corruption, and depending on how your encryption works, that could be causing the issue.

[Vilnoc]

Besides giving it all the network bandwidth, you could do it on a more powerful computer with more RAM and CPU power. You can also start Bitcoin Core with the -dbcache=<n> option where <n> is a value in Mb of RAM to give to the database. Increasing the dbcache will help it go faster.

Hmm, I think I may try that. The current system syncing it has 8 Logical Processors operating at 2.88ghz with 32gbs of total system memory. I could assign the process a higher memory priority and allocate maybe 10gbs worth of ram to the DB? would that make a significant difference? or would it be overkill?"

[achow101]

Besides giving it all the network bandwidth, you could do it on a more powerful computer with more RAM and CPU power. You can also start Bitcoin Core with the -dbcache=<n> option where <n> is a value in Mb of RAM to give to the database. Increasing the dbcache will help it go faster.

Hmm, I think I may try that. The current system syncing it has 8 Logical Processors operating at 2.88ghz with 32gbs of total system memory. I could assign the process a higher memory priority and allocate maybe 10gbs worth of ram to the DB? would that make a significant difference? or would it be overkill?"

It should make a significant difference. The default dbcache is 300 MB so assigning it 10 GB should greatly speed up the indexing part of the sync.

[Vilnoc]

It should make a significant difference. The default dbcache is 300 MB so assigning it 10 GB should greatly speed up the indexing part of the sync.

Well alrighty! I just tossed it 10gbs, It's got a year and a week to go as of 3:20 Pm Central. let's see how much we can get through!

[Vilnoc]

Since my last post it's down to 30 weeks, not sure if that's zippy given that I've tossed it a full pipe and 10gigs of ram or to be expected regardless.

[shorena]

Since my last post it's down to 30 weeks, not sure if that's zippy given that I've tossed it a full pipe and 10gigs of ram or to be expected regardless.

How exactly did you do this? (dbache=10240 in bitcoin.conf?) Does core use the 10 GB? Does it use more ram than previously?