Database corrupt question

[gabrio81]

hi guys,

this morning i was doing the "usual" sync, then the hard disk started clicking, most likely dust or bad cables (it's external), anyway the hdd is fine, did a good check on it and now i wanted to resync the blockchain.

Classic, I get "Error opening block database" Do you want to rebuild the database now?

Earlier on i had to kill the hard disk and unplug it as it was stuck so most likely the last block didn't write up correctly a closing.

If i click OK is it gonna resync 300+ gig worth of stuff or is he so smart that he can simply roll back the bad blocks say OK let's resume from here and be relatively short?

Or otherwise is there another command prompt thingie or anything to launch?

i have a backup so not end of the world if i have to resync etc but not ideal

thanks so much,
Gab

[gabrio81]

i checked log :

2020-10-18T16:51:39Z Fatal LevelDB error: Corruption: block checksum mismatch: X:\XXXX\chainstate/513482.ldb
2020-10-18T16:51:39Z You can use -debug=leveldb to get more complete diagnostic messages
2020-10-18T16:51:39Z Fatal LevelDB error: Corruption: block checksum mismatch: X:\XXXX\chainstate/513482.ldb
2020-10-18T16:51:39Z : Error opening block database.
Please restart with -reindex or -reindex-chainstate to recover.
2020-10-18T16:54:58Z Aborted block database rebuild. Exiting.

[jackg]

If you've got enough space on the drive. Try placing core and the backup (potentially in a new partition of the drive if you're used to tja) and see if it will be able to use that.. Make sure you run it with the -datadir extension from a command line linking to where you've placed the backup (don't work directly from the backup).

If not you'll have to reindex the whole chain at least once as it'll work through the full 300 GB.

[gabrio81]

got it, if i do "reindex" option on command prompt, is he gonna RE DOWNLOAD (which takes 1 week) or just re-index in fact, which i think is like 12 hours? sorry but want to be sure.

thanks!

[gabrio81]

if i just delete that bad block he pointed out, that won't work either?

[BitMaxz]

got it, if i do "reindex" option on command prompt, is he gonna RE DOWNLOAD (which takes 1 week) or just re-index in fact, which i think is like 12 hours? sorry but want to be sure.

thanks!

No, according to this it will only rebuilds the two leveldb databases for the block index and the chainstate, using the block files saved on disk.

Which means it doesn't need to redownload the whole blockchain and it will only rebuild the blocks that you already downloaded.

if i just delete that bad block he pointed out, that won't work either?

No you still need to use reindex command if you want to fix this issue.

[gabrio81]

got it, if i do "reindex" option on command prompt, is he gonna RE DOWNLOAD (which takes 1 week) or just re-index in fact, which i think is like 12 hours? sorry but want to be sure.

thanks!

No, according to this it will only rebuilds the two leveldb databases for the block index and the chainstate, using the block files saved on disk.

Which means it doesn't need to redownload the whole blockchain and it will only rebuild the blocks that you already downloaded.

if i just delete that bad block he pointed out, that won't work either?

No you still need to use reindex command if you want to fix this issue.

thanks, it's re-indexing now, see you in 1 week 😁

[bob123]

This happens relatively often with external drives.
Core needs a steady connection. Therefore an internal drive would be definitely better in terms of stability and speed.

If you want to circumvent this in the future, your best bet would be to use an internal drive.

[gabrio81]

just as a quick update, 24 hours after, just 61% reprocessed lol, maybe it will really take a week.

yes of course internal would be best but i don't have 300GB on my SSD unfortunately.

[nc50lc]

just as a quick update, 24 hours after, just 61% reprocessed lol, maybe it will really take a week.

yes of course internal would be best but i don't have 300GB on my SSD unfortunately.

How low is your dbcache size?
If it's on default, increasing it to at least half your RAM will help a lot.

But do that after your current task since that "61%" would be a waste if you stop it now.

[Jet Cash]

I would have copied the bad drive onto another disk, and used that to resync. Drives are so cheap now that it isn't worth the risk of hitting the same problem again. If you have a bad spot on the disk, then you need to flag it so that it win't be used again. I'm running a couple of nodes using external USB drives, and Core seems pretty good at coping with problems, as long as they aren't hardware errors.

[gabrio81]

just as a quick update, 24 hours after, just 61% reprocessed lol, maybe it will really take a week.

yes of course internal would be best but i don't have 300GB on my SSD unfortunately.

How low is your dbcache size?
If it's on default, increasing it to at least half your RAM will help a lot.

But do that after your current task since that "61%" would be a waste if you stop it now.

Interesting, finished sync and catchup new blocks, so it's finished. 173 blocks to go. i will check that. thanks

[gabrio81]

just as a quick update, 24 hours after, just 61% reprocessed lol, maybe it will really take a week.

yes of course internal would be best but i don't have 300GB on my SSD unfortunately.

How low is your dbcache size?
If it's on default, increasing it to at least half your RAM will help a lot.

But do that after your current task since that "61%" would be a waste if you stop it now.

BTW dbcache size, under Options > Main is 450 MiB right now. i have 16gb of ram on this machine. is that the one you mentioned to change?

[HCP]

BTW dbcache size, under Options > Main is 450 MiB right now. i have 16gb of ram on this machine. is that the one you mentioned to change?

Yes, try setting that value to 8192. It'll allow Bitcoin Core to cache more data in RAM and help alleviate some of the read/write load on the HDD and should theoretically help increase the sync speed.

[nc50lc]

How low is your dbcache size?
If it's on default, increasing it to at least half your RAM will help a lot.

But do that after your current task since that "61%" would be a waste if you stop it now.

BTW dbcache size, under Options > Main is 450 MiB right now. i have 16gb of ram on this machine. is that the one you mentioned to change?

Set a value depending on how much is left when your system's RAM usage is at peak, just make sure to give room for some spare space, half of 16 is a good amount.
As I've said, set that after finishing the -reindex process since it should be done by now.

But do not expect a lot of boost since you're using an external drive, it's the bottleneck.

[gabrio81]

How low is your dbcache size?
If it's on default, increasing it to at least half your RAM will help a lot.

But do that after your current task since that "61%" would be a waste if you stop it now.

BTW dbcache size, under Options > Main is 450 MiB right now. i have 16gb of ram on this machine. is that the one you mentioned to change?

Set a value depending on how much is left when your system's RAM usage is at peak, just make sure to give room for some spare space, half of 16 is a good amount.
As I've said, set that after finishing the -reindex process since it should be done by now.

But do not expect a lot of boost since you're using an external drive, it's the bottleneck.

ok thanks i will try yep it's finished. yeah but all things considered it's not a bad hdd since it's usb3 and peaks 100-120 mb/sec or 800-950 mbit.

actually on a normal day - when it doesn't crash, for 7 days to catch up new blocks, it takes around 30-40 mins max only. From 1 or 2 latest releases, core is much much faster to catch up. Downloads really well blocks first maxxing out my connection usually and then does the grunt work.

[bob123]

[...] yeah but all things considered it's not a bad hdd since it's usb3 and peaks 100-120 mb/sec or 800-950 mbit.

If you are fine with the speed, no reason to change something.

But an external drive will always be slower than an internal (HDD) one. The protocol overhead alone already makes a huge difference.
In addition to that, you'll only achieve best results if you only have your drive attached to the USB root as other devices will definitely influence the speed.

After all, the main issue with external drives is the stability. Core requires a constant connection which can not be guaranteed using external drives.

But if you only very rarely encounter an error and are fine with the (relatively acceptable) consequences, there is no reason to change your setup.

[gabrio81]

[...] yeah but all things considered it's not a bad hdd since it's usb3 and peaks 100-120 mb/sec or 800-950 mbit.

If you are fine with the speed, no reason to change something.

But an external drive will always be slower than an internal (HDD) one. The protocol overhead alone already makes a huge difference.
In addition to that, you'll only achieve best results if you only have your drive attached to the USB root as other devices will definitely influence the speed.

After all, the main issue with external drives is the stability. Core requires a constant connection which can not be guaranteed using external drives.

But if you only very rarely encounter an error and are fine with the (relatively acceptable) consequences, there is no reason to change your setup.

heheh yep, i was contemplating adding another HDD inside this laptop since there's a bay available, but it will add to the weight a bit ;-) but yeah for now is ok, in the end, as long as there's a backup there's hope hehehehe