Problems syncing this morning : invalid block, please help

[HCP]

Sorry... just to clarify... you're running 3x cryptocoin Core applications at the same time (eg. Bitcoin Core + Litecoin Core + Dogecoin Core) with all the blockdata stored on the same external drive?

Outside of the initial block download, I wouldn't have thought that running them all side by side would stress things too much... once nodes are synced up, the processing requirements aren't super high.

I've had multiple Core applications running before... I think 5 was my "personal best" (Bitcoin mainnet, Bitcoin testnet, BCH, Litecoin and Dogecoin) and never seemed to have any issues... but that was on a desktop with all the blockdata on the same (internal) SATA HDD.

[gabrio81]

yeah that's right, and they all have full blockchains running - i am gonna run only one at a time, finish what i have to do, close then run the other one, etc.

The private ones actually are much more intense but bitcoin always triggers the highest CPU usage and I/O on the hard disk....this is a fast Samsung SSD T7 but then at the same time, maybe that's too much.

anyway time will tell

[HCP]

The private ones actually are much more intense but bitcoin always triggers the highest CPU usage and I/O on the hard disk....this is a fast Samsung SSD T7 but then at the same time, maybe that's too much.

Interesting... but I'm not surprised that Bitcoin has the highest usage. I would assume that the amount of data (note: not just the number of blocks) being processed on the Bitcoin blockchain is quite a bit larger than the other private chains.

Still, running on a Samsung SSD... even an external one, I would have thought that it should be easy enough for the system to keep up, given that I have run multiple chains on an HDD.

I'm sure that running it one at a time will stop any issues, but could be annoying depending on how long it's been since that chain was synced and how long you have to wait for it to sync up.

[gabrio81]

it's actually not bad. on average if you do the BTC core sync once a week, it takes only 10-15 mins to sync with my system and this SSD. 

[gabrio81]

ha! i talked too soon.... while doing a sync just now, again i got the same thing, this time with only BTC core running, i guess i need to rescan o re-do the whole blockchain at some point soon.... that will need 1 week or so though... arghhh again invalid block.

this time was a bit weird, it started doing the sync, was a good point and only 369 blocks left

any tips to do a full rescan or complete blockchain sync again? of course i just ignored the block again and it worked again.

[HCP]

I guess it is possible that one of the peers you're getting data from may, for whatever reason (unknowingly or maliciously), be sending your node bad data... You could try "banning" all the current peers and see if it continues to happen.

Otherwise, it's likely some transient hardware issue with your setup... and those are an absolute MoFo to try and diagnose/rectify

[gabrio81]

ok, this happened again - i will take the plunge and do a rescan / reindex most likely taking a week...... something similar happened on a smaller chain and it fixed the issue

what's the command? is that - reindex something via command like?

can anybody help on this please?

thank you,
Gabrio

[nc50lc]

-snip-
what's the command? is that - reindex something via command like?

Exact same thing happened?

Yes? Then redo post number two but check your "debug.log" for the latest lines with:
<date/time> ERROR: AcceptBlockHeader: block xxxxxxxxxxxxxxxxxxBLOCK_HASHxxxxxxxxxxxxxxxxxx is marked invalid
And use that block hash to the command: reconsiderblock

[gabrio81]

ok thanks but i figured it out myself, i did a -reindex and it finished, took 17 hours, let's see if it happens again now.

[HCP]

In any case, this really should not be happening. It's not "normal".

I have both a desktop with a SATA HDD running Win10+Bitcoin Core, and a Pi with a USB3.0 external HDD running Umbrel, and neither exhibit this behaviour.

There seems to be something peculiar to your specific setup that is causing this, and I fear that you are going to continue to get these invalid block errors unless something in your setup (ie. hardware config) changes.

[gabrio81]

it happened again, used reconsiderblock and that's that - it's really strange though. oh well moving on - thought the reindex corrected the issue but apparently didn't. i cannot keep this plugged in all the time so that'll do.

in the end it works, i can operate when i need to. thanks all

[HCP]

A "reindex" won't prevent this from happening... something in your setup is causing data corruption and that corrupt data is being written on your disk.

It shouldn't be an invalid block being received from another node, that should be automatically invalidated and discarded by your node when it is received... so it is most likely something at your end (bad drive cable, bad hardware, bad power unit, faulty memory, faulty disk, bad chipset etc).

I would wager that this will continue to happen from time to time unless you substantially change your setup and replace hardware (the difficult part is that it will be very difficult to track down which specific bit of hardware is causing the error )...

Until then, you're going to need to continue using reconsiderblock whenever this happens.

[kano]

.. bad disk?

[kano]

.. bad disk?

On first post, OP mentioned that he ran disk (looks like with built-in windows tools) and RAM check, so it's unlikely. But if you have specific suggestion to test disk thoughtfully, you should mention it.

Well it keeps sounding like a bad disk, and no comment in the first post about that

He did mention CHKDSK 7 posts later,
but CHKDSK isn't by default a full check of any disk, and on a SSD even less so.

Make sure to backup the wallet (or that should have been done anyway) then,
at the very least run smartmon on the disk and check it's history and run a 'long' smartmon check of the disk.

Standard I/O to any disk wont cause errors no matter what you do, unless there's something wrong with the disk or the interface.
With an SSD, give it a few years at least before smartmon should start mentioning issues.

Though there are some altcoins that are VERY bad for any computer drives, so if he's 'mining' any of them, then the whole issue is simply to expect failure and don't dare run something important like a Bitcoin wallet on the same drive.

[gabrio81]

hi,

Well, it's very hard to say if it's the drive because, i ONLY use this ssd for crypto really, i have nothing else on it and it's also a very fast Samsung T7 SSD (1 gb/sec write and read) with built in encryption etc (which i am not using though).

Now, yes this happened many times now and i am quite clueless on why, I have never lost data on this ssd on other occasions. I am not mining at the moment but was helping the network with open port, etc. I have 2 others privacy coins and one of those also experienced same thing, reindex then now fine, the other one instead never had this issue,

I am gonna live with this, not ideal and again, the reason i am having this on external SSD is that i wanted a drive that does not stay on continuously like my "main" normal SSD where i do other stuff and my work, etc. As we all know these drives have an X lifespan and if they're on 24/7 that will get shortened every day.

Still, that's way bizarre so i am gonna send Samsung a message, maybe they can help or maybe this type of SSD is not suitable for crypto in general, hence the errors.

thanks for ideas and feedback anyway

[gabrio81]

hi guys,

I wanted to give an update on this issue...since this was last updated last year in Aug 2021.

So, this invalid block thing was happening quite often until i discovered something: I was always running my CPU with an undervolt flag for better temperature under heavy loads.

For some reason, at some point i disabled that flag while running Core, and again and again and i noticed that the issue with the invalid block NEVER happened again. I didn't change anything, i didn't change SSD drive, didn't format, didn't re-download the entire blockchain, nothing.

So i am quite happy i managed to find a solution that works. I still run the undervolt for other things and some other wallets but not for BTC core.

Still a mystery the reason why though ;-)

Best,
Gabrio

[nc50lc]

So i am quite happy i managed to find a solution that works. I still run the undervolt for other things and some other wallets but not for BTC core.

Still a mystery the reason why though ;-)

Glad that you found the source of the issue.

AFAIK, processors don't have consistent quality. Some are better some are worse, it's what they call "silicon lottery".
You may have picked the one which performs bad with the same underclock/undervolt setting as the others.

[NotATether]

So, this invalid block thing was happening quite often until i discovered something: I was always running my CPU with an undervolt flag for better temperature under heavy loads.

Are you doing the undervolt though software (e.g. cpupower) or are you setting a hardware setting?

Maybe you're using a hardware undervolt and so CPU cycles are being thrown away as a result without Core realizing it's being slown down. I never had any problems with underclocking with cpupower, you should look into that (but its underclock is system-wide).

[LoyceV]

So i am quite happy i managed to find a solution that works. I still run the undervolt for other things and some other wallets but not for BTC core.

It sounds like you've created a hardware problem by undervolting. You may not notice the problems in other applications, but they could still occur.
I'd say improve your cooling instead of using hardware outside it's specifications. For your next computer, consider a processor with lower TDP if possible.

So that's why Bitcoin Core gives a warning when started for the first time:

This initial synchronisation is very demanding, and may expose hardware problems with your computer that had previously gone unnoticed.

[gabrio81]

So i am quite happy i managed to find a solution that works. I still run the undervolt for other things and some other wallets but not for BTC core.

Still a mystery the reason why though ;-)

Glad that you found the source of the issue.

AFAIK, processors don't have consistent quality. Some are better some are worse, it's what they call "silicon lottery".
You may have picked the one which performs bad with the same underclock/undervolt setting as the others.

The weird part is that only BTC core is affected. This laptop has been a workhorse since 5 years already, poor thing hehe.