Core 0.20.0 sync stuck on a block, please help, has there been a fork?

[gabrio81]

hi guys,

I just launched my "usual" blockchain catch-up and core 0.20.0 is stuck from a while, he was crunching some data, CPU active, disk, network traffic, etc,. then all of a sudden all quiet, 0% network activity, etc. the disk is fine and PC as well, no hooks etc.

Sync is stuck at 2 days ago, Last Block time Fri Jul 31 09:50:27 2020 (99.89%)

Network is connected, i see all 4 dots connected, i have 10-12 connections active, though that then drops to 2 dots only, back to 4, very irregular. seems something is def up!

Can someone help me out please? i just had a peek at the debug log and here's what we have there:

2020-08-02T11:43:16Z ERROR: AcceptBlockHeader: block 00000000000000000007b86426ffa671d5f52a0cf22eb1a03a73c114695983c8 is marked invalid
2020-08-02T11:43:16Z ERROR: AcceptBlockHeader: block 00000000000000000007b86426ffa671d5f52a0cf22eb1a03a73c114695983c8 is marked invalid
2020-08-02T11:43:16Z ERROR: AcceptBlockHeader: block 00000000000000000007b86426ffa671d5f52a0cf22eb1a03a73c114695983c8 is marked invalid
2020-08-02T11:43:16Z ERROR: AcceptBlockHeader: block 00000000000000000007b86426ffa671d5f52a0cf22eb1a03a73c114695983c8 is marked invalid
2020-08-02T11:43:16Z ERROR: AcceptBlockHeader: block 00000000000000000007b86426ffa671d5f52a0cf22eb1a03a73c114695983c8 is marked invalid
2020-08-02T11:43:17Z ERROR: AcceptBlockHeader: block 00000000000000000007b86426ffa671d5f52a0cf22eb1a03a73c114695983c8 is marked invalid
2020-08-02T11:43:17Z ERROR: AcceptBlockHeader: block 00000000000000000007b86426ffa671d5f52a0cf22eb1a03a73c114695983c8 is marked invalid
2020-08-02T11:43:17Z New outbound peer connected: version: 70015, blocks=641879, peer=87 (full-relay)
2020-08-02T11:43:22Z ERROR: AcceptBlockHeader: block 00000000000000000007b86426ffa671d5f52a0cf22eb1a03a73c114695983c8 is marked invalid
2020-08-02T11:43:29Z ERROR: AcceptBlockHeader: block 00000000000000000007b86426ffa671d5f52a0cf22eb1a03a73c114695983c8 is marked invalid
2020-08-02T11:43:40Z New outbound peer connected: version: 70015, blocks=641879, peer=88 (full-relay)
2020-08-02T11:43:41Z New outbound peer connected: version: 70015, blocks=641879, peer=89 (full-relay)
2020-08-02T11:43:47Z New outbound peer connected: version: 70015, blocks=641879, peer=90 (full-relay)
2020-08-02T11:44:04Z New outbound peer connected: version: 70015, blocks=641879, peer=93 (full-relay)
2020-08-02T11:44:05Z New outbound peer connected: version: 70015, blocks=641879, peer=94 (full-relay)
2020-08-02T11:44:06Z New outbound peer connected: version: 70015, blocks=641879, peer=95 (full-relay)
2020-08-02T11:44:07Z New outbound peer connected: version: 70015, blocks=641879, peer=96 (block-relay)
2020-08-02T11:44:24Z New outbound peer connected: version: 70015, blocks=641879, peer=97 (block-relay)
2020-08-02T11:44:30Z New outbound peer connected: version: 70015, blocks=641879, peer=98 (block-relay)
2020-08-02T11:46:03Z ERROR: AcceptBlockHeader: block 00000000000000000007b86426ffa671d5f52a0cf22eb1a03a73c114695983c8 is marked invalid
2020-08-02T11:46:03Z ERROR: AcceptBlockHeader: block 00000000000000000007b86426ffa671d5f52a0cf22eb1a03a73c114695983c8 is marked invalid
2020-08-02T11:46:04Z ERROR: AcceptBlockHeader: block 00000000000000000007b86426ffa671d5f52a0cf22eb1a03a73c114695983c8 is marked invalid
2020-08-02T11:46:04Z ERROR: AcceptBlockHeader: block 00000000000000000007b86426ffa671d5f52a0cf22eb1a03a73c114695983c8 is marked invalid
2020-08-02T11:46:04Z ERROR: AcceptBlockHeader: block 00000000000000000007b86426ffa671d5f52a0cf22eb1a03a73c114695983c8 is marked invalid
2020-08-02T11:46:04Z ERROR: AcceptBlockHeader: block 00000000000000000007b86426ffa671d5f52a0cf22eb1a03a73c114695983c8 is marked invalid
2020-08-02T11:46:04Z ERROR: AcceptBlockHeader: block 00000000000000000007b86426ffa671d5f52a0cf22eb1a03a73c114695983c8 is marked invalid
2020-08-02T11:46:07Z ERROR: AcceptBlockHeader: block 00000000000000000007b86426ffa671d5f52a0cf22eb1a03a73c114695983c8 is marked invalid
2020-08-02T11:46:15Z ERROR: AcceptBlockHeader: block 00000000000000000007b86426ffa671d5f52a0cf22eb1a03a73c114695983c8 is marked invalid
2020-08-02T11:46:16Z New outbound peer connected: version: 70015, blocks=641880, peer=104 (full-relay)
2020-08-02T11:46:33Z New outbound peer connected: version: 70015, blocks=641880, peer=105 (full-relay)
2020-08-02T11:46:34Z New outbound peer connected: version: 70015, blocks=641880, peer=106 (full-relay)
2020-08-02T11:46:38Z New outbound peer connected: version: 70015, blocks=641879, peer=103 (full-relay)

many thanks,
Gab

[1714112781]

1714112781

[1714112781]

1714112781

[1714112781]

1714112781

[1714112781]

1714112781

[1714112781]

1714112781

[1714112781]

1714112781

[ranochigo]

Hmm. It most commonly occurs when you use an invalidate block command.

Try running

Code:

 reconsiderblock 00000000000000000007b86426ffa671d5f52a0cf22eb1a03a73c114695983c8 

.

[ABCbits]

AFAIK there's no fork and my Bitcoin Core are able to sync without any problem. Did you run any command that might break Bitcoin Core such as :

Code:

invalidateblock 00000000000000000007b86426ffa671d5f52a0cf22eb1a03a73c114695983c8

Edit : if you do, try to run command shown by @ranochigo

[gabrio81]

AFAIK there's no fork and my Bitcoin Core are able to sync without any problem. Did you run any command that might break Bitcoin Core such as :

Code:

invalidateblock 00000000000000000007b86426ffa671d5f52a0cf22eb1a03a73c114695983c8

Edit : if you do, try to run command shown by @ranochigo

hi both,

no i haven't ran any command line command at all, just launched usual sync, went to do some emails and wait in background. i tried to check the network state with a couple of commands but i haven't invalidated anything.

so, which would be the best course of action? wallet seems fine, i have backup, but i rather Not RE sync 300Gigs if i possible

thanks guys

[gabrio81]

so shall i do

reconsiderblock 00000000000000000007b86426ffa671d5f52a0cf22eb1a03a73c114695983c8

worst case nothing happens?

[gabrio81]

Ok, i took the plunge, panic mode ON and launched the command.... it worked! 

thanks guys!

guess it happens when you sync sometimes, right? always learn something new!

[ranochigo]

Ok, i took the plunge, panic mode ON and launched the command.... it worked! 

thanks guys!

guess it happens when you sync sometimes, right? always learn something new!

It shouldn't usually happen. Did you modify your bitcoin.conf or anything? Is there anything about this hash in the earlier parts of your debug.log?

[gabrio81]

Ok, i took the plunge, panic mode ON and launched the command.... it worked! 

thanks guys!

guess it happens when you sync sometimes, right? always learn something new!

It shouldn't usually happen. Did you modify your bitcoin.conf or anything? Is there anything about this hash in the earlier parts of your debug.log?

nah it started with 2020-08-02T11:08:20Z InvalidChainFound: invalid block=00000000000000000007b86426ffa671d5f52a0cf22eb1a03a73c114695983c8

then all the further errors, that's all i can see.

[gabrio81]

Ok, i took the plunge, panic mode ON and launched the command.... it worked! 

thanks guys!

guess it happens when you sync sometimes, right? always learn something new!

No, it never happens to me and this is first time i saw message " ERROR: AcceptBlockHeader: block [HASH] is marked invalid" without user doing anything.
It's more often to receive the block corrupted and Bitcoin Core download the block from another node.

it might have been lots of syncs on and off, disconnecting the hard disk on and off, oh well it's solved now. happy days.

[nc50lc]

nah it started with 2020-08-02T11:08:20Z InvalidChainFound: invalid block=00000000000000000007b86426ffa671d5f52a0cf22eb1a03a73c114695983c8

Perhaps your node followed a "stale block" sent by a pool right before you closed Bitcoin core.
Then when you opened to "blockchain catch-up" it was invalidated.

AFAIK, forks with 1-block-long chain happen often because of competing miners.
Previously, we can easily check it here as a chain diagram with height: blockchain.com/btc/orphaned-blocks but the link is now unavailable.
So we can't confirm it.

But it's now solved so, congrats!

[nc50lc]

nah it started with 2020-08-02T11:08:20Z InvalidChainFound: invalid block=00000000000000000007b86426ffa671d5f52a0cf22eb1a03a73c114695983c8

Perhaps your node followed a "stale block" sent by a pool right before you closed Bitcoin core.
Then when you opened to "blockchain catch-up" it was invalidated.
-snip-

I doubt it, IIRC Bitcoin Core would choose chain with most PoW.

It wouldn't know if it's the tip at that time then the user immediately close his node.

[nc50lc]

True, but i don't see how it makes other block invalid since IIRC Bitcoin Core would sync block header first if it's not 100% synced.

Assuming OP is always doing the "usual blockchain catch up", then the last time he did, his node had been syncing;
and after it was 100% synced, his node might have received a "will-be" stale block (new tip) then he closed the client right away because "blockchain had caught up".

But there's no much gain in finding the exact cause of the issue since this was solved now  

it might have been lots of syncs on and off, disconnecting the hard disk on and off, oh well it's solved now. happy days.

Probably.

[jnano]

Had something similar happen, with repeated "ERROR: AcceptBlockHeader: block ... is marked invalid" on the same block (which was valid).
It was on Litecoin Core 0.18, not Bitcoin Core, but the fix was the same: reconsiderblock.

The only strange thing that had happened in proximity was a previous out of space crash for the cachestate.
But a following run did sync new blocks for a minute or two, before hitting "marked invalid", so maybe it was unrelated.