Bitcoin Core Node Sync constantly fails at 97%

[0xhexhex]

Hi guys,

i not that tech savvy. meaning i cant do anything CLI, but am a windows user for 15 years and know my way around there and a bit in building PCs.

So im new to this but i wanted to do my part for the network and run a full node on my thin client.

Specs:
Win 10
CPU G3420
8GB Ram
256SSD
and second drive 1TB HDD (Node on here)

28.1 Downloaded at https://bitcoincore.org/en/download/ and verified


I synced for a fews weeks now. Its around 97% , when i open it, it progresses some 0,xx% butt then always fails after a minute or so, i get an
error saying:
"Error reading from database, shutting down."

What can i do? Any advice?
thank you guys in advance.

[NeuroticFish]

Please stop just pasting AI generated text.

"Error reading from database, shutting down."

I didn't encounter this error, but the internet seems to agree that reindex is the first thing to try. See these links/answers:

* https://bitcoin.stackexchange.com/a/56551

* https://www.reddit.com/r/Bitcoin/comments/1g66w8y/btc_core_error_reading_from_database_shutting_down/



PS. I feel that this topic is more suited for Bitcoin Technical Support, you may want to move it there

[odolvlobo]

I synced for a fews weeks now. Its around 97% , when i open it, it progresses some 0,xx% butt then always fails after a minute or so, i get an
error saying:
"Error reading from database, shutting down."

What can i do? Any advice?
thank you guys in advance.

Posting the contents of the log file where it starts reporting errors would help. The name of the file is debug.log and it is located in bitcoin core's data directory.

[0xhexhex]

Hi guys,

i not that tech savvy. meaning i cant do anything CLI, but am a windows user for 15 years and know my way around there and a bit in building PCs.

So im new to this but i wanted to do my part for the network and run a full node on my thin client.

Specs:
Win 10
CPU G3420
8GB Ram
256SSD
and second drive 1TB HDD (Node on here)

28.1 Downloaded at https://bitcoincore.org/en/download/ and verified


I synced for a fews weeks now. Its around 97% , when i open it, it progresses some 0,xx% butt then always fails after a minute or so, i get an
error saying:
"Error reading from database, shutting down."

What can i do? Any advice?
thank you guys in advance.

Never heard of this problem previous but I do think, is because of insufficient storage available on 1TB harddisk which contains blockchain data in, how about space availability?

Thank you for responding!
Free Space is: 893GB
used space is: 21GB

(is that normal with 97% downloaded blockchain??)

I synced for a fews weeks now. Its around 97% , when i open it, it progresses some 0,xx% butt then always fails after a minute or so, i get an
error saying:
"Error reading from database, shutting down."

What can i do? Any advice?
thank you guys in advance.

Posting the contents of the log file where it starts reporting errors would help. The name of the file is debug.log and it is located in bitcoin core's data directory.

Absolutely, here you are:


2025-01-17T17:05:55Z UpdateTip: new best=000000000000000000020ab79b52e94d7c67418063923b80e31c3d6b94d37d76 height=873702 version=0x200b4000 log2_work=95.312933 tx=1128647187 date='2024-12-07T22:16:18Z' progress=0.979447 cache=123.4MiB(925993txo)
2025-01-17T17:05:55Z Loading 0 mempool transactions from file...
2025-01-17T17:05:55Z Imported mempool transactions from file: 0 succeeded, 0 failed, 0 expired, 0 already there, 0 waiting for initial broadcast
2025-01-17T17:05:55Z initload thread exit
2025-01-17T17:05:55Z Synchronizing blockheaders, height: 879660 (~100.00%)
2025-01-17T17:05:56Z UpdateTip: new best=000000000000000000017f29f8a59f8e2cf4a9e6952b80bb1bf7413c183d800a height=873703 version=0x36000000 log2_work=95.312946 tx=1128649037 date='2024-12-07T22:24:54Z' progress=0.979450 cache=124.2MiB(933100txo)
2025-01-17T17:05:57Z LevelDB read failure: Corruption: block checksum mismatch: D:\BTCNode_Blockchain\chainstate/2510441.ldb
2025-01-17T17:05:57Z Fatal LevelDB error: Corruption: block checksum mismatch: D:\BTCNode_Blockchain\chainstate/2510441.ldb
2025-01-17T17:05:57Z You can use -debug=leveldb to get more complete diagnostic messages
2025-01-17T17:05:57Z [error] Error reading from database, shutting down.
2025-01-17T17:05:58Z [error] Error reading from database: Fatal LevelDB error: Corruption: block checksum mismatch: D:\BTCNode_Blockchain\chainstate/2510441.ldb

2025-01-19T17:02:20Z Bitcoin Core version v28.1.0 (release build)
2025-01-19T17:02:20Z Qt 5.15.14 (static), plugin=windows
2025-01-19T17:02:20Z Static plugins:
2025-01-19T17:02:20Z  QWindowsIntegrationPlugin, version 331520
2025-01-19T17:02:20Z  QWindowsVistaStylePlugin, version 331520
2025-01-19T17:02:20Z Style: windowsvista / QWindowsVistaStyle
2025-01-19T17:02:20Z System: Windows 10 Version 2009, x86_64-little_endian-llp64
2025-01-19T17:02:20Z Screen: \\.\DISPLAY1 1280x1024, pixel ratio=1.0
2025-01-19T17:02:22Z Script verification uses 1 additional threads
2025-01-19T17:02:22Z Using the 'sse4(1way),sse41(4way)' SHA256 implementation
2025-01-19T17:02:27Z Using RdRand as an additional entropy source
2025-01-19T17:02:27Z GUI: "registerShutdownBlockReason: Successfully registered: Bitcoin Core noch nicht sicher beendet…"
2025-01-19T17:02:27Z Default data directory C:\Users\bro02\AppData\Local\Bitcoin
2025-01-19T17:02:27Z Using data directory D:\BTCNode_Blockchain
2025-01-19T17:02:27Z Config file: D:\BTCNode_Blockchain\bitcoin.conf (not found, skipping)
2025-01-19T17:02:27Z Setting file arg: prune = "1907"
2025-01-19T17:02:27Z Setting file arg: server = true
2025-01-19T17:02:27Z Setting file arg: upnp = true
2025-01-19T17:02:27Z Using at most 125 automatic connections (2048 file descriptors available)
2025-01-19T17:02:27Z scheduler thread start
2025-01-19T17:02:27Z Binding RPC on address ::1 port 8332
2025-01-19T17:02:27Z Binding RPC on address 127.0.0.1 port 8332
2025-01-19T17:02:27Z Using random cookie authentication.
2025-01-19T17:02:27Z Generated RPC authentication cookie D:\BTCNode_Blockchain\.cookie
2025-01-19T17:02:27Z Permissions used for cookie: rw-rw-rw-
2025-01-19T17:02:27Z Starting HTTP server with 4 worker threads
2025-01-19T17:02:27Z Using wallet directory D:\BTCNode_Blockchain
2025-01-19T17:02:27Z init message: Überprüfe Wallet(s)...
2025-01-19T17:02:27Z Using /16 prefix for IP bucketing
2025-01-19T17:02:27Z init message: Lade P2P-Adressen...
2025-01-19T17:02:29Z Loaded 65701 addresses from peers.dat  1309ms
2025-01-19T17:02:29Z init message: Lade Bannliste…
2025-01-19T17:02:29Z SetNetworkActive: true
2025-01-19T17:02:29Z Script verification uses 1 additional threads
2025-01-19T17:02:29Z Cache configuration:
2025-01-19T17:02:29Z * Using 2.0 MiB for block index database
2025-01-19T17:02:29Z * Using 8.0 MiB for chain state database
2025-01-19T17:02:29Z * Using 440.0 MiB for in-memory UTXO set (plus up to 286.1 MiB of unused mempool space)
2025-01-19T17:02:29Z Using obfuscation key for blocksdir *.dat files (D:\BTCNode_Blockchain\blocks): 'd8b3906fc3bfc522'
2025-01-19T17:02:29Z Using 16 MiB out of 16 MiB requested for signature cache, able to store 524288 elements
2025-01-19T17:02:29Z Using 16 MiB out of 16 MiB requested for script execution cache, able to store 524288 elements
2025-01-19T17:02:29Z init message: Lade Block-Index...
2025-01-19T17:02:29Z Assuming ancestors of block 000000000000000000011c5890365bdbe5d25b97ce0057589acaef4f1a57263f have valid signatures.
2025-01-19T17:02:29Z Setting nMinimumChainWork=000000000000000000000000000000000000000088e186b70e0862c193ec44d6
2025-01-19T17:02:29Z Prune configured to target 1907 MiB on disk for block and undo files.
2025-01-19T17:02:29Z Opening LevelDB in D:\BTCNode_Blockchain\blocks\index
2025-01-19T17:02:29Z Opened LevelDB successfully
2025-01-19T17:02:29Z Using obfuscation key for D:\BTCNode_Blockchain\blocks\index: 0000000000000000
2025-01-19T17:02:35Z Shutdown requested. Exiting.
2025-01-19T17:02:35Z Shutdown: In progress...
2025-01-19T17:02:35Z scheduler thread exit
2025-01-19T17:02:35Z Flushed fee estimates to fee_estimates.dat.
2025-01-19T17:02:35Z Shutdown: done

[moderator's note: consecutive posts merged]

[BitMaxz]

As you can see on the logs, you have a chainstate or database corruption.
I think you can just delete that folder for chainstate and then shutdown the Bitcoin core and reopen it again; it should resync and generate a new database.

If it doesn't work, you can try using this command below through CMD or terminal. This won't need to redownload the blocks but to erase the chainstate and recreate a new one.

Code:

-reindex-chainstate

[Cricktor]

With only 8GB RAM your node can't hold the chainstate fully in memory and has to do a lot of I/O to your HDD (painfully slow at random access I/O) to properly build the UTXO set which is stored in the chainstate folder. Full sync on HDD without lots of RAM will be snail slow and your HDD won't like the workload. A synced chainstate folder is about 12GB large.

Have you checked if your HDD is still OK? By which interface is your HDD connected to your node?


Is it intentional that you have a pruned node which doesn't store more than 2GB of block data? You could store an unpruned blockchain on your 1TB HDD if it's not used for anything else.

~~~

If he deletes chainstate folder, his pruned node will have to re-download the whole blockchain again. Apparently the same is needed when he deletes the blocks folder.

[0xhexhex]

With only 8GB RAM your node can't hold the chainstate fully in memory and has to do a lot of I/O to your HDD (painfully slow at random access I/O) to properly build the UTXO set which is stored in the chainstate folder. Full sync on HDD without lots of RAM will be snail slow and your HDD won't like the workload. A synced chainstate folder is about 12GB large.

Have you checked if your HDD is still OK? By which interface is your HDD connected to your node?


Is it intentional that you have a pruned node which doesn't store more than 2GB of block data? You could store an unpruned blockchain on your 1TB HDD if it's not used for anything else.

~~~

If he deletes chainstate folder, his pruned node will have to re-download the whole blockchain again. Apparently the same is needed when he deletes the blocks folder.

So my HDD should be fine, i did run checkdisk.
it is connected internally via sata.

ok its good to know the ram thing, maybe i will upgrade it.

i just (in a thread above mentioned, with reddit link) tried to start "-reindex-chainstate". i gave my an error saying cant to it in pruned mode.
So i did just -reindex and its now doing it. guessing that will take quite some time...

i will report back if it fixed it. (and if not try the other methods you guys suggested.)

[Cricktor]

I don't think the chainstate folder on a pruned node requires the entire blockchain to be again downloaded if op deletes only a chain state folder. No? Yes, but only requires if he delete block's folder.

All I can say is, you're wrong with that assumption. To recreate or reconstruct the current UTXO set aka chainstate the node has to start from Genesis block and follow all transactions on the blockchain, unless you have a UTXO snapshot, e.g. created at a certain blockheight with Bitcoin Core's dumptxoutset command, which could be loaded later with loadtxoutset.

Loading a UTXO snapshot only gives you a defined starting point from a certain blockheight when the snapshot was saved. You still have to follow the blockchain from there block to block, transaction for transaction to get to the current UTXO set at current blockchain tip.

A pruned node doesn't have all the transactions to reconstruct the current UTXO set.

[satscraper]

"Error reading from database, shutting down."

What can i do? Any advice?
thank you guys in advance.

I had the similar issue ^{caused by corrupted database} around a year ago. Nothing helped except the building copy of blockchain ^{on my SSD} from the scratch. To accelerate IBD procedure I have used the following entries in my bitcoin.conf file : dbcache=1/4 RAM (for instance if RAM = 16 Gb then dbcache=4096) ^{rest of RAM is reserved for OS  operations} and add blocksonly=1.

[0xhexhex]

So currently reindexing is at around 51%
(first 40% were without HDD activity, but from there on its constantly 99% usage, is that how its supposed to be?)

i will report back if it fixed it.
otherwise will try satscraper's method too, thx btw

[alecfisker]

replace HDD with SSD, on big chunks of data > 4 GB HDD could lag behind the node file IO timeout making it hard sync fully
also OS try to virus check, search index attached drives all the time making concurrent file IO, to solve turn off windows defender, search indexer service for node drive
as an alternative attach SSD drive using USB adapter, weird but it will do IO faster because
windows will now think it is not system drive and do not occupy it with any unnecessary activity

[BitMaxz]

So currently reindexing is at around 51%
(first 40% were without HDD activity, but from there on its constantly 99% usage, is that how its supposed to be?)

That's normal because that is IBD, or we call it Initial Block Download during syncing; it uses high HDD usage; that is why we recommend using SSD if you want to speed the syncing process. There is a significant speed improvement; increasing dbcache will improve the syncing process slightly, but SSD will improve the overall performance of your PC as well as the syncing process.

[0xhexhex]

Hey, so im back and it worked. just now its fully synced.

But i only got outbound connections. console tells me:

 "connections": 10,
  "connections_in": 0,
  "connections_out": 10,


Checked via https://bitnodes.io/ if port 8333 is open and its not.

i got a vodafone box - Version 7 -  Firmware-Version AR01.04.137.08_072324_7249.SIP.10
and it tried opening or forwarding this port (there is just one menue where u can do that) but no luck, still tells me port closed.

is there anything else i can do or try? or am i stuck at his part, when using this DSL router?

[Cricktor]

Your node will work just fine with only outbound connections, it will still get transactions and blocks via its outbound connected nodes.

Is this a Vodaphone cable internet router thing? It's possible you only have DSLlite which means your box has no public IP4 address.

If you can switch to IP6, then it might solve your inbound connectivity. Unfortunately I can't give you out-of-box instructions to make your node use IP6. Someone else has to chime in for this.

[0xhexhex]

Thank you! Thats great to hear.

yes i already tried ip6, no change.

Also, it tells me its pruned. HDD used space ist just 20GB. It that something i should change? (Can i easily change it?)

[Cricktor]

How long did you wait to see if inbound connection might come in and stay?

It depends what you want to do with the node whether a pruned node is sufficient for you or not. Didn't you pay attention to the settings of your node?

A pruned node doesn't store all blocks of the blockchain, only the most recent ones up to the prune limit (or the blocks that are relevant for a loaded wallet's transactions, IIRC).

If you create a new wallet with the pruned node and keep that wallet loaded in your node, it won't cause problems with syncing your new wallet over time. It's only when you load an old wallet with a pruned node that issues come up, when blocks are missing to sync such a wallet. Your pruned node will tell you that it needs to download the whole blockchain again to sync such a "foreign" wallet.
Not exactly a nice situation.

Another no-go for a pruned node is when you want to sync an Electrum server to your node. This commonly doesn't work with a pruned node.

[0xhexhex]

How long did you wait to see if inbound connection might come in and stay?

It depends what you want to do with the node whether a pruned node is sufficient for you or not. Didn't you pay attention to the settings of your node?

A pruned node doesn't store all blocks of the blockchain, only the most recent ones up to the prune limit (or the blocks that are relevant for a loaded wallet's transactions, IIRC).

If you create a new wallet with the pruned node and keep that wallet loaded in your node, it won't cause problems with syncing your new wallet over time. It's only when you load an old wallet with a pruned node that issues come up, when blocks are missing to sync such a wallet. Your pruned node will tell you that it needs to download the whole blockchain again to sync such a "foreign" wallet.
Not exactly a nice situation.

Another no-go for a pruned node is when you want to sync an Electrum server to your node. This commonly doesn't work with a pruned node.

Thank you for the clarification.

Well i thought i paid attention (cause i wanted a full node in the first place) ... but idk honestly how it happened. is pruned the standard when installing?

ok, so i dont want to create a wallet, or load an old wallet. I just want to run a node for the sake of the network.

your question regarding how long i waited: it was a week or two.

[Cricktor]

I didn't pay too much attention to it, but AFAIR a pruned node is nowadays default (I could be wrong). I usually edit the bitcoin.conf file and don't make settings in the GUI version of Bitcoin Core for my node. I need a full node because I run my own Electrum server for my light wallets.

A full non-pruned Bitcoin Core node would currently need about 682GiB for blocks storage, 12GiB for chainstate and 67GiB for transaction index and other indices.

For the purpose you said, your pruned node should be mostly fine. Your pruned node just can't serve older blocks if another node asks it to send 'em. You likely can live with this.

[nc50lc]

Also, it tells me its pruned. HDD used space ist just 20GB. It that something i should change? (Can i easily change it?)

You choose, your node can still accept inbound peers even if your blockchain is pruned.
You don't have to if you don't want to sync again from start, your disk space is limited and you don't mind the limitations of a node in pruned mode.

ok, so i dont want to create a wallet, or load an old wallet. I just want to run a node for the sake of the network.

With a pruned blockchain, your node wont be able to serve historical blocks to new nodes that are in early IBD state (take this in consideration of your decision)
But it still verifies and relays transactions as well as serving the recent 288 blocks regardless of the prune size you selected.

Hey, so im back and it worked. just now its fully synced.
-snip-
But i only got outbound connections.

This is the part where you need specific help from that Internet Service Provider or users who are using the same router and ISP.

If you want a workaround, try setting up "Tor", that should enable your node to accept inbound connections even if your ISP wont.
First, read the basics in this documentation: github.com/bitcoin/bitcoin/blob/master/doc/tor.md
And or course, you need to install/setup Tor in your machine. (preferably the service and not the browser bundle, setup differs per OS)

If you can't figure something out, post a new topic in Bitcoin Technical Support board containing the details of what you done so far.