Bitcoin core is again downloading the whole blockchain

[marc3535]

Hi, I had Bitcoin Core up and running on my macbook. When I restarted the macbook this morning, Bitcoin Core started again downloading the entire blockchain. I selected the right folder, I have an external SSD drive on which I previously already had downloaded the whole blockchain. Also I tried to click "reset options", but after restarting bitcoin core it again doesn't connect with the existing blockchain on the SSD drive but starts downloading from scratch. Does anyone have any ideas how to fix this?

[1714470220]

1714470220

[1714470220]

1714470220

[1714470220]

1714470220

[Findingnemo]

Bitcoin core is not recognizing the data files for some reason which is why it is downloading the data again.

Check the status of your SSD drive, and make sure the files present in the directory is not corrupted or missing. Try ewscan or reindexing the blockchain data and check your 'bitcoin.conf' file, which is the configuration file for Bitcoin Core. Ensure that the 'datadir' parameter is correctly set to the directory where your blockchain data is stored. You can create or edit this file if necessary.

Another possible reason, restarting the device can lead to changes in the permission setting so please check that as well.

Using external hard drive as your main blockchain data storage is not a good option, you can use it for backups.

[marc3535]

OK thanks, will look into that but it sounds very complex. Are there any easier / better options? What is the better option to store the blockchain?

[BitMaxz]

I don't know this one will work but would you mind checking the Bitcoin directory if there is bitcoin.conf file or if you can't find add bitcoin.conf file and add this text inside the bitcoin.conf file below

Code:

datadir=$HOME/Library/Application Support/Bitcoin/blocks

Then save it we just adding this to force Bitcoin core to save and read blockchain under that path or you can point other directory path where you save your previous downloaded blockchain. Update here if it works  or not.

[marc3535]

BitMaxz, thanks, but it didn't work. I added this text inside the bitcoin.conf file but after restarting, Bitcoin Core won't start at all anymore. It says "Error reading configuration file: parse error on line 2: -datadir=$HOME/Library/Application Support/Bitcoin/blocks, options in configuration file must be specified without leading - "

[marc3535]

And FindingNemo, how I can ensure that the datadir parameter is correctly set to the directory where the blockchain data is stored?
PS What do you think what's the easiest way to run a bitcoin node? Bitcoin Core running by yourself has some problems obviously.

[BitMaxz]

BitMaxz, thanks, but it didn't work. I added this text inside the bitcoin.conf file but after restarting, Bitcoin Core won't start at all anymore. It says "Error reading configuration file: parse error on line 2: -datadir=$HOME/Library/Application Support/Bitcoin/blocks, options in configuration file must be specified without leading - "

Sorry the above parameter is wrong.

Remove the "-" on the datadir it should look like this datadir=yourblockdatafolder and then save.
It should look like this

Code:

datadir=$HOME/Library/Application Support/Bitcoin/

or replace it with this if you just want to point it to downloaded blocks.

Code:

blocksdir=$HOME/Library/Application Support/Bitcoin/blocks

[nc50lc]

When I restarted the macbook this morning, Bitcoin Core started again downloading the entire blockchain. I selected the right folder, I have an external SSD drive on which I previously already had downloaded the whole blockchain. Also I tried to click "reset options", but after restarting bitcoin core it again doesn't connect with the existing blockchain on the SSD drive but starts downloading from scratch. Does anyone have any ideas how to fix this?

Getting the logs may show some hints about the issue.
It's in your data directory, the file named "debug.log", open it as text to view the contents.

For now, if it's an issue with pathing of data directory, you can set it again in the GUI by starting bitcoin-qt with --choosedatadir command line option.
Or if you're using bitcoind, use --datadir=<path to custom 'bitcoin' data directory> arg like instructed above.
Or if you've set the default datadir, set datadir=<path to custom 'bitcoin' data directory> in the bitcoin.conf file inside bitcoin default data directory.

Then, if it started to resync again even after pointing to the correct directory, there may be issue in your blockchain, index or chainstate.
Such issue can be seen in the logs. Telling us your previous settings may help as well.

But to make this clear, does it show that it's downloading from block 0 again and not just catching-up to the tip from the last block height that it has?
Because Bitcoin Core has to do that once it's left closed for hours.

[satscraper]

Using external hard drive as your main blockchain data storage is not a good option,

I couldn't agree with this.

I use the  external drive ( now it is 870 EVO  1 TB SSD) for the copy of blockchain maintained by my node for many years, so  far so good, and go on with it.  

Why do you think it is a bad approach? Please, elaborate.

~

Check if bitcoin relevant directory on you external SSD has bitcoin.conf file.

Check if that file has the following entry datadir=<path to bitcoin relevant directory on you external SSD>

P.S. Some times I also have similar story when connect the wrong SSD (I have dozen of them). What I do - just reconnect my node to correct SSD .  

[goodbed]

Hi, I had Bitcoin Core up and running on my macbook. When I restarted the macbook this morning, Bitcoin Core started again downloading the entire blockchain. I selected the right folder, I have an external SSD drive on which I previously already had downloaded the whole blockchain. Also I tried to click "reset options", but after restarting bitcoin core it again doesn't connect with the existing blockchain on the SSD drive but starts downloading from scratch. Does anyone have any ideas how to fix this?

Ensure Bitcoin Core is closed, then open Terminal and navigate to the folder where Bitcoin Core is installed. Run the command to create a symbolic link to your blockchain data on the external SSD--

Code:

 ln -s /Volumes/YourSSDPath/Bitcoin ~/.bitcoin

Replace "YourSSDPath" with the actual path to your external SSD. After that, restart Bitcoin Core, and it should connect to the existing blockchain on the SSD.

[Medusah]

@marc3535, you can either create a symbolic link (with ln -s sourcepath linkpath) as proposed by goodbed or change the datadir in your bitcoin.conf and make it point to the directory that contains your bitcoin data.

[marc3535]

Unfortunately nothing functions. If I put the datadir to my ext SSD, it Bitcoin Core still starts from the beginning with downloading.
If I try to do it through terminal, then I get this output:
ln: illegal option -- /
usage: ln [-s [-F] | -L | -P] [-f | -i] [-hnv] source_file [target_file]
       ln [-s [-F] | -L | -P] [-f | -i] [-hnv] source_file ... target_dir
       link source_file target_file

So I don't what to do. Probably have to download the entire blockchain again.

[Findingnemo]

Using external hard drive as your main blockchain data storage is not a good option,

I couldn't agree with this.

I use the  external drive ( now it is 870 EVO  1 TB SSD) for the copy of blockchain maintained by my node for many years, so  far so good, and go on with it.  

Why do you think it is a bad approach? Please, elaborate.

The main reason why I said because external drives are prone to physical damage and often connection and disconnecting from the device can cause issues like OP had, I am not sure it is the actual reason but it could be a potential reason too.

The maximum Read and write speed of SSD can be achieved practically if we connect them using SATA ports.

[nc50lc]

So I don't what to do. Probably have to download the entire blockchain again.

Like I've mentioned above, checking the logs (debug.log) should point you to the right direction.
It contains command line args, configs, GUI settings that you might not have set correctly and specially errors that caused the issue.
You can post a few sessions inside [code][/code] tags, just remove some lines (for privacy purposes) if you want to.

Guessing the issue would only lead you to do unnecessary things that may even aggravate the problem.

[marc3535]

I understand what you say. But the debug.log file contains thousands and thousands of lines of information. I really wouldn't what to search for there...

[marc3535]

I let bitcoin core running again now and now I see in the SSD folder that it writes all kinds of .dat folders in the file chainstate at the moment, and not under blocks... could this mean something?

[marc3535]

This is a fragment of the debug.log file after restarting Bitcoin Core; does it contain any useful information?

2024-01-26T13:49:58Z Bitcoin Core version v25.1.0 (release build)
2024-01-26T13:49:58Z Qt 5.15.5 (static), plugin=cocoa (static)
2024-01-26T13:49:58Z Static plugins:
2024-01-26T13:49:58Z  QCocoaIntegrationPlugin, version 331520
2024-01-26T13:49:58Z  QMacStylePlugin, version 331520
2024-01-26T13:49:58Z Style: macintosh / QMacStyle
2024-01-26T13:49:58Z System: macOS 12.7, x86_64-little_endian-lp64
2024-01-26T13:49:58Z Screen: Colour LCD 1280x800, pixel ratio=2.0
2024-01-26T13:49:58Z GUI: Populating font family aliases took 231 ms. Replace uses of missing font family ".AppleSystemUIFont,13,-1,5,50,0,0,0,0,0" with one that exists to avoid this cost.
2024-01-26T13:49:59Z GUI: QFSFileEngine::open: No file name specified
2024-01-26T13:49:59Z GUI: QFSFileEngine::open: No file name specified
2024-01-26T13:49:59Z GUI: QFSFileEngine::open: No file name specified
2024-01-26T13:49:59Z GUI: QFSFileEngine::open: No file name specified
2024-01-26T13:49:59Z GUI: QFSFileEngine::open: No file name specified
2024-01-26T13:49:59Z Using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation
2024-01-26T13:49:59Z Using RdSeed as an additional entropy source
2024-01-26T13:49:59Z Using RdRand as an additional entropy source

2024-01-26T13:49:59Z Using data directory /Volumes/ext SSD
2024-01-26T13:49:59Z Config file: /Volumes/ext SSD/bitcoin.conf
2024-01-26T13:49:59Z Config file arg: datadir="/Volumes/ext SSD/blocks"
2024-01-26T13:49:59Z Config file arg: server="1"
2024-01-26T13:49:59Z Using at most 125 automatic connections (2560 file descriptors available)
2024-01-26T13:49:59Z Using 16 MiB out of 16 MiB requested for signature cache, able to store 524288 elements
2024-01-26T13:49:59Z Using 16 MiB out of 16 MiB requested for script execution cache, able to store 524288 elements
2024-01-26T13:49:59Z Script verification uses 3 additional threads
2024-01-26T13:49:59Z scheduler thread start
2024-01-26T13:49:59Z Binding RPC on address ::1 port 8332
2024-01-26T13:49:59Z Binding RPC on address 127.0.0.1 port 8332
2024-01-26T13:49:59Z [http] creating work queue of depth 16
2024-01-26T13:49:59Z Using random cookie authentication.
2024-01-26T13:49:59Z Generated RPC authentication cookie /Volumes/ext SSD/.cookie
2024-01-26T13:49:59Z [http] starting 4 worker threads
2024-01-26T13:49:59Z Using wallet directory /Volumes/ext SSD
2024-01-26T13:49:59Z init message: Verifying wallet(s)…
2024-01-26T13:49:59Z Using /16 prefix for IP bucketing
2024-01-26T13:49:59Z init message: Loading P2P addresses…
2024-01-26T13:49:59Z Loaded 63224 addresses from peers.dat  316ms
2024-01-26T13:49:59Z init message: Loading banlist…
2024-01-26T13:49:59Z SetNetworkActive: true
2024-01-26T13:49:59Z Cache configuration:
2024-01-26T13:49:59Z * Using 2.0 MiB for block index database
2024-01-26T13:49:59Z * Using 8.0 MiB for chain state database
2024-01-26T13:49:59Z * Using 440.0 MiB for in-memory UTXO set (plus up to 286.1 MiB of unused mempool space)
2024-01-26T13:49:59Z init message: Loading block index…
2024-01-26T13:49:59Z Assuming ancestors of block 000000000000000000035c3f0d31e71a5ee24c5aaf3354689f65bd7b07dee632 have valid signatures.
2024-01-26T13:49:59Z Setting nMinimumChainWork=000000000000000000000000000000000000000044a50fe819c39ad624021859
2024-01-26T13:49:59Z Opening LevelDB in /Volumes/ext SSD/blocks/index
2024-01-26T13:49:59Z Opened LevelDB successfully
2024-01-26T13:49:59Z Using obfuscation key for /Volumes/ext SSD/blocks/index: 0000000000000000
2024-01-26T13:50:00Z LoadBlockIndexDB: last block file = 116
2024-01-26T13:50:00Z LoadBlockIndexDB: last block file info: CBlockFileInfo(blocks=237, size=104983793, heights=285502...286626, time=2014-02-12...2014-02-19)
2024-01-26T13:50:00Z Checking all blk files are present...
2024-01-26T13:50:00Z Initializing chainstate Chainstate [ibd] @ height -1 (null)
2024-01-26T13:50:00Z Opening LevelDB in /Volumes/ext SSD/chainstate
2024-01-26T13:50:01Z Opened LevelDB successfully
2024-01-26T13:50:01Z Using obfuscation key for /Volumes/ext SSD/chainstate: c9419c6c1a4b94b7
2024-01-26T13:50:01Z Loaded best chain: hashBestChain=0000000000000000ebed3e2b0cf036d577ffec8a7c4015d64499e8a791b82c0b height=286523 date=2014-02-18T14:46:52Z progress=0.036239
2024-01-26T13:50:01Z [snapshot] allocating all cache to the IBD chainstate
2024-01-26T13:50:01Z Opening LevelDB in /Volumes/ext SSD/chainstate
2024-01-26T13:50:01Z Opened LevelDB successfully
2024-01-26T13:50:01Z Using obfuscation key for /Volumes/ext SSD/chainstate: c9419c6c1a4b94b7
2024-01-26T13:50:01Z [Chainstate [ibd] @ height 286523 (0000000000000000ebed3e2b0cf036d577ffec8a7c4015d64499e8a791b82c0b)] resized coinsdb cache to 8.0 MiB
2024-01-26T13:50:01Z [Chainstate [ibd] @ height 286523 (0000000000000000ebed3e2b0cf036d577ffec8a7c4015d64499e8a791b82c0b)] resized coinstip cache to 440.0 MiB
2024-01-26T13:50:01Z init message: Verifying blocks…
2024-01-26T13:50:01Z Verifying last 6 blocks at level 3
2024-01-26T13:50:01Z Verification progress: 0%
2024-01-26T13:50:01Z Verification progress: 16%
2024-01-26T13:50:01Z Verification progress: 33%
2024-01-26T13:50:01Z Verification progress: 50%
2024-01-26T13:50:01Z Verification progress: 66%
2024-01-26T13:50:01Z Verification progress: 83%
2024-01-26T13:50:01Z Verification progress: 99%
2024-01-26T13:50:01Z Verification: No coin database inconsistencies in last 6 blocks (2990 transactions)
2024-01-26T13:50:01Z  block index            2066ms
2024-01-26T13:50:01Z Setting NODE_NETWORK on non-prune mode
2024-01-26T13:50:01Z loadblk thread start
2024-01-26T13:50:01Z Reindexing block file blk00000.dat...
2024-01-26T13:50:01Z block tree size = 286627
2024-01-26T13:50:01Z nBestHeight = 286523
2024-01-26T13:50:01Z torcontrol thread start
2024-01-26T13:50:01Z Bound to 127.0.0.1:8334
2024-01-26T13:50:01Z Bound to [::]:8333
2024-01-26T13:50:01Z Bound to 0.0.0.0:8333
2024-01-26T13:50:01Z Loaded 2 addresses from "anchors.dat"
2024-01-26T13:50:01Z 2 block-relay-only anchors will be tried for connections.
2024-01-26T13:50:01Z init message: Starting network threads…
2024-01-26T13:50:01Z net thread start
2024-01-26T13:50:01Z dnsseed thread start
2024-01-26T13:50:01Z Waiting 300 seconds before querying DNS seeds.
2024-01-26T13:50:01Z addcon thread start
2024-01-26T13:50:01Z opencon thread start
2024-01-26T13:50:01Z msghand thread start
2024-01-26T13:50:01Z init message: Done loading
2024-01-26T13:50:01Z GUI: Platform customization: "macosx"
2024-01-26T13:50:01Z UpdateTip: new

[NotATether]

2024-01-26T13:50:01Z Opening LevelDB in /Volumes/ext SSD/chainstate
2024-01-26T13:50:01Z Opened LevelDB successfully
2024-01-26T13:50:01Z Using obfuscation key for /Volumes/ext SSD/chainstate: c9419c6c1a4b94b7
2024-01-26T13:50:01Z [Chainstate [ibd] @ height 286523 (0000000000000000ebed3e2b0cf036d577ffec8a7c4015d64499e8a791b82c0b)] resized coinsdb cache to 8.0 MiB
2024-01-26T13:50:01Z [Chainstate [ibd] @ height 286523 (0000000000000000ebed3e2b0cf036d577ffec8a7c4015d64499e8a791b82c0b)] resized coinstip cache to 440.0 MiB

Your volume and configuration is successfully detected by Bitcoin Core. It also looks like it saved your chainstate, which is at about 25% of the way there.

Paste the output of bitcoin-cli getblockchaininfo or show us a screenshot (if it's the GUI version) of the verification progress so we can check that it hasn't reset its block download progress again.

[marc3535]

 Thanks. This is the output of getblockchaininfo.  Can you see if it has reset the block download progress? Or maybe just again syncing?

"chain": "main",
  "blocks": 535936,
  "headers": 826689,
  "bestblockhash": "000000000000000000247fb1af3730f105ebf14b303d1c7eb6282cff46e94815",
  "difficulty": 5949437371609.53,
  "time": 1533817015,
  "mediantime": 1533815708,
  "verificationprogress": 0.3645248712057892,
  "initialblockdownload": true,
  "chainwork": "000000000000000000000000000000000000000002aae2c91fd6039d939eeb19",
  "size_on_disk": 567624183244,
  "pruned": false,
  "warnings": ""

[nc50lc]

This is a fragment of the debug.log file after restarting Bitcoin Core; does it contain any useful information?

Not much since the log before the restart isn't included for comparison.
But there are indications that the datadir that it loaded doesn't have the full blockchain,
since it's after the restart from an allegedly fully synced blockchain, it shouldn't be like than unless it's an issue with your Mac or a change in the previous setting.
Plus the date doesn't match, it looks like the logs after following the first few replies. (I see it in the config)

And there are some inconsistencies like this:

2024-01-26T13:49:59Z Config file arg: datadir="/Volumes/ext SSD/blocks"

That is the blocks directory and should be set as "blocksdir=" if you do not want to save the blocks inside the datadir.
Since you've set it like that, it would use "/Volumes/ext SSD/blocks" as the datadir and save the blockchain in "/Volumes/ext SSD/blocks/blocks".
Yet your node shows that it's using "/Volumes/ext SSD" as the datadir since you've set it in the GUI's welcome screen.
(assuming that there are no lines omitted in the gap in the middle of the shared logs)
Just remove the datadir setting in the bitcoin.conf file since it's incorrect and does nothing.

I understand what you say. But the debug.log file contains thousands and thousands of lines of information. I really wouldn't what to search for there...

You can paste it inside the code tags like I've instructed in my earlier reply.
That way, it wont consume a lot of vertical space.

You can also pick the sessions that are relevant like you did in the first log that you shared.

Thanks. This is the output of getblockchaininfo.  Can you see if it has reset the block download progress? Or maybe just again syncing?

  "blocks": 535936,
  "headers": 826689,
  "initialblockdownload": true,

It's syncing and currently at 535936 height.