Extremely slow reindexing

[BlackHatCoiner]

Bitcoin Client Software and Version Number: Bitcoin Core 26.0
Operating System: Raspbian Lite OS (64-bit)
System Hardware Specs: 4 GB memory, 2 TB external HDD.
Description of Problem:

TL;DR: I switched disks, and somehow either the blocks or the chainstate got corrupted, and I had to reindex. I'm one week now, at progress=0.313028. It seems like it goes much slower now. Yesterday it was in 0.30. At this rate, it will take more than a month to reach the tip. I was thinking if it's possible to eject the HDD, plug it a computer with better specs, so it could finish... this month. It should be possible to do, right? The disk is loaded with Raspbian, so I could theoretically load it during boot in any other machine.

(Asking just in case I don't mess things up)

[1714228720]

1714228720

[1714228720]

1714228720

[1714228720]

1714228720

[1714228720]

1714228720

[apogio]

I don't see a problem with doing what you suggest. I have never tried it though. To be honest, I don't understand how the directories were corrupted, but reindexing can be painful, indeed.

What I have tried in the past, and it worked, was to download the blockchain on my SSD using Windows 11 OS. Then I deleted Bitcoin Core, because I performed a format. Then I downloaded Core again and set it to look in the SSD again. It worked flawlessly. I guess it's a relevant situation, isn't it ?

[BlackHatCoiner]

What I have tried in the past, and it worked, was to download the blockchain on my SSD using Windows 11 OS.

I've considered doing that, but I don't have a terabyte of SSD, and I don't want to buy one just for that purpose. HDD satisfies my needs. Sure, Bitcoin Core would be less troublesome, but if it works, even slowly, then I don't care.

Then I downloaded Core again and set it to look in the SSD again. It worked flawlessly. I guess it's a relevant situation, isn't it ?

Hmm. Maybe if I used Bitcoin Core from my main computer without booting into Raspbian, and pointed to the Bitcoin data directory in the plugged HDD, then maybe it would reindex more rapidly. There are quite a lot of permission issues this way, though. (The datadir is read only, and I'll have to run Bitcoin Core as root)

[ranochigo]

Yes you can. Format of data directories between versions are the exact same and there is no compatibility issues whatsoever. Of course, remember to shut it down gracefully.

In fact, this was how I encouraged people to do IBD on a proper Desktop instead of their RPi. Resync on RPi was really long for me, and using the Desktop was much less frustrating in comparison.

[LoyceV]

I was thinking if it's possible to eject the HDD, plug it a computer with better specs, so it could finish...

That should work, although I'd prefer to use it as a secondary disk in another system, and not boot from it.
Your problem is the 4 GB RAM in combination with 11 GB chainstate. Add HDD, and it's a very bad combination.
As far as I know, reindexing chainstate requires almost as much writing to disk as a full fresh sync. It only saves writing 580 GB blocks. I wrote 4.58 GB to disk with 8 GB RAM during IBD. Remove the blocks, and you're left with 4 TB. And that was with twice the ram, so you'll need to write more than 4 TB. That's 4+ TB reading and writing more or less random access on a HDD. If it takes a month, that's 1.5+ MB/s on average, which could very well be the best the HDD can do with random reads and writes. This is why I prefer to put chainstate on SSD.
Progress will get worse once you're further ahead, see my graph.

[satscraper]

Bitcoin Client Software and Version Number: Bitcoin Core 26.0
Operating System: Raspbian Lite OS (64-bit)
System Hardware Specs: 4 GB memory, 2 TB external HDD.
Description of Problem:

TL;DR: I switched disks, and somehow either the blocks or the chainstate got corrupted, and I had to reindex. I'm one week now, at progress=0.313028. It seems like it goes much slower now. Yesterday it was in 0.30. At this rate, it will take more than a month to reach the tip. I was thinking if it's possible to eject the HDD, plug it a computer with better specs, so it could finish... this month. It should be possible to do, right? The disk is loaded with Raspbian, so I could theoretically load it during boot in any other machine.

(Asking just in case I don't mess things up)

I had the same experience some time ago. After  deterioration of chainstate data on my external SSD I have fallen for reindex. This  process  was as slow as it was  less than effective. Thus I had to start the sync from scratch.

Hence, to not waste time,  I would advocate  IBD rather than reindex in the case of any chainstate corruption.

[DaveF]

If you do it on a full desktop system, mind the file permissions. 99% of the time there is no issue that 1% will drive you nuts.

And I know it's been said but I'll say it again, get a SSD

And, with all the disk I/O and everything else make sure the RPi is not throttling for either temperature because the CPU being hammered or power because of the constant pull from the drive.

-Dave