Very hard to sync Core Client from scratch

[2J87qy5arXEXVhc9hfT2]

Recently I started running a Core 15.1 node with the blockchain on an external hard drive, and found it very hard to fully sync the blockchain.
The problem I keep running into is that (in about 90% of my attempts) the sync does not complete. A DB corruption occurs at some point (usually pretty late in the game, maybe 36 hours into it).

I did some digging in forums and saw that I am not the first user to notice this problem. Peter Wuille mentioned that he does a full sync on like a weekly basis and it always works, and he thus suspects that these type of issues are due to sub optimal hardware.

That seemed very reasonable at first because my initial attempt was done using a pretty old 400gig pocket hard drive. So I switched hard drives.
By now, about 2 months later, I have tried to sync the blockchain from scratch about 20 or 30 times and only twice managed to complete the process.
I did that on 4 different types of hard drives, including a brand new 1T Western Digital.

One may say that all 4 hard drives I used are "suboptimal", but I would object to that because
1. After the sync does complete, the node runs just fine.
2. I think it is desirable to be able to run a full node on a raspberry pi with a pocket size hard drive (note that I run the node on a raspberry pi but use my linux desktop to do the initial sync)

Now the issue I have here is not so much the crash during sync, but the fact that when I restart the client after the crash the only option is to basically start from scratch (If I understand correctly, the client is not necessarily downloading everything from scratch but that is not my point) and sync again from Genesis.
Ideally I would like the client to keep synching from the point it successfully got to and not re sync from scratch on such errors.

I am wondering if this resync from scratch on error is necessary. I assume that corruption occurs in one (or more) of the files in the blocks or chainstate directories. If that is so then why sync from Genesis? Is it not possible to, say, delete all files post the corrupted file and sync from there?

If there is no fundamental reason for this behaviour, I would also appreciate a pointer to the relevant code in the client.

Thanks!!

[ScripterRon]

You don't need to delete the blocks directory.  But you do need to delete the chainstate directory if it is corrupted.  LevelDB (which is used for chainstate) causes most of the disk activity and is the most likely source of any corruption, especially if you kill the node without going through the normal shutdown.

[2J87qy5arXEXVhc9hfT2]

Thank you for your answer.
So say I sync the node and get a db corruption when the node is, say, 90% synced. Now I delete the chainstate directory and restart the node. Will the node have to re-sync from Genesis (and take the usual 36~48 hours) or will it resume from 90%?

[ranochigo]

Now I delete the chainstate directory and restart the node. Will the node have to re-sync from Genesis (and take the usual 36~48 hours) or will it resume from 90%?

Your node would have to rebuild the chainstate by going through all the blocks that you've already downloaded and synchronized. I'm not sure if deleting it would actually require the blocks to be validated again (achow101 said Core does). Anyhow, you can execute -reindex-chainstate to rebuild the chainstate only and that wouldn't need to validate the blocks again since your client is getting the information from your already compiled block index.

If you have a spare hard drive, you can think about synchronized it first and copying the entire data directory to it. If it gets corrupted again, you can just replace the data directory.

[2J87qy5arXEXVhc9hfT2]

Now I delete the chainstate directory and restart the node. Will the node have to re-sync from Genesis (and take the usual 36~48 hours) or will it resume from 90%?

Your node would have to rebuild the chainstate by going through all the blocks that you've already downloaded and synchronized. I'm not sure if deleting it would actually require the blocks to be validated again (achow101 said Core does). Anyhow, you can execute -reindex-chainstate to rebuild the chainstate only and that wouldn't need to validate the blocks again since your client is getting the information from your already compiled block index.

If you have a spare hard drive, you can think about synchronized it first and copying the entire data directory to it. If it gets corrupted again, you can just replace the data directory.

Thanks.
Rsyncing from another hard drive is exactly what I did once I realised achieving a full sync from scratch is not likely to happen. Anyway, your remark about the -reindex-chainstate flag is interesting. I'll try that. However, I would still like to understand why the default behaviour is the way it is.