[SOLVED] Is there a quick solution to Blockchain Rescan Failed ?

[FredCailloux]

Bitcoin-qt 25.0.0 with complete up-to-date blockchain as been running fine for a while now on a Linux Fedora PC.
I closed the wallet and created another one, just to contain some funds from a paper wallet.
After "importprivkey" the wallet started to rescan the whole blockchain, which took quite a long time.
In the morning a "Rescan failed" error was on the screen.
Although, the funds where available. After a second attempt, same error.
Then, I closed that new wallet and re-openned the original wallet. Repeated the "rescanblockchain" and the same error popped-up again.

It appear the error is not dependent on the wallet, since both wallets do the same thing.
Error message: https://ibb.co/5KChMNP

I search this forum for potential solutions but none came out.

The Questions:
1- Is there something I can do to remedy the situation other than download the whole blockchain again ?
2- Could that error have an impact on further transactions ?

I'd like to avoid having to download the whole Blockchain again, that will take a long time.

[1714515674]

1714515674

[1714515674]

1714515674

[1714515674]

1714515674

[BitMaxz]

Take note that the current latest version of Bitcoin Core is 26.0 it would be better if you upgrade Bitcoin core to the most recent version.

Instead of using "rescanblockchain" have you tried to run Bitcoin core with "-rescan"? It won't take too long to rescan depending on your unit specs.

[FredCailloux]

run Bitcoin core with "-rescan"? It won't take too long to rescan depending on your unit specs.

I tried to call from Terminal : "bitcoin-qt -rescan" , but I guess this is not the correct method for
I am getting a "command not found" error.
Then it is offered to Install a package "bitcoin-core-desktop", which I answered No, for fear of breaking something.
Can you pls indicate how to properly call "bitcoin-qt -rescan" ?  Thanks

[achow101]

Please post the contents of your debug.log file.

[FredCailloux]

Please post the contents of your debug.log file.

I assume you want to see the debug.log file in the bitcoin folder ?
That file is relatively huge:  10 MB that's a lot of lines ! 61166 lines, to be specific.
Probably due to a "rescanblockchain" from yesterday.

Is there a way I can attach a file on this post ?

[achow101]

Please post the contents of your debug.log file.

I assume you want to see the debug.log file in the bitcoin folder ?
That file is relatively huge:  10 MB that's a lot of lines ! 61166 lines, to be specific.
Probably due to a "rescanblockchain" from yesterday.

No, rescanning doesn't output that much, it's probably mostly just all UpdateTip lines for every block that was received and validated

Search through the file for any lines that have the word "rescan" in them and copy those lines and a bunch around them.

Is there a way I can attach a file on this post ?

Upload it to a file sharing service and post the link here.

[FredCailloux]

Upload it to a file sharing service and post the link here.

I deleted the previous debug.log and re-started Bitcoin.
Then I performed a rescanblockchain. This is the debug.log file that came out.
https://drive.google.com/file/d/1FwFrrV-i2oB0ZqeTEOl1XRMW6yHl8B8W/view?usp=drive_link
It's a 20000 lines file with a Lot of "ERROR: ReadBlockFromDisk"
Strangely, Bicoin start OK and shows proper wallet balance.
Even more strange is the fact that in Menu/Help/AboutBitcoinCore, the version listed is 26 (cause I updated it)
Yet, the debug.log show version 25 ?
I'm eager to read you comments on this. thanks

[Cricktor]

Your debug.log has 4 starts of Bitcoin Core v25.0.0 that all were shutdown OK.

At 2024-03-02T21:38:10Z you started Core v26.0.0 and it was able to rescan until block 623709 without issues. After that you have numerous errors like

Code:

2024-03-03T02:57:59Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at FlatFilePos(nFile=2018, nPos=81489093)
2024-03-03T02:57:59Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at FlatFilePos(nFile=2018, nPos=86785969)
2024-03-03T02:57:59Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at FlatFilePos(nFile=2018, nPos=99727177)
2024-03-03T02:57:59Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at FlatFilePos(nFile=2018, nPos=103458026)
2024-03-03T02:57:59Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at FlatFilePos(nFile=2018, nPos=118177283)
2024-03-03T02:57:59Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at FlatFilePos(nFile=2018, nPos=120613759)
2024-03-03T02:57:59Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at FlatFilePos(nFile=2018, nPos=123082202)
2024-03-03T02:57:59Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at FlatFilePos(nFile=2019, nPos=10439794)
2024-03-03T02:57:59Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at FlatFilePos(nFile=2019, nPos=11967586)
2024-03-03T02:57:59Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at FlatFilePos(nFile=2019, nPos=14543115)
2024-03-03T02:57:59Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at FlatFilePos(nFile=2019, nPos=25973905)
2024-03-03T02:57:59Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at FlatFilePos(nFile=2019, nPos=60111692)
2024-03-03T02:58:00Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - Superfluous witness record: iostream error at FlatFilePos(nFile=2018, nPos=76480352)
...

To me it looks like your blockchain data files are corrupted from a certain point on.

And later in your debug.log there's total garbage that doesn't belong there. Didn't you notice that? Something's is severely wrong with your system (could be some issue with the filesystem or whatever caused that garbage to get dumped into the debug.log text file).
Another possibility is that the AfterRescan.log file got damaged in the process of uploading to your Google Drive, though I doubt that.

BTW, it's probably better to use pastebin.com or similar than the metadata hog Google.


Your blockchain data seems to be on an additional maybe external drive with is mounted at /mnt/DataE.
Did you always unmount this drive cleanly?
Did you experience any crashes of your system that might have corrupted your presumably external data drive?

If you perform a filesystem check, what's the result of that?

IMPORTANT: before you mess with valuable data, make backups!

[FredCailloux]

Did you experience any crashes of your system that might have corrupted your presumably external data drive?
If you perform a filesystem check, what's the result of that?

The PC is a Dell with an SSD 240GB and HDD 4TB. Both partitions DataD and DataE are on the HDD, 2TB each.
Bitcoin is on DataE with the whole blockchain.
Last week, after a request from Fedora 38, I proceeded with a system update.
The Fedora update went well and rebooted my machine. Now Fedora 39.
Right after the reboot the whole 4TB drive, both partition were invisible.
After some search I found the linux tool TestDisk. https://www.cgsecurity.org/wiki/TestDisk_Download
I ran TestDisk and it found my drive back and performed some commands which brought back everything.
After that Fedora 39 could read the drive as before and Bitcoin was working, seemingly, fine.
Except, of course, the rescanblockchain error, which is most probably due to this drive lost after Fedora update.

I ran fsck, following your suggestion, with instructions found online.
I'm not proficient in linux commands, let alone deep understanding.
What I got is an error stating "Bad magic number in super-block" and a suggestion to run e2fsck - b 8193 or other sizes.
I remember when running TestDisk, it was found that the drive was of NTFS type.
Which correspond with the initialization when the drive was first installed two year ago.
So now I'm hesitant to run any transformation on that drive,
with my limited knowledge and potential negative consequences.
I definitely need guidance in that direction, if anything could bring the drive back to normal.
Thanks

PS: Pastebin.com will not accept more than 512k. No good for this debug.log file
Unless there is a way to grab the whole file as a single link, which I could not find.

[LoyceV]

Then I performed a rescanblockchain. This is the debug.log file that came out.
https://drive.google.com/file/d/1FwFrrV-i2oB0ZqeTEOl1XRMW6yHl8B8W/view?usp=drive_link
It's a 20000 lines file with a Lot of "ERROR: ReadBlockFromDisk"

I'll quote a part:

Code:

2024-03-03T02:58:02Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at FlatFilePos(nFile=2020, nPos=49361069)
2024-03-03T02:58:02Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at FlatFilePos(nFile=2020, nPos=6576281)
2024-03-03T02:58:02Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at FlatFilePos(nFile=2020, nPos=37831041)
2024-03-03T02:58:02Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at FlatFilePos(nFile=2020, nPos=38842756)
2024-03-03T02:58:02Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at FlatFilePos(nFile=2020, nPos=118374383)
2024-03-03T02:58:02Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at FlatFilePos(nFile=2020, nPos=5309736)
2024-03-03T02:58:9±}©h∑ ¡ 
…ƒ#º°2rè≤äñ≥ ‘延_flÅ/Û ÔI–K	qãQ©è[ê5™P™•z JxW,πj†eBd fl¶F∏üMø∂bÅ2f U+§´ ∆É˙Iú9E[%@èè(«K	Úv≤Où¶(|*4m Œ ÙÆ&k/i‘ò),≈„Ó —Nóyˆ(@À' F&®Ò‚‡îáêÚ\+…ôdE∂L¥ß .É t  ‰∏x z›íà
’ò ”µ‰˜ >))ŒÚô
-ÉëùÏÚ¨ˆÔ•˘kÍíî ÿ≈€ü[´
ÕºÇ{/åXx÷  ÑØ åeQ∑¢>y ì 6gNt> … –æÿ% Œ'¥‡æ®+zÿñi$¬IKk¥‡⁄–e9G®LÌ_…]	À∫ €∞œ··\÷êpÒãø$
9 ¯‹Ùâ4ò~„u˜#‹Ô ~Zºù‹o±keÜ∆ø’Ôy¡ IGtT) 6G ~”ßƸ(9∫ºÔ am+né≥∑ÔúÈ—ñèE ∑G=6› ‡X¥\Æ u^ª 0Ûh>„é „á #»¶Ì â_¬ô PI€‚F €àIflØ%pH θ¢Wæ)√^æ“l!$´èIu íÇeC£Á E“∞∞'å§ êE%!)¬òk‡7?%	Q(ü ù뜂ƒ m±ÒB±ëŒƒÈcD “£˘utW´ Y ˝K5^ˆ3˛ìq ,üљ٥fi•˜í_€“ŒU ˛Gj®l∏fi∑	C«Ñ∞∑ç√O î–[$T:â ç‰ò@Œ°/Ω^™
XX˛Ò≠îv3Ω8†MY î»*˙!å
?ŸæJ≤õ1pmï‘∂ÖWÔ◊~RV<®≥Æu?>ùcRá}   ï-„U
CíÈö†,   Y ≈∫ƒ‚∆ ( 9ÕÛA %!√ë ûçäÖC J=ë˛D°>Åm É«flˆ0 ◊$v©Ià3. ı`6õ¢–Tô•°H
∆S>îf‘oœ‚X  ¸ËK]M˚tÌ9”¸@]ÖY€oô ! Je3ÃnÕû‡Âûä ◊ñ!T	±M  fl ºFíˇI= £ÜÄpÄ°∆e÷·¨˝ÕßSΩµD{Qá”ÜQπ mÙ—#“ kI∞¥øÌíÚVπ§öØƧ⁄d Œ– |®Á°wÿ/9ö ´Oi¡€
[O∫¡¯◊&Ç  2˚j:Ê◊u»ª˛RR{1~Ò qªbŸÍŸ°V ¡^Oúä9y -#'! Z•
O%, =e‹|Ar Lâ  w•Ç£gâ„ F€ÖT} ∏ æ刄¿c!7 ÉRC$∏⁄Ê  å¡hG ^Ñà+õ˙T™ óà àˆåqæå√o= öË`o–≤‰Ho? ¬6˚#M ?MPôdçäQ ›â‚Ò§„ü°0'wæ´c	 ªÑ_ÇÖ·  °Û „ ƒ ìÎ/xá'πA v ı∂àæ€Ö<û≤o: °€ ê÷Hîã ‚ A¢1Ï˚î‘ ˘ÚØ˘ôÁ 匙 Ã⁄Fw{Ï0ÀW|fi˘vïrT &|y'H÷;< y˘\˛™ÉP #Û8ıXN § ¥∆µ “≥Î ◊ ÈÛ_ ö  2.{• Yï$Ω e8≈» πÑxı≈ IÜ]ØC…∑Vc qÖ ,Ìøè  lph- µ˜‘;( ÷x|ÛsgÈûP-õ¡  •Z|-Òq∫
 Óî ∞0:Àu€í ü?¯Ã{fl	¯ÇÉãü(¨¬¥ÀF¯ dºQùˆûo ʧµ πfl][ÁÍ ˙OvcÙ/ŒÇ÷†j† ᡠAÊqì±åi∂æ• o Æ”ä ’à°î˘æÙØÙ;®Ø~'•Q‡è|dP-s–^‚"íeBUø:’◊  ãg9h˙$¨ï¶;Í˚R||WãH_ OãÒ. ÌN“öÖl ¬Ùz+∆ÑåÊ< ¬Ûƒ°ø¬M„ Œ ˛©s®vgò1@Ω…π7¶jÒf] >OÒ‹…ß˚TΩ¯fiA† o◊Mì∏ë∆Π õ›œ Í=À¶Rñ¬ØV£ ˙¥ÔK\2®;™' MiÑ?Ê¢,JÔà7{=ó8« W@  ƒZ . õ’äú  ÿ›æï2ÔÉn ÁØ
 Ü ¶ íÕ√àñè  ªAFõÕ™Ë∂6x1˜Ò \ O ∏ΩC c5 °´öæAæ€,R—c‰ÏπS8FÔ Ø≠mì˚)èØí'50"«˙ØÌ5›b l?xÚ ¬[≈     Â/?]M 'pV· 9v¨⁄ ∫A≈Úéí5»9 8ˇ|é÷E ¸Æj°òeç \√Ø˙æ
¥s.["Ü…Y <Œ>Õi ›‰˝Bnô Füæ v@m yã;ú≥ÍqØp≠GñAƒ™ b“≥ *QÜ¥E–Åò Àé flü0≥ñ ™H YßO &Xo,˚x™ººy´ì§GZã˚‰ Ò 9î æÒD∞x·	^cG Ä— Q’2b€´› 7ŸjÄÎè{»Û˘4U'.Ç8# Œ27EÑÂ1®Ωñ§“ΩW"*,¥éd2µ’¢^‰ª^R3∆±  ˛.o§–	‹ ñ *í≈>˙1’–ôπ fl ã ‘àö=£k¢ö¡≥≥,Gfl)flï∆K^ ˇ·A ≠\¡t™
 Rf›gnŸ•^[˜5œ: H;Å 

That doesn't look like a normal debug.log.

To me it looks like your blockchain data files are corrupted from a certain point on.

Agreed.

The PC is a Dell with an SSD 240GB and HDD 4TB. Both partitions DataD and DataE are on the HDD, 2TB each.
Bitcoin is on DataE with the whole blockchain.
Last week, after a request from Fedora 38, I proceeded with a system update.
The Fedora update went well and rebooted my machine. Now Fedora 39.
Right after the reboot the whole 4TB drive, both partition were invisible.

It sounds like you somehow messed up your file system, and/or your drive is failing.

This is what I would do: backup all data from the 4 TB drive you want to keep. Anyone should always have backups ready, but I'll mention it anyway. Backup your wallet.dat files (do it again, even though you should have several backups). And just in case, make another backup on another (brand) USB stick.

Test the disk. Start with the SMART status, and run a disk test. If you're more familiar with Windows, you could format it into a Windows format and test the drive there. If it passes all tests, restart your blockchain download. Put chainstate on the SSD and use as much dbcache as you can spare. Depending on specs it can take less than a day up to many weeks.
If the disk fails any tests, replace it.

[ABCbits]

Even more strange is the fact that in Menu/Help/AboutBitcoinCore, the version listed is 26 (cause I updated it)
Yet, the debug.log show version 25 ?

It's only weird if you delete the debug.log file just before you run Bitcoin Core 26.0. Anyway, as stated by @Cricktor, the log also shows you run Bitcoin Core version 26.0 once.

I remember when running TestDisk, it was found that the drive was of NTFS type.

If you only use Linux to access certain partition or drive in the future, i would recommend you to use ext4 instead. Based on my experience, it's more tricky to fix NFTS specific error on Linux rather than Windows.

[FredCailloux]

The corruption in the sampled file above is probably due to a bad upload on Google Drive. Cause when I see it directly from the Fedora computer, the file is clean.
Anyway, I decided to completely erase the HDD and perform thorough test on it and reformat.
I want to be sure that this HDD is healthy before I spend more time adding data to it.
Thanks to all for your valuable inputs.