Various error while "getrawtransaction"

[Tycek]

Hello.

I am running bitcoind.exe with the parameter of -txindex. I am trying to get transaction data by using getrawtransaction command in bitcoin-cli. However I am running into 3 errors:

• ERROR: FindTx: txid mismatch
• ERROR: FindTx: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error
• ERROR: FindTx: Deserialize or I/O error - non-canonical ReadCompactSize(): iostream error

The second error seems to be most common. I tried reindexing blockchain, but it did not solve my problem.

When I first downloaded the whole blockchain, I used GUI application, so I do not know, what parameters it used to run bitcoind to download the blockchain.

Would redownloading the whole blockchain solve my problem? And if so, what parameters of bitcoind should I use?

[1714101242]

1714101242

[ABCbits]

If you keep experience "Deserialize or I/O error", then most likely the problem is your storage drive.

Check your HDD for bad storage sector or check your SSD health

[BitMaxz]

Try to add this parameter "-txindex=1" with "=1"

I just got this idea from stackexchange question you can find the source from here below.

If you're using Bitcoin Core just for your own personal use, you probably don't need the -txindex=1 option. But, if you're using Bitcoin core for development or blockchain analysis of some sort, you will need to set -txindex=1 to be able to get transactions data for any transaction in the blockchain.

It means you can use the "getrawtransaction" command if it set to "-txindex=1"

[nc50lc]

Check your HDD for bad storage or check your SSD health

@Tycek Follow this ^

But if there isn't any problem with the disk, redownloading the blockchain might solve the issue.
You don't have to add any parameter, just delete everything inside blocks and chainstate folder in the data directory to sync from scratch.
You can also add txindex=1 to your bitcoin.conf file from the default data directory and specified data directory for it to work on both Bitcoind/cli and qt.

[Tycek]

I checked the disk with wmic, crystal disk info and tool from the manufacturer of the disk. All 3 test came without any problems on the disk. Before that I decided to redownload the blockchain. After that will be done, I will try again with the parameter -index=1.

I deleted the content of the folder blocks, but did not delete contents of the folder chainstate and run bitcoind with the parameter of -reindex.

[Tycek]

So I finished redownloading the whole blockchain, and the problem still persists. Not even -txindex=1 solved this problem. I noticed, that transactions that worked before, do not work now after the redownloading.

I am wondering. Is it even possible to get truly any transaction from the bitcoin core daemon? Or am I missing something?

[HCP]

Honestly sounds like your blocks are corrupted. This could be caused either by failures of the storage device, the motherboard, the CPU or the RAM.

What are the specs of the hardware you are using? I note you checked the storage device, but have you tried using Memtest86+ (https://www.memtest.org/) to test your RAM?

[BitMaxz]

If you have extra hard disk try to transfer the blockchain to another disk and then reconfigure it again.

I just got this solution from some post from github here https://github.com/bitcoin/bitcoin/issues/5668#issuecomment-113848906

Other users solved their problem by removing the whole blockchain and then redownload. Just do this if you don't have an extra hard disk to test the solution above but if this doesn't work you need to extra hard disk to test.

[Tycek]

I am wondering. Is it even possible to get truly any transaction from the bitcoin core daemon? Or am I missing something?

It's possible, but last time i tried to that, it's a bit annoying. You need to get to get raw transaction first (if you know hash of the transaction), then decode the the raw transaction to get the transaction info.

Well the command

Code:

getrawtransaction <transaction hash>

has another parameter for verbosity, which can output the transaction in JSON format.

Honestly sounds like your blocks are corrupted. This could be caused either by failures of the storage device, the motherboard, the CPU or the RAM.

What are the specs of the hardware you are using? I note you checked the storage device, but have you tried using Memtest86+ (https://www.memtest.org/) to test your RAM?

My specs are:

• CPU: Intel Core i9-7940X
• RAM: 128GB RAM DDR4
• MB: Asus rog rampage VI extreme
• GPU: 4x Nvidia GTX 1080 Ti

I did not try Memtest, since I am using this computer remotely.

If you have extra hard disk try to transfer the blockchain to another disk and then reconfigure it again.

I just got this solution from some post from github here https://github.com/bitcoin/bitcoin/issues/5668#issuecomment-113848906

Other users solved their problem by removing the whole blockchain and then redownload. Just do this if you don't have an extra hard disk to test the solution above but if this doesn't work you need to extra hard disk to test.

Right now I can't move the data folder to another disk. I will be able to do this, when I will have physical access to this computer, which I don't know when I will have. Right now, the data is on the same disk with my OS, but on a different partition.

It also seems, that the command

Code:

 getrawtransaction

only works on transactions that are fairly new. The older the transaction is, the smaller probability for the command to work.

[Tycek]

It also seems, that the command

Code:

 getrawtransaction

only works on transactions that are fairly new. The older the transaction is, the smaller probability for the command to work.

Did you disable txindex? If so, it only works on unconfirmed transaction on your mempool.

I did not. Bitcoind runs with parameter -txindex=1, as a post earlier mentioned, that it might help.

[HCP]

I'm able to retrieve transaction data from block 100001



Just as easily as I am able to get it from the latest block 609375:


And I'm running on an old i5 3570K, 8 Gigs RAM with block storage on an old HDD!


If you're still getting serialization errors, then chances are good that your block storage is corrupted. I can only recommend wiping everything (excluding your wallet.dat of course) and resyncing from the beginning and seeing if you are still getting serialization issues. If you continue to get them after a full wipe (blocks and chainstate etc) and resync, then it's highly likely to be a hardware issue of some description (bad RAM?) that is causing your problems.

It also seems, that the command

Code:

 getrawtransaction

only works on transactions that are fairly new. The older the transaction is, the smaller probability for the command to work.

Probably a stupid question... but you haven't accidentally enabled "pruning" have you?

[Tycek]

It also seems, that the command

Code:

 getrawtransaction

only works on transactions that are fairly new. The older the transaction is, the smaller probability for the command to work.

Did you disable txindex? If so, it only works on unconfirmed transaction on your mempool.

I did not. Bitcoind runs with parameter -txindex=1, as a post earlier mentioned, that it might help.

With exact same error (I/O error) ?

Yes, it is the same error.

https://imgur.com/a/vIC99kT

I can get the newer transaction from block 609375, but the transaction from block 100001 does not work for me. I included my configuration file too. There should be no pruning enabled. I tried dbcache parameter, but it did not help either.

I guess, I will just wipe the whole Bitcoin folder, and save just the configuration file, since I do not have enough space in the default directory. I do not have any Bitcoins on this wallet, so I can delete the wallet.dat too.

[Tycek]

Interesting. It seems that uninstalling and reinstalling the Bitcoin Core on the same disk solved my problem. Now it does not show the errors anymore and it seems, I can fetch any block and any transaction, even the transaction from the block 100001.

Now I just wonder, where the mistake was.

[nc50lc]

Does that include deleting the whole datadir?

Might have been a corrupted block file(s) because of improper shutdown,
have you encountered any sudden power-loss or have you been shutting down your PC without properly closing bitcoind?

[BitMaxz]

Interesting. It seems that uninstalling and reinstalling the Bitcoin Core on the same disk solved my problem. Now it does not show the errors anymore and it seems, I can fetch any block and any transaction, even the transaction from the block 100001.

Now I just wonder, where the mistake was.

That's good news.
Did you reinstall the same version or you installed the latest version that fixes your issue?

It might be a missing file or data corrupted.

How about the blocks did you download or sync it again?

[Tycek]

Does that include deleting the whole datadir?

Might have been a corrupted block file(s) because of improper shutdown,
have you encountered any sudden power-loss or have you been shutting down your PC without properly closing bitcoind?

Yes. I deleted the whole datadir too. I just backed up my config file. There was not any power-loss AFAIK and I did not shutdown the computer, since I was operating it remotely. I was closing bitcoind with the key combination CTRL + C.

That's good news.
Did you reinstall the same version or you installed the latest version that fixes your issue?

It might be a missing file or data corrupted.

How about the blocks did you download or sync it again?

I checked the latest version of Bitcoin Core, and it was the same that I already downloaded. All the blocks were redownloaded again.

[HCP]

Maybe try and use "bitcoin-cli stop" instead of just CTRL-C to shutdown bitcoind in the future?

It may or may not improve the situation... but it definitely won't make it worse as it will shutdown bitcoind in the most graceful manner possible and hopefully avoid any data corruption.

[nc50lc]

Maybe try and use "bitcoin-cli stop" instead of just CTRL-C to shutdown bitcoind in the future?

I tried to check the log if there's any difference with proper shutdown and CTRL+C and I didn't noticed any major difference:
Normal Shutdown:

Code:

2020-01-05T02:04:35Z tor: Thread interrupt
2020-01-05T02:04:35Z addcon thread exit
2020-01-05T02:04:35Z dnsseed thread exit
2020-01-05T02:04:35Z torcontrol thread exit
2020-01-05T02:04:35Z Shutdown: In progress...
2020-01-05T02:04:36Z net thread exit
2020-01-05T02:04:36Z msghand thread exit
2020-01-05T02:04:40Z opencon thread exit
2020-01-05T02:04:40Z scheduler thread interrupt
2020-01-05T02:04:40Z Dumped mempool: 0s to copy, 0.036256s to dump

Using Ctrl+C:

Code:

2020-01-05T02:06:03Z tor: Thread interrupt
2020-01-05T02:06:03Z dnsseed thread exit
2020-01-05T02:06:03Z torcontrol thread exit
2020-01-05T02:06:03Z addcon thread exit
2020-01-05T02:06:03Z Shutdown: In progress...
2020-01-05T02:06:03Z net thread exit
2020-01-05T02:06:03Z msghand thread exit
2020-01-05T02:06:07Z opencon thread exit
2020-01-05T02:06:07Z scheduler thread interrupt
2020-01-05T02:06:07Z Dumped mempool: 0s to copy, 0.022492s to dump
2020-01-05T02:06:08Z [default wallet] Releasing wallet
2020-01-05T02:06:08Z Shutdown: done

@Tycek So you deleted the datadir. You can assume that the issue was really a corrupted block after deducing the other attempts that you've made but it's still a mystery how.
If I were you, I'd check for any hardware problems specially the RAM or the Mobo if the HDD/Sata is not the issue.