bitcoin-qt: "ghost" addresses moving from v0.4.0-beta to v0.8.1-beta

[ner0]

The title itself is not self-explanatory so I'll have to get into the details of this weird thing. I'm not sure this is the right place but I'm restricted on other forums... Before that, just to sum-up what happened:
I used bitcoin-qt v0.8.1-beta with my wallet which was last used on v0.4.0-beta.
The first time I created new addresses bitcoin-qt lost them after closing it and 'ismine' function returned false on all of them.
I lost 2.00BTC because of this, mainly two reasons are involved, me being sloppy and me being victim of a random software bug.
In my attempt to avoid this happening to someone else (or to be the harbinger of bad news), I'll explain bellow how this happened:

1. Restored bitcoin-qt wallet backup from v0.4.0-beta;
2. Opened 'bitcoin-0.8.1-win32';
3. bitcoin-qt throws error message: "Wallet needed to be rewritten: restart Bitcoin to complete";
4. Opened bitcoin-qt again and created new address succesfully;
5. Closed bitcoin-qt (File > Exit);
6. Opened bitcoin-qt again and the previously created address is now gone;
7. Created new address again but the address was different from the one missing;
8. Closed bitcoin-qt (File > Exit);
9. Opened bitcoin-qt again and the previously created address is still there;


I've repeated the above process 10 times and in all of them the same thing happens. Also, when running 'validateaddress <address>' in the debug console for those ghost addresses 'ismine' returned false for all of them.
Any addresses created or passphrase change during the 1st run (2nd run if counting the wallet rewriting process) are not saved in my wallet.
This happens without any error or warning whatsoever, and also no significant or related entry on the debug log (http://pastebin.com/LFA1KJja).
Only after the 2nd run was bitcoin-qt able to retain the changes.


I'm not sure if this is a common error that might happen to any wallet in those conditions or if it is an isolated case. The fact that I didn't find any identical story tells me this is rather uncommon.
I'm convinced the 2.00BTC I sent to one of those "ghost" addresses are lost forever so I'm not hopeful of getting them back, but it would be nice if someone could track down the source of this problem and solve it.

Thanks

[old c coder]

Hello

Those 2 BTCs may not be lost. Do you still have the wallet.dat file that is the address of those 2 BTCs?

If you still have that old 0.4.0 wallet, you might try running the old 0.4.0 software to bring it back online, send the coins to a new address, and make a backup wallet.

Also try downloading the non-beta bitcoin 0.8.1 and using that. Here for bitcoind & bitcoin-qt
for windows:
http://sourceforge.net/projects/bitcoin/files/latest/download?source=files
or the latest version link at
http://sourceforge.net/projects/bitcoin/files/Bitcoin/bitcoin-0.7.0/
they go to the same place.

And 0.4.1 at
http://sourceforge.net/projects/bitcoin/files/Bitcoin/bitcoin-0.4.1/

Ron

[ner0]

Thanks for your reply, Ron.
Well, for one thing I can't find bitcoin-qt v0.4.0-beta, only v0.4.1 and higher. Is there somewhere where I could find it?

Also the address to where I sent the 2BTC doesn't exist in my wallet.
I had been using bitcoin-qt for a whole afternoon after rewriting my wallet and it discarded that address as soon as I closed bitcoin-qt. I haven't been able to generate the same address ever since.
A developer told me that the bitcoin-qt pre-loads 100 wallet addresses and if the address validation command doesn't indicate that the address is mine, then I could forget about ever getting back that money.
The thing is that I replicated that "bug" and created around 10 "ghost" addresses on several attempts, none of which would return "ismine" as true when validating the "ghost" addresses.

Anyway, I did try what you suggested but nothing worked. v0.4.1 doesn't show the 2BTC on any transaction, although it correctly grabbed other transactions before and after.
In the same way, the latest version (v0.8.1) didn't either, it also lists transactions before and after but not the one I am missing.
Just as a side note, the bug that made me lose those 2BTC is still present in the latest v0.8.1 - I replicated it again.
In fact, probably anybody can replicate this if they take and old wallet.dat and use it on a newer version, the first time bitcoin-qt runs properly it will not save any changes to the wallet whatsoever, whether it is a pass-phrase change or any newly generated addresses. Even if this bug only affects you under specific circumstances like using an old wallet format that has to be rewritten, it still is a bug that has not been addressed.

Sadly I felt forced to stop using bitcoin-qt as a result. Although it is a very well crafted tool, it surely is not good enough since it can give you random wallet addresses that do not belong to you and clears them after closing the software. It is also my understanding that "validateaddress" is unequivocal, this is why I lost hope in getting that money back altogether. Would it be possible that it could tell me 'ismine' is 'false' for that address and yet still appear in my wallet? Isn't the passphrase the only crucial element for recognizing an address as own?

I appreciate your tips and if you think there is anything else I could try or any other approach, I'm all ears.
Despite being convinced that I will not be able to get that transaction back, I will do anything possible to try to.

[old c coder]

Hello ner0

Did you see the text below the directory listing on
http://sourceforge.net/projects/bitcoin/files/Bitcoin/bitcoin-0.4.1/
where there is a discussion of what sounds like your issue. And that 0.4.1 addresses it? The section titled MAJOR BUG FIX (CVE-2011-4447)

wallet encryption...

Ron

[ner0]

Hello ner0

Did you see the text below the directory listing on
http://sourceforge.net/projects/bitcoin/files/Bitcoin/bitcoin-0.4.1/
where there is a discussion of what sounds like your issue. And that 0.4.1 addresses it? The section titled MAJOR BUG FIX (CVE-2011-4447)

wallet encryption...

Ron

Hello,

I am not sure this is the same issue.
At least it only seems to mention the security issue related to the insufficient encryption quality.
The issue I explained was triggered in step 3 (listed bellow), and there is no mention to what might have happened in my case.

Thanks anyway.

1. Run Bitcoin and let it rewrite the wallet.dat file

2. Run it again, then ask it for a new bitcoin address.
 Bitcoin-Qt: Address Book, then New Address...
 bitcoind: run the 'walletpassphrase' RPC command to unlock the wallet,
  then run the 'getnewaddress' RPC command.

3. If your encrypted wallet.dat may have been copied or stolen, send
 all of your bitcoins to the new bitcoin address.

4. Shut down Bitcoin, then backup the wallet.dat file.
 IMPORTANT: be sure to request a new bitcoin address before
 backing up, so that the 'keypool' is regenerated and backed up.

[old c coder]

The you'll just have to roll up your sleeves and look at that bitcoin-qt code your self! Even on windows, we can now compile on windows for windows! See
https://bitcointalk.org/index.php?topic=149479.msg2177060#msg2177060

It may not be that hard to find out what is going on!

Ron