Bitcoin-QT Core Problem - Keypool 100 transactions limi - Reward for help: 2 BTC

[shorena]

More of a question here for learning purposes

True, this is what a send to two addresses unknown to the wallet (no private key) would look like. Considering the time the transaction took place its likely that the change key was one 100+ ofter the january backup.

I am not getting how the return could go to an address about which your wallet does not know. I only see 2 options:

A: If the client knows that all keys in the old keypool have been used up (sync is recent enough), it will simply re-generate new private-public keys. This happens if you unlock the wallet to make a transaction. Hence return should go to a newly (at the time of transaction) generated address.

The client keeps track of the number of unused keys in the keypool, yes. This has little to do with syncing the blockchain, its entirely local. It requirest blockchain data to determine "used" though.

B: If the client does not know that all keys have been used up (sync not completed up to the point where the last address has been used), then it will send the return coins to an old address in the old keypool (say 99/100), which has already been used before (!), but for which you also ought to have the private key in the old wallet.dat. In this case the return address would have 2 transactions in it.

We are talking about a restoration from a backup. More below.

Please correct me if this is wrong, after all, I am here to learn.

Based on this and the transaction ID, the 13 BTC change was the only input to the return address, so I would say it is a new address generated when you sent the transaction. (Had 13 BTC been sent to a return address that was used sometime after your January backup (I do not see how that would be possible), that address would likely have another transaction in it as well. Being also old, it is quite probably that it was already used.)

BitcoinNewsBR
1. The transaction in the UI does look like if it was sent to an unknown address(mentioned by shorena), as generally return addresses are not shown in the UI. Are you perhaps using an extension/script to show these as well?
2. I think a "Private key for address X is unknown" also appears when you try to dumpprivkey with the wallet locked. I assume this is not the case.

No

Code:

dumpprivkey 1HxeCXGT11wVwy78gsVrzFfkfdijQE9rct 
Error: Please enter the wallet passphrase with walletpassphrase first. (code -13)

3. Have you tried to sign a message with the return address? I saw this recommended somewhere as to double-check if you don't have the private key for an address.

It should not make a difference.

4. If you haven't already, you could look at the commands listaddressgroupings and listreceivedbyaddress, they might tell you some additional info.

-


What I think happend is the following.

OP created a backup in January and kept using the wallet. The wallet kept refilling the keypool whenever needed, but these are not part of the January backup. Sometime in September the 101th key since the backup in Jan was used as a change address. When the wallet was restored from backup. It recognized the transaction because it had one of the keys that controlled the input that was used. Since it did not recognize any of the output addresses it marked them as external and thus is showing them in the overview, whereas it would not show a change output.

[Borisz]

First of all, Thank you!

What I think happend is the following.

OP created a backup in January and kept using the wallet. The wallet kept refilling the keypool whenever needed, but these are not part of the January backup. Sometime in September the 101th key since the backup in Jan was used as a change address. When the wallet was restored from backup. It recognized the transaction because it had one of the keys that controlled the input that was used. Since it did not recognize any of the output addresses it marked them as external and thus is showing them in the overview, whereas it would not show a change output.

I understand, you mean the transaction was made before the computer crash (that had a return address unknown to the January backup) and the restored backup doesn't recognize the address, hence marks it as outgoing.
(Any new keys generated in the keypool after the January backup are obviously lost.)

I interpreted the post such that the transaction was made with the wallet restored  from the January backup . In this case (if the wallet has caught up with the ledger far enough to determine that all the old 100 keys have been used) the wallet would generate new addresses, right?

[BitcoinNewsBR]

Hello, guys.

I HAVE GOOD NEWS! Not so much, but it is good news.

Someone saw this thread and discussed privately with me, sending all the necessary comments for I use on the Pywallet. Thanks you so much!

I'm on OSX, we used this command on Pywallet:

Code:

sudo /Users/XXX/Downloads/pywallet-master/pywallet.py --recover --recov_device /dev/rdisk0 --recov_size 1024Gio --recov_outputdir /Users/XXXX/Documents/Lucky

After I run this command, it shows this:

Found 12 possible wallets
Found 5862 possible encrypted keys
Found 0 possible unencrypted keys


And generate a .dat file on the folder /Users/XXXX/Documents/Lucky

So now, I need someone help:

1) I tried to open on Bitcoin-QT this .dat but didn't worked. How can I use this .dat file to see the private keys there?
2) How may I use this command again and generate a document showing the private key?
3) Usually, the keys would appear on the screen for Windows Users. For Mac, we will need to add something to the command for the output generate a txt.

Thanks for your attention!

[kolloh]

Hello, guys.

I HAVE GOOD NEWS! Not so much, but it is good news.

Someone saw this thread and discussed privately with me, sending all the necessary comments for I use on the Pywallet. Thanks you so much!

I'm on OSX, we used this command on Pywallet:

Code:

sudo /Users/XXX/Downloads/pywallet-master/pywallet.py --recover --recov_device /dev/rdisk0 --recov_size 1024Gio --recov_outputdir /Users/XXXX/Documents/Lucky

After I run this command, it shows this:

Found 12 possible wallets
Found 5862 possible encrypted keys
Found 0 possible unencrypted keys


And generate a .dat file on the folder /Users/XXXX/Documents/Lucky

So now, I need someone help:

1) I tried to open on Bitcoin-QT this .dat but didn't worked. How can I use this .dat file to see the private keys there?
2) How may I use this command again and generate a document showing the private key?
3) Usually, the keys would appear on the screen for Windows Users. For Mac, we will need to add something to the command for the output generate a txt.

Thanks for your attention!

Assuming the .dat file isn't corrupted. You should be able to simpy open the .dat file in a texteditor or use cat to view the contents of the file. The private keys will be in this file listed under xprv . If the wallet is encrypted, you may need to unencrypt it first.

If the file looks to be corrupt, I would try the following.  Run bitcoin application with the -salvagewallet parameter via command line. This command attempts to salvage public/private keys and master encryption keys (if the wallet is encrypted) into a new wallet.dat .

[achow101]

Assuming the .dat file isn't corrupted. You should be able to simpy open the .dat file in a texteditor or use cat to view the contents of the file. The private keys will be in this file listed under xprv . If the wallet is encrypted, you may need to unencrypt it first.

This is incorrect. The wallet.dat file is not a text file, it is a binary file in the Berkeley DB format.

If the file looks to be corrupt, I would try the following.  Run bitcoin application with the -salvagewallet parameter via command line. This command attempts to salvage public/private keys and master encryption keys (if the wallet is encrypted) into a new wallet.dat .

OP, try this. Put the wallet.dat file in your data directory. Start Bitcoin Core with the -salvagewallet option. If nothing is corrupted, it will work fine. If something is corrupted, it will attempt to fix it.

[BitcoinNewsBR]

Assuming the .dat file isn't corrupted. You should be able to simpy open the .dat file in a texteditor or use cat to view the contents of the file. The private keys will be in this file listed under xprv . If the wallet is encrypted, you may need to unencrypt it first.

This is incorrect. The wallet.dat file is not a text file, it is a binary file in the Berkeley DB format.

If the file looks to be corrupt, I would try the following.  Run bitcoin application with the -salvagewallet parameter via command line. This command attempts to salvage public/private keys and master encryption keys (if the wallet is encrypted) into a new wallet.dat .

OP, try this. Put the wallet.dat file in your data directory. Start Bitcoin Core with the -salvagewallet option. If nothing is corrupted, it will work fine. If something is corrupted, it will attempt to fix it.

When I try to open the wallet using the recovered .dat as wallet.dat it shows "Wallet corrupted. Recovered failed". I don't have even the option to open the console on the Bitcoin-QT.

So I tried to open using the terminal:

I entered this:

Code:

/Applications/Bitcoin-Qt.app/Contents/MacOS/Bitcoin-Qt -salvagewallet

It open the wallet and close in few seconds. Is this the command I need to enter to use this -salvagewallet option?

[achow101]

When I try to open the wallet using the recovered .dat as wallet.dat it shows "Wallet corrupted. Recovered failed". I don't have even the option to open the console on the Bitcoin-QT.

So I tried to open using the terminal:

I entered this:

Code:

/Applications/Bitcoin-Qt.app/Contents/MacOS/Bitcoin-Qt -salvagewallet

It open the wallet and close in few seconds. Is this the command I need to enter to use this -salvagewallet option?

It is something like that. The -salvagewallet option is a startup option, not a debug console command. You are supposed to do something like that but I don't have a Mac so I don't know the exact way you are supposed to do it.

[kolloh]

This is incorrect. The wallet.dat file is not a text file, it is a binary file in the Berkeley DB format.

Whoops. I was looking at an Electrum wallet.dat file which likely uses a different format.

So I tried to open using the terminal:

I entered this:

Code:

/Applications/Bitcoin-Qt.app/Contents/MacOS/Bitcoin-Qt -salvagewallet

It open the wallet and close in few seconds. Is this the command I need to enter to use this -salvagewallet option?

That should be the correct method. Try checking the debug.log file which should generate some output on the results of the salvage wallet command.

[jackjack]

None of these is the correct method
The thing is that pywallet is kinda old so it creates a wallet with an old format

Please post the full log (not only the "Found X possible Y" lines) of the pywallet recovery command, I must know whether the encrypted keys were decrypted or not (with the passphrases you must have provided)

If they were not decrypted, it means that you didn't provide the correct passphrases when pywallet asked you so (ie you must run the pywallet recovery command again and provide the passphrases of your old wallets when asked)
If they were decrypted, then the recovered wallet contains all those keys and you'll be able to extract the private keys using pywallet with dumpwallet on the recovered wallet
But first things first: please provide the full log (check beforehand that there is no private data in it)

[BitcoinNewsBR]

None of these is the correct method
The thing is that pywallet is kinda old so it creates a wallet with an old format

Please post the full log (not only the "Found X possible Y" lines) of the pywallet recovery command, I must know whether the encrypted keys were decrypted or not (with the passphrases you must have provided)

If they were not decrypted, it means that you didn't provide the correct passphrases when pywallet asked you so (ie you must run the pywallet recovery command again and provide the passphrases of your old wallets when asked)
If they were decrypted, then the recovered wallet contains all those keys and you'll be able to extract the private keys using pywallet with dumpwallet on the recovered wallet
But first things first: please provide the full log (check beforehand that there is no private data in it)

Thanks for your answer jackjack. Here is the log:

I open the terminal on OSX then enter this code:

Code:

sudo /Users/XXXX/Downloads/pywallet-master/pywallet.py --recover --recov_device /dev/rdisk0 --recov_size 1024Gio --recov_outputdir /Users/XXXX/Documents/sorte

Return:

Code:

iMac-de-XXX:~ XXX$ sudo /Users/XXXX/Downloads/pywallet-master/pywallet.py --recover --recov_device /dev/rdisk0 --recov_size 1024Gio --recov_outputdir /Users/XXXX/Documents/sorte
Password:  (My System password entered)

Enter the passphrase for the wallet that will contain all the recovered keys: XXXXXX  (No doubt this is the right passphrase)

Enter the possible passphrases used in your deleted wallets.
Don't forget that more passphrases = more time to test the possibilities.
Write one passphrase per line and end with an empty line.
Possible passphrase: XXXXXX (No doubt this is the right passphrase)
Possible passphrase:    (Enter)

Starting recovery.
0.10 Go read
0.20 Go read
0.30 Go read
0.40 Go read
0.50 Go read
0.60 Go read
0.70 Go read
0.80 Go read
0.90 Go read
1.00 Go read
1.10 Go read
1.20 Go read
1.30 Go read
1.40 Go read
1.50 Go read
1.60 Go read
1.70 Go read
1.80 Go read
1.90 Go read
2.00 Go read
2.10 Go read
2.20 Go read
2.30 Go read
2.40 Go read
2.50 Go read
2.60 Go read
2.70 Go read
2.80 Go read
2.90 Go read
3.00 Go read
3.10 Go read
3.20 Go read
3.30 Go read
.
.
.  (After 5 hours - 1Tera HD)

Read 1000.2 Go in 289.0 minutes

Found 12 possible wallets
Found 5862 possible encrypted keys
Found 0 possible unencrypted keys

This is all what happened when I use this command. And then generate the .dat on the folder I set.

Any idea how I could solve this Jackjack? Thanks!

[jackjack]

Don't you have anything more?
There should be an "Importing" section at the end, after the "found X Y" lines

[BitcoinNewsBR]

Don't you have anything more?
There should be an "Importing" section at the end, after the "found X Y" lines

Nop. Don't have these lines. A User who was helping me via private message also asked about the importing key screen:

Code:

Importing key    2/1065:

But it don't show after I finish the scan. After finished the scan, that was the message I saw (previous post)

Maybe because I'm on OSX?

Thanks for your answer.

[jackjack]

Don't you have anything more?
There should be an "Importing" section at the end, after the "found X Y" lines

Nop. Don't have these lines. A User who was helping me via private message also asked about the importing key screen:

Code:

Importing key    2/1065:

But it don't show after I finish the scan. After finished the scan, that was the message I saw (previous post)

Maybe because I'm on OSX?

Thanks for your answer.

Indeed OSX may be the reason
I'll make you a custom pywallet to take care of this

[BitcoinNewsBR]

Don't you have anything more?
There should be an "Importing" section at the end, after the "found X Y" lines

Nop. Don't have these lines. A User who was helping me via private message also asked about the importing key screen:

Code:

Importing key    2/1065:

But it don't show after I finish the scan. After finished the scan, that was the message I saw (previous post)

Maybe because I'm on OSX?

Thanks for your answer.

Indeed OSX may be the reason
I'll make you a custom pywallet to take care of this

Jackjack, thanks so much! You developed a fantastic app that helps the community and keep giving support to it! I really respect this attitude.

I'm confidence that I will recover the coins and make a good donation to support the Pywallet project and also tip the ones that helped me before!

I'm posting here the wallet, so everyone know if I move the coins: https://blockchain.info/address/1HxeCXGT11wVwy78gsVrzFfkfdijQE9rct

Thanks!

[jackjack]

Thank you for the kind words
I hope the missing pool keys are in these 5 thousands keys

The version you can try is here: https://framabin.org/?6501b12ab63d82e3#yN5vWu7fGCvx8RwGN6UpDSHpWXewPl/5NCHVyGBBfKQ=
It's basically the same with less "X Go read" lines (1 per GB, so around 3 per minute) and more verbose after the "Found X Y" lines to understand what's happening
You can check the diff (with http://diffchecker.com for exemple) between public pywallet and this one if you want

[BitcoinNewsBR]

Thank you for the kind words
I hope the missing pool keys are in these 5 thousands keys

The version you can try is here: https://framabin.org/?6501b12ab63d82e3#yN5vWu7fGCvx8RwGN6UpDSHpWXewPl/5NCHVyGBBfKQ=
It's basically the same with less "X Go read" lines (1 per GB, so around 3 per minute) and more verbose after the "Found X Y" lines to understand what's happening
You can check the diff (with http://diffchecker.com for exemple) between public pywallet and this one if you want

Hey, Jackjack.

Sorry my basically knowledge about it. But how I should proceed with it? How I execute this code? May you, please, guide me on it? Thanks

[jackjack]

Yes sorry, sure
You have to put this code in a new python file(e.g. pywalletz.py)
Then run the same recovery command, but with pywalletZ instead of pywallet

[BitcoinNewsBR]

I opened the pywalletz.py in the Xcode, deleted the code, copy and paste the new code.

Tried to run the same command again (as it is the new pywallet code on the same file now). It returns this error:

Code:

iMac-de-XX:~ XX$ sudo /Users/XXX/Downloads/pywallet-master/pywallet.py --recover --recov_device /dev/rdisk0 --recov_size 1024Gio --recov_outputdir /Users/XX/Documents/sorte
  File "/Users/XXX/Downloads/pywallet-master/pywallet.py", line 22
    except:
         ^
SyntaxError: invalid syntax

Any idea what I'm doing wrong? If you can upload the file in somewhere it is also fine to me.

Thanks, jackjack.

[jackjack]

Ok then download this: https://gist.githubusercontent.com/jackjack-jj/d3a9b74cef28d2f79ab113d8f3b96a82/raw/df64b98d2deadb577f7655d225abb8013329dd56/pywallet_BitcoinNewsBR_1.py
And run the recovery command using pywallet_BitcoinNewsBR_1.py

[BitcoinNewsBR]

Ok then download this: https://gist.githubusercontent.com/jackjack-jj/d3a9b74cef28d2f79ab113d8f3b96a82/raw/df64b98d2deadb577f7655d225abb8013329dd56/pywallet_BitcoinNewsBR_1.py
And run the recovery command using pywallet_BitcoinNewsBR_1.py

Hey, jackjack. Thanks again for the time you spent working on the code. Here is what I got:

Code:

$ sudo -i /Users/XXX/Downloads/pywallet_BitcoinNewsBR_1.py --recover --recov_device /dev/rdisk0 --recov_size 1024Gio --recov_outputdir /Users/XXX/Documents/sorte
Enter the passphrase for the wallet that will contain all the recovered keys: XXX

Enter the possible passphrases used in your deleted wallets.
Don't forget that more passphrases = more time to test the possibilities.
Write one passphrase per line and end with an empty line.
Possible passphrase: XXX
Possible passphrase: 

Starting recovery.
1.00 Go read
.
.
998.00 Go read
999.00 Go read
1000.00 Go read

Read 1000.2 Go in 317.6 minutes
Found 12 possible wallets
Found 5862 possible encrypted keys
Found 0 possible unencrypted keys
a
b
c0
Traceback (most recent call last):
  File "/Users/XXX/Downloads/pywallet_BitcoinNewsBR_1.py", line 4880, in <module>
    recoveredKeys=recov(device, passes, size, 10240, options.recov_outputdir)
  File "/Users/XXX/Downloads/pywallet_BitcoinNewsBR_1.py", line 1532, in recov
    print 'c0',tsincetz()
  File "/Users/XXX/Downloads/pywallet_BitcoinNewsBR_1.py", line 1531, in tsincetz
    return time.time()-tz
NameError: global name 'tz' is not defined

Any idea how to proceed?

Thanks for your attention!