Bitcoin Forum

Other => Beginners & Help => Topic started by: Flue on November 13, 2013, 04:37:42 PM



Title: Recover private key/coins
Post by: Flue on November 13, 2013, 04:37:42 PM
Hello!

I managed to format the hard drive with my wallet.dat file a long time ago. I quickly realized what had happend and tried to recover the file. I had written some few files to the drive after formatting it, but the drive is 2TB so I had my hopes up that I didn't overwrite the wallet.dat itself. I managed to recover 2 different wallet.dat files with a file recovery program. The program said both files were in great shape. Programs and movies recovered at the same time worked perfectly. I never managed to get the 2 wallet.dat files working...

When I didn't get them to work I ran a linux program from this forum which scanned the drive after keys and rebuilt a "recovered-wallet.dat" file. I didn't manage to get this one working either.

Back then the 10 coin's I had weren't worth too much so I quickly gave up, but now the coin's are worth almost 4000$!

So far I've tried to use pywallet and dump my private keys.

When I use the --dumpwallet command on the first 2 files I get the error message: ERROR:root:Couldn't open wallet.dat/main. Try quitting Bitcoin and running this again.

When using the same command on the "recovered-wallet.dat" file I get the error message:

Code:
 C:\Users\name\AppData\Roaming\Bitcoin>pywallet.py --dumpwallet > TEXT.txt
WARNING:root:pycrypto or libssl not found, decryption may be slow
Traceback (most recent call last):
  File "C:\Users\name\AppData\Roaming\Bitcoin\pywallet.py", line 2108, in p
arse_wallet
    d.update(parse_BlockLocator(vds))
TypeError: 'NoneType' object is not iterable

Can anyone help? :)


Title: Re: Recover private key - 1BTC reward!
Post by: Marinowner on November 13, 2013, 04:49:14 PM
do you know your bitcoin address , if you do we  can try to get private key!


Title: Re: Recover private key - 1BTC reward!
Post by: CrypticBits on November 13, 2013, 04:52:21 PM
I've sent you a PM mate, I've had about the same problem.


Title: Re: Recover private key - 1BTC reward!
Post by: El_Happy on November 13, 2013, 04:53:46 PM
With bitcoin-qt is just need to closed, then backup the wallet.dat and change of the restored file wallet.dat and start again with the command option -rescan.

In command window type bitcoin-qt.exe -rescan and hit enter.


Title: Re: Recover private key - 1BTC reward!
Post by: pontiacg5 on November 13, 2013, 05:03:31 PM
I can't help, but please be careful which who you choose to help. You seem pretty knowledgeable, so I'm sure you know why.

Good luck getting your coins back!


Title: Re: Recover private key - 1BTC reward!
Post by: Flue on November 13, 2013, 05:09:38 PM
Thanks for answering! :)

do you know your bitcoin address , if you do we  can try to get private key!

My bitcoin adress is: 13HeWt9mSkP6S6kMUzQu9pxh5bVazdw2sm

With bitcoin-qt is just need to closed, then backup the wallet.dat and change of the restored file wallet.dat and start again with the command option -rescan.

In command window type bitcoin-qt.exe -rescan and hit enter.

Running rescan with the "recovered.wallet.dat" file atleast makes bitcoin try to load. After some seconds of loading it gives me a message in my native language that says "Error loading wallet.dat. The wallet is damaged".

If I run the same command with the other files it doesn't even begin to load, and gives the error "wallet.dat corrupt, salvage failed" (in english).

I can't help, but please be careful which who you choose to help. You seem pretty knowledgeable, so I'm sure you know why.

Good luck getting your coins back!

Thanks for the warning! I will be careful. Already got one suspect PM :P


Title: Re: Recover private key - 1BTC reward!
Post by: Wipeout2097 on November 13, 2013, 05:10:24 PM
There is someone very reputable in this forum that may be able to help you technically, and to whom you can trust. I believe that is DannyHamilton ...


Title: Re: Recover private key - 1BTC reward!
Post by: Flue on November 13, 2013, 05:40:53 PM
There is someone very reputable in this forum that may be able to help you technically, and to whom you can trust. I believe that is DannyHamilton ...

Thank you! I've sent him a PM :)


Title: Re: Recover private key - 1BTC reward!
Post by: DannyHamilton on November 13, 2013, 05:44:43 PM
There is someone very reputable in this forum that may be able to help you technically, and to whom you can trust. I believe that is DannyHamilton ...

Thanks for the vote of confidence.  Unfortunately, this sort of wallet damage really isn't something I've got the proper skills to help with. I've been meaning to dig more into the actual structure of the wallet.dat file and learn how to parse it, but I just haven't had time for that yet. You could try asking jackjack (https://bitcointalk.org/index.php?action=profile;u=21053).  He's the guy who created and maintains pywallet.

I wouldn't give up hope yet.  It might still be possible to extract private keys from a significantly damaged wallet.dat file.

It might also be possible to extract a partial/damaged private key.  If there are only a handful of bits damaged or missing, a brute-force approach to repair/replace the damaged/missing parts might also be possible.


Title: Re: Recover private key - 1BTC reward!
Post by: Flue on November 13, 2013, 05:46:58 PM
Thanks alot! I will ask jackjack :)


Title: Re: Recover private key - 1BTC reward!
Post by: Jutarul on November 13, 2013, 05:51:02 PM
First look for evidence on the hard drive.

Also read this thread. https://bitcointalk.org/index.php?topic=91702.msg1016998#msg1016998


Title: Re: Recover private key - 1BTC reward!
Post by: Flue on November 13, 2013, 06:05:15 PM
First look for evidence on the hard drive.

Also read this thread. https://bitcointalk.org/index.php?topic=91702.msg1016998#msg1016998

Pubkey is the same as the bitcoin adress (the one you send bitcoins to)?

I'm gonna try to do a scan, but I fear it's long gone. The drive has been used alot the last year (after I recovered those other files). I acutally have a private key scan with pywallet already going, though I'm pretty sure everything there is overwritten...


I suggest you remove the "1BTC reward!" from your status.

Those with the skills to actually help you will do so simply because they can, and they like to help.

As you can see from the post above, offering a reward will only attract a LOT of random people who have no idea what your problem is or how to fix it.  They'll pop in here, barely even read what you've posted and then spout off with some random piece of advice they think they've heard someone else offer in the past, hoping that the right answer will be similar enough to what they've said that they can claim they deserve a part of the reward.  Your thread will fill up with useless, senseless, unhelpful confusion and it will become difficult to tell who is offering actual advice and help, and who is likely to waste your time.

If your problem is eventually resolved successfully, you can always still offer to make a donation to the person or people that you feel helped you as a way of saying thanks.

Done :)


Title: Re: Recover private key - 1BTC reward!
Post by: DannyHamilton on November 13, 2013, 06:07:56 PM
Thanks alot! I will ask jackjack :)

If he can't help or is too busy to respond at the moment, you could also see if deepceleron has the right skills to help.  He's knowledgeable, and usually willing to help out if he can.


Title: Re: Recover private key/coins
Post by: Flue on November 13, 2013, 06:31:12 PM
Thanks alot! I will ask jackjack :)

If he can't help or is too busy to respond at the moment, you could also see if deepceleron has the right skills to help.  He's knowledgeable, and usually willing to help out if he can.

Asked both of them now. Awaiting response :)


Title: Re: Recover private key - 1BTC reward!
Post by: Jutarul on November 13, 2013, 06:31:30 PM
I managed to format the hard drive with my wallet.dat file a long time ago.

First look for evidence on the hard drive.

???
1. It needs to be confirmed whether the restored wallet.dat files actually are the ones which contain the private keys for the claimed address.
2. If 1) fails, it needs to be confirmed that the claimed address is somewhere on the hard drive.

Such a confirmation was missing from the description. There are multiple reasons why the claimed address may not be in the restored wallet files or on the hard drive at all - if so the OP may actually be trying to restore a useless wallet.dat file. The script I referenced works with hard drives and files in the same way. After there is a positive confirm, then you can start thinking about data recovery.


Title: Re: Recover private key/coins
Post by: deepceleron on November 13, 2013, 06:39:15 PM
Asked both of them now. Awaiting response :)

The second that you say the word "format" or "repartition", the only hope is to yank the drive from service and scan the whole surface for any recoverable private keys. Data likely will no longer be where it should be, recovery software will find junk in the sectors where "wallet.dat" is indicated to have been.

After IMMEDIATELY discontinuing use of any drive with possibly-recoverable bitcoins on it, you need to boot another operating system off another drive, and use software that will mount and scan the entire original drive by raw disk sector to discover any recoverable private key data that could be left on the drive.

Get a second hard drive to run off of or bootable USB with a linux distro on it, and get pywallet running on it: https://bitcointalk.org/index.php?topic=34028.0

Then use the command below, pointed to the appropriate device corresponding to the old hard drive and the correct drive size/area to scan, pywallet will go through these steps: 1. read every byte on the disk to look for keys 2. create a recovery file of private keys which enables 3. attempt recovery of password-encrypted keys using possible phrases

python pywallet.py --recover --recov_device /dev/sdb --recov_size 120GB --recov_outputdir ~/foundkeys

It takes about 10 hours to fully 0-wipe a 2TB hard drive, expect longer for key recovery just to read and process the whole drive.

Several threads are around discussing this, but there is not one that is at "master tutorial" level.


Title: Re: Recover private key/coins
Post by: Flue on November 13, 2013, 06:55:47 PM
Asked both of them now. Awaiting response :)

The second that you say the word "format" or "repartition", the only hope is to yank the drive from service and scan the whole surface for any recoverable private keys. Data likely will no longer be where it should be, recovery software will find junk in the sectors where "wallet.dat" is indicated to have been.

After IMMEDIATELY discontinuing use of any drive with possibly-recoverable bitcoins on it, you need to boot another operating system off another drive, and use software that will mount and scan the entire original drive by raw disk sector to discover any recoverable private key data that could be left on the drive.

Get a second hard drive to run off of or bootable USB with a linux distro on it, and get pywallet running on it: https://bitcointalk.org/index.php?topic=34028.0

Then use the command below, pointed to the appropriate device corresponding to the old hard drive and the correct drive size/area to scan, pywallet will go through these steps: 1. read every byte on the disk to look for keys 2. create a recovery file of private keys which enables 3. attempt recovery of password-encrypted keys using possible phrases

python pywallet.py --recover --recov_device /dev/sdb --recov_size 120GB --recov_outputdir ~/foundkeys

Several threads are around discussing this, but there is not one that is at "master tutorial" level.


Thanks alot for answering!

This is exactly what I'm already doing. It's about half way through scanning the drive. I will post results when it's done. About 1TB left.

I used this tool: https://bitcointalk.org/index.php?topic=25091.0 to recover the "recovered-wallet.dat" file right after the disk was formated. Does this mean that the file MIGHT contain the right information?


Title: Re: Recover private key - 1BTC reward!
Post by: cr1776 on November 13, 2013, 07:06:48 PM
Whatever you do, keep copies of the recovered wallet.dat files indefinitely.  Just because you can't get it now (unless you end up being able to), doesn't mean you might not be able to do so in the future.

For example, depending upon what has been recovered from each, you MIGHT be able to combine the two into a working one.  Likewise, you might be able to recover part of your private key (or even all of it) eventually.  Or, tools may improve.  Either way, if you are not able to successfully recover something now, doesn't mean you won't be able to do so in the future.

The suggestion of talk to jackjack is a good one as is being careful about who you send information to.



There is someone very reputable in this forum that may be able to help you technically, and to whom you can trust. I believe that is DannyHamilton ...

Thanks for the vote of confidence.  Unfortunately, this sort of wallet damage really isn't something I've got the proper skills to help with. I've been meaning to dig more into the actual structure of the wallet.dat file and learn how to parse it, but I just haven't had time for that yet. You could try asking jackjack (https://bitcointalk.org/index.php?action=profile;u=21053).  He's the guy who created and maintains pywallet.

I wouldn't give up hope yet.  It might still be possible to extract private keys from a significantly damaged wallet.dat file.

It might also be possible to extract a partial/damaged private key.  If there are only a handful of bits damaged or missing, a brute-force approach to repair/replace the damaged/missing parts might also be possible.


Title: Re: Recover private key/coins
Post by: Flue on November 13, 2013, 08:48:58 PM
Whatever you do, keep copies of the recovered wallet.dat files indefinitely.  Just because you can't get it now (unless you end up being able to), doesn't mean you might not be able to do so in the future.

For example, depending upon what has been recovered from each, you MIGHT be able to combine the two into a working one.  Likewise, you might be able to recover part of your private key (or even all of it) eventually.  Or, tools may improve.  Either way, if you are not able to successfully recover something now, doesn't mean you won't be able to do so in the future.

The suggestion of talk to jackjack is a good one as is being careful about who you send information to.

Thanks for the advice!

I tried running pywallet --recover on just the "recovered-wallet.dat" file. It actually finds 306 keys, but it stops at importing 20/306.

Code:
Importing key   20/306:
Traceback (most recent call last):
  File "C:\Users\Name\AppData\Roaming\Bitcoin\pywallet.py", line 4910, in <
module>
    importprivkey(db, sec, "recovered: %s"%sec, None, True)
  File "C:\Users\Name\AppData\Roaming\Bitcoin\pywallet.py", line 2622, in i
mportprivkey
    pkey = EC_KEY(str_to_long(sec.decode('hex')))
  File "C:\Users\Name\AppData\Roaming\Bitcoin\pywallet.py", line 1056, in _
_init__
    self.pubkey = Public_key( generator, generator * secret )
  File "C:\Users\Name\AppData\Roaming\Bitcoin\pywallet.py", line 1000, in _
_init__
    raise RuntimeError, "Generator point has x or y out of range."
RuntimeError: Generator point has x or y out of range.

The dat file with only 19 keys actually works loading with the bitcoin client. Is there anyway to check if it contains any money without having to sync the bitcoin client?


Title: Re: Recover private key/coins
Post by: DeathAndTaxes on November 13, 2013, 08:50:47 PM
The dat file with only 19 keys actually works loading with the bitcoin client. Is there anyway to check if it contains any money without having to sync the bitcoin client?

You could export those private keys.  I can't remember if pywallet will export the address along with private key but if it doesn't you could use something like bitcoinaddress (offline) to generate the address from those private keys.  Then just check the balance of the 19 addresses on a site like blockchain.info.


Title: Re: Recover private key/coins
Post by: Flue on November 13, 2013, 09:35:28 PM
It doesn't seem like any of those 19 keys contains any money. Is there a way to recover the other 287 keys?


Title: Re: Recover private key/coins
Post by: Flue on November 14, 2013, 10:18:22 AM
The recover scan with pywallet on the 2TB drive found 4 possible wallets, 900 encrypted keys (newer wallets) and 1800 possible unencryptet keys. For some reason it only tried to recover the exact same 306 keys like the other scan, from the "recovered-wallet.dat" file, and it stopped at 20/306 this time too. Why didn't it try to recover the other keys? And why does it stop at 20?


Title: Re: Recover private key/coins
Post by: deepceleron on November 14, 2013, 03:45:11 PM

Thanks for the advice!

I tried running pywallet --recover on just the "recovered-wallet.dat" file. It actually finds 306 keys, but it stops at importing 20/306.

Quote
Importing key   20/306:
Traceback (most recent call last):
  File "C:\Users\Name\AppData\Roaming\Bitcoin\pywallet.py", line 4910, in <
module>
    importprivkey(db, sec, "recovered: %s"%sec, None, True)
  File "C:\Users\Name\AppData\Roaming\Bitcoin\pywallet.py", line 2622, in i
mportprivkey
    pkey = EC_KEY(str_to_long(sec.decode('hex')))
  File "C:\Users\Name\AppData\Roaming\Bitcoin\pywallet.py", line 1056, in _
_init__
    self.pubkey = Public_key( generator, generator * secret )
  File "C:\Users\Name\AppData\Roaming\Bitcoin\pywallet.py", line 1000, in _
_init__
    raise RuntimeError, "Generator point has x or y out of range."
RuntimeError: Generator point has x or y out of range.

The dat file with only 19 keys actually works loading with the bitcoin client. Is there anyway to check if it contains any money without having to sync the bitcoin client?

It doesn't just "stop" importing, it has a catastrophic failure with the value of one key being invalid as a elliptical curve private key. Likely there was a KeyA or other data prefix surrounding an area of a hard drive that made it look like a potential key, but the data was all 00s or all FFs, which would cause this error. Either the key will need to be edited out of the recovered-wallet, by hex editing the wallet to make that particular data a real (but no money) key, or fixing pywallet so it evaluates and skips invalid private keys.


Title: Re: Recover private key/coins
Post by: deepceleron on November 14, 2013, 04:09:11 PM
I'm fixing pywallet, but it is indented with tabs,  >:(. http://www.python.org/dev/peps/pep-0008/#tabs-or-spaces


Title: Re: Recover private key/coins
Post by: Flue on November 14, 2013, 05:05:40 PM
Thanks alot! I really appreciate it!

I get a syntax error at line 4261, "table in =", an arrow is pointing at the "=". Typo? :P


Title: Re: Recover private key/coins
Post by: deepceleron on November 14, 2013, 05:13:29 PM
Thanks alot! I really appreciate it!

I get a syntax error at line 4261, "table in =", an arrow is pointing at the "=". Typo? :P

Ooops, looks like I randomly hit "enter" in the middle of the file, fixed and reupped. I also ran it thru a code validator which removed some imports and reformatted the python.


Title: Re: Recover private key/coins
Post by: Flue on November 14, 2013, 05:24:00 PM
Now I get this error:

Traceback (most recent call last):
  File "C:\Users\Name\AppData\Roaming\Bitcoin\pywallet.py", line 27, in <mo
dule>
    pyw_filename = os.path.basename(__file__)
NameError: name 'os' is not defined


Title: Re: Recover private key/coins
Post by: deepceleron on November 14, 2013, 05:28:50 PM
Now I get this error:

Traceback (most recent call last):
  File "C:\Users\Name\AppData\Roaming\Bitcoin\pywallet.py", line 27, in <mo
dule>
    pyw_filename = os.path.basename(__file__)
NameError: name 'os' is not defined
The code (in)validator appears to have removed the os,sys import. Grr, I guess I'll just text-edit the python to patch it, and keep the tabs instead of using my IDE which bitches about the tabs and won't even let me type them. Sorry about all that...


Title: Re: Recover private key/coins
Post by: deepceleron on November 14, 2013, 05:48:28 PM
Ok, reupped, after actually running the code and recovering some real keys off my own hard drive:

Code:
C:\pywallet>diff.exe pywallet.py pywallet-keyskip.py
4910,4911c4910,4919
<                       importprivkey(db, sec, "recovered: %s"%sec, None, True)
<                       importprivkey(db, sec+'01', "recovered: %s"%sec, None, True)
---
>                       try:
>                               importprivkey(db, sec, "recovered: %s"%sec, None, True)
>                       except RuntimeError, e:
>                               print "%s error importing key %s" % (e, sec)
>                               continue
>                       try:
>                               importprivkey(db, sec+'01', "recovered: %s"%sec, None, True)
>                       except RuntimeError, e:
>                               print "%s error importing key %s" % (e, sec)
>                               continue


Title: Re: Recover private key/coins
Post by: Flue on November 14, 2013, 05:51:42 PM
Sure you reuploaded the right code? I get the same error with that one :P


Title: Re: Recover private key/coins
Post by: deepceleron on November 14, 2013, 06:00:00 PM
Sure you reuploaded the right code? I get the same error with that one :P
Yup, I renamed it just in case your browser is caching the old version or something:
http://we.lovebitco.in/pywallet-keyskip2.py

I modified this version of pywallet:
https://github.com/jackjack-jj/pywallet/blob/master/pywallet.py


Title: Re: Recover private key/coins
Post by: Flue on November 14, 2013, 06:06:50 PM
Now I've ran into a new problem which happens with both the new and the old version of pywallet. Sometimes it scannes the "recovered-wallet.dat" file, and actually tries to recover the keys to a new files, but most of the times it just skips everything without an error message:

Code:
Read 2.1 Go in 0.6 minutes

Found 0 possible wallets
Found 0 possible encrypted keys
Found 490 possible unencrypted keys


All the found encrypted private keys have been decrypted.
The wallet is encrypted and the passphrase is correct


Importing:


The new wallet H:\recover/recovered_wallet_1384452226.dat contains the 0 recove
red key

C:\Users\Name\AppData\Roaming\Bitcoin>

Any idea? Am I doing something wrong?


Title: Re: Recover private key/coins
Post by: deepceleron on November 14, 2013, 06:16:11 PM
Any idea? Am I doing something wrong?
I have no explanation; maybe move your original found-key file to a simple directory name such as C:\WALLET along with the script, specify the max size, etc. Here is the command line I just used with pywallet in the same directory as the keyfile:

pywallet-keyskip2.py --recover --recov_device recovered-wallet.dat --recov_size 1Mio --recov_outputdir .


Title: Re: Recover private key/coins
Post by: Flue on November 14, 2013, 06:29:49 PM
Yay! Managed to import all the keys now. Had to first do a normal --dumpwallet on the file, then run the --recover command. It probably reads from the text file instead of the wallet.

"The new wallet ./recovered_wallet_1384453446.dat contains the 306 recovered keys"

Which way is the fastest to check the balance?

Do I have to resync the bitcoin program to check balance, or just use the new wallet?


Title: Re: Recover private key/coins
Post by: DannyHamilton on November 14, 2013, 06:37:50 PM
Yay! Managed to import all the keys now. Had to first do a normal --dumpwallet on the file, then run the --recover command. It probably reads from the text file instead of the wallet.

"The new wallet ./recovered_wallet_1384453446.dat contains the 306 recovered keys"

Which way is the fastest to check the balance?

Do I have to resync the bitcoin program to check balance, or just use the new wallet?

Your wallet will need to do a rescan, but not a resync.  As long as your blockchain is already caught up, you won't have to download/synchronize it again.  You'll just have to wait for the client to scan through the blockchain that you already have so it can find any unspent outputs associated with those private keys.


Title: Re: Recover private key/coins
Post by: Flue on November 14, 2013, 06:43:36 PM
Bitcoin-QT freezes when I try to run it with the -rescan command...


Title: Re: Recover private key/coins
Post by: Flue on November 14, 2013, 06:55:55 PM
It worked now. Some old transactions came up, but they are from a newer wallet. Looks like I have to run another 10 hour scan on my 2TB drive. Thanks for the help so far!


Title: Re: Recover private key/coins
Post by: deepceleron on November 14, 2013, 06:57:51 PM
Bitcoin-QT freezes when I try to run it with the -rescan command...
It takes a long time to run, it can look like it locked up since there is no progress bar or such.
You can also start Bitcoin-Qt with the -salvagewallet option just for another round of validation.


Title: Re: Recover private key/coins
Post by: Flue on November 15, 2013, 10:48:20 AM
After a full scan it found 4 wallets, 900 encryptet keys and 1800 unencrypted keys. It only imported those 306 keys from last time. 0 coins. Guess my coins are lost forever. :(

I made 4 or 5 backups back then. But I forgot about bitcoin for a while because of the low value, and one disk after another failed. I'm even pretty sure I made an online backup, but I can't find it anywhere...

Thanks for all the help though!


Title: Re: Recover private key/coins
Post by: tom_o on November 15, 2013, 07:32:39 PM
After a full scan it found 4 wallets, 900 encryptet keys and 1800 unencrypted keys. It only imported those 306 keys from last time. 0 coins. Guess my coins are lost forever. :(

I made 4 or 5 backups back then. But I forgot about bitcoin for a while because of the low value, and one disk after another failed. I'm even pretty sure I made an online backup, but I can't find it anywhere...

Thanks for all the help though!

Contact http://www.datagenetics.com/ and offer them a portion of the hoard if they manage to open it?

Since 'change' privkeys are generated based on a sequential seed iirc, you might be able to get into others other using those and a smaller amount of bruteforcing, if not they may be able to take the wallet apart and avoid the corrupted areas crashing bitcoin-qt. Worth an email at least eh?

I sent them a random one just to tell them about bitcoin the other day, they might be interested.


Check out their blog for some of their pretty amazing work, and some comedy including this.

http://www.datagenetics.com/blog/november12013/test.png


Title: Re: Recover private key/coins
Post by: DannyHamilton on November 15, 2013, 07:35:27 PM
Since 'change' privkeys are generated based on a sequential seed iirc,

No.  They aren't.

In Bitcoin-Qt, all addresses are generated randomly.


Title: Re: Recover private key/coins
Post by: jackjack on November 29, 2013, 09:18:54 PM
Sorry to come so late
Any updates on this?
Did you contact HDD experts?