Pywallet 2.2: manage your wallet [Update required]

[jackjack]

Christmas update

New feature: Pywallet checks if it is up-to-date and if not, shows a link to update (automatically)
Encryption still in progress, cryptographic functions in python are such a pain...

[chunglam]

Hi jackjack, I have a corrupted wallet.dat with about 100 BTCs inside, it was encrypted, could this tool help me recover it? I offer 40 BTCs for anyone help me recover BTCs from this wallet. Details in the this topic:
https://bitcointalk.org/index.php?topic=55975.0

[etotheipi]

So I think I will help everyone with this wallet-extraction thing.  I was the one that found unencrypted keys in the 0.4.0 encrypted wallet, so I'm pretty sure I can find the encrypted keys  And Armory uses a very similar technique for encryption, so I'm basically already set up for this.  A few things I hope someone will be able to answer for me.  

(1)  I need to find nDeriveIterations.  Is there an easy way to find it for a hex-diver like me, or do I actually have to open the DB file with a BDB library to get to it?
(2)  I wish I'd thought of using the public key as the IV... that's pretty smart.  I assume it is the 65-byte private key that you double-sha256, correct?   Then, is it the first 16 bytes that are used for the IV?  AES256 IVs are only 16 bytes...
(3)  Is there any modification to the passphrase before applying X sha512 operations to it?  Is there any PBKDF2 calls somewhere?  (if there is, I'm not as well-prepared as I thought)

If anyone wants to simply use the work I've already done and split any appropriate bounties, I would be up for that  I have a python script which currently goes through the wallet.dat file pulls out every single public key, and then grabs any plaintext private keys nearby.  Of course, they private keys are not plaintext in this situation, but I can spit out the data where I expect the keys to be.  Then using CLI input params, I can actually apply the X sha512 operations and AES256-CBC decryption, provided with X and the passphrase.

I wouldn't mind doing this anyway, because I had planned on creating a wallet-conversion/import tool for Armory, but figured it was too much of a hassle.  Maybe now is a good time to deal with it

[jonathan]

I'm running python 2.7.1+ and jackjack-jj-pywallet-db5f0e9, on ubuntu 11.04 64 bit. I do...

$ sudo add-apt-repository ppa:twisted-dev/ppa
$ sudo aptitude install build-essential python-dev python-twisted python-bsddb3

... and so far, so good. But then...

$ sudo easy_install ecdsa
Processing ecdsa
error: Not a recognized archive type: ecdsa

I had a hunt around in synaptic, and the links in jackjack's first post in this thread, but no joy. Arghh, I have had this working before in debian squeeze! Anybody got any ideas how to fix this up for ubuntu 11.04?

edit: Just for the record,

# aptitude install build-essential python-dev python-twisted python-bsddb3 python-setuptools
# easy_install ecdsa

... does the job in debian squeeze (32 bit). I just tried it.

[Joric]

To whom it may concern - I added encrypted wallets support this morning - http://github.com/joric/pywallet
Here is the shorter, more readable version - PoC and simultaneously the unit test.

Code:

#!/usr/bin/env python

# Bitcoin wallet keys AES encryption / decryption (see http://github.com/joric/pywallet)
# Uses pycrypto or libssl or libeay32.dll or aes.py from http://code.google.com/p/slowaes

crypter = None

try:
    from Crypto.Cipher import AES
    crypter = 'pycrypto'
except:
    pass

class Crypter_pycrypto( object ):
    def SetKeyFromPassphrase(self, vKeyData, vSalt, nDerivIterations, nDerivationMethod):
        data = vKeyData + vSalt
        for i in range(nDerivIterations):
            data = hashlib.sha512(data).digest()
        self.SetKey(data[0:32])
        self.SetIV(data[32:32+16])
        return len(data)

    def SetKey(self, key):
        self.chKey = key

    def SetIV(self, iv):
        self.chIV = iv[0:16]

    def Encrypt(self, data):
        return AES.new(self.chKey,AES.MODE_CBC,self.chIV).encrypt(data)[0:32]
 
    def Decrypt(self, data):
        return AES.new(self.chKey,AES.MODE_CBC,self.chIV).decrypt(data)[0:32]

try:
    if not crypter:
        import ctypes
        import ctypes.util
        ssl = ctypes.cdll.LoadLibrary (ctypes.util.find_library ('ssl') or 'libeay32')
        crypter = 'ssl'
except:
    pass

class Crypter_ssl(object):
    def __init__(self):
        self.chKey = ctypes.create_string_buffer (32)
        self.chIV = ctypes.create_string_buffer (16)

    def SetKeyFromPassphrase(self, vKeyData, vSalt, nDerivIterations, nDerivationMethod):
        strKeyData = ctypes.create_string_buffer (vKeyData)
        chSalt = ctypes.create_string_buffer (vSalt)
        return ssl.EVP_BytesToKey(ssl.EVP_aes_256_cbc(), ssl.EVP_sha512(), chSalt, strKeyData,
            len(vKeyData), nDerivIterations, ctypes.byref(self.chKey), ctypes.byref(self.chIV))

    def SetKey(self, key):
        self.chKey = ctypes.create_string_buffer(key)

    def SetIV(self, iv):
        self.chIV = ctypes.create_string_buffer(iv)

    def Encrypt(self, data):
        buf = ctypes.create_string_buffer(len(data) + 16)
        written = ctypes.c_int(0)
        final = ctypes.c_int(0)
        ctx = ssl.EVP_CIPHER_CTX_new()
        ssl.EVP_CIPHER_CTX_init(ctx)
        ssl.EVP_EncryptInit_ex(ctx, ssl.EVP_aes_256_cbc(), None, self.chKey, self.chIV)
        ssl.EVP_EncryptUpdate(ctx, buf, ctypes.byref(written), data, len(data))
        output = buf.raw[:written.value]
        ssl.EVP_EncryptFinal_ex(ctx, buf, ctypes.byref(final))
        output += buf.raw[:final.value]
        return output

    def Decrypt(self, data):
        buf = ctypes.create_string_buffer(len(data) + 16)
        written = ctypes.c_int(0)
        final = ctypes.c_int(0)
        ctx = ssl.EVP_CIPHER_CTX_new()
        ssl.EVP_CIPHER_CTX_init(ctx)
        ssl.EVP_DecryptInit_ex(ctx, ssl.EVP_aes_256_cbc(), None, self.chKey, self.chIV)
        ssl.EVP_DecryptUpdate(ctx, buf, ctypes.byref(written), data, len(data))
        output = buf.raw[:written.value]
        ssl.EVP_DecryptFinal_ex(ctx, buf, ctypes.byref(final))
        output += buf.raw[:final.value]
        return output

try:
    if not crypter:
        from aes import *
        crypter = 'pure'
except:
    pass

class Crypter_pure(object):
    def __init__(self):
        self.m = AESModeOfOperation()
        self.cbc = self.m.modeOfOperation["CBC"]
        self.sz = self.m.aes.keySize["SIZE_256"]

    def SetKeyFromPassphrase(self, vKeyData, vSalt, nDerivIterations, nDerivationMethod):
        data = vKeyData + vSalt
        for i in range(nDerivIterations):
            data = hashlib.sha512(data).digest()
        self.SetKey(data[0:32])
        self.SetIV(data[32:32+16])
        return len(data)

    def SetKey(self, key):
        self.chKey = [ord(i) for i in key]

    def SetIV(self, iv):
        self.chIV = [ord(i) for i in iv]

    def Encrypt(self, data):
        mode, size, cypher = self.m.encrypt(data, self.cbc, self.chKey, self.sz, self.chIV)
        return ''.join(map(chr, cypher))
 
    def Decrypt(self, data):
        chData = [ord(i) for i in data]
        return self.m.decrypt(chData, self.sz, self.cbc, self.chKey, self.sz, self.chIV)

import hashlib

def Hash(data):
    return hashlib.sha256(hashlib.sha256(data).digest()).digest()

def main():

    #address
    addr = '1AJ3vE2NNYW2Jzv3fLwyjKF1LYbZ65Ez64'
    sec = '5JMhGPWc3pkdgPd9jqVZkRtEp3QB3Ze8ihv62TmmvzABmkNzBHw'
    secret = '47510706d76bc74a5d57bdcffc68c9bbbc2d496bef87c91de7f616129ac62b5f'.decode('hex')
    pubkey = '046211d9b7836892c8eef49c4d0cad7797815eff95108e1d30745c03577596c9c00d2cb1ab27c7f95c28771278f89b7ff40da49fe9b4ee834a3f6a88324db837d8'.decode('hex')
    ckey = '0f8c75e4c6ab3c642dd06786af80ca3a93e391637d029f1da919dad77d3c8e477efd479814ddf4c459aeba042420868f'.decode('hex')

    #master key
    crypted_key = '1e1d7ab34d8007f214eb528a1007c6721b9cd1d2c257adb25378ea8e47e3bdd22cfe93a8b6f18dcbe4206fe8c8178ff1'.decode('hex')
    salt = '3f94e3c670b695dd'.decode('hex')
    rounds = 47135
    method = 0
    password = '12345'

    global crypter

    if crypter == 'pycrypto':
        crypter = Crypter_pycrypto()
    elif crypter == 'ssl':
        crypter = Crypter_ssl()
        print "using ssl"
    elif crypter == 'pure':
        crypter = Crypter_pure()
        print "using slowaes"
    else:
        print("Need pycrypto of libssl or libeay32.dll or http://code.google.com/p/slowaes")
        exit(1)

    crypter.SetKeyFromPassphrase(password, salt, rounds, method)
    masterkey = crypter.Decrypt(crypted_key)
    crypter.SetKey(masterkey)
    crypter.SetIV(Hash(pubkey))
    d = crypter.Decrypt(ckey)
    e = crypter.Encrypt(d)

    print "masterkey:", masterkey.encode('hex')
    print 'c:', ckey.encode('hex')
    print 'd:', d.encode('hex')
    print 'e:', e.encode('hex')

if __name__ == '__main__':
    main()

[jackjack]

Many thanks Joric!
I didn't manage to find a working library for it and you found 3 of them...

I have exams this week, so I think I will put that in my fork this week-end or maybe next week

[mrx]

Having difficulties playing with ixcoin privkeys. They have a high addr version (138).

Code:

ixcoin/src/base58.h:160
#define ADDRESSVERSION   ((unsigned char)(fTestNet ? 111 : 138))  //ixcoin 

Code:

2012-01-26 17:23:19+0800 [HTTPChannel,0,172.16.0.3] 172.16.0.3 - - [26/Jan/2012:09:23:19 +0000] "GET /Info?key=MQhdkZfkga2JruxZxpUd39cB6THeTYBootwszNbjGuRUbRxxxx&msg=&pubkey=&sig=&vers=138&format=reg&need=1 HTTP/1.1" 200 18 "http://172.16.0.110:8989/" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1017.2 Safari/535.19"
2012-01-26 17:23:41+0800 [HTTPChannel,0,172.16.0.3] Unhandled Error
        Traceback (most recent call last):
          File "/usr/lib/python2.6/dist-packages/twisted/web/http.py", line 807, in requestReceived
            self.process()
          File "/usr/lib/python2.6/dist-packages/twisted/web/server.py", line 125, in process
            self.render(resrc)
          File "/usr/lib/python2.6/dist-packages/twisted/web/server.py", line 132, in render
            body = resrc.render(self)
          File "/usr/lib/python2.6/dist-packages/twisted/web/resource.py", line 210, in render
            return m(request)
        --- <exception caught here> ---
          File "pywalletm.py", line 1771, in render_GET
            pkey = regenerate_key(sec)
          File "pywalletm.py", line 448, in regenerate_key
            b = ASecretToSecret(sec)
          File "pywalletm.py", line 442, in ASecretToSecret
            if vch and vch[0] == chr(addrtype+128):
        exceptions.ValueError: chr() arg not in range(256)

138 + 128 > 255 so pywallet failed to translate those keys. Any workarounds?

[Joric]

Let me see, ixcoin - first commit aug 16 2011, showwallet - authored jul 12 2011.
Have no idea why they used addrtype 138 it's their problem now.
I believe 7-th bit has been reserved for "private addresses".
By the way, showwallet is already in the master branch (0.5.99).

Actual code (base58.h):

Code:

void SetSecret(const CSecret& vchSecret)
{
    SetData(fTestNet ? 239 : 128, &vchSecret[0], vchSecret.size());
}

There's no explicit 7-th bit anywhere though, so you may pick any byte you like.
With arithmetic overflow it would be chr((addrtype + 128) & 255) == 10.

[mrx]

Let me see, ixcoin - first commit aug 16 2011, showwallet - authored jul 12 2011.
Have no idea why they used addrtype 138 it's their problem now.
I believe 7-th bit has been reserved for "private addresses".
By the way, showwallet is already in the master branch (0.5.99).

Actual code (base58.h):

Code:

void SetSecret(const CSecret& vchSecret)
{
    SetData(fTestNet ? 239 : 128, &vchSecret[0], vchSecret.size());
}

There's no explicit 7-th bit anywhere though, so you may pick any byte you like.
With arithmetic overflow it would be chr((addrtype + 128) & 255) == 10.

well thanks, now it can dump the wallet without problem with SecretToASecret and reverse modified, but still can't process privkeys, insists saying "Bad private key".

[Joric]

well thanks, now it can dump the wallet without problem with SecretToASecret and reverse modified, but still can't process privkeys, insists saying "Bad private key".

IDK why but I tried to import public key first and it said "bad private key" too. Re-check what are you trying to import and try this with addrtype = 138 (line 23). It worked for me. Unfortunately it's not possible to autodetect (wallet doesn't contain addresses, only public keys).

Code:

pywallet.py --importprivkey=MXTf98yWq9KEovooAGDSjEWkVbYF5eiFZWKvWL6VX6vhpLZj7m
Address: xt5y9PhK1wnTT4fnf4SLVDaTCyMgAuNPXR
Privkey: MXTf98yWq9KEovooAGDSjEWkVbYF5eiFZWKvWL6VX6vhpLZj7m
Imported successfully

[mrx]

IDK why but I tried to import public key first and it said "bad private key" too. Re-check what are you trying to import and try this with addrtype = 138 (line 23). It worked for me. Unfortunately it's not possible to autodetect (wallet doesn't contain addresses, only public keys).

Code:

pywallet.py --importprivkey=MXTf98yWq9KEovooAGDSjEWkVbYF5eiFZWKvWL6VX6vhpLZj7m
Address: xt5y9PhK1wnTT4fnf4SLVDaTCyMgAuNPXR
Privkey: MXTf98yWq9KEovooAGDSjEWkVbYF5eiFZWKvWL6VX6vhpLZj7m
Imported successfully

It works, thanks.

However new pywallet can't use this method, still says bad key. Sometimes simple things works better than complex things.

[drakahn]

so i've been spammed with dust on namecoin and my wallet is 200mb... and won't load any more

i've tried deleting all transactions but there are still a lot of addresses made (200mb worth? the wallet size didn't change with deleting transactions...)

any idea on a solution? i think i will need to use pywallet so i'm asking here because it would be the best place to find people that know how to use it and what it can do...

[jackjack]

so i've been spammed with dust on namecoin and my wallet is 200mb... and won't load any more

i've tried deleting all transactions but there are still a lot of addresses made (200mb worth? the wallet size didn't change with deleting transactions...)

any idea on a solution? i think i will need to use pywallet so i'm asking here because it would be the best place to find people that know how to use it and what it can do...

The problem is that Namecoin download all the transactions when you launch it (it uses it to calculate the balance)
The better thing to do is to create a new address, send all your namecoins to this address, and import this address in a new wallet

[drakahn]

so i've been spammed with dust on namecoin and my wallet is 200mb... and won't load any more

i've tried deleting all transactions but there are still a lot of addresses made (200mb worth? the wallet size didn't change with deleting transactions...)

any idea on a solution? i think i will need to use pywallet so i'm asking here because it would be the best place to find people that know how to use it and what it can do...

The problem is that Namecoin download all the transactions when you launch it (it uses it to calculate the balance)
The better thing to do is to create a new address, send all your namecoins to this address, and import this address in a new wallet

i couldn't send without getting into it first...

but i found an old pre spam backup and deleted the spammed address, all fine now

[finway]

Watching.

[1QaZxSw2]

Very nice tool. Can this be used for litecoin wallets?

[jackjack]

Very nice tool. Can this be used for litecoin wallets?

If it uses the same type of wallet, yes.
Anyway, make a backup of your litecoin wallet and make a try

[viper1092]

Oh, wow! Thank you (and everyone else who has worked on it) for this awesome tool. It totally saved me, I was able to use it to dump JSON of my wallet. For some reason Bitcoin-qt would never actually open my wallet.

Then I realized it has a super awesome web interface via localhost.
This is brilliant.

[talpan]

Hello,

I've played a bit with vanitygen and I want to import the privkey now, but when I
 run pywallet the following exception is thrown:

Code:

  File "pywallet.py", line 10
    from bsddb.db import *
       ^
IndentationError: expected an indented block

I'm running ubuntu 11.10 and I've installed all the dependencies that are listed on github.
Any help is very appreciated

regards, talpan

[ForceField]

If Pywallet is still in development, suggestions for future versions:

1. Please implement backwards compatibility with previous versions of the Bitcoin client (such as v0.3.20.2) to enable Pywallet's use without closing the client.
2. Create a GUI, to simplify its use and enable a wider audience to be able to appreciate the easy functionality of Pywallet.
3. Add an option to combine two or more wallets.

Excellent Essential Tool. +1