Pywallet 2.2: manage your wallet [Update required]

[BitFanatic]

Great tool - incredibly useful!

I am currently importing a large amount of addresses via csv file.
Are there any stats/info on how long it takes for say every thousand addresses imported?

Thanks
If you have any comments/suggestions/critics I'd be happy  

It would be nice to have some sort of progress bar when executing larger tasks.
I'm still importing the massive csv file - but no idea how much longer it's going to take or if it has frozen.

Edit:
Infact you could add that quite easily - I just checked the dir with wallet.dat and can see that it is still increasing in size which means that the job has not frozen.
wallet.dat is now 80mb - I wonder how big it will get!

[jackjack]

Great tool - incredibly useful!

I am currently importing a large amount of addresses via csv file.
Are there any stats/info on how long it takes for say every thousand addresses imported?

Thanks
If you have any comments/suggestions/critics I'd be happy 

It would be nice to have some sort of progress bar when executing larger tasks.
I'm still importing the massive csv file - but no idea how much longer it's going to take or if it has frozen.

Edit:
Infact you could add that quite easily - I just checked the dir with wallet.dat and can see that it is still increasing in size which means that the job has not frozen.
wallet.dat is now 80mb - I wonder how big it will get!

Half of my post wasn't posted...
"I'd be happy... to consider them"
And I never imported that many private keys myself so I don't know the speed. Maybe PM the guy who asked me the feature (page 12 or 13) or give it a try.

The progress bar is a good idea, I'll at least put the import speed soon.
Would you mind posting the time it took, your CPU and the number of addresses you imported?
As for the size I'd say 150-200 Mo/Madresses

[K1773R]

Great tool - incredibly useful!

I am currently importing a large amount of addresses via csv file.
Are there any stats/info on how long it takes for say every thousand addresses imported?

Thanks
If you have any comments/suggestions/critics I'd be happy  

It would be nice to have some sort of progress bar when executing larger tasks.
I'm still importing the massive csv file - but no idea how much longer it's going to take or if it has frozen.

Edit:
Infact you could add that quite easily - I just checked the dir with wallet.dat and can see that it is still increasing in size which means that the job has not frozen.
wallet.dat is now 80mb - I wonder how big it will get!

i once imported 10m keys into a wallet.dat, the wallet was several gigs and bitcoind couldnt use it anymore and created a new one. so there is somewhere a upperlimit, i didnt research how much it is.

[fimp]

I'm trying to dump an encrypted wallet using the latest pywallet.

Code:

$ python pywallet.py --wallet=wallet.dat --dumpwallet --passphrase=xxxxxxxx
Segmentation fault: 11 (core dumped)
$

What can I do? Dumping it without the passphrase works, but gives me encrypted private keys.

[jackjack]

I never saw this...
What version of Python do you use?
Does it crash with a bad passphrase too?
Did you try with another encrypted wallet?
Is the wallet huge?

[fimp]

I never saw this...
What version of Python do you use?
Does it crash with a bad passphrase too?
Did you try with another encrypted wallet?
Is the wallet huge?

Python 2.7.5
Same error with bad passphrase.
Same error with another encrypted wallet.
Wallet is about 4.4 MB. Amount of addresses probably in the hundreds.

[mwm4bcf]

Just installed the packages on Win Vista. No obvious install errors.

The web interface comes up fine.

When I try to dump the wallet, I get the error "Error in dump page".  Any ideas?  Thanks!


Also, on the "Info" page, I try "Print the balance of a Bitcoin address", and get "Balance of <address> : Page not found".

[jackjack]

I never saw this...
What version of Python do you use?
Does it crash with a bad passphrase too?
Did you try with another encrypted wallet?
Is the wallet huge?

Python 2.7.5
Same error with bad passphrase.
Same error with another encrypted wallet.
Wallet is about 4.4 MB. Amount of addresses probably in the hundreds.

I'm sure I answered this post earlier! I don't get what happened...
What hardware is pywallet installed on? Small, like a RPi?
Is there something special written on the console before the segfault text?
Would you mind sending me an empty, encrypted wallet that can reproduce it?

Just installed the packages on Win Vista. No obvious install errors.

The web interface comes up fine.

When I try to dump the wallet, I get the error "Error in dump page".  Any ideas?  Thanks!

Also, on the "Info" page, I try "Print the balance of a Bitcoin address", and get "Balance of <address> : Page not found".

Thanks for reporting
What is written on the console before "Error in dump page"?

And indeed, I forgot a character, thanks! The update is on its way
Updated, you can upgrade to the last version. Maybe try the auto-update function, top-right of the pywallet page.

[mwm4bcf]

Thanks for the quick reply.  The problem isn't fixed.

Right after I click on the web page to dump my wallet, the console displays:

"root: couldn't open wallet.dat/main.  Try quitting Bitcoin and running this again."

followed by an unhandled error traceback.

Thanks again.

[jackjack]

This means the file doesn't exist. Check the path you wrote.

[mwm4bcf]

I did verify the path and filename.

Look at that error message again - it's trying to open "wallet.dat/main".  I certainly didn't type in the "/main" - the program somehow added that to the default filename.  Make sense?

[jackjack]

I did verify the path and filename.

Look at that error message again - it's trying to open "wallet.dat/main".  I certainly didn't type in the "/main" - the program somehow added that to the default filename.  Make sense?

Yes it's ok, it's a part of code from the original pywallet because main is the database name, and this doesn't change with the actual wallet path (even savingswallet.dat would be labelled in this error as wallet.dat/main).
This error happens when pywallet cant' read the file. Are you sure the file is readable? Try opening notepad and open the path you put in pywallet.
Just to be sure, it's a wallet.dat from the satoshi client, right?

[DeathAndTaxes]

First of a massive thank you to JackJack.  Not only is this a useful tool but I find python a more expressive language than the C/C++ used in the reference client.  This makes pywallet a useful tool for learning the wallet format.

Is the address in your sig best address for donations for further pywallet development?

Also when pywallet exports keys it provides multiple values like this

Code:

        {
            "addr": "1BJ4WxK17Ty4KAUtPYqDwHxtk1W2QSoGjj", 
            "compressed": true, 
            "encrypted_privkey": "92a29468f382783fb7905be81a9c2aa7c22d8ad48e10b5dd33aa1593f5835318f04cfb1b668063537da550796c57055b", 
            "hexsec": "L4Je6yblahblahblahblahblah", 
            "pubkey": "03f30cf41f7aa258c700d486688a15eea564db1d82fb900c8a8af0e0042f60a8cc", 
            "reserve": 1, 
            "sec": "L4Je6yblahblahblahblahblah", 
            "secret": "d365d4430dcbblahblahblahblahblah"
        },

addr, compressed, encrypted_prvikey, pubkey, reserve are all self explanatory.

However a little confused on hexsec, sec, and secret. Hexsec and sec appear to be identical in my dump for all keys.  Will they always be?  At first I though secret was private key in hexadecimal and sec was private key in import format but I am confused by fact that there are three entries and the naming "hexsec" is obviously not in hexadecimal.

Lastly couldn't find it but is there an option to do a dump of an encrypted wallet (without providing passphrase).

As an example something like

Code:

        {
            "addr": "1BJ4WxK17Ty4KAUtPYqDwHxtk1W2QSoGjj", 
            "compressed": true, 
            "encrypted_privkey": "92a29468f382783fb7905be81a9c2aa7c22d8ad48e10b5dd33aa1593f5835318f04cfb1b668063537da550796c57055b", 
            "pubkey": "03f30cf41f7aa258c700d486688a15eea564db1d82fb900c8a8af0e0042f60a8cc", 
            "reserve": 1, 
        }, 

For disaster recovery I print out plain text versions of company wallets (as well as normal binary backups) an encrypted dump would be an awesome feature.

[jackjack]

Donation address for pywallet is in the about page of the WUI, it's 1AQDfx22pKGgXnUZFL1e4UKos3QqvRzNh5. Thank you for making me mention it.

However a little confused on hexsec, sec, and secret. Hexsec and sec appear to be identical in my dump for all keys.  Will they always be?  At first I though secret was private key in hexadecimal and sec was private key in import format but I am confused by fact that there are three entries and the naming "hexsec" is obviously not in hexadecimal.

hexsec is the hexadecimal private key (you must have made a mistake with your copypastings)
secret is hexsec alone if uncompressed address, and hexsec+'\x01' if compressed address
sec is WIF formatted private key

Note that I found a bug though, looks like all the keys have an additional \x01 at the end

Lastly couldn't find it but is there an option to do a dump of an encrypted wallet (without providing passphrase).

As an example something like

Code:

        {
            "addr": "1BJ4WxK17Ty4KAUtPYqDwHxtk1W2QSoGjj", 
            "compressed": true, 
            "encrypted_privkey": "92a29468f382783fb7905be81a9c2aa7c22d8ad48e10b5dd33aa1593f5835318f04cfb1b668063537da550796c57055b", 
            "pubkey": "03f30cf41f7aa258c700d486688a15eea564db1d82fb900c8a8af0e0042f60a8cc", 
            "reserve": 1, 
        }, 

Maybe I don't understand, but it's already what is happening when you dump an encrypted wallet without providing the passphrase.

[DeathAndTaxes]

Donation address for pywallet is in the about page of the WUI, it's 1AQDfx22pKGgXnUZFL1e4UKos3QqvRzNh5. Thank you for making me mention it.

No problem.  Coins sent https://blockchain.info/tx/02f632bea4b192f899108b3706e001e828eed546eac0b19321154278c034c7c3  

Wait nobody has ever donated?  WTF? Soapbox mode engaged.

Public service announcement.  If pywallet has made your life easier (and you know it has), then show some support by sending some coins.  Come on this is a great tools to crack the byzantine mess that is wallet.dat.  Eventually the wallet will move to leveldb which is going to require a significant rewrite of pywallet.  Hint: developers tend not to continually support projects they see no support from the community.

BTW if you are reading this PSA, in the "Pywallet: manage your wallets/addresses/keys/tx's" thread, well the odds are it probably applies to you (otherwise you wouldn't be here).

hexsec is the hexadecimal private key (you must have made a mistake with your copypastings)
secret is hexsec alone if uncompressed address, and hexsec+'\x01' if compressed address
sec is WIF formatted private key

Weird that is not what I am getting.

Here is a unmodifed copy & paste of one key (a heads up to scammers, I removed this unused key from my keypool so nothing to steal)

Code:

E:\bitcoin\pywallet> pywallet.py --passphrase="NotMyPassphrase" --dumpwallet > wallet.decrypted.txt

...
        {
            "addr": "1MM4NpjXnWKohws4KCw9NpVPndmf9jB1Ho", 
            "compressed": true, 
            "encrypted_privkey": "65957014a15092d8329cd661d1a1acc38bf0eaec32b544b9e2fe38390697e2196c93a2b9e3a3d373701e67387ec015c4", 
            "hexsec": "KyFz1DzUkaCB53oM5VJhcp8Qg7aFWn88BuSqPg4rhLimuH3LMFv9", 
            "pubkey": "03ecdbccf53acedc9d283c2bd93b123f1bcd6ec07e5ce65babc81644e60a3e6c80", 
            "reserve": 1, 
            "sec": "KyFz1DzUkaCB53oM5VJhcp8Qg7aFWn88BuSqPg4rhLimuH3LMFv9", 
            "secret": "3cd80f8d99a883b0d48a9f844d94e8e875fbde18c5a6af1489dc9c4a8363c92401"
        }, 
...

This is on pywallet running on Windows, using a wallet exported from v0.8.3 client in case that somehow makes a difference.

Maybe I don't understand, but it's already what is happening when you dump an encrypted wallet without providing the passphrase.

DOH, I never tried.  I assumed passphrase was required for encrypted wallets.  Yeah it does exactly that and that is exactly what I was looking for.  I feel a little stupid that I needed assistance with that, but still glad I asked.  Pure awesome.

[jackjack]

No problem.   Coins sent https://blockchain.info/tx/02f632bea4b192f899108b3706e001e828eed546eac0b19321154278c034c7c3 

Wait nobody has ever donated?  WTF? Soapbox mode engaged.

Public service announcement.  If pywallet has made your life (and you know it has), then show some support and send some coins.  Come on this is a great tools to crack the byzantine mess that is wallet.dat.  Eventually wallet.dat will move to leveldb which is going to require a significant rewrite of pywallet and developers tend not to do that for projects they see support for.

BTW if you are reading this PSA, in the "Pywallet: manage your wallets/addresses/keys/tx's" thread, well the odds are it probably applies to you (otherwise you wouldn't be here).

Thanks a lot!
As the first donator you can ask me any feature you want to be included in pywallet.
Oh wait

Weird that is not what I am getting.

Here is a unmodifed copy & paste of one key (a heads up to scammers, I removed this unused key from my keypool so nothing to steal)

Code:

            "addr": "1MM4NpjXnWKohws4KCw9NpVPndmf9jB1Ho", 
            "compressed": true, 
            "encrypted_privkey": "65957014a15092d8329cd661d1a1acc38bf0eaec32b544b9e2fe38390697e2196c93a2b9e3a3d373701e67387ec015c4", 
            "hexsec": "KyFz1DzUkaCB53oM5VJhcp8Qg7aFWn88BuSqPg4rhLimuH3LMFv9", 
            "pubkey": "03ecdbccf53acedc9d283c2bd93b123f1bcd6ec07e5ce65babc81644e60a3e6c80", 
            "reserve": 1, 
            "sec": "KyFz1DzUkaCB53oM5VJhcp8Qg7aFWn88BuSqPg4rhLimuH3LMFv9", 
            "secret": "3cd80f8d99a883b0d48a9f844d94e8e875fbde18c5a6af1489dc9c4a8363c92401"

This is on pywallet running on Windows, using a wallet exported from v0.8.3 client in case that somehow makes a difference.

I had two bugs in the dump... Both solved, thanks for pointing that out.
I'll release a semi-private version soon (that will include the new wallet recovery, I just finished it)

Maybe I don't understand, but it's already what is happening when you dump an encrypted wallet without providing the passphrase.

DOH, I never tried.  I assumed passphrase was required for encrypted wallets.  Yeah it does exactly that and that is exactly what I was looking for.  I feel a little stupid that I needed assistance with that, but still glad I asked.  Pure awesome.

The truth is I should make http://pywallet.tk clearer

[jackjack]

New version with:

• Dumping fixed
• Recovery of unencrypted AND encrypted wallets on Linux AND Windows

2.1.0b1 here:
http://pastebin.com/raw.php?i=zU2x7amw

[piotr_n]

Thanks a lot!
As the first donator you can ask me any feature you want to be included in pywallet.

Have some more, since it makes you so happy

I don't need a feature, but just want to let you know that IMO your tool is undoubtedly the most useful bitcoin tool out there.
Though, it would not be needed, if the actual client had the same commands just built in..
But well, at least soon it will have a great secured payment protocol. Everybody who wants to receive bitcoins will be able to submit his personal details directly to mossad, in order to receive a free certificate, which will guarantee that the bitcoin address he's sending to you is indeed his. Myself, I've been waiting for such a feature for years - using PGP signatures was so much of a hassle, and it was just too anonymous

[DeathAndTaxes]

A quick question when you have the time (no rush), looking for clarification on tx_k & tx_v.

tx_k?
Looks like tx_k is the raw hex dump of the key as stored in the wallet db.  I would just be tx_id (tx_hash) however it looks like it is 02 + "tx" + reversed tx hash.  With the 02 being the length of the following string "tx".  Is that correct?

I assume this means that given an arbitrary tx hash one needs to reverse it and prepend 0x027478 to find the key/value pair in the db.  I know you didn't write the wallet but any insight as to why?  Why not just make the key the tx_id?  I mean if we start having hash collisions it means a lot more problems then just incorrect db reads.   


tx_v?
tx_v is the raw hex dump of the value stored in the wallet.db which should be the complete raw tx without any additional encoding.
Right?

(if it helps line 1687 in pywallet.py)

[piotr_n]

bitcoin hashes are always in reversed order.
somebody came with an idea that lsb is faster on the machines we have, so why not to extend it to 256 bits.
I don't mean to criticize him, but IMHO it has not turned out to be the most optimal solution after all