Getting private keys + creation date/ first use date out of Satoshi client

[jim618]

Hello All,

I have just put into MultiBit the ability to output the private keys of a wallet to a file and then reimport them to a different wallet.   

For maximum interoperability the format used for the private keys is the sipa wallet import format.  Both Andreas Schildbach and I are using the same format for these keys file so that Android Bitcoin Wallet and MultiBit keys can be swapped.

An interesting possibility is getting the data out of the Satoshi client and into MultiBit effectively "cloning" a wallet.dat - hence this post. If I include a simple example of the key file (which includes a disposable private key I made on bitaddress.org - again the same format) :

---- file starts ----
# KEEP YOUR PRIVATE KEYS SAFE !
# Anyone who can read this file can spend your bitcoin.
#
# Format:
#   <Base58 encoded private key>[<whitespace>[<key createdAt>]]
#
#   The Base58 encoded private keys are the same format as
#   produced by the Satoshi client/ sipa dumpprivkey utility.
#
#   Key createdAt is in UTC format as specified by ISO 8601
#   e.g: 2011-12-31T16:42:00Z . The century, 'T' and 'Z' are mandatory
#
5HpaKaHJXma9ESuV2X2V4e1NovA6WXC9JmUhkVu117y6tQKN8eE 2012-01-22T00:00:00Z
# End of private keys
---- file ends ----

The lines beginning with # are comments.   There is only one 'real' line in there - the key and the createdAt date.

Why the createdAt date?
Well, the stored blocks in bitcoinj do not store the transactions so at the moment we replay the blocks from a point before the private key creation date.   That is what the UTC date is for - it tells MultiBit how far back in time to replay the blocks.   It is not strictly the date of creation of the key - it just has to be before the first ever transaction that uses that key. It is in UTC format mainly to make it easy for people to handcraft the file if they need to.

My question is:
Is there a way to extract this information from the Satoshi wallet.dat ?


[Security Note: At the moment MultiBit does not have encrypted wallets and the key files are not encrypted. Both of these are planned enhancements but obviously you do not want to export any valuable encrypted Satoshi client keys into plain text.
Ideally it would be possible to transport private keys ENCRYPTED and have the same KDF and encryption algorithm at "both ends" but that is a question for another day.]

[DiThi]

Yes, it saves all transactions for each address, so just check the date of the first transaction.

[Pieter Wuille]

Version 0.6 of the Satoshi client will have the exportprivkey RPC call, which can be used to extract the private key corresponding to an address. This should allow you to import it, but it requires probably too much manual work to retrieve all keys one by one. Also, you'd need to scan all transactions in the wallet to find out where they were first used.

Additionally, my showwallet branch allows exporting the entire wallet (organised in information per key, and optionally which blocks and/or transactions they were used in, in a human-readable JSON file format. It still needs a bit of work, but I hope to get this merged soon as well.

Finally, note that I recently sent a mail to the bitcoin-development mailing list about an extension to the base58 private key format to indicate compressed public key addresses, which will also be used as of the 0.6 release.

[jim618]

Hi Pieter,

Thanks for your response.
Your showwallet branch would be the thing for me to use I think.  With block usage/ transactions it gives me the data I need.

I will have a look at your base58 private key format extension too as obviously we will have to support that.

There is plenty to do to improve the security on MultiBit which needs doing first so in a way I am glad there is no easy way to export all the keys out of the Satoshi client currently.   


From a user experience point of view:
Getting your private keys out of any bitcoin software is an excellent form of backup, but the ability to 'clone' wallets does present some problems with how it is presented to the end user.   Having a wallet on one machine which faithfully reflects the spends and receives on a different machine is very powerful but it is a quite different to what people are used to in financial software.

[DiThi]

For that reason it's better to have a "sweep funds" option instead of a "import private key".

Btw, what I said about the dates of saved transactions doesn't apply when you import a key with pywallet and use -rescan, as it records the date of the rescan instead. At least you would know which is the first transaction and look for the block it's located in.

[jim618]

@DiThi - yes the thread on sweeping/importing keys covers this pretty comprehensively.

I think this post by DeathAndTaxes summarises it quite well:

https://bitcointalk.org/index.php?topic=54603.msg659372#msg659372


RE: your clarification - knowing the first transaction and the block it is in would be enough.   The main thing with MultiBit/ bitcoinj is avoiding a complete replay from the genesis block which is costly.

[pointbiz]

Finally, note that I recently sent a mail to the bitcoin-development mailing list about an extension to the base58 private key format to indicate compressed public key addresses, which will also be used as of the 0.6 release.

Is there somewhere I can learn what you mean by compressed vs uncompressed public keys? What is a compressed public key and what is it used for?

[jim618]

In this post Pieter explains what a compressed public key is:

http://sourceforge.net/mailarchive/forum.php?thread_name=20111121114819.GB7261%40ulyssis.org&forum_name=bitcoin-development

My elliptic curve math is not really up to speed but it would appear that out of the X and Y coordinates you normally specify for a point on a curve in 2D, you can get away with specifying X and then recalculating the Y point.

It looks like there are two solutions for Y for a given X (think solving for Y in Y = X ^ 0.5) - it looks like only one is chosen but Pieter does not mention in that post how this is done.

The amount of shortening you could expect from this would be the order of 50% I presume as you are only specifying one point rather than two (excluding any constant prefix / signalling bits)