[PULL] private key and wallet export/import

[runeks]

Testing is currently the bottleneck for getting new features...

IMHO, making absolutely sure that new patches are safe is of utmost importance.
Hurrying a patch through might, at best, give us a feature one month ahead of time in the main client. At worst it might introduce an exploit into the main client that would crush any existing trust in Bitcoin. Security is so vital when it comes to a program like the main Bitcoin client. If we expect people to at some point start using Bitcoin as a currency, I can see no higher priority than security.
One security hole can potentially rob people of the equivalent of thousands of dollars. One extra feature lacking for a month is a very tiny advantage compared to this.

[btc_artist]

Any news on getting private key import into the main client?

[Gavin Andresen]

No issues with export wallets/private keys. I share gmaxwell's concerns about making it easy to shoot yourself in the foot, but most of us are grown-ups and if you're talking using the RPC interface there are already plenty of ways to shoot your feet.

Remove private key I had issues with, because if you're using the 'accounts' feature then removing keypairs from a wallet (and their associated transactions) does unpredictable things to account balances. At the very least, I think it should tell you what effects it had.  Maybe a JSON result that tells you how account balances changed, e.g. { "" : 1, "John's Wallet" : 6.2, etc.}. That way, if it had an unexpected effects you would know to restore the wallet from backup.

And it seems like 'sweep private key' and 'merge wallets' is really the functionality most people want, not import private key/wallet keys. The only issue I have with them is they are slow because of the rescanning of the block chain, and they may not work or may not be secure if you don't happen to have the whole block chain downloaded.

[btc_artist]

Thanks for the update Gavin.  I'm not sure what you're referring to with "sweep private key", but the functionality I would like to see is (1) merge wallets, and (2) import private key.  Merge wallets is pretty self-explanatory. Importing private keys is needed to be able to "redeem" bitcoins from offline storage such as paper wallets or physical bitcoins. Rescanning the block chain and/or waiting for the rest of the block chain to download is slow, but it's inevitable. Have a progress bar with a note saying it may take a while, and let the user interact with the rest of the application while it's working.

[SgtSpike]

Thanks for the update Gavin.  I'm not sure what you're referring to with "sweep private key", but the functionality I would like to see is (1) merge wallets, and (2) import private key.  Merge wallets is pretty self-explanatory. Importing private keys is needed to be able to "redeem" bitcoins from offline storage such as paper wallets or physical bitcoins. Rescanning the block chain and/or waiting for the rest of the block chain to download is slow, but it's inevitable. Have a progress bar with a note saying it may take a while, and let the user interact with the rest of the application while it's working.

Yes this.

Importing private keys is a must for things like offline storage of bitcoins, etc.  It may seem like most users just want merged wallets, but I know there's a bunch who need the ability to import private keys easily as well!

[bulanula]

I would like functionality to actually see your private keys without additional software. Just using the main client. Is this possible now ?

[SgtSpike]

I would like functionality to actually see your private keys without additional software. Just using the main client. Is this possible now ?

That would be nice too... just so long as it requests your wallet encryption password for every private key request.

[casascius]

And it seems like 'sweep private key' and 'merge wallets' is really the functionality most people want, not import private key/wallet keys. The only issue I have with them is they are slow because of the rescanning of the block chain, and they may not work or may not be secure if you don't happen to have the whole block chain downloaded.

They wouldn't be slow if there were an index to speed it up.  Such an index would have numerous other uses besides sweeping.

This would be an optional index that would be disabled by default, because it would consume a lot of disk space but benefit only the relatively small percentage of users who would use it.  Of course, for those who use it regularly, having Bitcoin take up another 20% of disk space is far better than having to wait 3-5 minutes each time you want to import a key.

I've put up a bounty for this index to be implemented.  jarpiain appears to have made considerable progress on it in his github fork of bitcoin.  I probably should up the bounty to account for decrease in BTC value since it was posted.  

[btc_artist]

Just to make sure I understand, the index would be to look up a bitcoin address and get all the blocks that that address has transactions in, so only those blocks would need to be scanned for transactions to add to your wallet, correct?

[N.Z.]

I didn`t get anyway...any chance to see official import feature in 0.5? 0.6? When?

[casascius]

Just to make sure I understand, the index would be to look up a bitcoin address and get all the blocks that that address has transactions in, so only those blocks would need to be scanned for transactions to add to your wallet, correct?

Exactly.

It would be fine if this index were built on the beginning of the address (e.g. the first 32 bits) rather than the entire thing, in order to save on space, at the minor expense that a few blocks might occasionally be consulted that don't actually contain transactions for the bitcoin address being searched.  My estimate of 20% of the block chain was very liberal in the side of "biggest impact" - practically, I think it would be far less.  The index would point to all blocks that contained any kind of reference to the address, whether input, output, or whatever, possibly even in orphan blocks.

[SgtSpike]

Just to make sure I understand, the index would be to look up a bitcoin address and get all the blocks that that address has transactions in, so only those blocks would need to be scanned for transactions to add to your wallet, correct?

Exactly.

It would be fine if this index were built on the beginning of the address (e.g. the first 32 bits) rather than the entire thing, in order to save on space, at the minor expense that a few blocks might occasionally be consulted that don't actually contain transactions for the bitcoin address being searched.  My estimate of 20% of the block chain was very liberal in the side of "biggest impact" - practically, I think it would be far less.  The index would point to all blocks that contained any kind of reference to the address, whether input, output, or whatever, possibly even in orphan blocks.

You're probably safe pulling it just based on the first 7 chars of any address.  There are very few addresses that share the same first 7 chars.  8 would be virtually none.  I don't know how many bits there are per char in the address though...  Is it just the same 8 bits per char as ASCII?

[runeks]

^ It's base58 so about 5.85 bits per char.
We should also consider vanity addresses though. Not all addresses are "random". Some are selected based on aesthetics.

If we have to search through the whole block chain in order to know the balance of a single address, how can the Bitcoin network verify or discard transactions as fast as it can? If it has to search through the whole block chain when a transaction comes in, to see if the sending address has at least the amount it wishes to send, how can the network ever handle multiple transactions per second?

[SgtSpike]

^ It's base58 so about 5.85 bits per char.
We should also consider vanity addresses though. Not all addresses are "random". Some are selected based on aesthetics.

If we have to search through the whole block chain in order to know the balance of a single address, how can the Bitcoin network verify or discard transactions as fast as it can? If it has to search through the whole block chain when a transaction comes in, to see if the sending address has at least the amount it wishes to send, how can the network ever handle multiple transactions per second?

Well, technically, yes, 5.85 bits.  I was talking more about how those chars are stored though, since that's what is relevant when casascius says "the first 32 bits".  Are they actually stored in some 6-bit formation, or are they stored in 8-bit ASCII chars, or 16-bit Unicode, or something else?

Also, how do vanity addresses change anything?

[runeks]

^ Ah, I see your point. The address is stored so there are no redundant bits, or else the block chain would take up too much space. Looks like he uses a variable called uint160, which is a base_uint class with the size of 160 bits. So it should work just like a 160 bit unsigned integer variable.

Vanity addresses can make the "take first x chars"-approach match more addresses than what would be calculated if we assumed all random address generation.
Ie., using the first 32 bits of an address as the ID doesn't give us a 1/2^32 chance of that matching two addresses (or whatever probability random address generation would lead to, I'm not that good a probability math ), because of vanity addresses. But this may not be relevant since, as casascius already said, we would just be searching through maybe two or three times as many blocks, but that would still be way way less than searching through all blocks.

[SgtSpike]

^ Ah, I see your point. The address is stored so there are no redundant bits, or else the block chain would take up too much space. Looks like he uses a variable called uint160, which is a base_uint class with the size of 160 bits. So it should work just like a 160 bit unsigned integer variable.

Vanity addresses can make the "take first x chars"-approach match more addresses than what would be calculated if we assumed all random address generation.
Ie., using the first 32 bits of an address as the ID doesn't give us a 1/2^32 chance of that matching two addresses (or whatever probability random address generation would lead to, I'm not that good a probability math ), because of vanity addresses. But this may not be relevant since, as casascius already said, we would just be searching through maybe two or three times as many blocks, but that would still be way way less than searching through all blocks.

Interesting, thanks for the info!

[mndrix]

It would be fine if this index were built on the beginning of the address (e.g. the first 32 bits) rather than the entire thing, in order to save on space, at the minor expense that a few blocks might occasionally be consulted that don't actually contain transactions for the bitcoin address being searched

If key import is the use case, you could also make the index smaller by pointing at only the first block that mentions an address.  If I send coins today, such an index would immediately eliminate 152,000 blocks from the search.  I'd guess that imported keys typically have a short time between initial funding and sweeping to a semi-permanent address.

[mndrix]

If we have to search through the whole block chain in order to know the balance of a single address, how can the Bitcoin network verify or discard transactions as fast as it can?

Transactions don't spend funds from an address.  They spend funds from a previous transaction.  So it's using a different index than what we'd need here.

[runeks]

^ I see. I really should go read more about the inner workings of Bitcoin.
Or maybe I should post questions in various threads on the questions I have about it until I understand it fully...

[runeks]

I get the following error when I try to build the showwallet branch with gcc 4.6:

Code:

g++ -c -pthread -Wno-invalid-offsetof -Wformat -g -DNOPCH  -DUSE_SSL -fno-stack-protector -fstack-protector-all -Wstack-protector -Wl,-z,relro -Wl,-z,now -D_FORTIFY_SOURCE=2 -O2 -MMD -o obj/nogui/bitcoinrpc.o bitcoinrpc.cpp
bitcoinrpc.cpp: In function 'void ThreadRPCServer2(void*)':
bitcoinrpc.cpp:2213:9: error: 'filesystem' has not been declared
bitcoinrpc.cpp:2213:26: error: expected ';' before 'certfile'
bitcoinrpc.cpp:2214:14: error: 'certfile' was not declared in this scope
bitcoinrpc.cpp:2214:49: error: 'filesystem' has not been declared
bitcoinrpc.cpp:2215:13: error: 'filesystem' has not been declared
bitcoinrpc.cpp:2215:32: error: 'certfile' was not declared in this scope
bitcoinrpc.cpp:2217:9: error: 'filesystem' has not been declared
bitcoinrpc.cpp:2217:26: error: expected ';' before 'pkfile'
bitcoinrpc.cpp:2218:14: error: 'pkfile' was not declared in this scope
bitcoinrpc.cpp:2218:45: error: 'filesystem' has not been declared
bitcoinrpc.cpp:2219:13: error: 'filesystem' has not been declared
bitcoinrpc.cpp:2219:32: error: 'pkfile' was not declared in this scope
make: *** [obj/nogui/bitcoinrpc.o] Error 1

I'm using gcc 4.6.1, libc6 2.13 and libboost 1.46 from Debian sid (the Linuxcoin distribution).
The regular bitcoin branch builds without issues.

EDIT: I just tried with gcc 4.5.2 in Ubuntu Natty as well, gives me the same error.