Show Message/Counter when address pool since last backup has been depleted.

[Mushoz]

Currently there's no easy way to see if the address pool of 100 new addresses since your last backup has been used up. Bitcoin-qt currently has a backup feature build in, so wouldn't it be quite easy to implement either a counter or message that notifies the user when a new backup is needed to keep his/her funds safe? This counter could be reset each time a new backup is created inside Bitcoin-qt. Now I know some people use different/automatic backup procedures, but a simple command in the config file could disable this feature.

Thoughts/suggestions?

[casascius]

One could just as easily increase the key pool to 2000 or 5000 addresses and know that the average user will probably die before they run out of addresses.  Then such a feature reminding them they're low on pool addresses wouldn't be needed.

[Diapolo]

Is there an RPC command to see how many keys are used / free?

RPC getinfo shows the keypoolsize!

Dia

[flower1024]

i would love this feature
even better: show a warning message if there is a privkey going to be used which is not contained in the last backup.

[DeathAndTaxes]

i would love this feature
even better: show a warning message if there is a privkey going to be used which is not contained in the last backup.

This.  From a usability standpoint having a counter is just another stats, number, widget which adds screen clutter and confusion.  However storing the last key included in the backup and then warning the user when trying to use (either by change address or new address button) a key not included in the last backup would be useful.

This operation will send bitcoins to a key not included in your last backup.  Funds may be lost if wallet is lost or corrupted.  
                It is strongly recommended you create a backup of your wallet before making this transaction.   Do you want to backup your wallet now?

[Yes] [No (Loss of funds possible)] [Backup after this transaction is completed]
[ ] Do not warn me about the potential for key loss again

Hell maybe in "noob mode" the wallet simply wouldn't allow you to do anything until you make a backup.  

The backup of your wallet is out of date.  This transaction can not be recovered from your existing backup.  Funds may be lost if the wallet becomes lost or corrupt.
You must make a backup of your wallet before continuing with this transaction.

[Make Backup Now]

If the second method is used I would include a "nag screen" (can be disabled in options) which starts when there are 10 or less backed up keys in the keypool.  Granted it isn't "foolproof"; a user can click make backup and then not safely store it.  User can make backups outside the GUI and get false warnings but it would provide "pretty good protection".

Another easy precaution would be to simply increase the default size of the keypool.  The default is currently 100. Even a casual user may use 1 or 2 addresses per day.  That means a backup is only good for roughly 60 days worth of addresses.   More active users (satoshi dice) could obsolete their backup in a matter of days.  Pretty easy for an inexperienced user to be running on a wallet no longer protected by a backup and not know it.  Given how tiny keys are I think making the default size 500 or even 1000 keys would be very useful.  Even 1,000 keys in a keypool makes the initially wallet only ~300KB.   Compared to a block chain in GB trying to save a couple hundred KB and putting funds at risk seems silly.

Advanced users who need ultra-small wallet sizes can modify the keypool to be smaller (even a keypool of zero) just like they could disable the backup warnings.

[DeathAndTaxes]

Is there an RPC command to see how many keys are used / free?

RPC getinfo shows the keypoolsize!

Dia

No such concept.  The keypool is continually refilled.

If you have a keypool of 100 keys and use a key the oldest key in the keypool is added to the active key portion of the wallet and the a new key created and added to the keypool.  The keypool will always be 100 keys.  The question is are there any keys in the keypool which aren't in your last backup.

[Diapolo]

Is there an RPC command to see how many keys are used / free?

RPC getinfo shows the keypoolsize!

Dia

No such concept.  The keypool is continually refilled.

If you have a keypool of 100 keys and use a key the oldest key in the keypool is added to the active key portion of the wallet and the a new key created and added to the keypool.  The keypool will always be 100 keys.  The question is are there any keys in the keypool which aren't in your last backup.

Strange that I have 103 in my testnet-wallet ... but thanks for your explanation anyway . I also like the suggested feature!

Dia

[flower1024]

if nobody is faster i'll try to implement it this weekend.

[Diapolo]

if nobody is faster i'll try to implement it this weekend.

That would be a very good thing .

Dia

[DeathAndTaxes]

Strange that I have 103 in my testnet-wallet ... but thanks for your explanation anyway .

Well you made me check and "getinfo" from PRC on our company cold wallet shows keypool 4630.  I know altered that to 5,000 before using the wallet (although wallet was already created before keypool was raised).

So that makes me unsure of the meaning and/or accurate of the "keypool" line in RPC call "getinfo".
Or maybe my understanding of keypool is flawed.  <- this would be bad because a lot of our wallet protection procedures are based on it.

I think I will start a new thread unless someone knows.

[Diapolo]

Strange that I have 103 in my testnet-wallet ... but thanks for your explanation anyway .

Well you made me check and "getinfo" from PRC on my wallet shows keypool 4630 however I know I set it to 5,000 even when I created this wallet.  I am not sure if keypool line in "getinfo" is showing the size of the keypool or is inaccurate?

Or maybe my understanding of keypool is flawed.  <- this would be bad because a lot of our wallet protection procedures are based on it.

Afaik getinfo should show the unused keys, which is strange for my number of 103 ^^.

Dia

[DeathAndTaxes]

Afaik getinfo should show the unused keys, which is strange for my number of 103 ^^.

Yeah very strange.

[Diapolo]

Afaik getinfo should show the unused keys, which is strange for my number of 103 ^^.

Yeah very strange.

Another wallet I use (mainnet) shows 101 ...

Dia

[Mushoz]

Great ideas guys! Looking forward to seeing something get implemented.
Personally I think DeathAndTaxes ideas are really good. No matter how annoying those messages are, proper backups are really needed to get Bitcoin to the masses. Losing coins due to computer crashes will turn people off, even if it's their own fault, so forcing backups is a good thing in my opinion. I also agree that the default pool size should be increased. Two more things come to mind though:

-Do you guys think a warning message should be displayed if someone is backing up a non-encrypted wallet.dat? People could lose their coins if they backup their unencrypted wallet to an insecure place.
-Do you guys think there should be a message of some kind that explains to backup the wallet to separate locations? It won't surprise me to see lots of people backing their wallet up to C:\Backups\ 

[Gavin Andresen]

RE: 101 or 103 unused keys in the keypool:  That's normal.  Here's the sequence of events that causes it:

+ You do something that requests a new key from the keypool. Several things do that (including the 'getinfo' RPC call -- it requests a key from the keypool so it can report the keypoololdest time).
+ The keypool automatically adds new keys so there are always at least 100 (by default)
+ The something you did returns the key back to the keypool, so now there are 100+1

I tend to have 104 keys in my keypools, because I do a lot of 4-core CPU mining on testnet-in-a-box setups, and the 4 miner threads each grab a keypool key that is released when bitcoind quits.

RE: better backup:  good idea. However, the keypool might not survive for much longer; they're likely to be replaced by Hierarchical Deterministic Wallets (see https://en.bitcoin.it/wiki/BIP_0032 ).

[Mushoz]

RE: 101 or 103 unused keys in the keypool:  That's normal.  Here's the sequence of events that causes it:

+ You do something that requests a new key from the keypool. Several things do that (including the 'getinfo' RPC call -- it requests a key from the keypool so it can report the keypoololdest time).
+ The keypool automatically adds new keys so there are always at least 100 (by default)
+ The something you did returns the key back to the keypool, so now there are 100+1

I tend to have 104 keys in my keypools, because I do a lot of 4-core CPU mining on testnet-in-a-box setups, and the 4 miner threads each grab a keypool key that is released when bitcoind quits.

RE: better backup:  good idea. However, the keypool might not survive for much longer; they're likely to be replaced by Hierarchical Deterministic Wallets (see https://en.bitcoin.it/wiki/BIP_0032 ).

Wow, that sounds even better. This means a single backup is sufficient to never lose any coins, correct? Ofcourse backups are still required to keep labels intact, but this would be way better than thread number 132497 with a person complaining he lost private keys to his funds. Awesome job, Gavin!

[TangibleCryptography]

RE: 101 or 103 unused keys in the keypool:  That's normal.  Here's the sequence of events that causes it:

+ You do something that requests a new key from the keypool. Several things do that (including the 'getinfo' RPC call -- it requests a key from the keypool so it can report the keypoololdest time).
+ The keypool automatically adds new keys so there are always at least 100 (by default)
+ The something you did returns the key back to the keypool, so now there are 100+1

I tend to have 104 keys in my keypools, because I do a lot of 4-core CPU mining on testnet-in-a-box setups, and the 4 miner threads each grab a keypool key that is released when bitcoind quits.

Excellent explanation and now it won't keep bugging me.

RE: better backup:  good idea. However, the keypool might not survive for much longer; they're likely to be replaced by Hierarchical Deterministic Wallets (see https://en.bitcoin.it/wiki/BIP_0032 ).

All I can say is .... YES PLEASE!

[etotheipi]

This is what deterministic wallets are for.  I hope the main devs are putting some priority on that.  (on a side note, I also want to upgrade my own wallet format in Armory and want to match the new Satoshi client wallets... if they ever get upgraded)

I am of the opinion that anything that requires persistent actions from the user will lead to failure.  Even fairly diligent users make mistakes -- they set up their backup drive once, but drive lettering gets switched for some reason and backups stop working, and the user is too lazy or doesn't even realize it needs to be corrected.  "Oh what's this error?  Eh, I'll deal with it later.".  Alternatively, for fear of losing your wallet, you backup to tons of different places, all the time, to be totally sure you got it, and now your wallets are scattered all over the place, and who knows if you can even find the latest one when you need it.  That can be a mess to sort out.  Not to mention a security issue to have your wallets everywhere...

I bet improper backups of non-deterministic wallets will lead to many more users losing coins than any online hacking/theft.  Yet, it's completely avoidable.  Use a deterministic wallet.  Back it up once, forever.  Never worry again.

On that note... @Gavin, any idea for when this might be implemented in Bitcoin-Qt?

[flower1024]

@Gavin: deterministic wallets sounds way better.

should i give the backup implementation a try? would be my first contact to bitcoind.
i've planned to attach an "IsBackuped"-Attribute to every address inside the keypool and to show a warning as there are now more (or only a few) "backuped" addresses left.

^^ do you think this is still a good idea? how fast is this bip going to be implemented?

[Diapolo]

@Gavin: deterministic wallets sounds way better.

should i give the backup implementation a try? would be my first contact to bitcoind.
i've planned to attach an "IsBackuped"-Attribute to every address inside the keypool and to show a warning as there are now more (or only a few) "backuped" addresses left.

^^ do you think this is still a good idea? how fast is this bip going to be implemented?

A quick and small patch doesn't hurt if it's a good one IMHO. Btw. IsInBackup sounds less weird!?

Dia