Bitcoin Forum
December 05, 2024, 03:35:41 AM *
News: Latest Bitcoin Core release: 28.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1]
  Print  
Author Topic: Key pool feature for safer wallet backup  (Read 8767 times)
satoshi (OP)
Founder
Sr. Member
*
qt
Offline Offline

Activity: 364
Merit: 7424


View Profile
October 09, 2010, 08:19:33 PM
Merited by ABCbits (1)
 #1

SVN rev 163 (ver 0.3.13.3) has the key pool feature.  Pre-generated new keys are aged in a queue before use, so that backups of wallet.dat hold keys you'll use in the future.

For now I made the default pool size 100.  It can be configured with -keypool=.  Be aware, it takes a little time to increase the pool size, so don't go crazy with it.  Disk space is about 1K per key.

I have not addressed the recovery side of this yet.  If you actually did restore an old wallet.dat, I think you may have to delete blk*.dat to rediscover your own transactions during the redownload.

I've only tested this moderately.  You might not want to use this for a website server until it's had some more testing.
nelisky
Legendary
*
Offline Offline

Activity: 1540
Merit: 1002


View Profile
October 09, 2010, 08:41:18 PM
 #2

Much appreciated, this feature will be Smiley

Now, is there/will there be a way to know how many free keys are in the pool at some moment in time, and trigger generation of a new pool of addresses in a determined time point?

The idea is to be able to schedule backups when new adresses are going to be generated, and to never use an address not already in a backup. These keys will be used for both change and new addresses, correct?

Thank you!
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 2311


Chief Scientist


View Profile WWW
October 17, 2010, 11:02:14 PM
Merited by ABCbits (2)
 #3

RE: how many free keys are in the pool:

By default, there are at least 100 free keys in the pool, always.

When a key is taken out, if the number of free keys drops below 100 (or the -keypool= number) another is generated.

Keys are put back if they're unused-- for example, a key is needed for every miner hashing thread, so if you're on a 4-core machine and turn on coin generation and then turn it back off you'll wind up with 104 keys in the free pool.

But for most people most of the time there will be exactly 100 free keys.


How often do you get the chance to work on a potentially world-changing project?
nelisky
Legendary
*
Offline Offline

Activity: 1540
Merit: 1002


View Profile
October 18, 2010, 12:02:52 AM
Merited by ABCbits (1)
 #4

RE: how many free keys are in the pool:

By default, there are at least 100 free keys in the pool, always.

When a key is taken out, if the number of free keys drops below 100 (or the -keypool= number) another is generated.

Keys are put back if they're unused-- for example, a key is needed for every miner hashing thread, so if you're on a 4-core machine and turn on coin generation and then turn it back off you'll wind up with 104 keys in the free pool.

But for most people most of the time there will be exactly 100 free keys.



So, 100 transactions from now and I'm in the same exact backup problem. It just moves the safety zone into the future "backup now and you'll be safe for X time" X being really small in a very busy service.

Is it possible to have hysteresis in the pool? once it reaches 20 it generates the other 80, or something or the kind? That and getfreekeys on the rpc side and I can make sure I don't need to do backups for every transaction, I don't need to do backups based on time and I'm 100% safe. Or I can do time based pooled backup and be 99% safe.

Even better, hysteresis as described and after every time the server generates new keys for the pool, it does a wallet backup with time signature. My backup job can simply detect new files as save them, a simple rsync would do the job in a guaranteed safe way.
LZ
Legendary
*
Offline Offline

Activity: 1722
Merit: 1072


P2P Cryptocurrency


View Profile
October 18, 2010, 01:43:31 AM
 #5

As I understood generated coins will not be in the key pool. Am I right?

My OpenPGP fingerprint: 5099EB8C0F2E68C63B4ECBB9A9D0993E04143362
theymos
Administrator
Legendary
*
Offline Offline

Activity: 5404
Merit: 13461


View Profile
October 18, 2010, 01:55:09 AM
 #6

As I understood generated coins will not be in the key pool. Am I right?

They will be. Everything is.

Every received IP transaction creates a new address (taken from the pool), so be careful if you are accepting those. Someone could empty your pool in a few minutes.

1NXYoJ5xU91Jp83XfVMHwwTUyZFK64BoAD
LZ
Legendary
*
Offline Offline

Activity: 1722
Merit: 1072


P2P Cryptocurrency


View Profile
October 18, 2010, 02:01:28 AM
Last edit: October 23, 2010, 10:03:46 AM by lzsaver
 #7

Okay. Seems I will see Generated string after 120 confirmations.

I am not accepting IP transactions. Thanks.

My OpenPGP fingerprint: 5099EB8C0F2E68C63B4ECBB9A9D0993E04143362
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 2311


Chief Scientist


View Profile WWW
October 18, 2010, 02:17:51 AM
Merited by ABCbits (1)
 #8

Backup every 30 sendtoaddress or generatenewaddress and you'll be fine-- you should always have at least 3 backup copies of all your keys.

If you're running a very busy service so backing up every 30 is too often, then run with -keypool=1000 and backup at least every 300 sends/generates.

I worry about bitcoin accumulating too many features and not doing any of them very well.  I suppose it wouldn't hurt to add an option so it automatically creates timestamped wallet backups... but should it erase old backups?  (if it doesn't, I KNOW people will forget to erase them and will be upset when their disk fills up with wallet backups and they're left to figure out how to clean up the resulting mess).  Should it encrypt them?  What should it do if an automatic time-stamped wallet backup fails?  When encrypted wallets are implemented, what should happen to old backups if the wallet encryption key is changed?

How often do you get the chance to work on a potentially world-changing project?
nelisky
Legendary
*
Offline Offline

Activity: 1540
Merit: 1002


View Profile
October 18, 2010, 11:02:31 AM
 #9

Backup every 30 sendtoaddress or generatenewaddress and you'll be fine-- you should always have at least 3 backup copies of all your keys.


You obviously understand that from the software perspective, the two approaches are different, right? Having a fixed trigger to backup is very easy and can be completely stateless, but backup every X whatevers means keeping a running count, harder to get redundant work with redundant data, etc.
If you're running a very busy service so backing up every 30 is too often, then run with -keypool=1000 and backup at least every 300 sends/generates.

I worry about bitcoin accumulating too many features and not doing any of them very well.  I suppose it wouldn't hurt to add an option so it automatically creates timestamped wallet backups... but should it erase old backups?  (if it doesn't, I KNOW people will forget to erase them and will be upset when their disk fills up with wallet backups and they're left to figure out how to clean up the resulting mess).  Should it encrypt them?  What should it do if an automatic time-stamped wallet backup fails?  When encrypted wallets are implemented, what should happen to old backups if the wallet encryption key is changed?


That's why I like simple, the kiss approach, within limits. The perfect approach for me would be manual triggering of key pool refilling, because that way the backup agent makes sure that there are enough keys from the last backup and, failing that, generates a new (variable size based on the average needed in the past) batch. This is very simple on the bitcoind side and very powerful for me.

I just need a way to know if the addresses on the wallet are covered by the last backup, and if there are still leftover addresses in the wallet, without depending on running counts and persistent data not stored in bitcoin directly.
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 2311


Chief Scientist


View Profile WWW
October 18, 2010, 11:49:30 AM
 #10

I just need a way to know if the addresses on the wallet are covered by the last backup, and if there are still leftover addresses in the wallet, without depending on running counts and persistent data not stored in bitcoin directly.

Keys in the keypool store when they're generated (and the oldest are always used first).   Asking "what's the oldest key in the key pool" seems reasonable, and I think it would give you what you want-- you could compare that timestamp to the timestamp of your last backup to see if you're covered.

Although if you're going to periodically check to see what the oldest timestamp is (or you're going to periodically check to see if bitcoin has written a new timestamped backup file or periodically check WHATEVER) then it seems simpler to me to just periodically always call backupwallet.  Disk space and bandwidth is cheap these days...

How often do you get the chance to work on a potentially world-changing project?
nelisky
Legendary
*
Offline Offline

Activity: 1540
Merit: 1002


View Profile
October 18, 2010, 01:30:28 PM
 #11

I just need a way to know if the addresses on the wallet are covered by the last backup, and if there are still leftover addresses in the wallet, without depending on running counts and persistent data not stored in bitcoin directly.

Keys in the keypool store when they're generated (and the oldest are always used first).   Asking "what's the oldest key in the key pool" seems reasonable, and I think it would give you what you want-- you could compare that timestamp to the timestamp of your last backup to see if you're covered.

Although if you're going to periodically check to see what the oldest timestamp is (or you're going to periodically check to see if bitcoin has written a new timestamped backup file or periodically check WHATEVER) then it seems simpler to me to just periodically always call backupwallet.  Disk space and bandwidth is cheap these days...


Yes, I will keep calling backupwallet regularly regardless, but how often do I do it? That's the answer this whole thing will be able to provide, as it's not a static thing. I can then check every minute but backup every day, unless I hit a peak usage and have to put an intermediate backup. And if these happen a few times in a row, I'll adjust the auto backup time frame.
bitcoinex
Sr. Member
****
Offline Offline

Activity: 350
Merit: 252


probiwon.com


View Profile WWW
October 22, 2010, 12:21:36 AM
Last edit: October 22, 2010, 12:47:20 AM by bitcoinex
 #12

How can I ensure that the pool is generated? debug.log

--pool is not listed in the "usage screen" bitcoind --help (0.3.14 beta)

New bitcoin lottery: probiwon.com
- Moжeт, ты eщё и в Heвидимyю Pyкy Pынкa вepyeшь? - Зaчeм жe вepoвaть в тo, чтo мoжнo нaблюдaть нeпocpeдcтвeннo?
mintymark
Sr. Member
****
Offline Offline

Activity: 286
Merit: 251


View Profile
August 05, 2011, 09:17:17 AM
 #13

It really does seem to me that the OP has a point here, and that control over keyy generation is very important and that the client does not do this in a satisfactory manner.

Once suggestion is to just generate a very large number, perhaps 20000 or even 200000 as a one off operation and leave it at that. Assume you will never generate any more. 20000 keys is about 15Mb. But it is a bit clunky. I tried this with keypool=10000 and when I generated a new address, there was a 10 minute delay and the wallet.dat grew, I then put keypool back to 1, and everything seemed fine, but I have not done a good test.

What happens if you put keypool=0 ?  Intuition suggests that this would generate a new address every time you ask for one, with none in the pool. Setting to 0 did not seem to give any strange behaviour.

What happens with keypool=-1 ? I tried this and could not get any strange behavior.

What I would like is that there is a setting where generating a new key just fails. So I could generate 100 or 1000 or 10000 keys and then set keypool=-1 and NO MORE WOULD BE GENERATED. Any attempt to generate new key would would take them from the pool or fail if the pool was empty.

Is there such a setting or combination of settings? I dont know, but I dont think so.

Can we have it?


[[ All Tips gratefully received!!  ]]
15ta5d1N8mKkgC47SRWmnZABEFyP55RrqD
bitplane
Sr. Member
****
Offline Offline

Activity: 321
Merit: 250

Firstbits: 1gyzhw


View Profile WWW
August 05, 2011, 11:41:23 AM
 #14

I'd personally prefer a cyclic pool of keys (along with an anonymity warning) rather than having to back up my wallet every time I make a transaction.
Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!