Bitcoin Forum
May 24, 2022, 03:34:52 AM *
News: Latest Bitcoin Core release: 23.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1] 2 »  All
  Print  
Author Topic: Increasing the number of keys in key pool  (Read 3733 times)
jl2012
Legendary
*
Offline Offline

Activity: 1792
Merit: 1012


View Profile
September 30, 2012, 08:03:27 AM
 #1

Again, someone lost >1300BTC due to misconception about wallet backup: https://bitcointalk.org/index.php?topic=110781.0

It might be difficult to implement deterministic wallet in the next release. However, it is very easy to increase the key pool from 100 keys to 10000 keys. Adding 2 or 0 zeros in the source code will solve 99% of the problem

Donation address: 374iXxS4BuqFHsEwwxUuH3nvJ69Y7Hqur3 (Bitcoin ONLY)
LRDGENPLYrcTRssGoZrsCT1hngaH3BVkM4 (LTC)
PGP: D3CC 1772 8600 5BB8 FF67 3294 C524 2A1A B393 6517
1653363292
Hero Member
*
Offline Offline

Posts: 1653363292

View Profile Personal Message (Offline)

Ignore
1653363292
Reply with quote  #2

1653363292
Report to moderator
1653363292
Hero Member
*
Offline Offline

Posts: 1653363292

View Profile Personal Message (Offline)

Ignore
1653363292
Reply with quote  #2

1653363292
Report to moderator
1653363292
Hero Member
*
Offline Offline

Posts: 1653363292

View Profile Personal Message (Offline)

Ignore
1653363292
Reply with quote  #2

1653363292
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1653363292
Hero Member
*
Offline Offline

Posts: 1653363292

View Profile Personal Message (Offline)

Ignore
1653363292
Reply with quote  #2

1653363292
Report to moderator
CIYAM
Legendary
*
Offline Offline

Activity: 1890
Merit: 1023


Ian Knowles - CIYAM Lead Developer


View Profile WWW
September 30, 2012, 08:23:27 AM
 #2

Actually it wouldn't matter how many keys were in the key pool in the case you have linked to as his backup was unencrypted then he encrypted his wallet and then sent BTC.

He lost the BTC because it was sent to a "change" address and that address (for security reasons) is not taken from the unencrypted pool but instead from a new pool that is created when you encrypt the wallet.

The lesson is to immediately backup after encrypting (and I believe the software does warn you that you need to).

With CIYAM anyone can create 100% generated C++ web applications in literally minutes.

GPG Public Key | 1ciyam3htJit1feGa26p2wQ4aw6KFTejU
runeks
Legendary
*
Offline Offline

Activity: 966
Merit: 1006



View Profile WWW
September 30, 2012, 08:35:36 AM
 #3

The lesson is to immediately backup after encrypting (and I believe the software does warn you that you need to).
I don't think it does. At least there's an open issue regarding this on the bug tracker: https://github.com/bitcoin/bitcoin/issues/1884
CIYAM
Legendary
*
Offline Offline

Activity: 1890
Merit: 1023


Ian Knowles - CIYAM Lead Developer


View Profile WWW
September 30, 2012, 08:41:27 AM
 #4

I don't think it does. At least there's an open issue regarding this on the bug tracker: https://github.com/bitcoin/bitcoin/issues/1884

Ouch - if that is true then certainly it needs to be changed (and the notice should probably be presented to the end user in a bold font with a bright color).

YOU NEED TO BACKUP YOUR ENCRYPTED WALLET BEFORE MAKING ANY TRANSACTION OR YOU COULD LOSE ALL YOUR BITCOINS!

With CIYAM anyone can create 100% generated C++ web applications in literally minutes.

GPG Public Key | 1ciyam3htJit1feGa26p2wQ4aw6KFTejU
runeks
Legendary
*
Offline Offline

Activity: 966
Merit: 1006



View Profile WWW
September 30, 2012, 09:43:25 AM
 #5

I just hacked together a quick patch in this pull request: https://github.com/bitcoin/bitcoin/pull/1890
Here's what the dialog looks like now:

If the devs wish they can make the text red.
CIYAM
Legendary
*
Offline Offline

Activity: 1890
Merit: 1023


Ian Knowles - CIYAM Lead Developer


View Profile WWW
September 30, 2012, 12:47:47 PM
 #6

Good stuff!

I think that should get the message across (red text is probably not necessary).

With CIYAM anyone can create 100% generated C++ web applications in literally minutes.

GPG Public Key | 1ciyam3htJit1feGa26p2wQ4aw6KFTejU
Stephen Gornick
Legendary
*
Offline Offline

Activity: 2506
Merit: 1004


View Profile
September 30, 2012, 07:39:29 PM
 #7

it is very easy to increase the key pool from 100 keys to 10000 keys. Adding 2 or 0 zeros in the source code will solve 99% of the problem

As has been described, a larger key pool wouldn't have solved the problem described where enabling wallet key encryption flushes the entire pool, but if you want to the key pool can size be controlled with a config setting:

 -keypool=<n>       Set key pool size to <n> (default: 100)
 
 - http://en.bitcoin.it/wiki/Running_Bitcoin

Unichange.me

            █
            █
            █
            █
            █
            █
            █
            █
            █
            █
            █
            █
            █
            █
            █
            █


Stephen Gornick
Legendary
*
Offline Offline

Activity: 2506
Merit: 1004


View Profile
September 30, 2012, 07:41:35 PM
 #8

I just hacked together a quick patch in this pull request: https://github.com/bitcoin/bitcoin/pull/1890

Should the instructions really be to "replace" your old backups?

Instead you might want to describe how no prior backups will have the new encrypted keys and thus it is recommended to make new backups now.  Or something to that effect.

Unichange.me

            █
            █
            █
            █
            █
            █
            █
            █
            █
            █
            █
            █
            █
            █
            █
            █


FreeMoney
Legendary
*
Offline Offline

Activity: 1246
Merit: 1014


Strength in numbers


View Profile WWW
September 30, 2012, 09:53:20 PM
 #9

That warning isn't quite right because it implies that the old backups won't be dangerous in the wrong hands, but they will until you have used all existing inputs.

Play Bitcoin Poker at sealswithclubs.eu. We're active and open to everyone.
BC12345
Newbie
*
Offline Offline

Activity: 57
Merit: 0


View Profile
September 30, 2012, 10:25:49 PM
 #10

1. +1 to this whole backup/encrypt wallet issue. I think this is really important, especially to people who are new with bitcoins (but apparently not only for them)

2. I do not think that flushing the key pool when the wallet is encrypted is a good idea. I understand why it's done, but when a program gives you the option to encrypt something I expect it to do right that: encrypt. I don't expect it to somehow modify/change my data.
kjj
Legendary
*
Offline Offline

Activity: 1302
Merit: 1016



View Profile
September 30, 2012, 10:45:41 PM
 #11

1. +1 to this whole backup/encrypt wallet issue. I think this is really important, especially to people who are new with bitcoins (but apparently not only for them)

2. I do not think that flushing the key pool when the wallet is encrypted is a good idea. I understand why it's done, but when a program gives you the option to encrypt something I expect it to do right that: encrypt. I don't expect it to somehow modify/change my data.

The keys existed on your disk in an unencrypted state.  They are not safe, they should not be used.  Marking them and generating new ones is the right thing to do.  Key encryption has been around for something like a year, and we are just now noticing that a few people have lost keys in strange situations.

17Np17BSrpnHCZ2pgtiMNnhjnsWJ2TMqq8
I routinely ignore posters with paid advertising in their sigs.  You should too.
Dabs
Legendary
*
Offline Offline

Activity: 3234
Merit: 1885


The Concierge of Crypto


View Profile
October 01, 2012, 06:50:52 AM
 #12

Actually, I prefer that the default keypool be made smaller. But that's just me, I don't have 100 transactions, and it is convenient for me to just take care of a dozen keys.

In my experiment, which anyone can duplicate, the newly encrypted wallet contains all the old keys AND 100 more new keys. The old keys are not useless, but the new keys will not yet have been backed up.

It's possible that the client just picked a key at random to send the change and it just so happens that it was not one of the old keys.

DeathAndTaxes
Donator
Legendary
*
Offline Offline

Activity: 1218
Merit: 1031


Gerald Davis


View Profile
October 01, 2012, 06:59:26 AM
 #13

Actually, I prefer that the default keypool be made smaller. But that's just me, I don't have 100 transactions, and it is convenient for me to just take care of a dozen keys.

In my experiment, which anyone can duplicate, the newly encrypted wallet contains all the old keys AND 100 more new keys. The old keys are not useless, but the new keys will not yet have been backed up.

It's possible that the client just picked a key at random to send the change and it just so happens that it was not one of the old keys.

Not exactly it is important to understand there is a difference between keys in the "active/used" wallet and keys in the keypool.

The keys (i.e. used keys) in the wallet transfer over.  Obviously they would have to or you would lose all coins associated with those keys.   The entire keypool is erased and new keys generated.  The Satoshi client always uses the next key in the keypool for change.   It is never up to chance.   Also a larger keypool doesn't increase your administrative burden.  If you are doing manual extraction/key work on a wallet the keypool represents "future" keys.  100 or 10,000 it doesn't really matter.  If you only have a dozen active keys you simply need those keys.  

TL/DR
Used/active keys =/= keypool.
The satoshi client always uses the "next" key from the keypool for change and new addresses.
Dabs
Legendary
*
Offline Offline

Activity: 3234
Merit: 1885


The Concierge of Crypto


View Profile
October 01, 2012, 07:44:58 AM
 #14

If the satoshi client discarded unused keys, then I would not have all the old unused keys inside the new encrypted wallet. My experiment had 108 old keys (I think I have 8 transactions). The new encrypted wallet has 208 keys, 108 from the old wallet including used and unused keys, and 100 more new keys.

I used bitcoin version 0.7 (the one just released) for Windows, and pywallet to export the keys from both the old unencrypted wallet, and the new encrypted wallet.

The difference between the two is I just encrypted the wallet, then I made a backup of the encrypted wallet.

Or maybe it's just my wallet since I created it when it was version 0.5 or so at the time (March 2012). Maybe the behavior for new wallets is different than for old wallets created with an old client? But I remember upgrading my wallet to a newer format (maybe it was version 0.6)

CIYAM
Legendary
*
Offline Offline

Activity: 1890
Merit: 1023


Ian Knowles - CIYAM Lead Developer


View Profile WWW
October 01, 2012, 10:20:30 AM
 #15

From what I've gathered it doesn't throw away those keys but after the encryption doesn't use them (at least for change addresses) either (I guess in the off chance somehow you managed to get funds sent to one of those addresses).

With CIYAM anyone can create 100% generated C++ web applications in literally minutes.

GPG Public Key | 1ciyam3htJit1feGa26p2wQ4aw6KFTejU
DeathAndTaxes
Donator
Legendary
*
Offline Offline

Activity: 1218
Merit: 1031


Gerald Davis


View Profile
October 01, 2012, 12:51:09 PM
 #16

If the satoshi client discarded unused keys, then I would not have all the old unused keys inside the new encrypted wallet. My experiment had 108 old keys (I think I have 8 transactions). The new encrypted wallet has 208 keys, 108 from the old wallet including used and unused keys, and 100 more new keys.

I used bitcoin version 0.7 (the one just released) for Windows, and pywallet to export the keys from both the old unencrypted wallet, and the new encrypted wallet.

The difference between the two is I just encrypted the wallet, then I made a backup of the encrypted wallet.

Or maybe it's just my wallet since I created it when it was version 0.5 or so at the time (March 2012). Maybe the behavior for new wallets is different than for old wallets created with an old client? But I remember upgrading my wallet to a newer format (maybe it was version 0.6)

Weird.  Maybe CIYAM is right.   To avoid losing funds in the event a key was used in a non-standard way (say from a keydump) it probably doesn't delete them just marks them so they will never be used in future tx by the client.  

I didn't notice another part of your question.  You can make the keypool smaller if you wish.  100 is the default (IMHO should be larger given the trivial amount of space) but you can set it to any value even 0.  With keypool=0 you have no keypool the only keys in the wallet will be existing keys and new keys (when needed) will be "created on the fly".  NOOB WARNING (not you dabs but anyone who happens to read this): with a keypool of 0 you would need an updated backup after every tx involving a new key ("new address button", or spend w/ change) to avoid the risk of irrecoverable loss of funds.

Still all of this is somewhat academic.  Deterministic wallets are the future.  The ability to backup and/or print an encrypted deterministic seed and thus avoid a whole category of potential data loss scenarios provides huge value in making bitcoin "easier to use" while providing no real risk/downside. 
Dabs
Legendary
*
Offline Offline

Activity: 3234
Merit: 1885


The Concierge of Crypto


View Profile
October 01, 2012, 01:52:55 PM
 #17

The risk with deterministic wallets is obvious. If someone cracks the deterministic seed used, he has all your money, now and into the future, for that particular wallet. With a randomly generated wallet, someone would have to crack or bruteforce every private key. Of course, if someone cracks the passphrase used for encrypted wallets, he has everything in your current wallet up to the key pool maximum (or future 100 transactions) but not your 101st transaction. (for default keypool=100).

I say, give the user the choice. I prefer randomly generated wallets.

I think an option for the client would be to set a maximum number of keys used, so that, for example, if you have already 1000 keys, and you have made 1000 transactions, the client would just re-use an old key. This way, you can keep a backup of your wallet once (after generating 1000 keys).

Take the example of BitcoinSpinner. It only uses 1 key. That's an extreme, but you can have the satoshi client fix your maximum keys to an arbitrary number of your choice (and probably force it to pre-generate that number already.)

To prevent the client from generating too many keys and taking over your computer, maybe the satoshi client will also have a hard coded limit on what the maximum wallet size would be, like 65000 keys is pretty large for one wallet, for one person.

kjj
Legendary
*
Offline Offline

Activity: 1302
Merit: 1016



View Profile
October 01, 2012, 02:01:34 PM
 #18

The risk with deterministic wallets is obvious. If someone cracks the deterministic seed used, he has all your money, now and into the future, for that particular wallet. With a randomly generated wallet, someone would have to crack or bruteforce every private key. Of course, if someone cracks the passphrase used for encrypted wallets, he has everything in your current wallet up to the key pool maximum (or future 100 transactions) but not your 101st transaction. (for default keypool=100).

I say, give the user the choice. I prefer randomly generated wallets.

I prefer random keys too, but realistically speaking, cracking the master seed is equivalent to either 1) breaking all EC math, including bitcoin, or 2) stealing your wallet and cracking your password.  (I'm pretty sure armory encrypts the master seed at least as well as the reference client encrypts the private keys.)

In either case, you have big problems.  EC wallets have plenty of advantages, and only insignificant theoretical disadvantages.  They should probably be the default.

17Np17BSrpnHCZ2pgtiMNnhjnsWJ2TMqq8
I routinely ignore posters with paid advertising in their sigs.  You should too.
runeks
Legendary
*
Offline Offline

Activity: 966
Merit: 1006



View Profile WWW
October 01, 2012, 09:26:13 PM
 #19

I just hacked together a quick patch in this pull request: https://github.com/bitcoin/bitcoin/pull/1890

Should the instructions really be to "replace" your old backups?

Instead you might want to describe how no prior backups will have the new encrypted keys and thus it is recommended to make new backups now.  Or something to that effect.

That warning isn't quite right because it implies that the old backups won't be dangerous in the wrong hands, but they will until you have used all existing inputs.
I considered making the message longer and more descriptive. But I'm afraid people will just ignore it if a giant wall of text appears.

Please feel free to compose a better message, and post it on the Github page for the issue. I think you are right, but I'm not quite sure how to get it across without the message becoming too verbose.
Dabs
Legendary
*
Offline Offline

Activity: 3234
Merit: 1885


The Concierge of Crypto


View Profile
October 02, 2012, 03:01:03 AM
 #20

Make it simple:

Warning: After encryption, make a backup of your new wallet and discard all previous backups. See *insert*webpage* for detailed explanation.

You have 17.29382 bitcoins from your previous unencrypted wallet, would you like to send them to a new address in your new encrypted wallet? (click yes / no / maybe / i'm not sure.)

Pages: [1] 2 »  All
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!