Help me understand how private keys work in wallet.dat

[BusyBeaverHP]

I read that wallet.dat in Bitcoin Core has 100 private keys in it.

I also know that everytime you spend, a private key is used up by mechanics of sending and reallocation. For simple assumption, suppose that I've used my 99th private key, what would actually happen once the keypool refreshes itself?

Example: Suppose I have 2 BTC stored in a wallet.dat that already has 99 transactions, what would happen when the keypool refresh itself?

[cp1]

It keeps 100 unused keys.  So every time you use one it will generate another one.  You should back it up frequently.

[DannyHamilton]

I read that wallet.dat in Bitcoin Core has 100 private keys in it.

Starts with a default of 100 private keys.  Additional keys are generated as needed.

I also know that everytime you spend, a private key is used up by mechanics of sending and reallocation. For simple assumption, suppose that I've used my 99th private key, what would actually happen once the keypool refreshes itself?

Example: Suppose I have 2 BTC stored in a wallet.dat that already has 99 transactions, what would happen when the keypool refresh itself?

The keypool remains refreshed all the time.  It acts like a queue.  Each time the software needs a new address and private key (if you request a new receiving address, or if the software needs an address for the change from the transaction), it takes the next one from the front of the queue.  At the same time, it generates a new address and private key and adds it to the back of the queue.

[odolvlobo]

It keeps 100 unused keys.  So every time you use one it will generate another one.  You should back it up frequently.

You don't need to back it up frequently. You only need to back it up as often as every 100 keys. One key is used every time you send bitcoins and one is used every time you allocate a new key to receive bitcoins.

[DannyHamilton]

It keeps 100 unused keys.  So every time you use one it will generate another one.  You should back it up frequently.

You don't need to back it up frequently.

I suppose that depends on how many transactions you are sending, how many receiving addresses you are generating, and what you mean when you say "frequently".

You only need to back it up as often as every 100 keys. One key is used every time you send bitcoins and one is used every time you allocate a new key to receive bitcoins.

While this is technically true, I tend to be concerned about the possibility that the most recent backup is unusable (for example, backing up a corrupted file before realizing that it is corrupted).

Instead, I prefer to back up every 25 keys, and keep the 3 most recent backups.  If you use a new address for every transaction (like you should), and you average 4 transactions per day, then backing up weekly would make sense.  Some people might use the word "frequent" to describe weekly backups.

[blacksails]

Is this something that a "normal" user needs to know, or is this more advanced stuff?
And why would you need 100 unused keys when it generates a new one when you need it anyway?

[cp1]

Is this something that a "normal" user needs to know, or is this more advanced stuff?
And why would you need 100 unused keys when it generates a new one when you need it anyway?

You'd have to back it up every transaction otherwise.

[DannyHamilton]

Is this something that a "normal" user needs to know, or is this more advanced stuff?
And why would you need 100 unused keys when it generates a new one when you need it anyway?

A "normal" user of the Bitcoin-Qt (or Bitcoin Core) wallet needs to know that they have to backup their wallet regularly.

Otherwise, they will lose all their bitcoins if their hard drive crashes (or their computer is stolen, burns in a fire, or is destroyed in a flood, tornado, or earthquake).

Since "regularly" can mean different things to different people, it would be good for a "normal" user of the Bitcoin-Qt (or Bitcoin Core) wallet to keep track of how many new addresses they've used (including change addresses) and backup their wallet every 25 addresses or so. Then keep the most recent 3 backups in 3 separate secure locations.

If you don't ever store enough bitcoins to care whether they are permanently lost, then the backups are not important.

[blacksails]

Is this something that a "normal" user needs to know, or is this more advanced stuff?
And why would you need 100 unused keys when it generates a new one when you need it anyway?

A "normal" user of the Bitcoin-Qt (or Bitcoin Core) wallet needs to know that they have to backup their wallet regularly.

Otherwise, they will lose all their bitcoins if their hard drive crashes (or their computer is stolen, burns in a fire, or is destroyed in a flood, tornado, or earthquake).

Since "regularly" can mean different things to different people, it would be good for a "normal" user of the Bitcoin-Qt (or Bitcoin Core) wallet to keep track of how many new addresses they've used (including change addresses) and backup their wallet every 25 addresses or so. Then keep the most recent 3 backups in 3 separate secure locations.

If you don't ever store enough bitcoins to care whether they are permanently lost, then the backups are not important.

Okay, that cleared it up pretty good. Thank you!

[boumalo]

It is still a bit complicated to safely keep bitcoins on your computer long term

[cp1]

If you use a deterministic wallet such as electrum then you don't need to worry about how many keys you've used.  It will keep track of as many as you want using a seed phrase.  Just write down that phrase and it's all you need.

[far9090]

https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&cad=rja&uact=8&ved=0CDYQFjAB&url=https%3A%2F%2Fen.bitcoin.it%2Fwiki%2FHow_to_import_private_keys&ei=fo5XU-uiFYuP5ASBzoDABg&usg=AFQjCNEcOmZQ89bxEmtCKgfzlWTsIs7s9w&bvm=bv.65177938,d.bGE

[Tammy Chan]

https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&cad=rja&uact=8&ved=0CDYQFjAB&url=https%3A%2F%2Fen.bitcoin.it%2Fwiki%2FHow_to_import_private_keys&ei=fo5XU-uiFYuP5ASBzoDABg&usg=AFQjCNEcOmZQ89bxEmtCKgfzlWTsIs7s9w&bvm=bv.65177938,d.bGE

This messy thing is indeed just https://en.bitcoin.it/wiki/How_to_import_private_keys, and it doesn't really help answering OP's question.

[Tammy Chan]

If you use a deterministic wallet such as electrum then you don't need to worry about how many keys you've used.  It will keep track of as many as you want using a seed phrase.  Just write down that phrase and it's all you need.

Exactly.

And just a side note, you could set the keypool size to whatever you want with "-keypool=<n>".
So, if you don't want to make backup so "frequently", you could use the argument to set a larger keypool.

[DeathAndTaxes]

They client doesn't wait until the keypool is exhausted before refreshing.  It refreshes the keypool whenever possible if the current size is less than the preset seize. 

If the wallet is not encrypted the keypool refreshes each time a new key is used.
If the wallet is locked (encrypted) the client will refresh the keypool the next time the wallet is unlocked.  
This means in situations where the wallet is not routinely unlocked (like a server watching wallet) they keypool will eventually be exhausted.  

You can get the current size of the keypool by the rpc command getinfo.

Eventually the Bitcoin-core wallet will probably use deterministic wallets making the concept of a keypool obsolete.

[BusyBeaverHP]

With the keypool queuing 100 unused private keys after every transaction, does this mean that a wallet will grow in size (because of used private/public keypairs) after every transaction?

One wallet example:

1. Received BTC (Generated new request code)
Wallet total: 1 Pub - 1 Priv - 100 unused Priv

2. Spent BTC
Wallet total: 2 Pub - 2 Priv - 100 unused Priv

3.  Received BTC (Generated new request code)
Wallet total: 3 Pub - 3 Priv - 100 unused Priv

4. Spent BTC
Wallet total: 4 Pub - 4 Priv - 100 unused Priv

... and so on...

Is this correct?

[odolvlobo]

With the keypool queuing 100 unused private keys after every transaction, does this mean that a wallet will grow in size (because of used private/public keypairs) after every transaction?
...

Is this correct?

That is correct.

[deepceleron]

If you plan on making a long-term backup, for the purpose of safeguard against data loss, after encrypting with a passphrase, you might consider starting Bitcoin once with a large keypool option such as bitcoin-qt -keypool=2000. This will fill the wallet with future keys that will keep your backup from becoming obsolete for a long time.

[BusyBeaverHP]

If you plan on making a long-term backup, for the purpose of safeguard against data loss, after encrypting with a passphrase, you might consider starting Bitcoin once with a large keypool option such as bitcoin-qt -keypool=2000. This will fill the wallet with future keys that will keep your backup from becoming obsolete for a long time.

Good idea. Do I change it in the shortcut properties, or somewhere within the program itself?

Another question: If I kept my wallet in cold storage, and I deposit more coins to a known address in that cold storage wallet, does that change the number of keys in use? How would that affect the existing unused keypool?

[DeathAndTaxes]

If you plan on making a long-term backup, for the purpose of safeguard against data loss, after encrypting with a passphrase, you might consider starting Bitcoin once with a large keypool option such as bitcoin-qt -keypool=2000. This will fill the wallet with future keys that will keep your backup from becoming obsolete for a long time.

Good idea. Do I change it in the shortcut properties, or somewhere within the program itself?

Neither since you only need to set it once.  In windows the easiest way is probably use the command prompt and type the command exactly as shown above.  Once the keypool is set loading it in the future won't unset it so there is no need to add it to a shortcut.  Understand it can take a while to generate a large number of keys and there will be no progress indicated by the system so just be patient eventually once the keypool is expanded the client will load normally.

Another question: If I kept my wallet in cold storage, and I deposit more coins to a known address in that cold storage wallet, does that change the number of keys in use? How would that affect the existing unused keypool?

It wouldn't use a new address thus it won't change the number of available keys in the keypool.