Need more clarification on backing up wallet

[yyohko]

Hi, I would like to know if we need to keep taking backups of our wallet.dat as we keep having transactions or we only need to backup once for the entire lifetime?

Let's say I backup my wallet.dat in January, have some transactions in February, then my PC crashed in March.
When I restore my wallet.dat, are my February transactions gone?

[ThomasV]

if you use a deterministic wallet, you can back it up once and for all.
if you use the official client, then you need to back it up every 100 transactions

[DMetcalfe92]

I don't think so. As the BTC Wallet downloads new blocks (blk001.dat & blkindex.dat) it will see your bitcoin address (as stored in your wallet.dat) and update the transactions?

I think the way it works is, All transactions EVER made, are in the block container, then bitcoin.exe just filters the results by wallet.dat

So no matter what, you'll always have your transaction history as it's on the interwebs!

[yyohko]

ThomasV, How do I use a deterministic wallet?
Also, what will happen if I fail to backup every 100 transactions? (when my PC crash)

[jake262144]

I don't think so. As the BTC Wallet downloads new blocks (blk001.dat & blkindex.dat) it will see your bitcoin address (as stored in your wallet.dat) and update the transactions?

Already discussed over, and over, and over again, e.g. here
Do your research first, post advice later.

The traditional wallet contains, among other goodies, private keys for your bitcoin addresses. A pool of 100 yet-unused keys is stored inside it by default.
Every transaction you send, some of the incoming transactions, and every receive address you generate inside the client use one of those pre-generated keys which causes the key-pool to be repopulated with new keys.
A sufficiently old wallet backup will be out of keys which might render some of your bitcoins unaccessible.


Deterministic wallets are only supported by alternative bitcoin clients.

[ThomasV]

ThomasV, How do I use a deterministic wallet?
Also, what will happen if I fail to backup every 100 transactions? (when my PC crash)

here are the deterministic wallets I know of:
 - Electrum (lightweight client written by me) https://bitcointalk.org/index.php?topic=50936
 - Armory (full client)
 - Bitcoinspinner (lightweight client for mobile phones)

if you fail to backup your wallet with the official client, you might lose some coins when your PC crashes.
but I guess the client warns you when its pool of addresses gets deprecated.

[jake262144]

if you fail to backup your wallet with the official client, you might lose some coins when your PC crashes.
but I guess the client warns you when its pool of addresses gets deprecated.

If it only were that simple... The pool is always being repopulated, no problem there. Nothing for the client to warn about.
The problem arises when one has to rely on a very old wallet backup missing some of the recently-generated keys.

[ThomasV]

if you fail to backup your wallet with the official client, you might lose some coins when your PC crashes.
but I guess the client warns you when its pool of addresses gets deprecated.

If it only were that simple... The pool is always being repopulated, no problem there. Nothing for the client to warn about.
The problem arises when one has to rely on a very old wallet backup missing some of the recently-generated keys.

exactly. but I was under the impression that the client warns you when the pool gets repopulated.
if it doesn't, how do you know when you need to back up?

[DeathAndTaxes]

if you fail to backup your wallet with the official client, you might lose some coins when your PC crashes.
but I guess the client warns you when its pool of addresses gets deprecated.

If it only were that simple... The pool is always being repopulated, no problem there. Nothing for the client to warn about.
The problem arises when one has to rely on a very old wallet backup missing some of the recently-generated keys.

exactly. but I was under the impression that the client warns you when the pool gets repopulated.
if it doesn't, how do you know when you need to back up?

It doesn't. The pool is ALWAYS being repopulated.  The client also has no idea when your last backup was.

As example might help:
Your keypool by default has 100 keys.  These are unused keys, the "next 100".
You make a backup today.  The backup contains all existing transactions plus the next 100 into the future.
If you lose your wallet.dat in the future as long as no more than 100 new addresses have been used you lose nothing.
If >100 new addresses have been used any coins at any addresses beyond the 100 are lost forever.

EVERYTIME you click [New Address] button or send funds (new address used for change) one address from keypool is used.  The client then puts 1 new address into the keypool so the keypool size doesn't change however the addresses in it do.  Thus your backup eventually will be out of date.  

You can set a larger keypool and honestly given how small keys are I don't know why client doesn't use a more "noob safe" number like 500 keys in the keypool by default.

[ThomasV]

if you fail to backup your wallet with the official client, you might lose some coins when your PC crashes.
but I guess the client warns you when its pool of addresses gets deprecated.

If it only were that simple... The pool is always being repopulated, no problem there. Nothing for the client to warn about.
The problem arises when one has to rely on a very old wallet backup missing some of the recently-generated keys.

exactly. but I was under the impression that the client warns you when the pool gets repopulated.
if it doesn't, how do you know when you need to back up?

It doesn't the pool is ALWAYS being repopulated.  The client also has no idea when your last backup was.

As example might help.

Your keypool by default has 100 keys.  These are unused keys, the "next 100".

You make a backup today.  The backup contains all existing transactions plus the next 100 into the future.

EVERYTIME you click NEW ADDRESS or receive funds one address from keypool is used and the client automatically puts 1 new key into the keypool.  So as long as your backup occured within the last 100 "new keys" being used you are safe.

You can set a larger keypool and honestly given how small keys are I don't know why client doesn't use a more "noob safe" number like 500 keys in the keypool by default.

This is not how I thought it worked.
Given that 100 addresses are pre-generated, I thought it would wait until they are all used until it generates another batch of 100.
This would be way safer, because you could warn the user that it is time to do another backup.

[DeathAndTaxes]

This is not how I thought it worked.
Given that 100 addresses are pre-generated, I thought it would wait until they are all used until it generates another batch of 100.
This would be way safer, because you could warn the user that it is time to do another backup.

No problem.  Keypool confuses everyone at least once. 
A non-refilling pool would require backups exactly at every 100 transaction increments and forgetting would make your client operating without backup immediately.  The pool continual refill method is better because you can make backups at anytime and regain 100 (or 1000) addresses of future protection.  

One way to combine both methods is to have a backup button in the client.  Click backup, client makes a copy of wallet.dat and records the last valid key in keypool.  Thus it could warn the user when last backup is out of date. 

I run with a keypool of 1000 keys.  Each key takes up about 1KB of space so my backup is ~1MB larger but ensures than any backup in last 1000 addresses is valid.  Given the trivial cost to store 1MB it seems silly to run w/ a smaller keypool.

[jake262144]

...This would be way safer, because you could warn the user that it is time to do another backup.

Way safer? No way
Your approach does not resolve the issue of having to fall back to an ancient backup of the wallet file.

It's the end user's responsibility to properly manage wallet backups.
I can't say this is the prudent approach as only a minority of users can be expected to follow a secure backup routine.

EDIT::*nods head* Hello DAT. I requested that auto backups be done by default once the wallet has been encrypted but the devs seem to have turned the deaf ear.

[ThomasV]

... only a minority of users can be expected to follow a secure backup routine.

are you saying that the official client is written for a minority of users?
why doesn't it have a deterministic wallet?

[DeathAndTaxes]

... only a minority of users can be expected to follow a secure backup routine.

are you saying that the official client is written for a minority of users?
why doesn't it have a deterministic wallet?

The official client likely will never be casual user friendly.  It is the reference design.  It tests out new functionality but it likely will never be the best client for casual users.  Personally I think this is a good thing.  We don't want one defacto client. 

[yyohko]

Er... thanks guys, this is suddenly too much information for me to digest.
So basically if I have no transactions or not expecting any, my backup will be good for years?
(I am using the official client)

[jake262144]

are you saying that the official client is written for a minority of users?
why doesn't it have a deterministic wallet?

Unfortunately, such seems to be the truth. The official client is convoluted about the most important aspects. It buries the wallet.dat file where the average non-techie user can't find it, it does ZERO backups, and offers NO encryption by default. I raised all those issues at least twice but no one seems to want to listen.
Yeah, why didn't the devs (or any other user) write the code and merge it into the client? I suspect insufficient motivation.

[ThomasV]

... only a minority of users can be expected to follow a secure backup routine.

are you saying that the official client is written for a minority of users?
why doesn't it have a deterministic wallet?

The official client likely will never be casual user friendly.  It is the reference design.  It tests out new functionality but it likely will never be the best client for casual users.  Personally I think this is a good thing.  We don't want one defacto client. 

well, the official Bitcoin client is the defacto client proposed at bitcoin.org
it does not look like their policy is to redirect noobs to more user-friendly clients.

[jake262144]

Er... thanks guys, this is suddenly too much information for me to digest.
So basically if I have no transactions or not expecting any, my backup will be good for years?
(I am using the official client)

Pretty much.
I suggest you encrypt the wallet (Settings -> Encrypt Wallet) and do a backup once a month. Just don't store the backups on the same machine.

[DeathAndTaxes]

... only a minority of users can be expected to follow a secure backup routine.

are you saying that the official client is written for a minority of users?
why doesn't it have a deterministic wallet?

The official client likely will never be casual user friendly.  It is the reference design.  It tests out new functionality but it likely will never be the best client for casual users.  Personally I think this is a good thing.  We don't want one defacto client. 

well, the official Bitcoin client is the defacto client proposed at bitcoin.org
it does not look like their policy is to redirect noobs to more user-friendly clients.

No but in my opinion Bitcoin isn't casual user ready yet.  There is a reason the major version number is 0.

An analogy.  Last time you went looking for a credit card did you go to VISA.COM?

The reality is the official client is a reference implementation.  A basis for designing other client and something to reference.  It likely will never be casual user friendly.  I don't think that is a deal killer.  No client is going to be optimal for every user and the Satoshi client has the obligation of being the reference model.

[ThomasV]

Er... thanks guys, this is suddenly too much information for me to digest.
So basically if I have no transactions or not expecting any, my backup will be good for years?
(I am using the official client)

yes.
do not forget to wait until the blockchain is fully synchronized everytime you use it; if you use it rarely the synchronization will take some time.