When to backup / What to backup

[nelisky]

Hey, so making my Lottery more disaster resilient, and taking on satoshi's backupwallet feature, I just wanted to double check I'm doing the right thing (tm).

My understanding is that the only important information on the whole set of bitcoin client data files is the private keys, one per incoming address. And all of these are stored in wallet.dat. Everything else, including the transactions themselves, is stored in every other bitcoin client running on the network. Is this correct?

Now, when does the information on the wallet.dat change, in a way that it makes our last backup ineffective? I'm guessing on every sent transaction, as bitcoin automatically creates a new receiving address for the transaction change portion, and everytime we create a new address.
Incoming transactions don't create new keys, do they? So I can safely receive without creating a new backup...

As a side thought, the import / export of keys has been mentioned now and then on the threads I follow. Would it be possible / practical to export all new keys from a wallet (passing an id from last backup, or the last key from the last backup) so we have granularity, and then be able to recreate a wallet by importing each of the pieces?

webservices in mind, this (efficient and effective backup) is the most critical point imho.

[theymos]

You need to back up after sending bitcoins and making new addresses.

The wallet contains local copies of all sent and received transactions. If the network loses one of your transactions, you'll rebroadcast it. Transactions shouldn't ever be lost if you're waiting for a few confirmations, but if you're paranoid you might also want to back up after received transactions.

[nelisky]

You need to back up after sending bitcoins and making new addresses.

Check.

The wallet contains local copies of all sent and received transactions. If the network loses one of your transactions, you'll rebroadcast it. Transactions shouldn't ever be lost if you're waiting for a few confirmations, but if you're paranoid you might also want to back up after received transactions.

Ok, so there are 2 elements in play here, addresses and transactions. Addresses are safe so as long as I do as described above, but transactions may be lost? Outgoing transactions are not a problem, obviously, as loosing them means only that the coins don't go out, so using a previous backup would show the correct amount before transaction.
But incoming transactions could be a problem? So someone sends me coins, I receive the transaction and then loose my wallet. I still have the address and key, so shouldn't the transaction be replayed when I parse all blocks again?

The means by which transactions get transported are the blocks, right? There's no out of band communicating of transactions between clients?

[theymos]

I still have the address and key, so shouldn't the transaction be replayed when I parse all blocks again?

Due to a bug or an attack on the network, blocks might be rewritten. Honest generators will rebroadcast your transaction for you if this happens, but this system might break down as well. It's very unlikely, of course, but you might want to account for this in situations requiring very high security. I wouldn't bother with it if I were you; I just wanted to mention the possibility.

The means by which transactions get transported are the blocks, right? There's no out of band communicating of transactions between clients?

That's right.

[nelisky]

I still have the address and key, so shouldn't the transaction be replayed when I parse all blocks again?

Due to a bug or an attack on the network, blocks might be rewritten. Honest generators will rebroadcast your transaction for you if this happens, but this system might break down as well. It's very unlikely, of course, but you might want to account for this in situations requiring very high security. I wouldn't bother with it if I were you; I just wanted to mention the possibility.

The means by which transactions get transported are the blocks, right? There's no out of band communicating of transactions between clients?

That's right.

I have a new usecase that is somewhat related to this thread... I have multiple bitcoin clients running, on multiple platforms. Specifically, I have one GUI based Mac client that holds addresses I have advertised, and I was trying to get rid of it.

The funny thing is that wallet.dat, if just moved to another client (even a custom compiled bitcoind on the same machine) simple refuses to work, screaming at me about corrupted databases and such. There's probably some easy fix for that but, what if I want to take 2 of my addresses out of that wallet and put them on another wallet, or on my mybitcoin account? I can transfer the balance, sure, but say I am receiving donations on that address and can't really ask people to change the address?

So what I would like to do is to export the privkey for one address in one wallet and then import it into another wallet. I know it's not implemented, but is it technically doable? I may take a stab at doing this, but just wanted to avoid a wild goose chase.

[theymos]

It's possible. Bitcointools can give you the public/private keys (you need to adjust it to show the non-abbreviated keys), but you'll have to do insertion yourself. See AddKey in main.cpp.

[Stephen Gornick]

I still have the address and key, so shouldn't the transaction be replayed when I parse all blocks again?

Due to a bug or an attack on the network, blocks might be rewritten. Honest generators will rebroadcast your transaction for you if this happens, but this system might break down as well. It's very unlikely, of course, but you might want to account for this in situations requiring very high security. I wouldn't bother with it if I were you; I just wanted to mention the possibility.

The means by which transactions get transported are the blocks, right? There's no out of band communicating of transactions between clients?

That's right.

I'm still struggling to understand what happens in a fairly typical scenario:

Day 1: New Bitcoin install and wallet.dat created
No transactions
Day 2: Incoming 10 BTC
Day 3: Wallet.dat backed up to flash drive, stored in safe.
Day 4: Outgoing 4 BTC
Day 5: Incoming 5 BTC
Day 6: Hard drive crashed
Day 7: New hard drive, new install of Bitcoin
-------------------------------------
So ..., what steps are next, and is there anything that I lose?

[nelisky]

If you're using the latest bitcoin client, nothing was lost as there's a pool of 100 addresses pre-created when you first started.

With the old client what would happen is:
- incoming 10 -> address X
- backup -> address X is safe
- outgoing 4 -> 10 out of address X, 4 to some external address, 6 into new address Y
- incoming 5 (I'm assuming to address X)
- crash
- recover address X which has 5 coins, address Y and its 6 coins lost.

[Stephen Gornick]

Thank you nelisky, that clears up the part that I wasn't understanding -- as long as my addresses get backed up, I won't lose any incoming Bitcoin sent to those addresses.  So if the first 100 Bitcoin addresses are generated when the Bitcoin client is created, how do I know when more are generated, and thus need to make another backup of my wallet.dat?

[nelisky]

Thank you nelisky, that clears up the part that I wasn't understanding -- as long as my addresses get backed up, I won't lose any incoming Bitcoin sent to those addresses.  So if the first 100 Bitcoin addresses are generated when the Bitcoin client is created, how do I know when more are generated, and thus need to make another backup of my wallet.dat?

Well, I haven't looked at the code yet, but from what was explained to me there's an initial pool of 100 and every time an address is taken out of that pool, a new one is inserted. So you don't have a simple way of knowing you used X% of the pool from last backup. You get a new address taken out every time you send coins, or you create an address manually, or you generate a block, so as long as you know approx how many of these actions you do in a certain period of time you can get a notion of the safe timespan between backups.

[Gavin Andresen]

Designing and implementing user-friendly wallet backup for the GUI bitcoin seems like it would be a great little project for somebody who knows C++ and knows, or is willing to learn, wxWidgets programming.  The non-GUI code already has the 'backupwallet' RPC method, and 'getinfo' will tell you the age of the oldest key in your keypool.

For extra credit, the code could keep track of the last successful backup and warn you that it is time to backup again when the keypool age gets close to the last backup time.

[nelisky]

Designing and implementing user-friendly wallet backup for the GUI bitcoin seems like it would be a great little project for somebody who knows C++ and knows, or is willing to learn, wxWidgets programming.  The non-GUI code already has the 'backupwallet' RPC method, and 'getinfo' will tell you the age of the oldest key in your keypool.

For extra credit, the code could keep track of the last successful backup and warn you that it is time to backup again when the keypool age gets close to the last backup time.

How does the oldest key help in deciding when to backup? Looking for a timestamp newer than the last backup would mean that we're already running with non-backed up keys, or am I reading this wrong?

We could say 'backup if oldest is less than a week old' or something like that, but I assume that is never a replacement for regular wallet backups.

[theymos]

How does the oldest key help in deciding when to backup? Looking for a timestamp newer than the last backup would mean that we're already running with non-backed up keys, or am I reading this wrong?

If your last backup was before "keypoololdest", you need to back up again. The time will get progressively closer to your latest backup as you use keys, and you should be able to back up in advance.

[balboah]

Why not link the file to a dropbox account?
As soon as it changes, it gets uploaded to dropbox

[Stephen Gornick]

Linking an unencrypted wallet.dat to Dropbox?  You might want to consider doing that.

I have Dropbox on both my desktop and laptop.  I don't want my desktop's wallet.dat anywhere near my laptop.

Occasionally I'll log into my Dropbox from someone else's computer, to pull a saved document.  With my wallet on dropbox I would be even more worried about them having pasword-capturing spyware.

And even assuming I trust everything is secure on my end, I must then trust that staff (and contractors ?) at Dropbox who have at least read-only access to storage won't scan for all wallet.dat files?

Now instead of linking to the wallet.dat, I would periodically create an encrypted copy of the wallet (wish that were automatic or part of the client) and then copy that to Dropbox.  Then, to get that to Dropbox on Windows I'd use something ike MS SyncToy.

[balboah]

Linking an unencrypted wallet.dat to Dropbox?  You might want to consider doing that.

I have Dropbox on both my desktop and laptop.  I don't want my desktop's wallet.dat anywhere near my laptop.

Occasionally I'll log into my Dropbox from someone else's computer, to pull a saved document.  With my wallet on dropbox I would be even more worried about them having pasword-capturing spyware.

And even assuming I trust everything is secure on my end, I must then trust that staff (and contractors ?) at Dropbox who have at least read-only access to storage won't scan for all wallet.dat files?

Now instead of linking to the wallet.dat, I would periodically create an encrypted copy of the wallet (wish that were automatic or part of the client) and then copy that to Dropbox.  Then, to get that to Dropbox on Windows I'd use something ike MS SyncToy.

That works too, if you have a lot of bitcoins I wouldn't put it there unencrypted either.
It's just a matter of how experienced you are with all the tools and how much bitcoins are at stake.

I hope the bitcoin client will have some crypto for the wallet sometime in the future, as it's a big risk if you get hacked, and the more "money" that are out there the bigger the motivation for hackers to try get it