Privacy versus Safety: handling change

[Gavin Andresen]

Here's how you can lose coins by backing up and restoring your wallet file:

Lets say you have one shiny 1,000 Bitcoin coin in your wallet (it's actually just a transaction for 1,000 bitcoins paid to a public key that's stored in your wallet).

You backup that file.

Now you spend 1 Bitcoin.  Your shiny 1,000 BTC coin is broken into 1BTC, plus 999BTC in change.  That change is given a new, different public key.

Now if you restore your wallet file, Bitcoin sees that the 1,000BTC coin has been spent-- 1BTC was sent somewhere, and the other 999BTC was sent somewhere else.  Because you don't have the key for the 999BTC, it has no idea that those coins belong to you.

So they're lost.

Trying out alternative policies for handling change on the TEST network would be a good idea, in my opinion.

Maybe change transactions should always get signed with the same public key, so you wouldn't lose coins when restoring your wallet... although that would give you less privacy because it would tend to tie all your transactions together.

Maybe your wallet should get initially populated with 100 "change" addresses, with one randomly chosen as needed.  And some super-geeky way of replacing them with another, new, 100 addresses.

Maybe there's an even better way of handling the "I lost BTC when I restored my wallet" problem; ideas?

[1713896511]

1713896511

[1713896511]

1713896511

[Insti]

Isn't this just a result of restoring to an out of date backup?

That aside,

Simple possible change return methods:

a) change gets returned to the address it came from.
b) change gets returned to a newly generated bitcoin address.

Most people wont care and will expect the a) behavior, so make this the default.
Those who care about the privacy of their transactions can enable b) through some option setting in the client.

[satoshi]

We should queue up a supply of pre-made addresses in the wallet to use when a new address is needed.  They aren't very big, so it wouldn't hurt to have a lot of them.  This would more generally cover the case also where someone backs up, then requests a new address and receives a big payment with it.  Maybe there should be separate queues so one type of demand on addresses doesn't deplete it for the others.

The addresses would be created and stored in the normal place, but also listed on a separate list of created-but-never-used addresses.  When an address is requested, the address at the front of the never-used queue is handed out, and a new address is created and added to the back.

There's some kind of rescan in the block loading code that was made to repair the case where someone copied their wallet.dat.  I would need to check that the rescan handles the case of rediscovering received payments in blocks that were already received, but are forgotten because the wallet was restored.

[FreeMoney]

If you can't restore a backup that is out of date at all then won't you have to backup after every transaction?

Is this a possible solution? I have two computers, I use one all the time for small transactions. I occasionally sock away a large amount to my other machine just by transfering then I also immediately backup that machine's wallet.dat to a secure place.

[dete]

Surely the "coins" stored in wallet.dat are essentially just a cache, though, right?  The "real" count of BTC that I can spend are encoded into the chain.

If I still have all my private keys, I should be able to scan the block list to recover that "cache".

Suggestion: Include a block number/id in the wallet which is the last-known-good point for that wallet.  When a wallet is restored from backup, you can rescan the new blocks to be sure it's up-to-date.

Second Suggestion: All the user to rebuild a wallet without re-downloading all the keys.

Final Suggestion: Separate the "real" critical data (the private keys), from the "cache" (my current coins).  There is no reason to conflate the data that is absolutely critical for me to access my coins from a system designed to efficiently find my coins and let me spend them.

(Unless I'm fundamentally misunderstanding the inner workings of BitCoin!)

[theymos]

dete: You are losing the private keys in the case. Almost every time you send coins, you're also sending some coins back to yourself, to a brand new keypair. Since this new private key is not in the backup, you lose the coins.

[dete]

Thanks Theymos: That clears things up a lot!  (I wasn't sure who the change-handling code worked...)

[Insti]

Has there been any progress on this?
What do we need to do to make it happen?
A user recently lost 8999 BTC due to this problem, it would be good if we could eliminate it.

[lachesis]

I would need to check that the rescan handles the case of rediscovering received payments in blocks that were already received, but are forgotten because the wallet was restored.

Yeah, that would be useful behaviour. I think better key management - the ability to import, export, create, and delete addresses at will - is also in order. The current system only let you create them - you cannot delete them or export them to a file.

[kiba]

Is it possible to get an address collision?

[throughput]

We should queue up a supply of pre-made addresses in the wallet to use when a new address is needed.  They aren't very big, so it wouldn't hurt to have a lot of them.  This would more generally cover the case also where someone backs up, then requests a new address and receives a big payment with it.  Maybe there should be separate queues so one type of demand on addresses doesn't deplete it for the others.

The addresses would be created and stored in the normal place, but also listed on a separate list of created-but-never-used addresses.  When an address is requested, the address at the front of the never-used queue is handed out, and a new address is created and added to the back.

There's some kind of rescan in the block loading code that was made to repair the case where someone copied their wallet.dat.  I would need to check that the rescan handles the case of rediscovering received payments in blocks that were already received, but are forgotten because the wallet was restored.

It should be better to allow exporting of keypairs in text form, as is done with PGP, and importing them back.
It would be also useful to add more control over transaction outputs (and inputs) in, say, no-warranty-expert-mode.
I request manual selection of tx inputs and specification of tx outputs. Let it only exist in command-line mode with -do-as-i-say option.

[ByteCoin]

Is it possible to get an address collision?

Possible - yes.
Practical - not at the moment.

The easiest way to prevent effectively permanent bitcoin loss is to backup and jealously protect the  private keys which are one roughly 78 digit decimal numbers or 64 digit hexadecimal number for all your bitcoin addresses. Everything else is recoverable.

ByteCoin

[silverman]

Make the Bitcoin wallet a directory with a series of snapshots of state, each transaction saving its new state?

An optional offsite backup cache would be nice too.