wallet restore question using up keypool

[zef]

Ive spent a lot of time reading the wiki and searching the forums, i cant seem to find an answer to this question:
Under restoring your wallet the wiki says, "Assuming your backup is recent enough that you haven't used up all of your key pool... restoring a wallet to a new (or old) location and rescanning the block chain should leave you with all your coins."

The wiki does not explain why it matters if you use up your keypool in reference to backing up your wallet. What happens if you backed up your wallet a year ago, had over 100 transactions since then, and restored your backup?  It seems as if there is a limit to a backups lifetime; i saw a thread saying you should backup your wallet every 30 transactions, but this just does not seem practical to me.  Can anyone carify?

[1715025584]

1715025584

[1715025584]

1715025584

[1715025584]

1715025584

[tubro]

read further up in the explanation. yes, if you backup your wallet and then keep making transfers from that wallet, then lose your wallet, the backup will only help you restore the first 100 transactions made since you made the backup. not bad, I find. If I make a backup of something I write, then write 100 lines more, then lose everything, my backup would not contain these new 100 lines

[zef]

Thanks for the response. I found the section you reference.

So if you create a backup, do more than 100 things that cause a new key to be used, and then restore from the backup, some Bitcoins will be lost. Bitcoin has not deleted any keys (keys are never deleted) – it has created a new key that is not in your old backup and then sent Bitcoins to it.

I get it now. The 100 keys you have are only needed for receiving money, not sending money. I mistakenly thought that you used up your keys for spending coins, but obviously that is not the case. Each amount received is tied to a certain key, if that key is not in your wallet, the network will not recognize that amount in your total account, and adjust accordingly. Therefore it is critical you maintain a list of ALL keys you have ever received money on.  

If my understanding is true, there should be a big bold warning as to what constitutes a full wallet backup at the top of the Wiki for wallet management. When I read it I was under the impression that all I needed to do was backup my wallet.dat file.

Obviously the average person will probably not hit that 100 transaction limit soon, but that just makes it even more dangerous as they may only ever make one backup assuming they will be fine, or forgetting to ever update it.  If bitcoins is to be mainstream, wallet managment will be a huge concern, since it can literally contain someones life savings. Clarity and ease of use on how to protect losing any part of your assets should be critical.

[zef]

On a related note, I don't really like the idea that bitcoin is unable to verify that you own certain public keys using your private key if they don't explicitly exist in your wallet.  Isn't it possible to verify that you are the owner of a public key using your private key, even if there is no record of you generating it, ie its not in your wallet?  This makes more sense to me, and means the wallet only needs to contain your private key information; ownership of transactions can be "deduced" by a public key relationship to your private key. 

[bittrader]

On a related note, I don't really like the idea that bitcoin is unable to verify that you own certain public keys using your private key if they don't explicitly exist in your wallet.  Isn't it possible to verify that you are the owner of a public key using your private key, even if there is no record of you generating it, ie its not in your wallet?  This makes more sense to me, and means the wallet only needs to contain your private key information; ownership of transactions can be "deduced" by a public key relationship to your private key. 

There actually isn't a single private key which has multiple public keys. Each private key has one public key and your wallet stores several sets of private/public keys. So that's why, if you're missing a public key from your wallet, you can't have ownership of that key's transactions....since the public key is gone from the wallet, the private key is gone as well (they're stored together).

Having multiple public/private key sets per user allows for more pseudonymity in the Bitcoin network since there isn't a single key that always identifies a user. Rather, when receiving money, a user can generate a new keypair unrelated to previous keypairs.

[zef]

Ok that makes sense, thanks.

[theymos]

I get it now. The 100 keys you have are only needed for receiving money, not sending money. I mistakenly thought that you used up your keys for spending coins, but obviously that is not the case. Each amount received is tied to a certain key, if that key is not in your wallet, the network will not recognize that amount in your total account, and adjust accordingly. Therefore it is critical you maintain a list of ALL keys you have ever received money on.

You've got it backwards. Each send creates a new key. You can receive as many transactions as you want and not use up any keys.

[zef]

You've got it backwards. Each send creates a new key. You can receive as many transactions as you want and not use up any keys.

Are you sure? This goes back to my original question, heres an example:

Lets assume that we use new keys when we SEND money.  So i backup my wallet, use up my keypool by sending 100 small amounts of coin, then on the 101th transaction, i send my remaining balance somewhere, say 100 coins.  I then restore my backed up wallet, since it has no key for my 101th transaction, it should restore my balance to 100, right?  Thats what is confusing me, obviously this can't be the case. 

[bittrader]

Lets assume that we use new keys when we SEND money.  So i backup my wallet, use up my keypool by sending 100 small amounts of coin, then on the 101th transaction, i send my remaining balance somewhere, say 100 coins.  I then restore my backed up wallet, since it has no key for my 101th transaction, it should restore my balance to 100, right?  Thats what is confusing me, obviously this can't be the case. 

Right, that's not the case. What'll actually happen is that, after you restore your backup, your client will download all the newest blocks from the network. One of those blocks will show your 100 BTC being transfered away from the addresses in your wallet. This will cause your client to know that your wallet no longer has any BTC.

[theymos]

So i backup my wallet, use up my keypool by sending 100 small amounts of coin, then on the 101th transaction, i send my remaining balance somewhere, say 100 coins.  I then restore my backed up wallet, since it has no key for my 101th transaction, it should restore my balance to 100, right?  Thats what is confusing me, obviously this can't be the case. 

It will detect the new transaction and set your balance to zero.

However, if that 101^st send did not empty your wallet, you would lose some random amount of additional BTC beyond what you sent. After 120 sends, you'd probably lose most/all of your BTC.

[zef]

Right, that's not the case. What'll actually happen is that, after you restore your backup, your client will download all the newest blocks from the network. One of those blocks will show your 100 BTC being transfered away from the addresses in your wallet. This will cause your client to know that your wallet no longer has any BTC.

My point is though, my backup wallet does not contain the private/public key pair i used for my 101th transaction, so how will the network know to update me on it?

It will detect the new transaction and set your balance to zero.

However, if that 101st send did not empty your wallet, you would lose some random amount of additional BTC beyond what you sent. After 120 sends, you'd probably lose most/all of your BTC.

It just doesnt make any sense to me why this would happen? Again, how can it detect my transaction if i am restoring a wallet that does not contain the keys that made the transfer?

[theymos]

It'll detect the transfer from the keys that you do have. If there was no transfer from the keys that you do have, then you'll still have that BTC.

[bittrader]

My point is though, my backup wallet does not contain the private/public key pair i used for my 101th transaction, so how will the network know to update me on it?

The network doesn't know which keys you have in your wallet. It doesn't ever send you updates about specific keys. Instead, all blocks are sent to your client. Then, your client examines each block and checks to see if any of the transactions in the block correspond to keys in your wallet. This allows the client to calculate how much BTC is in your wallet without revealing to the network which keys are yours.

[zef]

I think I may have found the explanation for what you guys are saying.  When you send BTC somewhere, more may get sent then you specified, the remainder coming back to you as another transaction to a new address, keeping the expected balance correct.  This phenomenon is mentioned here: http://blockexplorer.com/address/19p3N7TwiJkWDcAcRce6tBLkS2pMhwPQcc  

The official wiki does not mention this effect in the "transaction" section, which is probably why I have been so confused.  This answers my questions in regards to backing up the wallet and keypool, but now I have questions as to why bitcoins does something so strange.  Sends more money then u specify, and then sends the remainder back to a different address.  Is it for security or something?  How is the extra amount determined, and what happens if you don't have enough?

edit:
Actually i see in the wiki where it discusses this

An output contains instructions for sending bitcoins. Value is the number of Satoshi (1 BTC = 100,000,000 Satoshi) that this output will be worth when claimed. ScriptPubKey is the second half of a script (discussed later). There can be more than one output, and they share the combined value of the inputs. Because an output can only ever be referenced by a single input, the entire combined input value needs to be sent in an output if you don't want to lose it. If the input is worth 50 BTC but you only want to send 25 BTC, Bitcoin will create two outputs worth 25 BTC: one to the destination, and one back to you (known as "change", though you send it to yourself). Any input bitcoins not redeemed in an output is considered a transaction fee; whoever generates the block will get it.

[bittrader]

Yes, the whole amount from the previous transaction needs to be sent somewhere, even if most of it is only back to yourself at a new address.

It also happens to serve as a means to make it a little harder to tell how much BTC is sitting in a wallet without being spent.