Word of caution using Bitcoin-qt client and managing your own private keys

[Rygon]

I nearly had a heart attack because I thought I had lost nearly 200 BTC tonight, so I can't fall asleep and instead decided to write up what was nearly a horrible loss of 200 BTC, forever. I figure the newbie forum is a good place to post this, as it might help someone avoid an accident of their own.

After using bitcoin for several month, sending and receiving transactions, even buying a few goods on line (woot for bitcoinstore.com!), I decided I wanted to learn about cold storage to truly store my bitcoin. My plan of attack was to be as simple as possible, generate a private key using bitaddress, or another tool, running on a machine off the internet, then print out the private key for safekeeping, and the public key so I could send money to that address. When I wanted access to that money, I would use the importprivkey function to manually type in the private key in bitcoin-qt, and I'd have access to my sweet sweet coinage. Sounds easy enough, right?

So, to experiment and get the feel of things, I generated a random private key and imported it into bitcoin-qt. Sure enough, it figured out what the correct public key was, so I must have typed it in correctly. It didn't show up in my "Send Bitcoins' button as a receiving address, which was weird. So I send 0.01 BTC to this new address, and to my relief, it processes it like a transaction to myself. My wallet balance doesn't change, so Bitcoin-qt must be working right

This is when I start to get cocky. Even though I backed up my wallet a few weeks ago, I decide to use the dumpprivkey command in bitcoin-qt to see what the private key is for my main wallet address that I use for receiving coins (this is what shows up on the "send coins" screen). I copy down that address, and decide to test it out. Could I actually restore my bitcoin balance from scratch using that private key? I was feeling so hax0r. I didn't even need the wallet.dat file if I just had that private key, stenciled in stone or something. First though, I needed to start out bitcoin-qt from scratch. I had done this before, it works by removing the wallet.dat file from the bitcoin directory. When the program starts back up again, it creates a new wallet.dat file, but won't know the previous keys.

Since I'd backed up the wallet file a few weeks ago, I didn't see any harm in just deleting the wallet.dat file at the command line instead of moving the file like a normal person.

That was a terrible idea. I just didn't know it yet.

I open up bitcoin-qt, import my old private key, and it finds the public key, I'm momentarily happy. But something it wrong. It says my overall balance is only 0.1 BTC. Huh? I quickly figure out that when I sent the last transaction, bitcoin-qt sent the remaining 199 btc to another address. One that I didn't have the private key for. I checked blockchain, and this address had never been used before. I've noticed before that sometimes when sending coins from bitcoin-qt, it will send odd amounts to other addresses that it presumably has the private keys for. I occasionally "clean up" by sending everything to my main address. I came to the conclusion that bitcoin-qt had just created another address within the wallet.dat file, and I had deleted that file like a fool, and my old backup of wallet.dat wouldn't have the private key for that address.

I scrambled for an hour trying to recover a file deleted in a Linux terminal, which is nearly impossible. I got more and more desperate, even went to Dwolla to start sending money to Mtgox for new coins. Even though I had convinced myself that all was lost, I finally uploaded my backup of wallet.dat, even though I was sure I wouldn't have access to that particular address.

Thankfully I was wrong. My balance showed back up again because my previous backup of wallet.dat had that private key. Apparently bitcoin-qt generates a whole bunch of addresses that aren't visible on the main screen. So even if you receive every single coin at the same address, if you send any amount out, it could transfer the remaining coins to some other address to chill for a while. This isn't very well documented anywhere obvious to me, and I'm not sure why it needs to do all that. It makes understanding bitcoin just a little bit harder to grasp, imo. Does it ever generate more addresses after the initial wallet build? I have no idea. But I do think that if the BTC is stored in any public key other than the initial one that is set up, that should show up in the wallet GUI.

I guess my lesson is to always check blockchain so that you know exactly which address your coins are stored in. Oh, and backup wallet.dat. Perhaps even regularly!

[1715285925]

1715285925

[nobbynobbynoob]

"Change" addresses?

Good to know your BTC are fine, rich person!

[DeathAndTaxes]

It is called a change address.  You can't spend part of an output.  So if you have an output worth 10 BTC and want to send 6 BTC the transaction has 6 BTC sent to your destination and another 4 BTC sent back to a change address.  Why a new address?  To preserve anonymity.  If the change address was always an existing address it would be trivial to track transactions.

Still your right if you don't know what you are doing..... don't frak around with private keys and make frequent backups.  Honestly the point of a cold wallet is to keep it as an offline backup, not to import it, and play around periodically.  You gain nothing and run the risk of doing something silly.

[casascius]

I believe Bitcoin-qt should be reduced to being a Bitcoin "knowledge daemon" that can be run on a server, and get out of the business of being a wallet and a GUI client.  I consider it egregious that a "wallet" is a feature of the Bitcoin "daemon" in the first place.

Let somebody else handle this job the right way, with deterministic wallets, so development on this can be focused.

[hashkey]

It is a good practice to backup your wallet.dat for every 100 new addresses added to it https://en.bitcoin.it/wiki/Securing_your_wallet#Securing_the_Bitcoin-QT_or_bitcoind_wallet

[setkeh]

And every one though i was mad for having a daily cronjob to backup my wallet

[Foxpup]

And every one though i was mad for having a daily cronjob to backup my wallet

To a different file each day, I trust. It's always real fun when your wallet gets corrupted and your daily backup job overwrites your only backup with the corrupt data.

[Rygon]

It is called a change address.  You can't spend part of an output.  So if you have an output worth 10 BTC and want to send 6 BTC the transaction has 6 BTC sent to your destination and another 4 BTC sent back to a change address.  Why a new address?  To preserve anonymity.  If the change address was always an existing address it would be trivial to track transactions.

Still your right if you don't know what you are doing..... don't frak around with private keys and make frequent backups.  Honestly the point of a cold wallet is to keep it as an offline backup, not to import it, and play around periodically.  You gain nothing and run the risk of doing something silly.

Yeah, I still don't understand the whole change address thing or why it is necessary. Which is probably just one of many Bitcoin topics I don't understand, which is why I shouldn't be messing around with anything in the first place. I know just enough to be dangerous to myself, lol.

Although I have trust in the bitcoin-qt program, I'm going to try out some of the other programs like Armory for a better solution to cold storage. I'll still run bitcoin-qt since I want to support the network and the bandwidth and space are no big deal to me. Clearly I shouldn't have more coins on that computer than I would be angry to lose.

[nobbynobbynoob]

Armory requires QT to run online anyway AFAIK.

[casascius]

Clearly I shouldn't have more coins on that computer than I would be angry to lose.

For all the rest of your coins, there's cold storage on paper.

I haven't heard of a single bitcoin being lost or stolen from a paper wallet.

It's so simple: print a paper wallet, and then send the coins to the address printed on it.  When you need the coins, import the private key.

http://bitaddress.org

[DeathAndTaxes]

Clearly I shouldn't have more coins on that computer than I would be angry to lose.

For all the rest of your coins, there's cold storage on paper.

I haven't heard of a single bitcoin being lost or stolen from a paper wallet.

It's so simple: print a paper wallet, and then send the coins to the address printed on it.  When you need the coins, import the private key.

http://bitaddress.org

That is exactly what the OP did.  However he then imported that private key into the QT client and sent funds from it.

The larger issue is that one would either treat a wallet file as a complete unit (i.e. forget about individual keys) and treat the wallet as a single object which needs to be be backed up in its entirety OR use individual private keys in systems like offline paper wallets.

Anyone mixing both systems without understanding how it works under the hood is running the risk of losing funds.

[casascius]

I make sure that I always send all coins back to paper when I'm not actively transacting them.  By doing this, I never need to back up my wallet, since it (almost) never has coins in it.

[Rygon]

Clearly I shouldn't have more coins on that computer than I would be angry to lose.

For all the rest of your coins, there's cold storage on paper.

I haven't heard of a single bitcoin being lost or stolen from a paper wallet.

It's so simple: print a paper wallet, and then send the coins to the address printed on it.  When you need the coins, import the private key.

http://bitaddress.org

That is exactly what the OP did.  However he then imported that private key into the QT client and sent funds from it.

The larger issue is that one would either treat a wallet file as a complete unit (i.e. forget about individual keys) and treat the wallet as a single object which needs to be be backed up in its entirety OR use individual private keys in systems like offline paper wallets.

Anyone mixing both systems without understanding how it works under the hood is running the risk of losing funds.

+1
Clearly the paper wallet is the most straightforward way to do cold storage, which I at least feel confident about using at this point. It’s simple, and easy to avoid malware attacks. The only risk is losing the paper wallet - which is just a private key, the way I think about it. The confusion on my part is that the “wallet” in bitcoin-qt is more than just one private key. The terminology could be a lot clearer.

As a side note, I just came up with an thought for double or triple checking any process that involves transferring a large amount of coins. I’m a DYI kind of person, but it might be worth the peace of mind to have a trusted individual with at least equivalent knowledge of bitcoin to oversee the process and look out for “stupid human” errors. Just because a 0.01% error rate is still too high when it does involve significant BTC. That’s probably overkill, but if I ever cross over to having more money in bitcoin than what I could afford to lose, it might very well be worth it. Just a thought from a non-superuser.

[casascius]

Just to throw in a plug: I've released a methodology for doing encrypted private keys on paper wallets, and am evangelizing to get other bitcoin-related services to support it.

With encrypted paper wallets, you can make multiple copies of the paper wallet, as well as make one or more written copies of the passphrase, and store them in a way that makes sense.  (example: paper wallets in safe deposit box, passphrase at home/in brain/with important papers/etc.) so you can strategize on how to keep them safe without needing to worry about any technology.

[bitfarmer]

sounds like a disaster averted!

[dystopiandrift]

@casascius That is really cool and exciting. What is the decryption method like tho? I guess if you are able to convince the client maintainers to adopt this new method of yours (and it is secure, I don't doubt you!) it won't really matter to the end user to much... unless they are like me and want to know how everything in the world works. 

[casascius]

@casascius That is really cool and exciting. What is the decryption method like tho? I guess if you are able to convince the client maintainers to adopt this new method of yours (and it is secure, I don't doubt you!) it won't really matter to the end user to much... unless they are like me and want to know how everything in the world works.  

Decryption methods:

Available now: download my utility and decrypt it
Coming soon: point your iPhone at an encrypted QR code (program works, needs improvement before going to App Store)
Hopefully coming soon: a javascript decryption script so decryption can happen in a browser (I've opened a bounty and some people are working on it)
Hopefully leading to: that javascript incorporated into every website that accepts private keys, so the browser can do the decryption on the fly.

[dystopiandrift]

very very nice. browser would obviously be the best. cross platform compatibility and what have you. I would offer my help if I felt I had any to offer. By the time I learn enough java to get anything done, I'm sure you will have it all finished.  innovation and expansion of bitcoin for the win!

[setkeh]

And every one though i was mad for having a daily cronjob to backup my wallet

To a different file each day, I trust. It's always real fun when your wallet gets corrupted and your daily backup job overwrites your only backup with the corrupt data.

Yeah Of course i have a floder for odd days and even days so its very hard to loose my wallet the cronjob moves the backup to my nas before it makes the new one

[notme]

Your savior here is called the "keypool".   By default there will always be 100 unused keys in your wallet.  When you ask for a new address, it grabs one from this keypool (which has already been backed up hopefully) and then generates a new one to stick at the back of the queue.  There is a keypoolsize parameter you can set to alter the default size of 100 keys.