Lost large number of bitcoins

[Stone Man]

Here are the details.

1) Bought 9,000 BTC on one of the exchanges over time.
2) Transferred them to my client running on a linux live CD distro of Debian.
3) Backed up the wallet file to a flash drive.
4) Sent 1 BTC to myself
5) Closed client before any confirmations
6) Shut down system (wiped system disk loaded into memory and therefore the ./bitcoin folder
7) Loaded system back up
Copied old wallet.dat file into ./bitcoin folder
9) After some confirmations appeared the balance was 1 BTC and there was a transaction saying I spent 8,900 BTC to an address I did not recognize
10) I read on the forum threads that people have had problems like this but it seemed only when they were trying to double-spend by sending coins to another address and reloading an old wallet file


Is there anything I can do?

I do have the address that the 8,900 were supposedly sent to but the old wallet file is gone for good.

Thanks,
Stone Man

[tcatm]

Do you still have the wallet.dat that once "contained" the 9000 BTC? I might be able to help you.

[Insti]

It sounds like they are lost for good.



Lesson to other people: Remember you need to back your wallet up after every transaction!
(The wallet metaphor is actually a bad one, because it makes problems like this more common than they should be.)

Edit: (more explaination)

When you do a transaction you always spend ALL of the coins in that address, part of the money goes to whoever you're paying, and there rest is returned to you an a new 'change' address. (This helps your anonymity as someone watching cannot tell which amount you spent and which amount you kept.)

Before transaction:
Address1: 9000
Address2: 0

After transaction:
Address1: 0
Address2: 1
Address3: 8999

It sounds like you have wallet.dat that contains the private keys for Address1 and Address2, but not Address3.
(Although the numbers in your story don't add up.)
What were the Bitcoin addresses? (If we know these we can do some more in-depth block chain research)

[FreeMoney]

Oh, man. Why did you send the 1? Were you just testing to see how the backup worked? Why with so many coins?

That sucks.

[Stone Man]

Do you still have the wallet.dat that once "contained" the 9000 BTC? I might be able to help you.

Yes, I do.

I saw on another thread that gavinanderson has made some tools to do more in depth research but did not know how to proceed.

If you could point me in the right direction I would greatly appreciate it.

[Stone Man]

It sounds like you have wallet.dat that contains the private keys for Address1 and Address2, but not Address3.
(Although the numbers in your story don't add up.)
What were the Bitcoin addresses? (If we know these we can do some more in-depth block chain research)

Address1: 157PiPgqphedUvrco3mKU3Xoof7yzhj9pW
Address2: 157PiPgqphedUvrco3mKU3Xoof7yzhj9pW
Address3: 167ZWTT8n6s4ya8cGjqNNQjDwDGY31vmHg

[Insti]

I used bitcointools: http://github.com/gavinandresen/bitcointools
To look for your transactions in the block chain.

(Here are the edited highlights)

Code:

python dbdump.py --search-blocks 157PiPgqphedUvrco3mKU3Xoof7yzhj9pW 

Block: 73261
Time: Mon Aug  9 22:08:34 
734 tx in, 1 out
['TxIn: .... <snipped. There were a lot> ']
['TxOut: value: 9000.00 pubkey: 157PiPgqphedUvrco3mKU3Xoof7yzhj9pW Script: DUP HASH160 20:2d15...8e7a EQUALVERIFY CHECKSIG']

Block: 73272
Time: Mon Aug  9 23:35:11 2010 
1 tx in, 2 out
['TxIn: prev(8bda...4675:0) pubkey: 157PiPgqphedUvrco3mKU3Xoof7yzhj9pW sig: 72:3045...5301 65:04bc...2d42']
['TxOut: value: 1.00 pubkey: 157PiPgqphedUvrco3mKU3Xoof7yzhj9pW Script: DUP HASH160 20:2d15...8e7a EQUALVERIFY CHECKSIG', 
 'TxOut: value: 8999.00 pubkey: 167ZWTT8n6s4ya8cGjqNNQjDwDGY31vmHg Script: DUP HASH160 20:3816...7437 EQUALVERIFY CHECK SIG']

So the transactions are definitly in there.
Your only hope is to have the private key for the 167ZWTT8n6s4ya8cGjqNNQjDwDGY31vmHg address.

If you can get bitcointools working you can dump your wallet.dat using:

Code:

python dbdump.py --wallet

which will show you all the keys you know about in your wallet.
But you'll probably find that 167ZWTT8n6s4ya8cGjqNNQjDwDGY31vmHg is not in there

On the upside, (although it is probably of no consolation to you) there is now a decent bounty for anyone who can find an address collision...

_{Edit: I'd pasted the wrong block number and date for the first block. Fixed.}

[Stone Man]

Oh, man. Why did you send the 1? Were you just testing to see how the backup worked? Why with so many coins?

That sucks.

Actually, the sad part was I wasn't even testing the wallet backup when this happened. I was trying to watch when the network should have confirmed a payment to a website that takes bitcoins by paying myself 1 coin at about the same time. I never dreamed it could compromise my whole balance, especially since I was sending to myself.

[FreeMoney]

Oh, man. Why did you send the 1? Were you just testing to see how the backup worked? Why with so many coins?

That sucks.

Actually, the sad part was I wasn't even testing the wallet backup when this happened. I was trying to watch when the network should have confirmed a payment to a website that takes bitcoins by paying myself 1 coin at about the same time. I never dreamed it could compromise my whole balance, especially since I was sending to myself.

Ugh, that's awful.

[gridecon]

Until this thread I was totally unaware of this behavior and risk. The usual behavior of a "backup" of a file is to preserve the contents of that file, and the intuitive understand of the wallet.dat file is that it "contains" the coins. I understand this is a case of the system behaving as designed, but the information that you need to backup your wallet after every transaction and each transaction makes your old wallet backups useless needs to be communicated much more clearly to users, I think. It also seems to me that it would be useful to add features for 'automatic backup' of a wallet after each transaction and easy support for multiple wallets - a "savings account" and a "checking account" so to speak.

Sorry about your loss, Stone Man.

[sirius]

I added to the FAQ the warning to back up after each transaction. Is it necessary btw to stop the client before making a backup? That's a bit inconvenient. Automatic backups would be useful indeed.

[BioMike]

This has the implication that if you create a receiving address and receive coins to that address, then restore the wallet from an earlier backup, before the address was generated, then the coins associated with that address are lost. If you lose your wallet entirely, all of your coins are lost and can never be recovered.

On the upside, (although it is probably of no consolation to you) there is now a decent bounty for anyone who can find an address collision...

So, they aren't lost forever? I guess it is more likely to generate the key of the address of someone else then from this specific address.

[Insti]

Until this thread I was totally unaware of this behavior and risk. The usual behavior of a "backup" of a file is to preserve the contents of that file, and the intuitive understand of the wallet.dat file is that it "contains" the coins. I understand this is a case of the system behaving as designed, but the information that you need to backup your wallet after every transaction and each transaction makes your old wallet backups useless needs to be communicated much more clearly to users, I think. It also seems to me that it would be useful to add features for 'automatic backup' of a wallet after each transaction and easy support for multiple wallets - a "savings account" and a "checking account" so to speak.

If you backup a spreadsheet, do you not expect to lose changes you made after that backup when you restore?

There are many types of transactions that will survive a wallet restore. This is one of the circumstances in which you will lose coins.
If he'd transferred all 9000 to the address he would still have them.

I'm not trying to blame the user. I agree this is a problem that needs to be fixed.

This problem has been previously identified and there are plans to implement a fix, which has been talked about in the Privacy versus Safety: handling change thread. But it has not been implemented yet.

Perhaps a bounty is in order?

On the upside, (although it is probably of no consolation to you) there is now a decent bounty for anyone who can find an address collision...

So, they aren't lost forever? I guess it is more likely to generate the key of the address of someone else then from this specific address.

According to the theory, generating a address collision is so remotely unlikely as to never happen.
(You're more likely to get struck by lightning, win all the worlds lotteries, and then get abducted by aliens on the same day.)

However if you manage to find a weakness in a) the encryption method or  b) Bitcoins implementation of it. it may make it easy enough that you don't have to get abducted by aliens as well.

So they practically are lost forever.

[caveden]

Until this thread I was totally unaware of this behavior and risk. The usual behavior of a "backup" of a file is to preserve the contents of that file, and the intuitive understand of the wallet.dat file is that it "contains" the coins. I understand this is a case of the system behaving as designed, but the information that you need to backup your wallet after every transaction and each transaction makes your old wallet backups useless needs to be communicated much more clearly to users, I think. It also seems to me that it would be useful to add features for 'automatic backup' of a wallet after each transaction and easy support for multiple wallets - a "savings account" and a "checking account" so to speak.

Sorry about your loss, Stone Man.

+1

This shocked me actually. The poor guy didn't do anything stupid.

Automatic backups are really important after knowing that. And while that cannot be implemented, I think a sort of information message explained this in short words should be displayed every time someone sends coins. A popup saying something like, if I have T coins and I send S coins:
"S coins where sent to address X.
(T-S) coins where sent to address Y, which belongs to you, in order to improve privacy. Please, realize that old backups you might have of your wallet will now be useless. You should make a new one."

This simple message would have avoided this awful lost. (of course, you may also add a "Don't display this again" check box)

Regards

[QuantumMechanic]

I think Satoshi proposed a protection against this in the following post:

We should queue up a supply of pre-made addresses in the wallet to use when a new address is needed.  They aren't very big, so it wouldn't hurt to have a lot of them.  This would more generally cover the case also where someone backs up, then requests a new address and receives a big payment with it.  Maybe there should be separate queues so one type of demand on addresses doesn't deplete it for the others.

The addresses would be created and stored in the normal place, but also listed on a separate list of created-but-never-used addresses.  When an address is requested, the address at the front of the never-used queue is handed out, and a new address is created and added to the back.

There's some kind of rescan in the block loading code that was made to repair the case where someone copied their wallet.dat.  I would need to check that the rescan handles the case of rediscovering received payments in blocks that were already received, but are forgotten because the wallet was restored.

[caveden]

This is indeed a good solution. Solves more than one problem actually.

[skull88]

Good to know this, I thougt I was save with my backups, but it could have happened to me to.
Stoneman didn't really do anything stupid. 

[Ground Loop]

News to me is that *all* your coins are at risk.  I thought it was just clumps of coins (previously received transactions) involved in the transaction, not my aggregate balance.  Yikes.

As a countermeasure, I've manually split my wallet up into five new wallets of distributed value.  Now only one is at risk at any given time.

[vess]

A simple solution to this would be to store your wallet.dat on an encrypted partition that goes to an S3-based storage system, like Dropbox. They'll keep versions for you on update.

This would make a nice opt-in service that could be sold in the native client as a way to help fund research, by the way. The data storage needs are incredibly small.

[lachesis]

News to me is that *all* your coins are at risk.  I thought it was just clumps of coins (previously received transactions) involved in the transaction, not my aggregate balance.  Yikes.

You were right before. The reason all of his coins were lost is that he first transfered all ฿9000 to himself, merging them into a single TxIn. If he had skipped that step and gone straight to sending himself ฿1, he would have only lost the smallest payment that he had previously received that was over ฿1.

I think the client needs to communicate TxIns and TxOuts better to the user. I don't know how to do that without being confusing, but there are real privacy, safety, and security implications in which coins the client chooses to transfer.