Bitcoin Forum
March 19, 2024, 11:48:09 AM *
News: Latest Bitcoin Core release: 26.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1] 2 »  All
  Print  
Author Topic: Wallet encryption issue  (Read 9599 times)
etotheipi (OP)
Legendary
*
expert
Offline Offline

Activity: 1428
Merit: 1093


Core Armory Developer


View Profile WWW
November 10, 2011, 03:53:00 PM
Last edit: February 07, 2012, 06:59:20 PM by etotheipi
Merited by ABCbits (1)
 #1

2012 Update:  It's now been a few months since this event, so I think it's safe to describe, for the record, what actually happened.  The Bitcoin wallet file is managed by BSDDB, which is a database engine.  Presumably, a database engine was used, in order to ensure ACID transactions (atomicity, consistency, isolation, durability), which makes sure that the database won't be corrupted by inconveniently-timed power outtages, etc.  Most importantly, it should protect your private keys from ever getting corrupted.

The problem with a database manager, though, is that it does its own thing with the file storage.  It's a black box in terms of its behavior when you tell it to "delete" something.  You'd hope it would actually remove the data, or at least blank it out, but that's not necessarily true.  In the case of BSDDB, when Satoshi client says "delete this key and replace it with the encrypted key" it seems to not actually do that.  Instead, it simply marks the entry deleted, and then appends the new data to the end of the file.  It doesn't even overwrite the original data.  

From the user perspective, this is completely transparent, because the user must ask the BSDDB interface for the data, and BSDDB reports that it's gone, since the pointer to the data was discarded.  But, it's still in the binary file!

Before wallet encryption, I had written a pure-binary key-extractor script, to pull my private keys out of the wallet file (it was an elaborate search using the ECDSA library to find public-keys and find the associated private-key nearby).  After Bitcoin 0.4.0 was released with wallet encryption, I re-ran my script for fun.  To my surprise, it found 36 of 112 private keys!  Whoops!

I wasn't too involved in the fix for the problem, since I have no experience with BSDDB.  But the solution had to do with convincing the database to actually remove/overwrite the data, and allow for wallets to be "born" encrypted to avoid any risk of it happening again.   I don't know the gory details though, you'd have to ask the devs for specific details about how they fixed it.




Original Post:
I think I might've found a bug in the wallet encryption.

Edit: This problem has been fixed in the client version 0.5.0.  Please see the release notes for information on fixing your wallet.   The most important information is:

Quote
If you have a previously encrypted wallet.dat, the first time you run bitcoin-qt or bitcoind the wallet will be rewritten, Bitcoin will shut down, and you will be prompted to restart it to run with the new, properly encrypted file.

If you had a previously encrypted wallet.dat that might have been copied or stolen (for example, you backed it up to a public location) you should send all of your bitcoins to yourself using a new bitcoin address and stop using any previously generated addresses.



Founder and CEO of Armory Technologies, Inc.
Armory Bitcoin Wallet: Bringing cold storage to the average user!
Only use Armory software signed by the Armory Offline Signing Key (0x98832223)

Please donate to the Armory project by clicking here!    (or donate directly via 1QBDLYTDFHHZAABYSKGKPWKLSXZWCCJQBX -- yes, it's a real address!)
1710848889
Hero Member
*
Offline Offline

Posts: 1710848889

View Profile Personal Message (Offline)

Ignore
1710848889
Reply with quote  #2

1710848889
Report to moderator
Make sure you back up your wallet regularly! Unlike a bank account, nobody can help you if you lose access to your BTC.
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction.
RodeoX
Legendary
*
Offline Offline

Activity: 3066
Merit: 1145


The revolution will be monetized!


View Profile
November 10, 2011, 04:05:57 PM
 #2

Wow. That is a problem. So we still need to encrypt our wallet file, even if it is encrypted by the client?

The gospel according to Satoshi - https://bitcoin.org/bitcoin.pdf
Free bitcoin in ? - Stay tuned for this years Bitcoin hunt!
jojkaart
Member
**
Offline Offline

Activity: 97
Merit: 10


View Profile
November 10, 2011, 04:29:59 PM
 #3

Wow. That is a problem. So we still need to encrypt our wallet file, even if it is encrypted by the client?

If you move all your coins out of the old keys and only use new keys (created after encrypting the wallet), there should be no problem.

- Joel
etotheipi (OP)
Legendary
*
expert
Offline Offline

Activity: 1428
Merit: 1093


Core Armory Developer


View Profile WWW
November 10, 2011, 04:36:25 PM
Last edit: November 12, 2011, 01:58:49 AM by etotheipi
 #4

If you move all your coins out of the old keys and only use new keys (created after encrypting the wallet), there should be no problem.

This is where casascius' recommendation for "sweepprivkey" becomes extremely useful.  

Founder and CEO of Armory Technologies, Inc.
Armory Bitcoin Wallet: Bringing cold storage to the average user!
Only use Armory software signed by the Armory Offline Signing Key (0x98832223)

Please donate to the Armory project by clicking here!    (or donate directly via 1QBDLYTDFHHZAABYSKGKPWKLSXZWCCJQBX -- yes, it's a real address!)
DeathAndTaxes
Donator
Legendary
*
Offline Offline

Activity: 1218
Merit: 1063


Gerald Davis


View Profile
November 10, 2011, 04:38:28 PM
 #5

Wow. That is a problem. So we still need to encrypt our wallet file, even if it is encrypted by the client?

If you move all your coins out of the old keys and only use new keys (created after encrypting the wallet), there should be no problem.

- Joel

If that is true then the implementation is "borked".  Nothing in client docs indicate that limitation.  Users should at least be warned, even better would be for the client to recommend a transfer of all balances to a "new" address.
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 2164


Chief Scientist


View Profile WWW
November 10, 2011, 05:06:59 PM
 #6

Discussing how to fix this in IRC now.

This IS a show-stopper issue for the 0.5 release.

How often do you get the chance to work on a potentially world-changing project?
etotheipi (OP)
Legendary
*
expert
Offline Offline

Activity: 1428
Merit: 1093


Core Armory Developer


View Profile WWW
November 10, 2011, 05:10:36 PM
 #7

Has anyone verified this?  I don't want to stop any shows unless someone else can confirm it.

BTW, I am on Ubuntu 10.04, 64-bit, if that matters...

Founder and CEO of Armory Technologies, Inc.
Armory Bitcoin Wallet: Bringing cold storage to the average user!
Only use Armory software signed by the Armory Offline Signing Key (0x98832223)

Please donate to the Armory project by clicking here!    (or donate directly via 1QBDLYTDFHHZAABYSKGKPWKLSXZWCCJQBX -- yes, it's a real address!)
pc
Sr. Member
****
Offline Offline

Activity: 253
Merit: 250


View Profile
November 10, 2011, 05:29:56 PM
 #8

When I read this, I thought it was a report about the encryption in the existing 0.4 client. With Gavin's comment that it's holding up 0.5, it makes me unsure what version(s) this may be issue for.
etotheipi (OP)
Legendary
*
expert
Offline Offline

Activity: 1428
Merit: 1093


Core Armory Developer


View Profile WWW
November 10, 2011, 05:34:57 PM
 #9

This is a 0.4.0 issue.  Gavin was trying to release 0.5 in the immediate future, but I think he wants to hold it up until we find a fix for this problem.  We should get this fixed ASAP.

Founder and CEO of Armory Technologies, Inc.
Armory Bitcoin Wallet: Bringing cold storage to the average user!
Only use Armory software signed by the Armory Offline Signing Key (0x98832223)

Please donate to the Armory project by clicking here!    (or donate directly via 1QBDLYTDFHHZAABYSKGKPWKLSXZWCCJQBX -- yes, it's a real address!)
makomk
Hero Member
*****
Offline Offline

Activity: 686
Merit: 564


View Profile
November 10, 2011, 06:44:35 PM
 #10

I seem to recall that this was a known issue at the time wallet encryption was first implemented - was there meant to be something that fixed it since then?

Quad XC6SLX150 Board: 860 MHash/s or so.
SIGS ABOUT BUTTERFLY LABS ARE PAID ADS
kokjo
Legendary
*
Offline Offline

Activity: 1050
Merit: 1000

You are WRONG!


View Profile
November 10, 2011, 06:52:20 PM
 #11

fix:
load all keys in to memory.
delete wallet file.
make new encrypted wallet.
put back keys from memory.


"The whole problem with the world is that fools and fanatics are always so certain of themselves and wiser people so full of doubts." -Bertrand Russell
makomk
Hero Member
*****
Offline Offline

Activity: 686
Merit: 564


View Profile
November 10, 2011, 07:20:31 PM
 #12

fix:
load all keys in to memory.
delete wallet file.
make new encrypted wallet.
put back keys from memory.
That will of course leave behind unencrypted copies of the keys in the unallocated space on the disk - not to mention the risk of data loss if Bitcoin or the computer crashes at the wrong moment - but it might be better than nothing.

Quad XC6SLX150 Board: 860 MHash/s or so.
SIGS ABOUT BUTTERFLY LABS ARE PAID ADS
DeathAndTaxes
Donator
Legendary
*
Offline Offline

Activity: 1218
Merit: 1063


Gerald Davis


View Profile
November 10, 2011, 07:24:08 PM
 #13

fix:
load all keys in to memory.
delete wallet file.
make new encrypted wallet.
put back keys from memory.
That will of course leave behind unencrypted copies of the keys in the unallocated space on the disk - not to mention the risk of data loss if Bitcoin or the computer crashes at the wrong moment - but it might be better than nothing.

I think a better method is to load all keys in to memory, flag all existing keys as "potentially insecure", create new set of keys and transfer all balances to new set of keys.  Then encrypt all keys.

Granted you still run the risk of receiving money on an old "insecure" key but your current balances are secure and the client could ensure it never re-used one of those old addresses again.
kokjo
Legendary
*
Offline Offline

Activity: 1050
Merit: 1000

You are WRONG!


View Profile
November 10, 2011, 07:24:57 PM
 #14

fix:
load all keys in to memory.
delete wallet file.
make new encrypted wallet.
put back keys from memory.
That will of course leave behind unencrypted copies of the keys in the unallocated space on the disk - not to mention the risk of data loss if Bitcoin or the computer crashes at the wrong moment - but it might be better than nothing.
then delete the wallet file after putting back back keys from memory.

for unencrypted keys on freespace on disks:
Code:
sudo dd if=/dev/urandom of=/dev/sda
DO NOT TRY THIS AT HOME.

"The whole problem with the world is that fools and fanatics are always so certain of themselves and wiser people so full of doubts." -Bertrand Russell
jojkaart
Member
**
Offline Offline

Activity: 97
Merit: 10


View Profile
November 10, 2011, 07:33:37 PM
 #15

then delete the wallet file after putting back back keys from memory.

for unencrypted keys on freespace on disks:
Code:
sudo dd if=/dev/urandom of=/dev/sda
DO NOT TRY THIS AT HOME.

I seem to remember seeing some software at one point that securely overwrites all empty space in a filesystem. That was for Windows and I no longer even remember the name of the program. Something like that would be ideal to suggest to people who wish to secure their hard drives.

Would anyone happen to know the names of programs that can do this?
kokjo
Legendary
*
Offline Offline

Activity: 1050
Merit: 1000

You are WRONG!


View Profile
November 10, 2011, 07:37:25 PM
 #16

then delete the wallet file after putting back back keys from memory.

for unencrypted keys on freespace on disks:
Code:
sudo dd if=/dev/urandom of=/dev/sda
DO NOT TRY THIS AT HOME.

I seem to remember seeing some software at one point that securely overwrites all empty space in a filesystem. That was for Windows and I no longer even remember the name of the program. Something like that would be ideal to suggest to people who wish to secure their hard drives.

Would anyone happen to know the names of programs that can do this?

well you could:
Code:
dd if=/dev/random of=/junkfile
sync
rm /junkfile
it will consume all free space on your root fs.

"The whole problem with the world is that fools and fanatics are always so certain of themselves and wiser people so full of doubts." -Bertrand Russell
Etlase2
Hero Member
*****
Offline Offline

Activity: 798
Merit: 1000


View Profile
November 10, 2011, 07:40:16 PM
 #17

I seem to remember seeing some software at one point that securely overwrites all empty space in a filesystem. That was for Windows and I no longer even remember the name of the program. Something like that would be ideal to suggest to people who wish to secure their hard drives.

Would anyone happen to know the names of programs that can do this?


The one for windows is called sdelete. I remember reading somewhere that this utility wasn't possible to do identically in linux because of the file system for some reason.

Matt Corallo
Hero Member
*****
expert
Offline Offline

Activity: 755
Merit: 515


View Profile
November 10, 2011, 08:48:42 PM
 #18

fix:
load all keys in to memory.
delete wallet file.
And the power goes out...
make new encrypted wallet.
put back keys from memory.

Bitcoin Core, rust-lightning, http://bitcoinfibre.org etc.
PGP ID: 07DF 3E57 A548 CCFB 7530  7091 89BB B866 3E2E65CE
nibor
Sr. Member
****
Offline Offline

Activity: 438
Merit: 291


View Profile
November 10, 2011, 09:48:23 PM
 #19


Other option is to remove the ability to encrypt an old wallet.
You can only create a new encrypted one.

As by definition if you had an un-encrypted wallet the keys will be spread all over your hard disk so encrypting it just gives you a false sense of security.

Use case would be:
1) If no wallet file there offer user option to create new unencrypted or encrypted wallet.

2) If unencrypted wallet there and user clicks on encrypt wallet show screen with instructions on how to create a new wallet then send coins from old to new.

3) If encrypted wallet there and was from version v0.4 show a warning box every time you open it telling you to move to a new wallet and explain why.
dooglus
Legendary
*
Offline Offline

Activity: 2940
Merit: 1327



View Profile
November 12, 2011, 12:09:56 AM
 #20

for unencrypted keys on freespace on disks:
Code:
sudo dd if=/dev/urandom of=/dev/sda
DO NOT TRY THIS AT HOME.

I don't know if this was meant as a joke, but it's probably worth pointing out that this will overwrite all the partitions on the first hard drive with random junk, not just the "freespace".

Just-Dice                 ██             
          ██████████         
      ██████████████████     
  ██████████████████████████ 
██████████████████████████████
██████████████████████████████
██████████████████████████████
██████████████████████████████
██████████████████████████████
██████████████████████████████
██████████████████████████████
██████████████████████████████
██████████████████████████████
    ██████████████████████   
        ██████████████       
            ██████           
   Play or Invest                 ██             
          ██████████         
      ██████████████████     
  ██████████████████████████ 
██████████████████████████████
██████████████████████████████
██████████████████████████████
██████████████████████████████
██████████████████████████████
██████████████████████████████
██████████████████████████████
██████████████████████████████
██████████████████████████████
    ██████████████████████   
        ██████████████       
            ██████           
   1% House Edge
Pages: [1] 2 »  All
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!