Armory - Discussion Thread

[marcus_of_augustus]

So out of interest, what is the basic strategy for Armory's physical handling of wallets?

Specifically;

1) are unencrypted private keys ever held on disk (or only ever in RAM)?

2) how are any old copies of wallet files that reside on disk at any point deleted/erased?

[1714200101]

1714200101

[1714200101]

1714200101

[1714200101]

1714200101

[Foxpup]

Except for the fact that you're not compiling, you may be right.

Also, what you mean by remove the dependency, is it completely removing it or changing it to libpython2.6?

It is a combination of C++ and Python, so it is compiled.

And yes, I do mean remove the dependency completely, so that it only depends on python-qt4 and python-twisted, not libpython. python-qt4 depends on an appropriate version of libpython anyway, so unless Armory requires a specific version of libpython (and it doesn't look it does, when compiled against python 2.7), there shouldn't be a problem.

[Raoul Duke]

Except for the fact that you're not compiling, you may be right.

Also, what you mean by remove the dependency, is it completely removing it or changing it to libpython2.6?

It is a combination of C++ and Python, so it is compiled.

And yes, I do mean remove the dependency completely, so that it only depends on python-qt4 and python-twisted, not libpython. python-qt4 depends on an appropriate version of libpython anyway, so unless Armory requires a specific version of libpython (and it doesn't look it does, when compiled against python 2.7), there shouldn't be a problem.

Got it know.
The compile part I understood it wrong. Thought you were refering to the process you used but now that I read it again realized you were describing to etho what he should do. Sorry about that.

[etotheipi]

So out of interest, what is the basic strategy for Armory's physical handling of wallets?

Specifically;

1) are unencrypted private keys ever held on disk (or only ever in RAM)?

2) how are any old copies of wallet files that reside on disk at any point deleted/erased?

(1) Plaintext keys are only written to disk if the state of the wallet is ever "unencrypted." 

• If you create the wallet encrypted, only the encrypted versions of the keys ever touch the hard-drive. 
• If you add encryption to an unencrypted wallet, the old keys are overwritten in-place so that the new file is guaranteed not to contain unencrypted data.  My wallet file format and technique was motivated by my discovery of the 0.4.X wallet-not-actually-encrypted bug -- I designed it to be simple and maintainable without external tools so I have full control over the data.
• Decrypting the keys to use them for signing only decrypts them into RAM, where they are held in "SecureBinaryData" containers which are page-locked and blanked out in RAM when they go out of scope or explicitly using key.destroy().   

The only catch with #3 is that Python is handling the SecureBinaryData objects, and python sometimes does weird stuff, memory-management-wise.  But the worst that could happen is that the keys are held in RAM longer than they should be.  They should still obey page-locked rules (which almost guarantees that the data won't be swapped, but most operating systems can't guarantee that).

(2)  Wallets are all kept in the .armory or AppData/Armory directory.  A backup is always created & maintained in the same directory for redundancy and auto-restore-on-corruption.  When you decide to delete a wallet through the wallet properties, both main and backup will be deleted.  When you import one, a copy is made into your .armory or AppData/Armory directory, leaving the original intact (option coming soon to add a remote wallet so you can keep it on a USB key or network mount).

[Foxpup]

• If you add encryption to an unencrypted wallet, the old keys are overwritten in-place so that the new file is guaranteed not to contain unencrypted data.

How exactly do you guarantee that files will in fact be overwritten "in-place"? This is not only operating-system and filesystem dependent, but even you can manage to do this in all operating system and on all filesystems, the hardware is not guaranteed to write data where the operating-system tells it to. Solid-state drives in particular are guaranteed not to do this due to their wear-levelling mechanism. I'm sorry, but if an unencrypted private key is ever written to disc, the only secure options are to stop using that private key or start using thermite.

[etotheipi]

• If you add encryption to an unencrypted wallet, the old keys are overwritten in-place so that the new file is guaranteed not to contain unencrypted data.

How exactly do you guarantee that files will in fact be overwritten "in-place"? This is not only operating-system and filesystem dependent, but even you can manage to do this in all operating system and on all filesystems, the hardware is not guaranteed to write data where the operating-system tells it to. Solid-state drives in particular are guaranteed not to do this due to their wear-levelling mechanism. I'm sorry, but if an unencrypted private key is ever written to disc, the only secure options are to stop using that private key or start using thermite.

This topic has been beaten to death on the mailing list and on IRC: this issue is out of scope (that's the stance of the main devs, too).  We could spend our lives trying to accommodate all the different filesystems and journaling paradigms to try to guarantee data was overwritten, but it's 1000x more effort than the value you get out if it.  

The important part is that the filesystem reports that the file has been overwritten in place.  If so, then the attacker will require root access to your hard-drive to get bit-level access to your drive to get the stale data.  This means one of the following is true:

(1) You're already compromised -- he can also pull the key out of memory next time you unlock
(2) You discarded the drive and you should've blanked the data before doing so
(3) Your drive was stolen

In the end, everyone who cares about this tiny attack vector uses wallets that were born encrypted.   And Armory does guarantee that born-encrypted wallets are secure.

If you're worried about (2) and (3) and electron-tunneling microscopes pulling your key off even after shredding: create your wallet encrypted and never remove the passphrase!   Problem solved!

[Foxpup]

In the end, everyone who cares about this tiny attack vector uses wallets that were born encrypted.   And Armory does guarantee that born-encrypted wallets are secure.

If you're worried about (2) and (3) and electron-tunneling microscopes pulling your key off even after shredding: create your wallet encrypted and never remove the passphrase!   Problem solved!

The problem is not solved for people who don't know about these things, create an unencrypted wallet, then later learn about the danger and encrypt the wallet, then their computer gets stolen, but it's okay because their wallet is encrypted, so they restore from a backup only to find that all their addresses have been cleaned out completely, then they sue you because you "guaranteed" that their unencrypted wallet would be overwritten. I am not suggesting that you find a way to ensure that unencrypted wallets are completely destroyed (since that's almost impossible), just that it's probably not the best idea to make guarantees with other people's money that you can't back up.

[etotheipi]

In the end, everyone who cares about this tiny attack vector uses wallets that were born encrypted.   And Armory does guarantee that born-encrypted wallets are secure.

If you're worried about (2) and (3) and electron-tunneling microscopes pulling your key off even after shredding: create your wallet encrypted and never remove the passphrase!   Problem solved!

The problem is not solved for people who don't know about these things, create an unencrypted wallet, then later learn about the danger and encrypt the wallet, then their computer gets stolen, but it's okay because their wallet is encrypted, so they restore from a backup only to find that all their addresses have been cleaned out completely, then they sue you because you "guaranteed" that their unencrypted wallet would be overwritten. I am not suggesting that you find a way to ensure that unencrypted wallets are completely destroyed (since that's almost impossible), just that it's probably not the best idea to make guarantees with other people's money that you can't back up.

I never guaranteed it!  Or rather, I never intended to.  What I "guarantee" is the file format does not allow for leftover plaintext keys when you encrypt an unencrypted wallet.  I was very careful about that.  If the filesystem has in-place file modifications, then so do my wallets.

Plus, that whole open-source license thing prevents me from being sued, in the event that it doesn't work as expected .  But I poured my heart into this wallet for weeks, and 1000 lines of unit-testing.  I'm very confident it's as good as it can get.

On that note, there's probably a good chance that encrypting in place will work, because the file size isn't changing at all when the keys are encrypted.  So the filesystem has one less reason to rewrite the data to a new location on disk.  Again, no guarantees, but it's a pretty solid solution!

P.S. -- you'll notice that my quote was "the new file is guaranteed not to contain unencrypted key data"  I didn't say anything about the hard-disk

[Foxpup]

[marcus_of_augustus]

So out of interest, what is the basic strategy for Armory's physical handling of wallets?

Specifically;

1) are unencrypted private keys ever held on disk (or only ever in RAM)?

2) how are any old copies of wallet files that reside on disk at any point deleted/erased?

(1) Plaintext keys are only written to disk if the state of the wallet is ever "unencrypted." 

• If you create the wallet encrypted, only the encrypted versions of the keys ever touch the hard-drive. 
• If you add encryption to an unencrypted wallet, the old keys are overwritten in-place so that the new file is guaranteed not to contain unencrypted data.  My wallet file format and technique was motivated by my discovery of the 0.4.X wallet-not-actually-encrypted bug -- I designed it to be simple and maintainable without external tools so I have full control over the data.
• Decrypting the keys to use them for signing only decrypts them into RAM, where they are held in "SecureBinaryData" containers which are page-locked and blanked out in RAM when they go out of scope or explicitly using key.destroy().   

The only catch with #3 is that Python is handling the SecureBinaryData objects, and python sometimes does weird stuff, memory-management-wise.  But the worst that could happen is that the keys are held in RAM longer than they should be.  They should still obey page-locked rules (which almost guarantees that the data won't be swapped, but most operating systems can't guarantee that).

Thanks for your answers. Good to see you the ability and have taken the time to drill down to this level of detail. My confidence in Armory just went up by leaps.

(2)  Wallets are all kept in the .armory or AppData/Armory directory.  A backup is always created & maintained in the same directory for redundancy and auto-restore-on-corruption.  When you decide to delete a wallet through the wallet properties, both main and backup will be deleted.  When you import one, a copy is made into your .armory or AppData/Armory directory, leaving the original intact (option coming soon to add a remote wallet so you can keep it on a USB key or network mount).

When you say "deleted" here, do you use something like a "srm" tool or similar multiple overwrites?

Thanks again.

[etotheipi]

(2)  Wallets are all kept in the .armory or AppData/Armory directory.  A backup is always created & maintained in the same directory for redundancy and auto-restore-on-corruption.  When you decide to delete a wallet through the wallet properties, both main and backup will be deleted.  When you import one, a copy is made into your .armory or AppData/Armory directory, leaving the original intact (option coming soon to add a remote wallet so you can keep it on a USB key or network mount).

When you say "deleted" here, do you use something like a "srm" tool or similar multiple overwrites?

Thanks again.

Like the description above, I don't spend much time thinking about filesystem issues.  I just use the regular OS delete operation, and let it deal with it.

But now that you mention it, there's no reason not to do something more elaborate, such as overwriting with random data before the OS deletes it.  Worst case: there is no benefit.  But, there is a chance that the overwrite happens in-place and that's an improvement! 

I guess the important part is that the final file you get can't have any key data left over, which makes it safe[er] to back it up to a network location if you wanted to (though, I still wouldn't myself, but that was the problem with the 0.4.0 bug:  people backed their files up to dropbox, etc, because they thought their key data was encrypted).

[marcus_of_augustus]

Right. I'm just a little concerned that since on most OS "delete" actually just changes (removes) the file header to make that space available for new writes and the file data is hardly ever changed then there is a trail of remnant wallets all over the place waiting for a disk recovery tool to come along. Even remnant encrypted wallet data may become a target if valuable enough.

Unless someone is assiduously creating new wallets and moving all their coins to the new wallets as they go then wallets that are moved around need to be aware of where all the old copies (including remnants) have been left behind, since any of those copies, live or "deleted", will potentially be a vulnerability. I would make "only secure delete" a policy of any wallet handling tool.

[etotheipi]

Right. I'm just a little concerned that since on most OS "delete" actually just changes (removes) the file header to make that space available for new writes and the file data is hardly ever changed then there is a trail of remnant wallets all over the place waiting for a disk recovery tool to come along. Even remnant encrypted wallet data may become a target if valuable enough.

Unless someone is assiduously creating new wallets and moving all their coins to the new wallets as they go then wallets that are moved around need to be aware of where all the old copies (including remnants) have been left behind, since any of those copies, live or "deleted", will potentially be a vulnerability. I would make "only secure delete" a policy of any wallet handling tool.

Agreed.  I just feel like there's very little I can do to accommodate this, besides making a good-faith effort to overwrite the data in place instead of just doing a vanilla delete operation.  As I mentioned in my previous post:  this attack vector is basically nil if you wallet was born encrypted.  Any users with any significant amount of money that don't encrypt their wallets: well they are probably at greater risk of other attack vectors.

Perhaps I'll add a warning (along with the millions of others) which warns about "residue" wallets even after deletion.  I don't think there's anything else I can/should do about it.  Unless you have recommendations...? 

[Foxpup]

Secure deletion requires knowledge of the filesystem and root access to the drive, and doesn't work in all cases anyway. Also, in most operating systems, "overwriting" doesn't mean what you think it does. Usually, the file to be overwritten is deleted and the new file is written to the largest available block of contiguous free space, which is not guaranteed or even likely to be where the original file was located. Actually overwriting a file securely generally requires root access to the drive and even that isn't guaranteed to work. So, yeah, there's not really much you can do, other than warn people that using unencrypted wallets at any time for any reason is a Bad Idea except in very specific circumstances.

[marcus_of_augustus]

Right. I'm just a little concerned that since on most OS "delete" actually just changes (removes) the file header to make that space available for new writes and the file data is hardly ever changed then there is a trail of remnant wallets all over the place waiting for a disk recovery tool to come along. Even remnant encrypted wallet data may become a target if valuable enough.

Unless someone is assiduously creating new wallets and moving all their coins to the new wallets as they go then wallets that are moved around need to be aware of where all the old copies (including remnants) have been left behind, since any of those copies, live or "deleted", will potentially be a vulnerability. I would make "only secure delete" a policy of any wallet handling tool.

Agreed.  I just feel like there's very little I can do to accommodate this, besides making a good-faith effort to overwrite the data in place instead of just doing a vanilla delete operation.  As I mentioned in my previous post:  this attack vector is basically nil if you wallet was born encrypted.  Any users with any significant amount of money that don't encrypt their wallets: well they are probably at greater risk of other attack vectors.

Perhaps I'll add a warning (along with the millions of others) which warns about "residue" wallets even after deletion.  I don't think there's anything else I can/should do about it.  Unless you have recommendations...? 

Fair enough. Small recommendation, maybe you could also have a warning for people importing currently unencrypted wallets (or at any time previously unencrypted) to transfer the contents of these wallets into encrypted wallets. A warning to "Begin as you mean to go on" so to speak.

Foxpup:

The "srm" (secure remove) tool in Linux doesn't require root access nor knowledge of the filesystem. The default 35 times overwrite is over the top but the -dod (7 times) or -doe (3 times) flags are prudent security. https://en.wikipedia.org/wiki/Srm_%28Unix%29 This would cover Unix, Linux and MacOSX users, but it introduces a dependency.

[cypherdoc]

So out of interest, what is the basic strategy for Armory's physical handling of wallets?

Specifically;

1) are unencrypted private keys ever held on disk (or only ever in RAM)?

2) how are any old copies of wallet files that reside on disk at any point deleted/erased?

i would just use an offline wallet which is why i chose Armory to begin with. 

[marcus_of_augustus]

So out of interest, what is the basic strategy for Armory's physical handling of wallets?

Specifically;

1) are unencrypted private keys ever held on disk (or only ever in RAM)?

2) how are any old copies of wallet files that reside on disk at any point deleted/erased?

i would just use an offline wallet which is why i chose Armory to begin with. 

Well thanks for the useless, random, tangential comment. Offline or online is not relevant to my question, if you don't realise that yet.

[etotheipi]

So out of interest, what is the basic strategy for Armory's physical handling of wallets?

Specifically;

1) are unencrypted private keys ever held on disk (or only ever in RAM)?

2) how are any old copies of wallet files that reside on disk at any point deleted/erased?

i would just use an offline wallet which is why i chose Armory to begin with. 

Well thanks for the useless, random, tangential comment. Offline or online is not relevant to my question, if you don't realise that yet.

I think his point was that the questions you ask are irrelevant if your computer is offline.  Stored in RAM or HDD, encrypted or not, doesn't really matter.  The only thing to worry about is eventually discarding it, but if you stored any significant coins on it, ever, thermite is the only real solution

[Raoul Duke]

So out of interest, what is the basic strategy for Armory's physical handling of wallets?

Specifically;

1) are unencrypted private keys ever held on disk (or only ever in RAM)?

2) how are any old copies of wallet files that reside on disk at any point deleted/erased?

i would just use an offline wallet which is why i chose Armory to begin with. 

Well thanks for the useless, random, tangential comment. Offline or online is not relevant to my question, if you don't realise that yet.

Here is a relevant question: Have you asked the same questions to the regular Bitcoin client developers?

[etotheipi]

So out of interest, what is the basic strategy for Armory's physical handling of wallets?

Specifically;

1) are unencrypted private keys ever held on disk (or only ever in RAM)?

2) how are any old copies of wallet files that reside on disk at any point deleted/erased?

Here is a relevant question: Have you asked the same questions to the regular Bitcoin client developers?

I have, and the answer by them is:  "This problem is out of scope of our software.  This is an OS/filesystem issue, not an application issue." 

And of course:  after the wallet-not-actually-encrypted but 0.4.0, their wallets are now born encrypted, too.  If your wallet is unencrypted and then encrypted, they just delete the old wallet and rewrite a new, born-encrypted wallet.  But that's because in their case, they don't have a way to truly purge the old wallet using BSDDB.   Nonetheless, their implementation would suffer from the same filesystem issues.