So out of interest, what is the basic strategy for Armory's physical handling of wallets?
Specifically;
1) are unencrypted private keys ever held on disk (or only ever in RAM)?
2) how are any old copies of wallet files that reside on disk at any point deleted/erased?
Here is a relevant question: Have you asked the same questions to the regular Bitcoin client developers? I have, and the answer by them is: "This problem is out of scope of our software. This is an OS/filesystem issue, not an application issue." And of course: after the wallet-not-actually-encrypted but 0.4.0, their wallets are now born encrypted, too. If your wallet is unencrypted and then encrypted, they just delete the old wallet and rewrite a new, born-encrypted wallet. But that's because in their case, they don't have a way to truly purge the old wallet using BSDDB. Nonetheless, their implementation would suffer from the same filesystem issues. |
|
|
|
So out of interest, what is the basic strategy for Armory's physical handling of wallets?
Specifically;
1) are unencrypted private keys ever held on disk (or only ever in RAM)?
2) how are any old copies of wallet files that reside on disk at any point deleted/erased?
i would just use an offline wallet which is why i chose Armory to begin with. Well thanks for the useless, random, tangential comment. Offline or online is not relevant to my question, if you don't realise that yet. I think his point was that the questions you ask are irrelevant if your computer is offline. Stored in RAM or HDD, encrypted or not, doesn't really matter. The only thing to worry about is eventually discarding it, but if you stored any significant coins on it, ever, thermite is the only real solution  |
|
|
|
Right. I'm just a little concerned that since on most OS "delete" actually just changes (removes) the file header to make that space available for new writes and the file data is hardly ever changed then there is a trail of remnant wallets all over the place waiting for a disk recovery tool to come along. Even remnant encrypted wallet data may become a target if valuable enough.
Unless someone is assiduously creating new wallets and moving all their coins to the new wallets as they go then wallets that are moved around need to be aware of where all the old copies (including remnants) have been left behind, since any of those copies, live or "deleted", will potentially be a vulnerability. I would make "only secure delete" a policy of any wallet handling tool.
Agreed. I just feel like there's very little I can do to accommodate this, besides making a good-faith effort to overwrite the data in place instead of just doing a vanilla delete operation. As I mentioned in my previous post: this attack vector is basically nil if you wallet was born encrypted. Any users with any significant amount of money that don't encrypt their wallets: well they are probably at greater risk of other attack vectors. Perhaps I'll add a warning (along with the millions of others) which warns about "residue" wallets even after deletion. I don't think there's anything else I can/should do about it. Unless you have recommendations...? |
|
|
|
(2) Wallets are all kept in the .armory or AppData/Armory directory. A backup is always created & maintained in the same directory for redundancy and auto-restore-on-corruption. When you decide to delete a wallet through the wallet properties, both main and backup will be deleted. When you import one, a copy is made into your .armory or AppData/Armory directory, leaving the original intact (option coming soon to add a remote wallet so you can keep it on a USB key or network mount).
When you say "deleted" here, do you use something like a "srm" tool or similar multiple overwrites? Thanks again. Like the description above, I don't spend much time thinking about filesystem issues. I just use the regular OS delete operation, and let it deal with it. But now that you mention it, there's no reason not to do something more elaborate, such as overwriting with random data before the OS deletes it. Worst case: there is no benefit. But, there is a chance that the overwrite happens in-place and that's an improvement! I guess the important part is that the final file you get can't have any key data left over, which makes it safe[er] to back it up to a network location if you wanted to (though, I still wouldn't myself, but that was the problem with the 0.4.0 bug: people backed their files up to dropbox, etc, because they thought their key data was encrypted). |
|
|
|
In the end, everyone who cares about this tiny attack vector uses wallets that were born encrypted. And Armory does guarantee that born-encrypted wallets are secure.
If you're worried about (2) and (3) and electron-tunneling microscopes pulling your key off even after shredding: create your wallet encrypted and never remove the passphrase! Problem solved!
The problem is not solved for people who don't know about these things, create an unencrypted wallet, then later learn about the danger and encrypt the wallet, then their computer gets stolen, but it's okay because their wallet is encrypted, so they restore from a backup only to find that all their addresses have been cleaned out completely, then they sue you because you "guaranteed" that their unencrypted wallet would be overwritten. I am not suggesting that you find a way to ensure that unencrypted wallets are completely destroyed (since that's almost impossible), just that it's probably not the best idea to make guarantees with other people's money that you can't back up.  I never guaranteed it! Or rather, I never intended to. What I "guarantee" is the file format does not allow for leftover plaintext keys when you encrypt an unencrypted wallet. I was very careful about that. If the filesystem has in-place file modifications, then so do my wallets. Plus, that whole open-source license thing prevents me from being sued, in the event that it doesn't work as expected . But I poured my heart into this wallet for weeks, and 1000 lines of unit-testing. I'm very confident it's as good as it can get. On that note, there's probably a good chance that encrypting in place will work, because the file size isn't changing at all when the keys are encrypted. So the filesystem has one less reason to rewrite the data to a new location on disk. Again, no guarantees, but it's a pretty solid solution! P.S. -- you'll notice that my quote was "the new file is guaranteed not to contain unencrypted key data" I didn't say anything about the hard-disk |
|
|
|
- If you add encryption to an unencrypted wallet, the old keys are overwritten in-place so that the new file is guaranteed not to contain unencrypted data.
How exactly do you guarantee that files will in fact be overwritten "in-place"? This is not only operating-system and filesystem dependent, but even you can manage to do this in all operating system and on all filesystems, the hardware is not guaranteed to write data where the operating-system tells it to. Solid-state drives in particular are guaranteed not to do this due to their wear-levelling mechanism. I'm sorry, but if an unencrypted private key is ever written to disc, the only secure options are to stop using that private key or start using thermite. This topic has been beaten to death on the mailing list and on IRC: this issue is out of scope (that's the stance of the main devs, too). We could spend our lives trying to accommodate all the different filesystems and journaling paradigms to try to guarantee data was overwritten, but it's 1000x more effort than the value you get out if it. The important part is that the filesystem reports that the file has been overwritten in place. If so, then the attacker will require root access to your hard-drive to get bit-level access to your drive to get the stale data. This means one of the following is true: (1) You're already compromised -- he can also pull the key out of memory next time you unlock (2) You discarded the drive and you should've blanked the data before doing so (3) Your drive was stolen In the end, everyone who cares about this tiny attack vector uses wallets that were born encrypted. And Armory does guarantee that born-encrypted wallets are secure. If you're worried about (2) and (3) and electron-tunneling microscopes pulling your key off even after shredding: create your wallet encrypted and never remove the passphrase! Problem solved! |
|
|
|
So out of interest, what is the basic strategy for Armory's physical handling of wallets?
Specifically;
1) are unencrypted private keys ever held on disk (or only ever in RAM)?
2) how are any old copies of wallet files that reside on disk at any point deleted/erased?
(1) Plaintext keys are only written to disk if the state of the wallet is ever "unencrypted." - If you create the wallet encrypted, only the encrypted versions of the keys ever touch the hard-drive.
- If you add encryption to an unencrypted wallet, the old keys are overwritten in-place so that the new file is guaranteed not to contain unencrypted data. My wallet file format and technique was motivated by my discovery of the 0.4.X wallet-not-actually-encrypted bug -- I designed it to be simple and maintainable without external tools so I have full control over the data.
- Decrypting the keys to use them for signing only decrypts them into RAM, where they are held in "SecureBinaryData" containers which are page-locked and blanked out in RAM when they go out of scope or explicitly using key.destroy().
The only catch with #3 is that Python is handling the SecureBinaryData objects, and python sometimes does weird stuff, memory-management-wise. But the worst that could happen is that the keys are held in RAM longer than they should be. They should still obey page-locked rules (which almost guarantees that the data won't be swapped, but most operating systems can't guarantee that). (2) Wallets are all kept in the .armory or AppData/Armory directory. A backup is always created & maintained in the same directory for redundancy and auto-restore-on-corruption. When you decide to delete a wallet through the wallet properties, both main and backup will be deleted. When you import one, a copy is made into your .armory or AppData/Armory directory, leaving the original intact (option coming soon to add a remote wallet so you can keep it on a USB key or network mount). |
|
|
|
Teaser:Gonna try to make this and URI handling part of Armory version 0.75. And it will be the first version distributed with fully automated installers (with uninstallers), in both Linux and Windows.  
|
|
|
|
Any ideas on how I might create an all-in-one offline installation in Ubuntu? For now, maybe I'll just "endorse" 10.04 32-bit, execute the offline package download on 10.04 and tar up all the dependencies into one downloadable thing.
Well, you could include the dependencies, but the dependencies have dependencies of their own, so you have to include those, as well as their dependencies... If you keep going on like that you'll end up creating a whole distribution just for Armory. Which is a perfectly valid option, and probably a good idea if you want to ensure a completely secure software environment, but it's probably more effort than it's worth at this stage. Other than that I don't think there's any way to guarantee that all the dependencies will be met without having to go online. That's easy. I create a fresh installation, then execute the offline-package-installation -- I select the two top-level dependencies in Synaptic and it collects the entire dependency tree for me. When I select "Generate Download List", I get a wget-list of all 65 packages (it's 65 if I do the build-dependencies). I run the script and zip it up the directory. I think it's about 100 MB. It's not small, but it's acceptable. But it will be tied the particular distro I did it for... As for the python thing: check that you don't have both pythons on your system. As I said before, even though 11.10 is python2.7, it did work with my 2.6 build, because it apparently has libpython2.6.so on it (no idea why). Maybe I should try the other way: build everything on 2.7 and it might work on lower...? |
|
|
|
Ok, I removed the previous package. Instealled the new amd64-python2.7... ...and it works As soon as I get some extra BTC in my wallet gonna throw you some for all the trouble. After all I was the only one to complaint. FYI, you're not going crazy. I just tested the python2.6 package on a fresh install of 12.04 and it failed. I had to recompile the package without the (= python2.6) in order to get it to install, and it appears that the dependency check is valid. Argh! So this means I will have to have 4 different packages for Ubuntu: {i386, amd64}{python2.6, python2.7}. Anyone who cannot use one of those packages can just follow the regular build-from source instructions: they've been fairly unobtrusive so far, just needed the desktop icons. If you're using a different distribution than one of the modern Ubuntus, your probably capable of checking your python version or compiling yourself (I'll list Ubuntu release versions with the packages). Any ideas on how I might create an all-in-one offline installation in Ubuntu? For now, maybe I'll just "endorse" 10.04 32-bit, execute the offline package download on 10.04 and tar up all the dependencies into one downloadable thing. |
|
|
|
Also I tested your custom port version and tried a handful of 62,200 ports and it always gave that problem but 61,119 and below appear to work. No idea whats up with the 62200 and above.
Good to know! Maybe I will choose a port that is in a more-normal range: I assumed that picking a huge number would decrease the likelihood that any other program was using that port, but apparently not all systems like those big ports... Any guidance on how I should select a port? Bitcoin uses 8333 and 18333 (testnet)... how were those chosen and how did we know they weren't going to be consumed by other processes? Any other comments about it working in Windows? I'm curious if there's any continuing artifacts due to RAM-reduction. It seems to work fine on my VM, but many others have reported issues I can't reproduce  |
|
|
|
So I remember seeing you talking about adding the no forced transfer fee https://bitcointalk.org/index.php?topic=22434.100 into this. Are you still planning this? I spent many hours trying to compile that, but couldn't get it to work on windows and if you are planning on incorporating it sometime soon then I would be able to stop messing with it. I believe, in that same discussion, I mentioned how it wasn't going to be possible until I get independent networking. Because the default Satoshi client behavior rejects the zero-fee tx, and Satoshi is Armory's only link to the Bitcoin network. There's no way around it without implementing something hack-y, or just waiting until I can get time to do independent networking. Unfortunately, that may be a while... However, Armory has the same fee rules programmed into it as the Satoshi nodes, so if you always set it to zero, and accept the recommended fee if it pops up, you're paying the minimum necessary to meet those rules. Beyond that, there's not much I can offer you (other than a promise that it will be implemented when independent networking is). |
|
|
|
For reference, I have spent a lot of time in Thailand and even learning the Thai language. If you wander around you'll find lots of poverty and homeless. However what is amazing is that every single one of them has a cell-phone! Even when we went out to the middle of nowhere in a desert area with little huts/tents. I didn't see any electricity hookups, but they all had cellphones. I have no idea how they even charge them! One of them using it to play music on speakerphone. But I don't know if they have service plans with those phones... maybe they just play games an music with them? Seems unlikely. Wiki has a list of cellphone ownership rates, showing Thailand with 105% ownership. From looking at the list, there's actually quite a few such countries, and some of them are places that have high poverty rates, and likely low bank-account-owning rates. So contrary to some of the replies to this thread, I think there are quite a few places where Bitcoin on the phone can really catch on. Only problem with Thailand is that their currency has almost an identical symbol  |
|
|
|
Ok, I removed the previous package. Instealled the new amd64-python2.7... ...and it works As soon as I get some extra BTC in my wallet gonna throw you some for all the trouble. After all I was the only one to complaint. Fantastic! I wonder if this means that I have to maintain separate packages based on python version... You might've been one of the only people to try it, too! I don't know how widespread the issue is when I only have a couple people try and one fails... For reference, what OS are you running? |
|
|
|
Hey, wait a minute, if that's all it was, then it shouldn't have worked at all, because I don't have python 2.7, so the python version obviously doesn't make any difference at all. Where exactly did psy's error come from? You know what, just remove libpython from the dependency list, put up one version per architecture, and see if that works on both my and psy's system. If it does, problem solved. Except for the problem of why it didn't work on psy's system originally after he hacked the dependencies, that trick should have worked...
Yeah, I was hoping that my dependencies hack would work. After all it asks for libpython2.6 or superior, doesn't it? Isn't that what >=2.6 means? Equal or superior? I will try it again and post results in a while. Well, the dependency in that case was wrong. It needs to be =2.6, because the library was compiled with libpython2.6, which causes fatal problems with the program if the user only has python2.7. Make sure you "dpkg -r armory" before trying. I just want to make sure you don't add complications by having mixed version of the package. I haven't figured out how to get dpkg-builder to increment the debian build numeber (i.e. increase it from "armory_0.74-1" to "armory_0.74-2"). One more thing to work on... |
|
|
|
Strictly speaking, it requires any localhost version of bitcoin to be running. It can be bitcoin-qt or bitcoind. As long as there is a localhost node to do blockchain validation and networking fanciness (peer-finding, blacklisting, address collection, etc). So some form of bitcoin is technically required, but I can't put it in the depends list for fear that a distribution will not have it whatsoever, and then the package will be essentially uninstallable. However, with packages out of the way, I can get on with doing my dev plans to get to Beta, which then will lead me to dealing with networking-independence (which will prob be the first priority after Beta). Such update will also likely to lead to me removing some of my blockchain code and using the underlying library (libcoin, libbitcoin?) which will then allow it to work on all architectures smoothly: Win32 included. I believe I can stop thinking about debian packages for now. If you are using Ubuntu, you can use the packages (which seem to work with mixed python no problem), and otherwise you can build from source. You should be able to "make install" now, and then "chmod 755 dpkgfiles/postinst; dpkgfiles/postinst configure" in order to install the menu entries. I haven't tested this, but if you're in a non-standard distribution, you are probably capable of handling the subtleties (please email me to work it out if there are problems). |
|
|
|
So, for the purposes of offline systems: users want to be able to setup their offline system without ever connecting it to the internet. My solution to this, so far, has been --(1) Set it up with an internet connection
--(2) Disconnect network cable forever
--(3) Create first wallet
However, why should users have to do this? The Windows version bundles everything into the .msi file. Of course, the user could go to the offline system with the package, and use the "Generate Download Script" to get the wget-list of dependencies to execute online and bring back... but that doesn't work if they don't own any online linux machines. And even if they do, it's a mild PITA. My other option would be to actually distribute the packages, or at least the shared-objects. But I wonder if there's licensing issues with that. Plus the packages would probably huge, and might require a separate package for each supported distribution (one for 10.04, one for 10.10, 11.04 11.10, and then double that for 32-bit and 64-bit). So, what options do I have to enable a single-step installation on a freshly-installed offline system having none of the current dependencies met? EDIT: On second thought, maybe I can add all the .so files to the make-install step. It would copy the .so files from my build system right into the execution directory. Though, I'm concerned about the python version issues: combining a libpython2.6.so file with a system full of python2.7 libraries...? |
|
|
|
The python 2.7 version works fine on my system even though I have python 2.6, so either you've screwed up the dependencies again... Package: armory Version: 0.74-python 2.7-1 Architecture: amd64 Maintainer: Alan C. Reiner < alan.reiner@gmail.com> Installed-Size: 13296 Depends: libpython 2.6, python-qt4, python-twisted ...yup, you've screwed up again  ...or it doesn't depend on a particular version of python at all, in which case you can safely remove it from the dependency list, and I have no idea what was causing psy's problems. Gah! I added python to the version number, but forgot to automate the depends line. This really is a PITA! I just replaced that version on the downloads page. It should be correct. At least I'm getting these stupid little things out the way before I declare a real release... |
|
|
|
Okay! I think I got it! Please execute "sudo dpkg -r armory" before attempting to install this package. Since the version number didn't increment, your system may be confused by this... If you had problems before, please try it again by downloading the appropriate version from the downloads page. - It looks like dependencies worked this time. I used the short list produced by Foxpup. I tested it on a mostly-fresh system and got the appropriate errors. Double-clicking and letting the Ubuntu package manager deal with it did result in those dependencies being installed.
- Menu options now show up in the "Internet" menu on my computer. Thanks again to Foxpup for that.
- There is now a separate version for python2.6 and python2.7. For me, the python2.6 version worked on my 11.10 system with python2.7, but clearly others were having problems. So I compiled it again on the 11.10 system and posted the .deb file as well. The packages are marked with arch and python version.
As for the menu options: it is just my main system where the menu got screwed up. When I install the package on a [mostly-]fresh new system, the menu options appear as expected. Please let me know... P.S. -- Foxpup, I installed nothing extra in the Ubuntu 11.10 system. "python --version" replies with "Python 2.7.2+". And when I run the .deb package everything goes smoothly and it works. It does look like I have python2.6 libraries on that system... that would explain it... (libpython2.6.so.1.0)
|
|
|
|
Clearly something was wrong with the dependencies, but it still worked on my system, so I thought I'd throw it out there to find out that no one else could use it, anyway. I'll try to get that all straightened out today. If you used the package and you were successful, please let me know! All I've heard is unsuccess!
Well of course it worked on your system, you built it after all, so obviously you have all the dependencies already. It also works on my system, because I already downloaded the dependencies when I was building from the source. It's the people who don't have the dependencies already who need the package to contain a correct dependency list, otherwise the package will install without the correct dependencies, and that won't work at all. I think the following dependencies are correct (for the binary package, compiling from source requires extra stuff): libpython2.6 python-qt4 python-twisted (all other dependencies are children of these dependenies, so can be excluded from the list) Actually, I meant that my .deb file produced on 10.04 with python2.6 worked on Ubuntu 11.10 which uses python 2.7 and it worked when I manually installed the dependencies. Which is why I posted the apt-get installer line in my original .deb posting. As for dependencies, I noticed that I had all my dependencies on the "Build-Depends" line, and no "Depends" line. I'm trying to get dependencies worked out, but every time I compile the .deb, it overwrites my depends line and puts it's own in there. Definitely needs some improvement. Btw, your suggestion for Categories seems to have worked... almost. I now have 2 of the 3 icons under my "Internet" tab, but the other one somehow ends up in "Other" anyway. I think I may have messed up my menu Working on the dependencies thing now... |
|
|
|
|
Show Posts
