Indeed!  It's on the webpage:  http://bitcoinarmory.com/index.php/armory-wallet-files

However, I'm working on a new wallet format now, so that will be changing pretty substantially in the near future, but there's also a lot of work left to be done (so maybe not near-near future...)

Hahah,

Actually Armory already does this.  The algorithm could be improved, but it will try to collect dust and throw it on top, as long as it doesn't induce a fee, and it doesn't increase the address linkages (it is from addresses already on the input side).

This will be improved in the future, as someone pointed out that I can treat addresses that have already been linked, as a single "group."  Thus, I can throw in dust from all groups of addresses already represented on the input side, without damaging the input linkages.

BIP 32 was developed by the Bitcoin-Qt devs, and will hopefully make it's way into Bitcoin-Qt in 0.8.  But it's looking like it'll be a later release.  This will provide multi-chain deterministic addresses (create multiple wallets built from one seed).  I plan to implement the same thing in Armory so it will be compatible.

But at the moment, you are correct:  this cannot be done in Bitcoin-Qt, and it's a real shame because requiring-regular-backups-at-the-cost-of-losing-your-money is a real problem in the world of end-user software.  It's never worth dealing with backups until it's too late, i.e. "ehhh, I'll set it up next week."  Plus, that means that your backup solution can't be extremely secure, because it still needs to be convenient since you're doing it multiple times.  At least with a deterministic wallet, you can backup once, drive to the bank and put it in your safe-deposit box.  Once.

Personally, I think this is the number one reason to use these alt clients, because the ability to backup once, forever is such a powerful feature.

Ente

Actually, I think you just need to recompile.  Just type "make".  And you shouldn't do a checkout on the remotes/origin directly.   Just "git checkout dev" then do a "git pull origin dev".

Otherwise,  that's the right spirit!  Just compile it first,  *then* look for bugs :-)

I would only be concerned if I make updates to the wallet code, thus risking the possibility of making errors in computing addresses, etc.  But the wallet code hasn't been touched in months (except for some tweaks to the keypool). 

When I make the new wallet, I expect people will want to test on testnet first, or with smaller amounts of coins.  For these types of releases, though, the worst thing that will happen is that pressing some buttons will throw errors strange errors, or maybe even crash Armory.  If that happens, please send me the log file.  But I wouldn't worry about losing coins...

It's like VIM:  it's got a bit of a learning curve to be able to use it efficiently, but there's so many shortcuts and hotkeys that you can really fly once you have some experience (and yes, I do 100% of my code development in vim )


I think we're saying the same thing:  I showed partitioning at the second level, but it really would be any level low enough to meet some kind of criteria (though I'm not sure what that criteria is, if you don't have any of the data yet).  It was intended to be a "partitioning from the bottom" and then filling up to the root once you have it all.  

I imagine there would be a P2P command that says "RequestHashes | HeaderHash | Prefix".  If you give it an empty prefix, that means start at root:  it will give you the root hash, followed by the 256 child hashes.  If you give it a prefix "\x01" it gives you the hash of the node starting at '\x01' and the hashes of its 256 children. This is important, because I think for this to work, you have to have a baseline for what the tree is going to look like for your particular target block.  I think it gets significantly more complicated if you are aiming for partitions that are from different blocks...

Then there'd be another command that says "RequestBranch | HeaderHash | Prefix | StartNode".  The header hash/height would be included only so that peers that are significantly detached from your state won't start feeding you their data.  i.e. Maybe because they don't recognize your hash, or somehow they are more than 100 blocks from the state you are requesting.  If the peer's state is within 100 blocks, they start feeding you that partition, ordered lexicographically.  They'll probably be transferred in chunks of 1000 nodes, and then you put in the next request using the 1000th node as the start node to get the next chunk.  Since we have branch independence and insert order independence, the transfer should be stupid simple.

Also something to note:  I think that the raw TxOut script should be the key for this tree.  Sure, a lot of these will have a common prefix, but PATRICIA trees will compress those anyway.  What I'm concerned about is something like multiple variations of the same address, such as a TxOut using hash160 vs using full public key.  That can lead to stupid side-effects if you are only requesting by addr.  

Actually, there's a bug fix for that in 0.86.2 -- please verify for me that the issue is resolved (without breaking anything).  Thanks!

@d'aniel:  You might be right that it's possible to reconstruct the tree from an amalgamation of closely related states.  Though, I'm concerned that there's too many ways for that to go wrong.  Let's start a thought-experiment:  I have a fresh slate, with no Tx and no UTXO.  I then execute 65,536 requests for data, and download each one from a different peer (each request is for a different 2-byte prefix branch).  I will assume for a moment that all requests execute successfully, and we end up with something like the following:



A couple notes/assumptions about my drawing:  

• (1) I have drawn this as a raw trie, but the discussion is the same (or very close to the same) when you transition to the Patricia/Brandais hybrid.  Let me know if you think that's a bad assumption.
• (2) We have headers up to block 1000.  So we ask one peer that is at block 1000 for all 65,536 trie-node hashes.  We verify it against the meta-chain header.
• (3) We make attempts to download all 65,536 subtrees from a bunch of peers, and end up mostly with those for block 1000, but a few for 995-999, and a couple have given us block 1001-1002 because that was what they had by the time we asked them to send us that branch.  We assume that peers tell us what block they are serving from.
• (4) Some branches don't exist.  Even though the second layer on the main network they will always be at 100% density, there may be various optimization-related reasons to do this operation at a lower branch level where it's not 100% density.
• (4a) I've used green to highlight four situations that I don't think are difficult, but need to be aware of them.  Branch \x0202 is where the node hashes at block 1000 say it's an empty node, but is reported as having data by the peer serving us from block 1001.  \x0203 is the same, but with a peer serving block 993 telling us there is data there.  \x0302 and \x0303 are the inverse:  block 1000 has hashes for those trie-nodes, but when requested from peers serving at other points in time, they report empty.
• (5) Downloading the transactions-with-any-unspent-txouts from sources at different blocks also needs to be looked at.  We do eventually need to end up with a complete list of tx for the tree at block 1000 (or 1002?).  I'm expecting that any gaps can be filled with subsequent requests to other nodes.

So, as a starter algorithm, we acquire all this data and an almost-full UTXO tree.  We also acquire all of the blocks between 993 and 1002.   One branch at a time, we fast forward or rewind that branch based on the tx in blocks 993-1002.  It is possible we will be missing block data needed (due to #5), but I assume we will be able to acquire that info from someone -- perhaps this warrants keeping tx in the node's database for some number of blocks after it is depleted, to make sure it can still be served to other nodes catching up (among other reasons).

On the surface, this looks workable and actually not terribly complicated.  And no snapshots required!  Just ask peers for their data, and make sure you know what block their UTXO tree is at.  But my brain is at saturation, and I'm going to have to look at this with a fresh set of eyes later this week, to make sure I'm not neglecting something stupid.

I'll tag it when it's a real release -- right now it's still in dev branch because it's still a testing release.  Second, I have been pretty lazy about tagging versions, but I will start doing so more religiously, now that there are a couple build systems relying on them.  So, I will be sure to tag all future full-releases.   Third, this will actually be 0.86.2 -- it's really just a bugfix/polishing release.

On the topic of testing... anyone currently using it?  Any problems with it?  It's mostly small updates, so I don't expect a lot to go wrong with it.  But I still need some feedback to know for sure.

Perhaps it's a single-bit error!  I documented my experience with this here.  It was maddening, to be able to confirm 1.5 million tx hashes, and have exactly one fail.  It took me a while, but I eventually narrowed it down.  I was able to manually compare the tx from a different source to the one in my blkfile. 

Well, I am not seeing a way around using snapshots.  I was hoping someone more insightful than myself would point out something simpler, but it hasn't happened yet...

Also, as mentioned earlier, I think snapshots are wildly expensive to store.  I think if a node wants block X and the snapshot is only for block X+/-100, then he can get that snapshot at X and the 100 blocks in between and rewind or fast forward the UTXO tree on his own.  The rewinding and fast-forwarding should be extremely fast once you have the block data. 

Although this does open the question of how nodes intend to use this data.  If it turns out they will want to understand how the blockchain looked at multiple points in time, then perhaps it's worth the effort to store all these snapshots.  If it never happens, then the fast-foward/rewind would be better.  My thoughts on this is:

(1) The gap between snapshots should be considered relative to the size of a snapshot.  My guess is that 100 blocks is smaller than a snapshot, and thus you never need more snapshots than that
(2) Snapshots at various points in time actually won't be that useful, other than helping other nodes download.  These kinda-full-nodes only care about the latest state of the UTXO tree, nothing else.  If you think there's other reasons, please point them out.

Well, I'm hoping that it will be possible to not need to "catch up with the network" in the current sense.  Certain types of nodes will only care about having the final UTXO set, not replaying 100 GB of blockchain history just to get their 2 GB of UTXO data.  I'd like it if such nodes had a way of sharing these UTXO trees without using too much resources, and without too much complication around the fact that the tree is changing as you are downloading. 

One core benefit of the trie structure is that nodes can simply send a raw list of UTXOs, since insertion order doesn't matter (and thus deleted UTXOs don't need to be transferred).  Sipa tells me there's currently about 3 million UTXOs, so at 36 bytes each, that's about 100 MB to transfer.  There is, of course, the raw transactions with any remaining UTXO that need to be transferred, too -- currently 1.3 million out of about 10million total tx in the blockchain.  So that's probably another few hundred MB.  But still only a fraction of the 4.5 GB blockchain.   



That's a generalization of what I proposed:  if(blockheight mod 2016 == 0) {storeSnapshotFor2016Blocks}.  Clearly, the modulus needs to be calibrated...  The problem is these snapshots are very expensive, so we would prefer not to do snapshots at all.  But one may be necessary.  Hopefully not more than that.  Although it would be great if I just overlooked something and we could do this without snapshots at all.

A block, as it is stored on disk is very straightforward and parsed very easily:


The blk*.dat files are just a list of binary sequences like this

Good idea.  It's a very good idea ...

...and I secretly came up with this idea a couple weeks ago  Inspired by a user coming to me for help because he couldn't download the blockchain from his crappy internet connection.  He wanted to send me his watching-only wallet and I would generate the tx for him.  I've been pondering this idea for the last couple weeks, and it's something I'm keeping my eyes open for.  I was keeping it secret, and maybe it would just show up in some random Armory release one day

I seriously don't have any plans for this in the next one month.  But 2+ months, it's a distinct possibility...

Btw, I have a BIP 32 implementation in the "newwallet" branch of Armory.  It's only the crypto part -- I haven't been able to integrate it into a new wallet format yet (and thus, not usable in Armory yet).  But it includes the ChildKeyDeriv() source code, and a unit test for both HMAC-SHA512 and the ChildKeyDeriv().

The unit tests may not be entirely accurate, because I made them before sipa decided that all derivations should use compressed public keys.  But the algorithm is otherwise 98% what is described in the BIP.

Try some Unbalanced Oil & Vinegar.

Quantum computers indeed would break ECDSA, allowing private keys to be derived from public keys in polynomial time.   I suppose a mathematical breakthrough in finite field theory could do the same, but astoundingly unlikely.  But Bitcoin is the least of the world's problems when this happens:  the entire web of trust, SSL/PKI/CA architecture would be broken, rendering useless most internet crypto that we all rely on.  The world will have a lot of problems when QCs start to come into regular existence.  In particular, there will be a gap between when they start to become available to the wealthy, but are not ubiquitous enough to enable widespread quantum encryption to replace it.

But it still wouldn't be that bad.  There are quantum-resistant algorithms that can be run on classical computers.  So it's pretty ridiculous to single out Bitcoin for being susceptible to QCs without mentioning that all sensitive communications on the internet will be susceptible.  And without mentioning that there are alternatives.  

In fact, Bitcoin is wholly prepared to deal with this:

(1) By avoiding address reuse, you're 99% safe even in a world full of quantum computers -- because your address is the hash of your public key, and thus no one knows your public key until you've already submitted a signed transaction to the network.  Even in 50 years when QCs are "fast", they probably won't be fast see your public key, back out your private key, sign your coins to themselves, and then broadcast that tx ... all before your transaction has propagated to a majority of nodes.  Yes yes, isolation attacks... but see #2

(2) The Bitcoin network has the property that you can actually change stuff like crypto algorithms, hashing algorithms, etc, by incrementing a version number and encouraging all nodes to use the new [QC-resistant] algorithms.  Of course, it's not a trivial change, but the network would survive and users would be able to continue using the old encryption for a short time until the transition is made, as long as they never reuse addresses.   Especially large transactions could be submitted directly to miners to avoid isolation issues.

I would say the OP quote is simply under-informed.

P.S. -- Also, these worst case scenarios assume that QCs just pop out of nowhere and suddenly exist on everyone's desktops.  The fact is, we'll see QCs coming decades in advance... so all this "OMG need to swap crypto algorithms immediately!" stuff is overblown.

Please help test version 0.86.2-beta!  Lots of bug fixes and small improvements that should make the Armory experience smoother.  Please help test and give me feedback.

Luckily, most of the changes in this version are isolated, so there's not a lot of bugs expected.  But you never know until people start using it!  But the point was, it should be fairly stable already.

---

Windows 64 installer:   armory_0.86.2-beta_win64.msi
Windows 32 installer:   armory_0.86.2-beta_windows_all.msi
Ubuntu/Debian 64-bit installer:  armory_0.86.2-beta_amd64.deb
Ubuntu/Debian 32-bit installer:  armory_0.86.2-beta_i386.deb

For those compiling from source, the latest is on the "dev" branch i nthe git repo.  I just realized I should start a "testing" branch, and use that as a holding-cell for soon-to-be-master upgrades, and then I don't have to keep telling you guys what branch to use.  Not sure why I didn't do this sooner...


All features new to 0.86.2:

   - Added Root Key to "Backup Individual Keys"
        So you can backup your imported keys and deterministic "seed"
        from one operation, instead of two.  Key pool addresses are now
        accessible, too.

   - Right-click Ledger Options
        Added right-click menu to ledger for quick access to transaction
        and wallet information.  Also includes options for opening your
        web-browser right to tx or address information in blockchain.info.

   - Offline-Sign Confirmation & Warnings
        Offline signing now displays appropriate warnings about what users
        should verify for before signing and broadcasting.

   - Added Comments to Coin Control (Expert Mode)
        Abbreviated comments are now show in the coin control selection
        window, with full comments available via mouse-over text.

   - Bugfix: Disappearing Addresses
        Some startup operations were inadvertantly "rewinding" wallets with
        unused addresses, causing those addresses to disappear from the
        address list, and then shown again when the user requested another
        address.  Resolved.

   - Bugfix: Ledger Sorting
        All fields in the primary ledger are sortable.  Some fields become
        unsortable as a side-effect of ledger optimizations in v0.85.

Use the "windows_all" version.  That version is actually built on WinXP 32-bit, though it works fine on Win7-64bit too.

I have had multiple users request this in Armory.  My response is controversial, but I want to throw it out there as food for thought, and you can ignore it if you don't like it:

If you have an encrypted wallet and all your backups are encrypted as well, including encrypted paper backups -- you have a brain-wallet.  Not exactly a brain-wallet, just all the downsides of brain-wallets.  You are at significant risk of losing your coins no matter how good you think you are.  Either because you forget your encryption passphrase because you only used it once five years earlier when you made the backup, or because you get hit by a bus and take the passphrase (and BTC) to your grave with you.  If the encryption is implemented properly, the backup will be useless without the passphrase.

I have no problem with having encrypted backups in addition to an unencrypted backup stored somewhere secure such as a safe or safe-deposit box.  But I think if the option is there, a lot of users will make 100% of their backups encrypted, and a lot of BTC will be permanently lost.