I don't want to get side-tracked on the philosophy and use-cases of P2SH (though, I do agree, I much prefer plain multi-sig for escrow/contracts).  Even if users agreed to use plain multi-sig and avoid necessitating such backups, there's probably plenty of information in their wallet that is nearly-critical to have backed up -- perhaps contact information stored with the wallet, order numbers of various transactions, identities associated with non-wallet addresses, etc.  A lot of this isn't widely used now, but I expect it will be with the URI spec -- I've been pushing for contact information and order numbers to be included in URI strings so that when a merchant requests payment for something, they can auto-document the transaction for the buyer by pre-filling those fields which will end up in the user's wallet.  Even without P2SH, the user has a strong interest in securing that data (and users always fail if effort expenditure is required for a seemingly-non-essential operation).

I was under the impression that P2SH was intended for all multi-sig transactions, though you are right there is no reason plain-text multi-sig can't be used (is there? they were probably never added to isStandard...).  One reason for desiring P2SH, in general, is that very long scripts become part of the TxIns instead of the TxOuts.  This means that they will never bloat a pruned version of the blockchain.  If everyone only used P2SH, then no TxOut script would ever be longer than 22 bytes.

Re: "Relationship Objects"

I like the idea of defining "relationships" and having that be defined somewhere. However, where would it be stored?  I really think that however it is done, it must be coded into the wallet:  the user should never be in the position that they recover this wallet and think it's a single-sig wallet.  I want wallets to be dedicated to multi-sig or not.  If a user doesn't want multi-sig, create a regular wallet.  (easy for me to say, Armory is a multi-wallet app)

To me, it would be ideal if two-factor auth wallets were always generated by an offline computer.  It would create both wallets and generate 3 things to print:   paper backup of AB, A'B and AB' (where A is full wallet and A' is watching-only wallet).  AB goes into a safe-deposit box, A'B is entered into one device (desktop), AB' entered into the other device (smartphone).  Those could be backed up, too, in separate locations, with less security each than the AB backup.  A physical burglar would have to find both to compromise the funds.   I don't want to see A backed up separately from B. I really want the user to be backing up the wallet after the linkage has been defined in the wallet, so it's obvious from the backup that it is a multi-sig wallet and to which other wallet it is linked.  

The problem is, not all users want to do this with an offline system.  The desktop could make both and then just delete B (after it's backed up).  But some users want the keys to never be on the same device, ever.  There will probably have to be an interface to convert a regular wallet to a paired wallet, and allow users to merge them at a later time.  Perhaps, it is as simple as popping up a warning, suggesting they make a new backup when a link is created.

For escrow transactions and contracts, etc, I think you have to allow a regular, single-sig wallet to generate a private key intended to be used for this purpose.  This would be signified by the stored P2SH script in the wallet file, which would contain both public keys (and your own will be easily identifiable).  When the wallet is loaded, it will search all stored P2SH scripts and identify how they relate to your wallet.  But that will need to be backed up the moment you generate it.  I just don't see another way to be totally safe about this without using something as persistent as Dropbox (what other backup options are there for this, besides keeping this extra file on a network drive on another system, or external device?)

Done!  I just committed and pushed the updated code to the threading branch.  Anyone testing on OSX please pull and recompile and try again.  Hopefully that resolves the problem.

The good news is that I have had zero problems/bug reports about 0.84.5 except for this OSX problem.   The bad news is, I just realized that I have made some changes to master since I started the threading branch, and one of those changes is really important (the Bitcoin-Qt devs may start considering some Armory transactions non-std if this change isn't pulled).  This also means I'll have to do a tad bit more testing before release.  But so far, it's been going really well!

Horserider,

There is a couple different ways I am considering going.  I would like to do all of them, but I'm not sure what I'll have time for.  One thing will be to switch from RAM-heavy implementation to HDD-heavy (Armory duplicates blockchain data)... that doesn't solve your problem, but it simplifies it for a lot of other people (mostly people with less RAM).  It will also speed load times and allow you to connect to arbitrary [trusted] nodes.

The other thing is that I wanted to pioneer the blockchain pruning idea I posted about a while ago.  Since Armory is focused on security, I'm not really all that excited about making a pure lite-client and the associated risks with that.  But if I can get my pruning idea implemented, it would be extremely lightweight, and have 99% of the same security properties of a full client.  Not to mention, implementing and proving that concept would be quite a boon for Bitcon...

But I don't see either of these happening in the next month or two.  But I definitely want to do something that brings down the system requirements, considerably.  It will be on my mind after I finish the new wallet stuff...

Ooooh, cool.  That's exactly what I needed!  I guess that's a good reason alone to make changes to 0.84.5 before renaming it to beta.  And, it's very easy to branch based on OS.

Thanks picobit!  You're a lifesaver!

Crap.  I thought it was going to work with the latest Bitcoin-Qt.  I guess I should actually download it and make sure it works myself (yeah yeah, more testing...)

Sure, I was using Dropbox as a common example of a reliable backup system someone would want to use, but hesitate because of the security concerns.   But there's lot of other systems people could use to do this backup -- but with this method, it doesn't have to be a secure channel to maintain your privacy.

I am in-process of doing a full rewrite of Armory wallets; not just to improve the design, but to enable tons of new features.   However, I think this time I want to collect input from others before I go ahead and just do it my own way.  I'd like some feedback before I'm too committed to it my way.  

There are two major features I want to implement in the wallet that are "forward-thinking."  There's a lot of other stuff going in, but these are the two that warrant a real discussion.   Some quick background:  the new wallets will be based on BIP 0032 so that they will be compatible with future Satoshi wallets, and get all the nice features of hierarchical deterministic wallets.  There will have the ability to store multiple wallets/chains/accounts per file.  There will also be a way to merge wallets.  And, I want the wallet to handle P2SH scripts elegantly.  

However, P2SH engenders some complications in the wallet design.  P2SH was designed to hide scripts, thus you cannot just search the blockchain for multi-sig tx involving you -- you must actually save the (TxID, p2shScript) pair somewhere so you can recognize and find it later.  This is really annoying from the wallet-backup perspective:  all P2SH scripts must be backed up immediately to avoid potential of losing coins, but you also don't want your primary wallet to be copied around everywhere.  For instance, I think Dropbox is ideal for backing up P2SH scripts and tx/address comments, but should the user should never backup his entire wallet to Dropbox.  And I don't trust users to manually&responsibly set up reliable backups.   With that in mind:

• (1) Paired-wallet support (two-factor auth):  Each wallet/chain/account will have a field to specify another wallet/chain/account.  If that field is present, it signifies this is a "paired" wallet -- Armory will expect to have a watching-only copy of the second wallet in the same file.  It will never generate single-sig addresses -- it will only generate P2SH scripts requiring a signature from both wallets (design will also accommodate 2-of-3 and 3-of-3 wallet combos).  All multi-sig scripts will have public keys added to them in the order of the wallet IDs -- the wallet with the lower binary fingerprint/ID will always have its address first, etc.  By doing this, devices with complimentary wallet combinations (A'B and AB') will generate the same, completely deterministic chain of 2-of-2 payment addresses.    This means that in either wallet, I can say "get me address index 37", and it will fetch PubKeyA[37] and PubKeyB[37], and put them into a P2SH 2-of-2 script and return the associated payment address.  This makes P2SH for two-factor-authentication schemes completely deterministic, and will look almost identical to a regular wallet.  (for two-factor auth, you don't need ((A and B) or C) transactions, you just use (A and B) and backup both wallets to where you would otherwise backup wallet C)
• (2) "Lightly encrypted" P2SH and Comments files:  There are still situations where you need to backup your scripts:  escrow transactions with strangers on the internet, or various other contracts, etc.  Also, it would be nice to backup your comments/labels too, but not worth risking your whole wallet to do it.  So the user will be able to enable an external "wallet" file containing only these scripts & comments ("wallet" in quotes because it stores no keys).  Whenever you save a comment or P2SH script, it will save it to both files. Most importantly, these scripts and comments will be encrypted with AES256 using HMAC(walletRootPubKey, walletRootChainCode) as the encryption key.  Therefore, this external file can be backed up just about anywhere, even Dropbox, because the holder needs [at least] the watching-only wallet to decrypt it.  Someone who compromises only your Dropbox account will not have that wallet, and thus your privacy is maintained.  If your HDD crashes, you can restore your deterministic wallets/chains/accounts from your backup, and merge the Dropbox'd file into it and you're back to where you were before the HDD crash.  This not only saves your P2SH scripts from failure, but gives you a way to preserve all your address comments/labels, too (i.e. -- it's useful even for non-P2SH wallets).  The default behavior will be to not have an external file, but to allow the user to specify a location to create it that will be backed up regularly.

Edit:  Some might suggest you could always get the P2SH script from the other party if you lose your wallet.  My response is:  what if you stored their contact information as a comment in your wallet file?  This system would preserve both comments and P2SH scripts on an insecure channel without compromising privacy.

Number 2 is a bit crazy, but I think it solves a unique problem introduced by P2SH, as well as providing extra options for users deciding how to keep their P2SH/comments file backed up.

I spent a lot of time implementing coin selection in Armory.  A few of your assumptions will be incorrect if the tx was created with Armory.  Here's how Armory creates transactions:

(1) Coin-selection-evaluation method.   This is a method that scores a coin-selection based on various factors:  How many unique addresses are linked on the input side, the size of the transaction, the required fee, and... the output anonymity of the transaction (explained below).  All factors are scored between 0 (unfavorable) and 1 (favorable).

(2) I generate a dozen deterministic coin selection methods that accommodate different, specific wallet structures.  Those are added to the list. Then I generate some semi-random solutions (deterministic solutions like above, but with some perturbation).  

(3) Score all the solutions in part (2) with the eval method in part (1).  Default weights are hard-coded into armoryengine.py, but the user could modify them.  It defines how important each factor is:  most important is not using zero-conf.  Next is whether the transaction can be free (not unrelated to zero-conf).  Then number of input addresses linked.  Then tx priority, tx size, and output anonymity.  However, the user is free to make output anonymity most important (I do!  but only for fun, not because I actually care ).

(4) A new address is retrieved from the wallet, a change TxOut is added to the recipient list, then the recipient list is shuffled.  Transaction is signed and broadcast.

The parts you care about:  

(1)  If the transaction is small relative to the wallet balance, it a couple of the deterministic solutions are approx 2x the output value and 3x, so that change and recip are approximately the same or at least same order of magnitude.  It may help clean up dust, it may help anonymity (but no guarantee for either).  But when the tx amount is small relative to the wallet balance, you will frequently get equal or reversed size outputs, in random order.  This violates your assumption that the change address is always smaller.  Even if Armory didn't do it, a wallet with one very large TxOut, must have a large change output if you want to send only a small amount to someone.  So even without Armory's technique, you'll see that assumption violated with any client.

(2) Recipients are always shuffled, though only with python RNG (don't need cryptographic RNG for this )

(3) The fun part of Armory's "output anonymity" scoring is that it actually takes into account the number of significant digits in both change output and recipient.  Even though (recip, change) = (12.4, 11.8320385) are the same order of magnitude, it's pretty obvious which one is change which is output.  As such, I actually count the number of significant digits, and give the selection a higher score if it's closer, like (12.4, 11.83).  It gets an extra bonus -- a score actually above 1.0 -- if it finds a solution in which the change address has less sigfigs than the recip:  (12.4, 10.0).  i.e. deceptively-sized outputs is favored.  

How often does (3) trigger?  I don't know.  I've tested the evaluation code, though, and it definitely scores those solutions higher, but I don't know how often such deceptive solutions are ever found/included.  But if it ever happens, then your assumption about sig figs would be violated, as well.

Can you export a log file and email it to me?  etotheipi at gmail dot com.

If Armory isn't seeing new blocks found by Bitcoin-Qt, then there will definitely be something indicative of the problem in the log file.

That is definitely not by design.  What version and OS?

No, definitely not normal.  Since you're using Arch Linux, I'll assume you're familiar enough with the FS to understand what I'm about to say:

Armory installation directory:  If you checked out from git, it's the git directory.  If it was installed with a package, probably /usr/share/armory.  This is the directory that holds the source code, and all the .py files that actually execute the app/GUI.  Clearly you need read access to this directory, but it sounds like you do, since it runs without root access.
~/.bitcoin directory:  This is the directory that holds all the data from Bitcoin-Qt, including the blkXXXX.dat files needed by Armory.  But Armory only needs to read the files in this directory, it never writes anything.  Say, if you run Bitcoin-Qt with sudo, then it's possible it is setting permissions on the blkXXXX.dat files that allow only root to read them.  That would be a very good explanation for what you see, though not very likely.  Do you run Bitcoin-Qt with sudo?
~/.armory directory:  This directory needs both read and write access.  This is where all the settings and wallet files are kept.  This directory is also created the first time Armory runs. If you ran Armory the first time with sudo, it is possible this directory was created with root permissions.  This would prevent Armory from doing quite a few things, though I expect Armory would flat out crash if it couldn't write the .armory dir... but weirder things have happened.

Also, please run the latest version:  version 0.84.5 (it's the current head of the "threading" branch).  This version has a lot more auto-detection capability and tells you what is missing (if it can figure it out).  If it does work, it will help narrow this down...

The  BlockDataManager thread runs in its own little loop, and only executes non-Qt-related code in its own little bubble... via Queue.Queue objects.  The main thread dumps "work" for it onto the BDM.inputQueue, and it dumps the result to the BDM.outputQueue -- it does not use a callback structure because my computer almost imploded when I attempted Qt/Qapp calls from the BDM thread.

Thanks for getting your hands dirty with this.  I just got back from an extended vacation and don't have a lot of time to deal with this.  If you have some time, do you think you could construct a very simple example worthy of posting in a bug report (or maybe we discover a workaround)?  I'm thinking -- super simple C++ code that does nothing (basically a single main.cpp file), make an importable swig module with it using the "-threads" option, then creating a blank app that only calls getSaveFileName().  I will get to it eventually, if you don't.  But I may not have time until this weekend...

I did some more specific searching for this problem, and I still found nothing.  I suspect it's not a widely triggered bug:  only SWIG+threading+PyQt+OSX.

It is disappointing but not surprising.  I ran into some bizarre but documented issues with vector<>  iteration when using swig with -threads option.  Unfortunately, I can't figure out how I would work around it, and I haven't found anything searching the internet for related discussion.  Suggestions are welcome...

I thought about it, it seems doable to actually have a button on the screen where you save the unsigned transaction, that says "Lock these coins".  The tooltip will explain that it will "lock" the coins for 24 hours to prevent subsequent transactions from using conflicting coins.  I suppose that's not too bad...

picobit,

That is unfortunate about the save-file dialogs.  I'm really not sure how I can do anything about that, since the chunk of code that runs those dialogs is entirely out of my control.  Perhaps it is a Qt library upgrade on OSX?  What version of OSX?  @RedEmerald: do you see this too?  I am not near an OSX machine or VM, so I can't really do anything at the moment.  I will examine the code around those calls and see if I can find anything that looks suspicious.  But I doubt it -- those save dialogs are 100% part of the Qt library (I'll check to make sure I'm not doing something stupid before I invoke them).

The question about offline transactions is a good one.  The answer is:  nothing is saved.  You could go either way with the design, but I think they are both annoying, and actually putting in the work to save off the maybe-used-coins info might get the user into more of a mess than the way I did it.  So the answer is that Armory does not remember anything at all about unsigned transactions it saves.  Thus, if you create two consecutive ones without broadcasting the first, it is likely to create an invalid second transaction.   One solution is to make a single, multi-output transaction...

The converse is not only more work, but could put the user into states of unusability.  If they create an offline tx and then decide not to ever send it, then Armory would treat those coins as "locked," yet the user would believe they should still have access to the coins.  Maybe they were just messing around with the interface to see if things would work, and then canceled out when it told them to take it to the offline computer.  Their balance would be artificially low, and they'd have to do something to unlock the coins.  I could see that being confusing.  (I could probably find a way to make the interface accommodate both, but it won't be high on priority list, right now).

The way it's done right now:  if the user takes two transactions over and only one is valid, then one will go through, the second one won't, and then they'll be annoyed, but they'll try the second one again... which will now go through.  It was a bit extra work, but it gets done.

If you have the dependencies installed, then you might have to change two characters in the makefile, then type "make".  Then it's done.  The dependencies are usually where the issues are, but Armory's dependencies are totally standard and not picky about version.  And I'm working on how to autodetect the condition when you have to change the two characters (I'm really bad with makefiles).

Please make a post in the discussion thread and I'm sure someone will give you step-by-step instructions, and my guess is it will only be a couple commands.  And when they do, I'll post them on the website.

On the other side, Bitcoin-Qt is a royal P.I.T.A to compile.  I believe it's very picky about versions, and trying to get those versions installed are likely to fail and/or break other things on your system.  While I can't help you compile it, you don't need to compile it yourself to use Armory (I assume since you are here that you have a pre-compiled version you are using already).

  

To be fair, the compilation instructions are very easily translated to other *nix OS'es.  I have a few RH users who said it was quite easy to compile.  Unfortunately, my experience with other Linux distros is slim, but if you are using other distros, you're probably already familiar with copying a few commands into a terminal to get things to work.  However, I don't know if Bitcoin-Qt is the same...

FYI, I am the Armory developer.  I am the only one.  So, not only am I short on time, but my experience is limited to what I use.  I've spent quite a bit of time trying to accommodate other OS'es, but I really need other people (like you?) to help me figure out how to prepare it for other OS & architectures.  Red Emerald has basically taken the helm like this for OSX, and I would be thrilled if someone helped me figure out how to make an RPM, etc.

I know that's not the answer you're looking, but it's the best I can do by myself, at the moment...

Not merged yet.  The threading branch is the correct place to be.  Head of the "threading" branch is currently 0.84.5-alpha+, which hopefully will just be renamed to 0.85-beta... but there are probably a few more minor bugs/polishing details that will be fleshed out before then.

Speaking of that, I never even posted here that I did a semi-official release of 0.84.5!   I posted about it in the base forum, but never mentioned it here (sorry).  So read it about in the new thread!.

I am pretty comfortable with 0.84.5, but I needed a wider audience to test it before the official beta release.  And I couldn't get that audience without doing a semi-official, GPG-signed release so that people are comfortable with it.  It's kind of a catch-22, but it seems to be working so far