Armory 0.93 Testing Release  (0.92.99.7-testing)
Bug Bounty Started:  0.05 BTC per bug!

---

REMEMBER:  Download links are at the bottom but you can get the new version through the secure downloader.  Armory will check the signatures for you!

NOTE:  If you upgrade Core to 0.10 you must use this new version of Armory.  Armory 0.92.3 and older does not work with headers-first!

---

Hi all,

Finally, we have a solid testing release for 0.93.  This version has taken us quite a long time because it represents a substantial reworking of the blockchain engine as well as the inter-thread communication.  Much of Armory looks the same as it did in previous versions, but now it's 100% scalable, handling wallets that even have a 1,000,000+ addresses and transactions in their history.  It also does things in the background such as address and wallet imports, which no longer force you offline.  Because of the new threading engine, there's a whole class of stability problems that don't exist anymore (specifically BDM timeout errors that cascaded into unusability and required restarting Armory;  usually with larger wallets).  

Special thanks to goatpig and njaard who have spent many months the new blockchain engine!

Especially important for our more-hardcore users, we now have a "supernode" mode, that doubles Armory's DB size, but indexes all scripts on the blockchain.  If you delete your databases and start Armory with "--supernode", you'll build a DB that does instant importing of addresses and wallets.  It could be used to build a block explorer or an Electrum-style server for feeding lite-nodes (for everyone who's asked about lite-Armory, completion of supernode was a critical milestone that needed to be crossed to even think about it).

KEEP IN MIND:  This new version uses a new DB engine (LMDB instead of LevelDB).  This means that the 0.93 databases are incompatible with any already-built databases.  And it won't delete them automatically -- you must do it manually if you don't want both versions.  This was done to accommodate users that have to jump back to 0.92 if the latest version doesn't work for them.  We recommend simply using a new datadir for testing.

---

REMEMBER:  You can always get the new version through the secure downloader.  Armory will check the signatures for you!  Download links are provided here as a convenience for those testing in non-secure environments.

NOTE2:  I have no expectations of the OSX build working.  If it does, it would be a blessing.  We've had to stir some things up, and didn't have time to solidify before this testing release.  It'll come soon!

  Armory 0.92.99.7-testing for Windows XP, Vista, 7, 8+ (32- and 64-bit)
  Armory 0.92.99.7-testing for MacOSX 10.7+ (64bit)
  Armory 0.92.99.7-testing for Ubuntu 12.04+ (32bit)
  Armory 0.92.99.7-testing for Ubuntu 12.04+ (64bit)
  Armory 0.92.99.7-testing for RaspberryPi  (armhf)

  Armory 0.92.99.7-testing Offline Bundle for Ubuntu 12.04 exact (32bit)
  Armory 0.92.99.7-testing Offline Bundle for Ubuntu 12.04 exact (64bit)
  Armory 0.92.99.7-testing Offline Bundle for RaspberryPi  (armhf)

  Armory 0.92.99.7-testing: Signed hashes of all installers

---

Semi-official changelog:


VERSION 0.92.99.7
Released Jan 9, 2015


   - New Scalable Blockchain Engine
        This version of Armory will look very similar to previous versions,
        but actually has a completely new, scalable database engine (using
        LMDB instead of LevelDB).  The engine can now handle wallets with
        millions of addresses and transactions!

   - Compatible with Bitcoin Core 0.10 and "headers-first"
        The most recent version of Bitcoin Core includes a parallel network
        synchronization feature called "headers-first" which is incompatible
        with previous versions of Armory.  This version is required to use
        with Bitcoin Core 0.10 and newer.  Torrent-based bootstrapping will
        be removed in the next version of Armory.

   - Improved Threading and Reliability
        With the new backend comes overhauled inter-thread messaging.  This
        resolves a ton of stubborn stability issues that Armory has had in
        the past, especially with large wallets and transaction histories.  

   - Address and Wallet Importing in Background
        Importing wallets and addresses now induces background scans which
        do not disable the interface (previously forced you into offline
        mode for the duration of the scan).  
        
   - Supernode Mode
        Use the "--supernode" flag before creating the databases to create
        a supernode database that enables instant address & wallet import.
        This is an enabling feature for high-volume, consumer facing apps
        and services to be built on Armory, such as exchanges and block-
        explorers.  NOTE: Running with --supernode will result in an Armory
        database approximately double the size of the blockchain!  
        
   - Improved armoryd.py
        The daemon version of armory, armoryd.py, has been fully updated and
        tested with the new database engine.  Also includes a new "webshop"
        sample application that demonstrates basic usage of armoryd.py to
        collect & verify payments, generate unsigned transactions, etc.

I have been thoroughly distracted and I totally lost track of this effort.  However, I believe that the stated goals have been achieved, and certainly roman.z has put together an awesome tool.  Looking through the code I see it is very clean, commented and includes unit tests.  

Amusingly, after all of Roman's effort, Newar found the minimodem library which somehow evaded detection in this thread for many months (years?).   I think it's a slower-but-sufficient solution for linux-to-linux communication.  For this, I think Newar deserves an "Honorable mention" of 1 BTC.

Therefore, I will split the bounty 24 BTC to Roman and 1 BTC Newar.

Due to some concerns in the past that RAM errors could result in erroneous key calculations we decided to be extra safe and save off the byproduct of the deterministic key calculation into this file (the multipliers).  The data in that file cannot be used to figure out any private keys if you don't know them, but does allow you to confirm/regenerate the private keychain if you have the root private key.  Under nominal conditions, it's irrelevant.  But if there was a RAM error that calculated a bad public key on a WO wallet, you could use multipliers.txt and the paper backup to compute the private key for the erroneous public key.

We've never had to use this file on anyone's wallet, so we don't even have a script that does what I said above.  But we have seen wallet consistency errors that led us to believe this is possible, which is why we added it along with doing many key calculations multiple times and confirming they match.

This m might be a simple question for a namecoin guru:  what is the difference in block formats between Bitcoin and namecoin?  It might be something stupid and simple.  Could it have to do with the name reservation system?  Do transactions have a different structure when they register namecoin names? 

C:\Users\<user>\AppData\Roaming\Bitcoin

It sounds like you moved your Bitcoin home directory 7 months ago to a new datadir, and didn't tell Armory about the new location using --satoshi-datadir.  It's reading the old leftover blockchain files.

If you are letting Armory run Bitcoin for you, then you can point to the correct directory in the Armory settings.  Otherwise you have to use the --satoshi-datadir argument in the command line.

FYI, here's the old transitional code for the same method:

https://github.com/etotheipi/BitcoinArmory/blob/v0.88-beta/cppForSwig/BtcUtils.h#L736

And as njaard said:  you don't access python from C++.  If you need something, put it in the C++ side, and call it from python.  SWIG is amazing (being able to call C++ objects and methods as if they're native python objects), but it doesn't go both ways, and in fact it can get quite complicated to do so.

It's kind of fun watching you guys attempt to reverse engineer this stuff  

Looks like inf got it except for one mistake (if I read his text formatting correctly):  the lines in the backup data actually have 2-bytes of checksum per line, instead of 4 bytes of checksum per two lines.  i.e. you wrote:


But it should be:


You can confirm your understanding of it by look up the two functions "makeSixteenBytesEasy()" and "readSixteenEasyBytes()" (both are in ArmoryUtils.py)

In a general sense, they are referring to something very similar:  part of a cold-storage scheme. 

WIthin Armory, "watch-only" means you're simply watching a wallet, probably not owned by you.
"Offline" means you're watching a wallet that is owned by you, but the keys are on an offline computer, and you're simply watching it from this online computer.

When you import a watching-only wallet, it is by default not marked as your own (and thus labeled "Watch-Only").  You have to explicitly tell Armory it is your wallet in the wallet properties under "Belongs to:".  Then it will switch to "Offline" and its balance will be added to your global balance.  But changing that value doesn't really change anything:  it's still a wallet that only has public keys, and you can't use it to spend money, only see transaction history and receive payments.

When Armory's wallet format was originally created, compressed keys didn't exist in any Bitcoin software.  I never accommodated for the possibility in the wallet format.   Since then, the wallet format has been so thoroughly tested and battle-hardened, I didn't want to mess with it to support a non-critical feature like compressed public keys. 

Finishing up a new wallet format now that will support a lot of new features, including both compressed and uncompressed keys.  Still a ways off from an official release, but it's coming.

Ack.  Good catch.  I tend to check all the new downloads before pushing (or immediately after) version updates to make sure it was all smooth.  It seems I actually did it right but they pulled the rug out from under me... thanks for finding that commit that changed and vindicating me from fault

It will be updated soon.  Until then, you can still use the secure-downloader to confirm, just not execute the full download:

• In the secure downloader, select the OSX package, and instead of clicking "Download", click "Download Info".  You'll see the (incorrect) download link and the hash that Armory Tech has pre-verified and signed.
• Go to the bitcoin.org website and download the file from there (or just use the link posted, such as here:  
  https://bitcoin.org/bin/0.9.3/bitcoin-0.9.3-macosx.dmg
• In the terminal, you can shasum -a256 bitcoin*.dmg.  This should give you the same hash that is shown in Armory

That's an excellent point.  I wonder if maybe we can simply make the installer not overwrite shortcuts if they already exist, but only add them if they don't.  Alternatively, it would be nice to have the dbdir set in the settings.  I bet that explains a lot of support tickets we get when users upgrade -- nothing changed, it just overwrote their CLI args that they normally use...

One is where the bitcoind executable is located (installation directory), the other is where your Bitcoin wallets and databases are held.  I know it's not necessarily intuitive for users unfamiliar with it, but there's not much we can do to clarify other than teaching users what they are.  The wallets and databases are always stored separately, so that you can upgrade the app without touching your wallets, and redownloading the blockchain, etc.  The installation directory is intended to hold no temporary data so it can be safely wiped on reinstalls, etc.

In Windows, the installation directory is, by default:  C:\Program Files\Bitcoin (maybe (x86)).  It will find the daemon directory as necessary.
In Windows, the default location for your home dir is C:\Users\<user>\AppData\Roaming\Bitcoin.

Both of those need to be updated if you moved both.  If you're saving space, you most likely only moved the home directory which contains the huge databases.

Can you please confirm the sizes of those files.  I'm like 99% sure that what I just described is the problem, but I'd like confirmation.  If so, we can tweak the code to avoid this.  The whole point of that code block is to prevent this from happening, but apparently it can't handle once it has happened (perhaps from leaving Armory running for 3 months with a bunch of constant messages being written to log.

Don't worry about the RAM.  my guess is the log file somehow became 20 GB and Armory is choking trying to read it (which is ironic because it's failing while trying to cut it down to 1 MB)

Maybe you ran out of RAM?  Maybe it's actually a completely overflowing log file and it can't read the whole thing in? 

In your Armory home dir, check the size of armorylog.txt and armorycpplog.txt.  How big are they?  Try deleting them.

Does the transaction show up on blockchain.info?  If so, it actually succeeded.  Blockchain.info won't show it unless it's a valid transaction.  It might just be taking forever to be mined...?   Maybe it's showing it as empty because you just moved the coins out of it with a valid transaction?

If you aren't talking about blockchain.info, and the coins haven't moved and it appears to be trying to spend coins that don't exist, then it sounds like the database state is out of whack.  You can use Help->Rescan Databases.   If that doesn't work, try Help-Rebuild & Rescan.

A rebuild is a last resort, taking a couple hours, but should resolve all issues related to the DB.

Can you do me a favor:  do whatever it is that triggers the error, then go look at the Bitcoin Core logs:  /home/<user>/bitcoin/debug.log .  As soon as you see the disconnect message in Armory, you'll see a message in the logfile telling you why the transaction was rejected.  This is the most direct way to determine why it's failing.

Does your wallet have tons of small inputs?  Meaning, is your 0.5 BTC from receiving 500 donations of 0.001 each?  That frequently causes failures to send because of transactions that are too big.

It sounds like your blockchain is not synchronized.  You are attempting to send coins that are already spent, and thus the local Bitcoin Core instance is disconnecting you for broadcasting invalid transactions.  First thing is to make sure that the block count in the bottom-right corner of the Armory window matches the top block on blockchain.info.