Armory 0.96.3 released

[goatpig]

Notes

This is a point release to fix a vulnerability with fragmented backups. If you are using fragmented backups on for your wallets, read the dedicated topic:

https://bitcointalk.org/index.php?topic=2199659.0

Binaries

https://github.com/goatpig/BitcoinArmory/releases/tag/v0.96.3

Changelog

== Vulnerability Fix ==
   - Fragmented backups were using a faulty implementation of Shamir's Secret Sharing (SSS).
     One of the requirement of SSS security parameters is that the coefficients of the curve are chozen randomly. The implementation
     up to this point was deriving these coefficients deterministically.

   - While it is hard to determine how far the deterministic coefficient generation erodes the security of SSS, and how exploitable
     the vulnerability is, the recommendation for users of fragmented backups is to treat the wallets backed up in this fashion as
     compromised and to migrate all funds to a new wallet.

   - The fragmented backup code now properly randomizes the SSS coefficients. Fragmented backups created with version 0.96.3 and later
     are safe to use.

   - The result of this change is that fragmented backups will no longer be deterministic. The previous behavior guaranteed a given
     wallet will always return the same set of fragments for a given M-of-N scheme. Since it deteriorates SSS security properties,
     the behavior has to be rolled back.
   - Fragment sets are now generated randomly, therefor an unique ID has been added to each set to identify them. You cannot mix
     and match sets.
   - While Armory can no longer generate deterministic fragments, it can still restore wallets from deterministic fragments.

   - Many thanks to Gregory Maxwell (greg@xiph.org) for identifying and reporting the vulnerability as well as reviewing the fix.

== Fixed ==
   - Fixed faulty version packet deserialization revealed by Core 0.15.0.1

[1715059082]

1715059082

[1715059082]

1715059082

[1715059082]

1715059082

[1715059082]

1715059082

[droark]

FYI, this version of Armory is the last one built to run on OS X 10.7. Starting with 0.96.4, you'll have to roll your own version if you insist on running Armory on 10.7. Long & technical story short, Armory uses code that can have issues on 10.7. Supporting 10.7 made it more difficult for people on more recent versions to compile the code. So, 10.7 is out the door.

Also, goatpig didn't explicitly mention this but I believe this will fix the issues people were seeing when they ran Core 0.15.0.1 (i.e., the ones where goatpig told them to go back to 0.15). If you're still having issues, post your logs!

Thanks.

[achow101]

Also, goatpig didn't explicitly mention this but I believe this will fix the issues people were seeing when they ran Core 0.15.0.1 (i.e., the ones where goatpig told them to go back to 0.15). If you're still having issues, post your logs!

Yes, it does fix that problem, hence that one line under "== Fixed =="

[Searinox]

I have completely deleted the blockchain, let 14.0.2 redownload all of it which took over a day, let ArmoryDB rebuild its database, and when I launch Qt I get the Same Old issue. Yes I am on 0.96.3 now, and nothing's changed. Really, bitcoind seems to be behaving fine, and ADB is also building up its database normally, I'd wager the issue is with Qt.

EDIT:

I went ahead and I moved the wallets away from ArmoryQt's folder, and deleted the settings file. Started fresh with a new wallet. The issue remains UNCHANGED.

When not using any wallet, the Qt shows Connected and correct number of blocks loaded, and keeps up to date. The problem always seems to appear when having to do anything with scanning transaction history, which never goes ahead. Qt will also close normally when there is no history to scan, as opposed to when there is, in which case it will seem to operate normally right up to being Quit, at which point it will freeze up and require forced termination.

Rechap:

Core 0.14.0.2 - fresh everything - 1.5 days old (I even deleted peers.dat, banlist.dat etc.)
ArmoryDB - fresh everything(including old files deleted and replaced before this update) - few hours old
Qt - same as DB, with fresh config as of - 30min ago



EDIT2:

I updated Core to 1.15.0.1. No dice. So what I have is, whenever Qt launches and connects to ArmoryDB, ADB's log gets this:

Code:

-INFO  - 16:56:45.234: (..\BDM_Server.cpp:1114) registered bdv: 9763f8e6ac8df2c46d7a
-INFO  - 16:56:45.406: (..\BDM_supportClasses.cpp:401) Starting address registration process

Nothing more follows. From what I see it's supposed to start showing "Scanned from height #" events as it processes the blockchain. It does not proceed. Yet the blockchain is loaded properly enough to actually exist in bitcoind, up to date, and with no errors reported by it. The blockchain is also loaded properly enough by DB to build its database out of it. Then when Qt wants to scan an address, suddenly it cannot load it?

bitcoind - all ok, downloaded blockchain up to date, maintaining it, no errors in log
armorydb - all ok, log shows loading all blk files up to date, no errors reported
amroryqt - requests history for wallet, armorydb chokes on the request, qt cannot find/load blockchain - SAME WITH FRESH CONFIG AND NEW EMPTY CLEAN WALLET

What gives?

[HCP]

Are you running on Windows? If so, is your Bitcoin Core blockchain data in the "default" location? (C:\Users\USERNAME\AppData\Roaming\Bitcoin) or did you set it to a custom directory?

[goatpig]

Sounds like your blockchain data is corrupt. Probably somewhere in the first 5 or so files. Could you upload these somewhere so that I can take a look?

[Searinox]

Are you running on Windows? If so, is your Bitcoin Core blockchain data in the "default" location? (C:\Users\USERNAME\AppData\Roaming\Bitcoin) or did you set it to a custom directory?

Windows 10 x64 Creators Update. Location is non-standard. But it's the same folder it's been for years. There have been no changes to paths.

Bitcoin core: satoshi-datadir is "F:\Program Files\Armory\Data_BTC\blocks" where all 140+GB of blockchain are.
ArmoryDB: satoshi-datadir is "F:\Program Files\Armory\Data_BTC\blocks", datadir is "F:\Program Files\Armory\Data_ArmoryDB"
ArmoryQt: satoshi-datadir is "F:\Program Files\Armory\Data_BTC\blocks", datadir is "F:\Program Files\Armory\Data_ArmoryQt"

All 3 executables have under properties -> compatibility -> run as administrator checked. I verify that I can read and write randomly created files in the folders. I'm not getting prompted to confirm any permissions and it works fine.

Sounds like your blockchain data is corrupt. Probably somewhere in the first 5 or so files. Could you upload these somewhere so that I can take a look?

I'm guessing the blocks from Data_BTC\blocks right? Do you want the blk###### files? They're 127MB each. I reiterate that I have a fresh blockchain.

[Searinox]

Rather than wasting time to upload, I hashed them. See if this is enough for you.

Code:

  File: blk00000.dat
CRC-32: 1c7370f0
   MD4: 35d999d57683ec6b8096fc4738ed9d76
   MD5: b1d473ee9ad05a7c5d8c3094840b6295
 SHA-1: 693a0a3466424ca633f297e61e6d496a3a72f207

  File: blk00001.dat
CRC-32: 847b6315
   MD4: 4a0c34dd784338da7e0a13fe9d995ef5
   MD5: ed020c6d24cb0dbf06354f0a92ac10bc
 SHA-1: e09334553bc79a765866fc964cd280555da11dbb

  File: blk00002.dat
CRC-32: 6ac18b34
   MD4: 92c0f8cd783fe7bd8b1bfb193542aa55
   MD5: 97fa4ff5428a15282e304a831e68e312
 SHA-1: b36449e2914ede693cae4983d59ea19b6180b509

  File: blk00003.dat
CRC-32: efee16af
   MD4: f795bd0172260995f1e6c8bc95863ce9
   MD5: 8a057ef398031e4bc75cacf7639219c8
 SHA-1: 6d0828d4655701a8926a4b2acbffbe209eb38992

  File: blk00004.dat
CRC-32: 430c67b8
   MD4: 27a9dd6f59c9ccbae7211756afd0efbf
   MD5: d7141d2a74eac77b77c7c8611cc656a2
 SHA-1: 39db004eeeb7f77b2db431d4af3c455e57151d18

[goatpig]

Hashes are pointless, that data is not downloaded sequentially, no one has the same hash for this stuff.

I have no idea why your setup does not work. You're the only person with this issue so I have no clue to go with. You could give me the full log of a fresh DB bootstrap one last time, but if that doesn't yield anything, at some point I'm going to need to see some backtraces or look at your block files. Later is easier to deliver.

[Searinox]

Blocks 0-4 can be found here: https://drive.google.com/open?id=0B5_lN_xRgoTVU0N0djUtX05QWmM

[goatpig]

There was no issue with the block files. Something in your setup is preventing the DB from progressing forward. You're going to have to get me some debug data.

You can setup a Ubuntu VM and run the DB in there, or make in a different version of Windows, while your client remains on the host, or you need to whip out MSVC, build a debug version of the DB and set some key breakpoints to figure out what's going on. Pick your poison. I'm out of idea about how to do this easily.

[Searinox]

@@; Kay I'm just uh... gonna do some generic troubleshooting and cleanup and wait for everything to work at some point. As an extra bit of info though, bitcoind runs independent.

Code:

-INFO  - 19:24:54.343: (..\BDM_Server.cpp:1114) registered bdv: a2f604a580439d1dc0c0
-INFO  - 19:24:54.515: (..\BDM_supportClasses.cpp:401) Starting address registration process
-INFO  - 19:25:25.437: (..\BlockchainScanner.cpp:52) no history to scan
-INFO  - 19:30:02.500: (..\BlockchainScanner.cpp:52) no history to scan
-INFO  - 19:31:26.625: (..\BlockchainScanner.cpp:52) no history to scan
-INFO  - 19:34:33.015: (..\BlockchainScanner.cpp:52) no history to scan
-INFO  - 19:35:34.781: (..\BlockchainScanner.cpp:52) no history to scan
-INFO  - 19:39:23.625: (..\BlockchainScanner.cpp:52) no history to scan
-INFO  - 19:41:06.250: (..\BlockchainScanner.cpp:52) no history to scan
-INFO  - 19:57:16.109: (..\BlockchainScanner.cpp:52) no history to scan

WAIT A MONENT!

I accidentally started Qt before ADB and it began building its own DB in its own foloder. I used to start bitcoind, armoryDB and armoryQt each individually with its own command line; since when is DB managed by Qt??? Let's see how this works.

[goatpig]

since when is DB managed by Qt???

Which Qt? BitcoinQt or ArmoryQt?

-INFO  - 19:25:25.437: (..\BlockchainScanner.cpp:52) no history to scan

That means the DB has seen a new block but it has not address data to scan against it.

[Searinox]

since when is DB managed by Qt???

Which Qt? BitcoinQt or ArmoryQt?

-INFO  - 19:25:25.437: (..\BlockchainScanner.cpp:52) no history to scan

That means the DB has seen a new block but it has not address data to scan against it.

ArmoryQt(I don't use BitcoinQt). I used to be able to launch ADB independently of AQt. Now when I start AQt it also launches -another- instance of ADB with its own separate database folder. Is there a way to stop AQt from doing this? Or is this the only way AQt will run properly?

[goatpig]

ArmoryQt will first look for a DB on the given IP:port. If it can't find one, it will spawn a DB. If you are running a DB but ArmoryQt can't find it, you aren't giving it the proper IP:port to look at.

[Searinox]

ArmoryQt will first look for a DB on the given IP:port. If it can't find one, it will spawn a DB. If you are running a DB but ArmoryQt can't find it, you aren't giving it the proper IP:port to look at.

ArmoryDB is running on localhost with no exotic IP:port setup. But it is running independently. What commandline should Qt use to connect?

[droark]

Also, goatpig didn't explicitly mention this but I believe this will fix the issues people were seeing when they ran Core 0.15.0.1 (i.e., the ones where goatpig told them to go back to 0.15). If you're still having issues, post your logs!

Yes, it does fix that problem, hence that one line under "== Fixed =="

Sorry, what I meant was that, IMO, the phrasing wasn't as clear as it could've been. I definitely saw the entry. I just wanted to make it clear to those who don't speak technobabble.

[goatpig]

ArmoryDB is running on localhost with no exotic IP:port setup. But it is running independently. What commandline should Qt use to connect?

It should find the DB instance by default.

[Searinox]

ArmoryDB is running on localhost with no exotic IP:port setup. But it is running independently. What commandline should Qt use to connect?

It should find the DB instance by default.

DB managed by QT builds the tx history. I tried to replicate its command line in my own version of it. The one thing I could not do was connect QT to the DB once it starts with --cookie, because it randomizes the listen port.

[goatpig]

Something is off with how you pass your custom arguments. The cookie arg is not mandatory, if you start an instance of the DB and give that arg to your client you should be fine. Are you using something else than 127.0.0.1? Is localhost resolved to some other IP? You need to list out all parts of you setup that deviate from default settings and isolate them one after another.