Armory 0.95 testing phase

[goatpig]

It's that time again. Calling on testers to build the dev branch for the time being. You can see the current change set here:

https://github.com/goatpig/BitcoinArmory/blob/dev/changelog.txt

This covers the core of the update, i.e. litenode. Some GUI stuff is broken, mainly the address book, the wallet filter drop down list, progress bars and the some data in the wallet properties dialog.

Also unit tests are out whack and there a few pull requests I'm going over at the moment, notably one for SegWit scanning support submitted by achow101 (many thanks =D).

There are still a few convenience features I haven't implemented yet, albeit they should be done quickly, notably:
- a change to guardian to prevent ungraceful bitcoind shutdowns. This should fix a lot of corruption issues.
- a dust-be-gone feature for coin control.
- a fee per byte feature in spend dialogs with tx size projections.

Once the current the outstanding issues (and anything you guys find in the process) are fixed, we'll move to testing binaries.

Stuff that won't be in the scope of this release and when to expect them:
- New wallets: That's for 0.96, with full SegWit support (create and spend from SW transactions).
- HW wallets support: Most likely a point release for 0.96.
- Fee estimates: Expect a point release on top of 0.95 (.1~2)
- Blocks over P2P. 0.97 most likely
- DB to Client encryption and auth. Again, probably 0.97
- Code modularization: most of it for 0.96
- Supernode: 0.98

This is rather massive change to the code base with big overhauls under the hood. Expect a decent amount of bugs. It will make implementing and testing the new features a lot faster and safer as a return.

Happy testing =)

P.S. I'll be away on the 25-26th. Will go over bug reports starting Wednesday

[1695982274]

1695982274

[1695982274]

1695982274

[1695982274]

1695982274

[Iconiplex]

Just to clarify some changes in the first step of this P2P process as I think the litenode is something a ton of people will be thrilled about: since ArmoryQt relies on ArmoryDB, I imagine this means remote connections are only compatible with nodes running ArmoryDB and so end-users will need to wait until people begin hosting reliable Armory nodes? In other words, remote connections to nodes exclusively running Bitcoin Classic/Core/etc. are not supported, though this issue will be later mitigated by P2P node support in ArmoryDB (essentially allowing a user to run ArmoryDB locally but pull from a Classic node) and/or implementation of the supernode?

[Carlton Banks]

I shall be firing up my compiler a little later.

Good to get an outline on the development schedule too, although I'm aware of how/why sometimes things can slip.

Re: 0.93.3 users, any plans to get a website yet?

[achow101]

Anyone else getting this error:

Code:

/home/andy/bitcoin/BitcoinArmory/armoryengine/Transaction.py:2657: SyntaxWarning: import * only allowed at module level
  def PyCreateAndSignTx_old(srcTxOuts, dstAddrsVals):
(ERROR) Traceback (most recent call last):
  File "ArmoryQt.py", line 6678, in method_signal
    method()
  File "ArmoryQt.py", line 6718, in completeBlockchainProcessingInitialization
    self.setupLedgerViews()
  File "ArmoryQt.py", line 6733, in setupLedgerViews
    self.ledgerModel.setLedgerDelegate(TheBDM.bdv().getLedgerDelegateForWallets())
  File "/home/andy/bitcoin/BitcoinArmory/CppBlockUtils.py", line 1247, in getLedgerDelegateForWallets
    def getLedgerDelegateForWallets(self): return _CppBlockUtils.BlockDataViewer_getLedgerDelegateForWallets(self)
DbErrorMsg: <CppBlockUtils.DbErrorMsg; proxy of <Swig Object of type 'DbErrorMsg *' at 0x7f81fe1044e0> >

Traceback (most recent call last):
  File "ArmoryQt.py", line 6678, in method_signal
    method()  
  File "ArmoryQt.py", line 6718, in completeBlockchainProcessingInitialization
    self.setupLedgerViews()
  File "ArmoryQt.py", line 6733, in setupLedgerViews
    self.ledgerModel.setLedgerDelegate(TheBDM.bdv().getLedgerDelegateForWallets())
  File "/home/andy/bitcoin/BitcoinArmory/CppBlockUtils.py", line 1247, in getLedgerDelegateForWallets
    def getLedgerDelegateForWallets(self): return _CppBlockUtils.BlockDataViewer_getLedgerDelegateForWallets(self)
<class 'CppBlockUtils.DbErrorMsg'>: <CppBlockUtils.DbErrorMsg; proxy of <Swig Object of type 'DbErrorMsg *' at 0x7f81fe1044e0> >

And a dialog with

Code:

The DB has returned the following error:

invalid magic word

Armory will not shutdown.

Also, a few notes. That dialog message above should read "Armory will now shutdown".

And this line: https://github.com/goatpig/BitcoinArmory/blob/dev/SDM.py#L458 should be "pargs" and not "pars"


Edit: Why port 9050? AFAIK this makes it impossible to use Tor since Tor also uses port 9050 by default.

[bitpop]

Thanks. Is .94.1 compatible with core .13?

[mocacinno]

Builds perfectly on Ubuntu 14.04.4 LTS using core 0.12.1.

 I can confirm the errors knightdk saw tough, altough armory didn't shut itself down (yet), it just kept on "preparing the database" untill i shut it down myself

Code:

/home/bitcoin/armorydemo/BitcoinArmory/armoryengine/Transaction.py:2657: SyntaxWarning: import * only allowed at module level
  def PyCreateAndSignTx_old(srcTxOuts, dstAddrsVals):
No systemtrayicon available
-ERROR - 1469526445: (SwigClient.cpp:58) unexpected return value
(ERROR) Traceback (most recent call last):
  File "ArmoryQt.py", line 6678, in method_signal
    method()
  File "ArmoryQt.py", line 6716, in completeBlockchainProcessingInitialization
    self.setupBDV()
  File "ArmoryQt.py", line 6689, in setupBDV
    TheBDM.registerBDV()
  File "/home/bitcoin/armorydemo/BitcoinArmory/armoryengine/BDM.py", line 181, in registerBDV
    self.bdv_.registerWithDB(MAGIC_BYTES)
  File "/home/bitcoin/armorydemo/BitcoinArmory/CppBlockUtils.py", line 1251, in registerWithDB
    def registerWithDB(self, *args): return _CppBlockUtils.BlockDataViewer_registerWithDB(self, *args)
RuntimeError: unexpected return value

Traceback (most recent call last):
  File "ArmoryQt.py", line 6678, in method_signal
    method()
  File "ArmoryQt.py", line 6716, in completeBlockchainProcessingInitialization
    self.setupBDV()
  File "ArmoryQt.py", line 6689, in setupBDV
    TheBDM.registerBDV()
  File "/home/bitcoin/armorydemo/BitcoinArmory/armoryengine/BDM.py", line 181, in registerBDV
    self.bdv_.registerWithDB(MAGIC_BYTES)
  File "/home/bitcoin/armorydemo/BitcoinArmory/CppBlockUtils.py", line 1251, in registerWithDB
    def registerWithDB(self, *args): return _CppBlockUtils.BlockDataViewer_registerWithDB(self, *args)
RuntimeError: unexpected return value

Thanks

[Carlton Banks]

Thanks. Is .94.1 compatible with core .13?

Can't think of a reason why not. Don't think Bitcoin Core is expected to break compatibility in any intended way in the near future, but you never know about unintendeds popping up (like having to change to stricter signature acceptance at least twice). But 0.13 should be okay. Back everything up anyway etc if you try it, unknown territory regardless.


Re: 0.95 testing

Same as mocacinno and knightdk, pretty much. My command prompt error output is a little different, but very similar. Armory GUI loads, but fails to begin any meaningful Db work, although some initial files are created for the database. Debian 8.5 running on Qubes.

[achow101]

Thanks. Is .94.1 compatible with core .13?

Can't think of a reason why not. Don't think Bitcoin Core is expected to break compatibility in any intended way in the near future, but you never know about unintendeds popping up (like having to change to stricter signature acceptance at least twice). But 0.13 should be okay. Back everything up anyway etc if you try it, unknown territory regardless.

Actually segwit will break compatibility once it is deployed (so 0.13.1 and 0.12.2 and everything later). This is due to a change in transaction serialization which will cause all current versions of Armory to break with segwit. This is being fixed with my segwit compatibility PR (https://github.com/goatpig/BitcoinArmory/pull/47) that goatpig is reviewing and will probably go into 0.95.

[goatpig]

Every body please start the DB on its own, see what it has to say for itself. The binary is called ArmoryDB, resides in the root project folder (where ArmoryQt.py is) and takes essentially the same set of command line arguments as the client.

Say, to start the DB in testnet with the default folder paths, just do

Code:

./ArmoryDB --testnet

[achow101]

Every body please start the DB on its own, see what it has to say for itself. The binary is called ArmoryDB, resides in the root project folder (where ArmoryQt.py is) and takes essentially the same set of command line arguments as the client.

Say, to start the DB in testnet with the default folder paths, just do

Code:

./ArmoryDB --testnet

This is the output of that command:

Code:

/home/andy
-INFO  - 1469564234: (main.cpp:30) Running on 8 threads
-INFO  - 1469564234: (main.cpp:31) Ram usage level: 4
-INFO  - 1469564234: (BlockUtils.cpp:1195) blkfile dir: /media/andy/Data/Programs/Bitcoin/data
-INFO  - 1469564234: (BlockUtils.cpp:1196) lmdb dir: /home/andy/.armory/testnet3/databases
-INFO  - 1469564234: (lmdb_wrapper.cpp:388) Opening databases...
-INFO  - 1469564234: (BlockUtils.cpp:1386) Executing: doInitialSyncOnLoad
bind/listen: Address already in use

Edit, actually for some reason ArmoryDB was already running beforehand for some reason, I don't know why. I just ran that command again and it is fine.

Edit2: ArmoryQt.py works when the DB run separately.

Edit3: So it looks like the problem I was having might be that the DB wasn't starting in testnet mode, even though I ran the normal command in testnet mode. I just did the DB in testnet and Qt not, and I got the same error.

[goatpig]

Edit3: So it looks like the problem I was having might be that the DB wasn't starting in testnet mode, even though I ran the normal command in testnet mode. I just did the DB in testnet and Qt not, and I got the same error.

Most likely chocking on arg parsing. Will look into it.

[Carlton Banks]

Actually segwit will break compatibility once it is deployed (so 0.13.1 and 0.12.2 and everything later). This is due to a change in transaction serialization which will cause all current versions of Armory to break with segwit. This is being fixed with my segwit compatibility PR (https://github.com/goatpig/BitcoinArmory/pull/47) that goatpig is reviewing and will probably go into 0.95.

Ah, much appreciated you've already dealt with it. I'd assumed it was the same back-compatibility issues Core users face (i.e. witness tx are unavailable, but witness users can still send and receive p2pkh/p2sh to pre-witness wallets and vice-versa)



Executing ./ArmoryDB yields this:

Code:

-DEBUG - 1469567562: (Blockchain.cpp:232) Organizing chain
-INFO  - 1469567562: (DatabaseBuilder.cpp:53) updated HEADERS db in 0.006948s
-INFO  - 1469567562: (BlockUtils.cpp:1497) Enabling zero-conf tracking
Host localhost has multiple addresses ---
you must choose one explicitly!!!

What's my argument for specifying localhost for ArmoryDB?

[T-rage_11]

hi,

I'm using a old Windows XP 32-bit machine for Armory v0.92.3 cold storage ..
Can I still use the offline machine with newer online versions (0.94 and 0.95) ?

[goatpig]

Ah, much appreciated you've already dealt with it. I'd assumed it was the same back-compatibility issues Core users face (i.e. witness tx are unavailable, but witness users can still send and receive p2pkh/p2sh to pre-witness wallets and vice-versa)

Both 0.93 and 0.94 could be patched to ignore SW tx (like any non SW compliant client should, it's a soft fork after all). However I don't think anyone wants to go down that path. 0.95 will properly scan SW tx at the very least.

Executing ./ArmoryDB yields this:

Code:

-DEBUG - 1469567562: (Blockchain.cpp:232) Organizing chain
-INFO  - 1469567562: (DatabaseBuilder.cpp:53) updated HEADERS db in 0.006948s
-INFO  - 1469567562: (BlockUtils.cpp:1497) Enabling zero-conf tracking
Host localhost has multiple addresses ---
you must choose one explicitly!!!

What's my argument for specifying localhost for ArmoryDB?

Ahh, this thing again.

First, the DB is hardcoded to listen to localhost:9050. You can't make it listen to another IP (ill add custom ports at some point). If you want to enable the DB to serve remote clients, you need to set a http daemon to connect to it over the FCGI protocol (same as how stuff like PHP interface with a web stack) and manage the network parameters around the daemon, not the ArmoryDB per se.

The issue you are seeing seems to be common to Linux. I ran into it when I first started implementing around the FCGI library. It appears localhost is bound both in IPv4 or IPv6, and the fcgi lib uses listen in an old enough way that it doesn't try to distinguish between the two.

At first I took that for a quirk of my debian install (it's quite aged), researched the issue and fixed it fairly quickly (something to do with /etc/hosts file maybe? can't remember).

If you are hitting this error too, it seems this setup is more of a default than a stand alone case. I will setup a fresh VM, check against that, and most likely fix the FCGI lib directly.

[goatpig]

hi,

I'm using a old Windows XP 32-bit machine for Armory v0.92.3 cold storage ..
Can I still use the offline machine with newer online versions (0.94 and 0.95) ?

Yes

[achow101]

The behavior is kind of inconsistent. Sometimes it works, sometimes it doesn't. Usually it doesn't.

You should also fix that "pars" "pargs" thing. I think it prevents the db from being launched by qt.

[goatpig]

The behavior is kind of inconsistent. Sometimes it works, sometimes it doesn't. Usually it doesn't.

You should also fix that "pars" "pargs" thing. I think it prevents the db from being launched by qt.

Fixed the typos, will bundle a more consistent fix tomorrow

[mocacinno]

Ahh, this thing again.

First, the DB is hardcoded to listen to localhost:9050. You can't make it listen to another IP (ill add custom ports at some point). If you want to enable the DB to serve remote clients, you need to set a http daemon to connect to it over the FCGI protocol (same as how stuff like PHP interface with a web stack) and manage the network parameters around the daemon, not the ArmoryDB per se.

The issue you are seeing seems to be common to Linux. I ran into it when I first started implementing around the FCGI library. It appears localhost is bound both in IPv4 or IPv6, and the fcgi lib uses listen in an old enough way that it doesn't try to distinguish between the two.

At first I took that for a quirk of my debian install (it's quite aged), researched the issue and fixed it fairly quickly (something to do with /etc/hosts file maybe? can't remember).

If you are hitting this error too, it seems this setup is more of a default than a stand alone case. I will setup a fresh VM, check against that, and most likely fix the FCGI lib directly.

I'll also be waiting on this fix, had the same problem as Carlton Banks, so i cannot proceed to testing either

[goatpig]

I'll also be waiting on this fix, had the same problem as Carlton Banks, so i cannot proceed to testing either

Was stupid, should be fixed now.

[mocacinno]

I'll also be waiting on this fix, had the same problem as Carlton Banks, so i cannot proceed to testing either

Was stupid, should be fixed now.

Seems like the problem is fixed, altough i can't manage to get ArmoryDB running, seems like it can't bind... I'll see if i can find out what's wrong when i get home from work.

Code:

./ArmoryDB --testnet
/home/bitcoin
-INFO  - 1469689140: (main.cpp:30) Running on 8 threads
-INFO  - 1469689140: (main.cpp:31) Ram usage level: 4
-INFO  - 1469689140: (BlockUtils.cpp:1195) blkfile dir: /home/bitcoin/.bitcoin/testnet3/blocks
-INFO  - 1469689140: (BlockUtils.cpp:1196) lmdb dir: /home/bitcoin/.armory/testnet3/databases
-INFO  - 1469689140: (lmdb_wrapper.cpp:388) Opening databases...
bind/listen: Address already in use