dev branch work

[goatpig]

I can't see how it makes sense to scan the blocks for transactions in that case, but that happened, and all was well. Sadly, there was no surprise BTC

Failure on my part, "isNew" flag not being propagated correctly I expect.

It seems there are methods in cppforswig/BridgeAPI/WalletManager to convert the 0.96.x format to the new one? Can I call them somehow, or is there no userspace interface yet?

CppBridge loads all wallets in the datadir and converts them to the new format on the fly. Old wallet files are ignored past that point. Those wallets that carry encrypted private keys will lead to a prompt in GUI to unlock them. The same password used to unlock them will be used to encrypt the new file. There is no client facing API to call the conversion code directly. It is all baked into loadWallets (https://github.com/goatpig/BitcoinArmory/blob/dev/cppForSwig/BridgeAPI/WalletManager.cpp#L199).

It will be part of the last major new piece of GUI I need to implement. I want to add a settings dialog that precedes ArmoryQt itself to do a few things:

---------
- Wallets: need a tab that lists all wallets found in the data dir and offer users the choice to convert. I don't like the tacit conversion but it was a cheap solution at the time. I'm very bad at GUI development =D. Also need to offer the option to unlock wallets. The new wallet container encrypts public data with a different passphrase. This is set to a cleartext key by default, but with some minor GUI at wallet creation time, users can have the option to set a passphrase of their own choosing.

With fully encrypted wallets, users need unlock the public data prior to loading Armory itself. I could have gone with a simple message box prompt but that would get very obnoxious with more than 2-3 wallets loading.

---------
- AEAD key management: CppBridge couldn't talk to ArmoryDB because ArmoryDB wasn't set to --public. All sockets are encrypted now (this means from ArmoryDB to CppBridge and from CppBridge to ArmoryQt). The AEAD allows both 1-way auth (client authenticates the server) and 2-way auth (server authenticates the client back).

It defaults to 2-way, meaning ArmoryDB's key vault needs to know of CppBridge's key prior to the connection. As for ArmoryDB's key, if it's missing from CppBridge's vault, the user should be prompted to accept the key, but since I'm lazy, I defaulted the accept to true instead of writing a prompt (https://github.com/goatpig/BitcoinArmory/blob/dev/cppForSwig/BridgeAPI/CppBridge.cpp#L324).

Since ArmoryQt spawns CppBridge, adhoc keys are setup every run, and discarded after use.I haven't looked at automating ArmoryDB yet, so you're left running it yourself. As it is missing the keys in its vault, it will reject all clients, unless you run it with --public (accept all clients). The 1-way and 2-way handshakes are incompatible on purpose, there is no downgrading of a 2-way handshake to 1-way for convenience purposes.

There's a command prompt tool to manage keys, aptly dubbed "KeyManager" (I'm good with names 'n stuff) -> https://github.com/goatpig/BitcoinArmory/blob/dev/cppForSwig/KeyManager.cpp

This needs some GUI around it, considering it isn't just for your local instances, but remote ones as well (considering running a supernode for those people who can't get fullnode Armory up locally, just so they can move their coins out). With 2-way auth, you can actually manage who gets in a WAN exposed DB and/or CppBridge.

This tab will also be a good place to manage IPs for remote DB/Bridge. The bridge handles all things wallets, ArmoryQt is more or less a dumb interface now, so you could run CppBridge in say a Docker container and plug ArmoryQt to it from somewhere else.

---------
- DB/Core management: Need some GUI for people to discover block/db paths to ease the pain around initial setup. Also a place to reintroduce the settings for Core and DB automation. It's a lot harder to make the bootstrap consistent if ArmoryQt starts before the DB or Core. It's simpler to assume these pieces of software have to run before ArmoryQt is up and handle the complexity in this new pre-start dialog.

[Carlton Banks]

CppBridge loads all wallets in the datadir and converts them to the new format on the fly. Old wallet files are ignored past that point. Those wallets that carry encrypted private keys will lead to a prompt in GUI to unlock them. The same password used to unlock them will be used to encrypt the new file.

some issues I found:

1. one (encrypted) wallet throws this exception:

Code:

(INFO) ArmoryQt.py:4486 - Dashboard switched to "Scanning" mode
terminate called after throwing an instance of 'Armory::Wallets::IdException'
  what():  [EncryptionKeyId] invalid key size
(INFO) ArmoryQt.py:5277 - BDM is safe for clean shutdown

2. Another converts, then scans, then logs these errors when resolving tx hints:

Code:

-INFO  - 2022-04-06 - 23:08:29.411: (BlockchainScanner.cpp:1671) xxx blocks hit by tx filters
-ERROR - 2022-04-06 - 23:08:34.169: (BlockchainScanner.cpp:1468) Block deser error while processing tx filters: 
-ERROR - 2022-04-06 - 23:08:34.184: (BlockchainScanner.cpp:1469)   raw data does not match expected block hash
-ERROR - 2022-04-06 - 23:08:34.184: (BlockchainScanner.cpp:1470) Skipping this block
-ERROR - 2022-04-06 - 23:08:34.186: (BlockchainScanner.cpp:1468) Block deser error while processing tx filters: 
-ERROR - 2022-04-06 - 23:08:34.186: (BlockchainScanner.cpp:1469)   raw data does not match expected block hash
-ERROR - 2022-04-06 - 23:08:34.186: (BlockchainScanner.cpp:1470) Skipping this block
-ERROR - 2022-04-06 - 23:08:34.186: (BlockchainScanner.cpp:1468) Block deser error while processing tx filters: 
-ERROR - 2022-04-06 - 23:08:34.187: (BlockchainScanner.cpp:1469)   raw data does not match expected block hash
-ERROR - 2022-04-06 - 23:08:34.187: (BlockchainScanner.cpp:1468) Block deser error while processing tx filters: 
-ERROR - 2022-04-06 - 23:08:34.187: (BlockchainScanner.cpp:1469)   raw data does not match expected block hash
-ERROR - 2022-04-06 - 23:08:34.187: (BlockchainScanner.cpp:1470) Skipping this block
-ERROR - 2022-04-06 - 23:08:34.187: (BlockchainScanner.cpp:1470) Skipping this block

3. Upon file->quit, we have this:

Code:

(INFO) ArmoryQt.py:5277 - BDM is safe for clean shutdown
[2022/04/06 23:18:39:3578] N: __lws_lc_untag:  -- [wsi|0|pipe] (0) 22.654s
[2022/04/06 23:18:39:3826] N: __lws_lc_untag:  -- [vh|1|default||-1] (1) 22.677s
[2022/04/06 23:18:39:3826] N: __lws_lc_untag:  -- [wsicli|0|WS/h1/default/127.0.0.1] (0) 22.677s
[2022/04/06 23:18:39:3826] N: __lws_lc_untag:  -- [vh|0|netlink] (0) 22.677s
(ERROR) CppBridge.py:220 - Socket error: [Errno 9] Bad file descriptor
(ERROR) ArmoryQt.py:5288 - Strange error during shutdown
Traceback (most recent call last):
  File "/home/user/bitcoinarmory/./ArmoryQt.py", line 5280, in closeForReal
    TheBridge.shutdown()
  File "/home/user/bitcoinarmory/armoryengine/CppBridge.py", line 390, in shutdown
    self.clientFut.result()
  File "/usr/lib/python3.9/concurrent/futures/_base.py", line 440, in result
    return self.__get_result()
  File "/usr/lib/python3.9/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/usr/lib/python3.9/concurrent/futures/thread.py", line 52, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/user/bitcoinarmory/armoryengine/CppBridge.py", line 259, in readBridgeSocket
    response = self.bip15xConnection.decrypt(\
  File "/home/user/bitcoinarmory/armoryengine/BIP15x.py", line 312, in decrypt
    raise AEAD_Error("failed to decrypt payload: " + str(decryptionResult))
armoryengine.BIP15x.AEAD_Error: failed to decrypt payload: -1
(INFO) ArmoryQt.py:5290 - Attempting to close the main window!

the ArmoryQt.py process does terminate ok.


It seems like porting to a different Qt version/lib was pretty clean in terms of outcome, so that's something. The dashboard wallet scan animation was always a little "janky" under Xen, so some gui bugs are already fixed!

[goatpig]

1. Haven't imported a wallet with private keys in a while, time to retest it I guess.

2. How old is your DB? Current code received a set of fixes for txfilters vs what 0.96.5 puts out. Did you setup from scratch or used an existing db state?

3. I forgot this button even existed o.o', will fix

Thanks for the report.

It seems like porting to a different Qt version/lib was pretty clean in terms of outcome, so that's something. The dashboard wallet scan animation was always a little "janky" under Xen, so some gui bugs are already fixed!

There are some minor syntax changes from Qt5 to Qt4, but its essentially the same lib. I was surprised how little impact the migration had on looks. I was expecting a struggle just to port it and then end up with botched dialogs. Py2 to Py3 was a lot more disruptive in the end.