Armory + Bitcoin Core GUI doesn't work, Armory + bitcoind does ??? - Win10 x64

[HCP]

First up... the setup:

OS: Windows 10 Pro x64 (with latest patches, FCU etc)
Armory 0.96.4 RC3 (aka 0.96.3.992)
Bitcoin Core 0.15.1

ArmoryDB commandline:

Code:

ArmoryDB.exe --satoshi-datadir=E:\Bitcoin --cookie

bitcoind commandline:

Code:

bitcoind.exe -datadir=E:\Bitcoin

bitcoin.conf (in E:\Bitcoin) is:

Code:

walletrbf=1
dbcache=750
server=1

So, given the above does anyone have any idea why the Bitcoin Core GUI = Node Offline... whereas bitcoind.exe = Node Online ??!?

I've tried running Bitcoin Core GUI... and then running ArmoryDB from the commandline and it seems that ArmoryDB is "dying" silently after it attempts to enable zero-conf tracking:

Code:

-INFO  - 21:09:40.000: (..\main.cpp:32) Running on 4 threads
-INFO  - 21:09:40.000: (..\main.cpp:33) Ram usage level: 50
-INFO  - 21:09:40.000: (..\BlockUtils.cpp:915) blkfile dir: E:\Bitcoin/blocks
-INFO  - 21:09:40.000: (..\BlockUtils.cpp:916) lmdb dir: C:\Users\USER\AppData\Roaming/Armory/databases
-INFO  - 21:09:40.000: (..\lmdb_wrapper.cpp:388) Opening databases...
-INFO  - 21:09:40.047: (e:\users\goat\code\armory6\cppforswig\BDM_Server.h:263) Listening on port 56896
-INFO  - 21:09:40.047: (..\nodeRPC.cpp:57) RPC connection established
-INFO  - 21:09:40.047: (..\BlockDataManagerConfig.cpp:895) waiting on node sync: 0.999999%
-INFO  - 21:09:40.047: (..\nodeRPC.cpp:425) Node is ready
-INFO  - 21:09:40.047: (..\BlockUtils.cpp:1108) Executing: doInitialSyncOnLoad
-INFO  - 21:10:01.609: (..\DatabaseBuilder.cpp:199) Reading headers from db
-INFO  - 21:10:03.766: (..\DatabaseBuilder.cpp:238) Found 504986 headers in db
-INFO  - 21:10:06.000: (..\DatabaseBuilder.cpp:64) Rewinding 100 blocks
-INFO  - 21:10:06.000: (..\DatabaseBuilder.cpp:71) updating HEADERS db
-INFO  - 21:10:06.094: (..\DatabaseBuilder.cpp:493) Found next block after skipping 1089130bytes
-INFO  - 21:10:07.781: (..\DatabaseBuilder.cpp:281) parsed block file #1144
-INFO  - 21:10:07.156: (..\DatabaseBuilder.cpp:281) parsed block file #1145
-INFO  - 21:10:07.156: (..\Blockchain.cpp:248) Organizing chain 
-INFO  - 21:10:07.203: (..\Blockchain.cpp:370) Organized chain in 0s
-INFO  - 21:10:07.203: (..\DatabaseBuilder.cpp:76) updated HEADERS db in 1s
-INFO  - 21:10:07.219: (..\lmdb_wrapper.cpp:388) Opening databases...
-INFO  - 21:10:07.234: (..\DatabaseBuilder.cpp:1231) verifying txfilters integrity
-INFO  - 21:10:07.469: (..\DatabaseBuilder.cpp:1314) done checking txfilters
-INFO  - 21:10:07.516: (..\DatabaseBuilder.cpp:134) scanning new blocks from #504944 to #504946
-INFO  - 21:10:07.547: (..\BlockchainScanner.cpp:852) scanned from block #504944 to #504946
-INFO  - 21:10:07.625: (..\lmdb_wrapper.cpp:388) Opening databases...
-INFO  - 21:10:07.641: (..\DatabaseBuilder.cpp:186) scanned new blocks in 0s
-INFO  - 21:10:07.641: (..\DatabaseBuilder.cpp:190) init db in 27s
-INFO  - 21:10:07.656: (..\BDM_supportClasses.cpp:1891) Enabling zero-conf tracking

Debug output just stops at this point (left it for over 30 minutes and several new blocks)


The SAME setup, but running bitcoind instead of the GUI... and then, again, running ArmoryDB from the commandline yields:

Code:

-INFO  - 21:20:55.000: (..\main.cpp:32) Running on 4 threads
-INFO  - 21:20:55.000: (..\main.cpp:33) Ram usage level: 50
-INFO  - 21:20:55.000: (..\BlockUtils.cpp:915) blkfile dir: E:\Bitcoin/blocks
-INFO  - 21:20:55.000: (..\BlockUtils.cpp:916) lmdb dir: C:\Users\USER\AppData\Roaming/Armory/databases
-INFO  - 21:20:55.000: (..\lmdb_wrapper.cpp:388) Opening databases...
-INFO  - 21:20:55.032: (e:\users\goat\code\armory6\cppforswig\BDM_Server.h:263) Listening on port 59100
-INFO  - 21:20:55.391: (..\nodeRPC.cpp:57) RPC connection established
-INFO  - 21:21:04.422: (..\BlockDataManagerConfig.cpp:895) waiting on node sync: 0.999996%
-INFO  - 21:21:04.422: (..\BitcoinP2P.cpp:947) Connected to Bitcoin node
-INFO  - 21:21:04.422: (..\nodeRPC.cpp:425) Node is ready
-INFO  - 21:21:04.422: (..\BlockUtils.cpp:1108) Executing: doInitialSyncOnLoad
-INFO  - 21:21:04.469: (..\DatabaseBuilder.cpp:199) Reading headers from db
-INFO  - 21:21:07.750: (..\DatabaseBuilder.cpp:238) Found 504989 headers in db
-INFO  - 21:21:09.172: (..\DatabaseBuilder.cpp:64) Rewinding 100 blocks
-INFO  - 21:21:09.172: (..\DatabaseBuilder.cpp:71) updating HEADERS db
-INFO  - 21:21:09.172: (..\DatabaseBuilder.cpp:493) Found next block after skipping 1029916bytes
-INFO  - 21:21:09.500: (..\DatabaseBuilder.cpp:281) parsed block file #1144
-INFO  - 21:21:10.829: (..\DatabaseBuilder.cpp:281) parsed block file #1145
-INFO  - 21:21:10.829: (..\Blockchain.cpp:248) Organizing chain 
-INFO  - 21:21:10.860: (..\Blockchain.cpp:370) Organized chain in 0s
-INFO  - 21:21:10.860: (..\DatabaseBuilder.cpp:76) updated HEADERS db in 0s
-INFO  - 21:21:10.875: (..\lmdb_wrapper.cpp:388) Opening databases...
-INFO  - 21:21:10.891: (..\DatabaseBuilder.cpp:1231) verifying txfilters integrity
-INFO  - 21:21:10.141: (..\DatabaseBuilder.cpp:1314) done checking txfilters
-INFO  - 21:21:10.188: (..\DatabaseBuilder.cpp:134) scanning new blocks from #504947 to #504948
-INFO  - 21:21:10.219: (..\BlockchainScanner.cpp:852) scanned from block #504947 to #504948
-INFO  - 21:21:10.297: (..\lmdb_wrapper.cpp:388) Opening databases...
-INFO  - 21:21:10.329: (..\DatabaseBuilder.cpp:186) scanned new blocks in 0s
-INFO  - 21:21:10.329: (..\DatabaseBuilder.cpp:190) init db in 5s
-INFO  - 21:21:10.329: (..\BDM_supportClasses.cpp:1891) Enabling zero-conf tracking
-WARN  - 21:22:18.610: (..\BDM_supportClasses.cpp:1944) running 5 zc parser threads
-WARN  - 21:22:18.610: (..\BDM_supportClasses.cpp:1944) running 10 zc parser threads
-WARN  - 21:22:18.610: (..\BDM_supportClasses.cpp:1944) running 15 zc parser threads
-WARN  - 21:22:18.610: (..\BDM_supportClasses.cpp:1944) running 20 zc parser threads
-WARN  - 21:22:18.610: (..\BDM_supportClasses.cpp:1944) running 25 zc parser threads
-WARN  - 21:22:18.610: (..\BDM_supportClasses.cpp:1944) running 30 zc parser threads
-WARN  - 21:25:41.547: (..\BDM_supportClasses.cpp:1944) running 35 zc parser threads
-WARN  - 21:25:41.547: (..\BDM_supportClasses.cpp:1944) running 40 zc parser threads
-WARN  - 21:25:46.094: (..\BDM_supportClasses.cpp:1944) running 45 zc parser threads
-WARN  - 21:25:46.094: (..\BDM_supportClasses.cpp:1944) running 50 zc parser threads
-WARN  - 21:26:14.750: (..\BDM_supportClasses.cpp:1944) running 55 zc parser threads
-WARN  - 21:26:14.750: (..\BDM_supportClasses.cpp:1944) running 60 zc parser threads
-WARN  - 21:26:14.750: (..\BDM_supportClasses.cpp:1944) running 65 zc parser threads
-WARN  - 21:26:15.250: (..\BDM_supportClasses.cpp:1944) running 70 zc parser threads
-WARN  - 21:26:15.250: (..\BDM_supportClasses.cpp:1944) running 75 zc parser threads
-WARN  - 21:26:15.250: (..\BDM_supportClasses.cpp:1944) running 80 zc parser threads
-WARN  - 21:34:48.454: (..\BDM_supportClasses.cpp:1944) running 85 zc parser threads
-WARN  - 21:34:48.454: (..\BDM_supportClasses.cpp:1944) running 90 zc parser threads
-WARN  - 21:34:48.454: (..\BDM_supportClasses.cpp:1944) running 95 zc parser threads
-WARN  - 21:34:48.454: (..\BDM_supportClasses.cpp:1944) running 100 zc parser threads
-ERROR - 21:43:26.907: (..\BitcoinP2P.cpp:1037) caught StopBlockingLoop in processDataStackThread
-INFO  - 21:43:26.907: (..\BitcoinP2P.cpp:969) Disconnected from Bitcoin node
-INFO  - 21:43:36.985: (..\BitcoinP2P.cpp:947) Connected to Bitcoin node
-INFO  - 22:01:44.719: (..\BlockchainScanner.cpp:852) scanned from block #504949 to #504952
-INFO  - 22:19:02.047: (..\BDM_Server.cpp:1121) registered bdv: 54538a2d22a57f102c2b
-INFO  - 22:21:45.735: (..\BlockchainScanner.cpp:857) scanned block #504953
-INFO  - 22:22:19.735: (..\BlockchainScanner.cpp:857) scanned block #504954
-ERROR - 22:24:25.000: (..\BitcoinP2P.cpp:1037) caught StopBlockingLoop in processDataStackThread
-INFO  - 22:24:25.000: (..\BitcoinP2P.cpp:969) Disconnected from Bitcoin node
-INFO  - 22:24:32.282: (..\BitcoinP2P.cpp:947) Connected to Bitcoin node

Anyone have any ideas for anything I can try so I can get this working with the GUI... or maybe try and figure out why GUI doesn't work but bitcoind does so maybe the devs can fix it?

[1714496403]

1714496403

[1714496403]

1714496403

[1714496403]

1714496403

[1714496403]

1714496403

[PhoenixFire]

I'd try adding "listen=1" to your bitcoin.conf

[goatpig]

Consider using addnode=127.0.0.1 in your node .conf. If your node fills up its default incoming slots, it may just kick ArmoryDB out as it identifies itself as basically a useless node.

[PhoenixFire]

Consider using addnode=127.0.0.1 in your node .conf. If your node fills up its default incoming slots, it may just kick ArmoryDB out as it identifies itself as basically a useless node.

Would whitelist=127.0.0.1 be better, perhaps? Not sure if that'd really help keep a slot open for it, though.
Using addnode=127.0.0.1 unfortunately has the side effect of Core constantly trying to connect to itself. Might be benign, but does flood the logs.

Incidentally, I set up a Windows 10 x64 VM, for testing, given that we've had what looks like a trend of issues on Win 10.
As is the case with HCP... bitcoind connects and plays nice once everything is up to date. When running bitcoin-qt manually the best I can get is Connected, but Armory thinks the RPC server is down. I can run the usual RPC commands with bitcoin-cli, which suggests the RPC server is, in fact available? (And it is enabled in bitcoin.conf).
Doesn't seem to be down to any internal firewalling, and it is running 0.96.4 RC3 and Core 0.15.1, of course.

I'm not wanting to confuse and hijack the thread - I don't need this Win10 VM to work - but think there is a common issue here.

[HCP]

Incidentally, I set up a Windows 10 x64 VM, for testing, given that we've had what looks like a trend of issues on Win 10.
As is the case with HCP... bitcoind connects and plays nice once everything is up to date. When running bitcoin-qt manually the best I can get is Connected, but Armory thinks the RPC server is down. I can run the usual RPC commands with bitcoin-cli, which suggests the RPC server is, in fact available? (And it is enabled in bitcoin.conf).
Doesn't seem to be down to any internal firewalling, and it is running 0.96.4 RC3 and Core 0.15.1, of course.

I'm not wanting to confuse and hijack the thread - I don't need this Win10 VM to work - but think there is a common issue here.

Indeed, the only reason I have been testing this was the increasing number of folks having issues with Win10 and Armory.

I'd try adding "listen=1" to your bitcoin.conf

Having listen=1 seems to work with the GUI, but then drastically ramps up the bandwidth requirements. I'm still unsure as to why the GUI needs listen=1, but bitcoind does not... very curious.

[PhoenixFire]

Having listen=1 seems to work with the GUI, but then drastically ramps up the bandwidth requirements. I'm still unsure as to why the GUI needs listen=1, but bitcoind does not... very curious.

So with just that added you get to the green connected state? No RPC issues?

The GUI is there to act as a wallet and a node, together, and so it doesn't need to open itself up to anything external (including other processes on the same computer) in order to function usefully.
In contrast bitcoind is a daemon - daemons are headless and work in the background. If it didn't accept incoming connections or some form of external input by default, whether that be via bitcoin-cli or RPC, it wouldn't serve a useful function.
That is my understanding of it, at least.

[goatpig]

On Windows I use BitcoinQt exclusively and I don't use listen=1. listen=1 is to allow your node to be discovered by others on the network, resulting in incoming connections. Armory connects as an incoming node so maybe listen=1 affects the default connections ceiling (8 connections max).

I recommend use addnode because Armory does not present itself as an archival node over the P2P interface, which leads me to believe a Core instance limited to the default 8 outgoing connections would choose to suppress the ArmoryDB connection in favor of a real network node. I've often noticed my db gets booted off of my node, even in testnet, without the use of addnode. Addnode invariably fixes this issue.

I'm highlighting this issue because your logs demonstrate your node getting booted and reconnecting when using BitcoinQt.

Would whitelist=127.0.0.1 be better, perhaps? Not sure if that'd really help keep a slot open for it, though.
Using addnode=127.0.0.1 unfortunately has the side effect of Core constantly trying to connect to itself. Might be benign, but does flood the logs.

I have never experienced my node trying to loopback to itself. Whiltelist allows for the connection (even if the node score would have the DB booted otherwise). It does not reserve a slot for the DB over other valid nodes.

[HCP]

It is indeed very strange. There must be some issue with my setup that is preventing ArmoryDB from working as it should.

With listen=1, it works... without that and with just addnode=127.0.0.1, it just seems to stop at the "-INFO  - 11:05:57.281: (..\BDM_supportClasses.cpp:1891) Enabling zero-conf tracking" line in debug...

I'm honestly at a total loss as to why bitcoind works and QT doesn't... obviously something in my setup is "non standard" or Windows10 is just borked

[goatpig]

"-INFO  - 11:05:57.281: (..\BDM_supportClasses.cpp:1891) Enabling zero-conf tracking" line in debug...

That's the last line the DB puts out once it's done initializing. You saying clients can't connect past that point? Is there not a line saying the Node is ready? Does it just miss all new blocks past that point (which would be a pathing issue, not a socketing one).

[HCP]

With BitcoinQT open, and only addnode=127.0.0.1, (no "listen"), this is full dbLog.txt:

Code:

Log file opened at 22:00:49.000: C:\Users\USER\AppData\Roaming\Armory\dbLog.txt
-INFO  - 22:00:49.000: (..\main.cpp:32) Running on 4 threads
-INFO  - 22:00:49.000: (..\main.cpp:33) Ram usage level: 50
-INFO  - 22:00:49.000: (..\BlockUtils.cpp:915) blkfile dir: E:\Bitcoin\blocks
-INFO  - 22:00:49.000: (..\BlockUtils.cpp:916) lmdb dir: C:\Users\USER\AppData\Roaming\Armory\databases
-INFO  - 22:00:49.000: (..\lmdb_wrapper.cpp:388) Opening databases...
-INFO  - 22:00:49.031: (e:\users\goat\code\armory6\cppforswig\BDM_Server.h:263) Listening on port 61388
-INFO  - 22:00:49.046: (..\nodeRPC.cpp:57) RPC connection established
-INFO  - 22:00:49.109: (..\BlockDataManagerConfig.cpp:895) waiting on node sync: 0.999997%
-INFO  - 22:00:49.109: (..\nodeRPC.cpp:425) Node is ready
-INFO  - 22:00:49.109: (..\BlockUtils.cpp:1108) Executing: doInitialSyncOnLoad
-INFO  - 22:00:50.062: (..\BDM_Server.cpp:1121) registered bdv: 53de366ba5150f469baa
-INFO  - 22:02:20.687: (..\DatabaseBuilder.cpp:199) Reading headers from db
-INFO  - 22:02:23.187: (..\DatabaseBuilder.cpp:238) Found 505543 headers in db
-INFO  - 22:02:26.843: (..\DatabaseBuilder.cpp:64) Rewinding 100 blocks
-INFO  - 22:02:26.843: (..\DatabaseBuilder.cpp:71) updating HEADERS db
-INFO  - 22:02:26.890: (..\DatabaseBuilder.cpp:493) Found next block after skipping 1021719bytes
-INFO  - 22:02:31.843: (..\DatabaseBuilder.cpp:281) parsed block file #1149
-INFO  - 22:02:31.843: (..\Blockchain.cpp:248) Organizing chain 
-INFO  - 22:02:31.875: (..\Blockchain.cpp:370) Organized chain in 0s
-INFO  - 22:02:31.875: (..\DatabaseBuilder.cpp:76) updated HEADERS db in 5s
-INFO  - 22:02:31.890: (..\lmdb_wrapper.cpp:388) Opening databases...
-INFO  - 22:02:31.921: (..\DatabaseBuilder.cpp:1231) verifying txfilters integrity
-INFO  - 22:02:35.015: (..\DatabaseBuilder.cpp:1314) done checking txfilters
-INFO  - 22:02:35.078: (..\DatabaseBuilder.cpp:134) scanning new blocks from #505501 to #505500
-INFO  - 22:02:35.078: (..\BlockchainScanner.cpp:52) no history to scan
-INFO  - 22:02:35.078: (..\BlockchainScanner.cpp:1021) no SSH to scan
-INFO  - 22:02:35.171: (..\lmdb_wrapper.cpp:388) Opening databases...
-INFO  - 22:02:35.187: (..\DatabaseBuilder.cpp:186) scanned new blocks in 0s
-INFO  - 22:02:35.187: (..\DatabaseBuilder.cpp:190) init db in 106s
-INFO  - 22:02:35.187: (..\BDM_supportClasses.cpp:1891) Enabling zero-conf tracking

Nothing else happens... I've left it for several blocks... it shows "Node Offline (505500 Blocks)" in red, which is what the block height was when I opened Armory. No new blocks are scanned by ArmoryDB...

ArmoryLog is here: https://pastebin.com/uM8LzpH7

As you can see, there are "RPC connection established" and "Node is Ready" messages...

Code:

-INFO  - 22:00:49.046: (..\nodeRPC.cpp:57) RPC connection established
-INFO  - 22:00:49.109: (..\BlockDataManagerConfig.cpp:895) waiting on node sync: 0.999997%
-INFO  - 22:00:49.109: (..\nodeRPC.cpp:425) Node is ready

But then nothing after that "zero-conf tracking" line... It is like ArmoryDB just stops.  Bitcoin Core itself is happily updating and syncing new blocks... and if I close Armory and restart it, it'll sync up the new blocks, but again, "Node Offline"


As for pathing... ArmoryQt.conf is:

Code:

satoshi-datadir="E:\Bitcoin\blocks"
dbdir="C:\Users\USER\AppData\Roaming\Armory\databases"

If all I do is run bitcoind instead... everything works. The only obvious difference I can see in the Bitcoin Core debug log when running BitcoinQT vs bitcoind is:

BitcoinQT:

2018-01-22 08:53:31 AddLocal([2001:0:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx]:8333,1)
2018-01-22 08:53:31 Discover: MyPC - 2001:0:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx
2018-01-22 08:53:31 init message: Loading P2P addresses...

vs.
bitcoind:

2018-01-22 09:22:08 AddLocal([2001:0:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx]:8333,1)
2018-01-22 09:22:08 Discover: MyPC - 2001:0:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx
2018-01-22 09:22:08 Bound to [::]:8333
2018-01-22 09:22:08 Bound to 0.0.0.0:8333
2018-01-22 09:22:08 init message: Loading P2P addresses...

and if I use listen=1, and run BitcoinQT:

2018-01-22 09:37:19 AddLocal([2001:0:9d38:6abd:249c:1a7a:d0b7:40c2]:8333,1)
2018-01-22 09:37:19 Discover: HardCorePC - 2001:0:9d38:6abd:249c:1a7a:d0b7:40c2
2018-01-22 09:37:19 Bound to [::]:8333
2018-01-22 09:37:19 Bound to 0.0.0.0:8333
2018-01-22 09:37:19 init message: Loading P2P addresses...

One last difference I just noted in dbLog.txt... when running bitcoind or QT with listen=1, I see this in the debug:

Code:

-INFO  - 22:23:30.547: (..\BitcoinP2P.cpp:947) Connected to Bitcoin node

which (as the log file above shows) isn't there when I run QT without "listen"...

[goatpig]

1) Observe the DB without "listen=1". Is it eating a lot of RAM or maxing out 1 CPU core?

2) With the same setup, run the DB manually instead of letting the client spawn it.

[HCP]

1) Observe the DB without "listen=1". Is it eating a lot of RAM or maxing out 1 CPU core?

I wouldn't call 330+ Meg "a lot"...


The 1.7gigs that Chrome is currently eating on the other hand!

2) With the same setup, run the DB manually instead of letting the client spawn it.

Similar sort of result...

[goatpig]

I don't really know what else to look at then. I do not experience this issue at all on Win10 and I don't use bitcoind. There has to be something at socketing level. Do you run BitcoinQt with or without the wallet? Mine is always with disablewallet=1

[HCP]

I know right?... so weird... I just tried with "disablewallet=1"... Armory starts up and I still get the "disconnected" notification in the System Tray.

Obviously something in my Win10 setup isn't quite "right"... likely some random setting I have made that is preventing it from working. I'll keep experimenting with various Windows Settings (I've tried disabling all the Firewall and Windows Defender stuff) and see if I can figure it out.

Until then, I have some workarounds so it's not a show stopper... thanks for the help, I'll update if I figure it out.

[goatpig]

Next thing I would have gone to is the Windows firewall but it seems you beat at it. If anything compare the BitcoinQt and bitcoind firewall rules. Who knows if you can fully disable that thing, Win10 and all...

I guess the only real way for you to know what's going on is to look at the connection loop in BitcoinP2P.cpp in a debugger.