"Parsing Tx Hashes" all night, hung or wait?

[jojo69]

terribly sorry for the delay

Code:

stg@UD5:~
$ cd /usr/bin
stg@UD5:/usr/bin
$ ./ArmoryDB --dbdir="/media/BITCORE/armory" --satoshi-datadir="/media/BITCORE/Blockchain"
/home/stg
logging in /home/stg/.armory/dbLog.txt
-INFO  - 19:34:04: (main.cpp:32) Running on 8 threads
-INFO  - 19:34:04: (main.cpp:33) Ram usage level: 50
-INFO  - 19:34:04: (BlockUtils.cpp:915) blkfile dir: /media/BITCORE/Blockchain/blocks
-INFO  - 19:34:04: (BlockUtils.cpp:916) lmdb dir: /media/BITCORE/armory
-INFO  - 19:34:04: (lmdb_wrapper.cpp:388) Opening databases...
-INFO  - 19:34:04: (BDM_Server.h:263) Listening on port 9001
-INFO  - 19:34:04: (BlockUtils.cpp:1108) Executing: doInitialSyncOnLoad
-INFO  - 19:34:04: (DatabaseBuilder.cpp:199) Reading headers from db
-WARN  - 19:34:04: (lmdb_wrapper.cpp:1241) No headers in DB yet!
-INFO  - 19:34:04: (DatabaseBuilder.cpp:238) Found 1 headers in db
-INFO  - 19:34:04: (DatabaseBuilder.cpp:71) updating HEADERS db
-INFO  - 19:34:04: (Blockchain.cpp:248) Organizing chain 
-INFO  - 19:34:04: (Blockchain.cpp:370) Organized chain in 0s
-INFO  - 19:34:04: (DatabaseBuilder.cpp:76) updated HEADERS db in 0s
-INFO  - 19:34:04: (lmdb_wrapper.cpp:388) Opening databases...
-INFO  - 19:34:04: (DatabaseBuilder.cpp:1231) verifying txfilters integrity
-INFO  - 19:34:04: (DatabaseBuilder.cpp:1314) done checking txfilters
-INFO  - 19:34:04: (BDM_supportClasses.cpp:1891) Enabling zero-conf tracking

that seems to return the correct pathing no?

[HCP]

So I created an armorydb.conf in /home/stg/.armory/

Code:

--dbdir="/media/BITCORE/armory"
--satoshi-datadir="/media/BITCORE/Blockchain"

That armorydb.conf looks wrong... I believe it should just be:

Code:

dbdir="/media/BITCORE/armory"
satoshi-datadir="/media/BITCORE/Blockchain"

You don't include the --'s in the .conf files. If you do, the options get ignored. It seems the comment in the pathing docs is slightly incorrect:

You do not need to prepend args in the config file with double dashes (–), but their use is legal

I'm fairly sure that the double dashes causes issues in the .conf files... best not to use them.

that seems to return the correct pathing no?

Yep, that seems to be picking up the correct pathing when you run it from the commandline with the "/media/BITCORE/..." paths.

[jojo69]

thanks HCP,

just to be clear, am I good to ctrl C the DB process and close that console?  The DB doesn't need any special shutdown does it?

[bob123]

just to be clear, am I good to ctrl C the DB process and close that console?  The DB doesn't need any special shutdown does it?

Ctrl + C sends a SIGINT signal to the process. This is not a termination signal like SIGKILL or SIGSTOP.
What the software does after receiving a SIGINT completely depends on the software. It can be ignored, or used as a "shutdown command".

Pressing ctrl + c does not terminate the process. It tells the process to stop / terminate itself. So, yes. you are fine using ctrl + c as its just asking the software to stop.

[jojo69]

thanks, I did not know that

[jojo69]

Code:

-ERROR - : (StringSockets.cpp:351) FcgiSocket::writeAndRead FcgiError: unknown fcgi header request byte

is this a clue?

[goatpig]

Code:

-ERROR - : (StringSockets.cpp:351) FcgiSocket::writeAndRead FcgiError: unknown fcgi header request byte

is this a clue?

Something else than ArmoryDB is listening on the port that ArmoryQt connection to.

[jojo69]

ok, is there a guide to modifying the port settings?

[bob123]

ok, is there a guide to modifying the port settings?

You can start armorydb with a custom port by using the --fcgi-port=X argument:

Code:

armorydb.exe --fcgi-port=X

(Replace X with your port, e.g. 55555).


Or you can add this to your config file:

Code:

fcgi-port=X

(Replace X with your port, e.g. 55555).

[jojo69]

would I also have to change the port that QT is sending on, or am I misunderstanding the whole thing?

[bob123]

would I also have to change the port that QT is sending on, or am I misunderstanding the whole thing?

I don't think so, no.
The problem seems to be between the 2 armory processes: armoryqt and armorydb.

Armoryqt tries to connect to armorydb on a specific port but receives a weird reply -> Some other process is listening on that port.
As far as i am aware, there is no problem between the communication of armory and bitcoin core.

[jojo69]

I know that this is veering off topic of strictly Armory support into general linux tutoring, but is there a way to discover what this other process is that is squatting my port?

[HCP]

You could try using netstat...

Code:

netstat -ltnp

Have a look through the list for the port number you're interested in... You may need to run it using sudo to see all the system processes etc.

Or maybe one of the methods here: https://www.tecmint.com/find-out-which-process-listening-on-a-particular-port/


The thing is that if you let Armory spawn ArmoryDB (ie. you just start with armory command), then it should spawn ArmoryDB with a "random" port number... and it won't use 9001.

Code:

hcp@hcp-VirtualBox:~$ sudo netstat -ltnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 127.0.0.1:52811         0.0.0.0:*               LISTEN      4965/ArmoryDB       
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      536/systemd-resolve 
tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN      4720/cupsd          
tcp        0      0 127.0.0.1:8223          0.0.0.0:*               LISTEN      4852/python2        
tcp6       0      0 ::1:631                 :::*                    LISTEN      4720/cupsd          
hcp@hcp-VirtualBox:~$

on next run:

Code:

hcp@hcp-VirtualBox:~$ sudo netstat -ltnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 127.0.0.1:62284         0.0.0.0:*               LISTEN      5234/ArmoryDB       
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      536/systemd-resolve 
tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN      4720/cupsd          
tcp        0      0 127.0.0.1:8223          0.0.0.0:*               LISTEN      5121/python2        
tcp6       0      0 ::1:631                 :::*                    LISTEN      4720/cupsd          
hcp@hcp-VirtualBox:~$ 

You can see the port changed from 52811 to 62284... however, if I spawn ArmoryDB manually using ArmoryDB from the terminal, it always tries for 9001 unless I use the --fcgi-port command:

Code:

hcp@hcp-VirtualBox:~$ sudo netstat -ltnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      536/systemd-resolve 
tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN      4720/cupsd          
tcp        0      0 127.0.0.1:9001          0.0.0.0:*               LISTEN      5314/ArmoryDB       
tcp6       0      0 ::1:631                 :::*                    LISTEN      4720/cupsd          
hcp@hcp-VirtualBox:~$ 

and with ArmoryDB --fcgi-port=52828:

Code:

hcp@hcp-VirtualBox:~$ sudo netstat -ltnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      536/systemd-resolve 
tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN      4720/cupsd          
tcp        0      0 127.0.0.1:52828         0.0.0.0:*               LISTEN      5326/ArmoryDB       
tcp6       0      0 ::1:631                 :::*                    LISTEN      4720/cupsd          
hcp@hcp-VirtualBox:~$ 

[jojo69]

netstat -ltnp returns

Code:

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN      2443/cupsd          
tcp        0      0 0.0.0.0:445             0.0.0.0:*               LISTEN      2915/smbd           
tcp        0      0 127.0.0.1:8223          0.0.0.0:*               LISTEN      6385/python2        
tcp        0      0 0.0.0.0:139             0.0.0.0:*               LISTEN      2915/smbd           
tcp        0      0 0.0.0.0:49709           0.0.0.0:*               LISTEN      1876/rpc.statd      
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      1847/rpcbind        
tcp6       0      0 ::1:631                 :::*                    LISTEN      2443/cupsd          
tcp6       0      0 :::445                  :::*                    LISTEN      2915/smbd           
tcp6       0      0 :::36579                :::*                    LISTEN      1876/rpc.statd      
tcp6       0      0 :::6566                 :::*                    LISTEN      2744/saned          
tcp6       0      0 :::139                  :::*                    LISTEN      2915/smbd           
tcp6       0      0 :::111                  :::*                    LISTEN      1847/rpcbind   

this while armorycplog.txt is spamming

Code:

-ERROR - : (StringSockets.cpp:351) FcgiSocket::writeAndRead FcgiError: unknown fcgi header request byte

fresh armorylog.txt   https://pastebin.com/0ZCpn4ua

fresh dblog.txt 

Log file opened at 10:59:34: /home/stg/.armory/dbLog.txt
-INFO  - 10:59:34: (main.cpp:32) Running on 8 threads
-INFO  - 10:59:34: (main.cpp:33) Ram usage level: 50
-INFO  - 10:59:34: (BlockUtils.cpp:915) blkfile dir: /media/BITCORE/Blockchain/blocks
-INFO  - 10:59:34: (BlockUtils.cpp:916) lmdb dir: /home/stg/.armory/databases
-INFO  - 10:59:34: (lmdb_wrapper.cpp:388) Opening databases...
-INFO  - 10:59:34: (BDM_Server.h:263) Listening on port 59328
-INFO  - 10:59:34: (BlockUtils.cpp:1108) Executing: doInitialSyncOnLoad
-INFO  - 10:59:34: (DatabaseBuilder.cpp:199) Reading headers from db
-WARN  - 10:59:34: (lmdb_wrapper.cpp:1241) No headers in DB yet!
-INFO  - 10:59:34: (DatabaseBuilder.cpp:238) Found 1 headers in db
-INFO  - 10:59:34: (DatabaseBuilder.cpp:71) updating HEADERS db
-INFO  - 10:59:34: (Blockchain.cpp:248) Organizing chain
-INFO  - 10:59:34: (Blockchain.cpp:370) Organized chain in 0s
-INFO  - 10:59:34: (DatabaseBuilder.cpp:76) updated HEADERS db in 0s
-INFO  - 10:59:34: (lmdb_wrapper.cpp:388) Opening databases...
-INFO  - 10:59:34: (DatabaseBuilder.cpp:1231) verifying txfilters integrity
-INFO  - 10:59:34: (DatabaseBuilder.cpp:1314) done checking txfilters
-INFO  - 10:59:34: (BDM_supportClasses.cpp:1891) Enabling zero-conf tracking
-INFO  - 10:59:34: (BDM_Server.cpp:1121) registered bdv: efd166360b74055c3d4d
-INFO  - 10:59:34: (BDM_supportClasses.cpp:401) Starting address registration process
-ERROR - 10:59:34: (lmdb_wrapper.cpp:1503) Headers DB has no block at height: 0
-ERROR - 10:59:34: (lmdb_wrapper.cpp:1483) No headers at height 0
-ERROR - 10:59:34: (BlockchainScanner.cpp:445) Missing file map for output scan, this is unexpected
-ERROR - 10:59:34: (BlockchainScanner.cpp:447) Has the following block files:
-ERROR - 10:59:34: (BlockchainScanner.cpp:451) Was looking for id #4294967295


Log file opened at 11:17:09: /home/stg/.armory/dbLog.txt
-INFO  - 11:17:09: (main.cpp:32) Running on 8 threads
-INFO  - 11:17:09: (main.cpp:33) Ram usage level: 50
-INFO  - 11:17:09: (BlockUtils.cpp:915) blkfile dir: /media/BITCORE/Blockchain/blocks
-INFO  - 11:17:09: (BlockUtils.cpp:916) lmdb dir: /home/stg/.armory/databases
-INFO  - 11:17:09: (lmdb_wrapper.cpp:388) Opening databases...
-INFO  - 11:17:09: (BDM_Server.h:263) Listening on port 50407
-INFO  - 11:17:09: (BlockUtils.cpp:1108) Executing: doInitialSyncOnLoad
-INFO  - 11:17:09: (DatabaseBuilder.cpp:199) Reading headers from db
-WARN  - 11:17:09: (lmdb_wrapper.cpp:1241) No headers in DB yet!
-INFO  - 11:17:09: (DatabaseBuilder.cpp:238) Found 1 headers in db
-INFO  - 11:17:09: (DatabaseBuilder.cpp:71) updating HEADERS db
-INFO  - 11:17:09: (Blockchain.cpp:248) Organizing chain
-INFO  - 11:17:09: (Blockchain.cpp:370) Organized chain in 0s
-INFO  - 11:17:09: (DatabaseBuilder.cpp:76) updated HEADERS db in 0s
-INFO  - 11:17:09: (lmdb_wrapper.cpp:388) Opening databases...
-INFO  - 11:17:09: (DatabaseBuilder.cpp:1231) verifying txfilters integrity
-INFO  - 11:17:09: (DatabaseBuilder.cpp:1314) done checking txfilters
-INFO  - 11:17:09: (BDM_supportClasses.cpp:1891) Enabling zero-conf tracking
-INFO  - 11:17:09: (BDM_Server.cpp:1121) registered bdv: bbff82b8b99c0c3a6db6
-INFO  - 11:17:09: (BDM_supportClasses.cpp:401) Starting address registration process
-ERROR - 11:17:09: (lmdb_wrapper.cpp:1503) Headers DB has no block at height: 0
-ERROR - 11:17:09: (lmdb_wrapper.cpp:1483) No headers at height 0
-ERROR - 11:17:09: (BlockchainScanner.cpp:445) Missing file map for output scan, this is unexpected
-ERROR - 11:17:09: (BlockchainScanner.cpp:447) Has the following block files:
-ERROR - 11:17:09: (BlockchainScanner.cpp:451) Was looking for id #4294967295

I have gone so far as to uninstall armory and reinstall it and this behavior persists.

[HCP]

This is your problem right here:

2021-01-07 11:02:07 (WARNING) -- SDM.py:518 - bitcoind exited, bitcoind STDERR:
2021-01-07 11:02:07 (WARNING) -- SDM.py:520 - : You need to rebuild the database using -reindex to go back to unpruned mode.  This will redownload the entire blockchain.
2021-01-07 11:02:07 (WARNING) -- SDM.py:520 - Please restart with -reindex or -reindex-chainstate to recover.
2021-01-07 11:02:07 (WARNING) -- SDM.py:520 -

-ERROR - 10:59:34: (lmdb_wrapper.cpp:1503) Headers DB has no block at height: 0
-ERROR - 10:59:34: (lmdb_wrapper.cpp:1483) No headers at height 0
-ERROR - 10:59:34: (BlockchainScanner.cpp:445) Missing file map for output scan, this is unexpected
-ERROR - 10:59:34: (BlockchainScanner.cpp:447) Has the following block files:
-ERROR - 10:59:34: (BlockchainScanner.cpp:451) Was looking for id #4294967295

It seems that you're running Bitcoin Core in "pruned" mode... Armory needs Bitcoin Core to have a full, unpruned copy of the blockchain... do you have 350+Gigs of free diskspace to be able to sync the full blockchain?

[jojo69]

well god dammit, lol

This whole exercise was precipitated by trying to NOT re-download the entire chain, but alas, it appears you are correct and while I do have ample space, it appears that the folder is only 6 gigs in size...

ok then...see you all in a few days after I get that sorted

[PhoenixZephyrus]

well god dammit, lol

This whole exercise was precipitated by trying to NOT re-download the entire chain, but alas, it appears you are correct and while I do have ample space, it appears that the folder is only 6 gigs in size...

ok then...see you all in a few days after I get that sorted

I did the same exact mistake...pruned it and downloaded ~85% of the blockchain and then read on here that pruned mode wont work, so had to redownload the whole thing again for 12 hours. Even after that it was stuck on a few steps, like "Scanning Transaction history" and "Parsing Tx Hashes". So, if you run into further troubles, here's a list of things you should ensure (just so that you dont have to troubleshoot for hours like I did lol):

1. Check off the auto bitcoind in armory and run Bitcoin Core yourself (and just make sure you're on latest version, I think the latest version is 0.96.5 on Github)
2. Make sure you dont run Chrome in the background with a lot of stuff open, that tends to eat up a huge chunk of RAM and Armory used up ~14GB/16GB for me.
3. Building database seems to take quite a lot of time, even on my Ryzen 7 4800H 8c/16t CPU it took ~8 mins with an all core turbo of 4.17 Ghz for the whole almost (though I was running some other tasks in the background)
4. If nothing seems to work even after making sure of these, rebuild the database. That seemed to fix everything for me.