Starting preliminary 0.94 testing - "Headless fullnode"

[Carlton Banks]

Just got to the point with re-building supernode Db where the scanning begins. I think the threading sync is a little wrong for my platform somehow, range of CPU usage is 5-15%, and I state it that way because it's very bursty. Still getting long estimates for a complete scan (1.5 days currently).

[1715134584]

1715134584

[1715134584]

1715134584

[1715134584]

1715134584

[1715134584]

1715134584

[goatpig]

What's your disk bandwidth at?

[Carlton Banks]

Nothing installed that measures disk bandwidth, unless there's something in standard Gnome Debian that does. Using an SSD for the databases, not more than 2/3rds full. The drive responds as it ever has to other workloads.

[qertoip2]

PLEASE checkout the ffreeze branch and help test!  Once we get a little bit more testing we'll do a semi-official testing release with a proper bug bounty!

After successful sync and several successful restarts, I run into the following error. This is happening with the Bitcoin Core 0.11 fully synced and running fine in the background.

Code:

➜  git log
commit 083fc5b (HEAD -> ffreeze, origin/ffreeze)
Author: Alan Reiner <alan@bitcoinarmory.com>
Date:   Fri Aug 7 19:34:08 2015 -0400

    Added download script to get offline bundle deps

Code:

➜  uname -a
Linux qertoip 3.13.0-37-generic #64-Ubuntu SMP Mon Sep 22 21:28:38 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

Code:

➜  ~ python /ArmoryQt.py
(ERROR) ArmoryQt.py:1329 - 4 attempts to load blockchain failed.  Remove mempool.bin.
(ERROR) ArmoryQt.py:1334 - File mempool.bin does not exist. Nothing deleted.
-INFO  - 1439063351: (BlockUtils.cpp:873) blkfile dir: /home/qertoip/.bitcoin/blocks
-INFO  - 1439063351: (BlockUtils.cpp:874) lmdb dir: /home/qertoip/.armory/databases
-INFO  - 1439063351: (lmdb_wrapper.cpp:446) Opening databases...
-INFO  - 1439063351: (BlockUtils.cpp:1230) Executing: doInitialSyncOnLoad
"sni-qt/7281" WARN  21:49:11.804 void StatusNotifierItemFactory::connectToSnw() Invalid interface to SNW_SERVICE 
-INFO  - 1439063351: (BlockUtils.cpp:1314) Total number of blk*.dat files: 317
-INFO  - 1439063351: (BlockUtils.cpp:1315) Total blockchain bytes: 42,434,701,647
-INFO  - 1439063351: (BlockUtils.cpp:1823) Reading headers from db
(ERROR) announcefetch.py:312 - Could not verify data in signed message block
Traceback (most recent call last):
  File "/home/qertoip/Projects/BitcoinArmory/announcefetch.py", line 304, in __runFetchSequence
    sig, msg = readSigBlock(digestData)
  File "/home/qertoip/Projects/BitcoinArmory/jasvet.py", line 589, in readSigBlock
    name = r.split(BEGIN_MARKER)[1].split(DASHX5)[0]
IndexError: list index out of range
-INFO  - 1439063355: (BlockUtils.cpp:1849) Found 369016 headers in db
-DEBUG - 1439063355: (Blockchain.cpp:214) Organizing chain w/ rebuild
-WARN  - 1439063356: (BlockUtils.cpp:1343) --- Fetching SSH summaries for 346 registered addresses
-INFO  - 1439063356: (BlockUtils.cpp:1356) Left off at file 316, offset 71970929
-INFO  - 1439063356: (BlockUtils.cpp:1359) Reading headers and building chain...
-INFO  - 1439063356: (BlockUtils.cpp:1360) Starting at block file 316 offset 71970929
-INFO  - 1439063356: (BlockUtils.cpp:1362) Block height 368994
-INFO  - 1439063356: (BlockUtils.cpp:345) parsing headers in file 316
-DEBUG - 1439063357: (Blockchain.cpp:214) Organizing chain w/ rebuild
-INFO  - 1439063358: (BlockUtils.cpp:1399) Looking for first unrecognized block
-INFO  - 1439063358: (BlockUtils.cpp:1403) Updating Headers in DB
-INFO  - 1439063358: (BlockUtils.cpp:1677) Loading block data... file 316 offset 71970929
-INFO  - 1439063358: (BlockUtils.cpp:395) reading blocks from file 316
-INFO  - 1439063358: (BlockUtils.cpp:1417) Wrote blocks to DB in 0.102265s
-WARN  - 1439063358: (BlockUtils.cpp:1113) Scanning from 368991 to 368996
-ERROR - 1439063358: (BlockWriteBatcher.cpp:359) Header heigh&dup is not in BLKDATA DB
-ERROR - 1439063358: (BlockWriteBatcher.cpp:359) Header heigh&dup is not in BLKDATA DB
-ERROR - 1439063358: (BlockWriteBatcher.cpp:360) (368994, 0)
-ERROR - 1439063358: (BlockWriteBatcher.cpp:360) (368992, 0)
-ERROR - 1439063358: (BlockWriteBatcher.cpp:359) Header heigh&dup is not in BLKDATA DB
-ERROR - 1439063358: (BlockWriteBatcher.cpp:360) (368993, 0)
-ERROR - 1439063358: (BlockWriteBatcher.cpp:359) Header heigh&dup is not in BLKDATA DB
-ERROR - 1439063358: (BlockWriteBatcher.cpp:360) (368991, 0)
-ERROR - 1439063358: (BlockWriteBatcher.cpp:2175) hit interruption marker from pull threads
-INFO  - 1439063358: (BlockUtils.cpp:1457) checking scan integrity
-WARN  - 1439063358: (BlockUtils.cpp:1462) Top scanned block does not match top block header
-WARN  - 1439063358: (BlockUtils.cpp:1525) Issue is benign, moving on
-ERROR - 1439063358: (BDM_mainthread.cpp:430) BDM thread failed: bad block meta value

(python:7281): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion 'value >= min && value <= max' failed

If you need anything else please let me know.

[goatpig]

Does it fix itself after a restart?

[qertoip2]

Does it fix itself after a restart?

This happens every time I run Armory. I could probably reset this to working by removing ~/.armory/databases and starting over.

In the GUI a popup shows "bad block meta value". Once OK-ed the Armory shuts down.

This is 100% reproducible. Perhaps my "databases" would be useful for you? I can put it online if you need.

[goatpig]

This happens every time I run Armory. I could probably reset this to working by removing ~/.armory/databases and starting over.

Yes, you will have to rebuild & rescan or delete the DB folder

Perhaps my "databases" would be useful for you? I can put it online if you need.

I would like that very much. If I can experience the issue first hand I can improve the DB repair code to catch that one too.

[qertoip2]

I would like that very much. If I can experience the issue first hand I can improve the DB repair code to catch that one too.

Armory 0.94 databases.zip with the issue:

https://mega.co.nz/#!ws0w3ZiK!T0wpFyeHV8yJ9f-BASgsHqsJ3g48IZp4pwUJfbN-AQo

Please find the ZIP password PM-ed. I've encrypted the ZIP just in case it has some privacy implications.

[goatpig]

Thanks, that will help a lot.

Please find the ZIP password PM-ed. I've encrypted the ZIP just in case it has some privacy implications.

It does, encrypting it is proper practice.

[goatpig]

Try the latest commit with your damaged DB.

[qertoip2]

Try the latest commit with your damaged DB.

It fixed the issue, thank you so much!

Although there was a minor glitch during the recovery.

On the first run I got another error about missing headers, a suggestion to do a factory reset, and a forced close. I choose to not do a factory reset.

On the second run it fully recovered (very quickly).

The logs:

Code:

git pull origin ffreeze
make clean
make

Code:

➜  BitcoinArmory git:(ffreeze) python ArmoryQt.py 
********************************************************************************
Loading Armory Engine:
   Armory Version:       0.93.99.1
   Armory Build:         a840b70352
   PyBtcWallet  Version: 1.35
Detected Operating system: Linux
   OS Variant            : ('LinuxMint', '17.1', 'rebecca')
   User home-directory   : /home/qertoip
   Satoshi BTC directory : /home/qertoip/.bitcoin/
   Armory home dir       : /home/qertoip/.armory/
   ArmoryDB directory     : /home/qertoip/.armory/databases
   Armory settings file  : /home/qertoip/.armory/ArmorySettings.txt
   Armory log file       : /home/qertoip/.armory/armorylog.txt
   Do wallet checking    : True
/home/qertoip/Projects/BitcoinArmory/armoryengine/Transaction.py:2675: SyntaxWarning: import * only allowed at module level
  def PyCreateAndSignTx_old(srcTxOuts, dstAddrsVals):
-INFO  - 1439193620: (BlockUtils.cpp:874) blkfile dir: /home/qertoip/.bitcoin/blocks
-INFO  - 1439193620: (BlockUtils.cpp:875) lmdb dir: /home/qertoip/.armory/databases
-INFO  - 1439193620: (lmdb_wrapper.cpp:446) Opening databases...
"sni-qt/19177" WARN  10:00:20.523 void StatusNotifierItemFactory::connectToSnw() Invalid interface to SNW_SERVICE 
-INFO  - 1439193620: (BlockUtils.cpp:1231) Executing: doInitialSyncOnLoad
-INFO  - 1439193620: (BlockUtils.cpp:1315) Total number of blk*.dat files: 318
-INFO  - 1439193620: (BlockUtils.cpp:1316) Total blockchain bytes: 42,500,991,204
-INFO  - 1439193620: (BlockUtils.cpp:1837) Reading headers from db
(ERROR) announcefetch.py:312 - Could not verify data in signed message block
Traceback (most recent call last):
  File "/home/qertoip/Projects/BitcoinArmory/announcefetch.py", line 304, in __runFetchSequence
    sig, msg = readSigBlock(digestData)
  File "/home/qertoip/Projects/BitcoinArmory/jasvet.py", line 589, in readSigBlock
    name = r.split(BEGIN_MARKER)[1].split(DASHX5)[0]
IndexError: list index out of range
-INFO  - 1439193626: (BlockUtils.cpp:1863) Found 369156 headers in db
-DEBUG - 1439193626: (Blockchain.cpp:214) Organizing chain w/ rebuild
-WARN  - 1439193627: (BlockUtils.cpp:1344) --- Fetching SSH summaries for 346 registered addresses
-INFO  - 1439193628: (BlockUtils.cpp:1357) Left off at file 316, offset 126056295
-INFO  - 1439193628: (BlockUtils.cpp:1360) Reading headers and building chain...
-INFO  - 1439193628: (BlockUtils.cpp:1361) Starting at block file 316 offset 126056295
-INFO  - 1439193628: (BlockUtils.cpp:1363) Block height 369133
-INFO  - 1439193628: (BlockUtils.cpp:345) parsing headers in file 316
-INFO  - 1439193628: (BlockUtils.cpp:345) parsing headers in file 317
-DEBUG - 1439193628: (Blockchain.cpp:214) Organizing chain w/ rebuild
-INFO  - 1439193629: (BlockUtils.cpp:1400) Looking for first unrecognized block
-INFO  - 1439193629: (BlockUtils.cpp:1404) Updating Headers in DB
-INFO  - 1439193629: (BlockUtils.cpp:1691) Loading block data... file 316 offset 126056295
-INFO  - 1439193629: (BlockUtils.cpp:395) reading blocks from file 316
-INFO  - 1439193629: (BlockUtils.cpp:395) reading blocks from file 317
-INFO  - 1439193629: (BlockUtils.cpp:1418) Wrote blocks to DB in 0.100643s
-WARN  - 1439193629: (BlockUtils.cpp:1114) Scanning from 368991 to 369213
-ERROR - 1439193629: (BlockWriteBatcher.cpp:359) Header heigh&dup is not in BLKDATA DB
-ERROR - 1439193629: (BlockWriteBatcher.cpp:359) Header heigh&dup is not in BLKDATA DB
-ERROR - 1439193629: (BlockWriteBatcher.cpp:360) (368993, 0)
-ERROR - 1439193629: (BlockWriteBatcher.cpp:359) Header heigh&dup is not in BLKDATA DB
-ERROR - 1439193629: (BlockWriteBatcher.cpp:360) (368991, 0)
-ERROR - 1439193629: (BlockWriteBatcher.cpp:360) (368992, 0)
-ERROR - 1439193629: (BlockWriteBatcher.cpp:359) Header heigh&dup is not in BLKDATA DB
-ERROR - 1439193629: (BlockWriteBatcher.cpp:360) (368994, 0)
-ERROR - 1439193629: (BlockWriteBatcher.cpp:2175) hit interruption marker from pull threads
-INFO  - 1439193629: (BlockUtils.cpp:1458) checking scan integrity
-WARN  - 1439193629: (BlockUtils.cpp:1463) Top scanned block does not match top block header
-WARN  - 1439193629: (BlockUtils.cpp:1567) Issue is significant, attempting repairs
-INFO  - 1439193629: (BlockUtils.cpp:1579) Checking dupIDs from block 368992 onward

(python:19177): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion 'value >= min && value <= max' failed
-ERROR - 1439193631: (lmdb_wrapper.cpp:1585) Block height exceeds DupID lookup table
-ERROR - 1439193631: (BlockUtils.cpp:1618) missing 1 block headers
-ERROR - 1439193631: (BDM_mainthread.cpp:430) BDM thread failed: Missing headers! This is unexpected, Armory will have to close. If the error persists, do a factory reset.

(python:19177): Gtk-CRITICAL **: IA__gtk_widget_get_direction: assertion 'GTK_IS_WIDGET (widget)' failed

(python:19177): Gtk-CRITICAL **: IA__gtk_widget_get_direction: assertion 'GTK_IS_WIDGET (widget)' failed


➜  BitcoinArmory git:(ffreeze) python ArmoryQt.py 
********************************************************************************
Loading Armory Engine:
   Armory Version:       0.93.99.1
   Armory Build:         a840b70352
   PyBtcWallet  Version: 1.35
Detected Operating system: Linux
   OS Variant            : ('LinuxMint', '17.1', 'rebecca')
   User home-directory   : /home/qertoip
   Satoshi BTC directory : /home/qertoip/.bitcoin/
   Armory home dir       : /home/qertoip/.armory/
   ArmoryDB directory     : /home/qertoip/.armory/databases
   Armory settings file  : /home/qertoip/.armory/ArmorySettings.txt
   Armory log file       : /home/qertoip/.armory/armorylog.txt
   Do wallet checking    : True
-INFO  - 1439193687: (BlockUtils.cpp:874) blkfile dir: /home/qertoip/.bitcoin/blocks
-INFO  - 1439193687: (BlockUtils.cpp:875) lmdb dir: /home/qertoip/.armory/databases
-INFO  - 1439193687: (lmdb_wrapper.cpp:446) Opening databases...
-INFO  - 1439193687: (BlockUtils.cpp:1231) Executing: doInitialSyncOnLoad
"sni-qt/19238" WARN  10:01:27.265 void StatusNotifierItemFactory::connectToSnw() Invalid interface to SNW_SERVICE 
-INFO  - 1439193687: (BlockUtils.cpp:1315) Total number of blk*.dat files: 318
-INFO  - 1439193687: (BlockUtils.cpp:1316) Total blockchain bytes: 42,500,991,204
-INFO  - 1439193687: (BlockUtils.cpp:1837) Reading headers from db
(ERROR) announcefetch.py:312 - Could not verify data in signed message block
Traceback (most recent call last):
  File "/home/qertoip/Projects/BitcoinArmory/announcefetch.py", line 304, in __runFetchSequence
    sig, msg = readSigBlock(digestData)
  File "/home/qertoip/Projects/BitcoinArmory/jasvet.py", line 589, in readSigBlock
    name = r.split(BEGIN_MARKER)[1].split(DASHX5)[0]
IndexError: list index out of range
-INFO  - 1439193689: (BlockUtils.cpp:1863) Found 369235 headers in db
-DEBUG - 1439193689: (Blockchain.cpp:214) Organizing chain w/ rebuild
-WARN  - 1439193691: (BlockUtils.cpp:1344) --- Fetching SSH summaries for 346 registered addresses
-INFO  - 1439193691: (BlockUtils.cpp:1357) Left off at file 317, offset 15768567
-INFO  - 1439193691: (BlockUtils.cpp:1360) Reading headers and building chain...
-INFO  - 1439193691: (BlockUtils.cpp:1361) Starting at block file 317 offset 15768567
-INFO  - 1439193691: (BlockUtils.cpp:1363) Block height 369213
-INFO  - 1439193691: (BlockUtils.cpp:345) parsing headers in file 317
-DEBUG - 1439193691: (Blockchain.cpp:214) Organizing chain w/ rebuild
-INFO  - 1439193692: (BlockUtils.cpp:1400) Looking for first unrecognized block
-INFO  - 1439193692: (BlockUtils.cpp:1404) Updating Headers in DB
-INFO  - 1439193692: (BlockUtils.cpp:1691) Loading block data... file 317 offset 15768567
-INFO  - 1439193692: (BlockUtils.cpp:395) reading blocks from file 317
-INFO  - 1439193692: (BlockUtils.cpp:1418) Wrote blocks to DB in 0.000427s
-WARN  - 1439193692: (BlockUtils.cpp:1114) Scanning from 368991 to 369213
-ERROR - 1439193692: (BlockWriteBatcher.cpp:359) Header heigh&dup is not in BLKDATA DB
-ERROR - 1439193692: (BlockWriteBatcher.cpp:360) (368992, 0)
-ERROR - 1439193692: (BlockWriteBatcher.cpp:359) Header heigh&dup is not in BLKDATA DB
-ERROR - 1439193692: (BlockWriteBatcher.cpp:360) (368993, 0)
-ERROR - 1439193692: (BlockWriteBatcher.cpp:359) Header heigh&dup is not in BLKDATA DB
-ERROR - 1439193692: (BlockWriteBatcher.cpp:360) (368994, 0)
-ERROR - 1439193692: (BlockWriteBatcher.cpp:359) Header heigh&dup is not in BLKDATA DB
-ERROR - 1439193692: (BlockWriteBatcher.cpp:360) (368991, 0)
-ERROR - 1439193692: (BlockWriteBatcher.cpp:2175) hit interruption marker from pull threads
-INFO  - 1439193692: (BlockUtils.cpp:1458) checking scan integrity
-WARN  - 1439193692: (BlockUtils.cpp:1463) Top scanned block does not match top block header
-WARN  - 1439193692: (BlockUtils.cpp:1567) Issue is significant, attempting repairs
-INFO  - 1439193692: (BlockUtils.cpp:1579) Checking dupIDs from block 368992 onward

(python:19238): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion 'value >= min && value <= max' failed
-ERROR - 1439193694: (BlockUtils.cpp:1626) Missing block data, attempting to repair the DB
-INFO  - 1439193694: (BlockUtils.cpp:421) reading block file 317
-INFO  - 1439193694: (BlockUtils.cpp:421) reading block file 316
-INFO  - 1439193698: (BlockUtils.cpp:2298) BLKDATA DB was repaired successfully
-WARN  - 1439193698: (BlockUtils.cpp:1114) Scanning from 368992 to 369213
-INFO  - 1439193699: (BlockUtils.cpp:1458) checking scan integrity
-INFO  - 1439193699: (BlockUtils.cpp:1646) --- bwbDtor: 0s
-INFO  - 1439193699: (BlockUtils.cpp:1647) Scanned Block range in 9.09197s
-INFO  - 1439193699: (BlockUtils.cpp:1653) Finished loading at file 317, offset 15768567
-INFO  - 1439193699: (BlockDataViewer.cpp:157) Enabling zero-conf tracking 

[goatpig]

On the first run I got another error about missing headers, a suggestion to do a factory reset, and a forced close. I choose to not do a factory reset

This error marker is very aggressive, you shouldn't do a factory reset unless you run into it a few times in a row. I should do something about it really, at least not let it trigger on missing headers only.

[m.fridge]

Hi,

my previously reported bug (post #156 in this thread) has been fixed. Thanks for that! However, I've got new issues here:

When I first import a wallet before the database is built, then armory seems to stuck in an endless loop when beginning to scan transactions. The program consumes CPU and has high I/O, but it keeps staying at 4% scanning progress (even when running for one night).

When I first build the database and import the wallet afterwards, armory crashes when it comes to transaction scanning. Here is backtrace of such a crash:

Code:

[Thread 0xabcf0b40 (LWP 15657) exited]
[New Thread 0xabcf0b40 (LWP 15658)]
[Thread 0xabcf0b40 (LWP 15658) exited]
[Thread 0xab2ffb40 (LWP 15283) exited]
[New Thread 0xab2ffb40 (LWP 15659)]
terminate called after throwing an instance of 'LMDBException'
  what():  Failed to insert: need transaction

Program received signal SIGABRT, Aborted.
[Switching to Thread 0xab2ffb40 (LWP 15659)]
0xb7fdda0c in ?? ()
(gdb) backtrace
#0  0xb7fdda0c in ?? ()
#1  0x65637845 in ?? ()
(gdb)

[goatpig]

I fixed this issue in the last commit a couple days ago. Pull again and give it a spin.

[Searinox]

On the first run I got another error about missing headers, a suggestion to do a factory reset, and a forced close. I choose to not do a factory reset

This error marker is very aggressive, you shouldn't do a factory reset unless you run into it a few times in a row. I should do something about it really, at least not let it trigger on missing headers only.

Although I am running on 0.93.2 I can report as much. My desktop has been running Armory for more than a year now and I've had no issues or crashes at all. Zero. My laptop however, is a different issue. Whenever I wake it up from days of inactivity, Armory quickly throws a "missing headers" error and quits, only to return with full functionality on next launch. At first I didn't know if it was an issue with sleep or just it not being able to keep up with bitcoind but today for the first time it happened on the desktop.

We had a power failure yesterday and after restoration there was no internet until just about now. During all this time Armory had been running uninterrupted. When the internet came back, bitcoind began catching up with a day's worth of blocks, and Armory threw the "missing headers" error.

Could you please investigate if there are issues with Armory keeping up with bitcoind when it's behind by say 2-3 days and suddenly it gets a chance to catch up?

I am re-reporting this in the ongoing bigger Armory thread to keep a 0.93 error off the 0.94 post but since you are investigating this now, I figured I'd mention it here as well.

[goatpig]

Although I am running on 0.93.2 I can report as much. My desktop has been running Armory for more than a year now and I've had no issues or crashes at all. Zero. My laptop however, is a different issue. Whenever I wake it up from days of inactivity, Armory quickly throws a "missing headers" error and quits, only to return with full functionality on next launch. At first I didn't know if it was an issue with sleep or just it not being able to keep up with bitcoind but today for the first time it happened on the desktop.

We had a power failure yesterday and after restoration there was no internet until just about now. During all this time Armory had been running uninterrupted. When the internet came back, bitcoind began catching up with a day's worth of blocks, and Armory threw the "missing headers" error.

Could you please investigate if there are issues with Armory keeping up with bitcoind when it's behind by say 2-3 days and suddenly it gets a chance to catch up?

I am re-reporting this in the ongoing bigger Armory thread to keep a 0.93 error off the 0.94 post but since you are investigating this now, I figured I'd mention it here as well.

That's a 0.93 specific issue that has been fixed in 0.94

0.93 can't tell the difference between a mangled block and a block in the process of being written to disk (which appears mangled when Armory reads it). 0.94 side steps the issue by rereading partial blocks at each iteration until the partials either fill up or a valid block further in the file extends the chain.

The true solution would be to listen on the new block P2P notification and grab data over the socket, but that's a whole new effort.

[Searinox]

Yet another reason to anticipate 0.94. If I had the knowledge, I'd compile the source for Windows myself.

[m.fridge]

I fixed this issue in the last commit a couple days ago. Pull again and give it a spin.

The problem still exists with the latest version. Here is the log, where the crash seems to have another cause than before. I also got a popup saying "Cannot find block with hash".

Code:

-ERROR - 1439730764: (BlockWriteBatcher.cpp:711) error grabbing block 0|0 in file #0, offset: 8, with a size of 285 bytes
-ERROR - 1439730764: (BlockWriteBatcher.cpp:715) error: std::bad_alloc
-ERROR - 1439730764: (BlockWriteBatcher.cpp:403) No block in DB at height 0
[Thread 0xaad70b40 (LWP 24086) exited]
[New Thread 0x93e6eb40 (LWP 24087)]
-ERROR - 1439730764: (BlockWriteBatcher.cpp:711) error grabbing block 1|0 in file #0, offset: 301, with a size of 215 bytes
-ERROR - 1439730764: (BlockWriteBatcher.cpp:715) error: std::bad_alloc
-ERROR - 1439730764: (BlockWriteBatcher.cpp:403) No block in DB at height 1
[Thread 0x93e6eb40 (LWP 24087) exited]
[Thread 0x94ff0b40 (LWP 24084) exited]
[New Thread 0x94ff0b40 (LWP 24088)]
-ERROR - 1439730764: (BlockWriteBatcher.cpp:2175) hit interruption marker from pull threads
[New Thread 0x934ffb40 (LWP 24089)]
[Thread 0x934ffb40 (LWP 24089) exited]
[Thread 0x94ff0b40 (LWP 24088) exited]
[Thread 0x959ffb40 (LWP 24085) exited]
-WARN  - 1439730764: (BlockWriteBatcher.cpp:602) --- feedSleep: 0 s
-WARN  - 1439730764: (BlockWriteBatcher.cpp:605) --- workers: 0 s
-WARN  - 1439730764: (BlockWriteBatcher.cpp:608) --- writeStxo: 0.000131 s
-WARN  - 1439730764: (BlockWriteBatcher.cpp:610) --- writeStxo_grabMutex: 0 s
-WARN  - 1439730764: (BlockWriteBatcher.cpp:613) --- waitingOnSerThread: 0.004589 s
-WARN  - 1439730764: (BlockWriteBatcher.cpp:615) --- waitForDataToWrite: 0.872389 s
-WARN  - 1439730764: (BlockWriteBatcher.cpp:619) --- checkForCollisions: 0 s
-WARN  - 1439730764: (BlockWriteBatcher.cpp:621) --- getExistingKeys: 0.023769 s
-WARN  - 1439730764: (BlockWriteBatcher.cpp:623) --- getNewKeys: 0 s
-WARN  - 1439730764: (BlockWriteBatcher.cpp:625) --- getSSHHeadersLock: 1.7e-05 s
-WARN  - 1439730764: (BlockWriteBatcher.cpp:627) --- computeDBKeys: 0 s
-WARN  - 1439730764: (BlockWriteBatcher.cpp:629) --- getSshHeaders: 5.6e-05 s
-WARN  - 1439730764: (BlockWriteBatcher.cpp:631) --- finalizeSerialization: 0.004755 s
-WARN  - 1439730764: (BlockWriteBatcher.cpp:636) --- serializeBatch: 0.007962 s
-WARN  - 1439730764: (BlockWriteBatcher.cpp:638) --- serializeSubSsh: 3.6e-05 s
-WARN  - 1439730764: (BlockWriteBatcher.cpp:640) --- waitOnSSHser: 0.004153 s
-WARN  - 1439730764: (BlockWriteBatcher.cpp:644) --- waitOnWriteThread: 1.9e-05 s
-WARN  - 1439730764: (BlockWriteBatcher.cpp:647) --- putSSH: 0.000573 s
-WARN  - 1439730764: (BlockWriteBatcher.cpp:649) --- putSTX: 0.000375 s
-WARN  - 1439730764: (BlockWriteBatcher.cpp:652) --- getnextfeed: 0.814203 s
-WARN  - 1439730764: (BlockWriteBatcher.cpp:654) --- inControlThread: 1.09172 s
-WARN  - 1439730764: (BlockWriteBatcher.cpp:658) --- cleanup: 0 s
-WARN  - 1439730764: (BlockWriteBatcher.cpp:660) --- finishCleanup: 0 s
-INFO  - 1439730764: (BDM_supportClasses.cpp:259) Done with side scan of wallet <wallet ID>
[Thread 0x947efb40 (LWP 24081) exited]
[Thread 0xacabfb40 (LWP 24083) exited]
-ERROR - 1439730764: (BDM_mainthread.cpp:430) BDM thread failed: Cannot find block with hash 
[Thread 0xabe84b40 (LWP 20427) exited]
[New Thread 0xacabfb40 (LWP 24090)]
[Thread 0xacabfb40 (LWP 24090) exited]
[New Thread 0xacabfb40 (LWP 24091)]
[Thread 0xacabfb40 (LWP 24091) exited]
[New Thread 0xacabfb40 (LWP 24092)]
[Thread 0xacabfb40 (LWP 24092) exited]
[New Thread 0xacabfb40 (LWP 24093)]
[Thread 0xacabfb40 (LWP 24093) exited]
[New Thread 0xacabfb40 (LWP 24094)]
[Thread 0xacabfb40 (LWP 24094) exited]
[New Thread 0xacabfb40 (LWP 24095)]
[Thread 0xacabfb40 (LWP 24095) exited]
[New Thread 0xacabfb40 (LWP 24096)]
[Thread 0xacabfb40 (LWP 24096) exited]
[New Thread 0xacabfb40 (LWP 24097)]
[Thread 0xacabfb40 (LWP 24097) exited]
[New Thread 0xacabfb40 (LWP 24098)]
[Thread 0xacabfb40 (LWP 24098) exited]
[New Thread 0xacabfb40 (LWP 24099)]
[Thread 0xacabfb40 (LWP 24099) exited]
[Thread 0xb7c0a940 (LWP 20123) exited]
[Inferior 1 (process 20123) exited normally]
(gdb) backtrace
No stack.
(gdb)

[goatpig]

This looks severe. Do a Help -> Rescan and see if it you get the same output.

[m.fridge]

I did a rebuild and rescan and now end up with "std::bad_alloc". I thought that my harddrive might have problems and used a usb drive for the armory datadir, where I got the same error. Any ideas what's wrong here?

Code:

[New Thread 0xabe84b40 (LWP 24368)]
[Thread 0xabe84b40 (LWP 24368) exited]
[New Thread 0xabe84b40 (LWP 24369)]
[Thread 0xabe84b40 (LWP 24369) exited]
-INFO  - 1439844030: (BlockUtils.cpp:1691) Loading block data... file 0 offset 0
[New Thread 0xabe84b40 (LWP 24370)]
-INFO  - 1439844030: (BlockUtils.cpp:395) reading blocks from file 0
[Thread 0xabe84b40 (LWP 24370) exited]
-ERROR - 1439844030: (BDM_mainthread.cpp:430) BDM thread failed: std::bad_alloc

"std::bad_alloc" is shown as a popup. After okay-ing it, armory exits normally according to gdb.