Starting preliminary 0.94 testing - "Headless fullnode"

[Carlton Banks]

On the list of transactions, clicking on the header fields of the table (saying "Date", Wallet", "Comments", etc.), I can see an arrow being moved, switched up/down, moved to the column I clicked, etc, but the displayed transactions are not re-ordered accordingly.

In fact, they don't change at all.

currently feature-not-bug.

[fenwick]

Blockchain not scanned up to the top!

So, I have compiled Armory from github, using the ffreeze branch.
(My exact revision is 65dafec4ad888219abaa8897929364423f71ea05 )

I could successfully sync up my DB, and I also did two successful transactions.
I know they were successful, because they show up (with plenty of confirmations) on blockchain.info.

But then, after restarting Armory, the transactions don't show up.

Here is the full story:

 - I did a successful transaction, which was immediately confirmed a few times.
 - However, I noticed that armory still showed 0 confirmations.
 - I did another transaction, and Armory said that this transaction might have
   been refused by the network. I want to blockchain.info to check, and
   I saw that the transaction waiting there with 0 confirmations.
   A few blocks have gone by, and it wasn't confirmed, so I suspected that
   maybe I should have set higher fees, and so I wanted to "undo" this
   transactions ... so I told Armory (in the help menu) to
   "Clear All Unconfirmed".
 - Meanwhile, I see that eventually, my transaction _were_ confirmed OK
 - However, after restarting armory, these transactions don't show up at all.
   Neither the first one, nor the second one. Likewise, the displayed
   balance is the one before the transactions. According to armory, it's
   as if those transactions have never happened.
 - Also, the label in the bottom right corner says "Connected (353000 blocks)",
   whereas according to the .bitcoin/debug.log, the current block number is
   371598. Ha Armory forgot to scan the last 20000 blocks?

That is going on here?

After restarting Armory, and waiting for a long, long time again, the same situation returned.
(Scanning finished at block 353055)

This is the end of of the output:

-INFO  - 1440596237: (BlockUtils.cpp:395) reading blocks from file 322
-INFO  - 1440596239: (BlockUtils.cpp:395) reading blocks from file 323
-INFO  - 1440596241: (BlockUtils.cpp:395) reading blocks from file 324
-INFO  - 1440596243: (BlockUtils.cpp:395) reading blocks from file 325
-INFO  - 1440596244: (BlockUtils.cpp:1418) Wrote blocks to DB in 24.3227s
-WARN  - 1440596245: (BlockUtils.cpp:1114) Scanning from 353055 to 353055
-INFO  - 1440596247: (BlockUtils.cpp:1458) checking scan integrity
-INFO  - 1440596247: (BlockUtils.cpp:1646) --- bwbDtor: 0s
-INFO  - 1440596247: (BlockUtils.cpp:1647) Scanned Block range in 0.300441s
-INFO  - 1440596247: (BlockUtils.cpp:1653) Finished loading at file 325, offset 73062124
-INFO  - 1440596247: (BlockDataViewer.cpp:157) Enabling zero-conf tracking
-DEBUG - 1440596422: (Blockchain.cpp:214) Organizing chain

And indeed, it says the status is "Connected (353055 blocks)", ignoring the last ~20k blocks in the network.
(While my Satoshi's log says:  height=371612 )

[goatpig]

You are missing some block data starting 353055, which is why Armory can't scan any futher.

In your blocks folder, you should delete the last few blk files, probably up to blk00320.dat (315 to be on safe side) and the associated rev files (rev00320.dat and so on). Then start BitcoinQt, it will tell you its DB is corrupt and rebuild it from scratch, download the last few missing files in the process.

Once that is done, start Armory, it should recover from that.

[fenwick]

You are missing some block data starting 353055, which is why Armory can't scan any futher.

In your blocks folder, you should delete the last few blk files, probably up to blk00320.dat (315 to be on safe side) and the associated rev files (rev00320.dat and so on). Then start BitcoinQt, it will tell you its DB is corrupt and rebuild it from scratch, download the last few missing files in the process.

Once that is done, start Armory, it should recover from that.

OK, I will do that ... but I find it really weird that neither Satoshi,
nor Armory detects the missing/corrupted block data.

(Or at least none of them complains.)

[Carlton Banks]

Suggestion (possibly devs already thought this):

Create a package that isn't "armory" for the testing releases (armory-testing?).

Rationale: the majority of users still use a monolithic OS; only one instance of a package can be installed on such a machine. There are alot of people out there who might benefit from using this upcoming test release instead of the main release, and the process would benefit from as wide a potential testing base as can be. A separate package would allow those who run release Armory to see whether 0.94, for instance, alleviates their issues or not, without having to contemplate uninstalling "armory-testing" and re-installing Armory main release.

[doug_armory]

Suggestion (possibly devs already thought this):

Create a package that isn't "armory" for the testing releases (armory-testing?).

Rationale: the majority of users still use a monolithic OS; only one instance of a package can be installed on such a machine. There are alot of people out there who might benefit from using this upcoming test release instead of the main release, and the process would benefit from as wide a potential testing base as can be. A separate package would allow those who run release Armory to see whether 0.94, for instance, alleviates their issues or not, without having to contemplate uninstalling "armory-testing" and re-installing Armory main release.

Interesting idea. Admittedly, I'm not entirely sold on the idea of a separate package. There's too great a chance that people will be lazy and leave two copies on their system, potentially leading to weirdness down the road. For example, if somebody had 0.93-testing and tried to go back to 0.92, the Armory DB would keep getting nuked and lead to constant rebuilds.

Unfortunately, I don't think "But only people who know what they're doing will install it!" is correct. It wouldn't be hard for us to miss posts on, say, Reddit, and end up with somebody suggesting people try a test version that fixes one problem but introduces another.

(Just FYI, Reddit was just an example. Some of us do check Reddit on occasion. We typically prefer this forum, though.)

That said, it's something to keep in mind moving forward. Pros and cons, as always.

[Carlton Banks]

Interesting idea. Admittedly, I'm not entirely sold on the idea of a separate package. There's too great a chance that people will be lazy and leave two copies on their system, potentially leading to weirdness down the road. For example, if somebody had 0.93-testing and tried to go back to 0.92, the Armory DB would keep getting nuked and lead to constant rebuilds.

Unfortunately, I don't think "But only people who know what they're doing will install it!" is correct. It wouldn't be hard for us to miss posts on, say, Reddit, and end up with somebody suggesting people try a test version that fixes one problem but introduces another.

Agreed. I'm looking at all these user complaints from 0.92-0.93 and trying to think of a way to wish them away without actual magic, but there is no good compromise (as the terminally pessimistic will tell you about everything  ). It'd be nice if more people were accustomed to using virtualisation, but I guess it's not that time yet. I'll probably be waiting impatiently for everyone to adopt some other new sophistication by then  

(Just FYI, Reddit was just an example. Some of us do check Reddit on occasion. We typically prefer this forum, though.)

Really? Slightly amazed. I guess the Armory sub is much quieter than /r/bitcoin

[trev]

What's the total size of your DB? The headers DB alone?

https://github.com/etotheipi/BitcoinArmory/blob/ffreeze/cppForSwig/mdb/mdb.c#L588

Reduce this value to 16MB and rebuild a new DB with that, see if it suffers from the bad_alloc.

I reduced MAX_MAPSIZE_INCEREMENT from 256MB to 16MB, re-compiled, and --rebuild.
The databases are now:
  20819968 2015-08-31 00:57 blocks
107724800 2015-08-31 00:57 headers
After a couple of days of uptime I got the std::bad_alloc crash.

I'm running an identical copy of bitcoin-qt/Armory on a newer desktop that has 16GB dram.
I got the std::bad_alloc crash on that machine too, just like the laptop.

I also got the std::alloc crash when sending to a "normal" wallet.

In one instance so far, I started Armory at 23:50 and sent BTC at 00:10 and got std::alloc.
Is there any code in Armory that uses "current time - previous time" and doesn't handle
a negative value?

[goatpig]

What's the total size of your DB? The headers DB alone?

https://github.com/etotheipi/BitcoinArmory/blob/ffreeze/cppForSwig/mdb/mdb.c#L588

Reduce this value to 16MB and rebuild a new DB with that, see if it suffers from the bad_alloc.

I reduced MAX_MAPSIZE_INCEREMENT from 256MB to 16MB, re-compiled, and --rebuild.
The databases are now:
  20819968 2015-08-31 00:57 blocks
107724800 2015-08-31 00:57 headers
After a couple of days of uptime I got the std::bad_alloc crash.

I'm running an identical copy of bitcoin-qt/Armory on a newer desktop that has 16GB dram.
I got the std::bad_alloc crash on that machine too, just like the laptop.

I also got the std::alloc crash when sending to a "normal" wallet.

In one instance so far, I started Armory at 23:50 and sent BTC at 00:10 and got std::alloc.
Is there any code in Armory that uses "current time - previous time" and doesn't handle
a negative value?

Comment out this line:

https://github.com/etotheipi/BitcoinArmory/blob/ffreeze/ArmoryQt.py#L2619

It will prevent the ZC container from parsing any new tx. This is the most likely culprit atm.

[trev]

Comment out this line:

https://github.com/etotheipi/BitcoinArmory/blob/ffreeze/ArmoryQt.py#L2619

It will prevent the ZC container from parsing any new tx. This is the most likely culprit atm.

Since the crash occurs intermittently, I commented out addNewZeroConfTx() in one system and
ran the second system unpatched. The patched system has not crashed so far, but the unpatched
system has crashed several times. So, it looks like the problem is in the "zero confirmations" code.
I will use the fix on both systems. Thanks for providing a good work-around patch.

[pop9119]

Hey Armory folks,

In my occasional view of both bitcoin core and armory's code, I noticed that a recent commit to armory's ffreeze branch that put it in a gitian branch. I'm curious if this means that we are getting close to getting pre-built test builds of 0.94?

[qertoip2]

The advertised branch ffreeze is missing: https://github.com/etotheipi/BitcoinArmory/tree/ffreeze

In the similar gitian-ffreeze branch make is broken (no 'Makefile', Makefile.old doesn't work)

[zombieslayer9099]

I am on Debian based Linux.

Code:

./autogen.sh
./configure
make

Code:

python ArmoryQt.py
  File "ArmoryQt.py", line 2
    exec python /usr/local/lib/armory/ArmoryQt.py "$@"
                                                     ^
SyntaxError: invalid syntax

Is there a fix for this?

[m.fridge]

They made Armory closed-source, which is why you don't find the latest tree on github anymore. There won't be any open releases. As far as I noticed, this step was done without any public communication.

[goatpig]

Is there any chance that the 0.94 version will be released soon ?

Late Jan/early Feb.

[Carlton Banks]

Is there any chance that the 0.94 version will be released soon ?

Late Jan/early Feb.

Counting down... 

[goatpig]

Counting down... 

oh boy, I'm in trouble now.

[T-rage_11]

Is there any chance that the 0.94 version will be released soon ?

Late Jan/early Feb.

But we can't trust it anymore because it is closed source ...

or am I wrong?

[goatpig]

Is there any chance that the 0.94 version will be released soon ?

Late Jan/early Feb.

But we can't trust it anymore because it is closed source ...

or am I wrong?

That's an ETA for a fully open source release.

[Deth]

Is there any chance that the 0.94 version will be released soon ?

Late Jan/early Feb.

But we can't trust it anymore because it is closed source ...

or am I wrong?

That's an ETA for a fully open source release.

It is February 1st, got any progress on this? ~120G "waste of space" is becoming real annoying in my 256G laptop's SSD, while I have tons of HDD space on my servers.