Using Armory on the BCH chain

[TierNolan]

Once it writes to a blk file, it never checks files lower than that anymore.  That is a RAM variable though, so it gets set back to zero each time a node is restarted.

Why would it need to check for previous files appends on new blocks? If the issue only happens on fresh node start (when free space in older block files is eligible), as long as Armory sees that one new block, it will work.

I am not sure if they fully thought out the logic.  I may have missed something when looking at the code on core that protects against the issue.

I think it is a pretty bad policy no matter what.  It's not like keeping all the blk files at the same size actually matters, and keeping nearby blocks in the same file is likely a help.

They want a way to fill in spaces in the old block files but once a file overflows since last restart, they just don't bother with that one anymore.

The sequence is

- on startup set nLastBlockFile to 0
- when a new block arrives scan for a blk file with space starting at index = nLastBlockFile
- Once it jumps to a new file, it sets nLastBlockFile to that value

This means that nLastBlockFile never decreases, so once the node is running for a while it only writes to the latest blk file.

On the next reboot, different spaces could end up being filled.

Armory does not expect blocks to appear in order in files. It will check block data on disk from the last known block to have extended the chain. Therefor if you have up to blk01000.dat and the last top is in blk00997.dat, it will check #977 to #1000 on every new block notifications until it finds a block that extends the chain again.

Cool, so re-scanning problematic blk files wouldn't be an issue.  The only question is detecting them. 

It looks like c++ doesn't have a portable way to get the last modified time.

The assumption that later numbered blk files contain later data is violated by the code.  In that situation, blk #800 could be updated and no amount of scanning after 977 will fix things.

The ultimate solution would be blocks over P2P, but idk what kind of mess BCH will be by the time I'm done with that. Chances are Armory won't be compatible anymore at that point.

I don't think it is only restricted to Bitcoin Cash.  The problem is much more likely to be exposed by the variable block sizes, but I think it is possible even if they are mostly stable blocks.  You could get a 950kB space that doesn't get filled for a while and then someone is unlucky that they start their node just before a block that can fill that space arrives.

It might actually "settle down" once it has synced and then been restarted a few times.  This would "fill in" many of the spaces.  As long as the first block found after a restart is bigger than any of the spaces, then it will write to the file with the highest number and that jumps nLastBlockFile to the max value.

I think the workaround of wiping the armory database is probably acceptable, since the problem is rare and doesn't seem to affect the main chain.

[1715662382]

1715662382

[goatpig]

Blocks over P2P would actually fix this, because it would get the longest chain from your node, then grab block data by hash from that chain of headers, all over the P2P socket, completely ignoring the on disk file structure. I initially wanted this change for security reasons: Core needs to be exposed to the WAN but Armory does not need the WAN unless you want to connect to the DB remotely, and even then you could tunnel to it or put it behind a daemon. Limiting ArmoryDB's interactions with Core to just a couple sockets (P2P and RPC) would make way for more secure setups. Dodging this edge case is an added bonus now.

[TierNolan]

I initially wanted this change for security reasons: Core needs to be exposed to the WAN but Armory does not need the WAN

Seems reasonable.  Even if Armory accidentally connected to a remote node and received a false block hash, the local node's filesystem wouldn't have that block.  In that case, Armory stalls rather than trusting the block data from the remote host.

[TierNolan]

Hah, you learn something new everyday.

In c++, variables can be changed by passing them to functions, thanks to the reference notation.

I checked the validation.cpp file and could only see one place where there is a change to the variable.

It is also changed using this code.

Code:

pblocktree->ReadLastBlockFile(nLastBlockFile);

ReadLastBlockFile takes int& nFile as its input.

I thought references like that were inherently constants.

I think this effect is due to the changes to blockstorage.cpp (including some refactoring away from validation.cpp).  The call to ReadLastBlockFile doesn't change the nLastBlockFile global variable.  It just targets a local variable in the function.

This means the global variable stays at its default of zero.

This is what allows the node to write to previously completed blk*.dat files rather than only looking at new blk files.

In the Bitcoin Core client, the call to ReadLastBlockFile has a side effect that initializes the nLastBlockFile variable.

This is a Bitcoin Unlimited only behavior, I think, since Bitcoin Core and Bitcoin ABC haven't made the changes.

[goatpig]

Hah, you learn something new everyday.

In c++, variables can be changed by passing them to functions, thanks to the reference notation.

This is something you need to pay attention to when reading the method declaration. const& is used to avoid copies when passing arguments, and means the arg is an "in". Plain & implies the method is supposed to modify this reference, in which case it's considered an "out" argument.

This is common practice when you want to use the return value of the method as a status report. Typically your return an int, with 0 for success and negative values for error codes. You can also use this approach if the method has to provide several results, but that style is mostly for C. In C++ you tend to report errors by throwing, and write a class/struct for return types that require several members.

As seen in the code you linked to, the style is actually confusing: the semantics do not outright suggest ReadLastBlockFile takes a reference, as it is not being checked for a status return. This kind of semantics suggest you are setting some internal value for object pblocktree instead. The following snippet would be a little easier to read:

Code:

int result = pblocktree->ReadLastBlockFile(nLastBlockFile)
if(result != 0)
{
   //error handling
}

//do something with nLastBlockFile

With that in mind, the change in BU could very well be a mistake that led to a benign change in behavior, assuming the author of the change did not bother to acquire a broad understanding of this code before modifying it. He possibly saw a global variable changed by a local object and figured that's bad style, correcting for style without caring for the substance.

[TierNolan]

As seen in the code you linked to, the style is actually confusing: the semantics do not outright suggest ReadLastBlockFile takes a reference, as it is not being checked for a status return. This kind of semantics suggest you are setting some internal value for object pblocktree instead. The following snippet would be a little easier to read:

Code:

int result = pblocktree->ReadLastBlockFile(nLastBlockFile)
if(result != 0)
{
   //error handling
}

//do something with nLastBlockFile

Right.  Even better would be to create a local variable to indicate things.

Code:

int storedLastBlockFile;
int result = pblocktree->ReadLastBlockFile(storedLastBlockFile)
if(result != 0)
{
   //error handling
}

nLastBlockFile = storedLastBlockFile;

He kind of did that, since the new variable is called loadedblockfile, it just isn't used to update the global variable.

With that in mind, the change in BU could very well be a mistake that led to a benign change in behavior, assuming the author of the change did not bother to acquire a broad understanding of this code before modifying it. He possibly saw a global variable changed by a local object and figured that's bad style, correcting for style without caring for the substance.

I agree that this was possibly overlooked.  Having a pointer to the last file doesn't really do much if it isn't persisted over restarts.

He may have decided that it was more efficient to fill in the free spaces.  This change would make pruning less efficient too.  Ideally, each block should have a set of files from approx the same time, so they can be deleted roughly in order.

I wonder if the BU node writes to blk00000.dat on each restart if pruning is enabled.

[goatpig]

This change would make pruning less efficient too.

And possibly reorg speed? The revXXXXX.dat files hold the snapshot for the utxo changeset per block. Is it in sync with block alignment in blk files?

[TierNolan]

And possibly reorg speed? The revXXXXX.dat files hold the snapshot for the utxo changeset per block. Is it in sync with block alignment in blk files?

Yeah, the block's undo data is put in the corresponding rev file.  Re-orging each block could require looking in a separate file for each block.

[TierNolan]

Bitcoin Unlimited has fixed the blk file bug in their latest release.

In theory, this means that they should be compatible again and not cause 45 minute startup due to having to rescan everything.

They added an "xversion" message at some point and it breaks things again (sigh).  They technically broke protocol backwards compatibility with this change.

f11092713 add missing DB_LAST_BLOCK write in WriteBatchSync (#1504) (Greg Griffith)

I have created a branch that maintains compatibility.  I mostly coded this for myself, but it shows what you need to change to get things compatible.  (Use at own risk etc.)

(You need 1.5.1.0 to get the fix for the blk files.)

There aren't that many changes needed

- adds a -bch command line for Armory that switches to Bitcoin Cash magic net code
- satoshi-port command line fix (I think this is already fixed with latest release)
- sends ping message immediately after an verack to indicate that Armory node doesn't support xversion

[trademonkey]

I have around 14.2 BCH in a lockbox in armory that I can't get to. I'm offering a 10 % reward to the person who helps me get them out. I have tried many hours using the information on this forum, but it hasn't been enough. Is there anyone willing to help me? preferably someone known on the forum. Thanks in advance.

Edit: I just realized it was at least a year ago that I tried last time. Has any of the updates solved the problem?

[Carlton Banks]

I have around 14.2 BCH in a lockbox in armory that I can't get to. I'm offering a 10 % reward to the person who helps me get them out. I have tried many hours using the information on this forum, but it hasn't been enough. Is there anyone willing to help me? preferably someone known on the forum. Thanks in advance.

Edit: I just realized it was at least a year ago that I tried last time. Has any of the updates solved the problem?

So I think you could use BCH client software from 2017 to do this still

1. Get a copy of that software (ABC 0.14)
2. Sync the BCH chain up to the block that ABC 0.14 got hard forked into ABC 0.15 (it will do this without you telling it to)
3. Open Armory 0.96.5 (which has a lockboxs fix). Sync it against the ABC chain you have
4. Use the wallet on 0.14 ABC to get a new address
5. Compose a transaction sending the BCH to that new address from step 4 (Armory won't work with anything more than 0.14, so this must be done)
6. Save that tx in Armory as a hex string
7. Get latest version of ABC
8. Sync it against the 0.14 blockchain
9. When it's caught up, send your hex string saved tx using bitcoin-cli sendrawtransaction or on the console in ABC gui
10. profit


You might have problems with step 2. It might get stuck syncing the first ever BCH block (their first hard fork basically didn't work, lol). Later versions of ABC might have this fixed, so you could solve it "simply" by switching to latest ABC at that point, then when you're sure it's now switched to the BCH fork, IMMEDIATELY (or at least very quickly) quit the ABC latest and switch back to ABC 0.14. Armory will not sign the transaction with any version after 0.14


You can do the above in a "proper" way rather than the "basic" way as outlined above, but as there's money involved, you might find the "proper" way more frustrating. The outcome will be identical in either case.

Also, I think Tier Nolan figured out a patch to latest Armory (0.96.5) that bypasses the above rigmarole, but that involves compiling his code. FWIW, he's known and of good standing on bitcointalk/in bitcoin generally

[trademonkey]

Thank you Carlton, I might have another go at it. I've tried some versions of syncing the blockchain up until some point, but I could not get that to work. Then there was something about a certain script-sig(?) to move the BCH if I remember correctly. I learned a lot, but still failed in the end so I gave up temporarily. I'll start checking out Tier Nolan's code.

I'm still open to letting someone else do it, if anybody is willing.

[ghrunda]

I want to move all off my BTC and BCH (in chunks if possible) from my wallet and I havn't any transaction since 2014. What is the easiest way to do this?
To not mess it up with my BTC, can I begin to make transaction(s) to move my BTC to another wallet or do I need to begin with BCH?

Since I will move to a new wallet without any need of bitcoin core, is it then possible to
1. move my BTC from current armory installation
2. remove bitcoin core, armory and the blockchain (or use another user).
3. install BCH node and armory again
4. restore my wallet from a backup
5. move coins and sign it with bitcoin cash signer

[droark]

I'm not sure if it's possible to access the coins anymore, at least via Armory. Bcash has had several hard forks, and has one scheduled for later this month, actually. You may have to export your keys and try another tool.

[ghrunda]

I'm not sure if it's possible to access the coins anymore, at least via Armory. Bcash has had several hard forks, and has one scheduled for later this month, actually. You may have to export your keys and try another tool.

Yes, I've read all the post and it's a good option for me anyway.

[goatpig]

This feature is long dead and I'll remove all BCH signer code in the next release to boot. Therefor, I'm going to unsticky this thread.

[aa7356]

I have around 14.2 BCH in a lockbox in armory that I can't get to. I'm offering a 10 % reward to the person who helps me get them out. I have tried many hours using the information on this forum, but it hasn't been enough. Is there anyone willing to help me? preferably someone known on the forum. Thanks in advance.

Edit: I just realized it was at least a year ago that I tried last time. Has any of the updates solved the problem?

So I think you could use BCH client software from 2017 to do this still

1. Get a copy of that software (ABC 0.14)
2. Sync the BCH chain up to the block that ABC 0.14 got hard forked into ABC 0.15 (it will do this without you telling it to)
3. Open Armory 0.96.5 (which has a lockboxs fix). Sync it against the ABC chain you have
4. Use the wallet on 0.14 ABC to get a new address
5. Compose a transaction sending the BCH to that new address from step 4 (Armory won't work with anything more than 0.14, so this must be done)
6. Save that tx in Armory as a hex string
7. Get latest version of ABC
8. Sync it against the 0.14 blockchain
9. When it's caught up, send your hex string saved tx using bitcoin-cli sendrawtransaction or on the console in ABC gui
10. profit


You might have problems with step 2. It might get stuck syncing the first ever BCH block (their first hard fork basically didn't work, lol). Later versions of ABC might have this fixed, so you could solve it "simply" by switching to latest ABC at that point, then when you're sure it's now switched to the BCH fork, IMMEDIATELY (or at least very quickly) quit the ABC latest and switch back to ABC 0.14. Armory will not sign the transaction with any version after 0.14


You can do the above in a "proper" way rather than the "basic" way as outlined above, but as there's money involved, you might find the "proper" way more frustrating. The outcome will be identical in either case.

Also, I think Tier Nolan figured out a patch to latest Armory (0.96.5) that bypasses the above rigmarole, but that involves compiling his code. FWIW, he's known and of good standing on bitcointalk/in bitcoin generally

Hi Carlton,

I'm trying to follow your script, I am at step 2 ( Sync the BCH chain up to the block that ABC 0.14 )
I have a copy of the binaries of the version bitcoin-0.14.6-x86_64-linux-gnu that I downloaded from

https://download.bitcoinabc.org/0.14.6/linux/

the sha256sum of the binaries files I got on my storage is

9f37b8ec36a37944b016bbbf07340adb8ba644abb897b2d2e0edeb99ccf709c0  bitcoin-0.14.6-x86_64-linux-gnu.tar.gz

the weird thing is that in one machine those files starts get in sync with other peers, but the same binary does not in another machine located in the same IP subnet.

it should be a problem for both copies of the same binary, any hint?

Thanks

;
The funny thing is, in one machine

[Carlton Banks]

the weird thing is that in one machine those files starts get in sync with other peers, but the same binary does not in another machine located in the same IP subnet.

it should be a problem for both copies of the same binary, any hint?

you only need 1 copy of the BCH blockchain post-fork#1, so just copy it to any machine you would like to use it on. Remember that there were bugs in the original BCH hardforking logic, it didn't always manifest, and so you're kinda lucky that it even worked on 1 machine at this late stage tbh. Finding peers to connect to from between BCH fork#1 and BCH fork #2 is probably like looking for a needle in a haystack at this juncture.

[overmyhead]

This feature is long dead and I'll remove all BCH signer code in the next release to boot. Therefor, I'm going to unsticky this thread.

I did a lot of reading trying to understand this to wind up reading this. Not that I had enough understanding to try and acquire my BCH anyway.

So, if I have an armory wallet with BTC from before the fork how do I get my BCH now? In my reading I also found out there was a fork for BTG is that something I can/should try to get as well?

[goatpig]

This feature is long dead and I'll remove all BCH signer code in the next release to boot. Therefor, I'm going to unsticky this thread.

I did a lot of reading trying to understand this to wind up reading this. Not that I had enough understanding to try and acquire my BCH anyway.

So, if I have an armory wallet with BTC from before the fork how do I get my BCH now? In my reading I also found out there was a fork for BTG is that something I can/should try to get as well?

Did you move these coins since the BCH fork? Depending on that, the solution will differ a bit.