Question on blockheight behaviour

[Herbert]

Hi,

i am trying to make bitcoinmonitor.net more automated failsafe. One thing that currently requires manual intervention by admin is downtime of the main application while the bitcoind is still running. In this scenario bitcoind will try to notify the app about new blocks. As the app is down, these notifications get lost.
Result: When the app is running again admin has to manually make sure that the missed blocks are collected and handled apropriately.

Idea to automate this process:
- App stores for each received block the hash and height.
- If a new block comes in app calculates delta between new block's height and last seen block's height.
- Normally the delta should be exactly 1, but in case blocks have been missed by the app the delta would be larger.
- If blocks are missed the app requests the missing blocks using getblockhash(height) and getblock(hash) rpc commands

So if the last received block has height 10 and new block of height 15 comes in the app will try to get additionally blocks of height 11,12,13 and 14.

I think this should work fine in the normal case, but i am not sure if this would be safe for blockchain reorg scenarios.
Could it happen that after a reorg the newest block has a height than the last seen one?
Or could it happen that after a re-org a new block is accepted at the same height of the last block?

Note that my bitcoind patch triggeres notification on new blocks only when they are accepted and part of the main chain (Triggering is done at the end of CBlock::AcceptBlock())

[1715267623]

1715267623

[1715267623]

1715267623

[1715267623]

1715267623

[1715267623]

1715267623

[Pieter Wuille]

CBlock::AcceptBlock() is called for each block that gets attached to the block tree on disk, i.e. every valid block that has a history leading back to the genesis block. It is also called for blocks that fail to connect afterwards (those with double spends in them) or stale blocks that just never make it into the best chain.

I think you're better off hooking in ConnectBlock() and maybe DisconnectBlock(). These get called for each block joining and leaving the current best chain.

[Herbert]

Well, bitcoind only notifies my app in case the accepted block is part of the current best chain:

Code:

if (hashBestChain == hash)
{
    monitorBlock(*this, pindexBest);
}

So I will not get notification of blocks that are not on the main chain. But looking at the code indeed it seems I might miss new blocks during reorg.

Some more background info:
Normally i take a new block notification just as a trigger to check the monitored transactions if they got more confirmations now. I only look at the contained transactions of a new block to catch cases where the initial, unconfirmed transaction was not received. This might happen if bitcoind or the main app was offline for any reason or the user set up an address to monitor only after the transaction came in.

So this leads back to the original question: To catch up in these offline cases, can i just take the current height, calculate delta with the last seen block's height and get height-delta to height-1 blocks from bitcoind to catch up? Or could there be cases when the blockheight is "jumpy", e.g. not linear?

[twobitcoins]

In theory, there may be a new best chain where the block height stays the same or decreases.  The best chain is defined as the chain with the greatest total work, not the most blocks, so it is possible for a chain to contain fewer blocks yet represent more work (the blocks had a higher difficulty).

In practice it is unlikely to occur on the main bitcoin network.  Perhaps if two miners simultaneously generate the final block in a difficulty period with different block timestamps and thus different difficulties for the next difficulty period, and then two miners who chose to build on different blocks simultaneously generate blocks once again, you could see a new best chain that is the same height as the previous best chain.  That would be quite rare though.

[randomproof]

Couldn't you add code to bitcoind to add all the data to database on it's own, without notifying your app.  I'm assuming your app is doing that when it gets notified, so why not cut out the middleman?  If that is not possible, because your app is using some kind of internal database, you could have bitcoind save the data into a flat file if it can't connect to the app and when your app comes back online it can look for that file and import from there automatically.

[Herbert]

Couldn't you add code to bitcoind to add all the data to database on it's own, without notifying your app.  I'm assuming your app is doing that when it gets notified, so why not cut out the middleman?  If that is not possible, because your app is using some kind of internal database, you could have bitcoind save the data into a flat file if it can't connect to the app and when your app comes back online it can look for that file and import from there automatically.

Hmm, that would work, but i want to keep my patchset to bitcoind as small as possible.
As this is anyway only for emergency cases of unplanned downtime I implemented the initial proposal depending on the blockheight. It seems very unlikely that i have a unplanned downtime of the webapp and at the same time a blockchain reorg. And IF that happens I can still clean that up manually.
Thank you all for your answers

[Gavin Andresen]

On restart, you should be able to ask bitcoind for the top-of-chain block (using getblockcount/getblockhash/getblock).

If it's not in your database, then add it.  Then look at the previous block hash, and if it's not in your database ask bitcoind for it (and so on).

To be extra safe, after you're all done ask bitcoind for the top-of-chain block again to see if there was a block re-org while you were catching up.