getblockcount RPC fluctuation/errors

[dindi]

Hello,

Background:
I am developing an app that partially relies on getblockcount RPC call. It is polling for the last N blocks in the chain. (Yes, I have block notify too, this being a crawl-back mechanism for missing blocks).

Issues: technically everything bitcoind based (litecoin, namecoind, terracoin, max ..etc)  give me inconsistent results over RPC from time to time.


I would get 300,000   300,000,   300,001,  300,001, then suddenly 157,000...  then back to 300,001

When pulling e.g. 100 bocks, then crawling them with Transactions, Vin, Vout, this can get really expensive in my app. Of course I can do a "vote" or last good number and so on, but the question stands:


Does anyone  else see this behaviour? Is this something I can fix? Is there a condition that promotes these occurrences?

Here is a random data set, showing wild swings among all the different coin daemons. All that happens in the app, is that I do a getblockcount call, then log it, when the difference is larger that 1 in any direction, it is logged...


Any thoughts?

Thanks in advance!

ps:  RPC traffic is heavy and I see and handle empty responses. The numbers are only returned when in a valid success response...

 <message>-leap (BTC.2): 595972 - 308763 low:308753</message>
  <message>+leap (MAX.2):397879 - 397881 low:397879</message>
  <message>+leap (NVC.2):105530 - 397883 low:397873</message>
  <message>-leap (NVC.2): 397883 - 105530 low:105520</message>
  <message>+leap (LOT.2):569623 - 569625 low:569623</message>
  <message>+leap (MAX.2):397885 - 569625 low:569615</message>
  <message>-leap (MAX.2): 569625 - 397885 low:397875</message>
  <message>-leap (XPM.2): 610335 - 397885 low:397875</message>
  <message>+leap (XPM.2):397885 - 610335 low:610325</message>
  <message>+leap (VTC.2):113983 - 292384 low:292374</message>
  <message>-leap (VTC.2): 292384 - 113985 low:113975</message>
  <message>+leap (NMC.2):184929 - 359160 low:359150</message>
  <message>-leap (NMC.2): 359160 - 184929 low:184919</message>
  <message>+leap (VTC.2):113985 - 397890 low:397880</message>
  <message>-leap (VTC.2): 397890 - 113985 low:113975</message>
  <message>+leap (VTC.2):113985 - 359161 low:359151</message>
  <message>-leap (VTC.2): 359161 - 113985 low:113975</message>
  <message>+leap (FTC.2):292385 - 308767 low:308757</message>
  <message>-leap (FTC.2): 308767 - 292385 low:292375</message>
  <message>-leap (NMC.2): 184931 - 105532 low:105522</message>
  <message>+leap (NMC.2):105532 - 184931 low:184921</message>
  <message>+leap (NMC.2):184931 - 359161 low:359151</message>
  <message>-leap (NMC.2): 359161 - 184931 low:184921</message>
  <message>-leap (TRC.2): 359161 - 184931 low:184921</message>
  <message>+leap (TRC.2):184931 - 359161 low:359151</message>
  <message>-leap (TRC.2): 359161 - 113989 low:113979</message>
  <message>+leap (TRC.2):113989 - 359161 low:359151</message>
  <message>-leap (TRC.2): 359161 - 105533 low:105523</message>
  <message>+leap (TRC.2):105533 - 359162 low:359152</message>
  <message>+leap (VTC.2):113992 - 292385 low:292375</message>
  <message>+leap (FTC.2):292385 - 595979 low:595969</message>
  <message>-leap (VTC.2): 292385 - 113992 low:113982</message>
  <message>-leap (FTC.2): 595979 - 292385 low:292375</message>
  <message>+leap (BTC.2):308767 - 610359 low:610349</message>
  <message>-leap (BTC.2): 610359 - 308767 low:308757</message>
  <message>+leap (DOGE.2):282393 - 292385 low:292375</message>
  <message>-leap (DOGE.2): 292385 - 282393 low:282383</message>
  <message>+leap (FTC.2):292385 - 397906 low:397896</message>
  <message>-leap (FTC.2): 397906 - 292385 low:292375</message>
  <message>+leap (MAX.2):397908 - 397910 low:397908</message>
  <message>+leap (NVC.2):105534 - 397912 low:397902</message>
  <message>+leap (TRC.2):359162 - 610366 low:610356</message>
  <message>-leap (NVC.2): 397912 - 105534 low:105524</message>
  <message>-leap (TRC.2): 610366 - 359163 low:359153</message>
  <message>+leap (NVC.2):105534 - 610366 low:610356</message>
  <message>-leap (NVC.2): 610366 - 105534 low:105524</message>
  <message>+leap (BTC.2):308767 - 359164 low:359154</message>
  <message>-leap (BTC.2): 359164 - 308767 low:308757</message>
  <message>-leap (XPM.2): 610372 - 184934 low:184924</message>
  <message>+leap (XPM.2):184934 - 610374 low:610364</message>

[dserrano5]

I called getblockcount 100 times and didn't see any problem:

Code:

$ for I in {1..100}; do bitcoin-cli getblockcount >>gbc; sleep 5; done
$ uniq -c gbc
    100 308769
$ _

[dindi]

I called getblockcount 100 times and didn't see any problem:

Code:

$ for I in {1..100}; do bitcoin-cli getblockcount >>gbc; sleep 5; done
$ uniq -c gbc
    100 308769
$ _

Yep, with the client I see the same, I ran a 10k loop, and no issues. I am debugging the RPC library on the client end to ...


And actually, until I copy pasted it here, into an editor, I didn't notice a pattern..  I have a scary bleed-over pattern right there in the examples....   different daemons getting each-other's values.....

Thanks for the post I thought of anything but this, going a little crazy after 2 days of debugging this 12 hours a day:O

Cheers & thanks

[dindi]

Well.. thanks again.

I have an RPC bitcoin client object that is filled with all the RPC parameters. I used to have one for each daemon and recently was packed into one object.

During this procedure some scopes became incorrect. So if there was a poll for BTC, and within that time there was a poll for TRC, I ended up on the wrong port, because the next call overwrote the global parameters except of using local ones.

Since it is development, all the daemons have the same user/pass, except for DOGE. DOGE for that reason exhibited an extremely large "RETRY" number in my app.


Anyway... thanks again for the response...   I am running a last 100 blocks crawl now and all the mysterious "block not found" errors disappeared. Of course, because of this "little mistake"   I was feeding LTC transactions from "rawmempool" into other coin's databases, that naturally resulted in millions of retries.

12 coin daemons, last 100 blocks randomly cross-fed to each-others processes.   ..

Cheers ... all I needed is someone to tell me that the RPC was fine to revise my mess