0 blocks received in the last 4 hours

[tuaris]

Ever since I upgraded to 0.11.x I have started having this problem on my node:
 

Code:

  [errors] => WARNING: check your network connection, 0 blocks received in the last 4 hours (24 expected)

The node is indeed behind in block count:

Code:

    [version] => 110200
    [protocolversion] => 70002
    [walletversion] => 60000
    [blocks] => 397697
    [timeoffset] => -1
    [connections] => 19
    [proxy] =>
    [difficulty] => 144116447847.35
    [testnet] =>
    [keypoololdest] => 1347181371
    [keypoolsize] => 102

I restart the node, it catches up but then falls behind again about a day or two later.  There is nothing wrong with my Internet connection, I have several other nodes of different altcoin running on the same connection and they do not have this problem.

[1713258874]

1713258874

[1713258874]

1713258874

[1713258874]

1713258874

[1713258874]

1713258874

[-XXIII-]

Would there be anything in debug.log that could point us in the right direction?

[tuaris]

I've uploaded the log here: http://phobos.morante.net/downloads/unibia/other/debug.log

[shorena]

There is not a single block accepted before the restart. I guess the log was trimmed though (by default core trims to 10MB on start) I couldnt find anything unusual.

Code:

2016-02-11 04:10:49 Shutdown: In progress...
2016-02-11 04:10:52 StopNode()
2016-02-11 04:16:03

[-XXIII-]

Have you tried rebuilding the blockchain index? ( -reindex )

Code:

2016-02-11 05:09:01 CreateNewBlock(): total size 1000
2016-02-11 05:09:39 CreateNewBlock(): total size 1000
2016-02-11 05:10:28 CreateNewBlock(): total size 1000
2016-02-11 05:10:28 socket recv error Connection reset by peer (54)
2016-02-11 05:10:28 socket error accept failed: Software caused connection abort (53)
2016-02-11 05:10:28 socket send error Bad file descriptor (9)
[...]
2016-02-11 05:11:56 CreateNewBlock(): total size 28483
2016-02-11 05:12:09 CreateNewBlock(): total size 29229
2016-02-11 05:12:18 CreateNewBlock(): total size 31758
2016-02-11 05:12:22 CreateNewBlock(): total size 31983
2016-02-11 05:12:40 CreateNewBlock(): total size 32242
This error is spat out after
[...]
2016-02-11 05:12:44 ERROR: CScriptCheck(): a828806083276dc49921f35161cca061c588e0c65269c847ac81e861db896fc4:0 VerifySignature failed: Non-canonical signature: S value is unnecessarily high
2016-02-11 05:12:44 ERROR: AcceptToMemoryPool: ConnectInputs failed a828806083276dc49921f35161cca061c588e0c65269c847ac81e861db896fc4
[...]
2016-02-11 05:13:12 CreateNewBlock(): total size 38857
2016-02-11 05:14:02 CreateNewBlock(): total size 32435
2016-02-11 05:19:20 CreateNewBlock(): total size 96597
2016-02-11 05:25:05 CreateNewBlock(): total size 92159
2016-02-11 05:30:55 CreateNewBlock(): total size 128513
2016-02-11 05:39:03 CreateNewBlock(): total size 132654
2016-02-11 05:44:03 CreateNewBlock(): total size 88904
2016-02-11 05:44:54 CreateNewBlock(): total size 110542

[tuaris]

Have you tried rebuilding the blockchain index? ( -reindex )

Yes I did that sometime in late November and again in early January when it was occurring much more frequently on the 0.11.0 version.  I recently upgraded (2 weeks ago) to 0.11.2 and hadn't had a problem since the upgrade until now.

I haven't done a reindex using 0.11.2.  If you believe that is worthwhile, I could go ahead and kick that off.  It may take more than a day to complete.

[-XXIII-]

Have you tried rebuilding the blockchain index? ( -reindex )

Yes I did that sometime in late November and again in early January when it was occurring much more frequently on the 0.11.0 version.  I recently upgraded (2 weeks ago) to 0.11.2 and hadn't had a problem since the upgrade until now.

I haven't done a reindex using 0.11.2.  If you believe that is worthwhile, I could go ahead and kick that off.  It may take more than a day to complete.

I'm not sure what else to suggest. Just looking at how the block stacking falls apart after that bad file descriptor makes me suspicious. What intrigues me even more is how CreateNewBlock(): total size goes from 1000 to over 9000!!! somethings definitely out of whack. You could wait for another suggestion, or resync that chain and see what happens

[tuaris]

It has happened again even after I have re-indexed.
I haven't restarted the client and the full debug log is available:

http://phobos.morante.net/downloads/unibia/other/debug.7z

[tuaris]

Also notice that the daemon's memory usage starts to grow tremendously

Code:

39843 nobody         15  22    2  6186M  1746M uwait   3  94.4H   0.00% bitcoind

Probably because all those transactions are piling up.

[shorena]

Also notice that the daemon's memory usage starts to grow tremendously

Code:

39843 nobody         15  22    2  6186M  1746M uwait   3  94.4H   0.00% bitcoind

Probably because all those transactions are piling up.

With an update to 0.12 you can limit the amount of RAM used for transactions (300 MB by default).