Bitcoin Core stuck after a while and RPC become unresponsive

[jenema]

We're running 0.21.1 in Docker on AWS. The setup has been running for 11 months without hiccup.

Today we had an issue. When restarting the node, we cannot bring it back in a functional state

We observe the following

- Starts up without errors
- Starts to sync blocks
- Eventually syncing stops with what seems like network errors
- The RPC becomes unresponsive


I've enabled net logging and can see activity but start to see the following:


Dec 18, 2023 @ 22:25:05.682   2023-12-18T21:25:05Z trying connection 151.25.251.232:8333 lastseen=273.8hrs

Dec 18, 2023 @ 22:25:05.180   2023-12-18T21:25:05Z connection to 69.124.212.197:8333 timeout

Dec 18, 2023 @ 22:25:00.178   2023-12-18T21:25:00Z trying connection 69.124.212.197:8333 lastseen=645.8hrs

Dec 18, 2023 @ 22:24:59.739   2023-12-18T21:24:59Z Cleared nodestate for peer=20

Dec 18, 2023 @ 22:24:59.726   2023-12-18T21:24:59Z socket closed for peer=20

Dec 18, 2023 @ 22:24:59.726   2023-12-18T21:24:59Z disconnecting peer=20

Dec 18, 2023 @ 22:24:59.678   2023-12-18T21:24:59Z send version message: version 70016, blocks=821828, us=[::]:0, peer=20

Dec 18, 2023 @ 22:24:59.678   2023-12-18T21:24:59Z sending version (102 bytes) peer=20

Dec 18, 2023 @ 22:24:53.454   2023-12-18T21:24:53Z Added connection peer=20

Dec 18, 2023 @ 22:24:53.357   2023-12-18T21:24:53Z trying connection 185.135.81.50:8333 lastseen=12.9hrs

Dec 18, 2023 @ 22:24:52.856   2023-12-18T21:24:52Z connection to 46.114.228.163:8333 timeout

Dec 18, 2023 @ 22:24:47.853   2023-12-18T21:24:47Z trying connection 46.114.228.163:8333 lastseen=99.8hrs

Dec 18, 2023 @ 22:24:47.352   2023-12-18T21:24:47Z connect() to 185.52.54.114:8333 failed after select(): Connection refused (111)

Dec 18, 2023 @ 22:24:47.334   2023-12-18T21:24:47Z trying connection 185.52.54.114:8333 lastseen=12576.1hrs

Dec 18, 2023 @ 22:24:46.833   2023-12-18T21:24:46Z connect() to 59.138.115.137:8333 failed after select(): No route to host (113)

Dec 18, 2023 @ 22:24:43.583   2023-12-18T21:24:43Z trying connection 59.138.115.137:8333 lastseen=7976.7hrs

Dec 18, 2023 @ 22:24:43.080   2023-12-18T21:24:43Z connection to 70.187.170.242:8333 timeout


Network connectivity works fine, ie I can do a "curl www.google.com" with no issues.

Any hints?

[1714626258]

1714626258

[1714626258]

1714626258

[1714626258]

1714626258

[NotATether]

Can you verify that your node can receive incoming traffic on port 8333, and not just outgoing?

I don't know if your node has a firewall from Docker or AWS, but you might want to check that out to make sure that's not the cause of your problem.

Is there any abnormal network usage on your node too?

[seek3r]

As @NotATether already mentioned: Ensure that the necessary port (like 8333 in your case) is open and properly forwarded if behind a NAT.

The logs indicate multiple timeouts and connection refusals. This could be due to issues with the peers your node is trying to connect to. You can manually add some stable peers that you are familiar with to see if the node can maintain a stable connection with them.
Your ndode might got a temporarily ban due to many connection attempts in that short period. Just have a look at the logs of the node to ensure that it hasnt been banned by other peers.