Bitcoin Forum
April 19, 2018, 10:31:29 AM *
News: Latest stable version of Bitcoin Core: 0.16.0  [Torrent]. (New!)
 
   Home   Help Search Donate Login Register  
Pages: [1]
  Print  
Author Topic: Bitcoin Node CPU Spikes  (Read 666 times)
udevNull
Newbie
*
Offline Offline

Activity: 12
Merit: 0


View Profile
March 01, 2017, 05:10:11 AM
 #1

Hey guys.

Quick question regarding running a node, not sure if this place to ask or if https://bitcointalk.org/index.php?board=6.0 would be better. Please let me know.

I'm monitoring CPU usage of the node using grafana which gets it's data, ultimately, from collectd. I'm running 0.13.2.
My question is regarding random CPU spikes that occur. Any ideas what could be causing that?
They can last for up to 10 minutes (often only spike for a minute or two) and reach around 70% - 80% CPU usage. I've been seeing this for a while now (the last 3/4 months of active monitoring) on different versions of bitcoin core and on different machines (different hardware, same OS, Ubuntu 16.04 LTS). Is this some kind of a clean up process that runs or is it something else?

Thanks guys.
1524133889
Hero Member
*
Offline Offline

Posts: 1524133889

View Profile Personal Message (Offline)

Ignore
1524133889
Reply with quote  #2

1524133889
Report to moderator
1524133889
Hero Member
*
Offline Offline

Posts: 1524133889

View Profile Personal Message (Offline)

Ignore
1524133889
Reply with quote  #2

1524133889
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
achow101
Moderator
Legendary
*
Offline Offline

Activity: 1372
Merit: 1268


3F1Y9yquzvY6RWvKbw2n2zeo9V5mvBhADU


View Profile WWW
March 01, 2017, 05:13:22 AM
 #2

It could be when there are spikes in transactions that need verification. Could also be new blocks coming in that need to be checked.

udevNull
Newbie
*
Offline Offline

Activity: 12
Merit: 0


View Profile
March 01, 2017, 05:31:12 AM
 #3

Yeah, I checked that first as well but they don't correspond with new blocks. i.e. last night I had a spike between 23:16 - 23:25 (@72% CPU Usage). The next spike occured at 00:13 (@63% CPU Usage). The next one at 01:20 (@81% CPU Usage).

Using collectd, I've been able to narrow it down specifically to the bitcoind process. If it were related to blocks, it would occur far more frequently. The server is also a C4 aws instance. So it's not like it's lacking in terms of CPU power.
-ck
Staff
Legendary
*
Offline Offline

Activity: 2492
Merit: 1047


Ruu \o/


View Profile WWW
March 01, 2017, 05:38:15 AM
 #4

You're on linux so bring up 'top' while it's happening, press H to enable thread mode, and see what thread(s) is/are consuming CPU at the time. The bitcoin daemon has lots of threads that are named so you may be able to track which particular thread is consuming CPU and hence figure out what it's doing.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
udevNull
Newbie
*
Offline Offline

Activity: 12
Merit: 0


View Profile
March 01, 2017, 05:41:06 AM
 #5

You're on linux so bring up 'top' while it's happening, press H to enable thread mode, and see what thread(s) is/are consuming CPU at the time. The bitcoin daemon has lots of threads that are named so you may be able to track which particular thread is consuming CPU and hence figure out what it's doing.

Right, okay. I can give that a shot. So nobody else has experienced similar CPU spikes?
shorena
Copper Member
Legendary
*
Offline Offline

Activity: 1442
Merit: 1209


No I dont escrow anymore.


View Profile WWW
March 01, 2017, 07:57:06 AM
 #6

You're on linux so bring up 'top' while it's happening, press H to enable thread mode, and see what thread(s) is/are consuming CPU at the time. The bitcoin daemon has lots of threads that are named so you may be able to track which particular thread is consuming CPU and hence figure out what it's doing.

Right, okay. I can give that a shot. So nobody else has experienced similar CPU spikes?

Yeah, not that long though. Turned out my node was connected to a service that was pushing several thousand transactions to it every X minutes. If you have a network spike (from a few kbit/s to several mbit/s) at the same times that might be it.

udevNull
Newbie
*
Offline Offline

Activity: 12
Merit: 0


View Profile
March 01, 2017, 08:49:53 AM
 #7

You're on linux so bring up 'top' while it's happening, press H to enable thread mode, and see what thread(s) is/are consuming CPU at the time. The bitcoin daemon has lots of threads that are named so you may be able to track which particular thread is consuming CPU and hence figure out what it's doing.

Right, okay. I can give that a shot. So nobody else has experienced similar CPU spikes?

Yeah, not that long though. Turned out my node was connected to a service that was pushing several thousand transactions to it every X minutes. If you have a network spike (from a few kbit/s to several mbit/s) at the same times that might be it.

That's quite interesting, how long did your spikes last and how did you pick that up?
HI-TEC99
Legendary
*
Offline Offline

Activity: 1190
Merit: 1018



View Profile
March 01, 2017, 10:28:07 AM
 #8

You're on linux so bring up 'top' while it's happening, press H to enable thread mode, and see what thread(s) is/are consuming CPU at the time. The bitcoin daemon has lots of threads that are named so you may be able to track which particular thread is consuming CPU and hence figure out what it's doing.

Right, okay. I can give that a shot. So nobody else has experienced similar CPU spikes?

Yeah, not that long though. Turned out my node was connected to a service that was pushing several thousand transactions to it every X minutes. If you have a network spike (from a few kbit/s to several mbit/s) at the same times that might be it.


A redditer was complaining about high node CPU usage a year ago. He identified a high number of connections correlating with high CPU usage, and speculated it might be due to suboptimally configured nodes pushing dust and spam transactions. His error log was full of nonstandard transaction/dust errors.

UdevNull's node's error log might contain the same errors if he's experiencing the same problem.

https://www.reddit.com/r/Bitcoin/comments/3xp22r/fully_synced_bitcoin_core_is_using_50_cpu_is/cy6l0oi/

Quote

I'm running bitcoind on several different servers, some of them are reachable via a public IP, other aren't.

With public IP:
version v0.11.0, minrelaytxfee=0.00004
CPU fluctuates between 20% and 50%
113 connections
37135 transactions in mempool, 463 MB total

I see a lot of ERROR messages in debug.log
.
2015-12-21 13:51:16 ERROR: AcceptToMemoryPool: free transaction rejected by rate limiter
2015-12-21 13:51:16 ERROR: AcceptToMemoryPool: nonstandard transaction: dust

Without public IP:
version v0.11.1, no config tweaks
CPU usage often stays in 0-2% range,
9 connections
35766 transactions in mempool, 428 MB

very few ERROR messages in debug.log

So it looks like CPU usage correlates with the number of connections. I'm not sure if there is any specific cause (possible causes: old/misconfigured nodes pushing dust & spam transactions; thin clients catching up with the blockchain using bloom queries), or just the sheer number of connections.

You can restrict the number of connection with maxconnections config parameter.
shorena
Copper Member
Legendary
*
Offline Offline

Activity: 1442
Merit: 1209


No I dont escrow anymore.


View Profile WWW
March 06, 2017, 10:35:42 AM
 #9

You're on linux so bring up 'top' while it's happening, press H to enable thread mode, and see what thread(s) is/are consuming CPU at the time. The bitcoin daemon has lots of threads that are named so you may be able to track which particular thread is consuming CPU and hence figure out what it's doing.

Right, okay. I can give that a shot. So nobody else has experienced similar CPU spikes?

Yeah, not that long though. Turned out my node was connected to a service that was pushing several thousand transactions to it every X minutes. If you have a network spike (from a few kbit/s to several mbit/s) at the same times that might be it.

That's quite interesting, how long did your spikes last and how did you pick that up?

Just a few seconds. I found the IP in question using tcptrack. The node was just regular XT node, nothing special about it. With nmap I found they had ports 80 and 443 (among others) open so I just entered the IP in a browser and it redirected me to their main HP.



If you want to check the debug.log for a possible origin node as HI-TEC99 suggested make sure you have logips=1 set

Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!