Running a full node is starting to be a pain

[halfawake]

I suspect someone started downloading the full blockchain off my full node.  Unfortunately, the effect was that my computer came to a screeching halt: moving the mouse and the pressing keys on my keyboard had absolutely no effect for several minutes.  When I regained use of my computer, I looked in the Task Manager and bitcoin-qt was using the most RAM, and a fair amount of processing power was being used as well.  I turned off bitcoin-qt and the problem went away; anecdotally, I'd say that's reason to believe that running the bitcoin-qt client was causing the problem.

So a little bit about my specs, both internet and computer: I'm using an internet connection of 25 Mbps down / 1.5 Mbps up - reasonable download speed (unfortunately I don't live in an area where gigabit internet is available), but not much on the upload front.  My computer has 5 GB of RAM, 250 GB of hard drive space, and a 3 Ghz processor speed.  The only part of my computer itself that I feel is underpowered is the RAM, I plan on eventually upgrading to 20 GB RAM, but haven't gotten around to it yet.

So here's my question: is there something about the way the bitcoin client is built that causes my computer to freeze up when people start to download the blockchain from me?  Or is this just a case of me not having enough upload bandwidth to handle someone downloading a full node from me?  I'd love to be able to support the network by running the bitcoin-qt client regularly, but this is starting to happen semi-regularly as bitcoin becomes more popular and I don't feel like my computer is THAT underpowered.

[matt4054]

I suspect someone started downloading the full blockchain off my full node.  Unfortunately, the effect was that my computer came to a screeching halt: moving the mouse and the pressing keys on my keyboard had absolutely no effect for several minutes.  When I regained use of my computer, I looked in the Task Manager and bitcoin-qt was using the most RAM, and a fair amount of processing power was being used as well.  I turned off bitcoin-qt and the problem went away; anecdotally, I'd say that's reason to believe that running the bitcoin-qt client was causing the problem.

So a little bit about my specs, both internet and computer: I'm using an internet connection of 25 Mbps down / 1.5 Mbps up - reasonable download speed (unfortunately I don't live in an area where gigabit internet is available), but not much on the upload front.  My computer has 5 GB of RAM, 250 GB of hard drive space, and a 3 Ghz processor speed.  The only part of my computer itself that I feel is underpowered is the RAM, I plan on eventually upgrading to 20 GB RAM, but haven't gotten around to it yet.

So here's my question: is there something about the way the bitcoin client is built that causes my computer to freeze up when people start to download the blockchain from me?  Or is this just a case of me not having enough upload bandwidth to handle someone downloading a full node from me?  I'd love to be able to support the network by running the bitcoin-qt client regularly, but this is starting to happen semi-regularly as bitcoin becomes more popular and I don't feel like my computer is THAT underpowered.

As you noticed yourself about your specs, I would say the problem is the 5 GB of RAM. You don't mention which OS you are using: a 32 bit version of the OS would typically limit user-space RAM to about 3 GB - that's a bit short for modern apps, especially while running a full node.

What you described (in bold in my quote) is probably caused by what I call "swapping to death". It's the symptom of the OS running out of memory, and because of this, it is forced to swap memory contents back and forth from/to the hard drive, which is about 3 orders of magnitude slower than RAM. This may cause your machine to become completely unresponsive for several minutes.

I always wondered why OS designers wouldn't implement a system warning message whenever the system needs to use the swap file to such an extent that the user experience becomes heavily degraded. Most users don't understand what's happening in the background, and what they can do about it (i.e. add more RAM)

[deepceleron]

You can do a few things to mitigate coin applications that cause problems:

1. Lower the CPU affinity and process priority. You can do this with task manager right-clicks in Windows if you are going to leave Bitcoin running, or you can make a .cmd file (batch file) to run with the start priority options. CPU is required to validate P2P transaction signatures, and someone could be targeting you specifically with attack transaction broadcasts.

2. If your rotating hard disk is fragmented, that means the disk heads will have to hunt all over the platter to find data, limiting the possible input-output operations per second. You can use defrag programs that completely move and defrag every file, but the new blocks every 10 minutes will again be fragmented. It would be best to give the Bitcoin datadir its own ~40gb disk partition to limit hunting. This problem is solved with a SSD - instead of a limit of 150 IOPS from moving heads, it's more like 40000 IOPS.

3. Monitor RAM usage. I'm not so sure this is an issue. I have Bitcoin Core and Namecoin both running. They use under 400MB each on a 8GB system, and don't impact anything else noticeably.

[halfawake]

As you noticed yourself about your specs, I would say the problem is the 5 GB of RAM. You don't mention which OS you are using: a 32 bit version of the OS would typically limit user-space RAM to about 3 GB - that's a bit short for modern apps, especially while running a full node.

What you described (in bold in my quote) is probably caused by what I call "swapping to death". It's the symptom of the OS running out of memory, and because of this, it is forced to swap memory contents back and forth from/to the hard drive, which is about 3 orders of magnitude slower than RAM. This may cause your machine to become completely unresponsive for several minutes.

I always wondered why OS designers wouldn't implement a system warning message whenever the system needs to use the swap file to such an extent that the user experience becomes heavily degraded. Most users don't understand what's happening in the background, and what they can do about it (i.e. add more RAM)

Ah, good point, I didn't mention the OS.  I'm using Windows 7 64 bit, so it should totally support the 20 GB of RAM that I'm planning on upgrading to eventually.  Thanks for the technical explanation of what's going on, so it sounds like the RAM upgrade will probably fix the issue then.  Unfortunately, the motherboard on my computer only has two RAM slots, so I'm going to have to get a 16 GB chip to upgrade to the 20 GB of RAM that I'm aiming for.  (The one on my Amazon wishlist is going for about $171 right now.  It was actually significantly cheaper (like $20 - 30) on bitcoinstore.com last I checked though, so I'll probably buy it there.)

You're right, it would be smart for OS designers to warn users about this kind of swap file need.  I just hope that this reliance on swap files that's gotten a bit more common lately isn't going to degrade the lifespan of my hard drive.

[halfawake]

You can do a few things to mitigate coin applications that cause problems:

1. Lower the CPU affinity and process priority. You can do this with task manager right-clicks in Windows if you are going to leave Bitcoin running, or you can make a .cmd file (batch file) to run with the start priority options. CPU is required to validate P2P transaction signatures, and someone could be targeting you specifically with attack transaction broadcasts.

2. If your rotating hard disk is fragmented, that means the disk heads will have to hunt all over the platter to find data, limiting the possible input-output operations per second. You can use defrag programs that completely move and defrag every file, but the new blocks every 10 minutes will again be fragmented. It would be best to give the Bitcoin datadir its own ~40gb disk partition to limit hunting. This problem is solved with a SSD - instead of a limit of 150 IOPS from moving heads, it's more like 40000 IOPS.

3. Monitor RAM usage. I'm not so sure this is an issue. I have Bitcoin Core and Namecoin both running. They use under 400MB each on a 8GB system, and don't impact anything else noticeably.

Hrm.  I was noticing high CPU usage, so it's possible that was an issue.  I ran disk defrag a month or two ago and it only reported 1% of fragmentation, which doesn't seem like that much.  That said, it didn't tell me which 1% of the hard drive was fragmented.  If that 2 GB or so was right smack in the middle of the blockchain, that'd cause some significant problems I'd imagine.

Granted, I do have an old fashioned hard drive, not an SSD.  I'm seriously thinking of upgrading and am wondering whether bumping up my RAM or adding in an SSD would be better off performance wise.  Any thoughts there?

[deepceleron]

A solid state drive as an OS drive is the single biggest improvement you will ever experience. I boot Ubuntu 14 in under 30 seconds on an otherwise seven-year-old computer. You do have to plan your data allocation, such as C:SSD 240GB and then D:HDD for multimedia. If your current hard drive is not just 250GB of free space, but is actually a 250GB drive, it is also 10-year-old tech (but with the benefit that you can just duplicate it to a new SSD).

I would only recommend Tier-1 SSD, such as Intel 530 or Samsung 840 EVO 250GB. You can get the latter at Best Buy or Newegg for $130.

[matt4054]

Granted, I do have an old fashioned hard drive, not an SSD.  I'm seriously thinking of upgrading and am wondering whether bumping up my RAM or adding in an SSD would be better off performance wise.  Any thoughts there?

If you seldom run several RAM-demanding apps at the same time, I would say you'll be better off with an SSD. Otherwise, considering your current 5 GB of RAM, I would go for the RAM extension. Of course, having both would be the best ;-)

If you go for the SSD option, make sure to have your system files (OS) and swap file on it, as it's what makes most of the difference. In other words, keep your hard drive for large data files. Where you will store the block chain depends on your own priorities and the capacity of your SSD; of course, it would be much faster on the SSD (regarding what deepceleron said about fragmentation, timings etc), but the blockchain is getting huge nowadays.

[doldgigger]

I wonder why there isn't more effort put in the direction of implementing QoS mechanisms (like throttling blockchain downloads) in the protocol. After all, the network as a whole does not become more efficient when single nodes hog down other nodes due to misconfigured blockchain downloading.

[CJYP]

You can do a few things to mitigate coin applications that cause problems:

1. Lower the CPU affinity and process priority. You can do this with task manager right-clicks in Windows if you are going to leave Bitcoin running, or you can make a .cmd file (batch file) to run with the start priority options. CPU is required to validate P2P transaction signatures, and someone could be targeting you specifically with attack transaction broadcasts.

2. If your rotating hard disk is fragmented, that means the disk heads will have to hunt all over the platter to find data, limiting the possible input-output operations per second. You can use defrag programs that completely move and defrag every file, but the new blocks every 10 minutes will again be fragmented. It would be best to give the Bitcoin datadir its own ~40gb disk partition to limit hunting. This problem is solved with a SSD - instead of a limit of 150 IOPS from moving heads, it's more like 40000 IOPS.

3. Monitor RAM usage. I'm not so sure this is an issue. I have Bitcoin Core and Namecoin both running. They use under 400MB each on a 8GB system, and don't impact anything else noticeably.

Hrm.  I was noticing high CPU usage, so it's possible that was an issue.  I ran disk defrag a month or two ago and it only reported 1% of fragmentation, which doesn't seem like that much.  That said, it didn't tell me which 1% of the hard drive was fragmented.  If that 2 GB or so was right smack in the middle of the blockchain, that'd cause some significant problems I'd imagine.

Granted, I do have an old fashioned hard drive, not an SSD.  I'm seriously thinking of upgrading and am wondering whether bumping up my RAM or adding in an SSD would be better off performance wise.  Any thoughts there?

There is such a thing as a hybrid SSD HDD. I don't really know anything about it (so you should look into it yourself), but it seems to allow you to boot quickly from SSD while performing most of your everyday tasks on the cheaper hard drive.

I wonder why there isn't more effort put in the direction of implementing QoS mechanisms (like throttling blockchain downloads) in the protocol. After all, the network as a whole does not become more efficient when single nodes hog down other nodes due to misconfigured blockchain downloading.

Downloading the blockchain is easy. A much bigger resource hog is verifying the blockchain.

[stryker]

For a while now I've happily ran a full node using an old pentium dual core with 4GB ram... around 3.4Ghz with debian linux.  In fact it does much more than just bitcoin and the processor is hardly touched.

As for bitcoin I allow up to 50 connections inbound and I limited the outbound bandwidth to half the upstream speed delivered to my house.

Its worked flawlessly thus far.

[Meuh6879]

1) set 10 or 5 Mb on cache setting (instead of 100Mb)
2) set 12 connexion (not less than 8 instead of 20).

by the way, download the entery blockchain take insane CPU job at the end (with no download).

example : 22h of download, 4h of 100% CPU job at the end (just after that, bitcoin core is OK and have finish).

[elebit]

Something must be wrong with your set up.

I run bitcoind on 1 GB RAM and it's a bit strained at times but works. RAM should be your limiting factor.

[DeathAndTaxes]

5GB is a weird memory amount.  I assume it is 1GB DIMM and 4GB DIMM.  I would either add a 4GB DIMM (8GB total) or remove both and add a pair of 8GB DIMMS (16GB total).  Save the rest of your funds towards an SSD. 

[halfawake]

5GB is a weird memory amount.  I assume it is 1GB DIMM and 4GB DIMM.  I would either add a 4GB DIMM (8GB total) or remove both and add a pair of 8GB DIMMS (16GB total).  Save the rest of your funds towards an SSD. 

You're right, it's a combination of a 1 GB DIMM and a 4 GB DIMM. The computer originally had 2 GB of RAM but I had the store I bought it from swap out one of the 1 GB for a 4 GB since 2 GB of RAM just doesn't cut it in Windows.  So do you think the 20 GB of RAM I was aiming to upgrade to is overkill?

[gmaxwell]

So here's my question: is there something about the way the bitcoin client is built that causes my computer to freeze up when people start to download the blockchain from me?

Sounds like you have a broken SATA driver/controller. Try starting up some disk benchmarks and see if your computer becomes unusable when there is the slightest activity.

[halfawake]

So here's my question: is there something about the way the bitcoin client is built that causes my computer to freeze up when people start to download the blockchain from me?

Sounds like you have a broken SATA driver/controller. Try starting up some disk benchmarks and see if your computer becomes unusable when there is the slightest activity.

Huh, you mean I could fix this with a driver update?  That would sure be convenient.  I'll look into it and report back, thanks for the tip.

[FPoBA]

Anyone know the suggested specs to run a full node? I want to run one of my own, without having the PC freeze up.

[Vortex20000]

Well, I have a computer with 16GBs of RAM and upload/download speed of 300mbps.

I'm running Bitcoin-qt, Namecoin-qt, Feathercoin-qt, Dogecoin-qt, and Digibyte-qt, all of which are full nodes.

I don't get lag.

[Abdussamad]

5GB is a weird memory amount.  I assume it is 1GB DIMM and 4GB DIMM.  I would either add a 4GB DIMM (8GB total) or remove both and add a pair of 8GB DIMMS (16GB total).  Save the rest of your funds towards an SSD.  

You're right, it's a combination of a 1 GB DIMM and a 4 GB DIMM. The computer originally had 2 GB of RAM but I had the store I bought it from swap out one of the 1 GB for a 4 GB since 2 GB of RAM just doesn't cut it in Windows.  So do you think the 20 GB of RAM I was aiming to upgrade to is overkill?

Upgrade to 8GB. It should be more than enough. But first check your hard disk. Run chkdsk/scan disk or whatever you have on windows. Do the long test - the one that takes hours

edit: it's chkdsk /r

[FPoBA]

Well, I have a computer with 16GBs of RAM and upload/download speed of 300mbps.

I'm running Bitcoin-qt, Namecoin-qt, Feathercoin-qt, Dogecoin-qt, and Digibyte-qt, all of which are full nodes.

I don't get lag.

How big is the Blockchain at the moment?