Most Efficient Way To Run A Full Node?

[Paleus]

I am wondering what is the best way to run a full node with limited computing resources. In order to do this we must download and synchronize a full copy of the blockchain (300+ GB). This is not for being competitive as a miner but only hosting a full network node that can remain up-to-date with the blockchain in real-time.

I have tried downloading the entire blockchain to an external hard drive. However, it is basically at a standstill with 25% or more of the blockchain still needing to be synchronized. Is this delay because I am hosting it on an external hard drive? (Western Digital External Hard Drive)

I have heard people on this forum recommend renting server space in order to host a full node. I am wondering if this is a better option than hosting locally and what the costs associated would be. Performance and security are top priorities. I am looking at running a full node continuously, not just for a temporary time.

[AB de Royse777]

From the last few months I am not running a full node but I had a node in the external device for a few months and I did not have a problem running it all the time.

I have heard people on this forum recommend renting server space in order to host a full node. I am wondering if this is a better option than hosting locally and what the costs associated would be. Performance and security are top priorities. I am looking at running a full node continuously, not just for a temporary time.

I never paid much attention to it however if there is some step-by-step instruction then I have hosting with namecheap and if this works then I can also run a node online. Not sure if I understood correctly by renting server space.

[Wind_FURY]

I have heard people on this forum recommend renting server space in order to host a full node. I am wondering if this is a better option than hosting locally and what the costs associated would be. Performance and security are top priorities. I am looking at running a full node continuously, not just for a temporary time.

For the purpose of running it for fun, OK, but to actually run one, don't do that, because what would be the point? Your hosting service can censor you by simply turning off your machine, or deny access.

Not your hardware, not your node.

[stompix]

I have tried downloading the entire blockchain to an external hard drive. However, it is basically at a standstill with 25% or more of the blockchain still needing to be synchronized. Is this delay because I am hosting it on an external hard drive? (Western Digital External Hard Drive)

I remember, this post of achow101 it was during the time I was also searching for a different set to host my node and my old hdd wasn't able to sync in this life so I was looking for alternatives, the SSD prices put me off, especially since I knew I would have to buy from the starts a bigger one but right now I would go for one without thinking twice.

As for the server version, yeah, most people go with that, Hetzerand DigitalOcean had 20% of the nodes at one point, I saw quite a number recommending them as it gets rid of a machine running 24/7 in the house and offers more stability. As for the price, that depends on what budget people are willing to pay for a node, for some 10$ is too much, for some 50$ a month doesn't matter.

[BitcoinFX]

...snip...

I have heard people on this forum recommend renting server space in order to host a full node. I am wondering if this is a better option than hosting locally and what the costs associated would be. Performance and security are top priorities. I am looking at running a full node continuously, not just for a temporary time.

Running a Bitcoin full node on a VPS server instance is much more costly (in the long-term) than running a full node on your 'home' PC.

Whilst you can host a pruned node (for around $5 a month), with VPS / linux server companies such as digital ocean and vultr, hosting a full node at 300+ GB requires block storage which will set you back around $1.00/mo per 10 GB of storage i.e. around $55 a month minimum for a 500 GB instance.

...

I'm actually in the process of writing a guide for setting up a $5 Linux VPS running a pruned Bitcoin node (as a dual stack node with Tor .onion, optional) for my 'Bitcoin on Tor addnodes project' see: https://bitcointalk.org/index.php?topic=5177001.0 - however the guide likely won't be published until after the Tor v3 .onion rollout in mid 2021.

It is also possible to effectively CPU mine various altcoins utilizing cpulimit at 75% max (so that the VPS companies won't ban you!), to offset the cost of running a Bitcoin pruned VPS node 'cloud' instance.

...

TBH I find it rather surprising that the OP has recently posted this topic;

Tutorial: How To Fork Bitcoin
- https://bitcointalk.org/index.php?topic=5295647.0

... and yet seemingly is unaware of what is required in terms of 'hosting' a full Bitcoin server in the cloud !?

...

Whilst we are all always learning, in regards to Bitcoin, as one of the oldest and active members of this forum, topics like this one just highlight to me the often great disparity between users technical knowledge and what gets interest / help / funding to move forward with what folks seemingly need and want to achieve.

[barto123]

http://mynodebtc.com/

Bitcoin + Lightning plus all the useful necessities to become self-sovereign.

Will step you through exactly what you need to do.

I have a basic technical ability & I found it easy.

Raspberry PI 4, HDD, MicroSD card, case etc ~$200 on Amazon.

Software is FREE unless you want access to premium features (which I highly recommend).

[BitcoinFX]

http://mynodebtc.com/

Bitcoin + Lightning plus all the useful necessities to become self-sovereign.

Will step you through exactly what you need to do.

I have a basic technical ability & I found it easy.

Raspberry PI 4, HDD, MicroSD card, case etc ~$200 on Amazon.

Software is FREE unless you want access to premium features (which I highly recommend).

Yes indeed. This is a very good all-in-one 'home' solution for many Bitcoin users.

[Welsh]

Single board computers are the way to go. They run fairly efficiently so you aren't going to be forking out huge amounts of money on running costs, and there's usually just a single payment for the SBC, and hard drive. Anything else will be overkill, and will cost you more in the long term. If there was ever a global blanket censorship on Bitcoin, then we rely on nodes to keep the network secure, and that's exactly why you should never use a third party to host it. Of course, there's also the issue that you are technically pissing money down the drain every month for running costs. So get a SBC, there are cheaper versions than a Raspberry Pi out there,  but the Raspberri Pi has the better compatibility, and documentation.

Plus, the good news is if you ever want to host another website or anything like that the Raspberry Pi becomes very useful outside of hosting a node. I've got maybe 10 of these little things lying around doing various tasks for me.

[Paleus]

...snip...

I have heard people on this forum recommend renting server space in order to host a full node. I am wondering if this is a better option than hosting locally and what the costs associated would be. Performance and security are top priorities. I am looking at running a full node continuously, not just for a temporary time.

Running a Bitcoin full node on a VPS server instance is much more costly (in the long-term) than running a full node on your 'home' PC.

Whilst you can host a pruned node (for around $5 a month), with VPS / linux server companies such as digital ocean and vultr, hosting a full node at 300+ GB requires block storage which will set you back around $1.00/mo per 10 GB of storage i.e. around $55 a month minimum for a 500 GB instance.

...

I'm actually in the process of writing a guide for setting up a $5 Linux VPS running a pruned Bitcoin node (as a dual stack node with Tor .onion, optional) for my 'Bitcoin on Tor addnodes project' see: https://bitcointalk.org/index.php?topic=5177001.0 - however the guide likely won't be published until after the Tor v3 .onion rollout in mid 2021.

It is also possible to effectively CPU mine various altcoins utilizing cpulimit at 75% max (so that the VPS companies won't ban you!), to offset the cost of running a Bitcoin pruned VPS node 'cloud' instance.

...

TBH I find it rather surprising that the OP has recently posted this topic;

Tutorial: How To Fork Bitcoin
- https://bitcointalk.org/index.php?topic=5295647.0

... and yet seemingly is unaware of what is required in terms of 'hosting' a full Bitcoin server in the cloud !?

...

Whilst we are all always learning, in regards to Bitcoin, as one of the oldest and active members of this forum, topics like this one just highlight to me the often great disparity between users technical knowledge and what gets interest / help / funding to move forward with what folks seemingly need and want to achieve.

I suppose the crux of my questioning on efficiency revolves around storage of the blockchain and the experiences of bitcointalk users storing the blockchain in an external HDD and SSD. It would seem due to the continuous requirement of read/write access, using a SSD is almost always better performance wise. In order to have a forward-compatible node, we would need a SSD of at least 350 GB at this point.

Someone here made the comment that a full node shouldn't be hosted in the cloud in the very place but should only be hosted on hardware that you actually own and can physically access. I tend to agree with the observation that if you are using someone else's hardware than it is not truly YOUR node. In terms of efficiency, it seems in the long run it would actually be most cost-efficiency to pony up the costs for hardware that you yourself host locally, and in the case of storage, this would require a SSD of at least 350 GB - might as well buy 1 TB while you are at it.

Thank you all for your input.

[Paleus]

http://mynodebtc.com/

Bitcoin + Lightning plus all the useful necessities to become self-sovereign.

Will step you through exactly what you need to do.

I have a basic technical ability & I found it easy.

Raspberry PI 4, HDD, MicroSD card, case etc ~$200 on Amazon.

Software is FREE unless you want access to premium features (which I highly recommend).

May I ask why HDD instead of SSD?

Additionally, because MyNodeBTC comes preloaded with software, etc., how can we know for certain that it has not been tampered with or a trojan/keylogger, etc. has been installed on the device before it reaches us?

[ranochigo]

May I ask why HDD instead of SSD?

HDD is more cost effective than SSD in terms of per gb costs. While it does run a bit louder, Bitcoin nodes has a lot of I/O activity with it due to the constant fetching of block info to supply to the nodes. Flash storage can degrade over time with constant reading and writing. HDD is more reliable in this aspect.

Additionally, because MyNodeBTC comes preloaded with software, etc., how can we know for certain that it has not been tampered with or a trojan/keylogger, etc. has been installed on the device before it reaches us?

I would recommend for you to run one with Raspberry Pi.

Do not put your files on the MicroSD card if you're going that route. The high I/O will wear out the MicroSD card very quickly. You would want to synchronize your files on a computer before putting it on a Raspberry Pi. It's too underpowered and will take quite some time, perhaps up to a week to synchronize. You will want to use the options to disable wallet and all the unnecessary stuff to try to save the resources and perhaps limit the peer count. RPi is very underpowered for a node but it performs well for it's price, it'll work just fine for a medium sized full node.

[Paleus]

May I ask why HDD instead of SSD?

HDD is more cost effective than SSD in terms of per gb costs. While it does run a bit louder, Bitcoin nodes has a lot of I/O activity with it due to the constant fetching of block info to supply to the nodes. Flash storage can degrade over time with constant reading and writing. HDD is more reliable in this aspect.

Additionally, because MyNodeBTC comes preloaded with software, etc., how can we know for certain that it has not been tampered with or a trojan/keylogger, etc. has been installed on the device before it reaches us?

I would recommend for you to run one with Raspberry Pi.

Do not put your files on the MicroSD card if you're going that route. The high I/O will wear out the MicroSD card very quickly. You would want to synchronize your files on a computer before putting it on a Raspberry Pi. It's too underpowered and will take quite some time, perhaps up to a week to synchronize. You will want to use the options to disable wallet and all the unnecessary stuff to try to save the resources and perhaps limit the peer count. RPi is very underpowered for a node but it performs well for it's price, it'll work just fine for a medium sized full node.

I already have ~145 GB of the blockchain synched on a HDD but the rest is very slow to synch. How can I download the remaining on my local hard drive then move it over to the 1 TB HDD?

[barto123]

http://mynodebtc.com/

Bitcoin + Lightning plus all the useful necessities to become self-sovereign.

Will step you through exactly what you need to do.

I have a basic technical ability & I found it easy.

Raspberry PI 4, HDD, MicroSD card, case etc ~$200 on Amazon.

Software is FREE unless you want access to premium features (which I highly recommend).

May I ask why HDD instead of SSD?

Additionally, because MyNodeBTC comes preloaded with software, etc., how can we know for certain that it has not been tampered with or a trojan/keylogger, etc. has been installed on the device before it reaches us?

SDD is better, but obviously more money

MyNode software is opensource; but yes, you are putting an element of trust into the hands on the lead developer - his reputation is on the line though - he has more of an incentive to preserve it if he wants to continue earning good money from his code.

[pixie85]

May I ask why HDD instead of SSD?

HDD is more cost effective than SSD in terms of per gb costs. While it does run a bit louder, Bitcoin nodes has a lot of I/O activity with it due to the constant fetching of block info to supply to the nodes. Flash storage can degrade over time with constant reading and writing. HDD is more reliable in this aspect.

That's true but even if you use it 24/7 it's going to take years before the disk degrades. It should survive at least 3 years of heavy load.

Blockchain requires over 300 GB, a cheap 512GB SSD will cost you about 50EUR. It's the average price of a computer game

If you want it to be cheap to run you can set up a node on raspberry pi or a micro pc. A micro pc with 4gb ram will cost you about 100 EUR

[Carlton Banks]

+1 Raspberry Pi

I'm using an Rpi 3 (which shares the USB bandwidth with the ethernet), and it's more than enough to do the job

my advice would be to buy 2 external disks; one small SSD (even 32 GB is enough) for speed and reliability, and then one regular HDD with big capacity (+2TB for the future). Put the your Linux installation on the SSD, also Bitcoin's chainstate folder, but then keep the actual blockchain blocks on the slow HDD.

then, run it like a real server: use. the. command line. you'll find you will eventually want to tweak the kernel options to improve the reliability, or maybe for security to run a Lightning daemon alongside bitcoind (Lightning requires a hot wallet)

[ranochigo]

I already have ~145 GB of the blockchain synched on a HDD but the rest is very slow to synch. How can I download the remaining on my local hard drive then move it over to the 1 TB HDD?

What I'll do is to synchronize the client completely on a normal computer and then transfer the entire data directory to the HDD. It'll save a lot of time as you can validate faster on the computer due to its better processor.

That's true but even if you use it 24/7 it's going to take years before the disk degrades. It should survive at least 3 years of heavy load.

Blockchain requires over 300 GB, a cheap 512GB SSD will cost you about 50EUR. It's the average price of a computer game

I don't think anyone actually did an actual test on it but a good priced SSD should withstand the degradation pretty well. In terms of the cost and benefit, you'll probably have a faster synchronization and roughly a similar experience of running a node. I would argue that given the only process that would have any noticeable benefits from an SSD is the IBD, a HDD would provide a marginal cost savings and provide roughly the same user experience.

[MoreStamps.Global]

A dedicated VPS will be the finest solution, I think. I am running ETH 2.0 testnet on GCP and I am satisfied. You don't have to worry about constant internet and electricity supply. Besides, if you are determined to host a node at least for two years and luckily if you are running a business, AWS is here to provide you $5000 credits which is a win-win. There are also resellers you can get such accounts from and utilize it for hosting a node.

[Carlton Banks]

AWS is here to provide you $5000 credits which is a win-win

no that's spending money for almost no gain

AWS VPS instances are run out of the same datacenters occupying the same range of IP addresses, and they're overwhelmingly over-represented as part of the overall infrastrcuture of the internet. The purpose of people running their own Bitcoin node is of security/reliability through redundancy, and since there are probably several hundred+ Bitcoin nodes running on AWS instances, you wouldn't be diversifying the distribution of the Bitcoin network at all, rather more you would be concentrating Bitcoin nodes in the datacenter intranet of just one very large (the largest) hosting company.

sure, use a VPS. Don't use the biggest hosting company for VPS's, that's not making the Bitcoin network in the slightest bit more resilient.

[MoreStamps.Global]

AWS is here to provide you $5000 credits which is a win-win

no that's spending money for almost no gain

AWS VPS instances are run out of the same datacenters occupying the same range of IP addresses, and they're overwhelmingly over-represented as part of the overall infrastrcuture of the internet. The purpose of people running their own Bitcoin node is of security/reliability through redundancy, and since there are probably several hundred+ Bitcoin nodes running on AWS instances, you wouldn't be diversifying the distribution of the Bitcoin network at all, rather more you would be concentrating Bitcoin nodes in the datacenter intranet of just one very large (the largest) hosting company.

sure, use a VPS. Don't use the biggest hosting company for VPS's, that's not making the Bitcoin network in the slightest bit more resilient.

You're right in diversifying the distribution of the Bitcoin network but being a validator, we also need to avoid a burning hole in our pockets, right? Getting $5000 credit account for mere $300 is something very economical for any validator. Now just check the cost of other cloud hosting providers for this specs: v4 CPU, 16 GB RAM and 1 TB SSD. That's all I wanted to say!

People don't give a shit when it comes to helping Bitcoin network to become strong and secure as long as they are getting profits!

[ranochigo]

You're right in diversifying the distribution of the Bitcoin network but being a validator, we also need to avoid a burning hole in our pockets, right? Getting $5000 credit account for mere $300 is something very economical for any validator. Now just check the cost of other cloud hosting providers for this specs: v4 CPU, 16 GB RAM and 1 TB SSD. That's all I wanted to say!

People don't give a shit when it comes to helping Bitcoin network to become strong and secure as long as they are getting profits!

How about running it at home? It'll surely be way more economical that way.

If you're talking about helping the network, I don't think there really is that big of a need for you to run a node if you're just going to run it on the IP range as the others. After all, the benefits that others can reap from a node which is running at the same location is significantly lower. You don't really get profits from running a node at all though so you'll have to consider that as well.

However, if you're doing some data analytics like me, I could understand your need for a cloud instance that could support additional tasks other than running a node.