Cheap Node Self Hosting: Just because you CAN does not mean you SHOULD

[n0nce]

Since the same things keep coming up:

1) Yes you can force a RPi2 or 3 to run a node but it is going to be a slow miserable time and you will have lots of issues.

I don't visit this subforum a lot to be honest, so I didn't know this was a common question. Honestly, as you pointed out, with used parts and around 50 bucks of any currency you can get a decent node.

n0nce made a nice post about doing a node for under $60. https://bitcointalk.org/index.php?topic=5364742
You can do it for a bit more, you can do it for a bit less.

Don't try to cut that price in 1/2 unless you are technically inclined and can work around the issues. Otherwise you are going to have a miserable time.

I honestly don't see it happening for anything under 50 as long as you don't have a lot of parts already.

Generally speaking though, used laptops and other used hardware will be better value than single board computers. I'm playing with the Futurebit Apollo's built-in Orange Pi 4 at the moment, with 6 ARM cores and whatnot and it feels slower than my '$50 node'.

It's worth noting I bought a 1TB SSD and 8GB of RAM which made it an excellent node, with an insane uptime due to its built-in battery.

Regarding the cooling required for RPi4, I ran into some throttling with RPi4 during synchronization and slapped a heatsink, which more or less solved the issue. Active cooling wasn't really necessary in my case. Never really an issue with the previous revisions though (7w vs 5w). It is still painfully slow regardless.

The "painfully slow" part is indeed useful (I'd guess though that it happens in the initial sync, or when it's a couple days behind), but I was more curious if anybody had the (expectedly bad) experience with lower spec Pi (3,2,1, or even Zero).

I had tried with a Raspberry Pi 3B in the past (when it was recent). Even though it was many years ago with smaller IBD size, with a USB HDD, it was running for weeks. At one point there was a power issue and the HDD was corrupted. Reindexing would have taken just as long again, so I had to hook up the HDD to an actual PC for reindexing and finishing IBD and then plug it back into the Raspberry Pi.
So essentially I never finished initial block download on the Pi itself. I don't think this USB power issue is brought up often enough; since it's pretty common, yet there are tons of guides, 3D printed cases and whatnot for Raspberry nodes to be used with USB HDDs.

In the Raspberry Pi 'community', it seems a long and well-known fact that they struggle with USB power.

[DaveF]

n0nce made a nice post about doing a node for under $60. https://bitcointalk.org/index.php?topic=5364742
You can do it for a bit more, you can do it for a bit less.

Don't try to cut that price in 1/2 unless you are technically inclined and can work around the issues. Otherwise you are going to have a miserable time.

I honestly don't see it happening for anything under 50 as long as you don't have a lot of parts already.

That is part of the issue. As more RPi4 come back into stock and other embedded systems get retired I can see a lot of older slower RPi hitting the market.
Either RPi3 or the 1GB RPi4 that people bought during the shortage.
Even now you are seeing posts about people building a pruned node to fit it into a smaller drive because they don't want to spend the money for a used 1TB drive.

So telling people not to try is worth a mention.
Personally I would like to see them get rid of the pruned node option entirely. It's just not worth it.

-Dave

[NeuroticFish]

Personally I would like to see them get rid of the pruned node option entirely. It's just not worth it.

Most people tend to learn from their own mistakes. If somebody tells them, they think they know better.
This being said, people usually find out that pruned nodes don't worth it when they want to try out something and change the wallet.

[LoyceV]

Personally I would like to see them get rid of the pruned node option entirely. It's just not worth it.

Many new laptops are sold with 256 or 512 GB storage. Since SSDs took over HDDs, this hasn't gone up much. That would mean a brand new laptop can barely install Bitcoin Core without pruning, and I don't think that would be a good thing.

[n0nce]

I don't think this USB power issue is brought up often enough; since it's pretty common, yet there are tons of guides, 3D printed cases and whatnot for Raspberry nodes to be used with USB HDDs.

In the Raspberry Pi 'community', it seems a long and well-known fact that they struggle with USB power.

That's good point, although some of the issue caused by poor power supply or USB cable rather than RPi fault.

It's actually the Raspberry Pi's fault.. I'm not sure if it's the case on the latest models, but pretty confident model 3B only supplied max. of 0.9A, which is not enough to power an external 2.5" HDD. Even though these drives are basically made for expanding a laptop's storage, so they're made with portability in mind and reduced power draw compared to a 3.5" HDD, a Pi can't always reliably power it.

Edit: It appears the Model 4 can pull 1.2A. So that may work. But most people who make these Raspberry Pi node guides use SSDs, which draw much less power and remark like 'you can replace it with an HDD to save some money', while not having tried it and not realizing this problem. I noticed this problem way back in Raspberry Pi model 2 era.

Combined with a microSD card as an OS drive (more prone to memory corruptions & limited r/w cycles compared to an SSD or onboard flash), it doesn't make for the most reliable system - you know, something you'd like to run a Bitcoin node on..

[n0nce]

I use Pi 4 and can confirm i can use USB HDD. But AFAIK Pi 3B+ partially solve power problem.

That's good to know! With my 3B+ (last Pi I tried running a node on) and various HDDs, it wouldn't work. It surely also depends on the model of HDD used.

Good point. Although if you could afford 1TB SSD (for blockchain), surely you can afford USB flash drive or another SSD (with lower capacity) for OS.

USB flash drive will be the same as the microSD - not suited as an OS drive - a second SSD would be an option; though then we might get into power issues again, or figure out booting from the one SSD that holds the blockchain. I believe the Pi 4 is the first model that can do that.


I'd point out another issue with SBC nodes: as described in my Futurebit Apollo BTC guide, compilation times on these single board computers are extremely long. Building electrs took three times as long as it did on my $50 node and probably 10 times as long as on a modern desktop PC. Therefore people probably more often resort to prebuilt binaries (like I did in that guide) which I'm not really a fan of. If we already have the source and aim to minimize trust, we should compile everything.

Cross-compilation is a thing, but even trickier to set up and get right, especially for newcomers. That's why I think sticking to old x86 hardware is the easiest and fastest way to get up and running cheaply and reliably. A while back my laptop 'home node' powered through a night-long power outage due to its internal battery. To be fair, a Raspberry Pi in the same network powered back up without issues, but in the past I've had power cuts negatively affect SBCs (i.e. corrupted microSD drive).

[vv181]

Good point. Although if you could afford 1TB SSD (for blockchain), surely you can afford USB flash drive or another SSD (with lower capacity) for OS.

USB flash drive will be the same as the microSD - not suited as an OS drive - a second SSD would be an option; though then we might get into power issues again, or figure out booting from the one SSD that holds the blockchain. I believe the Pi 4 is the first model that can do that.

I was once running the node with SSD(Kingston A400 SATA SSD ) + HDD(My Passport from WD) and it works quite fine. Though if I plug the HDD when the Pi 4 is already on, it will always restart the Pi, so it needed to be plugged in before turning it on. I'm sure it drains all the power as it can get since even if I either connect a USB keyboard or a display, the OS just won't booted up and the Pi keep restarting.

Interestingly enough, the blockchain never got corrupted when it regularly got power outages. But due to the outages keep coming, it only lasts roughly 10 months, and the HDD started to make some noise before it fails.

As of now, I run the node with 2 SSD, and sometimes with a USB keyboard and connected it to a display, it never got any issues.

[NeuroticFish]

1) Yes you can force a RPi2 or 3 to run a node but it is going to be a slow miserable time and you will have lots of issues.

Since I am not good in hardware stuff and since I somewhat eyeing a configuration to move my bitcoin node away from my main computer... how bad is this CPU: Intel Pentium G4400T 2.90GHz?
Would I hate myself if I start using a mini PC with that CPU and 4GB RAM?

[DaveF]

1) Yes you can force a RPi2 or 3 to run a node but it is going to be a slow miserable time and you will have lots of issues.

Since I am not good in hardware stuff and since I somewhat eyeing a configuration to move my bitcoin node away from my main computer... how bad is this CPU: Intel Pentium G4400T 2.90GHz?
Would I hate myself if I start using a mini PC with that CPU and 4GB RAM?

Hate is a strong word. Only 2 cores is very little by today's standards as is 4GB of ram.
Remember it's a 7 year old CPU. A little more RAM for the IBD and an SSD will make it livable.

But, and this is an important but. You are still doing this on what I assume is a 6 or 7 year old system. What is the condition of the rest of the hardware.
Bitcoin sync does push some HW to it's limits, which by itself is not a bad thing. But it will bring what would be smaller issues to the front to become larger issues.

-Dave

[NeuroticFish]

A little more RAM for the IBD and an SSD will make it livable.

IBD is - by far - not an issue, I've already done that with a much more powerful machine, the blockchain is already on the designated HDD.
This 6-7 years old system would just sit running 24/7, hopefully without creating too much useless heat.

Only Electrs (or ElectrumX?) will have to rebuild its indexes (Bitcoin Core will have his own, built on the main PC).
And I hope I'll do this without pre-made (and not as much configurable) things like Umbrel and so on.

[LoyceV]

Since I am not good in hardware stuff and since I somewhat eyeing a configuration to move my bitcoin node away from my main computer... how bad is this CPU: Intel Pentium G4400T 2.90GHz?
Would I hate myself if I start using a mini PC with that CPU and 4GB RAM?

I would add 4 GB (8 GB RAM total), and go for it! With SSD, it should be able to synchronize 450 GB blockchain within a day.

This 6-7 years old system would just sit running 24/7, hopefully without creating too much useless heat.

I wouldn't like the 35W TDP. In summer because of the heat, the rest of the year because electricity prices are outrageous here nowadays.

The laptop I'm using now has a slower CPU (but more RAM). I should really upgrade, but don't like the hassle.

[NeuroticFish]

Thank you both for the input. My conclusion is that since there's no IBD to take care of, the setup is not that bad, although it could use some more RAM.
The only really worrying part may the consumption.

[LoyceV]

The only really worrying part may the consumption.

Measure it
Here, prices easily go up to 50 cent (euro/dollar, it's the same nowadays) per kWh. That means 30W costs 11 euro/dollar per month. If not for the disk space requirements, a VPS could be cheaper (but also more risky for running a node).
Back to the 11 dollar per month: that's 400 dollar in 3 years, and could very well justify buying a much more energy efficient second hand laptop to use as a node.

This got me thinking: how cool would it be to run a node on a phone? Replace Android by Linux, add a 512 GB microSD (around 50 bucks), and login remotely.
Or even without replacing Android: stackexchange mentions installing Bitcoin Core in Termux.
Phones take at most a few watt power, and modern ones have more RAM than low-end laptops. You may want to turn off data

[NeuroticFish]

The only really worrying part may the consumption.

Measure it
Here, prices easily go up to 50 cent (euro/dollar, it's the same nowadays) per kWh. That means 30W costs 11 euro/dollar per month. If not for the disk space requirements, a VPS could be cheaper (but also more risky for running a node).
Back to the 11 dollar per month: that's 400 dollar in 3 years, and could very well justify buying a much more energy efficient second hand laptop to use as a node.

I wish it would be easy to measure. Now the electricity price is subsidized, until next spring at least. There are so many taxes and then reductions I cannot properly measure.
By not-so-strict calculations 4 months ago I paid around 15 cents/kWh, 2 months ago I paid some 7 cents/kWh. This month... I will find out in some 10 days.
Since I expect the 7 cents may be an error, I'll go to a middle 11 cents for my calculation, which (based on yours) means 2.42 EUR/month and 88 EUR for 4 years.
While you did scare me, since I feel like CISC processors tend to be more power hungry than RISC ones (eg ARM), hence a RPi4 would have been better for the job, this setup doesn't look like a bad business after all...
I will do the math again after the next invoice arrives.

This got me thinking: how cool would it be to run a node on a phone? Replace Android by Linux, add a 512 GB microSD (around 50 bucks), and login remotely.
Or even without replacing Android: stackexchange mentions installing Bitcoin Core in Termux.
Phones take at most a few watt power, and modern ones have more RAM than low-end laptops.

microSD may not be best suited for such load. From my experience it tends to heat up and start failing (but maybe the adaptors I've been using were not top quality either).
And heavy duty microSD are overly expensive. Plus the 512 is just barely enough. I also guess that a more powerful smartphone may be more power hungry too, but I may be awfully wrong.
Interesting research though.
[Also, fyi: that SH system I consider buying, with that Intel G proc, 4GB RAM and 1TB HDD is just a tad over 100 EUR].

than low-end laptops. You may want to turn off data

You should have no SIM at all in it. Then no risk to have data
And I would not take into account systems that have own screen. There's usually no reason to buy screen for a machine supposed to run in unattended mode. Remote desktop - in a way or another - should do.
But this is also because I was talking about buying hardware.


Fun fact: before this thread I was considering to play with Bitcoin core on a RPi zero W (32bit), just to see how bad it runs. Afterwards I decided I should not waste my time.

[DaveF]

...
[Also, fyi: that SH system I consider buying, with that Intel G proc, 4GB RAM and 1TB HDD is just a tad over 100 EUR].
...

Fun fact: before this thread I was considering to play with Bitcoin core on a RPi zero W (32bit), just to see how bad it runs. Afterwards I decided I should not waste my time.

A few thoughts.
1st and I don't know how it is where you are but here (NY) there tends to be a 'base' price of working PCs between $70 to $125 which puts them at about what you are paying for about those specs. However, there is a jump to $175 to $200 where they get a lot better.

Smaller then needed SSD but more ram and a faster processor and you can get a 1TB spinning drive for under $50 for the blockchain:
https://www.microcenter.com/product/622158/hp-elitedesk-800-g1-usff-desktop-computer-(refurbished)

How do prices look for stuff like that where you are?

As for the RPi, the 4 is the only one worth running at this point.
https://magpi.raspberrypi.com/articles/raspberry-pi-4-specs-benchmarks

Yes it would probably be fun to tinker with the smaller / slower / less power hungry ones a bit but it's going to take a ton of work to get it to work and then it will not work well.

But still keep in mind, although there is a higher cost (RPI4 + case + SATA adapter + m.2 SATA drive + SD card + wall power adapter) the power pull from the nodes in a box that I like to build is usually a rounding error in most peoples electric use:



https://bitcointalk.org/index.php?topic=5364113

-Dave

[NeuroticFish]

A few thoughts.
1st and I don't know how it is where you are but here (NY) there tends to be a 'base' price of working PCs between $70 to $125 which puts them at about what you are paying for about those specs. However, there is a jump to $175 to $200 where they get a lot better.

Smaller then needed SSD but more ram and a faster processor and you can get a 1TB spinning drive for under $50 for the blockchain:
https://www.microcenter.com/product/622158/hp-elitedesk-800-g1-usff-desktop-computer-(refurbished)

How do prices look for stuff like that where you are?

I don't know that good the prices. I have some sort of feeling that new hardware is cheaper in US and SH is a little bit cheaper here.
With IBD already done I don't find SSD an useful expense, hence I tried to prefer bigger HDD.
I've seen here too i3 and i5 refurb or SH, but they are more expensive and I expect them also be more power hungry. The configuration I've put is almost 110$ and it already has 1 TB HDD on it.

As for the RPi, the 4 is the only one worth running at this point.

Yes, and it's difficult and expensive to get hands on one nowadays. The only one official reseller in my country is out of stock for maybe a year.

Yes it would probably be fun to tinker with the smaller / slower / less power hungry ones a bit but it's going to take a ton of work to get it to work and then it will not work well.

But still keep in mind, although there is a higher cost (RPI4 + case + SATA adapter + m.2 SATA drive + SD card + wall power adapter) the power pull from the nodes in a box that I like to build is usually a rounding error in most peoples electric use

While I expect everybody calculates only the power drew by the Pi, and indeed, there's more components to actually buy (plus if I get a Pi, I get 1TB SSD to keep the system fanless/noiseless).
And yes, I expect to have more to work at it. Still, since I already have a (much smaller) Pi, I started to really like those little buggers 
And over years the difference in power consumption may (wishful thinking) pay the difference in acquisition price.

Also consider Fulcrum[1] which have better performance[2].

I've been eyeing Fulcrum before Electrs. I've been thinking to run it under Windows. But I gave it up. I didn't find anybody really using it, and it also seems to be primarily developed for BCH not BTC. And since those guys tend to say "Bitcoin" when they actually refer to their altcoin, it can get confusing and problematic, so I decided not to risk even more headaches.

Fun fact: before this thread I was considering to play with Bitcoin core on a RPi zero W (32bit), just to see how bad it runs. Afterwards I decided I should not waste my time.

I found someone on reddit doing that[3] and managed to run it with GUI. But obviously some configuration on Bitcoin Core and the OS needed to fit everything on 1/2GB RAM.

Nice find! Still, I don't think that I'll go back to that path. My Pi Zero W has became print server in my home and if I ever get to the point I no longer need that.. it may become the hardware for a SeedSigner.
It is too weak for what Bitcoin Core needs.

[LoyceV]

I wish it would be easy to measure.

There are devices that do this, but usually the surface temperature is a pretty good estimate. If it's not warm, it's not worth worrying about.

Now the electricity price is subsidized, until next spring at least. There are so many taxes and then reductions I cannot properly measure.
By not-so-strict calculations 4 months ago I paid around 15 cents/kWh, 2 months ago I paid some 7 cents/kWh.

Here 2 of the taxes on electricity are also temporarily reduced, but it's still more expensive than ever. I didn't realize until yesterday how "local" high natural gas prices are: apparently European prices are 9 times higher than in the US.

While you did scare me, since I feel like CISC processors tend to be more power hungry than RISC ones (eg ARM), hence a RPi4 would have been better for the job, this setup doesn't look like a bad business after all...

The Raspberry Pi only consumes a few watts indeed.

microSD may not be best suited for such load. From my experience it tends to heat up and start failing (but maybe the adaptors I've been using were not top quality either).

I've broken cards in the past, but that happened by overwriting the entire card many times. The IBD writes all blocks once, and reads a lot of data, but it's only the chainstate directory I would worry about. If you're really going for a phone-setup (again: not easy but very cool), you could symlink chainstate to the phone's own storage.

I also guess that a more powerful smartphone may be more power hungry too, but I may be awfully wrong.

My assumption was the average phone battery holds about 10 Wh. It lasts more than 10 hours (especially with the screen off), so it can't use more than 1W on average.

[Also, fyi: that SH system I consider buying, with that Intel G proc, 4GB RAM and 1TB HDD is just a tad over 100 EUR].

I'd go for a laptop instead.

And I would not take into account systems that have own screen. There's usually no reason to buy screen for a machine supposed to run in unattended mode. Remote desktop - in a way or another - should do.

Somehow I always end up carrying a monitor for trouble shooting. Again:

I'm a sucker for laptops nowadays

Ideally one with DVD drive (to replace for a HDD).

you can get a 1TB spinning drive for under $50

I only recently found out that many well known harddrive brands use something called SMR (Shingled Magnetic Recording):

Western Digital, Toshiba, and Seagate have sold SMR drives without labeling them as such, generating a large controversy, as SMR drives behave much more slowly under some circumstances (such as random writes)

My 2 TB HDD has this problem, which means that writing a lot of data leads to terrible performance. I've seen it down to writing just kilobytes per second:

in order to write a single byte, it might have to rewrite multiple gigabytes.

In "normal usage patterns" (as designated so by HDD vendors, not by users!) this creates not much of a problem - the data is written to a CMR cache on the outer rim of the disk. Later, when disk usage goes down, the firmware will move the date to its final place in an SMR band.

When writing larger quantities of data at a time, this CMR cache is exhausted and the process of I/O to SMR bands has to take over - this is slower by orders of magnitude.

Since I now know about it, I'd never buy an SMR disk again.

[NeuroticFish]

I've broken cards in the past, but that happened by overwriting the entire card many times. The IBD writes all blocks once, and reads a lot of data, but it's only the chainstate directory I would worry about. If you're really going for a phone-setup (again: not easy but very cool), you could symlink chainstate to the phone's own storage.

I didn't break the cards, but the data was lost. Symlink is, however, a cool idea I agree with. (Also having Bitcoin core on a smartphone sounds overly cool!).

My assumption was the average phone battery holds about 10 Wh. It lasts more than 10 hours (especially with the screen off), so it can't use more than 1W on average.

From what I know it depends greatly on the usage. It's not the same if it sits in the pocket doing almost nothing or you play a game. It's not the same even if you have a lot of widgets refreshing all the time from the internet or not having them.
And I expect Bitcoin core will use some of that CPU...

[Also, fyi: that SH system I consider buying, with that Intel G proc, 4GB RAM and 1TB HDD is just a tad over 100 EUR].

I'd go for a laptop instead.

You really make it hard for me to decide 

And I would not take into account systems that have own screen. There's usually no reason to buy screen for a machine supposed to run in unattended mode. Remote desktop - in a way or another - should do.

Somehow I always end up carrying a monitor for trouble shooting.

Of course, but I already have monitor for that.

Since I now know about it, I'd never buy an SMR disk again.

Indeed, lately when I look for HDDs I also check for this (of course, in case of SH I don't get much detail, however, read below).


Another direction which you may like more: I may be able to get my hands for free onto an ASUS Pro57V (CPU = Intel T5800, 4GB RAM) and then I'll just buy a proper external HDD and insert it into this baby (which now has an old school 500GB WD Black in it). What you say? [PS. I will need tips on how to turn off and on the screen of that ASUS laptop]

[LoyceV]

I didn't break the cards, but the data was lost.

In that case, I would indeed blame the adapter. I've had several shitty microSD to USB adapters that disconnect when you look at them. I assume you didn't have this problem with the card in a phone, right?

From what I know it depends greatly on the usage. It's not the same if it sits in the pocket doing almost nothing or you play a game.

Agreed. But even when playing a game (with the screen on) it lasts many hours if not the whole day. So keeping up with 1 block every 10 minutes shouldn't consume a lot of power.

I'd go for a laptop instead.

You really make it hard for me to decide 

How about this: get a damaged laptop! They're awfully cheap, and for the purpose of a node it doesn't matter if the screen fell off.

Of course, but I already have monitor for that.

I have them, but it's annoying to hook up (especially with proper cable management that doesn't allow for quick disconnecting). Maybe a laptop with broken screen isn't the best idea after all

Another direction which you may like more: I may be able to get my hands for free onto an ASUS Pro57V (CPU = Intel T5800, 4GB RAM) and then I'll just buy a proper external HDD and insert it into this baby (which now has an old school 500GB WD Black in it). What you say?

It's older than my current laptop, and this one is ancient already, so I wouldn't expect too much from it. But it will work (at 35W tdp).
Why an external HDD, instead of replacing the internal disk?

[PS. I will need tips on how to turn off and on the screen of that ASUS laptop]

On: press a button.
Off: Try xset dpms force off. Or easier: close the lid, usually that turns off the screen.

[NeuroticFish]

In that case, I would indeed blame the adapter. I've had several shitty microSD to USB adapters that disconnect when you look at them. I assume you didn't have this problem with the card in a phone, right?

It has happened with USB adaptors, not in the phone, although one of my phones has odd restarts for some months and a new microSD has cured it, but there may have been other reasons back then.

How about this: get a damaged laptop! They're awfully cheap, and for the purpose of a node it doesn't matter if the screen fell off.

No. Sorry. I just don't have the connections to look for such good deals.

Of course, but I already have monitor for that.

I have them, but it's annoying to hook up (especially with proper cable management that doesn't allow for quick disconnecting). Maybe a laptop with broken screen isn't the best idea after all

LOL!

Why an external HDD, instead of replacing the internal disk?

One important reason is that I am not sure I'll be able to put it correctly back in place after opening it.
Another one may be that it looks like there aren't big enough CMR laptop HDDs available.

[PS. I will need tips on how to turn off and on the screen of that ASUS laptop]

On: press a button.
Off: Try xset dpms force off. Or easier: close the lid, usually that turns off the screen.

Thank you. Under Windows closing the lid tends to get the computer into sleep mode, which is not what I want. If Linux shuts the screen off for me at closing the lid, that's just perfect.
I'll write down that command too just in case.