LoyceV's 0.1 sat/vbyte Electrum Server Adventure

[DireWolfM14]

Any server suggestions?

Lol, don't be like me?   

My current system is a HP Proliant DL360 Gen7 with 12 cores, 96GB of ram, a 128GB SSD for the OS and programs, and three 2TB SSDs in a raid5 array for the data.  There was a time when those HP systems flooded Ebay so the were dirt cheap, and I'm a sucker for a bargain.

[joker_josue]

The biggest difficulty I've had in my research is finding a server with enough disk space so I don't have to worry about it.
This was the one I found that was at a minimally affordable price. Otherwise, everything I've found is very expensive.

A hybrid server may work: Bitcoin Core's blocks directory can be on a cheap HDD, as long as the rest is on SSD. Mine is on slow "SAN" network storage, and after the initial sync that isn't much of a limitation.

Is this mode feasible to use the block explorer?

My current system is a HP Proliant DL360 Gen7 with 12 cores, 96GB of ram, a 128GB SSD for the OS and programs, and three 2TB SSDs in a raid5 array for the data.  There was a time when those HP systems flooded Ebay so the were dirt cheap, and I'm a sucker for a bargain.

I've also thought about buying a second-hand server (I even find it at interesting prices), but to make it publicly accessible - for some projects, it can be more complicated in terms of stability and internal security.

[LoyceV]

Is this mode feasible to use the block explorer?

I've never installed my own block explorer. I assume it would create it's own database as it can't search Bitcoin Core's blocks directly, so in that case I assume it still needs Bitcoin Core's blocks directory only once, and can have it's own database on faster storage.

I've also thought about buying a second-hand server (I even find it at interesting prices), but to make it publicly accessible - for some projects, it can be more complicated in terms of stability and internal security.

I wouldn't self-host anything Bitcoin related from my home.

[DireWolfM14]

Is this mode feasible to use the block explorer?

I've never installed my own block explorer. I assume it would create it's own database as it can't search Bitcoin Core's blocks directly, so in that case I assume it still needs Bitcoin Core's blocks directory only once, and can have it's own database on faster storage.

I've only used mempool as a block explorer, and it's very low impact.  It draws data from Bitcoind and an Electrum server as it runs.  The production version of mempool (mempool.space, for example) requires Blockstream Electrs.  I've only run the personal version myself, with ElectrumX and Romanz Electrs as the backend.  I can't imagine it not being compatible with Fulcrum.  It does use it's own database structure (maria,) but as far as I know it doesn't store data in an exclusive database so you wouldn't be able to run it without at least having Bitcoind running.

I've also thought about buying a second-hand server (I even find it at interesting prices), but to make it publicly accessible - for some projects, it can be more complicated in terms of stability and internal security.

I wouldn't self-host anything Bitcoin related from my home.

I wouldn't want my home IP open to the public either, but I was wondering if a remote web server with a WirdGuard client and local server hardened with nftables would be a safe way to have specific local services/ports accessible from the remotely hosted domain name.  I'm only just starting to learn about using nftables for security, so I wouldn't risk it with my current knowledge.

[joker_josue]

I've also thought about buying a second-hand server (I even find it at interesting prices), but to make it publicly accessible - for some projects, it can be more complicated in terms of stability and internal security.

I wouldn't self-host anything Bitcoin related from my home.

I wouldn't want my home IP open to the public either, but I was wondering if a remote web server with a WirdGuard client and local server hardened with nftables would be a safe way to have specific local services/ports accessible from the remotely hosted domain name.  I'm only just starting to learn about using nftables for security, so I wouldn't risk it with my current knowledge.

I've already explored this topic a bit. Well, once configured, it's possible to isolate the network to protect home equipment from the outside world.

But I think the biggest problem, besides ensuring a good initial network setup, is its maintenance. It requires the person to always be very attentive to what is happening on the network, to check for updates, and to keep security levels at the highest possible level. Sometimes an update can ruin a configuration, which needs to be completely revised to maintain the desired levels.

So it's about stability. Ensuring it's 100% online, that it's not under-loaded. We are not 100% home to guarantee this supervision. There may be power outages, and other external factors of that nature, that can affect the entire structure, even if it is properly constructed.

Impossible? No, it's not, but it requires a very high level of "work" to ensure that everything is always functioning properly.
Ultimately, the question remains: is all this work and risk worthwhile, just to save money on a server that guarantees all of this?

[LoyceV]

I wouldn't want my home IP open to the public either, but I was wondering if a remote web server with a WirdGuard client and local server hardened with nftables would be a safe way to have specific local services/ports accessible from the remotely hosted domain name.  I'm only just starting to learn about using nftables for security, so I wouldn't risk it with my current knowledge.

I wouldn't risk it either. You'd be just one mistake away from compromising your home IP address. I've thought about how nice would it be to ship a cheap old laptop, and place it somewhere in a data center until it breaks down, but that's not very realistic.

[DireWolfM14]

I've thought about how nice would it be to ship a cheap old laptop, and place it somewhere in a data center until it breaks down, but that's not very realistic.

That would be cool, but I don't know if such a service is a practical business model.  

I would love to host a public SPV server, but the cost is prohibitive and I don't have any other services to offer that would offset the cost.  Most of the big ones I see are hosted by the big block exporters that earn from advertising, and some by exchanges.  Some are hosted by developers, such as Emzy but I suspect he's sponsored or partially compensated due to his involvement in Bisq.

[DireWolfM14]

I want to upgrade to this offer, but they've been out of stock for a while:

Code:

4x FAT Slice 2	2 Dedicated Cores	8GB	500GB

That does look like a sweet deal.  There's also this one for $10/month, and it's currently "in stock":

KVM FAT Slice 4
CPU - 4 cores Dedicated
RAM - 16 GB
NVMe - 1000 GB Disk
Bandwidth - 250 mbps Unlimited (or)
10gbps upto 16TB then 10mbps unlimited
1 IPv4 Included
Server Location Montreal, Canada
Support 24*7

Or maybe 6x And some SAN storage. That way I'll have more NVMe storage. This should be in stock again next month.

Do you mean you can pay for multiples and "splice" them together?  For example, if I purchased two the above I could have 8 cores, 32GB of ram, and 2TB storage all the same server?

[joker_josue]

Do you mean you can pay for multiples and "splice" them together?  For example, if I purchased two the above I could have 8 cores, 32GB of ram, and 2TB storage all the same server?

It didn't make sense to buy two of those units. It would be more expensive ($20/month).
If you choose the KVM FAT Slice 8 option, it's double the size and costs $18.

KVM FAT Slice 8
CPU - 8 cores Dedicated
RAM - 32 GB
NVMe - 2000 GB Disk
Bandwidth - 250 mbps Unlimited (or)
10gbps upto 32TB then 10mbps unlimited
1 IPv4 Included
Server Location Montreal, Canada
Support 24*7

[LoyceV]

Do you mean you can pay for multiples and "splice" them together?

If you click their link all plans are here, you get more options. When I got mine, it had a drop-down menu in which I could select how many "slices" I wanted.

Note that I'm not really recommending them, but I've used them for about 14 months. Even though it shouldn't, it does CPU steal up to 60% sometimes. And one time, my server was turned off for no reason, so I had to turn it on again. Support couldn't tell me why that happened.
I guess that's what comes with budget deals: as long as it works, it works. But it's not really meant for "production". I like the price though, I could no doubt get a more reliable server if I pay 10 times more, but that's not worth it for me.

[DireWolfM14]

KVM FAT Slice 8
CPU - 8 cores Dedicated
RAM - 32 GB
NVMe - 2000 GB Disk
Bandwidth - 250 mbps Unlimited (or)
10gbps upto 32TB then 10mbps unlimited
1 IPv4 Included
Server Location Montreal, Canada
Support 24*7

I pulled the trigger.  They give you 10% discount if you pay for the annual plan, so it ends up being $197.76 for a year, or $16.48 per month.  Not bad for all that you get when compared to the big boys.

Note that I'm not really recommending them, but I've used them for about 14 months. Even though it shouldn't, it does CPU steal up to 60% sometimes. And one time, my server was turned off for no reason, so I had to turn it on again. Support couldn't tell me why that happened.
I guess that's what comes with budget deals: as long as it works, it works. But it's not really meant for "production". I like the price though, I could no doubt get a more reliable server if I pay 10 times more, but that's not worth it for me.

Yeah, not great for a high production environment, but now if I want to create a hobbyist public SPV server to share with the Bitcointalk community, at least it's not costing me an arm and a leg.  This is something I've been wanting to do for a while, so stay tuned.

[joker_josue]

Yeah, not great for a high production environment, but now if I want to create a hobbyist public SPV server to share with the Bitcointalk community, at least it's not costing me an arm and a leg.  This is something I've been wanting to do for a while, so stay tuned.

I will stay. 

I'm thinking of doing something similar, but with a different scope. I just need a little more time to finish the project.

[DireWolfM14]

Just a brief update; my VPS is almost ready for prime time.  Perhaps I'll have time to publish the IP and Tor links this weekend. 

Based on the earlier discussion in this thread about the SPV server choices, I too decided to go with Fulcrum for this project.  This is my first experience with Fulcrum, all my previous servers were built with either ElectrumX or Romanz Electrs.  Fulcrum is 92% synchronized as of this morning and seems to be synchronizing at about the same rate as ElectrumX would, which makes sense since it's a fork or ElectrumX under the hood.

I haven't decided if I should publish/open the ports for bitcoind.  I currently have the ports on another server open for myself to use for Bisq and Sparrow on my laptop when traveling, and thought it would be convenient to switch to this server for that purpose.  It occurred to me that others might find it convenient as well for those reasons.  What do you guys think?

[Cricktor]

...

Nice, you likely won't regret your choice to go with Fulcrum. If you ever deal with larger address histories, you will appreciate the speed difference of Fulcrum.

What ports for bitcoind are you speaking of? I wouldn't want e.g. to expose the RPC interface to the public (not sure if you meant these ports).

[LoyceV]

If you ever deal with larger address histories, you will appreciate the speed difference of Fulcrum.

Let's test this: I took 1000 random addresses (taken from List of all Bitcoin addresses with a balance), and loaded them watch-only in Electrum. This may be a stretch, I have no idea how many transactions those addresses made. It could be thousands, it might be millions. Fulcrum is slowly eating more memory while loading, Electrum is Synchronizing... (8582/20844) and counting. To be updated
Note that my server doesn't have the fastest CPUs on the market

Update: after 5 minutes, Electrum is syncing at (16547/20844). Fulcrum's CPU consumption varies between 6 and 125%, python3 on my own system goes from 30 to 60%. I'm not sure what the bottleneck is: maybe disk access on my server? Or maybe Electrum itself gets exponentially more inefficient when loading many addresses?

Update: after Electrum was unresponsive for a few minutes, and python3 consumed 100% CPU and 28% memory on my desktop, it completed syncing. For whoever wants to reproduce this experiment for performance testing: Balance: 373 BTC looks nice This list of addresses has 9422 transactions and contains 4024 unspent transaction outputs. The wallet is 514 MB in size. It took about 15 minutes from the start until I closed Electrum. Note that python3 doesn't release that memory until you close the last open Electrum instance.

[nc50lc]

The wallet is 514 MB in size. It took about 15 minutes from the start until I closed Electrum.

That took a while, but certainly fast considering it's Electrum.

I'm not sure what the bottleneck is: maybe disk access on my server? Or maybe Electrum itself gets exponentially more inefficient when loading many addresses?

Sometimes, it's also the other operations that the client has to do with those transaction like their fiat-equivalent values and historical rates.
Not noticeable at small number of transactions, but a resource-hog at peak.
If you have any of it enabled in the settings, you could try another benchmark with it disabled.

[joker_josue]

~

Are you able to perform a test to see the addresses associated with 5 hops from another address?
I've been wondering how long it might take for him to generate that list, so to speak.

[LoyceV]

If you have any of it enabled in the settings, you could try another benchmark with it disabled.

I'm done testing for now, but I'm more curious why less than 10k transactions lead to a 500 MB wallet. That's 50 kB added per transaction on average. But again, I'm not curious enough to redo it and check the file

Are you able to perform a test to see the addresses associated with 5 hops from another address?

I'm not sure what you mean....

[joker_josue]

Are you able to perform a test to see the addresses associated with 5 hops from another address?

I'm not sure what you mean....

Let's say I want to know the addresses from which the satellites of a specific address came. Up to 5 hops back.

Final Address
1hop: Address A + Address B
2hop: Address A = Address C + D & Address B = Address E + F
3hop: Address C = Address G + H & Address D = Address I + J & Address E = K & Address F = L + M + N
etc... etc...

Did I manage to explain the idea?