Umbrel — Discussion, issues, solutions

[BlackHatCoiner]

I shall open a general thread about Umbrel, which is an implementation of an operating system that runs Bitcoin & Lightning Network node. You simply install it on your Raspberry Pi and enjoy the benefits of a personal node. Your node runs through Tor (as I've noticed it's mandatory) and can also been administrated via WAN (not just LAN!) if you enter your onion url wherever you are.

Other, similar, implementations of such OS are myNode, RaspiBolt and RaspiBlitz.


So, I'm facing a syncing problem. It takes way too long than what I expected. I set it up yesterday (24 hours must have passed) and it's still on 14.48%. Is this normal or the verification of the blocks takes more than it should? Is there any way I could speed it up? Like verifying the rest of the blocks in my PC which should be faster than with Raspberry and then plugging my external disk back to it again once it's done.

I have:

• 1 TB SSD external disk. (is this enough for both Bitcoin Core and Electrum?)
• Internet speed: ~100 Mbits/sec.
• 32GB micro SD card.

[ranochigo]

So, I'm facing a syncing problem. It takes way too long than what I expected. I set it up yesterday (24 hours must have passed) and it's still on 14.48%. Is this normal or the verification of the blocks takes more than it should?

It's normal. Raspberry Pi is notoriously slow and the record for me was 5 days with a fairly fast SSD and a RPi 4, can take up to weeks depending on which RPi you have. The CPU, the RAM and USB (again depending on your Pi) are bottlenecks.

Is there any way I could speed it up? Like verifying the rest of the blocks in my PC which should be faster than with Raspberry and then plugging my external disk back to it again once it's done.

That's what I did with my subsequent synchronization after it got corrupted a few times. Either that or just copy the entire dir of a copy that you've already fully synchronized (if you have).

[BlackHatCoiner]

It's normal. Raspberry Pi is notoriously slow and the record for me was 5 days with a fairly fast SSD and a RPi 4, can take up to weeks depending on which RPi you have.

I have RPi 4 so it'll probably take a century less.

The CPU, the RAM and USB (again depending on your Pi) are bottlenecks.

Indeed. In order to test all those OSes you'll have to wait for a month, although I may stay with Umbrel as it seems better than myNode (that I had tried). The syncing didn't take that much on that OS, though. It finished in ~2.5 days. Does this have to do with the mandate usage of Tor? (without Tor it should sync faster)

That's what I did with my subsequent synchronization after it got corrupted a few times.

It got corrupted? Why? Don't I just plug my SSD to the PC's usb port, start the daemon using bitcoind --datadir=<...> and safely shut it down with bitcoin-cli --datadir=<...> stop?

[ranochigo]

Indeed. In order to test all those OSes you'll have to wait for a month, although I may stay with Umbrel as it seems better than myNode (that I had tried). The syncing didn't take that much on that OS, though. It finished in ~2.5 days. Does this have to do with the mandate usage of Tor? (without Tor it should sync faster)

Probably not. The only way to check is by looking at the resource usage, htop should show you how much ram is getting utilized and the CPU as well. I've never really had a bottleneck while using Tor, or at least it isn't that obvious to me at all.

It got corrupted? Why? Don't I just plug my SSD to the PC's usb port, start the daemon using bitcoind --datadir=<...> and safely shut it down with bitcoin-cli --datadir=<...> stop?

As in my own RPi got corrupted due to an unclean shutdown. It has nothing to do with me synchronizing using my computer.

[BlackHatCoiner]

As in my own RPi got corrupted due to an unclean shutdown.

Like a power outage? What happens then? Will you wait for another week to sync?

[ranochigo]

Like a power outage? What happens then? Will you wait for another week to sync?

Yes. I rebuilt the block data by plugging the SSD in my computer and treating it as an external storage. It would be faster to simply copy another uncorrupted block dir onto the SSD though.

[BlackHatCoiner]

Yes. I rebuilt the block data by plugging the SSD in my computer and treating it as an external storage. It would be faster to simply copy another uncorrupted block dir onto the SSD though.

Are you sure there isn't a smarter solution in case of an unclean shutdown? Can't I just delete the last block in my data and retry with bitcoind?

[odolvlobo]

So, I'm facing a syncing problem. It takes way too long than what I expected. I set it up yesterday (24 hours must have passed) and it's still on 14.48%. Is this normal or the verification of the blocks takes more than it should? Is there any way I could speed it up? Like verifying the rest of the blocks in my PC which should be faster than with Raspberry and then plugging my external disk back to it again once it's done.

I set up a RPi node with a hard disk (not SSD) and TOR, and it took about a week to do the initial block download.

I have a UPS because power outages are common where I live.

[HCP]

So, I'm facing a syncing problem. It takes way too long than what I expected. I set it up yesterday (24 hours must have passed) and it's still on 14.48%. Is this normal or the verification of the blocks takes more than it should?

As mentioned... IBD on a Pi can be torturously slow...

Is there any way I could speed it up? Like verifying the rest of the blocks in my PC which should be faster than with Raspberry and then plugging my external disk back to it again once it's done.

If your PC is Linux based and/or can read EXT4 formatted drives... then yes. From memory I was actually able to do this using Ubuntu running under "WSL" (Windows Subsystem for Linux)... the trick is to remember to modify the directory and file permissions after you copy them to make sure Umbrel can read them. User/Group 1000:1000 from memory.


If your PC can't read EXT4 formatted drives, you can use ssh/scp and copy the blockchain data from your PC to your external drive connected to the Pi (I have also successfully transferred the blocks this way).

Are you sure there isn't a smarter solution in case of an unclean shutdown? Can't I just delete the last block in my data and retry with bitcoind?

You'd probably need to delete the corrupted block and then do a reindex... that should work.

[BlackHatCoiner]

Is there any way I can verify the blocks with my GPU instead? Both RPi's and my PC's CPUs are very slow and it'll take ages to finish. At the moment it's 18.62%. Also, in Umbrel I don't have many connections with other (possibly) Tor nodes which means that I may also have a slow downloading, too. Right now, I'm connected with one Bitcoin node and it sometimes, becomes, zero.

I set up a RPi node with a hard disk (not SSD) and TOR, and it took about a week to do the initial block download.

Hmm, that means around 14% per day. So I'm in a much worse situation. I have to do something (maybe using another Bitcoin node OS?) otherwise, it may take more than two months.

You'd probably need to delete the corrupted block and then do a reindex... that should work.

Does reindexing mean re-verifying each block?

[BlackHatCoiner]

Question: Will my electrum server be personal or public? I want to provide in the peer discovery of the electrum servers (that run over Tor).

[HCP]

Does reindexing mean re-verifying each block?

Essentially, yes... The short answer is that it starts at the beginning and verifies the block data stored on disk. If an invalid block/file is encountered, it will start syncing (downloading blocks from peers) from that point to ensure the validity of all the blocks.

Question: Will my electrum server be personal or public? I want to provide in the peer discovery of the electrum servers (that run over Tor).

I'm not sure if the .onion servers are included in peer discovery or if you have to explicitly set them. The Electrum docs regarding Tor seems to indicate that you have to either set up as connecting to 1 server (ie. .onion only) or you run the risk of actually connecting to clearnet peers for blockheader download:

Thus the two different options are, Connect to 1 server ONLY and get block headers and transaction info from that server. Or Connect to 8 block header servers and connect to 1 .onion server for the general use.

However, when I look through the list of "known" servers... I so see a couple of .onion servers listed:


So, it's possible that .onion nodes are included in peer discovery.


Note: Setting Electrum to use my Umbrel node (from a remote location)... it is connecting to my .onion address, but the other connected nodes are all clearnet.

[BlackHatCoiner]

So, I deleted Umbrel, burnt an Ubuntu OS to the SD card, synced with Bitcoin Core until ~450,000 and then, due to some issues, moved back to Umbrel. I forgot to add txindex=1, so I had to reindex it. I haven't done so far, because it requires much time. I copied, though, all my block files (directories: blocks, chainstate, indexes) to the Umbrel's bitcoin directory and I wanted from Umbrel to index them for me.

While it does read it that I'm using 151 GBs out of 1 TB, I don't understand if it resyncs or just re-indexes.

That 0.13% scares me.

However, when I look through the list of "known" servers... I so see a couple of .onion servers listed:

Umbrel runs electrs so I guess it should allow others to connect with my .onion url.

[HCP]

While it does read it that I'm using 151 GBs out of 1 TB, I don't understand if it resyncs or just re-indexes.

Have you tried connecting to the Pi using ssh/terminal and checking what the Bitcoin Core logs say?

If you make sure you're in /home/umbrel/umbrel and then use:

Code:

sudo docker-compose logs -f --tail=100 bitcoin

That'll show you the last 100 lines of the debug.log file and then continue to output new lines (ctrl+c to stop). That should hopefully give you some idea of what it is actually doing.


Alternatively you should be able to check the network usage of the device to see if it is actually downloading blocks.

[cygan]

i now also have an umbrel full node running


https://twitter.com/cryptocygan/status/1492930549861826563

the blockchain is now completely synchronized but somehow my node does not get more than 11 inbound connections (peers) ... port 8333 is enabled in my router.
or is it just the outbound connections that umbrel shows me?

[DaveF]

Mine is smaller:
https://bitcointalk.org/index.php?topic=5364113

Is that a RPi in there? I am on mobile and I can't tell if it's that or something else.
I do like Umbrel, but as I have pointed out the 'app store' thing allowing it to do much more then BTC makes me worry about the security of it.
We know what the security risks are for a lot of the crypto apps, but what if you installed Pi-Hole and someone found and exploited something with that....

-Dave

[cygan]

Mine is smaller:
https://bitcointalk.org/index.php?topic=5364113

Is that a RPi in there? I am on mobile and I can't tell if it's that or something else.
✂

yes, there are even 2 rpi 4 built in and installed
but again back to my actual question ... how can i see how many inbound connections my umbrel node has?

✂
I do like Umbrel, but as I have pointed out the 'app store' thing allowing it to do much more then BTC makes me worry about the security of it.
We know what the security risks are for a lot of the crypto apps, but what if you installed Pi-Hole and someone found and exploited something with that....

-Dave

the app thing is such a question i share with you - actually you take your own risk there if you would install additional apps here...
and then immediately another question about the apps - has anyone here already installed any apps on the umbrel node and can recommend which ones here?

[vv181]

the blockchain is now completely synchronized but somehow my node does not get more than 11 inbound connections (peers) ... port 8333 is enabled in my router.
or is it just the outbound connections that umbrel shows me?

but again back to my actual question ... how can i see how many inbound connections my umbrel node has?

I don't use Umbrel but looking at their code^[1], the number on the dashboard shows the total connection of your node received(inbound+outbound). So you have to either check on some explorer(BTC RPC explorer) or directly SSH'ing into the server. Anyway, isn't Umbrel default network are using Tor?

[cygan]

the blockchain is now completely synchronized but somehow my node does not get more than 11 inbound connections (peers) ... port 8333 is enabled in my router.
or is it just the outbound connections that umbrel shows me?

but again back to my actual question ... how can i see how many inbound connections my umbrel node has?

I don't use Umbrel but looking at their code^[1], the number on the dashboard shows the total connection of your node received(inbound+outbound). So you have to either check on some explorer(BTC RPC explorer) or directly SSH'ing into the server. Anyway, isn't Umbrel default network are using Tor?

what is the best way to do this?
i am a bit inexperienced in this respect...

[vv181]

the blockchain is now completely synchronized but somehow my node does not get more than 11 inbound connections (peers) ... port 8333 is enabled in my router.
or is it just the outbound connections that umbrel shows me?

but again back to my actual question ... how can i see how many inbound connections my umbrel node has?

I don't use Umbrel but looking at their code^[1], the number on the dashboard shows the total connection of your node received(inbound+outbound). So you have to either check on some explorer(BTC RPC explorer) or directly SSH'ing into the server. Anyway, isn't Umbrel default network are using Tor?

what is the best way to do this?
i am a bit inexperienced in this respect...

You can try SSH into the server.

Use Powershell if you are on Windows or if you are running on macOS/Linux you can directly type below command on the terminal.

Code:

ssh umbrel@umbrel.local

Then enter your password.

Password: The same password you use to log in to the Umbrel dashboard

If you haven't yet run through the setup process, the password will be set to moneyprintergobrrr.

After you've connected to the node, then proceed to see the network info by typing this on the terminal:

Code:

bitcoin-cli getnetworkinfo
#or
~/umbrel/bin/bitcoin-cli getnetworkinfo

And then, the output should show you how many connections you got.
e.g.

Code:

 ~
"connections": xx,
  "connections_in": xx,
  "connections_out": xx,
~