Bitcoin + Electrum server + Block explorer under Windows (with WSL and Debian)

[HCP]

Because I'm an idiot with bad time management... I decided that I could spare some time getting WSL and electrs up and running again... but I seem to be stuck getting electrs to start up

I don't remember having to do anything too special to make it work... but when I attempt to start electrs I get:

Code:

hcp@HCP-PC:~/electrs$ ./target/release/electrs
Starting electrs 0.9.6 on x86_64 linux with Config { network: Bitcoin, db_path: "/home/hcp/electrs/bitcoin", daemon_dir: "/mnt/e/Bitcoin", daemon_auth: CookieFile("/mnt/e/Bitcoin/.cookie"), daemon_rpc_addr: 127.0.0.1:8332, daemon_p2p_addr: 127.0.0.1:8333, electrum_rpc_addr: 127.0.0.1:50001, monitoring_addr: 127.0.0.1:4224, wait_duration: 10s, jsonrpc_timeout: 15s, index_batch_size: 10, index_lookup_limit: None, reindex_last_blocks: 0, auto_reindex: true, ignore_mempool: false, sync_once: false, disable_electrum_rpc: false, server_banner: "Welcome to electrs 0.9.6 (Electrum Rust Server)!", args: [] }
[2022-03-30T23:47:35.040Z INFO  electrs::metrics::metrics_impl] serving Prometheus metrics on 127.0.0.1:4224
[2022-03-30T23:47:35.040Z INFO  electrs::server] serving Electrum RPC on 127.0.0.1:50001
[2022-03-30T23:47:35.053Z INFO  electrs::db] "/home/hcp/electrs/bitcoin": 9 SST files, 0.000007245 GB, 0.000000009 Grows
[2022-03-30T23:47:35.056Z INFO  electrs::db] closing DB at /home/hcp/electrs/bitcoin
Error: electrs failed

Caused by:
    0: bitcoind RPC polling failed
    1: daemon not available
    2: JSON-RPC error: transport error: Couldn't connect to host: Connection refused (os error 111)
hcp@HCP-PC:~/electrs$

Really not sure what is causing the issue... any one else come across this?

[NeuroticFish]

Code:

Error: electrs failed

Caused by:
    0: bitcoind RPC polling failed
    1: daemon not available
    2: JSON-RPC error: transport error: Couldn't connect to host: Connection refused (os error 111)
hcp@HCP-PC:~/electrs$

Really not sure what is causing the issue... any one else come across this?

I can easily reproduce this when bitcoin core is running, but it's not synced.
So I'd start with getting full sync.

If that's not the cause, maybe take a look at rpcbind or rpcallowip in bitcoin.conf...

[NeuroticFish]

Maintenance: updating Electrs

Now and then it makes sense to update the server, since fixes and improvements are being added to the newer versions.
The details for updating are pretty much described at: https://github.com/romanz/electrs/blob/master/doc/upgrading.md still, I thought it won't hurt to have also a newbie friendly one here, focused on what we've installed.

Version
If you start your local electrs it will show its version. Mine is now 0.9.3 (yeah, I've done at some point in the past another/newer install).
In any git page of the project (like https://github.com/romanz/electrs ) there's a combobox telling "master". If you click it, and click tags, you'll see all the versions; more important, you'll see which is the latest one. In my case it's 0.9.6, so I'll update to that.

Update OS
Well, this should be run more often than updaing electrs, however, here we are:

Code:

sudo apt update
sudo apt upgrade

Optional prerequisites
Although it's not a required step to verify the signature, I will do that.
However, for that to work, we have to make sure our pgp "knows" the signature of the developer.

Code:

curl https://romanzey.de/pgp.txt
curl -o romanz.pgp https://romanzey.de/pgp.txt
gpg --import romanz.pgp

It should tell: "imported: 1".

Update Electrs
Since we have manual install, that path is what I'll follow.

Keep in mind that this code is for version 0.9.6; if you'll find newer versions on git, please adapt to that one.

Code:

cd ~
cd electrs
git checkout master
git pull

Now there's the optional step of verifying what we get. If you didn't import the pgp key, you will probably also skip this.

Code:

git verify-tag v0.9.6

It should tell things like "good signature" and "Primary key fingerprint: 15C8 C357 4AE4 F1E2 5F3F  35C5 87CA E5FA 4691 7CBB"
It may also give a warning that it's not the most trusted signature, but it's fine.

Let's proceed:

Code:

git checkout v0.9.6
ROCKSDB_INCLUDE_DIR=/usr/include ROCKSDB_LIB_DIR=/usr/lib cargo build --locked --release

If, back at install time, your install of librocksdb-dev has failed and have used a workaround, you'll have to look in git and adapt this last command (probably go for static linked path).

And we're done! If you also have the sh file I've created here, then:

Code:

cd ~
./run_electrs.sh

[NeuroticFish]

Maintenance: updating BTC RPC Explorer
Warning: it was not the nicest and cleanest update, but it works. You may want to read the warnings at the end.

Unlike Electrs, I didn't find any clear tutorial on how to update BTC RPC Explorer. But it's not difficult either.

Version
When you start your local BTC RPC Explorer it will show, at very start, its version. Or it can be seen at the bottom of web page. Mine is now 3.2.0.
In the git page of the project ( https://github.com/janoside/btc-rpc-explorer ) there's a combobox telling "master". If you click it, and click tags, you'll see all the versions; more important, you'll see which is the latest one. Now it shows 3.3.0, so I'll update to that.

Update OS
Well, this should be run more often and you may want to skip it if you've just done that when updating Electrs.

Code:

sudo apt update
sudo apt upgrade

Now.. let's go:

Code:

cd btc-rpc-explorer
git checkout master
git pull

Here I was told that I have local changes. If you don't, skip this part.

Code:

git reset --hard
git pull

Back on track; keep in mind that unlike Electrs, here I couldn't find a way to verify.

Code:

git checkout v3.3.0
npm install

It gave multiple warnings, but it seems to be installed (I've checked it and it works, so maybe you'll skip the fix warnings part.)

The Warnings
As I wrote, although it's well updated, it gave me warnings. Amongst those, it asked me to run

Code:

npm audit fix

and it didn't do much. So the next step I've done was the more risky (allegedly it can break things!)

Code:

npm audit fix --force

and it looked much better. I ran again, for a test

Code:

npm install

and seems happier now. Yay!

...It also wrote that it would be nice if I update to newer npm, but I've failed badly that part so I'll leave it out

Run
If you've made the shell file like here, then:

Code:

cd ~
./run_explorer.sh

[e8180kimo]

Do you know how to setup with Tor or how to connect to my node with your setup remotely?

[NeuroticFish]

Do you know how to setup with Tor or how to connect to my node with your setup remotely?

Sorry, I didn't do such tests, I use this configuration locally.
But if you install this Electrs and want to use from outside of the local computer, I think that a first step would be to set the correct address you're accessing it from, or, if you want to allow all addresses then probably

Code:

electrum_rpc_addr = "0.0.0.0:50001"

instead of

Code:

electrum_rpc_addr = "127.0.0.1:50001"

But I advise this only on LAN. If you go out of your LAN then you'll probably have to configure SSL too (and allow some port forwarding in your router too).


Tor, sorry, I really don't know...

[BlackHatCoiner]

Do you know how to setup with Tor or how to connect to my node with your setup remotely?

Update and install tor:

Code:

sudo apt update
sudo apt install tor

Check if Tor is installed:

Code:

tor --version
> Tor version 0.4.X
[...]

Edit the configuration file:

Code:

sudo nano /etc/tor/torrc

Add the following at the end of the file:

Code:

HiddenServiceDir /var/lib/tor/hidden_service_electrs/
HiddenServiceVersion 3
HiddenServicePort 50001 127.0.0.1:50002

(I think it needs to listen to 50001, because I see no SSL in this guide)

Reload tor:

Code:

sudo systemctl reload tor

Get the onion address with:

Code:

sudo cat /var/lib/tor/hidden_service_electrs/hostname

You connect to abc...xyz.onion:50002.

[NeuroticFish]

I may need help from more experienced people.

It was not only my WSL affected by the badly configured/partitioned SSD and I had the opportunity to pretty much wipe everything and start anew.
I've installed everything per this tutorial - obviously the latest versions - and I've got a strange surprise I don't like: no matter how long I leave it running, Electrs doesn't want to show me the "serving Electrum RPC on 127.0.0.1:50001" message.

Electrum connects to it, BTC-RPC-Explorer also connects to it, all seem to be fine, but I am missing that message I knew it announced that everything is up and running.
Does anybody know how to make it show up again? Is this a new feature? Can I enable it from config or command line?
Thanks.

[BlackHatCoiner]

Does anybody know how to make it show up again? Is this a new feature? Can I enable it from config or command line?

It should be showing it to you, because it does on raspibolt, and that uses v0.9.10.

What's your log? It should log you stuff if you've enabled it at the configuration (just add these two lines[1]). What is it showing you after you enter "./electrs"?

[1]

Code:

log_filters = "INFO"
timestamp = true

[NeuroticFish]

It should be showing it to you, because it does on raspibolt, and that uses v0.9.10.

What's your log? It should log you stuff if you've enabled it at the configuration (just add these two lines[1]). What is it showing you after you enter "./electrs"?

The config is the one from this post. I've added now your timestamp line, but no change.
After running electrs I see only the starting line containing basically the configuration (I've replaced this or that in case it's supposed to be private).

Starting electrs 0.9.10 on x86_64 linux with Config { network: Bitcoin, db_path: "/mnt/x/some/folder", daemon_dir: "/mnt/y/another/folder", daemon_auth: CookieFile("/mnt/x/some/folder/.somecookie"), daemon_rpc_addr: 127.0.0.1:8332, daemon_p2p_addr: 127.0.0.1:8333, electrum_rpc_addr: 127.0.0.1:50001, monitoring_addr: 127.0.0.1:4224, wait_duration: 10s, jsonrpc_timeout: 15s, index_batch_size: 10, index_lookup_limit: None, reindex_last_blocks: 0, auto_reindex: true, ignore_mempool: false, sync_once: false, disable_electrum_rpc: false, server_banner: "Electrs @ NeuroticFish", signet_magic: 3somenumber1, args: [] }

[NeuroticFish]

I've found the problem and it is, as in many similar cases, between the keyboard and the chair.
Yep, you were right and I am stupid. I managed to be careless with the config key.
I had, by mistake

Code:

log-filters = "INFO"

instead of the correct

Code:

log_filters = "INFO"

All fixed now. Thanks.

[NeuroticFish]

I've done some tests with having an username and a password for RPC in Bitcoin Core instead of cookie file and I will post them here in case somebody wants to do that.
I will comment out (not delete) the config lines related to RPC cookie file.
Let's say that the username will be UsErNaMe and the password will be PaSsWoRd (please use much better ones though, just I would avoid using # or : in them).

This would make x:\BitcoinDataDir\bitcoin.conf (see this post) look like:

Code: (bitcoin.conf)

txindex=1
server=1
rpcbind=127.0.0.1
rpcallow=127.0.0.1
#rpccookiefile=.mycookie
rpcuser=UsErNaMe
rpcpassword=PaSsWoRd

For Electrs, the content of $HOME/electrs/target/release/electrs.toml (see this post) will become:

Code: (electrs.toml)

#cookie_file = "/mnt/x/BitcoinDataDir/.mycookie"
auth="UsErNaMe:PaSsWoRd"
daemon_dir = "/mnt/x/BitcoinDataDir"
daemon_rpc_addr = "127.0.0.1:8332"
daemon_p2p_addr = "127.0.0.1:8333"
db_dir = "/mnt/y/ElectrsData"
network = "bitcoin"
electrum_rpc_addr = "127.0.0.1:50001"
#verbose = 2 -> obsolete, was working with version 0.9.0, no longer works in 0.9.3
#use log-filters for version 0.9.3+
log_filters = "INFO"
server_banner = "Electrs @ NeuroticFish, yay!"

And for BTC RPC Explorer, $HOME/btc-rpc-explorer/.env (see this post) becomes:

Code: (.env)

BTCEXP_BASEURL=/exp/
BTCEXP_BITCOIND_HOST=127.0.0.1
BTCEXP_BITCOIND_PORT=8332
#BTCEXP_BITCOIND_COOKIE=/mnt/X/BitcoinDataDir/.mycookie
BTCEXP_BITCOIND_USER=UsErNaMe
BTCEXP_BITCOIND_PASS=PaSsWoRd
BTCEXP_BITCOIND_RPC_TIMEOUT=5000
BTCEXP_ADDRESS_API=electrum
BTCEXP_ELECTRUM_SERVERS=tcp://127.0.0.1:50001
BTCEXP_ELECTRUM_TXINDEX=true
BTCEXP_SLOW_DEVICE_MODE=false
BTCEXP_PRIVACY_MODE=true
BTCEXP_NO_RATES=true

However, as said in Electrs pages, Electrs is meant for local networks, not exposed to the internet. So this setup is still not safe for the "outer world".
For that you'll probably need Blockstream's fork of electrs and maybe mempool space as explorer. And, clearly, SSL.
I wanted to make this clear, since some may expect that using Bitcoin Core RPC with user/password simply makes everything safe.

[jdoaktown]

Code:

Error: electrs failed

Caused by:
    0: bitcoind RPC polling failed
    1: daemon not available
    2: JSON-RPC error: transport error: Couldn't connect to host: Connection refused (os error 111)
hcp@HCP-PC:~/electrs$

Really not sure what is causing the issue... any one else come across this?

I can easily reproduce this when bitcoin core is running, but it's not synced.
So I'd start with getting full sync.

If that's not the cause, maybe take a look at rpcbind or rpcallowip in bitcoin.conf...

I had same problem. I am using Debian under WSL2 (but I'd guess Ubuntu would work as well).  
The issue is WSL2 is different from WSL1 because WSL2 uses a VM that is not perfectly bidirectional in its implementation of ip ports:
  https://learn.microsoft.com/en-us/windows/wsl/networking


To understand this better, on the Windows side use:  

Code:

 resmon 

  and  

Code:

 ss -l 

 on the Debian side to monitor port status.


To fix,
1) in Debian run:

Code:

ip route show | grep -i default | awk '{ print $3}'

this returns the ip to use w/in Debian to refer to the Windows host (which is NOT 127.0.0.1) eg.:

Code:

172.27.176.1

then, change electrs.toml to:

Code:

#cookie_file = "/mnt/x/BitcoinDataDir/.mycookie"
auth="rt1:rt1"
daemon_dir = "/mnt/x/BitcoinDataDir"

  # ip route show | grep -i default | awk '{ print $3}'
daemon_rpc_addr = "172.27.176.1:8332"
daemon_p2p_addr = "172.27.176.1:8333"


#daemon_p2p_addr = "127.0.0.1:18444"
db_dir = "/mnt/y/ElectrsData"
network = "bitcoin"
electrum_rpc_addr = "127.0.0.1:50001"
#verbose = 2 -> obsolete, was working with version 0.9.0, no longer works in 0.9.3
#use log-filters for version 0.9.3+


# OFF TRACE DEBUG INFO WARN ERROR
log_filters = "TRACE"
server_banner = "Electrs @ NeuroticFish, -jd"

and in bitcoin.conf:

Code:

rpcallowip=0.0.0.0/0
rpcbind=0.0.0.0

The above allows me to run my Bitcoin in Windows and my Electrs in WSL2 Debian. It accounts for the lopsided ip ports directionality between Win11 & WSL2.

Further troubleshooting tools:
On wsl2 side try both the windows .exe bitcoin

Code:

/mnt/c/BTCwin/daemon/bitcoin-cli.exe -rpcuser=rt1 -rpcpassword=rt1 -rpcport=8332 getblockchaininfo

...AND the Debian binary bitcoin:

Code:

/mnt/d/bitcoin-27.0/bin/bitcoin-cli -rpcuser=rt1 -rpcpassword=rt1 -rpcconnect=172.27.176.1  -rpcport=8332 getblockchaininfo

[MasterSa]

Hi,

I hope you're doing well.

Over the past few days, I've been trying to run a Bitcoin Core node as a Bitcoin explorer, but I've encountered some issues. I'm reaching out for your help once again to resolve them.

Here’s a brief overview of the situation:

I'm using Windows 11 with WSL installed. Inside WSL, I have set up a Bitcoin Core node (fully synced) along with Electrs. However, when I try to retrieve simple information using the following command, I receive an error:

Command>> curl http://127.0.0.1:50001/address/1Address
Error>>>>> curl: (1) Received HTTP/0.9 when not allowed

For bitcoin I am using cookie authentication.

Here is my bitcoin.conf file:

server=1
daemon=1
txindex=1
rpcbind=127.0.0.1
rpcallowip=127.0.0.1
cookieauth=1
txindex=1

And here is config.toml file:

network = "bitcoin"
db_dir = "/root/.electrs/db"
daemon_rpc_addr = "127.0.0.1:8332"
electrum_rpc_addr = "127.0.0.1:50001"
monitoring_addr = "127.0.0.1:4224"
jsonrpc_import = true

I'm not sure what I might be missing or doing wrong. Your guidance would be greatly appreciated.

Bitcoin Client Software and Version: Bitcoin Core 28.01.0
Operating System: Windows 11 64-bit
System Specs: Intel(R) Core(TM) i9-14900K, 3200 MHz, 24 Core(s), 1TB SSD, 64GB RAM
Description of the problem: right way to fetch data from Bitcoin core wallet that are not controlled by my Bitcoin node?
Example Address: 1HARAVuqP6V5t4h7Z6TWysj8rabog1HEfy
Related Transaction IDs: N/A
Screenshot of the Problem: https://postimg.cc/sMXJFR7c
Log Files from the Bitcoin Client: N/A

[mcdouglasx]

Here’s a brief overview of the situation:

I'm using Windows 11 with WSL installed. Inside WSL, I have set up a Bitcoin Core node (fully synced) along with Electrs. However, when I try to retrieve simple information using the following command, I receive an error:

Command>> curl http://127.0.0.1:50001/address/1Address
Error>>>>> curl: (1) Received HTTP/0.9 when not allowed

Hello, I think that the port 50001 you're trying to access does not use the HTTP protocol. Electrs operates on port 50001 but communicates using the Electrum protocol, not HTTP. You need to use a client that is compatible with the Electrum protocol.

[MasterSa]

thanks for your replay. Could you please provide some more details on how to do it?

[mcdouglasx]

thanks for your replay. Could you please provide some more details on how to do it?

Since it is directly unfeasible due to protocol differences to use curl, you need a kind of interpreter since they are 'speaking different languages'. So, you need an interface that translates your HTTP requests to the Electrum protocol, such as REST or Esplora.

https://theroadtonode.com/bitcoin-core-extensions/electrs

https://github.com/Blockstream/esplora

But if you prefer not to use curl to query an address with Electrs, you can use specific Electrum protocol clients that are designed to interact directly with Electrs, such as: Electrum CLI, ElectrumX.

[MasterSa]

thanks for your replay. Could you please provide some more details on how to do it?

Since it is directly unfeasible due to protocol differences to use curl, you need a kind of interpreter since they are 'speaking different languages'. So, you need an interface that translates your HTTP requests to the Electrum protocol, such as REST or Esplora.

https://theroadtonode.com/bitcoin-core-extensions/electrs

https://github.com/Blockstream/esplora

But if you prefer not to use curl to query an address with Electrs, you can use specific Electrum protocol clients that are designed to interact directly with Electrs, such as: Electrum CLI, ElectrumX.

Thanks a lot for the response! I really appreciate it. I'll dive deeper into this in the coming days, and I might have some more questions later on. Thanks again!

[ABCbits]

thanks for your replay. Could you please provide some more details on how to do it?

Since it is directly unfeasible due to protocol differences to use curl, you need a kind of interpreter since they are 'speaking different languages'. So, you need an interface that translates your HTTP requests to the Electrum protocol, such as REST or Esplora.

https://theroadtonode.com/bitcoin-core-extensions/electrs

https://github.com/Blockstream/esplora

But if you prefer not to use curl to query an address with Electrs, you can use specific Electrum protocol clients that are designed to interact directly with Electrs, such as: Electrum CLI, ElectrumX.

FWIW, Electrum protocol actually use JSON RPC[1]. It's better alternative if @MasterSa and other reader doesn't want to install/run another software, but willing to read it's documentation[2] which could use more example and tutorial.

[1] https://electrum-protocol.readthedocs.io/en/latest/protocol-basics.html
[2] https://electrum-protocol.readthedocs.io/en/latest/index.html