Bitcoin + Electrum server + Block explorer under Windows (with WSL and Debian)

[NeuroticFish]

This may be useful to others too, since I've spent quite a lot of time figuring out how to have everything working on a machine that runs a lot, but it's on Windows.

This walkthough is meant to show how to install an Electrum server (electrs) and a block explorer (BTC RPC Explorer) on (more or less) Windows. It's "more or less" because I actually use WSL (windows subsystem for linux) to get a Debian run under Windows.

If you have only one wallet and you don't want your block explorer, Electrum Personal Server may be enough for you. Start reading the other thread from here, I have 2 posts on that topic.

Electrs is not restricted to one wallet and also has useful data for the block explorer, hence this more generic setup was what I was looking for.

I will post all the details I remember and I will post also my config files too. Improvements are welcome, since I'm not really a Linux guy and, as said, most of this setup is still under Linux.

Bitcoin

Bitcoin is easy to install under Windows, it just runs, make sure your data directory is set somewhere with plenty of space, I use an external drive (X:\BitcoinDataDir)
I've created in X:\BitcoinDataDir a file bitcoin.conf with the following content:

Code: (bitcoin.conf)

txindex=1
server=1
rpcbind=127.0.0.1
rpcallow=127.0.0.1
rpccookiefile=.mycookie

Some extras:
1. If you want to not be too generous with the network, you may be tempted to use maxconnections in your config. Although I've read you should put minimum value 12, it's not enough. When I've tried that the block explorer may not be able to connect. With a value like 25 it was OK.
2. Make sure that if you use maxuploadtarget in your config, you also add whitelist=download@127.0.0.1

Of course, you can just keep it simple and don't add those extras.

I've also made 2 batch files because I'm lazy. Both are in the same folder ad bitcoind.exe and I've made for both shortcuts onto desktop.

Code: (start_bitcoin.bat )

bitcoind -datadir=X:\BitcoinDataDir

Code: (stop_bitcoin.bat)

bitcoin-cli.exe -datadir=X:\BitcoinDataDir stop

As a note, keep in mind that bitcoind will "discuss" with electrs and the block explorer only after the console shows progress=1.000000
And yes, you should leave at least bitcoind run as much as possible.

LE: If you prefer to use all this with RPC user/password instead of RPC cookie file, the configs are adapted in this later post.

[1714147604]

1714147604

[1714147604]

1714147604

[1714147604]

1714147604

[NeuroticFish]

WSL on Windows, Debian

Under windows (10), go to Start menu and type Turn Windows features on or off
Near the end of the list there's Windows Subsystem for Linux. Set that checked, OK, let it install/restart and so on.

When that's done, Start menu and type Run, and in the run box type https://aka.ms/wslstore <enter>
There install Debian (I had troubles with Ubuntu, so Debian it is).
Keep in mind that Debian and what's on it will take some 4.5 GB on the Windows drive. [Edit 2022-02-17: if the size on windows partition is a problem like it was for me, here's the solution.]

Start Debian, create an user and password for yourself and we can continue the real deal.

If you have an aggressive antivirus like myself (e.g. Comodo), it may be a good idea to disable the auto containment component. And don't forget to re-renable it after all the install is done.
I didn't do this at start and I've lost a lot of time because this or that was blocked from running under WSL.

It worth mentioning (thank you HCP, you have spared me a lot of time) that for each Windows drive, under Debian you'll have /mnt/<drive letter>/your path on that drive

Update the OS:

Code:

sudo apt update
sudo apt dist-upgrade

[NeuroticFish]

Electrs (from https://github.com/romanz/electrs)

The steps are based on the official install page and my struggle.
Some of the tools (curl, git) were missing and I had to install them myself.

I had to do all the following commands, one by one:

Code:

sudo apt install curl
sudo curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source $HOME/.cargo/env

sudo apt install cargo

sudo apt update
sudo apt install clang cmake build-essential

sudo apt install librocksdb-dev=6.11.4-3

sudo apt install git
git clone https://github.com/romanz/electrs
cd electrs
ROCKSDB_INCLUDE_DIR=/usr/include ROCKSDB_LIB_DIR=/usr/lib cargo build --locked --release

[later note: if sudo apt install librocksdb-dev=6.11.4-3 fails on Debian you can try sudo apt install librocksdb-dev=6.11.4*]
[later note: if sudo apt install librocksdb-dev=6.11.4-3 still fails (probably on Ubuntu), read this post for workaround]

I've edited under Windows (as e:\electrs.toml, keep this path in mind, you'll have to copy it to Linux) the config file; it was based on the config from the official example and mine looks like this (I also kept only the useful lines)

Code: (electrs.toml)

cookie_file = "/mnt/x/BitcoinDataDir/.mycookie"
daemon_dir = "/mnt/x/BitcoinDataDir"
daemon_rpc_addr = "127.0.0.1:8332"
daemon_p2p_addr = "127.0.0.1:8333"
db_dir = "/mnt/y/ElectrsData"
network = "bitcoin"
electrum_rpc_addr = "127.0.0.1:50001"
#verbose = 2 -> obsolete, was working with version 0.9.0, no longer works in 0.9.3
#use log-filters for version 0.9.3+
log_filters = "INFO"
server_banner = "Electrs @ NeuroticFish, yay!"

[Edit: version 0.9.3 doesn't like verbose in the config, using log-filters instead.]

This means that the Electrs data (up to 70 GB, according to some docs, but right now that folder on my computer is only 30 GB) will be on drive Y:, as Y:\ElectrsData
Under Debian, please create that folder:

Code:

mkdir /mnt/y/ElectrsData

Go into the correct folder and copy the config (you've saved it, right?). And don't forget the ending space and point

Code:

cd target/release
cp /mnt/e/electrs.toml .

Start electrs and let it sync, it'll take some hours in the first run.

Code:

./electrs

At a later point it will show the sweet:

Code:

[2021-10-16T13:31:39.392Z INFO  electrs::server] serving Electrum RPC on 127.0.0.1:50001

Edit 2022-04-17: If, at a later point, you might want to update Electrs, you can take a look here: https://bitcointalk.org/index.php?topic=5367296.msg59888827#msg59888827

[NeuroticFish]

BTC RPC Explorer (from https://github.com/janoside/btc-rpc-explorer)

I expect this to be installed after Electrs is completely synced, and it's running.
Open a new Debian (R-Click on the Debian rectangle on the task bar and select Debian) and let's go.
Again some Linux commands to be ran:

Code:

sudo su
curl -fsSL https://deb.nodesource.com/setup_lts.x | bash -
apt-get install -y nodejs
exit
git clone https://github.com/janoside/btc-rpc-explorer
cd btc-rpc-explorer
npm install

Now I've made, again under Windows, and again on my E: drive a file called .env with the content based on the official sample:

Code: (.env)

BTCEXP_BASEURL=/exp/
BTCEXP_BITCOIND_HOST=127.0.0.1
BTCEXP_BITCOIND_PORT=8332
BTCEXP_BITCOIND_COOKIE=/mnt/X/BitcoinDataDir/.mycookie
BTCEXP_BITCOIND_RPC_TIMEOUT=5000
BTCEXP_ADDRESS_API=electrum
BTCEXP_ELECTRUM_SERVERS=tcp://127.0.0.1:50001
BTCEXP_ELECTRUM_TXINDEX=true
BTCEXP_SLOW_DEVICE_MODE=false
BTCEXP_PRIVACY_MODE=true
BTCEXP_NO_RATES=true

Now copy .env into the Linux folder and start

Code:

cp /mnt/e/.env .
npm start

Edit 2022-04-17: If, at a later point, you might want to update BTC RPC Explorer, you can take a look here: https://bitcointalk.org/index.php?topic=5367296.msg59888845#msg59888845

[NeuroticFish]

Fine tuning and other info.

Electrum

From this moment onwards, you'll be able to use Electrum with your own server.
Although I use Electrum on the same computer, I use the portable one (yeah, I'm lazy)
I've made a batch file which I run from now on instead of Electrum's exe and it looks like this:

Code: (run-electrum.bat)

electrum-4.1.5-portable.exe --oneserver --server 127.0.0.1:50001:t

Linux batch files and others

Open a new Debian console. This time we will create 2 batch files under Linux, because we are lazy. Just now we will make them directly under Linux:

Code:

nano $HOME/run_electrs.sh

And inside nano editor:

Code: (run_electrs.sh)

cd $HOME/electrs/target/release
./electrs

CTRL-X and Save

again

Code:

nano $HOME/run_explorer.sh

Code: (run_explorer.sh)

cd $HOME/btc-rpc-explorer
npm start

CTRL-X and Save

two more lines to execute:

Code:

chmod +x $HOME/run_electrs.sh
chmod +x $HOME/run_explorer.sh

Now, when you want to exit Electrs or BTC RPC Explorer, you press CTRL+C in those consoles, then exit.
When you want to run them, you start ./run_electrs.sh in one console then ./run_explorer.exe in another

Web page for block explorer

http://localhost:3002/exp/


Thank you

• HCP for setting me on the right track
• ETFbitcoin for hinting the correct block explorer
• DaveF and NotATether for making me understand VirtualBox is not such a great idea
• DireWolfM14 for trying to help me with auto-starting services, although I still don't know how to do that  
• everybody active in the Dev & Tech from whom I've learned a lot
• and obviously the teams implementing all the great software I've installed and using now

It may be better to have a way to start everything from one click but I don't know how.
I've noticed that if I started bitcoind and immediately electrs, the second has failed, so now I just check when bitcoid is synced and then start electrs, I wait for electrs show it's ready and then start blockchain explorer and/or electrum. It's not optimal, but it's OK for my needs and... this is how much I've manged to do.

Edit 1: corrected/improved wording
Edit 2: corrected the .sh files per ETFbitcoin suggestion and ensured their paths too.

[DaveF]

Side note.
Disable automatic updates on Windows, and install the Windows updates manually.
When done then shutdown Debian
Reboot Windows and bring up Debian.

In theory, Windows should cleanly save the linux subsystem to disk, and then reboot.
In reality, at times something locks and it's not done cleanly. And bad things can happen....

-Dave

[MixMAx123]

Great instructions! This is something that many people can use well, thank you for your effort!
I will try it with your instructions, sobal my system runs correctly again.

[DireWolfM14]

Good job, I hope you had fun.  You're on the right path, and you'll pick up tricks that work for you as you go along.  WSL is a great way for Windows users to immerse themselves in Linux.

I prefer to run my node on dedicated hardware which I can leave running full time.  All my services are started on boot by systemd, so I just need to make sure the machine is powered on.  Unlike Linux on a physical or virtual machine, WSL doesn't have a system startup daemon like systemd.  It shouldn't be confused with a virtual machine, it's a virtual environment.

It's bugging me that you are unable to install Ubuntu.  I noticed that you used the Control Panel gui app to enable WSL, did also enable virtualization?  I've always installed WSL manually, you can find the installation instructions here (don't forget to start powershell as administrator):  https://docs.microsoft.com/en-us/windows/wsl/install-manual

Also, you can use powershell to interact with your WSL instances and, run them with alternate options.  For starters:

Code:

wsl --help

[NeuroticFish]

In theory, Windows should cleanly save the linux subsystem to disk, and then reboot.
In reality, at times something locks and it's not done cleanly. And bad things can happen....

Thanks for the heads up. I try to always exit the consoles as gracefully as I can (CTRL-C then exit) before leaving windows do shutdown or restart. I hope that I'm on the safe side.

Good job, I hope you had fun.  You're on the right path, and you'll pick up tricks that work for you as you go along.  WSL is a great way for Windows users to immerse themselves in Linux.

Thank you for the kind words.

I prefer to run my node on dedicated hardware which I can leave running full time.  All my services are started on boot by systemd, so I just need to make sure the machine is powered on.  Unlike Linux on a physical or virtual machine, WSL doesn't have a system startup daemon like systemd.  It shouldn't be confused with a virtual machine, it's a virtual environment.

I may get there too, but for now I cannot: even the oldest working laptop in the house is being used by the kids for online classes and when I bought a RasPi for my needs I was not smart enough and bought a weak one (Zero W), from what I've read it's too weak for this job.
Maybe I'll get to buy in the near future a RasPi 4 and a 1-2 TB HDD (I find SSD overpriced for the job at this size), maybe I open a new topic on that, since Black Friday is pretty close.

It's bugging me that you are unable to install Ubuntu.

I was able to install Ubuntu. I had other problems there:

• it didn't find for me librocksdb-dev=6.11.4-3
• it failed in creating files/folders onto my Windows partition for electrs data

Of course, it was my first install and the problems may have been caused by something I didn't do right or I didn't do at all. Or my antivirus blocking something I forgot to unblock.
Maybe I was just luckier with Debian. However, this worked, I've noted down this and that (for the case I have to do "backtracking" until I get it right) and.. this is what I have.
If people want to try it out on any other distro, be my guest, really  

I noticed that you used the Control Panel gui app to enable WSL, did also enable virtualization?

If you mean Hyper-V, no, it's not enabled and I don't intend to unless I really have to.
I've understood that it may interfere badly with VBox, which I still need now and then for my own tests (for simulating a fresh Windows or for running suspect apps) and my CPU is i7 so it has some in-built features, I think.

I've always installed WSL manually, you can find the installation instructions here (don't forget to start powershell as administrator):  https://docs.microsoft.com/en-us/windows/wsl/install-manual

Also, you can use powershell to interact with your WSL instances and, run them with alternate options.  For starters:

Code:

wsl --help

Although I work on Windows since Windows 95 (even 3.1, but that was for too short time), I have little experience with Power Shell. I find it too... Linux like and I didn't really need it; most of the time the normal command prompt or apps done by myself could do all I've needed.

Great instructions! This is something that many people can use well, thank you for your effort!
I will try it with your instructions, sobal my system runs correctly again.

Thank you! Please let me know if I've missed or didn't explain good enough something.

[ABCbits]

And inside nano editor:

Code:

cd ./electrs/target/release
./electrs

CTRL-X and Save

This script will fail if you run it from different location (location when you run nano to create the script). You should replace it with absolute path.

[NeuroticFish]

This script will fail if you run it from different location (location when you run nano to create the script). You should replace it with absolute path.

In my defense , if it's done as I expect, the .sh files are created in user home folder, the same folder the user gets in when he starts the Linux console, so he should not need to run that from somewhere else.
Also I tried to avoid having path containing the user name, but I've learned meanwhile about $HOME.

But you're right, users may do things in a way I don't expect  
Fixed it. Thank you.


...And if I started this, I've also ensured the paths for the .sh files are exactly where I expect them to be. So double thanks 

[MixMAx123]

As threatened, I try to imitate this now.
At this point, the first problems occur:

Code:

sudo apt install cargo

error: "Problems can not be corrected, they have retained defective packages "
https://de.share-your-photo.com/img/d80a1b2e3a.png

annotation
I'm trying this on Ubuntu 18.04.3 LTS.
I reinstalled Ubuntu yesterday. BitcoinCore now runs properly. No further software is installed.
I am an absolute Linux failure.
Things that you think for yourself, I have probably done wrong.
So put the worst case in advance.

Yesterday I also made a backup from the system. Every time something is defective, I simply set up the backup again.
And believe me, that happens very often!
So because of the security, I do not worry, I repeat and test as long as it goes.

Another note:
I do not know if that's normal, but if I want to install something with "sudo apt install xxx", this almost always fails with various errors.

[NeuroticFish]

I'm trying this on Ubuntu 18.04.3 LTS.

Do you have something older than Win 10? I'm asking because I see on the Windows store Ubuntu 20.04 LTS...
However, I'm not surprised, I had my own share of surprises with Ubuntu.

Unfortunately I am also far from good at Linux; this tutorial was made simply because I didn't find any reasonably good/complete one.
For your problems I have only one possible advice, if you didn't do this at start. Update your binaries:

Code:

sudo apt update
sudo apt dist-upgrade

Also, I'm a bit surprised. If you are not great at Linux, why didn't you try with Debian as I did?

Another note:
I do not know if that's normal, but if I want to install something with "sudo apt install xxx", this almost always fails with various errors.

I don't find it normal. But the environment variables may be different between your user and the super user (I hope that I didn't say something silly) and that could be one reason.
I'm not sure that it will help, but you can try to do sudo su, do the things without sudo in front and when you're done do exit.

[MixMAx123]

Also, I'm a bit surprised. If you are not great at Linux, why didn't you try with Debian as I did?

Well, it's easy, I decided on Ubuntu at that time because people said it was the best for me.
Now I have used it for a few years and you need good arguments or a weapon to bring me to learn again a new system.
I have now found and resolve many mistakes. You are right, it lay on the outdated packet. Two days I only searched for this error.
NodeJS and BTC Explorer work now.
I'll continue and now try to install electrumX or an elevated tool that creates a database for the bitcoin-addresses.
I believe I have to install Python3. At this point I had the last time my ubuntu destroyed. So now I'm here where I was the last time. :-(
When I'm done and everything works well, I will post a detailed description here as it is to set up on Ubuntu.

[MixMAx123]

So I came far now, but at this point I get a mistake:

Code:

sudo apt install librocksdb-dev=6.11.4-3

I did it that way:

Code:

sudo apt install librocksdb-dev

This has been installed version (5.8.8-1).
I tried "sudo apt update" but it did not work.
I can not update this version.
I tried to continue with this version.
But fatal mistakes are the result, it can not be right.
Can you help me again how I can install the version "librocksdb-dev=6.11.4-3"?
Thanks very much! :-)

[NeuroticFish]

Can you help me again how I can install the version "librocksdb-dev=6.11.4-3"?

I also couldn't install that library on Ubuntu in my tests. You can see me complaining here, it was the "smaller" problem I had.
The good news is that you can go without that lib, just then instead of dynamic linking (see the lines in my initial post)

Code:

$ ROCKSDB_INCLUDE_DIR=/usr/include ROCKSDB_LIB_DIR=/usr/lib cargo build --locked --release

you have to go with static linking

Code:

cargo build --locked --release

and you get error here too, you may have to set

Code:

$ RUSTFLAGS="-C link-args=-latomic" cargo build --locked --release

and retry with

Code:

cargo build --locked --release

Source/details: https://github.com/romanz/electrs/blob/master/doc/install.md#static-linking

[MixMAx123]

Thanks @NeuroticFish
I have done it now and it runs.
As announced I created a guide for Ubuntu:
https://bitcointalk.org/index.php?topic=5368219.0

[NeuroticFish]

I was told how to handle the 4.5GB Debian takes on windows drive, I will add also here the relevant information in order to Move after Install that Debian anywhere else you want.

Check this answer https://stackoverflow.com/a/64219885. But 4.5GB is rather small compared with Windows itself and some popular application on Windows (such as Ms Office), so i wouldn't bother move it.

I've installed that app with chocolatey, ran the command line, got some warnings, but went through. Now that Debian is successfully moved to HDD, I don't even mind if it's slower.

More detailed:

1. Installed Chocolatey to my Windows

1.1. Ran PowerShell as administrator and pasted there

1.2 Ran this command in that PowerShell (source: https://chocolatey.org/install ; make sure you use the latest instructions/command line from there)

Code:

Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://community.chocolatey.org/install.ps1'))

2. Installed LxRunOffline with chocolatey

In PowerShell started as administrator ran the command:

Code:

choco install lxrunoffline

3. Opened a completely new command prompt (cmd) as admin and there:

Code:

lxrunoffline move -n Debian -d x:\the\new\folder\

And waited for some time. Got some warnings, but went through. (Thanks again ETFbitcoin)

[NeuroticFish]

Glad it works. But what kind of warnings you received? Reduced performance? Might broken when windows release update for WSL/WSL2?

Code:

[WARNING] Ignoring an unsupported file "\\?\C:\Users\<my_user_name>\AppData\Local\Packages\TheDebianProject.DebianGNULinux_<something>\LocalState\rootfs\dev\full" of type 0020000.
[WARNING] Ignoring an unsupported file "\\?\C:\Users\<my_user_name>\AppData\Local\Packages\TheDebianProject.DebianGNULinux_<something>\LocalState\rootfs\dev\null" of type 0020000.
[WARNING] Ignoring an unsupported file "\\?\C:\Users\<my_user_name>\AppData\Local\Packages\TheDebianProject.DebianGNULinux_<something>\LocalState\rootfs\dev\ptmx" of type 0020000.
[WARNING] Ignoring an unsupported file "\\?\C:\Users\<my_user_name>\AppData\Local\Packages\TheDebianProject.DebianGNULinux_<something>\LocalState\rootfs\dev\random" of type 0020000.
[WARNING] Ignoring an unsupported file "\\?\C:\Users\<my_user_name>\AppData\Local\Packages\TheDebianProject.DebianGNULinux_<something>\LocalState\rootfs\dev\tty" of type 0020000.
[WARNING] Ignoring an unsupported file "\\?\C:\Users\<my_user_name>\AppData\Local\Packages\TheDebianProject.DebianGNULinux_<something>\LocalState\rootfs\dev\urandom" of type 0020000.
[WARNING] Ignoring an unsupported file "\\?\C:\Users\<my_user_name>\AppData\Local\Packages\TheDebianProject.DebianGNULinux_<something>\LocalState\rootfs\dev\zero" of type 0020000.

These are my warnings. I've hide replaced 2 parts, for privacy/security.

* Yesterday after the move I've checked that Debian, the Electrum server and the block explorer, all were working fine.
* I've had yesterday evening a Windows update and worked (I don't know what it contained though).

[ABCbits]

Glad it works. But what kind of warnings you received? Reduced performance? Might broken when windows release update for WSL/WSL2?

Code:

[WARNING] Ignoring an unsupported file "\\?\C:\Users\<my_user_name>\AppData\Local\Packages\TheDebianProject.DebianGNULinux_<something>\LocalState\rootfs\dev\full" of type 0020000.
[WARNING] Ignoring an unsupported file "\\?\C:\Users\<my_user_name>\AppData\Local\Packages\TheDebianProject.DebianGNULinux_<something>\LocalState\rootfs\dev\null" of type 0020000.
[WARNING] Ignoring an unsupported file "\\?\C:\Users\<my_user_name>\AppData\Local\Packages\TheDebianProject.DebianGNULinux_<something>\LocalState\rootfs\dev\ptmx" of type 0020000.
[WARNING] Ignoring an unsupported file "\\?\C:\Users\<my_user_name>\AppData\Local\Packages\TheDebianProject.DebianGNULinux_<something>\LocalState\rootfs\dev\random" of type 0020000.
[WARNING] Ignoring an unsupported file "\\?\C:\Users\<my_user_name>\AppData\Local\Packages\TheDebianProject.DebianGNULinux_<something>\LocalState\rootfs\dev\tty" of type 0020000.
[WARNING] Ignoring an unsupported file "\\?\C:\Users\<my_user_name>\AppData\Local\Packages\TheDebianProject.DebianGNULinux_<something>\LocalState\rootfs\dev\urandom" of type 0020000.
[WARNING] Ignoring an unsupported file "\\?\C:\Users\<my_user_name>\AppData\Local\Packages\TheDebianProject.DebianGNULinux_<something>\LocalState\rootfs\dev\zero" of type 0020000.

These are my warnings. I've hide replaced 2 parts, for privacy/security.

* Yesterday after the move I've checked that Debian, the Electrum server and the block explorer, all were working fine.
* I've had yesterday evening a Windows update and worked (I don't know what it contained though).

IIRC files in /dev represent your device and generated/removed during boot. Looks like my worry was for nothing.