Electrum server on Windows: Fulcrum

[apogio]

I had deleted the data dir for Fulcrum and still the same issue. Here's my Fulcrum.conf:

Code:

datadir = G:\FulcrumData_MainNet
bitcoind = 127.0.0.1:8332
rpcuser = randomuser
rpcpassword = randompass
tcp = 127.0.0.1:50001
peering = false
announce = false
public_tcp_port = 50001
admin = 8000
stats = 8080
db_max_open_files = 80
fast-sync = 8000

I think you 'll need to re-index Bitcoin Core.
Make sure to stop fulcrum before doing it.
I don't run fulcrum, unfortunately, but the error seems straightforward. Especially since it reads some blocks but it fails on the rest.

[fairmuffin]

I had deleted the data dir for Fulcrum and still the same issue. Here's my Fulcrum.conf:

Code:

datadir = G:\FulcrumData_MainNet
bitcoind = 127.0.0.1:8332
rpcuser = randomuser
rpcpassword = randompass
tcp = 127.0.0.1:50001
peering = false
announce = false
public_tcp_port = 50001
admin = 8000
stats = 8080
db_max_open_files = 80
fast-sync = 8000

I think you 'll need to re-index Bitcoin Core.
Make sure to stop fulcrum before doing it.
I don't run fulcrum, unfortunately, but the error seems straightforward. Especially since it reads some blocks but it fails on the rest.

Thanks for looking, I am proceeding with that; hopefully it won't take long, it took me a whole week to sync it :s

[apogio]

Thanks for looking, I am proceeding with that; hopefully it won't take long, it took me a whole week to sync it :s

Unfortunately, it will take some time, so be prepared with patience.
After doing that, you should be ok, but feel free to post here again, after finishing the re-indexing process.

[Cricktor]

Code:

datadir = G:\FulcrumData_MainNet
bitcoind = 127.0.0.1:8332
rpcuser = randomuser
rpcpassword = randompass
tcp = 127.0.0.1:50001
peering = false
announce = false
public_tcp_port = 50001
admin = 8000
stats = 8080
db_max_open_files = 80
fast-sync = 8000

I have the following settings for my Fulcrum running on a Raspi 4B with 8GB RAM (conf isn't complete, only relevant parts listed):

Code:

bitcoind = 127.0.0.1:8332
rpcuser = <redacted>
rpcpassword  = <redacted>

# RPi optimizations
# avoid 'bitcoind request timed out'
bitcoind_timeout = 600
db_mem=2048

# for 8GB RAM
#db_max_open_files=500
db_max_open_files=300
#fast-sync = 2048
fast-sync = 1024
txhash_cache = 512

Not sure and confident if my settings are any better, but they worked to sync without errors.

[fairmuffin]

Thanks for looking, I am proceeding with that; hopefully it won't take long, it took me a whole week to sync it :s

Unfortunately, it will take some time, so be prepared with patience.
After doing that, you should be ok, but feel free to post here again, after finishing the re-indexing process.

I have just let my bitcoin code sync up to 606010 and added this line to the Bitcoin config file:

Code:

connect=0

As I don't need to reach current block height. Fulcrum seems to be fully synced, as I am getting this:

Code:

[2025-01-26 18:20:49.006] Loading database ...
[2025-01-26 18:20:49.202] DB memory: 512.00 MiB
[2025-01-26 18:20:49.206] Coin: BTC
[2025-01-26 18:20:49.207] Chain: main
[2025-01-26 18:20:49.207] Verifying headers ...
[2025-01-26 18:20:49.727] Initializing header merkle cache ...
[2025-01-26 18:20:49.963] Checking tx counts ...
[2025-01-26 18:20:51.178] 479411650 total transactions
[2025-01-26 18:20:51.179] UTXO set: 63772347 utxos, 5356.877 MB
[2025-01-26 18:20:51.194] DB version: v3
[2025-01-26 18:20:51.204] BitcoinDMgr: starting 3 bitcoin RPC clients ...
[2025-01-26 18:20:51.204] BitcoinDMgr: started ok
[2025-01-26 18:20:51.205] Stats HTTP: starting 1 server ...
[2025-01-26 18:20:51.205] Starting listener service for HttpSrv 127.0.0.1:8080 ...
[2025-01-26 18:20:51.209] Service started, listening for connections on 127.0.0.1:8080

However, I am not sure how to use it to query balances for examples? I tried using it with BTC RPC Explorer but I get this error:

Code:

[2025-01-26 18:52:41.213] <HttpSrv 127.0.0.1:8080> Client: 127.0.0.1:49646; Invalid request: {"id": 1, "method": "blockchain.address.get_balance", "params": ["1A1zP1eP5QGefi2DMPTfTL5SLmv7DivfNa"]}

I also tried to use it as an Electrum server, but then the console prints:

Code:

[2025-01-26 20:24:18.297] <HttpSrv 127.0.0.1:8080> Client: 127.0.0.1:62656; Invalid request:

What am I doing wrong?

[NeuroticFish]

I could not say 100% what is the problem, but one thing I can see is that the console doesn't say anything about the tcp server on port 50001 being started.
So it's not matching with what I've written back then:

After starting Fulcrum (start.bat) it might take an awful lot of time until everything is sync-ed. You may want to look for lines like:

Code:

[2023-02-27 21:42:48.964] Starting listener service for TcpSrv 127.0.0.1:50001 ...
[2023-02-27 21:42:48.965] Service started, listening for connections on 127.0.0.1:50001

or

Code:

[2023-02-27 22:03:32.758] <Controller> Block height 778553, up-to-date

[fairmuffin]

I could not say 100% what is the problem, but one thing I can see is that the console doesn't say anything about the tcp server on port 50001 being started.
So it's not matching with what I've written back then:

After starting Fulcrum (start.bat) it might take an awful lot of time until everything is sync-ed. You may want to look for lines like:

Code:

[2023-02-27 21:42:48.964] Starting listener service for TcpSrv 127.0.0.1:50001 ...
[2023-02-27 21:42:48.965] Service started, listening for connections on 127.0.0.1:50001

or

Code:

[2023-02-27 22:03:32.758] <Controller> Block height 778553, up-to-date

Yeah I can confirm that it only tells me about listening for the HTTP thing, and not opening 50001. I think because Fulcrum still thinks there's more blocks that should be there. I've added maxblockheight=606010 to Fulcrum confguration file, and deleted Fulcrum data directory, I am syncing Fulcrum from scratch again and will see how this goes.

EDIT: Apparently that thing doesn't exist, so I am not sure it will help :S

[fairmuffin]

So I synced Fulcrum from scratch and now it's stuck like this (for three hours now):

Code:

[2025-01-27 00:42:51.127] <Controller> Processed height: 606000, 68.8%, 18.0 blocks/sec, 38790.7 txs/sec, 136442.5 addrs/sec
[2025-01-27 00:42:51.637] <Controller> Processed 606011 new blocks with 479411650 txs (1178064770 inputs, 1278776241 outputs, 1810346733 addresses), verified ok.
[2025-01-27 00:42:51.637] <Controller> Initial sync ended, flushing and deleting UTXO Cache ...
[2025-01-27 00:42:51.637] <Controller> Storage UTXO Cache: Flushing to DB ...

Looking at the task manager it's not using disk space apparently, and just some memory (RAM) so I think it's safe to stay it's not doing anything? Also the Processed height: 606000, 68.8% makes me think that Fulcrum is aware that my Bitcoin node was stopped at a specific block height that's missing a lot of recent blocks. Not sure what to do with this thing now really, I'd be happy to have it work correctly with up toblock height 606,010 (which is where I have turned connect=0 for my node), but I don't know if that's doable.

[apogio]

So I synced Fulcrum from scratch and now it's stuck like this (for three hours now):

Code:

[2025-01-27 00:42:51.127] <Controller> Processed height: 606000, 68.8%, 18.0 blocks/sec, 38790.7 txs/sec, 136442.5 addrs/sec
[2025-01-27 00:42:51.637] <Controller> Processed 606011 new blocks with 479411650 txs (1178064770 inputs, 1278776241 outputs, 1810346733 addresses), verified ok.
[2025-01-27 00:42:51.637] <Controller> Initial sync ended, flushing and deleting UTXO Cache ...
[2025-01-27 00:42:51.637] <Controller> Storage UTXO Cache: Flushing to DB ...

Looking at the task manager it's not using disk space apparently, and just some memory (RAM) so I think it's safe to stay it's not doing anything? Also the Processed height: 606000, 68.8% makes me think that Fulcrum is aware that my Bitcoin node was stopped at a specific block height that's missing a lot of recent blocks. Not sure what to do with this thing now really, I'd be happy to have it work correctly with up toblock height 606,010 (which is where I have turned connect=0 for my node), but I don't know if that's doable.

You need to wait for flushing to finish. Painful and long, but it will pass. [1]

[1] https://bitcoin.stackexchange.com/questions/116776/why-is-there-a-need-to-flush-the-utxo-set-when-you-prune-blk-dat-and-rev-dat-f

[ZinyProxy]

If one wants to have an Electrum server, one of the options is Fulcrum.

I already have an Electrum server (Electrs), which I use and for which I've made a more complete tutorial in the past, but at some point I've made tests with Fulcrum too and it would be a shame to remove it without providing a tutorial for the others who want it.

So... one major advantage of Fulcrum comes on Windows: the user can just get compiled binaries for Bitcoin Core, Fulcrum and Electrum, configure and run them all, without the need of stepping into Windows Subsystem for Linux (WSL). Another good thing is it's performance: it's processing the requests significantly faster than Electrs. Just in my experience (HDD, infrequent use for rather short tasks) Fulcrum gets in the state I can access it significantly slower than Electrs.

But, tbh, I don't like Fulcrum as much as the other option I have (Electrs)
* Fulcrum is made by somebody with BCH ties and it may be much better optimized for BCH (to say the least)
* As I said, Fulcrum starts/syncs slower than Electrs and that's not OK for my use case

Something more about this tutorial. This is made for a Fulcrum as I use it: me, for myself. No discussions with other servers, not announcing my server, also no SSL. I've kept it simple. If one wants to expand the tutorial with that, be my guest.

Since an Electrum server works on top of a Bitcoin Core node, I'll handle Bitcoin Core first, but shortly.

Bitcoin Core is with pretty much the same as in my other tutorial, but with some tiny bit of change in config:
* already in that topic, later on, I've switched for use with rpc user and password instead of cookie. If you want to keep using cookie, that's fine too, just pay attention to the config files (both of them)
* Bitcoin core's config needs one more line: zmqpubhashblock=tcp://127.0.0.1:8433

So, I will not insist with Bitcoin Core, it's in the other tutorial, I'll just put here my config:

Code: (bitcoin.conf)

txindex=1
server=1
rpcbind=127.0.0.1
rpcallow=127.0.0.1
rpcallowip=127.0.0.1
rpcuser=UsErNaMe
rpcpassword=PaSsWoRd
zmqpubhashblock=tcp://127.0.0.1:8433

Fulcrum (https://github.com/cculianu/Fulcrum)

You download Fulcrum from https://github.com/cculianu/Fulcrum/releases ; you will get something like Fulcrum-1.9.0-win64.zip
It would be nice to verify your download, which is done with the correcponding *.asc file, like for Electrum.

Unpack the zip into a new folder, maybe FulcrumBinaries. We won't touch that folder from now on.
Now we create a file called Fulcrum.conf with the content:

Code: (Fulcrum.conf)

datadir = x:\FulcrumData_MainNet
bitcoind = 127.0.0.1:8332
rpcuser = UsErNaMe
rpcpassword = PaSsWoRd
tcp = 127.0.0.1:50001
peering = false
announce = false
public_tcp_port = 50001
admin = 8000
stats = 8080
db_max_open_files = 80
fast-sync = 8000

Some details:
* The username and password has to match with the one from Bitcoin
* x:\FulcrumData_MainNet is a folder you create, preferably somewhere fast (SSD), since this is where Fulcrum will keep its data; mine has now ~113 GB, but it may be safe to have some more space there, especially at start
* that fast-sync line should be commented (put a # in front of it) after the initial sync finishes
* the server is set to not discuss with other servers, not announce itself for other clients and so on; also no SSL

I've made a batch file for start and one for stop, but this stop works only after the sync is done. Else you better press the good old CRTL-C.
The content of the batch file is not very useful as it is, maybe as example, because it contains the path to the exe and, as parameter, the path to the config.
In my case it's:

Code: (start.bat)

"x:\Fulcrum\FulcrumBinaries\Fulcrum.exe" x:\Fulcrum\Fulcrum.conf

Same goes for the stop.

Code: (stop.bat)

"x:\Fulcrum\FulcrumBinaries\FulcrumAdmin" -p 8000 stop

After starting Fulcrum (start.bat) it might take an awful lot of time until everything is sync-ed. You may want to look for lines like:

Code:

[2023-02-27 21:42:48.964] Starting listener service for TcpSrv 127.0.0.1:50001 ...
[2023-02-27 21:42:48.965] Service started, listening for connections on 127.0.0.1:50001

or

Code:

[2023-02-27 22:03:32.758] <Controller> Block height 778553, up-to-date

Then you can start Electrum. As I wrote in the other tutorial I'm lazy and I'm using the portable Electrum. For me the command line is:

Code:

electrum-4.3.4-portable.exe --oneserver --server 127.0.0.1:50001:t

I’ve been considering setting up Fulcrum, but it looks like the initial sync and hardware requirements are something to plan for. Might stick with Electrs for now since it works fine for my needs

[ABCbits]

So I synced Fulcrum from scratch and now it's stuck like this (for three hours now):

Code:

[2025-01-27 00:42:51.127] <Controller> Processed height: 606000, 68.8%, 18.0 blocks/sec, 38790.7 txs/sec, 136442.5 addrs/sec
[2025-01-27 00:42:51.637] <Controller> Processed 606011 new blocks with 479411650 txs (1178064770 inputs, 1278776241 outputs, 1810346733 addresses), verified ok.
[2025-01-27 00:42:51.637] <Controller> Initial sync ended, flushing and deleting UTXO Cache ...
[2025-01-27 00:42:51.637] <Controller> Storage UTXO Cache: Flushing to DB ...

Looking at the task manager it's not using disk space apparently, and just some memory (RAM) so I think it's safe to stay it's not doing anything? Also the Processed height: 606000, 68.8% makes me think that Fulcrum is aware that my Bitcoin node was stopped at a specific block height that's missing a lot of recent blocks. Not sure what to do with this thing now really, I'd be happy to have it work correctly with up toblock height 606,010 (which is where I have turned connect=0 for my node), but I don't know if that's doable.

You need to wait for flushing to finish. Painful and long, but it will pass. [1]

[1] https://bitcoin.stackexchange.com/questions/116776/why-is-there-a-need-to-flush-the-utxo-set-when-you-prune-blk-dat-and-rev-dat-f

CMIIW, but it seems Fulcrum is performing flushing on it's own DB rather than passing message from Bitcoin Core.

--snip--

I’ve been considering setting up Fulcrum, but it looks like the initial sync and hardware requirements are something to plan for. Might stick with Electrs for now since it works fine for my needs

If you already use Electrs, there's no strong reason switch to Fulcrum. But while time for initial sync is definitely longer, you could configure it to use less hardware resource.

[fairmuffin]

So I synced Fulcrum from scratch and now it's stuck like this (for three hours now):

Code:

[2025-01-27 00:42:51.127] <Controller> Processed height: 606000, 68.8%, 18.0 blocks/sec, 38790.7 txs/sec, 136442.5 addrs/sec
[2025-01-27 00:42:51.637] <Controller> Processed 606011 new blocks with 479411650 txs (1178064770 inputs, 1278776241 outputs, 1810346733 addresses), verified ok.
[2025-01-27 00:42:51.637] <Controller> Initial sync ended, flushing and deleting UTXO Cache ...
[2025-01-27 00:42:51.637] <Controller> Storage UTXO Cache: Flushing to DB ...

Looking at the task manager it's not using disk space apparently, and just some memory (RAM) so I think it's safe to stay it's not doing anything? Also the Processed height: 606000, 68.8% makes me think that Fulcrum is aware that my Bitcoin node was stopped at a specific block height that's missing a lot of recent blocks. Not sure what to do with this thing now really, I'd be happy to have it work correctly with up toblock height 606,010 (which is where I have turned connect=0 for my node), but I don't know if that's doable.

You need to wait for flushing to finish. Painful and long, but it will pass. [1]

[1] https://bitcoin.stackexchange.com/questions/116776/why-is-there-a-need-to-flush-the-utxo-set-when-you-prune-blk-dat-and-rev-dat-f

I have restarted the process again but with utxo_cache = 0. I also left my node (while Fulcrum was closed) to sync up to block height 650,543 and then I stopped it again with connect=0. After around 9-11 hours, it's stuck here:

Code:

[2025-01-28 05:03:18.834] <Controller> Processed height: 641000, 72.8%, 5.92 blocks/sec, 13515.6 txs/sec, 58349.8 addrs/sec
[2025-01-28 05:06:15.310] <Controller> Processed height: 642000, 72.9%, 5.67 blocks/sec, 12933.7 txs/sec, 56693.3 addrs/sec
[2025-01-28 05:09:17.796] <Controller> Processed height: 643000, 73.0%, 5.48 blocks/sec, 11590.1 txs/sec, 53866.4 addrs/sec
[2025-01-28 05:11:55.158] <Controller> Processed height: 644000, 73.1%, 6.35 blocks/sec, 13634.1 txs/sec, 60727.5 addrs/sec
[2025-01-28 05:14:34.317] <Controller> Processed height: 645000, 73.2%, 6.28 blocks/sec, 14011.8 txs/sec, 61212.1 addrs/sec
[2025-01-28 05:17:14.472] <Controller> Processed height: 646000, 73.3%, 6.24 blocks/sec, 13882.7 txs/sec, 60763.5 addrs/sec
[2025-01-28 05:20:07.502] <Controller> Processed height: 647000, 73.4%, 5.78 blocks/sec, 12677.3 txs/sec, 56857.9 addrs/sec
[2025-01-28 05:22:45.779] <Controller> Processed height: 648000, 73.5%, 6.32 blocks/sec, 12950.7 txs/sec, 59231.9 addrs/sec
[2025-01-28 05:25:16.626] <Controller> Processed height: 649000, 73.7%, 6.63 blocks/sec, 13106.6 txs/sec, 59100.7 addrs/sec
[2025-01-28 05:27:57.033] <Controller> Processed height: 650000, 73.8%, 6.23 blocks/sec, 13441.1 txs/sec, 59500.2 addrs/sec
[2025-01-28 05:29:20.571] <Controller> Processed 650544 new blocks with 572976592 txs (1411013606 inputs, 1523970373 outputs, 2185459747 addresses), verified ok.

So even if I avoid having it flushing it, it does stop at the block height of my node, but then it does nothing.

[fairmuffin]

Finished syncing everything, including Fulcrum and now it's working; however, I am a bit disappointed of the speed. For example I am using:

Code:

FulcrumAdmin -p 8000 query

I am getting about an average of 1 response every 2-3 seconds. I expected this to be faster, is there a way to benefit of more performance? My setup should be doing better than this since I am running on my main desktop. Also is there a better explorer than BTC RPC Explorer? Especially for its API.

[Cricktor]

Bumping the thread because I've a specific question regarding Fulcrum v2 which I run in the most current version as my personal Electrum server.

Currently I have a Virtualbox VM in which Umbrel runs (not the most recent version of it) and manages docker containers with a non-pruned Bitcoin Core node and a Fulcrum instance that talks to the Core node as blockchain data provider. There are a few other Bitcoin related docker containers on the Umbrel instance, too.

I want to setup a new and up-to-date Umbrel 1.7.4 instance on my Proxmox machine and migrate the Fulcrum database from the Virtualbox VM to the new Proxmox Umbrel. That should be fairly easy, at least I don't expect or anticipate any obstacles.

I'm not too keen to migrate the Virtualbox VM to Proxmox (I don't see real obstacles for this), mainly because it would mean to update and bump the Umbrel version over a few recent version to the current one (Umbrel on the Virtualbox VM is still 1.4.2, my bad).

What I don't want to migrate is the Bitcoin Core node of the Virtualbox VM. I'd rather want to use and copy data from another Bitcoin Core that is older than my Virtualbox Core and has seen some invalid blocks in the past which (I think) the Virtualbox VM Core hasn't seen at all. Call me crazy, but I'm a bit sentimental on the live blockchain history of that older Bitcoin Core full node instance.
(My Bitcoin Core nodes usually run 24/7/365, only interrupted for OS or other mandatory updates.)


Now my specific question (not really sure if someone else except Calin Culianu, the Fulcrum main developer, can confidently give an answer):
Would it be any problem if a Fulcrum v2.x (fully synced by Bitcoin Core node A) will have to talk to another fully synced Bitcoin Core node B in the future?

Both Core nodes serve the same accepted Bitcoin blockchain but their blockchain data files aren't necessarily the same especially when both nodes did their IBD at considerably different times.

I don't expect it matters but I don't fully understand how a Fulcrum instance relies on the Bitcoin Core node it has synced from.

I'm not sure if Calin reads here on bitcointalk.org, but it's worth a try and there are also some other quite bright and knowledgeable people here.