Electrum server discussion thread

[Red Emerald]

Okay, I've been trying out Electrum as a portable solution, but find I seem to be inhibited in many public places.  I presume due to firewalls that prohibit high number/non-standard ports.  Is there a public server that can fake being a standard webserver using port 80?  Can this even ben done?

Theres comments in the config file for doing exactly this and I seem to recall seeing a partial readme for it.  I don't know if any servers have done this though.

Once I get my server up publicly, I'll have it listening on 80 and 443

[1714469258]

1714469258

[1714469258]

1714469258

[1714469258]

1714469258

[1714469258]

1714469258

[1714469258]

1714469258

[EagleTM]

electrum.no-ip.org is listening on 80 and 443 specifically for this reason. Feel free to use it, just select HTTP or HTTPS and the server from the list.

I've added the report_* config options a while back (see sample config and README.IRC) to be able to have the server report a different port on IRC than it's bound to locally. This is useful to run the server as an unprivileged user and then to NAT, rinetd or haproxy, the standard high port (8081 for example) to 80.


Edit: spelling and clarification

[BkkCoins]

I've added the report_* config options a while back (see sample config and README.IRC) to be able to have the server report a different port on IRC than it's bound to locally. This is useful to run the server as an unprivileged user and then to NAT, rinetd or haproxy, the standard high port (8081 for example) to 80.

Does this support reporting multiple different hosts and also multiple ports? Both would be very useful and IMO needed. Reporting a secondary name would be useful for running a Tor hidden service alongside a normal one. Reporting more than one port would be useful for more flexible access. eg. port 80 and 110. Or configuring a tunnel to hop thru a proxy/firewall for certain users, exposing a secondary access location.

Or perhaps clients don't support more than one port since currently it doesn't present as a dropdown box for port number. I suppose port # could be manually configured as "special knowledge" but I still see multiple hosts as very useful.

[EagleTM]

Does this support reporting multiple different hosts and also multiple ports? Both would be very useful and IMO needed. Reporting a secondary name would be useful for running a Tor hidden service alongside a normal one. Reporting more than one port would be useful for more flexible access. eg. port 80 and 110. Or configuring a tunnel to hop thru a proxy/firewall for certain users, exposing a secondary access location.

Each server joins #electrum with a unique nickname prefixed by E_ and publishes its info in its name field which can be queried /whois . The limit here is the IRC name field - which has a maximum of 50 chars. Already at this time if your hostname is too long the ports for TCP, TLS, HTTP, HTTPS don't fit in.

Adding multiple hostnames and ports would be fairly complicated - moreover the clients would need to decode this and so forth. The KISS approach is one server instance, one user on IRC, one hostname and a specific set of ports.

What could certainly be done (but no priority) is mutliple server instances sharing one leveldb database with one master instance updating the db while the others just use the same and talk to bitcoind to send out tx.

[BkkCoins]

Does this support reporting multiple different hosts and also multiple ports? Both would be very useful and IMO needed. Reporting a secondary name would be useful for running a Tor hidden service alongside a normal one. Reporting more than one port would be useful for more flexible access. eg. port 80 and 110. Or configuring a tunnel to hop thru a proxy/firewall for certain users, exposing a secondary access location.

Each server joins #electrum with a unique nickname prefixed by E_ and publishes its info in its name field which can be queried /whois . The limit here is the IRC name field - which has a maximum of 50 chars. Already at this time if your hostname is too long the ports for TCP, TLS, HTTP, HTTPS don't fit in.

Adding multiple hostnames and ports would be fairly complicated - moreover the clients would need to decode this and so forth. The KISS approach is one server instance, one user on IRC, one hostname and a specific set of ports.

What could certainly be done (but no priority) is mutliple server instances sharing one leveldb database with one master instance updating the db while the others just use the same and talk to bitcoind to send out tx.

Or couldn't the server code just join IRC as a new user for each different host name? So when parsing the config it joins IRC once for each report line it encounters, with maybe some suffix on nickname.
eg. nick E_<name>_1 etc.

I don't know much at all about IRC so maybe there's some limitation preventing that but this seems like fairly KISS as well. I never logged into #electrum but doing this just now I was wondering what name my server would use when I don't set a nickname in the config?

edit: Oh, I found myself. It's just some random gibberish name. Maybe I'll go add a proper nick then.
edit2: Hmm. Maybe this is why my https port doesn't show up - not enough space. Isn't there a better way to do this?

[Red Emerald]

Does this support reporting multiple different hosts and also multiple ports? Both would be very useful and IMO needed. Reporting a secondary name would be useful for running a Tor hidden service alongside a normal one. Reporting more than one port would be useful for more flexible access. eg. port 80 and 110. Or configuring a tunnel to hop thru a proxy/firewall for certain users, exposing a secondary access location.

Each server joins #electrum with a unique nickname prefixed by E_ and publishes its info in its name field which can be queried /whois . The limit here is the IRC name field - which has a maximum of 50 chars. Already at this time if your hostname is too long the ports for TCP, TLS, HTTP, HTTPS don't fit in.

Adding multiple hostnames and ports would be fairly complicated - moreover the clients would need to decode this and so forth. The KISS approach is one server instance, one user on IRC, one hostname and a specific set of ports.

What could certainly be done (but no priority) is mutliple server instances sharing one leveldb database with one master instance updating the db while the others just use the same and talk to bitcoind to send out tx.

Or couldn't the server code just join IRC as a new user for each different host name? So when parsing the config it joins IRC once for each report line it encounters, with maybe some suffix on nickname.
eg. nick E_<name>_1 etc.

I don't know much at all about IRC so maybe there's some limitation preventing that but this seems like fairly KISS as well. I never logged into #electrum but doing this just now I was wondering what name my server would use when I don't set a nickname in the config?

edit: Oh, I found myself. It's just some random gibberish name. Maybe I'll go add a proper nick then.
edit2: Hmm. Maybe this is why my https port doesn't show up - not enough space. Isn't there a better way to do this?

I know some IRC servers have limits on the number of connections allowed from a single IP.

[EagleTM]

I don't know much at all about IRC so maybe there's some limitation preventing that but this seems like fairly KISS as well. I never logged into #electrum but doing this just now I was wondering what name my server would use when I don't set a nickname in the config?

edit: Oh, I found myself. It's just some random gibberish name. Maybe I'll go add a proper nick then.
edit2: Hmm. Maybe this is why my https port doesn't show up - not enough space. Isn't there a better way to do this?

Good idea - yes you could implement the irc thread to open multiple connections. It'd need a smart way of reading / iterating this from the server config without having to define lots of defaults. The limit IRC servers place for connections by IP is somewhere around 4-5 and it applies to a single server - connecting to irc.freenode.net will spread the connection to various servers. I'd assume most operators only need at most one secondary nick (currently no one else has this requirement). As for Tor - this would yet again need more thought because publishing a Tor node on public IRC will give people a bad experience by trying to connect to it outside of Tor. I don't really see the necessity either - people can and do connect to public servers from Tor and this works just fine. You could just publish the Tor address in your server info tab.

About the space issue: If you switch to Port 80 and 443 you will save a couple of chars. You're currently lacking 5.  Changing h8080 to h80 gives you two, so you might need to change t50001 or s50002 to a lower port number or try to shorten your hostname. There was the idea to make port numbers implicit - i.e. if they are not published the client assumes it's t50001 for example but this needs to be implemented in the server and all servers need to update their codebase in a relatively short timeframe to make this work.

Edit: Note about Tor address publishing on info tab, server propagation for implicit port names (not client, they don't query IRC)

[BkkCoins]

I'd assume most operators only need at most one secondary nick (currently no one else has this requirement). As for Tor - this would yet again need more thought because publishing a Tor node on public IRC will give people a bad experience by trying to connect to it outside of Tor. I don't really see the necessity either - people can and do connect to public servers from Tor and this works just fine. You could just publish the Tor address in your server info tab.

I think probably one would be enough. I can't think of another reason for having multiple names off hand. My idea with publishing a Tor address was that it would only show up in the list if a socks proxy was enabled. That way any client wanting to use Tor would choose socks mode and immediately see a list of available onions. Maybe even better, add a Tor proxy mode (HTTP, SOCKS4, SOCKS5, TOR) that is just the same as SOCKS5 but only lists onions. This makes it dead easy for someone wanting to connect this way. I guess this is a pretty small tweak as those keen to do this can probably manually enter onion addresses - it's just kind of nice if the one you use goes down to be able to choose an alternate from a list without going out to research it.

About the space issue: If you switch to Port 80 and 443 you will save a couple of chars. You're currently lacking 5.  Changing h8080 to h80 gives you two, so you might need to change t50001 or s50002 to a lower port number or try to shorten your hostname.

I have a shorter domain name available so I may just switch back to that one. It's too bad there isn't another field available in IRC to store some user comment or data. Perhaps an alternate approach would be if no port info is present the client could hit the server itself with an rpc call that provides details. In fact, this could provide an onion name if available. (Or maybe not as that would mean every client hitting a server and thus potentially a significant load).

Anyway these are just ideas for consideration as development advances. I've provided pull requests in the past on the client and if there was some general agreement that an idea is worth doing then I'd be happy to implement it and send a pull on the server as well. I'm trying to get to a backlog of features I have I'd like to sync with the new version client and then send pulls for them. Soon I hope.

[ThomasV]

on january 13th, a silent bug was introduced in the server code.
I fixed it today.
please pull the new code if you upgraded after this date!

[Red Emerald]

on january 13th, a silent bug was introduced in the server code.
I fixed it today.
please pull the new code if you upgraded after this date!

Oops. Sorry about that.

After I get some time, I'll get a PR in for my logging branch.  Then I'll work on unit tests.

[ThomasV]

another bugfix: if your bitcoind is older than this commit, please update/rebase :

https://github.com/bitcoin/bitcoin/commit/cd7fb7d1deece9da15d7750b3e05f729555a2cbe

[ThomasV]

one more bugfix commited this morning, this time for abe servers:
https://github.com/spesmilo/electrum-server/commit/4d465a7b3cb02c668440c03dd71d82b98f947637

[Red Emerald]

will you loose any coins?

can you explain your question?

When I update it, will I still have my coins afterword?

Updating the server will do nothing to your coins.

[piuk]

Thomas, please could you have a look at https://github.com/spesmilo/electrum-server/pull/23

[ajze]

I'm having issues getting Electrum server to sync with bitcoind.  It works fine for a short time then says this:

Code:

17/02/2013-05:20:34] catch_up: block 23100 (0.639s)
[17/02/2013-05:20:34] catch_up: block 23200 (0.638s)
Traceback (most recent call last):
  File "/home/bitcoind/electrum/electrum-server/backends/bitcoind/blockchain_processor.py",$
    respdata = urllib.urlopen(self.bitcoind_url, postdata).read()
  File "/usr/lib/python2.7/urllib.py", line 88, in urlopen
    return opener.open(url, data)
  File "/usr/lib/python2.7/urllib.py", line 209, in open
    return getattr(self, name)(url, data)
  File "/usr/lib/python2.7/urllib.py", line 344, in open_http
    h.endheaders(data)
  File "/usr/lib/python2.7/httplib.py", line 954, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python2.7/httplib.py", line 814, in _send_output
    self.send(msg)
  File "/usr/lib/python2.7/httplib.py", line 776, in send
    self.connect()
  File "/usr/lib/python2.7/httplib.py", line 757, in connect
    self.timeout, self.source_address)
  File "/usr/lib/python2.7/socket.py", line 571, in create_connection
 raise err
IOError: [Errno socket error] [Errno 110] Connection timed out
[17/02/2013-05:20:56] Stopping Stratum
Exception in thread Thread-4:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 755, in run
    self.function(*self.args, **self.kwargs)
  File "/home/bitcoind/electrum/electrum-server/backends/bitcoind/blockchain_processor.py",$
    threading.Timer(0, lambda: self.catch_up(sync=False)).start()
  File "/home/bitcoind/electrum/electrum-server/backends/bitcoind/blockchain_processor.py",$
    next_block_hash = self.bitcoind('getblockhash', [self.height + 1])
  File "/home/bitcoind/electrum/electrum-server/backends/bitcoind/blockchain_processor.py",$
    r = loads(respdata)
UnboundLocalError: local variable 'respdata' referenced before assignment

Looks like a time out issue.  I have it running on a virtual server that may be the source of the delays in communicating w/ bitcoind.

I don't speak python, but this seems to magically make the time out 60s instead of 1s:

electrum-server/backends/bitcoind/blockchain_processor.py line 87:

was:

Code:

        try:
            respdata = urllib.urlopen(self.bitcoind_url, postdata).read()
        except:
            traceback.print_exc(file=sys.stdout)
            self.shared.stop()

Changed to:

Code:

       try:
            respdata = urllib.urlopen(self.bitcoind_url, postdata).read()
        except:
            try:
                respdata = urllib.urlopen(self.bitcoind_url, postdata).read()
            except:
                traceback.print_exc(file=sys.stdout)
                self.shared.stop()

After changing this, it runs much longer before it dies with the same message.  I'm sure there's a much better way to do this.  Maybe some who knows about this stuff can fix it or at least change the time out to a value that will survive the server being overloaded temporarily.

[Edit]
Moved leveldb to shm to see if it helped performance.  It didn't.  The issue appears to be caused by high cpu usage.

[phantastisch]

I'm having issues getting Electrum server to sync with bitcoind.  It works fine for a short time then says this:

Code:

17/02/2013-05:20:34] catch_up: block 23100 (0.639s)
[17/02/2013-05:20:34] catch_up: block 23200 (0.638s)
Traceback (most recent call last):
  File "/home/bitcoind/electrum/electrum-server/backends/bitcoind/blockchain_processor.py",$
    respdata = urllib.urlopen(self.bitcoind_url, postdata).read()
  File "/usr/lib/python2.7/urllib.py", line 88, in urlopen
    return opener.open(url, data)
  File "/usr/lib/python2.7/urllib.py", line 209, in open
    return getattr(self, name)(url, data)
  File "/usr/lib/python2.7/urllib.py", line 344, in open_http
    h.endheaders(data)
  File "/usr/lib/python2.7/httplib.py", line 954, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python2.7/httplib.py", line 814, in _send_output
    self.send(msg)
  File "/usr/lib/python2.7/httplib.py", line 776, in send
    self.connect()
  File "/usr/lib/python2.7/httplib.py", line 757, in connect
    self.timeout, self.source_address)
  File "/usr/lib/python2.7/socket.py", line 571, in create_connection
 raise err
IOError: [Errno socket error] [Errno 110] Connection timed out
[17/02/2013-05:20:56] Stopping Stratum
Exception in thread Thread-4:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 755, in run
    self.function(*self.args, **self.kwargs)
  File "/home/bitcoind/electrum/electrum-server/backends/bitcoind/blockchain_processor.py",$
    threading.Timer(0, lambda: self.catch_up(sync=False)).start()
  File "/home/bitcoind/electrum/electrum-server/backends/bitcoind/blockchain_processor.py",$
    next_block_hash = self.bitcoind('getblockhash', [self.height + 1])
  File "/home/bitcoind/electrum/electrum-server/backends/bitcoind/blockchain_processor.py",$
    r = loads(respdata)
UnboundLocalError: local variable 'respdata' referenced before assignment

Looks like a time out issue.  I have it running on a virtual server that may be the source of the delays in communicating w/ bitcoind.

I don't speak python, but this seems to magically make the time out 60s instead of 1s:

electrum-server/backends/bitcoind/blockchain_processor.py line 87:

was:

Code:

        try:
            respdata = urllib.urlopen(self.bitcoind_url, postdata).read()
        except:
            traceback.print_exc(file=sys.stdout)
            self.shared.stop()

Changed to:

Code:

       try:
            respdata = urllib.urlopen(self.bitcoind_url, postdata).read()
        except:
            try:
                respdata = urllib.urlopen(self.bitcoind_url, postdata).read()
            except:
                traceback.print_exc(file=sys.stdout)
                self.shared.stop()

After changing this, it runs much longer before it dies with the same message.  I'm sure there's a much better way to do this.  Maybe some who knows about this stuff can fix it or at least change the time out to a value that will survive the server being overloaded temporarily.

[Edit]
Moved leveldb to shm to see if it helped performance.  It didn't.  The issue appears to be caused by high cpu usage.

I have the same problem on a dedicated server with a lot of power and there is no cpu usage. I was able to sync up to the latest blocks since of 15:51 then this started happening.

[EagleTM]

The crash message looks like bitcoind quit or is busy answering which will in turn make electrum server bail out...

Make sure you're running the latest bitcoind from git and let bitcoind sync up first before running electrum. If it still doesn't work then, we'll have to give it a close look.

[ThomasV]

bitcoind quit or is busy answering which will in turn make electrum server bail out...

yes, please note that this is expected behaviour.
the Electrum server shuts itself down in order to avoid sending lagging information to its clients

[phantastisch]

So it is me again ,

I am trying to run an electrum server on my node but i can not get irc to work.

If I set it to yes in the options I get 100% CPU load on 1 Core and no peers.

There are no ports closed, i asked my hosting provider and i made sure to disable the firewall.

I am able to connect to it with Electrum 1.6.2 at electrum.cpsh.me per tcp 50001 and http 8081

When i run electrum peers i get this message in the log :

[22/02/2013-17:55:22] Starting Electrum server on electrum.cpsh.me
[22/02/2013-17:55:22] hist [('0000000000000113d3646e4eb83c7392c046d7145bd6b80a1f2ee4b5aa650310', 222560, 0)]
[22/02/2013-17:55:22] catching up missing headers: 222516 222560
[22/02/2013-17:55:23] blockchain is up to date.
[22/02/2013-17:55:24] TCP server started.
[22/02/2013-17:55:24] HTTP server started.
[22/02/2013-17:55:35] blockchain: 222560 (0.028s)
[22/02/2013-17:56:26] blockchain: 222561 (10.456s)
[22/02/2013-18:04:01] unknown method {u'params': [u'secret'], u'id': 2, u'method': u'server.peers'}

Things I already investigated :

• changed the connection to the irc server on another port accordingly to freenode servers manually in the irc init file.
• monitored the ports
• reverted the last commit

things i suspect :

• the hostname

the hostname file says : electrum.cpsh.me

the hosts file says : <IpeditedoutbyMe> electrum.cpsh.me

I would appreciate any help,

thank you

[BkkCoins]

I'm having trouble getting electrum-server to run with the latest bitcoind from github.

I keep getting this,

Code:

Exception in thread Thread-4:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 755, in run
    self.function(*self.args, **self.kwargs)
  File "/usr/local/src/electrum-server/backends/bitcoind/blockchain_processor.py", line 72, in <lambda>
    threading.Timer(0, lambda: self.catch_up(sync=False)).start()
  File "/usr/local/src/electrum-server/backends/bitcoind/blockchain_processor.py", line 652, in catch_up
    next_block_hash = self.bitcoind('getblockhash', [self.height + 1])
  File "/usr/local/src/electrum-server/backends/bitcoind/blockchain_processor.py", line 95, in bitcoind
    raise BaseException(r['error'])
BaseException: {u'message': u'Block number out of range.', u'code': -1}

"Block number out of range" - what?

I updated my bitcoind today and it starts ok but electrum doesn't want to play with it.

edit: Just noticed that blocks being logged in bitcoin/debug.log are from 2011. Is this right - the new version has to reload the whole blockchain or something? It was up to date with previous pre-0.8 beta and I thought it would be a good idea to catch up with new release.

edit again: Oh. Well, now that the blockchain has fully re-indexed/downloaded it does work ok. For anyone about to update note that going from bitcoind 0.7.1 to 0.8.0 does this and took quite a few hours on my server.

BTW I changed my server URL from electrum.spiderfish.net back to electrum.tk (a name I briefly used before). I like spiderfish better but the name is too long to allow the SSL/HTTPS details to be passed via IRC. Not sure why that changed today as SSL was working before, and then today it stopped, until I restarted with the shorter name. I updated electrum server as well so maybe some changes occurred.