[Electrum] Independent monitoring of running servers?

[slush]

Hello all,

weeks ago I knocked up simple script which crawl over all public electrum nodes, connect to them and periodically check if they're up to date with their blockchains:

http://pastebin.com/UsBWCU4n
(use easy_install twisted stratum to install dependencies)

Is there somebody interested in providing (public?) service which will:

a) Show current status of all known servers (up/down)
b) Send mail to electrum operator if his node crashed

It should be quite trivial for some pythonist by using my code, but I don't have free time to finalize it into some usable form.

Such service would be really useful, because electrum servers are sometimes crashing (you know, it's still beta software) and it may took some time to admin to realize that his node is down.

Edit 03.03.2013: http://electrum.be/

[1713984727]

1713984727

[1713984727]

1713984727

[1713984727]

1713984727

[1713984727]

1713984727

[1713984727]

1713984727

[1713984727]

1713984727

[Tachikoma]

I would love to build this. I think it's really necessary since Electrum servers sometime lag behind, which can be quite annoying. This should be right up my alley to see I am a web-developer by day

[slush]

That code on pastebin uses Twisted library, which may twist someone's head for first look :-), but it should be really easy to hack that code to report broken server to database/logfile.

[wabber]

I think it would also be useful if that was included in the electrum client. That way ppl could compare the blockchain length of the nodes directly in the server list.

[slush]

I think that patches are welcome .

Although the primary reason for all this is notifying server admins about issues on their servers.

[marcus_of_augustus]

This seems like a good basis for a module in the client that does server 'auditing' and etc for more decentralisation of Electrum layer (e.g. choose m of n servers in some kind of multi-party server selection protocol)

[BkkCoins]

This seems like a good basis for a module in the client that does server 'auditing' and etc for more decentralisation of Electrum layer (e.g. choose m of n servers in some kind of multi-party server selection protocol)

If this is done it should be possible to be disabled by users. Connecting to many servers may represent information leakage for users who want to avoid disclosure that they are using Electrum. Right now users can choose who to connect to, and as far as we know there are no malicious servers. But perhaps later when it's difficult to monitor server authenticity someone could setup a server with the intent of logging who is using Electrum and at what IPs, and what addresses they subscribe. For security reasons users should control who they allow connection to.

[marcus_of_augustus]

This seems like a good basis for a module in the client that does server 'auditing' and etc for more decentralisation of Electrum layer (e.g. choose m of n servers in some kind of multi-party server selection protocol)

If this is done it should be possible to be disabled by users. Connecting to many servers may represent information leakage for users who want to avoid disclosure that they are using Electrum. Right now users can choose who to connect to, and as far as we know there are no malicious servers. But perhaps later when it's difficult to monitor server authenticity someone could setup a server with the intent of logging who is using Electrum and at what IPs, and what addresses they subscribe. For security reasons users should control who they allow connection to.

I agree.

All good points. It's again that balance between dealing with a known authenticated entity who you trust with your info or dealing with many who you semi (or don't have to) trust ... something to keep a look out for further down the track for sure.

[Tachikoma]

Just a little update, I'm working on it but since time it's limited It's going slowly.

Here is a little W.I.P. screenie:

[slush]

Nice! Are you using my script internally or did you implement checks youself?

[Deafboy]

I love you guys! (not in a gay way )
I was thinking about something like this for a few weeks. I'm using Nagios for electrum and bitcoind monitoring, but tool like this is needed.

[slush]

I was thinking about something like this for a few weeks. I'm using Nagios for electrum and bitcoind monitoring, but tool like this is needed.

Can you publish these nagios scripts, please? I had quite sophisticated scripts for nagios for the pool, but I lost them during Linode issue and I still didn't find a time to set it up back, at least in some limited form :-/. Some template for Electrum monitoring would motivate me to set up nagios again...

[Tachikoma]

Nice! Are you using my script internally or did you implement checks youself?

Your script, I assessed the options but the quickest way to get this online was using your script. So that's what I decided on.

I have thought about how to subscribe yourself to notifications and decided on a single token signup kind of thing.

So if you want to get alerts about one of the servers you fill in your email address you would like to receive the notifications on. A email will be send with a confirmation token, once you click the link and fill in the confirmation token you will start receiving emails. Every alert email will have an "unsubscribe" link to stop getting alerts.

I think I like this over full user accounts since this is probably something you will only do once and then forget about it, creating a whole new account for yet an other website seems silly. Ideas?

[slush]

Yes, I think that confirmation tokens are fine for it.

[Tachikoma]

Ok! First prototype is online at Eyelectrum.herokuapp.com. You can subscribe to your own server, or others, and should receive one email once your server lags two blocks behind.

There is no check build in yet to see if a server is actually online. I will try to work that in over the next few days.

[BkkCoins]

Looks great! Just a small detail. On my Firefox 15.0.1/Ubuntu 11.04 combination the "online/lagging" indicator is cut off by the block # so that you cannot read it fully.

[Tachikoma]

Looks great! Just a small detail. On my Firefox 15.0.1/Ubuntu 11.04 combination the "online/lagging" indicator is cut off by the block # so that you cannot read it fully.

Ah thanks! I only check in Chrome :$, will fix it in the next update.

[ThomasV]

Electrum servers can have a lag of 1 (or even two) blocks caused by propagation delays in the bitcoin network. This is normal, and I would not consider this as an issue.

For larger lags, the current "official" server code also makes sure that the server does not go online until it has caught up with the blockchain.
If this mechanism fails, I would rather try to work on fixing/improving it, rather than asking users to use a monitoring website.
I believe that it would be very bad for Electrum usability if we cannot solve this problem at the server level.

now, this monitoring website could be useful for something else: to measure the percentage of uptime of servers, and maybe their ping time. I think this would be useful.

[Tachikoma]

Keeping track of uptime / lagtime would be the next step. I agree that fixing the server code would be much better. Sadly at this moment nobody is doing it, and I don't have the expertise to do it. So in the meantime this is a viable solution.

[slush]

If this mechanism fails, I would rather try to work on fixing/improving it, rather than asking users to use a monitoring website.

Such monitoring is helpful mostly for server administrators than for end users. Server may fail not only because of bug in server code. For example my node went out of memory and stopped responding. I didn't have a monitoring and I didn't noticed it for a week.

Having easy server monitoring doesn't mean that we cannot improve server code itself. We definitely need both - stable server AND independent monitoring.