[SOLVED] Email Alerts for Headless Ubuntu Rig

[Johnny-Gear]

Hi All,

I am running a 4 card headless box ubuntu box using phoenix miner.

I have an issue where I get the following on some cards occasionally:

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 361, in callback
    self._startRunCallbacks(result)
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 455, in _startRunCallbacks
    self._runCallbacks()
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 542, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1076, in gotResult
    _inlineCallbacks(r, g, deferred)
--- <exception caught here> ---
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1020, in _inlineCallbacks
    result = g.send(result)
  File "/home/bcm/phoenix-1.48/minerutil/RPCProtocol.py", line 220, in _requestComplete
    result = RPCPoller.parse(data)
  File "/home/bcm/phoenix-1.48/minerutil/RPCProtocol.py", line 166, in parse
    raise ServerMessage(message)
minerutil.RPCProtocol.ServerMessage: Long polling is restarting, this doesn't effect mining


I don't really much understand that so if someone could help me with what the actual issue is I would also appreciate.

The main reason I am posting though is that I was wondering if anyone has any experience with integrating mail alerts into the miners or even perhaps sending triggers to a zabbix client?

I am happy to figure things out for myself but don't really have a whole lot of coding/scripting experience and would love some pointers.

Rebooting the box is easy enough to do from my mobile and if I had email alerts would save me a whole lot of downtime.

Thanks,

JG

[1714952105]

1714952105

[1714952105]

1714952105

[1714952105]

1714952105

[1714952105]

1714952105

[kripz]

dpkg-reconfigure exim4-config
nano /etc/exim4/passwd.conf

make some sort of script that does this:

#!/bin/bash
temp=aticonfig --odgt
if temp < 90
echo miner died bro | mail -s miner died, panick support@microsoft.com

shove it into cron

profit

[Johnny-Gear]

dpkg-reconfigure exim4-config
nano /etc/exim4/passwd.conf

make some sort of script that does this:

#!/bin/bash
temp=aticonfig --odgt
if temp < 90
echo miner died bro | mail -s miner died, panick support@microsoft.com

shove it into cron

profit

I didn't think of that. Excellent idea, thank you so much. You are a scholar and a gentleman kind Sir.

JG

[Johnny-Gear]

dpkg-reconfigure exim4-config
nano /etc/exim4/passwd.conf

make some sort of script that does this:

#!/bin/bash
temp=aticonfig --odgt
if temp < 90
echo miner died bro | mail -s miner died, panick support@microsoft.com

shove it into cron

profit

Hi again,

I am trying to implement your suggestion and am a little bit stuck.

I don't have much experience with sed or regexp and am thinking I will need to use sed to get the temp variable.

Does anyone know how I would get just the int or float temperature value from the following lines using sed?

Code:

Adapter 0 - ATI Radeon HD 5800 Series 
            Sensor 0: Temperature - 77.50 C

I have tried reading through the seemingly limitless texts on sed, but I am not having much luck implementing it. I would really appreciate some concise tips.

JG

[Ali]

The new page miningmonitor.com has mining alerts via SMS or mail.
It has been up for only a few hours but should offers exactly what you are asking for.

Please use my affiliate link to sign up: http://www.miningmonitor.com/Hedge

[Johnny-Gear]

The new page miningmonitor.com has mining alerts via SMS or mail.
It has been up for only a few hours but should offers exactly what you are asking for.

Please use my affiliate link to sign up: http://www.miningmonitor.com/Hedge

Thanks for the info.

Unfortunately, I currently run 4 workers, and would rather not pay 1 BTC a month for something I should be able to figure out in not too long. I hope to share my solution for free as well.

Guess I will be reading up on regex

JG

[Ali]

Yeah, you can of course do it yourself at well. I can't help you with this though (sorry, no linux-man here).
For me the silver-membership (1 BTC a year) is enough.

[Johnny-Gear]

Yeah, you can of course do it yourself at well. I can't help you with this though (sorry, no linux-man here).
For me the silver-membership (1 BTC a year) is enough.

I am getting closer - I can remove the variable I am trying to capture... so atleast I have the regex pattern right.

Code:

DISPLAY=:0 aticonfig --odgt --adapter=0 | sed 's/[0-9][0-9]\.[0-9][0-9]//'

All I need to do now is to figure out how to make sed only output the pattern I matched and I should be able to implement mail alerts pretty easily.

Help still welcome

JG

[Johnny-Gear]

Yeah, you can of course do it yourself at well. I can't help you with this though (sorry, no linux-man here).
For me the silver-membership (1 BTC a year) is enough.

I am getting closer - I can remove the variable I am trying to capture... so atleast I have the regex pattern right.

Code:

DISPLAY=:0 aticonfig --odgt --adapter=0 | sed 's/[0-9][0-9]\.[0-9][0-9]//'

All I need to do now is to figure out how to make sed only output the pattern I matched and I should be able to implement mail alerts pretty easily.

Help still welcome

JG

Code:

DISPLAY=:0 aticonfig --odgt --adapter=0 | sed 's/.*\([0-9][0-9]\.[0-9][0-9]\).*/\1/'

That is getting me just the float on the last line. I just can't get rid of the first line

JG

[Johnny-Gear]

Yeah, you can of course do it yourself at well. I can't help you with this though (sorry, no linux-man here).
For me the silver-membership (1 BTC a year) is enough.

I am getting closer - I can remove the variable I am trying to capture... so atleast I have the regex pattern right.

Code:

DISPLAY=:0 aticonfig --odgt --adapter=0 | sed 's/[0-9][0-9]\.[0-9][0-9]//'

All I need to do now is to figure out how to make sed only output the pattern I matched and I should be able to implement mail alerts pretty easily.

Help still welcome

JG

Code:

DISPLAY=:0 aticonfig --odgt --adapter=0 | sed 's/.*\([0-9][0-9]\.[0-9][0-9]\).*/\1/'

That is getting me just the float on the last line. I just can't get rid of the first line

JG

YAY!

Code:

DISPLAY=:0 aticonfig --odgt --adapter=0 | sed -n '/[0-9][0-9]\.[0-9][0-9]/,$p' | sed 's/.*\([0-9][0-9]\.[0-9][0-9]\).*/\1/'

piped twice but atleast it works. I am sure a true sed / regex guru could clean it up, but feels good getting there myself.

JG

[Johnny-Gear]

Hi All,

I am just posting in case anyone else requires a solution like this.

There are a couple of pre-requisites; Some sort of local SMTP server or smart host setup (I am using Exim4), and, a working aticonfig installation. As per the heading it should be pretty obvious this was done on Ubuntu, but any debian based distro would probably work. Heck, I can't really imagine there being too many issues with a RH based distro either; driver support may be more difficult though.

Here is the simple script I use (some changes may be required as this is customized for my setup):

Code:

#!/bin/sh
cd /home/<user>
MIN="55"
TEMP0=$( DISPLAY=:0 aticonfig --odgt --adapter=0 | grep -oE '[0-9]{2}[.][0-9]{2}' | sed -r 's/([0-9]*).*/\1/g' )
TEMP1=$( DISPLAY=:0 aticonfig --odgt --adapter=1 | grep -oE '[0-9]{2}[.][0-9]{2}' | sed -r 's/([0-9]*).*/\1/g' )
TEMP2=$( DISPLAY=:0 aticonfig --odgt --adapter=2 | grep -oE '[0-9]{2}[.][0-9]{2}' | sed -r 's/([0-9]*).*/\1/g' )
TEMP3=$( DISPLAY=:0 aticonfig --odgt --adapter=3 | grep -oE '[0-9]{2}[.][0-9]{2}' | sed -r 's/([0-9]*).*/\1/g' )

if [ "$TEMP0" -lt "$MIN" -o "$TEMP1" -lt "$MIN" -o "$TEMP2" -lt "$MIN" -o "$TEMP3" -lt "$MIN" ]; then
	echo "ALERT: FAILURE ON <host> - A MINER HAS DIED" | mail -s "ALERT: FAILURE ON <host> - A MINER HAS DIED" <email@address.com>
else
	echo "$(date) - gpu0 = $TEMP0, gpu1 = $TEMP1, gpu2 = $TEMP2, gpu3 = $TEMP3" >> tempLog
fi

Here is my crontab entry (to add one for yourself ~$ crontab -e):

Code:

*/15 * * * * /home/<user>/tempCheck.sh

Make sure you specify correct directories in your script as cron doesn't operate from your home folder. Also make sure the script is executable so cron can run it.

JG

[Johnny-Gear]

Hey All,

Don't mean to bump old threads, but just thought I would add some changes I made to the script I have been running as it was slightly annoying and costly when I couldn't get to the box quick enough to reboot it, so here is the new script:

Code:

#!/bin/sh
cd /home/<user>
MIN="40"
TEMP0=$( DISPLAY=:0 aticonfig --odgt --adapter=0 | grep -oE '[0-9]{2}[.][0-9]{2}' | sed -r 's/([0-9]*).*/\1/g' )
TEMP1=$( DISPLAY=:0 aticonfig --odgt --adapter=1 | grep -oE '[0-9]{2}[.][0-9]{2}' | sed -r 's/([0-9]*).*/\1/g' )
TEMP2=$( DISPLAY=:0 aticonfig --odgt --adapter=2 | grep -oE '[0-9]{2}[.][0-9]{2}' | sed -r 's/([0-9]*).*/\1/g' )
TEMP3=$( DISPLAY=:0 aticonfig --odgt --adapter=3 | grep -oE '[0-9]{2}[.][0-9]{2}' | sed -r 's/([0-9]*).*/\1/g' )

FAILVAL=$(cat /home/<user>/.TCStore)

if [ "$TEMP0" -lt "$MIN" -o "$TEMP1" -lt "$MIN" -o "$TEMP2" -lt "$MIN" -o "$TEMP3" -lt "$MIN" ]; then
	if [ "$FAILVAL" -eq "0" ]; then
		echo > .TCStore
		echo "1" > .TCStore
	elif [ "$FAILVAL" -eq "1" ]; then
		echo > .TCStore
		echo "2" > .TCStore
	elif [ "$FAILVAL" -eq "2" ]; then
		echo > .TCStore
		echo "0" > .TCStore
		reboot
		echo "ALERT: FAIL VALUE REACHED 3 ON <host> - REBOOTING" | mail -s "ALERT: FAIL VALUE REACHED 3 ON <host> - REBOOTING" <user>@<domain>
	else
		echo "$(date) - ERROR: FAILVAL = $FAILVAL - setting it back to 0" >> .FailValLog
		echo > .TCStore
		echo "0" > .TCStore
else
        echo > .TCStore
	echo "0" > .TCStore
	echo "$(date) - gpu0 = $TEMP0, gpu1 = $TEMP1, gpu2 = $TEMP2, gpu3 = $TEMP3" >> tempLog
fi

Still fairly primitive but seems to get the job done. I have also slightly changed times cron runs from every 15 minutes to every 20 minutes which I feel is better due to the fact the actual pool could be the cause of GPU's not mining and not the GPU's themselves. Basically having this script and cron running it every 20 minutes would mean that if there was any 60 minute period where any of the GPU temps were down to a level I can safely say there is no mining going on the rig reboots itself. This could obviously been refined so that separate GPU temps and their consecutive low temps were logged making it more accurate, however, this currently works for me and I don't really care enough to refine it.

Happy mining folks,

JG

EDIT: Forgot to mention, obviously I have some start-up scripts than start my miners and do some overclocking - If you don't have this, you would need to implement it for this script and technique to be useful.