"Rug Monitor" - Monitor your mining rigs (graphs for hashrate, temps, etc.)

[disq]

I've been developing this get-hashrate-generate-graphs solution since yesterday, it's finally at a releaseable stage.
Grab it from GitHub

Requirements: php-cli and rrdtool (and working aticonfig)

Webserver is not required in mining rigs or even at all (You can look at the graphs via sftp etc. if you want, or you can generate the graphs on a separate box which has a webserver running)

Auto-detects the cards you have installed and generates a config file, which then you can edit and rename the cards, turn off the fan display (maybe that particular card doesn't report fan speed correctly for some reason) etc.

To get the hashrate, you need a patch to Phoenix miner. Other miners are not supported right now, but it also supports a 'hashrate' file to read the hash rate from. Details in the README.

Unfortunately won't work on windows since it heavily depends on aticonfig.

WARNING
If running getstatus.php or update.php takes more than 2 seconds for you, or you get occasional gpu at 0% reports (jitters in the charts, etc) you should disable fan monitoring for now. Edit your config.json file and change the "fan":"yes" lines (there are one for each card) to "fan":"no".

Sample Graphs

Hashrate (Two rigs)
https://i.imgur.com/kWVDJ.png

Hashrate (Rigs combined)
https://i.imgur.com/SuRjv.png

GPU Load
https://i.imgur.com/0LyOn.png

Shares Submitted
https://i.imgur.com/82ThX.png

Individual Load vs. Fan vs. Temp Graphs for each card
https://i.imgur.com/cMvJg.png

Planned Features & Known Bugs

• Event notifications (Send e-mail when cards are idle etc)
• Needs better colors
• Check why shares submitted graph doesn't always make sense
• Possible bug in gen-rrd (The RRA's created might be way off)
• Fix fan monitoring, possibly by switching to ADL instead of aticonfig

[1714844209]

1714844209

[1714844209]

1714844209

[1714844209]

1714844209

[1714844209]

1714844209

[memvola]

I put this on a remote rig and it's reporting everything I need. I'll have to wait to see if "Shares Submitted" is working correctly, but overall it's a very useful tool. Thank you!

who tips to forum signatures anyway?

I did.

[PcChip]

Maybe Jeffrey Lebowski can use this program to make sure his rug is never stolen again

[Anonymous]

Heres my rug monitor

[shat]

Is there a modification of the patch for phoenix-1.48 yet?

[mikegogulski]

Heres my rug monitor

I lol'd.

[disq]

Is there a modification of the patch for phoenix-1.48 yet?

Sorry about that, I've just uploaded a patch for phoenix-1.48. Get it from github, put in phoenix dir, apply with "patch -p1 <logtotext-1.48.diff"

[mikegogulski]

This is good stuff, disq. I'm currently thinking about how to modify poclbm output in order to provide input for a Nagios sensor. I'll share it if I come up with anything useful.

[disq]

I recently tried this with 4 gpus (3 cards) on one box. Fan query seems to be a bottleneck, at least for my setup (sdk 2.1)

If running getstatus.php takes more than 2 seconds for you, or you get occasional gpu at 0% reports (jitters in the charts, etc) you should try disabling fan monitoring.

Edit your config.json file and change "fan":"yes" lines to "fan":"no" and see if it changes anything for you.

[Ajego]

nice script

however i had to set the -t /tmp/* path to the home dir of the user, cause i wanna start phoenix through a script, and so the privilege management was easier.

is there a way to create bigger images? i only found the width value, but no height value. you could may add the values to the config?

[disq]

nice script

however i had to set the -t /tmp/* path to the home dir of the user, cause i wanna start phoenix through a script, and so the privilege management was easier.

is there a way to create bigger images? i only found the width value, but no height value. you could may add the values to the config?

If you're running from a usb thumb drive, I highly recommend putting the phoenix output to tmpfs, since writes on flash memory are limited. (btw, noticed that in stock debian6, /tmp isn't tmpfs, it just resides on disk. this is bad of usb sticks as well. you can fix this by mounting /tmp as tmpfs in fstab)

Anyway, to change the chart configuration, currently one has to edit the gen-graph.php file. In later versions I could make the rrdgraph parameters configurable via the config file. But for now, here's a quick reference from RRDtool website: rrdgraph manpage (Scroll down to "Size" section)

[Ajego]

ah ok, very easy to edit the script

one last question, the "shares submitted" graph shows XXm on the y-axis. where is this value calculated? (i think somewhere around line 220)
i thin the 'm' does not stand for 'million'

[mikegogulski]

Rug monitor:

[Zagitta]

It would be nice if you left the files default setting files there so we could edit them manually rather than have to fight with Apache about write premissions and what not :<

[PulsedMedia]

awesome! Tipped you

Next week it's time to start building our cluster mgmt software and was just thinking howto monitor, thanks for solving it!
Rest should be easy

[disq]

one last question, the "shares submitted" graph shows XXm on the y-axis. where is this value calculated? (i think somewhere around line 220)
i thin the 'm' does not stand for 'million'

m is 'milli' (thousandth) the units have SI symbols. The shares submitted graph shows shares per second. The negative Y-axis (below 0) is shares rejected, positive is shares accepted. I had a problem with the shares submitted graph that, sometimes it would get confused and get spikes. Since rrdgraph automatically "zooms out" to view all the data, when this happens the actual shares graph is useless (since it's all zoomed out). Sorry for that, still looking for an explanation/solution.

It would be nice if you left the files default setting files there so we could edit them manually rather than have to fight with Apache about write premissions and what not :<

rrdgraph creates new image files with each generation. You could play with 'umask', but it could be easier (depending on your setup) to schedule a chmod each time after gen-graph runs.

awesome! Tipped you

Next week it's time to start building our cluster mgmt software and was just thinking howto monitor, thanks for solving it!
Rest should be easy

Thanks and good luck!

[tcpip4lyfe]

Very cool.  Anyone have a Cacti template for it?

[cyberchriss]

very nice scripting
I am using manual patched phoenix 1.50.

I noticed, that hasrates per device values were incorrect. (rates were too low)
I am not very familiar in php coding but I found this line in update.php

line 25

Code:

if (is_array($st) && time() - $st['timestamp']<3)

 

I increased the value from 3 to 15 seconds and now the rates seem to be correct.
Is there a reason, why you set the interval of file change date so low?


 

[disq]

very nice scripting
I am using manual patched phoenix 1.50.

I noticed, that hasrates per device values were incorrect. (rates were too low)
I am not very familiar in php coding but I found this line in update.php

line 25

Code:

if (is_array($st) && time() - $st['timestamp']<3)

 

I increased the value from 3 to 15 seconds and now the rates seem to be correct.
Is there a reason, why you set the interval of file change date so low?

fyi, the 1.48 patch can be applied to 1.50 as well. the reason behind that code is the assumption that if the stats isn't updated within the last 3 seconds than there's something wrong with the miner, maybe the card/miner froze or miner is not running. (as phoenix by default updates the file/dumps status every second)

of course, if phoenix updates the stats less frequently, this assumption isn't true. are you running phoenix with a custom avgsamples (-a) setting? i should probably extend the time to something like 30 seconds to cover most possibilities.

[cyberchriss]

I start my miners with

Code:

-t /tmp/gpu1.log -k phatk DEVICE=1 VECTORS BFI_INT AGGRESSION=15 WORKSIZE=64 FASTLOOP=false

I have seen in the code, that phoenix dumps the stats every second.
When I do a "tail -f /tmp/gpu1.log" I recognized that the file updates irregularly, sometimes with >8s delay.
I think 30 seconds for testing phoenix is still alive is okay for most setups.

After I  changed the time for above setting to 15s, my "shares submitted" Graph do not recognize the shares anymore.