Depending on the mobo I can get 25/26 MHs per 1060 under genoil with the watchdog script with -75 and +1450 with power limit at 82W.
With Claymore I was able to crank it up to 170/1700/82 and get another 1-2 MHs per card and it was relatively stable but when it did crash it did not recover without intervention  so for now I'm going with Genoil and finding different ways to deal with the remaining instability (which to me means a reboot ever 1-2 days at the moment).

Hey monit looks nice, I'm about half way there to taking the output of genoil and turing it into a hearbeat, then to remotely cycle the server, the hard part is how to define the conditions of when to trigger that event.

@ tempgoga - I'm curious on your approach, this is next on my plate (without a remote power switch) the easiest  - that seems applicable to my setup - is to filter the Genoil watchdog script for a reduced hash rate threshold instead of just 'error' since that script filters stdout from Genoil so technically the info is there to capture already.
I haven't gotten around to this yet but I'm hoping this weekend.

Another idea that I'm thinking of is some sort of port-knocking from a remote machine - it could be enough since the rigs usually are responsive to ssh or local scripts after they "soft crash" with the video cards - but this won't help in case of a complete freeze.
Then you need a remote power cycle ability which is a whole different level of infra.

Cheers!

First of all big thank you to fullzero and everyone contributing to this distro!

I've been struggling with the Genoil crash issue and lack of watchdog implementation for the past few days and I have a bandaid solution that seems to be actually working quite well, perhaps it can help others in the community:

Essentially you need to split the Genoil output to a file, grep it (we only care about 'error' instances only ;  and then this output as input for a monitoring script that kills and restarts the misbehaving process.

So we have 2 scripts launched in screen as daemons "ltail" script and "ett" script

$screen -dmS ltail sh ~/eth/Genoil-U/ltail
and
$screen -dmS ett bash ~/ett

ltail:
--------------------------
#!/bin/bash
echo listening...
cd ~/eth/Genoil-U/
tail -fn0 err.log | \
while read line ; do
        DATE=$(date +%d-%m-%Y" "%H:%M:%S)
        echo "$DATE $line" | grep "error" | tee -a ~/eth/Genoil-U/timestamp.log
        if [ $? = 0 ]
        then
                kill $(ps aux | grep '[e]thminer' | awk '{print $2}')
                sleep 1
                screen -dmS ett bash ~/ett
        fi
done
-------------------------
ett:
-------------------------
#!/bin/bash
cd ~/eth/Genoil-U
./ethminer -U -F eth-us.dwarfpool.com:80/0xBEbd092a03827C37B75cd4ea314b207AA65c348f/208 2>&1 | tee >(grep error --color=never --line-buffered | tee -a err.log)

-------------------------

finally I also send output of ltail to timestamp.log to track how many times Genoil fails per hour - with roughly aiming at 1 crash per hour this gives me about 130MHs out of 5xGTX1060 which is a good 20+ MHs higher then Claymore... most importantly it gives stable hashing despite the OC introduced errors. The recovery is literally seconds.
Oh yeah and I also run
$tail -f ~/eth/Genoil-U/timestamp.log in a screen as well as watch -n 5 'sensors |grep Core' in another screen to fine tune the OC vs crash per hour vs temp
Hope this helps, and I hope the message is not too chaotic.
Cheers!

BTC: 13PnEKpfVzNseWkrm6LoueKcCMPj74zPv7
ETH: 0xBEbd092a03827C37B75cd4ea314b207AA65c348f