[OS] nvOC easy-to-use Linux Nvidia Mining

[kk003]

This one is for coders:

Code:

function genoil_statics ()

{
#### Arrange the output for Genoil statics

if [[ "$ETHMINER_or_GENOIL_or_CLAYMORE" == "GENOIL" ]]; then

    # Check if Genoil is running. Note that I have to search the full path as there is a few miners with the same exec's name
    ps aux | grep -v grep | grep miner | grep -q "$KEY_GENOIL"
    if  [[ $? -eq 0 ]]; then
        GENOIL_IS_RUNNING="YES"
        TIMEOUT_FOR_TIMEOUT_IN_SECONDS=20       # The timeout for the command timeout to wait
        GENOIL_NUMBER_OF_HASHRATES_TO_SHOW=3    # Default 3, and I want to keep this value independent for each miner
        echo
        echo "It seems that GENOIL is running!!"

        # Extract the output from screen
        echo "Running timeout+script+screen for $TIMEOUT_FOR_TIMEOUT_IN_SECONDS seconds"
        timeout $TIMEOUT_FOR_TIMEOUT_IN_SECONDS script -q ~/kk003_telegram_data/files/output_miner_from_screen --command "screen -dr miner"
        sleep 2
#      screen -d miner # does not seem to need to be Detached

        # I don't see any valuable information than the total hashrate (I'll show a few at last)
        # Surprise!!!, sed 's/ /\n/g' no convierte espacios en saltos de linea, pero la misma linea en la consola si lo hace!!!
        TOTAL_HASHRATE=$(cat ~/kk003_telegram_data/files/output_miner_from_screen | grep "Mining on PoWhash" | cut -d":" -f4 |  cut -d" " -f2 | sed 's/ /\n/g' | tail -$GENOIL_NUMBER_OF_HASHRATES_TO_SHOW

        # Mount the little thing Genoil gives us
        #
        echo "Latest total hashrates : " > ~/kk003_telegram_data/files/miner_statics.txt
        echo $TOTAL_HASHRATE >> ~/kk003_telegram_data/files/miner_statics.txt

        echo "Latest total hashrates   : " > ~/kk003_telegram_data/files/miner_statics_log_file.txt
        echo $TOTAL_HASHRATE >> ~/kk003_telegram_data/files/miner_statics_log_file.txt


        # Insert the Title for Genoil mining information
        sed -i "1s/^/** Genoil mining  Information **\n/" ~/kk003_telegram_data/files/miner_statics.txt
        sed -i "1s/^/\n/" ~/kk003_telegram_data/files/miner_statics.txt

        sed -i "1s/^/** Genoil mining  Information **\n/" ~/kk003_telegram_data/files/miner_statics_log_file.txt
        sed -i "1s/^/\n/" ~/kk003_telegram_data/files/miner_statics_log_file.txt

    else
        echo "Genoil does not seem to be running!!. Skiping this bit."
        GENOIL_IS_RUNNING="NO"
    fi
fi
}

I found out a way to capture the output from screen in real time. See the lines:

Code:

# Extract the output from screen
        echo "Running timeout+script+screen for $TIMEOUT_FOR_TIMEOUT_IN_SECONDS seconds"
        timeout $TIMEOUT_FOR_TIMEOUT_IN_SECONDS script -q ~/kk003_telegram_data/files/output_miner_from_screen --command "screen -dr miner"
        sleep 2
#      screen -d miner # does not seem to need to be Detached

This run screen for 20 seconds (timeout command) and saves output to file ~/kk003_telegram_data/files/output_miner_from_screen (script command). -dr forces to attach miner (so if user have screen open in a terminal will close it and the script will do its job). After 20 seconds miner is detached without need to send CTRL+d.
This works better if ran from a non interactive terminal, ejem: from nvoc system  .

This means it is possible to capture the output of any miner even if all of it shows only on terminal (no log file, web server, argument -L on screen, etc).
I hope it will be useful for you too.

PD: if you use "tail -f file" to watch the output in real time don't trust what you see. The real output is in the file (do a cat file or whatever). Keep in main, if you log the output of your script probably you'll get those 20 seconds from screen in the log file too.

Nice; this is very useful.  Even as a methodology it can be applied to many different features. 

I am sorry to say that I have tested this method all night and it is unstable. At any given moment stop writing the file.

Finally I think this method will work to start screen's log file "on the fly"

Code:

screen -dr miner -X log

So some code like this would do the trick:

Code:

# First check if screen is running miner with the -L flag!!!
......
......

# End of first check if screen is running miner with the -L flag!!!

if [[ -f ~/screenlog.0 ]]; then
  > ~/screenlog.0
  screen -dr miner -X log       # Start login output from miner
  sleep 20                      # keep saving data for 20 seconds   
  screen -dr miner -X log       # Stop login output from miner
  # Get the miner statics
  TOTAL_HASHRATE=$(tail -60 ~/screenlog.0 | grep "Mining on PoWhash" | cut -d":" -f4 |  cut -d" " -f2 | sed 's/ /\n/g' | tail -3)
  .....
  .....

fi

I will add this method to kk003_telegram ASP + some new features and miners for statics.
Hope it helps.

[1713307434]

1713307434

[1713307434]

1713307434

[1713307434]

1713307434

[1713307434]

1713307434

[CryptAtomeTrader44]

Maybe its me but the speeds to download the OS are incredibly slow. I built a torrent and I am seeding it from my server and personal desktop here: https://www.cryptocurrencyfreak.com/2017/10/30/nvoc-mining-os-torrent/

Feel free to help seed!

Is this 1.3 or the name is wrong ?

That torrent really is for the 1.3 image; I think he DL'ed just before fullzero posted 1.4, and then came back to post about the torrent just after fullzero posted.

This one is for the 1.4 image:
http://s000.tinyupload.com/index.php?file_id=00585421597763893895

I've added it in comments on cryptocurrencyfreak 's blog; it's awaiting moderation or for him to notice it here.

Seeding with a gigabit seedbox and a 500 MB 2nd seedbox for the next 7 days.
Enjoy !

AWESOME!   Thanks!!

7zip say it is not a valid zip file ...dwld with synology download station and say the same error
3 seeds. not enough seeds...

[VoskCoin]

Any input on what my issue is?

2400w server psu powering 11 gpu and 11 riser
750w atx powering mobo cpu 2x molex (1 cable 2 connects) powering primary gpu and primary gpu riser
h110 btc pro 13 gpu mobo

wont boot with smos or nvoc

[medi91]

Tried my best to search for an answer, but no luck so far...

has anyone successfully gotten rid of the login loop issue?

Ive console logged in, did a re-image, tried different versions, and every ubuntu based fix i could find online (clear and .Xauthority fixes)

Board is a HP z400 board

Try this:
press CTRL- ALT- F1 together, terminal window will open. Login as m1, pass miner1

type:

Code:

sudo apt update && sudo apt upgrade && sudo reboot

Please reply with results

Unfortunately did not get rid of the login loop

[leenoox]

So which memories are the good and which one are bad ?
I know Samsung is good, but other good ones ?

Samsung memory is best, then Micron, then Hynix.
From my experience with 1060's mining ETH
Samsung: 25 MH/s
Micron: 22 MH/s
Hynix: 19 MH/s

Unfortunately there is no way to check memory vendor under linux (yet). You have to check under Windows with GPU-Z

[SergGT001]

Hi! I recorded the image on a flash drive as described in the first post of the topic. But I get the compand string grub. The image is not loaded. What am I doing wrong?
Screenshot https://cdn1.savepice.ru/uploads/2017/10/31/5205d8dee4f2371e5a84d12d54231a0b-full.jpg

[leenoox]

Finally I think this method will work to start screen's log file "on the fly"

Code:

screen -dr miner -X log

So some code like this would do the trick:

Code:

# First check if screen is running miner with the -L flag!!!
......
......

# End of first check if screen is running miner with the -L flag!!!

if [[ -f ~/screenlog.0 ]]; then
  > ~/screenlog.0
  screen -dr miner -X log       # Start login output from miner
  sleep 20                      # keep saving data for 20 seconds   
  screen -dr miner -X log       # Stop login output from miner
  # Get the miner statics
  TOTAL_HASHRATE=$(tail -60 ~/screenlog.0 | grep "Mining on PoWhash" | cut -d":" -f4 |  cut -d" " -f2 | sed 's/ /\n/g' | tail -3)
  .....
  .....

fi

I will add this method to kk003_telegram ASP + some new features and miners for statics.
Hope it helps.

It's really nice to add more features to nvOC and I would encourage everyone to try and contribute to this great project but I would strongly advice against using unnecessary extreme writing in logs to accomplish the proposed feature. Especialy recording the screen in screenlog...

You see people complainig every day of corrupted USB sticks, freezing, running out of space, boot problems, etc. USB stick as a medium is not designed for such operations, constant writing to USB stick is not recommended, it messes the controller, makes USB perform very slow after a while and kills memory cells which leads to corruption and lock-ups.

If nvOC was explicitly recommended as SSD/HDD OS, the logs would be fine but the reality is that 90% of people use nvOC from USB sticks.

To all devs: please avoid usage of logs as much as possible or at least log as little as possible.

Hint: if you want to show output from the miner in your script/program/telegram/web page (definetely a nice feature to show), record the output from the miner (or nvidia api) into memory then show it in your program on a per need basis. This is more easily done with PHP and Perl but it can be accomplished in Bash as well. Record stdout and stderr into rotating array (you can use shift for the array). You can record 10, 20, 100, even 500 lines and you won't run out of memory, then show the live output from the array into your featured program.

[papampi]

It's really nice to add more features to nvOC and I would encourage everyone to try and contribute to this great project but I would strongly advice against using unnecessary extreme writing in logs to accomplish the proposed feature. Especialy recording the screen in screenlog...

You see people complainig every day of corrupted USB sticks, freezing, running out of space, boot problems, etc. USB stick as a medium is not designed for such operations, constant writing to USB stick is not recommended, it messes the controller, makes USB perform very slow after a while and kills memory cells which leads to corruption and lock-ups.

If nvOC was explicitly recommended as SSD/HDD OS, the logs would be fine but the reality is that 90% of people use nvOC from USB sticks.

To all devs: please avoid usage of logs as much as possible or at least log as little as possible.

Hint: if you want to show output from the miner in your script/program/telegram/web page (definetely a nice feature to show), record the output from the miner (or nvidia api) into memory then show it in your program on a per need basis. This is more easily done with PHP and Perl but it can be accomplished in Bash as well. Record stdout and stderr into rotating array (you can use shift for the array). You can record 10, 20, 100, even 500 lines and you won't run out of memory, then show the live output from the array into your featured program.

We miners are spending thousands on our rigs and when it comes to stability and life span we forget the most important part and thats where the OS is installed on, we think small and go cheap with a 5-10$ USB when we can have much smoother system with a 30$ SSD.

Using USB as daily use is not recommended in any distro, The most important  difference other than USB read/write limit is there's no way for a USB drive to mark bad cells as 'do not use'  whereas the SSD may develop bad areas it can ignore them. With a USB drive one cell goes bad and the whole drive is useless.

I'm Not saying that you shouldn't use a USB for bootable OS, just that it's not a good idea to have it as your main boot drive and only use them for diagnostic / testing purposes.
No mining system should be run from USB as a daily hardcore mining system, but run it on usb, test OS and see if you like it, then install it on SSD.

I think mining with nvOC on USB is only good for those who want to mine when their pc is not in use, so they boot with usb, mine for a while then reboot back to their main OS...

[eruman34]

Hi,

Is there any chance to;

1- Get full support for Asus b250 / ETA? (18 GPU with OC)
2- Onboard gpu to monitor view (currently primary gpu only and i dont have display ports on mining p106-100)
3- INNO 3D P106-100 driver support (if already pardon me, i couldn't try due to monitor - gpu issue above)

Bdw thanks for such great effort dev.

[SergGT001]

Hi! I recorded the image on a flash drive as described in the first post of the topic. But I get the compand string grub. The image is not loaded. What am I doing wrong?
Screenshot https://cdn1.savepice.ru/uploads/2017/10/31/5205d8dee4f2371e5a84d12d54231a0b-full.jpg

[papampi]

Tried my best to search for an answer, but no luck so far...

has anyone successfully gotten rid of the login loop issue?

Ive console logged in, did a re-image, tried different versions, and every ubuntu based fix i could find online (clear and .Xauthority fixes)

Board is a HP z400 board

Try this:
press CTRL- ALT- F1 together, terminal window will open. Login as m1, pass miner1

type:

Code:

sudo apt update && sudo apt upgrade && sudo reboot

Please reply with results

Unfortunately did not get rid of the login loop

Broken nvidia driver

Code:

sudo dpkg-reconfigure --all

[papampi]

So which memories are the good and which one are bad ?
I know Samsung is good, but other good ones ?

Samsung memory is best, then Micron, then Hynix.
From my experience with 1060's mining ETH
Samsung: 25 MH/s
Micron: 22 MH/s
Hynix: 19 MH/s

Unfortunately there is no way to check memory vendor under linux (yet). You have to check under Windows with GPU-Z

Thanks a lot mate
looks like I should setup a windows on an ssd to test the new cards

[Stubo]

I have been spending a bit of time reading through all of the bash scripts that make up nvOC. I urge everybody to do that because then you will learn both how it works and how you may be to take advantage of some of the vast functionality that is offered. To make that easier to do, I found a python script "beautify_bash.py" that you can run against them (1bash, 2unix, 3main, etc.) to make them more readable.

https://github.com/ewiger/beautify_bash

It basically just reformats the code with the proper syntax indentation for loops and if statements. The first thing that I did with the 19-1.4 release the other day was to beautify all of the scripts, then make my personal miner tweaks to those versions and finally deploy those to my miners. I have found no issues from doing this. The basic steps to get it going are:

-Download the zip file to your PC, unzip it to get to beautify_bash.py
-Use WinSCP or suitable substitute to sftp it down to your rig(s) in /home/m1 directory
-Login to rig, make the script executable "chmod 750 beautify_bash.py"
-Execute against the scripts you use:
./beautify_bash.py 1bash
./beautify_bash.py 2unix
./beautify_bash.py 3main
./beautify_bash.py 4update
./beautify_bash.py IAmNotAJeep_and_Maxximus007_WATCHDOG
..etc

It will beautify the script and leave a copy of the original as <filename>~. Perhaps there is some reason for releasing the scripts with no syntax formatting as they are but I have yet to find it.

Hope this helps.

[kk003]

It's really nice to add more features to nvOC and I would encourage everyone to try and contribute to this great project but I would strongly advice against using unnecessary extreme writing in logs to accomplish the proposed feature. Especialy recording the screen in screenlog...

You see people complainig every day of corrupted USB sticks, freezing, running out of space, boot problems, etc. USB stick as a medium is not designed for such operations, constant writing to USB stick is not recommended, it messes the controller, makes USB perform very slow after a while and kills memory cells which leads to corruption and lock-ups.

If nvOC was explicitly recommended as SSD/HDD OS, the logs would be fine but the reality is that 90% of people use nvOC from USB sticks.

To all devs: please avoid usage of logs as much as possible or at least log as little as possible.

Hint: if you want to show output from the miner in your script/program/telegram/web page (definetely a nice feature to show), record the output from the miner (or nvidia api) into memory then show it in your program on a per need basis. This is more easily done with PHP and Perl but it can be accomplished in Bash as well. Record stdout and stderr into rotating array (you can use shift for the array). You can record 10, 20, 100, even 500 lines and you won't run out of memory, then show the live output from the array into your featured program.

We miners are spending thousands on our rigs and when it comes to stability and life span we forget the most important part and thats where the OS is installed on, we think small and go cheap with a 5-10$ USB when we can have much smoother system with a 30$ SSD.

Using USB as daily use is not recommended in any distro, The most important  difference other than USB read/write limit is there's no way for a USB drive to mark bad cells as 'do not use'  whereas the SSD may develop bad areas it can ignore them. With a USB drive one cell goes bad and the whole drive is useless.

I'm Not saying that you shouldn't use a USB for bootable OS, just that it's not a good idea to have it as your main boot drive and only use them for diagnostic / testing purposes.
No mining system should be run from USB as a daily hardcore mining system, but run it on usb, test OS and see if you like it, then install it on SSD.

I think mining with nvOC on USB is only good for those who want to mine when their pc is not in use, so they boot with usb, mine for a while then reboot back to their main OS...

I agree. I have to say that if system is using a hdd or ssd (better) it should not have any problem with log files as long as the system is healthy. If system has a very high cpu load/gpu instability etc, then is better not to log, at less in a productive system.
Linux is good at login but system must be healthy.
Got a rig (installed in a old hdd) with high cpu load (2 cores and 2.5 cpu load average) and log files for miners go around 150MB+... no problem. I don't think this would work on a usb stick.

[cryptosg]

Hi anyone know to connect to Ubiq pool?

UBQ_WORKER=$WORKERNAME
UBQ_ADDRESS="replace_with_your_UBIQ_address"
UBQ_POOL="stratum+tcp://us.ubiqpool.io:8008"
UBQ_EXTENSION_ARGUMENTS=""


Do we need to change the pool address?
Try change to below 3, dun works
Getwork
http://us.ubiqpool.io:8888

Stratum
stratum+tcp://us.ubiqpool.io:8008

Stratum 2
stratum+tcp://us2.ubiqpool.io:8008

[kk003]

I have been spending a bit of time reading through all of the bash scripts that make up nvOC. I urge everybody to do that because then you will learn both how it works and how you may be to take advantage of some of the vast functionality that is offered. To make that easier to do, I found a python script "beautify_bash.py" that you can run against them (1bash, 2unix, 3main, etc.) to make them more readable.

https://github.com/ewiger/beautify_bash

It basically just reformats the code with the proper syntax indentation for loops and if statements. The first thing that I did with the 19-1.4 release the other day was to beautify all of the scripts, then make my personal miner tweaks to those versions and finally deploy those to my miners. I have found no issues from doing this. The basic steps to get it going are:

-Download the zip file to your PC, unzip it to get to beautify_bash.py
-Use WinSCP or suitable substitute to sftp it down to your rig(s) in /home/m1 directory
-Login to rig, make the script executable "chmod 750 beautify_bash.py"
-Execute against the scripts you use:
./beautify_bash.py 1bash
./beautify_bash.py 2unix
./beautify_bash.py 3main
./beautify_bash.py 4update
./beautify_bash.py IAmNotAJeep_and_Maxximus007_WATCHDOG
..etc

It will beautify the script and leave a copy of the original as <filename>~. Perhaps there is some reason for releasing the scripts with no syntax formatting as they are but I have yet to find it.

Hope this helps.

I hate mal formated scripts such as when I upload a well formated one to pastebin.com and it makes a mess of it.
This will help, thx.

[leenoox]

It's really nice to add more features to nvOC and I would encourage everyone to try and contribute to this great project but I would strongly advice against using unnecessary extreme writing in logs to accomplish the proposed feature. Especialy recording the screen in screenlog...

You see people complainig every day of corrupted USB sticks, freezing, running out of space, boot problems, etc. USB stick as a medium is not designed for such operations, constant writing to USB stick is not recommended, it messes the controller, makes USB perform very slow after a while and kills memory cells which leads to corruption and lock-ups.

If nvOC was explicitly recommended as SSD/HDD OS, the logs would be fine but the reality is that 90% of people use nvOC from USB sticks.

To all devs: please avoid usage of logs as much as possible or at least log as little as possible.

Hint: if you want to show output from the miner in your script/program/telegram/web page (definetely a nice feature to show), record the output from the miner (or nvidia api) into memory then show it in your program on a per need basis. This is more easily done with PHP and Perl but it can be accomplished in Bash as well. Record stdout and stderr into rotating array (you can use shift for the array). You can record 10, 20, 100, even 500 lines and you won't run out of memory, then show the live output from the array into your featured program.

We miners are spending thousands on our rigs and when it comes to stability and life span we forget the most important part and thats where the OS is installed on, we think small and go cheap with a 5-10$ USB when we can have much smoother system with a 30$ SSD.

Using USB as daily use is not recommended in any distro, The most important  difference other than USB read/write limit is there's no way for a USB drive to mark bad cells as 'do not use'  whereas the SSD may develop bad areas it can ignore them. With a USB drive one cell goes bad and the whole drive is useless.

I'm Not saying that you shouldn't use a USB for bootable OS, just that it's not a good idea to have it as your main boot drive and only use them for diagnostic / testing purposes.
No mining system should be run from USB as a daily hardcore mining system, but run it on usb, test OS and see if you like it, then install it on SSD.

I think mining with nvOC on USB is only good for those who want to mine when their pc is not in use, so they boot with usb, mine for a while then reboot back to their main OS...

I agree. I have to say that if system is using a hdd or ssd (better) it should not have any problem with log files as long as the system is healthy. If system has a very high cpu load/gpu instability etc, then is better not to log, at less in a productive system.
Linux is good at login but system must be healthy.
Got a rig (installed in a old hdd) with high cpu load (2 cores and 2.5 cpu load average) and log files for miners go around 150MB+... no problem. I don't think this would work on a usb stick.

I agree, nvOC should be run from SSD and USB sticks should be used for evaluation or short term use only. Maybe fullzero should note/recommend this in the OP.

However, as a programmers we should do due diligence and optimize the code to run as smooth as possible including running off USB sticks

[leenoox]

I have been spending a bit of time reading through all of the bash scripts that make up nvOC. I urge everybody to do that because then you will learn both how it works and how you may be to take advantage of some of the vast functionality that is offered. To make that easier to do, I found a python script "beautify_bash.py" that you can run against them (1bash, 2unix, 3main, etc.) to make them more readable.

https://github.com/ewiger/beautify_bash

It basically just reformats the code with the proper syntax indentation for loops and if statements. The first thing that I did with the 19-1.4 release the other day was to beautify all of the scripts, then make my personal miner tweaks to those versions and finally deploy those to my miners. I have found no issues from doing this. The basic steps to get it going are:

-Download the zip file to your PC, unzip it to get to beautify_bash.py
-Use WinSCP or suitable substitute to sftp it down to your rig(s) in /home/m1 directory
-Login to rig, make the script executable "chmod 750 beautify_bash.py"
-Execute against the scripts you use:
./beautify_bash.py 1bash
./beautify_bash.py 2unix
./beautify_bash.py 3main
./beautify_bash.py 4update
./beautify_bash.py IAmNotAJeep_and_Maxximus007_WATCHDOG
..etc

It will beautify the script and leave a copy of the original as <filename>~. Perhaps there is some reason for releasing the scripts with no syntax formatting as they are but I have yet to find it.

Hope this helps.

I hate mal formated scripts such as when I upload a well formated one to pastebin.com and it makes a mess of it.
This will help, thx.

Great idea!!!

[Stubo]

Great idea!!!

The one minor tweak I made to beautify_bash.py is to the indentation. I have always used 3 spaces whereas the script uses 2. Of course, I read it and it is very easy to change to your own liking. Just change:

Code:

self.tab_size = 2

to

Code:

self.tab_size = 3

on line 33.

[hurvajs77]

@damNmad:

I found a small bug in your telegram configuration. Command which use for getting GPUs count is invalid when rig have installed over 9 cards...

Your command:

Code:

GPU_COUNT=$(nvidia-smi -L | tail -n 1| cut -c 5 |awk '{ SUM += $1+1} ; { print SUM }')

and result:

Code:

m1@rig-bafomet:~$ nvidia-smi -L | tail -n 1| cut -c 5 |awk '{ SUM += $1+1} ; { print SUM }'
2

There isn't any reason, why use awk. Easier way is use wc only.

So, fix:

Code:

GPU_COUNT=$(nvidia-smi -L | wc -l')

and result:

Code:

m1@rig-bafomet:~$ nvidia-smi -L | wc -l
11