Looks great Spinx!
Thanks!
Nice system.
I am trying Hive OS on my DL580 G7 with 4x E7-4830's and 4x GTX 750. It is up and running but something must be wrong with my configuration for the CPUs. I am getting some affinity errors on a few CPU cores and those cores only produce 11 H/s instead of the 33 H/s on cores where affinity works. I have disabled hyper-threading in the BIOS so the 32 cores shown are real CPU cores.
Could you post the CPU configuration that you used with Hive OS. Hopefully that will fix my problem.
Also is the rack that the DL580 and GPUs are mounted in a commercial rack or home made?
If home made can you provide assembly instructions.
Thanks
Reason i left hiveOS was that i never got it stable. Constant freeze and reboots, but it has a lot more features, on paper its far superior to SMOS.
I don't have the configs left i think, but please have a look at my post here on settings for E7-8837 with xmr-stak:
https://bitcointalk.org/index.php?topic=1955358.msg28646698#msg28646698Thanks for the link. Your settings and mine were the same so after further investigation I found what the
major problem that was causing the affinity errors and lower than expected hash rates.
Hive OS (and probably Linux in general) assigns 10 cores to each processor from the E7-4800 and E7-8800 family. So Processor #0 has cores 0 - 9, Processor #1 has Cores 10 - 19, Processor #2 has Cores 20 - 29, and Processor #3 has Cores 10 - 39. I was very confused in why mpstat -P ALL showed 40 cores even though the 4x E7-4830 (with HT disabled) only had 32 (4x 8 cores). It turns out that Linux disables two of the cores in each set of 10 for the 8-core processors. The two cores disabled are not at the end but core #4 & #5 from each group. If you have 6-core processors two more cores would be disabled.
So your configuration from the link above and what I was using would try to use non-existent cores (which caused the affinity errors) and would run more threads on some processors than what we thought and less on other processors. That is why I saw hashes of 11.5 instead of 33.5 that I was expecting.
This is the correct CPU threads table for 8-core E7-4800's and E7-8800's (including the E7-8837):
"cpu_threads_conf" :
[
{ "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 1 },
{ "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 2 },
{ "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 3 },
{ "low_power_mode" : true, "no_prefetch" : true, "affine_to_cpu" : 6 },
{ "low_power_mode" : true, "no_prefetch" : true, "affine_to_cpu" : 7 },
{ "low_power_mode" : true, "no_prefetch" : true, "affine_to_cpu" : 8 },
{ "low_power_mode" : true, "no_prefetch" : true, "affine_to_cpu" : 9 },
{ "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 10 },
{ "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 11 },
{ "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 12 },
{ "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 13 },
{ "low_power_mode" : true, "no_prefetch" : true, "affine_to_cpu" : 16 },
{ "low_power_mode" : true, "no_prefetch" : true, "affine_to_cpu" : 17 },
{ "low_power_mode" : true, "no_prefetch" : true, "affine_to_cpu" : 18 },
{ "low_power_mode" : true, "no_prefetch" : true, "affine_to_cpu" : 19 },
{ "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 20 },
{ "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 21 },
{ "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 22 },
{ "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 23 },
{ "low_power_mode" : true, "no_prefetch" : true, "affine_to_cpu" : 26 },
{ "low_power_mode" : true, "no_prefetch" : true, "affine_to_cpu" : 27 },
{ "low_power_mode" : true, "no_prefetch" : true, "affine_to_cpu" : 28 },
{ "low_power_mode" : true, "no_prefetch" : true, "affine_to_cpu" : 29 },
{ "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 30 },
{ "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 31 },
{ "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 32 },
{ "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 33 },
{ "low_power_mode" : true, "no_prefetch" : true, "affine_to_cpu" : 36 },
{ "low_power_mode" : true, "no_prefetch" : true, "affine_to_cpu" : 37 },
{ "low_power_mode" : true, "no_prefetch" : true, "affine_to_cpu" : 38 },
{ "low_power_mode" : true, "no_prefetch" : true, "affine_to_cpu" : 39 },
],
Using this the "affinity errors" went away and my overall CPU hash rate went from 1038 H/s to 1358 Hs for the processors. A 31% increase with no additional power used.
With 4x GTX 750's I am now hashing at 2250 H/s.
NOTE: the above configuration does not use Processor #0 Core #0 because I have that affined to from the Nvidia configuration file. If you have no GPU's then you can add this:
{ "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 0 },
to the top of the config file for extra hashes.
--------------------------
EDIT: 04/08/2018
Well the above worked on Rig #1 but when I brought up Rig #2 I again had affinity errors on 4 of the cores.
Can I say Linux really really sucks because it really really does.
Even though both Rigs are EXACTLY the same Linux disables different 2 cores out of the 10 that it uses on each of the 8 physical cores of the E7-8837.
On Rig #1 of the 40 cores Linux defines it disables core # 4, 5 on Processor #0, cores #14, 15 on Processor #1, cores #24, 25 on Processor #2 and cores #34, 35 On Processor #3
But on Rig #2 it does this: disables core # 1, 8 on Processor #0, cores #11, 18 on Processor #1, cores #24, 25 on Processor #2 and cores #34, 35 On Processor #3
That means that the custom XMR CPU configuration that I set in HiveOS cannot be used because what worked on Rig #1 fails on Rig #2 and more that likely on my future Rig #3.
So those who are using HiveOS and XMR-Stak do not use any overrides in the XMR CPU configuration in HiveOS just let XMR-Stak auto-generate it.
EDIT: 04/26/2018
The above issues of dummy cores has been solved with HiveOS going to Linux Kernel 4.13. mpstat -P ALL now shows the correct 32 cores.
