@SixLitr
Yes there really is something wrong with that last processor and I believe that it is probably memory related.
Your tests are showing that "sudo ./TurionPowerControl -psmax 1" is working properly.
The other three processors go from 1220 H/s per node to 1310 H/s per node or a gain of 7.4%.
As for that last processor is the system reporting any defective memory modules or complaining about non-optimal memory configuration when booting?
You can set in the BIOS under memory settings that a memory test is always run when booting.
Also to check for bad and or mismatched memory you could swap the memory between processors and see if the problem moves to another processor.
You can also use TurionPowerControl to check the DRAM Configuration Status
sudo ./TurionPowerControl -dram
Example from my quad Opteron 6348 Dell R815:
DCT0: DDR3 frequency: 1333 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=10 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=9 TrwtTO=8 Twtr=5 Twrrd=1 Twrwrsdsc=1 Trdrdsdsc=1 Tref=2 Trfc0=4 Trfc1=2 Trfc2=4 Trfc3=4 MaxRdLatency=63
LDIMM0=EMPTY/EMPTY LDIMM1=OK/OK LDIMM2=EMPTY/EMPTY LDIMM3=EMPTY/EMPTY
DCT1: DDR3 frequency: 1333 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=10 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=9 TrwtTO=8 Twtr=5 Twrrd=1 Twrwrsdsc=1 Trdrdsdsc=1 Tref=2 Trfc0=4 Trfc1=2 Trfc2=4 Trfc3=4 MaxRdLatency=63
LDIMM0=EMPTY/EMPTY LDIMM1=OK/OK LDIMM2=EMPTY/EMPTY LDIMM3=EMPTY/EMPTY
Node 1 ---
DCT0: DDR3 frequency: 1333 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=10 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=9 TrwtTO=8 Twtr=5 Twrrd=1 Twrwrsdsc=1 Trdrdsdsc=1 Tref=2 Trfc0=4 Trfc1=2 Trfc2=4 Trfc3=4 MaxRdLatency=61
LDIMM0=EMPTY/EMPTY LDIMM1=OK/OK LDIMM2=EMPTY/EMPTY LDIMM3=EMPTY/EMPTY
DCT1: DDR3 frequency: 1333 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=10 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=8 TrwtTO=7 Twtr=5 Twrrd=1 Twrwrsdsc=1 Trdrdsdsc=1 Tref=2 Trfc0=4 Trfc1=2 Trfc2=4 Trfc3=4 MaxRdLatency=60
LDIMM0=EMPTY/EMPTY LDIMM1=OK/OK LDIMM2=EMPTY/EMPTY LDIMM3=EMPTY/EMPTY
Node 2 ---
DCT0: DDR3 frequency: 1333 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=10 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=9 TrwtTO=8 Twtr=5 Twrrd=1 Twrwrsdsc=1 Trdrdsdsc=1 Tref=2 Trfc0=4 Trfc1=2 Trfc2=4 Trfc3=4 MaxRdLatency=64
LDIMM0=EMPTY/EMPTY LDIMM1=OK/OK LDIMM2=EMPTY/EMPTY LDIMM3=EMPTY/EMPTY
DCT1: DDR3 frequency: 1333 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=10 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=9 TrwtTO=8 Twtr=5 Twrrd=1 Twrwrsdsc=1 Trdrdsdsc=1 Tref=2 Trfc0=4 Trfc1=2 Trfc2=4 Trfc3=4 MaxRdLatency=61
LDIMM0=EMPTY/EMPTY LDIMM1=OK/OK LDIMM2=EMPTY/EMPTY LDIMM3=EMPTY/EMPTY
Node 3 ---
DCT0: DDR3 frequency: 1333 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=10 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=9 TrwtTO=8 Twtr=5 Twrrd=1 Twrwrsdsc=1 Trdrdsdsc=1 Tref=2 Trfc0=4 Trfc1=2 Trfc2=4 Trfc3=4 MaxRdLatency=61
LDIMM0=EMPTY/EMPTY LDIMM1=OK/OK LDIMM2=EMPTY/EMPTY LDIMM3=EMPTY/EMPTY
DCT1: DDR3 frequency: 1333 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=10 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=9 TrwtTO=8 Twtr=5 Twrrd=1 Twrwrsdsc=1 Trdrdsdsc=1 Tref=2 Trfc0=4 Trfc1=2 Trfc2=4 Trfc3=4 MaxRdLatency=60
LDIMM0=EMPTY/EMPTY LDIMM1=OK/OK LDIMM2=EMPTY/EMPTY LDIMM3=EMPTY/EMPTY
Node 4 ---
DCT0: DDR3 frequency: 1333 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=10 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=9 TrwtTO=8 Twtr=5 Twrrd=1 Twrwrsdsc=1 Trdrdsdsc=1 Tref=2 Trfc0=4 Trfc1=2 Trfc2=4 Trfc3=4 MaxRdLatency=62
LDIMM0=EMPTY/EMPTY LDIMM1=OK/OK LDIMM2=EMPTY/EMPTY LDIMM3=EMPTY/EMPTY
DCT1: DDR3 frequency: 1333 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=10 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=9 TrwtTO=8 Twtr=5 Twrrd=1 Twrwrsdsc=1 Trdrdsdsc=1 Tref=2 Trfc0=4 Trfc1=2 Trfc2=4 Trfc3=4 MaxRdLatency=63
LDIMM0=EMPTY/EMPTY LDIMM1=OK/OK LDIMM2=EMPTY/EMPTY LDIMM3=EMPTY/EMPTY
Node 5 ---
DCT0: DDR3 frequency: 1333 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=10 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=9 TrwtTO=8 Twtr=5 Twrrd=1 Twrwrsdsc=1 Trdrdsdsc=1 Tref=2 Trfc0=4 Trfc1=2 Trfc2=4 Trfc3=4 MaxRdLatency=61
LDIMM0=EMPTY/EMPTY LDIMM1=OK/OK LDIMM2=EMPTY/EMPTY LDIMM3=EMPTY/EMPTY
DCT1: DDR3 frequency: 1333 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=10 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=8 TrwtTO=7 Twtr=5 Twrrd=1 Twrwrsdsc=1 Trdrdsdsc=1 Tref=2 Trfc0=4 Trfc1=2 Trfc2=4 Trfc3=4 MaxRdLatency=61
LDIMM0=EMPTY/EMPTY LDIMM1=OK/OK LDIMM2=EMPTY/EMPTY LDIMM3=EMPTY/EMPTY
Node 6 ---
DCT0: DDR3 frequency: 1333 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=10 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=9 TrwtTO=8 Twtr=5 Twrrd=1 Twrwrsdsc=1 Trdrdsdsc=1 Tref=2 Trfc0=4 Trfc1=2 Trfc2=4 Trfc3=4 MaxRdLatency=63
LDIMM0=EMPTY/EMPTY LDIMM1=OK/OK LDIMM2=EMPTY/EMPTY LDIMM3=EMPTY/EMPTY
DCT1: DDR3 frequency: 1333 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=10 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=9 TrwtTO=8 Twtr=5 Twrrd=1 Twrwrsdsc=1 Trdrdsdsc=1 Tref=2 Trfc0=4 Trfc1=2 Trfc2=4 Trfc3=4 MaxRdLatency=63
LDIMM0=EMPTY/EMPTY LDIMM1=OK/OK LDIMM2=EMPTY/EMPTY LDIMM3=EMPTY/EMPTY
Node 7 ---
DCT0: DDR3 frequency: 1333 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=10 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=9 TrwtTO=8 Twtr=5 Twrrd=1 Twrwrsdsc=1 Trdrdsdsc=1 Tref=2 Trfc0=4 Trfc1=2 Trfc2=4 Trfc3=4 MaxRdLatency=61
LDIMM0=EMPTY/EMPTY LDIMM1=OK/OK LDIMM2=EMPTY/EMPTY LDIMM3=EMPTY/EMPTY
DCT1: DDR3 frequency: 1333 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=10 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=8 TrwtTO=7 Twtr=5 Twrrd=1 Twrwrsdsc=1 Trdrdsdsc=1 Tref=2 Trfc0=4 Trfc1=2 Trfc2=4 Trfc3=4 MaxRdLatency=59
LDIMM0=EMPTY/EMPTY LDIMM1=OK/OK LDIMM2=EMPTY/EMPTY LDIMM3=EMPTY/EMPTY
Hopefully you can spot the memory problem from this data. MaxRdLatency and DDR3 frequency are probable the most important along with the LDIMM results.
In my system I have four 2GB memory sticks for each Opteron 6348. That means two sticks for each of the two nodes in each Opteron. Thus the LDIMM1=OK/OK shown above.
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
On the subject of TurionPowerControl I have found more useful commands.
When I am testing I open three shellinthebox command windows to the server.
In the first one I run the actual benchmark. In the second I run nmon with the "c" key when it opens to observe which cores are active.
In the third one I cd into turionpowercontrol and run various TurionPowerControl commands while the benchmark is running.
This command: sudo ./TurionPowerControl -CM
shows the actual Pstate for all cores in real time. If you run this during testing without doing the "TurionPowerControl -psmax 1" you will see that the Pstate flips from PS1 and PS2 all the time. But if you have done "TurionPowerControl -psmax 1" it stays locked in PS1 which is why you get the gain.
To show useful information about Hardware Thermal Control features of the processors in the system
sudo ./TurionPowerControl -htc
The Dell R815 shows 70c as the high temperature limit and that that temp was exceeded before (which throttles the processor).
This command sets the high temperature limit on active node as a threshold for thermal control activity
sudo ./TurionPowerControl -htctemplimit <value>
I did "sudo ./TurionPowerControl -htctemplimit 75" and reran the benchmark but it resulted in no gain in H/s so I left it at the default 70c