[LOCKED] cpuminer-opt v3.12.3, open source optimized multi-algo CPU miner

[hmage]

The bottom line is the same build is being seen as AES on westmere but non AES on my skylake.
The difference with checking AES is that is the compiler macro that is now showing conflicting results.
 __AES__ returns true on westmere and false on skylake. And that's where I'm stuck.

__AES__ does not change between processors if the binary does not change. Same for __SSE2__ and similar.

Observe: https://godbolt.org/g/WdT8c5

Try changing compile flags from -march=skylake to -march=core2 to -march=native or others from this list. You'll see that resulting assembler code changes in how many times print is called and what string constants are defined.

When __AES__ is not defined, then printf("AES\n") doesn't even make it into the binary. Same for others.

When the user said he's using the same binary as yours, he might said that by mistake -- you need to compare sha1sums of the binaries to make sure -- he might be running different .exe than yours.

Care to take another shot? Even when you miss the mark you make me think a little differently.

No comments.

[1714934678]

1714934678

[1714934678]

1714934678

[1714934678]

1714934678

[joblo]

When the user said he's using the same binary as yours, he might said that by mistake -- you need to compare sha1sums of the binaries to make sure -- he might be running different .exe than yours.

I considered that but it's more than one user. And I downloaded the same binaries they did, and they stated they used
the westmere build, filename cpuminer.exe. So it is a copy of the same file not a reproduction. Also that is the only
build that does not support AES+AVX. Had the users tried the wrong file it would crash. Some did and reported same.

I don't know precisely how it was compiled but assuming I'm not being similarly misinformed by several users
my observations seem correct. The same file shows a different result for __AES__ on different CPU architectures.
I don't know which is correct but they should be the same.

Here is a bit of direct output of the variables used in the logic compiled for corei7

Code:

         **********  cpuminer-opt 3.3.4  *********** 
     A CPU miner with multi algo support and optimized for CPUs
     with AES_NI and AVX extensions.
     BTC donation address: 12tdvfF7KmAsihBXQXynT6E6th2c2pByTT
     Forked from TPruvot's cpuminer-multi with credits
     to Lucas Jones, elmad, palmd, djm34, pooler, ig0tik3d,
     Wolf0 and Jeff Garzik.

cpu has_aes 1
cpu has_avx 1
sw_has_aes 0
sw_has_sse2 1
Checking CPU capatibility...
        Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz
   CPU arch supports AES-AVX...YES.
   SW built for AES-AVX........NO.
   Algo supports AES-AVX.......YES.
Starting mining without AES-AVX optimizations...

I've removed the warning messages otherwise no change from 3.3.3.

The result of the logic is not relevent in this example because the CPU has AES but the software check
is what's important. In this case sw_has_aes taken directly from __AES__ is false. This seems reasonable
for a westmere build and it agrees with my results from testing the CMB binary on the same CPU.

So I think I've gone in a complete circle and back to the same point. The CMB westmere binary displays a
different result for the __AES__ macro on different CPUs. Three users have reported he same problem with
actual westmere class CPUs because, for them, __AES__ returned false.

I am confident in the data reported by the users.
I am less confident about the nature of the CMB build. I don't know if they built native on westmere HW or
crosscompiled using -march=corei7. I could only verify indirectly by crosscompiling myself.

But even ignoring how the bin was built __AES__ should still produce the same result on any HW.

[joblo]

I'm starting to prepare v3.3.4 for release. I will check for new posts before I publish

As it stands there are two changes:

- removed incompatible SW build check to fix problems for users with Westmere class CPUs.
- optimized m7m AES +65%, SSE2 %59%

Edit: This is what v3.3.4 should look like when compiler for and run on westmere. It is not actual
program output because I don't have a westmere CPU.

Code:

   CPU arch supports AES-AVX...NO.
   CPU arch supports SSE2......YES.
Starting mining without AES-AVX optimizations...

 

[My9bot]

I'm starting to prepare v3.3.4 for release. I will check for new posts before I publish

As it stands there are two changes:

- removed incompatible SW build check to fix problems for users with Westmere class CPUs.
- optimized m7m AES +65%, SSE2 %59%

very nice work
waiting for v3.3.4
thank you a lot

[joblo]

I'm starting to prepare v3.3.4 for release. I will check for new posts before I publish

As it stands there are two changes:

- removed incompatible SW build check to fix problems for users with Westmere class CPUs.
- optimized m7m AES +65%, SSE2 %59%

Edit: This is what v3.3.4 should look like when compiler for and run on westmere. It is not actual
program output because I don't have a westmere CPU.

Code:

   CPU arch supports AES-AVX...NO.
   CPU arch supports SSE2......YES.
Starting mining without AES-AVX optimizations...

 

cpuminer-opt v3.3.4 released.

https://drive.google.com/file/d/0B0lVSGQYLJIZVTFNSkZ0elRQZ2M/view?usp=sharing

[th3.r00t]

AMD

Code:

         **********  cpuminer-opt 3.3.4  ***********
     A CPU miner with multi algo support and optimized for CPUs
     with AES_NI and AVX extensions.
     BTC donation address: 12tdvfF7KmAsihBXQXynT6E6th2c2pByTT
     Forked from TPruvot's cpuminer-multi with credits
     to Lucas Jones, elmad, palmd, djm34, pooler, ig0tik3d,
     Wolf0 and Jeff Garzik.

Checking CPU capatibility...
        AMD Phenom(tm) II X4 940 Processor
   CPU arch supports AES-AVX...NO.
   CPU arch supports SSE2.....YES.
YES.
Starting mining without AES-AVX optimizations...

Something is missing there...

Intel

Code:

         **********  cpuminer-opt 3.3.4  ***********
     A CPU miner with multi algo support and optimized for CPUs
     with AES_NI and AVX extensions.
     BTC donation address: 12tdvfF7KmAsihBXQXynT6E6th2c2pByTT
     Forked from TPruvot's cpuminer-multi with credits
     to Lucas Jones, elmad, palmd, djm34, pooler, ig0tik3d,
     Wolf0 and Jeff Garzik.

Checking CPU capatibility...
        Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz
   CPU arch supports AES-AVX...YES.
   SW built for AES-AVX........YES.
   Algo supports AES-AVX.......YES.
Start mining with AES-AVX optimizations...

[joblo]

AMD

Code:

         **********  cpuminer-opt 3.3.4  ***********
     A CPU miner with multi algo support and optimized for CPUs
     with AES_NI and AVX extensions.
     BTC donation address: 12tdvfF7KmAsihBXQXynT6E6th2c2pByTT
     Forked from TPruvot's cpuminer-multi with credits
     to Lucas Jones, elmad, palmd, djm34, pooler, ig0tik3d,
     Wolf0 and Jeff Garzik.

Checking CPU capatibility...
        AMD Phenom(tm) II X4 940 Processor
   CPU arch supports AES-AVX...NO.
   CPU arch supports SSE2.....YES.
YES.
Starting mining without AES-AVX optimizations...

Something is missing there...

Intel

Code:

         **********  cpuminer-opt 3.3.4  ***********
     A CPU miner with multi algo support and optimized for CPUs
     with AES_NI and AVX extensions.
     BTC donation address: 12tdvfF7KmAsihBXQXynT6E6th2c2pByTT
     Forked from TPruvot's cpuminer-multi with credits
     to Lucas Jones, elmad, palmd, djm34, pooler, ig0tik3d,
     Wolf0 and Jeff Garzik.

Checking CPU capatibility...
        Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz
   CPU arch supports AES-AVX...YES.
   SW built for AES-AVX........YES.
   Algo supports AES-AVX.......YES.
Start mining with AES-AVX optimizations...

Nothing missing just something extra, two answers to the same question.
I'm in no rush to fix it but thanks for pointing it out.

[USScrypto]

L5630 Westmere works just fine with AES-NI and AVX with the latest Win binaries from CMB, performance however seems a bit weird going between 10 and 100 H/s with CPU load being low. Wolfs miner works fine with max load on the CPU...

[joblo]

L5630 Westmere works just fine with AES-NI and AVX with the latest Win binaries from CMB, performance however seems a bit weird going between 10 and 100 H/s with CPU load being low. Wolfs miner works fine with max load on the CPU...

Is that with hodl? I think your CPU is an Ivybridge based on the date of release.

http://ark.intel.com/products/47927/Intel-Xeon-Processor-L5630-12M-Cache-2_13-GHz-5_86-GTs-Intel-QPI?q=L5630

I think you should be getting around 150 H with 8 threads, have you tried the other bins?

[USScrypto]

Yes, for HODL. It is a Westmere-EP CPU, not Ivybridge. It is a lower power model and has less hashrate
It only works with cpuminer.exe, the others all crash when I try them including the amd one.

Just rebooted the computer and it seems to work normally again, the cpu load is maxed out and the hashrate is high., will monitor it for a while just in case.
Before rebooting the load was getting to minimum and maximum all the time with cpuminer.exe, but with wolfs miner it was maxing out at 100% and providing normal hashtrate.

Ok it did the same thing again, started at high CPU load and max hashrate, found a couple of shares and then the performance and CPU load started dropping to about 10% of the normal value...

[joblo]

Yes, for HODL. It is a Westmere-EP CPU, not Ivybridge. It is a lower power model and has less hashrate
It only works with cpuminer.exe, the others all crash when I try them including the amd one.

Just rebooted the computer and it seems to work normally again, the cpu load is maxed out and the hashrate is high., will monitor it for a while just in case.
Before rebooting the load was getting to minimum and maximum all the time with cpuminer.exe, but with wolfs miner it was maxing out at 100% and providing normal hashtrate.

Yours is the first Westmere that reports support for AES and AVX. Did you get all YESes? If so that's another data point in
to the previous problems.

If you got a YES for AES SW build it helps explain the problems some of the other Westmeres have had. Their CPUs say
NO to AES then fail to detect a comtabible SW build. It fails for SSE2 because it's actually AES.

With v3.3.4 the miner will no longer error out in a situation where it believes the CPU does not support AES but the
SW does. It will try to run anyway. If the CPU Is in fact missing AES it will crash. This would also mean that some
westmere's (like yours) support AES and others don't. Still a confusing situation.

This may leave some westmere users without a useable binary for cpuminer-opt unless they compile their own.

Waiting for feedback from them.

[USScrypto]

Yes, I'm getting all YESes with cpuminer.exe

[joblo]

Yes, I'm getting all YESes with cpuminer.exe

Thanks.

[Roolieman]

Joblo, can you make Windows version so I can test on Xeon E5649 (Westmere-EP, AES without AVX)

[joblo]

Joblo, can you make Windows version so I can test on Xeon E5649 (Westmere-EP, AES without AVX)

Cryptomining Blog has made binaries, follow the link in my sig for details. They have a westmere build.
You should  also try the sandybridge build. I'd like to see your results.

[Roolieman]

sry just saw it on the first page.

Westmere Xeon E5649 now works fine, 2 CPU's are throwing ~700kH/s (using cpuminer.exe)

Sandybridge build on i5-2500K gets ~250kH/s

Older Xeon's E5540 (Gainestown) with only SSE2 support are starting miner but crashes every time after receiving first data (using cpuminer.exe)

Checking CPU capatibility...
        Intel(R) Xeon(R) CPU           E5540  @ 2.53GHz
   CPU arch supports AES-AVX...NO.
   CPU arch supports SSE2.....YES.
YES.
Starting mining without AES-AVX optimizations...


Problem signature:
  Problem Event Name:   APPCRASH
  Application Name:   cpuminer.exe
  Application Version:   0.0.0.0
  Application Timestamp:   00130158
  Fault Module Name:   cpuminer.exe
  Fault Module Version:   0.0.0.0
  Fault Module Timestamp:   00130158
  Exception Code:   c000001d
  Exception Offset:   00000000000b7bd1
  OS Version:   6.3.9600.2.0.0.272.7
  Locale ID:   1033
  Additional Information 1:   3644
  Additional Information 2:   3644a7a02c1327208b46c488f6992045
  Additional Information 3:   0f4d
  Additional Information 4:   0f4d7611d008af45087a270ad4ae60b1

[hmage]

AMD

Code:

         **********  cpuminer-opt 3.3.4  ***********
     A CPU miner with multi algo support and optimized for CPUs
     with AES_NI and AVX extensions.
     BTC donation address: 12tdvfF7KmAsihBXQXynT6E6th2c2pByTT
     Forked from TPruvot's cpuminer-multi with credits
     to Lucas Jones, elmad, palmd, djm34, pooler, ig0tik3d,
     Wolf0 and Jeff Garzik.

Checking CPU capatibility...
        AMD Phenom(tm) II X4 940 Processor
   CPU arch supports AES-AVX...NO.
   CPU arch supports SSE2.....YES.
YES.
Starting mining without AES-AVX optimizations...

Something is missing there...

Intel

Code:

         **********  cpuminer-opt 3.3.4  ***********
     A CPU miner with multi algo support and optimized for CPUs
     with AES_NI and AVX extensions.
     BTC donation address: 12tdvfF7KmAsihBXQXynT6E6th2c2pByTT
     Forked from TPruvot's cpuminer-multi with credits
     to Lucas Jones, elmad, palmd, djm34, pooler, ig0tik3d,
     Wolf0 and Jeff Garzik.

Checking CPU capatibility...
        Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz
   CPU arch supports AES-AVX...YES.
   SW built for AES-AVX........YES.
   Algo supports AES-AVX.......YES.
Start mining with AES-AVX optimizations...

Nothing missing just something extra, two answers to the same question.
I'm in no rush to fix it but thanks for pointing it out.

First snippet says no to AES, yes to SSE2, and yes for nothing.

Second snippet says yes to AES, yes to AES, yes to AES.

See that second snippet doesn't mention SSE2 at all?

To avoid user confusion, I suggest you simplify the logic and always display yes/no answers to sse2, aes and avx, like this:

Code:

Checking compatibility of this cpuminer and CPU (Intel(R) Core(TM) i5-4570S CPU @ 2.90GHz)
   cpuminer expects SSE2:           YES.
   cpuminer expects AES:            YES.
   cpuminer expects AVX:            YES.
   CPU supports SSE2:               YES.
   CPU supports AES:                YES.
   CPU supports AVX:                YES.
   requested algo supports AES+AVX: YES.
[2016-06-03 04:53:49] 4 miner threads started, using 'hodl' algorithm.

Diff is here -- https://github.com/hmage/cpuminer-opt/commit/ddcf4d6d5bf2ace2582947eaf9ff565114460f2f

[hmage]

By the way, after that patch, I've found a bug:

Code:

Checking compatibility of this cpuminer and CPU (Intel(R) Core(TM)2 Quad CPU    Q9400  @ 2.66GHz)
   cpuminer expects SSE2:           YES.
   cpuminer expects AES:            no
   cpuminer expects AVX:            no
   CPU supports SSE2:               YES.
   CPU supports AES:                no
   CPU supports AVX:                YES.
   requested algo supports AES+AVX: YES.
[2016-06-03 04:57:04] 4 miner threads started, using 'hodl' algorithm.

See that "CPU supports AVX: yes"? That's a bug in has_avx(). For some reason the code you inherited returns yes if CPU either has AVX or OSXSAVE flag. Fix — https://github.com/hmage/cpuminer-opt/commit/5975fcf2ecad7057a254cc358e55434059124971

cpuid on that machine reports that OSXSAVE flag is supported by this cpu but not AVX:

Code:

$ cpuid -1|egrep -i 'avx|xsave'
      FXSAVE/FXRSTOR                         = true
      XSAVE/XSTOR states                      = true
      OS-enabled XSAVE/XSTOR                  = true
      AVX: advanced vector extensions         = false
      AVX2: advanced vector extensions 2       = false
      AVX512F: AVX-512 foundation instructions = false
      AVX512PF: prefetch instructions          = false
      AVX512ER: exponent & reciprocal instrs   = false
      AVX512CD: conflict detection instrs      = false
   XSAVE features (0xd/0):
         XCR0 field supported: AVX state      = false
         XCR0 field supported: AVX-512 state  = false
      bytes required by XSAVE/XRSTOR area     = 0x00000240 (576)
   XSAVE features (0xd/1):
      XSAVEOPT instruction                        = false
      XSAVEC instruction                          = false
      XSAVES/XRSTORS instructions                 = false

[joblo]

By the way, after that patch, I've found a bug:

Code:

Checking compatibility of this cpuminer and CPU (Intel(R) Core(TM)2 Quad CPU    Q9400  @ 2.66GHz)
   cpuminer expects SSE2:           YES.
   cpuminer expects AES:            no
   cpuminer expects AVX:            no
   CPU supports SSE2:               YES.
   CPU supports AES:                no
   CPU supports AVX:                YES.
   requested algo supports AES+AVX: YES.
[2016-06-03 04:57:04] 4 miner threads started, using 'hodl' algorithm.

See that "CPU supports AVX: yes"? That's a bug in has_avx(). For some reason the code you inherited returns yes if CPU either has AVX or OSXSAVE flag. Fix — https://github.com/hmage/cpuminer-opt/commit/5975fcf2ecad7057a254cc358e55434059124971

cpuid on that machine reports that OSXSAVE flag is supported by this cpu but not AVX:

Code:

$ cpuid -1|egrep -i 'avx|xsave'
      FXSAVE/FXRSTOR                         = true
      XSAVE/XSTOR states                      = true
      OS-enabled XSAVE/XSTOR                  = true
      AVX: advanced vector extensions         = false
      AVX2: advanced vector extensions 2       = false
      AVX512F: AVX-512 foundation instructions = false
      AVX512PF: prefetch instructions          = false
      AVX512ER: exponent & reciprocal instrs   = false
      AVX512CD: conflict detection instrs      = false
   XSAVE features (0xd/0):
         XCR0 field supported: AVX state      = false
         XCR0 field supported: AVX-512 state  = false
      bytes required by XSAVE/XRSTOR area     = 0x00000240 (576)
   XSAVE features (0xd/1):
      XSAVEOPT instruction                        = false
      XSAVEC instruction                          = false
      XSAVES/XRSTORS instructions                 = false

I need to make two points. In v3.3.4 the checks are all non-enforcing, regardless of the results
of the capabilities check. Without a Westmere of my own to test with I need to know how it really works
so I had to make it permissive and risk the miner crashing.

Yes you found a bug, mine by the way. I just assumed the AVX1 flag was what it says. It could result in a false positive
CPU AES check only if the CPU has AES but not AVX. If the CPU does not have AES the false AVX reading has no effect.
This bug may have a bearing on the Westmere problem but is no worse than the previous check that only read the
AES bit of CPUID.

I need to fully understand the architecture of Westmere as well as why the __AES__ macro seems to return different
results on different CPUs with the same SW build. The problem with westmere CPUs is all about the __AES__ macro.

I repeat it's all about the __AES__ macro on Westmere CPUs using the Westmere binary. At this point I don't really care
what the flags say, I just want the miner to work on Westmere. Once the Westmere architecture is characterized  and
if the __AES__ discrepency is understood, only then will I think about revisiting the flags. I've already broken enough
shit with my inability to test on Westmere. These guys need a working build.

The key questions are:

1. Are there two versions of Westmere CPUs with differing support for AES and AVX but using the same compile arch.

2. Was the CMB Westmere build compiled natively on Westmere or crosscompiled using -marh=corei7.

3. What CPU capabilities does the corei7 compile arch support and is it compatible with all Westmere CPUs.

I need those questions answered before trying anything else.

[joblo]

First snippet says no to AES, yes to SSE2, and yes for nothing.

Wrong. First snippet says yes to AES CPU, second snippet says YES to SSE2 CPU,
third snippet say YES to SSE2 CPU again.