[LOCKED] cpuminer-opt v3.12.3, open source optimized multi-algo CPU miner

[Larvitar]

I've uploaded a new windows binaries package of v3.7.7 to git with support for SHA. I also trimmed some
of the file names to remove redundancy.

4way includes avx2
avx2 includes avx
avx includes aes and sse4.2

It's avaiable on the releases page or this direct link:

https://github.com/JayDDee/cpuminer-opt/files/1569739/cpuminer-opt-3.7.7-windows-v2.zip

Be careful with sha, only the AMD Ryzen family supports it at this time.

Consider this a beta for the new Windows build system.

Great! SHA works great! Your miner is blazing fast! I will do the benchmarks like I did with 4ward build.

Apologize me, but can I ask one more build? SHA AVX (not AVX2) version.

I explain:
Ryzen doesn't have a "fine" AVX2 implementation. In some algos, AVX is faster than AVX2. We are trying to find the best setup with SHA, so maybe AVX setup can give one more option to try. What do you think?

[1714023865]

1714023865

[1714023865]

1714023865

[1714023865]

1714023865

[NameTaken]

How fast is the 1950X at these SHA supported algorithms?

[joblo]

I've uploaded a new windows binaries package of v3.7.7 to git with support for SHA. I also trimmed some
of the file names to remove redundancy.

4way includes avx2
avx2 includes avx
avx includes aes and sse4.2

It's avaiable on the releases page or this direct link:

https://github.com/JayDDee/cpuminer-opt/files/1569739/cpuminer-opt-3.7.7-windows-v2.zip

Be careful with sha, only the AMD Ryzen family supports it at this time.

Consider this a beta for the new Windows build system.

Great! SHA works great! Your miner is blazing fast! I will do the benchmarks like I did with 4ward build.

Apologize me, but can I ask one more build? SHA AVX (not AVX2) version.

I explain:
Ryzen doesn't have a "fine" AVX2 implementation. In some algos, AVX is faster than AVX2. We are trying to find the best setup with SHA, so maybe AVX setup can give one more option to try. What do you think?

That's a very reasonable request and doable. However I'm trying to reduce the number of binaries I build, 8 is too many.
Also I'm hesitant to "downgrade" the technology, it just doesn't feel right.

Let me think about it and I'll do something for next release.

I'm considering eliminating the sse42 build. There are no specific optimizations targetting sse42 so there should no
performance loss when using the sse2 build.

If there are no reports showing a benefit to the sse42 build it will be removed.

Edit: I could replace the avx2-sha build with avx-sha. It won't affect any Intel users until Cannonlake next year and Ryzen users
can choose the 4way build if they prefer AVX2. Thoughts?

[joblo]

One thing I couldn't resolve is that neoscrypt always fails to run (same in My9bot's version)

I can look into this now.

I suspect the neoscrypt problem is related to its use of asm. It is controlled by a NOASM
compile flag. I didn't find any hooks for Windows so it should compile the same as Linux.

It works with the old mingw environment used to build previous binaries. The only obvious
difference is the mingw host, The old way was built on a Windows host, the new ones cross
compiled on a Linux host. I don't know if that makes any difference.

Digging deeper will be very time consuming and considering neoscrypt performance on CPU
it's hardly worth the effort, especially if the result is to disable ASM and performance drops.

An easy test would be to recompile with -DNOASM and see if it works and how the performance
compares with the old mingw build. I'll leave it up to you if you're interested. The results will
help determine how to proceed.

[Larvitar]

I've uploaded a new windows binaries package of v3.7.7 to git with support for SHA. I also trimmed some
of the file names to remove redundancy.

4way includes avx2
avx2 includes avx
avx includes aes and sse4.2

It's avaiable on the releases page or this direct link:

https://github.com/JayDDee/cpuminer-opt/files/1569739/cpuminer-opt-3.7.7-windows-v2.zip

Be careful with sha, only the AMD Ryzen family supports it at this time.

Consider this a beta for the new Windows build system.

Great! SHA works great! Your miner is blazing fast! I will do the benchmarks like I did with 4ward build.

Apologize me, but can I ask one more build? SHA AVX (not AVX2) version.

I explain:
Ryzen doesn't have a "fine" AVX2 implementation. In some algos, AVX is faster than AVX2. We are trying to find the best setup with SHA, so maybe AVX setup can give one more option to try. What do you think?

That's a very reasonable request and doable. However I'm trying to reduce the number of binaries I build, 8 is too many.
Also I'm hesitant to "downgrade" the technology, it just doesn't feel right.

Let me think about it and I'll do something for next release.

I'm considering eliminating the sse42 build. There are no specific optimizations targetting sse42 so there should no
performance loss when using the sse2 build.

If there are no reports showing a benefit to the sse42 build it will be removed.

There is two possibilities:
if
- Make a 3.7.7 build with only AVX and SHA to see if SHA-like algos takes advantage of AVX instead AVX2;
If yes
- Split into two packages: normal and SHA zips (or Ryzen zips). SHA doubles the build count, so split into different package could reduce the "useless" executables (to non-Ryzen users).
If no
- Remove AVX-SHA from future builds.

[joblo]

Making 2 packages is more work for me, I'm trying to reduce the work.

I'm leaning toward replacing avx2-sha with avx-sha unless someone shows a good
reason for avx2-sha.

Edit: a couple more points

AVX2 and SHA improve different algos and different parts of the same algos. AVX won't have
any effect on SHA code on Ryzen CPUs. There are no technical concerns with AVX-SHA.

The only question is performance on algos that have use AVX2. The best algo to test this is
lyra2v2, it is almost 100% AVX2 and not too hard on memory so it will expose any weaknesses
in AVX2 on Ryzen.

[4ward]

One thing I couldn't resolve is that neoscrypt always fails to run (same in My9bot's version)

I can look into this now.

I suspect the neoscrypt problem is related to its use of asm. It is controlled by a NOASM
compile flag. I didn't find any hooks for Windows so it should compile the same as Linux.

It works with the old mingw environment used to build previous binaries. The only obvious
difference is the mingw host, The old way was built on a Windows host, the new ones cross
compiled on a Linux host. I don't know if that makes any difference.

Digging deeper will be very time consuming and considering neoscrypt performance on CPU
it's hardly worth the effort, especially if the result is to disable ASM and performance drops.

An easy test would be to recompile with -DNOASM and see if it works and how the performance
compares with the old mingw build. I'll leave it up to you if you're interested. The results will
help determine how to proceed.

-DNOASM didn't solve the issue, it still fails.

Code:

[2017-12-19 18:11:05] 4 miner threads started, using 'neoscrypt' algorithm.
[2017-12-19 18:11:05] Starting Stratum on stratum+tcp://neoscrypt.mine.zpool.ca:4233
[2017-12-19 18:11:05] Binding thread 0 to cpu 0 (mask 1)
[2017-12-19 18:11:05] Binding thread 1 to cpu 1 (mask 2)
[2017-12-19 18:11:05] Binding thread 2 to cpu 2 (mask 4)
[2017-12-19 18:11:05] Binding thread 3 to cpu 3 (mask 8)
[2017-12-19 18:11:07] Stratum session id: 67d8d8d809e8d6ce65058b9626f51cf4
[2017-12-19 18:11:07] Stratum difficulty set to 512
[2017-12-19 18:11:11] DEBUG: job_id='347' extranonce2=00000000 ntime=123a395a
[2017-12-19 18:11:11] neoscrypt block 2011810, diff 110.548

But you are right, this algo is definitely not the best performer, so I don't know if its worth investing much time in it

[joblo]

One thing I couldn't resolve is that neoscrypt always fails to run (same in My9bot's version)

I can look into this now.

-DNOASM didn't solve the issue, it still fails.

But you are right, this algo is definitely not the best performer, so I don't know if its worth investing much time in it

I found a potential problem that could cause data misalignment.

In algo/neoscrypt.c line 56:

Code:

#if (WINDOWS)
/* sizeof(unsigned long) = 4 for MinGW64 */
typedef unsigned long long ulong;
#else
typedef unsigned long ulong;
#endif
typedef unsigned int  uint;

Further down in the code some data defined as uint is accessed as ulong which has stricter
alignment requirements if defined as long long. If the condition is removed and ulong is the
same as uint it shouldn't crash.

I found no other compilation divergence and alignment bugs can slip through unnoticed for
a time. The trigger seems to be the different compile environment.

Can you test to confirm?

[4ward]

One thing I couldn't resolve is that neoscrypt always fails to run (same in My9bot's version)

I can look into this now.

-DNOASM didn't solve the issue, it still fails.

But you are right, this algo is definitely not the best performer, so I don't know if its worth investing much time in it

I found a potential problem that could cause data misalignment.

In algo/neoscrypt.c line 56:

Code:

#if (WINDOWS)
/* sizeof(unsigned long) = 4 for MinGW64 */
typedef unsigned long long ulong;
#else
typedef unsigned long ulong;
#endif
typedef unsigned int  uint;

Further down in the code some data defined as uint is accessed as ulong which has stricter
alignment requirements if defined as long long. If the condition is removed and ulong is the
same as uint it shouldn't crash.

I found no other compilation divergence and alignment bugs can slip through unnoticed for
a time. The trigger seems to be the different compile environment.

Can you test to confirm?

tried this

Code:

// #if (WINDOWS)
/* sizeof(unsigned long) = 4 for MinGW64 */
// typedef unsigned long long ulong;
// #else
typedef unsigned long ulong;
// #endif
typedef unsigned int  uint;

same problem, unless I'm doing it wrong )

[Larvitar]

Making 2 packages is more work for me, I'm trying to reduce the work.

I'm leaning toward replacing avx2-sha with avx-sha unless someone shows a good
reason for avx2-sha.

Edit: a couple more points

AVX2 and SHA improve different algos and different parts of the same algos. AVX won't have
any effect on SHA code on Ryzen CPUs. There are no technical concerns with AVX-SHA.

The only question is performance on algos that have use AVX2. The best algo to test this is
lyra2v2, it is almost 100% AVX2 and not too hard on memory so it will expose any weaknesses
in AVX2 on Ryzen.

I'm getting around 1.6~2MH/s in Lyra2z. But I'm getting low difficult share errors on every share:

Code:

[2017-12-19 17:45:42] Starting Stratum on stratum+tcp://us-east.lyra2z-hub.miningpoolhub.com:20581
[2017-12-19 17:45:42] 16 miner threads started, using 'lyra2rev2' algorithm.
[2017-12-19 17:45:43] Stratum difficulty set to 10
[2017-12-19 17:46:05] CPU #9: 2097.15 kH, 116.83 kH/s
[2017-12-19 17:46:05] CPU #11: 2097.15 kH, 116.65 kH/s
[2017-12-19 17:46:05] CPU #7: 2097.15 kH, 116.33 kH/s
[2017-12-19 17:46:06] CPU #15: 2097.15 kH, 114.68 kH/s
[2017-12-19 17:46:06] CPU #6: 2097.15 kH, 111.85 kH/s
[2017-12-19 17:46:06] CPU #3: 2097.15 kH, 111.58 kH/s
[2017-12-19 17:46:06] CPU #14: 2097.15 kH, 110.05 kH/s
[2017-12-19 17:46:07] CPU #10: 2097.15 kH, 108.93 kH/s
[2017-12-19 17:46:07] CPU #5: 2097.15 kH, 107.02 kH/s
[2017-12-19 17:46:07] CPU #8: 2097.15 kH, 106.52 kH/s
[2017-12-19 17:46:07] CPU #2: 2097.15 kH, 106.20 kH/s
[2017-12-19 17:46:09] CPU #1: 2097.15 kH, 98.64 kH/s
[2017-12-19 17:46:10] CPU #13: 2097.15 kH, 93.44 kH/s
[2017-12-19 17:46:16] CPU #4: 2097.15 kH, 73.31 kH/s
[2017-12-19 17:46:16] CPU #0: 2097.15 kH, 73.11 kH/s
[2017-12-19 17:46:20] CPU #12: 2097.15 kH, 64.31 kH/s
[2017-12-19 17:46:39] Stratum difficulty set to 7
[2017-12-19 17:46:42] CPU #13: 3174.84 kH, 98.57 kH/s
[2017-12-19 17:46:42] Rejected 1/1 (100.0%), 34.63 MH, 1634.59 kH/s
[2017-12-19 17:46:42] reject reason: low difficulty share of 8.935987400308036e-8
[2017-12-19 17:46:42] factor reduced to : 0.67

Is it miner-related or pool-related?

[joblo]

Making 2 packages is more work for me, I'm trying to reduce the work.

I'm leaning toward replacing avx2-sha with avx-sha unless someone shows a good
reason for avx2-sha.

Edit: a couple more points

AVX2 and SHA improve different algos and different parts of the same algos. AVX won't have
any effect on SHA code on Ryzen CPUs. There are no technical concerns with AVX-SHA.

The only question is performance on algos that have use AVX2. The best algo to test this is
lyra2v2, it is almost 100% AVX2 and not too hard on memory so it will expose any weaknesses
in AVX2 on Ryzen.

I'm getting around 1.6~2MH/s in Lyra2z. But I'm getting low difficult share errors on every share:

Code:

[2017-12-19 17:45:42] Starting Stratum on stratum+tcp://us-east.lyra2z-hub.miningpoolhub.com:20581
[2017-12-19 17:45:42] 16 miner threads started, using 'lyra2rev2' algorithm.
[2017-12-19 17:45:43] Stratum difficulty set to 10
[2017-12-19 17:46:05] CPU #9: 2097.15 kH, 116.83 kH/s
[2017-12-19 17:46:05] CPU #11: 2097.15 kH, 116.65 kH/s
[2017-12-19 17:46:05] CPU #7: 2097.15 kH, 116.33 kH/s
[2017-12-19 17:46:06] CPU #15: 2097.15 kH, 114.68 kH/s
[2017-12-19 17:46:06] CPU #6: 2097.15 kH, 111.85 kH/s
[2017-12-19 17:46:06] CPU #3: 2097.15 kH, 111.58 kH/s
[2017-12-19 17:46:06] CPU #14: 2097.15 kH, 110.05 kH/s
[2017-12-19 17:46:07] CPU #10: 2097.15 kH, 108.93 kH/s
[2017-12-19 17:46:07] CPU #5: 2097.15 kH, 107.02 kH/s
[2017-12-19 17:46:07] CPU #8: 2097.15 kH, 106.52 kH/s
[2017-12-19 17:46:07] CPU #2: 2097.15 kH, 106.20 kH/s
[2017-12-19 17:46:09] CPU #1: 2097.15 kH, 98.64 kH/s
[2017-12-19 17:46:10] CPU #13: 2097.15 kH, 93.44 kH/s
[2017-12-19 17:46:16] CPU #4: 2097.15 kH, 73.31 kH/s
[2017-12-19 17:46:16] CPU #0: 2097.15 kH, 73.11 kH/s
[2017-12-19 17:46:20] CPU #12: 2097.15 kH, 64.31 kH/s
[2017-12-19 17:46:39] Stratum difficulty set to 7
[2017-12-19 17:46:42] CPU #13: 3174.84 kH, 98.57 kH/s
[2017-12-19 17:46:42] Rejected 1/1 (100.0%), 34.63 MH, 1634.59 kH/s
[2017-12-19 17:46:42] reject reason: low difficulty share of 8.935987400308036e-8
[2017-12-19 17:46:42] factor reduced to : 0.67

Is it miner-related or pool-related?

User error, look carefullly at the algo.

Another note about lyra2z, 4way is likely slower due to previously mentioned issues with it.

[joblo]

tried this

Code:

// #if (WINDOWS)
/* sizeof(unsigned long) = 4 for MinGW64 */
// typedef unsigned long long ulong;
// #else
typedef unsigned long ulong;
// #endif
typedef unsigned int  uint;

same problem, unless I'm doing it wrong )

It's not wrong, thanks. I'm at a loss to explain it.

I could try to find out where it's crasshing but it may not be necessary. I noticed some optimization
opportunities that I hadn't noticed before but it will require a complete rewrite.

[Larvitar]

Making 2 packages is more work for me, I'm trying to reduce the work.

I'm leaning toward replacing avx2-sha with avx-sha unless someone shows a good
reason for avx2-sha.

Edit: a couple more points

AVX2 and SHA improve different algos and different parts of the same algos. AVX won't have
any effect on SHA code on Ryzen CPUs. There are no technical concerns with AVX-SHA.

The only question is performance on algos that have use AVX2. The best algo to test this is
lyra2v2, it is almost 100% AVX2 and not too hard on memory so it will expose any weaknesses
in AVX2 on Ryzen.

I'm getting around 1.6~2MH/s in Lyra2z. But I'm getting low difficult share errors on every share:

Code:

[2017-12-19 17:45:42] Starting Stratum on stratum+tcp://us-east.lyra2z-hub.miningpoolhub.com:20581
[2017-12-19 17:45:42] 16 miner threads started, using 'lyra2rev2' algorithm.
[2017-12-19 17:45:43] Stratum difficulty set to 10
[2017-12-19 17:46:05] CPU #9: 2097.15 kH, 116.83 kH/s
[2017-12-19 17:46:05] CPU #11: 2097.15 kH, 116.65 kH/s
[2017-12-19 17:46:05] CPU #7: 2097.15 kH, 116.33 kH/s
[2017-12-19 17:46:06] CPU #15: 2097.15 kH, 114.68 kH/s
[2017-12-19 17:46:06] CPU #6: 2097.15 kH, 111.85 kH/s
[2017-12-19 17:46:06] CPU #3: 2097.15 kH, 111.58 kH/s
[2017-12-19 17:46:06] CPU #14: 2097.15 kH, 110.05 kH/s
[2017-12-19 17:46:07] CPU #10: 2097.15 kH, 108.93 kH/s
[2017-12-19 17:46:07] CPU #5: 2097.15 kH, 107.02 kH/s
[2017-12-19 17:46:07] CPU #8: 2097.15 kH, 106.52 kH/s
[2017-12-19 17:46:07] CPU #2: 2097.15 kH, 106.20 kH/s
[2017-12-19 17:46:09] CPU #1: 2097.15 kH, 98.64 kH/s
[2017-12-19 17:46:10] CPU #13: 2097.15 kH, 93.44 kH/s
[2017-12-19 17:46:16] CPU #4: 2097.15 kH, 73.31 kH/s
[2017-12-19 17:46:16] CPU #0: 2097.15 kH, 73.11 kH/s
[2017-12-19 17:46:20] CPU #12: 2097.15 kH, 64.31 kH/s
[2017-12-19 17:46:39] Stratum difficulty set to 7
[2017-12-19 17:46:42] CPU #13: 3174.84 kH, 98.57 kH/s
[2017-12-19 17:46:42] Rejected 1/1 (100.0%), 34.63 MH, 1634.59 kH/s
[2017-12-19 17:46:42] reject reason: low difficulty share of 8.935987400308036e-8
[2017-12-19 17:46:42] factor reduced to : 0.67

Is it miner-related or pool-related?

User error, look carefullly at the algo.

Another note about lyra2z, 4way is likely slower due to previously mentioned issues with it.

Damn! You are right

Too many "Lyras" 

[joblo]

Damn! You are right

Too many "Lyras" 

If you're comparing AVX2 vs AVX performance lyra2v2 is better, less memory hard.

[thin]

I have  a problem running cpuminer-opt  v3.7.x on a couple of machines with win 10 x64. when started, it silently waits several seconds, then exit, without writing  symbol. any advice? do I miss some runtime libs ?

[joblo]

I have  a problem running cpuminer-opt  v3.7.x on a couple of machines with win 10 x64. when started, it silently waits several seconds, then exit, without writing  symbol. any advice? do I miss some runtime libs ?

Advice? Yes, provide proper information. What CPU, algo, command line options?
It's all displayed when the program starts, always provide that when reporting a problem.

[thin]

it does not depend on algo, even if I trying to get help noting displayed. CPU Celeron G3930

C:\App\cpuminer-opt-3.7.7-windows-v2>cpuminer-avx2.exe --help

C:\App\cpuminer-opt-3.7.7-windows-v2>cpuminer-avx.exe --help

C:\App\cpuminer-opt-3.7.7-windows-v2>cpuminer-4way.exe --help

C:\App\cpuminer-opt-3.7.7-windows-v2>cpuminer-4way.exe

C:\App\cpuminer-opt-3.7.7-windows-v2>

[joblo]

it does not depend on algo, even if I trying to get help noting displayed. CPU Celeron G3930

C:\App\cpuminer-opt-3.7.7-windows-v2>cpuminer-avx2.exe --help

C:\App\cpuminer-opt-3.7.7-windows-v2>cpuminer-avx.exe --help

C:\App\cpuminer-opt-3.7.7-windows-v2>cpuminer-4way.exe --help

C:\App\cpuminer-opt-3.7.7-windows-v2>cpuminer-4way.exe

C:\App\cpuminer-opt-3.7.7-windows-v2>

First of all your CPU doesn't have AVX or AVX2 so stay away from those and 4way.
You should be using aes-sse42. But there's a bigger problem if it won't display help.
I've never seen that kind of a problem, it looks like it's your system. No one else is
complaining so the problem is at your end.

[thin]

it does not depend on algo, even if I trying to get help noting displayed. CPU Celeron G3930

C:\App\cpuminer-opt-3.7.7-windows-v2>cpuminer-avx2.exe --help

C:\App\cpuminer-opt-3.7.7-windows-v2>cpuminer-avx.exe --help

C:\App\cpuminer-opt-3.7.7-windows-v2>cpuminer-4way.exe --help

C:\App\cpuminer-opt-3.7.7-windows-v2>cpuminer-4way.exe

C:\App\cpuminer-opt-3.7.7-windows-v2>

First of all your CPU doesn't have AVX or AVX2 so stay away from those and 4way.
You should be using aes-sse42. But there's a bigger problem if it won't display help.
I've never seen that kind of a problem, it looks like it's your system. No one else is
complaining so the problem is at your end.

that's why I ask if I miss some runtime libs ). I never seen such behavior before - it starts, waits silently several seconds, exit. quite unusual.
thanks anyway

[rukez]

Looks like there is some performance problem with yescript16 implementation on coffee-lake cpus
Intel i5 4440 (stock 3.3GHz) @4 threads generates ~600h/s
Intel i7 5820k (no overclock, 3.6GHz) @12 threads generates ~1200h/s in pool and up to 1400 in solo mining (6 threads generate little less) with both cpuminer-opt (3.7.6 and 3.7.7v2) under windows64 and under ubuntu64
Intel i7 8700k (stock 4.3GHz) @12 threads generates only 950h/s (both pool and solo), overclocking to 5Ghz (50x100) with cache overclock to 4.6Ghz (stock is 4.2) gives no profit, even more usually performance degrade (power limit disabled, core temperatures are ~75C so no throttling involved), 1-2-3-4-5-6 threads gives less results, overclocking bus to 130Mhz (also with ram) gives no result - maximum is about 950h/s
Even more funny - on stock frequency, switching from AVX to SSE2 gives some performance boost from 950 to 1000-1050h/s

I understand that 8700k lacks quad-channel RAM and has little bit less L3 cache (12 vs 15Mb), compared to 5820, but bottleneck is obviously something different because ram overclock gives no result (so double channel is not a problem, we should see performance boost when overclocking bus and ram) and cache is also not a problem (25% cache is gone but we gain >30% frequency bonus (when overclocked) so our smaller cache works at higher speeds together with cpu cores - we can put less but more frequent and calc it in less time - ) also, compared to 4440, if cache was a bottleneck, we have twice more (12 vs 6Mb), taking in mind much higher speed and optimized pipelane, if cache only matters, we should have 2x gain, compared to 4440

I hope for a fix