[LOCKED] cpuminer-opt v3.12.3, open source optimized multi-algo CPU miner

[joblo]

cpuminer-opt v3.5.4 is released.

- x11evo fixed (broken in v3.5.2) and optimized 23% faster
- Small improvements of 1% to 3% on many algos including timetravel,
   xevan and cryptonight.
- More code cleanup and compiler warning reduction.
- Improved checking for missing command line arguments.

Errata: AMD users have reported the miner exits silently after 50 share submissions
when mining timetravel algo. This problem is also present in previous versions
and is under investigation.

https://bitcointalk.org/index.php?topic=1326803.msg13542056#msg13542056

[Pennywis3]

Im using a few e5 2690 v2 ( 10 Core / 20 Threads ).

Using 12 threads for mining, geting about 580 h/s on Windows.
Am i doing something wrong? Anyway to speed this up? ( with Claymore i get about 740h/s ).

Optimum thread count for cryptonight is cache size in MB /  2. On an i7 ( 8 MB) 4 threads is best, an i5 (6 MB) 3 threads, etc,
You need 24 MB of cache for 12 threads, try reducing it to match your cache.

e5 2690 v2 has 25MB of cache

Well, ill try to figure something out.

[joblo]

Im using a few e5 2690 v2 ( 10 Core / 20 Threads ).

Using 12 threads for mining, geting about 580 h/s on Windows.
Am i doing something wrong? Anyway to speed this up? ( with Claymore i get about 740h/s ).

Optimum thread count for cryptonight is cache size in MB /  2. On an i7 ( 8 MB) 4 threads is best, an i5 (6 MB) 3 threads, etc,
You need 24 MB of cache for 12 threads, try reducing it to match your cache.

e5 2690 v2 has 25MB of cache

Well, ill try to figure something out.

OK, so I didn't check the specs before replying. I just answered your question directly, what you could be doing wrong.

You also could have mentioned the cache size in your OP knowing it was important. You could also have mentioned what
version and whether there is a change from previous versions, or if this is your first time, dude.

It helps if you list the possibilities you eliminated, comparing CPU usage etc. To the best of my knowledge Claymore
performs slightly better than cpuminer-opt on cryptonight but most of that has been using 2 channel memory.
I don't know how they compare with 4 channels. Has he released a new faster version?

[felixbrucker]

@joblo, i also just tested the amd issue with a sse2 bin, same problem, so its not some aes or avx code stuff (which amd might have done differently)

[joblo]

This can be a starting point for the investigation into the problem of the miner exiting after
50 submits when mining timetravel algo using an AMD CPU:

Problem description:

- silent exit after 50 submits
- only timetravel algo and AMD CPU
- multiple CPUs and architectures (SSE2 only or AES/AVX)
- reproduced by 3 users in versions 3.5.3 and 3.5.4

Analysis:

- there is no code to discriminate based on CPU type.
- there is no code to discriminate based on algo
- there is no code that makes decisions based on share count
- there is no code that exits voluntarilly without displaying a message
- other than a crash there is no timetravel specific code that can cause the program to exit

Based on the analysis above the only possibilities are a silent crash or an external event.
Reproducing the problem with SSE2 only eliminates a crash due to AES or AVX instructions.
A silent crash in timetravel code is still a remote possibility, but why only AMD.

An external event seems like a long shot but must be considered. That external force would
need to know the share count so that eliminates any other local software. The only entity
other than the miner itself that has this knowledge is the pool through the stratum connection.

I don't know if it is possible for the remote stratum server to somehow instruct or cause the miner
to exit and if so whether the miner would do so silently.

Plan:

I will do more analysis of the stratum code to see if it could exit silently.

I would like users to test using other timetravel miners on AMD CPUs. There is cpuminer-multi as
well as one or two others mentioned in the machinecoin thread

For Linux users it would be worth testing with gdb. At least with a backtrace we can see where the
program exited.

[felixbrucker]

cpuminer-multi 1.3.1 x64 (referenced in the mac thread) does not exit on my amd ( but is a lot slower )

[joblo]

cpuminer-multi 1.3.1 x64 (referenced in the mac thread) does not exit on my amd ( but is a lot slower )

The plot thickens. It means the problem is in the miner. A silent crash seems more likely. I had thought
of an incompatibility with some optimized code but you eliminated that. That it happens only with
timetravel means it probably crashed in timetravel code. Much of that code is cloned from other algos so
I will focus on the code that is unique.

[Epsylon3]

50 ... the benchmark submit ?

about that, its nice to receive all your bench stats guys, but joblo, could you add the "stats" presence check in the password to enable/allow it

https://github.com/tpruvot/cpuminer-multi/blob/linux/cpu-miner.c#L3446

[felixbrucker]

50 ... the benchmark submit ?

about that, its nice to receive all your bench stats guys, but joblo, could you add the "stats" presence check in the password to enable/allow it

https://github.com/tpruvot/cpuminer-multi/blob/linux/cpu-miner.c#L3446

nice find, just checked yiimp, no amd stats available

[joblo]

50 ... the benchmark submit ?

about that, its nice to receive all your bench stats guys, but joblo, could you add the "stats" presence check in the password to enable/allow it

https://github.com/tpruvot/cpuminer-multi/blob/linux/cpu-miner.c#L3446

nice find, just checked yiimp, no amd stats available

I think I might be getting part of the picture. I found no mention of 50 submits in the miner code so I presume
it comes from the pool. I assume the pool requests the data after 50 submits in the session.

I can't yet connect it to AMD or timetravel or the program exiting.

After a quick look at the stratum code it doesn't look like it would exit or exit without a message
if stats fail. I also didn't find anything that would single out AMD other than reporting it in the stats.
And I didn't find anything that would connect to timetravel other than simly reporting it.

It looks like we found the trigger only. Removing the trigger may make the problem disappear for
now but I think this needs to be understood better.

I do have a lead in this area. Timetravel has a local function to swap 2 variables using a temp. I replaced
it with an oiptimized version that doesn't require the use of a temp. X11evo has a similar function and when
I tried to implement the optimized version it segfaulted. I'm wondering if there is a macro issue affecting
operator precedence. I've never had a good understanding of all the parentheses needed in c but it seems
like I should encapsulate the args in the macro...

Code:

#define swap_vars(a,b) a^=b; b^=a; a^=b;

Code:

#define swap_vars(a,b) (a)^=(b); (b)^=(a); (a)^=(b);

I'd like to leave the trigger in place  to help get to the root problem.

Edit:

It still segfaults on x11evo with the macro change so it's a problem with the macro. This makes the
connection to timetravel. There is still no connection to AMD. That technique for swapping variables is tried and
true so it's starting to look like a compiler issue, possibly related to passing an expression as the arg instead
of a simple arg. Encapsulating the args should ensure they are fully evaluated before being used by the macro.
That this macro also has problems on Intel suggests it's not limited to AMD. Encapsulating the macro's args
should have eliminated any expansion weirdness due to the args being expressions rather than simple variables.
It's starting to look like the compiler messed up unless I missing something.

Any thoughts on passing expressions as arguments to macros?

Edit: more weird behaviour. Evaluating the args to local variables then passing the variables to the macro changed the
behaviour. It no longer crashed but produced only rejects. This change had an effect but it disn't fix iteven though the
change should have had no effect. It looks like this macro is cursed.

[joblo]

I just pushed a change to git that restores the old swap function. Awaiting test results.

I  still can't connect how stats collection triggers the problem with the macro even though
it's clear there is a problem with it. It's also not clear why it works on timetravel on Intel CPUs,
works for a while on AMD and doesn't work at all on x11evo. Bizarre.

Epsylon3, regarding the change you suggested, how does it affect things? Have there been
problems with stats collection from Intel CPUs?

[Epsylon3]

the pool always asks for stats at 50 100 150 and 200 shares, but the miner can ignore the request...

[joblo]

the pool always asks for stats at 50 100 150 and 200 shares, but the miner can ignore the request...

So your suggested code change enables the miner to ignore the request?

[Epsylon3]

by default its disabled in my miners, only -p stats allow it or -p "c=MAC,stats" etc

[joblo]

by default its disabled in my miners, only -p stats allow it or -p "c=MAC,stats" etc

I'm trying to understand what the change does.

Edit: it looks like opt_stratum_stats default will always be overridden by your code change. Also there is no command line
arg for the user to change it, and even if there was your code would always override it. It doesn't look right.

Edit: I've changed your code to do  &= instead of = to prevent it from overriding a previous disable setting.
Not pushed to git and not included in 3.5.5-pre.

Code:

if (rpc_pass && rpc_user)
     opt_stratum_stats &= (strstr(rpc_pass, "stats") != NULL) || (strcmp(rpc_user, "benchmark") == 0);

But it still needs a command line option be implemented to allow users to enable stats.
And I still don't know what this code does.

[Epsylon3]

the default (to false) is here to highlight the fact its disabled by default... also -p is optional

http://yiimp.ccminer.org/site/benchmarks

[joblo]

the default (to false) is here to highlight the fact its disabled by default... also -p is optional

http://yiimp.ccminer.org/site/benchmarks

OK I get it now. The password is how the user enables stats submission. I had the default set to true
which appears to be why you were getting so many stats reports. Fortunately this allowed the discovery
of a more serious issue.

I'll add your code as is and change the default to false in the next release.

[oldDIN]

joblo
 -"x11evo"
Checked on version 3.5.1 - it works on amd and intel well
version 3.5.5 - all rejects

-"MAC"
Yes, v 3.5.5 with code

Code:

if (rpc_pass && rpc_user)
     opt_stratum_stats &= (strstr(rpc_pass, "stats") != NULL) || (strcmp(rpc_user, "benchmark") == 0);

works good

remind-fx-6300, i3-2120

[joblo]

joblo
 -"x11evo"
Checked on version 3.5.1 - it works on amd and intel well
version 3.5.5 - all rejects

-"MAC"
Yes, v 3.5.5 with code

Code:

if (rpc_pass && rpc_user)
     opt_stratum_stats &= (strstr(rpc_pass, "stats") != NULL) || (strcmp(rpc_user, "benchmark") == 0);

works good

remind-fx-6300, i3-2120

Thanks for testing MAC but can you repeat the test with v3.5.5-pre exatcly as it exists in GIT? Unfotunately using the
two line above made your test invalid. v3.5.5-pre should be used only for testing timetravel on AMD.

As far as x11evo is concerned it works fine for me in 3.5.4 and I made no changes since so you should use 3.5.4.
Since x11evo works for me it may be a cpu architecture issue.  Please show the miner's startup messages showing
which features are available to see if I can reproduce the problem. Maybe Felix can help out with different AMD rchitectures
now that binaries are available. Here is the output from my i5-2400,  it is the same architecture as your i3-2130.

Code:

coin@sys16 ~/miners/cpu/cpuminer-opt/cpuminer-opt-3.5.4 $ ./cpuminer -a x11evo -o [...]

         **********  cpuminer-opt 3.5.4  *********** 
     A CPU miner with multi algo support and optimized for CPUs
     with AES_NI and AVX extensions.
     BTC donation address: 12tdvfF7KmAsihBXQXynT6E6th2c2pByTT
     Forked from TPruvot's cpuminer-multi with credits
     to Lucas Jones, elmad, palmd, djm34, pooler, ig0tik3d,
     Wolf0, Jeff Garzik and Optiminer.

CPU:         Intel(R) Core(TM) i5-2400 CPU @ 3.10GHz
CPU features: SSE2 AES AVX
SW built on Feb  2 2017 with GCC 4.8.4
SW features: SSE2 AES AVX
Algo features: SSE2 AES AVX AVX2
Start mining with SSE2 AES AVX

[2017-02-02 09:48:04] Starting Stratum on stratum+tcp://x11evo.mine.zpool.ca:3553
[2017-02-02 09:48:04] 4 miner threads started, using 'x11evo' algorithm.
[2017-02-02 09:48:05] x11evo block 357418, diff 0.741
[2017-02-02 09:48:12] x11evo block 357419, diff 0.918
[2017-02-02 09:48:12] CPU #0: 454.97 kH, 66.37 kH/s
[2017-02-02 09:48:12] CPU #2: 456.36 kH, 66.61 kH/s
[2017-02-02 09:48:12] CPU #3: 454.54 kH, 66.36 kH/s
[2017-02-02 09:48:12] CPU #1: 456.28 kH, 66.62 kH/s
[2017-02-02 09:49:05] CPU #1: 3575.55 kH, 67.11 kH/s
[2017-02-02 09:49:05] CPU #0: 3564.84 kH, 66.91 kH/s
[2017-02-02 09:49:05] CPU #3: 3560.21 kH, 66.82 kH/s
[2017-02-02 09:49:05] CPU #2: 3577.08 kH, 67.14 kH/s
[2017-02-02 09:49:08] x11evo block 357420, diff 0.741
[2017-02-02 09:49:08] CPU #0: 156.91 kH, 66.90 kH/s
[2017-02-02 09:49:08] CPU #3: 156.89 kH, 66.89 kH/s
[2017-02-02 09:49:08] CPU #1: 157.40 kH, 67.10 kH/s
[2017-02-02 09:49:08] CPU #2: 157.36 kH, 67.09 kH/s
[2017-02-02 09:49:35] CPU #1: 1819.90 kH, 67.11 kH/s
[2017-02-02 09:49:35] Accepted 1/1 (100%), 2291.07 kH, 267.99 kH/s, 68C
[2017-02-02 09:49:38] x11evo block 357421, diff 0.913
[2017-02-02 09:49:38] CPU #3: 2047.45 kH, 66.81 kH/s

Edit: also please include whether you compile yourself or use a windows binary and which
 architecture you use.

[felixbrucker]

i can test it if there is a full cmdline

i have somewhat lost track of all the different problems and version that should fix/be tested against, so a brief summary with the cmd to test would be great as well