[LOCKED] cpuminer-opt v3.12.3, open source optimized multi-algo CPU miner

[joblo]

And not a single one of you mentioned which operating system you are using. How 'bout it?

[4ward]

And not a single one of you mentioned which operating system you are using. How 'bout it?

Windows 10

ps. judging from "F:\Miners\CPU>", guytechie also uses windows

[joblo]

And not a single one of you mentioned which operating system you are using. How 'bout it?

Windows 10

ps. judging from "F:\Miners\CPU>", guytechie also uses windows

Thank you. That's important because I tested on Linux and it worked there. I now have a lead to the problem.
I suspect a data alignment problem as it has happened before with Windows and not Linux. I now know where
to look.

Edit: I've reproduced it but it's going to be a PITA to debug. I see no obvious alignment issues in the code so I have
to localize it by trial and error. It's cross compiled on a different machine on a different floor and I can't use gdb.

[somaton]

i'm using win 8.1 64-bit, but i think it does not matter that much which version of windows. Thanks.

[joblo]

i'm using win 8.1 64-bit, but i think it does not matter that much which version of windows. Thanks.

Thanks. I get frustrated when people ask for help and provide little data. It makes them look lazy,
makes more work for me and delays solving the problem.

[joblo]

I'm hesitant to bring this up with this Windows crash problem that has me stumped for now,
but there seems to be growing interest in Get Block Template (GBT) protocol for solo mining.

GBT code has existed in cpuminer-opt for some time but it is in an unknown state. I have never
tested it and few users have tried until now.

As usual it's difficult to get info from users but here is my understanding of the situation so far.

GBT was reported to work for hexxcoin using lyra2z330 but "it is sticky" whatever that means.
No other info was provided.

An issue was opened reporting it does not work for hodl.
https://github.com/JayDDee/cpuminer-opt/issues/77
This appears to be a GBT version issue and specific to hodl as hodl was not part of the initial
cpuminer-multi fork.

Another issue was opened reporting GBT doesn't work but no further info was provided,
not even the algo

If you have tried GBT and it works please report which algos. Similarly, if it doesn't work
please report with console debug log. It would also be useful if you know of others miners
that do work for those algos and provide console debug logs of both for comparison.

[joblo]

No quick fix for Windows crash problem in v3.8.1.

I've ruled out data alignment as the problem. the crash is occurring when calling a function.
There nothing special about the function, just a regular void function.

There are no compile warnings and it works fine on linux. I'm stuck at this point with no idea how
to proceed.

here's the code in question from algo/luffa/luffa-hash-2way.c

Code:

static void rnd512_2way( luffa_2way_context *state, __m256i msg1, __m256i msg0 )
{
    __m256i t[2];
    __m256i *chainv = state->chainv;
    __m256i tmp[2];
    __m256i x[8];

    t[0] = chainv[0];
    t[1] = chainv[1];

[snip]

int luffa_2way_update_close( luffa_2way_context *state,
                 void *output, const void *data, size_t inlen )
{
// Optimized for integrals of 16 bytes, good for 64 and 80 byte len
    __m256i *vdata  = (__m256i*)data;
    int i;
    int blocks = (int)( inlen / 32 );
    state->rembytes = inlen % 32;

    // full blocks
    for ( i = 0; i < blocks; i++, vdata+=2 )
       rnd512_2way( state, mm256_bswap_32( vdata[1] ),
                           mm256_bswap_32( vdata[0] ) );

It crashes when calling rnd512_2way. A printf as the first statement is never called.
I verified the alignment of vdata and that the function arguments could be evaluated.
The exception code is c0000005, a standard segfault. Given this occurred calling a function
it can be assumed it was a stack access that faulted but there was no report of stack smashing.
That's as far as I can take it, I can't find anything suspicious in the code.

[guytechie]

And not a single one of you mentioned which operating system you are using. How 'bout it?

Sorry, I keep thinking this is only for Windows.  I forgot about the Linux folks.

Other than the OS, I thought my info was pretty thorough.

I was about to test the other algos, but didn't have the time.  I just went back to 3.8.0 for now.


As for your stumpedness, I'm no coder, but I do a lot of troubleshooting.  What has changed from 3.8.0.1 (if that's the last version that worked) and 3.8.1?

If a lot has, start from 3.8.0.1 code base and start adding things back from 3.8.1 until you can reproduce the issue.

[joblo]

cpuminer-opt-3.8.1.1

https://github.com/JayDDee/cpuminer-opt/releases/tag/v3.8.1.1

Fixed Windows AVX2 crash in 3.8.1.

[joblo]

And not a single one of you mentioned which operating system you are using. How 'bout it?

Sorry, I keep thinking this is only for Windows.  I forgot about the Linux folks.

Other than the OS, I thought my info was pretty thorough.

I was about to test the other algos, but didn't have the time.  I just went back to 3.8.0 for now.


As for your stumpedness, I'm no coder, but I do a lot of troubleshooting.  What has changed from 3.8.0.1 (if that's the last version that worked) and 3.8.1?

If a lot has, start from 3.8.0.1 code base and start adding things back from 3.8.1 until you can reproduce the issue.

With me it's the opposite, Windows is an after thought.

Well that was a strange one. I knew precisely where the problem was, a function call, so there weren't very many
possibilities so I just tried changing things. It turns out Windows doesn't like passing vector arguments by value.
After I changed the function to pass by reference it was fixed. That's not a problem if the arguments are passed in
registers but it is a problem if passed on the stack and the stack isn't aligned properly. The Windows stack is
apparently alligned to 16 bytes so I have to assume the arguments were passed on the stack. This is a hit or
miss issue because the stack may be aligned naturally at times. I don't know if I got lucky on Linux or if
Windows just sucks.

[somaton]

Just tried new 3.8.1.1 and it is working now, only neoscrypt not working. With 3.8.0.1 myriad-groestl speed is 10.95 mh/s, but with 3.8.1 and 3.8.1.1 it dropped to 7.65 mh/s. Using cpuminer-avx2-sha.exe with Ryzen 1800x, Win 8.1 64-bit. Thanks.

[joblo]

Just tried new 3.8.1.1 and it is working now, only neoscrypt not working. With 3.8.0.1 myriad-groestl speed is 10.95 mh/s, but with 3.8.1 and 3.8.1.1 it dropped to 7.65 mh/s. Using cpuminer-avx2-sha.exe with Ryzen 1800x, Win 8.1 64-bit. Thanks.

Benchmark or pool?

Edit: my testing shows myr-gr only produces rejects and neoscrypt works on a pool. (AVX2 Linux)
I don't care so much about benchmark so can you retest on a real pool to confirm?

You should also see more speed on those algos that crashed on 3.8.1.

[im10er]

I keep running into this issue when I try to run cpuminer.. anyone have any ideas?

Code:

         **********  cpuminer-opt 3.8.1.1  ***********
     A CPU miner with multi algo support and optimized for CPUs
     with AES_NI and AVX2 and SHA extensions.
     BTC donation address: 12tdvfF7KmAsihBXQXynT6E6th2c2pByTT

CPU: Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz.
SW built on Feb 11 2018 with GCC 5.4.0.
CPU features: SSE2 AVX AVX2.
SW features: SSE2 AVX AVX2.
Algo features: SSE2 AES.
The SW build requires a CPU with AES and AVX2!

This is what I do to compile it..

Code:

apt-get update && apt-get -qy install \
 automake \
 build-essential \
 libcurl4-openssl-dev \
 libssl-dev \
 git \
 ca-certificates \
 libjansson-dev libgmp-dev g++ --no-install-recommends

./autogen.sh
CFLAGS="-O3 -march=native -Wall" CXXFLAGS="$CFLAGS -std=gnu++11" 
./configure --with-curl
make

This was my run

Code:

./cpuminer -a cryptonight -o stratum+tcp://cryptonight.usa.nicehash.com:3355 -u <wallet>.crypto-1

Any ideas on what's going wrong? I'm using this repo: https://github.com/JayDDee/cpuminer-opt

[im10er]

I keep running into this issue when I try to run cpuminer.. anyone have any ideas?

Code:

         **********  cpuminer-opt 3.8.1.1  ***********
     A CPU miner with multi algo support and optimized for CPUs
     with AES_NI and AVX2 and SHA extensions.
     BTC donation address: 12tdvfF7KmAsihBXQXynT6E6th2c2pByTT

CPU: Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz.
SW built on Feb 11 2018 with GCC 5.4.0.
CPU features: SSE2 AVX AVX2.
SW features: SSE2 AVX AVX2.
Algo features: SSE2 AES.
The SW build requires a CPU with AES and AVX2!

This is what I do to compile it..

Code:

apt-get update && apt-get -qy install \
 automake \
 build-essential \
 libcurl4-openssl-dev \
 libssl-dev \
 git \
 ca-certificates \
 libjansson-dev libgmp-dev g++ --no-install-recommends

./autogen.sh
CFLAGS="-O3 -march=native -Wall" CXXFLAGS="$CFLAGS -std=gnu++11" 
./configure --with-curl
make

This was my run

Code:

./cpuminer -a cryptonight -o stratum+tcp://cryptonight.usa.nicehash.com:3355 -u <wallet>.crypto-1

Any ideas on what's going wrong? I'm using this repo: https://github.com/JayDDee/cpuminer-opt

ooo maybe it's because this 2630 for some reason doesn't have AES? But according to specs it should: https://ark.intel.com/products/64593/Intel-Xeon-Processor-E5-2630-15M-Cache-2_30-GHz-7_20-GTs-Intel-QPI

[joblo]

I keep running into this issue when I try to run cpuminer.. anyone have any ideas?

Code:

         **********  cpuminer-opt 3.8.1.1  ***********
     A CPU miner with multi algo support and optimized for CPUs
     with AES_NI and AVX2 and SHA extensions.
     BTC donation address: 12tdvfF7KmAsihBXQXynT6E6th2c2pByTT

CPU: Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz.
SW built on Feb 11 2018 with GCC 5.4.0.
CPU features: SSE2 AVX AVX2.
SW features: SSE2 AVX AVX2.
Algo features: SSE2 AES.
The SW build requires a CPU with AES and AVX2!

ooo maybe it's because this 2630 for some reason doesn't have AES? But according to specs it should: https://ark.intel.com/products/64593/Intel-Xeon-Processor-E5-2630-15M-Cache-2_30-GHz-7_20-GTs-Intel-QPI

Maybe AES is disabled in the BIOS. Xeons being server CPUs may have that capability to reduce power
in applications where encryption is not required. Just guessing.

Edit: regardless of why your CPU doesn't have AES, if you can't enable it, the only recourse is to compile
with -march=core2.

[somaton]

Hello, here are some results from myr-gr with 3.8.1.1 and 3.8.0.1, same Ryzen 1800x with Win 8.1 64-bit, cpuminer-avx2-sha:

[2018-02-13 01:04:09] myr-gr block 1875839, diff 568884.412
[2018-02-13 01:04:09] CPU #3: 4045.67 kH, 482.97 kH/s
[2018-02-13 01:04:09] CPU #4: 3807.94 kH, 454.54 kH/s
[2018-02-13 01:04:09] CPU #5: 3983.73 kH, 475.52 kH/s
[2018-02-13 01:04:09] CPU #11: 4013.04 kH, 479.08 kH/s
[2018-02-13 01:04:09] CPU #12: 3937.05 kH, 470.06 kH/s
[2018-02-13 01:04:09] CPU #15: 4067.70 kH, 485.60 kH/s
[2018-02-13 01:04:09] CPU #2: 4058.00 kH, 484.44 kH/s
[2018-02-13 01:04:09] CPU #6: 3925.32 kH, 468.55 kH/s
[2018-02-13 01:04:09] CPU #0: 4011.98 kH, 478.89 kH/s
[2018-02-13 01:04:09] CPU #10: 4042.91 kH, 482.64 kH/s
[2018-02-13 01:04:09] CPU #7: 4059.21 kH, 484.53 kH/s
[2018-02-13 01:04:09] CPU #13: 4017.46 kH, 479.66 kH/s
[2018-02-13 01:04:09] CPU #9: 4046.48 kH, 483.07 kH/s
[2018-02-13 01:04:09] CPU #14: 4017.24 kH, 479.64 kH/s
[2018-02-13 01:04:09] CPU #1: 4026.06 kH, 480.63 kH/s
[2018-02-13 01:04:09] CPU #8: 3781.78 kH, 451.47 kH/s
[2018-02-13 01:04:15] CTRL_C_EVENT received, exiting

C:\cpuminer-opt-3.8.1.1-windows>

---

[2018-02-13 01:08:37] myr-gr block 2336945, diff 99474.286
[2018-02-13 01:08:39] CPU #5: 15.59 MH, 695.65 kH/s
[2018-02-13 01:08:39] CPU #7: 15.43 MH, 688.54 kH/s
[2018-02-13 01:08:39] CPU #6: 15.43 MH, 688.56 kH/s
[2018-02-13 01:08:39] CPU #10: 15.33 MH, 684.17 kH/s
[2018-02-13 01:08:39] CPU #14: 15.40 MH, 687.02 kH/s
[2018-02-13 01:08:39] CPU #4: 14.03 MH, 625.81 kH/s
[2018-02-13 01:08:39] CPU #12: 13.11 MH, 584.91 kH/s
[2018-02-13 01:08:39] CPU #1: 15.50 MH, 691.68 kH/s
[2018-02-13 01:08:39] CPU #2: 15.41 MH, 687.30 kH/s
[2018-02-13 01:08:39] CPU #0: 14.62 MH, 652.03 kH/s
[2018-02-13 01:08:39] CPU #15: 15.49 MH, 691.23 kH/s
[2018-02-13 01:08:39] CPU #8: 14.21 MH, 633.89 kH/s
[2018-02-13 01:08:39] CPU #11: 15.40 MH, 687.28 kH/s
[2018-02-13 01:08:39] CPU #9: 15.36 MH, 685.27 kH/s
[2018-02-13 01:08:39] CPU #3: 15.42 MH, 688.01 kH/s
[2018-02-13 01:08:39] CPU #13: 15.06 MH, 672.08 kH/s
[2018-02-13 01:08:44] CTRL_C_EVENT received, exiting

C:\cpuminer-opt-3.8.0.1-windows>

---

here is output for neoscrypt, no errors given, same story with 3.8.1.1:

         **********  cpuminer-opt 3.8.0.1  ***********
     A CPU miner with multi algo support and optimized for CPUs
     with AES_NI and AVX2 and SHA extensions.
     BTC donation address: 12tdvfF7KmAsihBXQXynT6E6th2c2pByTT

CPU: AMD Ryzen 7 1800X Eight-Core Processor         .
SW built on Feb  5 2018 with GCC 5.3.1.
CPU features: SSE2 AES AVX AVX2 SHA.
SW features: SSE2 AES AVX AVX2 SHA.
Algo features: None.
Start mining with no optimizations.

[2018-02-13 01:15:46] Starting Stratum on stratum+tcp://neoscrypt.mine.ahashpool.com:4233
[2018-02-13 01:15:46] 16 miner threads started, using 'neoscrypt' algorithm.
[2018-02-13 01:15:47] Stratum difficulty set to 2048
[2018-02-13 01:15:57] neoscrypt block 116272, diff 163.966

C:\cpuminer-opt-3.8.0.1-windows>


Thanks!

[phuocduong]

good, thanks dev for new version

[joblo]

Hello, here are some results from myr-gr with 3.8.1.1 and 3.8.0.1, same Ryzen 1800x with Win 8.1 64-bit, cpuminer-avx2-sha:

You're myr-gr test shows nothing, no share was ever submitted. If it had it would have been rejected.
I discovered this when optimizing it for 4 way. Next release it will be fixed and faster.

But I have reproduced that neoscrypt is broken on Windows, it works on Linux I will investigate.

[mangoo]

Does cpuminer-opt support CPU SHA functions for sha256d algo? Below, it doesn't seem to be the case.

Code:

# ./cpuminer --benchmark -a sha256d

         **********  cpuminer-opt 3.8.0  *********** 
     A CPU miner with multi algo support and optimized for CPUs
     with AES_NI and AVX2 and SHA extensions.
     BTC donation address: 12tdvfF7KmAsihBXQXynT6E6th2c2pByTT

CPU: AMD EPYC 7401P 24-Core Processor               .
SW built on Jan 25 2018 with GCC 7.2.0.
CPU features: SSE2 AES AVX AVX2 SHA.
SW features: SSE2 AES AVX AVX2.
Algo features: None.
Start mining with no optimizations.

[2018-02-13 05:20:19] 48 miner threads started, using 'sha256d' algorithm.

[joblo]

Does cpuminer-opt support CPU SHA functions for sha256d algo? Below, it doesn't seem to be the case.

Code:

# ./cpuminer --benchmark -a sha256d

         **********  cpuminer-opt 3.8.0  *********** 
     A CPU miner with multi algo support and optimized for CPUs
     with AES_NI and AVX2 and SHA extensions.
     BTC donation address: 12tdvfF7KmAsihBXQXynT6E6th2c2pByTT

CPU: AMD EPYC 7401P 24-Core Processor               .
SW built on Jan 25 2018 with GCC 7.2.0.
CPU features: SSE2 AES AVX AVX2 SHA.
SW features: SSE2 AES AVX AVX2.
Algo features: None.
Start mining with no optimizations.

[2018-02-13 05:20:19] 48 miner threads started, using 'sha256d' algorithm.

No, sha256d doesn't use the SHA extension. I'm doing some 4 way work with sha256
and sha512. 4 way sha256 might actually be faster than SHA. However, sha256d has
not been on my radar because it is hopelessly uncompetitive on a CPU. I might take a
look at it eventually but, so many more things to do.