Giulini
|
 |
April 22, 2016, 04:27:28 PM |
|
kim@spiel-2:~/cpuminer-opt-3.1.17$ ./configure CFLAGS="-O3 -march=btver1" --with-curl --with-crypto && make checking build system type... x86_64-unknown-linux-gnu checking host system type... x86_64-unknown-linux-gnu checking target system type... x86_64-unknown-linux-gnu checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for a thread-safe mkdir -p... /bin/mkdir -p checking for gawk... no checking for mawk... mawk checking whether make sets $(MAKE)... yes checking whether make supports nested variables... yes checking whether to enable maintainer-specific portions of Makefiles... no checking for style of include used by make... GNU checking for gcc... gcc checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking whether gcc understands -c and -o together... yes checking dependency style of gcc... gcc3 checking for gcc option to accept ISO C99... -std=gnu99 checking how to run the C preprocessor... gcc -std=gnu99 -E checking for grep that handles long lines and -e... /bin/grep checking for egrep... /bin/grep -E checking whether gcc -std=gnu99 needs -traditional... no checking dependency style of gcc -std=gnu99... gcc3 checking for ranlib... ranlib checking for g++... g++ checking whether we are using the GNU C++ compiler... yes checking whether g++ accepts -g... yes checking dependency style of g++... gcc3 checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking sys/endian.h usability... no checking sys/endian.h presence... no checking for sys/endian.h... no checking sys/param.h usability... yes checking sys/param.h presence... yes checking for sys/param.h... yes checking syslog.h usability... yes checking syslog.h presence... yes checking for syslog.h... yes checking for sys/sysctl.h... yes checking whether be32dec is declared... no checking whether le32dec is declared... no checking whether be32enc is declared... no checking whether le32enc is declared... no checking whether le16dec is declared... no checking whether le16enc is declared... no checking for size_t... yes checking for working alloca.h... yes checking for alloca... yes checking for getopt_long... yes checking whether we can compile AVX code... yes checking whether we can compile XOP code... yes checking whether we can compile AVX2 code... yes checking for json_loads in -ljansson... yes checking for pthread_create in -lpthread... yes checking whether __uint128_t is supported... yes checking that generated files are newer than configure... done configure: creating ./config.status config.status: creating Makefile config.status: creating compat/Makefile config.status: creating compat/jansson/Makefile config.status: creating cpuminer-config.h config.status: cpuminer-config.h is unchanged config.status: executing depfiles commands make all-recursive make[1]: Verzeichnis »/home/kim/cpuminer-opt-3.1.17« wird betreten Making all in compat make[2]: Verzeichnis »/home/kim/cpuminer-opt-3.1.17/compat« wird betreten make[3]: Verzeichnis »/home/kim/cpuminer-opt-3.1.17/compat« wird betreten make[3]: Für das Ziel »all-am« ist nichts zu tun. make[3]: Verzeichnis »/home/kim/cpuminer-opt-3.1.17/compat« wird verlassen make[2]: Verzeichnis »/home/kim/cpuminer-opt-3.1.17/compat« wird verlassen make[2]: Verzeichnis »/home/kim/cpuminer-opt-3.1.17« wird betreten gcc -std=gnu99 -DHAVE_CONFIG_H -I. -Iyes/include -Iyes/include -fno-strict-aliasing -I. -Iyes/include -Iyes/include -Wno-pointer-sign -Wno-pointer-to-int-cast -O3 -march=btver1 -Iyes/include -Iyes/include -MT algo/hodl/cpuminer-hodl-gate.o -MD -MP -MF algo/hodl/.deps/cpuminer-hodl-gate.Tpo -c -o algo/hodl/cpuminer-hodl-gate.o `test -f 'algo/hodl/hodl-gate.c' || echo './'`algo/hodl/hodl-gate.c algo/hodl/hodl-gate.c: In function ‘register_hodl_algo’: algo/hodl/hodl-gate.c:132:7: error: ‘algo_gate_t’ has no member named ‘set_data_size’ gate->set_data_size = (void*)&hodl_set_data_size; ^ make[2]: *** [algo/hodl/cpuminer-hodl-gate.o] Fehler 1 make[2]: Verzeichnis »/home/kim/cpuminer-opt-3.1.17« wird verlassen make[1]: *** [all-recursive] Fehler 1 make[1]: Verzeichnis »/home/kim/cpuminer-opt-3.1.17« wird verlassen make: *** [all] Fehler 2 kim@spiel-2:~/cpuminer-opt-3.1.17$
|
|
|
|
joblo (OP)
Legendary
Offline
Activity: 1470
Merit: 1114
|
 |
April 22, 2016, 04:39:40 PM |
|
algo/hodl/hodl-gate.c:132:7: error: ‘algo_gate_t’ has no member named ‘set_data_size’ gate->set_data_size = (void*)&hodl_set_data_size; ^ make[2]: *** [algo/hodl/cpuminer-hodl-gate.o] Fehler 1 make[2]: Verzeichnis »/home/kim/cpuminer-opt-3.1.17« wird verlassen make[1]: *** [all-recursive] Fehler 1 make[1]: Verzeichnis »/home/kim/cpuminer-opt-3.1.17« wird verlassen make: *** [all] Fehler 2 kim@spiel-2:~/cpuminer-opt-3.1.17$
That's a coding error. I just recompiled successfuly from the tarball I uploaded so don't know why you got that error.
|
|
|
|
hmage
Member

Offline
Activity: 83
Merit: 10
|
 |
April 22, 2016, 04:43:03 PM |
|
algo/hodl/hodl-gate.c:132:7: error: ‘algo_gate_t’ has no member named ‘set_data_size’ gate->set_data_size = (void*)&hodl_set_data_size; ^ make[2]: *** [algo/hodl/cpuminer-hodl-gate.o] Fehler 1 make[2]: Verzeichnis »/home/kim/cpuminer-opt-3.1.17« wird verlassen make[1]: *** [all-recursive] Fehler 1 make[1]: Verzeichnis »/home/kim/cpuminer-opt-3.1.17« wird verlassen make: *** [all] Fehler 2 kim@spiel-2:~/cpuminer-opt-3.1.17$
That's a coding error. I just recompiled successfuly from the tarball I uploaded so don't know why you got that error. This line happens when you're compiling for core2. You probably removed set_data_size everywhere but on that line. I had to remove it in my fork. Tried it out to delete the sourcecode line, then "SW built for SSE2..........NO." change to "YES", but the miner stops working immediately; tried also several -march versions same here with: Sempron145 CPU, configure und make with no mistakes ********** cpuminer-opt 3.1.16 *********** A CPU miner with multi algo support and optimized for CPUs with AES_NI extension. BTC donation address: 12tdvfF7KmAsihBXQXynT6E6th2c2pByTT Forked from TPruvot's cpuminer-multi with credits to Lucas Jones, elmad, palmd, djm34, pooler, ig0tik3d, Wolf0 and Jeff Garzik.
Checking CPU capatibility... AMD Phenom(tm) II X4 940 Processor CPU arch supports AES_NI...NO. CPU arch supports SSE2.....YES. SW built for SSE2..........NO. Incompatible SW build, rebuild with "-march=native" Why? Try using this fork -- https://github.com/hmage/cpuminer-opt -- it should report SSE and AES properly. Let us know if it works, I'll offer the diff to joblo.
|
|
|
|
joblo (OP)
Legendary
Offline
Activity: 1470
Merit: 1114
|
 |
April 22, 2016, 04:56:04 PM Last edit: April 22, 2016, 05:12:05 PM by joblo |
|
algo/hodl/hodl-gate.c:132:7: error: ‘algo_gate_t’ has no member named ‘set_data_size’ gate->set_data_size = (void*)&hodl_set_data_size; ^ make[2]: *** [algo/hodl/cpuminer-hodl-gate.o] Fehler 1 make[2]: Verzeichnis »/home/kim/cpuminer-opt-3.1.17« wird verlassen make[1]: *** [all-recursive] Fehler 1 make[1]: Verzeichnis »/home/kim/cpuminer-opt-3.1.17« wird verlassen make: *** [all] Fehler 2 kim@spiel-2:~/cpuminer-opt-3.1.17$
That's a coding error. I just recompiled successfuly from the tarball I uploaded so don't know why you got that error. This line happens when you're compiling for core2. You probably removed set_data_size everywhere but on that line. I had to remove it in my fork. This is getting really weird. I reproduced the compile error with core2. It doesn't make any sense because set_data_size is not conditional on the arch. I need to really dig into this. Edit: I take that back, I found some weird code I need to sort out. Edit2: set_data_size is a coding error on my part. When hodl was introduced set_data_size became targetted. In 3.1.16 I eliminated the need for set_data_size but missed one reference in targetted code and failed to test that target. At this point with so many versions flying around I'm hesitant to rush out another one. Hmage seems to have posted a version with the set_data_size fix so I'll wait to see if that solves both problems before I build another release.
|
|
|
|
joblo (OP)
Legendary
Offline
Activity: 1470
Merit: 1114
|
 |
April 22, 2016, 05:36:54 PM |
|
@hmage
I see the difference in your logic in the sse2 check. I checked for an inferiour build missing sse2 (since removed in 3.1.17) and you check for a superiour build including aes_ni. In practical terms the superiour check seems more important as that scenario could occur if cpuminer-opt was cross compiled on a CPU with aes-ni. On the other hand cross compiling on a CPU without sse2 would have failed.
Removing the check for inferiour build, as done in 3.1.17 and your fork, is harmless and will become permanent. I will add a check for superiour build because that is a possible scenario. Both pending test results from users with real AMD CPUs with sse2 but without aes-ni.
|
|
|
|
th3.r00t
|
 |
April 22, 2016, 07:19:20 PM |
|
@hmage
I see the difference in your logic in the sse2 check. I checked for an inferiour build missing sse2 (since removed in 3.1.17) and you check for a superiour build including aes_ni. In practical terms the superiour check seems more important as that scenario could occur if cpuminer-opt was cross compiled on a CPU with aes-ni. On the other hand cross compiling on a CPU without sse2 would have failed.
Removing the check for inferiour build, as done in 3.1.17 and your fork, is harmless and will become permanent. I will add a check for superiour build because that is a possible scenario. Both pending test results from users with real AMD CPUs with sse2 but without aes-ni.
So if I read correctly - we should use github fork of hmage, which is with fixed CPU, OS and algo checks. If everything is OK on AMD CPUs it will become the main branch of cpuminer-opt?
|
|
|
|
joblo (OP)
Legendary
Offline
Activity: 1470
Merit: 1114
|
 |
April 22, 2016, 07:28:05 PM |
|
@hmage
I see the difference in your logic in the sse2 check. I checked for an inferiour build missing sse2 (since removed in 3.1.17) and you check for a superiour build including aes_ni. In practical terms the superiour check seems more important as that scenario could occur if cpuminer-opt was cross compiled on a CPU with aes-ni. On the other hand cross compiling on a CPU without sse2 would have failed.
Removing the check for inferiour build, as done in 3.1.17 and your fork, is harmless and will become permanent. I will add a check for superiour build because that is a possible scenario. Both pending test results from users with real AMD CPUs with sse2 but without aes-ni.
So if I read correctly - we should use github fork of hmage, which is with fixed CPU, OS and algo checks. If everything is OK on AMD CPUs it will become the main branch of cpuminer-opt? It's up to hmage whether he wants to maintain his fork. I intend to implement the fix on mine once it has been properly tested. Even with the fix I believe AMD will require a workaround for -march
|
|
|
|
th3.r00t
|
 |
April 22, 2016, 08:17:35 PM |
|
When using the git fork with this command line ./autogen.sh && CXXFLAGS="-std=gnu++11" ./configure CFLAGS="-O3 -march=btver1" --with-curl --with-crypto && make I got a successful compile and working binary on AMD ********** cpuminer-multi 1.2-dev *********** A CPU miner with multi algo support and optimized for CPUs with AES_NI extension. BTC donation address: 12tdvfF7KmAsihBXQXynT6E6th2c2pByTT Forked from TPruvot's cpuminer-multi with credits to Lucas Jones, elmad, palmd, djm34, pooler, ig0tik3d, Wolf0 and Jeff Garzik.
Checking CPU capatibility of AMD Phenom(tm) II X4 940 Processor... CPU arch supports AES_NI... NO. CPU arch supports SSE2..... YES. SW built with AES_NI....... NO. SW built with SSE2......... YES. Algo supports AES_NI....... YES. Starting mining without AES_NI optimisations...
|
|
|
|
joblo (OP)
Legendary
Offline
Activity: 1470
Merit: 1114
|
 |
April 22, 2016, 08:39:35 PM Last edit: April 22, 2016, 09:28:38 PM by joblo |
|
When using the git fork with this command line ./autogen.sh && CXXFLAGS="-std=gnu++11" ./configure CFLAGS="-O3 -march=btver1" --with-curl --with-crypto && make I got a successful compile and working binary on AMD ********** cpuminer-multi 1.2-dev *********** A CPU miner with multi algo support and optimized for CPUs with AES_NI extension. BTC donation address: 12tdvfF7KmAsihBXQXynT6E6th2c2pByTT Forked from TPruvot's cpuminer-multi with credits to Lucas Jones, elmad, palmd, djm34, pooler, ig0tik3d, Wolf0 and Jeff Garzik.
Checking CPU capatibility of AMD Phenom(tm) II X4 940 Processor... CPU arch supports AES_NI... NO. CPU arch supports SSE2..... YES. SW built with AES_NI....... NO. SW built with SSE2......... YES. Algo supports AES_NI....... YES. Starting mining without AES_NI optimisations... Thanks. I presume -march=native still fails. I will soon release 3.1.18 with the following: - Implement CPU check logic from hmage - remove check for inferior SW build on SSE2 CPU - add check for superior build - Suppress messages not used in decision making - document AMD workaround - fix set_data_size bug Edit: here is a draft of the test I intend to add to the build instructions: Some users with AMD CPUs without AES_NI have reported problems compiling with build.sh or "-march=native". Problems have included compile errors and poor performance. These users are recommended to compile manually specifying "-march=btver1" on the configure command line.
|
|
|
|
th3.r00t
|
 |
April 22, 2016, 08:52:15 PM |
|
Yep, -march=native still fails.
Also ./build.sh produces slightly slower binary for me on Intel and AMD (with changed -march=), maybe too many CFLAGS?
|
|
|
|
joblo (OP)
Legendary
Offline
Activity: 1470
Merit: 1114
|
 |
April 22, 2016, 09:20:09 PM |
|
Yep, -march=native still fails.
Also ./build.sh produces slightly slower binary for me on Intel and AMD (with changed -march=), maybe too many CFLAGS?
Thanks. -march=native chooses a lower arch than your CPU is capable of. You can confirm this with this command gcc -march=native -Q --help=target You will likely see amdfam10 but will get better performance with btver1.
|
|
|
|
joblo (OP)
Legendary
Offline
Activity: 1470
Merit: 1114
|
 |
April 23, 2016, 03:44:49 AM |
|
|
|
|
|
th3.r00t
|
 |
April 23, 2016, 08:06:32 PM |
|
Yep, -march=native still fails.
Also ./build.sh produces slightly slower binary for me on Intel and AMD (with changed -march=), maybe too many CFLAGS?
Thanks. -march=native chooses a lower arch than your CPU is capable of. You can confirm this with this command gcc -march=native -Q --help=target You will likely see amdfam10 but will get better performance with btver1. cpuminer-opt v3.1.18 on AMD Command: ./autogen.sh && ./configure CFLAGS="-O3 -march=amdfam10" --with-curl --with-crypto && make Result: Build Failed, gcc tried to compile AES-NI binary. Command: ./autogen.sh && ./configure CFLAGS="-O3 -march=btver1" --with-curl --with-crypto && make Result: Build Successful
|
|
|
|
Giulini
|
 |
April 23, 2016, 08:17:29 PM |
|
with -march=native o.k., with "btver1" negative
Checking CPU capatibility... AMD Sempron(tm) 145 Processor CPU arch supports AES_NI...NO. CPU arch supports SSE2.....YES. SW built for SSE2..........YES. Starting mining without AES_NI optimizations...
[2016-04-23 22:06:15] Starting Stratum on stratum+tcp://hodl.suprnova.cc:4693 [2016-04-23 22:06:15] 1 miner threads started, using 'hodl' algorithm. [2016-04-23 22:06:18] Stratum difficulty set to 1 [2016-04-23 22:07:07] hodl.suprnova.cc:4693 hodl block 43933 [2016-04-23 22:07:07] CPU #0: 373 H, 7.66 H/s [2016-04-23 22:08:17] CPU #0: 529 H, 7.63 H/s [2016-04-23 22:08:17] accepted: 1/1 (100%), 529 H, 7.63 H/s yes!
|
|
|
|
joblo (OP)
Legendary
Offline
Activity: 1470
Merit: 1114
|
 |
April 23, 2016, 09:06:49 PM Last edit: April 24, 2016, 01:49:37 AM by joblo |
|
with -march=native o.k., with "btver1" negative
Checking CPU capatibility... AMD Sempron(tm) 145 Processor CPU arch supports AES_NI...NO. CPU arch supports SSE2.....YES. SW built for SSE2..........YES. Starting mining without AES_NI optimizations...
[2016-04-23 22:06:15] Starting Stratum on stratum+tcp://hodl.suprnova.cc:4693 [2016-04-23 22:06:15] 1 miner threads started, using 'hodl' algorithm. [2016-04-23 22:06:18] Stratum difficulty set to 1 [2016-04-23 22:07:07] hodl.suprnova.cc:4693 hodl block 43933 [2016-04-23 22:07:07] CPU #0: 373 H, 7.66 H/s [2016-04-23 22:08:17] CPU #0: 529 H, 7.63 H/s [2016-04-23 22:08:17] accepted: 1/1 (100%), 529 H, 7.63 H/s yes!
Edit @Giulini: what do you get from "gcc -march=native -Q --help=target"? Edit2: I just realized there is a typo, it should be "-march=bdver1". Give it a try, it might be faster for some algos.This might be the best I can come up with. Now that you both have it figured out for your own situation are the tips in README.md clear enough for other users? I've added another phrase to the existing in italic. Some users with AMD CPUs without AES_NI have reported problems compiling with build.sh or "-march=native". Problems have included compile errors and poor performance. These users are recommended to compile manually specifying "-march= btver1bdver1" on the configure command line. If all else fails "-march=core2" will provide the best compatibility but the lowest performance".
|
|
|
|
joblo (OP)
Legendary
Offline
Activity: 1470
Merit: 1114
|
 |
April 23, 2016, 09:46:53 PM |
|
With things, hopefully, settling down after a period of rapid development and many algo additions algo-gate has evolved significantly and delelopped some mutations. Some functions developped multiple personalities or had names that made no sense which deviated from the highly structured goals of the gate system. Nevertheless it proved its worth when merging new algos by isolating all the algo specific code. It also showed promise in gating other functions such as jsonrpc2.
I'll start working on v3.2 and do some genetic engineering on algo-gate so it doesn't turn into a monster. I will also convert jsonrpc2 functions to use the gate and possibly others
In the meantime only bug fixes and high priority features will be added to the 3.1 stream.
|
|
|
|
th3.r00t
|
 |
April 23, 2016, 10:12:16 PM |
|
cpuminer-opt v3.1.18 is one of the best in recent version history. Talking both Intel and AMD wise. Thank you for your hard work! Can't wait to see what branch 3.2 will offer. 
|
|
|
|
joblo (OP)
Legendary
Offline
Activity: 1470
Merit: 1114
|
 |
April 23, 2016, 10:39:40 PM |
|
cpuminer-opt v3.1.18 is one of the best in recent version history. Talking both Intel and AMD wise. Thank you for your hard work! Can't wait to see what branch 3.2 will offer.  The algo-gate work won't be user visible but will make it easier to add new algos. Thanks for your support and invaluable help in getting to the bottom of the compile issue.
|
|
|
|
AlexGR
Legendary
Offline
Activity: 1708
Merit: 1049
|
 |
April 24, 2016, 02:13:41 AM |
|
3.1.18 is kind'a buggy in cpu detection. In a Q8200 it says: Checking CPU capatibility... Intel(R) Core(TM) i5 CPU 670 @ 3.47GHz CPU arch supports AES_NI...YES. SW built for AES_NI........NO. Algo supports AES_NI.......YES. CPU and algo support AES_NI, but SW build does not. Rebuild with "-march=native" for better performance. Starting mining without AES_NI optimizations... ...so naturally it thinks I have AES, even though the build is not for AES (-march=native / no AES in Intel quad core q8200). Anyway I did a profile run in x11 to check the slowdowns:  A couple of them, with echo512 being the biggest culprit, dominate the process in terms of time wasted. It seems there is a very optimized AES version for it: https://bench.cr.yp.to/impl-hash/echo512.html https://bench.cr.yp.to/impl-hash/echo512.htmlhttps://github.com/floodyberry/supercop/tree/master/crypto_hash/echo512/aes/aes64
|
|
|
|
joblo (OP)
Legendary
Offline
Activity: 1470
Merit: 1114
|
 |
April 24, 2016, 03:13:34 AM |
|
Images snipped. 3.1.18 is kind'a buggy in cpu detection. In a Q8200 it says: Checking CPU capatibility... Intel(R) Core(TM) i5 CPU 670 @ 3.47GHz CPU arch supports AES_NI...YES. SW built for AES_NI........NO. Algo supports AES_NI.......YES. CPU and algo support AES_NI, but SW build does not. Rebuild with "-march=native" for better performance. Starting mining without AES_NI optimizations... ...so naturally it thinks I have AES, even though the build is not for AES (-march=native / no AES in Intel quad core q8200). Anyway I did a profile run in x11 to check the slowdowns: A couple of them, with echo512 being the biggest culprit, dominate the process in terms of time wasted. It seems there is a very optimized AES version for it: https://bench.cr.yp.to/impl-hash/echo512.htmlhttps://bench.cr.yp.to/impl-hash/echo512.htmlhttps://github.com/floodyberry/supercop/tree/master/crypto_hash/echo512/aes/aes64Let me get this straight. You compiled with -march=native on a core2 that thinks it's a i5-670. The compile succeeded and the miner ran ok. That's pretty special. The CPU model and AES support comes directly from CPUID and has been reliable until now. Even the AMD guys haven't reported CPUID problems. Can you confirm CPUID is correct: cat /proc/cpuinfo |grep model It would be interesting to see what the compiler thought: gcc -march=native -Q --help=target | fgrep march Regarding echo512, yes you will get the slow version. Unfortunately I'm unaware of a SSE2 optimized version and the AES version is already used by cpuminer-opt on capable CPUs. I have yet to study your data in any detail but the following may put performance into perspective. Echo512 and groestl have AES optimizations for most algos. Cryptonight and hodl have their own unique AES optimizations. The rest of the x11 chain, including groestl but excluding echo, have SSE2 optimized versions. The algos in the longer X chains, as well as non-aes echo, are filled with slow SPH versions.
|
|
|
|
|