When the user said he's using the same binary as yours, he might said that by mistake -- you need to compare sha1sums of the binaries to make sure -- he might be running different .exe than yours.
I considered that but it's more than one user. And I downloaded the same binaries they did, and they stated they used the westmere build, filename cpuminer.exe. So it is a copy of the same file not a reproduction. Also that is the only build that does not support AES+AVX. Had the users tried the wrong file it would crash. Some did and reported same. I don't know precisely how it was compiled but assuming I'm not being similarly misinformed by several users my observations seem correct. The same file shows a different result for __AES__ on different CPU architectures. I don't know which is correct but they should be the same. Here is a bit of direct output of the variables used in the logic compiled for corei7 ********** cpuminer-opt 3.3.4 ***********
A CPU miner with multi algo support and optimized for CPUs
with AES_NI and AVX extensions.
BTC donation address: 12tdvfF7KmAsihBXQXynT6E6th2c2pByTT
Forked from TPruvot's cpuminer-multi with credits
to Lucas Jones, elmad, palmd, djm34, pooler, ig0tik3d,
Wolf0 and Jeff Garzik.
cpu has_aes 1
cpu has_avx 1
sw_has_aes 0
sw_has_sse2 1
Checking CPU capatibility...
Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz
CPU arch supports AES-AVX...YES.
SW built for AES-AVX........NO.
Algo supports AES-AVX.......YES.
Starting mining without AES-AVX optimizations...
I've removed the warning messages otherwise no change from 3.3.3. The result of the logic is not relevent in this example because the CPU has AES but the software check is what's important. In this case sw_has_aes taken directly from __AES__ is false. This seems reasonable for a westmere build and it agrees with my results from testing the CMB binary on the same CPU. So I think I've gone in a complete circle and back to the same point. The CMB westmere binary displays a different result for the __AES__ macro on different CPUs. Three users have reported he same problem with actual westmere class CPUs because, for them, __AES__ returned false. I am confident in the data reported by the users. I am less confident about the nature of the CMB build. I don't know if they built native on westmere HW or crosscompiled using -march=corei7. I could only verify indirectly by crosscompiling myself. But even ignoring how the bin was built __AES__ should still produce the same result on any HW. |
|
|
|
Thanks. You're the third person with Westmere with this problem. The main problem is "SW built for SSE2..........NO."
When I use the same file on my Skylake it answers YES, as it should. It isn't the way it was compiled because
I'm using the same file downloaed from CMB.
The software build test is supposed to reflect the compilation, not the HW support, however, the same SW reports
different results on different HW. I can only conclude the CPU does affect the SW build test. Without a relaible way
to test the SW build I have no choice but to disable the test.
if ( cpu_has_sse2 )
{
printf("%s\n", grn_yes );
printf(" SW built for SSE2..........");
if ( sw_has_aes )
{
printf("%s\n", ylw_no );
See anything out of place?  You say you're checking for SSE2, but in code you're checking for sw_has_aes. It's reverse logic. I'm actually checking to make sure it was not built with AES. An AES build is not compatible on a non-AES CPU. A non-AES build is assumed to be SSE2. But that logic had escaped me. In fact I never check sw_has_sse2 in the code. Given that new information I still struggle with the fact that the westmere build for Windows shows different results on my CPU (SSE2==YES), and real westmere/nehalem CPUs (SSE2==NO). Given the actual code is testing for AES the result (SSE2==NO) actually means (AES==YES), the build supports AES. Since an AES build won't work on a non-AES CPU the miner errors out with an incompatible build. The bottom line is the same build is being seen as AES on westmere but non AES on my skylake. The difference with checking AES is that is the compiler macro that is now showing conflicting results. __AES__ returns true on westmere and false on skylake. And that's where I'm stuck. Care to take another shot? Even when you miss the mark you make me think a little differently. |
|
|
|
Just tested v.3.3.3 on Xeon E5649 (Westmere-EP, AES without AVX) and got same error like on v3.3.2
********** cpuminer-opt 3.3.3 ***********
A CPU miner with multi algo support and optimized for CPUs
with AES_NI and AVX1 extensions.
BTC donation address: 12tdvfF7KmAsihBXQXynT6E6th2c2pByTT
Forked from TPruvot's cpuminer-multi with credits
to Lucas Jones, elmad, palmd, djm34, pooler, ig0tik3d,
Wolf0 and Jeff Garzik.
Checking CPU capatibility...
Intel(R) Xeon(R) CPU E5649 @ 2.53GHz
CPU arch supports AES-AVX1...NO.
CPU arch supports SSE2.....YES.
SW built for SSE2..........NO.
Incompatible SW build, rebuild with "-march=native"
Thanks. You're the third person with Westmere with this problem. The main problem is "SW built for SSE2..........NO." When I use the same file on my Skylake it answers YES, as it should. It isn't the way it was compiled because I'm using the same file downloaed from CMB. The software build test is supposed to reflect the compilation, not the HW support, however, the same SW reports different results on different HW. I can only conclude the CPU does affect the SW build test. Without a relaible way to test the SW build I have no choice but to disable the test. |
|
|
|
|
The plan for v3.3.4
All known issues will be resolved except 1.
hodl on Windows for non-AES CPUs is still not working and will remain disabled.
V3.3.3 seems to have finally solved all the low diff problems with hodl.
My attempt to help users by detecting when and under-oprtimized build is being used has
been a major flop. It's over. v3.3.4 will still display the detected CPU architecture and
expected performance but it will got give any advice or take preemptive action.
The capabilities check will be informative only, the miner will alway attempt to mine.
Also there are significant improvement to m7m algo thanks to Wolf0.
I intend to wait a while longer before releasing v3.3.4 in case there any more reports from
users about v3.3.3.
|
|
|
|
Cpuminer is working and Im planing to use it on some xeon servers.
On Haswell Xeon it works like charm but cpuminer-opt-3-3-2-windows is not working on Westmere Xeon, I've got error message asking me to recompile. Same error on some older Xeons, with only SSE2 support.
Can some good soul explain how to do that.
Which model CPU do you have and can you confirm it has both AES_NI and AVX. The rebuild message you saw was changed in v3.3.2 to also check for AVX. This means the miner thinks the CPU has AVX as well as AES_NI, need to confirm that. If the miner is wrong it's a bug I need to investigate. If your CPU does indeed have AES_NI and AVX then the message is correct and the Westmere build was not compiled with AES-AVX1 support. In the meantime you can ignore the message and keep mining. Edit: The best thing to do, IMO, is to install mingw on your Westmere and compile for youself for the native architecture. This should ensure you get an optimum build regardless if your CPU has AVX1 or not, assuming the capabilities check is working properly. If not it will help define the problem better. As I said I don't have the proper CPU to test with so I'm dependent on users like you to confirm the AVX1 check is working properly. I appreciate your time and efforts. I recommend this to any users with CPUs in the grey zone between core2 and Sandybridge. Hello Joblo thx for help. I will try to describe all scenarios I have, feel free to ask anything. First server has 2 Xeon E5540 (Gainestown), only SSE instructions (no AES, no AVX). Error that it throws: Checking CPU capatibility... Intel(R) Xeon(R) CPU E5540 @ 2.53GHz CPU arch supports AES-AVX1...NO. CPU arch supports SSE2.....YES. SW built for SSE2..........NO. Incompatible SW build, rebuild with "-march=native" Second server has 2 Xeon E5649 (Westmere-EP), SSE, AES (no AVX). Error that it throws: Checking CPU capatibility... Intel(R) Xeon(R) CPU E5649 @ 2.53GHz CPU arch supports AES-AVX1...NO. CPU arch supports SSE2.....YES. SW built for SSE2..........NO. Incompatible SW build, rebuild with "-march=native" Third server has 2 Xeon E5-2687W v3 (Haswell E/EP), SSE, AES, AVX2, and it works like charm throwing around 2 MH/s. On this one I'm using cpuminer-haswell-broadwell.exe (For some reason I have to start two separate miners for both processors to fire) Checking CPU capatibility... Intel(R) Xeon(R) CPU E5-2687W v3 @ 3.10GHz CPU arch supports AES-AVX1...YES. SW built for AES-AVX1........YES. Algo supports AES-AVX1.......YES. Start mining with AES-AVX1 optimizations... [2016-06-01 23:44:35] 20 miner threads started, using 'x11evo' algorithm. I have also tested miner on my gaming rig with i5-2500K where it also works throwing around 200KH/s, using cpuminer-sandybridge-ivybridge.exe On servers where miner fails I have tried all executables but with same error. Can you pls point me to some tutorial about using mingw and link to source code? Thank you very much for the detailed data. It is very helpful. The main problem is this line: " SW built for SSE2..........NO.". This is supposed to indicate how it was compiled. I tried the precompiled Westmere binary on my Skylake and it said it was compiled for SSE2. Another user reported the same problem with a Nehalem CPU. I encourage you to try to use mingw. There are instructions in the README.md file. If you have problem you can always post questions in the cpuminer-opt thread. I'm fed up with those cpability checks. They were supposed to help users but have had the opposite effect. It will be removed from v3.3.4. I'll wait a litle longer to ensure there are no other issues with v3.3.3 |
|
|
|
thanks, working perfectly fine now  Glad to hear it. Thanks for testing. |
|
|
|
I tried this out and I get error: Checking CPU capatibility... Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz CPU arch supports AES-AVX1...NO. CPU arch supports SSE2.....YES. SW built for SSE2..........NO. Incompatible SW build, rebuild with "-march=native"Which exe did you use? Which algo? Edit: I tried ccminer.exe on a skylake which has AES and AVX and the capabilities check said the SW was built for AES. If you used the same file and it said it was not built for SSE2 then there is an inconsistency. The same file was built for AES but not SSE2. When I compile with -march=native both are true. It's becoming more clear that this check should be removed. I used cpuminer.exe (I have Nehalem architecture), on cryptonight and on nist5. Same error on both. OK, Thanks. I can't explain why it displays "SW built for SSE2..........NO." on your CPU and YES on mine. This is supposed to indicate how it was compiled. I've had enough trying to get it to work. I'm removing those checks in the next release. |
|
|
|
I tried this out and I get error: Checking CPU capatibility... Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz CPU arch supports AES-AVX1...NO. CPU arch supports SSE2.....YES. SW built for SSE2..........NO. Incompatible SW build, rebuild with "-march=native"Which exe did you use? Which algo? Edit: I tried ccminer.exe on a skylake which has AES and AVX and the capabilities check said the SW was built for AES. If you used the same file and it said it was not built for SSE2 then there is an inconsistency. The same file was built for AES but not SSE2. When I compile with -march=native both are true. It's becoming more clear that this check should be removed. |
|
|
|
|
Second issue is hodl on CPUs without AES on Windows. It still doesn't compile.
It works fine on Linux.
I am not testing this on a real core2, just forcing a core2 compile on an i7. I don't
think this is a factor.
The problem is endian.h. It's missing when compiling on Windows and is needed by two
files used for non-AES hodl. I can't find any endian.h in any of the mingw libraries. I have
searched the net and didn't find any endian library for mingw.
I tried a few different variations of endian.h such as sys/endian.h and a file I found on the
net called portable_endian.h. Some didn't exist, some didn't have the necessary definitions
and some compiled but produced only rejects, none of them worked.
The endianness on Windows might be a factor.
I need to find an endian library for mingw that Works with Windows little endian. Any suggestions?
|
|
|
|
There are a couple of issues that I would like some advice on from the community. I will make seperate posts so the discussion doesn't get garbled. The first issue is compiling an optimum build for early CPUs with AES support. These are Intel CPUs between core2 and Sandybridge including Westmere as well as similar AMD CPUs. This is primarilly intended for users mining with such CPUs but may also be of interest to others. I recently discovered the AES optimizations I have been using extensively use AVX1 instsructions as well as AES. AES was implemented on CPUs before AVX1. The CPU capabilities check in cpuminer-opt prior to v3.3.2 did not make this disctinction. It assumed if the CPU has AES it could use the AES code in cpuminer-opt. This results in users with these CPUs, particularly user using the CMB binaries, seeing the following message on miner startup: CPU and algo support AES-AVX1, but SW build does not.
Rebuild with "-march=native" for better performance.
One user has reported that the Westmere precompiled binary appears not to be compiled for AES and AVX1. It isn't clear why. That's where i need your help if you have one of these CPU. Linux, Windows, self compiled or prebuilt binaries, it doesn't matter. If you have a CPU in the Westmere family between core2 and Sandybridge you can help a lot. Please post: - your CPU model and confirmation whether it has AES and or AVX (may also be called AVX1) if possible. - whether and how you compiled or if you used a binary which one - the miner output at startup showing the CPU capabilities check - whether it mined successfully - any other info you think is useful Thank you in advance. Edit: Another option is to abandon the rebuild check. It is intended to catch precicely the type of situation that may currently exist with Westmere CPUs possibly getting lower than optimum performance. However, it seems to be causing more grief than it's worth, partly because it was not correctly implemented. Users thought the rebuild message was an error and meant the miner wasn't working. Even if the rebuild check is working properly an attempt to use the existing Westmere Windows build will result in complaints likely coming to me when they should be directed at the builder. I'll wait to see the reaction is to v3.3.3, if there are still problems with the rebuild check, it's gone. |
|
|
|
Cpuminer is working and Im planing to use it on some xeon servers.
On Haswell Xeon it works like charm but cpuminer-opt-3-3-2-windows is not working on Westmere Xeon, I've got error message asking me to recompile. Same error on some older Xeons, with only SSE2 support.
Can some good soul explain how to do that.
Which model CPU do you have and can you confirm it has both AES_NI and AVX. The rebuild message you saw was changed in v3.3.2 to also check for AVX. This means the miner thinks the CPU has AVX as well as AES_NI, need to confirm that. If the miner is wrong it's a bug I need to investigate. If your CPU does indeed have AES_NI and AVX then the message is correct and the Westmere build was not compiled with AES-AVX1 support. In the meantime you can ignore the message and keep mining. Edit: The best thing to do, IMO, is to install mingw on your Westmere and compile for youself for the native architecture. This should ensure you get an optimum build regardless if your CPU has AVX1 or not, assuming the capabilities check is working properly. If not it will help define the problem better. As I said I don't have the proper CPU to test with so I'm dependent on users like you to confirm the AVX1 check is working properly. I appreciate your time and efforts. I recommend this to any users with CPUs in the grey zone between core2 and Sandybridge. |
|
|
|
i got low diff shares only on this machine, so most likely number 2. tried your suggestion to restart the miner (even multiple times), still the same result sadly - mining starts with diff 8 (as on every start) - factor reduced to 0.XX after the first reject - more rejected shares with low diff (to be precise diff is always 0.0019530951976776123) see http://ctrlv.in/764598 for screen br I just released v3.3.3 with a fix for this, assuming you still got 2 GB RAM usage. https://drive.google.com/file/d/0B0lVSGQYLJIZQ1puZWRjWUdIRVE/view?usp=sharingGive it a try and let me know if you still see the problem. |
|
|
|
fyi: latest cpuminer-opt-3-3-2-windows binaries from cryptomining-blog sadly still generate low diff refused shares (tested with cpuminer-amd.exe and cpuminer.exe and an A6-6400K, 0/5 accepted till i quit and started using wolfs miner again)
Could you be more specific? There are two seperate issues one I'm still working on. 1. occasional low diff shares mixed with mostly valid shares always, should be fixed in 3.3.2. 2. All low diff shares from startup occasionally, still working on fix, stop and restart miner should work. |
|
|
|
Curious. How often does it occur? I have seen this very infrequently but have seen it before. Restarting the miner
usually fixes it. If it's happening more often that's a new issue.
Nice observation about the 2 GB RAM usage. HOdl uses a 1 GB buffer so somehow there were 2 allocated.
You didn't have two instances running at the same time did you? That wouldn't explain th erejects but would
explain the RAM usage.
My opinion is it occurs rather infrequent but annoying enough. No I didn't run 2 instances of the miner. That's great info. If one miner instance can allocate two buffers it's a mutex issue and I know exactly where. I've got a better solution. Take the buffer alloc out of the threads (don't know why it was there in the first place) and move it to main. |
|
|
|
@joblo Think I have found a bug. Using the windows build from cryptomining-blog, get rejects: share above target and noticed the memory usage spikes to 2GB of ram. When it works normally only 1GB ram is used. The problem occurs occasionally at miner launch, restarting can get it to work.  Curious. How often does it occur? I have seen this very infrequently but have seen it before. Restarting the miner usually fixes it. If it's happening more often that's a new issue. Nice observation about the 2 GB RAM usage. HOdl uses a 1 GB buffer so somehow there were 2 allocated. You didn't have two instances running at the same time did you? That wouldn't explain th erejects but would explain the RAM usage. |
|
|
|
So, in this particular case, moving the assignments to the left will solve the issue: CFLAGS="-O3 -march=native -Wall" CXXFLAGS="$CFLAGS -std=gnu++11" ./configure --with-curl Thanks I'll make the change. |
|
|
|
Thanks, you may have noticed that I implemented unordered_map as you suggested a while back.
Yeap, I just needed to be patient  There is an (albeit small) issue with C++ files not being optimized: ./configure --with-curl CFLAGS="-O3 -march=native -Wall" CXXFLAGS="$CFLAGS -std=gnu++11" Unless you previously set CFLAGS bash variable, $CFLAGS becomes nothing, so you're effectively calling this: ./configure --with-curl CFLAGS="-O3 -march=native -Wall" CXXFLAGS=" -std=gnu++11" As a result, c++ files are compiled without any optimizations: hmage@dhmd:~/src/cpuminer-opt$ ./build.sh 2>&1 | fgrep g++
checking for g++... g++
checking whether g++ accepts -g... yes
checking dependency style of g++... gcc3
g++ -DHAVE_CONFIG_H -I. -Iyes/include -fno-strict-aliasing -I./compat/jansson -I. -Iyes/include -std=gnu++11 -MT crypto/cpuminer-magimath.o -MD -MP -MF crypto/.deps/cpuminer-magimath.Tpo -c -o crypto/cpuminer-magimath.o `test -f 'crypto/magimath.cpp' || echo './'`crypto/magimath.cpp
g++ -DHAVE_CONFIG_H -I. -Iyes/include -fno-strict-aliasing -I./compat/jansson -I. -Iyes/include -std=gnu++11 -MT algo/hodl/cpuminer-hodl.o -MD -MP -MF algo/hodl/.deps/cpuminer-hodl.Tpo -c -o algo/hodl/cpuminer-hodl.o `test -f 'algo/hodl/hodl.cpp' || echo './'`algo/hodl/hodl.cpp
g++ -DHAVE_CONFIG_H -I. -Iyes/include -fno-strict-aliasing -I./compat/jansson -I. -Iyes/include -std=gnu++11 -MT algo/hodl/cpuminer-hodl_arith_uint256.o -MD -MP -MF algo/hodl/.deps/cpuminer-hodl_arith_uint256.Tpo -c -o algo/hodl/cpuminer-hodl_arith_uint256.o `test -f 'algo/hodl/hodl_arith_uint256.cpp' || echo './'`algo/hodl/hodl_arith_uint256.cpp
g++ -DHAVE_CONFIG_H -I. -Iyes/include -fno-strict-aliasing -I./compat/jansson -I. -Iyes/include -std=gnu++11 -MT algo/hodl/cpuminer-hodl_uint256.o -MD -MP -MF algo/hodl/.deps/cpuminer-hodl_uint256.Tpo -c -o algo/hodl/cpuminer-hodl_uint256.o `test -f 'algo/hodl/hodl_uint256.cpp' || echo './'`algo/hodl/hodl_uint256.cpp
g++ -DHAVE_CONFIG_H -I. -Iyes/include -fno-strict-aliasing -I./compat/jansson -I. -Iyes/include -std=gnu++11 -MT algo/hodl/cpuminer-hash.o -MD -MP -MF algo/hodl/.deps/cpuminer-hash.Tpo -c -o algo/hodl/cpuminer-hash.o `test -f 'algo/hodl/hash.cpp' || echo './'`algo/hodl/hash.cpp
g++ -DHAVE_CONFIG_H -I. -Iyes/include -fno-strict-aliasing -I./compat/jansson -I. -Iyes/include -std=gnu++11 -MT algo/hodl/cpuminer-hmac_sha512.o -MD -MP -MF algo/hodl/.deps/cpuminer-hmac_sha512.Tpo -c -o algo/hodl/cpuminer-hmac_sha512.o `test -f 'algo/hodl/hmac_sha512.cpp' || echo './'`algo/hodl/hmac_sha512.cpp
g++ -DHAVE_CONFIG_H -I. -Iyes/include -fno-strict-aliasing -I./compat/jansson -I. -Iyes/include -std=gnu++11 -MT algo/hodl/cpuminer-sha256.o -MD -MP -MF algo/hodl/.deps/cpuminer-sha256.Tpo -c -o algo/hodl/cpuminer-sha256.o `test -f 'algo/hodl/sha256.cpp' || echo './'`algo/hodl/sha256.cpp
g++ -DHAVE_CONFIG_H -I. -Iyes/include -fno-strict-aliasing -I./compat/jansson -I. -Iyes/include -std=gnu++11 -MT algo/hodl/cpuminer-sha512.o -MD -MP -MF algo/hodl/.deps/cpuminer-sha512.Tpo -c -o algo/hodl/cpuminer-sha512.o `test -f 'algo/hodl/sha512.cpp' || echo './'`algo/hodl/sha512.cpp
g++ -DHAVE_CONFIG_H -I. -Iyes/include -fno-strict-aliasing -I./compat/jansson -I. -Iyes/include -std=gnu++11 -MT algo/hodl/cpuminer-utilstrencodings.o -MD -MP -MF algo/hodl/.deps/cpuminer-utilstrencodings.Tpo -c -o algo/hodl/cpuminer-utilstrencodings.o `test -f 'algo/hodl/utilstrencodings.cpp' || echo './'`algo/hodl/utilstrencodings.cpp
g++ -DHAVE_CONFIG_H -I. -Iyes/include -fno-strict-aliasing -I./compat/jansson -I. -Iyes/include -std=gnu++11 -MT cpuminer-uint256.o -MD -MP -MF .deps/cpuminer-uint256.Tpo -c -o cpuminer-uint256.o `test -f 'uint256.cpp' || echo './'`uint256.cpp
To fix this, you need to either duplicate contents of CFLAGS manually in CXXFLAGS, or use shell variables (the way I do it in my fork): CFLAGS="-O3 -march=native -Wall" CXXFLAGS="$CFLAGS -std=gnu++11" ./configure --with-curl bash will expand $CFLAGS into contents of the variable it was set in same line previously, and then configure will pick up shell variables CFLAGS and CXXFLAGS and use them accordingly. The problem is marginal, because it affects only hodl and magimath, and hodl is hand-optimized into AES anyway. I did not benchmark the difference in performance yet, but plan to -- when back from work. Thanks. I can't take credit for the HOdl optimizations, that's Wolf0's work, but yes it is hand optimized. Regarding the propagation of CFLAGS I assumed that the first part (assigning CFLAGS) would run before assigning CXXFLAGS. Is that incorrect? Do I need to set CFLAGS outside of the configure command line? |
|
|
|
See my previous post, fix the minimum limit. Edit: it seems I was mistaken. The limit is actually 1 kH, not 1 GH, much better. |
|
|
|
|
Show Posts
