[LOCKED] cpuminer-opt v3.12.3, open source optimized multi-algo CPU miner

[joblo]

CRYPTOMINING-BLOG ARTICLE--

Congratulations!  Your software was featured on the blog!  I tried it, and found that it does at least 100% better than CPU-Multi on Yescript!  I do see that Yescript software algo is not yet optimized.  I hope that you will find the time to do that bit of code soon.

Running on i7 2600 here, I get about 2kh/s for Yescript.  With CPU-Multi, it was closer to 800H/s.

Thank you!       --scryptr

Unfortunately I can't optimize scrypt algos, only the chained algos like x11, quark etc. You get 2x multi, not sure why,
I just took whatever was there and it worked. Never compared performance.

Pretty cool to be featured on CMB, they already updated to v3.3.1.

[joblo]

I think the low difficulty rejects in hodl were left over from the mess in v3.2.  v3.1.18 seems to be clean.
If anyone has evidence to the contrary please let me know.

[Epsylon3]

For the version, you need to update that :

configure.ac:1:AC_INIT([cpuminer-multi], [1.2-dev])

sample from ccminer :

AC_INIT([ccminer], [1.8-dev], [], [ccminer], [http://github.com/tpruvot/ccminer])


then autoreconf && make

./cpuminer --version

         **********  cpuminer-opt 3.3.1  ***********

Code:

-AC_INIT([cpuminer-multi], [1.2-dev])
+AC_INIT([cpuminer-opt], [3.3.1])

[ryen123]

It looks like you are trying to solo mine the wallet. I have not been able to test this and didn't know if it worked.
Thanks for being my tester. I'll look into it to see what I can do.

Edit: can you provide the output without -q?

Code:

         **********  cpuminer-opt 3.3  ***********
     A CPU miner with multi algo support and optimized for CPUs
     with AES_NI extension.
     BTC donation address: 12tdvfF7KmAsihBXQXynT6E6th2c2pByTT
     Forked from TPruvot's cpuminer-multi with credits
     to Lucas Jones, elmad, palmd, djm34, pooler, ig0tik3d,
     Wolf0 and Jeff Garzik.

Checking CPU capatibility...
               Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz
   CPU arch supports AES_NI...YES.
   SW built for AES_NI........YES.
   Algo supports AES_NI.......YES.
Start mining with AES_NI optimizations...

[2016-06-01 13:25:04] Binding thread 0 to cpu 0 (mask 1)
[2016-06-01 13:25:04] Binding thread 1 to cpu 1 (mask 2)
[2016-06-01 13:25:04] Binding thread 2 to cpu 2 (mask 4)
[2016-06-01 13:25:04] 4 miner threads started, using 'hodl' algorithm.
[2016-06-01 13:25:04] Binding thread 3 to cpu 3 (mask 8)
[2016-06-01 13:25:05] Current block is 64418
[2016-06-01 13:25:05] Unrecognized block version: 4
[2016-06-01 13:25:05] Long-polling on http://192.168.1.8:11989
[2016-06-01 13:25:06] json_rpc_call failed, retry after 10 seconds
[2016-06-01 13:25:07] JSON-RPC call failed: Method not found
[2016-06-01 13:25:17] Current block is 64418
[2016-06-01 13:25:17] Unrecognized block version: 4
[2016-06-01 13:25:18] json_rpc_call failed, retry after 10 seconds
[2016-06-01 13:25:29] Current block is 64418
[2016-06-01 13:25:29] Unrecognized block version: 4
[2016-06-01 13:25:30] json_rpc_call failed, retry after 10 seconds

The output with Debug option without -q.

Also regarding the low difficulty share issue, it also occurred to me before. i5-2500K on ubuntu 14.04 x64.

Edit: Output with protocol dump (-P)

Code:

[2016-06-01 13:44:32] JSON protocol request:
{"method": "getblocktemplate", "params": [{"capabilities": ["coinbasetxn", "coinbasevalue", "longpoll", "workid"]}], "id":0}


* Rebuilt URL to: http://192.168.1.8:11989/
* Found bundle for host 192.168.1.8: 0x7fead8013d00
* Re-using existing connection! (#0) with host 192.168.1.8
* Connected to 192.168.1.8 (192.168.1.8) port 11989 (#0)
* Server auth using Basic with user 'hodl'
> POST / HTTP/1.1
Authorization: Basic Zm9ydHJlc3Nob2RsOjdBMjV2TjlGbGljSnhwcVk=
Host: 192.168.1.8:11989
Accept: */*
Accept-Encoding: deflate, gzip
Content-Type: application/json
Content-Length: 126
User-Agent: cpuminer-opt/3.3
X-Mining-Extensions: longpoll reject-reason
Expect: 100-continue

* Done waiting for 100-continue
< HTTP/1.1 200 OK
< Date: Wed, 01 Jun 2016 05:44:36 +0000
< Connection: keep-alive
< Content-Length: 583
< Content-Type: application/json
* Server bitcoin-json-rpc/v1.0.0 is not blacklisted
< Server: bitcoin-json-rpc/v1.0.0
<
* Connection #0 to host 192.168.1.8 left intact
[2016-06-01 13:44:33] JSON protocol response:
{
   "error": null,
   "result": {
      "transactions": [],
      "sizelimit": 1000000,
      "previousblockhash": "00000018958d65d48ce5f3ef329ab9b8a49eb9e6ad8c56c67cd4518b992841f5",
      "mutable": [
         "time",
         "transactions",
         "prevblock"
      ],
      "longpollid": "00000018958d65d48ce5f3ef329ab9b8a49eb9e6ad8c56c67cd4518b992841f5331",
      "version": 4,
      "mintime": 1464759040,
      "height": 64428,
      "capabilities": [
         "proposal"
      ],
      "coinbaseaux": {
         "flags": ""
      },
      "sigoplimit": 20000,
      "coinbasevalue": 5000000000,
      "target": "0000004556e10000000000000000000000000000000000000000000000000000",
      "curtime": 1464759870,
      "noncerange": "00000000ffffffff",
      "bits": "1d4556e1"
   },
   "id": 0
}
[2016-06-01 13:44:33] Current block is 64428
[2016-06-01 13:44:33] Unrecognized block version: 4
[2016-06-01 13:44:33] JSON protocol request:
{"method": "getmininginfo", "params": [], "id":8}


* Rebuilt URL to: http://192.168.1.8:11989/
* Found bundle for host 192.168.1.8: 0x7fead8013d00
* Re-using existing connection! (#0) with host 192.168.1.8
* Connected to 192.168.1.8 (192.168.1.8) port 11989 (#0)
* Server auth using Basic with user 'hodl'
> POST / HTTP/1.1
Authorization: Basic Zm9ydHJlc3Nob2RsOjdBMjV2TjlGbGljSnhwcVk=
Host: 192.168.1.8:11989
Accept: */*
Accept-Encoding: deflate, gzip
Content-Type: application/json
Content-Length: 51
User-Agent: cpuminer-opt/3.3
X-Mining-Extensions: longpoll reject-reason
Expect: 100-continue

* Done waiting for 100-continue
< HTTP/1.1 200 OK
< Date: Wed, 01 Jun 2016 05:44:37 +0000
< Connection: keep-alive
< Content-Length: 246
< Content-Type: application/json
* Server bitcoin-json-rpc/v1.0.0 is not blacklisted
< Server: bitcoin-json-rpc/v1.0.0
<
* Connection #0 to host 192.168.1.8 left intact
[2016-06-01 13:44:34] JSON protocol response:
{
   "error": null,
   "result": {
      "difficulty": 0.01356886,
      "errors": "",
      "currentblocksize": 1000,
      "currentblocktx": 0,
      "pooledtx": 0,
      "blocks": 64427,
      "genproclimit": -1,
      "networkhashps": 402233,
      "generate": false,
      "testnet": false,
      "chain": "main",
      "hashespersec": 0
   },
   "id": 8
}
[2016-06-01 13:44:34] json_rpc_call failed, retry after 10 seconds

[joblo]

It looks like you are trying to solo mine the wallet. I have not been able to test this and didn't know if it worked.
Thanks for being my tester. I'll look into it to see what I can do.

Edit: can you provide the output without -q?

Code:

         **********  cpuminer-opt 3.3  ***********
     A CPU miner with multi algo support and optimized for CPUs
     with AES_NI extension.
     BTC donation address: 12tdvfF7KmAsihBXQXynT6E6th2c2pByTT
     Forked from TPruvot's cpuminer-multi with credits
     to Lucas Jones, elmad, palmd, djm34, pooler, ig0tik3d,
     Wolf0 and Jeff Garzik.

Checking CPU capatibility...
               Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz
   CPU arch supports AES_NI...YES.
   SW built for AES_NI........YES.
   Algo supports AES_NI.......YES.
Start mining with AES_NI optimizations...

[2016-06-01 13:25:04] Binding thread 0 to cpu 0 (mask 1)
[2016-06-01 13:25:04] Binding thread 1 to cpu 1 (mask 2)
[2016-06-01 13:25:04] Binding thread 2 to cpu 2 (mask 4)
[2016-06-01 13:25:04] 4 miner threads started, using 'hodl' algorithm.
[2016-06-01 13:25:04] Binding thread 3 to cpu 3 (mask 8)
[2016-06-01 13:25:05] Current block is 64418
[2016-06-01 13:25:05] Unrecognized block version: 4
[2016-06-01 13:25:05] Long-polling on http://192.168.1.8:11989
[2016-06-01 13:25:06] json_rpc_call failed, retry after 10 seconds
[2016-06-01 13:25:07] JSON-RPC call failed: Method not found
[2016-06-01 13:25:17] Current block is 64418
[2016-06-01 13:25:17] Unrecognized block version: 4
[2016-06-01 13:25:18] json_rpc_call failed, retry after 10 seconds
[2016-06-01 13:25:29] Current block is 64418
[2016-06-01 13:25:29] Unrecognized block version: 4
[2016-06-01 13:25:30] json_rpc_call failed, retry after 10 seconds

The output with Debug option without -q.

Also regarding the low difficulty share issue, it also occurred to me before. i5-2500K on ubuntu 14.04 x64.

Edit: Output with protocol dump (-P)

Code:

[2016-06-01 13:44:32] JSON protocol request:
{"method": "getblocktemplate", "params": [{"capabilities": ["coinbasetxn", "coinbasevalue", "longpoll", "workid"]}], "id":0}


* Rebuilt URL to: http://192.168.1.8:11989/
* Found bundle for host 192.168.1.8: 0x7fead8013d00
* Re-using existing connection! (#0) with host 192.168.1.8
* Connected to 192.168.1.8 (192.168.1.8) port 11989 (#0)
* Server auth using Basic with user 'hodl'
> POST / HTTP/1.1
Authorization: Basic Zm9ydHJlc3Nob2RsOjdBMjV2TjlGbGljSnhwcVk=
Host: 192.168.1.8:11989
Accept: */*
Accept-Encoding: deflate, gzip
Content-Type: application/json
Content-Length: 126
User-Agent: cpuminer-opt/3.3
X-Mining-Extensions: longpoll reject-reason
Expect: 100-continue

* Done waiting for 100-continue
< HTTP/1.1 200 OK
< Date: Wed, 01 Jun 2016 05:44:36 +0000
< Connection: keep-alive
< Content-Length: 583
< Content-Type: application/json
* Server bitcoin-json-rpc/v1.0.0 is not blacklisted
< Server: bitcoin-json-rpc/v1.0.0
<
* Connection #0 to host 192.168.1.8 left intact
[2016-06-01 13:44:33] JSON protocol response:
{
   "error": null,
   "result": {
      "transactions": [],
      "sizelimit": 1000000,
      "previousblockhash": "00000018958d65d48ce5f3ef329ab9b8a49eb9e6ad8c56c67cd4518b992841f5",
      "mutable": [
         "time",
         "transactions",
         "prevblock"
      ],
      "longpollid": "00000018958d65d48ce5f3ef329ab9b8a49eb9e6ad8c56c67cd4518b992841f5331",
      "version": 4,
      "mintime": 1464759040,
      "height": 64428,
      "capabilities": [
         "proposal"
      ],
      "coinbaseaux": {
         "flags": ""
      },
      "sigoplimit": 20000,
      "coinbasevalue": 5000000000,
      "target": "0000004556e10000000000000000000000000000000000000000000000000000",
      "curtime": 1464759870,
      "noncerange": "00000000ffffffff",
      "bits": "1d4556e1"
   },
   "id": 0
}
[2016-06-01 13:44:33] Current block is 64428
[2016-06-01 13:44:33] Unrecognized block version: 4
[2016-06-01 13:44:33] JSON protocol request:
{"method": "getmininginfo", "params": [], "id":8}


* Rebuilt URL to: http://192.168.1.8:11989/
* Found bundle for host 192.168.1.8: 0x7fead8013d00
* Re-using existing connection! (#0) with host 192.168.1.8
* Connected to 192.168.1.8 (192.168.1.8) port 11989 (#0)
* Server auth using Basic with user 'hodl'
> POST / HTTP/1.1
Authorization: Basic Zm9ydHJlc3Nob2RsOjdBMjV2TjlGbGljSnhwcVk=
Host: 192.168.1.8:11989
Accept: */*
Accept-Encoding: deflate, gzip
Content-Type: application/json
Content-Length: 51
User-Agent: cpuminer-opt/3.3
X-Mining-Extensions: longpoll reject-reason
Expect: 100-continue

* Done waiting for 100-continue
< HTTP/1.1 200 OK
< Date: Wed, 01 Jun 2016 05:44:37 +0000
< Connection: keep-alive
< Content-Length: 246
< Content-Type: application/json
* Server bitcoin-json-rpc/v1.0.0 is not blacklisted
< Server: bitcoin-json-rpc/v1.0.0
<
* Connection #0 to host 192.168.1.8 left intact
[2016-06-01 13:44:34] JSON protocol response:
{
   "error": null,
   "result": {
      "difficulty": 0.01356886,
      "errors": "",
      "currentblocksize": 1000,
      "currentblocktx": 0,
      "pooledtx": 0,
      "blocks": 64427,
      "genproclimit": -1,
      "networkhashps": 402233,
      "generate": false,
      "testnet": false,
      "chain": "main",
      "hashespersec": 0
   },
   "id": 8
}
[2016-06-01 13:44:34] json_rpc_call failed, retry after 10 seconds

I think this is your problem.

[2016-06-01 13:25:05] Unrecognized block version: 4

I think there is a new block version for hodl and believe there is a bounty for someone to write the code to support it.

Until then you'll have to mine in a pool.

[ryen123]

I think this is your problem.

[2016-06-01 13:25:05] Unrecognized block version: 4

I think there is a new block version for hodl and believe there is a bounty for someone to write the code to support it.

Until then you'll have to mine in a pool.

Ok thanks.

[Epsylon3]

the block version is only for the gbt mode, which is only compatible with bitcoin as far as i know... (remains from original cpuminer)

gbt rules are too much variable per coin to handle them in a multi purpose miner

[joblo]

I have an update on a couple of issues that have existed for a while but are now better understood.

1. The definition of AES

The miner's AES optimized code also includes AVX instructions that are not included in the first generation
of AES CPUs, however the miner's CPU capabilities check uses the pure drefinition of AES (without AVX).
This results in the following message being displayed on startup on CPUs with AES but not AVX:

Code:

Rebuild with "-march=native" for better performance.

This is not an error, the miner will work, but not at AES performance levels. This message should be ignored
for Nehalem series CPUs and similar AMD. These CPUs can't mine using the miner's AES & AVX code.

I other words the miner will mine at the best rate possible for that CPU in spite of the suggestion otherwise.

The minimum requirement for AES mining will be changed to Sandybridge to conform with the actual limitations
of the AES mining code and the capabilities check will be changed to also check for AVX support in the CPU.

Update: I have changed the terminology to be more precise. It will now be called AES-AVX1 to more clearly
exclude CPUs that have AES but not AVX1. Also the rebuild warning will not be displayed for those CPUs.
Documentation will also be updated.

2. Low difficulty shares mining hodl

Since v3.2 hodl mining has started producing low difficulty shares at a rate of 1 or 2%. Prior to that there were
virtually no rejects. I may have found the problem and am testing a fix. At this point the first 170 shares have
been valid.

Update: I am optimistic the fix works. The first test run produced no low diff shares in 268 accepts

3. Hodl support on Windows for non AES-AVX1 CPUs. This is a more challenging issue and will not be fixed
in the next release.

I will build a new release when these 2 issues are solved.

In the meantime users should ignore the warning to rebuild for faster performance and hodl users with Linux
can use v3.1.18 if the rejects are unacceptible. Hodl miners n Windows will have to live with the rejects until
the fix is released.

Note to Cryptimining Blog: You might want to drop the Westmere build for the next release as it seems to have
no practical use. A core2 build would expand the range of supported CPUs and would also work on Westmere.

Update:

It appears both issues are fixed. The fix to hodl was more of a trial-and-error proceess so I want to test longer
to raise the confidence level. I don't know why it broke and I don't know why the fix works.

The AES issue, on the other hand, is now well understood so I am confident in the fix although I don't have the
resources to test it properly.

An interesting contrast in processes.

V3.3.2 coming in a matter of hours.

[th3.r00t]

joblo, today I decided to test the built-in Magi XMG M7M algo and for my surprise the miner says the algo in not AES-NI.

Magi has AES-NI support in cpuminer from atleast an year ago, by Wolf0.

Here is the github link:
https://github.com/wolf9466/wolf-m7m-cpuminer

[joblo]

joblo, today I decided to test the built-in Magi XMG M7M algo and for my surprise the miner says the algo in not AES-NI.

Magi has AES-NI support in cpuminer from atleast an year ago, by Wolf0.

Here is the github link:
https://github.com/wolf9466/wolf-m7m-cpuminer

How does your performance compare?

[th3.r00t]

AMD FX7600 (SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, SSE4A, XOP, AVX, FMA, FMA4, AES)

cpuminer v3.3.1 - 15-17 khash
Wolf0 - 33-35 khash

[joblo]

AMD FX7600 (SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, SSE4A, XOP, AVX, FMA, FMA4, AES)

cpuminer v3.3.1 - 15-17 khash
Wolf0 - 33-35 khash

I will definitely follow up for v3.3.3, need to get the other fixes out first. Thanks.

This is precisely the issue I'm trying to solve, where to find the best miner for a specific
algo. I want cpuminer-opt to the the one stop shop for all CPU mining.

Thanks for that tip.

[joblo]

cpuminer-opt v3.3.2 is released.

https://drive.google.com/open?id=0B0lVSGQYLJIZczlQLV93RnUzV1U

- Fixed low difficulty shares mining hodl
- Changed minimum CPU requirement for AES optimized mining, now requires
  a CPU with AES and AVX1. This excludes first generation AES CPUs like
  Nehalem and similar AMD. These CPUs can still mine using SSE2 optimizations.
- Updated build.sh and README.md

Windows users who don't like to compile their own can wait for Cryptomining Blog
to provide precompiled Windows executables. I will post the link in the OP when
it becomes available.

[hmage]

cpuminer-opt v3.3.2 is released.

https://drive.google.com/open?id=0B0lVSGQYLJIZczlQLV93RnUzV1U

- Fixed low difficulty shares mining hodl
- Changed minimum CPU requirement for AES optimized mining, now requires
  a CPU with AES and AVX1. This excludes first generation AES CPUs like
  Nehalem and similar AMD. These CPUs can still mine using SSE2 optimizations.
- Updated build.sh and README.md

Windows users who don't like to compile their own can wait for Cryptomining Blog
to provide precompiled Windows executables. I will post the link in the OP when
it becomes available.

Incorporated your changes into the git repository -- https://github.com/hmage/cpuminer-opt

As of now, there are no more source-level changes between my fork and yours -- https://github.com/hmage/cpuminer-opt/compare/upstream...master

[ryen123]

@joblo cryptomining-blog have updated their original article for the 3.3.2 windows release.

http://cryptomining-blog.com/7900-windows-binaries-for-the-cpuminer-opt-3-3-cpu-miner/

[joblo]

cpuminer-opt v3.3.2 is released.

https://drive.google.com/open?id=0B0lVSGQYLJIZczlQLV93RnUzV1U

- Fixed low difficulty shares mining hodl
- Changed minimum CPU requirement for AES optimized mining, now requires
  a CPU with AES and AVX1. This excludes first generation AES CPUs like
  Nehalem and similar AMD. These CPUs can still mine using SSE2 optimizations.
- Updated build.sh and README.md

Windows users who don't like to compile their own can wait for Cryptomining Blog
to provide precompiled Windows executables. I will post the link in the OP when
it becomes available.

Incorporated your changes into the git repository -- https://github.com/hmage/cpuminer-opt

As of now, there are no more source-level changes between my fork and yours -- https://github.com/hmage/cpuminer-opt/compare/upstream...master

Thanks, you may have noticed that I implemented unordered_map as you suggested a while back.

[joblo]

@joblo cryptomining-blog have updated their original article for the 3.3.2 windows release.

http://cryptomining-blog.com/7900-windows-binaries-for-the-cpuminer-opt-3-3-cpu-miner/

Thanks, OP is updated with the link.

[hmage]

Thanks, you may have noticed that I implemented unordered_map as you suggested a while back.

Yeap, I just needed to be patient

There is an (albeit small) issue with C++ files not being optimized:

Code:

./configure --with-curl CFLAGS="-O3 -march=native -Wall" CXXFLAGS="$CFLAGS -std=gnu++11"

Unless you previously set CFLAGS bash variable, $CFLAGS becomes nothing, so you're effectively calling this:

Code:

./configure --with-curl CFLAGS="-O3 -march=native -Wall" CXXFLAGS=" -std=gnu++11"

As a result, c++ files are compiled without any optimizations:

Code:

hmage@dhmd:~/src/cpuminer-opt$ ./build.sh 2>&1 | fgrep g++
checking for g++... g++
checking whether g++ accepts -g... yes
checking dependency style of g++... gcc3
g++ -DHAVE_CONFIG_H -I.  -Iyes/include -fno-strict-aliasing -I./compat/jansson -I. -Iyes/include  -std=gnu++11 -MT crypto/cpuminer-magimath.o -MD -MP -MF crypto/.deps/cpuminer-magimath.Tpo -c -o crypto/cpuminer-magimath.o `test -f 'crypto/magimath.cpp' || echo './'`crypto/magimath.cpp
g++ -DHAVE_CONFIG_H -I.  -Iyes/include -fno-strict-aliasing -I./compat/jansson -I. -Iyes/include  -std=gnu++11 -MT algo/hodl/cpuminer-hodl.o -MD -MP -MF algo/hodl/.deps/cpuminer-hodl.Tpo -c -o algo/hodl/cpuminer-hodl.o `test -f 'algo/hodl/hodl.cpp' || echo './'`algo/hodl/hodl.cpp
g++ -DHAVE_CONFIG_H -I.  -Iyes/include -fno-strict-aliasing -I./compat/jansson -I. -Iyes/include  -std=gnu++11 -MT algo/hodl/cpuminer-hodl_arith_uint256.o -MD -MP -MF algo/hodl/.deps/cpuminer-hodl_arith_uint256.Tpo -c -o algo/hodl/cpuminer-hodl_arith_uint256.o `test -f 'algo/hodl/hodl_arith_uint256.cpp' || echo './'`algo/hodl/hodl_arith_uint256.cpp
g++ -DHAVE_CONFIG_H -I.  -Iyes/include -fno-strict-aliasing -I./compat/jansson -I. -Iyes/include  -std=gnu++11 -MT algo/hodl/cpuminer-hodl_uint256.o -MD -MP -MF algo/hodl/.deps/cpuminer-hodl_uint256.Tpo -c -o algo/hodl/cpuminer-hodl_uint256.o `test -f 'algo/hodl/hodl_uint256.cpp' || echo './'`algo/hodl/hodl_uint256.cpp
g++ -DHAVE_CONFIG_H -I.  -Iyes/include -fno-strict-aliasing -I./compat/jansson -I. -Iyes/include  -std=gnu++11 -MT algo/hodl/cpuminer-hash.o -MD -MP -MF algo/hodl/.deps/cpuminer-hash.Tpo -c -o algo/hodl/cpuminer-hash.o `test -f 'algo/hodl/hash.cpp' || echo './'`algo/hodl/hash.cpp
g++ -DHAVE_CONFIG_H -I.  -Iyes/include -fno-strict-aliasing -I./compat/jansson -I. -Iyes/include  -std=gnu++11 -MT algo/hodl/cpuminer-hmac_sha512.o -MD -MP -MF algo/hodl/.deps/cpuminer-hmac_sha512.Tpo -c -o algo/hodl/cpuminer-hmac_sha512.o `test -f 'algo/hodl/hmac_sha512.cpp' || echo './'`algo/hodl/hmac_sha512.cpp
g++ -DHAVE_CONFIG_H -I.  -Iyes/include -fno-strict-aliasing -I./compat/jansson -I. -Iyes/include  -std=gnu++11 -MT algo/hodl/cpuminer-sha256.o -MD -MP -MF algo/hodl/.deps/cpuminer-sha256.Tpo -c -o algo/hodl/cpuminer-sha256.o `test -f 'algo/hodl/sha256.cpp' || echo './'`algo/hodl/sha256.cpp
g++ -DHAVE_CONFIG_H -I.  -Iyes/include -fno-strict-aliasing -I./compat/jansson -I. -Iyes/include  -std=gnu++11 -MT algo/hodl/cpuminer-sha512.o -MD -MP -MF algo/hodl/.deps/cpuminer-sha512.Tpo -c -o algo/hodl/cpuminer-sha512.o `test -f 'algo/hodl/sha512.cpp' || echo './'`algo/hodl/sha512.cpp
g++ -DHAVE_CONFIG_H -I.  -Iyes/include -fno-strict-aliasing -I./compat/jansson -I. -Iyes/include  -std=gnu++11 -MT algo/hodl/cpuminer-utilstrencodings.o -MD -MP -MF algo/hodl/.deps/cpuminer-utilstrencodings.Tpo -c -o algo/hodl/cpuminer-utilstrencodings.o `test -f 'algo/hodl/utilstrencodings.cpp' || echo './'`algo/hodl/utilstrencodings.cpp
g++ -DHAVE_CONFIG_H -I.  -Iyes/include -fno-strict-aliasing -I./compat/jansson -I. -Iyes/include  -std=gnu++11 -MT cpuminer-uint256.o -MD -MP -MF .deps/cpuminer-uint256.Tpo -c -o cpuminer-uint256.o `test -f 'uint256.cpp' || echo './'`uint256.cpp

To fix this, you need to either duplicate contents of CFLAGS manually in CXXFLAGS, or use shell variables (the way I do it in my fork):

Code:

CFLAGS="-O3 -march=native -Wall" CXXFLAGS="$CFLAGS -std=gnu++11" ./configure --with-curl

bash will expand $CFLAGS into contents of the variable it was set in same line previously, and then configure will pick up shell variables CFLAGS and CXXFLAGS and use them accordingly.

The problem is marginal, because it affects only hodl and magimath, and hodl is hand-optimized into AES anyway.

I did not benchmark the difference in performance yet, but plan to -- when back from work.

[joblo]

Thanks, you may have noticed that I implemented unordered_map as you suggested a while back.

Yeap, I just needed to be patient

There is an (albeit small) issue with C++ files not being optimized:

Code:

./configure --with-curl CFLAGS="-O3 -march=native -Wall" CXXFLAGS="$CFLAGS -std=gnu++11"

Unless you previously set CFLAGS bash variable, $CFLAGS becomes nothing, so you're effectively calling this:

Code:

./configure --with-curl CFLAGS="-O3 -march=native -Wall" CXXFLAGS=" -std=gnu++11"

As a result, c++ files are compiled without any optimizations:

Code:

hmage@dhmd:~/src/cpuminer-opt$ ./build.sh 2>&1 | fgrep g++
checking for g++... g++
checking whether g++ accepts -g... yes
checking dependency style of g++... gcc3
g++ -DHAVE_CONFIG_H -I.  -Iyes/include -fno-strict-aliasing -I./compat/jansson -I. -Iyes/include  -std=gnu++11 -MT crypto/cpuminer-magimath.o -MD -MP -MF crypto/.deps/cpuminer-magimath.Tpo -c -o crypto/cpuminer-magimath.o `test -f 'crypto/magimath.cpp' || echo './'`crypto/magimath.cpp
g++ -DHAVE_CONFIG_H -I.  -Iyes/include -fno-strict-aliasing -I./compat/jansson -I. -Iyes/include  -std=gnu++11 -MT algo/hodl/cpuminer-hodl.o -MD -MP -MF algo/hodl/.deps/cpuminer-hodl.Tpo -c -o algo/hodl/cpuminer-hodl.o `test -f 'algo/hodl/hodl.cpp' || echo './'`algo/hodl/hodl.cpp
g++ -DHAVE_CONFIG_H -I.  -Iyes/include -fno-strict-aliasing -I./compat/jansson -I. -Iyes/include  -std=gnu++11 -MT algo/hodl/cpuminer-hodl_arith_uint256.o -MD -MP -MF algo/hodl/.deps/cpuminer-hodl_arith_uint256.Tpo -c -o algo/hodl/cpuminer-hodl_arith_uint256.o `test -f 'algo/hodl/hodl_arith_uint256.cpp' || echo './'`algo/hodl/hodl_arith_uint256.cpp
g++ -DHAVE_CONFIG_H -I.  -Iyes/include -fno-strict-aliasing -I./compat/jansson -I. -Iyes/include  -std=gnu++11 -MT algo/hodl/cpuminer-hodl_uint256.o -MD -MP -MF algo/hodl/.deps/cpuminer-hodl_uint256.Tpo -c -o algo/hodl/cpuminer-hodl_uint256.o `test -f 'algo/hodl/hodl_uint256.cpp' || echo './'`algo/hodl/hodl_uint256.cpp
g++ -DHAVE_CONFIG_H -I.  -Iyes/include -fno-strict-aliasing -I./compat/jansson -I. -Iyes/include  -std=gnu++11 -MT algo/hodl/cpuminer-hash.o -MD -MP -MF algo/hodl/.deps/cpuminer-hash.Tpo -c -o algo/hodl/cpuminer-hash.o `test -f 'algo/hodl/hash.cpp' || echo './'`algo/hodl/hash.cpp
g++ -DHAVE_CONFIG_H -I.  -Iyes/include -fno-strict-aliasing -I./compat/jansson -I. -Iyes/include  -std=gnu++11 -MT algo/hodl/cpuminer-hmac_sha512.o -MD -MP -MF algo/hodl/.deps/cpuminer-hmac_sha512.Tpo -c -o algo/hodl/cpuminer-hmac_sha512.o `test -f 'algo/hodl/hmac_sha512.cpp' || echo './'`algo/hodl/hmac_sha512.cpp
g++ -DHAVE_CONFIG_H -I.  -Iyes/include -fno-strict-aliasing -I./compat/jansson -I. -Iyes/include  -std=gnu++11 -MT algo/hodl/cpuminer-sha256.o -MD -MP -MF algo/hodl/.deps/cpuminer-sha256.Tpo -c -o algo/hodl/cpuminer-sha256.o `test -f 'algo/hodl/sha256.cpp' || echo './'`algo/hodl/sha256.cpp
g++ -DHAVE_CONFIG_H -I.  -Iyes/include -fno-strict-aliasing -I./compat/jansson -I. -Iyes/include  -std=gnu++11 -MT algo/hodl/cpuminer-sha512.o -MD -MP -MF algo/hodl/.deps/cpuminer-sha512.Tpo -c -o algo/hodl/cpuminer-sha512.o `test -f 'algo/hodl/sha512.cpp' || echo './'`algo/hodl/sha512.cpp
g++ -DHAVE_CONFIG_H -I.  -Iyes/include -fno-strict-aliasing -I./compat/jansson -I. -Iyes/include  -std=gnu++11 -MT algo/hodl/cpuminer-utilstrencodings.o -MD -MP -MF algo/hodl/.deps/cpuminer-utilstrencodings.Tpo -c -o algo/hodl/cpuminer-utilstrencodings.o `test -f 'algo/hodl/utilstrencodings.cpp' || echo './'`algo/hodl/utilstrencodings.cpp
g++ -DHAVE_CONFIG_H -I.  -Iyes/include -fno-strict-aliasing -I./compat/jansson -I. -Iyes/include  -std=gnu++11 -MT cpuminer-uint256.o -MD -MP -MF .deps/cpuminer-uint256.Tpo -c -o cpuminer-uint256.o `test -f 'uint256.cpp' || echo './'`uint256.cpp

To fix this, you need to either duplicate contents of CFLAGS manually in CXXFLAGS, or use shell variables (the way I do it in my fork):

Code:

CFLAGS="-O3 -march=native -Wall" CXXFLAGS="$CFLAGS -std=gnu++11" ./configure --with-curl

bash will expand $CFLAGS into contents of the variable it was set in same line previously, and then configure will pick up shell variables CFLAGS and CXXFLAGS and use them accordingly.

The problem is marginal, because it affects only hodl and magimath, and hodl is hand-optimized into AES anyway.

I did not benchmark the difference in performance yet, but plan to -- when back from work.

Thanks. I can't take credit for the HOdl optimizations, that's Wolf0's work, but yes it is hand optimized.

Regarding the propagation of CFLAGS I assumed that the first part (assigning CFLAGS) would run before assigning
CXXFLAGS. Is that incorrect? Do I need to set CFLAGS outside of the configure command line?

[hmage]

Thanks. I can't take credit for the HOdl optimizations, that's Wolf0's work, but yes it is hand optimized.

Regarding the propagation of CFLAGS I assumed that the first part (assigning CFLAGS) would run before assigning
CXXFLAGS. Is that incorrect? Do I need to set CFLAGS outside of the configure command line?

You are not assigning CFLAGS in the shell if you give it as command line argument. And $CFLAGS expansion is done by the shell, not configure.

Short explanation:

Code:

echo A=1 B=1

shell runs "echo" with arguments "A=1" and "B=1". You'll get "A=1 B=1" as program's output. (with newline)

Code:

A=1 B=1 echo

shell sets environment variables "A=1" and "B=1", then runs "echo" with no arguments. You'll get empty string as program's output. (with newline)

This is big difference.

Many unix shells can expand any environment variable anywhere by using dollar sign, and this is where this is behaving not the way you intuitively might expect.

Code:

echo A=1 B=$A

shell, before executing this, expands $A into contents of that environment variable. But since A is not set, it replaces it with nothing, so shell runs "program" with arguments "A=1" and "B=". You'll get "A=1 B=" as program's output. (with newline)

Code:

A=1 B=$A echo

Shell, before executing this, sets environment variable "A=1", then expands $A into 1, then sets environment variable "B=1", then runs "echo" with no arguments. You'll get empty string as program's output. (with newline)

"configure" is coded to read CFLAGS and CXXFLAGS from environment variables if they are set, since it is common to set them as a separate phase in package managers before building (ubuntu, debian) and in some environments like gentoo it is common to have them set globally.

As a way to override that behaviour, it is possible to override environment's CFLAGS and CXXFLAGS with your custom ones by passing them as command line arguments to configure.

So, in this particular case, moving the assignments to the left will solve the issue:

Code:

CFLAGS="-O3 -march=native -Wall" CXXFLAGS="$CFLAGS -std=gnu++11" ./configure --with-curl