[LOCKED] cpuminer-opt v3.12.3, open source optimized multi-algo CPU miner

[joblo]

after including the lib dir with "-L/opt/mingw64/lib" only -lz was missing, after crosscompiling and copying the bin/lib/and include files into mingw64 the following error is displayed:

Code:

x86_64-w64-mingw32-g++  -O3 -march=native -Wall -I/opt/mingw64/include -L/opt/mingw64/lib -std=gnu++11 -fpermissive -Lcompat/pthreads/x64  -Lyes/lib  -Lcompat/pthreads/x64  -Lyes/lib  -o cpuminer.exe cpuminer-cpu-miner.o cpuminer-util.o cpuminer-uint256.o cpuminer-api.o cpuminer-sysinfos.o cpuminer-algo-gate-api.o algo/groestl/cpuminer-sph_groestl.o algo/skein/cpuminer-sph_skein.o algo/bmw/cpuminer-sph_bmw.o algo/shavite/cpuminer-sph_shavite.o algo/shavite/cpuminer-shavite.o algo/echo/cpuminer-sph_echo.o algo/blake/cpuminer-sph_blake.o algo/heavy/cpuminer-sph_hefty1.o algo/blake/cpuminer-mod_blakecoin.o algo/luffa/cpuminer-sph_luffa.o algo/cubehash/cpuminer-sph_cubehash.o algo/simd/cpuminer-sph_simd.o algo/hamsi/cpuminer-sph_hamsi.o algo/fugue/cpuminer-sph_fugue.o algo/gost/cpuminer-sph_gost.o algo/jh/cpuminer-sph_jh.o algo/keccak/cpuminer-sph_keccak.o algo/keccak/cpuminer-keccak.o algo/sha3/cpuminer-sph_sha2.o algo/sha3/cpuminer-sph_sha2big.o algo/shabal/cpuminer-sph_shabal.o algo/whirlpool/cpuminer-sph_whirlpool.o crypto/cpuminer-blake2s.o crypto/cpuminer-oaes_lib.o crypto/cpuminer-c_keccak.o crypto/cpuminer-c_groestl.o crypto/cpuminer-c_blake256.o crypto/cpuminer-c_jh.o crypto/cpuminer-c_skein.o crypto/cpuminer-hash.o crypto/cpuminer-aesb.o crypto/cpuminer-magimath.o algo/argon2/cpuminer-argon2a.o algo/argon2/ar2/cpuminer-argon2.o algo/argon2/ar2/cpuminer-opt.o algo/argon2/ar2/cpuminer-cores.o algo/argon2/ar2/cpuminer-ar2-scrypt-jane.o algo/argon2/ar2/cpuminer-blake2b.o algo/cpuminer-axiom.o algo/blake/cpuminer-blake.o algo/blake/cpuminer-blake2.o algo/blake/cpuminer-blakecoin.o algo/blake/cpuminer-decred.o algo/blake/cpuminer-pentablake.o algo/bmw/cpuminer-bmw256.o algo/cubehash/sse2/cpuminer-cubehash_sse2.o algo/cryptonight/cpuminer-cryptolight.o algo/cryptonight/cpuminer-cryptonight-common.o algo/cryptonight/cpuminer-cryptonight-aesni.o algo/cryptonight/cpuminer-cryptonight.o algo/cpuminer-drop.o algo/echo/aes_ni/cpuminer-hash.o algo/cpuminer-fresh.o algo/groestl/cpuminer-groestl.o algo/groestl/cpuminer-myr-groestl.o algo/groestl/sse2/cpuminer-grso.o algo/groestl/sse2/cpuminer-grso-asm.o algo/groestl/aes_ni/cpuminer-hash-groestl.o algo/groestl/aes_ni/cpuminer-hash-groestl256.o algo/haval/cpuminer-haval.o algo/heavy/cpuminer-heavy.o algo/heavy/cpuminer-bastion.o algo/cpuminer-hmq1725.o algo/hodl/cpuminer-hodl.o algo/hodl/cpuminer-hodl-gate.o algo/hodl/cpuminer-hodl_arith_uint256.o algo/hodl/cpuminer-hodl_uint256.o algo/hodl/cpuminer-hash.o algo/hodl/cpuminer-hmac_sha512.o algo/hodl/cpuminer-sha256.o algo/hodl/cpuminer-sha512.o algo/hodl/cpuminer-utilstrencodings.o algo/hodl/cpuminer-hodl-wolf.o algo/hodl/cpuminer-aes.o algo/hodl/cpuminer-sha512_avx.o algo/hodl/cpuminer-sha512_avx2.o algo/cpuminer-lbry.o algo/luffa/cpuminer-luffa.o algo/luffa/sse2/cpuminer-luffa_for_sse2.o algo/lyra2/cpuminer-lyra2.o algo/lyra2/cpuminer-sponge.o algo/lyra2/cpuminer-lyra2rev2.o algo/lyra2/cpuminer-lyra2re.o algo/keccak/sse2/cpuminer-keccak.o algo/cpuminer-m7m.o algo/cpuminer-neoscrypt.o algo/cpuminer-nist5.o algo/cpuminer-pluck.o algo/quark/cpuminer-quark.o algo/qubit/cpuminer-qubit.o algo/ripemd/cpuminer-sph_ripemd.o algo/cpuminer-scrypt.o algo/scryptjane/cpuminer-scrypt-jane.o algo/sha2/cpuminer-sha2.o algo/simd/sse2/cpuminer-nist.o algo/simd/sse2/cpuminer-vector.o algo/skein/cpuminer-skein.o algo/skein/cpuminer-skein2.o algo/cpuminer-s3.o algo/tiger/cpuminer-sph_tiger.o algo/cpuminer-veltor.o algo/whirlpool/cpuminer-whirlpool.o algo/whirlpool/cpuminer-whirlpoolx.o algo/x11/cpuminer-x11.o algo/x11/cpuminer-x11evo.o algo/x11/cpuminer-x11gost.o algo/x11/cpuminer-c11.o algo/x13/cpuminer-x13.o algo/x14/cpuminer-x14.o algo/x15/cpuminer-x15.o algo/x17/cpuminer-x17.o algo/yescrypt/cpuminer-yescrypt.o algo/yescrypt/cpuminer-yescrypt-common.o algo/yescrypt/cpuminer-sha256_Y.o algo/yescrypt/cpuminer-yescrypt-simd.o algo/cpuminer-zr5.o asm/cpuminer-neoscrypt_asm.o  asm/cpuminer-sha2-x64.o asm/cpuminer-scrypt-x64.o asm/cpuminer-aesb-x64.o  compat/cpuminer-winansi.o -lcurl -lz compat/jansson/libjansson.a -lpthread -L/mingw/x86_64-w64-mingw32/lib -lws2_32 -lssl -lcrypto -lgmp
cpuminer-util.o:util.c:(.text+0x4e6): undefined reference to `localtime_s'
cpuminer-util.o:util.c:(.text+0x632): undefined reference to `localtime_s'
cpuminer-util.o:util.c:(.text+0x784): undefined reference to `localtime_s'
/opt/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/4.9.0/../../../../x86_64-w64-mingw32/bin/ld: cpuminer-util.o: bad reloc address 0x11 in section `.text.unlikely'
collect2: error: ld returned 1 exit status

Here's the relevant code from compat.h:

Code:

#include <windows.h>
#include <time.h>

#ifndef localtime_r
#define localtime_r(src, dst) localtime_s(dst, src)
#endif

It looks like a problem finding time.h.

Edit: Or the time.h it is using doesn't define localtime_s.

[felixbrucker]

hmm, including it with -I does not change anything

yeah the intel cpu isnt the problem, only amd

so close, yet so far, will wait/retry later to compile
i feel i got further than using windows and msys

cheers

edit: yes the time.h does not contain localtime_s, what to do in this case?

[felixbrucker]

ok, copied the time.h from my previous windows try (which has localtime_s) to linux, error is gone but a different one appeard. will try and see if i can get a newer mingw64 build on my linux and rerun everything with newer base files

cheers

[joblo]

hmm, including it with -I does not change anything

yeah the intel cpu isnt the problem, only amd

so close, yet so far, will wait/retry later to compile
i feel i got further than using windows and msys

cheers

edit: yes the time.h does not contain localtime_s, what to do in this case?

The angle brackets instead of quotes means it's a library file. I found a few copies of time.h in /msys/opt/windows_64/include
but I didn't find localtime_s defined in any of them. But it compiles. I don't know.

[felixbrucker]

now it tells me

Code:

algo/cpuminer-m7m.o:m7m.c:(.text+0x1cfe): undefined reference to `nanosleep'

commands to reproduce:

Code:

mkdir /opt/deps
mkdir /opt/mingw64


####################################
# https://curl.haxx.se/download.html
# https://www.openssl.org/source/
# http://zlib.net/
# https://gmplib.org/#DOWNLOAD
# https://sourceforge.net/projects/mingw-w64/files/External%20binary%20packages%20(Win64%20hosted)/pthreads/
# save to /opt/deps
#
# https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Automated%20Builds/
# save to /opt/mingw64
####################################


cd /opt/deps

cd curl-7.50.1
./configure --host=x86_64-w64-mingw32 --prefix=/opt/mingw64
make
make install
cd ..

cd openssl-1.0.2h
CROSS_COMPILE="x86_64-w64-mingw32-" ./Configure mingw64 no-asm shared --prefix=/opt/mingw64
make
make install
cd ..

cd zlib-1.2.8
nano win32/Makefile.gcc # PREFIX = x86_64-w64-mingw32-
make -f win32/Makefile.gcc
cp zlib1.dll /opt/mingw64/bin
cp zconf.h zlib.h /opt/mingw64/include
cp libz.a /opt/mingw64/lib
cd ..

cd gmp-6.1.1
./configure --host=x86_64-w64-mingw32 --prefix=/opt/mingw64
make
make install
cd ..

cd pthreads-20100604/mingw64
unzip pthreads-w64.zip -d /opt/mingw64
cd ../..

however ill let this rest for now, if you got some spare time and are willing to setup a vm/container to verify the steps you have done on your main machine that would be super great, its low priority, amd win bins work too, just not that fast

cheers, keep up the good work

[joblo]

now it tells me

Code:

algo/cpuminer-m7m.o:m7m.c:(.text+0x1cfe): undefined reference to `nanosleep'

commands to reproduce:

Code:

mkdir /opt/deps
mkdir /opt/mingw64


####################################
# https://curl.haxx.se/download.html
# https://www.openssl.org/source/
# http://zlib.net/
# https://gmplib.org/#DOWNLOAD
# https://sourceforge.net/projects/mingw-w64/files/External%20binary%20packages%20(Win64%20hosted)/pthreads/
# save to /opt/deps
#
# https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Automated%20Builds/
# save to /opt/mingw64
####################################


cd /opt/deps

cd curl-7.50.1
./configure --host=x86_64-w64-mingw32 --prefix=/opt/mingw64
make
make install
cd ..

cd openssl-1.0.2h
CROSS_COMPILE="x86_64-w64-mingw32-" ./Configure mingw64 no-asm shared --prefix=/opt/mingw64
make
make install
cd ..

cd zlib-1.2.8
nano win32/Makefile.gcc # PREFIX = x86_64-w64-mingw32-
make -f win32/Makefile.gcc
cp zlib1.dll /opt/mingw64/bin
cp zconf.h zlib.h /opt/mingw64/include
cp libz.a /opt/mingw64/lib
cd ..

cd gmp-6.1.1
./configure --host=x86_64-w64-mingw32 --prefix=/opt/mingw64
make
make install
cd ..

cd pthreads-20100604/mingw64
unzip pthreads-w64.zip -d /opt/mingw64
cd ../..

however ill let this rest for now, if you got some spare time and are willing to setup a vm/container to verify the steps you have done on your main machine that would be super great, its low priority, amd win bins work too, just not that fast

cheers, keep up the good work

All these errors are stuff the compiler can't find. I didn't have to do any compiling of 3rd party packages so there must be a simpler
explanation. I noticed you keep referring to /opt/mingw64. My setup has all the includes in /msys/opt/windows_64. This is the way
it was setup when I installed it. Mingw asks you whether you want to install for cygwin or msys as described in README.md.
Maybe this has something to do with it.

[felixbrucker]

the commands refer to linux cross compiling, in windows there were no such libs already included, will wait for your walkthrough, you might have used other downloads as i have which are pre built or something

cheers

[joblo]

The stale share reject problem mining cryptonight at Nicehash is a Nicehash problem, either with the pool itself
or the cpuminer mod. As previously mentioned these rejects only occur at Nicehash and further testing has confirmed
they occur with Nicehash's miner.

After several hours of reject free mining they've started again.

[felixbrucker]

update: the buffer overflow thingy happened again (again only on this system), will investigate

cheers

edit: other systems joined, so its definitely a bug, either in lxc or the miner itself, will continue investigating

it seems to occur when name resolution fails (repeatedly)

[felixbrucker]

with my limited knowledge of using gdb without an IDE it got this info:

Code:

(gdb) where
#0  0x00007ffff663b418 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#1  0x00007ffff663d01a in __GI_abort () at abort.c:89
#2  0x00007ffff667d72a in __libc_message (do_abort=do_abort@entry=2, fmt=fmt@entry=0x7ffff6794c7f "*** %s ***: %s terminated\n") at ../sysdeps/posix/libc_fatal.c:175
#3  0x00007ffff671e89c in __GI___fortify_fail (msg=<optimized out>, msg@entry=0x7ffff6794c10 "buffer overflow detected") at fortify_fail.c:37
#4  0x00007ffff671c8a0 in __GI___chk_fail () at chk_fail.c:28
#5  0x00007ffff671e7e7 in __fdelt_chk (d=<optimized out>) at fdelt_chk.c:25
#6  0x000000000041028a in ?? ()
#7  0x0000000000408411 in ?? ()
#8  0x000000000040887f in ?? ()
#9  0x00007ffff79526fa in start_thread (arg=0x7ffff17d1700) at pthread_create.c:333
#10 0x00007ffff670cb5d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

6,7 and 8 are in the cpuminer code which got terminated unfortunately, someone knows some gdb command to not kill the cpuminer if a buffer overflow is detected so i can check which functions get called?

cheers

[joblo]

with my limited knowledge of using gdb without an IDE it got this info:

Code:

(gdb) where
#0  0x00007ffff663b418 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#1  0x00007ffff663d01a in __GI_abort () at abort.c:89
#2  0x00007ffff667d72a in __libc_message (do_abort=do_abort@entry=2, fmt=fmt@entry=0x7ffff6794c7f "*** %s ***: %s terminated\n") at ../sysdeps/posix/libc_fatal.c:175
#3  0x00007ffff671e89c in __GI___fortify_fail (msg=<optimized out>, msg@entry=0x7ffff6794c10 "buffer overflow detected") at fortify_fail.c:37
#4  0x00007ffff671c8a0 in __GI___chk_fail () at chk_fail.c:28
#5  0x00007ffff671e7e7 in __fdelt_chk (d=<optimized out>) at fdelt_chk.c:25
#6  0x000000000041028a in ?? ()
#7  0x0000000000408411 in ?? ()
#8  0x000000000040887f in ?? ()
#9  0x00007ffff79526fa in start_thread (arg=0x7ffff17d1700) at pthread_create.c:333
#10 0x00007ffff670cb5d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

6,7 and 8 are in the cpuminer code which got terminated unfortunately, someone knows some gdb command to not kill the cpuminer if a buffer overflow is detected so i can check which functions get called?

cheers

cpuminer is trying to create a new thread. This occurs in main() and is very old and
very stable code that hasn't been modified in ages. It's not likely to have a bug.

This could be the compiler or the system (this is the VM right?). The system may be imposing an artificial limitation on resources
that is being exceeded by cpuminer.

I presume you compiled the same way as on other systems so unless there is something else significantly different between them
the obvious conclusion would be that it's because it's a VM.

[NeuroticFish]

I was so happy that this miner works on my computer (Nehalem, Windows), but it was too early.
I did test with your 3.4.3 and 3.4.4 too and Cryptonight is.. slow. And I talk about what the pools report to me.
The miner reports 45-55 H/s and the pool about half of it.

I tried long ago Wolf's miner and I had a similar (slowness) issue and I had to abandon it.
For some reason the very old cpuminer-multi 1.0.3 (https://github.com/lucasjones/cpuminer-multi/releases) still works best for me.

I don't know if it makes sense for you to invest time into this, but I thought it's better you know about it.

[felixbrucker]

its a lxc container actually, this is far more bound to limitations as it uses the same kernel and is no full virtualized system

will test with bare metal systems to verify

im using a socks5 proxy on all my machines affected, maybe this part of the code is not well tested, time will tell as i do my various tests

yes, compiled the same way on every system (build.sh)

cheers

[joblo]

its a lxc container actually, this is far more bound to limitations as it uses the same kernel and is no full virtualized system

will test with bare metal systems to verify

im using a socks5 proxy on all my machines affected, maybe this part of the code is not well tested, time will tell as i do my various tests

yes, compiled the same way on every system (build.sh)

cheers

I think I recall seeing a similar crash due to a stack overflow. Cryptonight was storing a large data structure on the stack and, although
it worked on Linux, it crashed on Windows. I made a change to cryptonight to reduce stack usage and it fixed it.

Can you increase the stack on your LXC?

[joblo]

I was so happy that this miner works on my computer (Nehalem, Windows), but it was too early.
I did test with your 3.4.3 and 3.4.4 too and Cryptonight is.. slow. And I talk about what the pools report to me.
The miner reports 45-55 H/s and the pool about half of it.

I tried long ago Wolf's miner and I had a similar (slowness) issue and I had to abandon it.
For some reason the very old cpuminer-multi 1.0.3 (https://github.com/lucasjones/cpuminer-multi/releases) still works best for me.

I don't know if it makes sense for you to invest time into this, but I thought it's better you know about it.

Thank you for your comments. Cryptonight uses AES. CPUs starting with Westmere have HW AES support built in. All the talk about
AES optimization is actually about using the HW AES support of the CPU.

Unfortunately Nehalem does not have HW AES support and must use the slower software implementation. There is no point is investing any
time in such an old architecture.

[felixbrucker]

its a lxc container actually, this is far more bound to limitations as it uses the same kernel and is no full virtualized system

will test with bare metal systems to verify

im using a socks5 proxy on all my machines affected, maybe this part of the code is not well tested, time will tell as i do my various tests

yes, compiled the same way on every system (build.sh)

cheers

I think I recall seeing a similar crash due to a stack overflow. Cryptonight was storing a large data structure on the stack and, although
it worked on Linux, it crashed on Windows. I made a change to cryptonight to reduce stack usage and it fixed it.

Can you increase the stack on your LXC?

i doubt one can increase the stack on lxc level, stack is usually increased in code or compile options, or am i wrong?

edit: seems its also settable by bash, will try after i confirmed its on lxc only

thanks for the hint

[joblo]

its a lxc container actually, this is far more bound to limitations as it uses the same kernel and is no full virtualized system

will test with bare metal systems to verify

im using a socks5 proxy on all my machines affected, maybe this part of the code is not well tested, time will tell as i do my various tests

yes, compiled the same way on every system (build.sh)

cheers

I think I recall seeing a similar crash due to a stack overflow. Cryptonight was storing a large data structure on the stack and, although
it worked on Linux, it crashed on Windows. I made a change to cryptonight to reduce stack usage and it fixed it.

Can you increase the stack on your LXC?

i doubt one can increase the stack on lxc level, stack is usually increased in code or compile options, or am i wrong?

edit: seems its also settable by bash, will try after i confirmed its on lxc only

thanks for the hint

There might be a configure option to set stack size in cpuminer but I couldn't find it when troubleshooting the cryptonight stack overflow.
But then I found the real problem and it became moot.

The current setting (or the default) for cpuminer is good enough on other systems. If it doesn't work on lxc it's likely because lxc has a lower stack limit.
Maybe there is a setting in lxc to increase it.

Thre bash setting would only affect the shell, not other processes.

[felixbrucker]

its a lxc container actually, this is far more bound to limitations as it uses the same kernel and is no full virtualized system

will test with bare metal systems to verify

im using a socks5 proxy on all my machines affected, maybe this part of the code is not well tested, time will tell as i do my various tests

yes, compiled the same way on every system (build.sh)

cheers

I think I recall seeing a similar crash due to a stack overflow. Cryptonight was storing a large data structure on the stack and, although
it worked on Linux, it crashed on Windows. I made a change to cryptonight to reduce stack usage and it fixed it.

Can you increase the stack on your LXC?

i doubt one can increase the stack on lxc level, stack is usually increased in code or compile options, or am i wrong?

edit: seems its also settable by bash, will try after i confirmed its on lxc only

thanks for the hint

There might be a configure option to set stack size in cpuminer but I couldn't find it when troubleshooting the cryptonight stack overflow.
But then I found the real problem and it became moot.

The current setting (or the default) for cpuminer is good enough on other systems. If it doesn't work on lxc it's likely because lxc has a lower stack limit.
Maybe there is a setting in lxc to increase it.

Thre bash setting would only affect the shell, not other processes.

actually it does for this session, eg all programs started by this shell

"ulimit provides control over the resources available to the shell and to processes started by it, on systems that allow such control."

unfortunately i dont have any way to reproduce the buffer overflow except just waiting so it will take a while, will post results

[ioglnx]

Is this the Cryptonite (M7) XCN support or some other cryptonite? Sorry for that question of a newbe

[joblo]

Is this the Cryptonite (M7) XCN support or some other cryptonite? Sorry for that question of a newbe

No, different spelling.

cpuminer-opt does cryptonight algo which is used my monero coin (XMR). cryptonight is also sometimes
called cryptonote.