An (even more) optimized version of cpuminer (pooler's cpuminer, CPU-only)

[linuxdoctor]

I'm new to Bitcoin and this forum and am trying to learn all I can about it. One way I've always found to learn how things work is to go through the code.  I've already slogged through the code standard Bitcoin implementation, bitcoind/bitcoin-qt, and some of BitcoinArmory. I'm now looking through cpuminer. The code for cpuminer is fairly easy to read being written in standard C.

I understand most of what's going on but I've got a few questions about what I've found in the code. Actually, it's a question about some curious code. Here's the extract from cpu-miner.c. I've removed the irrelevant portions.

Code:

static void *miner_thread(void *userdata)
{
. . .
	uint32_t end_nonce = 0xffffffffU / opt_n_threads * (thr_id + 1) - 0x20;
. . .
		if (memcmp(work.data, g_work.data, 76)) {
			memcpy(&work, &g_work, sizeof(struct work));
			work.data[19] = 0xffffffffU / opt_n_threads * thr_id;
		} else
			work.data[19]++;
. . .
}

This code appears to set up an initial start and end boundary for nonce checking on new work. It also seems to be attempting some cleverness to avoid doing expensive computation on 'long long int' type.  I'm assuming that the intent is to set start and end boundaries at the beginning of each thread's block (relative offset of 0) and then then end at one by before the start of the next thread's block. That's not what the effect is and the following block shows:

Code:

thread = 0 start_nonse = 0x00000000 end_nonse = 0x1fffffdf
thread = 1 start_nonse = 0x1fffffff end_nonse = 0x3fffffde
thread = 2 start_nonse = 0x3ffffffe end_nonse = 0x5fffffdd
thread = 3 start_nonse = 0x5ffffffd end_nonse = 0x7fffffdc
thread = 4 start_nonse = 0x7ffffffc end_nonse = 0x9fffffdb
thread = 5 start_nonse = 0x9ffffffb end_nonse = 0xbfffffda
thread = 6 start_nonse = 0xbffffffa end_nonse = 0xdfffffd9
thread = 7 start_nonse = 0xdffffff9 end_nonse = 0xffffffd8

I created a programme that output the boundaries of each thread block and this is what I got.

The question is: is this intentional, and if so to what end? I know that the end boundary is later adjusted (downwards) based on the hash rate of the machine that the programme calculates on a running basis. It just seems a little odd to me. If this is a bug, a simple fix will make the cleverness work.

Thanks for you insite.

[1713976969]

1713976969

[1713976969]

1713976969

[1713976969]

1713976969

[1713976969]

1713976969

[1713976969]

1713976969

[pooler]

I understand most of what's going on but I've got a few questions about what I've found in the code. Actually, it's a question about some curious code. Here's the extract from cpu-miner.c. I've removed the irrelevant portions.

Code:

static void *miner_thread(void *userdata)
{
. . .
	uint32_t end_nonce = 0xffffffffU / opt_n_threads * (thr_id + 1) - 0x20;
. . .
		if (memcmp(work.data, g_work.data, 76)) {
			memcpy(&work, &g_work, sizeof(struct work));
			work.data[19] = 0xffffffffU / opt_n_threads * thr_id;
		} else
			work.data[19]++;
. . .
}

This code appears to set up an initial start and end boundary for nonce checking on new work. It also seems to be attempting some cleverness to avoid doing expensive computation on 'long long int' type.  I'm assuming that the intent is to set start and end boundaries at the beginning of each thread's block (relative offset of 0) and then then end at one by before the start of the next thread's block. That's not what the effect is and the following block shows:

Code:

thread = 0 start_nonse = 0x00000000 end_nonse = 0x1fffffdf
thread = 1 start_nonse = 0x1fffffff end_nonse = 0x3fffffde
thread = 2 start_nonse = 0x3ffffffe end_nonse = 0x5fffffdd
thread = 3 start_nonse = 0x5ffffffd end_nonse = 0x7fffffdc
thread = 4 start_nonse = 0x7ffffffc end_nonse = 0x9fffffdb
thread = 5 start_nonse = 0x9ffffffb end_nonse = 0xbfffffda
thread = 6 start_nonse = 0xbffffffa end_nonse = 0xdfffffd9
thread = 7 start_nonse = 0xdffffff9 end_nonse = 0xffffffd8

I created a programme that output the boundaries of each thread block and this is what I got.

The purpose of that piece of code is to share the same work unit (block header) between all miner threads, so that we don't need to fetch different work (if using getwork) or to compute a different Merkle root hash (if using Stratum) for every thread. This is accomplished by partitioning the nonce space and by assigning a distinct nonce range to each thread, so that no two ranges overlap. Because of how mining works, the search over the nonce space doesn't have to be exhaustive, meaning that it is OK if we don't check all possible nonces before fetching new work.
The gap that you see between nonce ranges is needed to avoid checking some nonces twice, as some of the core algorithm implementations may compute more hashes than requested (because of parallelization). The fact that we may skip checking a tiny fraction (0.000004%) of the nonce space is far from being a problem.

[linuxdoctor]

The gap that you see between nonce ranges is needed to avoid checking some nonces twice, as some of the core algorithm implementations may compute more hashes than requested (because of parallelization). The fact that we may skip checking a tiny fraction (0.000004%) of the nonce space is far from being a problem.

From a practical viewpoint it is unlikely that anybody would be able to hash every nonce in a given block given the present state of available hardware with the possible exception of banks of super computers deep in the bowels of the NSA or other government entity somewhere. From that perspective the gap is irrelevant. By the same token, even the overlap (without the subtract of 32) is irrelevant also and amounts to about the number of threads and still leaves a gap at the upper end of the 32-bit range.

I just wondered if there was something more specific to mining than that that necessitated the gap. It is the algorithm you use to calculate the range for each thread that attracted my attention. It looks like something I used to do with small microprocessors in embedded applications where I needed to play with numbers larger than the available precision of the machine. Very clever and not something I see often any more. The purist in me, though, would want to see each block on a neat zero boundary with no overlap.

Just for fun I adjusted your algorithm so that each block starts on a neat zero boundary with no overlaps.

Code:

	uint32_t end_nonce = 0xffffffffU / opt_n_threads * (thr_id + 1) - 0x20;
becomes
	uint32_t end_nonce = 0xffffffffU / opt_n_threads * (thr_id + 1) + thr_id;

and

			work.data[19] = 0xffffffffU / opt_n_threads * thr_id;
becomes
 			work.data[19] = 0xffffffffU / opt_n_threads * thr_id + thr_id;

This produces (for 8 threads):

thread = 0 start_nonse = 0x00000000 end_nonse = 0x1fffffff
thread = 1 start_nonse = 0x20000000 end_nonse = 0x3fffffff
thread = 2 start_nonse = 0x40000000 end_nonse = 0x5fffffff
thread = 3 start_nonse = 0x60000000 end_nonse = 0x7fffffff
thread = 4 start_nonse = 0x80000000 end_nonse = 0x9fffffff
thread = 5 start_nonse = 0xa0000000 end_nonse = 0xbfffffff
thread = 6 start_nonse = 0xc0000000 end_nonse = 0xdfffffff
thread = 7 start_nonse = 0xe0000000 end_nonse = 0xffffffff

This is using gcc 4.8.2 with -O3 optimization on Fedora linux kernel 3.13.0 and AMD FX8350.

Thanks for your timely response.

[pooler]

From a practical viewpoint it is unlikely that anybody would be able to hash every nonce in a given block given the present state of available hardware with the possible exception of banks of super computers deep in the bowels of the NSA or other government entity somewhere.

That is incorrect. It only takes about 72 MH/s to check every possible nonce in less than a minute, and dual-CPU systems are already able to attain such rates for SHA-256d.

[linuxdoctor]

You're right. I must still be living with the dinosaurs. I'm getting about 5M on my FX8350 running at the standard clock speed of 4.0GHz which is roughly equivalent to running through all 2^32 possible hashes in 107 seconds when all 8 cores are running.

So, that makes the gap and overlap even more important.

[imwesigwa]

Hello,

Am a newbie trying to setup CPUMiner of a 64bit 4 core RedHat Environment.
I have run into the following errors while installing mining_proxy.

Any Help will be appreciated.

Code:

Extracting stratum_mining_proxy-1.5.2-py2.4.egg to /usr/lib/python2.4/site-packages
  File "/usr/lib/python2.4/site-packages/stratum_mining_proxy-1.5.2-py2.4.egg/mining_libs/stratum_listener.py", line 133
    result = (yield self._f.rpc('mining.authorize', [worker_name, worker_password]))
                  ^
SyntaxError: invalid syntax
  File "/usr/lib/python2.4/site-packages/stratum_mining_proxy-1.5.2-py2.4.egg/mining_libs/utils.py", line 63
    (yield d)
         ^
SyntaxError: invalid syntax
  File "/usr/lib/python2.4/site-packages/stratum_mining_proxy-1.5.2-py2.4.egg/mining_libs/midstate.py", line 80
    consts = K if rounds is None else K[:rounds]
                ^
SyntaxError: invalid syntax

[Plus]

hello dear, i need help
using Binaries for Linux: http://sourceforge.net/projects/cpuminer/files/pooler-cpuminer-2.3.2-linux-x86_64.tar.gz (x86-64)
os: Linux new 2.6.18-348.4.1.el5.centos.plus #1 SMP Tue Apr 16 18:51:49 EDT 2013 x86_64 x86_64 x86_64 GNU/Linux
./minerd --url=stratum+tcp://s.eu.vertcoin.org:3333 --userpass=Plus.s741:coolmegapass

and work problem:
Failed to get Stratum session id
and
DEBUG: reject reason: Share is above target

printscreen http://clip2net.com/s/6Lfb2Z


booooo/booooo/booooo/booooo/booooo ((((

[electricfeel]

hello pooler, I have used your cpuminer for doge and ltc. I just started using it for maxcoin and It pops up and runs fine but I get a 0.00% and boo, tried running it like the following below.

using your 2.3.2-win 32 and stratum

1: minerd.exe  -o  stratum+tcp://maxcoinpool.com:4000  -u worker.1 -password
2: minerd.exe  -o  stratum+tcp://www.maxcoinpool.com:4000  -u worker.1 -password

its a command line and not sure what to do, any advice?

[PSL]

hello pooler, I have used your cpuminer for doge and ltc. I just started using it for maxcoin and It pops up and runs fine but I get a 0.00% and boo, tried running it like the following below.

This version of cpuminer cannot be used to mine maxcoin because that coin needs a miner with support for "keccak" algorithm.

[Pontius]

Sun Fire T1000 6-core @ 1GHz
Oracle Solaris 11.1

at 24 threads each gets .06kh/s =1.44kh/s

They're $75 on ebay, 180W power consumption.

If I mine all month I can turn $10 of power into 75 cents worth of altcoins

compiling with -O3 made this .07 per thread, for a total of 1.68Kh/s

Compiling it with SunStudio instead of GCC will give you maybe even a little more (didn't test this for an "eternity" though).
BTW, running more threads then you a have physical cores on a Niagara based systems will gain you absolutely nothing. 

[primer-]

Pooler, can you create a new build which includes keccak ? Is the current MaxCoin cpuminer based on your latest version ?

[oliversl]

It would be nice to have an official build after the maxcoin launch, thanks

[HammerHedd]

I have a quick question about darkcoin:

I"m trying to run minerd with -a X11 for the darkcoin algorithm, and all I get is the message "type --help for more information". If I leave out -a X11, I can connect to the pool fine via stratum, but of course all my hashes are rejected.

I don't see any X11 specific source in the git clone, but I'm no programmer, so I might not know where to look if it isn't obvious.

What am I doing wrong?

BTW, it seems to work fine on windows with a pre-compiled version. This is failing on Ubuntu with me compiling from the latest git (2.3.2).

[splat44]

Using latest pooler for windows while mining dogecoin, cannot connect to server: ypool.net:8332 even after using xptProxy v0.2b

Can anyone double check this?

edited

Ok, I found out that I need using localhost:8332

[DarkCloud]

Hi

Here is a small script for multiple choices when using the cpuminer, for anyone interested.
I was tired of having multiple files for pools, and here is the result of pure boredom :lol:

Just copy to a .cmd file and replace MINER.URL:PORT and USER:PASSWORD, except for the last url where port is %port%
Added a few menu items as an example at the top.

have phun

Code:

@:Multipool Multi (ALL):7777
@:Multipool Litecoin (LTC):3334
@:Multipool FeatherCoin (FTC):3337
@:Multipool Mincoin (MNC):3339
@:Multipool WorldCoin (WDC):3342
@:Multipool DigitalCoin (DGC):3343
@:Multipool NovaCoin (NVC):3344
@:Multipool LuckyCoin (LKY):3345
@:Multipool Argentum (ARG):3346
@:Multipool PhenixCoin (PXC):3347
@:Multipool MegaCoin (MEC):3348
@:Multipool BottleCaps (CAP):3349
@:Multipool Cryptogenic Bullion (CGB):3336
@:Multipool DogeCoin (DOGE):3352
@:Multipool Diamond (DMD):3354
@:Multipool Grandcoin (GDC):3356
@:Multipool FedoraCoin (TIPS):3357
@:Multipool Mooncoin (MOON):3358
@:Multipool Earthcoin (EAC):3359
@:Multipool Lottocoin (LOT):3360
@:Coin-Pool.org (LTC):0000
@:DogeCoin Pool (DOGE):0001
@:NyanCoin Pool (NYAN):0002
@:PotCoin Pool (POT):0003
@echo off
:start
set "var=@"
for /f "tokens=1,2 delims=:@" %%a in ('findstr /n "%var%:" "%~f0" ') do (
echo %%a %%b
)
echo.
set /p "num=Select a number: "
for /f "tokens=1,2,3 delims=:@" %%a in ('findstr /n "%var%:" "%~f0" ^|findstr "^%num%:" ') do (
set "name=%%b"
set "port=%%c"
)
echo starting %name%:%port%
title We are mining "%name%"
if %port% EQU 0000 (
   minerd -a scrypt -o stratum+tcp://MINER.URL:PORT -O USER:PASSWORD -q
  )
if %port% EQU 0001 (
   minerd -a scrypt -o stratum+tcp://MINER.URL:PORT -O USER:PASSWORD -q
  )
if %port% EQU 0002 (
   minerd -a scrypt -o stratum+tcp://MINER.URL:PORT -O USER:PASSWORD -q
  )
if %port% EQU 0003 (
   minerd -a scrypt -o stratum+tcp://MINER.URL:PORT -O USER:PASSWORD -q
  ) else (
   minerd -a scrypt -o stratum+tcp://MINER.URL:%port% -O USER:PASSWORD -q
)

[Vaheliex]

So i have been having a serious problem with cpuminer for the past couple of days and have asked around a couple of places and have yet to get a fix. Whenever i try to run the minerd or a batch file that uses it the command window shows up for a brief second, displays the file path of minerd, then closes itself and minerd proceeds to vanish from the folder. No error messages, just opens and then closes. My only real method of mining is with my cpu so without this program i really have no other way to efficiently mine. If anyone has a solution for this that would be great.

[derbrause]

I absolutely need a 64 bit CPU Miner for mining vertcoin (scrypt-n). Any way to get one? :-))

[examiner]

Hi,

I am trying to compile on XP 32bit (that's all I currently have available) and I am having some problems:

I have installed MinGW32, MSYS, C, and C++ compilers.
Downloaded and compiled curl-7.30.0.tar.gz, and copied over the libcurl.m4 and curl-config files.
Downloaded 'pooler-cpuminer-2.3.2.tar.gz' from Sourceforge (I am assuming this is the source to use?).

I am trying to build for 32bit Windows, so I use the config command:

Code:

./configure --host=i686-w64-mingw32 CFLAGS="-O3"

This appears to complete without any issues, so I issue the make command and get the following error message:

Code:

noveske@xp /c/cpuminer-2.3.2
$ make
make  all-recursive
make[1]: Entering directory `/c/cpuminer-2.3.2'
Making all in compat
make[2]: Entering directory `/c/cpuminer-2.3.2/compat'
Making all in jansson
make[3]: Entering directory `/c/cpuminer-2.3.2/compat/jansson'
make[3]: Nothing to be done for `all'.
make[3]: Leaving directory `/c/cpuminer-2.3.2/compat/jansson'
make[3]: Entering directory `/c/cpuminer-2.3.2/compat'
make[3]: Nothing to be done for `all-am'.
make[3]: Leaving directory `/c/cpuminer-2.3.2/compat'
make[2]: Leaving directory `/c/cpuminer-2.3.2/compat'
make[2]: Entering directory `/c/cpuminer-2.3.2'
gcc -std=gnu99 -DHAVE_CONFIG_H -I.  -fno-strict-aliasing -I./compat/jansson -I/c/MinGW/include   -O3 -MT minerd-cpu-miner.o -MD -MP -MF .deps/minerd-cpu-miner.Tpo -c -o minerd-cpu-miner.o `test -f 'cpu-miner.c' || echo './'`cpu-miner.c
In file included from c:\mingw\include\curl\curlbuild.h:124:0,
                 from c:\mingw\include\curl\curl.h:34,
                 from cpu-miner.c:37:
c:\mingw\include\ws2tcpip.h:38:2: error: #error "ws2tcpip.h is not compatible with winsock.h. Include winsock2.h instead."
 #error "ws2tcpip.h is not compatible with winsock.h. Include winsock2.h instead."
  ^
In file included from c:\mingw\include\curl\curlbuild.h:124:0,
                 from c:\mingw\include\curl\curl.h:34,
                 from cpu-miner.c:37:
c:\mingw\include\ws2tcpip.h:147:8: error: redefinition of 'struct ip_mreq'
 struct ip_mreq {
        ^
In file included from c:\mingw\include\windows.h:93:0,
                 from cpu-miner.c:23:
c:\mingw\include\winsock.h:315:8: note: originally defined here
 struct ip_mreq {
        ^
In file included from cpu-miner.c:39:0:
miner.h:9:21: fatal error: pthread.h: No such file or directory
 #include <pthread.h>
                     ^
compilation terminated.
make[2]: *** [minerd-cpu-miner.o] Error 1
make[2]: Leaving directory `/c/cpuminer-2.3.2'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/c/cpuminer-2.3.2'
make: *** [all] Error 2

noveske@xp /c/cpuminer-2.3.2

Any help would be great, thanks

Ihave the same issue. I'm trying to compile with static curl.
Please Help

thanks.

*** SOLVED ***

change code at line 22 of cpu-miner.c with:

#ifdef WIN32
#define WIN32_LEAN_AND_MEAN 1

now i compile withot errors

[Mr.Dark-Core]

Hi ! 

I'm Trying to Compile cpuminer on windows xp 32-bit 

i've downloaded "curl-7.35.0.tar.gz" and compiled it, copied "libcurl.m4" and "curl-config"

when i run this command in MSYS Shell:

LIBCURL="-lcurldll" ./configure CFLAGS="-O3"

It Gives me this Error: "configure: error: Missing required libcurl >= 7.15.2" 

i know that i hadn't installed the curl lib properly in my system that's why this error shows up 

can someone tell me the right way to install libcurl in windows ? ( Note: I'm a newbie with C/C++ and compiling and all related things ) 

Thanks in advance !   

configure script results:

Code:

checking build system type... i686-pc-mingw32
checking host system type... i686-w64-mingw32
checking target system type... i686-w64-mingw32
checking for a BSD-compatible install... /bin/install -c
checking whether build environment is sane... yes
checking for i686-w64-mingw32-strip... no
checking for strip... strip
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether to enable maintainer-specific portions of Makefiles... no
checking for style of include used by make... GNU
checking for i686-w64-mingw32-gcc... no
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.exe
checking for suffix of executables... .exe
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking dependency style of gcc... gcc3
checking for gcc option to accept ISO C99... -std=gnu99
checking how to run the C preprocessor... gcc -std=gnu99 -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking whether gcc -std=gnu99 needs -traditional... no
checking whether gcc -std=gnu99 and cc understand -c and -o together... yes
checking dependency style of gcc -std=gnu99... gcc3
checking for i686-w64-mingw32-ranlib... no
checking for ranlib... ranlib
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking sys/endian.h usability... no
checking sys/endian.h presence... no
checking for sys/endian.h... no
checking sys/param.h usability... yes
checking sys/param.h presence... yes
checking for sys/param.h... yes
checking syslog.h usability... no
checking syslog.h presence... no
checking for syslog.h... no
checking for sys/sysctl.h... no
checking whether be32dec is declared... no
checking whether le32dec is declared... no
checking whether be32enc is declared... no
checking whether le32enc is declared... no
checking for size_t... yes
checking for working alloca.h... no
checking for alloca... yes
checking for getopt_long... yes
checking whether we can compile AVX code... yes
checking whether we can compile XOP code... yes
checking whether we can compile AVX2 code... yes
checking for json_loads in -ljansson... no
checking for pthread_create in -lpthread... no
checking for pthread_create in -lpthreadGC2... no
checking for pthread_create in -lpthreadGC1... no
checking for pthread_create in -lpthreadGC... no
checking for gawk... (cached) gawk
checking for curl-config... /mingw/bin/curl-config
checking for the version of libcurl... 7.35.0
checking for libcurl >= version 7.15.2... yes
checking whether libcurl is usable... no
configure: error: Missing required libcurl >= 7.15.2

[darkfriend77]

hmmm .... any one an idear why I'm getting no ... hashrate?

on the pool it's working but in the log I see always 0.000

cpuminer-2.3.2-GC3355-win32