Bitcoin Forum
April 21, 2014, 09:38:32 AM *
News: Due to the OpenSSL heartbleed bug, changing your forum password is recommended.
 
   Home   Help Search Donate Login Register  
Pages: [1] 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49
  Print  
Author Topic: An (even more) optimized version of cpuminer  (Read 902432 times)
pooler
Hero Member
*****
Offline Offline

Activity: 527


View Profile

Ignore
December 19, 2011, 04:27:00 PM
 #1

Current Version: 2.3.3 (February 27, 2013) - Changelog
Source tarball: http://sourceforge.net/projects/cpuminer/files/pooler-cpuminer-2.3.3.tar.gz
Binaries for Windows:
http://sourceforge.net/projects/cpuminer/files/pooler-cpuminer-2.3.3-win32.zip (32-bit)
http://sourceforge.net/projects/cpuminer/files/pooler-cpuminer-2.3.3-win64.zip  (64-bit)
Binaries for Linux:
http://sourceforge.net/projects/cpuminer/files/pooler-cpuminer-2.3.3-linux-x86.tar.gz (x86)
http://sourceforge.net/projects/cpuminer/files/pooler-cpuminer-2.3.3-linux-x86_64.tar.gz (x86-64)
Binaries for Mac OS X: (courtesy of shawnp0wers and Tachikoma)
https://github.com/downloads/pooler/cpuminer/pooler-cpuminer-2.1.2-osx32.zip (32-bit, outdated)
http://sourceforge.net/projects/cpuminer/files/pooler-cpuminer-2.3.3-osx64.zip (64-bit)

SHA-256 Checksums
Code:
5eb61dfc0c965dfae65678d8ae5f1d2c99f781f3aba2136c1c7a4e0375cadfd7  pooler-cpuminer-2.3.3-win32.zip
8af141adfe7a2823a89fe5cc87f4da1c6be34f49a690b15e565d8b5c36e32045  pooler-cpuminer-2.3.3-win64.zip
b9ea3ae108786a4ad7031d7736a9f78463e00ff9f13927e2e8b18d2fcc9acab6  pooler-cpuminer-2.3.3-linux-x86.tar.gz
5222c2178402ca94d59c9884bcea5d449120a1a10116eca5e50d1b90688bd0af  pooler-cpuminer-2.3.3-linux-x86_64.tar.gz
9bbe99a797c585fe84d0d42106abe6e52b758fbfa3ca6409685b3f3a05263dd3  pooler-cpuminer-2.1.2-osx32.zip
6affb3f4c93d70180ce6474406162cfd39a0d9120ba70748847b504ab090a495  pooler-cpuminer-2.3.3-osx64.zip

Basic usage examples
Code:
$ ./minerd --url=http://myminingpool.com:9332 --userpass=my.worker:password
$ ./minerd --url=stratum+tcp://myminingpool.com:3333 --userpass=my.worker:password
For more information:
Code:
$ ./minerd --help

Building instructions
Installing dependencies for building on Debian, Ubuntu and other APT-based distros:
Code:
$ sudo apt-get install build-essential libcurl4-openssl-dev
Installing dependencies for building on Fedora, RHEL, CentOS and other yum-based distros:
Code:
$ sudo yum install gcc make curl-devel
Installing dependencies for building on OpenSUSE and other ZYpp-based distros:
Code:
$ sudo zypper in gcc make libcurl-devel
Recipe for building on Linux:
Code:
$ wget http://sourceforge.net/projects/cpuminer/files/pooler-cpuminer-2.3.3.tar.gz
$ tar xzf pooler-cpuminer-*.tar.gz
$ cd cpuminer-*
$ ./configure CFLAGS="-O3"
$ make

FAQ / Troubleshooting

Q: Should I call this miner "cpuminer" or "minerd"?
A: The software package is called "cpuminer". "minerd" ("miner daemon") is just the name of the executable file provided by the package.

Q: My antivirus flags the Windows binary as malware.
A: That's a known false positive. More information here.

Q: When running configure I get the error "C compiler cannot create executables".
A: Make sure you typed CFLAGS="-O3" with a big O, not with a zero.

Q: autogen.sh dies with "error: possibly undefined macro: AC_MSG_ERROR".
Q: configure chokes on something like "LIBCURL_CHECK_CONFIG(, 7.15.2, ,'".
A: Make sure you have installed the development package for libcurl. If you have and you're still getting the error when compiling from git, try compiling from tarball instead.

Q: I'm trying to connect to a Stratum server, but I get "HTTP request failed: Empty reply from server".
A: Make sure you specified the correct protocol in the server URL (stratum+tcp://).

Q: Is there any command-line option I can play with to make it mine faster?
A: No. The miner automatically picks the best settings for the CPU it is run on.

Q: What's the difference between the two available algorithms, scrypt and sha256d?
A: They are completely different proof-of-work algorithms. You must use scrypt for Litecoin, and you must use sha256d for Bitcoin. The default algorithm is scrypt, so for Bitcoin mining you have to specify --algo=sha256d.

Q: Will this miner use a lot of RAM when using the scrypt algorithm?
A: No, that's a GPU thing.

Q: How do I make the miner write its output to a file instead of printing it to the screen?
A: Just redirect the standard error stream to file:
Code:
minerd [OPTIONS] 2> myfile
You may also want to use the --quiet/-q option to disable the per-thread hashmeter.
On *nix, you probably also want to use the --background/-B option to fork in the background.


Original post (December 19, 2011) follows. Please note that most of the technical details are now outdated.

I have recently rewritten the heart of the scrypt hashing function used by the jgarzik/ArtForz cpuminer in assembly language, to see if this could bring some more speed. Apparently it did. Smiley
The source code is now available at GitHub:
https://github.com/pooler/cpuminer
The build process for Linux should be the same as before.

In the new code I tried to take full advantage of SSE2 instructions, which are available since the Pentium 4. Unfortunately, AMD's implementation of these instructions is not as fast as Intel's... well, ok, sadly it's nearly two times slower. For this reason, I had to write separate versions of the hashing functions. You don't need to worry about this, though, since the new function should be able to auto-detect your cpu and automatically select the best algorithm.

Long polling patch
This release also includes a new --timeout option that I originally added to solve a problem with long polling. Apparently the LP thread doesn't behave nicely under certain network conditions, as reported by various users. So, if you experienced high stale rates with the previous miner, you should definitely try out this new version.
Many thanks to SockPuppet, aka shawnp0wers, who helped me nail down the issue!

Some Technical Details
The current release includes four different implementations of the scrypt core, each one designed for a different hardware.
  • A fallback plain x86 version, to be used when SSE2 instructions are not available (Pentium III, Athlon XP and earlier processors).
  • A 32-bit version using SSE2, for use on the Pentium 4, Pentium M, Core, Atom, plus all 64-bit cpus running in a 32-bit OS.
  • A 64-bit version for Intel processors, i.e. Core 2, i3, i5, i7. This version can in most cases double the speed of the previous miner.
  • A 64-bit version for AMD processors, i.e. Athlon 64, Phenom, Sempron and the like. The speed increase here can range from 5% to 80%.
The first two versions only get compiled in the 32-bit miner, the last two only in the 64-bit miner. The miner uses the CPUID instruction to choose which version to use.

Compiler Flags
One cool aspect of assembly code is that users no more need to play with compiler flags to get the best performance. Configuring the build with just CFLAGS="-O3" is now more than enough to get efficient code. This also means that we no more need separate specialized binaries for Intel and AMD cpus. Just a 32-bit and a 64-bit version.

Personal Notes
Someone on IRC asked me why I am releasing this miner, instead of keeping it for myself or for my pool. Well, that's exactly the point. It is important for Litecoin that everybody has access to the most efficient mining software!
Someone might worry about the effect of this release on market prices, but consider this: if everybody starts using the new miner, the hash rate will go up, but so will difficulty, so nothing will ultimately change. I actually think this new miner will be very beneficial to Litecoin, because it should make mining easier for beginners (see compiler flags).
As crazy_rabbit wrote in another thread, one big plus of Litecoin is that everybody can participate. Well, consider this: now you can effectively mine on an Atom! Smiley

Alright folks... I hope you enjoy the performance boost. Consider this as my Christmas present to the community! Cheesy

BTC: 15MRTcUweNVJbhTyH5rq9aeSdyigFrskqE · LTC: LTCPooLqTK1SANSNeTR63GbGwabTKEkuS7
1398073112
Hero Member
*
Offline Offline

Posts: 1398073112

View Profile Personal Message (Offline)

Ignore
1398073112
Reply with quote  #2

1398073112
Report to moderator
Unbeatable Service & Product Support
Grab Your Miners at GAWMiners.com
Order Before April 25th to receive
Double your Hashing Power for 1 week!

Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1398073112
Hero Member
*
Offline Offline

Posts: 1398073112

View Profile Personal Message (Offline)

Ignore
1398073112
Reply with quote  #2

1398073112
Report to moderator
1398073112
Hero Member
*
Offline Offline

Posts: 1398073112

View Profile Personal Message (Offline)

Ignore
1398073112
Reply with quote  #2

1398073112
Report to moderator
1398073112
Hero Member
*
Offline Offline

Posts: 1398073112

View Profile Personal Message (Offline)

Ignore
1398073112
Reply with quote  #2

1398073112
Report to moderator
wknight
Hero Member
*****
Offline Offline

Activity: 845


Bitcoin calls me an Orphan


View Profile WWW

Ignore
December 19, 2011, 04:34:23 PM
 #2

This increase is amazing! I have doubled my speed on most of my servers. AMD noticed very slight increase.. but INTEL just rocks!

Hyperthreading on or off does not effect speeds

Mining Both Bitcoin and Litecoin.
btc_artist
Full Member
***
Offline Offline

Activity: 154


Bitcoin!


View Profile WWW

Ignore
December 19, 2011, 04:37:04 PM
 #3

I'll give this a try.

BTC: 1CDCLDBHbAzHyYUkk1wYHPYmrtDZNhk8zf
LTC: LMS7SqZJnqzxo76iDSEua33WCyYZdjaQoE
ThiagoCMC
Staff
Hero Member
*****
Offline Offline

Activity: 980


฿itcoin: Currency of Resistance!


View Profile WWW

Ignore
December 19, 2011, 04:56:25 PM
 #4

Hi!!

 For Intel CPU, what should be the CFLAGS options?!

 CFLAGS = -g -O3

 --

 And for AMD?

 CFLAGS = -mtune=amdfam10 -O3 -ffast-math -mabm -msse4a -pipe

Thanks!
Thiago

* Bitfication.com! Turn your fiat, into bits!
r3v3rs3
Newbie
*
Offline Offline

Activity: 22


View Profile

Ignore
December 19, 2011, 04:58:10 PM
 #5

Atom 330: 2 kH/s -> 4.8 kH/s

Nice work!
pooler
Hero Member
*****
Offline Offline

Activity: 527


View Profile

Ignore
December 19, 2011, 04:58:48 PM
 #6

Hi!!

 For Intel CPU, what should be the CFLAGS options?!

 CFLAGS = -g -O3

 --

 And for AMD?

 CFLAGS = -mtune=amdfam10 -O3 -ffast-math -mabm -msse4a -pipe

Thanks!
Thiago

Good news: you don't need to worry too much about CFLAGS.
Just use "-O3". gcc cannot optimize assembly code anyway.

BTC: 15MRTcUweNVJbhTyH5rq9aeSdyigFrskqE · LTC: LTCPooLqTK1SANSNeTR63GbGwabTKEkuS7
wknight
Hero Member
*****
Offline Offline

Activity: 845


Bitcoin calls me an Orphan


View Profile WWW

Ignore
December 19, 2011, 04:59:04 PM
 #7

Hi!!

 For Intel CPU, what should be the CFLAGS options?!

 CFLAGS = -g -O3

 --

 And for AMD?

 CFLAGS = -mtune=amdfam10 -O3 -ffast-math -mabm -msse4a -pipe

Thanks!
Thiago

I used CFLAGS="-O3 -Wall -msse2" ./configure for intel.. worked well

Mining Both Bitcoin and Litecoin.
SockPuppet
Newbie
*
Offline Offline

Activity: 21



View Profile

Ignore
December 19, 2011, 04:59:42 PM
 #8

My part was little more than that of a trained monkey, but I was happy to help troubleshoot.

One huge thing we hashed out just last night was the OSX compatibility stuff. I'm not sure if pooler has updated the github repo with the changes required for OSX compilation, but if nothing else I can post an OSX 10.6 compatible binary here.  (It works with 10.7 too, 10.4 requires a bit more work and will be available later...)

http://dl.dropbox.com/u/828037/minerd_for_OSX_10.6-7.zip
ThiagoCMC
Staff
Hero Member
*****
Offline Offline

Activity: 980


฿itcoin: Currency of Resistance!


View Profile WWW

Ignore
December 19, 2011, 04:59:52 PM
 #9

WOW!!!!  From 1.73 to 2.68 khash/s !!!!!!

* Bitfication.com! Turn your fiat, into bits!
g2x3k
Full Member
***
Offline Offline

Activity: 147


PooL-X.eu


View Profile WWW

Ignore
December 19, 2011, 05:09:52 PM
 #10

ahh good news Smiley my stock phenom 940 3ghz went from 3,05 to 3,25 kh/s

http://PooL-X.eu/ join the crew, bring your slaves
http://wallet.it.cx/ Instant LTC Wallet service
coblee
Donator
Hero Member
*
Offline Offline

Activity: 826


firstbits.com/1ce5j


View Profile WWW

Ignore
December 19, 2011, 05:11:00 PM
 #11

I'm trying to compile this for Mac and got this error:

Quote
gcc -DHAVE_CONFIG_H -I. -pthread -fno-strict-aliasing -I./compat/jansson    -O3 -Wall -msse2 -msse3 -msse4.1 -msse4.2 -msse4 -g -march=core2 -MT minerd-scrypt-x64.o -MD -MP -MF .deps/minerd-scrypt-x64.Tpo -c -o minerd-scrypt-x64.o `test -f 'scrypt-x64.S' || echo './'`scrypt-x64.S
scrypt-x64.S:131:Alignment too large: 15. assumed.
scrypt-x64.S:11:expecting operand before ','; got nothing
scrypt-x64.S:11:expecting operand before ','; got nothing
scrypt-x64.S:11:expecting operand before ','; got nothing
scrypt-x64.S:11:expecting operand before ','; got nothing
scrypt-x64.S:11:suffix or operands invalid for `rol'
scrypt-x64.S:11:suffix or operands invalid for `rol'

... snip ...

scrypt-x64.S:566:expecting operand before ','; got nothing
scrypt-x64.S:566:expecting operand before ','; got nothing
scrypt-x64.S:566:suffix or operands invalid for `pshufd'
scrypt-x64.S:566:suffix or operands invalid for `pshufd'
scrypt-x64.S:566:suffix or operands invalid for `pshufd'
make[2]: *** [minerd-scrypt-x64.o] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

Any ideas?

Litecoin: http://litecoin.org - LhK2kQwiaAvhjWY799cZvMyYwnQAcxkarr
Casascius physical bitcoins database: http://casascius.appspot.com/
ThiagoCMC
Staff
Hero Member
*****
Offline Offline

Activity: 980


฿itcoin: Currency of Resistance!


View Profile WWW

Ignore
December 19, 2011, 05:11:23 PM
 #12

WOW!!!!  From 1.73 to 2.68 khash/s !!!!!!

This is one of my Intel CPU...

The next is one of my AMD CPU (AM3):

from 2.85 to 3.27!!

Awesome work!!! I want to donate some litecoins to you pooler!!  :-D

* Bitfication.com! Turn your fiat, into bits!
pooler
Hero Member
*****
Offline Offline

Activity: 527


View Profile

Ignore
December 19, 2011, 05:26:08 PM
 #13

I'm trying to compile this for Mac and got this error:

Quote
gcc -DHAVE_CONFIG_H -I. -pthread -fno-strict-aliasing -I./compat/jansson    -O3 -Wall -msse2 -msse3 -msse4.1 -msse4.2 -msse4 -g -march=core2 -MT minerd-scrypt-x64.o -MD -MP -MF .deps/minerd-scrypt-x64.Tpo -c -o minerd-scrypt-x64.o `test -f 'scrypt-x64.S' || echo './'`scrypt-x64.S
scrypt-x64.S:131:Alignment too large: 15. assumed.
scrypt-x64.S:11:expecting operand before ','; got nothing
scrypt-x64.S:11:expecting operand before ','; got nothing
scrypt-x64.S:11:expecting operand before ','; got nothing
scrypt-x64.S:11:expecting operand before ','; got nothing
scrypt-x64.S:11:suffix or operands invalid for `rol'
scrypt-x64.S:11:suffix or operands invalid for `rol'

... snip ...

scrypt-x64.S:566:expecting operand before ','; got nothing
scrypt-x64.S:566:expecting operand before ','; got nothing
scrypt-x64.S:566:suffix or operands invalid for `pshufd'
scrypt-x64.S:566:suffix or operands invalid for `pshufd'
scrypt-x64.S:566:suffix or operands invalid for `pshufd'
make[2]: *** [minerd-scrypt-x64.o] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

Any ideas?

That's the same exact problem SockPuppet and I faced yesterday. I still don't know why but apparently the assembler available on MacOS doesn't like my macros.
We finally got it to compile by expanding all macros in the source, but don't ask me to do that again Smiley (ok, if you insist I can send you the temporary patched file.)
I will try to solve the issue with SockPuppet as soon as possible, I am really curious about where the problem actually lies.

BTC: 15MRTcUweNVJbhTyH5rq9aeSdyigFrskqE · LTC: LTCPooLqTK1SANSNeTR63GbGwabTKEkuS7
SockPuppet
Newbie
*
Offline Offline

Activity: 21



View Profile

Ignore
December 19, 2011, 05:27:54 PM
 #14

I'm trying to compile this for Mac and got this error:
Coblee, yeah -- those are some of the issues we worked through last night. It took hours, because pooler doesn't have a mac, and I don't have any assembly skillz. Smiley

The short version is, all macros need to be expanded (ie, eliminated), and then if he hasn't changed them, a few MOVQ ops need to be changed to the incorrect-but-still-works-and-makes-apple-happy MOVD. Smiley

The reason I'm still having problems with 10.4 is that we didn't do any work on scrypt-x86.S, and it has even more macros than the 64 bit one. Smiley

BTW: SockPuppet = trunkboy = shawnp0wers = Shawn Powers from Linux Journal, for those playing at home...
pooler
Hero Member
*****
Offline Offline

Activity: 527


View Profile

Ignore
December 19, 2011, 05:40:54 PM
 #15

Binaries for Windows now available, thanks diki!
https://github.com/downloads/pooler/cpuminer/pooler-cpuminer-win32.zip
https://github.com/downloads/pooler/cpuminer/pooler-cpuminer-win64.zip
(Please let me know if the packages miss any dynamic libraries.)

BTC: 15MRTcUweNVJbhTyH5rq9aeSdyigFrskqE · LTC: LTCPooLqTK1SANSNeTR63GbGwabTKEkuS7
ArtForz
Sr. Member
****
Offline Offline

Activity: 406


View Profile

Ignore
December 19, 2011, 05:45:20 PM
 #16

In case you're wondering why the SSE2 version sucks on K8 and K10 ... reason is rather simple.
the salsa20 function is a long string of data dependent 4*32-bit vector integer operations (i.e. output of one operation is used as input to the next).
And the execution latencies for the most used instructions in the salsa20 core (shift r/l immediate, add, xor) are all 2 clocks on K8/K10, all 1 clock on Atom/Core/Core2/Nehalem/SB.
End result ... sse2 salsa20 needs roughly twice the clocks/round on AMD compared to any modern intel.

bitcoin: 1Fb77Xq5ePFER8GtKRn2KDbDTVpJKfKmpz
i0coin: jNdvyvd6v6gV3kVJLD7HsB5ZwHyHwAkfdw
SockPuppet
Newbie
*
Offline Offline

Activity: 21



View Profile

Ignore
December 19, 2011, 05:47:04 PM
 #17

Binaries for Windows now available, thanks diki!
https://github.com/downloads/pooler/cpuminer/pooler-cpuminer-win32.zip
https://github.com/downloads/pooler/cpuminer/pooler-cpuminer-win64.zip
(Please let me know if the packages miss any dynamic libraries.)
Yeah, on Win7 64, I can't find libeay32.dll. I grabbed one from a version of OpenSSL, but it complains about missing functions, etc.
pooler
Hero Member
*****
Offline Offline

Activity: 527


View Profile

Ignore
December 19, 2011, 06:00:03 PM
 #18

Binaries for Windows now available, thanks diki!
https://github.com/downloads/pooler/cpuminer/pooler-cpuminer-win32.zip
https://github.com/downloads/pooler/cpuminer/pooler-cpuminer-win64.zip
(Please let me know if the packages miss any dynamic libraries.)
Yeah, on Win7 64, I can't find libeay32.dll. I grabbed one from a version of OpenSSL, but it complains about missing functions, etc.

Thank you for reporting this, I have added the DLL to the package.

BTC: 15MRTcUweNVJbhTyH5rq9aeSdyigFrskqE · LTC: LTCPooLqTK1SANSNeTR63GbGwabTKEkuS7
Schwede65
Sr. Member
****
Offline Offline

Activity: 300


View Profile

Ignore
December 19, 2011, 06:05:45 PM
 #19

Binaries for Windows now available, thanks diki!
https://github.com/downloads/pooler/cpuminer/pooler-cpuminer-win32.zip
https://github.com/downloads/pooler/cpuminer/pooler-cpuminer-win64.zip
(Please let me know if the packages miss any dynamic libraries.)
Yeah, on Win7 64, I can't find libeay32.dll. I grabbed one from a version of OpenSSL, but it complains about missing functions, etc.

the same is missing here W7-64...

w7-32 works very fine:
i7 920 / 7 threads from 1.5 KH/s/thread to 3.1 KH/s/thread

WOW - more then doubled
Graet
VIP
Hero Member
*
Offline Offline

Activity: 924



View Profile WWW

Ignore
December 19, 2011, 06:09:58 PM
 #20

Binaries for Windows now available, thanks diki!
https://github.com/downloads/pooler/cpuminer/pooler-cpuminer-win32.zip
https://github.com/downloads/pooler/cpuminer/pooler-cpuminer-win64.zip
(Please let me know if the packages miss any dynamic libraries.)
Yeah, on Win7 64, I can't find libeay32.dll. I grabbed one from a version of OpenSSL, but it complains about missing functions, etc.

Thank you for reporting this, I have added the DLL to the package.
thanks for the quick fix Smiley

| Ozcoin Pooled Mining Pty Ltd https://ozcoin.net Double Geometric Reward System https://lc.ozcoin.net for Litecoin mining DGM | https://WeExchange.co in AUD USD and CAD PM | https://crowncloud.net VPS and Dedicated Servers for the BTC community
Pages: [1] 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!