Bitcoin Forum
October 16, 2024, 06:31:18 AM *
News: Latest Bitcoin Core release: 28.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1] 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 ... 71 »
  Print  
Author Topic: An (even more) optimized version of cpuminer (pooler's cpuminer, CPU-only)  (Read 1958496 times)
pooler (OP)
Hero Member
*****
Offline Offline

Activity: 840
Merit: 507


View Profile
December 19, 2011, 04:27:00 PM
Last edit: June 26, 2020, 03:29:37 PM by pooler
Merited by OgNasty (2)
 #1

Current Version: 2.5.1 (Jun 25, 2020) - Changelog
Source tarball: https://github.com/pooler/cpuminer/releases/download/v2.5.1/pooler-cpuminer-2.5.1.tar.gz
Binaries for Windows:
https://github.com/pooler/cpuminer/releases/download/v2.5.1/pooler-cpuminer-2.5.1-win32.zip (32-bit)
https://github.com/pooler/cpuminer/releases/download/v2.5.1/pooler-cpuminer-2.5.1-win64.zip  (64-bit)
Binaries for Linux:
https://github.com/pooler/cpuminer/releases/download/v2.5.1/pooler-cpuminer-2.5.1-linux-x86_64.tar.gz (x86-64)
Binaries for macOS:
https://github.com/pooler/cpuminer/releases/download/v2.5.1/pooler-cpuminer-2.5.1-osx64.zip (64-bit)

SHA-256 Checksums
Code:
337f04fdb32f34b85819d09d59f6d3cf62991ac2e656735c43661dd3d4c57631  pooler-cpuminer-2.5.1.tar.gz
5fc7219fbb72dad32d64f11cd579383e53d8872f95309594fad2a07554a541f7  pooler-cpuminer-2.5.1-linux-x86_64.tar.gz
595ff086193ff68d8923f716757cf6ba6bbf1b761ae8786f437e2834621bebc8  pooler-cpuminer-2.5.1-osx64.zip
8d2c14b46bda27e2efc971907164f3e301253fd3a2d23f02201a1b3a2cbeb156  pooler-cpuminer-2.5.1-win32.zip
a33f6d637a71a69b21d95aa0237e9e5227da62b437e986d28d964fd5395ace37  pooler-cpuminer-2.5.1-win64.zip

Basic usage examples
Code:
$ ./minerd --url=http://myminingpool.com:9332 --userpass=my.worker:password
$ ./minerd --url=stratum+tcp://myminingpool.com:3333 --userpass=my.worker:password
For more information:
Code:
$ ./minerd --help

Building instructions
Installing dependencies for building on Debian, Ubuntu and other APT-based distros:
Code:
$ sudo apt-get install make libcurl4-openssl-dev
Installing dependencies for building on Fedora, RHEL, CentOS and other yum-based distros:
Code:
$ sudo yum install gcc make curl-devel
Installing dependencies for building on OpenSUSE and other ZYpp-based distros:
Code:
$ sudo zypper in gcc make libcurl-devel
Recipe for building on Linux:
Code:
$ wget https://github.com/pooler/cpuminer/releases/download/v2.5.1/pooler-cpuminer-2.5.1.tar.gz
$ tar xzf pooler-cpuminer-*.tar.gz
$ cd cpuminer-*
$ ./configure CFLAGS="-O3"
$ make

FAQ / Troubleshooting

Q: Should I call this miner "cpuminer" or "minerd"?
A: The software package is called "cpuminer". "minerd" ("miner daemon") is just the name of the executable file provided by the package.

Q: My antivirus flags the Windows binary as malware.
A: That's a known false positive. More information here.

Q: When I click on minerd.exe a black window flashes up and then disappears.
A: This is a command-line application, it has no graphical interface. You'll need to learn how to use the command line interface (CLI) of your operating system first.

Q: Can I mine (insert your cryptocoin here) with this miner?
A: Only if its proof-of-work algorithm is scrypt or SHA-256d. This miner does not currently support other algorithms such as Keccak, scrypt-jane, X11, etc. Forks of this project may provide additional algorithms, but I do not maintain them and they are not discussed here, so if you have questions about them please contact their authors.

Q: When running configure I get the error "C compiler cannot create executables".
A: Make sure you typed CFLAGS="-O3" with a big O, not with a zero.

Q: autogen.sh dies with "error: possibly undefined macro: AC_MSG_ERROR".
Q: configure chokes on something like "LIBCURL_CHECK_CONFIG(, 7.15.2, ,'".
A: Make sure you have installed the development package for libcurl. If you have and you're still getting the error when compiling from git, try compiling from tarball instead.

Q: I'm trying to connect to a Stratum server, but I get "HTTP request failed: Empty reply from server".
A: Make sure you specified the correct protocol in the server URL (stratum+tcp://).

Q: Is there any command-line option I can play with to make it mine faster?
A: No. The miner automatically picks the best settings for the CPU it is run on.

Q: What's the difference between the two algorithms, scrypt and sha256d?
A: They are completely different proof-of-work algorithms. You must use scrypt for Litecoin, and you must use sha256d for Bitcoin. The default algorithm is scrypt, so for Bitcoin mining you have to specify --algo=sha256d.

Q: Will this miner use a lot of RAM when using the scrypt algorithm?
A: No, that's a GPU thing.

Q: How do I make the miner write its output to a file instead of printing it to the screen?
A: Just redirect the standard error stream to file:
Code:
minerd [OPTIONS] 2> myfile
You may also want to use the --quiet/-q option to disable the per-thread hashmeter.
On *nix, you probably also want to use the --background/-B option to fork in the background.


Original post (December 19, 2011) follows. Please note that most of the technical details are now outdated.

I have recently rewritten the heart of the scrypt hashing function used by the jgarzik/ArtForz cpuminer in assembly language, to see if this could bring some more speed. Apparently it did. Smiley
The source code is now available at GitHub:
https://github.com/pooler/cpuminer
The build process for Linux should be the same as before.

In the new code I tried to take full advantage of SSE2 instructions, which are available since the Pentium 4. Unfortunately, AMD's implementation of these instructions is not as fast as Intel's... well, ok, sadly it's nearly two times slower. For this reason, I had to write separate versions of the hashing functions. You don't need to worry about this, though, since the new function should be able to auto-detect your cpu and automatically select the best algorithm.

Long polling patch
This release also includes a new --timeout option that I originally added to solve a problem with long polling. Apparently the LP thread doesn't behave nicely under certain network conditions, as reported by various users. So, if you experienced high stale rates with the previous miner, you should definitely try out this new version.
Many thanks to SockPuppet, aka shawnp0wers, who helped me nail down the issue!

Some Technical Details
The current release includes four different implementations of the scrypt core, each one designed for a different hardware.
  • A fallback plain x86 version, to be used when SSE2 instructions are not available (Pentium III, Athlon XP and earlier processors).
  • A 32-bit version using SSE2, for use on the Pentium 4, Pentium M, Core, Atom, plus all 64-bit cpus running in a 32-bit OS.
  • A 64-bit version for Intel processors, i.e. Core 2, i3, i5, i7. This version can in most cases double the speed of the previous miner.
  • A 64-bit version for AMD processors, i.e. Athlon 64, Phenom, Sempron and the like. The speed increase here can range from 5% to 80%.
The first two versions only get compiled in the 32-bit miner, the last two only in the 64-bit miner. The miner uses the CPUID instruction to choose which version to use.

Compiler Flags
One cool aspect of assembly code is that users no more need to play with compiler flags to get the best performance. Configuring the build with just CFLAGS="-O3" is now more than enough to get efficient code. This also means that we no more need separate specialized binaries for Intel and AMD cpus. Just a 32-bit and a 64-bit version.

Final Notes
Someone on IRC asked me why I am releasing this miner, instead of keeping it for myself or for my pool. Well, that's exactly the point. It is important for Litecoin that everybody has access to the most efficient mining software!
Someone might worry about the effect of this release on market prices, but consider this: if everybody starts using the new miner, the hash rate will go up, but so will difficulty, so nothing will ultimately change. I actually think this new miner will be very beneficial to Litecoin, because it should make mining easier for beginners (see compiler flags).
As crazy_rabbit wrote in another thread, one big plus of Litecoin is that everybody can participate. Well, consider this: now you can effectively mine on an Atom! Smiley

Alright folks... I hope you enjoy the performance boost. Consider this as my Christmas present to the community! Cheesy

BTC: 15MRTcUweNVJbhTyH5rq9aeSdyigFrskqE · LTC: LTCPooLqTK1SANSNeTR63GbGwabTKEkuS7
wknight
Legendary
*
Offline Offline

Activity: 889
Merit: 1000


Bitcoin calls me an Orphan


View Profile WWW
December 19, 2011, 04:34:23 PM
 #2

This increase is amazing! I have doubled my speed on most of my servers. AMD noticed very slight increase.. but INTEL just rocks!

Hyperthreading on or off does not effect speeds

Mining Both Bitcoin and Litecoin.
btc_artist
Full Member
***
Offline Offline

Activity: 154
Merit: 102

Bitcoin!


View Profile WWW
December 19, 2011, 04:37:04 PM
 #3

I'll give this a try.

BTC: 1CDCLDBHbAzHyYUkk1wYHPYmrtDZNhk8zf
LTC: LMS7SqZJnqzxo76iDSEua33WCyYZdjaQoE
ThiagoCMC
Legendary
*
Offline Offline

Activity: 1204
Merit: 1000

฿itcoin: Currency of Resistance!


View Profile
December 19, 2011, 04:56:25 PM
 #4

Hi!!

 For Intel CPU, what should be the CFLAGS options?!

 CFLAGS = -g -O3

 --

 And for AMD?

 CFLAGS = -mtune=amdfam10 -O3 -ffast-math -mabm -msse4a -pipe

Thanks!
Thiago
r3v3rs3
Newbie
*
Offline Offline

Activity: 22
Merit: 0


View Profile
December 19, 2011, 04:58:10 PM
 #5

Atom 330: 2 kH/s -> 4.8 kH/s

Nice work!
pooler (OP)
Hero Member
*****
Offline Offline

Activity: 840
Merit: 507


View Profile
December 19, 2011, 04:58:48 PM
 #6

Hi!!

 For Intel CPU, what should be the CFLAGS options?!

 CFLAGS = -g -O3

 --

 And for AMD?

 CFLAGS = -mtune=amdfam10 -O3 -ffast-math -mabm -msse4a -pipe

Thanks!
Thiago

Good news: you don't need to worry too much about CFLAGS.
Just use "-O3". gcc cannot optimize assembly code anyway.

BTC: 15MRTcUweNVJbhTyH5rq9aeSdyigFrskqE · LTC: LTCPooLqTK1SANSNeTR63GbGwabTKEkuS7
wknight
Legendary
*
Offline Offline

Activity: 889
Merit: 1000


Bitcoin calls me an Orphan


View Profile WWW
December 19, 2011, 04:59:04 PM
 #7

Hi!!

 For Intel CPU, what should be the CFLAGS options?!

 CFLAGS = -g -O3

 --

 And for AMD?

 CFLAGS = -mtune=amdfam10 -O3 -ffast-math -mabm -msse4a -pipe

Thanks!
Thiago

I used CFLAGS="-O3 -Wall -msse2" ./configure for intel.. worked well

Mining Both Bitcoin and Litecoin.
SockPuppet
Newbie
*
Offline Offline

Activity: 21
Merit: 0



View Profile
December 19, 2011, 04:59:42 PM
 #8

My part was little more than that of a trained monkey, but I was happy to help troubleshoot.

One huge thing we hashed out just last night was the OSX compatibility stuff. I'm not sure if pooler has updated the github repo with the changes required for OSX compilation, but if nothing else I can post an OSX 10.6 compatible binary here.  (It works with 10.7 too, 10.4 requires a bit more work and will be available later...)

http://dl.dropbox.com/u/828037/minerd_for_OSX_10.6-7.zip
ThiagoCMC
Legendary
*
Offline Offline

Activity: 1204
Merit: 1000

฿itcoin: Currency of Resistance!


View Profile
December 19, 2011, 04:59:52 PM
 #9

WOW!!!!  From 1.73 to 2.68 khash/s !!!!!!
g2x3k
Full Member
***
Offline Offline

Activity: 147
Merit: 100

PooL-X.eu


View Profile WWW
December 19, 2011, 05:09:52 PM
 #10

ahh good news Smiley my stock phenom 940 3ghz went from 3,05 to 3,25 kh/s

http://PooL-X.eu/ join the crew, bring your slaves
http://wallet.it.cx/ Instant LTC Wallet service
coblee
Donator
Legendary
*
Offline Offline

Activity: 1654
Merit: 1351


Creator of Litecoin. Cryptocurrency enthusiast.


View Profile
December 19, 2011, 05:11:00 PM
 #11

I'm trying to compile this for Mac and got this error:

Quote
gcc -DHAVE_CONFIG_H -I. -pthread -fno-strict-aliasing -I./compat/jansson    -O3 -Wall -msse2 -msse3 -msse4.1 -msse4.2 -msse4 -g -march=core2 -MT minerd-scrypt-x64.o -MD -MP -MF .deps/minerd-scrypt-x64.Tpo -c -o minerd-scrypt-x64.o `test -f 'scrypt-x64.S' || echo './'`scrypt-x64.S
scrypt-x64.S:131:Alignment too large: 15. assumed.
scrypt-x64.S:11:expecting operand before ','; got nothing
scrypt-x64.S:11:expecting operand before ','; got nothing
scrypt-x64.S:11:expecting operand before ','; got nothing
scrypt-x64.S:11:expecting operand before ','; got nothing
scrypt-x64.S:11:suffix or operands invalid for `rol'
scrypt-x64.S:11:suffix or operands invalid for `rol'

... snip ...

scrypt-x64.S:566:expecting operand before ','; got nothing
scrypt-x64.S:566:expecting operand before ','; got nothing
scrypt-x64.S:566:suffix or operands invalid for `pshufd'
scrypt-x64.S:566:suffix or operands invalid for `pshufd'
scrypt-x64.S:566:suffix or operands invalid for `pshufd'
make[2]: *** [minerd-scrypt-x64.o] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

Any ideas?

ThiagoCMC
Legendary
*
Offline Offline

Activity: 1204
Merit: 1000

฿itcoin: Currency of Resistance!


View Profile
December 19, 2011, 05:11:23 PM
 #12

WOW!!!!  From 1.73 to 2.68 khash/s !!!!!!

This is one of my Intel CPU...

The next is one of my AMD CPU (AM3):

from 2.85 to 3.27!!

Awesome work!!! I want to donate some litecoins to you pooler!!  :-D
pooler (OP)
Hero Member
*****
Offline Offline

Activity: 840
Merit: 507


View Profile
December 19, 2011, 05:26:08 PM
 #13

I'm trying to compile this for Mac and got this error:

Quote
gcc -DHAVE_CONFIG_H -I. -pthread -fno-strict-aliasing -I./compat/jansson    -O3 -Wall -msse2 -msse3 -msse4.1 -msse4.2 -msse4 -g -march=core2 -MT minerd-scrypt-x64.o -MD -MP -MF .deps/minerd-scrypt-x64.Tpo -c -o minerd-scrypt-x64.o `test -f 'scrypt-x64.S' || echo './'`scrypt-x64.S
scrypt-x64.S:131:Alignment too large: 15. assumed.
scrypt-x64.S:11:expecting operand before ','; got nothing
scrypt-x64.S:11:expecting operand before ','; got nothing
scrypt-x64.S:11:expecting operand before ','; got nothing
scrypt-x64.S:11:expecting operand before ','; got nothing
scrypt-x64.S:11:suffix or operands invalid for `rol'
scrypt-x64.S:11:suffix or operands invalid for `rol'

... snip ...

scrypt-x64.S:566:expecting operand before ','; got nothing
scrypt-x64.S:566:expecting operand before ','; got nothing
scrypt-x64.S:566:suffix or operands invalid for `pshufd'
scrypt-x64.S:566:suffix or operands invalid for `pshufd'
scrypt-x64.S:566:suffix or operands invalid for `pshufd'
make[2]: *** [minerd-scrypt-x64.o] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

Any ideas?

That's the same exact problem SockPuppet and I faced yesterday. I still don't know why but apparently the assembler available on MacOS doesn't like my macros.
We finally got it to compile by expanding all macros in the source, but don't ask me to do that again Smiley (ok, if you insist I can send you the temporary patched file.)
I will try to solve the issue with SockPuppet as soon as possible, I am really curious about where the problem actually lies.

BTC: 15MRTcUweNVJbhTyH5rq9aeSdyigFrskqE · LTC: LTCPooLqTK1SANSNeTR63GbGwabTKEkuS7
SockPuppet
Newbie
*
Offline Offline

Activity: 21
Merit: 0



View Profile
December 19, 2011, 05:27:54 PM
 #14

I'm trying to compile this for Mac and got this error:
Coblee, yeah -- those are some of the issues we worked through last night. It took hours, because pooler doesn't have a mac, and I don't have any assembly skillz. Smiley

The short version is, all macros need to be expanded (ie, eliminated), and then if he hasn't changed them, a few MOVQ ops need to be changed to the incorrect-but-still-works-and-makes-apple-happy MOVD. Smiley

The reason I'm still having problems with 10.4 is that we didn't do any work on scrypt-x86.S, and it has even more macros than the 64 bit one. Smiley

BTW: SockPuppet = trunkboy = shawnp0wers = Shawn Powers from Linux Journal, for those playing at home...
pooler (OP)
Hero Member
*****
Offline Offline

Activity: 840
Merit: 507


View Profile
December 19, 2011, 05:40:54 PM
 #15

Binaries for Windows now available, thanks diki!
https://github.com/downloads/pooler/cpuminer/pooler-cpuminer-win32.zip
https://github.com/downloads/pooler/cpuminer/pooler-cpuminer-win64.zip
(Please let me know if the packages miss any dynamic libraries.)

BTC: 15MRTcUweNVJbhTyH5rq9aeSdyigFrskqE · LTC: LTCPooLqTK1SANSNeTR63GbGwabTKEkuS7
ArtForz
Sr. Member
****
Offline Offline

Activity: 406
Merit: 257


View Profile
December 19, 2011, 05:45:20 PM
 #16

In case you're wondering why the SSE2 version sucks on K8 and K10 ... reason is rather simple.
the salsa20 function is a long string of data dependent 4*32-bit vector integer operations (i.e. output of one operation is used as input to the next).
And the execution latencies for the most used instructions in the salsa20 core (shift r/l immediate, add, xor) are all 2 clocks on K8/K10, all 1 clock on Atom/Core/Core2/Nehalem/SB.
End result ... sse2 salsa20 needs roughly twice the clocks/round on AMD compared to any modern intel.

bitcoin: 1Fb77Xq5ePFER8GtKRn2KDbDTVpJKfKmpz
i0coin: jNdvyvd6v6gV3kVJLD7HsB5ZwHyHwAkfdw
SockPuppet
Newbie
*
Offline Offline

Activity: 21
Merit: 0



View Profile
December 19, 2011, 05:47:04 PM
 #17

Binaries for Windows now available, thanks diki!
https://github.com/downloads/pooler/cpuminer/pooler-cpuminer-win32.zip
https://github.com/downloads/pooler/cpuminer/pooler-cpuminer-win64.zip
(Please let me know if the packages miss any dynamic libraries.)
Yeah, on Win7 64, I can't find libeay32.dll. I grabbed one from a version of OpenSSL, but it complains about missing functions, etc.
pooler (OP)
Hero Member
*****
Offline Offline

Activity: 840
Merit: 507


View Profile
December 19, 2011, 06:00:03 PM
 #18

Binaries for Windows now available, thanks diki!
https://github.com/downloads/pooler/cpuminer/pooler-cpuminer-win32.zip
https://github.com/downloads/pooler/cpuminer/pooler-cpuminer-win64.zip
(Please let me know if the packages miss any dynamic libraries.)
Yeah, on Win7 64, I can't find libeay32.dll. I grabbed one from a version of OpenSSL, but it complains about missing functions, etc.

Thank you for reporting this, I have added the DLL to the package.

BTC: 15MRTcUweNVJbhTyH5rq9aeSdyigFrskqE · LTC: LTCPooLqTK1SANSNeTR63GbGwabTKEkuS7
Schwede65
Sr. Member
****
Offline Offline

Activity: 309
Merit: 250


View Profile
December 19, 2011, 06:05:45 PM
 #19

Binaries for Windows now available, thanks diki!
https://github.com/downloads/pooler/cpuminer/pooler-cpuminer-win32.zip
https://github.com/downloads/pooler/cpuminer/pooler-cpuminer-win64.zip
(Please let me know if the packages miss any dynamic libraries.)
Yeah, on Win7 64, I can't find libeay32.dll. I grabbed one from a version of OpenSSL, but it complains about missing functions, etc.

the same is missing here W7-64...

w7-32 works very fine:
i7 920 / 7 threads from 1.5 KH/s/thread to 3.1 KH/s/thread

WOW - more then doubled
Graet
VIP
Legendary
*
Offline Offline

Activity: 980
Merit: 1001



View Profile WWW
December 19, 2011, 06:09:58 PM
Last edit: December 19, 2011, 08:35:42 PM by Graet
 #20

Binaries for Windows now available, thanks diki!
https://github.com/downloads/pooler/cpuminer/pooler-cpuminer-win32.zip
https://github.com/downloads/pooler/cpuminer/pooler-cpuminer-win64.zip
(Please let me know if the packages miss any dynamic libraries.)
Yeah, on Win7 64, I can't find libeay32.dll. I grabbed one from a version of OpenSSL, but it complains about missing functions, etc.

Thank you for reporting this, I have added the DLL to the package.
thanks for the quick fix Smiley

| Ozcoin Pooled Mining Pty Ltd https://ozcoin.net Double Geometric Reward System https://lc.ozcoin.net for Litecoin mining DGM| https://crowncloud.net VPS and Dedicated Servers for the BTC community
Pages: [1] 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 ... 71 »
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!