[LOCKED] cpuminer-opt v3.12.3, open source optimized multi-algo CPU miner

[joblo]

This is the home of cpuminer-opt, the optimized CPU miner.

This thread is locked. all discussion of cpuminer-opt should now take place
in the new thread started by my alter ego JayDDee.

https://bitcointalk.org/index.php?topic=5226770.msg53865575#msg53865575

----------------------------------------------------------

Supporting over 90 agorithms with many optimized for CPUs with the latest technologies:

AMD Ryzen: AVX2, SHA
Intel Skylake X: AVX512
Intel Icelake: AVX512, SHA, and VAES  (source code only at this time)

Older CPUs with SSE2 are also supported.

Unique features

Block and share time estimates.

True hashrate reporting with comparison with actual share submission rate.

Download

Source code and Windows binaries can be downloaded from git.

https://github.com/JayDDee/cpuminer-opt/releases

Download only from the official JayDDee github repository. There is malware masquerading
as cpuminer-opt even with fake ANN posts that look just like this one.

New in v3.12.3

Issue #238: Fixed skunk AVX2.

Issue #239: Faster AVX2 & AVX512 for skein +44%, skein2 +30%, plus marginal
increases for skunk, x16r, x16rv2, x16rt, x16rt-veil, x16s, x21s.

Faster anime VAES +57%, AVX512 +21%, AVX2 +3%.

Redesigned code reponsible for issue #236.


New in v3.12.2

Fixed xevan, skein, skein2 AVX2, #238.

Reversed polarity of AVX2 vector bit test utilities, and all users, to be
logically and semantically correct. Follow up to issue #236.

New in v3.12.1

Fixed anime AVX2 low difficulty shares, git issue #236.

Periodic summary now reports lost hash rate due to rejected and stale shares,
displayed only when non-zero.

New in v3.12.0.1

Fixed hodl rejects, git issue #237.

Fixed debug code added in v3.12.0 to work with AVX2 to be enabled only
after low difficulty share have been seen to avoid unnecessarily excessive
log outout.

Added more digits of precision to diff in log output to help diagnose
low difficulty shares.

New in v3.12.0

Faster phi2 AVX2 +62%, AVX512 +150% on Intel CPUs. AMD Ryzen AVX2 is
YMMV due to its inferiour AVX2 implementation.

Fixed Hodl stats, rejects are still an issue since v3.9.5, git issue #237.

API can now be enabled with "-b port" or "--api-bind port".
It will use the default address 127.0.0.1.

Editorial: Short form options should only be used on the command line to save
typing. Configuration files and scripts should always use the long form
"--api-bind addr:port" without relying on any defaults. This is a general
recommendation that applies to all options for any application.

Removed obsolete cryptonight, all variants, and supporting code for more
size reduction and faster compiling.

Tweaked the timing of the CPU temperature and frequency log (Linux only).

Added some debug code to collect more info about low difficulty rejects,
git issue #236.

Requirements:

1. A x86_64 architecture CPU with a minimum of SSE2 support. This includes Intel Core2 and newer and AMD equivalents.
In order to take advantage of AES_NI optimizations a CPU with AES_NI is required. This includes Intel Westmere and newer
and AMD equivalents. Further optimizations are available on some algos for CPUs with AVX and AVX2, Sandybridge and
Haswell respectively.

32 bit CPUs and ARM CPUs are not supported.

2. 64 bit Linux OS. Ubuntu and Fedora based distributions, including Mint and Centos are known to work and have all
dependencies in their repositories. Others may work but may require more effort.

64 bit Windows OS is supported using the pre-compiled binaries package or may be compiled with mingw_w64 and msys.

Apple and Android operating systems are not supported. FreeBSD YMMV.

Older CPUs, other architectures and operating systems may be supported by TPruvot's cpuminer-multi.

Security warning

Miner programs are often flagged as malware by antivirus programs. This is usually a false positive, they are
flagged simply because they are cryptocurrency miners. However, some malware has been spread using the
cover that miners are known to be subject to false positives. Always be on alert. The source code of cpuminer-opt
is open for anyone to inspect. If you don't trust the software don't download it.

Some cryptographic code has been taken from trusted sources but has been modified for speed at the expense of
accepted security practices. This code should not be imported into applications where secure cryptography is
required.

Errata:

Old algorithms that are rarely used anymore will not get the latest optimizations.

Cryptonight and variants are no longer supported.

Neoscrypt crashes on Windows, use legacy version.

Hodl requires a CPU with AES, use legacy version if without.

Benchmark testing does not work on several algos.

Donations

cpuminer-opt has no fees of any kind but donations are accepted.

BTC: 12tdvfF7KmAsihBXQXynT6E6th2c2pByTT

[kama]

where can we see the sourcecode and download windows binaries ?
also advice from me compile the miner with mingw it makes the miner software more powerfull.

[joblo]

where can we see the sourcecode and download windows binaries ?
also advice from me compile the miner with mingw it makes the miner software more powerfull.

Thanks kama. The project is just getting off the ground officially, nothing is yet available publicly
or privately.

Source code will be available. I compile Windows with VS and am not inclined to change. But as
I editted in my first post Windows is not guaranteed in the first release. It Depend on how things go.

[sp_]

You should add it to github.

good work

[joblo]

You should add it to github.

good work

Thanks SP. Now if only I could code in cuda.

[zTheWolfz]

Just checking in. Thread is looking good. I looked threw my collection of CPU miners and everything is pretty old and outdated.
Except the ones you talked about earlier. I'll be reading along and will be ready for this when the time is right, thanks for your support.
I'll have to brush up my Linux skills as most of what I have running at the moment is windows based.

[joblo]

Just checking in. Thread is looking good. I looked threw my collection of CPU miners and everything is pretty old and outdated.
Except the ones you talked about earlier. I'll be reading along and will be ready for this when the time is right, thanks for your support.
I'll have to brush up my Linux skills as most of what I have running at the moment is windows based.

I'm starting to work on compiling on Windows, if it goes well it should b be ready for private beta in a couple of days.

[GKar]

Looking forward to the cpu miner. 
 
I'm willing to help test it.   I've some i-7, I-3, I-5, core 2 computers.
Along with some older various single core Intel & AMD I have sitting here gathering dust.

Also have a black box AMD FX=8320E 8-core.

Algos I'd like to see are : neos, spread X11, Ethereum and the algo XMG coin uses.

--GKar (former nic was Angora)

[joblo]

Looking forward to the cpu miner. 
 
I'm willing to help test it.   I've some i-7, I-3, I-5, core 2 computers.
Along with some older various single core Intel & AMD I have sitting here gathering dust.

Also have a black box AMD FX=8320E 8-core.

Algos I'd like to see are : neos, spread X11, Ethereum and the algo XMG coin uses.

--GKar (former nic was Angora)

I'm comfortable with the testers I have now but thanks for the offer. You shouldn't have to
wait very long for the public release.

You might want to talk to epsylon3 (tpruvot) about new algos, but I think he is more focussed on
his ccminer fork and yiimp pool (busy guy) at the moment.

[theLosers106]

This is interesting. Been trying to make the cpuminers faster myself but never got much done. To busy with other things. If you need help compiling with mingw I could help.

Good luck 

theLosers106.

[joblo]

I'm having some difficulty compiling on Windows with VS community 2015.

If there is anyone who can help it will speed up Windows support.

I editted  cpuminer.vcxprog with a text editor, not VS's built in editor.
I modelled the changes based on the sources list in Makefile.am,
but I really have no idea what I'm doing.

I added a bunch of entries to this list:

Code:

     <Optimization Condition="'$(Configuration)'=='Release'">Full</Optimization>
    </ClCompile>
    <ClCompile Include="algo\sibcoin.c" />
    <ClCompile Include="algo\skein.c" />
    <ClCompile Include="algo\skein2.c" />
    <ClCompile Include="algo\x11.c" />
    <ClCompile Include="algo\x11_aes.c" />
    <ClCompile Include="algo\x13.c" />
    <ClCompile Include="algo\x13_aes.c" />
    <ClCompile Include="algo\aes-ni\echo512\hash.c" />
    <ClCompile Include="algo\aes-ni\groestl\hash-groestl.c" />
    <ClCompile Include="algo\x14.c" />
    <ClCompile Include="algo\x14_aes.c" />
    <ClCompile Include="algo\x15.c" />
    <ClCompile Include="algo\x15_aes.c" />
    <ClCompile Include="algo\zr5.c" />
    <ClCompile Include="sha3\aes_helper.c">
    <ClCompile Include="algo\sse2x6\bmw.c" />               <------- error here
    <ClCompile Include="algo\sse2\keccak.c" />
    <ClCompile Include="algo\sse2\skein.c" />
    <ClCompile Include="algo\sse2\echo.c" />

I get the following error.: The attribute "Include" in element  </ClCompile> is unrecognized. I've shuffled
the order of the list and it errors on a different file everytime but the same line number.

Any ideas welcome.
    

[theLosers106]

I'm having some difficulty compiling on Windows with VS community 2015.

If there is anyone who can help it will speed up Windows support.

I editted  cpuminer.vcxprog with a text editor, not VS's built in editor.
I modelled the changes based on the sources list in Makefile.am,
but I really have no idea what I'm doing.

I added a bunch of entries to this list:

Code:

     <Optimization Condition="'$(Configuration)'=='Release'">Full</Optimization>
    </ClCompile>
    <ClCompile Include="algo\sibcoin.c" />
    <ClCompile Include="algo\skein.c" />
    <ClCompile Include="algo\skein2.c" />
    <ClCompile Include="algo\x11.c" />
    <ClCompile Include="algo\x11_aes.c" />
    <ClCompile Include="algo\x13.c" />
    <ClCompile Include="algo\x13_aes.c" />
    <ClCompile Include="algo\aes-ni\echo512\hash.c" />
    <ClCompile Include="algo\aes-ni\groestl\hash-groestl.c" />
    <ClCompile Include="algo\x14.c" />
    <ClCompile Include="algo\x14_aes.c" />
    <ClCompile Include="algo\x15.c" />
    <ClCompile Include="algo\x15_aes.c" />
    <ClCompile Include="algo\zr5.c" />
    <ClCompile Include="sha3\aes_helper.c">
    <ClCompile Include="algo\sse2x6\bmw.c" />               <------- error here
    <ClCompile Include="algo\sse2\keccak.c" />
    <ClCompile Include="algo\sse2\skein.c" />
    <ClCompile Include="algo\sse2\echo.c" />

I get the following error.: The attribute "Include" in element  </ClCompile> is unrecognized. I've shuffled
the order of the list and it errors on a different file everytime but the same line number.

Any ideas welcome.
    

Just wondering if the folder's name sse2x6 correct?

[joblo]

I'm having some difficulty compiling on Windows with VS community 2015.

If there is anyone who can help it will speed up Windows support.

I editted  cpuminer.vcxprog with a text editor, not VS's built in editor.
I modelled the changes based on the sources list in Makefile.am,
but I really have no idea what I'm doing.

I added a bunch of entries to this list:

Code:

     <Optimization Condition="'$(Configuration)'=='Release'">Full</Optimization>
    </ClCompile>
    <ClCompile Include="algo\sibcoin.c" />
    <ClCompile Include="algo\skein.c" />
    <ClCompile Include="algo\skein2.c" />
    <ClCompile Include="algo\x11.c" />
    <ClCompile Include="algo\x11_aes.c" />
    <ClCompile Include="algo\x13.c" />
    <ClCompile Include="algo\x13_aes.c" />
    <ClCompile Include="algo\aes-ni\echo512\hash.c" />
    <ClCompile Include="algo\aes-ni\groestl\hash-groestl.c" />
    <ClCompile Include="algo\x14.c" />
    <ClCompile Include="algo\x14_aes.c" />
    <ClCompile Include="algo\x15.c" />
    <ClCompile Include="algo\x15_aes.c" />
    <ClCompile Include="algo\zr5.c" />
    <ClCompile Include="sha3\aes_helper.c">
    <ClCompile Include="algo\sse2x6\bmw.c" />               <------- error here
    <ClCompile Include="algo\sse2\keccak.c" />
    <ClCompile Include="algo\sse2\skein.c" />
    <ClCompile Include="algo\sse2\echo.c" />

I get the following error.: The attribute "Include" in element  </ClCompile> is unrecognized. I've shuffled
the order of the list and it errors on a different file everytime but the same line number.

Any ideas welcome.
    

Just wondering if the folder's name sse2x6 correct?

It's not the first time I've been blind, many thanks.

Update: No joy. I still get the same error. I was suspicious about th etypo becase I did double check everything,
my editor has been acting up so I may have messed up the line without realizing. I confirmed with the VS editor
that it was using the right file, the line looks just like every other (asfter deleting "x6" and I confirmed the file is present.

Windows support may have to wait.

[joblo]

I'm looking for opinions on how to handle incompatible CPUs, during the test phase and then the first release

There are two parts to this issue. First is whether the CPU has AES_NI and the second is whether
cpuminer is compiled for that architecture. it's a biggger issue on Windows where the builds are
portable. I hope MS builds to support multiple architectures and select the best one at run time.
When multiple architectures are supported in cpuminer it is my intention to do exactly that.

But for the time being it only support ASES_NI so...

For the first part I'm considering 4 options.

1. If the CPU doesn't support AES_NI exit.
      probably most likely for the release

2. Damn the torpedos full speed ahead
      My preferred choice during testing

3. Add a comand line prompt to choose 1 or 2
        will become obsolete when more architectures are supported
        would probably delay the test release

4. Add a user prompt at startup
       This would require the work in 3 as well additional work to add a yes/no prompt and
       expand the new command line option for 3 choices if an incompatibility is found: Exit, Continue,
       or Prompt(default). This has all the cons of 3 but more so.

I'm not interested in poll, I prefer the reasons for the choice. And of course the beta testers can get what
they want.

[Elun]

I'm having some difficulty compiling on Windows with VS community 2015.

If there is anyone who can help it will speed up Windows support.

I editted  cpuminer.vcxprog with a text editor, not VS's built in editor.
I modelled the changes based on the sources list in Makefile.am,
but I really have no idea what I'm doing.

I added a bunch of entries to this list:

Code:

     <Optimization Condition="'$(Configuration)'=='Release'">Full</Optimization>
    </ClCompile>
    <ClCompile Include="algo\sibcoin.c" />
    <ClCompile Include="algo\skein.c" />
    <ClCompile Include="algo\skein2.c" />
    <ClCompile Include="algo\x11.c" />
    <ClCompile Include="algo\x11_aes.c" />
    <ClCompile Include="algo\x13.c" />
    <ClCompile Include="algo\x13_aes.c" />
    <ClCompile Include="algo\aes-ni\echo512\hash.c" />
    <ClCompile Include="algo\aes-ni\groestl\hash-groestl.c" />
    <ClCompile Include="algo\x14.c" />
    <ClCompile Include="algo\x14_aes.c" />
    <ClCompile Include="algo\x15.c" />
    <ClCompile Include="algo\x15_aes.c" />
    <ClCompile Include="algo\zr5.c" />
    <ClCompile Include="sha3\aes_helper.c">
    <ClCompile Include="algo\sse2x6\bmw.c" />               <------- error here
    <ClCompile Include="algo\sse2\keccak.c" />
    <ClCompile Include="algo\sse2\skein.c" />
    <ClCompile Include="algo\sse2\echo.c" />

I get the following error.: The attribute "Include" in element  </ClCompile> is unrecognized. I've shuffled
the order of the list and it errors on a different file everytime but the same line number.

Any ideas welcome.
    

Just wondering if the folder's name sse2x6 correct?

It's not the first time I've been blind, many thanks.

Update: No joy. I still get the same error. I was suspicious about th etypo becase I did double check everything,
my editor has been acting up so I may have messed up the line without realizing. I confirmed with the VS editor
that it was using the right file, the line looks just like every other (asfter deleting "x6" and I confirmed the file is present.

Windows support may have to wait.

Look for example at this line:
<ClCompile Include="algo\zr5.c" />
and this line one more time:
<ClCompile Include="sha3\aes_helper.c">

Looks like you forgot self closing "/" before ">" in line with "sha3\aes_helper.c"

[joblo]

AES function pointer - call CPUID, one AND, and one compare against zero. If true, function pointer = AES-NI one, else slow AES.

Hi Wolf0, thanks for contributing.

I've got the capabilities check working has_aes-ni() already existed and I wrote has_sse2. I'm now trying to get cpuid working to see
if I can display the actual cpu model.

[joblo]

Look for example at this line:
<ClCompile Include="algo\zr5.c" />
and this line one more time:
<ClCompile Include="sha3\aes_helper.c">

Looks like you forgot self closing "/" before ">" in line with "sha3\aes_helper.c"

There was a lot wrong with that file, some of it was me and some of it was editpad++.
I'm starting fresh.

Edit: It's loaded now.  I made minimal edits and avoided moving things around. I should have known
better than to get too adventurous with MS. Thanks again for your keen eye.

Edit2: 479 errors on first compile. An this is code that compiles flawlessly on Linux. Obviously Windows
support is out for the beta. Maybe I'll get it working in time for the release.

[joblo]

While waiting until I downgrade to VS 2013 let me indulge myself as I ramble on about who I am.

I'm a 58 year old computer geek who was laid off with a good severence package 8 years ago less than
a year before the company went bankrupt.

My education is somewhat limited, just a 3 year computer science diploma. In my final year in college
I and 2 other team members wrote an assembler for a microprosessor board that didn't have one.
It had no keyboard or display interface just an rs-232 serial port. So we had to write the assembler
twice on two different processor architectures. My teammate had an Apple 2e and it could connect
to the serial port and talk to the monitor program. It also had some rudimantary file transfer capability
so off we went. We got it done on tie with only one real all-nighter which was more by choice than
necessity. We were among the few that attended class or stayed awake in class the last week before
the formal project demonstration week. We actually had a live demo where we uploaded source code
to the board comlied it and executed it. And it worked. For a bit of humour we even deliberately
left a coding error so we could show the eror handling capability. They we all three graduated and
ended up at the same company, although at different times and in different departments. The same
company that went bankrupt after giving me a golden parachute. Ifeel sorry for the people layed off
after me because the company started cutting back on the packages. Then they declared bankruptcy
and all severance packages were taken off the table. They even reduced pension benefits to past
employees A real shame.

The proprietary system I worked on used Motorola 68k, 88k and powerpc cpus. I was most familiar with
the unpopular 88k. My claim to fame was discovering a bug in the branch prediction which ironically
improved performance once it was disabled. Apparently the BP conflicted with a compiler optimization
even if it had been working properly. The 88k, Moto's first venture into RISC, however, was short lived.

The software system was an integrated development system consisting of a run time real time operating system,
it's applications, a Linux based (originally IBM mainframe) multi-site program and documentation library,
and compiler all written in and for a proprietary high level language with similarities (ie strict type checking) with Pascal.
I really learned to love the language and it has some great features, with OO added later. One of my favorite quirks
in the language was the guzinta: "->". This is unlike similar c++ operator. It is the assigment operator
and works like this:  "value -> variable;".  Some of your may recognize it.

it had some important features like not defining the null pointer as 0. nul always pointed off to invalid address high
up in the address space. Address 0 was also invalid and would throw an exception if accessed. The languange
also had array index protection built into the compiler. An index out of range also would throw an exception.
There was also a built in data structure (nice to own your own compiler) called a descriptor that was essentially
a pointer to an array. It also had index protection. c++ probably has a class similar to the descriptor but without
index protection. That feature alone made the system so much more resilient to memory corruption and easier
to debug, but most importantly the concept of a buffer overflow did not exist. The compiler would not allow it.
Imagine how many fewer exploits there would have been had c/c++ had such protection (at a slight performance
cost).

I've never done any Linux or c++ development although I had some exposure to c++ at work. Many of the concepts
are so different that what I am familiar with and that slows me down a lot. I've used HP-UX, solaris and linux as
desktop workstations at work and used linux at home since Redhat 5.2.

Although I'm comfortable with assembly code I haven't touched Intel since the 8000 series. That will make the assembly
language files tough to work with. I have a couple of optimization thechniques I've used in the past but they require
intimate knowledge of the Intel architecture such as memory interface, cache organization, execution environment,
instruction issue and retirement throughput, and things I'm probably not aware exist. I have a lot to learn, but that's why
I'm here.

I know nothing about cryptographic algorithms so don't expect me to code new algos. I'll leave that up to others.

I had heard of Bitcoin but didn't get involved until very late (too late) in the darkcoin frenzy in April 2014. I started
with just one CPU but unknown to me at the time ASICS had taken over bitcoin and scrypt was next. And GPUs were taking
over the altcoin algos. I bought a gt730, a nice little card that performed on par with my i7-4790K. I bought a 750ti shortly
after that and a few other cards since then.

Well that's how I got here. I'm bored with lots of time on my hands so I've been poking around at miner code here and there.
I'm even less familiar so I took a shot with a cpuminer and was surprised. It is a credit to Pooler's design that I was able to
pull together code from 3 different miners and make it work smoothly and efficiently in a short time.

VS is still installing but I think it's time for a break.

I must be getting tired I just tried to post this to the wrong thread. I giess i don't have the stamina I did 25 years ago.

Thanks for reading and taking an interest in my project.

[theLosers106]

While waiting until I downgrade to VS 2013 let me indulge myself as I ramble on about who I am.

I'm a 58 year old computer geek who was laid off with a good severence package 8 years ago less than
a year before the company went bankrupt.

The proprietary system I worked on used Motorola 68k, 88k and powerpc cpus. I was most familiar with
the unpopular 88k. My claim to fame was discovering a bug in the branch prediction which ironically
improved performance once it was disabled. Apparently the BP conflicted with a compiler optimization
even if it had been working properly. The 88k, Moto's first venture into RISC, however, was short lived.

The software system was an integrated development system consisting of a run time real time operating system,
it's applications, a Linux based (originally IBM mainframe) multi-site program and documentation library,
and compiler all written in and for a proprietary high level language with similarities (ie strict type checking) with Pascal.
I really learned to love the language and it has some great features, with OO added later. One of my favorite quirks
in the language was the guzinta: "->". This is unlike similar c++ operator. It is the assigment operator
and works like this:  "value -> variable;".  Some of your may recognize it.

it had some important features like not defining the null pointer as 0. nul always pointed off to invalid address high
up in the address space. Address 0 was also invalid and would throw an exception if accessed. The languange
also had array index protection built into the compiler. An index out of range also would throw an exception.
There was also a built in data structure (nice to own your own compiler) called a descriptor that was essentially
a pointer to an array. It also had index protection. c++ probably has a class similar to the descriptor but without
index protection. That feature alone made the system so much more resilient to memory corruption and easier
to debug, but most importantly the concept of a buffer overflow did not exist. The compiler would not allow it.
Imagine how many fewer exploits there would have been had c/c++ had such protection (at a slight performance
cost).

I've never done any Linux or c++ development although I had some exposure to c++ at work. Many of the concepts
are so different that what I am familiar with and that slows me down a lot. I've used HP-UX, solaris and linux as
desktop workstations at work and used linux at home since Redhat 5.2.

Although I'm comfortable with assembly code I haven't touched Intel since the 8000 series. That will make the assembly
language files tough to work with. I have a couple of optimization thechniques I've used in the past but they require
intimate knowledge of the Intel architecture such as memory interface, cache organization, execution environment,
instruction issue and retirement throughput, and things I'm probably not aware exist. I have a lot to learn, but that's why
I'm here.

I know nothing about cryptographic algorithms so don't expect me to code new algos. I'll leave that up to others.

I had heard of Bitcoin but didn't get involved until very late (too late) in the darkcoin frenzy in April 2014. I started
with just one CPU but unknown to me at the time ASICS had taken over bitcoin and scrypt was next. And GPUs were taking
over the altcoin algos. I bought a gt730, a nice little card that performed on par with my i7-4790K. I bought a 750ti shortly
after that and a few other cards since then.

Well that's how I got here. I'm bored with lots of time on my hands so I've been poking around at miner code here and there.
I'm even less familiar so I took a shot with a cpuminer and was surprised. It is a credit to Pooler's design that I was able to
pull together code from 3 different miners and make it work smoothly and efficiently in a short time.

VS is still installing but I think it's time for a break.

I must be getting tired I just tried to post this to the wrong thread. I giess i don't have the stamina I did 25 years ago.

Thanks for reading and taking an interest in my project.

Good read (:

[joblo]

An update on the beta test.

Windows is definitely out for the beta test and so is CPU ID display. Arcitecture checking is working but will be
in permissive mode for the test build. I'm pretty beat tonight but will start packaging everything tomorrow.
I will be contacting the select beta testers by PM to make arangements for getting the beta build to
them. I expect it to be available friday. I expect the beta test to be quite short as the software has been
very reliable. I hope the first public release is out next week.