Bitcoin Forum
July 20, 2017, 10:54:16 PM *
News: The warning which may be displayed by Bitcoin Core about unknown versions is related to BIP91, and can be safely ignored.
 
   Home   Help Search Donate Login Register  
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 [24] 25 26 27 28 29 30 31 32 33 34 35 »
  Print  
Author Topic: Wolf's XMR/BCN/DSH CPUMiner - 2x speed compared to LucasJones' - NEW 06/20/2014  (Read 336665 times)
dewdeded
Legendary
*
Offline Offline

Activity: 938


Monero Evangelist


View Profile WWW
August 02, 2014, 08:16:06 PM
 #461

You have 48 cores? (2-Core-CPUs?)
1500591256
Hero Member
*
Offline Offline

Posts: 1500591256

View Profile Personal Message (Offline)

Ignore
1500591256
Reply with quote  #2

1500591256
Report to moderator
1500591256
Hero Member
*
Offline Offline

Posts: 1500591256

View Profile Personal Message (Offline)

Ignore
1500591256
Reply with quote  #2

1500591256
Report to moderator
1500591256
Hero Member
*
Offline Offline

Posts: 1500591256

View Profile Personal Message (Offline)

Ignore
1500591256
Reply with quote  #2

1500591256
Report to moderator
Decentralized search
Search for products or services and get paid for it
pre-sale Token CAT
25 July 50% discount
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1500591256
Hero Member
*
Offline Offline

Posts: 1500591256

View Profile Personal Message (Offline)

Ignore
1500591256
Reply with quote  #2

1500591256
Report to moderator
PeaMine
Hero Member
*****
Offline Offline

Activity: 667


View Profile
August 03, 2014, 12:24:26 AM
 #462

Each optimization I tested worked on my CPU - I don't have all CPUs, so I can't test. That much of a reduction is odd, though... are you sure the test wasn't affected by other factors?

It's quite a difference. I have 8 of those monster servers that are freshly installed (+updated) with Ubuntu 14.04.
They are all identical and unused (no running services, crons etc..) but the results are rather inconsistent across each so perhaps something else could be influencing things.
I'll dig a bit deeper and see if anything comes of it.

Going to check this out as well, running it without the optimizations.
sandor111
Hero Member
*****
Offline Offline

Activity: 490



View Profile WWW
August 03, 2014, 12:42:24 AM
 #463

Btw.. Just thought i'd mention this:

Latest CPUMiner compiled from git: (24 x Intel(R) Xeon(R) CPU X5650 @ 2.67GHz)

Over a 20 minute duration:

[2014-08-02 20:25:25] accepted: 191/234 (81.62%), 230.74 H/s at diff 980 (yay!!!)
[2014-08-02 20:25:29] accepted: 192/235 (81.70%), 230.63 H/s at diff 980 (yay!!!)
[2014-08-02 20:25:29] accepted: 193/236 (81.78%), 230.61 H/s at diff 980 (yay!!!)
[2014-08-02 20:25:31] accepted: 194/237 (81.86%), 230.16 H/s at diff 980 (yay!!!)
[2014-08-02 20:25:40] accepted: 195/238 (81.93%), 231.01 H/s at diff 980 (yay!!!)

Then i removed all of the "optimizations" from the Makefile:

From:

am__append_3 = -Ofast -flto -fuse-linker-plugin -funroll-loops -fvariable-expansion-in-unroller -ftree-loop-if-convert-stores -fmerg
e-all-constants -fbranch-target-load-optimize2 -fsched2-use-superblocks -maes

to :

am__append_3 = -maes

The result?

[2014-08-02 20:26:44] accepted: 189/231 (81.82%), 289.83 H/s at diff 1384 (yay!!!)
[2014-08-02 20:26:45] accepted: 190/232 (81.90%), 290.09 H/s at diff 1384 (yay!!!)
[2014-08-02 20:27:03] accepted: 191/233 (81.97%), 291.41 H/s at diff 1384 (yay!!!)
[2014-08-02 20:27:07] accepted: 192/234 (82.05%), 290.47 H/s at diff 1384 (yay!!!)

There are numerous posts/warnings on-line about over-aggressive compile-time "optimizations".
Just thought I'd post my results/findings which concur.

Regards,
Cami

You should be getting atleast 400 H/s out of those CPUs (mine are hashing at 451 H/s)

PeaMine
Hero Member
*****
Offline Offline

Activity: 667


View Profile
August 03, 2014, 01:01:30 AM
 #464

Each optimization I tested worked on my CPU - I don't have all CPUs, so I can't test. That much of a reduction is odd, though... are you sure the test wasn't affected by other factors?

It's quite a difference. I have 8 of those monster servers that are freshly installed (+updated) with Ubuntu 14.04.
They are all identical and unused (no running services, crons etc..) but the results are rather inconsistent across each so perhaps something else could be influencing things.
I'll dig a bit deeper and see if anything comes of it.

Going to check this out as well, running it without the optimizations.


Without optimizations:

[2014-08-03 00:58:53] accepted: 4/4 (100.00%), 195.91 H/s at diff 5000 (yay!!!)
[2014-08-03 00:59:04] accepted: 5/5 (100.00%), 195.29 H/s at diff 5000 (yay!!!)
[2014-08-03 00:59:07] accepted: 6/6 (100.00%), 194.45 H/s at diff 5000 (yay!!!)
[2014-08-03 00:59:08] accepted: 7/7 (100.00%), 194.69 H/s at diff 5000 (yay!!!)


With:
[2014-08-03 01:00:28] accepted: 3/3 (100.00%), 369.24 H/s at diff 5000 (yay!!!)
[2014-08-03 01:00:36] accepted: 4/4 (100.00%), 369.24 H/s at diff 5000 (yay!!!)
[2014-08-03 01:00:36] accepted: 5/5 (100.00%), 369.23 H/s at diff 5000 (yay!!!)
[2014-08-03 01:00:52] accepted: 6/6 (100.00%), 369.14 H/s at diff 5000 (yay!!!)

Crazy how efficient those optimizations are.
Though sadly needs to be around 440+ or so for current difficulty to break even.
Hotmetal
Sr. Member
****
Offline Offline

Activity: 266


View Profile
August 03, 2014, 04:07:58 AM
 #465

You should be getting atleast 400 H/s out of those CPUs (mine are hashing at 451 H/s)

How though? I've tried on a variety of *very expensive* hardware from Dell.
Each has 100gigs of ram, 24 cpus, etc..

Ubuntu 14.04:

# sysctl -w vm.nr_hugepages=72  (confirmed that it is set)
# git clone https://github.com/wolf9466/cpuminer-multi.git
# cd cpuminer-multi
# ./autogen.sh
# ./configure CFLAGS="-march=native"
# make
# make install
# screen -d -m -S minerd /usr/local/bin/minerd -o stratum+tcp://mro.pool.minergate.com:45560 -p x -u x
# screen -r (after a few hours of running)
[2014-08-03 06:05:25] accepted: 3952/7916 (49.92%), 288.94 H/s at diff 444 (yay!!!)

example of hardware: 24 CPUs - Intel(R) Xeon(R) CPU E5-2420 0 @ 1.90GHz
another example:       24 CPUs - Intel(R) Xeon(R) CPU X5650  @ 2.67GHz



sandor111
Hero Member
*****
Offline Offline

Activity: 490



View Profile WWW
August 03, 2014, 10:29:44 AM
 #466

You should be getting atleast 400 H/s out of those CPUs (mine are hashing at 451 H/s)

How though? I've tried on a variety of *very expensive* hardware from Dell.
Each has 100gigs of ram, 24 cpus, etc..

Ubuntu 14.04:

# sysctl -w vm.nr_hugepages=72  (confirmed that it is set)
# git clone https://github.com/wolf9466/cpuminer-multi.git
# cd cpuminer-multi
# ./autogen.sh
# ./configure CFLAGS="-march=native"
# make
# make install
# screen -d -m -S minerd /usr/local/bin/minerd -o stratum+tcp://mro.pool.minergate.com:45560 -p x -u x
# screen -r (after a few hours of running)
[2014-08-03 06:05:25] accepted: 3952/7916 (49.92%), 288.94 H/s at diff 444 (yay!!!)

example of hardware: 24 CPUs - Intel(R) Xeon(R) CPU E5-2420 0 @ 1.90GHz
another example:       24 CPUs - Intel(R) Xeon(R) CPU X5650  @ 2.67GHz





Don't set threads equal to number of CPU threads, that is wrong.
The general rule is threads = floor(L3 cache / 1024)
Use this: num_threads=$(($(cat /proc/cpuinfo | grep "cache size" | uniq | cut -d":" -f2 | cut -d" " -f2)/1024))
Should be 12 for X5650

Hotmetal
Sr. Member
****
Offline Offline

Activity: 266


View Profile
August 03, 2014, 11:21:22 AM
 #467

Don't set threads equal to number of CPU threads, that is wrong.
The general rule is threads = floor(L3 cache / 1024)
Use this: num_threads=$(($(cat /proc/cpuinfo | grep "cache size" | uniq | cut -d":" -f2 | cut -d" " -f2)/1024))
Should be 12 for X5650

You sir, are a legend. Why does it automatically set thread count to ~cpu count?

Using your formula:

[2014-08-03 13:19:33] accepted: 12/12 (100.00%), 514.93 H/s at diff 444 (yay!!!)
[2014-08-03 13:19:34] accepted: 13/13 (100.00%), 513.02 H/s at diff 444 (yay!!!)
[2014-08-03 13:19:35] accepted: 14/14 (100.00%), 528.15 H/s at diff 444 (yay!!!)
[2014-08-03 13:19:35] accepted: 15/15 (100.00%), 525.17 H/s at diff 444 (yay!!!)

Perhaps this should be added to the original post? Its a BIG improvement.

Thanks! (Now to deploy this on a motherload of machines :/)
Hotmetal
Sr. Member
****
Offline Offline

Activity: 266


View Profile
August 03, 2014, 12:01:05 PM
 #468

Why does it automatically set thread count to ~cpu count?

Using your formula:

[2014-08-03 13:19:33] accepted: 12/12 (100.00%), 514.93 H/s at diff 444 (yay!!!)
[2014-08-03 13:19:34] accepted: 13/13 (100.00%), 513.02 H/s at diff 444 (yay!!!)
[2014-08-03 13:19:35] accepted: 14/14 (100.00%), 528.15 H/s at diff 444 (yay!!!)
[2014-08-03 13:19:35] accepted: 15/15 (100.00%), 525.17 H/s at diff 444 (yay!!!)

Perhaps this should be added to the original post? Its a BIG improvement.


To answer my own question, it appears as though the improvement is noticed only when the server has a large number of cpus / cache etc.
Older machines seem to perform better with a larger amount of threads, even when cpu count is lower.

I'm breaking > 10 000 H/s without even breaking a sweat across a few machines.
Thank you
ballot
Hero Member
*****
Offline Offline

Activity: 899



View Profile
August 03, 2014, 02:30:50 PM
 #469

trying to compile for win 64 bit
with this instruction
https://bitcointalk.org/index.php?topic=594470.msg6542760#msg6542760
tried many different ./configure
everytime it success compile but the compiled binary crash while trying to run also i tried to strip the exe too  Undecided

   ▄ █     ▄
   ▀ █ ▄ ▀ █
 ▄ █ █ █ █ █ ▄
██ █ █ █▄█ █ ██
██▄███▄█ ██████
 ▀▀███▄█████▀▀
██▀▀███████▀▀██
 █▄  ▀███▀  ▄█
 ▀▀▀██▀ ▀██▀▀▀
   ███▀█▀███
   █ █▀█ █ █
     ▀ █ █
     ▀ █
   ▄ █     ▄
   ▀ █ ▄ ▀ █
 ▄ █ █ █ █ █ ▄
██ █ █ █▄█ █ ██
██▄███▄█ ██████
 ▀▀███▄█████▀▀
██▀▀███████▀▀██
 █▄  ▀███▀  ▄█
 ▀▀▀██▀ ▀██▀▀▀
   ███▀█▀███
   █ █▀█ █ █
     ▀ █ █
     ▀ █
   ▄ █     ▄
   ▀ █ ▄ ▀ █
 ▄ █ █ █ █ █ ▄
██ █ █ █▄█ █ ██
██▄███▄█ ██████
 ▀▀███▄█████▀▀
██▀▀███████▀▀██
 █▄  ▀███▀  ▄█
 ▀▀▀██▀ ▀██▀▀▀
   ███▀█▀███
   █ █▀█ █ █
     ▀ █ █
     ▀ █
.
●  Over 630 Cards
●  32 Heroes in 7 Classes
●  Innovative Dual System
.
●  Balanced & Varied
●  Great Community
●  FREE to play
[]   ▄ █     ▄
   ▀ █ ▄ ▀ █
 ▄ █ █ █ █ █ ▄
██ █ █ █▄█ █ ██
██▄███▄█ ██████
 ▀▀███▄█████▀▀
██▀▀███████▀▀██
 █▄  ▀███▀  ▄█
 ▀▀▀██▀ ▀██▀▀▀
   ███▀█▀███
   █ █▀█ █ █
     ▀ █ █
     ▀ █
5w00p
Hero Member
*****
Offline Offline

Activity: 630



View Profile
August 04, 2014, 01:44:03 AM
 #470

8 Core Xeon gets 370 hash/sec all day, all night with 8 threads in ubuntu 14.04 w/ hugepages=24.
5w00p
Hero Member
*****
Offline Offline

Activity: 630



View Profile
August 04, 2014, 01:48:16 AM
 #471

Why does it automatically set thread count to ~cpu count?

Using your formula:

[2014-08-03 13:19:33] accepted: 12/12 (100.00%), 514.93 H/s at diff 444 (yay!!!)
[2014-08-03 13:19:34] accepted: 13/13 (100.00%), 513.02 H/s at diff 444 (yay!!!)
[2014-08-03 13:19:35] accepted: 14/14 (100.00%), 528.15 H/s at diff 444 (yay!!!)
[2014-08-03 13:19:35] accepted: 15/15 (100.00%), 525.17 H/s at diff 444 (yay!!!)

Perhaps this should be added to the original post? Its a BIG improvement.


To answer my own question, it appears as though the improvement is noticed only when the server has a large number of cpus / cache etc.
Older machines seem to perform better with a larger amount of threads, even when cpu count is lower.

I'm breaking > 10 000 H/s without even breaking a sweat across a few machines.


Thank you

really, because 10,000/528 = ~19.  19 is a "few machines?"

edit: and that diff is pathetically low.  444 is for a pentium 2 or someshit.
Wolf0
Legendary
*
Offline Offline

Activity: 1624


Miner Developer


View Profile
August 04, 2014, 10:21:49 AM
 #472

trying to compile for win 64 bit
with this instruction
https://bitcointalk.org/index.php?topic=594470.msg6542760#msg6542760
tried many different ./configure
everytime it success compile but the compiled binary crash while trying to run also i tried to strip the exe too  Undecided

Shit, I don't even compile on Windows, I cross-compile.

Code:
Donations: BTC: 1WoLFdwcfNEg64fTYsX1P25KUzzSjtEZC -- XMR: 45SLUTzk7UXYHmzJ7bFN6FPfzTusdUVAZjPRgmEDw7G3SeimWM2kCdnDQXwDBYGUWaBtZNgjYtEYA22aMQT4t8KfU3vHLHG
primer-
Legendary
*
Offline Offline

Activity: 994



View Profile
August 04, 2014, 10:42:03 AM
 #473

Looking for instructions on how to enable MADV_HUGEPAGE on Centos 6.5 , anyone ??
Hotmetal
Sr. Member
****
Offline Offline

Activity: 266


View Profile
August 04, 2014, 11:41:33 AM
 #474


really, because 10,000/528 = ~19.  19 is a "few machines?"

edit: and that diff is pathetically low.  444 is for a pentium 2 or someshit.

Yes, 19 is a "few machines". There are plenty more to use across different pools / coins.
Difficulty is assigned by the network and will scale up and down. 444 is the starting difficulty for that pool and its gotten considerably higher now.

Lighten up, you sound like you're having a bad day Wink
Wolf0
Legendary
*
Offline Offline

Activity: 1624


Miner Developer


View Profile
August 04, 2014, 01:42:41 PM
 #475

Looking for instructions on how to enable MADV_HUGEPAGE on Centos 6.5 , anyone ??

You can't, because CentOS is garbage. I had to add in a flag to disable some optimizations for that shit OS.

Code:
Donations: BTC: 1WoLFdwcfNEg64fTYsX1P25KUzzSjtEZC -- XMR: 45SLUTzk7UXYHmzJ7bFN6FPfzTusdUVAZjPRgmEDw7G3SeimWM2kCdnDQXwDBYGUWaBtZNgjYtEYA22aMQT4t8KfU3vHLHG
primer-
Legendary
*
Offline Offline

Activity: 994



View Profile
August 04, 2014, 01:54:34 PM
 #476

Looking for instructions on how to enable MADV_HUGEPAGE on Centos 6.5 , anyone ??

You can't, because CentOS is garbage. I had to add in a flag to disable some optimizations for that shit OS.

I know it is, is there any other way to get it to run on centos ? If i compile on ubuntu and run on centos it complains about glibc version (2.14 required, centos runs 2.12). Any other way ? Is it possible to include glibc in the static binary ?

EDIT : Removing MADV_HUGEPAGE from cpu-miner.c fixes the problem but with a significant loss in speed. I need the extra speed!!
pallas
Legendary
*
Offline Offline

Activity: 1358


Black Belt Developer


View Profile
August 04, 2014, 01:57:25 PM
 #477

Looking for instructions on how to enable MADV_HUGEPAGE on Centos 6.5 , anyone ??

You can't, because CentOS is garbage. I had to add in a flag to disable some optimizations for that shit OS.

I know it is, is there any other way to get it to run on centos ? If i compile on ubuntu and run on centos it complains about glibc version (2.14 required, centos runs 2.12). Any other way ? Is it possible to include glibc in the static binary ?

EDIT : Removing MADV_HUGEPAGE from cpu-miner.c fixes the problem but with a significant loss in speed. I need the extra speed!!

the static binary can include everything.
did you try:

./configure --enable-static

or

CFLAGS='-static'

?

primer-
Legendary
*
Offline Offline

Activity: 994



View Profile
August 04, 2014, 01:59:39 PM
 #478

Looking for instructions on how to enable MADV_HUGEPAGE on Centos 6.5 , anyone ??

You can't, because CentOS is garbage. I had to add in a flag to disable some optimizations for that shit OS.

I know it is, is there any other way to get it to run on centos ? If i compile on ubuntu and run on centos it complains about glibc version (2.14 required, centos runs 2.12). Any other way ? Is it possible to include glibc in the static binary ?

EDIT : Removing MADV_HUGEPAGE from cpu-miner.c fixes the problem but with a significant loss in speed. I need the extra speed!!

the static binary can include everything.
did you try:

./configure --enable-static

or

CFLAGS='-static'

?

Yes i have, i can't get past curl requirements even though i compiled curl from source with static flags.. I am not sure glibc is included in the static bin, can anyone confirm, dont want to waste time troubleshooting curl
primer-
Legendary
*
Offline Offline

Activity: 994



View Profile
August 04, 2014, 02:04:02 PM
 #479

Curl 7.31.1 compiled with :
./configure --disable-shared --enable-static --prefix=/usr/local --disable-ldap --disable-sspi
make -j 4;make install;

miner still errors out:
./autogen.sh
./configure CFLAGS="-static'
checking for the version of libcurl... 7.37.1
checking for libcurl >= version 7.15.2... yes
checking whether libcurl is usable... no
configure: error: Missing required libcurl >= 7.15.2

Any ideas ?
Hotmetal
Sr. Member
****
Offline Offline

Activity: 266


View Profile
August 04, 2014, 06:24:01 PM
 #480

Curl 7.31.1 compiled with :
./configure --disable-shared --enable-static --prefix=/usr/local --disable-ldap --disable-sspi
make -j 4;make install;

miner still errors out:
./autogen.sh
./configure CFLAGS="-static'
checking for the version of libcurl... 7.37.1
checking for libcurl >= version 7.15.2... yes
checking whether libcurl is usable... no
configure: error: Missing required libcurl >= 7.15.2

Any ideas ?

./configure is looking at curl prefix in /usr (depending on the distro). You might want configure curl like:
./configure --disable-shared --enable-static --prefix=/tmp/curl --disable-ldap --disable-sspi
make ; make install

then the miner:
./autogen.sh
./configure CFLAGS="-static' --with-libcurl=/tmp/curl
make

Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 [24] 25 26 27 28 29 30 31 32 33 34 35 »
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!