Bitcoin Forum
March 19, 2024, 10:21:29 AM *
News: Latest Bitcoin Core release: 26.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: « 1 ... 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 [674] 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 ... 814 »
  Print  
Author Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool  (Read 2591571 times)
jtoomim
Hero Member
*****
Offline Offline

Activity: 818
Merit: 1005


View Profile WWW
October 05, 2015, 04:57:18 AM
 #13461

saw the update on irc ... thx Wink

here is the link to DL latest commit https://github.com/p2pool/p2pool

updating mine shortly.

The main branch only includes the BIP101 change and the payout_address web interface change. The performance modification has not been merged into the main branch yet. If you want to test out the performance improvements, you have to use the jtoomim_performance branch, which forrestv is currently testing out.

https://github.com/p2pool/p2pool/tree/jtoomim_performance

git pull
git checkout jtoomim_performance

Edit/note: if you use pypy instead of regular python to run p2pool, the performance benefit is likely to be greater than 40%, since most of the rest of the slowdowns that I've seen would be amenable to pypy acceleration, but the problem I fixed was not. I may do some benchmarks to see if this is true in a while.

Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power.
http://Toom.im
Activity + Trust + Earned Merit == The Most Recognized Users on Bitcointalk
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction.
1710843689
Hero Member
*
Offline Offline

Posts: 1710843689

View Profile Personal Message (Offline)

Ignore
1710843689
Reply with quote  #2

1710843689
Report to moderator
1710843689
Hero Member
*
Offline Offline

Posts: 1710843689

View Profile Personal Message (Offline)

Ignore
1710843689
Reply with quote  #2

1710843689
Report to moderator
1710843689
Hero Member
*
Offline Offline

Posts: 1710843689

View Profile Personal Message (Offline)

Ignore
1710843689
Reply with quote  #2

1710843689
Report to moderator
p3yot33at3r
Sr. Member
****
Offline Offline

Activity: 266
Merit: 250



View Profile
October 05, 2015, 08:05:46 AM
 #13462


The main branch only includes the BIP101 change and the payout_address web interface change. The performance modification has not been merged into the main branch yet. If you want to test out the performance improvements, you have to use the jtoomim_performance branch, which forrestv is currently testing out.

https://github.com/p2pool/p2pool/tree/jtoomim_performance

git pull
git checkout jtoomim_performance

Edit/note: if you use pypy instead of regular python to run p2pool, the performance benefit is likely to be greater than 40%, since most of the rest of the slowdowns that I've seen would be amenable to pypy acceleration, but the problem I fixed was not. I may do some benchmarks to see if this is true in a while.

Nice work jtoomim!  Grin

How did you manage to get p2pool running with pypy btw? I tried a while back, but got strange permissions issues on install & errors when running so decided to leave it. Google & reddit searches revealed nothing, & the guide on this thread didn't work for me either. Python is a complete mystery to me..... Tongue

Any help/guidance would be very much appreciated  Wink
jtoomim
Hero Member
*****
Offline Offline

Activity: 818
Merit: 1005


View Profile WWW
October 05, 2015, 09:45:06 AM
Last edit: October 05, 2015, 02:44:36 PM by jtoomim
 #13463

How did you manage to get p2pool running with pypy btw? I tried a while back, but got strange permissions issues on install & errors when running so decided to leave it. Google & reddit searches revealed nothing, & the guide on this thread didn't work for me either. Python is a complete mystery to me..... Tongue

I use Debian for my servers, so I start with:

Code:
sudo apt-get install pypy pypy-dev gcc build-essential

p2pool uses twisted, and twisted uses zope.interface, and in order to install either one you need setuptools, so let's start with that:

Code:
wget https://bootstrap.pypa.io/ez_setup.py -O - | sudo pypy
sudo rm setuptools-18.3.2.zip

Then zope.interface:

Code:
wget https://pypi.python.org/packages/source/z/zope.interface/zope.interface-4.1.3.tar.gz#md5=9ae3d24c0c7415deb249dd1a132f0f79
tar zxf zope.interface-4.1.3.tar.gz
cd zope.interface-4.1.3/
sudo pypy setup.py install
cd ..
sudo rm -r zope.interface-4.1.3*

Then Twisted:

Code:
wget https://pypi.python.org/packages/source/T/Twisted/Twisted-15.4.0.tar.bz2
tar jxf Twisted-15.4.0.tar.bz2
cd Twisted-15.4.0
sudo pypy setup.py install
cd ..
sudo rm -r Twisted-15.4.0*

and that should be it.

Keep in mind that pypy uses a ton of RAM. You should expect about 2.5 GB of RAM per pypy-p2pool process. If you have 4 GB of RAM on your server, that will barely be enough to run bitcoind and pypy-p2pool as long as you don't have any GUIs running (no X11, no Gnome, no KDE, no XFCE).

Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power.
http://Toom.im
nicklello
Member
**
Offline Offline

Activity: 193
Merit: 10


View Profile
October 05, 2015, 12:25:51 PM
 #13464

@jtoomim

Thankyou for the steps to get pypy running p2pool --- worked perfectly.
btcscot
Newbie
*
Offline Offline

Activity: 30
Merit: 0


View Profile
October 05, 2015, 12:27:10 PM
 #13465

That's a massive trade of 40% less cpu usage but your using 2.5gb of ram wow that is huge but a lot of vps will give you a crap proccessor and loads of ram so it could work for vps users
jtoomim
Hero Member
*****
Offline Offline

Activity: 818
Merit: 1005


View Profile WWW
October 05, 2015, 01:23:55 PM
Last edit: October 05, 2015, 02:05:57 PM by jtoomim
 #13466

That's a massive trade of 40% less cpu usage but your using 2.5gb of ram wow that is huge but a lot of vps will give you a crap proccessor and loads of ram so it could work for vps users

No, the 40% less CPU usage is due to the code changes and is not due to switching to pypy. You get 40% less CPU usage with the same RAM usage by running the new branch of p2pool (still under testing!) with the regular python 2.7.

If you switch to pypy, you get an unknown reduction in CPU usage for 5x higher RAM usage. I haven't benchmarked the pypy/CPython difference yet. I couldn't notice a benefit before I made the code changes, but I think it might be significant now.

Edit: I'm starting a head-to-head comparison of python vs pypy now. So far, pypy loads shares about 3x faster, taking about 20 seconds instead of 60 seconds on my Core i7 4790k machine.

Edit2: It does not look like there is a big difference in operating CPU usage. I'm not even sure if pypy is any faster than python after the shares have been loaded. I'll let it run for at least half a day before I post any more results.

Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power.
http://Toom.im
p3yot33at3r
Sr. Member
****
Offline Offline

Activity: 266
Merit: 250



View Profile
October 05, 2015, 01:30:21 PM
 #13467

I use Debian for my servers, so I start with:

Code:
sudo apt-get install pypy pypy-dev gcc

Would this be the same for Xubuntu?

Keep in mind that pypy uses a ton of RAM. You should expect about 2.5 GB of RAM per pypy-p2pool process. If you have 4 GB of RAM on your server, that will barely be enough to run bitcoind and pypy-p2pool as long as you don't have any GUIs running (no X11, no Gnome, no KDE, no XFCE).

I have 16GB RAM installed due to the amount of wallets I use, running a stripped down Xubuntu 64bit headless setup - so hopefully I should be OK.
jtoomim
Hero Member
*****
Offline Offline

Activity: 818
Merit: 1005


View Profile WWW
October 05, 2015, 01:40:16 PM
 #13468

Would this be the same for Xubuntu?
Probably. You're in a better position to answer that question than I am.

Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power.
http://Toom.im
p3yot33at3r
Sr. Member
****
Offline Offline

Activity: 266
Merit: 250



View Profile
October 05, 2015, 01:47:27 PM
 #13469

Would this be the same for Xubuntu?
Probably. You're in a better position to answer that question than I am.

Right, I'm gonna have another go. Were you logged in as root when you done this, or as a normal user?

This is an excellent guide my friend thank you!
jtoomim
Hero Member
*****
Offline Offline

Activity: 818
Merit: 1005


View Profile WWW
October 05, 2015, 01:54:05 PM
 #13470

Right, I'm gonna have another go. Were you logged in as root when you done this, or as a normal user?

This is an excellent guide my friend thank you!
The sudo command gives you superuser (root) privileges just for that command. In other words, you can do it as a normal user, as long as that user is allowed to use sudo.

Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power.
http://Toom.im
btcscot
Newbie
*
Offline Offline

Activity: 30
Merit: 0


View Profile
October 05, 2015, 02:52:20 PM
 #13471

That's a massive trade of 40% less cpu usage but your using 2.5gb of ram wow that is huge but a lot of vps will give you a crap proccessor and loads of ram so it could work for vps users

No, the 40% less CPU usage is due to the code changes and is not due to switching to pypy. You get 40% less CPU usage with the same RAM usage by running the new branch of p2pool (still under testing!) with the regular python 2.7.

If you switch to pypy, you get an unknown reduction in CPU usage for 5x higher RAM usage. I haven't benchmarked the pypy/CPython difference yet. I couldn't notice a benefit before I made the code changes, but I think it might be significant now.

Edit: I'm starting a head-to-head comparison of python vs pypy now. So far, pypy loads shares about 3x faster, taking about 20 seconds instead of 60 seconds on my Core i7 4790k machine.

Edit2: It does not look like there is a big difference in operating CPU usage. I'm not even sure if pypy is any faster than python after the shares have been loaded. I'll let it run for at least half a day before I post any more results.

Ok that makes sense was going to say why has this not being done before hahaha need to pull my finger out and get my server running
p3yot33at3r
Sr. Member
****
Offline Offline

Activity: 266
Merit: 250



View Profile
October 05, 2015, 05:13:35 PM
 #13472

How did you manage to get p2pool running with pypy btw? I tried a while back, but got strange permissions issues on install & errors when running so decided to leave it. Google & reddit searches revealed nothing, & the guide on this thread didn't work for me either. Python is a complete mystery to me..... Tongue

I use Debian for my servers, so I start with:

Code:
sudo apt-get install pypy pypy-dev gcc build-essential

p2pool uses twisted, and twisted uses zope.interface, and in order to install either one you need setuptools, so let's start with that:

Code:
wget https://bootstrap.pypa.io/ez_setup.py -O - | sudo pypy
sudo rm setuptools-18.3.2.zip

Then zope.interface:

Code:
wget https://pypi.python.org/packages/source/z/zope.interface/zope.interface-4.1.3.tar.gz#md5=9ae3d24c0c7415deb249dd1a132f0f79
tar zxf zope.interface-4.1.3.tar.gz
cd zope.interface-4.1.3/
sudo pypy setup.py install
cd ..
sudo rm -r zope.interface-4.1.3*

Then Twisted:

Code:
wget https://pypi.python.org/packages/source/T/Twisted/Twisted-15.4.0.tar.bz2
tar jxf Twisted-15.4.0.tar.bz2
cd Twisted-15.4.0
sudo pypy setup.py install
cd ..
sudo rm -r Twisted-15.4.0*

and that should be it.

Keep in mind that pypy uses a ton of RAM. You should expect about 2.5 GB of RAM per pypy-p2pool process. If you have 4 GB of RAM on your server, that will barely be enough to run bitcoind and pypy-p2pool as long as you don't have any GUIs running (no X11, no Gnome, no KDE, no XFCE).

Just to confirm that this worked perfectly for me on Xubuntu 64bit. Testing stable so far  Wink

Many thanks for this jtoomim! - bookmarked  Wink Cheesy
Dyak80
Newbie
*
Offline Offline

Activity: 10
Merit: 0


View Profile
October 05, 2015, 05:21:23 PM
 #13473

Hi dear gurus p2pool.
Please solve the problem.
Antminer S7 batch 1 - low hashrate: ~2200Gh/s

http://s13.postimg.org/9zqcdvaef/p2pool_s7_b1.jpg
idonothave
Full Member
***
Offline Offline

Activity: 238
Merit: 100


View Profile
October 05, 2015, 07:31:03 PM
 #13474

Hi dear gurus p2pool.
Please solve the problem.
Antminer S7 batch 1 - low hashrate: ~2200Gh/s



simmilar solution as with S4? set minimal difficulty?
jtoomim
Hero Member
*****
Offline Offline

Activity: 818
Merit: 1005


View Profile WWW
October 05, 2015, 07:58:30 PM
 #13475

Hi dear gurus p2pool.
Please solve the problem.
Antminer S7 batch 1 - low hashrate: ~2200Gh/s

S7s do not work well on p2pool. Until someone comes up with a new version of cgminer for the S7, they will lose hashrate when run on p2pool. I suggest moving it onto a traditional pool at least for testing.

However, in my testing, they only lose about 300 GH/s, not 2500 GH/s. I suspect something else may be wrong as well.

You're getting a high HW error rate, over 1%. Your power supply might not be putting out enough voltage (should be above 12.0 V, and ideally probably around 12.3 V), or you might have bad hardware. If you can't adjust your power supply's output voltage, I suggest you try reducing the clock speed as well to see if that reduces the HW error rate and improves hashrate.

Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power.
http://Toom.im
Meuh6879
Legendary
*
Offline Offline

Activity: 1512
Merit: 1011



View Profile
October 05, 2015, 09:35:38 PM
 #13476

Quote
S7s do not work well on p2pool.

 Roll Eyes and my question : you use a REMOTE P2Pool node (with others mining devices) or a LOCAL P2pool ?










it's a critical question.
Richy_T
Legendary
*
Offline Offline

Activity: 2380
Merit: 2100


1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k


View Profile
October 06, 2015, 03:59:15 AM
Last edit: October 06, 2015, 04:46:09 AM by Richy_T
 #13477


I'm not aware of anyone who's tried the S5 cgminer version on an S7. You might want to check with kano or ckolivas first?

Sorry, my bad. This is on an S5.
As far as I know kano didn't write a shiny new set of code for the S5 like he did for the S3.  Also as far as I know, the latest version of cgminer for the S5 is indeed the one listed on page 627 of this thread.

Getting weird chain info with that version of cgminer. Hashrate seems to be sane though so I may leave it for now. (Apologies for the poor formatting.)

Chain#    ASIC#    Frequency    Temp    ASIC status
1 30 350 63 oooooo oooooooo oooooooo oooooooo
2 113 350 59 xxxxxxxxxx xxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx xooooooo oooooooo oooooooo oooooooo

Any way to make the cgminer change permanent?

Edit: Restarting cgminer fixed the chain weirdness.

1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k
idonothave
Full Member
***
Offline Offline

Activity: 238
Merit: 100


View Profile
October 06, 2015, 06:21:05 AM
 #13478


I'm not aware of anyone who's tried the S5 cgminer version on an S7. You might want to check with kano or ckolivas first?

Sorry, my bad. This is on an S5.
As far as I know kano didn't write a shiny new set of code for the S5 like he did for the S3.  Also as far as I know, the latest version of cgminer for the S5 is indeed the one listed on page 627 of this thread.

Getting weird chain info with that version of cgminer. Hashrate seems to be sane though so I may leave it for now. (Apologies for the poor formatting.)

Chain#    ASIC#    Frequency    Temp    ASIC status
1 30 350 63 oooooo oooooooo oooooooo oooooooo
2 113 350 59 xxxxxxxxxx xxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx xooooooo oooooooo oooooooo oooooooo

Any way to make the cgminer change permanent?

Edit: Restarting cgminer fixed the chain weirdness.

read this: https://bitcointalk.org/index.php?topic=882402.0
there is howto set cgminer fixed after restart for S4. with another bitmain devices it should be similar
jtoomim
Hero Member
*****
Offline Offline

Activity: 818
Merit: 1005


View Profile WWW
October 06, 2015, 11:16:00 AM
Last edit: October 06, 2015, 11:28:59 AM by jtoomim
 #13479

CPU usage results for pypy vs python using branch jtoomim_performance of p2pool after 21.7 hours:

Regular python:
68 minutes 3 seconds, 5.22% average

pypy:
39 minutes 20 seconds, 3.82% average

That's 42% lower CPU usage for pypy.

This is multiplicative with the jtoomim_performance benefit, so using the jtoomim_performance branch with pypy has about 65% lower CPU usage than using the head branch with regular python.

Test conditions: Core i7 4790k processor, 12 peers, 76 to 80 TH/s load (SP30s). Both nodes were run on the same server at the same time, with a local BitcoinXT bitcoind.

Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power.
http://Toom.im
p3yot33at3r
Sr. Member
****
Offline Offline

Activity: 266
Merit: 250



View Profile
October 06, 2015, 01:19:52 PM
 #13480

CPU usage results for pypy vs python using branch jtoomim_performance of p2pool after 21.7 hours:

Regular python:
68 minutes 3 seconds, 5.22% average

pypy:
39 minutes 20 seconds, 3.82% average

That's 42% lower CPU usage for pypy.

This is multiplicative with the jtoomim_performance benefit, so using the jtoomim_performance branch with pypy has about 65% lower CPU usage than using the head branch with regular python.

Test conditions: Core i7 4790k processor, 12 peers, 76 to 80 TH/s load (SP30s). Both nodes were run on the same server at the same time, with a local BitcoinXT bitcoind.

That's a big improvement - impressive  Smiley

Tying it out now.
Pages: « 1 ... 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 [674] 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 ... 814 »
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!