jtoomim
|
|
October 05, 2015, 04:57:18 AM |
|
The main branch only includes the BIP101 change and the payout_address web interface change. The performance modification has not been merged into the main branch yet. If you want to test out the performance improvements, you have to use the jtoomim_performance branch, which forrestv is currently testing out. https://github.com/p2pool/p2pool/tree/jtoomim_performancegit pull git checkout jtoomim_performance Edit/note: if you use pypy instead of regular python to run p2pool, the performance benefit is likely to be greater than 40%, since most of the rest of the slowdowns that I've seen would be amenable to pypy acceleration, but the problem I fixed was not. I may do some benchmarks to see if this is true in a while.
|
Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power. http://Toom.im
|
|
|
p3yot33at3r
|
|
October 05, 2015, 08:05:46 AM |
|
The main branch only includes the BIP101 change and the payout_address web interface change. The performance modification has not been merged into the main branch yet. If you want to test out the performance improvements, you have to use the jtoomim_performance branch, which forrestv is currently testing out. https://github.com/p2pool/p2pool/tree/jtoomim_performancegit pull git checkout jtoomim_performance Edit/note: if you use pypy instead of regular python to run p2pool, the performance benefit is likely to be greater than 40%, since most of the rest of the slowdowns that I've seen would be amenable to pypy acceleration, but the problem I fixed was not. I may do some benchmarks to see if this is true in a while. Nice work jtoomim! How did you manage to get p2pool running with pypy btw? I tried a while back, but got strange permissions issues on install & errors when running so decided to leave it. Google & reddit searches revealed nothing, & the guide on this thread didn't work for me either. Python is a complete mystery to me..... Any help/guidance would be very much appreciated
|
|
|
|
jtoomim
|
|
October 05, 2015, 09:45:06 AM Last edit: October 05, 2015, 02:44:36 PM by jtoomim |
|
How did you manage to get p2pool running with pypy btw? I tried a while back, but got strange permissions issues on install & errors when running so decided to leave it. Google & reddit searches revealed nothing, & the guide on this thread didn't work for me either. Python is a complete mystery to me..... I use Debian for my servers, so I start with: sudo apt-get install pypy pypy-dev gcc build-essential p2pool uses twisted, and twisted uses zope.interface, and in order to install either one you need setuptools, so let's start with that: wget https://bootstrap.pypa.io/ez_setup.py -O - | sudo pypy sudo rm setuptools-18.3.2.zip Then zope.interface: wget https://pypi.python.org/packages/source/z/zope.interface/zope.interface-4.1.3.tar.gz#md5=9ae3d24c0c7415deb249dd1a132f0f79 tar zxf zope.interface-4.1.3.tar.gz cd zope.interface-4.1.3/ sudo pypy setup.py install cd .. sudo rm -r zope.interface-4.1.3* Then Twisted: wget https://pypi.python.org/packages/source/T/Twisted/Twisted-15.4.0.tar.bz2 tar jxf Twisted-15.4.0.tar.bz2 cd Twisted-15.4.0 sudo pypy setup.py install cd .. sudo rm -r Twisted-15.4.0* and that should be it. Keep in mind that pypy uses a ton of RAM. You should expect about 2.5 GB of RAM per pypy-p2pool process. If you have 4 GB of RAM on your server, that will barely be enough to run bitcoind and pypy-p2pool as long as you don't have any GUIs running (no X11, no Gnome, no KDE, no XFCE).
|
Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power. http://Toom.im
|
|
|
nicklello
Member
Offline
Activity: 193
Merit: 10
|
|
October 05, 2015, 12:25:51 PM |
|
@jtoomim
Thankyou for the steps to get pypy running p2pool --- worked perfectly.
|
|
|
|
btcscot
Newbie
Offline
Activity: 30
Merit: 0
|
|
October 05, 2015, 12:27:10 PM |
|
That's a massive trade of 40% less cpu usage but your using 2.5gb of ram wow that is huge but a lot of vps will give you a crap proccessor and loads of ram so it could work for vps users
|
|
|
|
jtoomim
|
|
October 05, 2015, 01:23:55 PM Last edit: October 05, 2015, 02:05:57 PM by jtoomim |
|
That's a massive trade of 40% less cpu usage but your using 2.5gb of ram wow that is huge but a lot of vps will give you a crap proccessor and loads of ram so it could work for vps users
No, the 40% less CPU usage is due to the code changes and is not due to switching to pypy. You get 40% less CPU usage with the same RAM usage by running the new branch of p2pool (still under testing!) with the regular python 2.7. If you switch to pypy, you get an unknown reduction in CPU usage for 5x higher RAM usage. I haven't benchmarked the pypy/CPython difference yet. I couldn't notice a benefit before I made the code changes, but I think it might be significant now. Edit: I'm starting a head-to-head comparison of python vs pypy now. So far, pypy loads shares about 3x faster, taking about 20 seconds instead of 60 seconds on my Core i7 4790k machine. Edit2: It does not look like there is a big difference in operating CPU usage. I'm not even sure if pypy is any faster than python after the shares have been loaded. I'll let it run for at least half a day before I post any more results.
|
Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power. http://Toom.im
|
|
|
p3yot33at3r
|
|
October 05, 2015, 01:30:21 PM |
|
I use Debian for my servers, so I start with: sudo apt-get install pypy pypy-dev gcc Would this be the same for Xubuntu? Keep in mind that pypy uses a ton of RAM. You should expect about 2.5 GB of RAM per pypy-p2pool process. If you have 4 GB of RAM on your server, that will barely be enough to run bitcoind and pypy-p2pool as long as you don't have any GUIs running (no X11, no Gnome, no KDE, no XFCE).
I have 16GB RAM installed due to the amount of wallets I use, running a stripped down Xubuntu 64bit headless setup - so hopefully I should be OK.
|
|
|
|
jtoomim
|
|
October 05, 2015, 01:40:16 PM |
|
Would this be the same for Xubuntu?
Probably. You're in a better position to answer that question than I am.
|
Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power. http://Toom.im
|
|
|
p3yot33at3r
|
|
October 05, 2015, 01:47:27 PM |
|
Would this be the same for Xubuntu?
Probably. You're in a better position to answer that question than I am. Right, I'm gonna have another go. Were you logged in as root when you done this, or as a normal user? This is an excellent guide my friend thank you!
|
|
|
|
jtoomim
|
|
October 05, 2015, 01:54:05 PM |
|
Right, I'm gonna have another go. Were you logged in as root when you done this, or as a normal user?
This is an excellent guide my friend thank you!
The sudo command gives you superuser (root) privileges just for that command. In other words, you can do it as a normal user, as long as that user is allowed to use sudo.
|
Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power. http://Toom.im
|
|
|
btcscot
Newbie
Offline
Activity: 30
Merit: 0
|
|
October 05, 2015, 02:52:20 PM |
|
That's a massive trade of 40% less cpu usage but your using 2.5gb of ram wow that is huge but a lot of vps will give you a crap proccessor and loads of ram so it could work for vps users
No, the 40% less CPU usage is due to the code changes and is not due to switching to pypy. You get 40% less CPU usage with the same RAM usage by running the new branch of p2pool (still under testing!) with the regular python 2.7. If you switch to pypy, you get an unknown reduction in CPU usage for 5x higher RAM usage. I haven't benchmarked the pypy/CPython difference yet. I couldn't notice a benefit before I made the code changes, but I think it might be significant now. Edit: I'm starting a head-to-head comparison of python vs pypy now. So far, pypy loads shares about 3x faster, taking about 20 seconds instead of 60 seconds on my Core i7 4790k machine. Edit2: It does not look like there is a big difference in operating CPU usage. I'm not even sure if pypy is any faster than python after the shares have been loaded. I'll let it run for at least half a day before I post any more results. Ok that makes sense was going to say why has this not being done before hahaha need to pull my finger out and get my server running
|
|
|
|
p3yot33at3r
|
|
October 05, 2015, 05:13:35 PM |
|
How did you manage to get p2pool running with pypy btw? I tried a while back, but got strange permissions issues on install & errors when running so decided to leave it. Google & reddit searches revealed nothing, & the guide on this thread didn't work for me either. Python is a complete mystery to me..... I use Debian for my servers, so I start with: sudo apt-get install pypy pypy-dev gcc build-essential p2pool uses twisted, and twisted uses zope.interface, and in order to install either one you need setuptools, so let's start with that: wget https://bootstrap.pypa.io/ez_setup.py -O - | sudo pypy sudo rm setuptools-18.3.2.zip Then zope.interface: wget https://pypi.python.org/packages/source/z/zope.interface/zope.interface-4.1.3.tar.gz#md5=9ae3d24c0c7415deb249dd1a132f0f79 tar zxf zope.interface-4.1.3.tar.gz cd zope.interface-4.1.3/ sudo pypy setup.py install cd .. sudo rm -r zope.interface-4.1.3* Then Twisted: wget https://pypi.python.org/packages/source/T/Twisted/Twisted-15.4.0.tar.bz2 tar jxf Twisted-15.4.0.tar.bz2 cd Twisted-15.4.0 sudo pypy setup.py install cd .. sudo rm -r Twisted-15.4.0* and that should be it. Keep in mind that pypy uses a ton of RAM. You should expect about 2.5 GB of RAM per pypy-p2pool process. If you have 4 GB of RAM on your server, that will barely be enough to run bitcoind and pypy-p2pool as long as you don't have any GUIs running (no X11, no Gnome, no KDE, no XFCE). Just to confirm that this worked perfectly for me on Xubuntu 64bit. Testing stable so far Many thanks for this jtoomim! - bookmarked
|
|
|
|
|
idonothave
|
|
October 05, 2015, 07:31:03 PM |
|
Hi dear gurus p2pool. Please solve the problem. Antminer S7 batch 1 - low hashrate: ~2200Gh/s simmilar solution as with S4? set minimal difficulty?
|
|
|
|
jtoomim
|
|
October 05, 2015, 07:58:30 PM |
|
Hi dear gurus p2pool. Please solve the problem. Antminer S7 batch 1 - low hashrate: ~2200Gh/s
S7s do not work well on p2pool. Until someone comes up with a new version of cgminer for the S7, they will lose hashrate when run on p2pool. I suggest moving it onto a traditional pool at least for testing. However, in my testing, they only lose about 300 GH/s, not 2500 GH/s. I suspect something else may be wrong as well. You're getting a high HW error rate, over 1%. Your power supply might not be putting out enough voltage (should be above 12.0 V, and ideally probably around 12.3 V), or you might have bad hardware. If you can't adjust your power supply's output voltage, I suggest you try reducing the clock speed as well to see if that reduces the HW error rate and improves hashrate.
|
Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power. http://Toom.im
|
|
|
Meuh6879
Legendary
Offline
Activity: 1512
Merit: 1012
|
|
October 05, 2015, 09:35:38 PM |
|
S7s do not work well on p2pool. and my question : you use a REMOTE P2Pool node (with others mining devices) or a LOCAL P2pool ? it's a critical question.
|
|
|
|
Richy_T
Legendary
Offline
Activity: 2562
Merit: 2264
1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k
|
|
October 06, 2015, 03:59:15 AM Last edit: October 06, 2015, 04:46:09 AM by Richy_T |
|
I'm not aware of anyone who's tried the S5 cgminer version on an S7. You might want to check with kano or ckolivas first?
Sorry, my bad. This is on an S5. As far as I know kano didn't write a shiny new set of code for the S5 like he did for the S3. Also as far as I know, the latest version of cgminer for the S5 is indeed the one listed on page 627 of this thread. Getting weird chain info with that version of cgminer. Hashrate seems to be sane though so I may leave it for now. (Apologies for the poor formatting.) Chain# ASIC# Frequency Temp ASIC status 1 30 350 63 oooooo oooooooo oooooooo oooooooo 2 113 350 59 xxxxxxxxxx xxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx xooooooo oooooooo oooooooo oooooooo Any way to make the cgminer change permanent? Edit: Restarting cgminer fixed the chain weirdness.
|
1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k
|
|
|
idonothave
|
|
October 06, 2015, 06:21:05 AM |
|
I'm not aware of anyone who's tried the S5 cgminer version on an S7. You might want to check with kano or ckolivas first?
Sorry, my bad. This is on an S5. As far as I know kano didn't write a shiny new set of code for the S5 like he did for the S3. Also as far as I know, the latest version of cgminer for the S5 is indeed the one listed on page 627 of this thread. Getting weird chain info with that version of cgminer. Hashrate seems to be sane though so I may leave it for now. (Apologies for the poor formatting.) Chain# ASIC# Frequency Temp ASIC status 1 30 350 63 oooooo oooooooo oooooooo oooooooo 2 113 350 59 xxxxxxxxxx xxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx xooooooo oooooooo oooooooo oooooooo Any way to make the cgminer change permanent? Edit: Restarting cgminer fixed the chain weirdness. read this: https://bitcointalk.org/index.php?topic=882402.0there is howto set cgminer fixed after restart for S4. with another bitmain devices it should be similar
|
|
|
|
jtoomim
|
|
October 06, 2015, 11:16:00 AM Last edit: October 06, 2015, 11:28:59 AM by jtoomim |
|
CPU usage results for pypy vs python using branch jtoomim_performance of p2pool after 21.7 hours:
Regular python: 68 minutes 3 seconds, 5.22% average
pypy: 39 minutes 20 seconds, 3.82% average
That's 42% lower CPU usage for pypy.
This is multiplicative with the jtoomim_performance benefit, so using the jtoomim_performance branch with pypy has about 65% lower CPU usage than using the head branch with regular python.
Test conditions: Core i7 4790k processor, 12 peers, 76 to 80 TH/s load (SP30s). Both nodes were run on the same server at the same time, with a local BitcoinXT bitcoind.
|
Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power. http://Toom.im
|
|
|
p3yot33at3r
|
|
October 06, 2015, 01:19:52 PM |
|
CPU usage results for pypy vs python using branch jtoomim_performance of p2pool after 21.7 hours:
Regular python: 68 minutes 3 seconds, 5.22% average
pypy: 39 minutes 20 seconds, 3.82% average
That's 42% lower CPU usage for pypy.
This is multiplicative with the jtoomim_performance benefit, so using the jtoomim_performance branch with pypy has about 65% lower CPU usage than using the head branch with regular python.
Test conditions: Core i7 4790k processor, 12 peers, 76 to 80 TH/s load (SP30s). Both nodes were run on the same server at the same time, with a local BitcoinXT bitcoind.
That's a big improvement - impressive Tying it out now.
|
|
|
|
|