raskul
|
|
June 17, 2014, 08:15:59 PM |
|
Haven't gone through the whole post yet but I think this is/was EXACTLY what I was looking for- an updated and easy way to setup a p2pool. I'll start working on this now good to see more and more p2pool nodes getting set up, best luck with yours waldohoover I'm happy to see people wanting to support the network as well and move towards p2pool, thanks Setting up the node (in my sig) wasn't too bad with Morblias initial guidance but I def want to get a pool going now. I run /r/TeamRedditMining on reddit and would love to have our own pool going. aha! a US node, i'll direct US friends to it.. my node is also in my sig (Central Cavern - which is an Amsterdam node) - but be careful, the page is noisy Very nice and thank you! No harm in making mining a bit of fun, and since murdof did hold my hand rather a lot in setting up that node, credit should go to him; he's taught me loads over the past couple of days... my only linux work previously was setting up nodes for coins - what I really want to do is get my head around DGM. Always mined on small DGM pools up 'til now and I'd love to be able to know the ins and outs of it - even if I don't essentially set up a DGM pool, it's all learning huh?
|
tips 1APp826DqjJBdsAeqpEstx6Q8hD4urac8a
|
|
|
KyrosKrane
|
|
June 17, 2014, 08:23:03 PM Last edit: June 20, 2014, 07:12:53 PM by KyrosKrane |
|
A few months ago, I was working on a script that would automate the installation and rollout of a VPS with a bitcoind full node and a p2pool install. I never got it finalized with all the bells and whistles that I wanted, nor really polished, but it was good enough that I was able to adapt it for my local and remote p2pool installs, with some obvious changes. I'm posting it here in the hope you (or others) find it useful. I'd encourage you to scavenge whatever bits you can incorporate into your already-excellent guide. ############################################################################### # Prerequisites ############################################################################### # # Before you run this, do the following: # * Log in as root # * Be in your home directory # * The parent of home directories has to be specified below (e.g., if user home directories are like /home/user1, then HOME_ROOT must be set to /home). # # * (Optional but recommended) Create a file with the name specified below in REGULAR_USER_PUBLIC_KEY_FILE that contains your SSH public key. This should be in the copy/paste format you get from PuttyGen, not the saved public key file.
# Assumptions: # * Running on a relatively recent version of Ubuntu (12.04 or later) -- not tested with anything else, but it might work anyway! # * Python is already installed. If not, the script attempts to install Python, but right now I can't guarantee if you'll get Python 2.7 or 3.x. P2Pool doesn't work with 3.x. # * This script was designed for use on a VM or VPS that's basically empty, and only has the root user created.
############################################################################### # This section has to be run as root ###############################################################################
PrintStatus() { echo echo "----- STATUS: $*" echo }
############# # Script settings - EdIT THESE before running. #############
HOME_ROOT=/home
# These are general script settings. Edit these before running the script. REGULAR_USER_NAME=heybrother # Randomly inspired by the song on the radio as I wrote this! REGULAR_USER_PUBLIC_KEY_FILE=$REGULAR_USER_NAME.pub # @TODO: Prompt the user for these two values.
############# # System updates and patches #############
# @TODO: Maybe create a swap file? For now, the script assumes the host system either has enough memory or has swap already set up. # For a sample command, see: # https://raw.github.com/XertroV/BitcoinAutoNode/master/bitcoinAutoNode.sh
PrintStatus "Updating the system" # Update your packages and package list apt-get update apt-get -y dist-upgrade # This does the functions of upgrade as well.
PrintStatus "Installing system tools" # @TODO: To get the add-apt-repository command, we may need to install one of the following, as per: # https://stackoverflow.com/questions/13018626/add-apt-repository-not-found # for Ubuntu <= 12.04 # sudo apt-get install python-software-properties # for Ubuntu >= 12.10 # sudo apt-get install software-properties-common # # At some point I'll have to figure out how to distinguish versions and load only the needed one, but for now, just load both. apt-get -y install python-software-properties software-properties-common
# Add the bitcoin repository - used for some bitcoind dependencies # So it seems this is an official bitcoin-core repository, or at least, official enough that the devs endorse downloads from it in their readme file. #So, I'll trust the binary download from here, rather than try to muddle through the insane mess required to build bitcoind cleanly from source on multiple arbitrary target systems. PrintStatus "Adding bitcoin repository" add-apt-repository -y ppa:bitcoin/bitcoin apt-get update
# Install some useful utilities and the pre-reqs for P2Pool and bitcoin PrintStatus "Installing utilities" apt-get -y install nano htop ufw monit git p7zip-full screen
############# # Python & P2Pool dependencies #############
# Make sure Python is installed PYTHON="$(which python)" if [ -z "$PYTHON" ] ; then PrintStatus "Installing Python." apt-get -y install python # @TODO: review this; make sure we don't install Python3 by mistake. PYTHON="$(which python)" fi
if [ -z "$PYTHON" ] ; then PrintStatus "WARNING! Failed to install Python. P2Pool may not run." # @TODO: Any special handling here for the error? else PYTHON_VERSION="$("$PYTHON" -V 2>&1)" PYTHON_26=$(awk 'BEGIN { if ("'"$PYTHON_VERSION"'" ~ /Python 2.6.*/) print "YES";}') #PYTHON_27=$(awk 'BEGIN { if ("'"$PYTHON_VERSION"'" ~ /Python 2.7.*/) print "YES";}') # Just for testing PYTHON_3x=$(awk 'BEGIN { if ("'"$PYTHON_VERSION"'" ~ /Python 3.*/) print "YES";}')
if [ -n "$PYTHON_26" ] ; then PrintStatus "Python 2.6.* requires an additional dependency, installing." apt-get -y install python-argparse elif [ -n "$PYTHON_3x" ] ; then PrintStatus "WARNING! Python 3.x detected. This may not work with P2Pool." # @TODO: Any special handling here for the error? elif [ -n "$PYTHON_27" ] ; then PrintStatus "Python 2.7 detected, all ok!" fi fi
# Install the pre-reqs for P2Pool PrintStatus "Installing P2Pool dependencies" apt-get -y install python-zope.interface python-twisted python-twisted-web
############# # bitcoind #############
# For the UI, not strictly needed, but nice to have PrintStatus "Installing Bitcoin dependencies" apt-get -y install libqtgui4 # Side note - this installs a metric crapton of dependencies...
# Download and install bitcoind PrintStatus "Installing bitcoind" apt-get -y install bitcoind
############# # User creation #############
# @TODO: Check if user already exists and skip this step if needed
# Create your regular user PrintStatus "Creating your regular user $REGULAR_USER_NAME" useradd $REGULAR_USER_NAME
# Grant your user sudo permission usermod -a -G sudo $REGULAR_USER_NAME
# Make the user's SSH directory and add the private key if [ -s "$REGULAR_USER_PUBLIC_KEY_FILE" ] ; then PrintStatus "Setting up SSH keys for your user"
# First ensure the .ssh directory exists. if [ ! -d $HOME_ROOT/$REGULAR_USER_NAME/.ssh ] ; then mkdir -p $HOME_ROOT/$REGULAR_USER_NAME/.ssh chown $REGULAR_USER_NAME $HOME_ROOT/$REGULAR_USER_NAME chown $REGULAR_USER_NAME $HOME_ROOT/$REGULAR_USER_NAME/.ssh chmod 700 $HOME_ROOT/$REGULAR_USER_NAME/.ssh fi
# Add the key and assign ownership of the file. cat ~/$REGULAR_USER_PUBLIC_KEY_FILE >> $HOME_ROOT/$REGULAR_USER_NAME/.ssh/authorized_keys chown $REGULAR_USER_NAME $HOME_ROOT/$REGULAR_USER_NAME/.ssh/authorized_keys chmod 600 $HOME_ROOT/$REGULAR_USER_NAME/.ssh/authorized_keys rm ~/$REGULAR_USER_PUBLIC_KEY_FILE # don't need it any more.
else PrintStatus "SSH key NOT found for user $REGULAR_USER_NAME. Please enter a login password." passwd $REGULAR_USER_NAME fi
# PrintStatus "Locking out root password." # @TODO
############# # Firewall setup #############
# Next, let's set up a firewall PrintStatus "Setting up the firewall" ufw default deny # Deny everything unless expressly permitted ufw allow 22/tcp # SSH ufw allow 8333/tcp # Bitcoind peer to peer # Note that we do NOT enable the Bitcoind RPC port! ufw allow 9333/tcp # P2pool peer to peer ufw allow 9332/tcp # P2Pool connections and Web interface ufw --force enable # Turn it on
############# # Monit setup #############
# set up monit, a monitoring tool. PrintStatus "Configuring Monit" # @TODO
# @TODO: Also check out Zabbix as per: # https://bitcointalk.org/index.php?topic=153232.0 # http://www.zabbix.com/
############# # Ideas for future growth: #############
# Log rotation: http://manpages.ubuntu.com/manpages/jaunty/man8/logrotate.8.html # Also https://bitcoin.stackexchange.com/questions/18857/how-to-limit-the-debug-log-file-while-running-bitcoind
############################################################################### # This section has to be run as your regular user ###############################################################################
# @TODO: Maybe run this directly as the user? # http://www.cyberciti.biz/open-source/command-line-hacks/linux-run-command-as-different-user/
PrintStatus() { echo echo "----- STATUS: $*" echo }
############# # Bitcoind setup #############
# Create the bitcoind configuration file PrintStatus "Creating the bitcoind configuration file" mkdir -p $HOME/.bitcoin config="$HOME/.bitcoin/bitcoin.conf" randUser=`< /dev/urandom tr -dc A-Za-z0-9 | head -c40` randPass=`< /dev/urandom tr -dc A-Za-z0-9 | head -c40`
cat <<EOF > $config server=1 daemon=1
rpcuser=$randUser rpcpassword=$randPass connections=40
blockmaxsize=1000000 mintxfee=0.00001 minrelaytxfee=0.00001 EOF
# Download the bitcoin bootstrap file. This saves a massive amount of time when starting bitcoind the first time - literally days. # The right way to do this is via bittorrent - the seed file is at:
# Torrent info hash: 2d4e6c1f96c5d5fb260dff92aea4e600227f1aea # Torrent magnet link: # magnet:?xt=urn:btih:2d4e6c1f96c5d5fb260dff92aea4e600227f1aea&dn=bootstrap.dat&tr=udp://tracker.openbittorrent.com:80&tr=udp://tracker.publicbt.com:80&tr=udp://tracker.ccc.de:80&tr=udp://tracker.istole.it:80
# Details: https://bitcointalk.org/index.php?topic=145386.0 (Check here for more recent versions every few months.)
# However, most VPS providers don't allow torrents. So alternately, we'll download using http. Not nearly as efficient, but still faster than syncing with the network. PrintStatus "Downloading Bitcoin bootstrap.dat file. This could take a while..." cd $HOME/.bitcoin
#wget https://bitfetch.com/static/bootstrap.7z # Note: as of this writing, the SSL certificate for bitfetch.com has expired. # If you just try to wget the file, wget will fail, complaining about it. # So, we have to force wget to ignore the expired certificate. Once the # certificate on the site has been properly renewed, delete the line below # and uncomment the line above. wget --no-check-certificate https://bitfetch.com/static/bootstrap.7z
PrintStatus "Uncompressing Bitcoin bootstrap.dat file." 7z x bootstrap.7z
# Start bitcoind so that it can begin processing the bootstrap and syncing with the network # actually don't do this; instead force a reboot at the end of the script. # PrintStatus "Starting bitcoind" # bitcoind -daemon
############# # P2Pool setup #############
# Download and install p2pool cd git clone https://github.com/forrestv/p2pool.git # no actual install needed
# Start P2pool in the background. # Note that if you enable this in this script, P2Pool will spit out a few hundred errors until bitcoind is caught up. # So, disabling this here. # nohup python $HOME/p2pool/run_p2pool.py &
############# # cron setup #############
# Add the entries for the cron jobs PrintStatus "Setting up cron jobs" crontab -l > $HOME/Old_cron.txt echo "@reboot bitcoind -daemon" >> $HOME/Old_cron.txt echo "@reboot python $HOME/p2pool/run_p2pool.py" >> $HOME/Old_cron.txt crontab $HOME/Old_cron.txt rm $HOME/Old_cron.txt
# @TODO: Auto update and restart P2Pool (may or may not be desirable) # http://pastebin.com/HtD0uku4
EDIT 2014-06-20: I just noticed that the download of the blockchain file was disabled. That's because I was futzing around with the script the last time I used it, and I forgot to re-enable that line. It's corrected now.
|
|
|
|
raskul
|
|
June 17, 2014, 08:41:40 PM |
|
breakthrough moment.
I used to get all confused and dizzy when I looked at code. scanning through that box, Kyros, it would seem I may very well have 'learned a new language' I'll devote some time over the weekend to look into this in more depth.
|
tips 1APp826DqjJBdsAeqpEstx6Q8hD4urac8a
|
|
|
KyrosKrane
|
|
June 17, 2014, 08:52:03 PM |
|
This is also very help and really great, thanks for the contribution!
(Were you the guy who posted something similar about this on reddit?)
No, but I remember that post. It inspired me to create this script, and in fact, the section for setting up the bitcoin configuration file was adapted from that post.
|
|
|
|
murdof (OP)
|
|
June 18, 2014, 07:06:16 AM |
|
Good stuff Kyros!
I don't have in the guide any information about firewalling - so I guess this could be in the extras. I did add rpcallow=127.0.0.1 - but still people could find an exploit to that - so firewalling will be the best way to go.
What I understand is that people want to just get on a Linux box and just type 2 commands and set up everything. I don't think that people prefer to build things (I don't personally but I have to). So that's why I went with the binaries and a mini script to execute all the commands to have you up and running ASAP. I have tried the script in 3 scenarios already and works great. The only thing that I found is that when tar extracts some files are created with root as owner. Maybe I'm missing a tar option to use current user? Haven't checked.
All in all I invite you to try it out the 2 line script (you can just wget the script and check the contents) and let me know what you think.
|
|
|
|
nreal
Full Member
Offline
Activity: 932
Merit: 100
arcs-chain.com
|
|
June 18, 2014, 09:27:14 AM |
|
All make commands could go like make -j12.... if you have lets say 8 cores, it makes it alot faster that way... time is coins make -j12 -f makefile.unix
|
|
|
|
murdof (OP)
|
|
June 18, 2014, 08:37:04 PM |
|
huntercoin is fixed and compiles ok
still though downloading the blockchain so haven't tried it (for some reason downloading is very slow)
so when it has finished and i confirm it works with merge mining I will add it to the guide.
|
|
|
|
jonnybravo0311
Legendary
Offline
Activity: 1344
Merit: 1024
Mine at Jonny's Pool
|
|
June 18, 2014, 09:44:09 PM |
|
Hey murdof... just wanted to say good job on this - especially the "Lazy Man's" version. Personally, I hacked my way through setting everything up (Disclaimer: I have everything running natively on OSX Mavericks). The work you've done to compress the installation on Ubuntu 14.04LTS down to two commands is impressive. Nicely done!
|
Jonny's Pool - Mine with us and help us grow! Support a pool that supports Bitcoin, not a hardware manufacturer's pockets! No SPV cheats. No empty blocks.
|
|
|
KyrosKrane
|
|
June 19, 2014, 12:58:23 PM |
|
Something I just noticed: I think in the second sentence, you meant to say, "do not install Ubuntu Desktop 14.04 LTS" -- but either way, this setup should work just fine even on a Desktop install of Ubuntu. You'd just need more disk space for the graphical stuff. In the past, I've run plenty of server-type software (including bitcoind and p2pool) on the Desktop versions of Ubuntu. I'll spin up a VM with Ubuntu Desktop 14.04 installed and see how the script runs.
|
|
|
|
murdof (OP)
|
|
June 19, 2014, 01:42:18 PM |
|
Thanks Kyros,
fixed.
I don't think there is a point to run a server with GUI. Some people might want to run it on their main desktop on the other hand - so yes it would be good to know if you tried it and worked. Normally it should work ok.
|
|
|
|
Collider
|
|
June 19, 2014, 01:56:53 PM |
|
Murdof, how much of your RAM are you actually using on p2pool and Bitcoin client (aswell as the server OS)?
|
|
|
|
murdof (OP)
|
|
June 19, 2014, 02:16:34 PM |
|
Murdof, how much of your RAM are you actually using on p2pool and Bitcoin client (aswell as the server OS)?
I'm running with 4GB and it utilizes about 2.5GB. Huntercoin is something that takes up all CPU + Memory so I stopped it cause anyway I found it useless. Too slow and too many resources.
|
|
|
|
wknight
Legendary
Offline
Activity: 889
Merit: 1000
Bitcoin calls me an Orphan
|
|
June 19, 2014, 03:35:34 PM |
|
With memory being so cheap now I found 8 Gigs to be VERY comfortable running my p2pool node. no need to pop into swap file EVER!
|
Mining Both Bitcoin and Litecoin.
|
|
|
Collider
|
|
June 19, 2014, 04:03:09 PM |
|
While memory used to be more expensive, it is stil nice to know what the actual requirements are. I would therefore think it is fair to say that you should get away with a 3 GB Ram VPS safely. Most VPS providers allow you to exceed the allocated memory by atleast 10-20% anyway
|
|
|
|
wknight
Legendary
Offline
Activity: 889
Merit: 1000
Bitcoin calls me an Orphan
|
|
June 19, 2014, 04:24:46 PM |
|
While memory used to be more expensive, it is stil nice to know what the actual requirements are. I would therefore think it is fair to say that you should get away with a 3 GB Ram VPS safely. Most VPS providers allow you to exceed the allocated memory by atleast 10-20% anyway Totally passed up the idea of a VPS concept.. good point.. though i like to keep my memory on a even number call me old fashioned. 4GB Ram would be ideal and yes I have run it on a vps at 4GB and no problems.. course it was for litecoin The only reason I stay away from VPS is i want something local. Local is always the best results with p2pool shares
|
Mining Both Bitcoin and Litecoin.
|
|
|
raskul
|
|
June 19, 2014, 04:34:30 PM |
|
While memory used to be more expensive, it is stil nice to know what the actual requirements are. I would therefore think it is fair to say that you should get away with a 3 GB Ram VPS safely. Most VPS providers allow you to exceed the allocated memory by atleast 10-20% anyway Totally passed up the idea of a VPS concept.. good point.. though i like to keep my memory on a even number call me old fashioned. 4GB Ram would be ideal and yes I have run it on a vps at 4GB and no problems.. course it was for litecoin The only reason I stay away from VPS is i want something local. Local is always the best results with p2pool shares definitely, 100% agree, unfortunately I don't have the ability to run a local node, so VPS is a good option for folks like me. and with murdof's guide and how-to, those who can, can, and those who can't, can too!
|
tips 1APp826DqjJBdsAeqpEstx6Q8hD4urac8a
|
|
|
wknight
Legendary
Offline
Activity: 889
Merit: 1000
Bitcoin calls me an Orphan
|
|
June 19, 2014, 04:49:58 PM |
|
definitely, 100% agree, unfortunately I don't have the ability to run a local node, so VPS is a good option for folks like me. and with murdof's guide and how-to, those who can, can, and those who can't, can too!
Since i have no done it with bitcoin on a VPS.. how are the results? Are you seeing a lot more stales/DOA?
|
Mining Both Bitcoin and Litecoin.
|
|
|
|
wknight
Legendary
Offline
Activity: 889
Merit: 1000
Bitcoin calls me an Orphan
|
|
June 19, 2014, 05:25:57 PM |
|
3.18TH/s (1.7% DOA) so that is a bit of difference
|
Mining Both Bitcoin and Litecoin.
|
|
|
raskul
|
|
June 19, 2014, 05:28:29 PM |
|
3.18TH/s (1.7% DOA) so that is a bit of difference very good indeed.please always mine p2pool
|
tips 1APp826DqjJBdsAeqpEstx6Q8hD4urac8a
|
|
|
|