Bitcoin Forum
November 09, 2024, 02:33:54 PM *
News: Latest Bitcoin Core release: 28.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1] 2 3 4 »  All
  Print  
Author Topic: complete CD/USB/PXE bootable p2pool miner - p2pcoin  (Read 24209 times)
kjj (OP)
Legendary
*
Offline Offline

Activity: 1302
Merit: 1026



View Profile
February 23, 2012, 04:29:25 AM
Last edit: December 18, 2013, 04:49:33 PM by kjj
 #1

I started with linuxcoin and my headless mining scripts, and I've added p2pool and some new tricks.  I'm calling it p2pcoin for lack of a better name.

* Network provisioning - checks DNS for a URL to fetch the config file - many config options can be specified in the file or fetched from URLs
* BFGminer pool list is localhost, then up to 10 PEER[0-9]= lines, then BACKUP_POOL=, then TEMP_POOL=
* uses "-S all" in bfgminer to find PGA/ASIC devices and "-S opencl:auto" to find Radeons
* full bitcoind node
* full namecoind node
* full p2pool node
* can use rsync to fetch both block chains
* can use persistence, but works much better using RAM drives (needs ~24 GB of RAM or flash or both if you want to run bitcoind/p2pool)
* works great from PXE
* works as a dumb miner if it can't run bitcoind

It still has a bunch of ugly hacks and quirks, but works quite well.  It attempts to handle local errors the best it can, and has a heartbeat function to report stats every minute.  I use the heartbeat stats to monitor each box and trigger a network power strip to reboot boxes that are seriously stuck.

Note:  The high share speed of p2pool might wear out your flash drive.  If this bothers you, don't use persistence, or just boot from CD or PXE.

Usage example (my mining farm):
I have one box with 32 GB RAM, and several boxes with much less.  All use PXE to boot, none have persistent storage.  All boxes fire up bfgminer as soon as they are booted.  The big box uses rsync to fetch a copy of the block chain from a server on my network, then reindexes it and eventually p2pool starts.  The smaller boxes detect that they are unable to run bitcoind, and just run as dumb miners.  While the p2pool box is starting up, all bfgminer instances (including on the p2pool box) fail down their list to an external pool.  Once p2pool is ready, they all switch back to the local pool.



Current - Version 0.7.0: p2pcoin-0.7.0.iso.torrent - 2013-12-18 - Updated bitcoind to 0.8.6, updated bfgminer to 3.8.1, updated p2pool to 13.4. 



Version 0.0: p2pcoin-0.0.iso.torrent - 2012-02-26 - initial early testing release
Version 0.1: p2pcoin-0.1.iso.torrent - 2012-02-28 - p2pool updated to 0.9.1 - still early testing
Version 0.2: p2pcoin-0.2.iso.torrent - 2012-03-09 - p2pool updated to 0.9.2, bitcoind updated to 0.6.0rc2 - fixed logrotate problem
Version 0.3: p2pcoin-0.3.iso.torrent - 2012-03-30 - p2pool updated to 0.10.3, bitcoind updated to 0.6.0final
Version 0.3.1: p2pcoin-0.3.1.iso.torrent - 2012-04-23 - fixed CD boot problem
Version 0.4: p2pcoin-0.4.iso.torrent - 2012-05-03 - updated p2pool to 0.11.1
Version 0.4.1: p2pcoin-0.4.1.iso.torrent - 2012-05-23 - changed RAMdisk settings, upgraded to p2pool 0.11.2
Version 0.6: p2pcoin-0.6.0.iso.torrent - 2013-01-26 - Tons of changes.  Now using BFGminer
Current - Version 0.6.4: p2pcoin-0.6.4.iso.torrent - 2013-07-07 - Updated bitcoind to 0.8.3, updated bfgminer to 3.1.1, updated p2pool to 13.0. 

17Np17BSrpnHCZ2pgtiMNnhjnsWJ2TMqq8
I routinely ignore posters with paid advertising in their sigs.  You should too.
ahitman
Sr. Member
****
Offline Offline

Activity: 302
Merit: 250


View Profile
February 25, 2012, 12:01:28 AM
 #2

Id be interested in testing it out, but Ive always used cgminer. I can try it out on my hardware for a bit while Im at home. Let me know how to set this up.
ahitman
Sr. Member
****
Offline Offline

Activity: 302
Merit: 250


View Profile
February 25, 2012, 02:22:33 AM
 #3

Also I would suggest renaming this thread to : p2pool on USB fully installed ready to go
or something along those lines, as Im sure people are interested in running p2pool from a usb and not have to worry about installing anything, they just missed this thread.
kjj (OP)
Legendary
*
Offline Offline

Activity: 1302
Merit: 1026



View Profile
February 26, 2012, 05:59:04 PM
 #4

p2pcoin-0.0.iso.torrent

I'm setting up the seeds for the torrent now.

This ISO works as a bootable CD, or you can use unetbootin to write it to a USB stick, or you can extract the files and set it up for PXE booting on your network.

The easiest way to use it is with network configuration.  To do that, you need to add a TXT record to your DNS.  This needs to coordinate with your DHCP server.  If your DHCP servers provide "example.com" as your default search domain, the record needs to be "p2pcoin.example.com", etc.  The answer returned needs to be a URL where the box can download the configuration file.  In bind 9, it would look like this:

Code:
p2pcoin   (tab)   TXT   (tab) "http://www.example.com/p2pcoin/options.txt"

When it boots, it will append "?id=" and the MAC address of eth0 to the URL and then download it.  This way, you can specify different options for different boxes, if you need to.

The configuration file takes a bunch of options.  The parser is really dumb, so you need to get the format right.  Every option needs to be the exact key in all caps, an equal sign, and the value.  No spaces, no tabs.

Look in /etc/p2pcoin.defaults for examples of what this file should look like.

BITCOIN_CHAIN_SOURCE=a URL that rsync can use to download blk0001.dat and blkindex.dat (and eventually blk0002.dat, etc)
NAMECOIN_CHAIN_SOURCE=a URL that rsync can use to download the chain files for namecoin (blk*.dat and nameindex.dat)
TEMP_POOL=the full URL for the regular mining pool that you want to use during bootup, before p2pool starts
BACKUP_POOL=the full URL for the backup mining pool that you want to use if there is a problem after p2pool starts, or if the TEMP_POOL has a problem during bootup
BTC_ADDRESS=a bitcoin address where you want your earnings to go
NMC_ADDRESS=a namecoin address where you want your earnings swept to
BTC_ADDR_SOURCE=a URL that returns a bitcoin address
NMC_ADDR_SOURCE=a URL that returns a namecoin address
DONATE=1  (the percentage that you want to donate to the author of p2pool (not to me))
PHOENIX_DEFAULTS_SOURCE=a URL that returns a file with phoenix configuration lines for your cards

You don't need both BTC_ADDRESS and BTC_ADDR_SOURCE, one or the other is fine.  Same goes for the NMC versions.  It adds ?id=MAC to those web requests if you include them.

If you include a PHOENIX_DEFAULTS_SOURCE, that format is tricky too.  It needs to be the exact output of aticonfig --list-adapters, then a tab, and the options to be passed to phoenix when mining on p2pool, then another tab, and the exact options to be passed to phoenix when mining on the startup pool.  Look at /etc/phoenix.defaults for examples of this file.

Side note, please send any good configs to me so that I can include them in future releases.

If you are not using network configuration, you should edit /etc/p2pcoin.defaults and /etc/phoenix.defaults directly, which means that you need to set up persistent storage.  To do that, you need to have a ext2/3/4 filesystem available on the machine labelled "live-rw".  If you are using a USB stick, you can just partition it with a 1 GB FAT partition bootable, and use the rest for persistence.  If you are booting from a CD, you can use a USB stick or a hard drive for storage.

If you have enough RAM, which is about 4 GB, it tries to run bitcoind and namecoind completely out of RAM disk, which is very fast.  If you don't have enough RAM for that, you will need about 4 GB of persistent storage available.

I've probably forgotten a bunch of stuff.  Let me know if anyone has any questions.

17Np17BSrpnHCZ2pgtiMNnhjnsWJ2TMqq8
I routinely ignore posters with paid advertising in their sigs.  You should too.
DeathAndTaxes
Donator
Legendary
*
Offline Offline

Activity: 1218
Merit: 1079


Gerald Davis


View Profile
February 26, 2012, 06:04:05 PM
 #5

p2pool and bitcoind generate a lot of small writes.  Nobody is worried about this killing usb drives?

I use usb linux on each of my rigs (6) and then point them all to a non-mining machine running bitcoind & p2pool on a HDD.
kjj (OP)
Legendary
*
Offline Offline

Activity: 1302
Merit: 1026



View Profile
February 26, 2012, 06:06:15 PM
 #6

Oh, one other thing.  Bitcoin earnings are generated directly to the address provided, so it is no big deal if the wallet created by bitcoind is lost every time you reboot when using the RAM drive.

The same is not true of namecoin.  There is a script that runs from cron and checks the wallet balance every hour.  If it finds anything, it attempts to send it to the address provided.  This script is not well tested, and even if it were, you'd still have a huge window where a reboot could eat your wallet, and thus coins.  Please keep that in mind.

17Np17BSrpnHCZ2pgtiMNnhjnsWJ2TMqq8
I routinely ignore posters with paid advertising in their sigs.  You should too.
kjj (OP)
Legendary
*
Offline Offline

Activity: 1302
Merit: 1026



View Profile
February 26, 2012, 06:08:28 PM
 #7

p2pool and bitcoind generate a lot of small writes.  Nobody is worried about this killing usb drives?

I use usb linux on each of my rigs (6) and then point them all to a non-mining machine running bitcoind & p2pool on a HDD.

If you have 4 GB of RAM, it uses ram disk.  Also, no, I'm not worried about killing the USB drive even a little bit.  It would take months or years, and you can get 16 GB sticks for under $20 now.

17Np17BSrpnHCZ2pgtiMNnhjnsWJ2TMqq8
I routinely ignore posters with paid advertising in their sigs.  You should too.
rjk
Sr. Member
****
Offline Offline

Activity: 448
Merit: 250


1ngldh


View Profile
February 26, 2012, 06:09:33 PM
Last edit: February 26, 2012, 06:19:52 PM by rjk
 #8

Oh, one other thing.  Bitcoin earnings are generated directly to the address provided, so it is no big deal if the wallet created by bitcoind is lost every time you reboot when using the RAM drive.

The same is not true of namecoin.  There is a script that runs from cron and checks the wallet balance every hour.  If it finds anything, it attempts to send it to the address provided.  This script is not well tested, and even if it were, you'd still have a huge window where a reboot could eat your wallet, and thus coins.  Please keep that in mind.
I don't know how BAMT deals with persistency, but couldn't you do something similar to prevent issues like this? Obviously this would need a USB stick and not a CD boot.

Mining Rig Extraordinaire - the Trenton BPX6806 18-slot PCIe backplane [PICS] Dead project is dead, all hail the coming of the mighty ASIC!
DeathAndTaxes
Donator
Legendary
*
Offline Offline

Activity: 1218
Merit: 1079


Gerald Davis


View Profile
February 26, 2012, 06:14:55 PM
 #9

p2pool and bitcoind generate a lot of small writes.  Nobody is worried about this killing usb drives?

I use usb linux on each of my rigs (6) and then point them all to a non-mining machine running bitcoind & p2pool on a HDD.

If you have 4 GB of RAM, it uses ram disk.  Also, no, I'm not worried about killing the USB drive even a little bit.  It would take months or years, and you can get 16 GB sticks for under $20 now.

Hmm. Interesting.  So it doesn't keep p2pool & blockchain persistent (written to usb drive).  What happens on reboots?
kjj (OP)
Legendary
*
Offline Offline

Activity: 1302
Merit: 1026



View Profile
February 26, 2012, 06:17:13 PM
 #10

Oh, one other thing.  Bitcoin earnings are generated directly to the address provided, so it is no big deal if the wallet created by bitcoind is lost every time you reboot when using the RAM drive.

The same is not true of namecoin.  There is a script that runs from cron and checks the wallet balance every hour.  If it finds anything, it attempts to send it to the address provided.  This script is not well tested, and even if it were, you'd still have a huge window where a reboot could eat your wallet, and thus coins.  Please keep that in mind.
I don't know how BAMT deals with persistency, but couldn't you do something similar to prevent issues like this? Obviously this would need a UBS stick and not a CD boot.

What I'm thinking of doing in a future release will be to give the option to download the wallet.dat for namecoin from an external source.  There are some security implications of that, of course, but since p2pcoin nodes should be running behind firewalls, it may be a good choice for people that care about the chance of losing their namecoin earnings.

17Np17BSrpnHCZ2pgtiMNnhjnsWJ2TMqq8
I routinely ignore posters with paid advertising in their sigs.  You should too.
rTech
Sr. Member
****
Offline Offline

Activity: 305
Merit: 250


Trust but confirm!


View Profile
February 26, 2012, 06:20:05 PM
 #11

Im not enought experienced to use this but i love to help new projects, so i dl this and keep it on my torrent client for future downloaders.
kjj (OP)
Legendary
*
Offline Offline

Activity: 1302
Merit: 1026



View Profile
February 26, 2012, 06:24:22 PM
 #12

p2pool and bitcoind generate a lot of small writes.  Nobody is worried about this killing usb drives?

I use usb linux on each of my rigs (6) and then point them all to a non-mining machine running bitcoind & p2pool on a HDD.

If you have 4 GB of RAM, it uses ram disk.  Also, no, I'm not worried about killing the USB drive even a little bit.  It would take months or years, and you can get 16 GB sticks for under $20 now.

Hmm. Interesting.  So it doesn't keep p2pool & blockchain persistent (written to usb drive).  What happens on reboots?

During a clean reboot, it tries to synchronize the bitcoin and namecoin chains to persistent storage, if you have it.  And then when it boots, if it sees a local copy, and if that local copy isn't more than 2 weeks old, it will try to use it, rather than fetching it from the network again.

No attempt is made to save the p2pool database.  Your shares are already in the cloud, so there isn't any point.

17Np17BSrpnHCZ2pgtiMNnhjnsWJ2TMqq8
I routinely ignore posters with paid advertising in their sigs.  You should too.
kjj (OP)
Legendary
*
Offline Offline

Activity: 1302
Merit: 1026



View Profile
February 26, 2012, 06:28:37 PM
 #13

Im not enought experienced to use this but i love to help new projects, so i dl this and keep it on my torrent client for future downloaders.

Cool, thanks for seeding.

And if you'd like to try it, I'd be happy to try to walk you through setting it up.

17Np17BSrpnHCZ2pgtiMNnhjnsWJ2TMqq8
I routinely ignore posters with paid advertising in their sigs.  You should too.
rTech
Sr. Member
****
Offline Offline

Activity: 305
Merit: 250


Trust but confirm!


View Profile
February 26, 2012, 07:11:45 PM
 #14

No problem mate im happy to help to share. But for thing itself i do not have atm time to try. Maybe next weekend. But anyway you should edit torrent file in your first post and start keeping revision list also. That way its more easier to download always latest.
kjj (OP)
Legendary
*
Offline Offline

Activity: 1302
Merit: 1026



View Profile
February 28, 2012, 12:20:26 PM
 #15

So, I think that about a half dozen people have downloaded this so far.  Did anyone get it working?  Does anyone have questions or comments?

17Np17BSrpnHCZ2pgtiMNnhjnsWJ2TMqq8
I routinely ignore posters with paid advertising in their sigs.  You should too.
Raoul Duke
aka psy
Legendary
*
Offline Offline

Activity: 1358
Merit: 1002



View Profile
February 28, 2012, 02:32:13 PM
 #16

p2pool and bitcoind generate a lot of small writes.  Nobody is worried about this killing usb drives?

I would be more worried about this killing Hard Disk Drives... Yes, the bitcoin blockchain already killed one of my 500GB drives. That's what I got for running it for months non-stop in a windows environment.

Simple explanation: Blockchain fragmentation killed the disk. <-- How do I know? Because I had to recover the blockchain from the disk and as soon as it hit the fragmented part it was sloooooooooow as hell!
rTech
Sr. Member
****
Offline Offline

Activity: 305
Merit: 250


Trust but confirm!


View Profile
February 28, 2012, 02:47:28 PM
 #17

Added another line to keep up share. Now im sharing it via 1mb upload and 10mb upload. It should give good speed atleast european downloaders.
lodcrappo
Hero Member
*****
Offline Offline

Activity: 616
Merit: 506


View Profile
February 28, 2012, 06:52:19 PM
 #18

p2pool and bitcoind generate a lot of small writes.  Nobody is worried about this killing usb drives?

I use usb linux on each of my rigs (6) and then point them all to a non-mining machine running bitcoind & p2pool on a HDD.

If you have 4 GB of RAM, it uses ram disk.  Also, no, I'm not worried about killing the USB drive even a little bit.  It would take months or years, and you can get 16 GB sticks for under $20 now.

I would be careful here.  After months of field testing with BAMT (a usb based linux that makes every attempt not to write to the USB more than required) we see that some USB keys will die after a surprisingly short time (sometimes only a few weeks, with considerably less writes than running p2pool would generate).  It varies greatly from one model of key to another, but I would expect crappy keys to die in a matter of days running p2pool.  Maybe you don't care, but it would be best to warn users of this potential clearly.




kjj (OP)
Legendary
*
Offline Offline

Activity: 1302
Merit: 1026



View Profile
February 28, 2012, 07:26:06 PM
 #19

p2pool and bitcoind generate a lot of small writes.  Nobody is worried about this killing usb drives?

I use usb linux on each of my rigs (6) and then point them all to a non-mining machine running bitcoind & p2pool on a HDD.

If you have 4 GB of RAM, it uses ram disk.  Also, no, I'm not worried about killing the USB drive even a little bit.  It would take months or years, and you can get 16 GB sticks for under $20 now.

I would be careful here.  After months of field testing with BAMT (a usb based linux that makes every attempt not to write to the USB more than required) we see that some USB keys will die after a surprisingly short time (sometimes only a few weeks, with considerably less writes than running p2pool would generate).  It varies greatly from one model of key to another, but I would expect crappy keys to die in a matter of days running p2pool.  Maybe you don't care, but it would be best to warn users of this potential clearly.

If you are doing everything you can to avoid writes, and you are still killing drives, the problem is the drives.  I'll put a note at the top, but the answer in p2pcoin is to boot from CD or PXE if anyone is concerned about their flash drive.  Or even just not bother taking the extra time to add persistent storage.

17Np17BSrpnHCZ2pgtiMNnhjnsWJ2TMqq8
I routinely ignore posters with paid advertising in their sigs.  You should too.
Red Emerald
Hero Member
*****
Offline Offline

Activity: 742
Merit: 500



View Profile WWW
February 28, 2012, 07:31:50 PM
 #20

The name is confusing.  Makes me think it's another altcoin.

Pages: [1] 2 3 4 »  All
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!