[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool

[lenny_]

Gyver, not quite. Share have a flag that indicates whether the node that generated it got a stale share in the past, and from those, nodes can compute the stale share percentage and then the total hash rate with an expression like: unstale_hash_rate / (1 - pool_stale_proportion)

Is it just a flag or a counter? We have whole network stats for DOA and Orphans too, are there 2 flags/counters?
With a counter the stale estimation should be pretty good. If it's a flag, multiple stales between shares would be miscounted as an unique stale.

What might be underestimated too is the amount of work from nodes which stop mining after stale shares.

Note that I don't think it's an actual problem. With around 9-10 GH/s, I've always seen above 100% PPS and I'm currently at around 115% PPS for the last active weeks (computed from my actual earnings vs actual hash power). It matches the :
- p2pool.info luck,
- my efficiency,
- the average tx fee percentage.
This is why I'm back mining on p2pool: hashpower promises at least 105% PPS when leasing and falls short due to high rejects, I compared my actual PPS over several weeks and stats are largely in favor of p2pool for me. I left p2pool because I had I/O issues and likes to test novelties, but now bitcoind and p2pool are both on SSDs and they are just flying (at least when I'm not rewiring the whole apartment...).

My first SSD should be arriving soon.  I'm seriously hoping it decreases my stale rate - like you, I think it's I/O related.

M

Have you got latest bitcoind from git? It's much faster than stable version, with new database. With latest bitcoind from git my GetWork Latency dropped about 5-10x times.

I'll try it on my server.  Sounds risky for my main wallet, but main wallet isn't used for mining.

Thanks!

M

That's are my results, on Intel Atom 230 1.6GHz HT + 1 GB RAM:

It was after switching from bitcoind 0.7.1 to newest from git

[rav3n_pl]

Remember to

Code:

strip bitcoind

[mdude77]

In case you didn't see my post, I wrote a .Net "widget" for Windows that allows you to monitor your miners on p2pool, eclipse, ozcoin, and 50btc.  More pools are coming as time permits (bitminter is next).

https://bitcointalk.org/index.php?topic=86502.0

M

[gyverlb]

If you have always seen above 100% PPS then you are missing including something.

Block+Fees average is currently 100.36% according to blockchain.info

Not sure which stat you are referring to (whole network over a period of time ?).

Anyway, my computations seem fine to me. That's not rocket science to compute expected income from one's hashrate and current difficulty and compare it to actual income over a large enough period (at least 4 weeks in my case). I have a whole spreadsheet dedicated to my mining activites and one sheet is dedicated to expected/actual mining income: I dutifully enter each mining related transaction and compute expected income for each day (with appropriate adjustments on the days difficulty changes).

If my results seems too good to you on p2pool, you miss one perfectly reasonable explanation. It could simply be luck the PPLNS system doesn't use 1-difficulty shares but far more difficult shares which introduce variance in your expected income (I've seen at least +/- 15%) for a block found. So I may just have been lucky when p2pool found blocks and unlucky when it didn't. But as I said my results are matching with my reported efficiency (between 102 and 106%, was mostly 105 and is around 104 recently), p2pool.info reported luck (~110%) and the 100.36% you report for block+fees (in fact I used 100.25%).

[gyverlb]

Nice explanation from forrestv on how p2pool is counting stales on #p2pool channel:

Code:

23:05 < forrestv> gyver, assuming the share chain is operating normally (no major
                  forks), the pool hash rate estimate is unbiased except for it being
                  a little lower than true due to people leaving before telling
                  everyone they got a stale by finding another share
23:06 < forrestv> each share has a flag that says either "i don't have any uncounted
                  dead/stale shares", "i have 1 uncounted dead share", or "i have 1
                  uncounted stale share"
23:07 < forrestv> your node keeps counts of how many dead/stale shares you've mined
                  and how many times you've successfully produced a share with one of
                  those flags
23:07 < forrestv> and it keeps producing shares with one of those flags until you've
                  notified everyone of your past dead/stale shares

I can live with the lower estimation. Stales to report, according to my experience from both the network as a whole and my nodes, are mostly between 6 and 12%. So on average we lose between 6% and 12% of the hashrate of people leaving p2pool (assuming their node isn't coming back later) for the average period between their last p2pool-valid shares. The unrealistic (and provably "not happening") worst case scenario is when everybody mining on p2pool would create nodes that would only mine for a period matching their own expected interval between shares. That could explain a lower estimate of hashrate matching an artificial 110% PPS result.
Conclusion for kano, stales don't explain the current 110%, variance is probably the biggest part and p2pool has an advantage vs other pools too: it broadcasts its blocks through a much larger set of bitcoind nodes. When we had latency problems we had quite frequent orphan blocks (and probably bad luck too) and averaged ~90% PPS. Now we rarely see orphans compared to what we saw at the time.

[rav3n_pl]

forrestv, please check my comment on last commit
https://github.com/forrestv/p2pool/commit/daa3ba3dfed955caceef49438f6d0067c7ea99a1

[mdude77]

Gyver, not quite. Share have a flag that indicates whether the node that generated it got a stale share in the past, and from those, nodes can compute the stale share percentage and then the total hash rate with an expression like: unstale_hash_rate / (1 - pool_stale_proportion)

Is it just a flag or a counter? We have whole network stats for DOA and Orphans too, are there 2 flags/counters?
With a counter the stale estimation should be pretty good. If it's a flag, multiple stales between shares would be miscounted as an unique stale.

What might be underestimated too is the amount of work from nodes which stop mining after stale shares.

Note that I don't think it's an actual problem. With around 9-10 GH/s, I've always seen above 100% PPS and I'm currently at around 115% PPS for the last active weeks (computed from my actual earnings vs actual hash power). It matches the :
- p2pool.info luck,
- my efficiency,
- the average tx fee percentage.
This is why I'm back mining on p2pool: hashpower promises at least 105% PPS when leasing and falls short due to high rejects, I compared my actual PPS over several weeks and stats are largely in favor of p2pool for me. I left p2pool because I had I/O issues and likes to test novelties, but now bitcoind and p2pool are both on SSDs and they are just flying (at least when I'm not rewiring the whole apartment...).

My first SSD should be arriving soon.  I'm seriously hoping it decreases my stale rate - like you, I think it's I/O related.

M

Have you got latest bitcoind from git? It's much faster than stable version, with new database. With latest bitcoind from git my GetWork Latency dropped about 5-10x times.

I managed to get ubuntu up and running a vm, and managed to get bitcoin up and running without too much of the usual linux shenanigans.  However, I see a big bold warning on the client that says this is a test build and should not be used for mining.  How are you supposed to test it if you can't use it in prod?  I'm sure not pointing GPUs to test.

M

[cabin]

for you guys running the latest git versions of bitcoind, are you building these locally or is there someone trustworthy doing nightly builds?

[rav3n_pl]

for you guys running the latest git versions of bitcoind, are you building these locally or is there someone trustworthy doing nightly builds?

Once you make compiling machine up and running it is not a problem to compile new git yourself even everyday

Code:

cd bitcoin
git pull
cd src
make -f makefile.unix bitcoind
strip bitcoind

[Red Emerald]

for you guys running the latest git versions of bitcoind, are you building these locally or is there someone trustworthy doing nightly builds?

Once you make compiling machine up and running it is not a problem to compile new git yourself even everyday

Code:

cd bitcoin
git pull
cd src
make -f makefile.unix bitcoind
strip bitcoind

I usually build with multiple cores and without UPNP support since it's easier.

Code:

make -f makefile.unix -j2 USE_UPNP= bitcoind

You will also need some dependencies.  On Ubuntu 12.04, I use:

Code:

sudo apt-get install -y git-core build-essential libssl-dev libboost-all-dev libdb5.1-dev libdb5.1++-dev libgtk2.0-dev

[twmz]

I see a big bold warning on the client that says this is a test build and should not be used for mining. 

I am curious if others are mining with this anyway.  I, also, tried the latest and then went back to the stable version after seeing this message.  Mostly because I have used git-versions of bitcoind a lot in the past and they generally never had warnings like this.  So I took the presence of this warning to be an indication that it was especially unwise to mine with it.  But does anyone actually know if it has serious issues or if the message is mostly about prudence?

[rav3n_pl]

Last git is killer for litecoin pool
http://pastebin.com/EGbXXAps
Nodes are disconnected over and over.

[ChanceCoats123]

Updated to v9. It seems I'm always one of the last ones to move over (because I'm not as active on these forums anymore). Could I possibly get someone to shoot me an email when a new binary is released? That way I don't get left behind, and we can move the transition to the next version along?

[twmz]

Updated to v9. It seems I'm always one of the last ones to move over (because I'm not as active on these forums anymore). Could I possibly get someone to shoot me an email when a new binary is released? That way I don't get left behind, and we can move the transition to the next version along?

Sign up here:  http://groups.google.com/group/p2pool-notifications

forrestv uses this to send subscribers notifications anytime there is a required new version (or other urgent communications).

[Mobius]

Updated to v9. It seems I'm always one of the last ones to move over (because I'm not as active on these forums anymore). Could I possibly get someone to shoot me an email when a new binary is released? That way I don't get left behind, and we can move the transition to the next version along?

Why not add a notify on this thread, otherwise if your not willing to update yourself, then why should anyone make that effort for you. If you want to benefit, then take a proactive approach and stop waiting for someone to spoon feed you. Grow up, we are not your mom and dad.

[btharper]

Updated to v9. It seems I'm always one of the last ones to move over (because I'm not as active on these forums anymore). Could I possibly get someone to shoot me an email when a new binary is released? That way I don't get left behind, and we can move the transition to the next version along?

Why not add a notify on this thread, otherwise if your not willing to update yourself, then why should anyone make that effort for you. If you want to benefit, then take a proactive approach and stop waiting for someone to spoon feed you. Grow up, we are not your mom and dad.

Well someone beat you to the punch with a polite, helpful answer. To add insult to injury your solution isn't really adequate; a notification on this thread would work the first time anyone else posted in this thread (nearly daily) and have relatively little correlation to the new versions coming out.

As a curiosity, does the p2pool client support forwarding a signed message similarly to how bitcoind does? Could this be added to the low-priority wishlist if not?

[lenny_]

Updated to v9. It seems I'm always one of the last ones to move over (because I'm not as active on these forums anymore). Could I possibly get someone to shoot me an email when a new binary is released? That way I don't get left behind, and we can move the transition to the next version along?

New: Mailing list for urgent news (updates, bugs): http://groups.google.com/group/p2pool-notifications

Subscribe here, you will got your e-mails

[rav3n_pl]

Updated to v9. It seems I'm always one of the last ones to move over (because I'm not as active on these forums anymore). Could I possibly get someone to shoot me an email when a new binary is released? That way I don't get left behind, and we can move the transition to the next version along?

Why not add a notify on this thread, otherwise if your not willing to update yourself, then why should anyone make that effort for you. If you want to benefit, then take a proactive approach and stop waiting for someone to spoon feed you. Grow up, we are not your mom and dad.

One of ways to keep node automagically updated: https://bitcointalk.org/index.php?topic=18313.msg836949#msg836949
Edit to your config, put it to cron weekly and forget.

[gyverlb]

One of ways to keep node automagically updated: https://bitcointalk.org/index.php?topic=18313.msg836949#msg836949
Edit to your config, put it to cron weekly and forget.

Here's the updated version I'm currently using (launched by cron everyday and at @reboot, see your crontab manual...):

http://pastebin.com/HtD0uku4

The pool.sh script it uses launches p2pool in the background with screen. You should write it according to your needs, something like:

Code:

screen -U -S p2pool -d -m python run_p2pool.py <your_options>

Makes it easy to connect with

Code:

screen -r p2pool

if needed.

[rav3n_pl]

Update progress:
v9: 81%
v8: 12%
v5: 6%
Rest is under 1%

I`ts final git pull time!

Latest binary 9.0-6-gf18f763 for win32 and win64 on my skydrive if any1 interested