[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool

[organofcorti]

I think there's something wrong with the p2Pool 7 day stats. If you calculate manually, the last 7 days is better than average luck. In fact, there's been better than average luck at p2Pool for the last 6 weeks now. Check the last 7 days for yourself:

Code:

          Timestamp   ActualShares ExpectedShares
1  1344745354.000000  547055.000000 2036671.000000
2  1344740792.000000 9116832.000000 2036671.000000
3  1344649552.000000  134207.000000 2036671.000000
4  1344648269.000000 2121584.000000 2036671.000000
5  1344627801.000000 3184026.000000 2036671.000000
6  1344596756.000000 7983068.000000 2036671.000000
7  1344487657.000000 3886934.000000 2036671.000000
8  1344430970.000000  839057.000000 2036671.000000
9  1344418762.000000 1225598.000000 2036671.000000
10 1344401496.000000  814265.000000 2036671.000000
11 1344389344.000000  395892.000000 2036671.000000
12 1344383399.000000 2019198.000000 2036671.000000
13 1344353201.000000  279105.000000 2036671.000000
14 1344349060.000000 1526951.000000 2036671.000000
15 1344325803.000000  644660.000000 2036671.000000
16 1344315777.000000   35732.000000 2036671.000000
17 1344315245.000000  271678.000000 2036671.000000
18 1344311286.000000 1395283.000000 2036671.000000
19 1344289928.000000 1986950.000000 2036671.000000
20 1344258323.000000  152132.000000 2036671.000000
21 1344256008.000000  815393.000000 2036671.000000
22 1344243762.000000 1675975.000000 2036671.000000

The average round length / difficulty = 0.91. This is better than average.

Your math and statistics always befuddle me.  But this one I think I can do.. but I'm not going to.  I'm going to assume your math is right, just question your conclusion.

p2pool.info shows the last 7 days as 0.89.  I could see rounding error being the difference between your 0.91 and p2pool.info's 0.89.

What I'm not understanding is how that's better than average?  I thought 1 was average?

M

What you do is take all the "ActualShares" and divide by all the "ExpectedShares", and then take the average (add all the ActualShares/ExpectedShares and divide by the number of rounds, 22 in this case). That average is 0.91. This means that the rounds were 9% shorter than expected which means you solved more blocks for your shares (or it took fewer shares to solve your blocks.)

Round length / D gets smaller with good luck and larger with bad luck, and is what I was referring to. I'm not sure how the luck is calculated on the p2Pool graphs or what it means, exactly.

Anyway, the available stats say the good luck is still rolling for p2Pool. Don't sell it short!

[1713919251]

1713919251

[organofcorti]

I think p2pool's hashrate is slightly underreported seeing as stales aren't included in the hashrate calculations even though stales can be blocks.

I don't think it's that. I'm using the same data the luck graphs should be using, and this is not just a few percent points of difference. This is going from shorter than average rounds (what the data shows) to longer than average rounds ( what the luck graphs show).

I'm not trying to pick a fight with the p2Pool.info guys - I think that website is invaluable. I just think the luck graphs need to be fixed.

[mdude77]

What you do is take all the "ActualShares" and divide by all the "ExpectedShares", and then take the average (add all the ActualShares/ExpectedShares and divide by the number of rounds, 22 in this case). That average is 0.91. This means that the rounds were 9% shorter than expected which means you solved more blocks for your shares (or it took fewer shares to solve your blocks.)

Round length / D gets smaller with good luck and larger with bad luck, and is what I was referring to. I'm not sure how the luck is calculated on the p2Pool graphs or what it means, exactly.

Anyway, the available stats say the good luck is still rolling for p2Pool. Don't sell it short!

It's just coincidental that your number is .02 different from the number on the stats page?

M

[organofcorti]

It's just coincidental that your number is .02 different from the number on the stats page?

M

Yes.

I don't know what the stats page is measuring for luck. At a guess, it's probably measuring the inverse of round length / D. So if you want to compare the actual result to anything, compare it to 1/0.89 which makes it about 20% different.

[racerguy]

I've been averaging 0.126btc/day over the last 37days on p2pool with my 5770, my guess is my avg hashrate is just below 200mhash/s(I game as well sometimes on the 5770).  This is well above 100%pps so I can confirm our recent good luck is real but the p2pool.info stats may be a bit off somehow.

On an unrelated point my 5770 is only 5$ away from paying for itself over those 37 days.

[mdude77]

I've been averaging 0.126btc/day over the last 37days on p2pool with my 5770, my guess is my avg hashrate is just below 200mhash/s(I game as well sometimes on the 5770).  This is well above 100%pps so I can confirm our recent good luck is real but the p2pool.info stats may be a bit off somehow.

On an unrelated point my 5770 is only 5$ away from paying for itself over those 37 days.

We were doing great until the last week.  We went from 3-7 blocks a day average to 1-3 blocks a day average, with lower payout per block.   (Note the p2pool.info stats show great numbers except for the last week.)

I'm going to do a week comparison.  I can now easily split my miners 50/50, so I have 2.6g/h going to ozcoin at 5% donation, and 2.6g/h going to p2pool. 

M

[racerguy]

I've been averaging 0.126btc/day over the last 37days on p2pool with my 5770, my guess is my avg hashrate is just below 200mhash/s(I game as well sometimes on the 5770).  This is well above 100%pps so I can confirm our recent good luck is real but the p2pool.info stats may be a bit off somehow.

On an unrelated point my 5770 is only 5$ away from paying for itself over those 37 days.

We were doing great until the last week.  We went from 3-7 blocks a day average to 1-3 blocks a day average, with lower payout per block.   (Note the p2pool.info stats show great numbers except for the last week.)

I'm going to do a week comparison.  I can now easily split my miners 50/50, so I have 2.6g/h going to ozcoin at 5% donation, and 2.6g/h going to p2pool. 

M

I don't know for sure but when talking about pools in the 300-500ghash range I don't think 1 week is long enough for luck to even out to a reasonable level(maybe organofcorti knows how long it would take). 

[organofcorti]

We were doing great until the last week.  We went from 3-7 blocks a day average to 1-3 blocks a day average, with lower payout per block.   (Note the p2pool.info stats show great numbers except for the last week.)

Comparing rounds per day isn't helpful unless you also compare hashrates. And yes, you earned less this week than last because there was even better luck last week. Even comparing your week's earnings with last weeks is tricky unless you have a large hashrate.

I'm going to do a week comparison.  I can now easily split my miners 50/50, so I have 2.6g/h going to ozcoin at 5% donation, and 2.6g/h going to p2pool. 

M

Good idea. But remember, your variance will be a bit higher at p2Pool so make sure you make the comparison over a substantial time period.

I don't know for sure but when talking about pools in the 300-500ghash range I don't think 1 week is long enough for luck to even out to a reasonable level(maybe organofcorti knows how long it would take). 

This too. I hadn't thought of that until racerguy brought it up. If p2pool has 1/4 the hashrate of Ozcoin, you should average your results for p2Pool over four times as long. Even at one week for Ozcoin and 4 weeks for p2Pool, you'd still have at 95% confidence interval of +/- 18%. And which week would you select for averaging at Ozcoin?

Still, it's a great idea mdude77, and getting more "hands on" in this fashion might help you get a feel for the differences between pools. Good luck, and a weekly update on your progress wouldn't go astray on the Weekly Pool Statistics thread.

[twmz]

I think there's something wrong with the p2Pool 7 day stats. If you calculate manually, the last 7 days is better than average luck. In fact, there's been better than average luck at p2Pool for the last 6 weeks now. Check the last 7 days for yourself:

Code:

          Timestamp   ActualShares ExpectedShares
1  1344745354.000000  547055.000000 2036671.000000
2  1344740792.000000 9116832.000000 2036671.000000
3  1344649552.000000  134207.000000 2036671.000000
4  1344648269.000000 2121584.000000 2036671.000000
5  1344627801.000000 3184026.000000 2036671.000000
6  1344596756.000000 7983068.000000 2036671.000000
7  1344487657.000000 3886934.000000 2036671.000000
8  1344430970.000000  839057.000000 2036671.000000
9  1344418762.000000 1225598.000000 2036671.000000
10 1344401496.000000  814265.000000 2036671.000000
11 1344389344.000000  395892.000000 2036671.000000
12 1344383399.000000 2019198.000000 2036671.000000
13 1344353201.000000  279105.000000 2036671.000000
14 1344349060.000000 1526951.000000 2036671.000000
15 1344325803.000000  644660.000000 2036671.000000
16 1344315777.000000   35732.000000 2036671.000000
17 1344315245.000000  271678.000000 2036671.000000
18 1344311286.000000 1395283.000000 2036671.000000
19 1344289928.000000 1986950.000000 2036671.000000
20 1344258323.000000  152132.000000 2036671.000000
21 1344256008.000000  815393.000000 2036671.000000
22 1344243762.000000 1675975.000000 2036671.000000

The average round length / difficulty = 0.91. This is better than average.

You didn't include this very long block in your calculation (and p2pool.info did):

Code:

23 1344216828.000000 11306170.000000 2036671.000000 

Probably you and p2pool.info just used a different definitions of "last 7 days".  p2pool.info includes rounds based on their end time (the timestamp of when the block was found).  So, the 7 day luck calculation include all blocks that were found in the 604,800 seconds (7 days) since the most recent block was found (not since "now").  

At the time of your post, the most recent block had been found at 1344745354.  So the cuttoff was 1344745354 - 604800 = 1344140554, which means that the long block found at 1344216828 was included in the 7 day range.

[organofcorti]

I should have thought of that, twmz. Yes, I take "now", subtract 7 days and include only rounds completed in that time. So i missed that stonking great round. Good news is your stats will show a much improved 7 day luck shortly

Which just goes to show that the 7 day luck isn't something mdude77 shouldn't be basing his mining decisions on just the last weeks rounds - the variance in such a short period of time can be huge.

[Krak]

My payout is higher than it should be for 1% of the pool hashpower.  Right now the pool is at 500g/h, I'm at 5 g/h.  That's 1%, so I should get a payout of .5btc.  The last couple days I've been around .6 to .65.  That tells me someone, or a bunch of someones out there aren't getting their fair share of hashes.

Include me in there, I've only found 5 shares in the past 24 hours.

What you do is take all the "ActualShares" and divide by all the "ExpectedShares", and then take the average (add all the ActualShares/ExpectedShares and divide by the number of rounds, 22 in this case). That average is 0.91. This means that the rounds were 9% shorter than expected which means you solved more blocks for your shares (or it took fewer shares to solve your blocks.)

Round length / D gets smaller with good luck and larger with bad luck, and is what I was referring to. I'm not sure how the luck is calculated on the p2Pool graphs or what it means, exactly.

Anyway, the available stats say the good luck is still rolling for p2Pool. Don't sell it short!

How is that done exactly? Does it count all of the high difficulty shares or does it translate them into difficulty 1 shares?

[organofcorti]

How is that done exactly? Does it count all of the high difficulty shares or does it translate them into difficulty 1 shares?

The pool hashrate and round duration are used to estimate the number of difficulty 1 shares.

[randomguy7]

My p2pool efficiency is only about 93%, someone has a clue what the reason for this could be?

I run my p2pool instance on a server with 100mbit/s connection, my latency to the server is about 20ms.
My miners are cgminer (with icarus boards) and btcminer (with ztex quads).
I rarely run any downloads on my local link to avoid an increase of latency and thereby stales. I even have qos set up to prefer my miners traffic to the normal web traffic while surfing. But efficiency still sucks hard.
Any ideas what else I could try?

[mdude77]

I should have thought of that, twmz. Yes, I take "now", subtract 7 days and include only rounds completed in that time. So i missed that stonking great round. Good news is your stats will show a much improved 7 day luck shortly

Which just goes to show that the 7 day luck isn't something mdude77 shouldn't be basing his mining decisions on just the last weeks rounds - the variance in such a short period of time can be huge.

It's not just the stats.  It's the results.  I know past results aren't a guarantee of the future.. but I'm seeing an awful lot of red, literally, because rounds are taking too long.  We used to get more blocks, and higher payouts.  Now it's less blocks, and smaller payouts.  That's a double whammy.

And it all started when the extra 120 g/h was added.

I'm hoping it's a bad streak.. but I have yet to see anyone indicate for sure that we don't have a scaling issue.

Any chance p2pool could be changed to auto adjust the share difficulty to a given miner based on hash rate?  I know I haven't taken the time to figure out what my ideal share difficulty is, and I'm sure I'm not the only one.

M

[Krak]

My p2pool efficiency is only about 93%, someone has a clue what the reason for this could be?

I run my p2pool instance on a server with 100mbit/s connection, my latency to the server is about 20ms.
My miners are cgminer (with icarus boards) and btcminer (with ztex quads).
I rarely run any downloads on my local link to avoid an increase of latency and thereby stales. I even have qos set up to prefer my miners traffic to the normal web traffic while surfing. But efficiency still sucks hard.
Any ideas what else I could try?

Try setting a cap on your incoming connections with --max-conns. I've noticed that nodes with a lot of incoming connections actually have higher dead/orphan rates than nodes with lower connections (I set my max connections to 5 and I almost always have perfect efficiency).

[mdude77]

My p2pool efficiency is only about 93%, someone has a clue what the reason for this could be?

I run my p2pool instance on a server with 100mbit/s connection, my latency to the server is about 20ms.
My miners are cgminer (with icarus boards) and btcminer (with ztex quads).
I rarely run any downloads on my local link to avoid an increase of latency and thereby stales. I even have qos set up to prefer my miners traffic to the normal web traffic while surfing. But efficiency still sucks hard.
Any ideas what else I could try?

For efficiency rate to be that low, it means your stale rate is pretty high.  Right?

I run two p2pools, on two different machines, with non standard ports.  I have them linked to each other with "-n".  I have my firewall set to forward both nonstandard ports to the respective pools.  My stale rate is ~5%, which right now gives me ~104% efficiency.   Right now I have 24 connections on one, and 11 on the other (was just restarted a few hours ago).

I've also found if I run p2pool on a machine with only one core (processor), my stale rate increases significantly.

Maybe that'll help.

M

[Krak]

Any chance p2pool could be changed to auto adjust the share difficulty to a given miner based on hash rate?  I know I haven't taken the time to figure out what my ideal share difficulty is, and I'm sure I'm not the only one.

Take a look here. https://bitcointalk.org/index.php?topic=18313.msg816322#msg816322

[randomguy7]

My p2pool efficiency is only about 93%, someone has a clue what the reason for this could be?

I run my p2pool instance on a server with 100mbit/s connection, my latency to the server is about 20ms.
My miners are cgminer (with icarus boards) and btcminer (with ztex quads).
I rarely run any downloads on my local link to avoid an increase of latency and thereby stales. I even have qos set up to prefer my miners traffic to the normal web traffic while surfing. But efficiency still sucks hard.
Any ideas what else I could try?

For efficiency rate to be that low, it means your stale rate is pretty high.  Right?

I run two p2pools, on two different machines, with non standard ports.  I have them linked to each other with "-n".  I have my firewall set to forward both nonstandard ports to the respective pools.  My stale rate is ~5%, which right now gives me ~104% efficiency.   Right now I have 24 connections on one, and 11 on the other (was just restarted a few hours ago).

I've also found if I run p2pool on a machine with only one core (processor), my stale rate increases significantly.

Maybe that'll help.

M

That's interesting, all three boxes (2 dedi servers, and my local miner box) I tried p2pool on had only one cpu core.

My stale rate usually isn't very high (the difficulty 1 shares), but the shares which meet the difficulty of the p2pool share chain have a higher stale rate than the pool average. Doesn't make a lot of sense to me but that are my numbers.

[Red Emerald]

It's not just the stats.  It's the results.  I know past results aren't a guarantee of the future.. but I'm seeing an awful lot of red, literally, because rounds are taking too long.  We used to get more blocks, and higher payouts.  Now it's less blocks, and smaller payouts.  That's a double whammy.

And it all started when the extra 120 g/h was added.

I'm hoping it's a bad streak.. but I have yet to see anyone indicate for sure that we don't have a scaling issue.

Any chance p2pool could be changed to auto adjust the share difficulty to a given miner based on hash rate?  I know I haven't taken the time to figure out what my ideal share difficulty is, and I'm sure I'm not the only one.

M

+1

It seems like we had our luck problem come up last time when the pool grew from 200 to 300 GH/s.  Maybe there is a scaling issue.

[eightbitbandit]

Did the latest update up the ram requirement significantly? I've had oom-killer kill the python process twice in 24hrs. I restarted it again this morning and it's already back up to 350mb. This box only has 1gb total, but it's had no problem running p2pool for weeks uninterrupted before. I finally just put an entry in crontab to restart the thing every 6 hrs.

on my Ubuntu-Server with 4GB Ram  bitcoind use 2.6% and Pyhthon 2.5% Memory

Odd. Our bitcoind's are only off by about 20mb, but my p2pool is using around 200mb more. I'm using the latest git pulls of both (as of yesterday anyway).

This is a Fedora 17 32bit box, if that matters.

Did you patch the Fedora yesterday?

I actually just did a few mins ago, but I haven't rebooted to apply the new kernel yet. According to my yum.log, my last python update was on June 03, to 2.7.3-3. Most of the stuff it pulled down today involved dhclient, perl, and the kernel.

For what it's worth, Fedora patched python on Aug 10th and my ballooning RAM dependencies disappeared. I'm holding steady at about 90mb, which is less than my instance of bitcoind (about 100mb). Just thought I'd drop a note to let others know. It's possible they fixed it earlier, because I updated to 2.7.3-7, vs the 2.7.3-3 I was having problems with. I just had a cron running every six hours that restarted my p2pool instance to keep things in check, heh.