[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool

[Ente]

Just my fucking luck...  My dedicated miner runs for 10 days straight without issue, and on the day that p2pool finds a billion blocks, I come back to a rig that won't give video. Restart and everything is fine and dandy again. WTF is that?!?! It's like every 10-12 days this shit happens and on the day p2pool has good luck, my share count is 1/6 what it typically is. 

..let me know a few hours before your rig goes down, next time, ok? I would like a "warning, blockfinding-spree ahead!" warning system! ;-)

It all evens out after some time.

Ente

Actually ... no it doesn't.
If you lose out in a good luck spree there is no guarantee whatsoever that you will get it back later.
There is no memory in bitcoin hashing.

There is no memory in bitcoin hashing.

Exactly!
That is exactly why it evens out after some time.
If you roll dice, and have a particulary unlucky streak with four "1" in a row, that is bad luck. And happens quite rarely too. But as you go on, after some time, you will eventually hit four "6" in a row. Then it will have evened out. Sure, you will have dozens of "three 1" and "three 6" in between, maybe even another "four 1" too. But if you roll long enough, and the dice *really* dont remember anything (i.e. not broken or manipulated), you will eventually have an average of exactly 3. It may take a few hundreds or many thousands of rolls, depending on how many decimals you want to converge to "3".

Anyway, I think we are talking about the same here ;-)

(And to me it is not entirely sure yet that the *whole* p2pool setup is working entirely as expected and calculated. There seem to be an additional, constant small-% loss in the way.. observing the "luck.png" graphs..)

Ente

[Smoovious]

um...

average out to 3.5 ... not 3

otherwise, correct

although, with dice, you're always there for each roll you make, so all of the luck is your own, good or bad... since the numbers in BTC continue to keep rolling whether you're playing or not, you could conceivably, consistently miss out on the good luck periods, and stay in on the bad luck periods. You'd never know when those periods come up tho, so that is another level of luck.


-- Smoov

[DeathAndTaxes]

Just my fucking luck...   My dedicated miner runs for 10 days straight without issue, and on the day that p2pool finds a billion blocks, I come back to a rig that won't give video. Restart and everything is fine and dandy again. WTF is that?!?! It's like every 10-12 days this shit happens and on the day p2pool has good luck, my share count is 1/6 what it typically is.  

You're not alone, I made the decision to use my computer to spend the day playing Borderlands and therefore not mining (so only .15BTC today)

The reason why pools exist in the first place - to reduce variance.
P2Pool just happens to reduce it way less than the other pools.

That is just weak. Name me a pool where he would have been compensated for NOT mining?

He wasn't mining, it isn't like he had a 10 minute reboot window and lost 100% of the block value for 3 blocks.  He still got credit on each block for the portion of time he was mining in the last 24 hours.  The n used by p2pool is ~4.  Most pools PPLNS pools use an n of ~1.  There is no pool under a similar scenario where he would have been compensated more.

[Ente]

um...
average out to 3.5 ... not 3

:-)

Ente

[organofcorti]

..let me know a few hours before your rig goes down, next time, ok? I would like a "warning, blockfinding-spree ahead!" warning system! ;-)

It all evens out after some time.

Ente

Actually ... no it doesn't.
If you lose out in a good luck spree there is no guarantee whatsoever that you will get it back later.
There is no memory in bitcoin hashing.

Kano: Central limits theorem.

[kano]

Just my fucking luck...   My dedicated miner runs for 10 days straight without issue, and on the day that p2pool finds a billion blocks, I come back to a rig that won't give video. Restart and everything is fine and dandy again. WTF is that?!?! It's like every 10-12 days this shit happens and on the day p2pool has good luck, my share count is 1/6 what it typically is.  

You're not alone, I made the decision to use my computer to spend the day playing Borderlands and therefore not mining (so only .15BTC today)

The reason why pools exist in the first place - to reduce variance.
P2Pool just happens to reduce it way less than the other pools.

That is just weak. Name me a pool where he would have been compensated for NOT mining?

He wasn't mining, it isn't like he had a 10 minute reboot window and lost 100% of the block value for 3 blocks.  He still got credit on each block for the portion of time he was mining in the last 24 hours.  The n used by p2pool is ~4.  Most pools PPLNS pools use an n of ~1.  There is no pool under a similar scenario where he would have been compensated more.

I can name no pool where he would be compensated for not mining.
Though DGM will compensate you after you stop - but that's for mining you have already done.
Edit: hmm though that's similar to p2pool in the respect of being paid for your work after you stop anyway.

As for compensation - well that's an easy one to explain.
It's called granularity (and rounding) when things change.

Say there's 10,000 items - in groups of 10 and each group takes 10s to find.
If I get 1 for every group I find I will of course get 1,000 if I find all of them and it will take 10,000s.

Say there's 10,000 items - in groups of 1,000 and each group takes 1,000s to find.
If I get 100 for every group I find I will of course still get 1,000 if I find all of them and it will take 10,000s again.

However, if I stop collecting 500s before I find them all what will I get?
In the first case I will get 950
In the second case I will get 900

Meaning of all this?
When you stop mining, the amount of time you spent on getting nothing (wasted) will on average be half the time for you to find a share unless you plan waiting and stopping WHEN you find a share.
For P2Pool the amount of wasted time will of course average out to be the P2Pool difficulty times more than a 1 difficulty pool for your hash rate.

All that above is of course what I was referring to

[DeathAndTaxes]

When you stop mining, the amount of time you spent on getting nothing (wasted) will on average be half the time for you to find a share unless you plan waiting and stopping WHEN you find a share.
For P2Pool the amount of wasted time will of course average out to be the P2Pool difficulty times more than a 1 difficulty pool for your hash rate.

That would be true if you worked TOWARDS a share.  However you don't each hash is independent.  Randomly turning your computer on and off wouldn't affect your payout on 1 diff shares vs 1000 diff shares as long as the "on time" is the same.  If you were on 50% you would in the long run earn 50% regardless of the share size.

[kano]

When you stop mining, the amount of time you spent on getting nothing (wasted) will on average be half the time for you to find a share unless you plan waiting and stopping WHEN you find a share.
For P2Pool the amount of wasted time will of course average out to be the P2Pool difficulty times more than a 1 difficulty pool for your hash rate.

That would be true if you worked TOWARDS a share.  However you don't each hash is independent.  Randomly turning your computer on and off wouldn't affect your payout on 1 diff shares vs 1000 diff shares as long as the "on time" is the same.  If you were on 50% you would in the long run earn 50% regardless of the share size.

Nope, that was the point of my example.
It's to do with when you stop mining and the average expected amount of time you will mine and gain nothing for it.
(which will be - on average half the average expected time it takes to find a share)

Edit: yes of course if you resume mining the total expected shares is still based on the time you mine - I'm just referring to if you stop mining.

[DeathAndTaxes]

Nope, that was the point of my example.
It's to do with when you stop mining and the average expected amount of time you will mine and gain nothing for it.
(which will be - on average half the average expected time it takes to find a share)

I get that it was your point but it is still wrong.

Nothing is lost.  Saying the amount of time mined and you gain nothing is a fallacy.  Also stopping right after you find a share or block to minimize "lost work" is also a fallacy.  The very next hash could just as likely be another share or block.

Still the issue you were "helping" with had nothing to do with granularity.  The user wasn't mining for HOURS.  In any PPLNS or DGM pool his compensation likely would have been ZERO.  p2pool due to the fact that it has n = ~4.5 kept shares earned in the last 24 hours and resulted in some compensation.

[kano]

Well I edited my reply - but you posted too fast
My example was just about stop mining.
I guess I ignored the start mining again bit - which (yes I agree) means it's simply just the amount of time he mines.

Hmm - well - then I guess my example is wrong/bad coz we aren't referring to a case of stop mining permanently.
OK ignore me then

[Gabi]

I don't know what you guys are doing but p2pool is at 401Gh/s right now, and today was like always at 380-390



Keep up the good work!

[DeathAndTaxes]

It seems to be bouncing around today.  Saw a high of 380 earlier then it fell down to 330 ish then popped back up to 400.  I guess some new users are trying it out.

[Aseras]

I threw my little farm at it for an hour or so to test something, may be back soon.

[Proofer]

..let me know a few hours before your rig goes down, next time, ok? I would like a "warning, blockfinding-spree ahead!" warning system! ;-)

It all evens out after some time.

Ente

Actually ... no it doesn't.
If you lose out in a good luck spree there is no guarantee whatsoever that you will get it back later.
There is no memory in bitcoin hashing.

Kano: Central limits theorem.

IMHO Kano is correct, and I don't see the relevance of the Central Limit Theorem to the question of whether bad luck in the past will be compensated for in the future.  The idea that tossing a fair coin and getting, say, six heads in a row will be "evened out" by disproportionally more tails in future tosses is known as the Gambler's Fallacy.  If you've had bad [or good] luck in the past, then your lifetime expectation for luck is bad [or good]; you cannot change the past -- it is the baseline for your future.  More generally, probability theory deals with unknown (often future) events, not known past events.

[Ente]

IMHO Kano is correct, and I don't see the relevance of the Central Limit Theorem to the question of whether bad luck in the past will be compensated for in the future.  The idea that tossing a fair coin and getting, say, six heads in a row will be "evened out" by disproportionally more tails in future tosses is known as the Gambler's Fallacy.  If you've had bad [or good] luck in the past, then your lifetime expectation for luck is bad [or good]; you cannot change the past -- it is the baseline for your future.  More generally, probability theory deals with unknown (often future) events, not known past events.

No, of course not "compensated" as a reaction to the bad luck streak. But "evened out" in the sense that this single bad luck streak has less and less influence as the time or number of draws rise.

I am pretty sure we all are more or less talking about the same thing, and actually agreeing. How about we say "there are bad times, there are good times, everything is fine", and leave that page-long non-p2pool-related statistics/probability stuff?

The still unanswered question remains:
Is p2pool in fact statistically "working", in the sense that 100 Th will yield the [statistically] same amount of blocks as 100 Th on a single bitcoind or centralized pool?

Ente

[Gabi]

Yes.

[Shadow383]

The still unanswered question remains:
Is p2pool in fact statistically "working", in the sense that 100 Th will yield the [statistically] same amount of blocks as 100 Th on a single bitcoind or centralized pool?

Ente

I'm beginning to suspect not.
back when the pool was at 91% luck over 90 days, the odds of it having such a poor reward after so much hashing was ~2%.
It's obviously lower than that now. I'm giving it another 48 hours then I'm going back to abcpool 

[Gabi]

Please stop spreading FUD  P2pool is open source and there is nothing suspect in the code. Do you know what variability and luck is?

[Shadow383]

Please stop spreading FUD  P2pool is open source and there is nothing suspect in the code. Do you know what variability and luck is?

Yes, what I'm suggesting is that we may be having issues with orphaned blocks or some such issue.
Just saying, the odds of luck as bad as p2pool now displays is less than 1%

[Ente]

Please stop spreading FUD   P2pool is open source and there is nothing suspect in the code. Do you know what variability and luck is?

All right.

This is what the "luck" graph looks like:

(as can be seen here: http://u.forre.st/p2pool/luck.png)


This is what "variability and luck" would make me expect it look instead:



But, in fact, it looks much more like this, which is not "variability and luck", but a inherent problem besides variability:


The red line is like the average block-finding rate. Do you see the angle between the black "hashes calculated" graph and the red "blocks found" graph? THAT is what I am talking about, not the stochastic ripple in the original blue line, which I crudely smoothed out with the red line.

It does not seem to be caused by block orphans, as people stated before that these are in the low % range. Neither has it anything to do with local orphans or local deads, as these don't add to the (black) "hashes calculated" neither.


I am not confident enough to calculate the "we are losing x % hashingpower" from this graphs, maybe someone else is. To me it seems clear to be a constant "loss", therefore the two graphs resemble two lines with an angle.
My wild guess would be we are effectively losing 15% hashingpower, in the sense that we have 15% less blocks, since 2 months.

Ente

edit: fixed images