Ah ha! (p2pool question I can't ask in the place where I want to ask it)

[Mogsington]

Only took me around 24 hours to figure out I can only post in this section!
(Brains r us).

Anyway! Hi all!

p2pool stuff that I've not managed to find answers for elsewhere:

First off, I've read and basically understand the whole p2pool system. With a couple of huge blank spaces...

When I mine for a standard pool, every share that gets accepted generates some micro BTC quantity. Nice and easy to keep track of, even if the amount varies with block luck. I'm good with all that. But ... in p2pool, it's a little different. I can sit and watch hundreds of accepted shares vanish in to the pool, but not get a single "share" according to p2pool.

I gather that's just how it works, but what I don't get is what on earth is happening to all those accepted non-share returns? Am I being sent deliberate duplicates just to keep my computer busy? .. Frankly I'd rather it didn't do that thanks. Just send me one share every 3 hours if that's the only one I'll get any credit for maybe?

Yes. You're right. I really don't understand that part of p2pool at all, and I've never seen it explained anywhere. I can accept that somehow when I do get lucky and send a good share in, for some reason it seems to get way more credit than it would do on a normal pool, but it makes using p2pool and trying to estimate returns / compare it to any other pool virtually impossible.

The other bit I don't get is ... It seems to be based on a rolling 24hr PPS type system. PPLNS or something? Anyway. That's fine in theory if a pool has enough hash power to turn a block over normally within 24hrs. What seems to happen at the moment though, is if you return a good share and get credit for it at the start of a long block, your credit vanishes before the end of the block gets reached two days later. That doesn't help either when you're trying to pin down how well p2pool works compared to other systems.

Finally! I've seen all the comments saying p2pool isn't suitable for low hash rate miners, but I've never seen anyone dare to state a cut off point for viability. Where do the returns begin to vanish? Is a 750M/hs set up viable? How about 1.2G/hs? What about 500M/hs? The only statement I've seen on low rate miners implied that CPU mining on it was pointless. O.o Rly?! Ya Rly! We knew that already!!!

I'd love to love p2pool, but the opacity for it on those points always makes me feel like leaving it running on p2pool is potentially a huge waste of time. The only conclusion I can draw is .. it's perhaps the ultimate dip in pool. You're barely ever going to get a credit for an accepted share anyway, so may as well just dip in for 15 mins once in a while, return a few on the off chance you get lucky, then dive back out to standard pools.

I wanted to post this to one of the main p2pool threads, in fact it's the only reason I signed up for yet another forum, but here we are!

[jdbtracker]

Yo noticed that too eh? I've been just mining in P2pool for a few days now(24 hours continuous), in the last few days no one has gotten a single share, the p2pool.info is saying 4 days now without a found block; I think I may have only of gotten some Bitcoins from a few donations since I started 2 weeks ago.

P2pool is working on a very different system then the larger pools, I think Slush's pool gives out a share for every work unit regardless if the pool finds a block or not. In the end on a bad week Slush is taking the brunt of the losses, hence the 2% infrastructure tax to make sure he keeps in business.

P2pool is acting more like one big solo miner, and with an average of only 300gh/s it takes a whole day for the whole system to crack a block. meanwhile slush's pool has 6000gh/s! The averages on wins and losses smooth out a lot better on slush's pool or on the gargantuant BTC pool, that one is huge!

It also doesn't help that the pieces sent to miners is quite large, it takes my rig 760mh/s 1 hour to hash them, it is very distrubing to me when I see the estimate amount at .065 and after a dead or orphaned block It pounds right down to .043, very painful, I haven't gotten a cent in 4 days. but thatws how p2pool works one giant congregation of solo miners, working together but facing the same benefits and challenges of solo mining with the rewards being distributed.

[Mogsington]

Uh huh.
So p2pool is very much like solo mining?

That's .... really not all that appealing for anyone who isn't in the several GH/s league.

Why on earth are you sticking it out with it if you know after a week the returns are virtually non existent?

Hmms. Well thanks for the reply. It didn't actually add much detail that I didn't know, but it did illustrate the "Hey I thought I earned that but it vanished" nature of it, and the "Hey I can mine here for days on end and get zilch" aspects.

I've deliberately not gone for the top mining pools because I understand the need for many mid sized pools rather than one huge dominant pool. There are plenty of good pools outside of the top 5 or 6 big players. Sadly I'm not going to count p2pool as one of them. I've seen people claim they get slightly better than ordinary pool returns out of it, so good luck to those who stick with it, but until a decentralised pool comes out that gives a lot less variance appears ... Nope. I'm not waiting the 6 months required to find out if that's true or not.

Note to p2pool devs : The main forum thread for it on here https://bitcointalk.org/index.php?topic=18313.0 really doesn't give anything like enough detail on the variance problems, or the apparently vanishing "earned" BTC side of it. The idea is great. The end product though is a bit opaque.

Seriously guys. Check out the mid sized pools though, the dominance of the main 3 players is getting worrying.

[jdbtracker]

It's the size of the pool that does it; to earn decent rewards the pools have to scale, if P2pools network doesn't grow in hash power our rewards will only get worse, well sort of... if you count it in USD... you reward will be the same as long as the price goes up.

I stick to it because I know eventually it will be fixed, we will have decent hashing power once the asic problem is resolved, besides I like the idea of a decentralized transparent pool, I can look at the code and know that there is no tomfoolery going on, it can be true for the pools too.

the pool is small and at the moment only the most adept can join p2pool, that's what is hurting it, but with further updates it'll get easier more accessible to regular people, for that reason I keep the 1% developer tip while I run. The more he's got in his pocket the more incentive he has to get more people on this pool.

[Mogsington]

And back to P2Pool he went ...

Ok here's what I've figured out.
The stats you get from http://127.0.0.1:9332/static/ when running p2pool are what you might call indicative at best. So if you're sat staring at those to see how it's going, you're not actually getting the real story.

The *only* way I can figure out to see how well p2pool works for you is to write down when you started mining on it, then watch your BTC wallet and add up the income from blocks as they appear. Divide your income by time and you can figure out a 24hr income rate to compare with other mining pools you may have used. (Don't forget pending payouts!) You also need to leave p2pool running for at the very least 3 days, more like a week to 10 days before you finally get to see what it all averages out at. Even then you're vulnerable to hitting a lucky or unlucky week.

The stats on p2pool are bizarre. I keep thinking I understand them, but then odd things happen that prove that I don't. I'm probably being dumb. I'll keep staring at them The "Current payout to default address" is actually pretty useful, so there is that.

I can't speak for everyone who's tried out p2pool and thought "nahhh", but I think for me, the main one was not having any clue how well it was doing even after 24hrs. Compared to any other pool out there it feels like gambling. On the other pools you can see per share pretty much what you've got in 24hrs. In p2pool it's cross your fingers, see how lucky you were in the last 24hrs, and don't expect it to do the same thing tomorrow. A good day is a very good day. A bad day is a very bad day.

Basically, I'm taking the leap of faith and seeing what returns I actually get out of the thing over an extended period, and lo and behold, so far it's paying out more than my previous pool. I know that will drop away as luck averages out, but it still looks like it will turn out marginally better than a standard pool.

The other thing to love about it is the mining transactions just appear in your BTC wallet as mining transactions. There's no minimum limit on withdrawals, no waiting for an auto-payout threshold to get reached, just what you got from each block as it appears. No MtRed (or insert any other pool name here) to fold on you unexpectedly either.

It sort of reminds me of the jump in the matrix. The numbers do work out, but you have to really believe!

[ElBurro]

P2pool takes a bit of work to get working at optimum efficiency. If you don't have time to dedicate to tweaking and monitoring it's better just to find a good pool. I'm in the process of setting up an Ubuntu VM to see if I can increase my return.

[Mogsington]

That's not what I've found at all.
As long as latency between the miner and p2pool is small (run them both on the same machine), then it just works as is.
I've never had a problem with DOA shares either. At the moment I'm at...

"Stale rate: ~0.0% (0-49%) Efficiency: ~120.8% (61-121%)"

Just running it with no tweaks.
I would suggest using IP numbers to point your miners at your local p2pool server instead of local lan names, but apart from that ... it's fine. And I'm on what most of the developed world would call a pretty slow DSL line. I get about 1.4MB/s max. Don't blame me, or my ISP, blame the monopoly on communications infrastructure we've had in the UK since dear Maggie Thatcher made a quick pile of cash from privatising it in the worst way possible.

[Mogsington]

I'll add to that ...

If you look at the graphs in p2pool, and scroll down to a miner, you can see the % given in total hashes to dead hashes. You do get the odd red blip on dead hashes, but you also get the mean value for your hash output. It seems to me a % of that is lost to the network (miner reports marginally higher hash rate than total of all hashes), but everyone else on the network is suffering that same loss as well, so again, the numbers even out when it comes to who makes the shares in to the block.

You can tell from my first post in this thread, I really was a "I don't trust p2pool" type. But once you take the plunge and begin to think about pooled mining from a different angle ... and see the payouts ... it's actually pretty darn good! In fact, I'm beginning to wonder why it hasn't attracted more power to the pool.

Probably the same reasons I dodged it to begin with. It seems very opaque and hard to understand. The only solution to that is figure out your returns for the pool you are currently on. Do the best job you can of it, include the processing up to the first payout, and any payout that came after you stopped when the block's cleared.

Now switch to p2pool and write down when you started. Keep a tally of income over time and wait a few days for it to level out. Don't forget to include the payments yet to clear, because you included them in the calc's for the original pool.

End result? I'm fast becoming a born again p2pool fanatic 

[jdbtracker]

The total payout per block is much better in p2pool, far better than any other pool out there, low or non-existent fees(don't stop tipping the developer). I know mathematically that I will have a great day when we hit a good week, or a bit disappointed like last week when the pool was having a bad streak.

But how do things change eh? last few blocks have been easy, that extra power on the network is really helping.
Now if we could only get at least 1 terahash/s performance, then our average bad luck streaks will start to even out much better, getting us to the mean average consistently. This week I am very happy with my returns, truly mind blowing.

There is a lot of problems with the network; I think that is caused by improperly setup machines, maybe someone should write up a basic primer on P2pool best practices for smooth p2p networking.

[Mogsington]

Bonus points.

While waiting for the long blocks on p2pool, you can make up lyrics for songs such as "Some day my block will come". 

[Gabi]

P2pool shares have a much higher difficulty than the shares of the other pools 

[lbr]

How long will it take me to generate a block?No-one can say exactly. There is a generation calculator that will tell you how long it might take.

What if I'm 1% towards calculating a block and...?There's no such thing as being 1% towards solving a block. You don't make progress towards solving it. After working on it for 24 hours, your chances of solving it are equal to what your chances were at the start or at any moment. Believing otherwise is what's known as the Gambler's fallacy [1].

It's like trying to flip 53 coins at once and have them all come up heads. Each time you try, your chances of success are the same.

https://en.bitcoin.it/wiki/Blocks#How_long_will_it_take_me_to_generate_a_block.3F

So.. mining at slush's pool also is a gamble.
Just it is "more stable" gamlbe ; )
And since p2pool overall hashrate is not so big, the stats could look bizzare.

Right?

What I don't get is
Lets assume there are only two miners in the network miner#1 has 10000GHs and miner#2 has 1Hs and some calculator tells that miner#2 supposed to mine a block in a 24hours.
Now, since each block is created every 10 minutes(in our case miner#1 will generate them) and after each new block the work should be restarted, is it fair to say that miner#2 never generates a block?

[jdbtracker]

the system will reject it for being slow, The dynamics of Bitcoin limit it in many ways, it is a very complex machine hell bent on one thing... increasing economic efficiency.

to be profitable you have to be close to the mean average of the entire network, if your slower your not getting the transaction done in time, holding up economic transactions is a threat to the network so you will get a DOA.

on p2pool like in solo mining the network will punish you, by asking you, to do it again, this time faster! dammit!
In the larger pools it is very friendly to small miners, they get a cut too, plus any DOA on them is probably cause the server is getting overloaded... p2pool has a bit of a disadvantage there, it seems.

[Mogsington]

No wait up.
It's appears for me at least trivially easy to be at the network average.

There are two points of variance in p2pool. How many valid shares probability allows your miner to submit in a time period, and how long the block is in a time period.

It doesn't seem to matter too much how far down the MH/s scale you are. If you leave it running 24/7, the probabilities for both work out.

In basic terms. When the mean luck of all the blocks you have processed on p2pool is at 50%, then hopefully! (Because that might take ages!), your mean luck at returning valid shares per time period will be near 50% as well. It's only at that point that you can actually say how much you get on average per day with a degree of certainty.

You can get lucky on shares in a long block even on a slow machine. You can equally miss shares in small blocks. I know. It's a head twister. You need to put faith in the raw probability curves at work and not what your brain tells you might or might not work.

If you have good efficiency and low DOA rates, then you don't seem to get screwed. The stats might be all over the shop, but when you sum up the income over 10 days, you should have a pretty good idea of what you actually get back. Bear in mind, you may pick a period with 4 two day blocks, or 14 short blocks. That's the nature of probability.

The p2pool stats you get from the local webserver are a best attempt at expressing those probabilities. Therefore they are always different to what you actually get for better or for worse. I'm sticking with it now. The end results are so far looking good on the curve to 50% luck.

[Mogsington]

Yo noticed that too eh? I've been just mining in P2pool for a few days now(24 hours continuous), in the last few days no one has gotten a single share

Woah there. That simply cannot be true. In fact even taken as you yourself not getting a share, over 24hrs, that's not impossible, but incredibly unlikely ... with your stated MH/s? It's out there in the thousands to one sort of field of probability. A rough head calc guess tells me you might have to mine for 8 years+ to see that kind of bad luck over a few days. Are you sure you've got the network side working?

[Mogsington]

What I don't get is
Lets assume there are only two miners in the network miner#1 has 10000GHs and miner#2 has 1Hs and some calculator tells that miner#2 supposed to mine a block in a 24hours.
Now, since each block is created every 10 minutes(in our case miner#1 will generate them) and after each new block the work should be restarted, is it fair to say that miner#2 never generates a block?

Er. Well first. the estimate for a 1Hs miner would be 0.<a LOT of zeros>something. In fact so close to zero, you'd stop bothering.
Second. 10 minutes is across the entire bitcoin block generating network, not any one pool.
Yes it would be fair to say a 1Hs miner isn't worth asking to generate a block even at stupidly low difficulty rates.
It *could* happen within a lifetime, but the odds are astronomical.

[Mogsington]

So.. mining at slush's pool also is a gamble.
Just it is "more stable" gamlbe ; )

Well yes. Built in with the deductions, 24hr PPLNS system and all that. You could look at it as a weighted gamble that pays a steady income at some possible cost to that income.

And since p2pool overall hashrate is not so big, the stats could look bizzare.

Right?

Sort of. If you point a low hash rate (within reason!) machine at it, then there is a fair bit of variance on how often you'll get a valid share as well as how long each individual block takes.

Yes it's a lot of variance. That doesn't make it any more or less of a gamble than slush's pool. Well. Except Slushi might go down like any number of mining pools before it. Slushi is a single point access that might get DoS'd or just have a bad day. Or the 2% fee. You also have no idea what network inefficiencies (if any) happen at the server end of a fast pool like Slushi's. Could be better, same or worse. Who knows?

Apart from that. p2pool is totally a gamble just the same as Slushi.

[jdbtracker]

I think Slush was getting a DOS attack for the past few days, A few people I know were trying to connect but could not, I attempted to try it and I was denied access, due to an IO error, first time that has ever happened.

I already had my GUIminer setup for Slush, so I know it worked before.
I'd rather take my chances  on p2pool, no ones ever going to effectively attack our network... though Id  wonder how well Tor can function covering p2pool.

has anyone tried doing a mass Tor test on the network to make sure it can operate during hostile conditions?

I hate these hackers with vested interest  in trying to bring down a pool to bring the network within attacking distance... despicable, those networks need to adopt a decentralized system to mantain Bitcoins security,

everyone should do a  Tor test as soon as possible, the future of the network may depend on it.

[Mogsington]

Something else I've noticed.

I've seen a couple of "big" rig owners do a write up on the shiny new multi GH/s machine they just got. Both of them seem to just point it at p2pool for a few hours expecting the same steady rate of return you get from a traditional pool. When they don't see it within a few hours and they don't understand the numbers they see, they seem to wander off to the standard server pools constant trickle rate because it's easier to quantify.

Heigh ho. Maybe p2pool works better with and for lots of smaller miners who appreciate the returns more anyway. It seems a bit odd though when most miners claim to understand bitcoin technically that they can't get their head around a distributed mining pool model. Yes p2pool is another layer of "huh?", it is different, but that doesn't seem to stop it working for a lot of people.

[jdbtracker]

No wait up.
It's appears for me at least trivially easy to be at the network average.

There are two points of variance in p2pool. How many valid shares probability allows your miner to submit in a time period, and how long the block is in a time period.

It doesn't seem to matter too much how far down the MH/s scale you are. If you leave it running 24/7, the probabilities for both work out.

In basic terms. When the mean luck of all the blocks you have processed on p2pool is at 50%, then hopefully! (Because that might take ages!), your mean luck at returning valid shares per time period will be near 50% as well. It's only at that point that you can actually say how much you get on average per day with a degree of certainty.

You can get lucky on shares in a long block even on a slow machine. You can equally miss shares in small blocks. I know. It's a head twister. You need to put faith in the raw probability curves at work and not what your brain tells you might or might not work.

If you have good efficiency and low DOA rates, then you don't seem to get screwed. The stats might be all over the shop, but when you sum up the income over 10 days, you should have a pretty good idea of what you actually get back. Bear in mind, you may pick a period with 4 two day blocks, or 14 short blocks. That's the nature of probability.

The p2pool stats you get from the local webserver are a best attempt at expressing those probabilities. Therefore they are always different to what you actually get for better or for worse. I'm sticking with it now. The end results are so far looking good on the curve to 50% luck.

Very true, the encryption is based on random probability. so a 1mh/s miner could do it, but I feel I need to test this, I"m going to activate my cpu and begin testing this, I'll run it for 1 week 24/7 and come back with the results, I'll make sure to optimise my server for maximum efficiency... though hopefully no one starts downloading massive files while i'm doing this.