p2pool - Decentralized, Absolutely DoS-Proof, Pool Hopping-Proof Pool [archival]

[EskimoBob]

Is there a known problem with poclbm (latest git) not checking the actual target of a getwork?

Ok, I have confirmed that poclbm will happily submit shares that don't meat the target requirements.  This explains the shares/hour being identical to past experience even though share difficulty is 3-4x higher with p2pool.

My next question is does p2pool correctly reject these shares?  I can tell that p2pool never tells the miner they are rejected (the "result" of the share submission is always "1" (true)), but there may be some place else in the p2pool code that notices the hash does not meet the target requirements.  I'm not that good at reading python code, so I can't tell by looking.

I think cgminer 1.3.0 has a similar problem. Can someone please confirm? It was mentioned in #p2pool last night.
 

[1714944679]

1714944679

[1714944679]

1714944679

[1714944679]

1714944679

[1714944679]

1714944679

[1714944679]

1714944679

[thirdlight]

I don't think the pool hash rate is reporting correctly.

I've started a census at

http://forum.bitcoin.org/index.php?topic=30683

Add your current hash rate please! 

[Shevek]

[...)
Also p2pool eats more and more memory. I stopped it at 600MB now. It didn't  even react to SIGINT anymore and I had to SIGKILL it.

I agree: p2pool eats too much memory:

Code:

 PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM   TIME+   COMMAND
1528 shevek    20   0  526m 512m 3516 S   31 17.6 191:32.14 python

So 17,6% of mem is a bit too much.

I think it's a problem of the design. As forrestv said, the whole sharechain is kept in memory, instead of a file or a db.

Perhaps it's time to reconsider this policy and think about files. Moreover, with files in mind, it is possible reconsider also the limit of 24,000 shares of payout, too low for home miners of ~100 MH/s power.

[jgarzik]

I think it's a problem of the design. As forrestv said, the whole sharechain is kept in memory, instead of a file or a db.

Yes.  That is always a bit silly, because the OS will cache frequently accessed bits of files anyway.

[Operations]

Hello,

does anybody still use this pool?

I'm using it with some breaks since it went live on the mainnet and i see a big issue.

- p2pool is running for some time
- stop p2pool
- wait some hours or only minutes
- start p2pool
- now lots of shares (~10%) are missing
- e.g. i stopped the pool today 7/23/2011 1:02 p.m. UTC with shown 5xx mhash/s and ~38500 shares
- then i started the pool 7/23/2011 1:04 p.m. UTC (2 minutes) later and now 20 minutes later it shows

Code:

Pool rate: 545 mhash/s 35167 shares Contribution: 11.50% >62 mhash/s

- so almost 3500 shares are missing
- i'm not sure but i think we should be around 45000-50000 shares already


Can some of the few p2pool users confirm this?


Thanks

Best Regards

Operations

[thirdlight]

I'm seeing 55603 shares right now, pool rate 600 MH/s.

I'd really like to see this succeed, but get the feeling there's only a few of us here!

Maybe when the namecoin difficulty changes (4 hours), people will remember about bitcoins!

[Un zafado cualquiera]

Am i Actually lone?

pool rate: 0 mhash/s

and I am with 1.1Mhash/s

[forrestv]

- now lots of shares (~10%) are missing
- e.g. i stopped the pool today 7/23/2011 1:02 p.m. UTC with shown 5xx mhash/s and ~38500 shares
- then i started the pool 7/23/2011 1:04 p.m. UTC (2 minutes) later and now 20 minutes later it shows

- so almost 3500 shares are missing
- i'm not sure but i think we should be around 45000-50000 shares already

Can some of the few p2pool users confirm this?

This is completely normal - the oldest shares are forgotten after a time. There nominally should be at least 34560 shares - this is the number that is remembered.

[Operations]

Ok with last changes in the p2pool Source code we are able to see number of Stale Shares. After updating the Source Code i've started the p2pool and the Diablo Miner. 1 hour later i can see the following Result.

- Gentoo Linux 64 bit stable
- Catalyst 11.6 + AMD SDK 2.4
- HD 5670 900/300 MHz (Core/Ram)
- ~101 mhash/s
- latest Diablo Miner and p2pool version

Code:

Pool: 978 mhash/s in 35799 shares Recent: 5.38% >52 mhash/s Known: 52 shares (so 16 stales)

That are a lot of stales. If p2pool and Diablo Miner runs 10 or more hours then i'm still seeing hashrates of 70-80 mhash/s. So i must assume that 20-30% of my contributed shares are stale shares.

If i'm using the Diablo Miner with Eligius pool i'm still getting around 10% of stales. And if i'm using cgminer with Eligius i'm getting lesser stales but around 10-15% of corrupted shares.

I don't know why i'm getting such a high stale rate. Maybe my Internet connection is not fast enough or the 5670 is too slow.

Can someone confirm this issue?


Thanks.

Best Regards.

Operations

[forrestv]

That are a lot of stales. If p2pool and Diablo Miner runs 10 or more hours then i'm still seeing hashrates of 70-80 mhash/s. So i must assume that 20-30% of my contributed shares are stale shares.

DiabloMiner has a bug where it waits half a second after receiving a long polling response. There is a 20% chance that a p2pool share will be generated in that window, resulting in all hashing until the next getwork being useless.

I have created a fork of DiabloMiner on GitHub that fixes this: https://github.com/forrestv/DiabloMiner Make sure to get the fast_long_polling branch!

Code:

git clone git@github.com:forrestv/DiabloMiner.git
cd DiabloMiner
git checkout fast_long_polling

Built with JAR and Windows binary: http://u.forre.st/u/xsvenrwa/DiabloMiner_2e73cc.zip

DiabloMiner is the currently the best known miner to use; poclbm and phoenix-miner both are incompatible with p2pool's mining!

[twmz]

DiabloMiner is the currently the best known miner to use; poclbm and phoenix-miner both are incompatible with p2pool's mining!

Can you clarify what they are doing that is incompatible?  Is it just that they don't actually verify a share against the given target?  I "fixed" that by modifying my proxy to check a submitted share against the target and silently drop them when they are not valid so that p2pool never sees them.

I only care because DiabloMiner now uses a kernel that is not compatible with my GPU on OS X (basically, any phatk-like kernel fails and only a poclbm-like kernel works).

[Operations]

I'm using your branch of Diablo Miner (http://u.forre.st/u/xsvenrwa/DiabloMiner_2e73cc.zip) now and i will report back.

Where can i see how many shares are "completed" of the current block/round?


Thanks

Best Regards

Operations

[forrestv]

DiabloMiner is the currently the best known miner to use; poclbm and phoenix-miner both are incompatible with p2pool's mining!

Can you clarify what they are doing that is incompatible?  Is it just that they don't actually verify a share against the given target?  I "fixed" that by modifying my proxy to check a submitted share against the target and silently drop them when they are not valid so that p2pool never sees them.

No, it's because their long polling implementations ignore updates pushed if the previous_block field in the getwork's block header didn't change. Shares have an internal previous_shares field that needs to be updated; This leads to a lot of stales because the miner works on expired shares.

Where can i see how many shares are "completed" of the current block/round?

That concept doesn't really apply to p2pool since it isn't a proportional pool. It simply pays all the shares in the last day, and forgets about ones before that. I can estimate that the equivalent of about 140815 normal difficulty-1 shares have been produced by looking at the time the pool's been active and the average hashrate: 7*24*60*60*1e9/2**32 (one week at about 1GH/s).

General notice: Also, I'd urge everyone to update now as there have been a lot of interface updates and a couple memory improvements. A new py2exe Windows executable has been posted on the first post.

[Operations]

OK p2pool updated and i used your version of Diablo Miner.

Code:

Pool: 1432MH/s in 35875 shares Recent: 6.34% >90MH/s Shares: 67 (8 stale)

It's better than before. Now i have to check long time results.


By the way. WOW almost 1.8 GH/s Pool rate.  

[Operations]

After 9 hours of runtime p2pool shows:

Code:

693MH/s in 41521 shares Recent: 10.93% >75MH/s Shares: 645 (112 stale)

And Diablo Miner shows:Pool:

Code:

 mhash: 101,5/101,7 | accept: 645 | reject: 0 | hw error: 5

So 112 of 645 shares are stale shares this is a 17,4% stale rate.


Something is still wrong and i don't know the root cause.

[burp]

Has p2pool found any block yet?

[forrestv]

After 9 hours of runtime p2pool shows:

Code:

Pool: 693MH/s in 41521 shares Recent: 10.93% >75MH/s Shares: 645 (112 stale)

So 112 of 645 shares are stale shares this is a 17,4% stale rate.

Something is still wrong and i don't know the root cause.

While this is a bit high, some amount of stales will always be normal since work is invalidated within an average of 5 seconds. However, 'p2pool shares' are not exactly the same as normal shares: While you don't get credit for them, they can still cause the pool to solve a block. Because of this, as long as everyone using p2pool has approximately the same amount of stales, nobody is losing anything because of them.

I'm working on more techniques in p2pool and the miners to reduce the amount of stales. The lowest I've gotten it is about 8%.

Has p2pool found any block yet?

No. :/ With the current pool power, there was only about a 10% chance of this happening before now.

[burp]

Has p2pool found any block yet?

No. :/ With the current pool power, there was only about a 10% chance of this happening before now.

I guess more people would try p2pool if it worked with most miners (like cgminer or poclbm).

[jgarzik]

I guess more people would try p2pool if it worked with most miners (like cgminer or poclbm).

+1 agreed, it should be compatible, not force changes to all miners

[forrestv]

I guess more people would try p2pool if it worked with most miners (like cgminer or poclbm).

+1 agreed, it should be compatible, not force changes to all miners

I hadn't tested cgminer before, working on it now. It doesn't seem to have the problems other miners have with long polling, but has something else causing quite a few stales ... maybe it can be fixed with a command line option.

Normal poclbm and DiabloMiner have a delay in long polling requests that can't be worked around - the creator made some assumptions about how work distribution would work ... But they still work, they just have a high percentage of stales.

Phoenix did the same thing (assumptions that can't be worked around), but they much more significantly hurt the stale ratio.