[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool

[nreal]

What is the doa% with s2? on pool level its 13% now, so anything below 13 is just fine.

I believe it's much higher than that.

M

25% ?
Local dead on arrival: ~      =?

[sconklin321]

What is the doa% with s2? on pool level its 13% now, so anything below 13 is just fine.

I believe it's much higher than that.

M

25% ?
Local dead on arrival: ~      =?

I think the problem isn't just DOA.  From what I've read, something appears to be slowing them down all around since, I believe stated earlier in the thread, even the Ants are showing the lower hashrate, not just p2pool.

[mdude77]

What is the doa% with s2? on pool level its 13% now, so anything below 13 is just fine.

I believe it's much higher than that.

M

25% ?
Local dead on arrival: ~      =?

When I run it through my proxy, the Ants report in excess of 50% rejects.  I'm not sure that number is right, but I haven't double checked it to confirm.

I'm suspecting that p2pool isn't properly returning matching reject messages, so the Ants don't really know the amount of rejects.  I'm not 100% of this, as the stratum specs leave a lot to interpretation/figuring out on your own.

What I'm referring to is this.

There's a unique ID # that's passed with each message from the miner to the pool.  That number is incremented for each message.  Each share is a message.  So say the Ant sends message #122 and it's a share submission.  If p2pool rejects it, the ID it uses on the reject is its own internal counter, so it could be #155.  I'm not sure cgminer knows how to match that up.

My proxy takes care of matching those up, so the message the Ant gets back from the proxy (Relayed from the pool) is the same # of the share from the Ant, in the example above, #122.

M

EDIT: I was looking at the wrong thing.. the return message from p2pool matches the originated message.

[MineForeman.com]

Yet another reason i like p2pool
I wouldn't imagine it possible to hide the fee is it?
In any case if the fee is hidden just don't use the node

Obviously you would not just hide the fee's, you would make it appear to be 0 while in reality you could set it to 1% or 5% or 100%  (the user would work it out fairly quickly if it was 100% though).

Neil

[raskul]

Yet another reason i like p2pool
I wouldn't imagine it possible to hide the fee is it?
In any case if the fee is hidden just don't use the node

Obviously you would not just hide the fee's, you would make it appear to be 0 while in reality you could set it to 1% or 5% or 100%  (the user would work it out fairly quickly if it was 100% though).

Neil

yeah, but ye wouldnae. really?

[-ck]

There's a unique ID # that's passed with each message from the miner to the pool.  That number is incremented for each message.  Each share is a message.  So say the Ant sends message #122 and it's a share submission.  If p2pool rejects it, the ID it uses on the reject is its own internal counter, so it could be #155.  I'm not sure cgminer knows how to match that up.

cgminer sends a unique ID message with every share submitted It then locally adds a record of the submitted share to a database. When it gets responses from whatever the upstream server is (be it a pool or a proxy) it matches the return response based on the ID and looks it up in its database to determine what the "server" thought of its share submission. cgminer does not assume messages return synchronously and uses the ID to look up the response to its submissions. If the upstream server corrupts the share submission ID by changing its return submission ID, then that's a fault and it should not do that. As far as I'm aware, no functional pool/proxy/server changes the share submission ID to give a different ID in response.

[nreal]

What is the doa% with s2? on pool level its 13% now, so anything below 13 is just fine.

I believe it's much higher than that.

M

25% ?
Local dead on arrival: ~      =?

When I run it through my proxy, the Ants report in excess of 50% rejects.  I'm not sure that number is right, but I haven't double checked it to confirm.

I'm suspecting that p2pool isn't properly returning matching reject messages, so the Ants don't really know the amount of rejects.  I'm not 100% of this, as the stratum specs leave a lot to interpretation/figuring out on your own.

What I'm referring to is this.

There's a unique ID # that's passed with each message from the miner to the pool.  That number is incremented for each message.  Each share is a message.  So say the Ant sends message #122 and it's a share submission.  If p2pool rejects it, the ID it uses on the reject is its own internal counter, so it could be #155.  I'm not sure cgminer knows how to match that up.

My proxy takes care of matching those up, so the message the Ant gets back from the proxy (Relayed from the pool) is the same # of the share from the Ant, in the example above, #122.

M

With walletadress/10000000 i can expect one share that pays in 20 h - with 4 ants1 pencil modded to 150gh - anything else is just waste of space. its the only 1 share in 20 h im interested here. if i get 2 of those in a day i call it a lucky day. The pool migth just be usable for s2s with similiar settings - some days you score bigtime some not so well. I think you are using those s2 with own walletaddress for each.

This is serious mining to go after bigger fishes. Im not worried about the variation - this is more fun than chase those smaller shares..

[mdude77]

There's a unique ID # that's passed with each message from the miner to the pool.  That number is incremented for each message.  Each share is a message.  So say the Ant sends message #122 and it's a share submission.  If p2pool rejects it, the ID it uses on the reject is its own internal counter, so it could be #155.  I'm not sure cgminer knows how to match that up.

cgminer sends a unique ID message with every share submitted It then locally adds a record of the submitted share to a database. When it gets responses from whatever the upstream server is (be it a pool or a proxy) it matches the return response based on the ID and looks it up in its database to determine what the "server" thought of its share submission. cgminer does not assume messages return synchronously and uses the ID to look up the response to its submissions. If the upstream server corrupts the share submission ID by changing its return submission ID, then that's a fault and it should not do that. As far as I'm aware, no functional pool/proxy/server changes the share submission ID to give a different ID in response.

That's how I interpreted that things should be working as well.

I just double checked my code, and it appears I misspoke.  I was looking at the return message after I matched it up with the miner, so of course they would differ.

Nevermind!

M

[ceslick]

At the moment my conclusion is a proxy isn't going to help getting S2s working with p2pool.

I believe the problem is the constant "drop EVERYTHING you are doing and restart" messages that p2pool sends every 30 seconds.  Then it rejects all the old work coming p2pool leading to large DOA.

My recommendations for p2pool to become more S2 friendly:

1 - Sending new work every 30 seconds is fine.  It's apparently not necessary to force the miner to restart "clean jobs" = true.  It also needs to gracefully accept work from prior job IDs for a period of time like other pools do.

2 - Start at a higher pseudo difficulty than 1.

3 - Allow a higher pseudo difficulty than 1000.

4 - Send the actual errors back for rejected work, instead of just "false".

If I understood python I'd take a stab at 2 through 4.  1 might be a big deal.

Here are 2 patches for p2pool that address point 2 and 3 from your list.

First a simple patch that allow higher than 1000 pseudo-difficulty.
https://github.com/jaketri/p2pool/commit/05b630f2c8f93b78093043b28c0c543fafa0a856

And another patch that add "--min-difficulty" parameter to p2pool. For my setup I use 256 as start pseudo difficulty.

https://github.com/jaketri/p2pool/commit/5f02f893490f2b9bfa48926184c4b1329c4d1554

How do I apply those patches? GIT noob here...

[JakeTri]

At the moment my conclusion is a proxy isn't going to help getting S2s working with p2pool.

I believe the problem is the constant "drop EVERYTHING you are doing and restart" messages that p2pool sends every 30 seconds.  Then it rejects all the old work coming p2pool leading to large DOA.

My recommendations for p2pool to become more S2 friendly:

1 - Sending new work every 30 seconds is fine.  It's apparently not necessary to force the miner to restart "clean jobs" = true.  It also needs to gracefully accept work from prior job IDs for a period of time like other pools do.

2 - Start at a higher pseudo difficulty than 1.

3 - Allow a higher pseudo difficulty than 1000.

4 - Send the actual errors back for rejected work, instead of just "false".

If I understood python I'd take a stab at 2 through 4.  1 might be a big deal.

Here are 2 patches for p2pool that address point 2 and 3 from your list.

First a simple patch that allow higher than 1000 pseudo-difficulty.
https://github.com/jaketri/p2pool/commit/05b630f2c8f93b78093043b28c0c543fafa0a856

And another patch that add "--min-difficulty" parameter to p2pool. For my setup I use 256 as start pseudo difficulty.

https://github.com/jaketri/p2pool/commit/5f02f893490f2b9bfa48926184c4b1329c4d1554

How do I apply those patches? GIT noob here...

You can cherry-pick (see "git help cherry-pick") the changes to your local git branch ...

... or you can manually apply the one line patch that allow higher than 1000 pseudo-difficulty and use "walletaddress+difficulty" as username to ask p2pool to disable dynamic pseud-difficulty and only use the requested difficulty.
For example to only receive jobs with pseudo-difficulty "2048" for your wallet-address "1Bitcoin552oR6LtAp55PMsfbEaFLjuTjg" you need to use as username "1Bitcoin552oR6LtAp55PMsfbEaFLjuTjg+2048"

You can use the stock p2pool without any patch if you need pseudo-difficulty of 1000 or less.

[Bulk-Neteller Ltd]

i love the GIT nobe. wonderful posts here.
educating.

[bryonp]

Hey All,
Can you look at these stats and tell me if I am close to the average or I am under performing??? Thanks
16 - S1's and 1 S2

Pool rate: 543TH/s (14% DOA+orphan) Share difficulty: 2060000

Node uptime: 5.5 days Peers: 6 out, 0 in

Local rate: 4.56TH/s (2.1% DOA) Expected time to share: 32.3 minutes

Shares: 254 total (37 orphaned, 8 dead) Efficiency: 95.31%

Payout if a block were found NOW: 0.19603371 BTC to XXXXXXXXXXXXXXXXXXXXXXXXX. Expected after mining for 24 hours: 0.210 BTC per block.

Current block value: 25.05644681 BTC Expected time to block: 1.5 days


(my Payout if a block were found NOW   2 days ago was hittinf .2400000000000 now it has moved down to .196   ??)

Thanks for any info....

[windpath]

Hi Bryon,

Local rate: 4.56TH/s (2.1% DOA)

16 x 200 GH/s = 3.2 TH/s expected
With the S2 = 4.2 TH/s expected

Your hashrate and DOA look great.

The difference in payout is due to the variance/luck of the number of shares you currently have in the share chain, the share chain is about 3 days long so it will take about 3 days for your payment to peak and then level out....

[windpath]

Mining Decentralisation: The Low Hanging Fruit

Mike Hearn posted today on the Bitcoin Foundation website about 2 methods of improving mining decentralization, p2pool was one of them...

He made 4 suggestions to improve p2pool

1. Problem: installation is too difficult
2. Problem: the p2pool website is hard to find
3. Problem: P2Pool feels clunky compared to centralised pools
4. Problem: P2Pool has an increasingly high share difficulty

Mike's problems 1-3 are relatively easy to fix, #4 is one we have all been discussing lately...

His proposed solution:

P2Pool could split itself into multiple independent p2pools, amoeba style, whenever it gets too large for small miners to benefit.

Without mass adoption of p2pool (and possibly even with it) this solution has 1 glaring flaw: Every time p2pool variance was significantly reduced by adding more miners/hashpower the "split" would increase the variance right back to high levels. Even at our current 400-500 TH/s we are still over a day to find a block.

What level of "expected time to block" would be acceptable?

Problem 4 needs a better solution. I already suggested what I thought was a decent starting point, but will take some serious work to get done.

Anyone else have any ideas on how to address high share difficulty?

Link to article:
https://bitcoinfoundation.org/2014/07/03/mining-decentralisation-the-low-hanging-fruit/

[PatMan]

He's not exactly stated anything new, it's been like this for over a year......

[MineForeman.com]

1. Problem: installation is too difficult

P2Pool comes complete with a partial solution for this one, you could have public servers so people don't have to go through all that it takes to setup and maintain your own.

The issue is paying for them.  I have been hashing through different ideas in my head but as of yet I have not come up with anything that floats.

Neil

[norgan]

1. Problem: installation is too difficult

P2Pool comes complete with a partial solution for this one, you could have public servers so people don't have to go through all that it takes to setup and maintain your own.

The issue is paying for them.  I have been hashing through different ideas in my head but as of yet I have not come up with anything that floats.

Neil

sign up to BizSpark or dreamspark and get access to free Azure credit. enough to run a dual core 3.5gb sever for the whole moth for free.

http://www.microsoft.com/bizspark/default.aspx
https://www.dreamspark.com/

[sconklin321]

4. Problem: P2Pool has an increasingly high share difficulty

I had a possible solution for this, though I don't know easy it would be to implement, or if it would even be worth it.  My inspiration for this comes from the bitcoin itself.  Our sharechain is designed to work just like the bitcoin blockchain only with a shorter retarget window.  One thing the bitcoin blockchain does that the sharechain doesn't (at least to my knowledge, other than when we find blocks) is transmit the bitcoin transaction.  What if we told the nodes to treat the pseudo shares of either 10% of current difficulty, or of a fixed difficulty, like the bitcoin network does when you send bitcoins to someone else?  The node accepting the share, would broadcast the share to the network, much like the bitcoin network does when you send bitcoin.  Then when a share is found, the node that finds it, bundles the "mini-shares" just like the bitcoin network does bitcoin being transmitted.  We could even include a bonus of say 0.5% of the share value submitted with that share to the finder like transaction fees.  

So at current difficulty, the node would broadcast shares of 200k or higher to the network to be included in the next share (block).  When the share is found they are broadcast to the network and thus included in the sharechain, while maintaining a 30 second share time.  This would allow lower hashrate miners to come in and contribute to the network.

[sconklin321]

1. Problem: installation is too difficult

P2Pool comes complete with a partial solution for this one, you could have public servers so people don't have to go through all that it takes to setup and maintain your own.

The issue is paying for them.  I have been hashing through different ideas in my head but as of yet I have not come up with anything that floats.

Neil

sign up to BizSpark or dreamspark and get access to free Azure credit. enough to run a dual core 3.5gb sever for the whole moth for free.

http://www.microsoft.com/bizspark/default.aspx
https://www.dreamspark.com/

That doesn't solve any issues, the blockchain download alone to run p2pool is greater than 3.5 gb

Edit: I'm sorry, I'm tired and didn't realize you were talking about RAM.  You could also use a service like linode.com.  $20/mth gets you 2gb or RAM and a dual core for a month, where Azure becomes $111 to $134 /mth after your free month.  You can also get 4gb of RAM and 4 cores from linode.com for $40/mth.  There is also digitalocean.com.

[KyrosKrane]

1. Problem: installation is too difficult

Murdof and I have made some tools to install p2pool more easily on Ubuntu; the thread is here:

https://bitcointalk.org/index.php?topic=651819.0

It would be trivial for someone to create a Windows installation package for p2pool; however, that would require trusting the package maker and having the user run an effectively arbitrary executable on their system.