[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool

[spiccioli]

Yes, I understand the percentage of actually seeing some if not any shares are; the thing that confuses me is how am I able to show accepted on the miner when my pool doesn't show any shares, unless it's mining so low that the pool already is on another block. :\

https://en.bitcoin.it/wiki/P2Pool#Frequently_Asked_Questions

Q: Why does my miner say it has found a lot of shares but p2pool say I have only found a few?!

A: The real P2Pool difficulty is hundreds of times higher than on normal pools, but p2pool essentially lies to your miner and tells it to work on relatively easy shares so that it submits shares every few seconds instead of every few hours. P2Pool then ignores any submitted shares that don't match the real share difficulty. By doing this, P2Pool can more accurately report your local hash rate and you can see if you are having problems with too many stale shares quickly

twmz,

question: could p2pool use difficulty 1 shares to calc miner payout instead of using high difficulty ones? This should lower miner's variance and still be able to use high difficulty shares to solve blocks.

spiccioli

No.

First high difficulty shares aren't used to solve a block.  They aren't used for anything other than payouts.  They need to be high to keep avg time between shares ~ 10 seconds.  For example if p2pool uses 1 difficulty shares instead of 0.1 shares per second it would have 60 shares per second.  The sharechain would simply collapse under the weight of dozens of forks a second.

Shares are completely worthless (in terms of real value).  This applies to any pool.  You don't work towards a block.  You either solve it or you don't.  If you don't the hash produced is worthless.  Pools simply use shares to record how much work was attempted and thus determine a fair split.

Thanks DAT,

I was missing (I didn't completely understand) this issue.

I was thinking that using a higher "sampling frequency" could give a better estimate of miner contribution to the pool.

I was wrong

spiccioli.

[DeathAndTaxes]

I was thinking that using a higher "sampling frequency" could give a better estimate of miner contribution to the pool.

Just to be precise.  A higher sampling frequency (lower diff shares) does provide a better estimate (less variance in short run).  This is why all traditional pools use a diff 1 shares.  

It just happens to be that given the limits of internet connectivity, latency, and node response times p2pool has a semi-unique restriction in that it needs to keep LP interval at ~ 10 seconds.  That puts an lower limit on difficulty of shares.  Hypothetically if all the nodes of p2pool were on internet backbone and had sub 100ms latency with every other node you could use a smaller LP window and thus a lower difficulty.

As indicated above you can support p2pool w/ 1 difficulty shares by using p2pmining.com.  It works as a "front end".  It records diff 1 shares submitted by miners and submits p2pool dif shares to p2pool.  When a block is found it splits any revenue by the diff 1 shares.  It is semi-centralized (in rewards not work generation).

[Red Emerald]

I was thinking that using a higher "sampling frequency" could give a better estimate of miner contribution to the pool.

Just to be precise.  A higher sampling frequency (lower diff shares) does provide a better estimate (less variance in short run).  This is why all traditional pools use a diff 1 shares. 

It just happens to be that given the limits of internet connectivity, latency, and node response times p2pool has a semi-unique restriction in that it needs to keep LP interval at ~ 10 seconds.  That puts an lower limit on difficulty of shares.  Hypothetically if all the nodes of p2pool were on internet backbone and had sub 100ms latency with every other node you could use a smaller LP window and thus a lower difficulty.

As indicated above you can support p2pool w/ 1 difficulty shares by using p2pmining.com.  It works as a "front end".  It records diff 1 shares submitted by miners and submits p2pool dif shares to p2pool.  When a block is found it splits any revenue by the diff 1 shares.  It is semi-centralized (in rewards not work generation).

Isn't the work generation centralized too? p2pmining runs bitcoind and namecoind and then gives you the work.

[DeathAndTaxes]

Isn't the work generation centralized too? p2pmining runs bitcoind and namecoind and then gives you the work.

Yes.  I guess I should say work can be verified.  It is possible for p2pmining to provide software which would allow user to create work locally.  No idea if the operator plans on implementing something like that.

[ragnard]

Hypothetically if all the nodes of p2pool were on internet backbone and had sub 100ms latency with every other node you could use a smaller LP window and thus a lower difficulty.

Makes me think... Could P2Pool protocol be setup to allow "super nodes" that have this good connectivity?  Users would then connect to those on a random or geographical basis.  That would give the benefit of allowing lower difficulty shares but still have a decentralized "core".  Kind of like TOR does with multiple entry/exit points on the network.  Anyone could setup a P2Pool super node, and the clients could decide if they want to keep using it based on latency, hops, etc...

As the popularity of P2Pool grows, the difficulty is only going to get higher and turn off small miners.  If this is indeed the future of BTC mining, and I hope it is, something needs to be done to address this.

[DeathAndTaxes]

Hypothetically if all the nodes of p2pool were on internet backbone and had sub 100ms latency with every other node you could use a smaller LP window and thus a lower difficulty.

Makes me think... Could P2Pool protocol be setup to allow "super nodes" that have this good connectivity?  Users would then connect to those on a random or geographical basis.  That would give the benefit of allowing lower difficulty shares but still have a decentralized "core".  Kind of like TOR does with multiple entry/exit points on the network.  Anyone could setup a P2Pool super node, and the clients could decide if they want to keep using it based on latency, hops, etc...

As the popularity of P2Pool grows, the difficulty is only going to get higher and turn off small miners.  If this is indeed the future of BTC mining, and I hope it is, something needs to be done to address this.

Yes it has been discussed.  There are a couple of methods of providing multiple layers to p2pool with the highest layer (highest difficulty, possibly 6000+ per TH/s) forming a backbone for major farms, sub p2pools and conventional pools.

[kano]

...
Yes it has been discussed.  There are a couple of methods of providing multiple layers to p2pool with the highest layer (highest difficulty, possibly 6000+ per TH/s) forming a backbone for major farms, sub p2pools and conventional pools.

However, as soon as you have sub-p2pools - they are no different to normal pools.
So why not use a normal pool with 1 difficulty instead and solve the whole issue up front?

[DeathAndTaxes]

However, as soon as you have sub-p2pools - they are no different to normal pools.

I think you misunderstand the concept of a sub-p2pool.  Everything would be completely decentralized just as p2pool is now.

[kano]

However, as soon as you have sub-p2pools - they are no different to normal pools.

I think you misunderstand the concept of a sub-p2pool.  Everything would be completely decentralized just as p2pool is now.

Hmm - well I guess it could be possible?

But certainly people opening up their own p2pool as a pool is no different to a normal pool.

However, I'm not sure how the sub-p2pool would pay out the earnings.
Either the master p2pool share chain would have to know every sub-p2pool address and pay them their micro-payments in the master p2pool block or there would have to be some centralised non p2p control to handle that ... or

[DeathAndTaxes]

However, I'm not sure how the sub-p2pool would pay out the earnings.
Either the master p2pool share chain would have to know every sub-p2pool address and pay them their micro-payments in the master p2pool block

Thats it.

Simple version.

1) Increase LP interval of p2pool to 60 seconds (and increase difficulty by factor of 6).
2) Expand protocol from 1 share = 1 address to 1 share = multiple addresses and weighted split
3) Build sub p2pool protocol which creates its own share chain (unknown and invisible to main p2pool).

When p2pool find a block it payouts as normal with the exception of the fact that 1 share may be paid to more than one address.

When subpool find a low diff share it adds it to its internal share chain.
When subpool finds a p2oool diff share it submits it to p2pool (which updates p2pool split)
When subpool finds a block it submits it to Bitcoin network.

[NothinG]

Yes, I understand the percentage of actually seeing some if not any shares are; the thing that confuses me is how am I able to show accepted on the miner when my pool doesn't show any shares, unless it's mining so low that the pool already is on another block. :\

https://en.bitcoin.it/wiki/P2Pool#Frequently_Asked_Questions

Q: Why does my miner say it has found a lot of shares but p2pool say I have only found a few?!

A: The real P2Pool difficulty is hundreds of times higher than on normal pools, but p2pool essentially lies to your miner and tells it to work on relatively easy shares so that it submits shares every few seconds instead of every few hours. P2Pool then ignores any submitted shares that don't match the real share difficulty. By doing this, P2Pool can more accurately report your local hash rate and you can see if you are having problems with too many stale shares quickly

That's the problem. I was able to see some shares even if they wen't getting submitted to the pool itself, but my pool wont even show those shares.

[gyverlb]

However, I'm not sure how the sub-p2pool would pay out the earnings.
Either the master p2pool share chain would have to know every sub-p2pool address and pay them their micro-payments in the master p2pool block

Thats it.

Simple version.

1) Increase LP interval of p2pool to 60 seconds (and increase difficulty by factor of 6).
2) Expand protocol from 1 share = 1 address to 1 share = multiple addresses and weighted split
3) Build sub p2pool protocol which creates its own share chain (unknown and invisible to main p2pool).

When p2pool find a block it payouts as normal with the exception of the fact that 1 share may be paid to more than one address.

When subpool find a low diff share it adds it to its internal share chain.
When subpool finds a p2oool diff share it submits it to p2pool (which updates p2pool split)
When subpool finds a block it submits it to Bitcoin network.

Just thinking out loud...

I'm not sure how the subpool would be notified of share difficulty changes and submit shares to the parent p2pool : each subpool node should be notified of the share difficulty changes so could probably maintain a connection to the parent p2pool to get it (or can it trust other subpool nodes to forward the information and only have a subset of the subpool in charge of broadcasting it?). If having "proxies" is a security problem and mandates a direct parent p2pool connection for each subpool node, is the p2pool protocol scalable enough to support this diff broadcast to thousands of nodes or even more (assuming we want every miner to be able to join p2pool) instead of the current 200+ nodes?
I can see how the share submission could be efficient : it's supposed to be low traffic.

Another problem would be to make sure one subpool node would have to reward other nodes having successfully found low-diff shares while having a valid p2pool share. I'm not familiar enough with Bitcoin to be sure it is doable.

If p2pool can scale to an arbitrary large number of nodes, and still get low latencies for broadcasting diff changes, the first class of problems don't really exist : just make all subpool nodes join both p2pool and the subpool and submit information to both according to the difficulty having been resolved. I suppose it should work, I'm only worried about latencies on large scale p2p nets (raising the target LP interval would help).

For the second problem, sorry no clue.

[Panda Mouse]

On another note, I'm trying to get my 7970 hashing in the pool, but I can't seem to get it configured correctly with diablominer.
It works fine with cgminer, but I get better hashrates with diablo.
Anyone got this combo (7970+diablominer+P2Pool) working?
I'd love a few pointers in the right direction.

Try use cgminer and diablominer as a kernel. This is the best solution.

Try: cgminer.exe -u user -p pass -o http://127.0.0.1:9332 --auto-fan --gpu-engine 450-1195 --gpu-memdiff -150 --gpu-powertune 0 --temp-target 75 -q -g 1 -I 8 -k diablo -v 1 -w 256

Panda Mouse

[spiccioli]

No you shouldn't have to port anything. PyPy should just work with normal Python code. It's simply a different implementation of the standard Python interpreter. No difference in the language it interprets, just how it does it.

For one thing it has a built-in Just-in-time compiler powered by LLVM, which means that instead of the writers of a Python implementation needing to worry about how to create an efficient Just-in-time compiler, the work of people from the LLVM project, who are already experts in constructing such a compiler, can be leveraged.

This made me curious, so I am running p2pool with pypy as we speak.  It seems to be working fine, but I'll watch it for a day or two and see if there are any abnormalities.

twmz,

I've used pypy for a couple of days, it works ok, I don't see a big difference in CPU usage compared to python, but it has a problem: it is not compatible with pygame so you cannot show/generate graphs.

spiccioli

[bitpop]

Where is a list of url resources I can look at such as http://bitpoppool.geekgalaxy.com:9332/fee

[twmz]

I've used pypy for a couple of days, it works ok, I don't see a big difference in CPU usage compared to python, but it has a problem: it is not compatible with pygame so you cannot show/generate graphs.

Bummer about both.  I don't use the graphs so hadn't noticed that pygame wasn't compatible.  But if it doesn't offer any significant CPU efficiency gain, it's probably irrelevant.

[forrestv]

Where is a list of url resources I can look at such as http://bitpoppool.geekgalaxy.com:9332/fee

https://en.bitcoin.it/wiki/P2Pool#Web_interface

[bitpop]

Thanks!

Where is a list of url resources I can look at such as http://bitpoppool.geekgalaxy.com:9332/fee

https://en.bitcoin.it/wiki/P2Pool#Web_interface

[bitpop]

Can anyone attack me by making my p2pool public? Or any private info leaking through the web interface?
http://bitpoppool.geekgalaxy.com:9332/fee

[bitpop]

Is this actually sending?
http://bitpoppool.geekgalaxy.com:9332/patron_sendmany/.001