[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool

[kano]

...
I guess also there needs to be clearly stated that anti-hop functionality is deployed on some pools - slush for example - that means you will not be paid your fair share if you disconnect from the pool either on purpose or due to an outage.

The expected loss is related to the amount of time you disconnect vs the % of the average block find time that the pool pays (whatever that may be)
If you simply just disconnected and don't come back, then it would be related to a % of the average expected loss.

You could compare that to a 'lock-in contract' where you lose out by leaving, except you don't gain much by staying.
There is possibly a very small gain when others lose out by leaving - the amount lost by anyone who disconnected may be distributed to everyone who didn't - so if the pool is 5PH and someone with 5TH lost 100% of their payout, everyone might gain about 0.1% more on that particular payout on their expected 100% less orphans less fees ... of course that depends on how the payout is calculated.

I'm not sure anyone has proven that to be the case, Kano. From your description you could as easily be talking about PPLNS with a very small N, which is still a fair reward method, and for any fair system expected value of a share should always be B/D.

The expected value of shares submitted over a period of time including disconnections should be the sum of the shares submitted.

Well just to use the simplest argument based on one assumption: that the pool's anti-hop deterrent does work:
The expected payout is E if you mine 24/7
The expected payout if you hop is expected to be E-e lower by some 'e' if the hopping deterrent works.
So E-e is less than E if e>0

[organofcorti]

...
I guess also there needs to be clearly stated that anti-hop functionality is deployed on some pools - slush for example - that means you will not be paid your fair share if you disconnect from the pool either on purpose or due to an outage.

The expected loss is related to the amount of time you disconnect vs the % of the average block find time that the pool pays (whatever that may be)
If you simply just disconnected and don't come back, then it would be related to a % of the average expected loss.

You could compare that to a 'lock-in contract' where you lose out by leaving, except you don't gain much by staying.
There is possibly a very small gain when others lose out by leaving - the amount lost by anyone who disconnected may be distributed to everyone who didn't - so if the pool is 5PH and someone with 5TH lost 100% of their payout, everyone might gain about 0.1% more on that particular payout on their expected 100% less orphans less fees ... of course that depends on how the payout is calculated.

I'm not sure anyone has proven that to be the case, Kano. From your description you could as easily be talking about PPLNS with a very small N, which is still a fair reward method, and for any fair system expected value of a share should always be B/D.

The expected value of shares submitted over a period of time including disconnections should be the sum of the shares submitted.

Well just to use the simplest argument based on one assumption: that the pool's anti-hop deterrent does work:
The expected payout is E if you mine 24/7
The expected payout if you hop is expected to be E-e lower by some 'e' if the hopping deterrent works.
So E-e is less than E if e>0

Bah! Humbug! Redefining expectation, that's cheating.

Moreover, that's what I said when I caught Bitclockers out. You shouldn't use a man's words against him.

[windpath]

Very excited about the blocks we have been finding lately...



Not so excited about the frequency of 0 transaction blocks....

If your "that guy" please PM me, I'll keep your identity private, and may be able to help you make your node more efficient and include some transactions in your blocks as well.

0 transaction blocks reflect poorly on our community, and with current transaction volume slow down the whole network.

[tomsanders]

Link to the P2Pool Node list?

[Geremia]

Code:

Local: 1946GH/s in last 5.8 minutes Local dead on arrival: ~37.2% (35-39%) Expected time to share: 4.0 hours

Why is my "Local dead on arrival" so high?

Update: It just now improved:

Code:

Local: 2017GH/s in last 10.0 minutes Local dead on arrival: ~1.8% (1-4%) Expected time to share: 3.8 hours

Was it just "adjusting" or something?

[tomsanders]

I may sound a little dumb for writing this.... with the p2p pool are you your own "private" pool e.g you keep the entire 25BTC apart from a little fee or is it a very big mining pool and you are supporting the pool by having your own node close to you? Im guess then the 25BTC would be then shared?

Thanks

[roy7]

I may sound a little dumb for writing this.... with the p2p pool are you your own "private" pool e.g you keep the entire 25BTC apart from a little fee or is it a very big mining pool and you are supporting the pool by having your own node close to you? Im guess then the 25BTC would be then shared?

Thanks

The 25BTC is shared throughout all of the p2pool network. It's like a distributed pool, where you mine off your own node (or a public node) with the advantages that can give you (zero latency if you run it local to your miners) but you also benefit from sharing hash power with the rest of the p2pool network to reduce variance.

[roy7]

I believe there's also work being done to use the proxy to be able to balance miner hashrate.  The idea being there to try to mitigate the effects of "swamping" a node if a large miner comes onto a node with a lot of smaller miners, they could be proxied to a different node where their hashrate is more appropriate.

I've been away a year so maybe you are referring to something else, but if you are talking about small miners having their difficulty targets skyrocket when a large miner joins the same public p2pool node, that is avoided with this pull request:

https://github.com/forrestv/p2pool/pull/174

Basically, it assigns difficulty targets to miners based on their payment address instead of the total hash rate of the local node. This way the difficulty target your miner is working on is identical to running your own private node, even if some other far larger miner is on the same public node with you. This is for the true difficulty target to get shares, a different pull request was submitted related to pseduo-shares.

Just been checking out ckpool and ckproxy, I think those came out after I left, quite cool. I'd have been all over ckproxy if I knew about it back then. I attempted to learn enough Haskell to run proxypool with p2pool for BTC mining, but it wasn't worth the pain.

[roy7]

Maybe p2pool could consider some (more complex) version of that in the share chain to force miners to use a diff related to their hash rate and thus allow smaller miners to reduce their variance.

Been looking around the past month of posts and saw this. I think I recall a conversation once that it'd be nice to scale the difficulty target for miners to target some preferred # of shares in the sharechain. It does this already in some regard as I think there's a cap the target for a hash rate can't be so low that you'll find more than 5% of the full share chain's % of shares on average. (Going by memory and haven't touched p2pool since last April.) However that's realistically way too high. For low variance payouts for a large miner, you only need to keep a few shares in the chain at all times. This is a reason "friendly" large miners could use /DIFF to kick their difficulty targets way higher... they'd have less shares in the share chain (leaving more room for smaller people and reducing their variance), but the shares they do have are worth that much more during payout.

Actually I used google and spent some time trying to find what I'm thinking of. I did. It's here, the bottom portion of this post:

https://bitcointalk.org/index.php?topic=18313.msg4830990#msg4830990

with small followup:

https://bitcointalk.org/index.php?topic=18313.msg4831519#msg4831519

What I provided was a different way to calculate target difficulties for large miners such that they'd always have shares in the sharechain (amount of payment would vary, but they'd almost never miss a payment totally). For large miners it might drop the # of shares in the sharechain they were gobbling up by a factor of 10-15. Which means the sharechain difficulty is lower, making it easier for smaller miners to find shares.

Of course it has the problem of abuse by larger miners splitting up their 'farm' to multiple addresses and thus acting like multiple smaller miners - so that would have to be resolved some way I've got no idea how

It wouldn't be in a large miner's financial interest to push out/hurt smaller miners. Just makes p2pool network speed lower for everyone. Granted, the community is full of vandals though.

Edit: Correction, the hard code in p2pool was to make a miner's minimum difficulty target even if they said /1 to be such that they can only find 1.67% of the total share chain's shares. This is far too many shares for 1 miner to have, of course...

[bhanu545]

Very excited about the blocks we have been finding lately...



Not so excited about the frequency of 0 transaction blocks....

If your "that guy" please PM me, I'll keep your identity private, and may be able to help you make your node more efficient and include some transactions in your blocks as well.

0 transaction blocks reflect poorly on our community, and with current transaction volume slow down the whole network.

Do you mean zero transaction blocks bring no extra transaction fee with block reward when it is found? and will the transaction fee be shared to all miners or only block founder?
How do you insert transactions in blocks? any reference to shed some light about this concept?

[jonnybravo0311]

Very excited about the blocks we have been finding lately...

...img snipped...

Not so excited about the frequency of 0 transaction blocks....

If your "that guy" please PM me, I'll keep your identity private, and may be able to help you make your node more efficient and include some transactions in your blocks as well.

0 transaction blocks reflect poorly on our community, and with current transaction volume slow down the whole network.

Do you mean zero transaction blocks bring no extra transaction fee with block reward when it is found? and will the transaction fee be shared to all miners or only block founder?
How do you insert transactions in blocks? any reference to shed some light about this concept?

Zero transaction blocks (well, technically they're reported as 1 transaction because of the block reward distribution) have zero transaction fees.  How could they have any fees since there are no transactions in the block?  In a block that actually does have transactions - like that last one p2pool found 353366 - there were 0.02990026BTC of fees.  Those fees are combined with the 25BTC block reward and distributed to all miners according to the shares the miners have on the payout list.

The transactions are inserted into the block by the node on which you're mining.  Each node runs its own copy of the Bitcoin Core.  If a node has setup their configuration to do something like setting a max block size of 10kb, or their mintxfee to something like 100BTC, then if they happen to find a share that satisfies the network difficulty the chances are exceptionally good the block will contain no transactions other than the block reward.

[kano]

...

Of course it has the problem of abuse by larger miners splitting up their 'farm' to multiple addresses and thus acting like multiple smaller miners - so that would have to be resolved some way I've got no idea how

It wouldn't be in a large miner's financial interest to push out/hurt smaller miners. Just makes p2pool network speed lower for everyone. Granted, the community is full of vandals though.

Edit: Correction, the hard code in p2pool was to make a miner's minimum difficulty target even if they said /1 to be such that they can only find 1.67% of the total share chain's shares. This is far too many shares for 1 miner to have, of course...

So basically what you are saying is that people with more mining power will be happy to have MUCH higher difficulty than p2pool's already very high difficulty, so that others can have lower difficulty, and thus people with more mining power will also have higher variance than most people mining on p2pool so the others can all have lower variance?
Unfortunately "people will do what is best for others at their own detriment" usually doesn't pan out too well.

In fact there's a very obvious example with the recent blocks where 2 were mined by people who put almost no transactions in them.
No transactions confirmed is bad for bitcoin in general, but anyone short sighted may well only care about the BTC reward and not the bitcoin network ... or p2pool's reputation.

[roy7]

So basically what you are saying is that people with more mining power will be happy to have MUCH higher difficulty than p2pool's already very high difficulty, so that others can have lower difficulty, and thus people with more mining power will also have higher variance than most people mining on p2pool so the others can all have lower variance?
Unfortunately "people will do what is best for others at their own detriment" usually doesn't pan out too well.

Yeah maybe I'm being naïve. Don't know. It just doesn't seem like the largest miners need to have potentially 150 shares in the share chain at a given time, when they could have 15 shares worth 10x as much instead. There isn't a real 'detriment' here as you'll still earn the same amount of income on average.

Also a big miner shouldn't view p2pool share difficulty as "very high".

I don't have the math to come up with a confidence interval on the # of shares someone will have in the share chain for a given hash power off the top of my head. However, for sake of example, lets say the huge miner with 150 share on average has high confidence to be in the 130-170 range. Likewise, if his diff target was ten times higher, he'd be 13-17 with 15 on average. If each share of normal difficulty paid out .01 BTC for simplicity (.1 BTC for the 10x higher target shares), then his income averages 150 * .01 or 15 * .1, 1.5 BTC either way. The question is if the steps within this range are steps of .01 BTC each or .1 BTC each.

When I used to run sha256 alt coin p2pool nodes and would mine on them with my S1, I'd set /DIFF as high as I could to not shut out GPU miners. I know, few people likely do that in the p2pool BTC world, which is why it makes more sense for p2pool to automatically scale the vardiff beyond just minimizing network traffic and be smart regarding the fact the sharechain is a fixed size.

I guess my point is the # of shares in the sharechain is a resource that needs to be managed as intelligently as possible for the health of the network. For each big miner with 100+ shares in the chain at any one time, there will be far more miners struggling to keep just 1 share active at all times.

Edit: On a sidenote I sent an email to windpath with various old links and github patches in case he wants to apply some nice front end improvements from last year. It seems p2pool-node-status went quiet before they moved from -devel to the normal branch, and the alt coins that also used the interface all faded away. So the cool improvements to show things like estimated time per share and miner-specific difficulty targets have been forgotten to the sands of time.

[Meuh6879]

interesting ...  

[Geremia]

Code:

Local: 1946GH/s in last 5.8 minutes Local dead on arrival: ~37.2% (35-39%) Expected time to share: 4.0 hours

Why is my "Local dead on arrival" so high?

Update: It just now improved:

Code:

Local: 2017GH/s in last 10.0 minutes Local dead on arrival: ~1.8% (1-4%) Expected time to share: 3.8 hours

Was it just "adjusting" or something?

Update: p2pool casuse my S5 to lock up after a certain amount of time! Why?
I don't get this problem with Eligius pool. Also, I get much higher hashrates with Eligius, which seems strange to me.

[tomsanders]

Anyone know why im getting this fault?

user@ubuntu:~$ bitcoin-cli getblockcount
error: {"code":-28,"message":"Verifying blocks..."}


Thanks

[Geremia]

Anyone know why im getting this fault?

user@ubuntu:~$ bitcoin-cli getblockcount
error: {"code":-28,"message":"Verifying blocks..."}


Thanks

That's what it does when bitcoind or bitcoin-qt first starts up.

[tomsanders]

Anyone know why im getting this fault?

user@ubuntu:~$ bitcoin-cli getblockcount
error: {"code":-28,"message":"Verifying blocks..."}


Thanks

That's what it does when bitcoind or bitcoin-qt first starts up.

Nothing to be concerned about then?

Thanks

[Geremia]

Code:

Local: 1946GH/s in last 5.8 minutes Local dead on arrival: ~37.2% (35-39%) Expected time to share: 4.0 hours

Why is my "Local dead on arrival" so high?

Update: It just now improved:

Code:

Local: 2017GH/s in last 10.0 minutes Local dead on arrival: ~1.8% (1-4%) Expected time to share: 3.8 hours

Was it just "adjusting" or something?

Update: p2pool casuse my S5 to lock up after a certain amount of time! Why?
I don't get this problem with Eligius pool. Also, I get much higher hashrates with Eligius, which seems strange to me.

Update #3: I corrected this issue by (1) making each of my miners mine to a separate user and (2) adding these options to `cgminer`:
--queue 0 --failover-only --expiry 1 --scan-time 1

Are these good settings?

[Geremia]

Anyone know why im getting this fault?

user@ubuntu:~$ bitcoin-cli getblockcount
error: {"code":-28,"message":"Verifying blocks..."}


Thanks

That's what it does when bitcoind or bitcoin-qt first starts up.

Nothing to be concerned about then?

Thanks

No, that's normal behavior. Once it's finishing verifying blocks, you will then be able to issue RPC calls just fine.