[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool

[IYFTech]

Mr. Jinx, occasional memory leaks have plagued P2Pool nodes for a while. Python/Twisted makes it pretty easy to leave a reference to something or leave a task running in the background... several of them have been patched, though.

Try looking through P2Pool's log file for lines that contain a ">" character, which indicates an uncaught exception/traceback. A task dying that periodically removes old shares was the cause of a previous memory leak; maybe this is something similar.

Ah, thanks. That makes it easier to troubleshoot!

This one happens alot. Always the same peers that gives this error (2).
What could be causing this?

Code:

2014-02-24 05:00:39.106919 > in handle_share_hashes:
2014-02-24 05:00:39.107328 > Traceback (most recent call last):
2014-02-24 05:00:39.107653 > Failure: twisted.internet.defer.TimeoutError: in GenericDeferrer
2014-02-24 05:00:39.109908 Lost peer x.x.x.x:36098 -
2014-02-24 05:00:39.110112     Connection was aborted locally, using
2014-02-24 05:00:39.110291     L{twisted.internet.interfaces.ITCPTransport.abortConnection}.
2014-02-24 05:00:39.110502
2014-02-24 05:00:39.110721     @since: 11.1
2014-02-24 05:00:39.110918     .[

Anyone who can shed some light on this?

This is p2pool disconnecting from older, incompatible versions of p2pool (v11.1) - it's a good thing. The others I'm not sure of, sorry.

[jedimstr]

So on the BTC side, P2Pool has been having horrible luck lately... and the Hashrate on the pool has essentially been flat or declining since the new year.

Anything we can do to encourage more people to use P2Pool (I saw a PR push on Reddit around the holidays and the GHash.io panic on Reddit), or to enhance our chances of getting a few blocks?  It seems every other block these days is GHash.io blocks now and the previous outrage has died down since BTCGuild and Eligius have caught up and split the top 3 spots: https://blockchain.info/pools

[Tegija]

So on the BTC side, P2Pool has been having horrible luck lately... and the Hashrate on the pool has essentially been flat or declining since the new year.

Anything we can do to encourage more people to use P2Pool (I saw a PR push on Reddit around the holidays and the GHash.io panic on Reddit), or to enhance our chances of getting a few blocks?  It seems every other block these days is GHash.io blocks now.

it would be nice if we could start some campaign to get more people to p2pool. i am trying hard and have found some new people, but considering that at this time every 2-3 days there come more new TH to the network than whole p2pool totally has, we need really good strategy to get new miner to p2pool.

it is pretty amazing that people do not understand that it is for everybody good if p2pool gets bigger and more healthy.

i hope it will change soon an we have more people mining with us.

[IYFTech]

This is an ongoing age old problem, scroll back & you'll see.

Not enough development, not enough (lately almost zero) support, not enough compatibility, not user friendly enough......the list goes on & on.

It's a shame.

[kano]

Hmm ... so how many TH/s is the 'development team' being paid?

[roy7]

Hmm ... so how many TH/s is the 'development team' being paid?

Since donations are stored on the share chain, you could actually whip up a tool to calculate that if you were curious enough.

(I realize you might be asking a rhetorical question in response to IYFTech.)

[PatMan]

Hmm ... so how many TH/s is the 'development team' being paid?

There's a dev team?........

[phillipsjk]

Edit: Since AFAIK, the getwork protocol does not allow workers to be interrupted with new work, we should be able to estimate the expected stales given a specific block frequency. If we assume a 13s worst case latency, that works out to at most 43% stales with a 30 second target. If we assume a 6.5s average, that works out to 21.7% stale. -- that does seem high.

Edit: Apparently Longpolling works around HTTP limitations by having the miner request new work immediately. The server then does not respond until new work is ready. Testing time.

See this:

https://bitcointalk.org/index.php?topic=18313.msg4556966#msg4556966

I don't know of anyone who has got slush's proxy to work with p2pool, I've tried countless times myself - it just doesn't work, in fact I have a feeling that that's what is now causing my p2pool start up error/warning. Your only choice with a blade is to use +1 at the end of your user name/addy which, tbh, is a bit of a waste.....

OK, I finally got P2Pool running and learned the truth. One thing that confused me is that the +1 option does not appear to be documented anywhere. What it does is tell the blade to report diff-1 shares. Apparently, the blades do not support higher difficulty.

Test results are in: no longpoll support. 21% stale for me, I guess. (confirmed with netstat -- only transient connections show up on port 9332) The reported DOA rate varies between 20-40% for some reason.

PS: Slush's proxy does not work because P2Pool does not appear to support the stratum protocol. I may try a different proxy if I think it will result in less CPU usage (for example, using compiled, rather than interpreted code). Better power savings would result from replacing the CPU though.

Edit: P2Pool does support Stratum. Slush's proxy not working is more complex. P2Pool tries to automatically detect whether the worker is using stratum or not. While at the same time, the slush proxy tries to automatically detect if the pool is using Stratum or not (using the getwork protocol). I tried removing that detection from the slush code, and it still didn't work.

[IYFTech]

Exactly.

If you do find a proxy that is compatible with p2pool without a daft DOA rate I'm sure there are a few hundred miners who would like to hear about it......

[phillipsjk]

Well I think I found out where the extra 6 seconds of lag came from:

Code:

2014-02-25 13:53:22.835376 Punishing share for 'Block-stale detected! height(6f2f28d90af36171b9bc41e7986c7bb24943c37bec4ef054) < height(151b94b7ca0744b491b3fcbf950435984bc36c5a97c9a08e9) or 
19015f53 != 19015f53'! Jumping from 1ce9c6ad to 212471a6!
...
2014-02-25 13:53:24.455802 New work for worker! Difficulty: 1.000000 Share difficulty: 529462.142225 Total block value: 25.027840 BTC including 106 transactions
.
.
.
2014-02-25 14:05:10.552019 Transaction db65517b5b4b7314e86aa52a1f9e0ea3be37044eab1a73e408ea07f74ec5e383 rescued from peer latency cache!
...
2014-02-25 14:05:17.463766 New work for worker! Difficulty: 1.000000 Share difficulty: 553972.015873 Total block value: 25.105139 BTC including 529 transactions
.
.
.
2014-02-25 14:14:57.135562 P2Pool: 17337 shares in chain (9424 verified/17341 total) Peers: 6 (0 incoming)
...
2014-02-25 14:15:00.183437 P2Pool: 17338 shares in chain (9425 verified/17342 total) Peers: 6 (0 incoming)
...
2014-02-25 14:15:05.433806 New work for worker! Difficulty: 1.000000 Share difficulty: 565095.416800 Total block value: 25.057434 BTC including 304 transactions

I may be misinterpreting things, but I appears to take my machine 1-7 seconds to put work together.

[VarDiff]

P2Pool added to mining pools database
Good luck, miners.

[paniczklos]

Hi guys, is there a way to check why I'm not getting fees from every mined block (and the total fees are much lower than the set 1%) ?
Right now avarage payout of two biggest miners only is ~4000BC, but I'm not getting any fees ;/ (It's not about greed - I just don't understand it).
Hashrate is constant (and growing).




http://freebtc.eu:8336/static/graphs.html?Day

[Mr. Jinx]

Hi guys, is there a way to check why I'm not getting fees from every mined block (and the total fees are much lower than the set 1%) ?
Right now avarage payout of two biggest miners only is ~4000BC, but I'm not getting any fees ;/ (It's not about greed - I just don't understand it).
Hashrate is constant (and growing).

I had the same question in another topic:

Should you get a 1% fee everytime the miners are paid? I haven't received any fee yet, while the active miners have received their share payments.
Or will this fee be paid on a daily base? I got the feeling something is setup wrong.

It isn't intuitive, but each share a miner submits to you has a 1% chance of becoming your share instead. So you'll see a lot of variance in your pool fee income, but long-term it should even out (at the expense of higher variance for the miner as well).

Maybe one day p2pool will allow the fee amount to be recorded in the share, like the donation amount is, so it can be paid exactly when each share is paid and all shares remain with the miner.

After a few weeks I can confirm this behavior. Sometimes there are days without a fee, and then suddenly you get a few in a row. Looking back a month it seems to be about 1%!

[sshapiroNJ]

Hi guys, is there a way to check why I'm not getting fees from every mined block (and the total fees are much lower than the set 1%) ?
Right now avarage payout of two biggest miners only is ~4000BC, but I'm not getting any fees ;/ (It's not about greed - I just don't understand it).
Hashrate is constant (and growing).

I had the same question in another topic:

Should you get a 1% fee everytime the miners are paid? I haven't received any fee yet, while the active miners have received their share payments.
Or will this fee be paid on a daily base? I got the feeling something is setup wrong.

It isn't intuitive, but each share a miner submits to you has a 1% chance of becoming your share instead. So you'll see a lot of variance in your pool fee income, but long-term it should even out (at the expense of higher variance for the miner as well).

Maybe one day p2pool will allow the fee amount to be recorded in the share, like the donation amount is, so it can be paid exactly when each share is paid and all shares remain with the miner.

After a few weeks I can confirm this behavior. Sometimes there are days without a fee, and then suddenly you get a few in a row. Looking back a month it seems to be about 1%!

It has also happened to me. I need like a couple of months to understand what is going on until I read on post on the site (another subforum) and finally understood what's going on

[roy7]

Fee % isn't a % of each block. It is a % of shares are randomly given to you instead of your miner. So in 100 shares, on average, 1 share will be 100% credit to you, and 99 shares will be 100% credit to your miners. It should average out long-term that you make 1% of the total income of the pool. But it is very high variance. I hope in a future major version of p2pool the fee % is applied to every block like donation %.

[roy7]

For those interested I just found an interesting patch the author mentioned on the HHTT thread:

https://github.com/iongchun/p2pool/tree/auto-worker-diff

Gives you two command line options to control the pseduo share difficulty. You can set a target difficulty per-miner or per-address. Using that per-address would seem to fit nicely with my pull request to set share difficulty based on address speed instead of local node speed.

[Mr. Jinx]

I can currently offer a 5 LTC bounty if someone can help me resolve this issue:

I'm still having this memory leak issue with p2pool which I would love to resolve:


I looked at the ">" errors in the logs, but I don't know if this is the problem:

Code:

2014-02-24 05:34:43.706844 > Couldn't link returned work's job id with its handler. This should only happen if this process was recently restarted!

Code:

2014-02-24 05:05:18.137532 Worker xxxxxxxxxxx submitted share with hash > target:
2014-02-24 05:05:18.137946     Hash:   xxxxxxxxxxxxxxx
2014-02-24 05:05:18.138166     Target: xxxxxxxxxx

Code:

2014-02-24 05:00:39.106919 > in handle_share_hashes:
2014-02-24 05:00:39.107328 > Traceback (most recent call last):
2014-02-24 05:00:39.107653 > Failure: twisted.internet.defer.TimeoutError: in GenericDeferrer
2014-02-24 05:00:39.109908 Lost peer x.x.x.x:36098 -
2014-02-24 05:00:39.110112     Connection was aborted locally, using
2014-02-24 05:00:39.110291     L{twisted.internet.interfaces.ITCPTransport.abortConnection}.
2014-02-24 05:00:39.110502
2014-02-24 05:00:39.110721     @since: 11.1

Right now I have to restart the node every few days, which the miners don't like.

[matthewh3]

I'm thinking of setting a P2Pool node up for around 2MH/s on Vertcoin.  Can you still auto donate 1% to the dev's?

[roy7]

I'm thinking of setting a P2Pool node up for around 2MH/s on Vertcoin.  Can you still auto donate 1% to the dev's?

I don't think Forrest has a Vertcoin address but you can ask him. The address hard coded into the repo wouldn't work anyway. Maybe best to just do a manual donation now and then to his BTC address.

[hasis]

Hi! I have the following problem. Sometimes, if the restart p2pool (quark-algoritm), when the power pool > ~ 150 MH / s, process starts to load the processor to 100% and all shares the dead. All logentries "DEAD ON ARRIVAL". Why this happens and how to solve this problem?