[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool

[twmz]

So ... there any sign of fixing the p2pool drastic performance problem that was the reason BFL sent forrestv a Jalapeno weeks ago?

What you mean? p2pool is working fine. With new bitcoin version (low latencies) especially.

Not with high hashrate ASICs and stratum.

[gyverlb]

So ... there any sign of fixing the p2pool drastic performance problem that was the reason BFL sent forrestv a Jalapeno weeks ago?

Why don't you ask him if he received it and started using it? No need to pollute this thread with this, only people with Jalapenos can help and I don't see any around here.

[kano]

So ... there any sign of fixing the p2pool drastic performance problem that was the reason BFL sent forrestv a Jalapeno weeks ago?

Why don't you ask him if he received it and started using it? No need to pollute this thread with this, only people with Jalapenos can help and I don't see any around here.

Yes feel free to stick your head in the sand and dream of unicorns.

[gyverlb]

So ... there any sign of fixing the p2pool drastic performance problem that was the reason BFL sent forrestv a Jalapeno weeks ago?

Why don't you ask him if he received it and started using it? No need to pollute this thread with this, only people with Jalapenos can help and I don't see any around here.

Yes feel free to stick your head in the sand and dream of unicorns.

I may have missed someone (sorry), but apart from you and maybe forrestv (didn't remember any post where he confirmed testing one on p2pool or even having received it, feel free to point me to it) I don't know of anyone with a Jalapeno in this thread. As you obviously don't use p2pool or tried to help fix any problem with it (given that you don't waste an occasion to troll about how p2pool not being good for Bitcoin) what's your point asking about "drastic performance problems" for hardware nearly nobody owns?

I have more than twice the hashrate of a Jalapeno mining on p2pool (with a single rig about half of its speed) with no problem and I'm far from being the largest miner on it: the problem is either due to a single connection to p2pool reaching a high enough hashrate or some peculiar behavior of the hardware that makes cgminer behave differently with it and trigger a p2pool bug.
Even Avalons are rare enough that finding both an Avalon and a developer interested in helping at the same time is difficult (seems ckolivas made progress in cgminer but couldn't help with the p2pool code). If forrestv doesn't have a working Jalapeno in hand you can stick your head in the sand and wish for support any day...

[forrestv]

I may have missed someone (sorry), but apart from you and maybe forrestv (didn't remember any post where he confirmed testing one on p2pool or even having received it, feel free to point me to it) I don't know of anyone with a Jalapeno in this thread.

Sorry for not posting about it, but I did in fact receive a 3.4GH/s single-core Jalapeno prototype (half the hash rate of a normal Jalapeno). I'm composing a more thorough report right now, but it suffices to say that there were no interesting results. It works with P2Pool about as well as was expected - ~1520% DoA due to ASIC batching latency, but no problems other than that.

EDIT: This was all using cgminer 3.2.0. If anyone who can demonstrate problems with Jalapenos can provide more details about their setup, I can try to replicate it.

[zvs]

well, about 10-15 nodes dropped offline for me with this latest batch of horse staple battery transactions

[gyverlb]

It's nearly confirmed: getblocktemplate latency doesn't influence efficiency with both latest p2pool and bitcoind. I can't even raise it to more than 0.2s now and I don't see any efficiency change between 0.01s and 0.2s.

What's more interesting is the optimal number of connections with other P2Pool node. I had bandwidth problems and lowered them to a total of 6 (3 in and 3 out) to fit both bitcoind and P2Pool's traffic in a ~400kb/s upload limit. My efficiency was between 110 and 115% with these 6 connections.

My BW problems were solved since then and I didn't touch my settings (an average of 112.5% efficiency is not too bad...).
I just tested with a total of 10 connections and my efficiency reached 120%. I'll now focus on testing these settings to make it a bit more clear the kind of compromise done by tweaking them.

I've seen that the 2 most recent P2Pool blocks were less than 200kB and 20kB in size.
There's no good reason to generate short blocks like that when there are plenty transactions to reap fees from anymore.
Please start upgrading to bitcoind-0.8.2 and if you don't have bandwidth limitations that make it impossible to do so for you, raise your maxblocksize and lower you mintxfee and minrelaytxtfee settings (see my guide for values to start from). You and everyone else will earn more BTC this way.

[kano]

I may have missed someone (sorry), but apart from you and maybe forrestv (didn't remember any post where he confirmed testing one on p2pool or even having received it, feel free to point me to it) I don't know of anyone with a Jalapeno in this thread.

Sorry for not posting about it, but I did in fact receive a 3.4GH/s single-core Jalapeno prototype (half the hash rate of a normal Jalapeno). I'm composing a more thorough report right now, but it suffices to say that there were no interesting results. It works with P2Pool about as well as was expected - ~1520% DoA due to ASIC batching latency, but no problems other than that.

EDIT: This was all using cgminer 3.2.0. If anyone who can demonstrate problems with Jalapenos can provide more details about their setup, I can try to replicate it.

Yes I've already requested a protocol change for p2pool and Jalapenos, but that wasn't the problem.

The problem was p2pool performance ... according to what's written there:
https://bitcointalk.org/index.php?topic=18313.5120

that gyverlb posted twice on that page, once before and once after, the 7 posts on the subject ...

[stepkrav]

anyone tried to run p2pool alongside an electrum server? Is that feasible?

[gyverlb]

I may have missed someone (sorry), but apart from you and maybe forrestv (didn't remember any post where he confirmed testing one on p2pool or even having received it, feel free to point me to it) I don't know of anyone with a Jalapeno in this thread.

Sorry for not posting about it, but I did in fact receive a 3.4GH/s single-core Jalapeno prototype (half the hash rate of a normal Jalapeno). I'm composing a more thorough report right now, but it suffices to say that there were no interesting results. It works with P2Pool about as well as was expected - ~1520% DoA due to ASIC batching latency, but no problems other than that.

EDIT: This was all using cgminer 3.2.0. If anyone who can demonstrate problems with Jalapenos can provide more details about their setup, I can try to replicate it.

Yes I've already requested a protocol change for p2pool and Jalapenos, but that wasn't the problem.

The problem was p2pool performance ... according to what's written there:
https://bitcointalk.org/index.php?topic=18313.5120

that gyverlb posted twice on that page, once before and once after, the 7 posts on the subject ...

Meaning that there was another user (ckolivas) with a Jalapeno I should have remembered? I suppose I should be ashamed...

So that's make it 3 now (and forrestv doesn't even have the same model you and ckolivas have, just peachy for someone trying to debug a problem).

Instead of trolling around:
What's interesting is that the same (or probably very similar, ckolivas might have tested with an earlier version of cgminer) software using the same driver doesn't exhibit the same behavior at all with different hashrates.
This points to only one of the 2 possible sources of problems with ASICs and p2pool I suspected :
a bug in p2pool when a single miner reaches a hashrate limit between 3.4GH/s and 6+GH/s (2^32 H/s comes to mind...).

[furball]

It's nearly confirmed: getblocktemplate latency doesn't influence efficiency with both latest p2pool and bitcoind. I can't even raise it to more than 0.2s now and I don't see any efficiency change between 0.01s and 0.2s.

What's more interesting is the optimal number of connections with other P2Pool node. I had bandwidth problems and lowered them to a total of 6 (3 in and 3 out) to fit both bitcoind and P2Pool's traffic in a ~400kb/s upload limit. My efficiency was between 110 and 115% with these 6 connections.

My BW problems were solved since then and I didn't touch my settings (an average of 112.5% efficiency is not too bad...).
I just tested with a total of 10 connections and my efficiency reached 120%. I'll now focus on testing these settings to make it a bit more clear the kind of compromise done by tweaking them.

I've seen that the 2 most recent P2Pool blocks were less than 200kB and 20kB in size.
There's no good reason to generate short blocks like that when there are plenty transactions to reap fees from anymore.
Please start upgrading to bitcoind-0.8.2 and if you don't have bandwidth limitations that make it impossible to do so for you, raise your maxblocksize and lower you mintxfee and minrelaytxtfee settings (see my guide for values to start from). You and everyone else will earn more BTC this way.

Thanks for the update, awesome work! Just wanted to clarify something with what you said above; you said you are seeing no correlation between latency and efficiency with the new versions, but you are working from a very low latency range. Do you have any reason to believe that if latencies on nodes are higher than 0.2s(say between 0.3 and 0.5) that efficiency will still NOT be affected? Just interested to see that you think.

BTW, I've followed your guide regarding latest versions, blockmaxsize/txfees and miner tuning and looks to be helping..thanks for that!

[twmz]

Meaning that there was another user (ckolivas) with a Jalapeno I should have remembered? I suppose I should be ashamed...

So that's make it 3 now (and forrestv doesn't even have the same model you and ckolivas have, just peachy for someone trying to debug a problem).

I have a 5.6 GH/s jalapeno, but I never tried it with p2pool because I got it after the previous reports of it not working (and so I assumed it wouldn't work).  I am traveling through next Thursday, but when I get back home, I'll try it with p2pool (with the latest versions of both p2pool and bitcoind) for next weekend and see how it does.

[gyverlb]

Meaning that there was another user (ckolivas) with a Jalapeno I should have remembered? I suppose I should be ashamed...

So that's make it 3 now (and forrestv doesn't even have the same model you and ckolivas have, just peachy for someone trying to debug a problem).

I have a 5.6 GH/s jalapeno, but I never tried it with p2pool because I got it after the previous reports of it not working (and so I assumed it wouldn't work).  I am traveling through next Thursday, but when I get back home, I'll try it with p2pool (with the latest versions of both p2pool and bitcoind) for next weekend and see how it does.

Ideally forrestv should run the node the Jalapeno is mining on to debug it, better synchronize with him or you have high chances are just confirming what others with regular Jalapenos got without any clue on how to help.

[gyverlb]

[...]
Thanks for the update, awesome work! Just wanted to clarify something with what you said above; you said you are seeing no correlation between latency and efficiency with the new versions, but you are working from a very low latency range. Do you have any reason to believe that if latencies on nodes are higher than 0.2s(say between 0.3 and 0.5) that efficiency will still NOT be affected? Just interested to see that you think.

BTW, I've followed your guide regarding latest versions, blockmaxsize/txfees and miner tuning and looks to be helping..thanks for that!

I couldn't test it with getblocktemplate latencies higher than 0.25s. But I suspect you should be fine with much larger values: people here had 30s latencies with 0.8.1 and IIRC mot of them didn't see efficiency plummet like it would have if this latency mattered anymore.

I believe this latency only matters for blocks generated just after another: p2pool probably uses an empty template in this case until it gets an answer from bitcoind (forrestv, can you confirm this?). So unless it becomes a substantial portion of the 10min block interval that shouldn't be a big problem for P2Pool's income, it would reduce the total tx fees a bit that's all.

[twmz]

Ideally forrestv should run the node the Jalapeno is mining on to debug it, better synchronize with him or you have high chances are just confirming what others with regular Jalapenos got without any clue on how to help.

My primary goal will be to confirm or not the reports that it doesn't actually work.  If it has problems, then we can explore options to debug.

[furball]

[...]
Thanks for the update, awesome work! Just wanted to clarify something with what you said above; you said you are seeing no correlation between latency and efficiency with the new versions, but you are working from a very low latency range. Do you have any reason to believe that if latencies on nodes are higher than 0.2s(say between 0.3 and 0.5) that efficiency will still NOT be affected? Just interested to see that you think.

BTW, I've followed your guide regarding latest versions, blockmaxsize/txfees and miner tuning and looks to be helping..thanks for that!

I couldn't test it with getblocktemplate latencies higher than 0.25s. But I suspect you should be fine with much larger values: people here had 30s latencies with 0.8.1 and IIRC mot of them didn't see efficiency plummet like it would have if this latency mattered anymore.

I believe this latency only matters for blocks generated just after another: p2pool probably uses an empty template in this case until it gets an answer from bitcoind (forrestv, can you confirm this?). So unless it becomes a substantial portion of the 10min block interval that shouldn't be a big problem for P2Pool's income, it would reduce the total tx fees a bit that's all.

Good to know...thanks for the reply!

[Richy_T]

Hi guys. I am not sure if this counts as a bug in p2pool or not but we are apparently adding a "strange" zero value transaction output at the end of our payouts. This affected a wallet app (that has since been fixed).

Details here:

https://bitcointalk.org/index.php?topic=52674.msg2327217#msg2327217

[twmz]

I have a 5.6 GH/s jalapeno, but I never tried it with p2pool because I got it after the previous reports of it not working (and so I assumed it wouldn't work).  I am traveling through next Thursday, but when I get back home, I'll try it with p2pool (with the latest versions of both p2pool and bitcoind) for next weekend and see how it does.

I had some free time tonight so I tried this.  My intent was to run a 24 hour long test to see if there were problems with p2pool or cgminer going crazy and blowing up, as I believe was reported with previous attempts with ASIC devices.  However, I aborted my test after an hour when it was apparent that Jalapeno devices can not be effectively used with p2pool.  In fact, I see now that forrestv reported the same problem that I ran into just a few posts up in the thread, and I didn't notice that detail until now.

The problem is the same as with the BFL Single FPGA devices.  Immediately after a p2pool share is found and stratum signals that new work should start (or a longpoll occurs), the Jalapeno will have a tendency to submit several stale shares (DOA in p2pool terms) because the BFL chips can't be interrupted mid-work and so they keep working on the old work for a couple seconds after stratum has signaled to start on a new work item.  The result is approximately 20% DOA and a 20% drop of effective hashrate, which makes a Jalapeno pretty much unusable with p2pool for anyone that cares about their earnings.

Although the BFL SC devices supposedly support the ZPX command that was added for MiniRig FPGA devices to allow it to work on shorter nonce ranges (and therefore wake up for new work much more frequently and therefore have much lower DOA %), neither cgminer nor bfgminer (I tried them both) use this command with the BFL SC devices and instead use the ZNX queue command.  With any other pool, ZNX is a better choice because it will minimize idle time of the hardware.  But with p2pool, ZNX is likely worse than ZPX would have been.  In a perfect world, BFL would have added a "partial nonce" version of the ZNX command so that we could have queued work items and smaller nonce ranges, but that isn't the case.

So, the verdict, at least for jalapeno devices, is that they are essentially unusable with p2pool and there is nothing that p2pool can change (other than something drastic like not having shares every 10 seconds) to make it work.  Perhaps if cgminer supported switching to ZPX things might get a little better, but there would still almost certainly be a reduction in hashrate as compared with other pools that were compatible with the more efficient ZNX command.

I'm not sure how things will turn out with the 60 GH/s SC Singles, but my guess is they will have exactly the same problem and it won't matter that they are faster because they are really just more of the same ASIC chips working in parallel and each chip is still going to be testing an entire nonce range before returning any results.  But I am not entirely confident on that, so I will certainly retest this when mine arrive in 4-6 weeks.

Edit: Also, perhaps obvious is that it's possible the same problem with the CPU spiking would have eventually happened to me and I just didn't bother waiting long enough.  So I think it is unclear if the problem that ckolivas saw with p2pool a while back still exists with the latest versions of bitcoind and p2pool.

[forrestv]

Although the BFL SC devices supposedly support the ZPX command that was added for MiniRig FPGA devices to allow it to work on shorter nonce ranges (and therefore wake up for new work much more frequently and therefore have much lower DOA %), neither cgminer nor bfgminer (I tried them both) use this command with the BFL SC devices and instead use the ZNX queue command.  With any other pool, ZNX is a better choice because it will minimize idle time of the hardware.  But with p2pool, ZNX is likely worse than ZPX would have been.  In a perfect world, BFL would have added a "partial nonce" version of the ZNX command so that we could have queued work items and smaller nonce ranges, but that isn't the case.

Ah, I assumed cgminer was doing something optimal-ish! I'm going to write a BFL interface within P2Pool for testing to see how much better it can get.

[-ck]

Although the BFL SC devices supposedly support the ZPX command that was added for MiniRig FPGA devices to allow it to work on shorter nonce ranges (and therefore wake up for new work much more frequently and therefore have much lower DOA %), neither cgminer nor bfgminer (I tried them both) use this command with the BFL SC devices and instead use the ZNX queue command.

Check the BFL forums. We asked them why they didn't implement the command because we tried it and it doesn't work on BFL SC devices.

They did not respond. I think BFL are too busy drowning in fail to respond to petty questions like these.