[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool

[-ck]

Actually this is a major problem. If p2pool is rolling ntime and telling the mining software it can roll ntime, there is nothing to guarantee they wont collide. The expire=10 seconds tells the mining software how long it can roll work for, not how far into the future it can roll time. Potentially the software can roll time up to 2 hours into the future within that 10 seconds, so there can be no guarantee the work source and mining software won't clash. If you're asking the mining software to roll time, you shouldn't be rolling it within p2pool, or vice versa. Since p2pool generates its own work locally, I see no point even asking the mining software to roll the time if it's going to roll it itself. Since rolling time requires a lot less CPU than generating fresh work, it really is a matter of where you want the time to be rolled - the source of the work (in this case p2pool) or the mining software. It should not be done at both ends.

[tucenaber]

Since p2pool generates its own work locally, I see no point even asking the mining software to roll the time if it's going to roll it itself.

So if I hack the p2pool code, and remove the X-Roll-NTime header altogether as a short term fix, that would make cgminer stop rolling and my problem would be gone? Super easy  I will try it right away.

Thank you!

[Smoovious]

Since p2pool generates its own work locally, I see no point even asking the mining software to roll the time if it's going to roll it itself.

So if I hack the p2pool code, and remove the X-Roll-NTime header altogether as a short term fix, that would make cgminer stop rolling and my problem would be gone? Super easy  I will try it right away.

Thank you!

What is the difference between using long-polling, and not using long-polling, and since it is a 10-second interval anyways, is there any real need for long-polling to begin with?

(requesting a little enlightenment about it)

-- Smoov

[DiabloD3]

Heh, DiabloMiner fixes this by just watching for the X-Is-P2Pool header.

[tucenaber]

Since p2pool generates its own work locally, I see no point even asking the mining software to roll the time if it's going to roll it itself.

So if I hack the p2pool code, and remove the X-Roll-NTime header altogether as a short term fix, that would make cgminer stop rolling and my problem would be gone? Super easy  I will try it right away.

Thank you!

What is the difference between using long-polling, and not using long-polling, and since it is a 10-second interval anyways, is there any real need for long-polling to begin with?

(requesting a little enlightenment about it)

-- Smoov

Long polling means that the pool is waiting with giving a reply until it actually has new work, and that is good even for short period lengths too, I guess. But this issue isn't about long-polling, I think, but whether the server or the client should increment the timestamp when the client runs out of nonces.

Anyhoooo...

Removing the header S-O-L-V-E-D it!  Man, I'm sooo happy!

[kano]

Since p2pool generates its own work locally, I see no point even asking the mining software to roll the time if it's going to roll it itself.

So if I hack the p2pool code, and remove the X-Roll-NTime header altogether as a short term fix, that would make cgminer stop rolling and my problem would be gone? Super easy   I will try it right away.

Thank you!

What is the difference between using long-polling, and not using long-polling, and since it is a 10-second interval anyways, is there any real need for long-polling to begin with?

(requesting a little enlightenment about it)

-- Smoov

The LP is not just a notification, it also provides new work that is valid.
So the miner will get the new work immediately rather than having the extra delay of requesting it.

[coinnewb]

Since p2pool generates its own work locally, I see no point even asking the mining software to roll the time if it's going to roll it itself.

So if I hack the p2pool code, and remove the X-Roll-NTime header altogether as a short term fix, that would make cgminer stop rolling and my problem would be gone? Super easy  I will try it right away.

Thank you!

What is the difference between using long-polling, and not using long-polling, and since it is a 10-second interval anyways, is there any real need for long-polling to begin with?

(requesting a little enlightenment about it)

-- Smoov

Long polling means that the pool is waiting with giving a reply until it actually has new work, and that is good even for short period lengths too, I guess. But this issue isn't about long-polling, I think, but whether the server or the client should increment the timestamp when the client runs out of nonces.

Anyhoooo...

Removing the header S-O-L-V-E-D it!  Man, I'm sooo happy!

Do you mind sharing the details of changes?  I see the same issue with my setup from time to time.
Thanks.

[tucenaber]

Do you mind sharing the details of changes?  I see the same issue with my setup from time to time.
Thanks.

In the p2pool directory, look for the file p2pool/bitcoin/worker_interface.py. Open it in an editor and remove the line containing "X-Roll-NTime" and remove it.

Code:

     @defer.inlineCallbacks
     def _getwork(self, request, data, long_poll):
         request.setHeader('X-Long-Polling', '/long-polling')
-        request.setHeader('X-Roll-NTime', 'expire=10')
         request.setHeader('X-Is-P2Pool', 'true')

done!

[rav3n_pl]

PM forrestv about that!

[tucenaber]

done

[mdude77]

Anyhoooo...

Removing the header S-O-L-V-E-D it!  Man, I'm sooo happy!

I made your change to my side, installed python 2.7.3 64-bit (windows) and all the associated apps needed for p2pool and fired it up.

The dupe message is definitely gone.

Now, however, cgminer on my main miner (4x7870 = 2.6g/h) is complaining non stop about p2pool not providing work fast enough.  I think if my phoenix miner (4x5870 = 1.6g/h) had a better UI, it would be complaining too.

I'm not sure this is an improvement.

M

[-ck]

I would think you're better off making p2pool not roll ntime and leave it to the mining software.

[kano]

Anyhoooo...

Removing the header S-O-L-V-E-D it!  Man, I'm sooo happy!

I made your change to my side, installed python 2.7.3 64-bit (windows) and all the associated apps needed for p2pool and fired it up.

The dupe message is definitely gone.

Now, however, cgminer on my main miner (4x7870 = 2.6g/h) is complaining non stop about p2pool not providing work fast enough.  I think if my phoenix miner (4x5870 = 1.6g/h) had a better UI, it would be complaining too.

I'm not sure this is an improvement.

M

However, what this is saying is that your p2pool can't handle a local miner without roll-n-time
So I'd guess your p2pool setup is very slow or has poor network connectivity to your miner since it really should be a local miner talking to a local p2pool and that REALLY should be fast?!?

[rav3n_pl]

Maybe make this time up to 30 or 60 sec instead of removing line?

[tucenaber]

I would think you're better off making p2pool not roll ntime and leave it to the mining software.

I'm basically with this, but I'm not sure how to achieve it. There seem to be no consensus on what expire=10 actually means.

I guess (and it is just a guess after looking at the code) the reason p2pool does it this way is because two different miners must be given unique work, and the way it does it - since the merkle root is the same - is by giving 10 second timstamp intervals to work on, assuming the expire=10 makes the miner ask for more instead of rolling past 10 seconds. This assumption is false though, which is the problem.

Now with miners rolling on their own, the load on the server would be smaller but since each miner will be working on the exact same work, the clashes will return in a different form, if you have more than one miner connected. Not a good solution...

One simple way to avoid the latter is to have unique payment addresses for each miner. That would make the merkle roots diffferent, right? Then we could do it the ckolivas way and everybody lives happily ever after

Forrest can give a better explanation of this, tho.

[mdude77]

Anyhoooo...

Removing the header S-O-L-V-E-D it!  Man, I'm sooo happy!

I made your change to my side, installed python 2.7.3 64-bit (windows) and all the associated apps needed for p2pool and fired it up.

The dupe message is definitely gone.

Now, however, cgminer on my main miner (4x7870 = 2.6g/h) is complaining non stop about p2pool not providing work fast enough.  I think if my phoenix miner (4x5870 = 1.6g/h) had a better UI, it would be complaining too.

I'm not sure this is an improvement.

M

However, what this is saying is that your p2pool can't handle a local miner without roll-n-time
So I'd guess your p2pool setup is very slow or has poor network connectivity to your miner since it really should be a local miner talking to a local p2pool and that REALLY should be fast?!?

You'd think that's what it means, but I don't think it's hardware related.  My local workstation is running at 2.2ghz with 8gig memory.  Not low on resources, it doesn't thrash the swap file.  Connected via 1gb switch, then to a 1mb switch.  The "not providing work fast enough" comes after the "server requested work restart message".  If there's a delay, it's p2pool getting work from somewhere, I don't think it's my workstation.

M

[forrestv]

I would think you're better off making p2pool not roll ntime and leave it to the mining software.

I'm basically with this, but I'm not sure how to achieve it. There seem to be no consensus on what expire=10 actually means.

I guess (and it is just a guess after looking at the code) the reason p2pool does it this way is because two different miners must be given unique work, and the way it does it - since the merkle root is the same - is by giving 10 second timstamp intervals to work on, assuming the expire=10 makes the miner ask for more instead of rolling past 10 seconds. This assumption is false though, which is the problem.

Now with miners rolling on their own, the load on the server would be smaller but since each miner will be working on the exact same work, the clashes will return in a different form, if you have more than one miner connected. Not a good solution...

I think the best solution is increasing the increment that P2Pool does from 12 to something huge (100?) is best. Roll-Ntime helps a lot because it lets the mining software generate enough work to feed all the GPUs with the single long poll response that P2Pool sends. P2Pool rolling the timestamp itself also helps a lot because it lets P2Pool not regenerate the merkle root which (at least now) is relatively slow, so P2Pool can broadcast the same work to every miner daemon that you have without having to regenerate it, which especially helps latency on long polls.

I doubt that any miner is ever going to roll the timestamp more than 100 seconds (this would only happen if they have more than 100 GPUs, I think?), so this preserves the benefits of rolling the timestamp in both ways. Adding a sanity check that warns if miners are rolling >90 seconds would let us know if there's anything wrong with this.

[tucenaber]

I think the best solution is increasing the increment that P2Pool does from 12 to something huge (100?) is best. Roll-Ntime helps a lot because it lets the mining software generate enough work to feed all the GPUs with the single long poll response that P2Pool sends. P2Pool rolling the timestamp itself also helps a lot because it lets P2Pool not regenerate the merkle root which (at least now) is relatively slow, so P2Pool can broadcast the same work to every miner daemon that you have without having to regenerate it, which especially helps latency on long polls.

I doubt that any miner is ever going to roll the timestamp more than 100 seconds (this would only happen if they have more than 100 GPUs, I think?), so this preserves the benefits of rolling the timestamp in both ways. Adding a sanity check that warns if miners are rolling >90 seconds would let us know if there's anything wrong with this.

That sounds like a hack to me   How about miner specific addresses? Would that solve the problem, and if so why is it a bad idea?
Wouldn't you want to distort the block timestamp as little as possible?

[forrestv]

I think the best solution is increasing the increment that P2Pool does from 12 to something huge (100?) is best. Roll-Ntime helps a lot because it lets the mining software generate enough work to feed all the GPUs with the single long poll response that P2Pool sends. P2Pool rolling the timestamp itself also helps a lot because it lets P2Pool not regenerate the merkle root which (at least now) is relatively slow, so P2Pool can broadcast the same work to every miner daemon that you have without having to regenerate it, which especially helps latency on long polls.

I doubt that any miner is ever going to roll the timestamp more than 100 seconds (this would only happen if they have more than 100 GPUs, I think?), so this preserves the benefits of rolling the timestamp in both ways. Adding a sanity check that warns if miners are rolling >90 seconds would let us know if there's anything wrong with this.

That sounds like a hack to me   How about miner specific addresses? Would that solve the problem, and if so why is it a bad idea?
Wouldn't you want to distort the block timestamp as little as possible?

Having different addresses forces the generation transaction and merkle root to be regenerated, with all the latency that that creates. It's equivalent to simply removing P2Pool's timestamp rolling and leaving X-Roll-Ntime in.

The block timestamps aren't very important - as long as they're centered around the correct time and within the bounds of Bitcoin's protocol rules, you can change them as much as you like. They're only really used by Bitcoin to recalculate the difficulty every 2016 blocks, and small changes won't affect that.

[mdude77]

I think the best solution is increasing the increment that P2Pool does from 12 to something huge (100?) is best. Roll-Ntime helps a lot because it lets the mining software generate enough work to feed all the GPUs with the single long poll response that P2Pool sends. P2Pool rolling the timestamp itself also helps a lot because it lets P2Pool not regenerate the merkle root which (at least now) is relatively slow, so P2Pool can broadcast the same work to every miner daemon that you have without having to regenerate it, which especially helps latency on long polls.

I doubt that any miner is ever going to roll the timestamp more than 100 seconds (this would only happen if they have more than 100 GPUs, I think?), so this preserves the benefits of rolling the timestamp in both ways. Adding a sanity check that warns if miners are rolling >90 seconds would let us know if there's anything wrong with this.

I changed mine to 100.  It significantly decreased the amount of dupe messages, but I still get them here and there.

M