Bitcoin Forum
December 09, 2016, 05:42:49 PM *
News: To be able to use the next phase of the beta forum software, please ensure that your email address is correct/functional.
 
   Home   Help Search Donate Login Register  
Pages: « 1 ... 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 [148] 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 ... 744 »
  Print  
Author Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool  (Read 2033445 times)
tucenaber
Sr. Member
****
Offline Offline

Activity: 336


View Profile
July 11, 2012, 12:45:07 PM
 #2941

Since p2pool generates its own work locally, I see no point even asking the mining software to roll the time if it's going to roll it itself.
So if I hack the p2pool code, and remove the X-Roll-NTime header altogether as a short term fix, that would make cgminer stop rolling and my problem would be gone? Super easy  Grin I will try it right away.

Thank you!
What is the difference between using long-polling, and not using long-polling, and since it is a 10-second interval anyways, is there any real need for long-polling to begin with?

(requesting a little enlightenment about it)

-- Smoov

Long polling means that the pool is waiting with giving a reply until it actually has new work, and that is good even for short period lengths too, I guess. But this issue isn't about long-polling, I think, but whether the server or the client should increment the timestamp when the client runs out of nonces.

Anyhoooo...

Removing the header S-O-L-V-E-D it!  Man, I'm sooo happy! Cheesy
1481305369
Hero Member
*
Offline Offline

Posts: 1481305369

View Profile Personal Message (Offline)

Ignore
1481305369
Reply with quote  #2

1481305369
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1481305369
Hero Member
*
Offline Offline

Posts: 1481305369

View Profile Personal Message (Offline)

Ignore
1481305369
Reply with quote  #2

1481305369
Report to moderator
1481305369
Hero Member
*
Offline Offline

Posts: 1481305369

View Profile Personal Message (Offline)

Ignore
1481305369
Reply with quote  #2

1481305369
Report to moderator
kano
Legendary
*
Offline Offline

Activity: 1932


Linux since 1997 RedHat 4


View Profile
July 11, 2012, 01:46:57 PM
 #2942

Since p2pool generates its own work locally, I see no point even asking the mining software to roll the time if it's going to roll it itself.
So if I hack the p2pool code, and remove the X-Roll-NTime header altogether as a short term fix, that would make cgminer stop rolling and my problem would be gone? Super easy  Grin I will try it right away.

Thank you!
What is the difference between using long-polling, and not using long-polling, and since it is a 10-second interval anyways, is there any real need for long-polling to begin with?

(requesting a little enlightenment about it)

-- Smoov

The LP is not just a notification, it also provides new work that is valid.
So the miner will get the new work immediately rather than having the extra delay of requesting it.

Pool: https://kano.is BTC: 1KanoiBupPiZfkwqB7rfLXAzPnoTshAVmb
CKPool and CGMiner developer, IRC FreeNode #ckpool and #cgminer kanoi
Help keep Bitcoin secure by mining on pools with Stratum, the best protocol to mine Bitcoins with ASIC hardware
coinnewb
Full Member
***
Offline Offline

Activity: 155



View Profile
July 11, 2012, 10:15:37 PM
 #2943

Since p2pool generates its own work locally, I see no point even asking the mining software to roll the time if it's going to roll it itself.
So if I hack the p2pool code, and remove the X-Roll-NTime header altogether as a short term fix, that would make cgminer stop rolling and my problem would be gone? Super easy  Grin I will try it right away.

Thank you!
What is the difference between using long-polling, and not using long-polling, and since it is a 10-second interval anyways, is there any real need for long-polling to begin with?

(requesting a little enlightenment about it)

-- Smoov

Long polling means that the pool is waiting with giving a reply until it actually has new work, and that is good even for short period lengths too, I guess. But this issue isn't about long-polling, I think, but whether the server or the client should increment the timestamp when the client runs out of nonces.

Anyhoooo...

Removing the header S-O-L-V-E-D it!  Man, I'm sooo happy! Cheesy

Do you mind sharing the details of changes?  I see the same issue with my setup from time to time.
Thanks.
tucenaber
Sr. Member
****
Offline Offline

Activity: 336


View Profile
July 11, 2012, 11:25:58 PM
 #2944

Do you mind sharing the details of changes?  I see the same issue with my setup from time to time.
Thanks.
In the p2pool directory, look for the file p2pool/bitcoin/worker_interface.py. Open it in an editor and remove the line containing "X-Roll-NTime" and remove it.
Code:
     @defer.inlineCallbacks
     def _getwork(self, request, data, long_poll):
         request.setHeader('X-Long-Polling', '/long-polling')
-        request.setHeader('X-Roll-NTime', 'expire=10')
         request.setHeader('X-Is-P2Pool', 'true')

done!
rav3n_pl
Legendary
*
Offline Offline

Activity: 1320


Don`t panic! Organize!


View Profile
July 11, 2012, 11:55:31 PM
 #2945

PM forrestv about that!

1Rav3nkMayCijuhzcYemMiPYsvcaiwHni  Bitcoin stuff on my OneDrive
My RPC CoinControl for any coin https://bitcointalk.org/index.php?topic=929954
My SatoshDice bot https://bitcointalk.org/index.php?topic=897685
tucenaber
Sr. Member
****
Offline Offline

Activity: 336


View Profile
July 12, 2012, 12:23:58 AM
 #2946

done
mdude77
Legendary
*
Offline Offline

Activity: 1358


View Profile
July 12, 2012, 12:50:38 AM
 #2947

Anyhoooo...

Removing the header S-O-L-V-E-D it!  Man, I'm sooo happy! Cheesy

I made your change to my side, installed python 2.7.3 64-bit (windows) and all the associated apps needed for p2pool and fired it up.

The dupe message is definitely gone.

Now, however, cgminer on my main miner (4x7870 = 2.6g/h) is complaining non stop about p2pool not providing work fast enough.  I think if my phoenix miner (4x5870 = 1.6g/h) had a better UI, it would be complaining too.

I'm not sure this is an improvement. Sad

M

MMinerMonitor author, monitor/auto/schedule reboots/alerts/remote/MobileMiner for Ants and Spondoolies! Latest (5.2). MPoolMonitor author, monitor stats/workers for most pools, global BTC stats (current/nxt diff/USD val/hashrate/calc)! Latest (v4.2) 
Buyer beware of Bitmain hardware and services.
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2002


Ruu \o/


View Profile WWW
July 12, 2012, 02:45:42 AM
 #2948

I would think you're better off making p2pool not roll ntime and leave it to the mining software.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
Pooled mine at kano.is, solo mine at solo.ckpool.org
-ck
kano
Legendary
*
Offline Offline

Activity: 1932


Linux since 1997 RedHat 4


View Profile
July 12, 2012, 03:28:33 AM
 #2949

Anyhoooo...

Removing the header S-O-L-V-E-D it!  Man, I'm sooo happy! Cheesy

I made your change to my side, installed python 2.7.3 64-bit (windows) and all the associated apps needed for p2pool and fired it up.

The dupe message is definitely gone.

Now, however, cgminer on my main miner (4x7870 = 2.6g/h) is complaining non stop about p2pool not providing work fast enough.  I think if my phoenix miner (4x5870 = 1.6g/h) had a better UI, it would be complaining too.

I'm not sure this is an improvement. Sad

M
However, what this is saying is that your p2pool can't handle a local miner without roll-n-time
So I'd guess your p2pool setup is very slow or has poor network connectivity to your miner since it really should be a local miner talking to a local p2pool and that REALLY should be fast?!?

Pool: https://kano.is BTC: 1KanoiBupPiZfkwqB7rfLXAzPnoTshAVmb
CKPool and CGMiner developer, IRC FreeNode #ckpool and #cgminer kanoi
Help keep Bitcoin secure by mining on pools with Stratum, the best protocol to mine Bitcoins with ASIC hardware
rav3n_pl
Legendary
*
Offline Offline

Activity: 1320


Don`t panic! Organize!


View Profile
July 12, 2012, 05:52:06 AM
 #2950

Maybe make this time up to 30 or 60 sec instead of removing line?

1Rav3nkMayCijuhzcYemMiPYsvcaiwHni  Bitcoin stuff on my OneDrive
My RPC CoinControl for any coin https://bitcointalk.org/index.php?topic=929954
My SatoshDice bot https://bitcointalk.org/index.php?topic=897685
tucenaber
Sr. Member
****
Offline Offline

Activity: 336


View Profile
July 12, 2012, 08:04:59 AM
 #2951

I would think you're better off making p2pool not roll ntime and leave it to the mining software.
I'm basically with this, but I'm not sure how to achieve it. There seem to be no consensus on what expire=10 actually means.

I guess (and it is just a guess after looking at the code) the reason p2pool does it this way is because two different miners must be given unique work, and the way it does it - since the merkle root is the same - is by giving 10 second timstamp intervals to work on, assuming the expire=10 makes the miner ask for more instead of rolling past 10 seconds. This assumption is false though, which is the problem.

Now with miners rolling on their own, the load on the server would be smaller but since each miner will be working on the exact same work, the clashes will return in a different form, if you have more than one miner connected. Not a good solution...

One simple way to avoid the latter is to have unique payment addresses for each miner. That would make the merkle roots diffferent, right? Then we could do it the ckolivas way and everybody lives happily ever after Smiley

Forrest can give a better explanation of this, tho.
mdude77
Legendary
*
Offline Offline

Activity: 1358


View Profile
July 12, 2012, 10:51:43 AM
 #2952

Anyhoooo...

Removing the header S-O-L-V-E-D it!  Man, I'm sooo happy! Cheesy

I made your change to my side, installed python 2.7.3 64-bit (windows) and all the associated apps needed for p2pool and fired it up.

The dupe message is definitely gone.

Now, however, cgminer on my main miner (4x7870 = 2.6g/h) is complaining non stop about p2pool not providing work fast enough.  I think if my phoenix miner (4x5870 = 1.6g/h) had a better UI, it would be complaining too.

I'm not sure this is an improvement. Sad

M
However, what this is saying is that your p2pool can't handle a local miner without roll-n-time
So I'd guess your p2pool setup is very slow or has poor network connectivity to your miner since it really should be a local miner talking to a local p2pool and that REALLY should be fast?!?

You'd think that's what it means, but I don't think it's hardware related.  My local workstation is running at 2.2ghz with 8gig memory.  Not low on resources, it doesn't thrash the swap file.  Connected via 1gb switch, then to a 1mb switch.  The "not providing work fast enough" comes after the "server requested work restart message".  If there's a delay, it's p2pool getting work from somewhere, I don't think it's my workstation.

M

MMinerMonitor author, monitor/auto/schedule reboots/alerts/remote/MobileMiner for Ants and Spondoolies! Latest (5.2). MPoolMonitor author, monitor stats/workers for most pools, global BTC stats (current/nxt diff/USD val/hashrate/calc)! Latest (v4.2) 
Buyer beware of Bitmain hardware and services.
forrestv
Hero Member
*****
Offline Offline

Activity: 510


View Profile
July 12, 2012, 03:03:48 PM
 #2953

I would think you're better off making p2pool not roll ntime and leave it to the mining software.
I'm basically with this, but I'm not sure how to achieve it. There seem to be no consensus on what expire=10 actually means.

I guess (and it is just a guess after looking at the code) the reason p2pool does it this way is because two different miners must be given unique work, and the way it does it - since the merkle root is the same - is by giving 10 second timstamp intervals to work on, assuming the expire=10 makes the miner ask for more instead of rolling past 10 seconds. This assumption is false though, which is the problem.

Now with miners rolling on their own, the load on the server would be smaller but since each miner will be working on the exact same work, the clashes will return in a different form, if you have more than one miner connected. Not a good solution...

I think the best solution is increasing the increment that P2Pool does from 12 to something huge (100?) is best. Roll-Ntime helps a lot because it lets the mining software generate enough work to feed all the GPUs with the single long poll response that P2Pool sends. P2Pool rolling the timestamp itself also helps a lot because it lets P2Pool not regenerate the merkle root which (at least now) is relatively slow, so P2Pool can broadcast the same work to every miner daemon that you have without having to regenerate it, which especially helps latency on long polls.

I doubt that any miner is ever going to roll the timestamp more than 100 seconds (this would only happen if they have more than 100 GPUs, I think?), so this preserves the benefits of rolling the timestamp in both ways. Adding a sanity check that warns if miners are rolling >90 seconds would let us know if there's anything wrong with this.

1J1zegkNSbwX4smvTdoHSanUfwvXFeuV23
tucenaber
Sr. Member
****
Offline Offline

Activity: 336


View Profile
July 12, 2012, 04:06:39 PM
 #2954

I think the best solution is increasing the increment that P2Pool does from 12 to something huge (100?) is best. Roll-Ntime helps a lot because it lets the mining software generate enough work to feed all the GPUs with the single long poll response that P2Pool sends. P2Pool rolling the timestamp itself also helps a lot because it lets P2Pool not regenerate the merkle root which (at least now) is relatively slow, so P2Pool can broadcast the same work to every miner daemon that you have without having to regenerate it, which especially helps latency on long polls.

I doubt that any miner is ever going to roll the timestamp more than 100 seconds (this would only happen if they have more than 100 GPUs, I think?), so this preserves the benefits of rolling the timestamp in both ways. Adding a sanity check that warns if miners are rolling >90 seconds would let us know if there's anything wrong with this.

That sounds like a hack to me Wink  How about miner specific addresses? Would that solve the problem, and if so why is it a bad idea?
Wouldn't you want to distort the block timestamp as little as possible?
forrestv
Hero Member
*****
Offline Offline

Activity: 510


View Profile
July 12, 2012, 05:21:12 PM
 #2955

I think the best solution is increasing the increment that P2Pool does from 12 to something huge (100?) is best. Roll-Ntime helps a lot because it lets the mining software generate enough work to feed all the GPUs with the single long poll response that P2Pool sends. P2Pool rolling the timestamp itself also helps a lot because it lets P2Pool not regenerate the merkle root which (at least now) is relatively slow, so P2Pool can broadcast the same work to every miner daemon that you have without having to regenerate it, which especially helps latency on long polls.

I doubt that any miner is ever going to roll the timestamp more than 100 seconds (this would only happen if they have more than 100 GPUs, I think?), so this preserves the benefits of rolling the timestamp in both ways. Adding a sanity check that warns if miners are rolling >90 seconds would let us know if there's anything wrong with this.

That sounds like a hack to me Wink  How about miner specific addresses? Would that solve the problem, and if so why is it a bad idea?
Wouldn't you want to distort the block timestamp as little as possible?

Having different addresses forces the generation transaction and merkle root to be regenerated, with all the latency that that creates. It's equivalent to simply removing P2Pool's timestamp rolling and leaving X-Roll-Ntime in.

The block timestamps aren't very important - as long as they're centered around the correct time and within the bounds of Bitcoin's protocol rules, you can change them as much as you like. They're only really used by Bitcoin to recalculate the difficulty every 2016 blocks, and small changes won't affect that.

1J1zegkNSbwX4smvTdoHSanUfwvXFeuV23
mdude77
Legendary
*
Offline Offline

Activity: 1358


View Profile
July 12, 2012, 11:58:51 PM
 #2956

I think the best solution is increasing the increment that P2Pool does from 12 to something huge (100?) is best. Roll-Ntime helps a lot because it lets the mining software generate enough work to feed all the GPUs with the single long poll response that P2Pool sends. P2Pool rolling the timestamp itself also helps a lot because it lets P2Pool not regenerate the merkle root which (at least now) is relatively slow, so P2Pool can broadcast the same work to every miner daemon that you have without having to regenerate it, which especially helps latency on long polls.

I doubt that any miner is ever going to roll the timestamp more than 100 seconds (this would only happen if they have more than 100 GPUs, I think?), so this preserves the benefits of rolling the timestamp in both ways. Adding a sanity check that warns if miners are rolling >90 seconds would let us know if there's anything wrong with this.

I changed mine to 100.  It significantly decreased the amount of dupe messages, but I still get them here and there.

M

MMinerMonitor author, monitor/auto/schedule reboots/alerts/remote/MobileMiner for Ants and Spondoolies! Latest (5.2). MPoolMonitor author, monitor stats/workers for most pools, global BTC stats (current/nxt diff/USD val/hashrate/calc)! Latest (v4.2) 
Buyer beware of Bitmain hardware and services.
coinnewb
Full Member
***
Offline Offline

Activity: 155



View Profile
July 13, 2012, 02:43:21 AM
 #2957

Thanks for sharing the details, tucenaber!

I also changed mine to 100 but still see the warning message from time to time.  Any idea on how to proceed?
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2002


Ruu \o/


View Profile WWW
July 13, 2012, 11:25:14 AM
 #2958

I don't understand. If p2pool can roll the time locally fast enough, there should be no need to tell the mining software to roll the time at all. Alternatively, if you let the mining software roll the time, why should p2pool roll time at all? There's some kind of catch 22 thing going on here, and yet it should be very low overhead to roll the time. Something is missing in this equation. The mining software should report if it supports nrolltime in X-Mining-Extensions: "longpoll midstate rollntime submitold". If it supports rollntime then p2pool should not be rolling the time itself and allow the miner to do it by advertising X-Roll-Ntime (potentially with some expire= but just "Y" should suffice here). If it does not support rollntime, then p2pool should roll the time itself and not advertise X-Roll-Ntime.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
Pooled mine at kano.is, solo mine at solo.ckpool.org
-ck
rav3n_pl
Legendary
*
Offline Offline

Activity: 1320


Don`t panic! Organize!


View Profile
July 13, 2012, 01:21:31 PM
 #2959

I don't understand. If p2pool can roll the time locally fast enough, there should be no need to tell the mining software to roll the time at all. Alternatively, if you let the mining software roll the time, why should p2pool roll time at all? There's some kind of catch 22 thing going on here, and yet it should be very low overhead to roll the time. Something is missing in this equation. The mining software should report if it supports nrolltime in X-Mining-Extensions: "longpoll midstate rollntime submitold". If it supports rollntime then p2pool should not be rolling the time itself and allow the miner to do it by advertising X-Roll-Ntime (potentially with some expire= but just "Y" should suffice here). If it does not support rollntime, then p2pool should roll the time itself and not advertise X-Roll-Ntime.
Looks like killing rolltime line is best option if using cgminer then Smiley

1Rav3nkMayCijuhzcYemMiPYsvcaiwHni  Bitcoin stuff on my OneDrive
My RPC CoinControl for any coin https://bitcointalk.org/index.php?topic=929954
My SatoshDice bot https://bitcointalk.org/index.php?topic=897685
Schleicher
Hero Member
*****
Offline Offline

Activity: 630



View Profile
July 13, 2012, 03:43:16 PM
 #2960

Looks like these are the options we have:
1. recalculate the merkle root each time p2pool is going to send work to the miner
  • this will increase CPU load
  • the miner can roll the time
2. p2pool rolls the time, don't let the mining client do it
  • this will result in more getwork requests

Bitcoin donations: 1H2BHSyuwLP9vqt2p3bK9G3mDJsAi7qChw
Pages: « 1 ... 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 [148] 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 ... 744 »
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!