[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool

[IYFTech]

How's it working out - did they do it yet?

Nothing yet......

It doesn't need to be faster.

Faster would just be a bonus by-product of being in C/++, it certainly won't do any harm. Having more coders able to play with it is the main advantage though.....

[1714280128]

1714280128

[1714280128]

1714280128

[1714280128]

1714280128

[1714280128]

1714280128

[1714280128]

1714280128

[mdude77]

It doesn't need to be faster.

Faster would just be a bonus by-product of being in C/++, it certainly won't do any harm. Having more coders able to play with it is the main advantage though.....

I don't think it'd be worth the trouble.  If you understand the principle behind how p2pool works, and its short comings, the best bet is to start from scratch with a new design that addresses the flaws in p2pool.

M

[IYFTech]

It doesn't need to be faster.

Faster would just be a bonus by-product of being in C/++, it certainly won't do any harm. Having more coders able to play with it is the main advantage though.....

I don't think it'd be worth the trouble.  If you understand the principle behind how p2pool works, and its short comings, the best bet is to start from scratch with a new design that addresses the flaws in p2pool.

M

Exactly. Been saying this for over a year.......

[IYFTech]

Well, if you believe the BitmainWarranty account, and can stomach the screenshot where they "tested" for only a minute, then the S4 works with p2pool: https://bitcointalk.org/index.php?topic=796839.msg9075999#msg9075999.

This. It's hardly an example of how it performs with p2pool is it.....& who is this user BitmainWarranty? I'll wait until I see some solid, hard evidence before making my mind up I think, although I've seen enough complaints about the S4's to pretty much come to a conclusion already.....

Bitmain are going to point an S4 at my node for 10 minutes shortly apparently, so we'll see.......

Well, initial results are in - and it ain't good:



Looks like they decided to switch it off pretty sharpish when they saw the reject/DOA rate too.....my DOA rate instantly climbed from ~2% to over 18% even before the hash rate reached 2Th, if it was ever going to get to 2Th - which I doubt. I wish they'd let it run for the 10 minutes they said they would, maybe they're changing some settings......but it looks like the S4 is really a badly disguised S2  

Oh dear.

[IYFTech]

Update: 2nd test.

Looks like Bitmain are tinkering with their settings:



Slightly higher hash rate, much higher DOA/Reject - over 30%....

[truten]

Help pull start, When you start kogad web page with statistics is not active is a log.

Code:

2014-10-06 02:46:57.773788  Local: 0H/s in last 0.0 seconds Local dead on arrival: ??? Expected time to share: ???

And when I do update statistics node, then fly these errors

Code:

 
2014-10-06 02:46:53.330462 > Error in DeferredResource handler:
2014-10-06 02:46:53.330741 > Traceback (most recent call last):
2014-10-06 02:46:53.330858 >   File "/home/ubuntu/p2pool-doge1-8/p2pool/util/deferred_resource.py", line 24, in render
2014-10-06 02:46:53.330955 >     defer.maybeDeferred(resource.Resource.render, self, request).addCallbacks(finish, finish_error)
2014-10-06 02:46:53.331061 >   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 139, in maybeDeferred
2014-10-06 02:46:53.331164 >     result = f(*args, **kw)
2014-10-06 02:46:53.331243 >   File "/usr/lib/python2.7/dist-packages/twisted/web/resource.py", line 250, in render
2014-10-06 02:46:53.331323 >     return m(request)
2014-10-06 02:46:53.331399 >   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1237, in unwindGenerator
2014-10-06 02:46:53.331479 >     return _inlineCallbacks(None, gen, Deferred())
2014-10-06 02:46:53.331554 > --- <exception caught here> ---
2014-10-06 02:46:53.331629 >   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1099, in _inlineCallbacks
2014-10-06 02:46:53.331717 >     result = g.send(result)
2014-10-06 02:46:53.331822 >   File "/home/ubuntu/p2pool-doge1-8/p2pool/web.py", line 198, in render_GET
2014-10-06 02:46:53.331901 >     res = yield self.func(*self.args)
2014-10-06 02:46:53.332012 >   File "/home/ubuntu/p2pool-doge1-8/p2pool/web.py", line 203, in <lambda>
2014-10-06 02:46:53.332106 >     web_root.putChild('rate', WebInterface(lambda: p2pool_data.get_pool_attempts_per_second(node.tracker, node.best_share_var.value, decent_height())/(1-p2pool_data.get_average_stale_prop(node.tracker, node.best_share_var.value, decent_height()))))
2014-10-06 02:46:53.332204 >   File "/home/ubuntu/p2pool-doge1-8/p2pool/data.py", line 573, in get_pool_attempts_per_second
2014-10-06 02:46:53.332310 >     assert dist >= 2
2014-10-06 02:46:53.332406 > exceptions.AssertionError:

Maybe that is not installed or not run? Help me.

[zvs]

Help pull start, When you start kogad web page with statistics is not active is a log.

Code:

2014-10-06 02:46:57.773788  Local: 0H/s in last 0.0 seconds Local dead on arrival: ??? Expected time to share: ???

And when I do update statistics node, then fly these errors

Code:

 
2014-10-06 02:46:53.330462 > Error in DeferredResource handler:
2014-10-06 02:46:53.330741 > Traceback (most recent call last):
2014-10-06 02:46:53.330858 >   File "/home/ubuntu/p2pool-doge1-8/p2pool/util/deferred_resource.py", line 24, in render
2014-10-06 02:46:53.330955 >     defer.maybeDeferred(resource.Resource.render, self, request).addCallbacks(finish, finish_error)
2014-10-06 02:46:53.331061 >   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 139, in maybeDeferred
2014-10-06 02:46:53.331164 >     result = f(*args, **kw)
2014-10-06 02:46:53.331243 >   File "/usr/lib/python2.7/dist-packages/twisted/web/resource.py", line 250, in render
2014-10-06 02:46:53.331323 >     return m(request)
2014-10-06 02:46:53.331399 >   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1237, in unwindGenerator
2014-10-06 02:46:53.331479 >     return _inlineCallbacks(None, gen, Deferred())
2014-10-06 02:46:53.331554 > --- <exception caught here> ---
2014-10-06 02:46:53.331629 >   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1099, in _inlineCallbacks
2014-10-06 02:46:53.331717 >     result = g.send(result)
2014-10-06 02:46:53.331822 >   File "/home/ubuntu/p2pool-doge1-8/p2pool/web.py", line 198, in render_GET
2014-10-06 02:46:53.331901 >     res = yield self.func(*self.args)
2014-10-06 02:46:53.332012 >   File "/home/ubuntu/p2pool-doge1-8/p2pool/web.py", line 203, in <lambda>
2014-10-06 02:46:53.332106 >     web_root.putChild('rate', WebInterface(lambda: p2pool_data.get_pool_attempts_per_second(node.tracker, node.best_share_var.value, decent_height())/(1-p2pool_data.get_average_stale_prop(node.tracker, node.best_share_var.value, decent_height()))))
2014-10-06 02:46:53.332204 >   File "/home/ubuntu/p2pool-doge1-8/p2pool/data.py", line 573, in get_pool_attempts_per_second
2014-10-06 02:46:53.332310 >     assert dist >= 2
2014-10-06 02:46:53.332406 > exceptions.AssertionError:

Maybe that is not installed or not run? Help me.

Mess with some of the difficulty/share settings?  I guess that's why it isn't working.

I don't care to fix stratum, since it's about 10% slower than long polling.    (ed: well, I wasted a little bit of time on it, until I decided it was better to just remove the logging spam from other ppl)

well and final ed, correction:  it's inefficient for end user, unless you care about using up a little more bandwidth, and having more connections open

[truten]

Mess with some of the difficulty/share settings?  I guess that's why it isn't working.

I don't care to fix stratum, since it's about 10% slower than long polling.    (ed: well, I wasted a little bit of time on it, until I decided it was better to just remove the logging spam from other ppl)

well and final ed, correction:  it's inefficient for end user, unless you care about using up a little more bandwidth, and having more connections open

I want to start so early.
And here's another one bug found. When connected to a pool of miners, so we get an error

Code:

2014-10-06 06:05:05.537252 > Unhandled Error
2014-10-06 06:05:05.537497 > Traceback (most recent call last):
2014-10-06 06:05:05.537644 > File "/home/ubuntu/p2pool-doge1-8/p2pool/main.py", line 595, in run
2014-10-06 06:05:05.537747 > reactor.run()
2014-10-06 06:05:05.537845 > File "/usr/lib/python2.7/dist-packages/twisted/internet/base.py", line 1192, in run
2014-10-06 06:05:05.537946 > self.mainLoop()
2014-10-06 06:05:05.538049 > File "/usr/lib/python2.7/dist-packages/twisted/internet/base.py", line 1201, in mainLoop
2014-10-06 06:05:05.538126 > self.runUntilCurrent()
2014-10-06 06:05:05.538196 > File "/usr/lib/python2.7/dist-packages/twisted/internet/base.py", line 824, in runUntilCurrent
2014-10-06 06:05:05.538268 > call.func(*call.args, **call.kw)
2014-10-06 06:05:05.538337 > --- ---
2014-10-06 06:05:05.538424 > File "/home/ubuntu/p2pool-doge1-8/p2pool/bitcoin/stratum.py", line 38, in _send_work
2014-10-06 06:05:05.538509 > x, got_response = self.wb.get_work(*self.wb.preprocess_request('' if self.username is None else self.username))
2014-10-06 06:05:05.538584 > File "/home/ubuntu/p2pool-doge1-8/p2pool/work.py", line 177, in preprocess_request
2014-10-06 06:05:05.538655 > raise jsonrpc.Error_for_code(-12345)(u'p2pool is not connected to any peers')
2014-10-06 06:05:05.538726 > p2pool.util.jsonrpc.NarrowError: -12345 p2pool is not connected to any peers

Miner gives this error

http://c2n.me/j2aGiI.png

[zvs]

I think for minerd, you just use --no-stratum, and it's better.  (for p2pool at least)

[truten]

I think for minerd, you just use --no-stratum, and it's better.  (for p2pool at least)

If you do do something it's OK, you do not know because of what is a problem?

[zvs]

I think for minerd, you just use --no-stratum, and it's better.  (for p2pool at least)

If you do do something it's OK, you do not know because of what is a problem?

Code:

[2014-10-06 02:21:50] 1 miner threads started, using 'scrypt' algorithm.
[2014-10-06 02:21:50] JSON-RPC call failed: {
   "data": null,
   "code": -32601,
   "message": "Method not found"
}
[2014-10-06 02:21:50] getblocktemplate failed, falling back to getwork
[2014-10-06 02:21:50] Starting Stratum on stratum+tcp://127.0.0.1:9327
[2014-10-06 02:21:50] Stratum requested work restart
[2014-10-06 02:21:50] Stratum requested work restart
[2014-10-06 02:21:51] thread 0: 1344 hashes, 12.26 khash/s
[2014-10-06 02:21:51] accepted: 1/1 (100.00%), 12.26 khash/s (yay!!!)
[2014-10-06 02:21:51] thread 0: 1704 hashes, 12.23 khash/s
[2014-10-06 02:21:51] accepted: 2/2 (100.00%), 12.23 khash/s (yay!!!)
[2014-10-06 02:21:51] thread 0: 72 hashes, 12.13 khash/s
[2014-10-06 02:21:51] accepted: 3/3 (100.00%), 12.13 khash/s (yay!!!)
[2014-10-06 02:21:51] thread 0: 24 hashes, 11.49 khash/s
[2014-10-06 02:21:51] accepted: 4/4 (100.00%), 11.49 khash/s (yay!!!)
[2014-10-06 02:21:51] thread 0: 1344 hashes, 12.31 khash/s
[2014-10-06 02:21:51] accepted: 5/5 (100.00%), 12.31 khash/s (yay!!!)
[2014-10-06 02:21:51] thread 0: 84 hashes, 12.00 khash/s
[2014-10-06 02:21:51] accepted: 6/6 (100.00%), 12.00 khash/s (yay!!!)
[2014-10-06 02:21:51] thread 0: 828 hashes, 12.23 khash/s
[2014-10-06 02:21:51] accepted: 7/7 (100.00%), 12.23 khash/s (yay!!!)
[2014-10-06 02:21:51] thread 0: 696 hashes, 12.18 khash/s
[2014-10-06 02:21:51] accepted: 8/8 (100.00%), 12.18 khash/s (yay!!!)
[2014-10-06 02:21:51] thread 0: 516 hashes, 12.18 khash/s
[2014-10-06 02:21:51] accepted: 9/9 (100.00%), 12.18 khash/s (yay!!!)
[2014-10-06 02:21:51] thread 0: 360 hashes, 12.24 khash/s
[2014-10-06 02:21:51] accepted: 10/10 (100.00%), 12.24 khash/s (yay!!!)
[2014-10-06 02:21:51] thread 0: 336 hashes, 12.30 khash/s
[2014-10-06 02:21:51] accepted: 11/11 (100.00%), 12.30 khash/s (yay!!!)
[2014-10-06 02:21:52] thread 0: 828 hashes, 12.32 khash/s
[2014-10-06 02:21:52] accepted: 12/12 (100.00%), 12.32 khash/s (yay!!!)
[2014-10-06 02:21:52] thread 0: 576 hashes, 12.34 khash/s
[2014-10-06 02:21:52] accepted: 13/13 (100.00%), 12.34 khash/s (yay!!!)
[2014-10-06 02:21:52] thread 0: 144 hashes, 12.20 khash/s
[2014-10-06 02:21:52] accepted: 14/14 (100.00%), 12.20 khash/s (yay!!!)
[2014-10-06 02:21:52] thread 0: 660 hashes, 12.27 khash/s
[2014-10-06 02:21:52] accepted: 15/15 (100.00%), 12.27 khash/s (yay!!!)

I never tried minerd before, so that's what it should look like with stratum.  So, I guess in your case, maybe it's a firewall?  the +0.0000000 seems out of place also, i'd think even with a cpu something like 0.000001 would be reasonable

[truten]

I never tried minerd before, so that's what it should look like with stratum.  So, I guess in your case, maybe it's a firewall?  the +0.0000000 seems out of place also, i'd think even with a cpu something like 0.000001 would be reasonable

I think he is not, because the system is Linux (Ubuntu) started in the log to meet the compound which is

Code:

2014-10-06 07:54:59.374809 P2Pool: 0 shares in chain (0 verified/5 total) Peers: 2 (0 incoming)
2014-10-06 07:54:59.375279  Local: 0H/s in last 0.0 seconds Local dead on arrival: ??? Expected time to share: ???
2014-10-06 07:55:02.563667 Peer sent entire transaction 1edf48c6eb2aaad76af659a2e6c2ae96eec7ad8f487f67ed17af40b371e02c25 that was already received

But there are no statistics

[naplam]

Who runs http://p2pool-nodes.info ? how can I tell them to list my public pool?

[IYFTech]

Who runs http://p2pool-nodes.info ? how can I tell them to list my public pool?

It will find your node automatically - might take a few days to propagate 

[windpath]

I've set up a private pool solution for a reasonably large miner using a combination of p2pool and ckpool technology. You should all see a decent increase in the overall pool size over the next 24-48 hours.

This hasher is now online. His hashrate should be obvious, right at the top of the list. Barring changes in plans, and provided the hardware continues to hash well, it should be remaining on this pool.

Now the interesting thing with this is, because I have connected the hardware via ckproxy instead of as 100 connections directly to the p2pool client, p2pool sees it as one client, which means that this miner's share target is more than 10 times larger than that for other miners. By doing this, even though I've dumped a large hashrate onto the pool, it won't substantially increase the target share rate for the smaller miners. This means smaller miners can benefit from the increased p2pool hashrate decreasing their variance without their share target increasing that much which normally increases their variance the same amount. If more larger miners did something similar on p2pool, it might keep the smaller miners. The large miner benefits from his p2pool client scaling where it otherwise couldn't and the smaller miners get to stay and benefit from his presence. While it's not a "fix" for the overall design, it might give p2pool some breathing space, allowing ever larger miners to join. That said, "small" these days is not really that small... Perhaps p2pool will actually end up being nothing but big miners (though that is what most of the network is now), provided their hardware is compatible :p

Sorry I'm late to the party, but this is great news and wanted to thank you.

While setting "the client" at a higher share diff does nothing for the overall pool diff, it certainly speeds up the pool as a whole vs submitting a bunch of sudo shares that don't meet the minimum diff requirement.

It is how p2pool was intended to be used. Your clients earnings will not change, his accepted shares will have a greater weight, but the overall pool efficiency is increased because he is submitting less shares to be processed by us all.

[norgan]

I've set up a private pool solution for a reasonably large miner using a combination of p2pool and ckpool technology. You should all see a decent increase in the overall pool size over the next 24-48 hours.

This hasher is now online. His hashrate should be obvious, right at the top of the list. Barring changes in plans, and provided the hardware continues to hash well, it should be remaining on this pool.

Now the interesting thing with this is, because I have connected the hardware via ckproxy instead of as 100 connections directly to the p2pool client, p2pool sees it as one client, which means that this miner's share target is more than 10 times larger than that for other miners. By doing this, even though I've dumped a large hashrate onto the pool, it won't substantially increase the target share rate for the smaller miners. This means smaller miners can benefit from the increased p2pool hashrate decreasing their variance without their share target increasing that much which normally increases their variance the same amount. If more larger miners did something similar on p2pool, it might keep the smaller miners. The large miner benefits from his p2pool client scaling where it otherwise couldn't and the smaller miners get to stay and benefit from his presence. While it's not a "fix" for the overall design, it might give p2pool some breathing space, allowing ever larger miners to join. That said, "small" these days is not really that small... Perhaps p2pool will actually end up being nothing but big miners (though that is what most of the network is now), provided their hardware is compatible :p

Sorry I'm late to the party, but this is great news and wanted to thank you.

While setting "the client" at a higher share diff does nothing for the overall pool diff, it certainly speeds up the pool as a whole vs submitting a bunch of sudo shares that don't meet the minimum diff requirement.

It is how p2pool was intended to be used. Your clients earnings will not change, his accepted shares will have a greater weight, but the overall pool efficiency is increased because he is submitting less shares to be processed by us all.

I missed the first part of this convo but going back I can see now. So if every miner with hash rate > 5% of total hash rate ran the ckproxy, we would see a far more efficient p2pool? that would be higher than around 150th/s right now yes?

[-ck]

It is how p2pool was intended to be used. Your clients earnings will not change, his accepted shares will have a greater weight, but the overall pool efficiency is increased because he is submitting less shares to be processed by us all.

I missed the first part of this convo but going back I can see now. So if every miner with hash rate > 5% of total hash rate ran the ckproxy, we would see a far more efficient p2pool? that would be higher than around 150th/s right now yes?

Yes that is correct, though it is awkward since you need to add yet another layer to the mining, but if you are an owner of hundreds of Terrahash and wish to mine on p2pool, I'm sure you'd be willing to go to the effort. Of course ideally this should happen within the client, or ckpool should take more of the work from the p2pool internals but this way the p2pool client is actually completely native and unmodified (for now). The farm I helped get on board has been off the pool for a little but will be back on again soon, and I anticipate to be helping yet another even larger miner do the same thing soon.

[norgan]

It is how p2pool was intended to be used. Your clients earnings will not change, his accepted shares will have a greater weight, but the overall pool efficiency is increased because he is submitting less shares to be processed by us all.

I missed the first part of this convo but going back I can see now. So if every miner with hash rate > 5% of total hash rate ran the ckproxy, we would see a far more efficient p2pool? that would be higher than around 150th/s right now yes?

Yes that is correct, though it is awkward since you need to add yet another layer to the mining, but if you are an owner of hundreds of Terrahash and wish to mine on p2pool, I'm sure you'd be willing to go to the effort. Of course ideally this should happen within the client, or ckpool should take more of the work from the p2pool internals but this way the p2pool client is actually completely native and unmodified (for now). The farm I helped get on board has been off the pool for a little but will be back on again soon, and I anticipate to be helping yet another even larger miner do the same thing soon.

This sounds pretty good to me. Could be an interesting concept to explore as a fork.

[mdude77]

It is how p2pool was intended to be used. Your clients earnings will not change, his accepted shares will have a greater weight, but the overall pool efficiency is increased because he is submitting less shares to be processed by us all.

I missed the first part of this convo but going back I can see now. So if every miner with hash rate > 5% of total hash rate ran the ckproxy, we would see a far more efficient p2pool? that would be higher than around 150th/s right now yes?

Yes that is correct, though it is awkward since you need to add yet another layer to the mining, but if you are an owner of hundreds of Terrahash and wish to mine on p2pool, I'm sure you'd be willing to go to the effort. Of course ideally this should happen within the client, or ckpool should take more of the work from the p2pool internals but this way the p2pool client is actually completely native and unmodified (for now). The farm I helped get on board has been off the pool for a little but will be back on again soon, and I anticipate to be helping yet another even larger miner do the same thing soon.

How does this differ from pointing all the workers to the same workername on the same node, all with high difficulty?

M

[-ck]

How does this differ from pointing all the workers to the same workername on the same node, all with high difficulty?

The p2pool client software has to generate multiple workers of multiple different data sets and track them all, and then process the shares as they come in. Setting the difficulty high only helps that latter part (the shares) which is actually the least work. If python wasn't single threaded and as inefficient as it was compared to a low level language, this workload would be irrelevant over and above the work of communicating with the p2pool network and the bitcoind to generate work, but it's not when everything is serialised, so there are always bursts of worst case latencies which are real-world problematic.