[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool

[gyverlb]

When I browsed through my P2Pool log, I noticed that the P2Pool stale rate varied between 17% and 20%+ in the last couple of days. Therefore the efficiency figure fluctuates even if your own stale / DOA rate doesn't change at all. That is why I think one should watch one's own stale rate more than the efficiency figure.

Both are interesting: your own stale rate varying shows that something changed on your node and help you pinpoint local problems. But you don't want your efficiency to fall too much. Depending on your use of merged-mining (Namecoin essentially), if you fall below the 90-95% range, a centralized pool begins to makes sense if you can't lower your stale rate and wants to maximize your income.

One problem with reaching conclusions on the data is that one needs quite a high hashrate in order to get a narrow confidence interval on the stale rate in a timeframe of a day or so.

For example, my mining rate is about 4.5 GH/s, and the stale rate interval reported by P2Pool is -+5% of the real stale rate after mining with the pool for three days or so.

Yep, mining with less than 10GH/s makes things difficult. I'm at ~9GH/s and I don't fully trust my stats. I often double-check configuration changes when differences are low enough to be explained by variance in my share/stale rate.

[1714002347]

1714002347

[1714002347]

1714002347

[1714002347]

1714002347

[Bitmong]

When I browsed through my P2Pool log, I noticed that the P2Pool stale rate varied between 17% and 20%+ in the last couple of days. Therefore the efficiency figure fluctuates even if your own stale / DOA rate doesn't change at all. That is why I think one should watch one's own stale rate more than the efficiency figure.

Both are interesting: your own stale rate varying shows that something changed on your node and help you pinpoint local problems. But you don't want your efficiency to fall too much. Depending on your use of merged-mining (Namecoin essentially), if you fall below the 90-95% range, a centralized pool begins to makes sense if you can't lower your stale rate and wants to maximize your income.

Well, I am merge-mining Namecoin, Devcoin and Ixcoin, so there is some extra there.

And actually my efficiency is over 100% now that the incoming bitcoind connection issue was solved. Now I'm only interested to see how the 0.8.2rc3 update and increased maxblocksize affected the efficiency. We shall see about that this week.

[walf_man]

need continue update...

[daemondazz]

need continue update...

Of?

[zvs]

if you eliminated the DOA's from the ASIC miner that has my pool as a backup, i have 501 shares, 12 orphans, and ~25-30 DOA (ed: more like 460-470 shares, 10-11 orphans, and ~25-30 DOA.  forgot to include the good ones too)

mining on my home connection nets me 0-1% DOA but 10%+ orphans.  i think it's due to 1) not being able to open that many connections (bandwidth), and 2) most of the hashing power being in europe and asia

anyway, for someone out there looking for a pool to mine at, the lowest latency one isnt necessarily the best.   i spent several days on my florida server (75ms latency) and ended up gonig back to germany, because the florida server's orphans >>> DOA on other server, even though it also had 50+ connections open.  like i said, i think that's probably due to all the nodes in europe, russia, china, etc

i imagine the default outgoing limit was set to such a small number w/ ppl like myself in mind (crap upstream from home connection), but with 10Mbit connection you should at least up that to 15 or so.   it looks like my average outgoing is around 200 KB/s with 70 connections, though on occasion it'll go > 500KB/s

i'm not that fond of incoming connections (due to all the messed up nodes that'll connect and disconnect + the nodes with 1000+ms latency), but if nobody accepted any incoming, then, well....   i set mine to 10 as a compromise.. it defaults to 40.   the --p2pool-node you put as additions don't count as outgoing connections towards your limit.    (i have 5 outgoing, 10 incoming, the rest are all from --p2pool-node)

[furball]

Just to recap, is this now the recommended settings:

Code:

blockmaxsize=1000000
blockminsize=400000
mintxfee=0.00001
minrelaytxfee=0.00001

And that's for bitcoin.conf right? And solo miners should/nt use this too?

I'm guessing this only works if my node finds the block so I hope if this is good, most people add this because it effects me.

That's for bitcoin.conf, yes.

blockminsize is a personnal choice (I left it out in the guide).
This works if your node finds a block, the minrelaytxfee might have a direct impact as it helps propagate fees that could be paid to you when someone else finds a block.

I've been testing some scenarios as well and found that for me, the txfees are very influential on the latency. I wanted to change to a bigger block size to hopefully make more on the pool but I found after that I had to tune the txfees to keep the latency low.

This is what I've settled on; this gives me circa 0.2s latency with an acceptable efficiency; maybe it will work for other people too.

blockprioritysize=27000
blockmaxsize=1000000
mintxfee=0.00007
minrelaytxfee=0.00007

[gyverlb]

I've been testing some scenarios as well and found that for me, the txfees are very influential on the latency. I wanted to change to a bigger block size to hopefully make more on the pool but I found after that I had to tune the txfees to keep the latency low.

This is what I've settled on; this gives me circa 0.2s latency with an acceptable efficiency; maybe it will work for other people too.

blockprioritysize=27000
blockmaxsize=1000000
mintxfee=0.00007
minrelaytxfee=0.00007

You are right: if there are enough tx to fill a block raising the minimum txfees avoids maintaining some that can't fit in the block in the memory pool which lowers your latency.

I raised mined from 0.00001 to 0.00002 and although my latency went from 0.25s to ~0.1s my efficiency wasn't raised noticeably (still ~105%). I believe that more and more P2Pool nodes are being setup correctly which makes everybody's efficiency converge towards 100% (which is fair and good).

[ok]

Thanks, but this still doesnt' help us. How to merge stratum-forrestv with stratum-mining-proxy?

stratum-mining-proxy uses the stratum package, which is somewhere on your computer if you're running stratum-mining-proxy, and is what the patch needs to be applied to.

Thank you! It works.
Now I successfully installed new stratum package and proxy is working, just look:

Code:

$ python mining_proxy.py -gp 5001 -sp 5002 -o localhost -p 9332
2013-05-27 10:22:17,862 INFO proxy jobs.<module> # Using C extension for midstate speedup. Good!
2013-05-27 10:22:17,871 ERROR proxy mining_proxy.main # Stratum host/port autodetection failed
Traceback (most recent call last):
  File "mining_proxy.py", line 178, in main
    new_host = (yield utils.detect_stratum(args.host, args.port))
  File "/usr/local/lib/python2.7/dist-packages/Twisted-13.0.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py", line 1070, in _inlineCallbacks
    result = g.send(result)
  File "/home/pioruns/stratum-mining-proxy/mining_libs/utils.py", line 69, in detect_stratum
    header = f.response_headers.get('x-stratum', None)[0]
TypeError: 'NoneType' object has no attribute '__getitem__'
2013-05-27 10:22:17,871 WARNING proxy mining_proxy.main # Stratum proxy version: 1.5.2
2013-05-27 10:22:17,873 WARNING proxy mining_proxy.test_update # Checking for updates...
2013-05-27 10:22:18,265 WARNING proxy mining_proxy.main # Trying to connect to Stratum pool at localhost:9332
2013-05-27 10:22:18,268 INFO stats stats.print_stats # 1 peers connected, state changed 1 times
2013-05-27 10:22:18,268 INFO proxy mining_proxy.on_connect # Connected to Stratum pool at localhost:9332
2013-05-27 10:22:18,268 INFO proxy mining_proxy.on_connect # Subscribing for mining jobs
2013-05-27 10:22:18,303 WARNING proxy mining_proxy.main # -----------------------------------------------------------------------
2013-05-27 10:22:18,304 WARNING proxy mining_proxy.main # PROXY IS LISTENING ON ALL IPs ON PORT 5002 (stratum) AND 5001 (getwork)
2013-05-27 10:22:18,304 WARNING proxy mining_proxy.main # -----------------------------------------------------------------------
2013-05-27 10:22:18,304 INFO proxy client_service.handle_event # Setting new difficulty: 0.999984741211
2013-05-27 10:22:18,306 INFO proxy client_service.handle_event # New job 8850090419252557308580900352527982298 for prevhash 385766c3, clean_jobs=True
2013-05-27 10:22:29,931 INFO proxy client_service.handle_event # Setting new difficulty: 0.999984741211
2013-05-27 10:22:29,933 INFO proxy client_service.handle_event # New job 82294000856594409674845997521737547736 for prevhash 385766c3, clean_jobs=True
2013-05-27 10:22:41,372 INFO proxy client_service.handle_event # Setting new difficulty: 0.999984741211
2013-05-27 10:22:41,374 INFO proxy client_service.handle_event # New job 285117993302263092594898950693479724933 for prevhash 385766c3, clean_jobs=True

Why do you need a stratum proxy any way?

Tomorrow (hopefully today if time permits) I will be testing BE Blade on this proxy

Hi there lenny, please let me know if you were successful with setting up you BE Blade with the stratum proxy on p2pool. Thx

[FlappySocks]

Anyone tried p2pool with shedskin or pypy to lighten the resources it uses?

[PrintMule]

Is there a point of me trying out my 600mh/swhile having quite weak upload speeds?



Do not want to go through all the hassle for nothing.
What rates are expected?

[stepkrav]

can i use a remote bitcoind instead of running it in the same machine with the miners?

Or could i have bitcoind and p2pool on remote machine and miners to a local one ?

[lenny_]

Thanks, but this still doesnt' help us. How to merge stratum-forrestv with stratum-mining-proxy?

stratum-mining-proxy uses the stratum package, which is somewhere on your computer if you're running stratum-mining-proxy, and is what the patch needs to be applied to.

Thank you! It works.
Now I successfully installed new stratum package and proxy is working, just look:

Code:

$ python mining_proxy.py -gp 5001 -sp 5002 -o localhost -p 9332
2013-05-27 10:22:17,862 INFO proxy jobs.<module> # Using C extension for midstate speedup. Good!
2013-05-27 10:22:17,871 ERROR proxy mining_proxy.main # Stratum host/port autodetection failed
Traceback (most recent call last):
  File "mining_proxy.py", line 178, in main
    new_host = (yield utils.detect_stratum(args.host, args.port))
  File "/usr/local/lib/python2.7/dist-packages/Twisted-13.0.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py", line 1070, in _inlineCallbacks
    result = g.send(result)
  File "/home/pioruns/stratum-mining-proxy/mining_libs/utils.py", line 69, in detect_stratum
    header = f.response_headers.get('x-stratum', None)[0]
TypeError: 'NoneType' object has no attribute '__getitem__'
2013-05-27 10:22:17,871 WARNING proxy mining_proxy.main # Stratum proxy version: 1.5.2
2013-05-27 10:22:17,873 WARNING proxy mining_proxy.test_update # Checking for updates...
2013-05-27 10:22:18,265 WARNING proxy mining_proxy.main # Trying to connect to Stratum pool at localhost:9332
2013-05-27 10:22:18,268 INFO stats stats.print_stats # 1 peers connected, state changed 1 times
2013-05-27 10:22:18,268 INFO proxy mining_proxy.on_connect # Connected to Stratum pool at localhost:9332
2013-05-27 10:22:18,268 INFO proxy mining_proxy.on_connect # Subscribing for mining jobs
2013-05-27 10:22:18,303 WARNING proxy mining_proxy.main # -----------------------------------------------------------------------
2013-05-27 10:22:18,304 WARNING proxy mining_proxy.main # PROXY IS LISTENING ON ALL IPs ON PORT 5002 (stratum) AND 5001 (getwork)
2013-05-27 10:22:18,304 WARNING proxy mining_proxy.main # -----------------------------------------------------------------------
2013-05-27 10:22:18,304 INFO proxy client_service.handle_event # Setting new difficulty: 0.999984741211
2013-05-27 10:22:18,306 INFO proxy client_service.handle_event # New job 8850090419252557308580900352527982298 for prevhash 385766c3, clean_jobs=True
2013-05-27 10:22:29,931 INFO proxy client_service.handle_event # Setting new difficulty: 0.999984741211
2013-05-27 10:22:29,933 INFO proxy client_service.handle_event # New job 82294000856594409674845997521737547736 for prevhash 385766c3, clean_jobs=True
2013-05-27 10:22:41,372 INFO proxy client_service.handle_event # Setting new difficulty: 0.999984741211
2013-05-27 10:22:41,374 INFO proxy client_service.handle_event # New job 285117993302263092594898950693479724933 for prevhash 385766c3, clean_jobs=True

Why do you need a stratum proxy any way?

Tomorrow (hopefully today if time permits) I will be testing BE Blade on this proxy

Hi there lenny, please let me know if you were successful with setting up you BE Blade with the stratum proxy on p2pool. Thx

Unfortunately, no success...
Blade just do not connect to stratum proxy on p2pool. There is not error message at all.
I would like really to have some developer in it, I can donate my Blade worktime to debug this.

[mdude77]

Unfortunately, no success...
Blade just do not connect to stratum proxy on p2pool. There is not error message at all.
I would like really to have some developer in it, I can donate my Blade worktime to debug this.

You know, I've tried using the stratum proxy on p2pool with normal GPUs.  It just doesn't work.  I don't think this is a blade thing.

M

[daemondazz]

Is there a point of me trying out my 600mh/swhile having quite weak upload speeds?



Do not want to go through all the hassle for nothing.
What rates are expected?

I run my p2pool node hosted in a data centre and slightly under 2GH/s of miners on my ADSL link which has a 600-700kbps upstream speed.

[daemondazz]

can i use a remote bitcoind instead of running it in the same machine with the miners?

Or could i have bitcoind and p2pool on remote machine and miners to a local one ?

Yes to both.

If you want to run p2pool on a different machine than bitcoind, then you need to configure bitcoind to listen on an IP address instead of loopback and then configured p2pool with the IP address. I've tried this over a LAN and it works - although I haven't tried it over the general Internet.

For the second question, this is the setup I have - bitcoind/p2pool in a data centre and miners at home.

[stepkrav]

thanks daemondazz, i'm interested in the scheme you use. Any caveat i should have in mind? For example it would be wise to use a payout address not from the bitcoind, right?

[yxt]

right!

[gyverlb]

Is there a point of me trying out my 600mh/swhile having quite weak upload speeds?



Do not want to go through all the hassle for nothing.
What rates are expected?

You probably can if you don't use your upload bandwidth for much else. Follow instructions in the guide in my signature, default settings would clog your upload pipe and give suboptimal results.

Edit: your ping time isn't bad (especially if it's not a ping with an host at your ISP) which is a good thing.

[gyverlb]

thanks daemondazz, i'm interested in the scheme you use. Any caveat i should have in mind? For example it would be wise to use a payout address not from the bitcoind, right?

It's always a good idea. Your wallets shouldn't have a public IP address but P2Pool certainly should.

[gyverlb]

can i use a remote bitcoind instead of running it in the same machine with the miners?

Or could i have bitcoind and p2pool on remote machine and miners to a local one ?

Yes to both.

If you want to run p2pool on a different machine than bitcoind, then you need to configure bitcoind to listen on an IP address instead of loopback and then configured p2pool with the IP address. I've tried this over a LAN and it works - although I haven't tried it over the general Internet.

For the second question, this is the setup I have - bitcoind/p2pool in a data centre and miners at home.

Your pool node (http://cryptominer.org) doesn't seem to publish its default pages (couldn't find them). What is the average efficiency you get with such a setup? I stopped using a remote node in the early days while I had large amounts of stales and low efficiency and I wonder if it got better since.

Getblocktemplate latency doesn't seem to matter anymore for example (I'll have to double-check, the network was a little unstable these past 48h and my results are not conclusive).