[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool

[gyverlb]

Just to recap, is this now the recommended settings:

Code:

blockmaxsize=1000000
blockminsize=400000
mintxfee=0.00001
minrelaytxfee=0.00001

And that's for bitcoin.conf right? And solo miners should/nt use this too?

I'm guessing this only works if my node finds the block so I hope if this is good, most people add this because it effects me.

blockminsize is 0 by default, i don't see why you'd want to change it unless there's some particular reason you want to reserve 400kb for (any) 0 fee transactions..  if you've been running bitcoind for more than a couple hours, i imagine all of your new blocks will start out at 400kb, as there are plenty of ppl sending free transactions out there trying to do double spends and what not

it does allow 27000 bytes of priority transactions, which are free.

the fees you have here are 1/10th of the default too, i dunno why you'd want to lower that

I removed the blockminsize vaue from my recommendations: using it is giving a free ride. It's more or less an individual decision: do you want to encourage fee-less transactions or not? This has nearly no impact on your income (as bitcoind will select the best combination of tx to maximize your income by default).

As I explained during my tests if you don't lower the mintxfee and minrelaytxfee in the current situation you can't even fill a 500kB block. If you want to maximize your income you have to lower them to include more transactions

[gyverlb]

Just to recap, is this now the recommended settings:

Code:

blockmaxsize=1000000
blockminsize=400000
mintxfee=0.00001
minrelaytxfee=0.00001

And that's for bitcoin.conf right? And solo miners should/nt use this too?

I'm guessing this only works if my node finds the block so I hope if this is good, most people add this because it effects me.

Keep in mind that your available bandwidth plays a big part here. People running p2pool on a crappy DSL connection probably won't be able to get away with 1MB blocks. 500KB might even be pushing it.

I agree, this is addressed in the guide in my signature.
TL;DR: to minimize your bandwidth usage you can lower maxconnections (125 by default which is insanely high for a miner with typical home upload bandwidths) there are trade-offs described in the guide, but lowering it to be able to increase blocmaxsize is usually a win.

[Subo1977]

is three a list of found Block per P2pool-Node?

I think it's one thing of including the maximum TX Fees in a Block , but how great is the chance of finding a Block on my node and earn the Fee's?

in my situation it's make's a differenz of 20 % income with / Without Fees, but i havnt found a Block in the last. So its better for me to not include the Fees's and increase my Efficiency.

[gyverlb]

Just to recap, is this now the recommended settings:

Code:

blockmaxsize=1000000
blockminsize=400000
mintxfee=0.00001
minrelaytxfee=0.00001

And that's for bitcoin.conf right? And solo miners should/nt use this too?

I'm guessing this only works if my node finds the block so I hope if this is good, most people add this because it effects me.

That's for bitcoin.conf, yes.

blockminsize is a personnal choice (I left it out in the guide).
This works if your node finds a block, the minrelaytxfee might have a direct impact as it helps propagate fees that could be paid to you when someone else finds a block.

[lenny_]

Thanks, but this still doesnt' help us. How to merge stratum-forrestv with stratum-mining-proxy?

stratum-mining-proxy uses the stratum package, which is somewhere on your computer if you're running stratum-mining-proxy, and is what the patch needs to be applied to.

Thank you! It works.
Now I successfully installed new stratum package and proxy is working, just look:

Code:

$ python mining_proxy.py -gp 5001 -sp 5002 -o localhost -p 9332
2013-05-27 10:22:17,862 INFO proxy jobs.<module> # Using C extension for midstate speedup. Good!
2013-05-27 10:22:17,871 ERROR proxy mining_proxy.main # Stratum host/port autodetection failed
Traceback (most recent call last):
  File "mining_proxy.py", line 178, in main
    new_host = (yield utils.detect_stratum(args.host, args.port))
  File "/usr/local/lib/python2.7/dist-packages/Twisted-13.0.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py", line 1070, in _inlineCallbacks
    result = g.send(result)
  File "/home/pioruns/stratum-mining-proxy/mining_libs/utils.py", line 69, in detect_stratum
    header = f.response_headers.get('x-stratum', None)[0]
TypeError: 'NoneType' object has no attribute '__getitem__'
2013-05-27 10:22:17,871 WARNING proxy mining_proxy.main # Stratum proxy version: 1.5.2
2013-05-27 10:22:17,873 WARNING proxy mining_proxy.test_update # Checking for updates...
2013-05-27 10:22:18,265 WARNING proxy mining_proxy.main # Trying to connect to Stratum pool at localhost:9332
2013-05-27 10:22:18,268 INFO stats stats.print_stats # 1 peers connected, state changed 1 times
2013-05-27 10:22:18,268 INFO proxy mining_proxy.on_connect # Connected to Stratum pool at localhost:9332
2013-05-27 10:22:18,268 INFO proxy mining_proxy.on_connect # Subscribing for mining jobs
2013-05-27 10:22:18,303 WARNING proxy mining_proxy.main # -----------------------------------------------------------------------
2013-05-27 10:22:18,304 WARNING proxy mining_proxy.main # PROXY IS LISTENING ON ALL IPs ON PORT 5002 (stratum) AND 5001 (getwork)
2013-05-27 10:22:18,304 WARNING proxy mining_proxy.main # -----------------------------------------------------------------------
2013-05-27 10:22:18,304 INFO proxy client_service.handle_event # Setting new difficulty: 0.999984741211
2013-05-27 10:22:18,306 INFO proxy client_service.handle_event # New job 8850090419252557308580900352527982298 for prevhash 385766c3, clean_jobs=True
2013-05-27 10:22:29,931 INFO proxy client_service.handle_event # Setting new difficulty: 0.999984741211
2013-05-27 10:22:29,933 INFO proxy client_service.handle_event # New job 82294000856594409674845997521737547736 for prevhash 385766c3, clean_jobs=True
2013-05-27 10:22:41,372 INFO proxy client_service.handle_event # Setting new difficulty: 0.999984741211
2013-05-27 10:22:41,374 INFO proxy client_service.handle_event # New job 285117993302263092594898950693479724933 for prevhash 385766c3, clean_jobs=True

Why do you need a stratum proxy any way?

Tomorrow (hopefully today if time permits) I will be testing BE Blade on this proxy

[bitpop]

Good guide even though I'm completely opposite with transactions. Anyway why does Bitcoin limit connections on the rpc side? That seems like a bug.

[bitpop]

Does the be blade use its own mining software and not cgminer etc?

[bitpop]

http://p2pool.info

is three a list of found Block per P2pool-Node?

I think it's one thing of including the maximum TX Fees in a Block , but how great is the chance of finding a Block on my node and earn the Fee's?

in my situation it's make's a differenz of 20 % income with / Without Fees, but i havnt found a Block in the last. So its better for me to not include the Fees's and increase my Efficiency.

[Subo1977]

http://p2pool.info

is three a list of found Block per P2pool-Node?

I think it's one thing of including the maximum TX Fees in a Block , but how great is the chance of finding a Block on my node and earn the Fee's?

in my situation it's make's a differenz of 20 % income with / Without Fees, but i havnt found a Block in the last. So its better for me to not include the Fees's and increase my Efficiency.

I know this site,
i mean a list of found Block per NODE

[zvs]

http://p2pool.info

is three a list of found Block per P2pool-Node?

I think it's one thing of including the maximum TX Fees in a Block , but how great is the chance of finding a Block on my node and earn the Fee's?

in my situation it's make's a differenz of 20 % income with / Without Fees, but i havnt found a Block in the last. So its better for me to not include the Fees's and increase my Efficiency.

I know this site,
i mean a list of found Block per NODE

besides just grepping BLOCK on the log file?  probably not..  i've found 3 in the last 2 months or so, but that's incredibly lucky.   just look at what the normal odds would be and calculate it that way, i guess.  for 10ghash, the average time to find a block would be about 2 months exactly.  so the chances of finding one before next difficulty change is, what, 25%?   so, if some 500kb block size increases the # of orphans and DOA you get by even .5% or something, it's a bad deal.  i still think the best thing to do here would be to give fees to block solver, but w/e

raising blocksize to something like 20,000 or 30,000 and also raising the minimum fee, i think you could argue that might be more efficient for both

ed:  i've run p2pool on a virtual ubuntu server on my home connection.  i have an i7 980 and 24GB RAM, so I keep entire blockchain in RAM, etc.  but I have to limit upstream to 50KB/s, because home connection is only 1Mbit upstream (or so they claim, it's more like 768Kbps)...  anything more and it starts to cause some jitter, anything past 70KB/s and I start getting packetloss.   so a 500KB block would take some 10 seconds to send out to 1 peer.  this also increases the chance of it being orphaned from bitcoin network itself, not just p2pool

[gyverlb]

  so, if some 500kb block size increases the # of orphans and DOA you get by even .5% or something, it's a bad deal.  i still think the best thing to do here would be to give fees to block solver, but w/e

raising blocksize to something like 20,000 or 30,000 and also raising the minimum fee, i think you could argue that might be more efficient for both

Please stop propagating deprecated numbers: these aren't valid anymore, I spent several days testing this and reporting results, the guide in my signature is updated with up to date information.
For example I use a maxblocksize of 1,000,000 and lower fee limits than the defaults and my node is currently at 111,7% efficiency (the number oscillates between 110 and 115) with a getblocktemplate latency averaging 0.26s for the last 24h.

ed:  i've run p2pool on a virtual ubuntu server on my home connection.  i have an i7 980 and 24GB RAM, so I keep entire blockchain in RAM, etc.  but I have to limit upstream to 50KB/s, because home connection is only 1Mbit upstream (or so they claim, it's more like 768Kbps)...  anything more and it starts to cause some jitter, anything past 70KB/s and I start getting packetloss.   so a 500KB block would take some 10 seconds to send out to 1 peer.  this also increases the chance of it being orphaned from bitcoin network itself, not just p2pool

My upstream is the same. But I limited the number of connections of both bitcoind and P2Pool to avoid them filling the pipe. I use maxconnections=10 for bitcoind and --max-conns 3 --outgoing-conns 3 for P2Pool. With your processor speed, with the same settings than my own configuration you should expect a better efficiency than mine (probably >115% unless all miners start updating according to my guide in the meantime: this should lower all efficiencies above 100%). Just read the guide and report if your results don't match mine.

Blocks found by a P2Pool node are broadcasted by every P2Pool nodes and IIRC as the transactions are already known, like P2Pool shares the whole block doesn't need to be transmitted between P2Pool nodes.
If it weren't the case we would have more orphans than other pools.

[Bitmong]

I don't know the exact reason why getblocktemplate affected efficiency and even if it's still the case today as forrestv might have changed something that removes this problem. It was still the case very recently (like less than 2 months ago) when getblocktemplate took more than 0.2s. I don't check often how it affects p2pool but I'm doing it right now (in fact I'm studying how the block size and fee limits affect getblocktemplate in the current situation, checking the efficiency is just a bonus). If the behavior of p2pool changed I'll know it in the following days and will be able to update my guide. For now I still recommend to keep it under 0.2s to be safe.

Some recent findings on P2Pool efficiency on my node.

My node is directly connected to the Internet with Ethernet, 100 Mbit/s downstream and 10 Mbit/s upstream. The node is a Phenom four-core processor, with SSD disk. I have 7 mining rigs connected to the node via LAN.

All numbers below are with current (April 2013) P2Pool from Github.

When my configuration was incorrect and Bitcoind could only make outgoing connections, my efficiency was between 95% and 99%.

After fixing the configuration problem, efficiency rose to 110-115% level. I have now 30-40 connections to the Bitcoin network.

When the getblocktemplate latency started to appear, my efficiency was still between 110-115%. My getblocktemplate latency was about 30 seconds at that time.

I have now upgraded to the 0.8.2rc3 version, and the getblocktemplate latency decreased to about 0.1 seconds, but it has increased to 0.9 seconds since the upgrade (in four hours).

Current efficiency after two hours from the upgrade is 102.4%. Well, I think one cannot deduce anything from that yet, maybe the stopping and restarting of bitcoind caused some orphans.

I'll report the efficiency back to this thread after 24 hours have passed with this new bitcoind version.

[Bitmong]

One more thing I noticed.. After the bitcoind upgrade, my DOA has doubled from ~ 4% to 8%.

EDIT: After closer inspection of the log file, I see that the DOA has varied between ~0% to 8% anyway before this.

[gyverlb]

When the getblocktemplate latency started to appear, my efficiency was still between 110-115%. My getblocktemplate latency was about 30 seconds at that time.

This is the worst latency I've seen reported so far (by nearly 3x, the worst I can rember was ~12s). With <0.8.2rc3 getblocktemplate depends heavily on you CPU speed and number of transactions in your memory pool. As you seem to have an adequate CPU, it probably doesn't explain the difference. Do you use non-default values in your bitcoin.conf?

You have relatively high bandwidth, but what is your link latency? If you use traceroute/mtr or similar tools, what is the time-to-hop for your ISP main routers and addresses in North America/Europe/China (where most nodes and probably miners are according to http://blockchain.info/fr/nodes-globe).

[davidkassa]

When the getblocktemplate latency started to appear, my efficiency was still between 110-115%. My getblocktemplate latency was about 30 seconds at that time.

This is the worst latency I've seen reported so far

My getblocktemplate latency hit about 30 seconds until I upgraded to the 0.8.2rc3.

Now I'm not sure if it's "bad luck" or something else but since I upgraded my latency is ~0.4 but my efficiency has been sitting in the low 80%s. I'm not quite sure how to debug this. My local DOA is ~2.5% but my orphans have been awful. I have both 8333 and 9333 traffic QOS'd. I guess I'm going to look closer at my network.

[daemondazz]

Hwoever has started mining on cryptominer.org in the last little while, please check your username as it appears you've got an extra 'M' at the start:

Code:

darryl@server1:~$ bitcoind validateaddress M13iETYjuQ1Dnhz4vRPBRHJj6mztu4i86Fs
{
    "isvalid" : false
}
darryl@server1:~$ bitcoind validateaddress 13iETYjuQ1Dnhz4vRPBRHJj6mztu4i86Fs
{
    "isvalid" : true,
    "address" : "13iETYjuQ1Dnhz4vRPBRHJj6mztu4i86Fs",
    "ismine" : false
}

Unless of course you're donating your hashrate to me

[gyverlb]

When the getblocktemplate latency started to appear, my efficiency was still between 110-115%. My getblocktemplate latency was about 30 seconds at that time.

This is the worst latency I've seen reported so far

My getblocktemplate latency hit about 30 seconds until I upgraded to the 0.8.2rc3.

Now I'm not sure if it's "bad luck" or something else but since I upgraded my latency is ~0.4 but my efficiency has been sitting in the low 80%s. I'm not quite sure how to debug this. My local DOA is ~2.5% but my orphans have been awful. I have both 8333 and 9333 traffic QOS'd. I guess I'm going to look closer at my network.

Check my guide if you see some low hanging fruits.

I just noticed that my own efficiency started to get down the last 24h (the first 24h with my settings where between 110 and 115% and now I'm at ~105%).
This may be because most P2Pool nodes are upgrading bitcoind and lowering their own getblocktemplate latency in the process. I'll have to test again to see if lowering my getblocktemplate latency helps my efficiency.
My current 0.25s might not be low enough now. One possibility is that this value actually influences efficiency when there's a noticeable difference between its value on a node and the average on all P2Pool nodes. Back to testing different bitcoin.conf values. Too bad: in its current configuration my node was including enough fees to make a 26.5BTC block! Fortunately with 0.8.2 I won't have to make much sacrifices in fees to reach very low latencies.

[Bitmong]

When the getblocktemplate latency started to appear, my efficiency was still between 110-115%. My getblocktemplate latency was about 30 seconds at that time.

This is the worst latency I've seen reported so far (by nearly 3x, the worst I can rember was ~12s). With <0.8.2rc3 getblocktemplate depends heavily on you CPU speed and number of transactions in your memory pool. As you seem to have an adequate CPU, it probably doesn't explain the difference. Do you use non-default values in your bitcoin.conf?

I didn't change any block size / tx fee settings in bitcoin.conf, only rpc settings and server settings.

Actually the latency was around 30 seconds when I used the bitcoind on that computer or when I set P2Pool to use the bitcoin-qt on my Windows computer with Core i7-3820 processor. So, the latency was definitely not caused by the CPU.

You have relatively high bandwidth, but what is your link latency? If you use traceroute/mtr or similar tools, what is the time-to-hop for your ISP main routers and addresses in North America/Europe/China (where most nodes and probably miners are according to http://blockchain.info/fr/nodes-globe).

I live in Finland btw.

MTR shows these results for some nodes:

First ISP router 2ms
Last ISP router 4ms
US (Sayreville) 111ms
Japan 294ms
Australia 430ms
Hong Kong 340ms
Netherlands 33ms
China 330ms
Switzerland 44ms
Ukraine 57ms

[Bitmong]

I just noticed that my own efficiency started to get down the last 24h (the first 24h with my settings where between 110 and 115% and now I'm at ~105%).
This may be because most P2Pool nodes are upgrading bitcoind and lowering their own getblocktemplate latency in the process. I'll have to test again to see if lowering my getblocktemplate latency helps my efficiency.
My current 0.25s might not be low enough now. One possibility is that this value actually influences efficiency when there's a noticeable difference between its value on a node and the average on all P2Pool nodes. Back to testing different bitcoin.conf values. Too bad: in its current configuration my node was including enough fees to make a 26.5BTC block! Fortunately with 0.8.2 I won't have to make much sacrifices in fees to reach very low latencies.

When I browsed through my P2Pool log, I noticed that the P2Pool stale rate varied between 17% and 20%+ in the last couple of days. Therefore the efficiency figure fluctuates even if your own stale / DOA rate doesn't change at all. That is why I think one should watch one's own stale rate more than the efficiency figure.

One problem with reaching conclusions on the data is that one needs quite a high hashrate in order to get a narrow confidence interval on the stale rate in a timeframe of a day or so.

For example, my mining rate is about 4.5 GH/s, and the stale rate interval reported by P2Pool is -+5% of the real stale rate after mining with the pool for three days or so.

[gyverlb]

When the getblocktemplate latency started to appear, my efficiency was still between 110-115%. My getblocktemplate latency was about 30 seconds at that time.

This is the worst latency I've seen reported so far (by nearly 3x, the worst I can rember was ~12s). With <0.8.2rc3 getblocktemplate depends heavily on you CPU speed and number of transactions in your memory pool. As you seem to have an adequate CPU, it probably doesn't explain the difference. Do you use non-default values in your bitcoin.conf?

I didn't change any block size / tx fee settings in bitcoin.conf, only rpc settings and server settings.

Actually the latency was around 30 seconds when I used the bitcoind on that computer or when I set P2Pool to use the bitcoin-qt on my Windows computer with Core i7-3820 processor. So, the latency was definitely not caused by the CPU.

Maybe I just upgraded soon enough to not see these kinds of latencies (it was just below 10s at the worst).

You have relatively high bandwidth, but what is your link latency? If you use traceroute/mtr or similar tools, what is the time-to-hop for your ISP main routers and addresses in North America/Europe/China (where most nodes and probably miners are according to http://blockchain.info/fr/nodes-globe).

I live in Finland btw.

MTR shows these results for some nodes:

First ISP router 2ms
Last ISP router 4ms
US (Sayreville) 111ms
Japan 294ms
Australia 430ms
Hong Kong 340ms
Netherlands 33ms
China 330ms
Switzerland 44ms
Ukraine 57ms

These looks good, I have better European and China results (most of the time <30ms in Europe and ~220ms with China) but I'm puzzled (I don't think it can explain our differences in efficiency).

Did you restart your p2pool node recently and let it run 24h to have a more accurate stat (efficiency isn't accurate with long runs : it compares your own rates of DOA/orphans since the p2pool process start with the average over 24h of the network).