[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool

[kano]

I still think they should implement this call on top of an asynchronous update of the block template to make the RPC call o(1) instead of o(nb_tx) or worse though. In the worst case, even if some params of getblocktemplate don't allow to precompute most of the template at least the common case scenario with no parameter can be sped up.

Suggested it, hopefully sipa will be interested enough to either explain why it can't work or make a mental note of the idea for the next performance roadblock.

gmaxwell's comment makes me think the discussion about async processing of templates should be taken here. I'm wondering if forrestv implemented a p2pool template cache with the same approach I suggested could be provided by bitcoind. Especially just after a new block found on the network (see the comment here). There's no explicit configuration of bitcoind done to make signal p2pool new blocks so is it detecting them by calling a fast RPC method of bitcoind or is there a delay that could be shortened by using the "blocknotify" bitcoind option for example?

Ah yeah - I forgot to completely ignore the fact (like you are) that the big pools are confirming the transactions that you aren't ...

So yet again your excuse for proving p2pool is worse for bitcoin is that ... there's apparently a problem with bitcoind that the big pools can work around ... and then you get posts like the one above this post ...

I also notice that gmaxwell is completely avoiding posting here ... yes he's an avid proponent of p2pool ... maybe there's a reason he's not commenting in here ...

[daemondazz]

I pulled down the latest bitcoind from Git yesterday and rebuilt it (rather than using the Ubuntu PPA).



The latency at a bit over 12s is version 0.8.1 from the PPA. No matter what changes I made to the config, it would always climb up to that within a couple of hours.

The next step is 0.8.2rc1 from Git with all values at the default. Definitely a lot better. I'm getting a lot of non-standard transaction type messages in the log, which I am assuming is transactions that fail the new dust test. I have deb files for Ubuntu 12.04 AMD64 if anyone is interested.

The last step is setting the mintxfee and minrelaytxfee to double the default values:

Code:

mintxfee=0.001
minrelaytxfee=0.001

I am mining blocks with transaction fees (133 transactions at the moment), although that's a bit moot anyway because I haven't actually solved a block in the whole time I've been mining, but still.

I'm hoping that's a reasonable tradeoff between a p2pool node that generates some income and contributing to the greater bitcoin network.

[centove]

Pulled the latest git... (reports itself as "version" : 80201)

Nice... Latency has dropped waaay down.

http://ask.gxsnmp.org/

[gyverlb]

Ah yeah - I forgot to completely ignore the fact (like you are) that the big pools are confirming the transactions that you aren't ...

So yet again your excuse for proving p2pool is worse for bitcoin is that ... there's apparently a problem with bitcoind that the big pools can work around ... and then you get posts like the one above this post ...

I also notice that gmaxwell is completely avoiding posting here ... yes he's an avid proponent of p2pool ... maybe there's a reason he's not commenting in here ...

What's your point again? Sorry kano, but your bullshitting is so thick I don't even know what you are bullshitting about now.

[centove]

Seen on the btcguild news page:

Stratum servers have recently had a patch applied to the bitcoind client which should reduce the time the servers spend waiting for new work to be generated when a block is found on the network. This should result in fewer stales for all users on Stratum, who have seen stale rates increasing due to performance issues in bitcoind the last week.

So I guess the ebil p2pool people weren't the only ones seeing the problem...

[Amph]

w8 perhaps i missed a part, but the config file is the same of the old version 8.1?
because nothing has changed when i set the new parameters, the latency remain the same, with or without setting

[daemondazz]

w8 perhaps i missed a part, but the config file is the same of the old version 8.1?
because nothing has changed when i set the new parameters, the latency remain the same, with or without setting

Did you upgrade to 0.8.1rc1?

As I posted from my latency graph screenshot, the latency about halved with the new version at default settings on my system.

[daemondazz]

Someone appears to be bringing more hardware online in the pool, approx 100GH/s in the last hour or so, my node is currently reporting 980GH/s

Are we going to break 1TH/s soon

[daemondazz]

Nice!

2013-05-24 14:52:34.464483  Pool: 1000GH/s Stale rate: 16.5% Expected time to block: 13.3 hours

[Amph]

w8 perhaps i missed a part, but the config file is the same of the old version 8.1?
because nothing has changed when i set the new parameters, the latency remain the same, with or without setting

Did you upgrade to 0.8.1rc1?

As I posted from my latency graph screenshot, the latency about halved with the new version at default settings on my system.

yeah latency is better
my concern is that even if i put those settings in the config file
mintxfee=0.01
minrelaytxfee=0.005
the latency remain the same as default

[rav3n_pl]

mintxfee=0.01 is too big imo, tx fee is calculate per 1kb of tx data, so if s1 is sending from p2pool mining account this can hurts.

[GrapeApe]

Not bad huh?

[rav3n_pl]

Not bad huh?

HOW?
Give us spec.

ps. update main topic, we passed 1000 GH/s

[GrapeApe]

WOW! How?

1) actual bitcoin from git ( moves  variables ...Fees.. from main.h to main.cpp)
 2)  set in src/main.cpp the parameters to
int64 CTransaction::nMinTxFee = 1000000000;    # Override with -mintxfee
int64 CTransaction::nMinRelayTxFee = 1000000000;



3) compile bitcoin

 
4) in bitcoin.conf:
blockmaxsize=5000
blockprioritysize=0
blockminsize=0

Greets

I did what is stated above to the latest 0.8.2rc2 (sipa changes). I had similar results with the 0.8.2rc1 as well (3.00ms).

Deja Vu no?

[gyverlb]

4) in bitcoin.conf:
blockmaxsize=5000
blockprioritysize=0
blockminsize=0

Greets

I did what is stated above to the latest 0.8.2rc2 (sipa changes). I had similar results with the 0.8.2rc1 as well (3.00ms).

Usually I agree to some blockmaxsize and tx fee tuning, but it's going too far here. forrestv suggested that the bitcoind latency doesn't influence your efficiency noticeably and I can confirm that below 0.2s I don't see any meaningful difference (I'm still at 110-115% efficiency).

In fact you may actually hurt your income at this point. Depending on your hashrate you will eventually hit a block and it will pay less to everyone, yourself included. If you only have hundreds of megahashes this doesn't change much for everyone as you probably won't hit a block in the foreseeable future but if you are one of the lucky few with ~100GH/s (there are at least 2 currently according to the payouts) it's 10% of the total p2pool network and nearly 10% of less fees for everyone yourself included. Fees are ~1% of the incomes currently.

This means ~0.1% less income for everyone. If you encourage others to do the same mistake, this is 1% less income for everyone.

My recommendations if you want to tune bitcoind:

• compile it from git head like GrapeApe: it has the latest performance enhancements and its default mintxfee and minrelaytxfee values are better for miners (don't touch the source, modifications of constants demonstrated earlier in this thread will only make things more difficult and only touch default values that can now be changed with the mintxfee and minrelaytxfee parameters)
• look at your average bitcoind latency as reported by p2pool for at least 24h
• if your latency is above 0.1s and only if it is try the following tunings (from first to last) measuring the result for at least 24h each time you apply one of them and stop when you reach <0.1s latency (it will give you some margin if something slows down bitcoind later and avoid its latency raising above 0.2s, you should monitor this value and many others with monitoring software if you value your income anyway):
  
  • give CPU and IO priority to your bitcoind and p2pool processes if your OS allows it (see the guide in my signature, I'll update it soon for Linux examples)
  • move the bitcoind and p2pool data directories to SSD storage
  • reduce blockmaxsize to 250000 instead of 500000
  • raise mintxfee and minrelaytxfee to 0.0005
  • if you are still above 0.1s, reapply the measure that had the largest impact on your getblocktemplate latency between the last 2 (blockmaxsize=125000 or mintxfee/minrelaytxfee=0.001 for example)

[rav3n_pl]

So for git and 0.8.2 it all can be set in bitcoin.conf
I`ll try this:

Code:

blockmaxsize=500000
blockprioritysize=10000
blockminsize=5000
mintxfee=0.0005
minrelaytxfee=0.0005

[GrapeApe]

Uuhh please don't call them GrapeApe modifications (just following directions from another post). With that said gyverlb I agree completely. My efficiency is the same with 2.5ms vs .05s latency. So yes all I am doing is ultimately cutting into my (everyones) income. I have done exactly as you recommended above. By the way your guide has been a huge help.

[gyverlb]

Uuhh please don't call them GrapeApe modifications (just following directions from another post). With that said gyverlb I agree completely. My efficiency is the same with 2.5ms vs .05s latency. So yes all I am doing is ultimately cutting into my (everyones) income. I have done exactly as you recommended above. By the way your guide has been a huge help.

I'll edit my post right away to remove "GrapeApe modifications" (I can't fit many posts in my short-term memory and I'm a little groggy from painkillers so I make mistakes, sorry).

[gyverlb]

4) in bitcoin.conf:
blockmaxsize=5000
blockprioritysize=0
blockminsize=0

Greets

I did what is stated above to the latest 0.8.2rc2 (sipa changes). I had similar results with the 0.8.2rc1 as well (3.00ms).

Usually I agree to some blockmaxsize and tx fee tuning, but it's going too far here. forrestv suggested that the bitcoind latency doesn't influence your efficiency noticeably and I can confirm that below 0.2s I don't see any meaningful difference (I'm still at 110-115% efficiency).

In fact you may actually hurt your income at this point. Depending on your hashrate you will eventually hit a block and it will pay less to everyone, yourself included. If you only have hundreds of megahashes this doesn't change much for everyone as you probably won't hit a block in the foreseeable future but if you are one of the lucky few with ~100GH/s (there are at least 2 currently according to the payouts) it's 10% of the total p2pool network and nearly 10% of less fees for everyone yourself included. Fees are ~1% of the incomes currently.

This means ~0.1% less income for everyone. If you encourage others to do the same mistake, this is 1% less income for everyone.

My recommendations if you want to tune bitcoind:

• compile it from git head like GrapeApe: it has the latest performance enhancements and its default mintxfee and minrelaytxfee values are better for miners (don't touch the source, modifications of constants demonstrated earlier in this thread will only make things more difficult and only touch default values that can now be changed with the mintxfee and minrelaytxfee parameters)
• look at your average bitcoind latency as reported by p2pool for at least 24h
• if your latency is above 0.1s and only if it is try the following tunings (from first to last) measuring the result for at least 24h each time you apply one of them and stop when you reach <0.1s latency (it will give you some margin if something slows down bitcoind later and avoid its latency raising above 0.2s, you should monitor this value and many others with monitoring software if you value your income anyway):
  
  • give CPU and IO priority to your bitcoind and p2pool processes if your OS allows it (see the guide in my signature, I'll update it soon for Linux examples)
  • move the bitcoind and p2pool data directories to SSD storage
  • reduce blockmaxsize to 250000 instead of 500000
  • raise mintxfee and minrelaytxfee to 0.0005
  • if you are still above 0.1s, reapply the measure that had the largest impact on your getblocktemplate latency between the last 2 (blockmaxsize=125000 or mintxfee/minrelaytxfee=0.001 for example)

Note that if you reach the latest steps, you probably have CPU and IO limitations that are hurting your efficiency elsewhere anyway.

If your node is in ideal conditions (very strong CPU and SSD dedicated to bitcoind + P2Pool) and your latency still needs heavy tuning (blocksize < 100000 or fees > 0.001) to reach 0.1s, this is starting to hurt the Bitcoin network (especially if you control large hashrates): you should start discussing it with other p2pool miners to see if it's a general problem worth reporting to the Bitcoin devs.

[zvs]

This server is using the latest git pull with unmodified settings on fees & a max blocksize of 500000 (isn't the default 1,000,000?  or is it still 500k for p2pool?)

http://nogleg.net:9332/static/graphs.html?Hour

granted, it's also running ltc @

http://nogleg.net:9327/static/graphs.html?Hour

but that doesn't seem to really take much resources anyway

    "blocks" : 237724,
    "currentblocksize" : 476689,
    "currentblocktx" : 812,
    "difficulty" : 11187257.46136079,
    "errors" : "",
    "generate" : false,
    "genproclimit" : -1,
    "hashespersec" : 0,
    "pooledtx" : 844,
    "testnet" : false

it benches about 5% higher than nogleg.com