[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool

[gyverlb]

This server is using the latest git pull with unmodified settings on fees & a max blocksize of 500000 (isn't the default 1,000,000?  or is it still 500k for p2pool?)

I believe the default is 500k. 1M is the limit allowed by the protocol and can be set if needed.

In fact if you have a very low getblocktemplate latency, you should increase maxblocksize as there's still 22MB of unconfirmed transaction as I write this according to blockchain.info: there may be fee-less transactions in these 22MB but I assume that there's a lot of fees to grab too.

If you are below 0.1s latency and want to optimize your income, increase maxblocksize gradually until you reach either 1MB or 0.1s latency.

Edit: typo, I meant 0.1s latency not 0.01.

[Mogumodz]

http://sourceforge.net/projects/bitcoin/files/Bitcoin/bitcoin-0.8.2/test/

Just seen a new rc3 4 hours ago. Anyone want to test?

I'm off to bed.

[kano]

Ah yeah - I forgot to completely ignore the fact (like you are) that the big pools are confirming the transactions that you aren't ...

So yet again your excuse for proving p2pool is worse for bitcoin is that ... there's apparently a problem with bitcoind that the big pools can work around ... and then you get posts like the one above this post ...

I also notice that gmaxwell is completely avoiding posting here ... yes he's an avid proponent of p2pool ... maybe there's a reason he's not commenting in here ...

What's your point again? Sorry kano, but your bullshitting is so thick I don't even know what you are bullshitting about now.

I think I've found a Luke-Jr V2.0

[zvs]

This server is using the latest git pull with unmodified settings on fees & a max blocksize of 500000 (isn't the default 1,000,000?  or is it still 500k for p2pool?)

I believe the default is 500k. 1M is the limit allowed by the protocol and can be set if needed.

In fact if you have a very low getblocktemplate latency, you should increase maxblocksize as there's still 22MB of unconfirmed transaction as I write this according to blockchain.info: there may be fee-less transactions in these 22MB but I assume that there's a lot of fees to grab too.

If you are below 0.01s latency and want to optimize your income, increase maxblocksize gradually until you reach either 1MB or 0.01s latency.

Well, it's hitting like .10-.15s avg, with .0001 fee and 500,000 blocksize, doesn't seem too bad. 

'course, someone could always waste some of their money to spam a lot of junk transactions.  Guess you'd just need to monitor it a bit

[Mogumodz]

While all you lot touch each other and fight about it.

No changes to Bitcoin.conf, simply updated to 0.8.3rc3 and I have some LOW LAT back, seems to be holding.

[mdude77]

While all you lot touch each other and fight about it.

No changes to Bitcoin.conf, simply updated to 0.8.3rc3 and I have some LOW LAT back, seems to be holding.

I still don't see what the big deal was about latency.  Everything seemed to be working fine, but people were panicking over apparent high latency.  Instead they'd rather use a pre-release version of bitcoind that specifics warns you not to use for mining.

<shrug>

M

[gyverlb]

Ah yeah - I forgot to completely ignore the fact (like you are) that the big pools are confirming the transactions that you aren't ...

So yet again your excuse for proving p2pool is worse for bitcoin is that ... there's apparently a problem with bitcoind that the big pools can work around ... and then you get posts like the one above this post ...

I also notice that gmaxwell is completely avoiding posting here ... yes he's an avid proponent of p2pool ... maybe there's a reason he's not commenting in here ...

What's your point again? Sorry kano, but your bullshitting is so thick I don't even know what you are bullshitting about now.

I think I've found a Luke-Jr V2.0

That's not really helping your point.

To help you understand my point of view: I'm bed-ridden for more than 3 weeks now and take painkillers that make concentration for long period of times difficult and result in my current short-temper. If you don't present your argument in a concise way or repeat arguments that have already countered, I simply call bullshit: I don't have the patience to repeat myself these days.

Calling me names (literally ) is just a waste of your time.

[Mogumodz]

While all you lot touch each other and fight about it.

No changes to Bitcoin.conf, simply updated to 0.8.3rc3 and I have some LOW LAT back, seems to be holding.

I still don't see what the big deal was about latency.  Everything seemed to be working fine, but people were panicking over apparent high latency.  Instead they'd rather use a pre-release version of bitcoind that specifics warns you not to use for mining.

<shrug>

M

Trouble started with 0.8.1 and 11.4 for me. Seems all back to normal now. Not had to change anything in Bitcoin.conf to affect tx and wotnot, waiting worked.

[gyverlb]

While all you lot touch each other and fight about it.

No changes to Bitcoin.conf, simply updated to 0.8.3rc3 and I have some LOW LAT back, seems to be holding.

I still don't see what the big deal was about latency.  Everything seemed to be working fine, but people were panicking over apparent high latency.  Instead they'd rather use a pre-release version of bitcoind that specifics warns you not to use for mining.

<shrug>

M

I already explained the problem. There are 2 reasons to upgrade:

• You use bitcoind's RPC interface yourself and use calls that took several seconds instead of <0.1s in your tools (like I do)
• P2Pool has shown less efficiency when the getblocktemplate latency is above 0.2s in the past (it's shown in the graphs to help study its influence)

If everyone has the same blocktemplate latency (even if its several seconds) it doesn't matter: the efficiency is relative to other P2Pool nodes. But if some have far less latency than others we could see the same problem appear again.
And efficiency vs other node set aside we have orphan blocks to worry about too: if other pools manage to get block templates faster than P2Pool our orphan rate will rise.

[mdude77]

While all you lot touch each other and fight about it.

No changes to Bitcoin.conf, simply updated to 0.8.3rc3 and I have some LOW LAT back, seems to be holding.

I still don't see what the big deal was about latency.  Everything seemed to be working fine, but people were panicking over apparent high latency.  Instead they'd rather use a pre-release version of bitcoind that specifics warns you not to use for mining.

<shrug>

M

I already explained the problem. There are 2 reasons to upgrade:

• You use bitcoind's RPC interface yourself and use calls that took several seconds instead of <0.1s in your tools (like I do)
• P2Pool has shown less efficiency when the getblocktemplate latency is above 0.2s in the past (it's shown in the graphs to help study its influence)

If everyone has the same blocktemplate latency (even if its several seconds) it doesn't matter: the efficiency is relative to other P2Pool nodes. But if some have far less latency than others we could see the same problem appear again.
And efficiency vs other node set aside we have orphan blocks to worry about too: if other pools manage to get block templates faster than P2Pool our orphan rate will rise.

And you know for sure HOW that when the latency goes up that it affects p2pool?  When everyone else tries saying there's a problem, you call them a troll, and it's just bad luck.  Yet when you say there's a problem, it's gospel, not bad luck?

I agree, everyone having different latencies should theoretically affect their performance vs others.  All the more reason to leave it alone and keep everyone the same!

M

[zvs]

While all you lot touch each other and fight about it.

No changes to Bitcoin.conf, simply updated to 0.8.3rc3 and I have some LOW LAT back, seems to be holding.

I still don't see what the big deal was about latency.  Everything seemed to be working fine, but people were panicking over apparent high latency.  Instead they'd rather use a pre-release version of bitcoind that specifics warns you not to use for mining.

<shrug>

M

I already explained the problem. There are 2 reasons to upgrade:

• You use bitcoind's RPC interface yourself and use calls that took several seconds instead of <0.1s in your tools (like I do)
• P2Pool has shown less efficiency when the getblocktemplate latency is above 0.2s in the past (it's shown in the graphs to help study its influence)

If everyone has the same blocktemplate latency (even if its several seconds) it doesn't matter: the efficiency is relative to other P2Pool nodes. But if some have far less latency than others we could see the same problem appear again.
And efficiency vs other node set aside we have orphan blocks to worry about too: if other pools manage to get block templates faster than P2Pool our orphan rate will rise.

And you know for sure HOW that when the latency goes up that it affects p2pool?  When everyone else tries saying there's a problem, you call them a troll, and it's just bad luck.  Yet when you say there's a problem, it's gospel, not bad luck?

I agree, everyone having different latencies should theoretically affect their performance vs others.  All the more reason to leave it alone and keep everyone the same!

M

1) they'll be slower on the uptake for new blocks, wasting the mining time of whatever their latency is/was
2) if they discover a block, it's more likely to be orphaned

indirectly, higher latency would imply more transactions, which would mean more orphans for you on the p2pool network

[daemondazz]

Also, the P2Pool block time is 10 seconds, so having a latency of >10 seconds means that when a new block is detected, you're going to completely miss the next block, which would half your actual effective rate.

[mdude77]

Also, the P2Pool block time is 10 seconds, so having a latency of >10 seconds means that when a new block is detected, you're going to completely miss the next block, which would half your actual effective rate.

Finally something that is factual.  Thanks!

The highest I saw mine was 3s.  And as I stated multiple times, restarting bitcoind put it back down where it was supposed to be, but it did start climbing back up.  I still fail to see how transactions have anything to do with it when restarting bitcoind lowers the latency.

M

[gyverlb]

And you know for sure HOW that when the latency goes up that it affects p2pool?

Maybe because I did exactly what I described in the guide in my signature.
Mind the fact that I said it affected p2pool. forrestv didn't communicate much on the subject, he might have changed p2pool's code since the problem was detected. This is easy to check by doing the same experiment that I did and described in the guide.

Currently I'm slowly raising maxblocksize and lowering mintxfee and minrelaytxfee on my node to find out how the new bitcoind code is behaving, how the getblocktemplate latency changes and how the efficiency reacts. I'll probably have to add some tips based on my results.
For example, even with the default fees, bitcoind doesn't fill a 500kB block on my node so raising maxblocksize doesn't change much, I'm just testing fee limits below the default value.
I suspect most of the 29MB of transactions currently around is made of fee-less or below recommended fees.

When everyone else tries saying there's a problem, you call them a troll, and it's just bad luck.  Yet when you say there's a problem, it's gospel, not bad luck?

Which problem and reported by whom? If it's still this nonsense about a bug in p2pool when we have bad luck, yes there are trolls. I don't even bother arguing on the subject now, organofcorti has already made the statistical analysis of p2pool luck and shown it was perfectly normal.

And if you didn't believe me when I said getblocktemplate latency was a problem, why don't you argue with the Bitcoin devs that included a patch to lower it this very week?

I agree, everyone having different latencies should theoretically affect their performance vs others.  All the more reason to leave it alone and keep everyone the same!

Yeah right, like it would work and nobody would try to profit from the situation if there's even a mirage of an advantage to gain. There are many people motivated by pure profit or even blinded by some assumption that tuning something as much as possible is good although it's often a compromise.
Just look at what I had to say when someone gave an example of maxblocksize=5000: there's clearly work to be done to avoid these extremes (in this case just explaining that it's counterproductive should be enough).

[bitpop]

Huge difference on mine with rc3 see sig

[bitpop]

Ps is there an explorer for unconfirmed transactions?

[Subo1977]

Ps is there an explorer for unconfirmed transactions?

blockchain.info/de/unconfirmed-transactions

[bitpop]

Sweet. What's the harm in making my node include them all.

[gyverlb]

Also, the P2Pool block time is 10 seconds, so having a latency of >10 seconds means that when a new block is detected, you're going to completely miss the next block, which would half your actual effective rate.

Finally something that is factual.  Thanks!

The highest I saw mine was 3s.  And as I stated multiple times, restarting bitcoind put it back down where it was supposed to be, but it did start climbing back up.  I still fail to see how transactions have anything to do with it when restarting bitcoind lowers the latency.

M

  There are such misconceptions floating around...

OK first mdude77 is wondering why the latency is lowered by restarting bitcoind and raises again.

When you restart bitcoind it doesn't save the list of unconfirmed transactions on disk: they are only maintained in what's called the memory pool.
When you restart it, at first it doesn't have any transaction in its memory pool. So any rpc call to bitcoind involving any parsing of the list of transaction in the memory pool is fast: getblocktemplate is obviously one of them as its goal is to build a template for a block including the tx in your memory pool that your bitcoind deems worth confirming.
Then your bitcoind learns of unconfirmed transactions at the rate at which it's neighboors (the nodes it's connected to) on the bitcoind P2P network transmit them. Only new transactions are automatically broadcasted and received by a freshly started node quickly, old ones are transmitted far slower to avoid network congestion. So a freshly started node gets new transactions appearing on the network quickly and slowly catches up through the backlog of unconfirmed transactions. The rate at which a node receives old transactions varies greatly: it depends on the nodes it is connected to (not every node is configured the same, some may relay more transactions than others).
This is why getblocktemplate latency is slowly rising and is perfectly consistent with getblocktemplate latency being heavily dependent on both the number of transactions your bitcoind accepts in its memory pool and the number of transactions it selects for confirmation (these being controlled by maxblocksize, minxtxfee and minrelaytxfee among others).

Second: the getblocktemplate and P2Pool orphan rate.
First a bit of vocabulary just to avoid any misunderstanding: what daemondazz calls P2Pool blocks are P2Pool shares.

AFAIK P2Pool doesn't need a fresh getblocktemplate to build a share. It can reuse a previous one (unless a Bitcoin block has just been found, then shares based on old templates would generate orphan blocks, see one of my earlier posts on why it's bad). The only drawback of using an old getblocktemplate result is that it doesn't include new transactions that were received after it (so it lowers the fee income and slow transaction confirmation a bit).

What the problem is is probably more subtle than that: P2Pool is an event-driven program (as opposed to multi-threaded or multi-process) with most of its event being handled asynchronously (waiting for a getblocktemplate result is probably done asynchronously).
What could happen but is very difficult to analyze is that the event-driven framework doesn't cope with long running queries as well for some obscure technical reason (like polling some event result in exponentially longer intervals instead of being woken up by a select or equivalent system call) or might have a timing problem when some task that can't be done asynchronously (like pure crypto computations) happens to be done right when the geblocktemplate comes back. There's even a system in place in P2Pool to punish shares computed from old templates when a newer block is known: P2Pool nodes generating them because of high getblocktemplate latency will most probably see them orphaned.

I don't know the exact reason why getblocktemplate affected efficiency and even if it's still the case today as forrestv might have changed something that removes this problem. It was still the case very recently (like less than 2 months ago) when getblocktemplate took more than 0.2s. I don't check often how it affects p2pool but I'm doing it right now (in fact I'm studying how the block size and fee limits affect getblocktemplate in the current situation, checking the efficiency is just a bonus). If the behavior of p2pool changed I'll know it in the following days and will be able to update my guide. For now I still recommend to keep it under 0.2s to be safe.

[gyverlb]

While all you lot touch each other and fight about it.

No changes to Bitcoin.conf, simply updated to 0.8.3rc3 and I have some LOW LAT back, seems to be holding.

I still don't see what the big deal was about latency.  Everything seemed to be working fine, but people were panicking over apparent high latency.  Instead they'd rather use a pre-release version of bitcoind that specifics warns you not to use for mining.

<shrug>

M

Trouble started with 0.8.1 and 11.4 for me. Seems all back to normal now. Not had to change anything in Bitcoin.conf to affect tx and wotnot, waiting worked.

As your post could be misinterpreted: to be accurate simply waiting didn't work: you had to update bitcoind to a version which includes the fix from sipa for the getblocktemplate low performance. The P2Pool version doesn't have anything to do with it and you verified it yourself by upgrading bitcoind and keeping the same P2Pool version.

Your trouble started when bitcoind 0.8.1 had to handle the current backlog of unconfirmed transactions. 0.8.1 could handle a normal block size worth of transactions (500kB to 1MB) without problems but it simply couldn't process more than 20MB of them efficiently until sipa fixed the code.