[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool

[Glasswalker]

Hey, I recently joined P2Pool with the mining cluster of the Bitcoin Syndicate (currently 6GHash, soon to jump to 12GHash.)

I have a question that came up in relation to BIP16 and how it affects P2Pool (brought it up on this thread: https://bitcointalk.org/index.php?topic=66514)

The question is basically:

BIP16 has a vulnerability, if miners use old bitcoin clients that don't properly handle it, someone could submit a "poisoned" tx into the network that is invalid under bip16 but is valid on older clients. In this case the miner using the old client would add the tx to a block thinking it's valid, but the rest of the network on newer clients would reject the block (Eventually outpacing it's fork and invalidating the miners income). If the transactions are injected regularly, this would amount to a DOS for any miner using older clients.

My question as it relates to P2Pool is:

Since P2Pool is Peer to Peer, if say half the miners have updated clients, and half are using older clients. I assume that the client who ultimately finds the block is the one that submits it to the network and handles validation of transactions? If so, 50% of the time would we be vulnerable to this flaw with BIP16?

In other words, do we require 100% of P2Pool users to upgrade their bitcoin clients to be BIP16 compliant to totally avoid this issue?

And if the answer to the above is yes, that's a huge vulnerability for P2Pool (because someone could maliciously join the pool and mine using an old client, to invalidate a percentage of blocks mined by the miners on P2Pool).

Can someone with more technical knowledge of exactly how P2Pool works dive into this a bit deeper please? Preferably BEFORE the launch of BIP16 (potentially 1st of April at this point).

Thanks!

[forrestv]

Since P2Pool is Peer to Peer, if say half the miners have updated clients, and half are using older clients. I assume that the client who ultimately finds the block is the one that submits it to the network and handles validation of transactions? If so, 50% of the time would we be vulnerable to this flaw with BIP16?

In other words, do we require 100% of P2Pool users to upgrade their bitcoin clients to be BIP16 compliant to totally avoid this issue?

And if the answer to the above is yes, that's a huge vulnerability for P2Pool (because someone could maliciously join the pool and mine using an old client, to invalidate a percentage of blocks mined by the miners on P2Pool).

All P2Pool users will definitely need to upgrade bitcoind before April 1st because of this issue. The question is of how to prevent people from not upgrading, which can be done by changing the rules of the P2Pool protocol.

This could be as simple as a new version of P2Pool that checks bitcoind's version and refuses to work with older versions, combined with a protocol change on April 1st that requires miners to use the new client. Of course, this is vulnerable to people patching out the version check, but P2Pool (along with all other pools) is already vulnerable to malicious block invalidating attacks, so that's not a problem.

[Frizz23]

I just checked my p2pool cruncher because of unusually low payouts.

Besides heaps of orphans in the log there are lots of lines like this here:

Code:

2012-03-08 20:57:32.886000 Error when requesting noncached value:
2012-03-08 20:57:32.887000 > Traceback (most recent call last):
2012-03-08 20:57:32.888000 >   File "twisted\internet\defer.pyc", line 388, in errback
2012-03-08 20:57:32.889000 >
2012-03-08 20:57:32.891000 >   File "twisted\internet\defer.pyc", line 455, in _startRunCallbacks
2012-03-08 20:57:32.892000 >
2012-03-08 20:57:32.893000 >   File "twisted\internet\defer.pyc", line 542, in _runCallbacks
2012-03-08 20:57:32.894000 >
2012-03-08 20:57:32.895000 >   File "twisted\internet\defer.pyc", line 1076, in gotResult
2012-03-08 20:57:32.896000 >
2012-03-08 20:57:32.897000 > --- <exception caught here> ---
2012-03-08 20:57:32.897000 >   File "twisted\internet\defer.pyc", line 1018, in _inlineCallbacks
2012-03-08 20:57:32.898000 >
2012-03-08 20:57:32.899000 >   File "twisted\python\failure.pyc", line 350, in throwExceptionIntoGenerator
2012-03-08 20:57:32.901000 >
2012-03-08 20:57:32.903000 >   File "p2pool\main.pyc", line 171, in <lambda>
2012-03-08 20:57:32.904000 >
2012-03-08 20:57:32.905000 >   File "twisted\internet\defer.pyc", line 1018, in _inlineCallbacks
2012-03-08 20:57:32.906000 >
2012-03-08 20:57:32.907000 >   File "twisted\python\failure.pyc", line 350, in throwExceptionIntoGenerator
2012-03-08 20:57:32.908000 >
2012-03-08 20:57:32.909000 >   File "p2pool\util\jsonrpc.pyc", line 67, in callRemote
2012-03-08 20:57:32.910000 >
2012-03-08 20:57:32.912000 > p2pool.util.jsonrpc.Error: -2 Safe mode: WARNING: Displayed transactions may not be correct!  You may need to upgrade, or other nod
es may need to upgrade.
2012-03-08 20:57:32.914000
2012-03-08 20:57:32.918000

What's going on? I updated p2pool last week! So do I really need to update again?

[DeathAndTaxes]

This is coming from bitcoind.
"WARNING: Displayed transactions may not be correct!  You may need to upgrade, or other nod
es may need to upgrade."

You running 0.6.0 RC1?  If so you need to go to RC2 or go back to 0.5.2

[DeathAndTaxes]

In other words, do we require 100% of P2Pool users to upgrade their bitcoin clients to be BIP16 compliant to totally avoid this issue?

And if the answer to the above is yes, that's a huge vulnerability for P2Pool (because someone could maliciously join the pool and mine using an old client, to invalidate a percentage of blocks mined by the miners on P2Pool).

Can someone with more technical knowledge of exactly how P2Pool works dive into this a bit deeper please? Preferably BEFORE the launch of BIP16 (potentially 1st of April at this point).

IF 51% of Bitcoin network accepts BIP16 then p2pool does need to also.  A miner producing non-compliant hashes will eventually produce a non-compliant block.  The % would be relative to the % of hashing power they have.

A couple ways to deal with it.
The simplest way would be to do version checking in p2pool.  Make new version of p2pool which will not work under any version of bitcoind that isn't BIP16 compliant.  It will simple fail to run with an error.  If someone updates their p2pool but not bitcoind they can't mine.  The new version would also reject 100% of shares from old versions.  Thus if someone doesn't update p2pool or bitcoind they can mine (because their software isn't aware of the issue) but they will see 100% reject rate.  Compensation won't go down because they will never earn anything for this "bad" shares to match the expected value of their "bad" blocks.

Summary:
new p2pool & new bitcoind (BIP16 compliant) = good (no problems)
new p2pool & old bitcoind = won't start
old p2pool & old bitcoind = 100% reject rate from new p2pool version nodes.

That deals with a "ignorant user".  It prevents them from accidentally hurting the network.  A malicious user could intentionally modify bitcoind and p2pool code to produce non-compliant blocks.  This is no different than any other block withholding attack  it generally doesn't make any economical sense because it reduces attacker's revenue also and in case of p2pool they also lose finder's reward.

[stevegee58]

it generally doesn't make any economical sense because it reduces attacker's revenue also and in case of p2pool they also lose finder's reward.

"...some men aren't looking for anything logical, like money. They can't be bought, bullied, reasoned, or negotiated with. Some men just want to watch the world burn."

There are people who would do this just for the sport.

 

[DeathAndTaxes]

"...some men aren't looking for anything logical, like money. They can't be bought, bullied, reasoned, or negotiated with. Some men just want to watch the world burn."

True.  non-economical attacks are always possible but I would point out they can happen on any pool.  The only way to be "safe" would be solo mining.  p2pool through the combination of PPLNS and finders fee is less vulnerable to non-economic block withholding attack.

The most vulnerable would be PPS pools.

[stevegee58]

I suppose a closed p2p pool could be established where each node is authenticated to be there.  Of course then you'd have to have someone administrating it, granting and revoking access, etc.

[DeathAndTaxes]

I suppose a closed p2p pool could be established where each node is authenticated to be there.  Of course then you'd have to have someone administrating it, granting and revoking access, etc.

So all the disadvantages of both a centralized pool AND a decentralized one? 

[kano]

Since P2Pool is Peer to Peer, if say half the miners have updated clients, and half are using older clients. I assume that the client who ultimately finds the block is the one that submits it to the network and handles validation of transactions? If so, 50% of the time would we be vulnerable to this flaw with BIP16?

In other words, do we require 100% of P2Pool users to upgrade their bitcoin clients to be BIP16 compliant to totally avoid this issue?

And if the answer to the above is yes, that's a huge vulnerability for P2Pool (because someone could maliciously join the pool and mine using an old client, to invalidate a percentage of blocks mined by the miners on P2Pool).

All P2Pool users will definitely need to upgrade bitcoind before April 1st because of this issue. The question is of how to prevent people from not upgrading, which can be done by changing the rules of the P2Pool protocol.

This could be as simple as a new version of P2Pool that checks bitcoind's version and refuses to work with older versions, combined with a protocol change on April 1st that requires miners to use the new client. Of course, this is vulnerable to people patching out the version check, but P2Pool (along with all other pools) is already vulnerable to malicious block invalidating attacks, so that's not a problem.

Hmmm - so 3 weeks for the bitcoin developers to ensure that rc2 isn't like rc1 ... and then force everyone to use it under the name of 0.6.0?

[DeathAndTaxes]

Hmmm - so 3 weeks for the bitcoin developers to ensure that rc2 isn't like rc1 ... and then force everyone to use it under the name of 0.6.0?

That assumes the network has the 60%? support which AFAIK it currently doesn't. 

[cabin]

anyone else getting tons of these.. or just me? I am on rc2, for sure. Earlier in this thread it was mentioned it is supposed to be 'ok', but still.. it isn't going away..

2012-03-08 19:22:44.788000 >     return g.throw(self.type, self.value, self.tb)
2012-03-08 19:22:44.788000 >   File "C:\data\miner2\p2pool-a15c106\p2pool\util\jsonrpc.py", line 67, in callRemote
2012-03-08 19:22:44.788000 >     raise Error(**resp['error'])
2012-03-08 19:22:44.789000 > p2pool.util.jsonrpc.Error: -5 Block not found
2012-03-08 19:22:44.790000

[tiker]

Every time I update p2pool or stop and start it again I get new errors...

This time I'm getting the following but don't know why or how to fix it.. any ideas?

Code:

2012-03-08 19:18:37.602900 Error when requesting noncached value:
2012-03-08 19:18:37.603161 > Traceback (most recent call last):
2012-03-08 19:18:37.603401 >   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 388, in errback
2012-03-08 19:18:37.603583 >     self._startRunCallbacks(fail)
2012-03-08 19:18:37.603746 >   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 455, in _startRunCallbacks
2012-03-08 19:18:37.603914 >     self._runCallbacks()
2012-03-08 19:18:37.604149 >   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 542, in _runCallbacks
2012-03-08 19:18:37.604342 >     current.result = callback(current.result, *args, **kw)
2012-03-08 19:18:37.604508 >   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1076, in gotResult
2012-03-08 19:18:37.604674 >     _inlineCallbacks(r, g, deferred)
2012-03-08 19:18:37.604861 > --- <exception caught here> ---
2012-03-08 19:18:37.605021 >   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1018, in _inlineCallbacks
2012-03-08 19:18:37.605215 >     result = result.throwExceptionIntoGenerator(g)
2012-03-08 19:18:37.605376 >   File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 350, in throwExceptionIntoGenerator
2012-03-08 19:18:37.605567 >     return g.throw(self.type, self.value, self.tb)
2012-03-08 19:18:37.605729 >   File "/home/ed/apps/p2pool/p2pool/main.py", line 171, in <lambda>
2012-03-08 19:18:37.605894 >     height_cacher = deferral.DeferredCacher(defer.inlineCallbacks(lambda block_hash: defer.returnValue((lambda x: x['blockcount'] if 'blockcount' in x else x['height'])((yield bitcoind.rpc_getblock('%x' % (block_hash,)))))))
2012-03-08 19:18:37.606063 >   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1018, in _inlineCallbacks
2012-03-08 19:18:37.606276 >     result = result.throwExceptionIntoGenerator(g)
2012-03-08 19:18:37.606439 >   File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 350, in throwExceptionIntoGenerator
2012-03-08 19:18:37.606632 >     return g.throw(self.type, self.value, self.tb)
2012-03-08 19:18:37.606794 >   File "/home/ed/apps/p2pool/p2pool/util/jsonrpc.py", line 67, in callRemote
2012-03-08 19:18:37.606978 >     raise Error(**resp['error'])
2012-03-08 19:18:37.607198 > p2pool.util.jsonrpc.Error: -5 Block not found

[TheHarbinger]

Well, I hate to do it, but I just pulled my 2GH/s from P2Pool.  Something is very seriously wrong.  I have been keeping it up to date, so don't tell me it's me version of p2pool or bitcoin, it's not.  Over the last 4 days I have watched my payout per block drop from around 0.4 to 0.15 on the last one, and p2pool.info is showing 950MH/s.  I will come back, but after what ever the hell is broken is fixed.

[stevegee58]

anyone else getting tons of these.. or just me? I am on rc2, for sure. Earlier in this thread it was mentioned it is supposed to be 'ok', but still.. it isn't going away..

2012-03-08 19:22:44.788000 >     return g.throw(self.type, self.value, self.tb)
2012-03-08 19:22:44.788000 >   File "C:\data\miner2\p2pool-a15c106\p2pool\util\jsonrpc.py", line 67, in callRemote
2012-03-08 19:22:44.788000 >     raise Error(**resp['error'])
2012-03-08 19:22:44.789000 > p2pool.util.jsonrpc.Error: -5 Block not found
2012-03-08 19:22:44.790000

Did you use the windows installer for rc2 or the zip file?  The installer is broken.  Use the zip file instead.

[Red Emerald]

Well, I hate to do it, but I just pulled my 2GH/s from P2Pool.  Something is very seriously wrong.  I have been keeping it up to date, so don't tell me it's me version of p2pool or bitcoin, it's not.  Over the last 4 days I have watched my payout per block drop from around 0.4 to 0.15 on the last one, and p2pool.info is showing 950MH/s.  I will come back, but after what ever the hell is broken is fixed.

The hashrate on p2pool.info is not from the last 10 minutes. It's based on shares submitted in the last 24 hours.  If you've been tinkering and moving things, then it will obviously be lower.

Note: Hashrates are very rough estimates based on the number of shares submitted in the past day. They may be off by 10-20% or more due to variance.

How large and bold should we make the word "variance" before people will read it?

Maybe try p2pmining https://bitcointalk.org/index.php?topic=66202.0

[TheHarbinger]

Well, I hate to do it, but I just pulled my 2GH/s from P2Pool.  Something is very seriously wrong.  I have been keeping it up to date, so don't tell me it's me version of p2pool or bitcoin, it's not.  Over the last 4 days I have watched my payout per block drop from around 0.4 to 0.15 on the last one, and p2pool.info is showing 950MH/s.  I will come back, but after what ever the hell is broken is fixed.

The hashrate on p2pool.info is not from the last 10 minutes. It's based on shares submitted in the last 24 hours.  If you've been tinkering and moving things, then it will obviously be lower.

Note: Hashrates are very rough estimates based on the number of shares submitted in the past day. They may be off by 10-20% or more due to variance.

How large and bold should we make the word "variance" before people will read it?

Maybe try p2pmining https://bitcointalk.org/index.php?topic=66202.0

My rig has been running for 14 days without any changes.  Not 10 minutes.  In that 14 days, both cgminer and my local p2pool constantly report between 1.8 and 2.2 GH/s.  In the last 4 days, my hashrate on p2pool.info and payouts have dropped steadily from 2.0GH/s and 0.4BTC/block to what they are now 943 MH/s and 0.16BTC/block.

That is not variance, that is something wrong.  It doesn't matter if I use 0.5.2 or 0.6.0r2.  The ONLY thing that has changed is the version of p2pool being run, now using p2pool_win32_1f87c32.

[Red Emerald]

Well, I hate to do it, but I just pulled my 2GH/s from P2Pool.  Something is very seriously wrong.  I have been keeping it up to date, so don't tell me it's me version of p2pool or bitcoin, it's not.  Over the last 4 days I have watched my payout per block drop from around 0.4 to 0.15 on the last one, and p2pool.info is showing 950MH/s.  I will come back, but after what ever the hell is broken is fixed.

The hashrate on p2pool.info is not from the last 10 minutes. It's based on shares submitted in the last 24 hours.  If you've been tinkering and moving things, then it will obviously be lower.

Note: Hashrates are very rough estimates based on the number of shares submitted in the past day. They may be off by 10-20% or more due to variance.

How large and bold should we make the word "variance" before people will read it?

Maybe try p2pmining https://bitcointalk.org/index.php?topic=66202.0

My rig has been running for 14 days without any changes.  Not 10 minutes.  In that 14 days, both cgminer and my local p2pool constantly report between 1.8 and 2.2 GH/s.  In the last 4 days, my hashrate on p2pool.info and payouts have dropped steadily from 2.0GH/s and 0.4BTC/block to what they are now 943 MH/s and 0.16BTC/block.

That is not variance, that is something wrong.  It doesn't matter if I use 0.5.2 or 0.6.0r2.  The ONLY thing that has changed is the version of p2pool being run, now using p2pool_win32_1f87c32.

Maybe you somehow got split from the rest of the network? How many peers is p2pool showing?

Are you 100% sure you upgraded to RC2? There have been multiple people that were sure, but the installer apparently doesn't always work properly and they were actually running RC1.

[tiker]

anyone else getting tons of these.. or just me? I am on rc2, for sure. Earlier in this thread it was mentioned it is supposed to be 'ok', but still.. it isn't going away..

2012-03-08 19:22:44.788000 >     return g.throw(self.type, self.value, self.tb)
2012-03-08 19:22:44.788000 >   File "C:\data\miner2\p2pool-a15c106\p2pool\util\jsonrpc.py", line 67, in callRemote
2012-03-08 19:22:44.788000 >     raise Error(**resp['error'])
2012-03-08 19:22:44.789000 > p2pool.util.jsonrpc.Error: -5 Block not found
2012-03-08 19:22:44.790000

Did you use the windows installer for rc2 or the zip file?  The installer is broken.  Use the zip file instead.

That's not it.. because I'm getting the same thing now on my linux box without upgrading to RC2..

Code:

ed@funkylinux4:~/apps$ ./clients/bitcoind getinfo
{
    "version" : 59900,
    "protocolversion" : 60000,
    "balance" : 0.00000000,
    "blocks" : 170270,
    "connections" : 8,
    "proxy" : "",
    "difficulty" : 1496978.59502557,
    "testnet" : false,
    "keypoololdest" : 1327627799,
    "keypoolsize" : 101,
    "paytxfee" : 0.00000000,
    "errors" : ""
}

[TurdHurdur]

I got some of those "p2pool.util.jsonrpc.Error: -5 Block not found" errors, but it doesn't seem to adversely effect my generation of shares. Maybe a peer sent me garbage.