[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool

[zvs]

i have a bitcoind that has 800 connections

This may be the cause of your DOA. It was already said here, but you shouldn't have that many connections on your bitcoind. The network code for bitcoin isn't yet optimized and with that many connections you're effectively harming yourself and the network as a whole...

I'll drop it to 500 and move everything back over tomorrow and see how it goes.  I'm 95% sure the +25 outgoing connections reduced the orphans by at least 50%.  The DOA rate is still a bit high, but I also get 150ms ping times to my hetzner server.  I have 24GB of RAM on my home computer, so I might just end up installing the p2pool server here.

[bitcoindaddy]

Is p2pool ready for ASIC's or is there still work that needs to be done? Can it already do variable difficulty?

[K1773R]

Is p2pool ready for ASIC's or is there still work that needs to be done? Can it already do variable difficulty?

yes, vardiff works already. stratum is "ComingSoonTM"

[bitcoindaddy]

Is p2pool ready for ASIC's or is there still work that needs to be done? Can it already do variable difficulty?

yes, vardiff works already. stratum is "ComingSoonTM"

Will stratum make the communication between cgminer and the local p2pool process more efficient or is that how variable difficult will be done?

[K1773R]

Is p2pool ready for ASIC's or is there still work that needs to be done? Can it already do variable difficulty?

yes, vardiff works already. stratum is "ComingSoonTM"

Will stratum make the communication between cgminer and the local p2pool process more efficient or is that how variable difficult will be done?

local communication has no real latencys therefore its always efficient. variable difficulty is already implemented (altough it increases the diff for all miners, not only the fast one). for the unreleased(/undeveloped) stratum stuff ul have to ask forrestv.

[bitcoindaddy]

Is p2pool ready for ASIC's or is there still work that needs to be done? Can it already do variable difficulty?

yes, vardiff works already. stratum is "ComingSoonTM"

Will stratum make the communication between cgminer and the local p2pool process more efficient or is that how variable difficult will be done?

local communication has no real latencys therefore its always efficient. variable difficulty is already implemented (altough it increases the diff for all miners, not only the fast one). for the unreleased(/undeveloped) stratum stuff ul have to ask forrestv.

So p2pool is not really ready for ASIC? Because the first one that comes along is going to screw it up for everyone else? Not really local variable difficulty?

[K1773R]

Is p2pool ready for ASIC's or is there still work that needs to be done? Can it already do variable difficulty?

yes, vardiff works already. stratum is "ComingSoonTM"

Will stratum make the communication between cgminer and the local p2pool process more efficient or is that how variable difficult will be done?

local communication has no real latencys therefore its always efficient. variable difficulty is already implemented (altough it increases the diff for all miners, not only the fast one). for the unreleased(/undeveloped) stratum stuff ul have to ask forrestv.

So p2pool is not really ready for ASIC? Because the first one that comes along is going to screw it up for everyone else? Not really local variable difficulty?

It is already with Vardiff. Later there will be stratum too.

[mdude77]

Is p2pool ready for ASIC's or is there still work that needs to be done? Can it already do variable difficulty?

yes, vardiff works already. stratum is "ComingSoonTM"

Will stratum make the communication between cgminer and the local p2pool process more efficient or is that how variable difficult will be done?

local communication has no real latencys therefore its always efficient. variable difficulty is already implemented (altough it increases the diff for all miners, not only the fast one). for the unreleased(/undeveloped) stratum stuff ul have to ask forrestv.

So p2pool is not really ready for ASIC? Because the first one that comes along is going to screw it up for everyone else? Not really local variable difficulty?

ASIC is going to mess everyone up who doesn't have an ASIC.  That isn't unique to p2pool.

M

[Exdeath]

Hello, everyone. Today I try joining to p2pool and I have some problems.
My OS: Ubuntu 12.04.1 64 bit
Bitcoind version:

Code:

$./bitcoind getinfo
....
 "version" : 70100,
  "protocolversion" : 60002,
  "walletversion" : 40000,
....

Python version: 2.7.3

Today I download the tgz-archive with p2pool v 9.4 for 64-bit Linux.
Then I installed required packages:
sudo apt-get install python-zope.interface python-twisted python-twisted-web

After the bitcoind successfully running, and all of the blocks are downloaded, I run file "run_p2pool.py", and:

Code:

$./run_p2pool.py
2012-12-26 07:36:45.305205 > Error while checking Bitcoin connection:
2012-12-26 07:36:45.305419 > Traceback (most recent call last):
2012-12-26 07:36:45.305524 >   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 545, in _runCallbacks
2012-12-26 07:36:45.305624 >     current.result = callback(current.result, *args, **kw)
2012-12-26 07:36:45.305726 >   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1095, in gotResult
2012-12-26 07:36:45.305871 >     _inlineCallbacks(r, g, deferred)
2012-12-26 07:36:45.305966 >   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1037, in _inlineCallbacks
2012-12-26 07:36:45.306088 >     result = result.throwExceptionIntoGenerator(g)
2012-12-26 07:36:45.306208 >   File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 382, in throwExceptionIntoGenerator
2012-12-26 07:36:45.306317 >     return g.throw(self.type, self.value, self.tb)
2012-12-26 07:36:45.306418 > --- <exception caught here> ---
2012-12-26 07:36:45.306519 >   File "~/forrestv-p2pool-6880123/p2pool/util/deferral.py", line 41, in f
2012-12-26 07:36:45.306623 >     result = yield func(*args, **kwargs)
2012-12-26 07:36:45.306721 >   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1037, in _inlineCallbacks
2012-12-26 07:36:45.306828 >     result = result.throwExceptionIntoGenerator(g)
2012-12-26 07:36:45.306930 >   File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 382, in throwExceptionIntoGenerator
2012-12-26 07:36:45.307052 >     return g.throw(self.type, self.value, self.tb)
2012-12-26 07:36:45.307158 >   File "/mnt/1/exdeath/forrestv-p2pool-6880123/p2pool/bitcoin/helper.py", line 13, in check
2012-12-26 07:36:45.307267 >     if not (yield net.PARENT.RPC_CHECK(bitcoind)):
2012-12-26 07:36:45.307366 >   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1037, in _inlineCallbacks
2012-12-26 07:36:45.307465 >     result = result.throwExceptionIntoGenerator(g)
2012-12-26 07:36:45.307557 >   File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 382, in throwExceptionIntoGenerator
2012-12-26 07:36:45.307661 >     return g.throw(self.type, self.value, self.tb)
2012-12-26 07:36:45.307761 >   File "~/forrestv-p2pool-6880123/p2pool/bitcoin/networks.py", line 16, in <lambda>
2012-12-26 07:36:45.307865 >     'bitcoinaddress' in (yield bitcoind.rpc_help()) and
2012-12-26 07:36:45.307963 >   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1037, in _inlineCallbacks
2012-12-26 07:36:45.308069 >     result = result.throwExceptionIntoGenerator(g)
2012-12-26 07:36:45.308166 >   File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 382, in throwExceptionIntoGenerator
2012-12-26 07:36:45.308313 >     return g.throw(self.type, self.value, self.tb)
2012-12-26 07:36:45.308415 >   File "~/forrestv-p2pool-6880123/p2pool/util/jsonrpc.py", line 64, in callRemote
2012-12-26 07:36:45.308519 >     raise e
2012-12-26 07:36:45.308617 > twisted.web.error.Error: 401 Authorization Required

I will be very grateful to anyone who help me fix this.

[K1773R]

Hello, everyone. Today I try joining to p2pool and I have some problems.
My OS: Ubuntu 12.04.1 64 bit
Bitcoind version:

Code:

$./bitcoind getinfo
....
 "version" : 70100,
  "protocolversion" : 60002,
  "walletversion" : 40000,
....

Python version: 2.7.3

Today I download the tgz-archive with p2pool v 9.4 for 64-bit Linux.
Then I installed required packages:
sudo apt-get install python-zope.interface python-twisted python-twisted-web

After the bitcoind successfully running, and all of the blocks are downloaded, I run file "run_p2pool.py", and:

Code:

$./run_p2pool.py
2012-12-26 07:36:45.305205 > Error while checking Bitcoin connection:
2012-12-26 07:36:45.305419 > Traceback (most recent call last):
2012-12-26 07:36:45.305524 >   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 545, in _runCallbacks
2012-12-26 07:36:45.305624 >     current.result = callback(current.result, *args, **kw)
2012-12-26 07:36:45.305726 >   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1095, in gotResult
2012-12-26 07:36:45.305871 >     _inlineCallbacks(r, g, deferred)
2012-12-26 07:36:45.305966 >   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1037, in _inlineCallbacks
2012-12-26 07:36:45.306088 >     result = result.throwExceptionIntoGenerator(g)
2012-12-26 07:36:45.306208 >   File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 382, in throwExceptionIntoGenerator
2012-12-26 07:36:45.306317 >     return g.throw(self.type, self.value, self.tb)
2012-12-26 07:36:45.306418 > --- <exception caught here> ---
2012-12-26 07:36:45.306519 >   File "~/forrestv-p2pool-6880123/p2pool/util/deferral.py", line 41, in f
2012-12-26 07:36:45.306623 >     result = yield func(*args, **kwargs)
2012-12-26 07:36:45.306721 >   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1037, in _inlineCallbacks
2012-12-26 07:36:45.306828 >     result = result.throwExceptionIntoGenerator(g)
2012-12-26 07:36:45.306930 >   File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 382, in throwExceptionIntoGenerator
2012-12-26 07:36:45.307052 >     return g.throw(self.type, self.value, self.tb)
2012-12-26 07:36:45.307158 >   File "/mnt/1/exdeath/forrestv-p2pool-6880123/p2pool/bitcoin/helper.py", line 13, in check
2012-12-26 07:36:45.307267 >     if not (yield net.PARENT.RPC_CHECK(bitcoind)):
2012-12-26 07:36:45.307366 >   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1037, in _inlineCallbacks
2012-12-26 07:36:45.307465 >     result = result.throwExceptionIntoGenerator(g)
2012-12-26 07:36:45.307557 >   File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 382, in throwExceptionIntoGenerator
2012-12-26 07:36:45.307661 >     return g.throw(self.type, self.value, self.tb)
2012-12-26 07:36:45.307761 >   File "~/forrestv-p2pool-6880123/p2pool/bitcoin/networks.py", line 16, in <lambda>
2012-12-26 07:36:45.307865 >     'bitcoinaddress' in (yield bitcoind.rpc_help()) and
2012-12-26 07:36:45.307963 >   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1037, in _inlineCallbacks
2012-12-26 07:36:45.308069 >     result = result.throwExceptionIntoGenerator(g)
2012-12-26 07:36:45.308166 >   File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 382, in throwExceptionIntoGenerator
2012-12-26 07:36:45.308313 >     return g.throw(self.type, self.value, self.tb)
2012-12-26 07:36:45.308415 >   File "~/forrestv-p2pool-6880123/p2pool/util/jsonrpc.py", line 64, in callRemote
2012-12-26 07:36:45.308519 >     raise e
2012-12-26 07:36:45.308617 > twisted.web.error.Error: 401 Authorization Required

I will be very grateful to anyone who help me fix this.

suply username and password, run p2pool with --help for a list of all parameters.

[zvs]

i'm at 188 shares and 1 orphan now, after modifying source to allow more outgoing connections

[K1773R]

i'm at 188 shares and 1 orphan now, after modifying source to allow more outgoing connections

u see i just improved ur mining alot

[Exdeath]

K1773R, thx 

[zvs]

i'm at 188 shares and 1 orphan now, after modifying source to allow more outgoing connections

u see i just improved ur mining alot

Well, now I'm at 20 shares and 3 orphans.  Not a huge sample, but..

my conclusion would be that the initial outgoing # is too low.  10 as max is also too low.  you should be able to set it up to 20-30.

but, most orphans come from the size of the blocks.  i did ~250 shares with 2 orphans with a maxblocksize of 10000, essentially making blocks of 5 or 6 transactions... later on I changed that to nothing, so all blocks had just 1 transaction.   that also lowered the "GetBlockTemplate Latency" to a couple milliseconds, as can be seen at:  http://nogleg.com:9332/static/graphs.html?Day

Now I've set it back to a 500kB max block size and am at that 20 blocks w/ 3 orphans.  My GetBlockTemplate latency has also increased, though tbh, I don't find that very relevant.  It's a good diagnostic for spotting out possible issues, like if you have maxblocksize set to 0 and it's taking half a second, then that's a problem, I guess.

That 1/4th or 1/3rd of a second later may matter in 1 out of 500 orphans.  The bigger issue would be network slowness & latency.  A bigger problem for p2pool than the network as a whole, since most pools will be run on dedicated servers on good networks.  p2pool is different, because it has all these people mining w/ many of them on crap connections.  For me to make the most bitcoins, then I should make all blocks with 0 transactions, to limit orphans.  

my block solved w/ maxblocksize of 10000:  http://blockchain.info/tx/971d3109bdc197d1bb8d1334896db2235941b1da884081dee9e94df666a37e84

i doubt getting 25.5 instead of 25.01 would make up for all the extra orphans that are caused by having transactions included (in p2pool)

It seems to me like if you're keen on p2pool, you'd be better off running a private network with a select group of people, rather than losing 5, 10, or 15% of your hashing power due to people with poor connections... or else just set your maxblocksize to 0.  

ps:  i'm changing my maxblocksize back to 0

[Krak]

my conclusion would be that the initial outgoing # is too low.  10 as max is also too low.  you should be able to set it up to 20-30.

You can already set this with a command line flag (--outgoing-conns). The default shouldn't be raised because with a slow DSL connection, 6 is already pushing it.

ps:  i'm changing my maxblocksize back to 0

This is extremely bad for the network in general and overall unnecessary with transaction pre-forwarding. Seriously, p2pool has had only one orphan in the last 3 months. That's pretty good.

[K1773R]

i'm at 188 shares and 1 orphan now, after modifying source to allow more outgoing connections

u see i just improved ur mining alot

Well, now I'm at 20 shares and 3 orphans.  Not a huge sample, but..

my conclusion would be that the initial outgoing # is too low.  10 as max is also too low.  you should be able to set it up to 20-30.

but, most orphans come from the size of the blocks.  i did ~250 shares with 2 orphans with a maxblocksize of 10000, essentially making blocks of 5 or 6 transactions... later on I changed that to nothing, so all blocks had just 1 transaction.   that also lowered the "GetBlockTemplate Latency" to a couple milliseconds, as can be seen at:  http://nogleg.com:9332/static/graphs.html?Day

Now I've set it back to a 500kB max block size and am at that 20 blocks w/ 3 orphans.  My GetBlockTemplate latency has also increased, though tbh, I don't find that very relevant.  It's a good diagnostic for spotting out possible issues, like if you have maxblocksize set to 0 and it's taking half a second, then that's a problem, I guess.

That 1/4th or 1/3rd of a second later may matter in 1 out of 500 orphans.  The bigger issue would be network slowness & latency.  A bigger problem for p2pool than the network as a whole, since most pools will be run on dedicated servers on good networks.  p2pool is different, because it has all these people mining w/ many of them on crap connections.  For me to make the most bitcoins, then I should make all blocks with 0 transactions, to limit orphans.  

my block solved w/ maxblocksize of 10000:  http://blockchain.info/tx/971d3109bdc197d1bb8d1334896db2235941b1da884081dee9e94df666a37e84

i doubt getting 25.5 instead of 25.01 would make up for all the extra orphans that are caused by having transactions included (in p2pool)

It seems to me like if you're keen on p2pool, you'd be better off running a private network with a select group of people, rather than losing 5, 10, or 15% of your hashing power due to people with poor connections... or else just set your maxblocksize to 0.  

ps:  i'm changing my maxblocksize back to 0

it depends how "good" the network is, for example:
BTC -> 25 connections are good (0% stale so far for me)
LTC -> 50 connections to get 70 submited, 6 stale (needs more connections).

The Coin got 2 sides, the more connection the more broadcast so send/validate/calc. This may lead to more DOA. so dont use astronomical numbers.

[Schleicher]

If the p2pool client is running on your computer at home then the upload speed is probably the bottleneck for most people.
I have 30 KB/s upload speed for example. A 500KB block would need almost 17 seconds to upload. With 10 p2pool connections this would be 170 seconds.

[gyverlb]

If the p2pool client is running on your computer at home then the upload speed is probably the bottleneck for most people.
I have 30 KB/s upload speed for example. A 500KB block would need almost 17 seconds to upload. With 10 p2pool connections this would be 170 seconds.

I believe p2pool doesn't transfer all the block's content: IIRC transactions are preemptively exchanged between nodes before a block is found and only a shorter representation of the block with references to these transactions should be transfered.

[zvs]

If the p2pool client is running on your computer at home then the upload speed is probably the bottleneck for most people.
I have 30 KB/s upload speed for example. A 500KB block would need almost 17 seconds to upload. With 10 p2pool connections this would be 170 seconds.

I believe p2pool doesn't transfer all the block's content: IIRC transactions are preemptively exchanged between nodes before a block is found and only a shorter representation of the block with references to these transactions should be transfered.

It may not transfer the whole thing, but from my experience w/ the larger block sizes, you get tons more orphans.  

I wish it showed local orphans/DOA on the graphs so that it could be analyzed more quickly..  

I'll dig through my logs later and check the orphan amts compared to block size sometime in the next few days...

I do see the reasoning behind the lower amt of outgoing connections though, it does make sense... since not everyone will be on a dedicated server like a 'pool'.   I still think it'd be nice if it were configurable up to 30 instead of maxing out at 10, though....  I believe the # of incoming connections is default capped at 30?  That should probably be lower, actually..

and I've solved two blocks out of equiv of 200,000 difficulty 1 shares so far....  -48 btc....  sadface

[spiccioli]

ps:  i'm changing my maxblocksize back to 0

please don't, this way you're not processing transactions which is the whole meaning of mining.

give blockmaxsize a low value, like 8kB, so that it does not create too many orphans but still processes transactions.

Btw, orphans that solve a block are as good as any other share you submit.

my 2c

spiccioli