[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool

[vindimy]

Guys,

I've just upgraded to 0.11.1 (did a git clone) after seeing the warning messages (like below). However, the warnings didn't go away after I restarted. Is that normal?

Code:

2012-05-09 18:28:18.833633  Pool: 357GH/s Stale rate: 10.9% Expected time to block: 5.8 hours
2012-05-09 18:28:21.877741 > ########################################
2012-05-09 18:28:21.877929 > >>> WARNING: A MAJORITY OF SHARES CONTAIN A VOTE FOR AN UNSUPPORTED SHARE IMPLEMENTATION! (v2 with 86% support)
2012-05-09 18:28:21.877989 > >>> An upgrade is likely necessary. Check http://p2pool.forre.st/ for more information.
2012-05-09 18:28:21.878055 > ########################################
2012-05-09 18:28:21.878105 P2Pool: 19106 shares in chain (19110 verified/19110 total) Peers: 11 (0 incoming)
2012-05-09 18:28:21.878146  Local: 1380MH/s in last 3.4 minutes Local dead on arrival: ~6.1% (2-15%) Expected time to share: 41.8 minutes
2012-05-09 18:28:21.878186  Shares: 0 (0 orphan, 0 dead) Stale rate: ??? Efficiency: ??? Current payout: 0.0970 BTC
2012-05-09 18:28:21.878231  Pool: 357GH/s Stale rate: 10.9% Expected time to block: 5.8 hours
2012-05-09 18:28:24.924752 > ########################################
2012-05-09 18:28:24.924897 > >>> WARNING: A MAJORITY OF SHARES CONTAIN A VOTE FOR AN UNSUPPORTED SHARE IMPLEMENTATION! (v2 with 86% support)
2012-05-09 18:28:24.924956 > >>> An upgrade is likely necessary. Check http://p2pool.forre.st/ for more information.
2012-05-09 18:28:24.925011 > ########################################
2012-05-09 18:28:24.925247 P2Pool: 19106 shares in chain (19110 verified/19110 total) Peers: 11 (0 incoming)
2012-05-09 18:28:24.925360  Local: 1401MH/s in last 3.5 minutes Local dead on arrival: ~5.9% (2-15%) Expected time to share: 41.1 minutes
2012-05-09 18:28:24.925405  Shares: 0 (0 orphan, 0 dead) Stale rate: ??? Efficiency: ??? Current payout: 0.0970 BTC
2012-05-09 18:28:24.925453  Pool: 357GH/s Stale rate: 10.9% Expected time to block: 5.8 hours
2012-05-09 18:28:25.560529 New work for worker! Difficulty: 0.999985 Share difficulty: 811.979872 Total block value: 50.276000 BTC including 190 transactions
2012-05-09 18:28:25.583851 New work for worker! Difficulty: 0.999985 Share difficulty: 811.979872 Total block value: 50.276000 BTC including 190 transactions
2012-05-09 18:28:25.608886 New work for worker! Difficulty: 1.000000 Share difficulty: 811.979872 Total block value: 50.276000 BTC including 190 transactions
2012-05-09 18:28:27.969760 > ########################################
2012-05-09 18:28:27.969910 > >>> WARNING: A MAJORITY OF SHARES CONTAIN A VOTE FOR AN UNSUPPORTED SHARE IMPLEMENTATION! (v2 with 86% support)
2012-05-09 18:28:27.969977 > >>> An upgrade is likely necessary. Check http://p2pool.forre.st/ for more information.
2012-05-09 18:28:27.970057 > ########################################
2012-05-09 18:28:27.970107 P2Pool: 19107 shares in chain (19111 verified/19111 total) Peers: 11 (0 incoming)
2012-05-09 18:28:27.970146  Local: 1401MH/s in last 3.5 minutes Local dead on arrival: ~7.2% (3-16%) Expected time to share: 41.1 minutes
2012-05-09 18:28:27.970185  Shares: 0 (0 orphan, 0 dead) Stale rate: ??? Efficiency: ??? Current payout: 0.0970 BTC
2012-05-09 18:28:27.970229  Pool: 357GH/s Stale rate: 10.9% Expected time to block: 5.8 hours
2012-05-09 18:28:31.016919 > ########################################
2012-05-09 18:28:31.017079 > >>> WARNING: A MAJORITY OF SHARES CONTAIN A VOTE FOR AN UNSUPPORTED SHARE IMPLEMENTATION! (v2 with 86% support)
2012-05-09 18:28:31.017137 > >>> An upgrade is likely necessary. Check http://p2pool.forre.st/ for more information.
2012-05-09 18:28:31.017192 > ########################################
2012-05-09 18:28:31.017240 P2Pool: 19107 shares in chain (19111 verified/19111 total) Peers: 11 (0 incoming)
2012-05-09 18:28:31.017279  Local: 1421MH/s in last 3.6 minutes Local dead on arrival: ~7.0% (3-16%) Expected time to share: 40.6 minutes
2012-05-09 18:28:31.017319  Shares: 0 (0 orphan, 0 dead) Stale rate: ??? Efficiency: ??? Current payout: 0.0970 BTC
2012-05-09 18:28:31.017363  Pool: 357GH/s Stale rate: 10.9% Expected time to block: 5.8 hours
2012-05-09 18:28:34.061658 > ########################################
2012-05-09 18:28:34.061830 > >>> WARNING: A MAJORITY OF SHARES CONTAIN A VOTE FOR AN UNSUPPORTED SHARE IMPLEMENTATION! (v2 with 86% support)
2012-05-09 18:28:34.061922 > >>> An upgrade is likely necessary. Check http://p2pool.forre.st/ for more information.

I am 100% sure that I'm not running an older version. My OS is latest 64-bit Debian.

[forrestv]

vindimy, what does run_p2pool.py --version print?

[P4man]

I got a question; is there a way to determine if a pool+worker combo is a p2pool worker using cgminer (api) or some other way?

IOW, if I give you an IP+port, workername and password, could you tell if its p2pool or a regular pool?

[twmz]

I got a question; is there a way to determine if a pool+worker combo is a p2pool worker using cgminer (api) or some other way?

IOW, if I give you an IP+port, workername and password, could you tell if its p2pool or a regular pool?

Without connecting to it?  I might guess it was p2pool if the port was 9332 since that is not commonly used for other bitcoin pools.

If I can connect to it, then p2pool includes a "X-Is-P2Pool: true" HTTP header in its response to a getwork request.

[vindimy]

vindimy, what does run_p2pool.py --version print?

Code:

dv@lax1:/opt/p2$ python run_p2pool.py --version
/opt/p2pool/forrestv-p2pool-eb0b203/p2pool/util/pack.py:187: DeprecationWarning: object.__new__() takes no parameters
  return Type.__new__(cls, bits, endianness)
eb0b203

dv@lax1:/opt/p2$ python --version
Python 2.6.6

dv@lax1:/opt/p2pool$ ll
total 680
-rw-r--r-- 1 dv   dv   352751 Mar 24 01:13 forrestv-p2pool-0.10.3-0-g71ef0aa.tar.gz
-rw-r--r-- 1 root root 318393 May  9 18:16 forrestv-p2pool-0.11.1-0-gd514a6c.tar.gz
drwxrwxr-x 8 root root   4096 May  3 07:28 forrestv-p2pool-04283e9/
drwxr-xr-x 9 dv   dv     4096 Mar 24 01:19 forrestv-p2pool-eb0b203/

Now I see that it's still running old version of p2pool. And I know why. I updates my symlink to point to new installation, but I didn't chdir out-and-in of the symlinked dir. After doing that I get:

Code:

dv@lax1:/opt/p2$ python run_p2pool.py --version
/opt/p2pool/git/p2pool/p2pool/util/pack.py:186: DeprecationWarning: object.__new__() takes no parameters
  return Type.__new__(cls, bits, endianness)
0.11.1-27-g245da11

All is well now. Thanks forrestv! Coins going your way.

EDIT:

When I started cgminer again, I got this exception. Not sure if important.

Code:

2012-05-09 21:58:32.480639
2012-05-09 21:58:38.464645
2012-05-09 21:58:38.464793 GOT BLOCK FROM PEER! Passing to bitcoind! 6be723ad bitcoin: http://blockexplorer.com/block/000000000000084045d435e306638d746ecd67438a6426e15c4aca0b6be723ad
2012-05-09 21:58:38.464845
2012-05-09 21:58:57.475813 Requesting parent share 84b554c3 from 72.14.191.28:9333
2012-05-09 21:58:57.476478 ... done processing 1001 shares. New: 1001 Have: 12028/~17280
2012-05-09 21:58:57.476928 > Watchdog timer went off at:
2012-05-09 21:58:57.476987 >   File "run_p2pool.py", line 5, in <module>
2012-05-09 21:58:57.477032 >     main.run()
2012-05-09 21:58:57.477077 >   File "/opt/p2pool/git/p2pool/p2pool/main.py", line 1073, in run
2012-05-09 21:58:57.477119 >     reactor.run()
2012-05-09 21:58:57.477160 >   File "/usr/lib/python2.6/dist-packages/twisted/internet/base.py", line 1165, in run
2012-05-09 21:58:57.477205 >     self.mainLoop()
2012-05-09 21:58:57.477246 >   File "/usr/lib/python2.6/dist-packages/twisted/internet/base.py", line 1177, in mainLoop
2012-05-09 21:58:57.477289 >     self.doIteration(t)
2012-05-09 21:58:57.477331 >   File "/usr/lib/python2.6/dist-packages/twisted/internet/selectreactor.py", line 140, in doSelect
2012-05-09 21:58:57.477374 >     _logrun(selectable, _drdw, selectable, method, dict)
2012-05-09 21:58:57.477638 >   File "/usr/lib/python2.6/dist-packages/twisted/python/log.py", line 84, in callWithLogger
2012-05-09 21:58:57.477748 >     return callWithContext({"system": lp}, func, *args, **kw)
2012-05-09 21:58:57.477794 >   File "/usr/lib/python2.6/dist-packages/twisted/python/log.py", line 69, in callWithContext
2012-05-09 21:58:57.477838 >     return context.call({ILogContext: newCtx}, func, *args, **kw)
2012-05-09 21:58:57.477880 >   File "/usr/lib/python2.6/dist-packages/twisted/python/context.py", line 59, in callWithContext
2012-05-09 21:58:57.477923 >     return self.currentContext().callWithContext(ctx, func, *args, **kw)
2012-05-09 21:58:57.477964 >   File "/usr/lib/python2.6/dist-packages/twisted/python/context.py", line 37, in callWithContext
2012-05-09 21:58:57.478006 >     return func(*args,**kw)
2012-05-09 21:58:57.478230 >   File "/usr/lib/python2.6/dist-packages/twisted/internet/selectreactor.py", line 146, in _doReadOrWrite
2012-05-09 21:58:57.478321 >     why = getattr(selectable, method)()
2012-05-09 21:58:57.478367 >   File "/usr/lib/python2.6/dist-packages/twisted/internet/tcp.py", line 460, in doRead
2012-05-09 21:58:57.478413 >     return self.protocol.dataReceived(data)
2012-05-09 21:58:57.478456 >   File "/opt/p2pool/git/p2pool/p2pool/p2p.py", line 57, in new_dataReceived
2012-05-09 21:58:57.478499 >     old_dataReceived(data)
2012-05-09 21:58:57.478539 >   File "/opt/p2pool/git/p2pool/p2pool/util/datachunker.py", line 40, in _DataChunker
2012-05-09 21:58:57.478581 >     wants = receiver.send(buf.get(wants))
2012-05-09 21:58:57.478625 >   File "/opt/p2pool/git/p2pool/p2pool/util/p2protocol.py", line 48, in dataReceiver
2012-05-09 21:58:57.478668 >     self.packetReceived(command, type_.unpack(payload))
2012-05-09 21:58:57.478710 >   File "/opt/p2pool/git/p2pool/p2pool/p2p.py", line 69, in packetReceived
2012-05-09 21:58:57.478752 >     p2protocol.Protocol.packetReceived(self, command, payload2)
2012-05-09 21:58:57.478794 >   File "/opt/p2pool/git/p2pool/p2pool/util/p2protocol.py", line 61, in packetReceived
2012-05-09 21:58:57.478836 >     handler(**payload2)
2012-05-09 21:58:57.478876 >   File "/opt/p2pool/git/p2pool/p2pool/p2p.py", line 205, in handle_shares
2012-05-09 21:58:57.478919 >     self.node.handle_shares([p2pool_data.load_share(share, self.node.net, self) for share in shares], self)
2012-05-09 21:58:57.478962 >   File "/opt/p2pool/git/p2pool/p2pool/main.py", line 265, in handle_shares
2012-05-09 21:58:57.479004 >     set_real_work2()
2012-05-09 21:58:57.479044 >   File "/opt/p2pool/git/p2pool/p2pool/main.py", line 183, in set_real_work2
2012-05-09 21:58:57.479086 >     best, desired = tracker.think(get_height_rel_highest, pre_current_work.value['previous_block'], pre_current_work.value['bits'])
2012-05-09 21:58:57.479165 >   File "/opt/p2pool/git/p2pool/p2pool/data.py", line 403, in think
2012-05-09 21:58:57.479208 >     if not self.attempt_verify(share):
2012-05-09 21:58:57.479250 >   File "/opt/p2pool/git/p2pool/p2pool/data.py", line 353, in attempt_verify
2012-05-09 21:58:57.479291 >     share.check(self)
2012-05-09 21:58:57.479331 >   File "/opt/p2pool/git/p2pool/p2pool/data.py", line 248, in check
2012-05-09 21:58:57.479374 >     share_info, gentx = self.generate_transaction(tracker, self.share_info['share_data'], self.header['bits'].target, self.share_info['timestamp'], self.share_info['bits'].target, self.common['ref_merkle_link'], self.net)
2012-05-09 21:58:57.479424 >   File "/opt/p2pool/git/p2pool/p2pool/data.py", line 134, in generate_transaction
2012-05-09 21:58:57.479466 >     65535*net.SPREAD*bitcoin_data.target_to_average_attempts(block_target),
2012-05-09 21:58:57.479507 >   File "/opt/p2pool/git/p2pool/p2pool/util/skiplist.py", line 55, in __call__
2012-05-09 21:58:57.479550 >     updates[x] = updates[x][0], self.combine_deltas(updates[x][1], delta) if updates[x][1] is not None else delta
2012-05-09 21:58:57.479594 >   File "/opt/p2pool/git/p2pool/p2pool/data.py", line 303, in combine_deltas
2012-05-09 21:58:57.479636 >     return share_count1 + share_count2, math.add_dicts(weights1, weights2), total_weight1 + total_weight2, total_donation_weight1 + total_donation_weight2
2012-05-09 21:58:57.479680 >   File "/opt/p2pool/git/p2pool/p2pool/util/math.py", line 69, in add_dicts
2012-05-09 21:58:57.479721 >     return dict((k, v) for k, v in res.iteritems() if v != zero)
2012-05-09 21:58:57.479762 >   File "/opt/p2pool/git/p2pool/p2pool/util/math.py", line 69, in <genexpr>
2012-05-09 21:58:57.479803 >     return dict((k, v) for k, v in res.iteritems() if v != zero)
2012-05-09 21:58:57.479844 >   File "/opt/p2pool/git/p2pool/p2pool/main.py", line 740, in <lambda>
2012-05-09 21:58:57.479886 >     sys.stderr.write, 'Watchdog timer went off at:\n' + ''.join(traceback.format_stack())
2012-05-09 21:58:57.505188 P2Pool: 12027 shares in chain (3387 verified/12028 total) Peers: 6 (0 incoming)
2012-05-09 21:58:57.505292  Local: 728MH/s in last 58.9 seconds Local dead on arrival: ~100.0% (72-100%) Expected time to share: 1.2 hours

[Frizz23]

How can I verify that merged mining is working for me?

I haven't found a single NMC block since I use p2pool (since January).

[kano]

How can I verify that merged mining is working for me?

I haven't found a single NMC block since I use p2pool (since January).

One way to know: If you find a BTC block then you should also find an NMC block
NMC blocks are expected to be more often, but certainly not less often than a BTC block
There are conditions where that wouldn't be true, but those conditions are not likely to ever happen.

[Smoovious]

How can I verify that merged mining is working for me?

I haven't found a single NMC block since I use p2pool (since January).

One way to know: If you find a BTC block then you should also find an NMC block
NMC blocks are expected to be more often, but certainly not less often than a BTC block
There are conditions where that wouldn't be true, but those conditions are not likely to ever happen.

...but it has to be you finding the BTC block, not the pool.

Try adding this to your namecoin.conf file: printtoconsole=1

Then you should be able to see P2Pool polling namecoind. Also, if P2Pool wasn't able to communicate with namecoind, you'd see a lot of error messages on the P2Pool end.

I don't know if namecoin has a testnet set up, but try switching to that for a little while too. It might have a much lower difficulty to test with.

-- Smoov

[Isokivi]

Is there any way to identify which rig, or even better which gpu is responsible for my orphans/stales ?

[twmz]

Is there any way to identify which rig, or even better which gpu is responsible for my orphans/stales ?

Use a different username for each rig or gpu and then look at the graphs.  There will be a separate graph for each username and you can see individual stale statistics for each.

[DeathAndTaxes]

Is there any way to identify which rig, or even better which gpu is responsible for my orphans/stales ?

Use a different username for each rig or gpu and then look at the graphs.  There will be a separate graph for each username and you can see individual stale statistics for each.

Good "p2pool pro-tip"!

That will only show DOA though.

Orphans (not to be confused with DOA) really aren't GPU/rig dependent.  All your rigs are sending shares to the p2pool instance.  If the p2pool instance finds the shares bad or old then it will be marked DOA.  If they aren't DOA they are all submitted to the network.  The orphan rate should be roughly the same for all GPUs/rigs.  Any rig/miner/GPU problems should show up in increased DOA rate and as twmz indicated using a seperate username for each rig can help spot that.

Also I would recommend ANUBIS (search the forum).  I nice web monitor to keep track of multiple GPU and multiple rigs by using the cgminer API.  Easy to spot a rig with above average rejects (DOAs) or communication problems (as well as overhot cards, or below avg hashrates).  If you want to do any per GPU analysis I recommend using static 1 diff shares locally.

Code:

user="username/A+B"

username is simply the identifier
/A is used to look for and submit higher diff shares (allows huge miners to helps smaller miners by taking on more variance)
+B is used to control the local share diff.

TL/DR
DOA = share is "bad" (it could be stale or it could just be invalid garbage) by the time YOUR p2pool node gets it.
Oprhan = your p2pool node saw the share as good but it lost the "race" w/ a competing share.
Stales = orphans + DOA.

[twmz]

That will only show DOA though.

Sorry, you are right.  I shouldn't answer questions early in the morning right after I wake up.

[Frizz23]

Try adding ...

Just wanted to say thanks! This really is a very response group

[coretechs]

I recently upgraded some of my GPU miners to cgminer 2.4 and the latest p2pool build.  After a day or so I noticed that my overall hash rate was significantly lower than it had been in the past.  I did some testing and found that lowering intensity even further from 7 to (5) 6 on some of my rigs resulted in a ~10%-15% improvement in the hash rate reported by p2pool (but lower in cgminer).  I just thought I'd mention it in case anyone else had a similar experience.  I suspect it is mostly due to upgrading cgminer since I was using a much older version.

The graphs in p2pool were very helpful in identifying trends between my rigs.  

edit - ran a bit longer to rule out some more variance and determined I may have been a bit optimistic with my initial findings, settled on -I 6

[bitpop]

wow never knew about printtoconsole=1
no more stupid blank dos boxes

[Frizz23]

Try adding this to your namecoin.conf file: printtoconsole=1

Then you should be able to see P2Pool polling namecoind.

What messages am I looking for in the namecoind logs?

[Smoovious]

Try adding this to your namecoin.conf file: printtoconsole=1

Then you should be able to see P2Pool polling namecoind.

What messages am I looking for in the namecoind logs?

ok, well, first off, in your P2Pool logs, if you are seeing lines like
2012-05-11 05:36:14.828000 > Error while calling merged getauxblock:
2012-05-11 05:36:14.831000 > Traceback (most recent call last):
2012-05-11 05:36:14.833000 > Failure: twisted.internet.error.ConnectionRefusedError: Connection was refused by other side: 10061: No connection could be made because the target machine actively refuse
d it..
then P2Pool isn't communicating with one of your daemons. Won't say which one, but this comes up when P2Pool doesn't find a daemon at the URL you supplied, nothing is answering the port.

Also, if NMC is the only one you're merge mining with, and you're seeing "Got new merged mining work!" messages, then you should be in good shape.

As for the option above, printtoconsole, I noticed different daemons giving more or less info than bitcoind does, and it looks like with that set, it doesn't give enough, which was my bad for not checking that specific daemon. Looking with the '-debug' command line option enabled, I'm not seeing anything obvious from the namecoind end showing P2P's communication, so I wasn't helpful there, but the P2P messages I've mentioned in this reply should help out.

-- Smoov

[Frizz23]

Also, if NMC is the only one you're merge mining with, ...

Is it possible to add more than just NMC to do merged mining? I thought the only possible option is BTC + NMC. No?

[ChanceCoats123]

  I still haven't set up merged mining with P2pool.

[Smoovious]

Also, if NMC is the only one you're merge mining with, ...

Is it possible to add more than just NMC to do merged mining? I thought the only possible option is BTC + NMC. No?

Yes, just add another --merged with another URL for the daemon.

You just have to make sure the block chains are compatable.

BTC and LTC for example, aren't compatable.

BTC, NMC, GG(now dead), are compatable, as well as a few others.

(for compatability, basically means, the same miner works for both)

-- Smoov