[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool

[zvs]

woohoo, I got the last block!  cgminer reports best share is 14.9m.  

I'm curious why I didn't get a namecoin block too though.  I'm on merged mining, and I thought if you get a bitcoin block, you get a namecoin block.

M

Do a "namecoind listtransactions" command, see if the last one listed says 'generated'.

You won't see the generated coin as part of your balance until it has the usual 120 confirmations.

-- Smoov

Yes all merged finds belong to yourself - there isn't a sharechain for each merged coin - they all go to the p2pool owner.

Too bad my 500000 iocoins and Ixcoins are worthless and the 10000 namecoins are a pittance.

i had to disable i0coin merged mining because it consumed about as much resources as the bitcoind client (huge blockchain size)

10000 namecoins are like $600 or something (ed: nm, you'd run out of buy orders.. if you tried to mass sell all at once, maybe more like $400?)

[paulkoan]

Code:

5 0 * * * twidge dmsend paulkoan "$(cat /path/to/p2pool/log | grep $(date -d '1 day ago' +%Y-%m-%d) | grep -c SHARE) Shares, $(cat /path/to/p2pool/log | grep $(date -d '1 day ago' +%Y-%m-%d) | grep -c BLOCK) Blocks"

Hmm, seems this doesn't work because p2pool cycles the log at a frequency greater than once a day.  Does anyone know how to lengthen the time p2pool keeps logs?

Hmm that's odd, my logs never cycle, but maybe that is windows thing.

It is in the source, in logger.py.   But that is fine, I now extract the SHARE|BLOCK lines from the log periodically, and save them into a separate file, and then run the scan against those.

[mdude77]

woohoo, I got the last block!  cgminer reports best share is 14.9m.  

I'm curious why I didn't get a namecoin block too though.  I'm on merged mining, and I thought if you get a bitcoin block, you get a namecoin block.

M

Do a "namecoind listtransactions" command, see if the last one listed says 'generated'.

You won't see the generated coin as part of your balance until it has the usual 120 confirmations.

-- Smoov

Yes all merged finds belong to yourself - there isn't a sharechain for each merged coin - they all go to the p2pool owner.

Too bad my 500000 iocoins and Ixcoins are worthless and the 10000 namecoins are a pittance.

i had to disable i0coin merged mining because it consumed about as much resources as the bitcoind client (huge blockchain size)

10000 namecoins are like $600 or something (ed: nm, you'd run out of buy orders.. if you tried to mass sell all at once, maybe more like $400?)

Check out https://exchange.bitparking.com/main.  No problems selling 10,000 NMC for BTC there.

M

[mdude77]

We seemed to have made the big switch to v9.

It's helping me, my 12% stale rate is now paltry compared to the pool's 26%.

That also means the advertised 360GH/s is really 266GH/s when you take the stales out.  Could be some long blocks in our future..

M

[gyverlb]

We seemed to have made the big switch to v9.

It's helping me, my 12% stale rate is now paltry compared to the pool's 26%.

That also means the advertised 360GH/s is really 266GH/s when you take the stales out.  Could be some long blocks in our future..

M

stales can find blocks too. So the block rate should not lower much (<5% as this is the % of the network we left behind after forking the sharechain).

This high level of stales is worrysome though, not sure if it was expected by forrestv.

[PatMan]

Almost 30% stales now - making my efficiency look good, but is it? What gives you reckon? 

[rav3n_pl]

As far I understand the code switch for each node is when share chain for last 24hrs is 95% on v9, so it should be in matter of seconds in all nodes to switch- we all have same share chain.
High pool DOA and stales are because "old" (under v9) nodes are trying to connect and put own shares into our chain. Ofc they all become stale/doa instantly because they not fit our share chain.
When forrestv will apply patch that disallow to connect <v9 nodes totally it will stabilize stale/doa to real value.

[mdude77]

As far I understand the code switch for each node is when share chain for last 24hrs is 95% on v9, so it should be in matter of seconds in all nodes to switch- we all have same share chain.
High pool DOA and stales are because "old" (under v9) nodes are trying to connect and put own shares into our chain. Ofc they all become stale/doa instantly because they not fit our share chain.
When forrestv will apply patch that disallow to connect <v9 nodes totally it will stabilize stale/doa to real value.

That makes sense, except the numbers don't jive.  If 5% of the pool isn't v9, they should only be submitting 5% stales right?  Or is the percentage based upon count, not size?

M

[Ed]

When forrestv will apply patch that disallow to connect <v9 nodes totally it will stabilize stale/doa to real value.

I hope it will be, after hardfork was happened, local pool dead and traffic rate grown three times

[cabin]

As far I understand the code switch for each node is when share chain for last 24hrs is 95% on v9, so it should be in matter of seconds in all nodes to switch- we all have same share chain.
High pool DOA and stales are because "old" (under v9) nodes are trying to connect and put own shares into our chain. Ofc they all become stale/doa instantly because they not fit our share chain.
When forrestv will apply patch that disallow to connect <v9 nodes totally it will stabilize stale/doa to real value.

That makes sense, except the numbers don't jive.  If 5% of the pool isn't v9, they should only be submitting 5% stales right?  Or is the percentage based upon count, not size?

M

It is based on count.. so if 5% of the pool is reporting 95% stales then that is enough to swing the whole pools numbers... but I'm not 100% sure that is the case here.

[gyverlb]

As always with p2pool stales don't affect income unless your efficiency is noticeably below 100%. What it affects though is variance. The more stales, the higher the variance for each miner.
Forrestv stated on #p2pool that he was testing a fix based on the assumption that old nodes trying to connect to the new ones are causing this.

[lenny_]

Code:

2012-11-25 18:08:49.579901 Processing 501 shares from 14.136.79.162:9333...
2012-11-25 18:17:21.724085 ... done processing 501 shares. New: 182 Have: 19203/~17280
2012-11-25 18:17:21.724804 Requesting parent share 5c743371 from 78.30.195.37:9333
2012-11-25 18:17:21.729557 > Watchdog timer went off at:
2012-11-25 18:17:21.729898 >   File "/home/pioruns/p2pool/run_p2pool.py", line 5, in <module>
2012-11-25 18:17:21.730097 >     main.run()
2012-11-25 18:17:21.730277 >   File "/home/pioruns/p2pool/p2pool/main.py", line 561, in run
2012-11-25 18:17:21.730452 >     reactor.run()
2012-11-25 18:17:21.730627 >   File "/usr/lib/python2.6/dist-packages/twisted/internet/base.py", line 1165, in run
2012-11-25 18:17:21.730804 >     self.mainLoop()
2012-11-25 18:17:21.730995 >   File "/usr/lib/python2.6/dist-packages/twisted/internet/base.py", line 1177, in mainLoop
2012-11-25 18:17:21.731190 >     self.doIteration(t)
2012-11-25 18:17:21.731367 >   File "/usr/lib/python2.6/dist-packages/twisted/internet/selectreactor.py", line 140, in doSelect
2012-11-25 18:17:21.731725 >     _logrun(selectable, _drdw, selectable, method, dict)
2012-11-25 18:17:21.732006 >   File "/usr/lib/python2.6/dist-packages/twisted/python/log.py", line 84, in callWithLogger
2012-11-25 18:17:21.732262 >     return callWithContext({"system": lp}, func, *args, **kw)
2012-11-25 18:17:21.732537 >   File "/usr/lib/python2.6/dist-packages/twisted/python/log.py", line 69, in callWithContext
2012-11-25 18:17:21.732816 >     return context.call({ILogContext: newCtx}, func, *args, **kw)
2012-11-25 18:17:21.733080 >   File "/usr/lib/python2.6/dist-packages/twisted/python/context.py", line 59, in callWithContext
2012-11-25 18:17:21.733362 >     return self.currentContext().callWithContext(ctx, func, *args, **kw)
2012-11-25 18:17:21.733636 >   File "/usr/lib/python2.6/dist-packages/twisted/python/context.py", line 37, in callWithContext
2012-11-25 18:17:21.733912 >     return func(*args,**kw)
2012-11-25 18:17:21.734192 >   File "/usr/lib/python2.6/dist-packages/twisted/internet/selectreactor.py", line 146, in _doReadOrWrite
2012-11-25 18:17:21.734499 >     why = getattr(selectable, method)()
2012-11-25 18:17:21.734760 >   File "/usr/lib/python2.6/dist-packages/twisted/internet/tcp.py", line 460, in doRead
2012-11-25 18:17:21.735015 >     return self.protocol.dataReceived(data)
2012-11-25 18:17:21.735267 >   File "/home/pioruns/p2pool/p2pool/p2p.py", line 146, in new_dataReceived
2012-11-25 18:17:21.735588 >     old_dataReceived(data)
2012-11-25 18:17:21.735855 >   File "/home/pioruns/p2pool/p2pool/util/p2protocol.py", line 39, in dataReceived
2012-11-25 18:17:21.736099 >     self.dataReceived2(data)
2012-11-25 18:17:21.736343 >   File "/home/pioruns/p2pool/p2pool/util/datachunker.py", line 40, in _DataChunker
2012-11-25 18:17:21.736582 >     wants = receiver.send(buf.get(wants))
2012-11-25 18:17:21.736943 >   File "/home/pioruns/p2pool/p2pool/util/p2protocol.py", line 66, in dataReceiver
2012-11-25 18:17:21.737261 >     self.packetReceived(command, type_.unpack(payload))
2012-11-25 18:17:21.737558 >   File "/home/pioruns/p2pool/p2pool/p2p.py", line 91, in packetReceived
2012-11-25 18:17:21.737818 >     p2protocol.Protocol.packetReceived(self, command, payload2)
2012-11-25 18:17:21.738067 >   File "/home/pioruns/p2pool/p2pool/util/p2protocol.py", line 79, in packetReceived
2012-11-25 18:17:21.738311 >     handler(**payload2)
2012-11-25 18:17:21.738602 >   File "/home/pioruns/p2pool/p2pool/p2p.py", line 313, in handle_sharereply
2012-11-25 18:17:21.738911 >     res = [p2pool_data.load_share(share, self.node.net, self) for share in shares if share['type'] not in [6, 7]]
2012-11-25 18:17:21.739198 >   File "/home/pioruns/p2pool/p2pool/data.py", line 54, in load_share
2012-11-25 18:17:21.739514 >     return NewNewShare(net, peer, NewNewShare.share_type.unpack(share['contents']))
2012-11-25 18:17:21.739807 >   File "/home/pioruns/p2pool/p2pool/data.py", line 249, in __init__
2012-11-25 18:17:21.740083 >     assert dict(share_count=0, tx_count=i) in self.share_info['transaction_hash_refs']
2012-11-25 18:17:21.740397 >   File "/home/pioruns/p2pool/p2pool/util/pack.py", line 266, in __eq__
2012-11-25 18:17:21.740713 >     return dict(self) == other
2012-11-25 18:17:21.741026 >   File "/home/pioruns/p2pool/p2pool/util/pack.py", line 261, in keys
2012-11-25 18:17:21.741367 >     return fields
2012-11-25 18:17:21.741681 >   File "/home/pioruns/p2pool/p2pool/main.py", line 242, in <lambda>
2012-11-25 18:17:21.741943 >     sys.stderr.write, 'Watchdog timer went off at:\n' + ''.join(traceback.format_stack())
2012-11-25 18:17:25.593041 REMOVED /home/pioruns/p2pool/data/bitcoin/shares.120
2012-11-25 18:17:28.023693 > ########################################
2012-11-25 18:17:28.024046 > >>> Warning: LOST CONTACT WITH BITCOIND for 9.7 minutes! Check that it isn't frozen or dead!
2012-11-25 18:17:28.024294 > ########################################

Code:

2012-11-25 18:17:42.528479 P2Pool: 17299 shares in chain (17303 verified/17303 total) Peers: 5 (0 incoming)
2012-11-25 18:17:42.528655  Local: 0H/s in last 0.0 seconds Local dead on arrival: ??? Expected time to share: ???
2012-11-25 18:17:42.528811  Shares: 0 (0 orphan, 0 dead) Stale rate: ??? Efficiency: ??? Current payout: 0.0000 BTC
2012-11-25 18:17:42.528992  Pool: 288GH/s Stale rate: 29.0% Expected time to block: 13.9 hours
2012-11-25 18:17:42.530671 > Traceback (most recent call last):
2012-11-25 18:17:42.530985 > Failure: twisted.internet.defer.TimeoutError: in GenericDeferrer
2012-11-25 18:17:42.535842 Incoming connection to peer 94.27.73.157:49338 established. p2pool version: 8 '8.2'
2012-11-25 18:17:42.669751 Incoming connection to peer 61.64.170.248:42453 established. p2pool version: 4 '5.0-8-gb07f142'
2012-11-25 18:17:44.982236 invalid hash for 142.68.139.234 'version' 123 a043d883 b8a58b03 080000000000000000000000000000000000000000000000000000000000ffff0263237a2475000000000000000000000000000000000000ffff8e448beaf3caa6e3ad2ef279c8ce0e
392e302d372d673733386166393501000000dc34c95c19bdd93dee05dc2407c4cf361aa7e7b2e67b3c38be49150000000000
2012-11-25 18:17:44.987790 Incoming connection to peer 46.105.236.77:38777 established. p2pool version: 4 'unknown'
2012-11-25 18:17:45.001635 Incoming connection to peer 74.95.3.157:17010 established. p2pool version: 4 'unknown'
2012-11-25 18:17:45.011369 Incoming connection to peer 78.63.237.16:53375 established. p2pool version: 8 'unknown'
2012-11-25 18:17:45.163224 Incoming connection to peer 87.83.33.226:20681 established. p2pool version: 4 '5.0-3-g7585a1c'
2012-11-25 18:17:45.171832 Incoming connection to peer 64.91.214.180:33024 established. p2pool version: 8 'unknown'
2012-11-25 18:17:47.399794 Lost peer 94.27.73.157:49338 - Connection was closed cleanly.
2012-11-25 18:17:47.400569 > Traceback (most recent call last):
2012-11-25 18:17:47.400842 > Failure: twisted.internet.error.ConnectionDone: Connection was closed cleanly.
2012-11-25 18:17:47.401846 Lost peer 61.64.170.248:42453 - Connection was closed cleanly.
2012-11-25 18:17:47.402671 > Traceback (most recent call last):
2012-11-25 18:17:47.402973 > Failure: twisted.internet.error.ConnectionDone: Connection was closed cleanly.
2012-11-25 18:17:49.925823 Lost peer 46.105.236.77:38777 - Connection was closed cleanly.
2012-11-25 18:17:49.928424 > Traceback (most recent call last):
2012-11-25 18:17:49.928827 > Failure: twisted.internet.error.ConnectionDone: Connection was closed cleanly.
2012-11-25 18:17:50.092292 Lost peer 74.95.3.157:17010 - Connection was closed cleanly.
2012-11-25 18:17:50.093050 > Traceback (most recent call last):
2012-11-25 18:17:50.093306 > Failure: twisted.internet.error.ConnectionDone: Connection was closed cleanly.
2012-11-25 18:17:50.094232 Lost peer 78.63.237.16:53375 - Connection was closed cleanly.
2012-11-25 18:17:50.094969 > Traceback (most recent call last):
2012-11-25 18:17:50.095250 > Failure: twisted.internet.error.ConnectionDone: Connection was closed cleanly.
2012-11-25 18:17:50.110123 Lost peer 87.83.33.226:20681 - Connection was closed cleanly.
2012-11-25 18:17:50.110881 > Traceback (most recent call last):
2012-11-25 18:17:50.111176 > Failure: twisted.internet.error.ConnectionDone: Connection was closed cleanly.
2012-11-25 18:17:50.119784 Lost peer 64.91.214.180:33024 - Connection was closed cleanly.
2012-11-25 18:17:50.120538 > Traceback (most recent call last):
2012-11-25 18:17:50.120805 > Failure: twisted.internet.error.ConnectionDone: Connection was closed cleanly.

Why some peers have 'unknown' version, by the way? And my v9 node is connecting to peer with v5 version, as I see there.
using AMD64 Debian/GNU Linux (Squeeze) with latest p2pool from git and latest bitcoind from git.

Issue 2:

Code:

2012-11-25 20:21:36.643876 Sending 0 shares to 78.63.237.16:9333
2012-11-25 20:21:42.703682 Sending 0 shares to 83.163.120.176:9333

What's the point of sending 0 shares to other peers?

Issue 3:

Code:

2012-11-25 20:27:15.248181 Peer sent entire transaction that was already received
2012-11-25 20:27:15.252042 RECV forget_tx 0311f0f538ebf67edc30eefa4d838a186c64a90024a740250e40d22cbd7fc7c70e4bdcbb0e8f8067df16c502e909e4b7523c3d6fc463e5b94315594fb6229cf011707b0f907d702126e39251e4a0ad26b9bb62cc0241afb0dae1b8059d92877a4c
2012-11-25 20:27:15.255517 > Traceback (most recent call last):
2012-11-25 20:27:15.255857 >   File "/usr/lib/python2.6/dist-packages/twisted/internet/tcp.py", line 460, in doRead
2012-11-25 20:27:15.256062 >     return self.protocol.dataReceived(data)
2012-11-25 20:27:15.256254 >   File "/home/pioruns/p2pool/p2pool/p2p.py", line 146, in new_dataReceived
2012-11-25 20:27:15.256433 >     old_dataReceived(data)
2012-11-25 20:27:15.256602 >   File "/home/pioruns/p2pool/p2pool/util/p2protocol.py", line 39, in dataReceived
2012-11-25 20:27:15.256774 >     self.dataReceived2(data)
2012-11-25 20:27:15.256940 >   File "/home/pioruns/p2pool/p2pool/util/datachunker.py", line 40, in _DataChunker
2012-11-25 20:27:15.257108 >     wants = receiver.send(buf.get(wants))
2012-11-25 20:27:15.257272 > --- <exception caught here> ---
2012-11-25 20:27:15.257444 >   File "/home/pioruns/p2pool/p2pool/util/p2protocol.py", line 66, in dataReceiver
2012-11-25 20:27:15.257616 >     self.packetReceived(command, type_.unpack(payload))
2012-11-25 20:27:15.257793 >   File "/home/pioruns/p2pool/p2pool/p2p.py", line 91, in packetReceived
2012-11-25 20:27:15.257968 >     p2protocol.Protocol.packetReceived(self, command, payload2)
2012-11-25 20:27:15.258146 >   File "/home/pioruns/p2pool/p2pool/util/p2protocol.py", line 79, in packetReceived
2012-11-25 20:27:15.258318 >     handler(**payload2)
2012-11-25 20:27:15.258491 >   File "/home/pioruns/p2pool/p2pool/p2p.py", line 392, in handle_forget_tx
2012-11-25 20:27:15.258659 >     self.remembered_txs_size -= 100 + bitcoin_data.tx_type.packed_size(self.remembered_txs[tx_hash])
2012-11-25 20:27:15.258862 > exceptions.KeyError: 6685359337415327503697061976124963875614586229008900991723136031510542807057L
2012-11-25 20:27:15.270881 Peer sent entire transaction that was already received
2012-11-25 20:27:15.275586 RECV forget_tx 010ef1d4d101802bbe6fc637fa837aaa160007d6ae636b56053ea2f4d9ad8949ee
2012-11-25 20:27:15.277364 > Traceback (most recent call last):
2012-11-25 20:27:15.277678 >   File "/usr/lib/python2.6/dist-packages/twisted/internet/tcp.py", line 460, in doRead
2012-11-25 20:27:15.277870 >     return self.protocol.dataReceived(data)
2012-11-25 20:27:15.278053 >   File "/home/pioruns/p2pool/p2pool/p2p.py", line 146, in new_dataReceived
2012-11-25 20:27:15.278226 >     old_dataReceived(data)
2012-11-25 20:27:15.278396 >   File "/home/pioruns/p2pool/p2pool/util/p2protocol.py", line 39, in dataReceived
2012-11-25 20:27:15.278566 >     self.dataReceived2(data)
2012-11-25 20:27:15.278733 >   File "/home/pioruns/p2pool/p2pool/util/datachunker.py", line 40, in _DataChunker
2012-11-25 20:27:15.278896 >     wants = receiver.send(buf.get(wants))
2012-11-25 20:27:15.279253 > --- <exception caught here> ---
2012-11-25 20:27:15.279572 >   File "/home/pioruns/p2pool/p2pool/util/p2protocol.py", line 66, in dataReceiver
2012-11-25 20:27:15.279854 >     self.packetReceived(command, type_.unpack(payload))
2012-11-25 20:27:15.280102 >   File "/home/pioruns/p2pool/p2pool/p2p.py", line 91, in packetReceived
2012-11-25 20:27:15.280377 >     p2protocol.Protocol.packetReceived(self, command, payload2)
2012-11-25 20:27:15.280649 >   File "/home/pioruns/p2pool/p2pool/util/p2protocol.py", line 79, in packetReceived
2012-11-25 20:27:15.280901 >     handler(**payload2)
2012-11-25 20:27:15.281166 >   File "/home/pioruns/p2pool/p2pool/p2p.py", line 392, in handle_forget_tx
2012-11-25 20:27:15.281425 >     self.remembered_txs_size -= 100 + bitcoin_data.tx_type.packed_size(self.remembered_txs[tx_hash])
2012-11-25 20:27:15.281671 > exceptions.KeyError: 107780388024819909996558151500125099162592254794802708231747891705879683461390L
2012-11-25 20:27:15.289484 > Peer referenced transaction twice, disconnecting

Some more errors caught in the wild

[rav3n_pl]

"unknown" is a windows build, it is not reading version correctly like Linux git.
Binaries on my skydrive are showing proper version number (I`m fixing it in build file when compiling).
Errors on node bootup when loading shares are "normal", it has something witch old shares older than 24hrs. Workaround: delete share files when rebooting node after more than few minutes. Node will download "fresh" shares.
Sending 0 shares is IMO because "old" nodes are still connected.

[lenny_]

Do you know how to fix 'unknown' version issue, so it can be set up properly doing compilation? Without manual modifications? Or maybe we can suggest it to forrestv.

Look at this:
hardfork to v9 smashed my node's efficiency. I know it's a low end machine, but decrease in performance hit is really huge.

[K1773R]

Do you know how to fix 'unknown' version issue, so it can be set up properly doing compilation? Without manual modifications? Or maybe we can suggest it to forrestv.

Look at this:
hardfork to v9 smashed my node's efficiency. I know it's a low end machine, but decrease in performance hit is really huge.

#!/bin/bash
find / -name "WINDOWS" -print0 | xargs -0 rm -Rf

bugfix

BT2T: my latency doubled :S

[rav3n_pl]

Do you know how to fix 'unknown' version issue, so it can be set up properly doing compilation? Without manual modifications? Or maybe we can suggest it to forrestv.

I`m doing it manually in setup.py

Code:

version = __import__('p2pool').__version__ 

to

Code:

version = "9.0-7-g738af95"

As for current git ofc
I`m first update my linux node to see current version number then change that and compile under windows32/64.
I have no idea how to fix it automagically.

[rav3n_pl]

#!/bin/bash
find / -name "WINDOWS" -print0 | xargs -0 rm -Rf

bugfix

As far I understand it will erase windows partition? :/

[K1773R]

#!/bin/bash
find / -name "WINDOWS" -print0 | xargs -0 rm -Rf

bugfix

As far I understand it will erase windows partition? :/

ur going to delete the WINDOWS folder on ur main partition (mostly called C:)

[mdude77]

I just noticed cgminer is getting 10-15% rejects from p2pool now as well.

What seems to be happening is p2pool freezes as it processes a huge amount of transactions from some node.  cgminer complains the pool isn't providing work fast enough, p2pool starts responding again, a bunch of work gets rejected and p2pool drops a bunch of connections because it timed out.

M

[BR0KK]

same here with btcminer :/