re: Punishing share for 'Block-stale detected!, etc.
I get a lot of orphans when this happens... because my bitcoind is too fast?
I went to several other pools (my other one in Florida, OVH in Canada, p2pool.org, some others) and they all had my share listed, which means that it was accepted and got orphaned later.
This occuring after several:
2013-06-09 21:34:35.132788 Skipping from block 4bc4be449a7de0c01cb29a9d661045a19393dea2cefc2e26b to block 9b14c16ea0d67eda77564adca4de0ac3e84b1efbd8cfb5adce!
2013-06-09 21:34:36.831062 Punishing share for 'Block-stale detected! 4bc4be449a7de0c01cb29a9d661045a19393dea2cefc2e26b < 9b14c16ea0d67eda77564adca4de0ac3e84b1efbd8cfb5adce'! Jumping from 2ba6238a to e8ee1640!
2013-06-09 21:34:36.917817 Punishing share for 'Block-stale detected! 4bc4be449a7de0c01cb29a9d661045a19393dea2cefc2e26b < 9b14c16ea0d67eda77564adca4de0ac3e84b1efbd8cfb5adce'! Jumping from 2ba6238a to e8ee1640!
2013-06-09 21:34:36.920728 Punishing share for 'Block-stale detected! 4bc4be449a7de0c01cb29a9d661045a19393dea2cefc2e26b < 9b14c16ea0d67eda77564adca4de0ac3e84b1efbd8cfb5adce'! Jumping from 2ba6238a to e8ee1640!
2013-06-09 21:34:40.064626 GOT SHARE! Brusthonin 32cdc8e4 prev e8ee1640 age 2.69s
... and then that share was orphaned. It seems to me like if I modified the source to turn off the whole 'punishing share' stuff, I'd get less orphans?
If I understand correctly, P2Pool doesn't want to build a share on top of one it knows to be stale (built on a previous Bitcoin block).
It may be because your bitcoind has better connections to the Bitcoin P2P network and learns of new blocks before others: if the majority of the network is slower it will not see the share as a "Block-stale" as you do and not punish it, refusing your share in the relatively rare case when you submit one just after this "Block-stale". You could lose a bit of income this way, but it should not be noticeable (this can only affect you during the ~5s a block needs to fully propagates the Bitcoin network, which is less than 1% of the block interval: in the worst case you shouldn't lose more than 0.5% of efficiency).
If you modified the source to turn off the 'punishing share', you would find your node at the opposite end of the spectrum: it would build on a potential stales and make orphans too.
You can find out if it's a problem by computing the number of shares you found just after a block that were orphaned and the number of shares that were accepted. If you have enough logs/hashrate you can get a meaningful sample (one hundred shares generated just after blocks would be adequate) and find if you get more orphans in this population than the orphans you get over the same time period. This would make sure there is some imbalance and I think we could even find out if reversing the logic would benefit you in your case (I'm a bit tired and going to bed, but I think it's simple probabilities).
OK, so I've been looking at my orphans for the last 10 days or so.... about 33% of them occur after a block solve.
Here's another example (editing out the multiple new work for workers messages, so there's just the one)
2013-06-21 22:31:58.822054 P2Pool: 17355 shares in chain (17359 verified/17359 total) Peers: 77 (16 incoming)
2013-06-21 22:31:58.822116 Local: 6968MH/s in last 10.0 minutes Local dead on arrival: ~2.0% (0-6%) Expected time to share: 11.7 minutes
2013-06-21 22:31:58.822136 Shares: 128 (2 orphan, 7 dead) Stale rate: ~7.0% (3-13%) Efficiency: ~107.9% (101-112%) Current payout: 0.2439 BTC
2013-06-21 22:31:58.822159 Pool: 582GH/s Stale rate: 13.9% Expected time to block: 1.6 days
2013-06-21 22:32:03.229133 Skipping from block 2d6aa3455150dc4ed11205a84269ec9d8248c74fba1760ca6c to block 35054271c485c0c1e7d0581d22196ef7a11b00ab308c9a20ad!
2013-06-21 22:32:03.273201 New work for worker! Difficulty: 15.000000 Share difficulty: 11481.054517 Total block value: 25.000000 BTC including 0 transactions
2013-06-21 22:32:03.873304 Punishing share for 'Block-stale detected! 2d6aa3455150dc4ed11205a84269ec9d8248c74fba1760ca6c < 35054271c485c0c1e7d0581d22196ef7a11b00ab308c9a20ad'! Jumping from ab17a876 to f4ee4862!
2013-06-21 22:32:04.180605 New work for worker! Difficulty: 15.000000 Share difficulty: 11414.202197 Total block value: 25.002500 BTC including 4 transactions
2013-06-21 22:32:05.143761 Punishing share for 'Block-stale detected! 2d6aa3455150dc4ed11205a84269ec9d8248c74fba1760ca6c < 35054271c485c0c1e7d0581d22196ef7a11b00ab308c9a20ad'! Jumping from ab17a876 to f4ee4862!
2013-06-21 22:32:05.146830 Punishing share for 'Block-stale detected! 2d6aa3455150dc4ed11205a84269ec9d8248c74fba1760ca6c < 35054271c485c0c1e7d0581d22196ef7a11b00ab308c9a20ad'! Jumping from ab17a876 to f4ee4862!
2013-06-21 22:32:06.846943 P2Pool: 17354 shares in chain (17359 verified/17359 total) Peers: 77 (16 incoming)
2013-06-21 22:32:06.847025 Local: 7054MH/s in last 10.0 minutes Local dead on arrival: ~2.0% (0-6%) Expected time to share: 11.5 minutes
2013-06-21 22:32:06.847051 Shares: 128 (2 orphan, 7 dead) Stale rate: ~7.0% (3-13%) Efficiency: ~107.7% (100-112%) Current payout: 0.2428 BTC
2013-06-21 22:32:06.847077 Pool: 581GH/s Stale rate: 13.7% Expected time to block: 1.7 days
2013-06-21 22:32:10.143508 Punishing share for 'Block-stale detected! 2d6aa3455150dc4ed11205a84269ec9d8248c74fba1760ca6c < 35054271c485c0c1e7d0581d22196ef7a11b00ab308c9a20ad'! Jumping from ab17a876 to f4ee4862!
2013-06-21 22:32:10.146594 Punishing share for 'Block-stale detected! 2d6aa3455150dc4ed11205a84269ec9d8248c74fba1760ca6c < 35054271c485c0c1e7d0581d22196ef7a11b00ab308c9a20ad'! Jumping from ab17a876 to f4ee4862!
2013-06-21 22:32:13.344146 New work for worker! Difficulty: 15.000000 Share difficulty: 11376.680741 Total block value: 25.002500 BTC including 4 transactions
2013-06-21 22:32:14.824266 New work for worker! Difficulty: 15.000000 Share difficulty: 11379.604632 Total block value: 25.002500 BTC including 4 transactions
2013-06-21 22:32:14.863067 P2Pool: 17356 shares in chain (17361 verified/17361 total) Peers: 77 (16 incoming)
2013-06-21 22:32:14.863146 Local: 6995MH/s in last 10.0 minutes Local dead on arrival: ~2.0% (0-6%) Expected time to share: 11.6 minutes
2013-06-21 22:32:14.863172 Shares: 128 (2 orphan, 7 dead) Stale rate: ~7.0% (3-13%) Efficiency: ~107.9% (101-112%) Current payout: 0.2428 BTC
2013-06-21 22:32:14.863199 Pool: 576GH/s Stale rate: 13.9% Expected time to block: 1.7 days
2013-06-21 22:32:15.729087 New work for worker! Difficulty: 15.000000 Share difficulty: 11319.620580 Total block value: 25.002500 BTC including 4 transactions
2013-06-21 22:32:17.027868 GOT SHARE! Drizztztzzztztzzzzztzz f3db321a prev 15a3746e age 1.30s
2013-06-21 22:32:17.139592 New work for worker! Difficulty: 15.000000 Share difficulty: 11282.391344 Total block value: 25.002500 BTC including 4 transactions
2013-06-21 22:32:22.883715 P2Pool: 17350 shares in chain (17354 verified/17354 total) Peers: 77 (16 incoming)
2013-06-21 22:32:22.883792 Local: 7138MH/s in last 10.0 minutes Local dead on arrival: ~2.0% (0-6%) Expected time to share: 11.4 minutes
2013-06-21 22:32:22.883817 Shares: 129 (2 orphan, 7 dead) Stale rate: ~7.0% (3-13%) Efficiency: ~108.3% (101-113%) Current payout: 0.2716 BTC
2013-06-21 22:32:22.883853 Pool: 591GH/s Stale rate: 14.1% Expected time to block: 1.6 days
2013-06-21 22:32:27.267897 New work for worker! Difficulty: 15.000000 Share difficulty: 11282.391344 Total block value: 25.001000 BTC including 2 transactions
2013-06-21 22:32:30.904216 P2Pool: 17350 shares in chain (17354 verified/17354 total) Peers: 77 (16 incoming)
2013-06-21 22:32:30.904297 Local: 6936MH/s in last 10.0 minutes Local dead on arrival: ~2.0% (0-6%) Expected time to share: 11.7 minutes
2013-06-21 22:32:30.904323 Shares: 129 (2 orphan, 7 dead) Stale rate: ~7.0% (3-13%) Efficiency: ~108.3% (101-113%) Current payout: 0.2716 BTC
2013-06-21 22:32:30.904352 Pool: 591GH/s Stale rate: 14.1% Expected time to block: 1.6 days
2013-06-21 22:32:33.207259 New work for worker! Difficulty: 15.000000 Share difficulty: 11301.302403 Total block value: 25.001000 BTC including 2 transactions
2013-06-21 22:32:38.925707 P2Pool: 17327 shares in chain (17332 verified/17332 total) Peers: 77 (16 incoming)
2013-06-21 22:32:38.925789 Local: 6976MH/s in last 10.0 minutes Local dead on arrival: ~2.0% (0-6%) Expected time to share: 11.5 minutes
2013-06-21 22:32:38.925825 Shares: 129 (3 orphan, 7 dead) Stale rate: ~7.8% (4-14%) Efficiency: ~107.4% (100-112%) Current payout: 0.2428 BTC
2013-06-21 22:32:38.925861 Pool: 574GH/s Stale rate: 14.1% Expected time to block: 1.7 days
Here's what it looks like on a server in Canada's OVH datacenter (100ms from my Hetzner server):
My share:
Time first seen: Fri Jun 21 2013 22:32:17 GMT-0500 (Central Daylight Time) (1371871937.19677)
Peer first received from: 5.9.24.81,37483
The one that replaced it:
Time first seen: Fri Jun 21 2013 22:32:31 GMT-0500 (Central Daylight Time) (1371871951.30414)
Peer first received from: xxx,55412
Since there was one other 'new work' thing in there, it means that a 2nd share built off my own and that 2-share chain wasn't invalidated until the 3rd share.
For this to happen, that would mean that three clients would have had to accept one of the shares that my node 'punished' for being invalid (probably the one that came 600ms after the block was detected)...
or is my logic here wrong somewhere?
ed: edited out a 3rd party IP
ed2: and here's the "offending" block:
http://blockchain.info/block/000000000000008f0ea3c9b07b3284ffdaad5b0dd31997ea1d02742bc9a185cbblockchain.info reports it being first seen from my IP, 5.9.24.81
hmm.. or, it could be the one before it,
http://blockchain.info/block-index/394345where it lists 5.9.24.81 as 10th or so
i think that one is probably the one. the other came @ 32:27.... which would be 4 seconds before the share that replaced mine, but 26 seconds after the last block... and would also mean that p2pool x's and p2pool y's were working off of different chains for at least those 26 seconds