So, we just got a share that would of found us a block, but it was stale. The tooltip says "Share was submitted 1 minute too late." Does this mean the miner submitted too late to the pool late, or did we simply legit lost the propagation race?
It wasn't an orphan after all, it was a stale share.
He submitted it 69 seconds late to kano.is
ckpool has (as yet before never tested) code to try submit stale block shares anyway ... just in case.
kano.is put out our 388788 block change to F2Pool's block on time:
[2015-12-17 11:38:04.925] Block hash changed to 00000000000000000497f638fc378166d2d14742d4b77187209fe5f32a57afe3
poolbench also shows they saw it 573ms later:
stratum.kano.is:3333 2015-12-16 19:38:05.498000
But the share came in 69 seconds later:
[2015-12-17 11:39:13.492] Possible block solve diff 182123374943.539276 !
[2015-12-17 11:39:13.604] SUBMIT BLOCK RETURNED: inconclusive
[2015-12-17 11:39:13.604] Submitted, but rejected block 388788
So yeah there was no chance in hell of it ever being a block
Edit: oh also CKDB's report on it (including my manual interventions 'Orphan' and 'Unworthy')
[2015-12-17 11:39:13.515+11] blocks_add(): BLOCK! Status: New, Block: 388788/...00006097e9a3b4d0 Diff 182G UTC:2015-12-17 00:39:13.492270+00
[2015-12-17 11:39:13.515+11] BLOCK! STALE Diff 230.2% (182123374944/79102380900.2) jbravo.rpi0166b Pool 92112911921.0 92.1G 116.45%
[2015-12-17 11:48:05.275+11] make_a_shift_mark() mark 6228334527803037660/e/x/Shift fin: 5ppdo charlotte/Block 388788 fin/
[2015-12-17 11:48:13.327+11] SS_to_MS() Processing: workmarkers 20140918511339//End 6228334527803037660/Stt 6228334527803037601/Shift fin: 5ppa6 belldandy++ to Shift fin: 5ppdo charlotte/x
[2015-12-17 11:48:15.960+11] SS_to_MS() Processed: 3684 ms 218670 ss 1817804 shares 2391717373 diff for workmarkers 20140918511339//End 6228334527803037660/Stt 6228334527803037601/Shift fin: 5ppa6 belldandy++ to Shift fin: 5ppdo charlotte/x
[2015-12-17 11:56:20.737+11] cmd_query() 1.block height=388788
[2015-12-17 11:56:57.677+11] cmd_query() 1.block height=388787
[2015-12-17 11:58:29.852+11] blocks_add(): Status: Orphan, Block: 388788/...00006097e9a3b4d0 Diff 182G
[2015-12-17 11:58:37.627+11] summarise_blocks() block 388788, stats confirmed 91908114882/297315718/77826749/276997/78216
[2015-12-17 11:58:38.628+11] process_pplns(): can't process block 388788/6228334527803037660/jbravo.rpi0166b/2559600823 status: O/Orphan
[2015-12-17 12:06:34.142+11] blocks_add(): Status: Unworthy, Block: 388788/...00006097e9a3b4d0 Diff 182G