I did some in-depth analysis of Bithopper Long-Poll timing method for identifying block-solving pools, initially to see how pool-delay corrections can be used to make round-finding results more accurate, I used the console log output of BH, which after some grepping, looks like this:
(round 141030 - found by deepbit)
[1295][23:39:34] LP Call mining.mainframe.nl:8343/LP
[1326][23:39:35] LP Call pit.deepbit.net:8332/listenChannel
[1333][23:39:35] LP Call mineco.in:3000/LP
[1334][23:39:35] LP Call eu1.triplemining.com:8344/LP
[1341][23:39:36] LP Call arsbitcoin.com:8344/LP
[1344][23:39:37] LP Call uscentral.btcguild.com:8332/LP
[1355][23:39:38] LP Call pool.bitclockers.com:8332/LP
[1356][23:39:38] LP Call pool.bitp.it:8334/longpoll
[1358][23:39:39] LP Call bitcoins.lc:8080/LP
[1392][23:39:42] LP Call su.mining.eligius.st:8337/LP
We see above, the first pool to respond is NOT the finding pool!
I have workers logging into 16 different pools; however only 11 of those reply with new block LP pushes. Current Bithopper logic simply guesses that the first LP is the finding pool; as we can see in above, the first LP was
not from deepbit in this case. As I had seen this behavior before, I requested the addition of an lp_penalty option - a per-pool option in Bithopper, where you can "delay" the LP reponse time on fast-reporting pools.
Now, we must classify what that delay should be. I used the LP timestamps from the console log as shown above. I was originally expecting the finding pool to clearly stand out, with a definitive earlier LP push after correction, but even this turned out
not to be the case, so I should have modded the output to be tenths of seconds for better analysis before I started..
Here is 11 rounds of data, with the actual finding pool verified with digbtc.com and pident.artefact2.com. The block delays for each pool and each round are normalized against the time of the finding pool, if present:
The time-delay table is the top, different analyses are in blue, and the actual correction I am using is on is on the far right. Note the lp_penalty setting would be the opposite of this correction - eligius might get lp_penalty:0 and mineco: 4.
The bottom table is the time delays after applying the best correction I could hand-tweak. We see that still the finding pool almost never is the fastest (highlighted in red). (This might improve with 1/10th second stats resolution).
However, we see that if we average
all pool LPs after learning a delay correction, the finding pool does begin to stand out.
The conclusions I have to make:-Per-pool LP delays are more random than one might expect; and some pools have more variation than others (likely related to number of miners, and their p2p distance from block-solving pool)
-Deepbit's block-find LPs do not clearly stand out from other non-finding LPs received from deepbit,
-The first reporting pool, even when testing many delay correction formulas, will not necessarily be the finding pool.
-Only with a time delay correction for each pool, and then comparison of each pool to the average of all pools can we hope to reliably identify the stand-out block-finding pool. We also need a confidence factor, so if no pools stand out, we can conclude none of LP-returning pools we check found the block.
We have also seen --p2pLP come to prominence, where LP block-finding information is shared on IRC, and then a vote helps further refine results. However this still is very rudimentary; and as nearly all bH users get LPs from deepbit vs only some getting other pools, deepbit tends to win votes undeservedly (and these biased voting results would be
more wrong if it wasn't for DB being the largest pool).
So to improve BH LP block finding to the point of reliability, we need:- Self-learning of per-pool LP delays (LP delay learning must remove the finding pool however, or else we bias deepbit's delays because they find so many blocks)
- Publishing of
all local corrected new block LP results to IRC, not just winning pool (perhaps JSON format, like "
** block 3fe33a: {"triple": 0.0, "bitp": 0.3, "deepbit": 0.5, "mineco": 1.3, "bclc", 1.7}"
-
Averaging (not voting or first response) all IRC results (including pools we don't actually check ourselves), to determine if there is a clear LP block winner; a confidence level can be assigned.
For the super-advanced, it may even be able to profile a 'signature' of block delays to determine who found the block, since pools also seem to have a different LP delay depending on their p2p distance from the finding pool.
