Edit: I've removed some of the irrelevant banter.
My KDB pool code reports
ANY work generated that is ever empty, of the work generated once every 30 seconds over the years.
Over the life of my pool, that would amount to less than you can count on one hand.
This, of course, is the same work that is available to EVERY pool at the same time, not just my pool.
Generating work:
Most pools, I imagine work the same as here - we do a call to bitcoind to get a template that already includes all the transactions we will mine.
Some pools may manipulate the transactions in their pool code - but we don't.
It is possible for us to mine an empty block under very rare circumstances out of the pool's control.
But that has never happened here for the 2429 blocks we have found.
Those circumstances are: a pool generates work that contains all the known transactions on the network.
That pool (here or elsewhere) then finds a block. During that short time (30s or less) no other transactions are seen on the network.
Thus after submitting the block found, the next call to get a work template from bitcoind will have no new transactions.
I log whenever the work is empty - I think it's happened less times than you can count on one hand in 6 years
Thus during those very few rare events someone would have to have found a block to produce an empty block out of their control.
So yeah maybe less than half a dozen possible empty blocks in 6 years for all pools on the network - the rest are them doing it on purpose.
I make my bitcoin debug.log report the transaction information for each work it generates, and I can also see in the bitcoin debug.log what is available in the first work after any block change (VERY relevant example further below)
----
"Mandatory" empty blocks have to do with the time between the last work generation sent to the miner that found the block, before finding a block, and the next work generation once your pool found a block.
The transactions that appear on the network during that time (or unused by the block, still in the mempool) decide it ...
It doesn't matter how big or how small my pool is, my pool is part of the bitcoin network and sees the same transactions as every other well connected pool on the network - though mine is most likely the most well connected of all pools
When a block change occurs, my pool will ALWAYS see every transaction that was in the block, that's how bitcoin works.
It will also see what transactions were left over and available to use mining for the next block, in the new work it will send out to the pool's miners.
This is the time gap/number of available transactions that decides if pools MUST generate an empty block with the new work they send to their miners.
This number is the "maybe" less than half a dozen in the past many years.
What the other pools do, to purposefully produce empty blocks, is to not use a template with transactions generated by bitcoind, but simply generate work based on the hash of the block just found on the network, and nothing more - they cannot include transactions in the work when they do this, since they don't already know which network transactions have already been included in the block they don't want to get the transaction details of.
i.e. they aren't verifying the full block before producing a new block work template for their miners.
This was the cause of the failed/incorrect fork by Bitmain and F2Pool in 2015 - they lost all their 5 or 6 blocks on that bad fork since they built new blocks, on a fork of the network based on an invalid block, produced by another pool that ignored the network rule changes.
You could probably force most of the current large pools onto an invalid fork by mining a block with an invalid transaction in it
---
It takes less than 200ms from receiving a block in bitcoind, until it can produce a full new block template with transactions.
I have a message in debug.log when the block arrives with a timestamp to microseconds, another when it has been processed, and another when the pool has been sent the new work.
The
largest block in the last 8 hours was
631950 it was
1783877 bytes.
bitcoind debug.log shows (as I mentioned)
2020-05-27 19:07:53.985131 ProcessNewBlock
2020-05-27 19:07:54.025189 Pre-allocating up to position 0x3000000 in blk02094.dat
2020-05-27 19:07:54.107985 UpdateTip: new best=000000000000000000067af76e3e524beabb9557f71413d8db9b88760e445d3b height=631950 version=0x20002000 log2_work=91.982404 tx=533594598 date='2020-05-27 19:07:33' progress=1.000000 cache=36.9MiB(236180txo) warning='75 of last 100 blocks have unexpected version'
2020-05-27 19:07:54.108131 Block 000000000000000000067af76e3e524beabb9557f71413d8db9b88760e445d3b provided by 107.191.117.193:8333
2020-05-27 19:07:54.158265 GetBlockTemplate called
2020-05-27 19:07:54.160724 CNB
2020-05-27 19:07:54.174358 CreateNewBlock(): block weight: 3964486 txs: 122 of 10917 fees: 0.04177550 sigops 20166
Also of interest in this case, it had to extend some disk space for the block.
Total time from when it arrived until the pool got new work 19:07:54.174358 - 19:07:53.985131 = 189227 microseconds
189 millisecondsAs for empty blocks, the one I see in the last 8 hours is 631926 by
ViaBTCI'll include the block before as well for more information:
2020-05-27 16:36:48.892330 ProcessNewBlock
2020-05-27 16:36:48.913863 Leaving block file 2093: CBlockFileInfo(blocks=99, size=133620045, heights=631826...631924, time=2020-05-26...2020-05-27)
2020-05-27 16:36:48.914006 Pre-allocating up to position 0x1000000 in blk02094.dat
2020-05-27 16:36:48.950898 Pre-allocating up to position 0x100000 in rev02094.dat
2020-05-27 16:36:49.011947 UpdateTip: new best=00000000000000000010dab51e5208c538fce5634104fbd059da24140911efe7 height=631925 version=0x27ffe000 log2_work=91.981925 tx=533547943 date='2020-05-27 16:36:37' progress=1.000000 cache=14.5MiB(52565txo) warning='72 of last 100 blocks have unexpected version'
2020-05-27 16:36:49.012075 Block 00000000000000000010dab51e5208c538fce5634104fbd059da24140911efe7 provided by 107.191.117.193:8333
2020-05-27 16:36:49.071051 GetBlockTemplate called
2020-05-27 16:36:49.071136 CNB
2020-05-27 16:36:49.088969 CreateNewBlock(): block weight: 3964991 txs: 1933 of 17715 fees: 0.18324281 sigops 13534
2020-05-27 16:37:08.482568 ProcessNewBlock
2020-05-27 16:37:08.490710 UpdateTip: new best=0000000000000000000f1b87afb1b95a5e681736ea387b60a8bd150b1ec8bb30 height=631926 version=0x3fff0000 log2_work=91.981944 tx=533547944 date='2020-05-27 17:06:23' progress=1.000012 cache=14.5MiB(52772txo) warning='73 of last 100 blocks have unexpected version'
2020-05-27 16:37:08.568545 CreateNewBlock(): block weight: 3964339 txs: 1903 of 17821 fees: 0.21419789 sigops 13493
In this case firstly, you can see the block before, it was 20 seconds before it.
Secondly, in this case the work kano.is generated included 1933 transactions and fees: 0.18324281 BTC
However as can be seen from the next block on the block chain, it was only 315 bytes ... instead of our "block weight: 3964991"
(You'll have to check the 315 bytes number on any block explorer to see it's an 'empty' block)
i.e. our work that miners were working on was a full block, but
ViaBTC miners were working on an empty block when they found it.
It took 196639 microseconds to process i.e.
197 milliseconds... and of course it
didn't take
197 milliseconds longer than ViaBTC - they didn't process it in 0ms
You can run a bitcoind yourself and you can see this information - though you'll have to patch it and recompile it to display the extra information
I've been around in bitcoin since 2011 working on code related to it.
Working on mining code, pool code, and even simple patches to bitcoin itself.
---
Now as to why not to mine on these "empty block" pools?
Well as is clear in all discussions about empty blocks, those pools don't fully process the block before sending out new work.
Thus they can fork bitcoin and cause all sorts of problems with exchanges and transactions, if the empty block they generate is based on an invalid block e.g. with an invalid transaction in it.
This occurred before in 2015 by Bitmain and F2Pool where they forked the bitcoin network and kept ahead of the correct fork for 5-6 blocks with a large % of the bitcoin network, until certain people contacted them and told them they were mining on an invalid fork (I contacted Bitmain ...)