the main reason for empty blocks has always been SPV mining, that is the fact that miners didn't verify the previous block completely before starting the next so they start with an empty block until they verify the previous one and be able to update their mempool and add new transactions to their block. sometimes they get lucky and find the answer to that empty block and publish that.
since during the past couple of years there has been a lot of improvements in the speed of transaction verification process with all the optimizations done by core team and SPV mining is not that common anymore we are seeing a much reduced number of empty blocks nowadays.
...
What has changed is that they simply update the work to the miners faster now than they used to do it.
Simple example:
BeforeBlockchange ... send out empty work.
30s later ... send out new work with transactions.
Over time I guess they realised how stupid this is and changed it to:
NowBlockchange ... send out empty work.
A few seconds later ... send out new work with transactions.
Magic! Now they generate 10% of the empty blocks they used to generate if they take 3s instead of 30s
If they take 0.3s, empty blocks drop to 1% ... etc.
You can see that most pools are still ignoring transactions on a block change.
If you can see the mempool after each block change ... you check that, any time, after you see an empty block on the network.
... I can coz I have extra information in the logs coded into all the bitcoinds that run on my pool so I can always see the size of all work generated.
P.S. I'm one of the VERY few pools that has NEVER mined an empty block
I can only think of one other pool - and that effectively died recently when they lost yet another block due to negligence by the guy who ran the pool.
P.P.S. the time to validate a block and generate a full block is less than 200ms