Why did miners waste much block space when there is need

[Huppercase]

I noticed a block confirmed less than an hour ago, but I also noticed that it contains only a single transaction, I was wondering why would they confirmed just a single transaction in a block amidst this period of large pending transactions.
This is the block that was found with a single transaction:
https://mempool.space/block/000000000000000000042e59020bb65cd4221498a9cb6fc81bb43a35bdf9d8b6

I was confused why will they mine a single transaction when everyone is looking for a way to move their bitcoin from one place to another, this doesn't make sense when they can allow others to include their transactions with the fees all together.

[LoyceV]

This usually happens when a block is found shortly after the previous block. Miners don't waste a block when they find one, even if they didn't have time to build the transactions yet.

[nc50lc]

This can also be an indication of a "Covert ASICBoost" since not including transactions is the only viable option to do it.

However, your example is definitely mined less than a minute after the pervious block
so the miner just got lucky and didn't had time to included transactions in it.

[paid2]

Yes, some pools even accept to mine totally empty blocks.
The biggest pools all do this : Binance Pool, Foundry, Antpool etc...

You can search on the forum, kano has explained many times why this is harmful for the BTC blockchain and why it is stupid to accept to validate empty blocks.

Some pools refuse to do this, including kano's pool. Unfortunately, profits before the interest of the blockchain itself takes precedence for the biggest pools.

[o_e_l_e_o]

I was confused why will they mine a single transaction when everyone is looking for a way to move their bitcoin from one place to another, this doesn't make sense when they can allow others to include their transactions with the fees all together.

To expand on the above answers:

When a node receives a block from somewhere else, it has to spend a little bit of time verifying that block, checking every transaction in the block is correct and accurate, and then updating its set of unconfirmed outputs to remove all the outputs which have just been spent and add all the new outputs which have just been created. This doesn't take long - usually in the order of a few seconds depending on your hardware - but it isn't instant.

While this is happening, a miner cannot create a new block filled with transactions to work on, because it doesn't know which transactions it can and cannot include until it verifies which transactions have just been mined in the block it just received. So for these few seconds, the miner's options are either to have their mining equipment sit idle and do nothing, or attempt to mine an empty block until they have fully verified the last block. Since having their equipment sit idle would be a waste of money, most miners attempt to mine an empty block for a few seconds until they create a normal block filled with transactions and then switched to trying to mine that instead. Very occasionally a miner will be successful in these few seconds and will mine an empty block.

You can search on the forum, kano has explained many times why this is harmful for the BTC blockchain and why it is stupid to accept to validate empty blocks.

I don't see how it is harmful at all. Even empty blocks add to the security of the network, and literally nothing is lost by an empty block being found - it does not make the next block any harder to find.

[LoyceV]

literally nothing is lost by an empty block being found - it does not make the next block any harder to find.

That's not entirely correct: if miners would skip the few seconds between full blocks, the difficulty would be a bit lower (after 2 weeks). On average, the same amount of blocks would contain more transactions. There's just no reason for individual miners to do this.

[o_e_l_e_o]

That's not entirely correct: if miners would skip the few seconds between full blocks, the difficulty would be a bit lower (after 2 weeks). On average, the same amount of blocks would contain more transactions. There's just no reason for individual miners to do this.

I deliberately didn't even mention this because the effect is negligible.

In the last 20,000 blocks, I count 44 empty blocks, for an average of 1 every 454.5 blocks, or 4.435 per difficulty epoch.
Removing those 4.435 blocks per epoch would result in an epoch taking 44.35 minutes longer (on average).
Diving two weeks plus 44.35 minutes by 2016 results in an average block time of 10 minutes and 1.3 seconds.

[DaveF]

Taking it a bit past what was said above.
IF they are doing it properly and that is a big IF a large pool is running several nodes all over the world, when a block comes in they all should start validating it. Once a certain percentage of them agree then and only then do they start building the new block for the pool.

Loosing the fees even now is better then building a invalid block that gets rejected.
AND not broadcasting an empty block as soon as possible risks loosing it too.

It just comes down to routing. It takes about 1/2 a second to send 1 packet of data around the world without doing any kind of firewall / security inspection in an ideal setting.
Allowing for the nodes to verify what is in the block, and sending it back out is another couple of seconds and then start building a new block. If all is ideal you should have a new block ready to go in 10 seconds or so.

BUT, if you are waiting for minimum of 3 of 5 to do their thing you might add a few seconds on top of that.

Add in a bloated memory pool and some DPI From firewalls and you add a few seconds again.

All of a sudden you are looking at more time to build a block.

Even now there are 2 schools of thought with what happened with what happened with foundry the other day.
Some people including Kano are saying that they tried to orphan a couple of blocks.
Others are saying that they saw foundrys blocks 1st.

It has always been the way it works.

And part of the problem is that even if you understand BTC perfectly, unless you understand the true issues of internet routing and the true delays that proper DPI can put into network performance then it's never going to seem logical as to what happened.

Dave's pool can put it's nodes out there in public.
A corporation with all kinds of security requirements probably has 3 layers of security devices looking at all data coming in before it even hits the node to be processed. Dave's node has seen, and processed the block before it has even gotten through the security devices of some places. On the flip side, it's a lot harder to take out big corps node(s) by flooding them with bad data since it never even makes it into the network.


-Dave

* DPI = deep packet inspection. https://www.fortinet.com/resources/cyberglossary/dpi-deep-packet-inspection

[mikeywith]

When a node receives a block from somewhere else, it has to spend a little bit of time verifying that block, checking every transaction in the block is correct and accurate, and then updating its set of unconfirmed outputs to remove all the outputs which have just been spent and add all the new outputs which have just been created. This doesn't take long - usually in the order of a few seconds depending on your hardware - but it isn't instant.

While this is happening, a miner cannot create a new block filled with transactions to work on, because it doesn't know which transactions it can and cannot include until it verifies which transactions have just been mined in the block it just received. So for these few seconds, the miner's options are either to have their mining equipment sit idle and do nothing, or attempt to mine an empty block until they have fully verified the last block. Since having their equipment sit idle would be a waste of money, most miners attempt to mine an empty block for a few seconds until they create a normal block filled with transactions and then switched to trying to mine that instead. Very occasionally a miner will be successful in these few seconds and will mine an empty block.

Downloading, verifying transactions, and adjusting the node's mempool to purge the already confirmed transactions takes no more than 200ms, a mining pool would waste on average 14400ms a day if they refrain from mining empty blocks, depending on the size of their hashrate it could be anywhere between a few thousands to a few million dollars a year, of course, this comes at an expense which is increased probability of orphan race.

[o_e_l_e_o]

Downloading, verifying transactions, and adjusting the node's mempool to purge the already confirmed transactions takes no more than 200ms

In a perfect scenario, maybe, but the network is not perfect. The mining pool also has to build a candidate block, calculate the Merkle root, and then update all its miners with the new candidate block hash. If the whole process genuinely took only 200ms, then we wouldn't see empty blocks at all. But it simply is not that quick. It takes a few seconds in the real world, and so we still occasionally see empty blocks being mined.

[mikeywith]

In a perfect scenario, maybe, but the network is not perfect. The mining pool also has to build a candidate block, calculate the Merkle root, and then update all its miners with the new candidate block hash. If the whole process genuinely took only 200ms, then we wouldn't see empty blocks at all. But it simply is not that quick. It takes a few seconds in the real world, and so we still occasionally see empty blocks being mined.

I initially had the same thoughts,  but turned out i was wrong, the  <200ms average isn't a random figure i invented it was reported by Kano from his own pool logs, his pool is one of the oldest pools and the data he reports is probably the most accurate you can find on the internet.

By the way, the figure includes EVERYTHING from knowing about the block to the moment the pool generates work for its miners, ya some will take over 200ms, some less, but the average is around that.

It takes less than 200ms from receiving a block in bitcoind, until it can produce a full new block template with transactions.
I have a message in debug.log when the block arrives with a timestamp to microseconds, another when it has been processed, and another when the pool has been sent the new work.

The largest block in the last 8 hours was 631950 it was 1783877 bytes.
bitcoind debug.log shows (as I mentioned)

Code:

2020-05-27 19:07:53.985131 ProcessNewBlock
2020-05-27 19:07:54.025189 Pre-allocating up to position 0x3000000 in blk02094.dat
2020-05-27 19:07:54.107985 UpdateTip: new best=000000000000000000067af76e3e524beabb9557f71413d8db9b88760e445d3b height=631950 version=0x20002000 log2_work=91.982404 tx=533594598 date='2020-05-27 19:07:33' progress=1.000000 cache=36.9MiB(236180txo) warning='75 of last 100 blocks have unexpected version'
2020-05-27 19:07:54.108131 Block 000000000000000000067af76e3e524beabb9557f71413d8db9b88760e445d3b provided by 107.191.117.193:8333
2020-05-27 19:07:54.158265 GetBlockTemplate called
2020-05-27 19:07:54.160724 CNB
2020-05-27 19:07:54.174358 CreateNewBlock(): block weight: 3964486 txs: 122 of 10917 fees: 0.04177550 sigops 20166

Also of interest in this case, it had to extend some disk space for the block.

Total time from when it arrived until the pool got new work 19:07:54.174358 - 19:07:53.985131 = 189227 microseconds 189 milliseconds

As for empty blocks, the one I see in the last 8 hours is 631926 by ViaBTC
I'll include the block before as well for more information:

Code:

2020-05-27 16:36:48.892330 ProcessNewBlock
2020-05-27 16:36:48.913863 Leaving block file 2093: CBlockFileInfo(blocks=99, size=133620045, heights=631826...631924, time=2020-05-26...2020-05-27)
2020-05-27 16:36:48.914006 Pre-allocating up to position 0x1000000 in blk02094.dat
2020-05-27 16:36:48.950898 Pre-allocating up to position 0x100000 in rev02094.dat
2020-05-27 16:36:49.011947 UpdateTip: new best=00000000000000000010dab51e5208c538fce5634104fbd059da24140911efe7 height=631925 version=0x27ffe000 log2_work=91.981925 tx=533547943 date='2020-05-27 16:36:37' progress=1.000000 cache=14.5MiB(52565txo) warning='72 of last 100 blocks have unexpected version'
2020-05-27 16:36:49.012075 Block 00000000000000000010dab51e5208c538fce5634104fbd059da24140911efe7 provided by 107.191.117.193:8333
2020-05-27 16:36:49.071051 GetBlockTemplate called
2020-05-27 16:36:49.071136 CNB
2020-05-27 16:36:49.088969 CreateNewBlock(): block weight: 3964991 txs: 1933 of 17715 fees: 0.18324281 sigops 13534
2020-05-27 16:37:08.482568 ProcessNewBlock
2020-05-27 16:37:08.490710 UpdateTip: new best=0000000000000000000f1b87afb1b95a5e681736ea387b60a8bd150b1ec8bb30 height=631926 version=0x3fff0000 log2_work=91.981944 tx=533547944 date='2020-05-27 17:06:23' progress=1.000012 cache=14.5MiB(52772txo) warning='73 of last 100 blocks have unexpected version'
2020-05-27 16:37:08.568545 CreateNewBlock(): block weight: 3964339 txs: 1903 of 17821 fees: 0.21419789 sigops 13493

In this case firstly, you can see the block before, it was 20 seconds before it.
Secondly, in this case the work kano.is generated included 1933 transactions and fees: 0.18324281 BTC
However as can be seen from the next block on the block chain, it was only 315 bytes  ... instead of our "block weight: 3964991"
(You'll have to check the 315 bytes number on any block explorer to see it's an 'empty' block)
i.e. our work that miners were working on was a full block, but ViaBTC miners were working on an empty block when they found it.

It took 196639 microseconds to process i.e. 197 milliseconds

... and of course it didn't take 197 milliseconds longer than ViaBTC - they didn't process it in 0ms

You can run a bitcoind yourself and you can see this information - though you'll have to patch it and recompile it to display the extra information

I've been around in bitcoin since 2011 working on code related to it.
Working on mining code, pool code, and even simple patches to bitcoin itself.

Now if you think <200ms isn't worth it to mine empty blocks, you can read my post in the same topic explaining how it IS worth it for mining pools to invest those 200ms

There is 86400000ms in a day, and 144 blocks to be mined, so by utilizing 100ms every 10 mins you get a total of 14400ms which is 0.0167% of the total 24 hours.

When a pool like F2pool saves 150ms for every potential block, that's 552510 terahash worth of mining, if that's hard to understand think of it this way.

When F2pool knows about a new block coming, and while going through that 150ms process they switch their hashrate to another PPS pool and say that is Viabtc, by the end of the day they would get paid $1500 from mining for 150ms every 10 mins or a total of 21.1 seconds a day with 22,100,420.00th worth of hash power, but instead of doing this, they would just mine to their own pool and still by the end of the day/week/month/year, they will theoretically make that same amount of money.

And it gets even better when you don't have to download the block in the first place when your pool is synced with the other pools via any other protocol and all you need is their hash (not even the block header) then that makes it well above 200ms, but even with these numbers F2pool can potentially make over half a million dollar in profit yearly, or otherwise, lose it if they use your code and not mine empty blocks.

[dzungmobile]

This usually happens when a block is found shortly after the previous block. Miners don't waste a block when they find one, even if they didn't have time to build the transactions yet.

It usually happens after an empty block.

Bitcoin's Empty Blocks Analaysis.
BTC empty blocks (2009 - 5 May 2020): miners, size, daily, monthly,yearly stats

Here is 30 first empty blocks and last 30 empty blocks of AntPool within the observed period.

Code:

     +------------------------------------------------+
     | height            minedon   sizekb        date |
     |------------------------------------------------|
  1. | 295637   2014-04-13 15:52     .201   13apr2014 |
  2. | 318180   2014-08-30 02:29      .25   30aug2014 |
  3. | 318202   2014-08-30 06:16      .25   30aug2014 |
  4. | 319252   2014-09-05 17:06      .25   05sep2014 |
  5. | 319412   2014-09-06 16:57      .25   06sep2014 |
     |------------------------------------------------|
  6. | 319591   2014-09-07 19:11      .25   07sep2014 |
  7. | 320010   2014-09-10 15:05      .25   10sep2014 |
  8. | 320724   2014-09-14 22:54      .25   14sep2014 |
  9. | 320857   2014-09-15 19:31      .25   15sep2014 |
 10. | 321174   2014-09-17 14:46      .25   17sep2014 |
     |------------------------------------------------|
 11. | 321739   2014-09-20 20:42      .25   20sep2014 |
 12. | 321788   2014-09-21 02:34      .25   21sep2014 |
 13. | 322407   2014-09-24 23:58      .25   24sep2014 |
 14. | 322420   2014-09-25 01:14      .25   25sep2014 |
 15. | 322919   2014-09-28 12:04      .25   28sep2014 |
     |------------------------------------------------|
 16. | 323667   2014-10-03 17:32      .25   03oct2014 |
 17. | 323713   2014-10-04 01:09      .25   04oct2014 |
 18. | 324519   2014-10-09 10:56      .25   09oct2014 |
 19. | 324619   2014-10-10 00:02      .25   10oct2014 |
 20. | 324847   2014-10-11 12:27      .25   11oct2014 |
     |------------------------------------------------|
 21. | 325496   2014-10-15 23:07      .25   15oct2014 |
 22. | 325839   2014-10-18 05:38      .25   18oct2014 |
 23. | 325994   2014-10-19 05:53      .25   19oct2014 |
 24. | 326053   2014-10-19 17:25      .25   19oct2014 |
 25. | 327574   2014-10-29 23:14     .248   29oct2014 |
     |------------------------------------------------|
 26. | 328525   2014-11-04 15:11     .248   04nov2014 |
 27. | 330046   2014-11-14 23:16     .248   14nov2014 |
 28. | 330108   2014-11-15 09:39     .248   15nov2014 |
 29. | 332277   2014-11-30 11:22     .248   30nov2014 |
 30. | 332553   2014-12-02 08:53     .206   02dec2014 |
     +------------------------------------------------+
..............................................................
      +------------------------------------------------+
      | height            minedon   sizekb        date |
      |------------------------------------------------|
1682. | 594573   2019-09-12 20:25     .379   12sep2019 |
1683. | 594584   2019-09-12 22:26      .38   12sep2019 |
1684. | 595057   2019-09-16 00:59     .379   16sep2019 |
1685. | 595265   2019-09-17 08:45     .378   17sep2019 |
1686. | 595366   2019-09-18 01:16     .379   18sep2019 |
      |------------------------------------------------|
1687. | 595920   2019-09-21 13:00     .379   21sep2019 |
1688. | 596059   2019-09-22 09:37     .379   22sep2019 |
1689. | 596245   2019-09-23 17:58     .379   23sep2019 |
1690. | 596270   2019-09-23 23:21     .379   23sep2019 |
1691. | 596525   2019-09-25 14:34     .379   25sep2019 |
      |------------------------------------------------|
1692. | 596588   2019-09-25 23:25     .378   25sep2019 |
1693. | 597126   2019-09-29 13:58     .379   29sep2019 |
1694. | 597980   2019-10-05 10:27      .38   05oct2019 |
1695. | 598289   2019-10-07 11:58      .38   07oct2019 |
1696. | 598356   2019-10-07 23:11      .38   07oct2019 |
      |------------------------------------------------|
1697. | 598411   2019-10-08 06:20      .38   08oct2019 |
1698. | 599327   2019-10-14 11:52     .378   14oct2019 |
1699. | 600871   2019-10-24 15:37     .378   24oct2019 |
1700. | 602493   2019-11-05 19:33      .38   05nov2019 |
1701. | 604106   2019-11-16 21:34      .38   16nov2019 |
      |------------------------------------------------|
1702. | 604202   2019-11-17 13:05      .38   17nov2019 |
1703. | 604285   2019-11-18 01:36     .379   18nov2019 |
1704. | 605320   2019-11-25 06:17      .38   25nov2019 |
1705. | 607190   2019-12-08 07:32      .38   08dec2019 |
1706. | 614346   2020-01-24 15:59      .38   24jan2020 |
      |------------------------------------------------|
1707. | 618512   2020-02-22 13:31      .38   22feb2020 |
1708. | 618861   2020-02-25 00:52      .38   25feb2020 |
1709. | 620233   2020-03-05 00:33      .38   05mar2020 |
1710. | 623149   2020-03-27 08:45      .38   27mar2020 |
1711. | 624652   2020-04-06 09:15     .379   06apr2020 |
      |------------------------------------------------|
1712. | 628428   2020-05-01 15:15     .379   01may2020 |
      +------------------------------------------------+