Bitcoin Forum
August 17, 2022, 04:58:32 PM *
News: Latest Bitcoin Core release: 23.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: « 1 [2]  All
  Print  
Author Topic: Bitcoin Empty Blocks  (Read 438 times)
DougM
Full Member
***
Offline Offline

Activity: 173
Merit: 116


View Profile
August 24, 2020, 07:54:46 PM
Merited by ranochigo (2), ETFbitcoin (2)
 #21

That's the other puzzling thing, how is it possible that a block could be successfully mined within seconds when it should take on average 10 minutes?
This is an average time range, it could be faster or slower depending on some factors such as the one given by @ranochigo; luck, cause finding a hash is based on randomness. The difficulty level is adjusted approximately every 2 weeks or 2016 blocks to maintain the average block time of 10 minutes.
I did some quick checks based on the earliest blocks (0  to 119,964 contained in blk00000.dat just because I had them handy).  Comparing the time deltas based on sequential blocks from the beginning of (blockchain) time the delta ranges from -119 to  7,719 minutes, but sure enough the average is ~10 minutes!  That is surprising because the difficulty didn't change until the end of 2009, but the first dat file covers from Jan 03 2009 (Genesis block 0) to April 24 2011 (119,964) so I guess maybe it compensated later if the average was off initially?

Maybe so...I checked the metrics from blocks 1 to 38,091 (Feb 4, 2010) around the time of the first difficulty change from 1 to 2:  Min -65 mins, Max 7,719 min (FYI delta 0 to 1 was an anomaly next largest was 1509 min)  and the average was close to 15 minutes versus target 10 minutes. This means the average after that must have been less than 10 minutes for a while to compensate.

Interestingly the timestamps for follow on blocks can be earlier in time...that was a bit of surprise to me at first, but I learned I can't assume sequential blocks are in temporal order.  For example:

https://www.blockchain.com/btc/block/32649  -->  2010-01-02 04:49
https://www.blockchain.com/btc/block/32650  -->  2010-01-02 02:50

1660755512
Hero Member
*
Offline Offline

Posts: 1660755512

View Profile Personal Message (Offline)

Ignore
1660755512
Reply with quote  #2

1660755512
Report to moderator
1660755512
Hero Member
*
Offline Offline

Posts: 1660755512

View Profile Personal Message (Offline)

Ignore
1660755512
Reply with quote  #2

1660755512
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
barrysty1e
Hero Member
*****
Offline Offline

Activity: 625
Merit: 515



View Profile WWW
August 24, 2020, 11:41:06 PM
Merited by ETFbitcoin (2), NotFuzzyWarm (1)
 #22

When it comes to validating a block, depending on the composition of transactions, it can take upwards of a few seconds (even more in some pathological/adversarial scenarios) to complete.

perhaps if you use a commodore64..

doublesha of the raw transaction data per transaction in block,
store in list, then hash the list,
then compare this hash with the one provided in the blockheader,
then hash the blockheader and ensure its below target,
we're talking milliseconds here..

heres an example from dash (x11 is a bit more harsh than sha256d, only enforcing the point)..
Code:
2020-08-24 23:37:04 received: block (452 bytes) peer=0
2020-08-24 23:37:04 received block 4829e011bc551411c903f35b1f436f92c2a021d89e3e9b2a12be1ffd632cb736 peer=0
2020-08-24 23:37:04   - Load block from disk: 0.00ms [0.01s]
2020-08-24 23:37:04     - Sanity checks: 0.02ms [0.01s]
2020-08-24 23:37:04     - Fork checks: 0.03ms [0.01s]
2020-08-24 23:37:04       - Connect 2 transactions: 0.01ms (0.007ms/tx, 0.015ms/txin) [0.01s]
2020-08-24 23:37:04     - Verify 1 txins: 0.03ms (0.026ms/txin) [0.02s]
2020-08-24 23:37:04       - IS filter: 0.01ms [0.01s]
2020-08-24 23:37:04       - GetBlockSubsidy: 0.01ms [0.00s]
2020-08-24 23:37:04       - IsBlockValueValid: 0.02ms [0.01s]
2020-08-24 23:37:04       - IsBlockPayeeValid: 1.63ms [0.62s]
2020-08-24 23:37:04         - Loop: 0.01ms [0.07s]
2020-08-24 23:37:04         - quorumBlockProcessor: 0.04ms [0.41s]
2020-08-24 23:37:04         - deterministicMNManager: 3.06ms [1.14s]
2020-08-24 23:37:04           - GetTxPayload: 0.00ms [0.00s]
2020-08-24 23:37:04             - BuildNewListFromBlock: 1.40ms [0.56s]
2020-08-24 23:37:04             - CSimplifiedMNList: 3.39ms [1.19s]
2020-08-24 23:37:04             - CalcMerkleRoot: 3.05ms [1.08s]
2020-08-24 23:37:04           - CalcCbTxMerkleRootMNList: 8.81ms [3.20s]
2020-08-24 23:37:04             - GetMinedAndActiveCommitmentsUntilBlock: 0.19ms [0.07s]
2020-08-24 23:37:04             - GetMinedCommitment: 0.01ms [0.09s]
2020-08-24 23:37:04             - Loop: 0.01ms [0.01s]
2020-08-24 23:37:04             - ComputeMerkleRoot: 0.01ms [0.00s]
2020-08-24 23:37:04           - CalcCbTxMerkleRootQuorums: 0.24ms [0.18s]
2020-08-24 23:37:04         - CheckCbTxMerkleRoots: 9.09ms [3.39s]
2020-08-24 23:37:04       - ProcessSpecialTxsInBlock: 12.22ms [5.01s]
2020-08-24 23:37:04     - Index writing: 0.07ms [0.03s]
2020-08-24 23:37:04     - Callbacks: 0.01ms [0.00s]
2020-08-24 23:37:04   - Connect total: 14.15ms [5.76s]
2020-08-24 23:37:04   - Flush: 0.09ms [0.04s]
2020-08-24 23:37:04   - Writing chainstate: 0.01ms [0.00s]


my father wears sneakers in the pool
barrysty1e
Hero Member
*****
Offline Offline

Activity: 625
Merit: 515



View Profile WWW
August 25, 2020, 12:01:39 AM
 #23

Quote from: DougM link=topic=5270775.msg55060150#msg55060150
Interestingly the timestamps for follow on blocks can be earlier in time...that was a bit of surprise to me at first, but I learned I can't assume sequential blocks are in temporal order.  For example:

https://www.blockchain.com/btc/block/32649  -->  2010-01-02 04:49
https://www.blockchain.com/btc/block/32650  -->  2010-01-02 02:50

Still reigns true now (https://github.com/bitcoin/bitcoin/blob/0.20/src/chain.h#L18-L30), unsure of the exact rational behind it..

my father wears sneakers in the pool
CardCoins
Jr. Member
*
Offline Offline

Activity: 31
Merit: 14


View Profile
August 25, 2020, 12:59:01 AM
 #24

When it comes to validating a block, depending on the composition of transactions, it can take upwards of a few seconds (even more in some pathological/adversarial scenarios) to complete.

perhaps if you use a commodore64..

doublesha of the raw transaction data per transaction in block,
store in list, then hash the list,
then compare this hash with the one provided in the blockheader,
then hash the blockheader and ensure its below target,
we're talking milliseconds here..

heres an example from dash (x11 is a bit more harsh than sha256d, only enforcing the point)..
Code:
2020-08-24 23:37:04 received: block (452 bytes) peer=0
2020-08-24 23:37:04 received block 4829e011bc551411c903f35b1f436f92c2a021d89e3e9b2a12be1ffd632cb736 peer=0
2020-08-24 23:37:04   - Load block from disk: 0.00ms [0.01s]
2020-08-24 23:37:04     - Sanity checks: 0.02ms [0.01s]
2020-08-24 23:37:04     - Fork checks: 0.03ms [0.01s]
2020-08-24 23:37:04       - Connect 2 transactions: 0.01ms (0.007ms/tx, 0.015ms/txin) [0.01s]
2020-08-24 23:37:04     - Verify 1 txins: 0.03ms (0.026ms/txin) [0.02s]
2020-08-24 23:37:04       - IS filter: 0.01ms [0.01s]
2020-08-24 23:37:04       - GetBlockSubsidy: 0.01ms [0.00s]
2020-08-24 23:37:04       - IsBlockValueValid: 0.02ms [0.01s]
2020-08-24 23:37:04       - IsBlockPayeeValid: 1.63ms [0.62s]
2020-08-24 23:37:04         - Loop: 0.01ms [0.07s]
2020-08-24 23:37:04         - quorumBlockProcessor: 0.04ms [0.41s]
2020-08-24 23:37:04         - deterministicMNManager: 3.06ms [1.14s]
2020-08-24 23:37:04           - GetTxPayload: 0.00ms [0.00s]
2020-08-24 23:37:04             - BuildNewListFromBlock: 1.40ms [0.56s]
2020-08-24 23:37:04             - CSimplifiedMNList: 3.39ms [1.19s]
2020-08-24 23:37:04             - CalcMerkleRoot: 3.05ms [1.08s]
2020-08-24 23:37:04           - CalcCbTxMerkleRootMNList: 8.81ms [3.20s]
2020-08-24 23:37:04             - GetMinedAndActiveCommitmentsUntilBlock: 0.19ms [0.07s]
2020-08-24 23:37:04             - GetMinedCommitment: 0.01ms [0.09s]
2020-08-24 23:37:04             - Loop: 0.01ms [0.01s]
2020-08-24 23:37:04             - ComputeMerkleRoot: 0.01ms [0.00s]
2020-08-24 23:37:04           - CalcCbTxMerkleRootQuorums: 0.24ms [0.18s]
2020-08-24 23:37:04         - CheckCbTxMerkleRoots: 9.09ms [3.39s]
2020-08-24 23:37:04       - ProcessSpecialTxsInBlock: 12.22ms [5.01s]
2020-08-24 23:37:04     - Index writing: 0.07ms [0.03s]
2020-08-24 23:37:04     - Callbacks: 0.01ms [0.00s]
2020-08-24 23:37:04   - Connect total: 14.15ms [5.76s]
2020-08-24 23:37:04   - Flush: 0.09ms [0.04s]
2020-08-24 23:37:04   - Writing chainstate: 0.01ms [0.00s]



In perfect circumstances block validation is quite fast, that's correct!

There are adversarial cases where block validation can take quite a bit longer then the block you have presented here. In fact, one of the most significant improvements that segwit introduced was designed to partially address this exact issue, as the signature hashing algorithm was O(n^2). Now it is O(n): https://github.com/bitcoin/bips/blob/master/bip-0143.mediawiki#motivation

While segwit addresses one of these complexities, there are still others (albeit much less vicious): https://bitslog.com/2017/04/17/new-quadratic-delays-in-bitcoin-scripts/

Even if you don't assume the block is intentionally adversarial, there are lots of other factors which can effect validation time, including how much of the UTXO set the validator has cached, and how much of the block's content is in that cache. The libraries and hardware they use is also an important factor. Sipa has a great stackexchange post which details some of this stuff here, he suggests blocks can take multiple seconds to validate: https://bitcoin.stackexchange.com/questions/50349/how-long-does-block-validation-take/50407#50407
Picard78
Member
**
Offline Offline

Activity: 68
Merit: 23


View Profile
August 25, 2020, 07:20:31 AM
 #25

There are 11 empty blocks in 966 blocks or 1.13% of the blocks are empty from Block 645,198 to 644,232 = 966.

645,198
645,098
645,090
645,027
644,937
644,928
644,644
644,543
644,498
644,394
644,232

Is it possible to download Block Header Data for analysis in a tool like Excel or MS Access?  I would like to analyze the average Block Size for the past 50,000 blocks.  Average Block Fee in satoshis for the past 100,000 blocks etc...   Also I am very curious how many Empty Blocks there are for the past 50,000 blocks.

Thank you
 
Picard78
Member
**
Offline Offline

Activity: 68
Merit: 23


View Profile
August 25, 2020, 07:53:13 AM
 #26

Andreas Antonopoulos discusses the reason for the Empty Blocks in the below video from 0 - 3:15 min.  According to him "it takes about 15 - 20 seconds to validate" a block.  He says this at the start of the video.

https://www.youtube.com/watch?v=dizF2S63RXY
NotFuzzyWarm
Legendary
*
Offline Offline

Activity: 2996
Merit: 2046


Evil beware: We have waffles!


View Profile
August 25, 2020, 04:01:52 PM
 #27

One has to wonder about the date of that video.
Per-Kano it takes around 120-150ms to fully validate blocks. Being the operator of a pool that only uses fully validated blocks and has NEVER mined an empty one, he knows what it takes.

- For bitcoin to succeed the community must police itself -    My info useful? Donations welcome! 1FuzzyWc2J8TMqeUQZ8yjE43Rwr7K3cxs9
How a miner mfgr SHOULD operate:  HagssFIN trip to Canaan
-Support Sidehacks miner development. Donations to:   1BURGERAXHH6Yi6LRybRJK7ybEm5m5HwTr
ranochigo
Legendary
*
Offline Offline

Activity: 2730
Merit: 3340

@ me if you need my response


View Profile
August 25, 2020, 04:27:16 PM
Last edit: August 25, 2020, 04:41:08 PM by ranochigo
Merited by NotFuzzyWarm (1), ETFbitcoin (1)
 #28


One has to wonder about the date of that video.
Per-Kano it takes around 120-150ms to fully validate blocks. Being the operator of a pool that only uses fully validated blocks and has NEVER mined an empty one, he knows what it takes.
Indeed. I remember Kano being an opponent of SPV mining and some drama between him and F2Pool. BIP152 would've improved the propagation timing further and the effects of SPV mining would've diminished from the past.

I can't comment too much on how Kano and Antpool handles their mining code nor the efficacy of their code because I can't have access to either. He does have sound arguments against SPV mining and miners are mining less empty blocks in the years after that incident.


Blockchair has database dumps here but you need to parse it to have anything useful :https://blockchair.com/dumps

I just remembered,  LoyceV has everything: https://bitcointalk.org/index.php?topic=5246271.0

NotFuzzyWarm
Legendary
*
Offline Offline

Activity: 2996
Merit: 2046


Evil beware: We have waffles!


View Profile
August 25, 2020, 06:52:16 PM
Last edit: August 26, 2020, 12:53:31 PM by NotFuzzyWarm
Merited by CardCoins (3), ranochigo (2)
 #29

Quote
Indeed. I remember Kano being an opponent of SPV mining and some drama between him and F2Pool.
As in f2pool and Antpool mining 4 empty blocks in a row in 2015(?) that were off-chain thanks to their jumping the gun to be 1st out the gate to start a new (un-validated) block by using results from their previous also empty and un-validated block? Cheesy

Turns out that someone else had won the Orphan race for that 1st empty & un-validated block that Antpool & f2 had based their chain on.... Oops. Thanks to the back-end monitoring that Kano has he was very quickly alerted to the issue and contacted Antpool ops about it who then quickly relayed the info to f2pool who promptly switched tracks back to the valid current chain and of course invalidating the off-chain blocks they found.

Kano.is uses a modified version of ckpool as its front-end in conjunction with KDB which also directly monitors the network via bitcoind. Most of the code is out there in gits. Mind you his changes to ckpool are not there despite him being one of the major developers of it (the 'k' stands for Kano) thanks to a certain someone kicking him off the ckpool github due to irreconcilable differences in their views on proper coding/testing & pool financial safety. Anywho, the core code is there for anyone to peruse & use but Kano's mods to ckpool and updates to his KDB remain private. Since he does not distribute his changes that does not violate the GPL .

- For bitcoin to succeed the community must police itself -    My info useful? Donations welcome! 1FuzzyWc2J8TMqeUQZ8yjE43Rwr7K3cxs9
How a miner mfgr SHOULD operate:  HagssFIN trip to Canaan
-Support Sidehacks miner development. Donations to:   1BURGERAXHH6Yi6LRybRJK7ybEm5m5HwTr
Picard78
Member
**
Offline Offline

Activity: 68
Merit: 23


View Profile
August 26, 2020, 07:05:37 AM
 #30

Quote
I just remembered,  LoyceV has everything: https://bitcointalk.org/index.php?topic=5246271.0

Thank you ranochigo.  This is exactly what I was looking for.

Since block 600,000 (Oct 19, 2019) or in the last 45,343 blocks there were:

- 184 empty blocks which is ~ 0.4% of 45,343 blocks.

- Average block size of 1,116,072 bytes    

- Average block fees USD $3,145.  Average Block Coinbase reward USD $90,806.  Average profit from fees ~3.5% per block.

- Maximum USD $ / KB is $23 on 2020-02-01 and 2020-07-28.   (Note: Fees were over $100 / KB for many blocks in Dec 2017)
Picard78
Member
**
Offline Offline

Activity: 68
Merit: 23


View Profile
August 26, 2020, 07:32:50 AM
Last edit: August 26, 2020, 07:46:52 AM by Picard78
 #31

Since block 600,000 (Oct 19, 2019) or in the last 45,343 blocks:

                   Blocks Mined  Percent   Empty Blocks
1   Unknown   8,448   18.63%     11
2   F2Pool   7,797   17.20%     56
3   BTC.com   5,765   12.71%     33
4   AntPool   5,078   11.20%     23
5   Poolin   4,475   9.87%        9
6   ViaBTC   2,818   6.21%        6
7   SlushPool   2,004   4.42%       10
8   Huobi   1,842   4.06%       14

Hmmm... this was formatted properly before I posted it.



Picard78
Member
**
Offline Offline

Activity: 68
Merit: 23


View Profile
August 26, 2020, 07:43:13 AM
 #32

Since block 600,000 (Oct 19, 2019) or in the last 45,343 blocks:

The average block mining time is 9.892152041 min

Very nice.
DarkDays
Legendary
*
Offline Offline

Activity: 2002
Merit: 1188


View Profile
August 31, 2020, 08:47:57 PM
 #33

It's perfectly normal and is usually a result of SPV mining.

When a block is discovered, time is taken for the block to propagate and for the miners to verify the entire block. This can take a few seconds and those time would be wasted if the ASICs and the miners were just to idle and wait for the pool to validate the block. Enter SPV mining, SPV mining takes the block header of the previous block and create a new block header without including any transactions so as to prevent any conflicting transactions.

After the block is fully validated, the pool goes ahead to do everything normally and compile the unconfirmed transactions from the mempool. These kinds of blocks are usually observed when the blocks are mined in quick succession.

But wouldn't this create many empty blocks eventually after the decades of transactions? I know right now the size limited by that single reward it gives out to the miners. But can't a sidechain be implemented for the rewards itself? I do count myself among those few members that think that empty blocks are indeed a waste of resources. All it would take to stop empty blocks from being mined is simply not allowing miners to solve a block's hash without downloading the entire content for that block. I think a simple line of code could even solve this. No?

I personally think the block reward should be stripped from empty or stuffed blocks. It's just causing inflation for now real benefit.
pooya87
Legendary
*
Offline Offline

Activity: 2814
Merit: 7229


#moon


View Profile
September 01, 2020, 03:54:37 AM
 #34

But wouldn't this create many empty blocks eventually after the decades of transactions?
no, because we are talking about finding the hash within a ~second and the chances of that happening is very small that it has happened very rarely in the past 646k blocks.

All it would take to stop empty blocks from being mined is simply not allowing miners to solve a block's hash without downloading the entire content for that block. I think a simple line of code could even solve this. No?
it is not that simple and there is already a much better solution: incentive. when the price is high and block reward is low and the total fee is a meaningful amount of money, it means if the miner finds an empty block they lose all that extra income.

I personally think the block reward should be stripped from empty or stuffed blocks. It's just causing inflation for now real benefit.
the bitcoin design is that you get paid for the "work" you have done and you do pretty much the same amount of work for an empty block or for a full block so you get paid for it all the same, meaning the reward can not change.

wanted sliter
Member
**
Offline Offline

Activity: 402
Merit: 10


View Profile
September 05, 2020, 10:42:27 AM
 #35

https://www.blockchain.com/btc/block/644927
https://www.blockchain.com/btc/block/644928
Observing two adjacent blocks 644927 and 644928, we can see them mined in the same minute. Block 644927 has 612 transactions and the next block 644928 has only 1 transaction.
Since the mining speed is too fast, transactions cannot be sent in this block, so only 1 transaction can be made.

█▐█ ██   100xCoin   ██ █▌█     {  On Our Way to $1B Market Cap & Beyond  }
⦁  Whitepaper  ⦁    Telegram    ⦁      Twitter      ⦁     Medium     ⦁      Reddit      ⦁
▬▬▬▬▬▬▬▬▬▬▬▬▬  ▬▬  Powered by BOUNTY DETECTIVE  ▬▬  ▬▬▬▬▬▬▬▬▬▬▬▬▬
Pages: « 1 [2]  All
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!