Bitcoin Forum
February 19, 2019, 10:26:44 PM *
News: Latest Bitcoin Core release: 0.17.1 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1] 2 3 4 5 6 7 8 9 10 »  All
  Print  
Author Topic: Empty blocks  (Read 22374 times)
jonnybravo0311
Legendary
*
Offline Offline

Activity: 1344
Merit: 1015


Mine at Jonny's Pool


View Profile WWW
June 09, 2015, 09:05:35 PM
Last edit: July 09, 2015, 03:23:40 PM by jonnybravo0311
 #1

There's been quite a bit of discussion on this topic throughout the forum.  Some feel empty blocks are valuable.  Others feel they are of no value.

I thought it might be fun to write a small program to see just how many of these blocks exist in the blockchain.  For those who don't know, an empty block is a block that is mined containing only the coinbase transaction that awards the new coins.  To be honest, I was quite frankly surprised by the number there were, especially since people seemed to think they were rare occurrences.  Sure, I expected there to be quite a few at the beginning of the blockchain when BTC was pretty much an unknown and only a select few were mining it.  However, I was not expecting there to be so many empty blocks as there are.

As of block 360189 there are 85295 empty blocks on the chain.  For the math challenged, 23.68% of all blocks are empty.  8 of the last 189 blocks are empty.  25 of the last 1000 are empty.

Maybe I'll get ambitious and write the output to a spreadsheet so I can get some good data points out of this to see trends.

Where do you stand on this debate?

EDIT: I've uploaded the data from my last run.

https://www.dropbox.com/s/2rt64pt6hyhr41b/btc_stats.zip?dl=0.  Has blocks up to 364144

Archive contains two files:

empty_blocks.csv has the raw data in the following format:

Block Height, NumTx, BlockTime (UTC), BlockSize (Bytes), WhoMined (if I could figure it out, BTC address of coinbase transaction if I couldn't, or unknown if no BTC address could be deciphered from coinbase transaction).

stats.csv has data on pools, how many blocks they've mined, how many were empty and percentage of empty to total.

Enjoy.

Jonny's Pool - Mine with us and help us grow!  Support a pool that supports Bitcoin, not a hardware manufacturer's pockets!  No SPV cheats.  No empty blocks.
1550615204
Hero Member
*
Offline Offline

Posts: 1550615204

View Profile Personal Message (Offline)

Ignore
1550615204
Reply with quote  #2

1550615204
Report to moderator
1550615204
Hero Member
*
Offline Offline

Posts: 1550615204

View Profile Personal Message (Offline)

Ignore
1550615204
Reply with quote  #2

1550615204
Report to moderator
Your Bitcoin transactions
The Ultimate Bitcoin mixer
made truly anonymous.
with an advanced technology.
Mix coins
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1550615204
Hero Member
*
Offline Offline

Posts: 1550615204

View Profile Personal Message (Offline)

Ignore
1550615204
Reply with quote  #2

1550615204
Report to moderator
1550615204
Hero Member
*
Offline Offline

Posts: 1550615204

View Profile Personal Message (Offline)

Ignore
1550615204
Reply with quote  #2

1550615204
Report to moderator
chalkboard17
Sr. Member
****
Offline Offline

Activity: 481
Merit: 250



View Profile
June 09, 2015, 09:20:47 PM
 #2

Empty blocks aren't useless as they confirm previous transactions and secure them further against attacks. Though, they are much less useful than normal blocks. I am assuming the 23% number is skewed by 2009 and 2010 blocks which were mosly empty due to not having actual transactions rather than being empty on purpose by miners. 25 out of 1000 last which makes 2.5% seem much more right to work on.
I think miners shouldn't mine empty blocks because they would lose tx fee and make people await more time for their tx to be included in the chain. Why do some pools do that? Is there an advantage?

                ▄▄  ▄▄                
            ██  ▀▀  ▀▀   ██           
        ██                   ██
       
                ██  ██  ▄▄            
     ██    ██           ▀▀  ▄▄        
                  ███       ▀▀        
   ██    ██   ███      ███     ██     
                          ███         
  ██   ██   ██    ███ ███    ▄▄   ██  
               ███           ▀▀       
  ██   ██  ███           ███  ██   ██ 
                     ███              
    ▄▄  ██    ███ ███     ▄▄  ██   ██ 
    ▀▀    ▄▄              ▀▀          
      ▄▄  ▀▀          ███    ██   ██  
      ▀▀      ██  ███                 
         ██              ███    ███   
             ██  ██  ███              
       ██                    ██       
           ███  ▄▄▄  ▄▄  ███          
                ▀▀▀  ▀▀               
 
STREAMITY
 

 

  Twitter
Facebook
Instagram
  Telegram
LinkedIn
Medium
os2sam
Legendary
*
Offline Offline

Activity: 2562
Merit: 1003


Think for yourself


View Profile
June 09, 2015, 09:27:33 PM
 #3

Can you tell what pool(s) are currently mining 0 transaction blocks consistently?
Thanks,
Sam

A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?
TheRealSteve
Hero Member
*****
Offline Offline

Activity: 686
Merit: 500

FUN > ROI


View Profile
June 09, 2015, 09:28:11 PM
 #4

Why do some pools do that? Is there an advantage?
Time.  Get the details of the last block mined, immediately start mining for the next - at this point 'empty' - block (and your chance at the block subsidy).  Then while things are busily mining away, build new headers that do include transactions, and at the next opportune moment start mining on that. It's just a small sliver of time, but against an otherwise equal opponent, collecting transactions by default would be a losing strategy in the long run.

jonnybravo0311
Legendary
*
Offline Offline

Activity: 1344
Merit: 1015


Mine at Jonny's Pool


View Profile WWW
June 09, 2015, 09:37:54 PM
 #5

Can you tell what pool(s) are currently mining 0 transaction blocks consistently?
Thanks,
Sam
Not from the data I pulled since the core API doesn't provide it - I ran this against my local node.  I don't know how happy blockchain.info would be if I pummeled them with 86k requests Tongue.

I am assuming the 23% number is skewed by 2009 and 2010 blocks which were mosly empty due to not having actual transactions rather than being empty on purpose by miners.
I'm sure the earlier blocks had far more empties.  Like I wrote, maybe I'll get ambitious and plot this in a spreadsheet.  I mostly wrote the code for fun just to see how many blocks are empty.

Why do some pools do that? Is there an advantage?
Time.  Get the details of the last block mined, immediately start mining for the next - at this point 'empty' - block (and your chance at the block subsidy).  Then while things are busily mining away, build new headers that do include transactions, and at the next opportune moment start mining on that. It's just a small sliver of time, but against an otherwise equal opponent, collecting transactions by default would be a losing strategy in the long run.
This is where there's plenty of disagreement between pool software writers.  On one side of the fence you've got luke-jr and wizkid who support the empty block theory.  On the other you've got ck/kano who do not support it at all.

Jonny's Pool - Mine with us and help us grow!  Support a pool that supports Bitcoin, not a hardware manufacturer's pockets!  No SPV cheats.  No empty blocks.
alani123
Legendary
*
Offline Offline

Activity: 1876
Merit: 1039



View Profile
June 09, 2015, 09:42:18 PM
 #6

Weren't 0 transaction blocks dominating the testnet recently because of a pool? Other than that, here are some stats by blockr

https://btc.blockr.io/trivia/block

Quote
Blocks with 1 transaction: 85294  Blocks with 2 transactions: 12438

Edit: 1 transaction in a block essentially means that it's empty.



███             ▄▄▄███████▄▄▄          ████                   ████          ▄▄▄███████▄▄▄         
███         ▄███████████████████▄       ████                 ████       ▄███████████████████▄     
███       ▄██████▀▀       ▀▀██████▄      ████               ████      ▄██████▀▀       ▀▀██████▄   
███      █████▀               ▀█████      ████             ████      █████▀               ▀█████ 
███    ▐████▀                   ▀████▌     ████           ████     ▐████▀                   ▀████▌
███    ████▌                     ▐████      ████         ████      ████▌                     ▐████
███    ████                       ████       ████       ████       ████                       ████
███    ████                       ████        ████     ████        ████                       ████
███    ████▌                     ▐████         ████   ████         ████▌                     ▐████
███    ▐████▄                   ▄████▌          ████ ████          ▐████▄                   ▄████▌
███      █████▄               ▄█████             ███████             █████▄               ▄█████ 
███       ▀██████▄▄       ▄▄██████▀               █████               ▀██████▄▄       ▄▄██████▀   
███         ▀███████████████████▀                  ███                  ▀███████████████████▀     
███             ▀▀▀███████▀▀▀                       █                       ▀▀▀███████▀▀▀         
INTERNET OF VALUE OMNILEDGER
|
   
   
 
 
|


             ▄████▄▄   ▄
█▄          ██████████▀▄
███        ███████████▀
▐████▄     ██████████▌
▄▄██████▄▄▄▄█████████▌
▀████████████████████
  ▀█████████████████
  ▄▄███████████████
   ▀█████████████▀
    ▄▄█████████▀
▀▀██████████▀
    ▀▀▀▀▀
M
 

             █▀▀▀▄▄▄██▄
             █     ▀██▀
            █
         ▄▄▄█▄▄▄
 ████▄▄███████████▄▄████
▐██████▀▀███████▀▀██████▌
 ▀████    █████    ████▀
  ████▄  ▄█████▄  ▄████
  ▀███████████████████▀
   ▀████▄▀█████▀▄████▀
     ▀▀███▄▄▄▄▄███▀▀
         ▀▀▀▀▀▀▀
 

    ▄█████
   ████▀▀▀
   ████
   ████
██████████
▀▀▀████▀▀
   ████
   ████
   ████
   ████
   ████
   ▀▀▀▀
|
jonnybravo0311
Legendary
*
Offline Offline

Activity: 1344
Merit: 1015


Mine at Jonny's Pool


View Profile WWW
June 09, 2015, 09:58:29 PM
 #7

Weren't 0 transaction blocks dominating the testnet recently because of a pool? Other than that, here are some stats by blockr

https://btc.blockr.io/trivia/block

Quote
Blocks with 1 transaction: 85294  Blocks with 2 transactions: 12438

Edit: 1 transaction in a block essentially means that it's empty.
That's a pretty cool site... didn't know it existed, or I wouldn't have bothered writing my own code to get the results Smiley.  At least we both match up (I included the genesis block whereas they didn't).

Jonny's Pool - Mine with us and help us grow!  Support a pool that supports Bitcoin, not a hardware manufacturer's pockets!  No SPV cheats.  No empty blocks.
TheRealSteve
Hero Member
*****
Offline Offline

Activity: 686
Merit: 500

FUN > ROI


View Profile
June 09, 2015, 10:04:23 PM
Last edit: June 11, 2015, 06:17:03 PM by TheRealSteve
 #8

Can you tell what pool(s) are currently mining 0 transaction blocks consistently?
Not from the data I pulled since the core API doesn't provide it - I ran this against my local node.  I don't know how happy blockchain.info would be if I pummeled them with 86k requests Tongue.
I ran my parser against a dataset - wasn't really designed for this - but here's what I've got:
Please see update post


Note: This post previously contained an image based on incorrect data.  See: https://bitcointalk.org/index.php?topic=1085800.msg11578412#msg11578412

The old image, for reference, can be found at: http://i.imgur.com/RlhKM97.png

Comments below up to kano's post identifying a problem are regarding this old image.



Why do some pools do that? Is there an advantage?
Time.
This is where there's plenty of disagreement between pool software writers.  On one side of the fence you've got luke-jr and wizkid who support the empty block theory.  On the other you've got ck/kano who do not support it at all.
Disagreement is healthy Smiley

os2sam
Legendary
*
Offline Offline

Activity: 2562
Merit: 1003


Think for yourself


View Profile
June 09, 2015, 10:19:09 PM
 #9

Can you tell what pool(s) are currently mining 0 transaction blocks consistently?
Not from the data I pulled since the core API doesn't provide it - I ran this against my local node.  I don't know how happy blockchain.info would be if I pummeled them with 86k requests Tongue.
I ran my parser against a dataset - wasn't really designed for this - but here's what I've got:


What is the "dataset"?

Doesn't look like anyone is purposely mining empty blocks on a regular basis.  So, that's a good thing.
Thanks,
Sam

A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?
TheRealSteve
Hero Member
*****
Offline Offline

Activity: 686
Merit: 500

FUN > ROI


View Profile
June 09, 2015, 10:22:13 PM
 #10

What is the "dataset"?
Last 10,197 blocks (at the time I ran it), data sourced from blocktrail.com .  They have a paginated API (200 results per page, so 51 requests) that at least has info on the block height, timestamp, number of transactions, size of the block, apparent pool that mined it and some other info.

achow101
Staff
Legendary
*
Offline Offline

Activity: 1680
Merit: 2083


bc1qshxkrpe4arppq89fpzm6c0tpdvx5cfkve2c8kl


View Profile WWW
June 09, 2015, 10:41:45 PM
 #11

Can we see the code that creates this?

TheRealSteve
Hero Member
*****
Offline Offline

Activity: 686
Merit: 500

FUN > ROI


View Profile
June 09, 2015, 10:55:31 PM
 #12

Can we see the code that creates this?

I could give you the pseudocode for the poller - all the rest is just LibreOffice Base + Calc bits - but that's not all that interesting either.  Basically, take this URL: https://api.blocktrail.com/v1/btc/all-blocks?sort_dir=desc&limit=200&api_key=YOUR_API_KEY_HERE&page=, concatenate with a page number that increments until either you find the page number of interest (or in my case, the block of interest).  Download the concatenated URL.  Parse result with whatever JSON library you're happy with.  I only read out height, is_orphan, block_time and miningpool_slug originally, added transactions and byte_size for this thread.
They offer libraries for Python, PHP and NodeJS, so.. pretty easy to get started with.

You'll need an API key, of course, but that's as easy as signing up, going to the relevant page, and off you go.
( Do keep quota and throttles in mind - they're pretty lax right now, but be a good API citizen and all that. This includes not sharing the actual polled data, as per their T&C. )

Mikestang
Legendary
*
Offline Offline

Activity: 1218
Merit: 1000



View Profile
June 09, 2015, 10:57:10 PM
 #13

There's been quite a bit of discussion on this topic throughout the forum.  Some feel empty blocks are valuable.  Others feel they are of no value.

I thought it might be fun to write a small program to see just how many of these blocks exist in the blockchain.  For those who don't know, an empty block is a block that is mined containing only the coinbase transaction that awards the new coins.  To be honest, I was quite frankly surprised by the number there were, especially since people seemed to think they were rare occurrences.  Sure, I expected there to be quite a few at the beginning of the blockchain when BTC was pretty much an unknown and only a select few were mining it.  However, I was not expecting there to be so many empty blocks as there are.

As of block 360189 there are 85295 empty blocks on the chain.  For the math challenged, 23.68% of all blocks are empty.  8 of the last 189 blocks are empty.  25 of the last 1000 are empty.

Maybe I'll get ambitious and write the output to a spreadsheet so I can get some good data points out of this to see trends.

Where do you stand on this debate?

Thank you for the research, the numbers are enlightening.  I tend to fall on the side of the discussion that holds that blocks should contain transactions, since that is their ultimate purpose.
kano
Legendary
*
Offline Offline

Activity: 2730
Merit: 1103


Linux since 1997 RedHat 4


View Profile
June 09, 2015, 11:03:08 PM
 #14

Can you tell what pool(s) are currently mining 0 transaction blocks consistently?
Not from the data I pulled since the core API doesn't provide it - I ran this against my local node.  I don't know how happy blockchain.info would be if I pummeled them with 86k requests Tongue.
I ran my parser against a dataset - wasn't really designed for this - but here's what I've got:
http://i.imgur.com/RlhKM97.png


Why do some pools do that? Is there an advantage?
Time.
This is where there's plenty of disagreement between pool software writers.  On one side of the fence you've got luke-jr and wizkid who support the empty block theory.  On the other you've got ck/kano who do not support it at all.
Disagreement is healthy Smiley
Your calculations must be wrong.
None of the blocks on my pool kano.is have 0 non-coinbase transactions.
One recently only had about 10 since it was a few seconds after another one on the internet.

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
TheRealSteve
Hero Member
*****
Offline Offline

Activity: 686
Merit: 500

FUN > ROI


View Profile
June 09, 2015, 11:06:36 PM
 #15

Thank you for the research, the numbers are enlightening.  I tend to fall on the side of the discussion that holds that blocks should contain transactions, since that is their ultimate purpose.
Oh I absolutely agree with that.  I thought you were referring to disagreement on whether or not the empty-block-first mining strategy was actually advantageous.  I seem to recall some reddit chatter about the impact being negligible compared to other factor and minor debate about whether there was in fact an actual advantage Smiley

TheRealSteve
Hero Member
*****
Offline Offline

Activity: 686
Merit: 500

FUN > ROI


View Profile
June 09, 2015, 11:10:11 PM
 #16

Your calculations must be wrong.
None of the blocks on my pool kano.is have 0 non-coinbase transactions.
One recently only had about 10 since it was a few seconds after another one on the internet.
Probably - let me check what it's doing.

TheRealSteve
Hero Member
*****
Offline Offline

Activity: 686
Merit: 500

FUN > ROI


View Profile
June 09, 2015, 11:27:59 PM
 #17

Probably - let me check what it's doing.
Stared myself blind on the code, couldn't find any issues.  Turns out it's in the API.  E.g. "height":360206,"block_time":"2015-06-09T22:17:46+0000",[...],"byte_size":230144,"confirmations":7,"transactions":1
That's not right: https://www.blocktrail.com/BTC/block/00000000000000000f757987312e482d3edd6a019d9eea5eb62e0b4a13e51e05

I'll drop them a line about this later.  In the mean time, it looks like the byte_size is correct, so I'll redo the stats image.

jonnybravo0311
Legendary
*
Offline Offline

Activity: 1344
Merit: 1015


Mine at Jonny's Pool


View Profile WWW
June 10, 2015, 12:18:03 AM
 #18

OK... this has probably been answered before, so forgive me for asking it.  Where do I find out which pool mined a block?  I can't see anything useful in the output of the core API, and the "relayed_by" value provided by the blockchain.info API doesn't appear to be what I'm looking for, either.  I figure it's probably in the output script for the coinbase transaction.  Can somebody give me a quick rundown of how to decode that (other than passing it to bitcoin-cli)... like some pseudocode, or a link with some info?

For example, looking at the block TheRealSteve used in his last response, I can clearly see that on blockchain.info it shows it was submitted by Eligius.  Looking at the coinbase output script it shows a bunch of garbled crap with Eligius buried in it.  Is there a nice way to extract the info from there, or is it just a messy thing like, "if decoded script contains eligius, then the block was found by Eligius.  If decoded script contains antpool, then it was found by antpool, etc"?

Jonny's Pool - Mine with us and help us grow!  Support a pool that supports Bitcoin, not a hardware manufacturer's pockets!  No SPV cheats.  No empty blocks.
TheRealSteve
Hero Member
*****
Offline Offline

Activity: 686
Merit: 500

FUN > ROI


View Profile
June 10, 2015, 12:33:33 AM
 #19

Is there a nice way to extract the info from there, or is it just a messy thing like, "if decoded script contains eligius, then the block was found by Eligius.  If decoded script contains antpool, then it was found by antpool, etc"?
It's a messy thing.  There's actually a write-up about it here: Don't believe everything you read on blockchain.info about block sources

Blocktrail also does the messy thing, as does whomined.com .  You can ready whomined's approach here: http://whomined.com/#how-it-works.

I'd imagine there's libraries that'll decode the scriptSig without going through Bitcoin Core.

kano
Legendary
*
Offline Offline

Activity: 2730
Merit: 1103


Linux since 1997 RedHat 4


View Profile
June 10, 2015, 02:14:56 AM
 #20

Just looked up kano.is smallest block:

29 transactions.
https://blockchain.info/block-index/878311/00000000000000000880afbc23f0176372837c3edc9251763a4268edac280299

---

When trying to find empty blocks, of course the transaction count is what matters.

If the data source is no good, the next best bet is the block reward.
If it's only 25BTC then it is very likely to be an empty block (I'd doubt there are few if any recent blocks that have only free transactions)

The block size doesn't help since there are 2 pools with very large coinbase transactions. p2pool is one of them.

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
Pages: [1] 2 3 4 5 6 7 8 9 10 »  All
  Print  
 
Jump to:  

Bitcointalk.org is not available or authorized for sale. Do not believe any fake listings.
Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!