jonnybravo0311 (OP)
Legendary
Offline
Activity: 1344
Merit: 1023
Mine at Jonny's Pool
|
![](https://bitcointalk.org/Themes/custom1/images/post/question.gif) |
June 09, 2015, 09:05:35 PM Last edit: July 09, 2015, 03:23:40 PM by jonnybravo0311 |
|
There's been quite a bit of discussion on this topic throughout the forum. Some feel empty blocks are valuable. Others feel they are of no value. I thought it might be fun to write a small program to see just how many of these blocks exist in the blockchain. For those who don't know, an empty block is a block that is mined containing only the coinbase transaction that awards the new coins. To be honest, I was quite frankly surprised by the number there were, especially since people seemed to think they were rare occurrences. Sure, I expected there to be quite a few at the beginning of the blockchain when BTC was pretty much an unknown and only a select few were mining it. However, I was not expecting there to be so many empty blocks as there are. As of block 360189 there are 85295 empty blocks on the chain. For the math challenged, 23.68% of all blocks are empty. 8 of the last 189 blocks are empty. 25 of the last 1000 are empty. Maybe I'll get ambitious and write the output to a spreadsheet so I can get some good data points out of this to see trends. Where do you stand on this debate? EDIT: I've uploaded the data from my last run. https://www.dropbox.com/s/2rt64pt6hyhr41b/btc_stats.zip?dl=0. Has blocks up to 364144 Archive contains two files: empty_blocks.csv has the raw data in the following format: Block Height, NumTx, BlockTime (UTC), BlockSize (Bytes), WhoMined (if I could figure it out, BTC address of coinbase transaction if I couldn't, or unknown if no BTC address could be deciphered from coinbase transaction). stats.csv has data on pools, how many blocks they've mined, how many were empty and percentage of empty to total. Enjoy.
|
Jonny's Pool - Mine with us and help us grow! Support a pool that supports Bitcoin, not a hardware manufacturer's pockets! No SPV cheats. No empty blocks.
|
|
|
chalkboard17
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
June 09, 2015, 09:20:47 PM |
|
Empty blocks aren't useless as they confirm previous transactions and secure them further against attacks. Though, they are much less useful than normal blocks. I am assuming the 23% number is skewed by 2009 and 2010 blocks which were mosly empty due to not having actual transactions rather than being empty on purpose by miners. 25 out of 1000 last which makes 2.5% seem much more right to work on. I think miners shouldn't mine empty blocks because they would lose tx fee and make people await more time for their tx to be included in the chain. Why do some pools do that? Is there an advantage?
|
|
|
|
os2sam
Legendary
Offline
Activity: 3582
Merit: 1094
Think for yourself
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
June 09, 2015, 09:27:33 PM |
|
Can you tell what pool(s) are currently mining 0 transaction blocks consistently? Thanks, Sam
|
A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing on usenet and in e-mail?
|
|
|
TheRealSteve
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
June 09, 2015, 09:28:11 PM |
|
Why do some pools do that? Is there an advantage? Time. Get the details of the last block mined, immediately start mining for the next - at this point 'empty' - block (and your chance at the block subsidy). Then while things are busily mining away, build new headers that do include transactions, and at the next opportune moment start mining on that. It's just a small sliver of time, but against an otherwise equal opponent, collecting transactions by default would be a losing strategy in the long run.
|
|
|
|
jonnybravo0311 (OP)
Legendary
Offline
Activity: 1344
Merit: 1023
Mine at Jonny's Pool
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
June 09, 2015, 09:37:54 PM |
|
Can you tell what pool(s) are currently mining 0 transaction blocks consistently? Thanks, Sam
Not from the data I pulled since the core API doesn't provide it - I ran this against my local node. I don't know how happy blockchain.info would be if I pummeled them with 86k requests ![Tongue](https://bitcointalk.org/Smileys/default/tongue.gif) . I am assuming the 23% number is skewed by 2009 and 2010 blocks which were mosly empty due to not having actual transactions rather than being empty on purpose by miners.
I'm sure the earlier blocks had far more empties. Like I wrote, maybe I'll get ambitious and plot this in a spreadsheet. I mostly wrote the code for fun just to see how many blocks are empty. Why do some pools do that? Is there an advantage? Time. Get the details of the last block mined, immediately start mining for the next - at this point 'empty' - block (and your chance at the block subsidy). Then while things are busily mining away, build new headers that do include transactions, and at the next opportune moment start mining on that. It's just a small sliver of time, but against an otherwise equal opponent, collecting transactions by default would be a losing strategy in the long run. This is where there's plenty of disagreement between pool software writers. On one side of the fence you've got luke-jr and wizkid who support the empty block theory. On the other you've got ck/kano who do not support it at all.
|
Jonny's Pool - Mine with us and help us grow! Support a pool that supports Bitcoin, not a hardware manufacturer's pockets! No SPV cheats. No empty blocks.
|
|
|
alani123
Legendary
Offline
Activity: 2506
Merit: 1468
Leading Crypto Sports Betting & Casino Platform
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
June 09, 2015, 09:42:18 PM |
|
Weren't 0 transaction blocks dominating the testnet recently because of a pool? Other than that, here are some stats by blockr https://btc.blockr.io/trivia/blockBlocks with 1 transaction: 85294 Blocks with 2 transactions: 12438
Edit: 1 transaction in a block essentially means that it's empty.
|
..Stake.com.. | | | ▄████████████████████████████████████▄ ██ ▄▄▄▄▄▄▄▄▄▄ ▄▄▄▄▄▄▄▄▄▄ ██ ▄████▄ ██ ▀▀▀▀▀▀▀▀▀▀ ██████████ ▀▀▀▀▀▀▀▀▀▀ ██ ██████ ██ ██████████ ██ ██ ██████████ ██ ▀██▀ ██ ██ ██ ██████ ██ ██ ██ ██ ██ ██ ██████ ██ █████ ███ ██████ ██ ████▄ ██ ██ █████ ███ ████ ████ █████ ███ ████████ ██ ████ ████ ██████████ ████ ████ ████▀ ██ ██████████ ▄▄▄▄▄▄▄▄▄▄ ██████████ ██ ██ ▀▀▀▀▀▀▀▀▀▀ ██ ▀█████████▀ ▄████████████▄ ▀█████████▀ ▄▄▄▄▄▄▄▄▄▄▄▄███ ██ ██ ███▄▄▄▄▄▄▄▄▄▄▄▄ ██████████████████████████████████████████ | | | | | | ▄▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▄ █ ▄▀▄ █▀▀█▀▄▄ █ █▀█ █ ▐ ▐▌ █ ▄██▄ █ ▌ █ █ ▄██████▄ █ ▌ ▐▌ █ ██████████ █ ▐ █ █ ▐██████████▌ █ ▐ ▐▌ █ ▀▀██████▀▀ █ ▌ █ █ ▄▄▄██▄▄▄ █ ▌▐▌ █ █▐ █ █ █▐▐▌ █ █▐█ ▀▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▀█ | | | | | | ▄▄█████████▄▄ ▄██▀▀▀▀█████▀▀▀▀██▄ ▄█▀ ▐█▌ ▀█▄ ██ ▐█▌ ██ ████▄ ▄█████▄ ▄████ ████████▄███████████▄████████ ███▀ █████████████ ▀███ ██ ███████████ ██ ▀█▄ █████████ ▄█▀ ▀█▄ ▄██▀▀▀▀▀▀▀██▄ ▄▄▄█▀ ▀███████ ███████▀ ▀█████▄ ▄█████▀ ▀▀▀███▄▄▄███▀▀▀ | | | ..PLAY NOW.. |
|
|
|
jonnybravo0311 (OP)
Legendary
Offline
Activity: 1344
Merit: 1023
Mine at Jonny's Pool
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
June 09, 2015, 09:58:29 PM |
|
Weren't 0 transaction blocks dominating the testnet recently because of a pool? Other than that, here are some stats by blockr https://btc.blockr.io/trivia/blockBlocks with 1 transaction: 85294 Blocks with 2 transactions: 12438
Edit: 1 transaction in a block essentially means that it's empty. That's a pretty cool site... didn't know it existed, or I wouldn't have bothered writing my own code to get the results ![Smiley](https://bitcointalk.org/Smileys/default/smiley.gif) . At least we both match up (I included the genesis block whereas they didn't).
|
Jonny's Pool - Mine with us and help us grow! Support a pool that supports Bitcoin, not a hardware manufacturer's pockets! No SPV cheats. No empty blocks.
|
|
|
TheRealSteve
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
June 09, 2015, 10:04:23 PM Last edit: June 11, 2015, 06:17:03 PM by TheRealSteve |
|
Can you tell what pool(s) are currently mining 0 transaction blocks consistently?
Not from the data I pulled since the core API doesn't provide it - I ran this against my local node. I don't know how happy blockchain.info would be if I pummeled them with 86k requests ![Tongue](https://bitcointalk.org/Smileys/default/tongue.gif) . I ran my parser against a dataset - wasn't really designed for this - but here's what I've got: Please see update post Note: This post previously contained an image based on incorrect data. See: https://bitcointalk.org/index.php?topic=1085800.msg11578412#msg11578412The old image, for reference, can be found at: https://i.imgur.com/RlhKM97.pngComments below up to kano's post identifying a problem are regarding this old image.
Why do some pools do that? Is there an advantage? Time. This is where there's plenty of disagreement between pool software writers. On one side of the fence you've got luke-jr and wizkid who support the empty block theory. On the other you've got ck/kano who do not support it at all. Disagreement is healthy ![Smiley](https://bitcointalk.org/Smileys/default/smiley.gif)
|
|
|
|
os2sam
Legendary
Offline
Activity: 3582
Merit: 1094
Think for yourself
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
June 09, 2015, 10:19:09 PM |
|
Can you tell what pool(s) are currently mining 0 transaction blocks consistently?
Not from the data I pulled since the core API doesn't provide it - I ran this against my local node. I don't know how happy blockchain.info would be if I pummeled them with 86k requests ![Tongue](https://bitcointalk.org/Smileys/default/tongue.gif) . I ran my parser against a dataset - wasn't really designed for this - but here's what I've got: What is the "dataset"? Doesn't look like anyone is purposely mining empty blocks on a regular basis. So, that's a good thing. Thanks, Sam
|
A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing on usenet and in e-mail?
|
|
|
TheRealSteve
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
June 09, 2015, 10:22:13 PM |
|
What is the "dataset"?
Last 10,197 blocks (at the time I ran it), data sourced from blocktrail.com . They have a paginated API (200 results per page, so 51 requests) that at least has info on the block height, timestamp, number of transactions, size of the block, apparent pool that mined it and some other info.
|
|
|
|
achow101
Staff
Legendary
Offline
Activity: 3486
Merit: 6824
Just writing some code
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
June 09, 2015, 10:41:45 PM |
|
Can we see the code that creates this?
|
|
|
|
TheRealSteve
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
June 09, 2015, 10:55:31 PM |
|
Can we see the code that creates this? I could give you the pseudocode for the poller - all the rest is just LibreOffice Base + Calc bits - but that's not all that interesting either. Basically, take this URL: https://api.blocktrail.com/v1/btc/all-blocks?sort_dir=desc&limit=200&api_key=YOUR_API_KEY_HERE&page=, concatenate with a page number that increments until either you find the page number of interest (or in my case, the block of interest). Download the concatenated URL. Parse result with whatever JSON library you're happy with. I only read out height, is_orphan, block_time and miningpool_slug originally, added transactions and byte_size for this thread. They offer libraries for Python, PHP and NodeJS, so.. pretty easy to get started with. You'll need an API key, of course, but that's as easy as signing up, going to the relevant page, and off you go. ( Do keep quota and throttles in mind - they're pretty lax right now, but be a good API citizen and all that. This includes not sharing the actual polled data, as per their T&C. )
|
|
|
|
Mikestang
Legendary
Offline
Activity: 1274
Merit: 1000
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
June 09, 2015, 10:57:10 PM |
|
There's been quite a bit of discussion on this topic throughout the forum. Some feel empty blocks are valuable. Others feel they are of no value.
I thought it might be fun to write a small program to see just how many of these blocks exist in the blockchain. For those who don't know, an empty block is a block that is mined containing only the coinbase transaction that awards the new coins. To be honest, I was quite frankly surprised by the number there were, especially since people seemed to think they were rare occurrences. Sure, I expected there to be quite a few at the beginning of the blockchain when BTC was pretty much an unknown and only a select few were mining it. However, I was not expecting there to be so many empty blocks as there are.
As of block 360189 there are 85295 empty blocks on the chain. For the math challenged, 23.68% of all blocks are empty. 8 of the last 189 blocks are empty. 25 of the last 1000 are empty.
Maybe I'll get ambitious and write the output to a spreadsheet so I can get some good data points out of this to see trends.
Where do you stand on this debate?
Thank you for the research, the numbers are enlightening. I tend to fall on the side of the discussion that holds that blocks should contain transactions, since that is their ultimate purpose.
|
|
|
|
kano
Legendary
Offline
Activity: 4578
Merit: 1851
Linux since 1997 RedHat 4
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
June 09, 2015, 11:03:08 PM |
|
Can you tell what pool(s) are currently mining 0 transaction blocks consistently?
Not from the data I pulled since the core API doesn't provide it - I ran this against my local node. I don't know how happy blockchain.info would be if I pummeled them with 86k requests ![Tongue](https://bitcointalk.org/Smileys/default/tongue.gif) . I ran my parser against a dataset - wasn't really designed for this - but here's what I've got: https://i.imgur.com/RlhKM97.pngWhy do some pools do that? Is there an advantage? Time. This is where there's plenty of disagreement between pool software writers. On one side of the fence you've got luke-jr and wizkid who support the empty block theory. On the other you've got ck/kano who do not support it at all. Disagreement is healthy ![Smiley](https://bitcointalk.org/Smileys/default/smiley.gif) Your calculations must be wrong. None of the blocks on my pool kano.is have 0 non-coinbase transactions. One recently only had about 10 since it was a few seconds after another one on the internet.
|
|
|
|
TheRealSteve
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
June 09, 2015, 11:06:36 PM |
|
Thank you for the research, the numbers are enlightening. I tend to fall on the side of the discussion that holds that blocks should contain transactions, since that is their ultimate purpose.
Oh I absolutely agree with that. I thought you were referring to disagreement on whether or not the empty-block-first mining strategy was actually advantageous. I seem to recall some reddit chatter about the impact being negligible compared to other factor and minor debate about whether there was in fact an actual advantage ![Smiley](https://bitcointalk.org/Smileys/default/smiley.gif)
|
|
|
|
TheRealSteve
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
June 09, 2015, 11:10:11 PM |
|
Your calculations must be wrong. None of the blocks on my pool kano.is have 0 non-coinbase transactions. One recently only had about 10 since it was a few seconds after another one on the internet.
Probably - let me check what it's doing.
|
|
|
|
TheRealSteve
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
June 09, 2015, 11:27:59 PM |
|
Probably - let me check what it's doing.
Stared myself blind on the code, couldn't find any issues. Turns out it's in the API. E.g. "height":360206,"block_time":"2015-06-09T22:17:46+0000",[...],"byte_size":230144,"confirmations":7,"transactions":1That's not right: https://www.blocktrail.com/BTC/block/00000000000000000f757987312e482d3edd6a019d9eea5eb62e0b4a13e51e05I'll drop them a line about this later. In the mean time, it looks like the byte_size is correct, so I'll redo the stats image.
|
|
|
|
jonnybravo0311 (OP)
Legendary
Offline
Activity: 1344
Merit: 1023
Mine at Jonny's Pool
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
June 10, 2015, 12:18:03 AM |
|
OK... this has probably been answered before, so forgive me for asking it. Where do I find out which pool mined a block? I can't see anything useful in the output of the core API, and the "relayed_by" value provided by the blockchain.info API doesn't appear to be what I'm looking for, either. I figure it's probably in the output script for the coinbase transaction. Can somebody give me a quick rundown of how to decode that (other than passing it to bitcoin-cli)... like some pseudocode, or a link with some info?
For example, looking at the block TheRealSteve used in his last response, I can clearly see that on blockchain.info it shows it was submitted by Eligius. Looking at the coinbase output script it shows a bunch of garbled crap with Eligius buried in it. Is there a nice way to extract the info from there, or is it just a messy thing like, "if decoded script contains eligius, then the block was found by Eligius. If decoded script contains antpool, then it was found by antpool, etc"?
|
Jonny's Pool - Mine with us and help us grow! Support a pool that supports Bitcoin, not a hardware manufacturer's pockets! No SPV cheats. No empty blocks.
|
|
|
TheRealSteve
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
June 10, 2015, 12:33:33 AM |
|
Is there a nice way to extract the info from there, or is it just a messy thing like, "if decoded script contains eligius, then the block was found by Eligius. If decoded script contains antpool, then it was found by antpool, etc"?
It's a messy thing. There's actually a write-up about it here: Don't believe everything you read on blockchain.info about block sourcesBlocktrail also does the messy thing, as does whomined.com . You can ready whomined's approach here: http://whomined.com/#how-it-works. I'd imagine there's libraries that'll decode the scriptSig without going through Bitcoin Core.
|
|
|
|
kano
Legendary
Offline
Activity: 4578
Merit: 1851
Linux since 1997 RedHat 4
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
June 10, 2015, 02:14:56 AM |
|
Just looked up kano.is smallest block: 29 transactions. https://blockchain.info/block-index/878311/00000000000000000880afbc23f0176372837c3edc9251763a4268edac280299--- When trying to find empty blocks, of course the transaction count is what matters. If the data source is no good, the next best bet is the block reward. If it's only 25BTC then it is very likely to be an empty block (I'd doubt there are few if any recent blocks that have only free transactions) The block size doesn't help since there are 2 pools with very large coinbase transactions. p2pool is one of them.
|
|
|
|
|