Bitcoin Forum
August 17, 2022, 05:18:04 PM *
News: Latest Bitcoin Core release: 23.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1] 2 3 »  All
  Print  
Author Topic: Bitcoin's Empty Blocks Analaysis.  (Read 740 times)
mikeywith
Legendary
*
Offline Offline

Activity: 1596
Merit: 4450


be constructive or S.T.F.U


View Profile
May 03, 2020, 12:34:57 AM
Last edit: May 03, 2020, 01:44:53 AM by mikeywith
Merited by LoyceV (10), ETFbitcoin (6), fillippone (5), tranthidung (4), philipma1957 (3), DdmrDdmr (3), pooya87 (2), ranochigo (2), 1miau (2), Heisenberg_Hunter (2), BitMaxz (1), Pmalek (1), nc50lc (1)
 #1

I was trying to find a recent analysis of empty blocks and I couldn't find any, It seems like most of the analysis is outdated, so I decided to make one using blockchair explorer with some filters.


The analysis is based on 5 years, starting from the first day of 2015 ending on the last day of 2019, I have purposely ignored 2020 since we don't have enough data for it.


I started by analyzing the monthly empty blocks.


Chart 1: Numbers of blocks found per month (total of 60 months)

Added a more detailed chart showing the total of empty blocks per year.


Chart 2: Numbers of blocks found per year 2015-2019


The following figure is similar to figure 2 but in a pie chart, showing the exact number of blocks found per year


Chart 3: Number of blocks found per year between 2015-2019


Now comes the interesting part, below is a chart of total empty blocks per mining pool.


Chart 4: Number of blocks per mining pool 2015-2019

A different representation of the same chart


Chart 5: Number of blocks per mining pool 2015-2019


Since we have a total of 26 mining pools, and most of them have a very small number of empty blocks, I have excluded 16 of them and kept only the top 10 mining pools, showing the number of empty blocks they found and the percentage (share)


Chart 6: Number of blocks and total share (%) per mining pool 2015-2019 for top 10 mining pools


To get a better idea in regards to mining pools: time ratio, the chart below shows the number of blocks per year for each mining pool.

Chart 7: Number of blocks a per mining pool per the year 2015-2019


Excluding 16 mining pools and keeping only the top 10 we get a clearer chart.


Chart 8: Number of blocks a per mining pool per the year 2015-2019 top 10 mining pools.





My interpretation:

1- The unusual numbers of empty block found during certain periods based on chart 1 does show some evidence that "someone" did use Covert Asicboost, and based on the fact that Bitmain pools (Antpool+BTC.com) found nearly 55% of those empty blocks, I don't believe that bitmain only tried Covert Asicboost on Testnet.

2- Aside from Covert Asicboost, there is a good possibility that some mining pools were purposely refusing to include transactions in the blocks they mined, causing a delay in transactions confirmation probably to give the impression of bitcoin "block is too small and we must increase it", this perfectly coincides with the spam attacks on the blockchain which happened in 2017, to be more accurate, May 2017 as shown in the chart below.



Comparing this chart against chart number 1, you can clearly see that May 2017 had a huge spike in empty blocks, it had exactly 75 empty blocks which is twice as high compared to the average number of empty blocks around that period, the majority of those empty blocks were obviously mined by Antpool and BTC.com (Bitmain), while the transactions went from 10-50k to just a bit over 150k during that month, someone was really trying to "prove" something, it's most likely Bitmain wanting larger blocks.


3- The number of empty blocks is decreasing, and since there is no more obvious drama coming from Bitmain and the big-blockers, taking 2019 numbers which had a total of 314 empty blocks or 0.86 blocks every day or a block every 1.16 days, the numbers in 2020 look even better, so it seems like the mining infrastructure is improving in terms of routing protocols, block propagation time, the confirmation of the transactions in the last block, speed of constructing blocks, etc.

4- Judging based on the point above, it seems like the time needed to validate block transactions for miners is well below 10 seconds.  



I would love to hear your interpretations/thoughts regarding the data shown in this topic.

Mikey.

 

I could do a 10 years analysis instead of just 5, the only issue is scraping the data from blockchair.com, it took me very long to get 5 years worth of data, however, if someone (maybe LoyceV or DdmrDdmr) can help me scrape the data I'll be willing to make a 10 years analysis.

1660756684
Hero Member
*
Offline Offline

Posts: 1660756684

View Profile Personal Message (Offline)

Ignore
1660756684
Reply with quote  #2

1660756684
Report to moderator
1660756684
Hero Member
*
Offline Offline

Posts: 1660756684

View Profile Personal Message (Offline)

Ignore
1660756684
Reply with quote  #2

1660756684
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1660756684
Hero Member
*
Offline Offline

Posts: 1660756684

View Profile Personal Message (Offline)

Ignore
1660756684
Reply with quote  #2

1660756684
Report to moderator
1660756684
Hero Member
*
Offline Offline

Posts: 1660756684

View Profile Personal Message (Offline)

Ignore
1660756684
Reply with quote  #2

1660756684
Report to moderator
philipma1957
Legendary
*
Offline Offline

Activity: 3486
Merit: 4586



View Profile
May 03, 2020, 02:12:47 AM
Last edit: May 03, 2020, 02:31:06 AM by philipma1957
 #2

  Just today there was a fee free block.

Let me pull it up.

628428 came in 1 second after 628427

they ripped off 3 in a row

628426  fees were 1.0282xxx

https://www.blockchain.com/btc/block/0000000000000000000254d12f670a0e5f3bdf579ce961f23295b0f7f1a6a783

628427  fees were 0.9683
https://www.blockchain.com/btc/block/00000000000000000007a4504c9d9aa57510a66ae0c995da60cfbf7cee451d84

628428 fees were zero 0.0

https://www.blockchain.com/btc/block/00000000000000000008c8fa0d79f31b44e6df494d20da165bf0f0b291e13d9a

here is bitmain's stats page

https://www.antpool.com/poolStats.htm

I see BTC as the super highway and alt coins as taxis and trucks needed to move transactions.
mikeywith
Legendary
*
Offline Offline

Activity: 1596
Merit: 4450


be constructive or S.T.F.U


View Profile
May 03, 2020, 02:30:33 AM
 #3

 Just today there was a fee free block.

Let me pull it up.

628428 came in 1 second after 628427

True block 628428 was empty and was mined by Antpool, there is a more recent empty block 628603, the block you mentioned was found yesterday unless of course, you are in the U.S or somewhere very west then it's "today".

Below is a list of empty blocks for the past few days

628603   2020-05-02 18:22   1THash&58COIN   
628428   2020-05-01 15:15   AntPool      
628336   2020-04-30 22:30   BTC.com   
628017   2020-04-28 15:34   BTC.com   
627867   2020-04-27 14:53   F2Pool   
https://blockchair.com/


it's pretty normal to find 1 empty block every day.

philipma1957
Legendary
*
Offline Offline

Activity: 3486
Merit: 4586



View Profile
May 03, 2020, 02:35:02 AM
Last edit: May 03, 2020, 03:06:20 AM by philipma1957
Merited by mikeywith (1)
 #4

bitmain had a 1 second block as per their stat page

 a normal block should be 600 seconds  odds for the network would be high  for any given block to be a 1 second block

you bring up an interesting point in saying we get 1 empty block a day.

that would be 1 in 144.

628603 came at 14:22:59
https://btc.com/000000000000000000015c0f2df9276237da03c568231b4f67d367ebdee91cd5


628602 came at 14:22:48

https://btc.com/00000000000000000003cdbcc1ed3d99bfb8c6db73a8db671a42a62624c30f7c


this is 11 seconds with zero fees

but ant pool block
628,428  came at 11:15:53

https://btc.com/00000000000000000008c8fa0d79f31b44e6df494d20da165bf0f0b291e13d9a

and block 628,427 came at 11:15:47

https://btc.com/00000000000000000007a4504c9d9aa57510a66ae0c995da60cfbf7cee451d84

this is 6 seconds or about 6/11 time to earn as the other example.

although on their stats page they claim it took 1 second.

I would love to know  the time gaps for every one  and how many per day.

____________________________________________________________________-
See below 32 second no earnings
_______________________________________________________________-
628336  18:30:32
https://btc.com/00000000000000000005fa95deb4a7815d38974c9aa55205794c21fae092c7da

628335 18:30:00
https://btc.com/0000000000000000000eef2b53bf510184ae045ee4ef9f8fffbed3a4bcb66bf2

that seems really long time to earn 0 tx fees

I checked the first 3 you listed

11  seconds 0 fees
 6   seconds 0 fees
32  seconds 0 fees



seems to me  if a pool is under 10 seconds it is not a big deal but 32 seconds seems really bad.

Obviously you need to check more then 3 to get a feel for fucked up issues.

A huge pool with 25% of the blocks could do a 10 second block vs 600 seconds that would not be unusual due to their hash amounts.  How many of those 10 second blocks are empty ?

a 25% pool makes 36 blocks a day or one every 40 minutes if they do a 10 second block every once in a while it will be empty.


I would love to see

1 second block
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
up to
60 second blocks    and tx fees they get.

I see BTC as the super highway and alt coins as taxis and trucks needed to move transactions.
ranochigo
Legendary
*
Offline Offline

Activity: 2730
Merit: 3340

@ me if you need my response


View Profile
May 03, 2020, 02:56:32 AM
 #5

Great analysis. Could you factor in the timestamp/the timing received by blockchair into the graph? It would be a lot clearer to see the timings between the empty blocks and the block before it.


philipma1957
Legendary
*
Offline Offline

Activity: 3486
Merit: 4586



View Profile
May 03, 2020, 03:09:56 AM
 #6

Great analysis. Could you factor in the timestamp/the timing received by blockchair into the graph? It would be a lot clearer to see the timings between the empty blocks and the block before it.



I am interested in time breakdowns  I would love to see how many zero fee blocks are due to the sheer size of the pool

ie lets say under 10 seconds
vs something seems wrong lets say 30 seconds.

JUST a fast check of three shows  a big difference in time 6 to 32 seconds.

people scream empty blocks are  pools being bad actors

is it simply the pools are really big?


@ mikey thanks you thread made me consider that size is a bigger factor then I realize.

BTW  I think covert asic was done bigly back in the day.

I also think if someone uses  time analysis along with empty block like you did you can get a better idea of what was happening.

I see BTC as the super highway and alt coins as taxis and trucks needed to move transactions.
nc50lc
Legendary
*
Offline Offline

Activity: 1778
Merit: 3567


Self-proclaimed Genius ㊙️


View Profile
May 03, 2020, 03:37:21 AM
 #7

people scream empty blocks are  pools being bad actors

is it simply the pools are really big?
The fact that the time to mine those blocks is within seconds and the number of transaction is '1' (Coinbase TX) tell that those are probably mined with 'Covert ASICBoost'.
Because not including any transaction is the easiest way to generate a Merkle Root Hash (hashMerkleRoot) with fixed last 4 bytes.

But there's no evidence so I'll leave it with "probably"  Smiley

mikeywith
Legendary
*
Offline Offline

Activity: 1596
Merit: 4450


be constructive or S.T.F.U


View Profile
May 03, 2020, 04:16:59 AM
 #8

Great analysis. Could you factor in the timestamp/the timing received by blockchair into the graph? It would be a lot clearer to see the timings between the empty blocks and the block before it.

I could but there are two problems, the first one is scraping the data of blockchain,took me forever to extract the data for empty blocks which is nothing compared to the total number of blocks, but if someone has a better way of getting those data, put them in a table format like excel and send them to me, I can do it and even more perhaps.

The second problem is that time-stamping isn't exactly accurate, they are accurate enough to be used to adjust the difficulty, but not accurate enough in a sense that we can use them to reach to any conclusions, maybe if we find empty blocks coming after a long enough time (enough to actually validate the transactions of the previous block) and still come in empty, we would confirm that a certain pool did actually have enough time but did not include transactions on purpose, however, since the time-stamp is actually put in the header of the block by whoever mines it, they can manipulate it, say I find a block 60 seconds after the last block was propagated (nobody knows about it yet), I need 30 seconds to validate the previous transactions and reconstruct the pool, but I decide not to include any transactions, I can conveniently lie and say I found the block in 10 seconds, who is going to stop me from doing so?


JUST a fast check of three shows a big difference in time 6 to 32 seconds.

I don't think anyone knows exactly how long it takes a mining pool to fully download and validate the previous block, but I am pretty sure it's less than 32 seconds, of course, more complex blocks take slightly longer, but 32 seconds can't be it, why do I think so?

Applying cumulative distribution function, the probability of blocks being found between 1 and 32 seconds can be derived by:

exp(−1/600)−exp(−32/600) = 5.36% of blocks would be empty, in other would we would "in general" have an empty block every 18.6 blocks, that's nearly 8 blocks a day which would result in 2920 blocks per year, which never happened even when was well-intended.

so really 32 seconds is more than enough to validate the previous block, deal with the memepool and reconstruct the next block WITH transactions in it, but where did the 32 seconds come from?

1- That node's memepool was empty. > very unlikely.
2- The time reported is actually "wrong" > Very likely.


I would still do the analysis, but really the results will be meaningless


people scream empty blocks are  pools being bad actors

is it simply the pools are really big?

The fact that the time to mine those blocks is within seconds and the number of transaction is '1' (Coinbase TX) tell that those are probably mined with 'Covert ASICBoost'.
Because not including any transaction is the easiest way to generate a Merkle Root Hash (hashMerkleRoot) with fixed last 4 bytes.

But there's no evidence so I'll leave it with "probably"  Smiley

I don't think anybody uses Covert Asicboost anymore, finding blocks in the next second at any given time always has a probability of 0.166%, if everyone was mining with a pen and paper, people would probably still mine empty blocks.

To explain even further, for the past 4 months of 2020 we had exactly 14 empty blocks every month, that's a bit less than 0.5 blocks a day, this is below normal as far as statistics and probability are concerned, applying the above formula and assuming for the sake of it that 5 seconds is more than enough to validate the transactions it is still OKAY to mine an empty block once every day, I think the number of empty blocks for the past 2 years is pretty normal.



nc50lc
Legendary
*
Offline Offline

Activity: 1778
Merit: 3567


Self-proclaimed Genius ㊙️


View Profile
May 03, 2020, 04:35:16 AM
 #9

-snip-
I don't think anybody uses Covert Asicboost anymore, finding blocks in the next second at any given time always has a probability of 0.166%, if everyone was mining with a pen and paper, people would probably still mine empty blocks.
-snip-
That probability explains the seconds difference in the timestamp but those "few seconds block" that are empty can't be a coincidence.
Because if it's just luckily mined, then it shouldn't be empty unless it's purposely made empty.

But since it's usually only seconds after the previous block, there's low to zero effect on the network when it comes with transaction validation.
So I agree with the second paragraph.

mikeywith
Legendary
*
Offline Offline

Activity: 1596
Merit: 4450


be constructive or S.T.F.U


View Profile
May 03, 2020, 04:51:49 AM
 #10

That probability explains the seconds difference in the timestamp but those "few seconds block" that are empty can't be a coincidence.
Because if it's just luckily mined, then it shouldn't be empty unless it's purposely made empty.

Nop, the probability explains exactly the chances of actually finding a block the very next second, it has nothing to do with the timestamp, in other words, if you were mining your own blockchain with 0 verifications needed, the chances of hitting a block in that very next second is always 0.166% (provided the mean time is 600 seconds), empty blocks are found in a very short time anywhere from => 1 second and < the time needed to validate with the previous block.   

We could also be talking about different things here, but the use of the word "purposely" creates confusion, miners want nothing more than extra profit, they have every interest in the world to NOT mine empty blocks WHEN they can fill them with transactions, but they have no problem with mining empty blocks when they happen to find one BEFORE being able to download the previous block, validate the transactions, clear the memepool, reconstruct the block and then propagate it, so this is not really "purposely" mining empty blocks, because "purposely" means they "CAN SAFELY" include transactions but they chose not to, which is not the case at all because nobody is mining empty blocks for the sake of it.

pooya87
Legendary
*
Offline Offline

Activity: 2814
Merit: 7229


#moon


View Profile
May 03, 2020, 05:11:01 AM
Merited by fillippone (2), ETFbitcoin (1), mikeywith (1)
 #11

the main reason for empty blocks has always been SPV mining, that is the fact that miners didn't verify the previous block completely before starting the next so they start with an empty block until they verify the previous one and be able to update their mempool and add new transactions to their block. sometimes they get lucky and find the answer to that empty block and publish that.
since during the past couple of years there has been a lot of improvements in the speed of transaction verification process with all the optimizations done by core team and SPV mining is not that common anymore we are seeing a much reduced number of empty blocks nowadays.

as for ASIC boost, it is an optimization that could only makes sense financially. using this optimization to the algorithm with which a miner computes the block hash will only give a very small "boost" that small boost could only make sense if mempool was empty or had very low paying fees.
ignoring the 2015 in your chart, the 2017 spike doesn't make any sense to be linked with ASIC boost since during that spike on your chart we were paying miners between 3 to 5 bitcoins in fees in total per block. in other words an empty block was losing somewhere around $8000 (25% of block reward).

p.s. why does all your charts say number of "blocks" instead of "empty blocks"? i'm confused...

mikeywith
Legendary
*
Offline Offline

Activity: 1596
Merit: 4450


be constructive or S.T.F.U


View Profile
May 03, 2020, 05:35:20 AM
 #12

as for ASIC boost, it is an optimization that could only makes sense financially. using this optimization to the algorithm with which a miner computes the block hash will only give a very small "boost" that small boost could only make sense if mempool was empty or had very low paying fees.

What do you mean by a small boost? using Asicboost yields up to 2000% more profit, even now with overt Asicboost mining gears run with 20% less power, giving the fact the profitability margin is too small a 20% savings on the power bill is a dozen time more profit, miners rather mine empty blocks using Asicboost than including your transactions even if you think you were paying high, overt Asicboost, however, does not have the tendency of generating empty blocks, so miners still use the boost and include your transactions, but the former is more important to them than the latter 99% of the time.

As far as May 2017, I did point that

Quote
Aside from Covert Asicboost, there is a good possibility that some mining pools were purposely refusing to include transactions in the blocks they mined, causing a delay in transactions confirmation probably to give the impression of bitcoin "block is too small and we must increase it", this perfectly coincides with the spam attacks on the blockchain which happened in 2017, to be more accurate, May 2017 as shown in the chart below.


Quote
p.s. why does all your charts say number of "blocks" instead of "empty blocks"?

Sorry, I forget to add the word "empty", figured that out later but then I thought that shouldn't be so confusing given the blocks number is too small and the obvious topic title and content which focuses on empty blocks, thanks for pointing that out anyway.

pooya87
Legendary
*
Offline Offline

Activity: 2814
Merit: 7229


#moon


View Profile
May 03, 2020, 06:18:45 AM
 #13

using Asicboost yields up to 2000% more profit,

that number makes no sense at all.
mining is calling the SHA256 compression function 3 times, this "boost" is changing these 3 calls by fixing the result of one for part of the computation and the result is far less than 2000% profit!!! it would at most increase it by 10%.

in any case let me ask you this, since you claim the reason for empty blocks is ASIC boost how do you explain lack of it nowadays? 2000% increased profit seems to be an excellent incentive to always use ASIC boost and mine empty blocks.

mikeywith
Legendary
*
Offline Offline

Activity: 1596
Merit: 4450


be constructive or S.T.F.U


View Profile
May 03, 2020, 07:16:22 AM
 #14

that number makes no sense at all.

Well, you can complain to "Antonopoulos" who mentioned the 2000%, or you can prove him wrong, or I can save you the trouble and explain to you why he is right.

Here is the calculation from whattomine.com for a single miner that does 10th and consumes 1000w ( pretty much similar efficiency to the most famous miner Antminer S9 taking the average rate of 6 cents per Kwh the miner earns exactly $0.02.

Now running the miner with Asicboost which can achieve up to 30% decrease in power consumption, the miner now uses only 700 wats and makes 0.46$ a day that's a 23 times more profit aka 2200% more profit, and by the way, I am not even trying to tweak any numbers to get to these results, i am using the average gear with the average power rate, results can be a lot better for many other miners.


Quote
in any case let me ask you this, since you claim the reason for empty blocks is ASIC boost how do you explain lack of it nowadays? 2000% increased profit seems to be an excellent incentive to always use ASIC boost and mine empty blocks.

Bitmain still uses Asicboost,  I personally use Asicboost, almost everyone and their grandmother uses ASICboost nowadays and get 20-30% saving on the power cost aka 2000% more profit, what you must understand is that there are TWO types of ASICboosts, the COVERT used previously by bitmain which bitmain then denied, that type of ASICboost has to do with empty blocks, the community opposed it because it's patented and it's not easy to detect and is not compatible with SegWit.

The OVERT Asicboost is a whole different story, it's the one we use now, has no effect on empty blocks, easily detectable, etc.


I hope that by now the confusion is gone.

tranthidung
Legendary
*
Offline Offline

Activity: 1638
Merit: 3174


Stop Putin - a liar, despot, dictator


View Profile WWW
May 03, 2020, 10:40:37 AM
 #15

I could do a 10 years analysis instead of just 5, the only issue is scraping the data from blockchair.com, it took me very long to get 5 years worth of data, however, if someone (maybe LoyceV or DdmrDdmr) can help me scrape the data I'll be willing to make a 10 years analysis.
I checked at https://blockchair.com/bitcoin/blocks?q=
It seems they only allow 10 blocks in a new batch of result. I don't find option to set up time window to get all time data. Or did I miss something? If there is such option, I can help you.

█▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀

█          ▄         ▄      ▄▄▄▄▄
█       ▄███      ▄███      █████
█        ████      ████     ▀▀▀▀▀
█         ████      ████
█          ████▄▄▄▄▄▄████▄▄▄▄▄▄▄▄
█           █████████████████████
█            ▀█████▄   ▀█████▄
█              ▀█████▀   ▀█████▀
█                 ▀▀        ▀▀

█▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
.....Your private Bitcoin wallet for desktop.....█▀▀▀▀▀▀











█▄▄▄▄▄▄
▀▀▀▀▀▀█











▄▄▄▄▄▄█
LoyceV
Legendary
*
Online Online

Activity: 2674
Merit: 11734


Thick-Skinned Gang Leader and Golden Feather


View Profile WWW
May 03, 2020, 11:24:07 AM
 #16

I could do a 10 years analysis instead of just 5, the only issue is scraping the data from blockchair.com, it took me very long to get 5 years worth of data, however, if someone (maybe LoyceV or DdmrDdmr) can help me scrape the data I'll be willing to make a 10 years analysis.
A year ago, I scraped all block headers from blockchain.com (for this topic). Around the same time someone posted a link to a site that allowed to quickly select this data, but I can't find it back.
I set a 1.8 second delay in between page loads. If you tell me exactly what you need, I can get it in about 2 weeks. Running on a VPS is not a problem, assuming Blockchain.com hasn't changed scraping restrictions.

tranthidung
Legendary
*
Offline Offline

Activity: 1638
Merit: 3174


Stop Putin - a liar, despot, dictator


View Profile WWW
May 03, 2020, 11:30:05 AM
Last edit: May 03, 2020, 11:43:43 AM by tranthidung
Merited by mikeywith (1)
 #17

I set a 1.8 second delay in between page loads. If you tell me exactly what you need, I can get it in about 2 weeks. Running on a VPS is not a problem, assuming Blockchain.com hasn't changed scraping restrictions.
https://blockchair.com/bitcoin/blocks?q=time(2020-05-02%2010:19:32..2020-05-03%2010:19:32)

I think these following indicators are enough (choose them from the General section):
  • Height
  • Mined on
  • Miner
  • Generation (BTC)
  • Reward (BTC)

Raw data will be like that, and I can import data with tab delimiter to find which blocks are empty and related details (miners & date time)  Smiley
Code:
Height Mined on Miner Generation (BTC) Reward (BTC)
628553 2020-05-02 10:15 OKEX 12.50000000 13.36347909
628552 2020-05-02 09:59 F2Pool 12.50000000 12.71661242

I don't know what does it mean with the part from the link above
Code:
(2020-05-02%2010:19:32..2020-05-03%2010:19:32)
Can we set up something to display all time data?

I tried but it only narrows down time window and only 10 lines of result are displayed.

█▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀

█          ▄         ▄      ▄▄▄▄▄
█       ▄███      ▄███      █████
█        ████      ████     ▀▀▀▀▀
█         ████      ████
█          ████▄▄▄▄▄▄████▄▄▄▄▄▄▄▄
█           █████████████████████
█            ▀█████▄   ▀█████▄
█              ▀█████▀   ▀█████▀
█                 ▀▀        ▀▀

█▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
.....Your private Bitcoin wallet for desktop.....█▀▀▀▀▀▀











█▄▄▄▄▄▄
▀▀▀▀▀▀█











▄▄▄▄▄▄█
mikeywith
Legendary
*
Offline Offline

Activity: 1596
Merit: 4450


be constructive or S.T.F.U


View Profile
May 03, 2020, 09:00:46 PM
 #18

If you tell me exactly what you need, I can get it in about 2 weeks. Running on a VPS is not a problem, assuming Blockchain.com hasn't changed scraping restrictions.

The problem now is that some members want to know the time difference between empty blocks and the block that came before them, to analyze this kind of data Blockchain.com is not enough as it doesn't show the full timestamp, it's missing the last two digits the represent the seconds, the only explorer I am aware of which provides the full timestamp is https://blockstream.info.

for this study, I will need the following columns.

Height
TimeStamp
Transactions (only the number)

If possible (depends on how you scrape the data, you could exclude all blocks that are not followed by a block which has 1 transaction (empty block), this will reduce the data by a tremendous amount, if you can't filter the results, it's okay, I can do it.


Now in regards to a similar study in the OP but with longer data, all I need is

Miner
Timestamp
Transactions ( number of transactions)

You still can't use blockchain.com because it doesn't show the timestamp, so really blockchain.com is no use, and you will need to use blockchair for that, here is a permalink of what I need https://blockchair.com/bitcoin/blocks?q=transaction_count(1)#f=time,guessed_miner,transaction_count


I think these following indicators are enough (choose them from the General section):

Height
Mined on
Miner
Generation (BTC)
Reward (BTC)


Three of those are not needed for the study, please refer to my explanation above

Quote
I tried but it only narrows down time window and only 10 lines of result are displayed.

Exactly, that's the problem, Imagine how time-consuming that was when I had to do that manually for 5 years of empty blocks   Grin, but I can only think that there is a way to automate this.


LoyceMobile
Sr. Member
****
Offline Offline

Activity: 1136
Merit: 410


LoyceV on the road. Or couch.


View Profile WWW
May 03, 2020, 09:09:08 PM
 #19

I'll see what I can do. I kinda want to just scrape everything, and make a huge csv. That'll come in handy for other analyses too.

     ▄█
   ▄██▌
 ▄████
▀▀▀█████▀
  ▐███▀
  ██▀
  ▀
.
▄▄▄███████▄▄▄
▄▄█████████████████▄▄
▄███████████████████████▄
███████████████████████████
██████████
███████████████████
██████████
█████████████████████
█████████████████████████████
█████████████████████████████
██
███████████████████████████
██
█████████████████████████
███████████████████████
▀▀█████████████████▀▀

▀▀▀███████▀▀▀
▄▄▄███████▄▄▄
▄▄█▀▀███████████▀▀█▄▄
▄████▄▄███████████▄▄████▄
█████
███▀▀▄▄▄▄▄▄▄▀▀████████
█████
██▀▄██████▀████▄▀███████
███████▀▄█████▀ ▐█████▄▀███████
██  ███ ████▀   ▀▀█████ ███  ██
██████▄▀█████  ▄█████▀▄██████
██████▄▀███▌▄██████▀▄██████
██
██████▄▄▀▀▀▀▀▀▀▄▄████████
▀█
███▀▀███████████▀▀████▀
▀▀█▄▄███████████▄▄█▀▀
▀▀▀███████▀▀▀
▄▀▀▀▀▀▀▀▀▀▀█████████
▀▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
█████████████████████

██████████▄▀▀▀▀▀▀▀▀▀

▄▄▄████████████████████▄▄▄
████████████████████████████
██████████████████████████████
███████████████████████████████
███████████████████████████████
███████████████████████████
▀██
█████████▀   ▀███████████▀
▀▀█████▀▀       ▀▀█████▀▀
.
SPORTS  │  CASINO  │  ESPORTS
.
philipma1957
Legendary
*
Offline Offline

Activity: 3486
Merit: 4586



View Profile
May 03, 2020, 11:19:04 PM
 #20

I'll see what I can do. I kinda want to just scrape everything, and make a huge csv. That'll come in handy for other analyses too.

It would be cool to see that a huge amount of empty blocks by antpool were 30-60 seconds

vs 1-10 seconds as it would expose a pattern of bad acting that they are accused of.


I see BTC as the super highway and alt coins as taxis and trucks needed to move transactions.
Pages: [1] 2 3 »  All
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!