Bitcoin Forum

I've recently been playing around with indexing the blockchain and producing some interesting charts.
I thought others may find them useful so I've uploaded them.

https://azeteki.github.io/charts/img/chart-blocksize.png (https://azeteki.github.io/charts)
https://azeteki.github.io/charts/img/chart-fees_per_kb.png (https://azeteki.github.io/charts)

More charts can be found at the following link: https://azeteki.github.io/charts (https://azeteki.github.io/charts)

The data was gleaned using a short python script to get historical block data from bitcoind.
This takes a little while but can be updated quickly with new blocks once historical information is indexed.
You can find it on my GitHub page if you are interested.

GitHub link: https://github.com/azeteki/bitcoind-chainstats (https://github.com/azeteki/bitcoind-chainstats)

Hey!! This is very interesting!

What's is a link to the GitHub??
Edited: I found a link in your profile!

I will take a look at it! It is really cool!


Regards.

Updated the original post with GitHub link.
Note that the gnuplot scripts are not included.
With full output, it will take a few hours and generate a delimited list of information about each block.
You can also get data for a range, e.g. block 300000-301000 if you so wish.

I believe you will need -txindex enabled to get coinbase fee data (it queries the coinbase transaction for each block).
Without that it should report 0 for all related items though I have not tested.

There are likely better programs available to perform the same task, however I like the simplicity of using bitcoind wherever possible.

Yeah.
That's very cool indeed.  A lot of data for data maniacs :)

Could you do the same in JavaScript? :)

Regards.

:)


Apologies, but I would rather nail myself to a rock, on Mars. :P

Ahh ok :)

I was asking because I am trying to do something similar in JS and it is not easy :D

One more time thanks for sharing this with us! This is really cool tool!

I hope you plan to do some more good stuff for Bitcoiners in the future!

Regards.

Really cool charts!

What's "Average transaction size"? Doesn't seem to be bytes per transaction and I can't think of anything else.

Good question!
I was charting inverse average transaction size. Not so useful!

Updated now.
Calculation: block_size/transactions_in_block.

Updated the charts page with individual charts for 2013 and 2014.

I've set the axes to show the most reasonable data, with the exception of coinbase reward which really isn't that interesting otherwise.

You can pick out quite a few instances in which large fees have been paid, presumably by mistake. Oops :(

I git cloned the repo today. Am I blind or you're not sharing the gnuplot scripts? I expected a wholly spoon-fed solution ;).

Consider it me giving you work to do. ;)

I will likely upload the scripts at some point, though I want to slim them down somewhat and ensure they don't break first.

Its worth taking a look.

It will be much better if more information or some analysis is presented along with the charts.

Are the high-up dots in the coinbase reward chart large fees, out of curiosity? Or is it something else I'm not understanding wholly?

if he would have used matplotlib (doesn't look like it) there is https://github.com/jakevdp/mpld3

Yes. The total coinbase reward (for the most part) will be equal to the block subsidy (50...25...12.5, etc), plus the sum of all fees in the blocks.

Most of the high up dots there represent mistakes or similar, like the ~200BTC fee that ASICMiner mined a while back.

Updated and added a chart for the total blockchain size.

In the space of a year we've gone from ~9GB to ~21GB. How time flies!

A chart with the size of the UTXO (# of transactions, size or both) would be interesting.  It is the critical resource and anecdotally it appears to grow slower than the overall blockchain but it would be interesting to see.

You mean like? http://bitcoin.sipa.be/pruning-size.png

Nice find.  I only wish sipa had continued it through the current block.  Also I assume this is more than just the UTXO (i.e. includes hashes of pruned txns to allow validation of the merkle tree).

All of my charts have been produced using bitcoind RPC only, no manipulation.

I can't really think of a way of producing a chart like sipa's without reindexing somehow.

If you could get bitcoin core to step through indexing block by block, you could do something like the following...

for i in blockcount:
    index_one_block
    gettxoutsetinfo

My machine is unfortunately too slow for both, really. A reindex would take weeks if not more for me, and gettxoutsetinfo is an extremely slow command to run each time.

Without specialist tools it will have to fall to someone else I'm afraid. :(

Updated and added a few more charts.

A few 'experimental' charts are up at the following link: https://azeteki.github.io/charts/test (https://azeteki.github.io/charts/test)

https://azeteki.github.io/charts/img/test/chart-blockinterval-2014.png (https://azeteki.github.io/charts)

These ones might be a little inaccurate as I haven't fully tested my moving average routines. They should be close enough for rendering though.

Nice work!

Updated for block 332749 2014-12-03 21:29:57.
Apologies for the delay. I have been away attending to various worldly matters.
Some of the charts may look a little odd as I have not had time to adjust the axes.

http://azeteki.github.io/charts/img/chart-coinbase.png
Not really understand this chart. The coinbase reward only can be 50 or 25 at this time. Why there are other value? ???

Transaction fees are part of the coinbase transaction.

As shorena says.

Strictly to my understanding the coinbase can (presently) be anything _up to_ 25 + sum of all transaction fees.

A miner can produce a block with a coinbase of 5BTC and it will be valid. It is just not really an economically sensible thing for them to do.

Alright. I understand now. But it's really hard to believe that someone make such high fees transactions. Would you please show me the hash of that kind of tx/blocks please? I want to have a look into them.

Probably messed up.

1d7749c65c90c32f5e2c036217a2574f3f4403da39174626b246eefa620b58d9

link for the lazy: https://www.blocktrail.com/BTC/tx/1d7749c65c90c32f5e2c036217a2574f3f4403da39174626b246eefa620b58d9

Its indeed rare and usually - at least from what I read - by accident. Some reported they got high fees back from the miners.

Thanks. I think it's caused by accident. I heard about that someone has got his fees back from asicminer's pool.

Yes. Most (all?) of these huge fees are caused by user error. People mucking around creating raw transactions, or coding bad wallets.
When creating a transaction by hand, the difference between inputs and outputs is given to miners as a fee.
A naive hacker can easily use a 50BTC input and assume that paying 3BTC to an address results in 47 remaining at the old address. Not so - the remainder of 47 goes off to mining land.

Updated charts for block 334725.

The most interesting changes recently to me are the leveling off of difficulty, and the continued increase in average block size. Blocks are roughly 375KB on average now, up from 225KB six months ago.

https://azeteki.github.io/charts/img/2014/chart-nethash.png
https://azeteki.github.io/charts/img/test/chart-blocksize-1008.png

Updated for 340709 (2015-01-27 15:54:27).

I have not yet bothered to create a 2015 page as there wouldn't really be much data yet. For now 2014 follows through to January.

As a side note, if anyone happens to have a host I could use that would be very much appreciated. Bandwidth usage should be utterly minimal.
The entire site including all graphs is about 500KB at present.