A bitcoin blockchain parser in a few (thousand) lines of C++

[Prospero]

If you don't mind C (versus C++), picocoin's "blkstats" utility parses the blockchain in under 3 minutes.

https://github.com/jgarzik/picocoin/blob/master/src/blkstats.c
https://bitcointalk.org/index.php?topic=128055.0
https://github.com/jgarzik/picocoin/

Can this be used to compute the bitcoin rich list (list of all addresses with balance greater than x)?

[1715462696]

1715462696

[micax1]

I just wrote a bitcoin blockchain parser in a tiny code snippet.  This is implemented as just two source files, one header and one CPP, without any external dependencies.  This code snippet does not use any memory allocation, templates, containers, STL, Boost, or anything more complicated than 'fopen, fread, fclose'.

I wrote this mostly as a learning tool so I could understand the blockchain format myself.

While writing it I ran into a couple of minor issues.  One, is that sometimes a blockchain data file will just run out of data, the rest of the file containing zeros.  I don't know if this is normal or expected, but I treat it as an end-of-file condition.  

The other is that some blocks contain less data than is indicated by the block length; meaning that after all of the transactions are read in, the file pointer has not advanced as far as the 'block length' which was indicated.  I am going to assume this is normal and expected?

This code snippet parses my copy of the blockchain (9.2gb) in roughly 95 seconds; which I figured was pretty good; though I don't know what I have to compare to.

At any rate, if anyone finds this code snippet useful or just wants to better understand the data layout of the bitcoin blockchain you can find it here:

http://codesuppository.blogspot.com/2013/07/a-bitcoin-blockchain-parser-as-single.html

Feedback, bugfixes, suggestions, all welcome.

Thanks,
John

Hi! your tool is definitely usefull! however since blockchain become huge (~30 GB) and more then 52 mln addresses and 200+ inputs outputs - tool is no longer working ( moreover it`s not possible to recompile it changeing just:

#define MAX_BITCOIN_ADDRESSES 48000000 // 40 million unique addresses.
#define MAX_TOTAL_TRANSACTIONS 90000000 // 40 million transactions.
#define MAX_TOTAL_INPUTS 268000000 // 200 million inputs.
#define MAX_TOTAL_OUTPUTS 268000000 // 200 million outputs
#define MAX_TOTAL_BLOCKS 600000      // 1/2 million blocks.

because of it leads to Warning   16   warning C4307: '*' : integral constant overflow

these are the highest numbers tool is still compile-able however
it won`t let you poarse blockchain after.

It would be much appreciated if you can recompile it to be compatible with current blockchain size! or tips on how to do it )))
Thank you.

[jgarzik]

If you don't mind C (versus C++), picocoin's "blkstats" utility parses the blockchain in under 3 minutes.

https://github.com/jgarzik/picocoin/blob/master/src/blkstats.c
https://bitcointalk.org/index.php?topic=128055.0
https://github.com/jgarzik/picocoin/

Can this be used to compute the bitcoin rich list (list of all addresses with balance greater than x)?

Yes.

[micax1]

If you don't mind C (versus C++), picocoin's "blkstats" utility parses the blockchain in under 3 minutes.

https://github.com/jgarzik/picocoin/blob/master/src/blkstats.c
https://bitcointalk.org/index.php?topic=128055.0
https://github.com/jgarzik/picocoin/

Can this be used to compute the bitcoin rich list (list of all addresses with balance greater than x)?

Yes.

I could not find a docs on how to do it - can you plz help? for example i want to get every address from blockchain and sort it in order of higher current balance.
Thank you!

[micax1]

Hi! your tool is definitely usefull! however since blockchain become huge (~30 GB) and more then 52 mln addresses and 200+ inputs outputs - tool is no longer working ( moreover it`s not possible to recompile it changeing just:

#define MAX_BITCOIN_ADDRESSES 48000000 // 40 million unique addresses.
#define MAX_TOTAL_TRANSACTIONS 90000000 // 40 million transactions.
#define MAX_TOTAL_INPUTS 268000000 // 200 million inputs.
#define MAX_TOTAL_OUTPUTS 268000000 // 200 million outputs
#define MAX_TOTAL_BLOCKS 600000      // 1/2 million blocks.

because of it leads to Warning   16   warning C4307: '*' : integral constant overflow

these are the highest numbers tool is still compile-able however
it won`t let you poarse blockchain after.

It would be much appreciated if you can recompile it to be compatible with current blockchain size! or tips on how to do it )))
Thank you.

good news - duatiugame  refactored code and it`s working again:
https://code.google.com/p/blockchain/source/checkout

#define MAX_BITCOIN_ADDRESSES 60000000 // 60 million unique addresses.
#define MAX_TOTAL_TRANSACTIONS 70000000 // 70 million transactions.
#define MAX_TOTAL_INPUTS 250000000 // 250 million inputs.
#define MAX_TOTAL_OUTPUTS 250000000 // 250 million outputs
#define MAX_TOTAL_BLOCKS 600000      // 600,000 blocks.

[micax1]

https://code.google.com/p/blockchain/source/checkout

recently was refactored and now works grerat! thank you!

[DieJohnny]

can this parser be used to do an audit of bitcoin total coins?

How do we know that there are exactly the number of coins in the blockchain as have been mined since the beginning?

[amaclin]

can this parser be used to do an audit of bitcoin total coins?

How do we know that there are exactly the number of coins in the blockchain as have been mined since the beginning?

Very simple. Sum of all unspent yet outputs.
The second question: how many bitcoins were lost?
The answer: sum of all unspent outputs which are provable unspendable