Get list of all addresses with a balance over x?

[DavidBloom_NY]

in short:

- get a Linux Box/VM with >=8GB RAM
- download blockparser
- compile it
- run "./parser allBalances  -w 250000 >allBalances.txt"

the parser run take around 2 minutes on my vSphere VM (2x3Ghz Xeon, 16GB of RAM)

now you have a list of the top 250.000 adresses sorted by BTC amount.

add some sed/awk/cut magic to make a csv and you are done.


not as easy for the avg windows joe but not that hard either...


quick'n'dirty

Edit: ofc you need a up-to-date blockchain in <user>/.bitcoin for it to parse

newbie here, this works, thanks a lot.

[starmyc]

in short:

- get a Linux Box/VM with >=8GB RAM
- download blockparser
- compile it
- run "./parser allBalances  -w 250000 >allBalances.txt"

the parser run take around 2 minutes on my vSphere VM (2x3Ghz Xeon, 16GB of RAM)

now you have a list of the top 250.000 adresses sorted by BTC amount.

add some sed/awk/cut magic to make a csv and you are done.


not as easy for the avg windows joe but not that hard either...


quick'n'dirty

Edit: ofc you need a up-to-date blockchain in <user>/.bitcoin for it to parse

newbie here, this works, thanks a lot.

Yeah, well, I'm interested on which version of blockparser you used, because the 1st link given by google is https://github.com/znort987/blockparser, and as last change was 3 years ago, I can safely say it doesn't include segwit transactions, bech32 addresses...

[mazertwo]

Hello!

Can you parse Bitcoin Private (btcprivate.org)?

[QRC]

Hello!

Can you parse Bitcoin Private (btcprivate.org)?

You can parse almost any coin very easily, because most of them using level db for keeping their utxo,
Just use python plyvel module to read db data.

Actually, bitcoin private used the same technique to get bitcoin utxo.

[starmyc]

Hello!

Can you parse Bitcoin Private (btcprivate.org)?

You can parse almost any coin very easily, because most of them using level db for keeping their utxo,
Just use python plyvel module to read db data.

Actually, bitcoin private used the same technique to get bitcoin utxo.

Yes, but this is not sufficient. You also have to uncompress those entries, and it requires some programming knowledge than using a library and print, it requires too to know how those coins works, their storage format (compact size, varint, etc)... Not hard, not trivial either.

Hello!

Can you parse Bitcoin Private (btcprivate.org)?

I've added Zcash & Bitcoin Private balances generation to my webservice. I've still a little work to do on my software though.

[QRC]

Yes, but this is not sufficient. You also have to uncompress those entries, and it requires some programming knowledge than using a library and print, it requires too to know how those coins works, their storage format (compact size, varint, etc)... Not hard, not trivial either.

Yes, it's true.
However, if coin uses bitcoin codebase from 0.15 or later, it's quiet easy, one db entry one utxo.
Before that it's somewhat complicated because one db entry contained the whole transaction.
But format of this entry was well documented in coin.h file:
https://github.com/bitcoin/bitcoin/blob/v0.10.1/src/coins.h#L20

Also, if I'm not mistaken BTCP script has function for decoding old format as well
https://github.com/BTCPrivate/utxo-dump/blob/master/utxo/chainstate.py
parse_ldb_value_old line 57

[starmyc]

Also, if I'm not mistaken BTCP script has function for decoding old format as well
https://github.com/BTCPrivate/utxo-dump/blob/master/utxo/chainstate.py
parse_ldb_value_old line 57

But you're spoiling all fun to learn. It is more interesting to code your own decoder to learn how it structured & how things work.

[mazertwo]

Can you parse MoneroV (XMV) blockchain? monerov.org
And Reden (REDN) interesting too reden.io

[MikaelBlomvist]

need this but my vps has 40GB of space, cant fit those damn blocks there  . Any other solution!?

[coder0x15]

@starmyc, can you tell, you use https://github.com/mycroft/chainstate to parse chainstate db? Or you work with full blockchain database?

[starmyc]

@starmyc, can you tell, you use https://github.com/mycroft/chainstate to parse chainstate db? Or you work with full blockchain database?

Sure I use it (as I wrote it). Parsing the whole blockchain is slow, I let this job to official clients.

[Banxter]

@starmyc, can you tell, you use https://github.com/mycroft/chainstate to parse chainstate db? Or you work with full blockchain database?

Sure I use it (as I wrote it). Parsing the whole blockchain is slow, I let this job to official clients.

good job! thx
How much does this parser use RAM?
(the old parser https://github.com/znort987/blockparser was good, but I used at least 64GB, I had to use expensive VPS)

[starmyc]

@starmyc, can you tell, you use https://github.com/mycroft/chainstate to parse chainstate db? Or you work with full blockchain database?

Sure I use it (as I wrote it). Parsing the whole blockchain is slow, I let this job to official clients.

good job! thx
How much does this parser use RAM?
(the old parser https://github.com/znort987/blockparser was good, but I used at least 64GB, I had to use expensive VPS)

The chainstate is not the blockchain. Those are 2 differents tools for 2 differents usages.

[coder0x15]

If you want to list all transactions from a blockchain, you need a block parser (and I've mine too, but it is closed source).

@starmyc please can you share the link to download that closed-source blockparser?

[starmyc]

If you want to list all transactions from a blockchain, you need a block parser (and I've mine too, but it is closed source).

@starmyc please can you share the link to download that closed-source blockparser?

I will, someday. Still need to clean up the really crappy code.

[coder0x15]

@starmyc is any idea for to make some cool stuff? Not only –Chainstate LDB API caller–. I made ldb files parser without of any APIs and third-parties frameworks. Only hardcore, only raw decoding.
P.S. Obfuscated keys are strong for unequipped user

[Adsixnine]

I'm curious as to what % of possible addresses have been used? 

[MikaelBlomvist]

I'm curious as to what % of possible addresses have been used? 

considering the amount of bitcoin addresses that can be generated, I'd say we have used around 0.000000000001%

[starmyc]

I'm curious as to what % of possible addresses have been used? 

considering the amount of bitcoin addresses that can be generated, I'd say we have used around 0.000000000001%

FYI, today I've done a full dump of all bitcoin addresses ever used at least once. I've reached the number of 401,407,598 unique addresses for all block files from blk00000 to blk01275. The % is still really small. Smaller than 10^-40 %, at least.

[starmyc]

@starmyc is any idea for to make some cool stuff? Not only –Chainstate LDB API caller–. I made ldb files parser without of any APIs and third-parties frameworks. Only hardcore, only raw decoding.
P.S. Obfuscated keys are strong for unequipped user

I do not parse ldb files by myself. I use google's library for this, as I want to focus more on Bitcoin's aspect rather than a third party nosql library (I would rather study cassandra or any distributed engine).

What's the issue with obfuscated datas ? The key is in the DB as well!