DavidBloom_NY
Newbie
Offline
Activity: 2
Merit: 0
|
|
May 14, 2018, 06:55:24 AM |
|
in short: - get a Linux Box/VM with >=8GB RAM - download blockparser - compile it - run "./parser allBalances -w 250000 >allBalances.txt" the parser run take around 2 minutes on my vSphere VM (2x3Ghz Xeon, 16GB of RAM) now you have a list of the top 250.000 adresses sorted by BTC amount. add some sed/awk/cut magic to make a csv and you are done. not as easy for the avg windows joe but not that hard either... quick'n'dirty Edit: ofc you need a up-to-date blockchain in <user>/.bitcoin for it to parse newbie here, this works, thanks a lot.
|
|
|
|
starmyc
Full Member
Offline
Activity: 198
Merit: 130
Some random software engineer
|
|
May 14, 2018, 07:05:36 AM |
|
in short: - get a Linux Box/VM with >=8GB RAM - download blockparser - compile it - run "./parser allBalances -w 250000 >allBalances.txt" the parser run take around 2 minutes on my vSphere VM (2x3Ghz Xeon, 16GB of RAM) now you have a list of the top 250.000 adresses sorted by BTC amount. add some sed/awk/cut magic to make a csv and you are done. not as easy for the avg windows joe but not that hard either... quick'n'dirty Edit: ofc you need a up-to-date blockchain in <user>/.bitcoin for it to parse newbie here, this works, thanks a lot. Yeah, well, I'm interested on which version of blockparser you used, because the 1st link given by google is https://github.com/znort987/blockparser, and as last change was 3 years ago, I can safely say it doesn't include segwit transactions, bech32 addresses...
|
|
|
|
mazertwo
Newbie
Offline
Activity: 54
Merit: 0
|
|
May 22, 2018, 08:43:42 AM |
|
Hello!
Can you parse Bitcoin Private (btcprivate.org)?
|
|
|
|
QRC
Newbie
Offline
Activity: 10
Merit: 2
|
|
May 24, 2018, 05:15:57 AM |
|
Hello!
Can you parse Bitcoin Private (btcprivate.org)?
You can parse almost any coin very easily, because most of them using level db for keeping their utxo, Just use python plyvel module to read db data. Actually, bitcoin private used the same technique to get bitcoin utxo.
|
|
|
|
starmyc
Full Member
Offline
Activity: 198
Merit: 130
Some random software engineer
|
|
May 24, 2018, 05:23:11 AM |
|
Hello!
Can you parse Bitcoin Private (btcprivate.org)?
You can parse almost any coin very easily, because most of them using level db for keeping their utxo, Just use python plyvel module to read db data. Actually, bitcoin private used the same technique to get bitcoin utxo. Yes, but this is not sufficient. You also have to uncompress those entries, and it requires some programming knowledge than using a library and print, it requires too to know how those coins works, their storage format (compact size, varint, etc)... Not hard, not trivial either. Hello!
Can you parse Bitcoin Private (btcprivate.org)?
I've added Zcash & Bitcoin Private balances generation to my webservice. I've still a little work to do on my software though.
|
|
|
|
QRC
Newbie
Offline
Activity: 10
Merit: 2
|
|
May 24, 2018, 05:44:36 AM |
|
Yes, but this is not sufficient. You also have to uncompress those entries, and it requires some programming knowledge than using a library and print, it requires too to know how those coins works, their storage format (compact size, varint, etc)... Not hard, not trivial either.
Yes, it's true. However, if coin uses bitcoin codebase from 0.15 or later, it's quiet easy, one db entry one utxo. Before that it's somewhat complicated because one db entry contained the whole transaction. But format of this entry was well documented in coin.h file: https://github.com/bitcoin/bitcoin/blob/v0.10.1/src/coins.h#L20Also, if I'm not mistaken BTCP script has function for decoding old format as well https://github.com/BTCPrivate/utxo-dump/blob/master/utxo/chainstate.pyparse_ldb_value_old line 57
|
|
|
|
starmyc
Full Member
Offline
Activity: 198
Merit: 130
Some random software engineer
|
|
May 24, 2018, 07:24:28 AM |
|
But you're spoiling all fun to learn. It is more interesting to code your own decoder to learn how it structured & how things work.
|
|
|
|
mazertwo
Newbie
Offline
Activity: 54
Merit: 0
|
|
May 28, 2018, 11:25:55 AM |
|
Can you parse MoneroV (XMV) blockchain? monerov.org And Reden (REDN) interesting too reden.io
|
|
|
|
MikaelBlomvist
Newbie
Offline
Activity: 24
Merit: 0
|
|
May 30, 2018, 07:39:38 PM |
|
need this but my vps has 40GB of space, cant fit those damn blocks there . Any other solution!?
|
|
|
|
|
starmyc
Full Member
Offline
Activity: 198
Merit: 130
Some random software engineer
|
|
June 01, 2018, 06:14:06 AM |
|
Sure I use it (as I wrote it). Parsing the whole blockchain is slow, I let this job to official clients.
|
|
|
|
Banxter
Jr. Member
Offline
Activity: 88
Merit: 2
|
|
June 01, 2018, 06:38:39 AM |
|
Sure I use it (as I wrote it). Parsing the whole blockchain is slow, I let this job to official clients. good job! thx How much does this parser use RAM? (the old parser https://github.com/znort987/blockparser was good, but I used at least 64GB, I had to use expensive VPS)
|
|
|
|
starmyc
Full Member
Offline
Activity: 198
Merit: 130
Some random software engineer
|
|
June 01, 2018, 07:29:31 AM |
|
Sure I use it (as I wrote it). Parsing the whole blockchain is slow, I let this job to official clients. good job! thx How much does this parser use RAM? (the old parser https://github.com/znort987/blockparser was good, but I used at least 64GB, I had to use expensive VPS) The chainstate is not the blockchain. Those are 2 differents tools for 2 differents usages.
|
|
|
|
coder0x15
Jr. Member
Offline
Activity: 36
Merit: 3
|
|
June 01, 2018, 08:19:42 AM |
|
If you want to list all transactions from a blockchain, you need a block parser (and I've mine too, but it is closed source).
@starmyc please can you share the link to download that closed-source blockparser?
|
|
|
|
starmyc
Full Member
Offline
Activity: 198
Merit: 130
Some random software engineer
|
|
June 01, 2018, 08:47:58 AM |
|
If you want to list all transactions from a blockchain, you need a block parser (and I've mine too, but it is closed source).
@starmyc please can you share the link to download that closed-source blockparser? I will, someday. Still need to clean up the really crappy code.
|
|
|
|
coder0x15
Jr. Member
Offline
Activity: 36
Merit: 3
|
|
June 02, 2018, 03:18:52 AM Last edit: June 02, 2018, 03:30:46 AM by coder0x15 |
|
@starmyc is any idea for to make some cool stuff? Not only –Chainstate LDB API caller–. I made ldb files parser without of any APIs and third-parties frameworks. Only hardcore, only raw decoding. P.S. Obfuscated keys are strong for unequipped user
|
|
|
|
Adsixnine
Newbie
Offline
Activity: 3
Merit: 0
|
|
June 02, 2018, 10:54:31 AM |
|
I'm curious as to what % of possible addresses have been used?
|
|
|
|
MikaelBlomvist
Newbie
Offline
Activity: 24
Merit: 0
|
|
June 02, 2018, 04:06:31 PM |
|
I'm curious as to what % of possible addresses have been used? considering the amount of bitcoin addresses that can be generated, I'd say we have used around 0.000000000001%
|
|
|
|
starmyc
Full Member
Offline
Activity: 198
Merit: 130
Some random software engineer
|
|
June 03, 2018, 08:30:42 PM |
|
I'm curious as to what % of possible addresses have been used? considering the amount of bitcoin addresses that can be generated, I'd say we have used around 0.000000000001% FYI, today I've done a full dump of all bitcoin addresses ever used at least once. I've reached the number of 401,407,598 unique addresses for all block files from blk00000 to blk01275. The % is still really small. Smaller than 10^-40 %, at least.
|
|
|
|
starmyc
Full Member
Offline
Activity: 198
Merit: 130
Some random software engineer
|
|
June 03, 2018, 08:35:25 PM |
|
@starmyc is any idea for to make some cool stuff? Not only –Chainstate LDB API caller–. I made ldb files parser without of any APIs and third-parties frameworks. Only hardcore, only raw decoding. P.S. Obfuscated keys are strong for unequipped user I do not parse ldb files by myself. I use google's library for this, as I want to focus more on Bitcoin's aspect rather than a third party nosql library (I would rather study cassandra or any distributed engine). What's the issue with obfuscated datas ? The key is in the DB as well!
|
|
|
|
|