Get all Bitcoin Addresses with balance >= 0.00000001

[Rintila]

Hi everyone,

How can I get a list of all Bitcoin addresses with the current balance >= 0.00000001?

I already searched for many ways, including coding, but I get nowhere.

Any ideas?

Thanks

[gmaxwell]

The system itself doesn't have balances in any direct way, so thats one reason it's not as simple as you might guess.

What precisely are you trying to accomplish?

[Rintila]

The system itself doesn't have balances in any direct way, so thats one reason it's not as simple as you might guess.

What precisely are you trying to accomplish?

"What precisely are you trying to accomplish?"

I work for the NSA and I need to be able to get this list ASAP, we are trying to crack Bitcoin.

Just kidding I just got into Bitcoin technically recently and I'm trying to figure out how everything works...

Basically what I'm searching for is for a way to query the blockchain locally.

But I find it very difficult to find a way, I use the blockchain.info API, but when you want to make millions of requests the API is obsolete since there are requests limits and obviously its slow.

I would like to do complex queries, but if I am just able to get all the current addresses with a balance > 0 I will be happy enough.

Maybe this info is available somewhere?

Any ideas?

[OnkelPaul]

If searching and coding didn't get you there, your searching and coding skills need some improvement  

Look here for some existing discussion on a very similar requirement:
https://bitcointalk.org/index.php?topic=762445.0
(found it using G with the search phrase "bitcoin find all addresses with positive balance", it isn't really that hard...)

Onkel Paul

[Rintila]

If searching and coding didn't get you there, your searching and coding skills need some improvement  

Look here for some existing discussion on a very similar requirement:
https://bitcointalk.org/index.php?topic=762445.0
(found it using G with the search phrase "bitcoin find all addresses with positive balance", it isn't really that hard...)

Onkel Paul

Yea, I stumbled upon it before, but problem is that I'm a C# guy, every time I need to compile other programming languages its always an odyssey...

And I'm not even a Linux user.. so I think I will have to start getting into Linux.

Also, when you have to build 50 libraries, plus troubleshoot 200 more compilation errors, its always a pain in the ass to have the motivation to go through it all.

But come on guys, Bitcoin is already 6 years old and to get this basic info is so complex?

There are lots of lazy people out there...

Making this kind of info accessible is part of the maturity of Bitcoin, if nothing like this is available yet, I will take care of building a website where this info will be accessible to everybody out there.

[Rintila]

btw, just for info if somebody is also looking for this kind of info, I found this post here where all address until 3rd January 2014 are available:

https://bitcointalk.org/index.php?topic=267618.msg4289057#msg4289057

Some other useful instructions are also included in that thread.

Any ideas on how the find the actual list?

[DannyHamilton]

btw, just for info if somebody is also looking for this kind of info, I found this post here where all address until 3rd January 2014 are available:

https://bitcointalk.org/index.php?topic=267618.msg4289057#msg4289057

Some other useful instructions are also included in that thread.

Any ideas on how the find the actual list?

The list changes continuously, so as soon as you started running any such query, it would already be incorrect.

It can help to limit your query to transactions that are included in the blockchain.  At least then the list will only change an average of every 10 minutes.

That being said, the blockchain does not store balances.  It ONLY stores transactions.  Every transaction since the beginning of Bitcoin.  To get the specific information you are looking for, you can try looking for tools that will provide you the current UTXO set (unspent transaction output set). Such a tool would start at the beginning of the blockchain and read through every transaction, removing outputs from the tool's list as they are spent and adding outputs as they are created.  When it gets to the end of the blockchain, you should have a complete list of all UTXO.  Keep in mind that bitcoins aren't always sent to addresses, so some of the outputs in the UTXO won't be to addresses.  You might be better off looking for a list of unique output scripts in the set of all UTXO.

Here's someone that wrote their own UTXO parser.  Perhaps if you look around you can find a similar free open source tool that you can use:

In validating a UTXO parser I started looking at various outputs which are . . .
- snip -

[Rintila]

btw, just for info if somebody is also looking for this kind of info, I found this post here where all address until 3rd January 2014 are available:

https://bitcointalk.org/index.php?topic=267618.msg4289057#msg4289057

Some other useful instructions are also included in that thread.

Any ideas on how the find the actual list?

The list changes continuously, so as soon as you started running any such query, it would already be incorrect.

It can help to limit your query to transactions that are included in the blockchain.  At least then the list will only change an average of every 10 minutes.

That being said, the blockchain does not store balances.  It ONLY stores transactions.  Every transaction since the beginning of Bitcoin.  To get the specific information you are looking for, you can try looking for tools that will provide you the current UTXO set (unspent transaction output set). Such a tool would start at the beginning of the blockchain and read through every transaction, removing outputs from the tool's list as they are spent and adding outputs as they are created.  When it gets to the end of the blockchain, you should have a complete list of all UTXO.  Keep in mind that bitcoins aren't always sent to addresses, so some of the outputs in the UTXO won't be to addresses.  You might be better off looking for a list of unique output scripts in the set of all UTXO.

Here's someone that wrote there own UTXO parser.  Perhaps if you look around you can find a similar free open source tool that you can use:

In validating a UTXO parser I started looking at various outputs which are . . .
- snip -

Yes, I know that the list changes all the time, but an up to date list is enough to have a actual estimation.

I'm investigating some parsers, but I'm sure to do all of this its not just clicking the Build button, there will be issues on the way to troubleshoot and so on.

I will do that later, but for now I would just need an actual list. That would make my day

[NeuroticFish]

I didn't read it yet, but I think that you should start with the (free) book Blockchain Programming in C#
I guess that you'll be able (after some steps) to reach your goal.

https://bitcointalk.org/index.php?topic=926087.0

[nuno12345]

First approach.
1-Setup a blockchain explorer like abe.
2-Search trough the sql to get all available addresses, if needed convert the hash160 to address, add them to sql.
3-Run a query to delete duplicates.
4-Query every address using abe's api addressbalance.
5-If balance > desired_amount drop the address from the new sql table.
6-Done.

Note: This may take some time, better to add a sleep there so it wont crash abe.

Second approach.
1-Setup bitcoind.
2-Set txindex=1 and rescan the blockchain.
3-Get latest block with getblockcount.
4-Use getblockhash block_number to get it's hash.
5-Use getblock hash to get that block transactions.
6-Parse every transaction with getrawtransaction hash 1 (1 for verbose).
7-Get the vout address of each transaction and add to sql.
8-Run a query to delete duplicates.
9-Use blockchain.info api to get each address balance.
10-If balance > desired_amount drop the address from the new sql table.
11-Done.

Note: This WILL take some time, better add a sleep there and go to sleep for like 1 week.

Hope this helps

[Rintila]

I didn't read it yet, but I think that you should start with the (free) book Blockchain Programming in C#
I guess that you'll be able (after some steps) to reach your goal.

https://bitcointalk.org/index.php?topic=926087.0

What a nice book to read at night, thanks

[Rintila]

First approach.
1-Setup a blockchain explorer like abe.
2-Search trough the sql to get all available addresses, if needed convert the hash160 to address, add them to sql.
3-Run a query to delete duplicates.
4-Query every address using abe's api addressbalance.
5-If balance > desired_amount drop the address from the new sql table.
6-Done.

Note: This may take some time, better to add a sleep there so it wont crash abe.

Second approach.
1-Setup bitcoind.
2-Set txindex=1 and rescan the blockchain.
3-Get latest block with getblockcount.
4-Use getblockhash block_number to get it's hash.
5-Use getblock hash to get that block transactions.
6-Parse every transaction with getrawtransaction hash 1 (1 for verbose).
7-Get the vout address of each transaction and add to sql.
8-Run a query to delete duplicates.
9-Use blockchain.info api to get each address balance.
10-If balance > desired_amount drop the address from the new sql table.
11-Done.

Note: This WILL take some time, better add a sleep there and go to sleep for like 1 week.

Hope this helps

You almost made me cum with your second option, that would be perfect, I would just set up a Bitcoin server with JSON-RPC... but when I got to point 9 I lost my fire

"9-Use blockchain.info api to get each address balance."

That would require millions of requests to the blockchain.info API... they have daily limits and stuff.

Thanks for the suggestions, I will investigate option 1.

Btw isn't there a way to calculate the Balance somehow with your second option? (without doing blockchain.info API requests)

[DannyHamilton]

- snip -
Second approach.
1-Setup bitcoind.
2-Set txindex=1 and rescan the blockchain.
3-Get latest block with getblockcount.
4-Use getblockhash block_number to get it's hash.
5-Use getblock hash to get that block transactions.
6-Parse every transaction with getrawtransaction hash 1 (1 for verbose).
7-Get the vout address of each transaction and add to sql.
8-Run a query to delete duplicates.
9-Use blockchain.info api to get each address balance.
10-If balance > desired_amount drop the address from the new sql table.
11-Done.
- snip -

- snip -
Btw isn't there a way to calculate the Balance somehow with your second option? (without doing blockchain.info API requests)

Yes.

Step 6 and 7 are incorrect.

getrawtransaction will return a hex representation of the raw transaction.  Here's an example of transaction ID 62886d0b5e7e88981839afb6996010d149f934f30a04f428456cd5e10a772846

Code:

010000000102651693fc8811883bd5f4a90d9825f1bc18a55ad9026264bcfc1e63a6b7f089000000006a4730440220673be38060c769b260ba9b0d0bee89fc0f9db1a3dbf09060c43adc499c082d7a02202aabd0eea4ee119c04234dee43c3543426fbd23547252e52fd4496d06b2e1fd4012102bc2af3ca047923b71f4e1d2aae0564ea97d0ec86f1aa7f4c2d4fb2acc3f9fe75ffffffff02411fc605000000001976a914c08407401f334b9e936391fee7753ea9bb45613688aca4688e00000000001976a914892f2d2752a0dc59f59be9ddffe56bccdd0440bc88ac00000000

Then if you run decoderawtransaction on that raw data, you'll get a json object with the data parsed out.  Here's an example of that same transaction:

Code:

{
    "txid" : "cc4aa7b2a873515edf55167126fb18919a67abb5f3cac35cd48c1e2b834c3edc",
    "version" : 1,
    "locktime" : 0,
    "vin" : [
        {
            "txid" : "89f0b7a6631efcbc646202d95aa518bcf125980da9f4d53b881188fc93166502",
            "vout" : 0,
            "scriptSig" : {
                "asm" : "30440220673be38060c769b260ba9b0d0bee89fc0f9db1a3dbf09060c43adc499c082d7a02202aabd0eea4ee119c04234dee43c3543426fbd23547252e52fd4496d06b2e1fd401 02bc2af3ca047923b71f4e1d2aae0564ea97d0ec86f1aa7f4c2d4fb2acc3f9fe75",
                "hex" : "4730440220673be38060c769b260ba9b0d0bee89fc0f9db1a3dbf09060c43adc499c082d7a02202aabd0eea4ee119c04234dee43c3543426fbd23547252e52fd4496d06b2e1fd4012102bc2af3ca047923b71f4e1d2aae0564ea97d0ec86f1aa7f4c2d4fb2acc3f9fe75"
            },
            "sequence" : 4294967295
        }
    ],
    "vout" : [
        {
            "value" : 0.96870209,
            "n" : 0,
            "scriptPubKey" : {
                "asm" : "OP_DUP OP_HASH160 c08407401f334b9e936391fee7753ea9bb456136 OP_EQUALVERIFY OP_CHECKSIG",
                "hex" : "76a914c08407401f334b9e936391fee7753ea9bb45613688ac",
                "reqSigs" : 1,
                "type" : "pubkeyhash",
                "addresses" : [
                    "1JYvuKZmqyyn8MTaTSwCQ6CrcJfcLGAruM"
                ]
            }
        },
        {
            "value" : 0.09332900,
            "n" : 1,
            "scriptPubKey" : {
                "asm" : "OP_DUP OP_HASH160 892f2d2752a0dc59f59be9ddffe56bccdd0440bc OP_EQUALVERIFY OP_CHECKSIG",
                "hex" : "76a914892f2d2752a0dc59f59be9ddffe56bccdd0440bc88ac",
                "reqSigs" : 1,
                "type" : "pubkeyhash",
                "addresses" : [
                    "1DWN69tNjUQJv3QeJMHVLSJfHXXr2QusST"
                ]
            }
        }
    ]
}

You can see here that the transaction defines an array inputs that are being spent "vin" and an array of new unspent of outputs that are being created "vout".

This particular transaction creates 2 outputs identified by "n".

Output 0 contains 0.96870209 BTC (identified by "value") and is associated with address 1JYvuKZmqyyn8MTaTSwCQ6CrcJfcLGAruM.
Output 1 contains 0.09332900 BTC and is associated with address 1DWN69tNjUQJv3QeJMHVLSJfHXXr2QusST.

This particular transaction also spends 1 previously unspent output from transaction 89f0b7a6631efcbc646202d95aa518bcf125980da9f4d53b881188fc93166502.  The "vout" in the "vin" indicates that it was the output from the transaction that had "n":0

Therefore, your program can remove output 0 of transaction 89f0b7a6631efcbc646202d95aa518bcf125980da9f4d53b881188fc93166502 from it's list of unspent outputs. And can add two new outputs (0 and 1) of transaction cc4aa7b2a873515edf55167126fb18919a67abb5f3cac35cd48c1e2b834c3edc to it's list of unspent outputs.

Once your program is done running, it can just dump the list of addresses from it's list of unspent outputs and remove any duplicate addresses.

[Rintila]

- porn -

Sounds cool, I will have a look at it later. Thanks

[Rintila]

- porn -

Sounds cool, I will have a look at it later. Thanks

Ok, checked this, looks like it works

@DannyHamilton I checked point 6 from nuno12345, it also works if we set the second parameter as 1, it spits out directly the json code.

Great, I think this is the best way to do what I want for now.

All I need now is to set a Bitcoin server and send the commands via http and I will be able to start doing tons of queries.

I didn't test yet the Balance calculations, but for now I will just fetch all addresses that are in the "addresses" part, like this I'm able to fetch all addresses that were ever in the blockchain, which means that they either have BTCs or had BTCs in the past, so like this I will be able to fetch all Bitcoin used addresses EVER on the mainnet, right?

I will start coding it soon... wondering how the speed of all of this will be.

Thanks guys

[rfcdejong]

If you are a C# guy then look at this post about a book from Nicolas Dorier

Blockchain Programming in C# (Part 1)
https://bitcointalk.org/index.php?topic=926087.0

It is a NuGet package NBitcoin, install using Visual Studio NuGet Package Manager.

Or look at his source for the fun of it
https://github.com/NicolasDorier

All access to the blockchain is possible, so also getting some sort of rich list descending to get 0.000001 and higher

[gmaxwell]

The system itself doesn't have balances in any direct way, so thats one reason it's not as simple as you might guess.

What precisely are you trying to accomplish?

"What precisely are you trying to accomplish?"

I work for the NSA and I need to be able to get this list ASAP, we are trying to crack Bitcoin.

Just kidding I just got into Bitcoin technically recently and I'm trying to figure out how everything works...

Basically what I'm searching for is for a way to query the blockchain locally.

But I find it very difficult to find a way, I use the blockchain.info API, but when you want to make millions of requests the API is obsolete since there are requests limits and obviously its slow.

I would like to do complex queries, but if I am just able to get all the current addresses with a balance > 0 I will be happy enough.

Maybe this info is available somewhere?

Any ideas?

People often ask questions like that out of ignorance about how the system works, so it's useful to ask why they're asking; as mentioned the data you seek is not relevant to the operation of the system.  You still haven't explained what you're trying to accomplish. Knowing it can help people provide more useful answers.

[DeathAndTaxes]

Listen to Danny he knows his stuff.  I would also add and this is a little cliche but to avoid wasting a lot of time you really need to understand how bitcoin works.  So try to find and read any articles that talk about Bitcoin under the hood as well as the developer resources on the bitcoin homepage.

A couple hints:
1) The blockchain doesn't record balances.
2) The blockchain doesn't record addresses only inputs and outputs.
3) Inputs always reference a previously unspent output.
4) Outputs are not addresses they are scripts.  The most common script is Pay2PubkeyHash but it isn't the only type of script.
5) Transactions "spend" discrete outputs not a specific value.
6) Outputs are either spent or unspent.  Unspent outputs can be "spent" in a new transaction.

If you don't have a good understanding of how Bitcoin actually works then you will probably end up just running around in circles. 

The historical blockchain builds the UTXO set.  To build the UTXO you start from block 1 (the output in the genesis block is unspenable) and for each block you add to the UTXO all the new txn outputs and you remove from the UTXO all those outputs referenced in txn inputs.  When you get to the current block you will have the current UTXO.  Note this process assumes the blockchain is already validated which is a safe assumption if you are pulling blocks from bitcoind.

The UTXO set has all the information you need.  When someone say address x has a balance of 1.234 BTC what they really mean is that the UTXO contains one or more unspent outputs which have a combined value of 1.234 BTC.

[Rintila]

- snip -

Nothing in concrete yet, but my main objective is to prove how secure a Bitcoin Address is and eventually to lunch a website with lots of statistics about Bitcoin addresses like how many are there, which balances they have from top to down, etc...

For now I'm just playing around and if I get all what I'm thinking right, then I will think about the website.

[Rintila]

Listen to Danny he knows his stuff.  I would also add and this is a little cliche but to avoid wasting a lot of time you really need to understand how bitcoin works.  So try to find and read any articles that talk about Bitcoin under the hood as well as the developer resources on the bitcoin homepage.

A couple hints:
1) The blockchain doesn't record balances.
2) The blockchain doesn't record addresses only inputs and outputs.
3) Inputs always reference a previously unspent output.
4) Outputs are not addresses they are scripts.  The most common script is Pay2PubkeyHash but it isn't the only type of script.
5) Transactions "spend" discrete outputs not a specific value.
6) Outputs are either spent or unspent.  Unspent outputs can be "spent" in a new transaction.

If you don't have a good understanding of how Bitcoin actually works then you will probably end up just running around in circles. 

The historical blockchain builds the UTXO set.  To build the UTXO you start from block 1 (the output in the genesis block is unspenable) and for each block you add to the UTXO all the new txn outputs and you remove from the UTXO all those outputs referenced in txn inputs.  When you get to the current block you will have the current UTXO.  Note this process assumes the blockchain is already validated which is a safe assumption if you are pulling blocks from bitcoind.

The UTXO set has all the information you need.  When someone say address x has a balance of 1.234 BTC what they really mean is that the UTXO contains one or more unspent outputs which have a combined value of 1.234 BTC.

Yes I listen to Danny, his advices were very helpful... you guys really know about this stuff.

But can you guys just confirm if what I said in the other post is correct?

All I need now is to set a Bitcoin server and send the commands via http and I will be able to start doing tons of queries.

I didn't test yet the Balance calculations, but for now I will just fetch all addresses that are in the "addresses" part [in the getrawtransaction or decoderawtransaction json responses],
like this I'm able to fetch all addresses that were ever in the blockchain, which means that they either have BTCs or had BTCs in the past,

so like this I will be able to fetch all Bitcoin used addresses EVER on the mainnet, right?

Is this assumption correct?