List of all Bitcoin addresses with a balance

[citb0in]

Sorry, I didn't mention that my question is related to the thread, but indeed.
The LoyceVs list contains the line. So, the question was "Why does the list contain the line if the address doesn't exist in the real blockchain?".
The same for these addresses (for convenience, the first one is also included):
bc1pqcmm7krqrmsjr4k84r85v6s9rhnl9warprrp0t
bc1sw50qgdz25j
--snip--

I see. But I can't find the Bitcoin address bc1qpcmm7krqrmsjr4k84r85v6s9rhnl9warprrp0t that you originally mentioned in the current LoyceV funded addresses list:

even up to this prefix there is no single hit

Code:

$ grep 'bc1qpcmm7k' blockchair_bitcoin_addresses_and_balance_LATEST.tsv

<nothing>

first hit occurs here, but this is not the address you mentioned before.

Code:

$ grep 'bc1qpcmm7' blockchair_bitcoin_addresses_and_balance_LATEST.tsv 

bc1qpcmm74pjafundkxm79pk4hl42skcy7dswa7skq   18000

Now to the other addresses you have listed...

The same for these addresses (for convenience, the first one is also included):
bc1pqcmm7krqrmsjr4k84r85v6s9rhnl9warprrp0t
bc1sw50qgdz25j
--snip--

The first line has a typo, see the address you posted at the beginning and see this address you mentioned:
bc1qpcmm7krqrmsjr4k84r85v6s9rhnl9warprrp0t
bc1pqcmm7krqrmsjr4k84r85v6s9rhnl9warprrp0t

Apparently you have misspelled the address in your initial question. You have a typo Being that said, the address with the prefix 'bc1p' is in the LoyceV list:

Code:

$ grep 'bc1pqcmm7krqrmsjr4k84r85v6s9rhnl9warprrp0t' blockchair_bitcoin_addresses_and_balance_LATEST.tsv 

bc1pqcmm7krqrmsjr4k84r85v6s9rhnl9warprrp0t   41354

Now we are one step further. These are special other address types. You can find more information HERE.

Hope this answers your question.
citb0in

[LoyceV]

Why do I see only the address bc1qpcmm7krqrmsjr4k84r85v6s9rhnl9warprrp0t on blockchair.com but nowhere else?

This address is empty and not included in my list of funded addresses.

The LoyceVs list contains the line.

No, it does not.

So, the question was "Why does the list contain the line if the address doesn't exist in the real blockchain?".

Search the differences:
https://blockchair.com/bitcoin/address/bc1qpcmm7krqrmsjr4k84r85v6s9rhnl9warprrp0t
https://blockchair.com/bitcoin/address/bc1pqcmm7krqrmsjr4k84r85v6s9rhnl9warprrp0t

Lesson learned: use copy paste and carefully check each character. If I type an address on an offline computer, I create a checksum on both just to be sure I typed it correctly.

[Ermitage]

Guys, thank you for your discovery and answers.
It looks a bit strange because I always use copy/paste and so on.
But the lines I see in the file I downloaded this morning.
I am ready to provide screenshots to all of mentioned addresses.
But, I already have an idea why it can be so. Bad if it affected not only me.
I just extracted the data using 7-zip and those text lines are in the result file. Maybe some bytes were mixed (shifted)
But I like more the idea of different witness versions.
Screenshot for some lines: https://ibb.co/tJJhGRL
Blockchain on the source site: https://blockchair.com/bitcoin/address/bc1pqcmm7krqrmsjr4k84r85v6s9rhnl9warprrp0t

[LoyceV]

I just extracted the data using 7-zip and those text lines are in the result file. Maybe some bytes were mixed (shifted)

If that's the case, write to 7-zip. I just use tar, and it works fine.

[efjay]

Many thanks for making this available, liked it enough to login to BCT and post for the first time in years!!


I started playing with the all addresses with balance data and made a poor version of a full ranked list.  Simple one line command in bash or git bash (and others) if people are interested.

For those that want to, download the data file locally and use something like this (probably more efficient ways to do this but I was in a hurry ) :

zcat datafile.tar.gz | grep -v address | sort -k2 -nr | nl -w8 - > outputfilename.txt

This will rank from largest as #1 down to smallest (over 50 million addresses currently).


Unfortunately it seems the BTC addresses that I have are off-chain / micro wallets.  Is this an area that interests you to expand the dataset?  (I don't know much about how these sort of wallets/systems work under the covers at this stage so not sure what is in the art of the possible)

Assuming the places that run these wallets provide access to the data in some way of course, at this stage I haven't gone down the path of opening any discussion with blockchair directly.

Enjoy!

[citb0in]

zcat datafile.tar.gz | grep -v address | sort -k2 -nr | nl -w8 - > outputfilename.txt

This will rank from largest as #1 down to smallest (over 50 million addresses currently).

I don't know which file you have downloaded but I am confused about your procedure because: the list, which also contains the balances, is sorted by default by balance with the highest at the top and the lowest at the bottom. So you don't have to sort anything YMMV

[efjay]

I don't know which file you have downloaded but I am confused about your procedure because: the list, which also contains the balances, is sorted by default by balance with the highest at the top and the lowest at the bottom. So you don't have to sort anything YMMV

Fair comment.  guess it future proofs any changes to the sorting of the source data as long as the balance data is in the same column

Cheers

[LoyceV]

Many thanks for making this available, liked it enough to login to BCT and post for the first time in years!!

You're welcome

Unfortunately it seems the BTC addresses that I have are off-chain / micro wallets.

What do you mean by "off-chain"? Bitcoin addresses can only be on-chain (unless they're unused, in that case they're nowhere).

[efjay]

Unfortunately it seems the BTC addresses that I have are off-chain / micro wallets.

What do you mean by "off-chain"? Bitcoin addresses can only be on-chain (unless they're unused, in that case they're nowhere).

An example being a deposit address on Faucetpay.io is not in the list of 50+ million addresses (source data http://addresses.loyce.club-blockchair/bitcoin_addresses_and_balance_LATEST.tsv.gz from a few days ago at least).

In this particular case the address starts with 1EZASQnt79J and can be sent to from many locations (do they all have their own faucetpay integration feature outside of the BTC blockchain - I don't know).

Cheers

[LoyceV]

An example being a deposit address on Faucetpay.io is not in the list

That's because the address was never used on-chain. I don't use Faucetpay, but I guess they use the Bitcoin address as "account number". Anyone can do anything they want off-chain, there's no way to index that.

[efjay]

An example being a deposit address on Faucetpay.io is not in the list

That's because the address was never used on-chain. I don't use Faucetpay, but I guess they use the Bitcoin address as "account number". Anyone can do anything they want off-chain, there's no way to index that.

Yeah I had a quick look at one of the addresses and blockchair has some history data, so I think they use it like a clearing account.

What I mean by that is that it is a real address available onchain but also offchain (some of my terminology might be a bit wrong but feel free to correct me if there's better ways to elaborate on how it works).

Offchain usage (or is that considered sidechain) is like the network of systems that can talk direct to the (in this case the one i looked at, faucetpay) network/system, allowing transfers to be fee free if the sending system chooses to do so (transactions don't show in blockchair in this case).

Onchain usage is normal real transfers via the block chain, registers in the historical data (eg on blockchair) and then at some stage (algo? time? amount? don't know) is moved out of that address to another address (hence the clearing) - but in the interface that I see the amount is still allocated to me.  So my balance to my (faucetpay) address is buried in another address, hopefully readily available to me when i need to spend it!

And that would explain why my address doesn't show in the list of all address and balances above zero, as when they clear it they move the whole balance out.

Cheers

[LoyceV]

What I mean by that is that it is a real address available onchain

It's easy to generate billions of "real" addresses, but they're meaningless until they receive an on-chain Bitcoin payment.

And that would explain why my address doesn't show in the list of all address and balances above zero

Bitcoin's blockchain is a ledger for anything that ever happened. Things made up by Faucetpay internally have nothing to do with Bitcoin.

[runlola]

The files haven't been updated in over a week, though the source files on blockchair are still updated. Don't see any maintenance status mentioned on the download site or this topic.

[marssystems]

The files haven't been updated in over a week

It looks like the daily updates didn't happen, which means the weekly update had no fresh data. I'm not sure why, I'll check it tomorrow.

Update: it's fixed. It was waiting for another stuck update. The weekly update is running now, this takes a few hours to complete. Thanks for reporting this.

Files haven't been updated since 19 April 2024.

[LoyceV]

The files haven't been updated in over a week

I get this data from Blockchair.com, and their last update is from April 19. Their other data is still being updated, so it looks like something is stuck with updating the Bitcoin addresses.

Files haven't been updated since 19 April 2024.

I posted both posts in the wrong topic. It's correct now, the other post is here. Both updates got stuck, one was waiting for the other, which was waiting for Blockchair. In short: the list of all addresses with a balance is not updated at the moment.

[LoyceV]

I get this data from Blockchair.com, and their last update is from April 19. Their other data is still being updated, so it looks like something is stuck with updating the Bitcoin addresses.

The daily updates are back to normal

[LoyceV]

I tried a lot and could not put to work these commands on linux mint

I've deleted your post because of the unnecessary long quote.

Your screenshot already shows the problem:
[/quote]

comm -12 Bitcoin_addresses_LATEST.txt <(cat wallet.txt | sort | uniq)

The file with balances is indeed not sorted. And you can't use it, because comm matches whole lines. Remove the balances and sort it, or download the file without balances.

comm -12 Bitcoin_addresses_LATEST.txt <(cat wallet.txt | fromdos | sort | uniq)

If you need fromdos, install tofrodos, as it tells you to do.

Running this as root is bad practice.

[pbies]

I use these 3 scripts:

Get addresses and balances and unpack them:

Code:

#!/usr/bin/env bash
wget http://addresses.loyce.club/blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz
pv blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz | gunzip > blockchair_bitcoin_addresses_and_balance_LATEST.tsv
>&2 echo -ne "\x7"

Take only addresses, sort and uniq them:

Code:

#!/usr/bin/env bash
pv -B 1M blockchair_bitcoin_addresses_and_balance_LATEST.tsv | cut -f 1 | sort -u > addrs-with-bal.txt
>&2 echo -ne "\x7"

Compare two files, one with addresses and second your one with addresses you are searching for:

Code:

#!/usr/bin/env bash
comm -12 <(pv -cN 1st "$1") <(pv -cN 2nd "$2")
>&2 echo -ne "\x7"

[silaslyma]

Compare two files, one with addresses and second your one with addresses you are searching for:

Code:

#!/usr/bin/env bash
comm -12 <(pv -cN 1st "$1") <(pv -cN 2nd "$2")
>&2 echo -ne "\x7"

Sorry, i do not know what i am doing wrong,
I tested with several addresses and only found 1

https://i.ibb.co/R3v2fJk/1.jpg

https://i.ibb.co/M9NXhgL/2.jpg

[silaslyma]

comm -12 Bitcoin_addresses_LATEST.txt <(cat wallet.txt | fromdos | sort | uniq)

If you need fromdos, install tofrodos, as it tells you to do.

Now it worked. Thank you very much