All used addresses

[mamuu]

How to dump  mainnet all used addresses without balance or tx ?

[NeuroticFish]

How to dump  mainnet all used addresses without balance or tx ?

If there are no balance and also no tx, the address is not used and you can't really know if somebody has it in his wallet or not.
So this makes you ask for pretty much all Bitcoin addresses, which is basically impossible to get, they're far too many.

[LoyceV]

How to dump  mainnet all used addresses without balance or tx ?

The easiest way is to download Blockchair Database Dumps, but at 10 KB/s it takes months. I'm already downloading it, and will publish the data for faster downloads once it's done. I don't know how many weeks/months it'll take though.

Update: It's done, see List of all Bitcoin addresses ever used!

Related data:
Bitcoin block data available in CSV format
List of all Bitcoin addresses with a balance

How to dump  mainnet all used addresses without balance or tx ?

If there are no balance and also no tx

He's asking for used addresses, so they must have received a transaction at some point.

[NeuroticFish]

How to dump  mainnet all used addresses without balance or tx ?

If there are no balance and also no tx

He's asking for used addresses, so they must have received a transaction at some point.

You are right. Still, he asked about "no tx" and I wanted to make sure he understands what he wants and maybe "refine" his request.
I somehow think that he may have confused the unused addresses he can see in his walled as "used, but with no transactions".

If indeed he wants addresses with "no tx" then they've never got any transaction ever. And in that case a dump of addresses will not help him much

[bob123]

How to dump  mainnet all used addresses without balance or tx ?

If there are no balance and also no tx

He's asking for used addresses, so they must have received a transaction at some point.

And at the same time he is asking for addresses without a balance/transaction, so they have to be unused.

Since the request itself is illogical, we need to wait for OP to clarify his request.

@OP
In case you are simply looking for used addresses: Download core and the whole blockchain with the txindex flag.
You will have the complete data.
Then you just need to extract what you are searching for.

This way you don't need to rely on a 3rd party which takes weeks to download from.
Depending on your processing power and bandwidth, syncing the whole blockchain might be much faster.

[mamuu]

thank you for your answers

i need just used address database in mainnet

sorry my english poor :/

actually i need a list of processed but empty wallet addresses

there is usually an address list with balance

[bob123]

In this case you could proceed as mentioned above.

Use core to sync the whole blockchain with the txindex flag.
Then build a database using this data.

[BTCW]

Try this. I use it all the time.

https://github.com/alecalve/python-bitcoin-blockchain-parser

[Tamarindei]

For 0.005 btc (~50$) you can contact the person here and get a recent dump:

https://balances.crypto-nerdz.org/

Maybe you can get it cheaper if you ask. He seems to be a fair guy.

Regards

[TheArchaeologist]

Get all balances here (currently unlocked, 771 megabyte):
https://balances.crypto-nerdz.org/balances/balances-bitcoin-20200708-0000-MXSyuyTD.gz

Please note: it's not my site/data and link will probably be locked again. So if you need it I suggest you get it now.

[naufragus]

I'm already downloading it, and will publish the data for faster downloads once it's done. I don't know how many weeks/months it'll take though.

Hey LoyceV
Thanks for downloading and offering to share that.
i am downloading it too, with a bash script, two days and downloaded 10GB..
i think those dumps amount to 1TB, so it will take too long to download from only one location..
if you can share the files when you are ready, i will be very ineterested.
cheers!

PS: yes, i think i added some zeroes when calculating download size. this data set size from blockchair, specifically, seems close to 100GB..

[LoyceV]

i am downloading it too, with a bash script, two days and downloaded 10GB..
i think those dumps amount to 1TB, so it will take too long to download from only one location..

Blockchair increased the download speed from 10 to 100 kB/s. I'm currently downloading November 2018. I'm not sure yet if I'll share the raw data, I'd need another VPS because of the size.
It's probably much less than 1 TB though, the entire blockchain isn't that big.

[naufragus]

hey LoyceV

yeah, and there is this new feature in 0.20 which seems it can dump snapshots of unspent transaction outputs with `dumptxoutset`..
when they update boincoind in my distro, i will check that out.
indeed my downloads are between 48KB/s and 98KB/s from blockchair, still if i could access from three different locations it should take less than 70 days to download everything.. if from one location, 200 days..
i really just wanted to grep those dumps, sort and uniq those addresses..

[LoyceV]

i really just wanted to grep those dumps, sort and uniq those addresses..

That's my plan I'm curious to see how many different addresses have been used.
Once I have it, I'll provide daily updates.

if i could access from three different locations

Getting 3 different VPSses (VPSs?) wouldn't even be very expensive, but it feels like cheating Blockchair's download restrictions. I don't think it will take 200 days at 100 kB/s. I'm currently at February 2019.

This is all I have so far, feel free to download 74 GB: http://loyceipv6.tk:20319/blockdata/

[TheBeardedBaby]

The easiest way is to download Blockchair Database Dumps, but at 10 KB/s it takes months.

I tried downloading it and here shows 100 KB/s. I can download them and upload them to google drive if you want I'll be away for a few weeks but I can do that job, just let me know.

[LoyceV]

I tried downloading it and here shows 100 KB/s. I can download them and upload them to google drive if you want I'll be away for a few weeks but I can do that job, just let me know.

Thanks, but no need, I downloaded 4 months since yesterday, so I should be done in a 4 days.

[naufragus]

Hey LoyceV

Thank you very much for your effort for uploading those dumps..
Sorry i can't repay.

I am downloading the files, should not take too long now..

As for cheating Blockchair's download restrictions, i don't think that would be bad or evil.

here is a shell script i'm using for downloading these files (requires gnu coreutils)..
https://github.com/mountaineerbr/scripts/blob/master/blockchair.btcoutputs.sh

Take care everyone!

[LoyceV]

Sorry i can't repay.

No problem. I'll run out of disk space on the VPS soon though, so I'll have to start deleting older files.
Judging by bandwidth consumption, someone else downloaded the data already. I'll also run out of monthly bandwidth if only a few people download all 84 GB, but since I have no other use for the VPS, that's okay too.

here is a shell script i'm using for downloading these files (requires curl and bash)..
https://github.com/mountaineerbr/scripts/blob/master/blockchair.btcblockhain.outputs.sh

Mine is a lot shorter, usually I type everything on just one line. I still have to adjust it for processing the data, but I'll do that when it's complete.

[LoyceV]

Answering this deleted post, I'm currently running this:

Code:

for file in *.tsv.gz; do gunzip -c $file | grep -v is_from_coinbase | cut -f 7 >> /tmp/addresses.txt; done

It takes a while, but doesn't consume a lot of memory. When done, I'll sort | uniq the file and get the result. Now that I think about it, I could have piped the whole thing through sort in the same line instantly.

When I have a bit more time, I'll create daily updates for all used Bitcoin addresses and make it available for download. It's going to be a big file though.

Blockchair's /bitcoin/outputs currently takes 106 GB, and grows 2 GB per month. At 100 KB/s, it takes just under 2 weeks to download from Blockchair. For $32 per year, I can run a VPS in Germany with 1 Gbit connection, enough disk space to keep up for a few years, and enough bandwidth to allow 9 downloads per month. If anyone can use this, let me know.

The list with all addresses is 49 GB in size. If you tried to load it to RAM, that's probably why you ran out of memory.
Total address count: 1,484,589,749
1... address count: 1,039,899,708
3... address count: 343,485,961
bc1q... address count: 55,006,904
...-... (with a "dash") address count: 46,197,161

Unique address count:
1... address count: 470,943,308
3... address count: 167,941,821
bc1q... address count: 39,137,878
...-... (with a "dash") weird address count: 15,157,808

And here it stops for now: after processing data for 5 hours, I made a mistake and accidentally overwrote my end-result. I'll restart later.
I'd like to see which address has received most transactions.

[naufragus]

really pleased with the GNU sort (that is called the genius sort)..

this ought to be about right.. will try and upload split unique address list files to git.

one feature of my lists is i tried to keep the original order in which addresses first appeared in the blockchair dumps..

you should not do it all at once, if you have got disc space.. i have produced some intermediate  files for processing..


dump files from blockchair (4210 files):

from 20090103 to 20200718

addresses total: 1483853800

unique addresses total: 692773144

https://github.com/mountaineerbr/bitcoin-all-addresses