List of all Bitcoin addresses ever used - weekly updates work again!

[PrimeNumber7]

I'm trying to figure out how much RAM it uses in the worst case.

I can limit sort's memory usage. I'm more worried about the grinding this causes on the hard drive. I have no idea how much data gets read and written to sort 30 GB, but I assume every bit gets pickup up at least several times.

I'm trying to figure out how much RAM it uses in the worst case.

I can limit sort's memory usage. I'm more worried about the grinding this causes on the hard drive. I have no idea how much data gets read and written to sort 30 GB, but I assume every bit gets pickup up at least several times.

On second thought, I just had one of my servers' disks fail a couple days ago (all data was lost), so I'm not comfortable running these updating scripts on the rest of my hardware with all that grinding until I can set up a proper backup plan for my TBs of data.

Have you considered creating a script to update your existing list once you have a list of addresses? Your script could start at block_n and add an address if it is not already in your list. This should reduce read/write operations pretty significantly.

[1714099872]

1714099872

[1714099872]

1714099872

[1714099872]

1714099872

[1714099872]

1714099872

[LoyceV]

Have you considered creating a script to update your existing list once you have a list of addresses? Your script could start at block_n and add an address if it is not already in your list. This should reduce read/write operations pretty significantly.

That's what I tried to do for the addresses in chronological order, but it gave inconsistent results (and due to the time it takes to test it, I'm still not sure what caused it). I could probably do the same for the sorted addresses, by using comm instead of sort. That means only reading the compressed big file from disk, reading and sorting the weekly addition, and writing the new compressed file to disk. This does make more sense and reduces disk writes

[BoraxTributary]

For anyone stumbling open this in the future, I have created a torrent for this file, which accesses the web link.

Web Link: https://files.notatether.com/public/loycev/addresses_sorted.txt.gz

Torrent Magnet URL

.torrent file, good until 2022-09

This is up to date as of 2021-08-12. It will not be updated in the future, but is being left here to help, at least for now.

[BoraxTributary]

Another way to generate the file yourself, from a local copy of the blockchain: https://github.com/graymauser/btcposbal2csv

[LoyceV]

This is up to date as of 2021-08-12.

If it's the same file notatether.com published, it was last updated in May.

I still haven't found a decent affordable replacement host.

[mrxtraf]

What the addres with prefix s- and prefix m- and prefix d- ?

[pooya87]

What the addres with prefix s- and prefix m- ?

They are not addresses, they are output scripts that don't have any corresponding address and blockchair.com explorer uses an undocumented method to convert them into these strange looking strings.

[mrxtraf]

What the addres with prefix s- and prefix m- ?

They are not addresses, they are output scripts that don't have any corresponding address and blockchair.com explorer uses an undocumented method to convert them into these strange looking strings.

I understand that these are scripts. But where to look at the principle of formation. And what is the difference, s, m, d?

[LoyceV]

But where to look at the principle of formation. And what is the difference, s, m, d?

Nobody knows But if you enter one of them into Blockchair.com's Search field, it shows the transaction.
Sorry, I forgot it works differently: if you edit them into the URL of a Bitcoin address on Blockchair.com, it shows the transaction.
Example: blockchair.com/bitcoin/address/s-ffd80dee5966fb23c1a483b28f6bfcbc.

[mrxtraf]

But where to look at the principle of formation. And what is the difference, s, m, d?

Nobody knows But if you enter one of them into Blockchair.com's Search field, it shows the transaction.

I try. But not found 

[LoyceV]

But not found  

My mistake, sorry. Read my updated post

[pooya87]

And what is the difference, s, m, d?

As I said these are undocumented. Basically this particular explorer decided to use their own convention for naming these types of scripts and never told anyone how they are doing it or what they mean.
If I had to guess, they are probably computing the hash of the script (possibly SHA1 or MD5) then enter that smaller hash in their database to search based on a hash. The starting letter (s, m, ...) may be a quick indicator on where in the database to look them up.

[LoyceV]

Finally, this service is back online! I got myself a new VPS:

The data
See alladdresses.loyce.club (new location)

I'm still working on an update, I expect to upload it tomorrow.

This server allows 8 TB of bandwidth per month (and I'm still hoping to have it doubled because of the Black Friday deal), but doesn't have the disk space to run updates. I'm not sure yet how often I'll update it (after tomorrow's update).
For now: enjoy!

An update on my statistics from August last year:

Some interesting (?) statistics (updated until blockchair_bitcoin_outputs_20211202)
Total address count: 1,967,537,866
1... address count: 1,227,646,688
3... address count: 543,236,056
bc1q... address count: 145,894,845
...-... (with a "dash") address count: 50,758,929

Unique address count: 927,366,160
1... address count: 547,749,478
3... address count: 272,205,566
bc1q... address count: 89,702,297
...-... (with a "dash") weird address count: 17,707,628

[icopress]

LoyceV, can you tell me where I can find a list of all companies represented on the coinmarketcap with verified accounts in third-party networks? For example, Twitter filter or Etherscan filter, etc. I apologize in advance for moving away from the topic of discussion, but I need advice ... and I figured that since you are a data guru, you can tell me where to look for what I need, (I would appreciate any feedback). 

[LoyceV]

LoyceV, can you tell me where I can find a list of all companies represented on the coinmarketcap with verified accounts in third-party networks?

No.

For example, Twitter filter or Etherscan filter, etc.

I'm so glad I barely know what "Twitter" or "Etherscan" means I stay away from shitcoins and spammy websites.

I apologize in advance for moving away from the topic of discussion, but I need advice ... and I figured that since you are a data guru, you can tell me where to look for what I need, (I would appreciate any feedback).  

Sorry, I have no useful feedback on this It seems more like a manual thing to collect custom data on custom networks.

[MrFreeDragon]

LoyceV, can you tell me where I can find a list of all companies represented on the coinmarketcap with verified accounts in third-party networks? For example, Twitter filter or Etherscan filter, etc. I apologize in advance for moving away from the topic of discussion, but I need advice ... and I figured that since you are a data guru, you can tell me where to look for what I need, (I would appreciate any feedback). 

Please take this list of approx. 7k coins represented on coinmarketcap with telegram groups and discord channels:

https://anonfiles.com/18Mcs300v5/coinmarketcap_chats_csv

[cixegz]

@LoyceV
i am download some tsv file those site
https://gz.blockchair.com/bitcoin/inputs/
http://addresses.loyce.club/
how to extract .tsv file  in terminal explain and best tool viewer suggest
thanks,

read this topic any update https://bitcointalk.org/index.php?topic=5379443.0
solve this i pay for 2022$

[LoyceV]

@LoyceV
i am download some tsv file those site
https://gz.blockchair.com/bitcoin/inputs/

That takes a while. I have the data, but still don't have a server big enough to offer faster downloads.

http://addresses.loyce.club/
how to extract .tsv file  in terminal explain and best tool viewer suggest
thanks,

After extracting the .gz file you don't need to extract the .tsv anymore: it's just plain text. I wouldn't suggest any "viewer", it's not really meant for humans to read. I use command line tools on Linux, a simple "head" or "tail" is all the viewing I need.

read this topic any update https://bitcointalk.org/index.php?topic=5379443.0
solve this i pay for 2022$

Sorry, that's out of my league.

[mynonce]

read this topic any update https://bitcointalk.org/index.php?topic=5379443.0
solve this i pay for 2022$

Sorry, that's out of my league.

...
iknow random private key: how to find y is small range or big range  atleast guess
...

If you have a private key, you can't know whether your y coordinate is in small range or big range before calculating it.
There is no relationship between your public key y value and your private key. Else you would be able to calculate the private key with the public key.
Or we could say:

... It is impossible ... because then ECDSA would be broken.

As of today 01/01/2022 

[LoyceV]

Finally, this service is back online! I got myself a new VPS:

The data
See alladdresses.loyce.club (new location)

I'm currently experimenting with updates. The VPS I'm using doesn't have enough space, so I use an external ("pay by the hour") host for updates. If all works out, I'll be able to provide monthly updates again with only a few hours of additional server time, and a few minutes of my own time.

The VPS got upgraded to 16 TB bandwidth per month, and downloads reach almost 1 Gbit/s.