Bitcoin Forum
September 03, 2026, 04:44:44 AM *
News: Latest Bitcoin Core release: 31.1 [Torrent]
 
   Home   Help Search Login Register More  
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 [17]  All
  Print  
Author Topic: List of all Bitcoin addresses with a balance  (Read 12762 times)
This is a self-moderated topic. If you do not want to be moderated by the person who started this topic, create a new topic. (15 posts by 2+ users deleted.)
albert0bsd
Hero Member
*****
Offline

Activity: 1126
Merit: 728



View Profile
August 28, 2026, 02:07:00 PM
 #321

I just try to download the lastest file and sadly the server is down, reading the recent messages i think i wil put a mirror ot the files as soon i can get a copy of them.

@LoyceV I will let you know when my mirror is ready.

₿ SIGNATURE SPACE AVAILABLE    HERO MEMBER    PM bc1qjyhcjacmpc9pg8wes82lktyrjcwk4sdvqtm7ky
LoyceV (OP)
Legendary
*
Offline

Activity: 4144
Merit: 22597


Thick-Skinned Gang Leader and Golden Feather 2021


View Profile WWW
August 28, 2026, 05:03:56 PM
 #322

i think i wil put a mirror ot the files as soon i can get a copy of them.
I can rsync them to you the second I update them if you want. I've only killed Apache2, my server and updates are still running.

¡uʍop ǝpᴉsdn pɐǝɥ ɹnoʎ ɥʇᴉʍ ʎuunɟ ʞool no⅄
albert0bsd
Hero Member
*****
Offline

Activity: 1126
Merit: 728



View Profile
August 28, 2026, 05:17:53 PM
 #323

I can rsync them to you the second I update them if you want.

Yes, absolutely. That would be great.

I can set up a dedicated account/SSH key with access only to the mirror directory, so you can rsync the files whenever you update them.

Let me know what you need from my side. Send me a DM if you like

₿ SIGNATURE SPACE AVAILABLE    HERO MEMBER    PM bc1qjyhcjacmpc9pg8wes82lktyrjcwk4sdvqtm7ky
albert0bsd
Hero Member
*****
Offline

Activity: 1126
Merit: 728



View Profile
August 29, 2026, 02:51:22 PM
Last edit: August 31, 2026, 06:11:36 PM by albert0bsd
Merited by LoyceV (4), NotATether (2)
 #324

Thank you loyceV for the rsync Grin

Here is my mirror:

Code:
http://addresses.albertobsd.dev/

I remove LATEST symlink file to avoid people trying to update it each 3 minutes, All other files will be there.

₿ SIGNATURE SPACE AVAILABLE    HERO MEMBER    PM bc1qjyhcjacmpc9pg8wes82lktyrjcwk4sdvqtm7ky
LoyceV (OP)
Legendary
*
Offline

Activity: 4144
Merit: 22597


Thick-Skinned Gang Leader and Golden Feather 2021


View Profile WWW
August 29, 2026, 03:21:56 PM
 #325

Here is my mirror:
Code:
https://addresses.albertobsd.dev/
Suggestion: sort the files by date instead of alphabet.
Enjoy your burned bandwidth Tongue
Updates are scheduled, they start right after my update finishes.

Quote
I remove LATEST symlink file to avoid people trying to update it each 3 minutes, All other files will be there.
I've excluded it from rsync. It should also remove older files.



It occurred to me....
What if I provide uncompressed text-files? Would that work with rsync? I've never used rsync to a public location. It would save so much bandwidth if only the removed/added addresses have to be downloaded each day!
Alternatively: what if I provide only a list of changes, that users can easily add on their own each day? It could even be a daily list of changes from the past week, so downloading once a week is enough to be up to date again.
Thoughts?

¡uʍop ǝpᴉsdn pɐǝɥ ɹnoʎ ɥʇᴉʍ ʎuunɟ ʞool no⅄
pbies
Sr. Member
****
Offline

Activity: 443
Merit: 272



View Profile
August 30, 2026, 12:37:23 AM
 #326

It occurred to me....
What if I provide uncompressed text-files? Would that work with rsync? I've never used rsync to a public location. It would save so much bandwidth if only the removed/added addresses have to be downloaded each day!
Alternatively: what if I provide only a list of changes, that users can easily add on their own each day? It could even be a daily list of changes from the past week, so downloading once a week is enough to be up to date again.
Thoughts?

You would need to host "diff" files. Much work on client side to update the file.

Also .txt goes full size thru Internet (can be even packed in the air but still bandwidth takes whole file) so loss of bandwidth.

I don't like neither of these solutions. My own opinion.

@albert0bsd HTTPS will kill your server fast when there will be few downloads at the same time. Maybe plain HTTP?

BTC: bc1qmrexlspd24kevspp42uvjg7sjwm8xcf9w86h5k
LoyceV (OP)
Legendary
*
Offline

Activity: 4144
Merit: 22597


Thick-Skinned Gang Leader and Golden Feather 2021


View Profile WWW
August 30, 2026, 07:38:15 AM
 #327

You would need to host "diff" files. Much work on client side to update the file.
I've never created diff files, but shouldn't it take only seconds to process from a script? I doubt it's slower than downloading and decompressing gz.

Quote
Also .txt goes full size thru Internet (can be even packed in the air but still bandwidth takes whole file) so loss of bandwidth.
Rsync can take care of that. Let's test it:
1. I've uncompressed Bitcoin_addresses_August_29_2026.txt.gz on my server to "test", and downloaded it through compressed rsync:
Code:
sent 43 bytes  received 1,75G bytes  1,07M bytes/sec
total size is 2,81G  speedup is 1,60
2. I've uncompressed Bitcoin_addresses_August_30_2026.txt.gz on my server to "test", and downloaded it through compressed rsync:
Quote
sent 424,24K bytes  received 1,54G bytes  1,03M bytes/sec
total size is 2,81G  speedup is 1,83
This is disappointing. I had higher hopes for rsync being smart enough to handle this with much less bandwidth.

Quote
I don't like neither of these solutions. My own opinion.
Thanks. I'm just looking for an option to keep it online.

@albert0bsd: are updates working on your mirror? Your listing was last updated yesterday, and files that should be deleted are still shown.



I don't know why downloads from this server are now sooooooo slow! It used to be much faster:
I also tested Bitcoin addresses with a balance:
Code:
-                    41%[=======>            ] 561.19M  55.5MB/s    eta 11s
This is a different server. Much smaller, but fast disk.

¡uʍop ǝpᴉsdn pɐǝɥ ɹnoʎ ɥʇᴉʍ ʎuunɟ ʞool no⅄
pbies
Sr. Member
****
Offline

Activity: 443
Merit: 272



View Profile
August 30, 2026, 04:02:36 PM
 #328

Diff files are short, but to make changes to whole addresses file is at least to once copy it (in the same folder) and apply the changes.
He would need to search in the file (so read it line by line) for single lines and apply the patch. Always it is at least to go once thru file line by line and make at least one copy of the whole file (the changed one beside the source file being in the same folder).

BTC: bc1qmrexlspd24kevspp42uvjg7sjwm8xcf9w86h5k
albert0bsd
Hero Member
*****
Offline

Activity: 1126
Merit: 728



View Profile
August 30, 2026, 09:29:35 PM
 #329

@albert0bsd HTTPS will kill your server fast when there will be few downloads at the same time. Maybe plain HTTP?

Thank you for the recommendation, I will monitor the CPU usage, right now is 3% usage, for the moment is OK.

The problem is all .dev top-level domains are built into the HSTS preload list, which forces all web browsers to connect via HTTPS only.
For scripts I can redirect HTTP but for browsers I cannot do anything.

@albert0bsd: are updates working on your mirror? Your listing was last updated yesterday, and files that should be deleted are still shown.

The rsync directory and the web root directory are different (sorry for that this is because the directory user permissions are different between your user and the apache user)

I just add a hourly cron job to update new files into the web directory. I will add some script later to work with the deleted files.

₿ SIGNATURE SPACE AVAILABLE    HERO MEMBER    PM bc1qjyhcjacmpc9pg8wes82lktyrjcwk4sdvqtm7ky
LoyceV (OP)
Legendary
*
Offline

Activity: 4144
Merit: 22597


Thick-Skinned Gang Leader and Golden Feather 2021


View Profile WWW
September 02, 2026, 11:35:02 AM
 #330

I've killed Apache2 again. This is my bandwidth usage in the past 2 days:
Image loading...

The problem is that, apart from a few obvious abusers, the bandwidth goes to 1529 different IP-addresses that all downloaded more than 1 GB since yesterday.
This is what a few minutes of my access.log (excluding IP-addresses) looks like:
Code:
- - [02/Sep/2026:12:35:24 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 206 136488 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:35:28 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 139944 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:35:42 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 72800 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:28:10 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 1769562525 "-" "Wget/1.21.4"
- - [02/Sep/2026:12:35:51 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 72800 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:35:55 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 265720 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:35:58 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 105120 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:36:00 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 169360 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:36:02 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 200376 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:36:05 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 206 136488 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:36:09 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 137088 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:36:22 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 72800 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:36:31 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 72800 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:36:36 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 268640 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:36:38 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 105120 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:36:41 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 201480 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:36:42 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 200376 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:36:46 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 206 328152 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:41:47 +0200] "HEAD /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 250 "-" "curl/7.88.1"
- - [02/Sep/2026:12:29:49 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 1769562525 "-" "Wget/1.21.4"
- - [02/Sep/2026:12:36:49 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 137088 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:11:53:12 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 988793760 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:37:03 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 72800 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:37:12 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 72800 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:37:16 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 137240 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:37:19 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 105120 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:37:21 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 201480 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:37:23 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 200376 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:37:26 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 206 139392 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:37:30 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 137088 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:37:43 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 72800 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:37:52 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 72800 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:37:57 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 137240 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:38:00 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 105120 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:38:02 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 201480 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:38:04 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 200376 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:38:07 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 206 136488 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:38:10 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 137088 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:38:24 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 72800 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:38:33 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 72800 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:38:37 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 137240 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:38:40 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 105120 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:38:42 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 201480 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:38:44 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 168432 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:38:48 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 206 136488 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:38:51 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 139944 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:39:04 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 72800 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:24:03 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 369806320 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:39:13 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 72800 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:39:17 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 268640 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
- - [02/Sep/2026:12:39:21 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 105120 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
Those requests come from many different IP-addresses.

When I sort those 1529 different IP-address (the ones that downloaded >1 GB), I get many IPs that belong to the same range, like 64.236.x.x (a Microsoft datacenter), 57.151.x.x (another Microsoft datacenter) and many more. Those ranges are all running Wget/1.21.4.

At this rate, it's unsustainable to keep sending them more than 1 TB per day.
They may or may not be legitimate users, but considering the bursts in bandwidth, I think it's a few entities that are completely wasting my bandwidth.

I'm open to suggestions Smiley

¡uʍop ǝpᴉsdn pɐǝɥ ɹnoʎ ɥʇᴉʍ ʎuunɟ ʞool no⅄
pbies
Sr. Member
****
Offline

Activity: 443
Merit: 272



View Profile
September 02, 2026, 11:50:34 AM
 #331

CAPTCHA or token.

BTC: bc1qmrexlspd24kevspp42uvjg7sjwm8xcf9w86h5k
LoyceV (OP)
Legendary
*
Offline

Activity: 4144
Merit: 22597


Thick-Skinned Gang Leader and Golden Feather 2021


View Profile WWW
September 02, 2026, 12:27:20 PM
 #332

CAPTCHA or token.
Captcha is terrible, it kills automated downloads. And spammers somehow still get a way around it.

Any idea how I could implement an authentication token? A thousand downloads per day wouldn't be any problem at all if they all pay $1 per year.

¡uʍop ǝpᴉsdn pɐǝɥ ɹnoʎ ɥʇᴉʍ ʎuunɟ ʞool no⅄
Mek
Member
**
Offline

Activity: 86
Merit: 11

mtc.mekweb.eu - mega transistor clock


View Profile WWW
September 02, 2026, 12:33:09 PM
 #333

Make a page that will issue a token valid for 3 complete downloads of the latest file per day per IP (for example). 3, not 1, because it's possibly hard to tell server-side if the download was successful. Who wants more, should contact you to arrange payment.
Or maybe automate the latest file to be hosted as a torrent.

My Scrabble game: skrebl.eu
My database of electronic parts: elparts.mekweb.eu
My DIY electronic kit - mega transistor clock: mtc.mekweb.eu
LoyceV (OP)
Legendary
*
Offline

Activity: 4144
Merit: 22597


Thick-Skinned Gang Leader and Golden Feather 2021


View Profile WWW
September 02, 2026, 12:42:54 PM
Last edit: September 02, 2026, 02:44:29 PM by LoyceV
 #334

With all this abuse downloading 100 MB/s, my own download (after killing Apache2) does less than 1 MB/s. I don't get it...

Make a page that will issue a token
This is where you lost me. I have no idea how to do "interactive" websites.

Quote
Or maybe automate the latest file to be hosted as a torrent.
I really don't like this option, as it exposes IP-addresses, but I may have to do it anyway. Most downloads come from servers anyway.
The next problem with torrents is that they "stay alive" when they're outdated, but it would be the user's own responsibility to get the latest version.



Thinking about the problem: ideally, I should have a "token" that can be added to an URL, and counts the total bandwidth used by each token. That way, it doesn't matter if the user makes a mistake and downloads the same file twice, or downloads all files I have. But if the token is shared, it'll be maxed out and blocked.
The problem is that I have no idea how to implement this. $1 for a token valid for 1 year or 1 TB bandwidth (whichever is reached first) would stop abuse and is cheap enough. I could also give away tokens to anyone who asks for it.

¡uʍop ǝpᴉsdn pɐǝɥ ɹnoʎ ɥʇᴉʍ ʎuunɟ ʞool no⅄
pbies
Sr. Member
****
Offline

Activity: 443
Merit: 272



View Profile
September 02, 2026, 03:54:37 PM
 #335

Torrent is not a bad idea but some short unique name and date in format YYMMDD, like 260902.

BTC: bc1qmrexlspd24kevspp42uvjg7sjwm8xcf9w86h5k
albert0bsd
Hero Member
*****
Offline

Activity: 1126
Merit: 728



View Profile
September 02, 2026, 04:12:53 PM
 #336

This is where you lost me. I have no idea how to do "interactive" websites.

I can help you with that.

If you guys are going to use torrent I can also share the files in tat way

₿ SIGNATURE SPACE AVAILABLE    HERO MEMBER    PM bc1qjyhcjacmpc9pg8wes82lktyrjcwk4sdvqtm7ky
LoyceV (OP)
Legendary
*
Offline

Activity: 4144
Merit: 22597


Thick-Skinned Gang Leader and Golden Feather 2021


View Profile WWW
September 02, 2026, 05:50:00 PM
 #337

Torrent is not a bad idea but some short unique name and date in format YYMMDD, like 260902.
If I use torrent, I was thinking of using the same filename for each update. That way, anyone can download the small torrent file as often as they want, and the torrent system does the rest.

This is where you lost me. I have no idea how to do "interactive" websites.
I can help you with that.
Great!

¡uʍop ǝpᴉsdn pɐǝɥ ɹnoʎ ɥʇᴉʍ ʎuunɟ ʞool no⅄
pbies
Sr. Member
****
Offline

Activity: 443
Merit: 272



View Profile
September 02, 2026, 09:47:20 PM
 #338

@LoyceV

Torrent duplication here arrives. Not good. It can fail to work.

Before seed propagates the file there will pass some time.
It will take time for people to spread the file for current day.

I advise to use both magnet link and .torrent file on website (I see it is down, just needed all addresses ever used, I will go for old one on my disk, EDIT: I don't have it so I need to download it).

BTC: bc1qmrexlspd24kevspp42uvjg7sjwm8xcf9w86h5k
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 [17]  All
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!