|
albert0bsd
|
 |
August 28, 2026, 02:07:00 PM |
|
I just try to download the lastest file and sadly the server is down, reading the recent messages i think i wil put a mirror ot the files as soon i can get a copy of them.
@LoyceV I will let you know when my mirror is ready.
|
₿ SIGNATURE SPACE AVAILABLE • HERO MEMBER • PM • bc1qjyhcjacmpc9pg8wes82lktyrjcwk4sdvqtm7ky
|
|
|
LoyceV (OP)
Legendary

Activity: 4144
Merit: 22597
Thick-Skinned Gang Leader and Golden Feather 2021
|
 |
August 28, 2026, 05:03:56 PM |
|
i think i wil put a mirror ot the files as soon i can get a copy of them. I can rsync them to you the second I update them if you want. I've only killed Apache2, my server and updates are still running.
|
¡uʍop ǝpᴉsdn pɐǝɥ ɹnoʎ ɥʇᴉʍ ʎuunɟ ʞool no⅄
|
|
|
|
albert0bsd
|
 |
August 28, 2026, 05:17:53 PM |
|
I can rsync them to you the second I update them if you want.
Yes, absolutely. That would be great. I can set up a dedicated account/SSH key with access only to the mirror directory, so you can rsync the files whenever you update them. Let me know what you need from my side. Send me a DM if you like
|
₿ SIGNATURE SPACE AVAILABLE • HERO MEMBER • PM • bc1qjyhcjacmpc9pg8wes82lktyrjcwk4sdvqtm7ky
|
|
|
|
albert0bsd
|
 |
August 29, 2026, 02:51:22 PM Last edit: August 31, 2026, 06:11:36 PM by albert0bsd Merited by LoyceV (4), NotATether (2) |
|
Thank you loyceV for the rsync  Here is my mirror: http://addresses.albertobsd.dev/ I remove LATEST symlink file to avoid people trying to update it each 3 minutes, All other files will be there.
|
₿ SIGNATURE SPACE AVAILABLE • HERO MEMBER • PM • bc1qjyhcjacmpc9pg8wes82lktyrjcwk4sdvqtm7ky
|
|
|
LoyceV (OP)
Legendary

Activity: 4144
Merit: 22597
Thick-Skinned Gang Leader and Golden Feather 2021
|
 |
August 29, 2026, 03:21:56 PM |
|
Here is my mirror: https://addresses.albertobsd.dev/ Suggestion: sort the files by date instead of alphabet. Enjoy your burned bandwidth  Updates are scheduled, they start right after my update finishes. I remove LATEST symlink file to avoid people trying to update it each 3 minutes, All other files will be there. I've excluded it from rsync. It should also remove older files.
It occurred to me....What if I provide uncompressed text-files? Would that work with rsync? I've never used rsync to a public location. It would save so much bandwidth if only the removed/added addresses have to be downloaded each day! Alternatively: what if I provide only a list of changes, that users can easily add on their own each day? It could even be a daily list of changes from the past week, so downloading once a week is enough to be up to date again. Thoughts?
|
¡uʍop ǝpᴉsdn pɐǝɥ ɹnoʎ ɥʇᴉʍ ʎuunɟ ʞool no⅄
|
|
|
|
pbies
|
 |
August 30, 2026, 12:37:23 AM |
|
It occurred to me.... What if I provide uncompressed text-files? Would that work with rsync? I've never used rsync to a public location. It would save so much bandwidth if only the removed/added addresses have to be downloaded each day! Alternatively: what if I provide only a list of changes, that users can easily add on their own each day? It could even be a daily list of changes from the past week, so downloading once a week is enough to be up to date again. Thoughts?
You would need to host "diff" files. Much work on client side to update the file. Also .txt goes full size thru Internet (can be even packed in the air but still bandwidth takes whole file) so loss of bandwidth. I don't like neither of these solutions. My own opinion. @albert0bsd HTTPS will kill your server fast when there will be few downloads at the same time. Maybe plain HTTP?
|
BTC: bc1qmrexlspd24kevspp42uvjg7sjwm8xcf9w86h5k
|
|
|
LoyceV (OP)
Legendary

Activity: 4144
Merit: 22597
Thick-Skinned Gang Leader and Golden Feather 2021
|
 |
August 30, 2026, 07:38:15 AM |
|
You would need to host "diff" files. Much work on client side to update the file. I've never created diff files, but shouldn't it take only seconds to process from a script? I doubt it's slower than downloading and decompressing gz. Also .txt goes full size thru Internet (can be even packed in the air but still bandwidth takes whole file) so loss of bandwidth. Rsync can take care of that. Let's test it: 1. I've uncompressed Bitcoin_addresses_August_29_2026.txt.gz on my server to "test", and downloaded it through compressed rsync: sent 43 bytes received 1,75G bytes 1,07M bytes/sec total size is 2,81G speedup is 1,60 2. I've uncompressed Bitcoin_addresses_August_30_2026.txt.gz on my server to "test", and downloaded it through compressed rsync: sent 424,24K bytes received 1,54G bytes 1,03M bytes/sec total size is 2,81G speedup is 1,83 This is disappointing. I had higher hopes for rsync being smart enough to handle this with much less bandwidth. I don't like neither of these solutions. My own opinion. Thanks. I'm just looking for an option to keep it online. @albert0bsd: are updates working on your mirror? Your listing was last updated yesterday, and files that should be deleted are still shown.
I don't know why downloads from this server are now sooooooo slow! It used to be much faster:
|
¡uʍop ǝpᴉsdn pɐǝɥ ɹnoʎ ɥʇᴉʍ ʎuunɟ ʞool no⅄
|
|
|
|
pbies
|
 |
August 30, 2026, 04:02:36 PM |
|
Diff files are short, but to make changes to whole addresses file is at least to once copy it (in the same folder) and apply the changes. He would need to search in the file (so read it line by line) for single lines and apply the patch. Always it is at least to go once thru file line by line and make at least one copy of the whole file (the changed one beside the source file being in the same folder).
|
BTC: bc1qmrexlspd24kevspp42uvjg7sjwm8xcf9w86h5k
|
|
|
|
albert0bsd
|
 |
August 30, 2026, 09:29:35 PM |
|
@albert0bsd HTTPS will kill your server fast when there will be few downloads at the same time. Maybe plain HTTP?
Thank you for the recommendation, I will monitor the CPU usage, right now is 3% usage, for the moment is OK. The problem is all .dev top-level domains are built into the HSTS preload list, which forces all web browsers to connect via HTTPS only. For scripts I can redirect HTTP but for browsers I cannot do anything. @albert0bsd: are updates working on your mirror? Your listing was last updated yesterday, and files that should be deleted are still shown.
The rsync directory and the web root directory are different (sorry for that this is because the directory user permissions are different between your user and the apache user) I just add a hourly cron job to update new files into the web directory. I will add some script later to work with the deleted files.
|
₿ SIGNATURE SPACE AVAILABLE • HERO MEMBER • PM • bc1qjyhcjacmpc9pg8wes82lktyrjcwk4sdvqtm7ky
|
|
|
LoyceV (OP)
Legendary

Activity: 4144
Merit: 22597
Thick-Skinned Gang Leader and Golden Feather 2021
|
 |
September 02, 2026, 11:35:02 AM |
|
I've killed Apache2 again. This is my bandwidth usage in the past 2 days:  The problem is that, apart from a few obvious abusers, the bandwidth goes to 1529 different IP-addresses that all downloaded more than 1 GB since yesterday. This is what a few minutes of my access.log (excluding IP-addresses) looks like: - - [02/Sep/2026:12:35:24 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 206 136488 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:35:28 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 139944 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:35:42 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 72800 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:28:10 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 1769562525 "-" "Wget/1.21.4" - - [02/Sep/2026:12:35:51 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 72800 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:35:55 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 265720 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:35:58 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 105120 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:36:00 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 169360 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:36:02 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 200376 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:36:05 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 206 136488 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:36:09 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 137088 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:36:22 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 72800 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:36:31 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 72800 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:36:36 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 268640 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:36:38 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 105120 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:36:41 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 201480 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:36:42 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 200376 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:36:46 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 206 328152 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:41:47 +0200] "HEAD /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 250 "-" "curl/7.88.1" - - [02/Sep/2026:12:29:49 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 1769562525 "-" "Wget/1.21.4" - - [02/Sep/2026:12:36:49 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 137088 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:11:53:12 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 988793760 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:37:03 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 72800 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:37:12 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 72800 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:37:16 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 137240 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:37:19 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 105120 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:37:21 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 201480 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:37:23 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 200376 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:37:26 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 206 139392 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:37:30 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 137088 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:37:43 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 72800 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:37:52 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 72800 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:37:57 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 137240 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:38:00 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 105120 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:38:02 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 201480 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:38:04 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 200376 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:38:07 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 206 136488 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:38:10 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 137088 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:38:24 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 72800 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:38:33 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 72800 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:38:37 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 137240 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:38:40 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 105120 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:38:42 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 201480 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:38:44 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 168432 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:38:48 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 206 136488 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:38:51 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 139944 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:39:04 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 72800 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:24:03 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 369806320 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:39:13 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 72800 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:39:17 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 268640 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" - - [02/Sep/2026:12:39:21 +0200] "GET /blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz HTTP/1.1" 200 105120 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" Those requests come from many different IP-addresses. When I sort those 1529 different IP-address (the ones that downloaded >1 GB), I get many IPs that belong to the same range, like 64.236.x.x (a Microsoft datacenter), 57.151.x.x (another Microsoft datacenter) and many more. Those ranges are all running Wget/1.21.4. At this rate, it's unsustainable to keep sending them more than 1 TB per day. They may or may not be legitimate users, but considering the bursts in bandwidth, I think it's a few entities that are completely wasting my bandwidth. I'm open to suggestions 
|
¡uʍop ǝpᴉsdn pɐǝɥ ɹnoʎ ɥʇᴉʍ ʎuunɟ ʞool no⅄
|
|
|
|
pbies
|
 |
September 02, 2026, 11:50:34 AM |
|
CAPTCHA or token.
|
BTC: bc1qmrexlspd24kevspp42uvjg7sjwm8xcf9w86h5k
|
|
|
LoyceV (OP)
Legendary

Activity: 4144
Merit: 22597
Thick-Skinned Gang Leader and Golden Feather 2021
|
 |
September 02, 2026, 12:27:20 PM |
|
CAPTCHA or token. Captcha is terrible, it kills automated downloads. And spammers somehow still get a way around it. Any idea how I could implement an authentication token? A thousand downloads per day wouldn't be any problem at all if they all pay $1 per year.
|
¡uʍop ǝpᴉsdn pɐǝɥ ɹnoʎ ɥʇᴉʍ ʎuunɟ ʞool no⅄
|
|
|
Mek
Member


Activity: 86
Merit: 11
mtc.mekweb.eu - mega transistor clock
|
 |
September 02, 2026, 12:33:09 PM |
|
Make a page that will issue a token valid for 3 complete downloads of the latest file per day per IP (for example). 3, not 1, because it's possibly hard to tell server-side if the download was successful. Who wants more, should contact you to arrange payment. Or maybe automate the latest file to be hosted as a torrent.
|
My Scrabble game: skrebl.eu My database of electronic parts: elparts.mekweb.eu My DIY electronic kit - mega transistor clock: mtc.mekweb.eu
|
|
|
LoyceV (OP)
Legendary

Activity: 4144
Merit: 22597
Thick-Skinned Gang Leader and Golden Feather 2021
|
 |
September 02, 2026, 12:42:54 PM Last edit: September 02, 2026, 02:44:29 PM by LoyceV |
|
With all this abuse downloading 100 MB/s, my own download (after killing Apache2) does less than 1 MB/s. I don't get it... Make a page that will issue a token This is where you lost me. I have no idea how to do "interactive" websites. Or maybe automate the latest file to be hosted as a torrent. I really don't like this option, as it exposes IP-addresses, but I may have to do it anyway. Most downloads come from servers anyway. The next problem with torrents is that they "stay alive" when they're outdated, but it would be the user's own responsibility to get the latest version.
Thinking about the problem: ideally, I should have a "token" that can be added to an URL, and counts the total bandwidth used by each token. That way, it doesn't matter if the user makes a mistake and downloads the same file twice, or downloads all files I have. But if the token is shared, it'll be maxed out and blocked. The problem is that I have no idea how to implement this. $1 for a token valid for 1 year or 1 TB bandwidth (whichever is reached first) would stop abuse and is cheap enough. I could also give away tokens to anyone who asks for it.
|
¡uʍop ǝpᴉsdn pɐǝɥ ɹnoʎ ɥʇᴉʍ ʎuunɟ ʞool no⅄
|
|
|
|
pbies
|
 |
September 02, 2026, 03:54:37 PM |
|
Torrent is not a bad idea but some short unique name and date in format YYMMDD, like 260902.
|
BTC: bc1qmrexlspd24kevspp42uvjg7sjwm8xcf9w86h5k
|
|
|
|
albert0bsd
|
 |
September 02, 2026, 04:12:53 PM |
|
This is where you lost me. I have no idea how to do "interactive" websites.
I can help you with that. If you guys are going to use torrent I can also share the files in tat way
|
₿ SIGNATURE SPACE AVAILABLE • HERO MEMBER • PM • bc1qjyhcjacmpc9pg8wes82lktyrjcwk4sdvqtm7ky
|
|
|
LoyceV (OP)
Legendary

Activity: 4144
Merit: 22597
Thick-Skinned Gang Leader and Golden Feather 2021
|
 |
September 02, 2026, 05:50:00 PM |
|
Torrent is not a bad idea but some short unique name and date in format YYMMDD, like 260902. If I use torrent, I was thinking of using the same filename for each update. That way, anyone can download the small torrent file as often as they want, and the torrent system does the rest. This is where you lost me. I have no idea how to do "interactive" websites. I can help you with that. Great!
|
¡uʍop ǝpᴉsdn pɐǝɥ ɹnoʎ ɥʇᴉʍ ʎuunɟ ʞool no⅄
|
|
|
|
pbies
|
 |
September 02, 2026, 09:47:20 PM |
|
@LoyceV
Torrent duplication here arrives. Not good. It can fail to work.
Before seed propagates the file there will pass some time. It will take time for people to spread the file for current day.
I advise to use both magnet link and .torrent file on website (I see it is down, just needed all addresses ever used, I will go for old one on my disk, EDIT: I don't have it so I need to download it).
|
BTC: bc1qmrexlspd24kevspp42uvjg7sjwm8xcf9w86h5k
|
|
|
|