[back on new server] List of all Bitcoin addresses with a balance

[LoyceV]

1. Cloudflare considered to behave just like MITM attack among some people with serious privacy or security concern. But it could be "easy" way to reduce successful abuse attempt.

It could also break any automated scripts that download it. And I really dislike Cloudflare.

2. Limit the maximum bandwidth per connection or IP. People who only need your data one time or on few occasion more likely to tolerate it than people who regularly download latest file.

I doubt that's going to stop people. Automated scripts don't mind the slower download, and most of the bandwidth goes to different IPs now.

I've restarted Apache2. The moment it's online, downloads started.

These are 10 second averages of outgoing bandwidth (in Mbyte/s):
80,183,775
74,460,381
77,756,118
75,537,366
78,059,662
82,381,260
80,515,415
75,926,504
74,763,770
83,997,099
84,626,537
82,675,131
80,837,847
79,588,349
80,269,984
81,104,141
80,695,840
72,825,527
77,831,898
83,120,619
74,202,390
76,550,071
80,254,527
76,089,717
76,394,077

There's no abuse anymore (like someone downloading the same file every 3 minutes). It's all different IP-addresses. It just looks like demand for this data went up A LOT recently.

I've added a warning on http://addresses.loyce.club/ about outages. Next time I take it offline (which won't take long), I'll just remove the "LATEST" symlink, forcing users to manually check things.

[LoyceV]

Torrent is not a bad idea but some short unique name and date in format YYMMDD, like 260902.

If I use torrent, I was thinking of using the same filename for each update. That way, anyone can download the small torrent file as often as they want, and the torrent system does the rest.

I've created Torrent files, let's try this for a while:
addresses.loyce.club/blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz.torrent (1689 MB)
addresses.loyce.club/Bitcoin_addresses_LATEST.txt.gz.torrent (1393 MB)

I've only turned the LATEST versions into .torrent files. This will for now break all automated daily (or more frequent) downloads, forcing users to check the site to continue.
My server's upload (per thread) is still terribly slow. For now, staying within my bandwidth limits without taking it offline is my main priority.
The LATEST .torrent files are updated daily. I'm hoping someone will post a script to automatically download the LATEST version through Torrent. Don't keep older Torrents seeding, let them die out the next day.

Feel free to suggest more Trackers to add to the .torrent.
Feel free to post anything that can be improved, I don't have much experience creating Torrents.
Please try the torrents! I need some downloaders

Earlier, while testing, I could connect to the .torrent, but now it doesn't connect anymore. Suggestions?

Update (a day later)
Everything seems to work fine: the new .torrent is created automatically, and starts downloading the moment I click it.
I've only had a few .torrent downloads yesterday, but I expect more users to realize the change in the coming days.

How to download the Torrent
On Linux, I use transmission-daemon with transmission-remote (part of transmission-cli) to create and upload Torrents. You can use the same to download them. Don't forget to remove the old Torrents before downloading the latest version:

Code:

transmission-remote -t all --remove

Bandwidth
My bandwidth consumption dropped about 90% to a few MByte/s. I've also wiped my iptables DROPlist. If "3 minute guy" wants to come back to get a 404 every 3 minutes, that's okay
Note that the only change I made, was for LATEST, not for all other files. Those are still available for incidental downloads. If someone clicks the link and downloads it, that's totally fine. If you want an update a month later, that's also fine. But if my bandwidth consumption skyrockets again, I'll start logging IP-addresses and automatically DROPping IP-addresses for anyone who should have used the Torrent (and by the way I drop them, it will also DROP your access to my website and Torrent daemon). So just don't abuse it

I'll update the OP when I decided how to keep this service long-term.

[bitmover]

There's no abuse anymore (like someone downloading the same file every 3 minutes). It's all different IP-addresses. It just looks like demand for this data went up A LOT recently.

I've added a warning on http://addresses.loyce.club/ about outages. Next time I take it offline (which won't take long), I'll just remove the "LATEST" symlink, forcing users to manually check things.

I just noticed you have a competitor:
https://bitref.com/bitcoin-addresses-with-balance/

But he charges $199 USD to have access to the information (all addresses with balance).
Maybe charging something wouldn't be a bad idea.

[LoyceV]

But he charges $199 USD to have access to the information (all addresses with balance).

If someone pays that much for something they can create or download for free, that's on them

Maybe charging something wouldn't be a bad idea.

I'd do that, if I'd know how. It was discussed on the previous page:

A thousand downloads per day wouldn't be any problem at all if they all pay $1 per year.

[bitmover]

But he charges $199 USD to have access to the information (all addresses with balance).

If someone pays that much for something they can create or download for free, that's on them

Maybe charging something wouldn't be a bad idea.

I'd do that, if I'd know how. It was discussed on the previous page:

A thousand downloads per day wouldn't be any problem at all if they all pay $1 per year.

It will demand some work from your side. AI certainly helps.

Easy way:
Put your email and give them the link to download once they send you the btc.

Longer way:
You would probably have to create an account/login system.
Genetate a single address for each account( maybe use btcpayserver or a bitcoin library)

[LoyceV]

It will demand some work from your side. AI certainly helps.

I don't really like using things I don't understand.

Easy way:
Put your email and give them the link to download once they send you the btc.

Then I'll have to keep track of who shares their link with whom, and it's terrible for privacy.

Longer way:
You would probably have to create an account/login system.
Genetate a single address for each account( maybe use btcpayserver or a bitcoin library)

It should still work for automated downloads, as that's how most people use this.

But there's a bigger "problem": I never created this with the intention of earning money from it. I kinda like how this data has become so established that even people on Github link to addresses.loyce.club. Charging money for it would scare off the large majority of users, even if it's just a small amount.
So for now, I'll just try the Torrent solution for a while.

[bitmover]

It will demand some work from your side. AI certainly helps.

I don't really like using things I don't understand.

I agree with you on this.

However, you don't have to do use AI this way (asking it do to things you don't understand).

You can use AI to guide you, and AI can teach you how to do it. And do little by little with AI support.

AI is a basically a dumb developer with infinite patience, you can ask AI any question, to the most basic level of understanding.

I am a much better developer since I started using AI, I learned a lot from it and learned much faster than I would without it.

Easy way:
Put your email and give them the link to download once they send you the btc.

Then I'll have to keep track of who shares their link with whom, and it's terrible for privacy.

You already have the IP of anyone who access your data. But you don't know which addresses they are searching for, so I don't see any additional privacy problem.

I also don't think people would be trying to cheat your system...

But there's a bigger "problem": I never created this with the intention of earning money from it. I kinda like how this data has become so established that even people on Github link to addresses.loyce.club. Charging money for it would scare off the large majority of users, even if it's just a small amount.
So for now, I'll just try the Torrent solution for a while.

That is indeed a good point. I also got many backlinks from chatgpt and github to bitcoindata.science, because my tools work and they are free to use.

But, there might be some way to earn from your tool (if you want to). Maybe free up to 5 downloads per month, and then ask for money after 5 downloads, or some other model. Or you can just let everything how it is, I just saw someone charging $199 for the same thing and remembered about your tool.

[LoyceV]

But, there might be some way to earn from your tool (if you want to). Maybe free up to 5 downloads per month, and then ask for money after 5 downloads, or some other model.

I enjoy projects much more without dealing with accounting systems

In theory, the Torrent solution would be perfect: if 1000 people per day download it, they'll share the majority of bandwidth consumption amongst themselves and my problem is gone But so far, I only get a few downloads per day. Let's see if this is a deal breaker before trying more drastic measures

However, you don't have to do use AI this way (asking it do to things you don't understand).

I've never used and always ignored all chatbots I see

[pbies]

What about some CDN = Content Delivery Network?
For example Amazon has such.

They should host some data with unlimited bw.

It may be more expensive but should work flawlessly...

[LoyceV]

It may be more expensive

You answered your own question. And they don't accept crypto.

[LoyceV]

I was afraid it would come to this:

Note that the only change I made, was for LATEST, not for all other files. Those are still available for incidental downloads. If someone clicks the link and downloads it, that's totally fine. If you want an update a month later, that's also fine. But if my bandwidth consumption skyrockets again, I'll start logging IP-addresses and automatically DROPping IP-addresses for anyone who should have used the Torrent (and by the way I drop them, it will also DROP your access to my website and Torrent daemon). So just don't abuse it

Instead of just playing nice and implementing the Torrent download, more and more people are downloading "September_12_2026.txt.gz" directly.

I hate that it has to come down to this. I'll collect data for a while, then implement automatic IP DROPs for the users who do this every day. I can already expect the next problem to be people using random IPs.

[LoyceV]

The LATEST .torrent files are updated daily. I'm hoping someone will post a script to automatically download the LATEST version through Torrent. Don't keep older Torrents seeding, let them die out the next day.

How's this (quick and dirty):

Code:

#!/bin/bash

# Requirements:
# sudo apt install transmission-daemon transmission-cli curl
# systemctl disable --now transmission-daemon   # Running the daemon as a user makes it easier to connect to

mkdir -p ~/addresses.loyce.club/incomplete; cd ~/addresses.loyce.club/

if [ -z "`ps aux | grep 'transmission-daemon' | grep -v grep | grep -v 'color=auto'`" ]
then  transmission-daemon --incomplete-dir ~/addresses.loyce.club/incomplete --download-dir ~/addresses.loyce.club                                                                                                                                    # Start the daemon if it's not running yet
fi

while test 1
do
      # Bitcoin_addresses_LATEST.txt.gz.torrent                                                                                                                                                                                                       # Remove this block if you only want the other file
            checksumaddresses=$(cat Bitcoin_addresses_LATEST.txt.gz.torrent 2> /dev/null | md5sum)
            curl -s -o Bitcoin_addresses_LATEST.txt.gz.torrent -z Bitcoin_addresses_LATEST.txt.gz.torrent http://addresses.loyce.club/Bitcoin_addresses_LATEST.txt.gz.torrent                                                                         # Download when newer
            if [ "$checksumaddresses" != "$(cat Bitcoin_addresses_LATEST.txt.gz.torrent | md5sum)" ]
            then  transmission-remote -t $(transmission-remote --list | grep Bitcoin_addresses_ | cut -c1-8 | cut -d'*' -f1 | sed s/\ //g) --remove 2>/dev/null 1>/dev/null                                                                           # Remove old torrent
                  transmission-remote -a ~/addresses.loyce.club/Bitcoin_addresses_LATEST.txt.gz.torrent                                                                                                                                               # Start new torrent
            fi

      # blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz.torrent                                                                                                                                                                                # Remove this block if you only want the other file
            checksumbalances=$(cat blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz.torrent 2> /dev/null | md5sum)
            curl -s -o blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz.torrent -z blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz.torrent http://addresses.loyce.club/blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz.torrent    # Download when newer
            if [ "$checksumbalances" !=  "$(cat blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz.torrent | md5sum)" ]
            then  transmission-remote -t $(transmission-remote --list | grep blockchair_bitcoin_addresses_and_balance_ | cut -c1-8 | cut -d'*' -f1 | sed s/\ //g) --remove 2>/dev/null 1>/dev/null                                                    # Remove old torrent
                  transmission-remote -a ~/addresses.loyce.club/blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz.torrent                                                                                                                        # Start new torrent
            fi

      sleep 18m
done

# Monitoring
# while test 1; do clear; transmission-remote -t all  --info-peers; echo; transmission-remote --list; sleep 10; done

I've tested the above script on a second server. It downloads both LATEST versions daily.
Start the script once and keep it running in the background. If there's a newer torrent available, it stops seeding the older one and start downloading the latest version. When it's done, it's moved to ~/addresses.loyce.club/.
I need a few testers before publishing it on my site.

My current VPS will be replaced by a new one. I'm hoping that will improve the download speed, but it means it'll be down for a while.

[Vod]

I tried to add both torrent links to seed but it got stuck on "Retrieving metadata...".  Have you moved or restricted it?

Edit - site is moving.

[LoyceV]

Edit - site is moving.

My site is back! Feel free to start torrenting
Today's data update will take a few hours to complete.

[Vod]

Edit - site is moving.

My site is back! Feel free to start torrenting
Today's data update will take a few hours to complete.

Running now.  I have a 3GB upload so I'll seed for you as needed.

[pbies]

@LoyceV

1. When to start the script you've provided? (crontab, daily? which hour is the best?)
2. Will it redownload the same torrent? (stupid question but need to ask; I need postprocessing)

I am getting when manually run:

Unexpected response: <h1>401: Unauthorized</h1>
Unexpected response: <h1>401: Unauthorized</h1>
Unexpected response: <h1>401: Unauthorized</h1>
Unexpected response: <h1>401: Unauthorized</h1>


Fixed script:

Code:

#!/bin/bash

# Keeps the two addresses.loyce.club torrents up to date.
#
# Requirements:
#   apt install transmission-daemon transmission-cli curl
#
# One-time setup (Ubuntu 24.04 / Transmission 4.1.x):
#   systemctl disable --now transmission-daemon          # packaged daemon holds 9091 and requires auth
#   cat > /etc/apparmor.d/local/transmission-daemon <<'EOF'
#   /root/addresses.loyce.club/ rw,
#   /root/addresses.loyce.club/** rwk,
#   EOF
#   apparmor_parser -r /etc/apparmor.d/transmission
#
# The AppArmor profile confines transmission-daemon to
# /var/lib/transmission-daemon/.config/transmission-daemon/ by default; without the
# local override above the daemon cannot read or write $BASE.

set -u

BASE="${BASE:-$HOME/addresses.loyce.club}"
URL="http://addresses.loyce.club"
RPC_PORT="${RPC_PORT:-9092}"
PEER_PORT="${PEER_PORT:-51420}"
CONF="$BASE/config"
LOG="$BASE/transmission.log"
INTERVAL="${INTERVAL:-18m}"
TR=(transmission-remote "127.0.0.1:$RPC_PORT")
FILES=(
	"Bitcoin_addresses_LATEST.txt.gz.torrent"
	"blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz.torrent"
)

log() { echo "$(date '+%F %T') $*"; }

mkdir -p "$BASE/incomplete" "$CONF" || exit 1
cd "$BASE" || exit 1

write_settings() {
	[ -f "$CONF/settings.json" ] && return
	cat > "$CONF/settings.json" <<EOF
{
	"rpc-enabled": true,
	"rpc-port": $RPC_PORT,
	"rpc-bind-address": "127.0.0.1",
	"rpc-authentication-required": false,
	"rpc-whitelist-enabled": true,
	"rpc-whitelist": "127.0.0.1,::1",
	"download-dir": "$BASE",
	"incomplete-dir": "$BASE/incomplete",
	"incomplete-dir-enabled": true,
	"peer-port": $PEER_PORT,
	"peer-port-random-on-start": false,
	"dht-enabled": true,
	"pex-enabled": true,
	"lpd-enabled": true,
	"utp-enabled": true,
	"encryption": 1,
	"download-queue-enabled": false,
	"queue-stalled-enabled": false
}
EOF
}

rpc_up() { "${TR[@]}" --session-info >/dev/null 2>&1; }

start_daemon() {
	write_settings
	transmission-daemon --config-dir "$CONF" --logfile "$LOG"
	for _ in $(seq 30); do
		rpc_up && return 0
		sleep 1
	done
	return 1
}

if ! rpc_up; then
	if ! start_daemon; then
		log "transmission RPC not reachable on 127.0.0.1:$RPC_PORT" >&2
		tail -n 20 "$LOG" >&2 2>/dev/null
		dmesg -T 2>/dev/null | grep -i 'apparmor.*DENIED' | tail -5 >&2
		exit 1
	fi
	log "daemon started (rpc $RPC_PORT, peer $PEER_PORT)"
fi

# IDs of torrents whose name contains $1
get_ids() {
	"${TR[@]}" --list 2>/dev/null | awk -v n="$1" 'NR > 1 && $1 != "Sum:" && index($0, n) { gsub(/\*/, "", $1); print $1 }'
}

update() {
	local f="$1"
	local prefix="${f%%LATEST*}"
	local tmp="$BASE/$f.tmp"
	local cond=()
	local id

	[ -f "$f" ] && cond=(-z "$f")

	rm -f "$tmp"
	if ! curl -fsSL -R "${cond[@]}" -o "$tmp" "$URL/$f"; then
		log "download failed: $f" >&2
		rm -f "$tmp"
		return
	fi

	# 304 Not Modified, or empty body
	if [ ! -s "$tmp" ]; then
		rm -f "$tmp"
		return
	fi

	# Unchanged content
	if [ -f "$f" ] && cmp -s "$tmp" "$f"; then
		rm -f "$tmp"
		return
	fi

	mv -f "$tmp" "$f"

	# Drop the old torrent; data stays on disk and the new torrent re-verifies it,
	# fetching only the changed pieces.
	for id in $(get_ids "$prefix"); do
		"${TR[@]}" -t "$id" --remove >/dev/null 2>&1
	done

	if "${TR[@]}" -a "$BASE/$f" >/dev/null 2>&1; then
		log "added: $f"
	else
		log "add failed: $f" >&2
	fi
}

while true; do
	for f in "${FILES[@]}"; do
		update "$f"
	done
	sleep "$INTERVAL"
done

# Monitoring:
#   watch -n10 'transmission-remote 127.0.0.1:9092 --list'
#   transmission-remote 127.0.0.1:9092 -t all --info

Defects corrected from the original:

ps aux | grep daemon check matched unrelated processes and the script's own pipeline; replaced with pgrep -u <uid> -f.
No wait for RPC startup — the first transmission-remote call after launching the daemon raced and failed silently.
curl -z against a nonexistent file made -z misbehave on first run; now conditional.
curl -s swallowed HTTP errors; a 404/500 body overwrote the good torrent file. -f plus temp-file staging prevents corruption.
md5sum on a missing file compared garbage; replaced with cmp on staged temp file.
transmission-remote -t $(...) ran with an empty -t argument when no torrent matched, which targets all torrents in some versions; now a loop over parsed IDs, skipping the header and Sum: row.
cut -c1-8 truncated names and broke for IDs of varying width; replaced with awk field extraction.
Duplicated blocks collapsed to a loop over FILES.
Missing set-level failure handling on cd.

[LoyceV]

1. When to start the script you've provided? (crontab, daily? which hour is the best?)

Just start it once:

Start the script once and keep it running in the background. If there's a newer torrent available, it stops seeding the older one and start downloading the latest version. When it's done, it's moved to ~/addresses.loyce.club/.

The time doesn't matter, it gets the latest version.

2. Will it redownload the same torrent? (stupid question but need to ask; I need postprocessing)

No.

I am getting when manually run:

Unexpected response: <h1>401: Unauthorized</h1>
Unexpected response: <h1>401: Unauthorized</h1>
Unexpected response: <h1>401: Unauthorized</h1>
Unexpected response: <h1>401: Unauthorized</h1>

It requires authentication when transmission-daemon runs as a system service. Just kill it and run it as a user.

Fixed script:

I prefer a short script, that makes it easier to read. This is a good addition though:

Code:

systemctl disable --now transmission-daemon

grep daemon check matched unrelated processes and the script's own pipeline

I can't account for all local processes, but grep -v grep makes sure it can't grep itself.

curl -z against a nonexistent file made -z misbehave on first run; now conditional.

Here too, I prefer a shorter script. It works when the file doesn't exist yet, it tells you about it, and you'll never see that message again. I'm okay with that

md5sum on a missing file compared garbage; replaced with cmp on staged temp file.

It worked Your script got 4 times longer, which makes it harder to read.

cut -c1-8 truncated names and broke for IDs of varying width; replaced with awk field extraction.

This one was annoying indeed. I couldn't just remove the torrent from transmission-daemon based on it's name.

[bitrecovery]

how do I remove extra characters?

You mean remove the balance? I just use cut. I got you 3 new files with only addresses.

Thank you LoyceV: how your address database powers free seed recovery and a suggestion to offload your server bandwidth

Hi LoyceV,

First and foremost, I want to express immense gratitude on behalf of our team and the broader Bitcoin recovery community for your dedication in compiling and maintaining the daily database of Bitcoin addresses with a positive balance on addresses.loyce.club.

Your data dumps serve as the backbone of our open-source desktop software, bitResurrector (https://ai-seedfinder.com/bitresurrector). The software is 100% free and runs completely offline on the user's local machine. When people lose 1 to 2 words from their BIP-39 seed phrase or face scrambled unnumbered lists, the primary hurdle is usually that they do not know their original receiving address. Standard tools like btcrecover simply fail without a known address.

We take your daily address dump, index the active UTXO addresses with balances greater than zero (currently over 58000000 funded addresses), and compile it into a compact 256MB in-memory Bloom filter. This allows regular Bitcoiners to test hundreds of thousands of candidate phrases per second on their own hardware and instantly identify their funded wallet in RAM without needing to remember their public address. Your continuous archive has directly enabled dozens of people to recover their locked coins completely free of charge.

Regarding the server load and bandwidth issues you mentioned in this thread:

I noticed you mentioned facing recurrent server traffic spikes, bandwidth quotas, and strain on your VPS from automated scrapers and high-volume downloads of multi-gigabyte files. Having handled similar data distribution challenges, I would like to offer a practical setup that could reduce your hosting load to near zero while keeping your dumps publicly accessible:

• 1. GitHub Releases as an automated global CDN: GitHub allows release assets up to 2 GB per file (your compressed .tsv.gz archive fits well within this limit at roughly 600 to 800 MB). GitHub provides virtually unlimited download bandwidth backed by Fastly and Azure edge networks at no charge. You can keep your scripts and maintenance private in a private GitHub repository, and use a simple GitHub Action or a 3-line Bash script with the official GitHub CLI (

Code:

gh release create

) to automatically upload your daily build as a public release asset. Users and scrapers download directly from GitHub's global edge at 10 Gbps speeds, completely bypassing your VPS bandwidth.[/li]

[li]2. Cloudflare R2 object storage ($0 egress fees): If you prefer not to use GitHub, you can set up a Cloudflare R2 bucket. Unlike AWS S3 or standard hosting providers, Cloudflare R2 charges exactly $0.00 for outbound bandwidth (zero egress fees), regardless of how many terabytes users download. You only pay standard storage ($0.015 per GB per month, which is literally pennies for your dump size). A nightly

Code:

rclone sync

or

Code:

aws s3 cp

command from your crontab pushes the updated archive to R2 in a few seconds, and all public download traffic hits Cloudflare CDN instead of your origin server.[/li]

Either of these approaches would permanently solve the bandwidth exhaustion problem, shield your server IP address from direct scraper traffic, prevent downtime, and ensure the Bitcoin community always has reliable, high-speed access to your invaluable work.

Thank you once again for your tireless contribution to the Bitcoin ecosystem over all these years. If you need any assistance setting up the automated GitHub or R2 push scripts, I would be more than happy to share our deployment configs.

Best regards, bitResurrector Team

[Vod]

Your data dumps serve as the backbone of our open-source desktop software, bitResurrector (https://ai-seedfinder.com/bitresurrector). The software is 100% free and runs completely offline on the user's local machine.

Why is a subscription needed if it's 100% free?