List of all Bitcoin addresses ever used - weekly updates work again!

[nullama]

This is great, thanks!

I have a question. Looking at the website, it shows this:

all_Bitcoin_addresses_ever_used_in_order_of_first_appearance.txt.gz   2021-12-08 20:07    23G   
all_Bitcoin_addresses_ever_used_sorted.txt.gz   2021-12-08 15:25    20G

Why is there a 3GB difference in what seems to be just an order difference? Is it because the compression of sorted addresses is able to compress the data more, or am I missing something and these files have different data?

[1696142507]

1696142507

[LoyceV]

Why is there a 3GB difference in what seems to be just an order difference? Is it because the compression of sorted addresses is able to compress the data more

Correct. Ordered data can be compressed more than random data.

Update complete!
See:
http://alladdresses.loyce.club/all_Bitcoin_addresses_ever_used_in_order_of_first_appearance.txt.gz (23 GB)
http://alladdresses.loyce.club/all_Bitcoin_addresses_ever_used_sorted.txt.gz (21 GB)
Updated up until yesterday's data dump.

[LoyceV]

I'm currently experimenting with updates. The VPS I'm using doesn't have enough space, so I use an external ("pay by the hour") host for updates. If all works out, I'll be able to provide monthly updates again with only a few hours of additional server time, and a few minutes of my own time.

I tried another update, this time with the "Premium KVM" (with NVMe) instead of "VDS" (with SSD). Unfortunately this was slower, but the update is complete again.

[LoyceV]

It's been more than a month, I'm running another update. I love 99 MB/s download speeds. Never mind, it dropped.
It should be done in a few hours.

Update: done!
See:
http://alladdresses.loyce.club/all_Bitcoin_addresses_ever_used_in_order_of_first_appearance.txt.gz
http://alladdresses.loyce.club/all_Bitcoin_addresses_ever_used_sorted.txt.gz

[LoyceV]

Bump: I've updated this data yesterday again. This VPS has 3.9 GB disk space left, so at some point it will run out. If that happens, I hope to move it to one that's big enough to handle updating it too (so I can do more frequent updates).

[LoyceV]

I need help, I may be going crazy!

I'm moving this project to a different server (a kindly donated dedicated Xeon server). I've transfered the data, updated the DNS, and updated Apache2's configuration for http://alladdresses.loyce.club/

This is what it shows:

However, as you can see in the console, I've updated the 2 large files: they now have a newer date. I've also added a "test" file and a "test2" directory. Both don't show up noline, but I can navigate into that directory so it exists.
It looks as if I'm viewing a cached version, except for that I'm using a private browser window. I tried Tor too, all with the same result.

What sorcery is this?

[seoincorporation]

I need help, I may be going crazy!

I'm moving this project to a different server (a kindly donated dedicated Xeon server). I've transfered the data, updated the DNS, and updated Apache2's configuration for http://alladdresses.loyce.club/

This is what it shows:

However, as you can see in the console, I've updated the 2 large files: they now have a newer date. I've also added a "test" file and a "test2" directory. Both don't show up noline, but I can navigate into that directory so it exists.
It looks as if I'm viewing a cached version, except for that I'm using a private browser window. I tried Tor too, all with the same result.

What sorcery is this?

Your server must have and index.html file, but your server doesn't have it under the alladdresses subdomain, that's why it opens like a folder.

If we take a look to http://alladdresses.loyce.club/indextest.html the site looks fine. So, just rename that file to index.html and it should work fine.

---------------------

Trying to understand the problem.

If we open the new folder test2 it shows an empty folder, maybe your htaccess file is configured to hide empty folders, not sure. but the folder is there.

[LoyceV]

Your server must have and index.html file, but your server doesn't have it under the alladdresses subdomain, that's why it opens like a folder.

That's intentional: I want to see the files (but some files aren't showing correctly).

If we open the new folder test2 it shows an empty folder, maybe your htaccess file is configured to hide empty folders

There is no .htaccess in use.

I use the same settings for different subdomains, but they don't have this problem.

[seoincorporation]

Ok, then lets make some tests.

Could you add a text file to that folder, lets see i that way it get listed.

And another reason for a folder to not show up is the permissions, but your permissions are fine as we can see in the image, so, we can discard this one.

The default Ubuntu document root is /var/www/html. You can make your own virtual hosts under /var/www.

Since you are working on a virtual host, the way to fix this is with the configuration:

/etc/apache2/apache2.conf

I just make a test with my local server:

Code:

root@root:/var/www/html# ls
index.html  info.php
root@root:/var/www/html# cat index.html > index2.html
root@root:/var/www/html# rm index.html
root@root:/var/www/html# mkdir test2

And it show the empty folder, so, the problem should be your apache2 config.

[LoyceV]

Could you add a text file to that folder, lets see i that way it get listed.

See http://alladdresses.loyce.club/test2/hi.txt
The file is visible on http://alladdresses.loyce.club/test2/
But the directory is not visible on http://alladdresses.loyce.club/

Since you are working on a virtual host, the way to fix this is with the configuration:

/etc/apache2/apache2.conf

I think that can't be the problem, because it works for other virtual hosts. I copied the files:

Code:

cp -a test* ../../blockdata.loyce.club/public_html/

They show up on http://blockdata.loyce.club/ as expected.

I just make a test with my local server:

That's how it should be indeed, hence the "I'm going crazy" It looks as if http://alladdresses.loyce.club/ still runs on my old server, but that can't be (otherwise hi.txt wouldn't be visible).

[seoincorporation]

...
That's how it should be indeed, hence the "I'm going crazy" It looks as if http://alladdresses.loyce.club/ still runs on my old server, but that can't be (otherwise hi.txt wouldn't be visible).

And now we are 2 going crazy, lol. As you mention, this is working in the new server, but the fact that isn't indexing the new folder doesn't have sense.

Did you try restarting the apache service?

Code:

sudo service apache2 restart

[LoyceV]

And now we are 2 going crazy, lol.

Lol.

Did you try restarting the apache service?

I did:

Code:

systemctl status apache2

Code:

sudo service apache2 restart

I tried this one too, just in case. No improvement.

[seoincorporation]

Ok, lets try with this one. It worked in my local server:

Code:

chmod 0755 test2

I hope this one do the magic.

[LoyceV]

Code:

chmod 0755 test2

That only reduced permissions (and didn't work).

[TryNinja]

I'm far from being an expert with Apache, but maybe this is related?

https://cwiki.apache.org/confluence/display/httpd/DirectoryListings

More specificially, "Directory Listings" and "Some files aren't listed".

[seoincorporation]

Ok, looks like i found something mate:

Code:

http://blockdata.loyce.club/

It show the test2 folder with the same modification time.



I have no idea why, but i know this is a nice hint for the solution

You are creating the folder under blockdata and some way it's linked to alladdresses.

-----------------------


Another solution is to build a site for that subdomain. That way you can manipulate all the links without problems.

[LoyceV]

I'm far from being an expert with Apache, but maybe this is related?

https://cwiki.apache.org/confluence/display/httpd/DirectoryListings

More specificially, "Directory Listings" and "Some files aren't listed".

None of the files are forbidden, they're just regular files, like the other files.

It's even weirder that the date of the 2 large files is incorrect. That's the date it had before I updated the file, the file as it shows it doesn't even exist anymore. If you download it, you get the correct updated file.

You are creating the folder under blockdata and some way it's linked to alladdresses.

I copied it there to show it works fine on another subdomain:

I think that can't be the problem, because it works for other virtual hosts. I copied the files:

Code:

cp -a test* ../../blockdata.loyce.club/public_html/

They show up on http://blockdata.loyce.club/ as expected.

I've deleted them from "blockdata" again.

Another solution is to build a site for that subdomain. That way you can manipulate all the links without problems.

FML! I feel sooooooooooooooooooooooo stupid now
I used wget to transfer files from the old server to the new server, and as you can see in my terminal screenshot, that created an index.html file. So instead of getting a fresh listing from Apache, it shows the old one. That explains everything, no scorcery involved, just my own stupidity.

Updates work again!
I've moved this data to a (kindly donated) dedicated server. The large files are updated each Tuesday. Enjoy!

[GR Sasa]

Hello LoyceV,

How/why could this be useful for us bitcoin users?

I mean yes we got a overview about all used addresses, but for me i am not sure how this could be helpful for us.

Thanks

[DaveF]

Hello LoyceV,

How/why could this be useful for us bitcoin users?

I mean yes we got a overview about all used addresses, but for me i am not sure how this could be helpful for us.

Thanks

For some people it's interesting statistical information. You can see how many new addresses were added / used in a particular timeframe.
Depending on what you need it for there are other ways of getting it but he is doing the back end work for you.
For you to use BTC day to day it's not important, but if you are doing blockchain research it's a handy thing to have.

-Dave

[LoyceV]

How/why could this be useful for us bitcoin users?

I'm not sure If you don't know what to do with this, I guess it's not for you. But some people download it, and it's actually consuming quite a bit of bandwidth so there must be something.

You can see how many new addresses were added / used in a particular timeframe.

There's no time stamp information included, so you'll need to get Bitcoin block data (728 GB): inputs, outputs and transactions for the full data.

Depending on what you need it for there are other ways of getting it but he is doing the back end work for you.

What can I say: I like data