Bitcoin Forum
May 02, 2024, 03:41:50 AM *
News: Latest Bitcoin Core release: 27.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1]
  Print  
Author Topic: How can I get data on where Bitcoin nodes are located?  (Read 116 times)
fortysxity (OP)
Newbie
*
Offline Offline

Activity: 4
Merit: 0


View Profile
April 13, 2023, 02:29:11 AM
 #1

How does this site do it?

https://bitnodes.io/

How could I do it?
1714621310
Hero Member
*
Offline Offline

Posts: 1714621310

View Profile Personal Message (Offline)

Ignore
1714621310
Reply with quote  #2

1714621310
Report to moderator
Even if you use Bitcoin through Tor, the way transactions are handled by the network makes anonymity difficult to achieve. Do not expect your transactions to be anonymous unless you really know what you're doing.
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction.
1714621310
Hero Member
*
Offline Offline

Posts: 1714621310

View Profile Personal Message (Offline)

Ignore
1714621310
Reply with quote  #2

1714621310
Report to moderator
DaveF
Legendary
*
Offline Offline

Activity: 3458
Merit: 6258


Crypto Swap Exchange


View Profile WWW
April 13, 2023, 02:45:31 AM
Merited by pooya87 (2), ABCbits (1)
 #2

Their code is here: https://github.com/ayeowch/bitnodes
And the install & configure guide is here: https://github.com/ayeowch/bitnodes/wiki/Provisioning-Bitcoin-Network-Crawler

Not much to it, if you have a fully patched and updated debian machine you can probably be up and running in under an hour.

-Dave

█▀▀▀











█▄▄▄
▀▀▀▀▀▀▀▀▀▀▀
e
▄▄▄▄▄▄▄▄▄▄▄
█████████████
████████████▄███
██▐███████▄█████▀
█████████▄████▀
███▐████▄███▀
████▐██████▀
█████▀█████
███████████▄
████████████▄
██▄█████▀█████▄
▄█████████▀█████▀
███████████▀██▀
████▀█████████
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
c.h.
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▀▀▀█











▄▄▄█
▄██████▄▄▄
█████████████▄▄
███████████████
███████████████
███████████████
███████████████
███░░█████████
███▌▐█████████
█████████████
███████████▀
██████████▀
████████▀
▀██▀▀
pooya87
Legendary
*
Offline Offline

Activity: 3444
Merit: 10521



View Profile
April 13, 2023, 04:18:14 AM
Merited by ABCbits (2)
 #3

bitnodes.io can only detect nodes that are accepting incoming connections not all bitcoin nodes. In order to have a better view of the whole network and see all nodes (or at least a lot more than just those nodes accepting incoming connection) you'd have to run strong servers with multiple nodes accepting incoming connections and receive connection from a large part of the network.

And the way to detect the first node types is by using their public announcements (addr message they send out) even though the nodes you connect to and send a getaddr message are not going to send you all the announcements which means your "crawler" has to connect to a lot of nodes and for a long time to be able to construct as large a database as possible.

Their location is a matter of categorizing IP addresses and mapping them to geography.

.
.BLACKJACK ♠ FUN.
█████████
██████████████
████████████
█████████████████
████████████████▄▄
░█████████████▀░▀▀
██████████████████
░██████████████
████████████████
░██████████████
████████████
███████████████░██
██████████
CRYPTO CASINO &
SPORTS BETTING
▄▄███████▄▄
▄███████████████▄
███████████████████
█████████████████████
███████████████████████
█████████████████████████
█████████████████████████
█████████████████████████
███████████████████████
█████████████████████
███████████████████
▀███████████████▀
█████████
.
PawGo
Legendary
*
Offline Offline

Activity: 952
Merit: 1367


View Profile
April 13, 2023, 05:37:30 PM
Merited by pooya87 (2)
 #4

Their location is a matter of categorizing IP addresses and mapping them to geography.

Which of course cannot be treated as a definitive solution. There are companies which sell updated information about IP ranges and their locations, so if you want to rely on that it will not be for free. I do not think there are a lot of public (not-tor) nodes which use VPN to hide their real public IP, but it is possible too.
DaveF
Legendary
*
Offline Offline

Activity: 3458
Merit: 6258


Crypto Swap Exchange


View Profile WWW
April 13, 2023, 08:09:04 PM
 #5

Their location is a matter of categorizing IP addresses and mapping them to geography.

Which of course cannot be treated as a definitive solution. There are companies which sell updated information about IP ranges and their locations, so if you want to rely on that it will not be for free. I do not think there are a lot of public (not-tor) nodes which use VPN to hide their real public IP, but it is possible too.

Due to the IPv4 exhaustion and the sale / transfer of IP space even between regions a lot of those services are best guesses.

Yes, the IP I have from my cable company is setup properly and the IP geolocation is accurate, BUT I have a friend using a small rural ISP who has an IP address that is still, as in YEARS later, linked back to an out of business ISP and their address & geolocation THAT IS NOT EVEN IN THE US. No matter what he does, a lot of places still think he is in Canada.

-Dave

█▀▀▀











█▄▄▄
▀▀▀▀▀▀▀▀▀▀▀
e
▄▄▄▄▄▄▄▄▄▄▄
█████████████
████████████▄███
██▐███████▄█████▀
█████████▄████▀
███▐████▄███▀
████▐██████▀
█████▀█████
███████████▄
████████████▄
██▄█████▀█████▄
▄█████████▀█████▀
███████████▀██▀
████▀█████████
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
c.h.
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▀▀▀█











▄▄▄█
▄██████▄▄▄
█████████████▄▄
███████████████
███████████████
███████████████
███████████████
███░░█████████
███▌▐█████████
█████████████
███████████▀
██████████▀
████████▀
▀██▀▀
RickDeckard
Legendary
*
Offline Offline

Activity: 1008
Merit: 3006



View Profile
April 13, 2023, 09:48:13 PM
Merited by pooya87 (2), ABCbits (1)
 #6

bitnodes.io can only detect nodes that are accepting incoming connections not all bitcoin nodes. In order to have a better view of the whole network and see all nodes (or at least a lot more than just those nodes accepting incoming connection) you'd have to run strong servers with multiple nodes accepting incoming connections and receive connection from a large part of the network.

And the way to detect the first node types is by using their public announcements (addr message they send out) even though the nodes you connect to and send a getaddr message are not going to send you all the announcements which means your "crawler" has to connect to a lot of nodes and for a long time to be able to construct as large a database as possible.

Their location is a matter of categorizing IP addresses and mapping them to geography.
The work being done on bitnodes.io is great since other companies/services use it as a data source for their website. Take, for instance, the case of bitrawr[1] who uses the data from bitnodes to showcase how Bitcoin is spread around the world. Right at the end of the page sits also an important information that is worth to be taken into account when looking at the ~17,700 active nodes:
Quote
Bitnodes uses Bitcoin protocol version 70001 (i.e. >= /Satoshi:0.8.x/), so nodes running an older protocol version will be skipped.

As a closing remark, I once kept these[2][3] links shared by Luke Dashjr[4] that supposedly were scraping/listening to the nodes and recording each entry. I don't know if it is still being updated but considering who did scrapper, for sure it may be seen as reliable. I do like this[3] section of his website where we can see the amount of nodes that is running each Bitcoin protocol. I wonder if there is some way to verify these numbers?

[1]https://www.bitrawr.com/terminal/bitcoin-node-map
[2]https://luke.dashjr.org/programs/bitcoin/files/charts/software.html
[3]https://luke.dashjr.org/programs/bitcoin/files/charts/services.html
[4]https://nitter.net/LukeDashjr

█▀▀▀











█▄▄▄
▀▀▀▀▀▀▀▀▀▀▀
e
▄▄▄▄▄▄▄▄▄▄▄
█████████████
████████████▄███
██▐███████▄█████▀
█████████▄████▀
███▐████▄███▀
████▐██████▀
█████▀█████
███████████▄
████████████▄
██▄█████▀█████▄
▄█████████▀█████▀
███████████▀██▀
████▀█████████
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
c.h.
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▀▀▀█











▄▄▄█
▄██████▄▄▄
█████████████▄▄
███████████████
███████████████
███████████████
███████████████
███░░█████████
███▌▐█████████
█████████████
███████████▀
██████████▀
████████▀
▀██▀▀
ABCbits
Legendary
*
Offline Offline

Activity: 2856
Merit: 7436


Crypto Swap Exchange


View Profile
April 14, 2023, 11:08:15 AM
Merited by pooya87 (2), RickDeckard (2)
 #7

--snip--
Right at the end of the page sits also an important information that is worth to be taken into account when looking at the ~17,700 active nodes:
Quote
Bitnodes uses Bitcoin protocol version 70001 (i.e. >= /Satoshi:0.8.x/), so nodes running an older protocol version will be skipped.

Satoshi:0.8.x (which represent Bitcoin-Qt 0.8.x) released 10 years ago[1]. And Bitcoin Qt version lower than 0.8 can't even perform initial block download anymore[2]. So i wouldn't worry too much about number of undetected node (which accept incoming connection).

As a closing remark, I once kept these[2][3] links shared by Luke Dashjr[4] that supposedly were scraping/listening to the nodes and recording each entry. I don't know if it is still being updated but considering who did scrapper, for sure it may be seen as reliable. I do like this[3] section of his website where we can see the amount of nodes that is running each Bitcoin protocol. I wonder if there is some way to verify these numbers?

It seems to be updated since current number is different from past snapshot on internet archive[3]. As for verifying the number, i guess it's impossible unless you run Bitcoin DNS seed node where your IP node/domain is added to list of DNS seed on Bitcoin Core (and optionally other full node software).



[1] https://bitcoin.org/en/release/v0.8.0
[2] https://blog.lopp.net/running-bitcoin-core-v0-7-and-earlier/
[3] https://web.archive.org/web/20230326174034/https://luke.dashjr.org/programs/bitcoin/files/charts/software.html

█▀▀▀











█▄▄▄
▀▀▀▀▀▀▀▀▀▀▀
e
▄▄▄▄▄▄▄▄▄▄▄
█████████████
████████████▄███
██▐███████▄█████▀
█████████▄████▀
███▐████▄███▀
████▐██████▀
█████▀█████
███████████▄
████████████▄
██▄█████▀█████▄
▄█████████▀█████▀
███████████▀██▀
████▀█████████
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
c.h.
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▀▀▀█











▄▄▄█
▄██████▄▄▄
█████████████▄▄
███████████████
███████████████
███████████████
███████████████
███░░█████████
███▌▐█████████
█████████████
███████████▀
██████████▀
████████▀
▀██▀▀
pooya87
Legendary
*
Offline Offline

Activity: 3444
Merit: 10521



View Profile
April 14, 2023, 11:30:49 AM
 #8

The work being done on bitnodes.io
As far as statistics go, theirs is a decent work. But some of the nodes they list specifically in the first page are spy nodes not real full nodes that are maliciously and intentionally placed there.

Quote
I don't know if it is still being updated
Considering that the charts showing number of nodes is updated on daily basis, I would assume the links you shared are also up to date.
https://luke.dashjr.org/programs/bitcoin/files/charts/historical-dygraph.html

.
.BLACKJACK ♠ FUN.
█████████
██████████████
████████████
█████████████████
████████████████▄▄
░█████████████▀░▀▀
██████████████████
░██████████████
████████████████
░██████████████
████████████
███████████████░██
██████████
CRYPTO CASINO &
SPORTS BETTING
▄▄███████▄▄
▄███████████████▄
███████████████████
█████████████████████
███████████████████████
█████████████████████████
█████████████████████████
█████████████████████████
███████████████████████
█████████████████████
███████████████████
▀███████████████▀
█████████
.
fortysxity (OP)
Newbie
*
Offline Offline

Activity: 4
Merit: 0


View Profile
April 18, 2023, 09:32:09 PM
 #9

Their code is here: https://github.com/ayeowch/bitnodes
And the install & configure guide is here: https://github.com/ayeowch/bitnodes/wiki/Provisioning-Bitcoin-Network-Crawler

Not much to it, if you have a fully patched and updated debian machine you can probably be up and running in under an hour.

-Dave

Thank you!
Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!