Bitcoin Forum

Bitcoin => Development & Technical Discussion => Topic started by: disclosure on December 29, 2014, 10:15:14 PM



Title: Bitnodes crawlers from 54.x.x.x (Amazon EC2 instances)
Post by: disclosure on December 29, 2014, 10:15:14 PM
Does anyone know about the Bitnodes crawlers from these addresses:

54.173.72.127 (ec2-54-173-72-127.compute-1.amazonaws.com)
54.183.64.54 (ec2-54-183-64-54.us-west-1.compute.amazonaws.com)
54.194.231.211 (ec2-54-194-231-211.eu-west-1.compute.amazonaws.com)
54.66.214.167 (ec2-54-66-214-167.ap-southeast-2.compute.amazonaws.com)
54.67.33.14 (ec2-54-67-33-14.us-west-1.compute.amazonaws.com)
54.77.251.214 (ec2-54-77-251-214.eu-west-1.compute.amazonaws.com)
54.94.195.96 (ec2-54-94-195-96.sa-east-1.compute.amazonaws.com)
54.94.200.247 (ec2-54-94-200-247.sa-east-1.compute.amazonaws.com)

They appear to have started around the same time so they may be operated by the same owner. If you are the owner of these crawlers, please consider running only one instance of the crawler to avoid taking up the available connection slots on each node.


Title: Re: Bitnodes crawlers from 54.x.x.x (Amazon EC2 instances)
Post by: theymos on December 30, 2014, 12:33:16 AM
I noticed those as well. They make me wonder whether Bitcoin Core should try to identify and drop useless peers when getting close to the connection limit.


Title: Re: Bitnodes crawlers from 54.x.x.x (Amazon EC2 instances)
Post by: windpath on December 30, 2014, 04:29:10 AM
I noticed those as well. They make me wonder whether Bitcoin Core should try to identify and drop useless peers when getting close to the connection limit.

I have all 8 connected to my testing node, what makes them useless?


Title: Re: Bitnodes crawlers from 54.x.x.x (Amazon EC2 instances)
Post by: disclosure on December 30, 2014, 04:51:04 AM
For the Bitnodes project (https://getaddr.bitnodes.io), I am using only 1 instance of the crawler which connect/disconnect from reachable nodes periodically. This should be sufficient to aggregate network metrics. Running more than 1 instance and taking up connection slots from peers will reduce the number of available open slots for actual peers to connect to. There are only 125 open slots available per node unless of course if you have configured it manually with maxconnections option in bitcoind.conf.


Title: Re: Bitnodes crawlers from 54.x.x.x (Amazon EC2 instances)
Post by: Luke-Jr on December 30, 2014, 10:42:08 AM
I noticed those as well. They make me wonder whether Bitcoin Core should try to identify and drop useless peers when getting close to the connection limit.
I have all 8 connected to my testing node, what makes them useless?
They're worse than useless - maintaining a persistent connection to every node is a serious privacy issue.
Each node can also only have a limited number of connections, each of which consume resources.