How is it possible to measure the amount of nodes that are just "listening" ?

[Brandsen]

This pie chart keeps on popping up everywhere:
http://luke.dashjr.org/programs/bitcoin/files/charts/software.html

It shows that the actual number of bitcoin core nodes is more that 67 000...

A very different number is shown at:
https://bitnodes.21.co/

My understanding is that the chart by Luke is also showing nodes that do not accept incoming connections...
In other words: Nodes that are only listening and do not forward transactions and blocks.

But how is it possible to measure the number of such nodes?
Can someone please explain..?

[achow101]

Those listening nodes still need to connect to the network. They usually, on first start, will connect to a DNS Seeder and ask it for some nodes to connect to. Generally these nodes are ones that have high uptimes and high bandwidth. The seeder will also record some information about the node that asked it for data in case it is a listening node so that it can send nodes to connect to that node as well. So there are then two ways to get information about non-listening nodes; run a DNS seeder, or operate a high uptime, high bandwidth node that gets lots of connections.

The second method doesn't really work that well since Bitcoin Core puts a maximum of 125 connections so it can't actually get thousands of nodes connecting in order to get accurate data. What Luke-Jr does is he operates a DNS seeder and builds his information off of there.

The DNS seeders are hard coded into the software. There are currently 6 DNS seeders, one of which is Luke-Jr's.

[nibor]

Or every node sends an Address Message over the network. And I think even if you are behind a NAT firewall it will send that with the real IP address of you router (i.e. NOT the 192.168.1.10 type addresss).

Every node keeps a database of these in the datadir. So anyone can just count the address in here.

[Luke-Jr]

My understanding is that the chart by Luke is also showing nodes that do not accept incoming connections...
In other words: Nodes that are only listening and do not forward transactions and blocks.

You have that backward. Listening nodes are that ones that do accept connections.
Also, whether a node is listening or not is entirely unrelated to whether it forwards transactions or blocks.

Those listening nodes still need to connect to the network. They usually, on first start, will connect to a DNS Seeder and ask it for some nodes to connect to. Generally these nodes are ones that have high uptimes and high bandwidth. The seeder will also record some information about the node that asked it for data in case it is a listening node so that it can send nodes to connect to that node as well. So there are then two ways to get information about non-listening nodes; run a DNS seeder, or operate a high uptime, high bandwidth node that gets lots of connections.

DNS seeding uses DNS because the server typically never sees the actual IP querying it. Running a DNS seed does not really get any useful information.
The data used for the DNS seeder (and also part of the data for my stats) is collected by crawling the Bitcoin network directly. Anyone can do this.

[sp00lin9]

Here's a pretty neat tool in case you haven't come across it.

https://coin.dance/nodes

[Luke-Jr]

Here's a pretty neat tool in case you haven't come across it.

https://coin.dance/nodes

That's missing all the non-listening nodes.

[TechPriest]

Listening nodes are that ones that do accept connections.

Doest it means that listening nodes can transmit transaction to other nodes? Or they just build "connection bridge" between full nodes?
Also (as i know) listening nodes can differ depend their type. I mean that you could compile node which will not download blockchain but can transmit transactions to other nodes. Or vice versa.

[2112]

A very different number is shown at:
https://bitnodes.21.co/

This site is buggy and undercounts the nodes. I still occasionally use it, but the numbers there are not even statistically reliable, they are skewed downwards.

Part of it is because of simple bugs (e.g. incorrect pruning of inactive nodes) or routing problems on their end.

Part of it is because of some rather strange ways it attempts to deal with Sybil attacks and cannot reliably distinguish them for legitimate address reuse (e.g. CG-NAT or certain forms of IPv6 deployment like DS-Lite).

[RustyShackleford1950]

Don't mean to revive an old topic but this isn't answered anywhere easy where I can find it.

Also, whether a node is listening or not is entirely unrelated to whether it forwards transactions or blocks.

So a non-listening node will forward blocks? Even if the user hasn't opened the port?

[achow101]

So a non-listening node will forward blocks? Even if the user hasn't opened the port?

Yes. Listening and non-listening nodes do exactly the same thing. They still send and receive blocks and transactions from their peers and will relay blocks and transactions to their peers. The only difference is that listening nodes allow other nodes to make the connection to it. The name inboud only means that someone else initiated the connection. The name outbound only means that you initiated the connection. Otherwise both connection types are exactly the same and the nodes do exactly the same things with both connection types.

[RustyShackleford1950]

But a non-listening node will not serve SPV nodes will it?

[Luke-Jr]

But a non-listening node will not serve SPV nodes will it?

Why is this relevant? SPV nodes are harmful to the network, so serving them isn't helping the network.

But not even listening nodes necessarily serve SPV nodes. That's an additional feature called "BLOOM"..

[RustyShackleford1950]

But a non-listening node will not serve SPV nodes will it?

Why is this relevant? SPV nodes are harmful to the network, so serving them isn't helping the network.

Because SPV nodes are the only realistic way of having users be able to self verify on a mobile device.

[Luke-Jr]

But a non-listening node will not serve SPV nodes will it?

Why is this relevant? SPV nodes are harmful to the network, so serving them isn't helping the network.

Because SPV nodes are the only realistic way of having users be able to self verify on a mobile device.

SPV doesn't self-verify. It blindly trusts.

[RustyShackleford1950]

SPV doesn't self-verify. It blindly trusts.

SPV does verify. I don't know what definition you're using when you say it doesn't self-verify, but it does verify transactions itself and doesn't rely on a trusted third party to verify.

In terms of trust, the trust is minimal in an SPV node.

Would you prefer the majority of users use web wallets where they must be centrally trusted by a single authority, or mobile SPV wallets where at least the level of trust required is minimal and users can verify?

[achow101]

SPV does verify. I don't know what definition you're using when you say it doesn't self-verify, but it does verify transactions itself and doesn't rely on a trusted third party to verify.

That is untrue. A SPV wallet cannot fully verify a transaction. It does not have the UTXO set, so it cannot fully verify it. Even if it does know the previous transactions, it does not know the transactions that those spend from and so on. If it did, it would basically be a full node, which they are not. Because an SPV node does not have the UTXO set and has not verified the chain of transactions that a given transaction is part of, it must trust the node that gave it the transaction and trust that that node gave it the right transaction and that the transaction was valid to it. Without the previous transactions, it cannot verify that the input script is correct. Without the previous transactions, it cannot verify that the amounts in the outputs are valid.

[RustyShackleford1950]

SPV does verify. I don't know what definition you're using when you say it doesn't self-verify, but it does verify transactions itself and doesn't rely on a trusted third party to verify.

That is untrue. A SPV wallet cannot fully verify a transaction. It does not have the UTXO set, so it cannot fully verify it. Even if it does know the previous transactions, it does not know the transactions that those spend from and so on. If it did, it would basically be a full node, which they are not. Because an SPV node does not have the UTXO set and has not verified the chain of transactions that a given transaction is part of, it must trust the node that gave it the transaction and trust that that node gave it the right transaction and that the transaction was valid to it. Without the previous transactions, it cannot verify that the input script is correct. Without the previous transactions, it cannot verify that the amounts in the outputs are valid.

Come on we're playing semantics. Yes, obviously a SPV wallet doesn't fully verify like a full node does, but it does simply verify transactions which is the point. It has limited trust to a third party and since the information it receives from a third party is cross verified with others and it is deep in the blockchain, there's a fair assumption of truth in it.

The point is that every person isn't going to run a full node, and you want all users to have a decentralised self verifying experience. SPV nodes on mobile are the best answer to this.

[achow101]

Come on we're playing semantics. Yes, obviously a SPV wallet doesn't fully verify like a full node does, but it does simply verify transactions which is the point. It has limited trust to a third party and since the information it receives from a third party is cross verified with others and

The point is that SPV wallets can't and don't "simply verify transactions". There is no verification that can be done without having the transactions that a transaction spends from.

it is deep in the blockchain, there's a fair assumption of truth in it.

That's also not true. A transaction is not necessarily deep in the blockchain. SPV wallets allow you to receive unconfirmed transactions and sometimes spend from them. You can also spend from transactions with one confirmation; those are definitely not "deep in the blockchain"

The point is that every person isn't going to run a full node, and you want all users to have a decentralised self verifying experience. SPV nodes on mobile are the best answer to this.

Yes, I agree that not everyone is going to run a full node, but current SPV wallets are not a decentralized self verifying experience at all.

[jnano]

Listening and non-listening nodes do exactly the same thing. They still send and receive blocks and transactions from their peers and will relay blocks and transactions to their peers.

Do non-listening nodes change their 8 peers more often than listening nodes?

[achow101]

Listening and non-listening nodes do exactly the same thing. They still send and receive blocks and transactions from their peers and will relay blocks and transactions to their peers.

Do non-listening nodes change their 8 peers more often than listening nodes?

Connections, both inbound and outbound, are rarely closed unless one peer goes offline. So listening and non-listening nodes will change their peers at about the same frequency. Outbound connections in general might change more frequently as we have stricter criteria for who we connect to and when to disconnect from them than we do with inbound peers.