[RFC] Separating leaf nodes and supernodes

[xf2_org]

Currently we advertise all nodes on the network.  As we are seeing from the huge influx of new users, many nodes do not have port 8333 properly forwarded (tip: use -upnp), leading to a passing around a lot of network addresses that ultimately are useless, because they do not accept incoming TCP connections.

Taking a page from gnutella, I propose separating nodes into leaf nodes and full nodes (or super-nodes).  Here is how it could work:

• All nodes start out as leaf nodes, unless -noleaf specified.
• All nodes may be upgraded to supernodes, unless -nosuper specified.
• Leaf nodes do not advertise their own address
• If X minutes have passed (60?), and no incoming connections have been seen ask a couple peers to connect-back to you.
• If X minutes have passed, periodically check for incoming connections.  If present, upgrade to supernode.

We may easily add other conditions to being a supernode, besides uptime and working incoming connections.

Since this behavior is coded on a per-client basis, users have total freedom to accept or reject these rules.  Older clients continue to work as before.

I haven't fully thought this out... this is just an early sketch.  Comments welcome.

[1713966629]

1713966629

[1713966629]

1713966629

[Matt Corallo]

Leaf nodes do not advertise their own address

This is my only concern.  How can you pull that off?  There was some discussion about adding a services flag to indicate that new nodes should attempt to connect to you, but currently, services are set in a nodes database by ORing previously known services with new ones.  Meaning you cant downgrade to a leafnode after you have advertised yourself as a supernode.  We might be able to change that on a flag-by-flag basis, but doing that has the potential for sybil-style attacks where you re-advertise all known nodes with the supernode flag off.

[xf2_org]

Leaf nodes do not advertise their own address

This is my only concern.  How can you pull that off?  There was some discussion about adding a services flag to indicate that new nodes should attempt to connect to you, but currently, services are set in a nodes database by ORing previously known services with new ones.  Meaning you cant downgrade to a leafnode after you have advertised yourself as a supernode.  We might be able to change that on a flag-by-flag basis, but doing that has the potential for sybil-style attacks where you re-advertise all known nodes with the supernode flag off.

This wasn't related to flags.  "advertise" simply refers to a leaf node sending out their own address onto the network.  You would need an additional connect-back message.

[Matt Corallo]

This wasn't related to flags.  "advertise" simply refers to a leaf node sending out their own address onto the network.  You would need an additional connect-back message.

Well then I suppose my question is how do you not advertise?  Old clients will always forward your IP because they have "seen" you, and unless there is a way to tell 3rd party new clients not to attempt to connect to you, they will just see you as another generic node.  You can not advertise on IRC, but I'd like to see that go away anyway, plus that is only a way to bootstrap, not make connections after having previously connected.

[nanotube]

This wasn't related to flags.  "advertise" simply refers to a leaf node sending out their own address onto the network.  You would need an additional connect-back message.

Well then I suppose my question is how do you not advertise?  Old clients will always forward your IP because they have "seen" you, and unless there is a way to tell 3rd party new clients not to attempt to connect to you, they will just see you as another generic node.  You can not advertise on IRC, but I'd like to see that go away anyway, plus that is only a way to bootstrap, not make connections after having previously connected.

as i understand it, this is not about leaf nodes being "anonymous", but simply about reducing the signal-to-noise ratio in the node addresses being passed around for bootstrapping purposes. "over time" as nodes get upgraded to the new code, fewer and fewer of them will pass around 'useless' addresses to others.

[Matt Corallo]

as i understand it, this is not about leaf nodes being "anonymous", but simply about reducing the signal-to-noise ratio in the node addresses being passed around for bootstrapping purposes.

Yep, when did anyone say otherwise?

"over time" as nodes get upgraded to the new code, fewer and fewer of them will pass around 'useless' addresses to others.

The hard part is in the implementation.  How do you do it in a mostly-backward compatible way where the 10 old nodes left dont cause issues for the rest?

[theymos]

A leaf/supernode distinction is definitely needed.

If this only changes whether a peer will broadcast itself, it seems that an attacker could break these rules and flood bad addresses. Maybe addresses need to be checked before being relayed (possibly with a random chance of checking).

[xf2_org]

If this only changes whether a peer will broadcast itself, it seems that an attacker could break these rules and flood bad addresses.

You mean...  do exactly what they can do now?

[theymos]

You mean...  do exactly what they can do now?

Yes. There's no point in making these changes if an attacker can easily undo their effects, though.

[xf2_org]

You mean...  do exactly what they can do now?

Yes. There's no point in making these changes if an attacker can easily undo their effects, though.

The changes are voluntary anyway, and old clients won't follow the rules.  So the situation can only get better, it won't get worse.

[Mike Hearn]

Yes please. This would be a great improvement.

IIRC the code does not broadcast an address unless a peer actually provides an addr message. I think it won't "synthesize" addr messages by itself. So this change means adding a new "dialback" message. When receiving that message, a node would initiate an outbound connection to the IP addresss of the connection that sent the message. If it doesn't succeed after 10 seconds it could send a "dialbackerror" message. Or it could just be implicit.

Nodes that successfully get inbound connections would then change their IRC nick to the format earlier clients recognize as valid.

[bluecmd]

You mean...  do exactly what they can do now?

Yes. There's no point in making these changes if an attacker can easily undo their effects, though.

That is a quite negative view I think. This could easily be resolved by requiring that the list of peers being sent from a supernode is up-to-date (as in, the nodes are still online). When the client begins to connect to the list of peers received and hits a certain number of failed connections, block all updates from that supernode.

[theymos]

That is a quite negative view I think. This could easily be resolved by requiring that the list of peers being sent from a supernode is up-to-date (as in, the nodes are still online). When the client begins to connect to the list of peers received and hits a certain number of failed connections, block all updates from that supernode.

I suggested something like that earlier in the thread...

I'm not saying the idea is bad. I'm just saying that not broadcasting your own IP will by itself not be very helpful in the long-term, and thinking only of that might lead down the wrong path. For example, if we know that supernodes will need to check peers, a new network command for "am I reachable?" will not be necessary. You can just see whether you appear in the addr lists of your peers (if their version is OK).

[bluecmd]

I suggested something like that earlier in the thread...

Ah, I didn't see that - my bad.

I'm not saying the idea is bad. I'm just saying that not broadcasting your own IP will by itself not be very helpful in the long-term, and thinking only of that might lead down the wrong path. For example, if we know that supernodes will need to check peers, a new network command for "am I reachable?" will not be necessary. You can just see whether you appear in the addr lists of your peers (if their version is OK).

I am not sure I understand you correctly, but here goes.

You can just see whether you appear in the addr lists of your peers (if their version is OK).

Being reachable is a bi-directional thing, being a peer is not. If I am in your peer list nothing is known about if you connected to me or I to you - and thus the reachable status is unknown for both peers.

What I think would be good is something like this:

• Client connects to IRC and selects one or many supernodes. (Implement some sort of tagging)
• Client connects to supernode per usual. If the node wants to become a supernode (Would be the default I think) it would mark this in the services field)
• After connection to the supernode, the supernode will then try to connect back to the node and verify the version information again (it should be the same obviously).
• If this is successful the node is marked as a super node and may join the IRC channel with supernode identity

This way introduces 2 services: "wants-to-be supernode" and "is supernode". If an incoming connection from a supernode succeeds, then the node is allowed to set the flag "is supernode". There is probably a much better way, but this is what I am currently is thinking.

Writing this I started to wonder: Isn't it enough to get one non-RFC1918 incoming connection to be promoted to supernode? That would mean that we do not need flags in the services field but only the tagging in the bootstrap channel.

EDIT: Reading the thread start again, this exactly what was proposed. Just ignore my random rants, original idea has my support!

[gmaxwell]

We may easily add other conditions to being a supernode, besides uptime and working incoming connections.

I would strongly recommend adding a blockheight check as well.  Right now you can _easily_ form cliques of empty-headed nodes that talk only among themselves and have never seen the rest of the network.

I'm not actually concerned with being able to provide the chain itself, but rather having the chain is evidence that you've been connected to the rest of the network before and therefore know some addresses that may get someone connected to it again.  I'd personally use the highest checkpoint coded into the software, but really— it should probably be fine to use the earliest and thus allow nodes to begin seeding a bit faster.

This measure also proxies for another thing: systems which are too overloaded or poorly connected to successfully download the blockchain should probably not get a bunch of leaf nodes connecting to them.

Also, jrmithdobbs has a pull request someplace with some reasonable RFC1918/private IP detection logic.

Separately, on IRC nodes which decide that they are successfully listening ought to join multiple random IRC channels.   Two would be sufficient for making partitioning unlikely, but three might be better.   (Not that this general proposal should depend on IRC— this should just be one part of its effect)

[jrmithdobbs]

Also, jrmithdobbs has a pull request someplace with some reasonable RFC1918/private IP detection logic.

Already merged.