Bitcoin Forum
December 09, 2016, 05:52:43 PM *
News: To be able to use the next phase of the beta forum software, please ensure that your email address is correct/functional.
 
   Home   Help Search Donate Login Register  
Pages: [1]
  Print  
Author Topic: [RFC] Separating leaf nodes and supernodes  (Read 1977 times)
xf2_org
Member
**
Offline Offline

Activity: 70


View Profile
June 05, 2011, 02:52:56 PM
 #1


Currently we advertise all nodes on the network.  As we are seeing from the huge influx of new users, many nodes do not have port 8333 properly forwarded (tip: use -upnp), leading to a passing around a lot of network addresses that ultimately are useless, because they do not accept incoming TCP connections.

Taking a page from gnutella, I propose separating nodes into leaf nodes and full nodes (or super-nodes).  Here is how it could work:

  • All nodes start out as leaf nodes, unless -noleaf specified.
  • All nodes may be upgraded to supernodes, unless -nosuper specified.
  • Leaf nodes do not advertise their own address
  • If X minutes have passed (60?), and no incoming connections have been seen ask a couple peers to connect-back to you.
  • If X minutes have passed, periodically check for incoming connections.  If present, upgrade to supernode.

We may easily add other conditions to being a supernode, besides uptime and working incoming connections.

Since this behavior is coded on a per-client basis, users have total freedom to accept or reject these rules.  Older clients continue to work as before.

I haven't fully thought this out... this is just an early sketch.  Comments welcome.
1481305963
Hero Member
*
Offline Offline

Posts: 1481305963

View Profile Personal Message (Offline)

Ignore
1481305963
Reply with quote  #2

1481305963
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1481305963
Hero Member
*
Offline Offline

Posts: 1481305963

View Profile Personal Message (Offline)

Ignore
1481305963
Reply with quote  #2

1481305963
Report to moderator
Matt Corallo
Hero Member
*****
expert
Offline Offline

Activity: 751


View Profile
June 05, 2011, 04:31:35 PM
 #2

Leaf nodes do not advertise their own address
This is my only concern.  How can you pull that off?  There was some discussion about adding a services flag to indicate that new nodes should attempt to connect to you, but currently, services are set in a nodes database by ORing previously known services with new ones.  Meaning you cant downgrade to a leafnode after you have advertised yourself as a supernode.  We might be able to change that on a flag-by-flag basis, but doing that has the potential for sybil-style attacks where you re-advertise all known nodes with the supernode flag off.

Bitcoin Ubuntu PPA maintainer - donate to me personally: 1JBMattRztKDF2KRS3vhjJXA7h47NEsn2c
http://bitcoinrelaynetwork.org maintainer
PGP ID: 07DF 3E57 A548 CCFB 7530  7091 89BB B866 3E2E65CE
xf2_org
Member
**
Offline Offline

Activity: 70


View Profile
June 05, 2011, 04:41:09 PM
 #3

Leaf nodes do not advertise their own address
This is my only concern.  How can you pull that off?  There was some discussion about adding a services flag to indicate that new nodes should attempt to connect to you, but currently, services are set in a nodes database by ORing previously known services with new ones.  Meaning you cant downgrade to a leafnode after you have advertised yourself as a supernode.  We might be able to change that on a flag-by-flag basis, but doing that has the potential for sybil-style attacks where you re-advertise all known nodes with the supernode flag off.

This wasn't related to flags.  "advertise" simply refers to a leaf node sending out their own address onto the network.  You would need an additional connect-back message.

Matt Corallo
Hero Member
*****
expert
Offline Offline

Activity: 751


View Profile
June 05, 2011, 04:59:01 PM
 #4

This wasn't related to flags.  "advertise" simply refers to a leaf node sending out their own address onto the network.  You would need an additional connect-back message.
Well then I suppose my question is how do you not advertise?  Old clients will always forward your IP because they have "seen" you, and unless there is a way to tell 3rd party new clients not to attempt to connect to you, they will just see you as another generic node.  You can not advertise on IRC, but I'd like to see that go away anyway, plus that is only a way to bootstrap, not make connections after having previously connected.

Bitcoin Ubuntu PPA maintainer - donate to me personally: 1JBMattRztKDF2KRS3vhjJXA7h47NEsn2c
http://bitcoinrelaynetwork.org maintainer
PGP ID: 07DF 3E57 A548 CCFB 7530  7091 89BB B866 3E2E65CE
nanotube
Hero Member
*****
Offline Offline

Activity: 485


View Profile WWW
June 05, 2011, 08:05:38 PM
 #5

This wasn't related to flags.  "advertise" simply refers to a leaf node sending out their own address onto the network.  You would need an additional connect-back message.
Well then I suppose my question is how do you not advertise?  Old clients will always forward your IP because they have "seen" you, and unless there is a way to tell 3rd party new clients not to attempt to connect to you, they will just see you as another generic node.  You can not advertise on IRC, but I'd like to see that go away anyway, plus that is only a way to bootstrap, not make connections after having previously connected.

as i understand it, this is not about leaf nodes being "anonymous", but simply about reducing the signal-to-noise ratio in the node addresses being passed around for bootstrapping purposes. "over time" as nodes get upgraded to the new code, fewer and fewer of them will pass around 'useless' addresses to others.

Join #bitcoin-market on freenode for real-time market updates.
Join #bitcoin-otc - an over-the-counter trading market. http://bitcoin-otc.com
OTC web of trust: http://bitcoin-otc.com/trust.php
My trust rating: http://bitcoin-otc.com/viewratingdetail.php?nick=nanotube
Matt Corallo
Hero Member
*****
expert
Offline Offline

Activity: 751


View Profile
June 05, 2011, 08:35:56 PM
 #6

as i understand it, this is not about leaf nodes being "anonymous", but simply about reducing the signal-to-noise ratio in the node addresses being passed around for bootstrapping purposes.
Yep, when did anyone say otherwise?
"over time" as nodes get upgraded to the new code, fewer and fewer of them will pass around 'useless' addresses to others.
The hard part is in the implementation.  How do you do it in a mostly-backward compatible way where the 10 old nodes left dont cause issues for the rest?

Bitcoin Ubuntu PPA maintainer - donate to me personally: 1JBMattRztKDF2KRS3vhjJXA7h47NEsn2c
http://bitcoinrelaynetwork.org maintainer
PGP ID: 07DF 3E57 A548 CCFB 7530  7091 89BB B866 3E2E65CE
theymos
Administrator
Legendary
*
expert
Offline Offline

Activity: 2506


View Profile
June 05, 2011, 10:30:12 PM
 #7

A leaf/supernode distinction is definitely needed.

If this only changes whether a peer will broadcast itself, it seems that an attacker could break these rules and flood bad addresses. Maybe addresses need to be checked before being relayed (possibly with a random chance of checking).

1NXYoJ5xU91Jp83XfVMHwwTUyZFK64BoAD
xf2_org
Member
**
Offline Offline

Activity: 70


View Profile
June 06, 2011, 12:00:23 AM
 #8

If this only changes whether a peer will broadcast itself, it seems that an attacker could break these rules and flood bad addresses.

You mean...  do exactly what they can do now?

theymos
Administrator
Legendary
*
expert
Offline Offline

Activity: 2506


View Profile
June 06, 2011, 12:18:00 AM
 #9

You mean...  do exactly what they can do now?

Yes. There's no point in making these changes if an attacker can easily undo their effects, though.

1NXYoJ5xU91Jp83XfVMHwwTUyZFK64BoAD
xf2_org
Member
**
Offline Offline

Activity: 70


View Profile
June 06, 2011, 12:41:17 AM
 #10

You mean...  do exactly what they can do now?

Yes. There's no point in making these changes if an attacker can easily undo their effects, though.

The changes are voluntary anyway, and old clients won't follow the rules.  So the situation can only get better, it won't get worse.

Mike Hearn
Legendary
*
expert
Offline Offline

Activity: 1526


View Profile
June 06, 2011, 09:09:40 AM
 #11

Yes please. This would be a great improvement.

IIRC the code does not broadcast an address unless a peer actually provides an addr message. I think it won't "synthesize" addr messages by itself. So this change means adding a new "dialback" message. When receiving that message, a node would initiate an outbound connection to the IP addresss of the connection that sent the message. If it doesn't succeed after 10 seconds it could send a "dialbackerror" message. Or it could just be implicit.

Nodes that successfully get inbound connections would then change their IRC nick to the format earlier clients recognize as valid.
bluecmd
Newbie
*
Offline Offline

Activity: 14


View Profile
June 06, 2011, 10:48:55 AM
 #12

You mean...  do exactly what they can do now?

Yes. There's no point in making these changes if an attacker can easily undo their effects, though.

That is a quite negative view I think. This could easily be resolved by requiring that the list of peers being sent from a supernode is up-to-date (as in, the nodes are still online). When the client begins to connect to the list of peers received and hits a certain number of failed connections, block all updates from that supernode.
theymos
Administrator
Legendary
*
expert
Offline Offline

Activity: 2506


View Profile
June 06, 2011, 11:17:39 AM
 #13

That is a quite negative view I think. This could easily be resolved by requiring that the list of peers being sent from a supernode is up-to-date (as in, the nodes are still online). When the client begins to connect to the list of peers received and hits a certain number of failed connections, block all updates from that supernode.

I suggested something like that earlier in the thread...

I'm not saying the idea is bad. I'm just saying that not broadcasting your own IP will by itself not be very helpful in the long-term, and thinking only of that might lead down the wrong path. For example, if we know that supernodes will need to check peers, a new network command for "am I reachable?" will not be necessary. You can just see whether you appear in the addr lists of your peers (if their version is OK).

1NXYoJ5xU91Jp83XfVMHwwTUyZFK64BoAD
bluecmd
Newbie
*
Offline Offline

Activity: 14


View Profile
June 06, 2011, 11:54:18 AM
 #14

I suggested something like that earlier in the thread...
Ah, I didn't see that - my bad.

I'm not saying the idea is bad. I'm just saying that not broadcasting your own IP will by itself not be very helpful in the long-term, and thinking only of that might lead down the wrong path. For example, if we know that supernodes will need to check peers, a new network command for "am I reachable?" will not be necessary. You can just see whether you appear in the addr lists of your peers (if their version is OK).

I am not sure I understand you correctly, but here goes.

You can just see whether you appear in the addr lists of your peers (if their version is OK).
Being reachable is a bi-directional thing, being a peer is not. If I am in your peer list nothing is known about if you connected to me or I to you - and thus the reachable status is unknown for both peers.

What I think would be good is something like this:
  • Client connects to IRC and selects one or many supernodes. (Implement some sort of tagging)
  • Client connects to supernode per usual. If the node wants to become a supernode (Would be the default I think) it would mark this in the services field)
  • After connection to the supernode, the supernode will then try to connect back to the node and verify the version information again (it should be the same obviously).
  • If this is successful the node is marked as a super node and may join the IRC channel with supernode identity

This way introduces 2 services: "wants-to-be supernode" and "is supernode". If an incoming connection from a supernode succeeds, then the node is allowed to set the flag "is supernode". There is probably a much better way, but this is what I am currently is thinking.

Writing this I started to wonder: Isn't it enough to get one non-RFC1918 incoming connection to be promoted to supernode? That would mean that we do not need flags in the services field but only the tagging in the bootstrap channel.

EDIT: Reading the thread start again, this exactly what was proposed. Just ignore my random rants, original idea has my support!
gmaxwell
Moderator
Legendary
*
qt
Online Online

Activity: 2030



View Profile
June 06, 2011, 08:25:12 PM
 #15

We may easily add other conditions to being a supernode, besides uptime and working incoming connections.

I would strongly recommend adding a blockheight check as well.  Right now you can _easily_ form cliques of empty-headed nodes that talk only among themselves and have never seen the rest of the network.

I'm not actually concerned with being able to provide the chain itself, but rather having the chain is evidence that you've been connected to the rest of the network before and therefore know some addresses that may get someone connected to it again.  I'd personally use the highest checkpoint coded into the software, but really— it should probably be fine to use the earliest and thus allow nodes to begin seeding a bit faster.

This measure also proxies for another thing: systems which are too overloaded or poorly connected to successfully download the blockchain should probably not get a bunch of leaf nodes connecting to them.

Also, jrmithdobbs has a pull request someplace with some reasonable RFC1918/private IP detection logic.

Separately, on IRC nodes which decide that they are successfully listening ought to join multiple random IRC channels.   Two would be sufficient for making partitioning unlikely, but three might be better.   (Not that this general proposal should depend on IRC— this should just be one part of its effect)
jrmithdobbs
Jr. Member
*
Offline Offline

Activity: 59


View Profile
June 06, 2011, 08:51:18 PM
 #16

Also, jrmithdobbs has a pull request someplace with some reasonable RFC1918/private IP detection logic.

Already merged.

1B8TSDzXdyTRX5eF77gWQoXujBaDtKFE6H
Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!