Accessing the network through a "lite" node

[etotheipi]

I've been looking at the BTC Protocol Specification to understand what I would need to do differently (network-wise) if I were to put a "lite" node on the network.  So far, all I see is a "services" bitfield, but it only has one bit defined:  whether the node will relay headers-only, or all block information.  Wouldn't this be the place where you would specify that you are a lite-node?  Is there another way to do this? 

Here's my understanding of how a lite-node will operate:

• Will not verify any information - assumes that headers and tx's it receives (especially those in the blockchain) are valid
• Will not relay/broadcast any information to other nodes because it cannot verify it (is it okay to relay headers if they have the right number of leading 0 bits?)
• Will connect to peers in the way a full node does, but with information identifying its limited capabilities
• Can request information from the nodes, such as headers and block data, but will not forward any of it
• Will receive broadcasted tx and headers without requesting them.
• Can construct and broadcast new tx packets to connected nodes (with tx-fee info exchange)
• Will provide the user payment verification

I guess my question is more generally about what capabilities/behaviors are appropriate for a lite-node and is it inappropriate to be leeching from the network like this without providing the support of a full node.  I can see how peers that are connected to you have less "influence" since your lite-node is filling the slot of a "connected peer" but won't be broadcasting important information, such a new blocks and tx data.  However, we all know that in the far future, all average users will be using lite-nodes, since the blockchain will be too big to store on an average HDD computer... so this isn't unreasonable at all...

[1713988565]

1713988565

[1713988565]

1713988565

[1713988565]

1713988565

[MoonShadow]

I don't understand what you are trying to do here.  There are a couple of levels of "lightweight" nodes that have been detailed on this forum that are compatible with the Bitcoin network.  Which are you shooting for?  What is the use case that you are aiming to satisfy?

[etotheipi]

I don't understand what you are trying to do here.  There are a couple of levels of "lightweight" nodes that have been detailed on this forum that are compatible with the Bitcoin network.  Which are you shooting for?  What is the use case that you are aiming to satisfy?

Perhaps I should search bitcointalk.org through google, I bet that would get me better results.  Regardless,  this probably should've been in the alternative clients subforum, so sorry about that...

I should've stuck to the specific question which is: how should the node identify itself on the network?  It needs to let other nodes know it's not a full node.  But I don't see anywhere on the specification page how this is actually achieved.  I see a 64-bit "services" bitfield in the version message, but I see nothing about how to identify that my node has the capabilities listed there.  Perhaps that specification is just not up to date.

The use-case here is like any other lite-client:   reduced resource utilization, and design focused on the human interface without having to reimplement the entire BTC protocol and risk forking the blockchain when I do it wrong.  Multi-sig transactions will be useless if no one can use them.  Wallet security could greatly benefit from a variety of features that aren't necessarily appropriate for the reference client, and will have questionable success without testing them on the network with real users. 

[MoonShadow]

I don't understand what you are trying to do here.  There are a couple of levels of "lightweight" nodes that have been detailed on this forum that are compatible with the Bitcoin network.  Which are you shooting for?  What is the use case that you are aiming to satisfy?

Perhaps I should search bitcointalk.org through google, I bet that would get me better results.  Regardless,  this probably should've been in the alternative clients subforum, so sorry about that...

I should've stuck to the specific question which is: how should the node identify itself on the network?  It needs to let other nodes know it's not a full node.  But I don't see anywhere on the specification page how this is actually achieved.  I see a 64-bit "services" bitfield in the version message, but I see nothing about how to identify that my node has the capabilities listed there.  Perhaps that specification is just not up to date.

I'm not a programmer, so I can't help on the specifics, but there generally isn't any reason to identify the node.  My understanding is that nodes operate on an announce then pull-request kind of model.  Your client simply would never announce transactions or blocks being available via it, and would only request data that concerns itself.

The use-case here is like any other lite-client:   reduced resource utilization, and design focused on the human interface without having to reimplement the entire BTC protocol and risk forking the blockchain when I do it wrong.  Multi-sig transactions will be useless if no one can use them.  Wallet security could greatly benefit from a variety of features that aren't necessarily appropriate for the reference client, and will have questionable success without testing them on the network with real users. 

What I'm asking is are you trying to make a light client with a full blockchain that doesn't participate, a headers only blockchain, or no blockchain that keeps only blocks that concern itself?  The three models operate entirely differently.

[etotheipi]

I don't understand what you are trying to do here.  There are a couple of levels of "lightweight" nodes that have been detailed on this forum that are compatible with the Bitcoin network.  Which are you shooting for?  What is the use case that you are aiming to satisfy?

Perhaps I should search bitcointalk.org through google, I bet that would get me better results.  Regardless,  this probably should've been in the alternative clients subforum, so sorry about that...

I should've stuck to the specific question which is: how should the node identify itself on the network?  It needs to let other nodes know it's not a full node.  But I don't see anywhere on the specification page how this is actually achieved.  I see a 64-bit "services" bitfield in the version message, but I see nothing about how to identify that my node has the capabilities listed there.  Perhaps that specification is just not up to date.

I'm not a programmer, so I can't help on the specifics, but there generally isn't any reason to identify the node.  My understanding is that nodes operate on an announce then pull-request kind of model.  Your client simply would never announce transactions or blocks being available via it, and would only request data that concerns itself.

This does not seem optimal.  The problem is that if a lot of people are using lightweight nodes that don't identify themselves as such, there's a risk for major network bottlenecks.  You try to propagate information on the network, but none of your peers will actually help you because you accidentally connected to too many lite nodes.  I see a necessity for nodes to be able to query the capabilities of their peers, to make sure that they are connected to a sufficient number of full-nodes that will help propagate important information.

The use-case here is like any other lite-client:   reduced resource utilization, and design focused on the human interface without having to reimplement the entire BTC protocol and risk forking the blockchain when I do it wrong.  Multi-sig transactions will be useless if no one can use them.  Wallet security could greatly benefit from a variety of features that aren't necessarily appropriate for the reference client, and will have questionable success without testing them on the network with real users. 

What I'm asking is are you trying to make a light client with a full blockchain that doesn't participate, a headers only blockchain, or no blockchain that keeps only blocks that concern itself?  The three models operate entirely differently.

At the moment, I'm working on the first option (full blockchain but no participation), with the intention to fork a branch of the second type (headers only, keeping only block data that is relevant to my wallet).  I currently envision having client software on your computer with the full blockchain and a phone app that only keeps headers.  In the future when the blockchain is big, both pieces would be headers-only, but perhaps with more information stored on the computer.

[MoonShadow]

I don't understand what you are trying to do here.  There are a couple of levels of "lightweight" nodes that have been detailed on this forum that are compatible with the Bitcoin network.  Which are you shooting for?  What is the use case that you are aiming to satisfy?

Perhaps I should search bitcointalk.org through google, I bet that would get me better results.  Regardless,  this probably should've been in the alternative clients subforum, so sorry about that...

I should've stuck to the specific question which is: how should the node identify itself on the network?  It needs to let other nodes know it's not a full node.  But I don't see anywhere on the specification page how this is actually achieved.  I see a 64-bit "services" bitfield in the version message, but I see nothing about how to identify that my node has the capabilities listed there.  Perhaps that specification is just not up to date.

I'm not a programmer, so I can't help on the specifics, but there generally isn't any reason to identify the node.  My understanding is that nodes operate on an announce then pull-request kind of model.  Your client simply would never announce transactions or blocks being available via it, and would only request data that concerns itself.

This does not seem optimal.  The problem is that if a lot of people are using lightweight nodes that don't identify themselves as such, there's a risk for major network bottlenecks.  You try to propagate information on the network, but none of your peers will actually help you because you accidentally connected to too many lite nodes.  I see a necessity for nodes to be able to query the capabilities of their peers, to make sure that they are connected to a sufficient number of full-nodes that will help propagate important information.

If none of your peers will request the data you announced, then they are all light clients and your node needs to connect to more or different nodes.  It is not optimal, but it is anonymous.  There are attacks that can be used more effectively if the malicious node knows the limitations of his peers or the peers of a target node.  There are good reasons for sharing this data, and good reasons for keeping it mum, so the option rests with the node and is not forced.  I'm not even sure if announcement of node facilities is even emplemented.

The use-case here is like any other lite-client:   reduced resource utilization, and design focused on the human interface without having to reimplement the entire BTC protocol and risk forking the blockchain when I do it wrong.  Multi-sig transactions will be useless if no one can use them.  Wallet security could greatly benefit from a variety of features that aren't necessarily appropriate for the reference client, and will have questionable success without testing them on the network with real users. 

What I'm asking is are you trying to make a light client with a full blockchain that doesn't participate, a headers only blockchain, or no blockchain that keeps only blocks that concern itself?  The three models operate entirely differently.

At the moment, I'm working on the first option (full blockchain but no participation), with the intention to fork a branch of the second type (headers only, keeping only block data that is relevant to my wallet).  I currently envision having client software on your computer with the full blockchain and a phone app that only keeps headers.  In the future when the blockchain is big, both pieces would be headers-only, but perhaps with more information stored on the computer.

A light client & trusted node pairing, then.  This has been discussed in much detail.  Use your search-fu and you will find much more than I can offer.  The bitcoinJ clients are of the semi-independent headers only type.  I suggest you start with understanding how they manage things.

[etotheipi]

As I linked in my first post, there is a "services" bitfield in the specification described as "bitfield of features to be enabled for this connection".  I assumed this was the place to identify yourself and your capabilities.  I'm not convinced that the network can really operate with a diverse field of node types without understanding his own peers.  If none of your peers are willing to identify their capabilities, and you aren't receiving any block/tx updates, how do you know that you aren't being attacked, or just connected to bunch of lightweight nodes? 

The fact that such a field exists in the specification page leads me to believe there was intent for nodes to advertise this information.  And it makes sense that the network (and users) would want nodes to specify this information.

And my search-fu sucks.  I don't know why I've always failed at this.  I've tried "lite node" and "lightweight".  I found a page labeled "RFC for lightweight nodes" but it didn't contain this information.  I'm not so concerned with how to manage it, though.  I was more interested with the network protocol around "services" identification. 

[theymos]

Not having NODE_NETWORK set indicates that you're a lightweight client. When Bitcoin doesn't set NODE_NETWORK, it means that Bitcoin is downloading full blocks but then throwing them away except for the Merkle trees. So Bitcoin lightweight-client nodes currently do relaying and as much verification as possible, but they don't have old blocks on hand.

There's not yet any way to show that you're running in an even more lightweight mode, though you should definitely not set NODE_NETWORK.

[etotheipi]

Not having NODE_NETWORK set indicates that you're a lightweight client. Exactly what this means is not yet well-defined.

By "not set" I assume you mean setting NODE_NETWORK to zero.  It sounded like the reference client doesn't really take this into account yet, but that's not a reason to ignore it...

You should definitely be downloading full blocks if you have it set, though.

Does this mean that I should be relaying full blocks if this value is set (i.e. other nodes expect me to respond to getdata requests)?  Or they won't give me full blocks unless this is set (i.e. they assume I'm a headers-only kind of node)?

[theymos]

By "not set" I assume you mean setting NODE_NETWORK to zero.  It sounded like the reference client doesn't really take this into account yet, but that's not a reason to ignore it...

Well, NODE_NETWORK is a constant always equal to 1, so you don't set that... It is taken into account in places, and Bitcoin will even unset the flag for itself when it is compiled to behave as a lightweight client.

Does this mean that I should be relaying full blocks if this value is set (i.e. other nodes expect me to respond to getdata requests)?

Right. Bitcoin expects to be able to download the entire block chain from you.

[etotheipi]

I'm talking about my personal client development.  I can set NODE_NETWORK to whatever I want... though it sounds like 0 or 1 are the only options that are meaningful.

[theymos]

I'm talking about my personal client development.  I can set NODE_NETWORK to whatever I want... though it sounds like 0 or 1 are the only options that are meaningful.

NODE_NETWORK is a constant flag that is applied to nServices, a bitfield. Not applying it leaves nServices at 0, but NODE_NETWORK remains at 1.

Without NODE_NETWORK, you are not a "network node".

[bitrick]

I'm talking about my personal client development.  I can set NODE_NETWORK to whatever I want... though it sounds like 0 or 1 are the only options that are meaningful.

NODE_NETWORK is a constant flag that is applied to nServices, a bitfield. Not applying it leaves nServices at 0, but NODE_NETWORK remains at 1.

Without NODE_NETWORK, you are not a "network node".

In the Satoshi client, there is a global variable in net.cpp called "fClient" that controls whether the local node is a full node or just a "client".
This variable is currently hardcoded to false.

If the variable is changed to true, then NODE_NETWORK will not be applied to the nLocalServices variable and the local client will not advertise itself as a full node when it sends the initial "version" message.

Other nodes will see this and not ask that node for blocks.

Note, you may want check with Gavin on the status of using fClient mode. Most of the flag variables are hooked up to command line parameters, but this one is not, probably because it is not ready and well tested. Quoting Gavin from the following thread last month:

http://sourceforge.net/mailarchive/forum.php?thread_name=CAJ1JLtu7UXz2X25iWLKUUmK%3Doapj8OzGRH964JY%3DPpzd8rOfTg%40mail.gmail.com&forum_name=bitcoin-development

"Stuff I'd like to see in the release-after-next:
fClient mode (download headers only, for faster initial startup; I've started the work, talk to me if you want to take over)"