Most transaction relaying is currently pointless and wastefull

[ByteCoin]

The original bitcoin design has transactions propagating across the network of peers essentially at random. This was a reasonable design as all peers were approximately indistinguishable.

With the advent of GPU mining and especially mining pools, the network partitions clearly into mining and essentially non-mining peers.
When a transaction is created, if it is relayed to non-mining peers the only effect is to force the recipient to waste time verifying it before passing it on. Only when the transaction reaches mining peers is there any chance that it might get into a block. Similarly, once all the miners know about a transaction there's little point relaying it to the remaining non-mining peers.

With the current mining situation it would be more efficient to have transactions sent directly to a random node on an interconnected mining network.

Also, there's little point in most non-mining clients being informed about new blocks unless they are waiting for a transaction to confirm.

It would be more efficient for new blocks to be pulled from the mining network rather than being pushed out indiscriminately as in the current situation.

ByteCoin

[1714103568]

1714103568

[1714103568]

1714103568

[1714103568]

1714103568

[1714103568]

1714103568

[JoelKatz]

I don't follow your reasoning. The second a new block comes out, everyone needs it. The client can hardly do anything without a current block chain.

Also, you need to see unconfirmed transactions to know if coins are heading your way. Otherwise, there will be a minimum 10 minute delay before you know a transaction has been initiated. How would you do a vending machine with a random delay around ten minutes?

[error]

Right. "Miners" are not the only ones who need to know about transactions as they occur. The recipient, for one, needs to know - and he also needs to know if someone is playing shenanigans by trying to spend the same coins. Various other services also have good reasons to monitor transaction activity in real time.

[theymos]

I agree that always relaying everything is a waste of resources. I propose that a new network message be created, "stoprelay", which would be transmitted immediately after the version exchange. The message would contain a list of things that are normally relayed ("block", "addr", etc.). The node receiving a stoprelay message would not relay the listed message types to the sender of the stoprelay except when explicitly requested with getblocks, etc.

Current lightweight clients can safely stop receiving relayed addr messages once they have enough good addresses in their databases. Current clients need to receive blocks in order to detect received transactions. Receiving transaction relays could perhaps be made optional for those people who do not care about 0-confirmation transactions.

I don't see any point in making clients poll for blocks. Everyone who wants to receive blocks will want to receive them as soon as possible, and they will therefore poll very frequently and waste resources. People who are listening for all transactions will likewise want them ASAP.

In the future, headers-only clients can maybe send a stoprelay for transactions, but then request that transactions meeting certain criteria be relayed. For example, these lightweight clients might request that all address transactions involving addresses starting in "1A" be relayed to them, except when these transactions are over 10 kB. (If remembering these criteria is burdensome on the nodes doing the relaying, polling might be necessary.)

More important than relaying, though, is the need to separate leaf and hub nodes. Leaf nodes should connect only to hubs, and hubs should make sure that they are well-connected to other hubs.

In the future, I imagine that Bitcoin will ask on first run whether you want to donate resources to help the network. If you say no, then you will run with headers only. Otherwise, you will download and verify all blocks and act as a hub. Hubs can then be identified by their NODE_NETWORK service indicator.

It is very useful for non-miner nodes to act as hubs. Right now there are less than a dozen "big miners". These nodes alone might not be able to (or might not want to) handle all of the necessary requests for data.

[gmaxwell]

The original bitcoin design has transactions propagating across the network of peers essentially at random. This was a reasonable design as all peers were approximately indistinguishable.

Because of the inv process relaying takes very little bandwidth. We're talking about a grand total of a few kbit/sec. Even with enough transactions to cause a maximum blocksize it isn't much bandwidth.

The relaying also improves security by more broadly propagating evidence of transaction activity (even if no one is currently doing much monitoring).

Most importantly, perhaps, is that it allows nodes to see incoming transactions before they are mined and it allows them to do so without constantly disclosing the identity of the keys they hold.

This is an area were lightweight nodes could probably be more lightweight, but I don't see any advantage in changing this for full validating nodes.

[realnowhereman]

I suggested in my first post that the 'services' field in the 'version' message was underused.

It would solve this problem.  Every node could announce (a) whether it was mining (b) whether it wanted relayed transactions.  There are 64 bits and only two are used, so there is plenty of flexibility.

[Steve]

Wouldn't this concern be solved by a lightweight, non block chain client that connected to some other node that was maintaining and validating the full block chain?  Inv messages make transaction announcement very lightweight.  Also, if you want to monitor the network to assess the risk of a particular transaction never being confirmed (i.e. if there is a conflicting transaction floating about), you not only want to hear about the transaction once, you want to gauge how widely distributed and accepted the transaction is (so, you want to hear about it from a large percentage of the peers with which you are connected).

[BTC_Junkie]

Wouldn't this concern be solved by a lightweight, non block chain client that connected to some other node that was maintaining and validating the full block chain?  Inv messages make transaction announcement very lightweight.  Also, if you want to monitor the network to assess the risk of a particular transaction never being confirmed (i.e. if there is a conflicting transaction floating about), you not only want to hear about the transaction once, you want to gauge how widely distributed and accepted the transaction is (so, you want to hear about it from a large percentage of the peers with which you are connected).

Doesn't that lead to a vulnerability if the super node is compromised?

[bji]

Also, you need to see unconfirmed transactions to know if coins are heading your way. Otherwise, there will be a minimum 10 minute delay before you know a transaction has been initiated. How would you do a vending machine with a random delay around ten minutes?

You don't do a vending machine with bitcoin, period.  If you can't wait ~ 1 hr for 6 block confirmations, then you are at extreme risk of being scammed, and vendors will not rely on promises (i.e. uncomfirmed transactions) to deliver goods.  Nobody is going to use a vending machine that you have for wait 1 hr to prove you've paid for your chocolate bar, ergo, there will be no bitcoin vending machines.

[bji]

Wouldn't this concern be solved by a lightweight, non block chain client that connected to some other node that was maintaining and validating the full block chain?  Inv messages make transaction announcement very lightweight.  Also, if you want to monitor the network to assess the risk of a particular transaction never being confirmed (i.e. if there is a conflicting transaction floating about), you not only want to hear about the transaction once, you want to gauge how widely distributed and accepted the transaction is (so, you want to hear about it from a large percentage of the peers with which you are connected).

Yes.  I believe that if bitcoin flourishes, this is inevitable, and it will happen this way.

[bji]

Wouldn't this concern be solved by a lightweight, non block chain client that connected to some other node that was maintaining and validating the full block chain?  Inv messages make transaction announcement very lightweight.  Also, if you want to monitor the network to assess the risk of a particular transaction never being confirmed (i.e. if there is a conflicting transaction floating about), you not only want to hear about the transaction once, you want to gauge how widely distributed and accepted the transaction is (so, you want to hear about it from a large percentage of the peers with which you are connected).

Doesn't that lead to a vulnerability if the super node is compromised?

Every client talks a a few supernodes and only trusts results that are confirmed by all.

[davout]

If you propagate txes to only a single or a couple of mining nodes it could act selfishly and not relay to other mining nodes in order to keep the potential fees resulting in slower tx validation.

[ByteCoin]

[a mining node] could act selfishly and not relay to other mining nodes in order to keep the potential fees resulting in slower tx validation.

This is an important point - especially when the mining reward drops to a level where fees make a more significant fraction of the miner's return.

I doubt that, with the currrent protocol, we could reliably detect whether a miner is not relaying transactions as it should. Even if we could detect it, there's a question of what sort of steps could be taken to discourage it. Any sort of collaborative censure of the non-relaying miner would have the tendancy to encourage the formation of mining cartels.

It's a completely rational mining behaviour however. I expect it to be the default eventually if no preventative measures are taken.

Also it's interesting to note that it's a scenario in which attaching a fee to a transaction makes it slower to confirm!

ByteCoin

[Gavin Andresen]

Half-baked thoughts on transaction relaying (I agree that it may become a major problem because there are no incentives to properly relay transactions right now):

I think writing code to reliably detect that a peer isn't relaying transactions is possible. Something like:
 Generating a new transaction:
  Pick a connected peer at random "P"
  Relay new transaction to all nodes EXCEPT P
  If, after a little while, P doesn't tell us about our new transaction then it is likely P is not relaying properly.
   (assumption is that we are not P's only connection, and it will get the transaction from its other peers)

And I think something like the above could be one of the metrics used to measure "ill-behaving peers" (other metrics might be number of double-spend transactions or orphan blocks received, number of spammy-looking transactions received, etc).  If a peer is too ill-behaved, the penalty could be shunning-- drop its connection and add its IP to a temporary refuse-connections list.

(maybe lesser penalties make sense, too-- maybe order-of-relaying is based on good behavior, so the code announces new blocks/transactions to better-behaved peers fore worse-behaved peers).

If cheating miners find themselves disconnected from the rest of the network, that is a strong incentive not to cheat.

I'd really like somebody with a lot more network design experience than me to run some simulations and see what network behavior would be like with/without rules like I'm proposing.  Or tell me that disconnecting ill-behaving nodes is a terrible idea because it makes it easy for an attacker to shatter the network into a gazillion pieces...