Bitcoin Forum

Bitcoin => Development & Technical Discussion => Topic started by: gmaxwell on May 28, 2019, 03:29:06 AM



Title: Bandwidth-Efficient Transaction Relay
Post by: gmaxwell on May 28, 2019, 03:29:06 AM

In a follow-up to the ideas discussed in https://bitcointalk.org/index.php?topic=1377345.0

Quote
Hi all,

We are making public our latest work on Erlay, an efficient transaction relay protocol for Bitcoin.
It is available here: https://arxiv.org/abs/1905.10518

The main idea is that instead of announcing every transaction to every peer, announcements are only sent directly over a small number of connections (only 8 outgoing ones). Further relay is achieved by periodically running a set reconciliation protocol over every connection between the sets of withheld announcements in both directions.

The set reconciliation protocol uses error correcting codes to communicate a set of transactions to a peer with an unknown but similar set using bandwidth only equal to the size of the difference and not the size of the sets themselves.

Results: we save half of the bandwidth a node consumes, allow increasing connectivity almost for free, and, as a side effect, better withstand timing attacks.
If outbound peer count were increased to 32, Erlay saves around 75% overall bandwidth compared to the current protocol.

This work uses Minisketch, an efficient library for set reconciliation, which we made public before: github.com/sipa/minisketch.

Some of you may already know about it from discussions with me, Scaling Bitcoin 18, or CoreDev in Tokyo. Our proposal has become more precise since then.

The next step here is to receive more feedback, have a broader discussion, and then write a BIP along with improving reference implementation. We are looking forward to hearing your suggestions or concerns regarding this work.

This protocol is a result of work by myself, Gregory Maxwell, Pieter Wuille, and my supervisors at UBC: Ivan Beschastnikh and Sasha Fedorova.
I would like to thank Tim Ruffing and Ben Woosley for contributions to the write-up, and Blockstream for supporting my work on this protocol.
gleb
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2019-May/016994.html


Title: Re: Bandwidth-Efficient Transaction Relay
Post by: bitmover on May 28, 2019, 02:35:44 PM
Is transaction broadcast really a problem that needs to be solved?

And won't this solution affect decentralization of transactions broadcast?


Title: Re: Bandwidth-Efficient Transaction Relay
Post by: ETFbitcoin on May 28, 2019, 03:50:06 PM
Is it exclusively used for relay transaction or block (either compact block or full block for node which left behind) as well?

Is transaction broadcast really a problem that needs to be solved?

I think we should perceive this works as optimization rather than a problem & i don't see any reason to reject optimization (as long as it don't causes new problem).

Besides, this would lesser the difficulty of on-chain scaling.


Title: Re: Bandwidth-Efficient Transaction Relay
Post by: gmaxwell on May 30, 2019, 02:28:36 AM
Is transaction broadcast really a problem that needs to be solved?

Erlay reduces node bandwidth usage by 38%, or by 75% if connectivity is increased from 8 to 32 for attack robustness reasons.

Quote
And won't this solution affect decentralization of transactions broadcast?
The opposite: transaction relay is more decenteralized and robust if nodes have more connections. Erlay allows having more connections without using significantly more bandwidth.


Title: Re: Bandwidth-Efficient Transaction Relay
Post by: Wind_FURY on May 30, 2019, 05:52:25 AM
I don't understand more than half of how it actually works, although I try to learn, but thank you gmaxwell and the other Core developers who worked on this. The network is better with you around.

#RealScaling 8)


Title: Re: Bandwidth-Efficient Transaction Relay
Post by: Lauda on May 30, 2019, 09:58:26 AM
Is transaction broadcast really a problem that needs to be solved?
Erlay reduces node bandwidth usage by 38%, or by 75% if connectivity is increased from 8 to 32 for attack robustness reasons.
Assuming this gets added to Bitcoin Core soon, would you say that it is likely that the default connectivity would also be increased?


Title: Re: Bandwidth-Efficient Transaction Relay
Post by: gmaxwell on May 30, 2019, 10:43:30 AM
Assuming this gets added to Bitcoin Core soon, would you say that it is likely that the default connectivity would also be increased?
Maybe not at the same time because there are other things like per-connection memory usage that need to be optimized too... but eventually, sure. Probably you'll see per connection memory usage get improved, then the default inbound connection limit get increased, and only after that's well deployed would the default outbound level get increased.


Title: Re: Bandwidth-Efficient Transaction Relay
Post by: Lauda on May 30, 2019, 02:18:28 PM
Assuming this gets added to Bitcoin Core soon, would you say that it is likely that the default connectivity would also be increased?
Maybe not at the same time because there are other things like per-connection memory usage that need to be optimized too... but eventually, sure. Probably you'll see per connection memory usage get improved, then the default inbound connection limit get increased, and only after that's well deployed would the default outbound level get increased.
How far away would you say that we're from seeing this be implemented/PR made?


Title: Re: Bandwidth-Efficient Transaction Relay
Post by: gmaxwell on June 02, 2019, 10:58:45 AM
How far away would you say that we're from seeing this be implemented/PR made?
I have no idea. I'm not planning on working on it, Gleb or Pieter might have some idea.