Bitcoin Forum
October 15, 2019, 07:57:21 AM *
News: Latest Bitcoin Core release: 0.18.1 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1]
  Print  
Author Topic: [minisketch] Draft Erlay tx relay BIP published  (Read 224 times)
This is a self-moderated topic. If you do not want to be moderated by the person who started this topic, create a new topic. (2 posts by 2 users deleted.)
Carlton Banks
Legendary
*
Offline Offline

Activity: 2520
Merit: 1976



View Profile
September 27, 2019, 01:11:34 PM
Last edit: October 02, 2019, 02:15:03 PM by Carlton Banks
Merited by Welsh (6), ETFbitcoin (1), squatter (1), TryNinja (1), hugeblack (1), Coding Enthusiast (1)
 #1

https://github.com/naumenkogs/bips/blob/bip-reconcil/bip-reconcil.mediawiki


exciting, I didn't expect progress so quickly since the announcement of the Minisketch library this Spring.

This implementation does something really smart; @naumenkogs has opted to snip the txids to a fraction of their full length, then adds a salt to guard against hash collisions. This saves more bandwidth than set reconciliation alone. The total bandwidth savings are apparently ~40% over the flood based transaction relaying that Bitcoin uses now (and that's the figure for a "private" node with 8 outgoing connections, public nodes with hundreds of connections will save far more bandwidth).

I think daily tx relay on average uses something like 300MB? Not sure. If so, that would mean cutting that to 180MB Cool


If this is an easy BIP to review, it could make it into Bitcoin next year Smiley

Vires in numeris
1571126241
Hero Member
*
Offline Offline

Posts: 1571126241

View Profile Personal Message (Offline)

Ignore
1571126241
Reply with quote  #2

1571126241
Report to moderator
1571126241
Hero Member
*
Offline Offline

Posts: 1571126241

View Profile Personal Message (Offline)

Ignore
1571126241
Reply with quote  #2

1571126241
Report to moderator
1571126241
Hero Member
*
Offline Offline

Posts: 1571126241

View Profile Personal Message (Offline)

Ignore
1571126241
Reply with quote  #2

1571126241
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1571126241
Hero Member
*
Offline Offline

Posts: 1571126241

View Profile Personal Message (Offline)

Ignore
1571126241
Reply with quote  #2

1571126241
Report to moderator
1571126241
Hero Member
*
Offline Offline

Posts: 1571126241

View Profile Personal Message (Offline)

Ignore
1571126241
Reply with quote  #2

1571126241
Report to moderator
Carlton Banks
Legendary
*
Offline Offline

Activity: 2520
Merit: 1976



View Profile
October 01, 2019, 02:06:07 PM
Last edit: October 02, 2019, 02:01:13 PM by Carlton Banks
 #2

video on Erlay, presented by @naumenkogs:

https://youtube.com/watch?v=YxsjdIl0034&t=13m00s


(jump to minute 0:13:00 to get to the start)

Vires in numeris
pereira4
Legendary
*
Offline Offline

Activity: 1554
Merit: 1150


View Profile
October 02, 2019, 01:51:03 PM
Merited by Carlton Banks (1)
 #3

https://arxiv.org/abs/1905.10518
Quote

Unlike block relay, transaction dissemination has received little attention in prior work. We propose a new transaction dissemination protocol, Erlay, that not only reduces the bandwidth consumption by 40% assuming current connectivity, but also keeps the bandwidth use almost constant as the connectivity increases. In contrast, the existing protocol increases the bandwidth consumption linearly with the number of connections. By allowing more connections at a small cost, Erlay improves the security of the Bitcoin network. And, as we demonstrate, Erlay also hardens the network against attacks that attempt to learn the origin node of a transaction. Erlay is currently being investigated by the Bitcoin community for future use with the Bitcoin protocol.

Yeah this looks great on paper... reduction of bandwith and more security? Could anyone make a case for this being controversial to add for any reason? Segwit was also on principle only benefits but a whole narrative against it was constructed. Even if this can be added without the miner signal clusterfuck, there's always people that find things to pick at.


video on Erlay, presented by @naumenkogs:

https://youtube.com/watch?v=YxsjdIl0034


(jump to minute 0:13:00 to get to the start)

You can link directly link in youtube now  Grin

https://youtube.com/watch?v=YxsjdIl0034&t=13m00s
Carlton Banks
Legendary
*
Offline Offline

Activity: 2520
Merit: 1976



View Profile
October 02, 2019, 02:12:18 PM
Last edit: October 02, 2019, 06:36:37 PM by Carlton Banks
 #4

You can link directly link in youtube now  Grin

https://youtube.com/watch?v=YxsjdIl0034&t=13m00s

corrected my link, thanks for pointing that out Cheesy


Yeah this looks great on paper... reduction of bandwith and more security? Could anyone make a case for this being controversial to add for any reason? Segwit was also on principle only benefits but a whole narrative against it was constructed. Even if this can be added without the miner signal clusterfuck, there's always people that find things to pick at.

no consensus changes are needed (erlay is a p2p layer change), so there's no scope for delaying or blocking this change. flood relaying will continue to be supported, and I think it's probably needed as a fallback mode in cases where set reconciliation doesn't work anyway (successful reconciliation is probabilistic)

it's also difficult to see what the objections could be. Maybe one could argue that erlay pushes transaction latency up high enough to erase gains in reduced block orphaning? if it came from the "small blocks bad" people, it would be hilarious hypocrisy, as their position was always that higher block orphaning rates was an acceptable trade-off for "big" blocks (i.e. >8MB). but I guess we can expect them to make a bunch of contradictory and self-refuting claims anyway, it's never stopped anybody before Grin

Vires in numeris
bitaps
Member
**
Offline Offline

Activity: 128
Merit: 39

https://bitaps.com/


View Profile WWW
October 02, 2019, 02:29:25 PM
 #5

Quote
Truncated transaction IDs
For announcing and relaying transaction outside of reconciliation, we need an unambiguous, unsalted way to refer to transactions to deduplicate transaction requests. As we're introducing a new scheme anyway, this is a good opportunity to switch to wtxid-based requests rather than txid-based ones.

What is the motivation to switch to wtx_id?

pereira4
Legendary
*
Offline Offline

Activity: 1554
Merit: 1150


View Profile
October 02, 2019, 05:33:40 PM
 #6

Quote
Results: we save half of the bandwidth a node consumes, allow increasing connectivity almost for free, and, as a side effect, better withstand timing attacks.
If outbound peer count were increased to 32, Erlay saves around 75% overall bandwidth compared to the current protocol.”

As far as I can tell, this could create another wave of people demanding a block size increase due the reduced costs in maintaining a node, so I can see pressure on raising the block size again making a case out of this update. The question is, will there be an agreement on increasing the block size this time? I bet that the outcome is a no. Looking forward to dumping "Bitcoin Erlay" for more Bitcoin Grin
Carlton Banks
Legendary
*
Offline Offline

Activity: 2520
Merit: 1976



View Profile
October 02, 2019, 06:35:17 PM
Merited by Welsh (4), bones261 (4)
 #7

this could create another wave of people demanding a block size increase due the reduced costs in maintaining a node, so I can see pressure on raising the block size again making a case out of this update.


sure, but people (myself, for instance) also said increasing the blocksize should be offset by mitigations that reduce it's impact. Erlay is one such mitigation, but there are also several more possible ways to do so.
Erlay gets us closer to making a future agreement on another blocksize increase a no-brainer, but I don't expect Erlay to break down all the resistance in the die-hard small block contingent, it's not the most significant scaling improvement on the table (and any improvement in scaling the Bitcoin network is a by-product of it's real objective)


The question is, will there be an agreement on increasing the block size this time? I bet that the outcome is a no.

You seem to be living in an alternate universe; agreement was reached last time, and the blocksize was quadrupled.


Looking forward to dumping "Bitcoin Erlay" for more Bitcoin Grin

I repeat: Erlay is not a consensus change, so there is no plausible fork.

any fork-coin to avoid Erlay couldn't possibly work, people could make an Erlay implementation for the fork, and it would interoperate with the flood relay on that coin no problem. It would be a completely pointless exercise.

Vires in numeris
figmentofmyass
Hero Member
*****
Offline Offline

Activity: 1176
Merit: 914



View Profile
October 02, 2019, 06:52:51 PM
 #8

Quote
Results: we save half of the bandwidth a node consumes, allow increasing connectivity almost for free, and, as a side effect, better withstand timing attacks.
If outbound peer count were increased to 32, Erlay saves around 75% overall bandwidth compared to the current protocol.”

As far as I can tell, this could create another wave of people demanding a block size increase due the reduced costs in maintaining a node, so I can see pressure on raising the block size again making a case out of this update. The question is, will there be an agreement on increasing the block size this time? I bet that the outcome is a no. Looking forward to dumping "Bitcoin Erlay" for more Bitcoin Grin

the elephant in the room is whether a hard fork is politically tenable at all. tbh it's hard to imagine the network not splitting in that case.

we could expand space for witness data with a soft fork, right? that might work well in a post-schnorr world since sig aggregation can drastically reduce the number of sigops in any given tx.

can anyone speak to the transaction/block latency aspect?

Carlton Banks
Legendary
*
Offline Offline

Activity: 2520
Merit: 1976



View Profile
October 02, 2019, 07:08:36 PM
Merited by figmentofmyass (1)
 #9

can anyone speak to the transaction/block latency aspect?

Erlay increases transaction latency some, but not to the extent that a 100% set-reconciliation based relay protocol would. But it's more latent than flood relay, which has optimal latency performance according to @naumenkogs' presentation.

It doens't affect block announcement latency, but could increase block orphaning rates at the margins.

Vires in numeris
pereira4
Legendary
*
Offline Offline

Activity: 1554
Merit: 1150


View Profile
October 04, 2019, 04:37:46 PM
 #10



You seem to be living in an alternate universe; agreement was reached last time, and the blocksize was quadrupled.


Agreement... more like a clusterfuck. Segwit got in via miracle. I don't see that ever happening again, ever.




I repeat: Erlay is not a consensus change, so there is no plausible fork.

any fork-coin to avoid Erlay couldn't possibly work, people could make an Erlay implementation for the fork, and it would interoperate with the flood relay on that coin no problem. It would be a completely pointless exercise.

My point is that someone would hardfork Bitcoin after Erlay and also include a 16MB blocksize or something and market it as the next big pump. The shitcoin market is all about marketing and taking advantage of events that raise attention. So pushing for the "blocksize increase because now we have Erlay" narrative, not getting the blocksize increase then going their way to market this fork of Bitcoin is something I see happening. However the hardfork is definitely pretty much dead but I predict in special situations someone will keep milking this cow, it's all about timing. And I will be there to dump for more actual BTC.
gmaxwell
Moderator
Legendary
*
qt
Offline Offline

Activity: 2842
Merit: 2513



View Profile
October 04, 2019, 11:28:21 PM
Merited by Welsh (10), bones261 (4), ETFbitcoin (1)
 #11

It doens't affect block announcement latency, but could increase block orphaning rates at the margins.
Simulation seems to show that it improves block propagation delay in fact.

Erlay's latency increase for propagation is almost entirely an increase in latency of the first hop.

Overall with the recommended parameters in simulation the result has transactions take a little longer to propagate but there is less variance in their overall propagation time.  This improves the block propagation speed because more often if a miner includes a txn everyone already has it. ... at least in simulation.

I wouldn't promote that as an advantage, because miners adding a small delay before including transactions would likely have a much greater non-additive effect ...  but it doesn't appear the an increase in orphaning is likely, at least.

My point is that someone would hardfork Bitcoin after Erlay and also include a 16MB blocksize or something and market it as the next big pump.
We're mostly past that point. The same people they could deceive by doing that can be even better deceived by pure jargon-technobabble, and the Bitcoin industry won't even bother to call out the fraud in that case.  If they just copy bitcoin they'll be constrained by the realism of the bitcoin discussion of the copied tech and also risk irritating bitcoin people calling out their garbage.

Doesn't look good for your erlay fuelled scam if the inventors of erlay step up and call it a scam, much safer to promote rolling-hypercube-slasher-coordinator-masternode-unls or whatever where you can dismiss any criticism as pure ignorance and "toxic maximalism".
Coding Enthusiast
Hero Member
*****
Offline Offline

Activity: 690
Merit: 1095


Novice C♯ Coder


View Profile WWW
October 05, 2019, 02:10:27 PM
 #12

Quote
Truncated transaction IDs
For announcing and relaying transaction outside of reconciliation, we need an unambiguous, unsalted way to refer to transactions to deduplicate transaction requests. As we're introducing a new scheme anyway, this is a good opportunity to switch to wtxid-based requests rather than txid-based ones.

What is the motivation to switch to wtx_id?

This is a very odd decision in my opinion. Wtxids are not used anywhere (so it shouldn't be pre-computed already) and they are more expensive to compute, since the purpose is "optimization" using them seems to add to the overall time.

Imagine the simplest SegWit transaction, one that is spending 1x P2WPKH and creates 2x new outputs. To compute txid you'd have to compute 2x SHA256 of ~116 bytes but for computing wtxid you'd have to compute 2x SHA256 of ~225 bytes. The first round of these two require: 2 blocks and 4 blocks respectively. So the speed of computing txid in this case is nearly twice the speed of computing wtxid for a simple tx.
If the number of inputs were higher or if this was a longer witness (eg. spending a P2WSH of a 2of3 multisig scheme) the difference would be drastically more.

Projects List+Suggestion box
Donation link using BIP21
Bech32 Donation link!
BitcoinTransactionTool (0.9.2):  Ann - Source Code
Watch Only Bitcoin Wallet (supporting SegWit) (3.1.0):  Ann - Source Code
SharpPusher (broadcast transactions) (0.10.0): Ann - Source Code

gmaxwell
Moderator
Legendary
*
qt
Offline Offline

Activity: 2842
Merit: 2513



View Profile
October 05, 2019, 09:30:09 PM
Last edit: October 08, 2019, 04:38:34 AM by gmaxwell
Merited by bones261 (4), Coding Enthusiast (2)
 #13

Wtxids are not used anywhere (so it shouldn't be pre-computed already) and they are more expensive to compute,
Sure they are, they're required to tell two different transactions apart.

When txs are only identified by txids I can take a valid transaction mutate its witness to make it invalid (or just too low a feerate), and it'll have the same txid, so if you fetch by txid you can't avoid fetching the same junk multiple times-- you can't negative cache the invalid values.

wtxid works much better in that respect.  They need to be computed in any case because otherwise block propagation latency would be much higher,  and the computational cost of hashing a single transaction is pretty negligible in any case (and as mentioned, it's already being done). Switching to wtxid for relay has also been in the works for over three years.
Carlton Banks
Legendary
*
Offline Offline

Activity: 2520
Merit: 1976



View Profile
October 05, 2019, 10:28:55 PM
 #14

It doens't affect block announcement latency, but could increase block orphaning rates at the margins.
Simulation seems to show that it improves block propagation delay in fact.

Erlay's latency increase for propagation is almost entirely an increase in latency of the first hop.

Overall with the recommended parameters in simulation the result has transactions take a little longer to propagate but there is less variance in their overall propagation time.  This improves the block propagation speed because more often if a miner includes a txn everyone already has it. ... at least in simulation.

hmm, so less variance in overall propagation leads to less variance in respective mempools contents over time, and so of course compact block success rate improves. That makes sense, nice consequential effect.

Vires in numeris
Carlton Banks
Legendary
*
Offline Offline

Activity: 2520
Merit: 1976



View Profile
October 07, 2019, 11:40:45 AM
Merited by gmaxwell (1)
 #15

just quickly pointing out the thread moderation policy:

  • no trolling
  • no trolls

because bitaps replied to a known troll account, their recent reply was wholly removed.

Vires in numeris
Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!