Carlton Banks (OP)
Legendary
Offline
Activity: 3430
Merit: 3080
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
September 27, 2019, 01:11:34 PM Last edit: October 02, 2019, 02:15:03 PM by Carlton Banks |
|
https://github.com/naumenkogs/bips/blob/bip-reconcil/bip-reconcil.mediawikiexciting, I didn't expect progress so quickly since the announcement of the Minisketch library this Spring. This implementation does something really smart; @naumenkogs has opted to snip the txids to a fraction of their full length, then adds a salt to guard against hash collisions. This saves more bandwidth than set reconciliation alone. The total bandwidth savings are apparently ~40% over the flood based transaction relaying that Bitcoin uses now (and that's the figure for a "private" node with 8 outgoing connections, public nodes with hundreds of connections will save far more bandwidth). I think daily tx relay on average uses something like 300MB? Not sure. If so, that would mean cutting that to 180MB ![Cool](https://bitcointalk.org/Smileys/default/cool.gif) If this is an easy BIP to review, it could make it into Bitcoin next year ![Smiley](https://bitcointalk.org/Smileys/default/smiley.gif)
|
Vires in numeris
|
|
|
|
pereira4
Legendary
Offline
Activity: 1610
Merit: 1183
|
https://arxiv.org/abs/1905.10518 Unlike block relay, transaction dissemination has received little attention in prior work. We propose a new transaction dissemination protocol, Erlay, that not only reduces the bandwidth consumption by 40% assuming current connectivity, but also keeps the bandwidth use almost constant as the connectivity increases. In contrast, the existing protocol increases the bandwidth consumption linearly with the number of connections. By allowing more connections at a small cost, Erlay improves the security of the Bitcoin network. And, as we demonstrate, Erlay also hardens the network against attacks that attempt to learn the origin node of a transaction. Erlay is currently being investigated by the Bitcoin community for future use with the Bitcoin protocol.
Yeah this looks great on paper... reduction of bandwith and more security? Could anyone make a case for this being controversial to add for any reason? Segwit was also on principle only benefits but a whole narrative against it was constructed. Even if this can be added without the miner signal clusterfuck, there's always people that find things to pick at. You can link directly link in youtube now https://youtube.com/watch?v=YxsjdIl0034&t=13m00s
|
|
|
|
Carlton Banks (OP)
Legendary
Offline
Activity: 3430
Merit: 3080
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
October 02, 2019, 02:12:18 PM Last edit: October 02, 2019, 06:36:37 PM by Carlton Banks |
|
corrected my link, thanks for pointing that out ![Cheesy](https://bitcointalk.org/Smileys/default/cheesy.gif) Yeah this looks great on paper... reduction of bandwith and more security? Could anyone make a case for this being controversial to add for any reason? Segwit was also on principle only benefits but a whole narrative against it was constructed. Even if this can be added without the miner signal clusterfuck, there's always people that find things to pick at. no consensus changes are needed (erlay is a p2p layer change), so there's no scope for delaying or blocking this change. flood relaying will continue to be supported, and I think it's probably needed as a fallback mode in cases where set reconciliation doesn't work anyway (successful reconciliation is probabilistic) it's also difficult to see what the objections could be. Maybe one could argue that erlay pushes transaction latency up high enough to erase gains in reduced block orphaning? if it came from the "small blocks bad" people, it would be hilarious hypocrisy, as their position was always that higher block orphaning rates was an acceptable trade-off for "big" blocks (i.e. >8MB). but I guess we can expect them to make a bunch of contradictory and self-refuting claims anyway, it's never stopped anybody before ![Grin](https://bitcointalk.org/Smileys/default/grin.gif)
|
Vires in numeris
|
|
|
bitaps
Member
![*](https://bitcointalk.org/Themes/custom1/images/star.gif)
Offline
Activity: 148
Merit: 45
https://bitaps.com/
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
October 02, 2019, 02:29:25 PM |
|
Truncated transaction IDs For announcing and relaying transaction outside of reconciliation, we need an unambiguous, unsalted way to refer to transactions to deduplicate transaction requests. As we're introducing a new scheme anyway, this is a good opportunity to switch to wtxid-based requests rather than txid-based ones.
What is the motivation to switch to wtx_id?
|
|
|
|
pereira4
Legendary
Offline
Activity: 1610
Merit: 1183
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
October 02, 2019, 05:33:40 PM |
|
Results: we save half of the bandwidth a node consumes, allow increasing connectivity almost for free, and, as a side effect, better withstand timing attacks. If outbound peer count were increased to 32, Erlay saves around 75% overall bandwidth compared to the current protocol.” As far as I can tell, this could create another wave of people demanding a block size increase due the reduced costs in maintaining a node, so I can see pressure on raising the block size again making a case out of this update. The question is, will there be an agreement on increasing the block size this time? I bet that the outcome is a no. Looking forward to dumping "Bitcoin Erlay" for more Bitcoin ![Grin](https://bitcointalk.org/Smileys/default/grin.gif)
|
|
|
|
Carlton Banks (OP)
Legendary
Offline
Activity: 3430
Merit: 3080
|
this could create another wave of people demanding a block size increase due the reduced costs in maintaining a node, so I can see pressure on raising the block size again making a case out of this update. sure, but people (myself, for instance) also said increasing the blocksize should be offset by mitigations that reduce it's impact. Erlay is one such mitigation, but there are also several more possible ways to do so. Erlay gets us closer to making a future agreement on another blocksize increase a no-brainer, but I don't expect Erlay to break down all the resistance in the die-hard small block contingent, it's not the most significant scaling improvement on the table (and any improvement in scaling the Bitcoin network is a by-product of it's real objective) The question is, will there be an agreement on increasing the block size this time? I bet that the outcome is a no. You seem to be living in an alternate universe; agreement was reached last time, and the blocksize was quadrupled. Looking forward to dumping "Bitcoin Erlay" for more Bitcoin ![Grin](https://bitcointalk.org/Smileys/default/grin.gif) I repeat: Erlay is not a consensus change, so there is no plausible fork. any fork-coin to avoid Erlay couldn't possibly work, people could make an Erlay implementation for the fork, and it would interoperate with the flood relay on that coin no problem. It would be a completely pointless exercise.
|
Vires in numeris
|
|
|
figmentofmyass
Legendary
Offline
Activity: 1652
Merit: 1483
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
October 02, 2019, 06:52:51 PM |
|
Results: we save half of the bandwidth a node consumes, allow increasing connectivity almost for free, and, as a side effect, better withstand timing attacks. If outbound peer count were increased to 32, Erlay saves around 75% overall bandwidth compared to the current protocol.” As far as I can tell, this could create another wave of people demanding a block size increase due the reduced costs in maintaining a node, so I can see pressure on raising the block size again making a case out of this update. The question is, will there be an agreement on increasing the block size this time? I bet that the outcome is a no. Looking forward to dumping "Bitcoin Erlay" for more Bitcoin ![Grin](https://bitcointalk.org/Smileys/default/grin.gif) the elephant in the room is whether a hard fork is politically tenable at all. tbh it's hard to imagine the network not splitting in that case. we could expand space for witness data with a soft fork, right? that might work well in a post-schnorr world since sig aggregation can drastically reduce the number of sigops in any given tx. can anyone speak to the transaction/block latency aspect?
|
|
|
|
Carlton Banks (OP)
Legendary
Offline
Activity: 3430
Merit: 3080
|
can anyone speak to the transaction/block latency aspect?
Erlay increases transaction latency some, but not to the extent that a 100% set-reconciliation based relay protocol would. But it's more latent than flood relay, which has optimal latency performance according to @naumenkogs' presentation. It doens't affect block announcement latency, but could increase block orphaning rates at the margins.
|
Vires in numeris
|
|
|
pereira4
Legendary
Offline
Activity: 1610
Merit: 1183
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
October 04, 2019, 04:37:46 PM |
|
You seem to be living in an alternate universe; agreement was reached last time, and the blocksize was quadrupled. Agreement... more like a clusterfuck. Segwit got in via miracle. I don't see that ever happening again, ever. I repeat: Erlay is not a consensus change, so there is no plausible fork.
any fork-coin to avoid Erlay couldn't possibly work, people could make an Erlay implementation for the fork, and it would interoperate with the flood relay on that coin no problem. It would be a completely pointless exercise.
My point is that someone would hardfork Bitcoin after Erlay and also include a 16MB blocksize or something and market it as the next big pump. The shitcoin market is all about marketing and taking advantage of events that raise attention. So pushing for the "blocksize increase because now we have Erlay" narrative, not getting the blocksize increase then going their way to market this fork of Bitcoin is something I see happening. However the hardfork is definitely pretty much dead but I predict in special situations someone will keep milking this cow, it's all about timing. And I will be there to dump for more actual BTC.
|
|
|
|
gmaxwell
Moderator
Legendary
Offline
Activity: 4242
Merit: 8647
|
It doens't affect block announcement latency, but could increase block orphaning rates at the margins.
Simulation seems to show that it improves block propagation delay in fact. Erlay's latency increase for propagation is almost entirely an increase in latency of the first hop. Overall with the recommended parameters in simulation the result has transactions take a little longer to propagate but there is less variance in their overall propagation time. This improves the block propagation speed because more often if a miner includes a txn everyone already has it. ... at least in simulation. I wouldn't promote that as an advantage, because miners adding a small delay before including transactions would likely have a much greater non-additive effect ... but it doesn't appear the an increase in orphaning is likely, at least. My point is that someone would hardfork Bitcoin after Erlay and also include a 16MB blocksize or something and market it as the next big pump.
We're mostly past that point. The same people they could deceive by doing that can be even better deceived by pure jargon-technobabble, and the Bitcoin industry won't even bother to call out the fraud in that case. If they just copy bitcoin they'll be constrained by the realism of the bitcoin discussion of the copied tech and also risk irritating bitcoin people calling out their garbage. Doesn't look good for your erlay fuelled scam if the inventors of erlay step up and call it a scam, much safer to promote rolling-hypercube-slasher-coordinator-masternode-unls or whatever where you can dismiss any criticism as pure ignorance and "toxic maximalism".
|
|
|
|
Coding Enthusiast
Legendary
Offline
Activity: 1040
Merit: 2785
Bitcoin and C♯ Enthusiast
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
October 05, 2019, 02:10:27 PM |
|
Truncated transaction IDs For announcing and relaying transaction outside of reconciliation, we need an unambiguous, unsalted way to refer to transactions to deduplicate transaction requests. As we're introducing a new scheme anyway, this is a good opportunity to switch to wtxid-based requests rather than txid-based ones.
What is the motivation to switch to wtx_id? This is a very odd decision in my opinion. Wtxids are not used anywhere (so it shouldn't be pre-computed already) and they are more expensive to compute, since the purpose is "optimization" using them seems to add to the overall time. Imagine the simplest SegWit transaction, one that is spending 1x P2WPKH and creates 2x new outputs. To compute txid you'd have to compute 2x SHA256 of ~116 bytes but for computing wtxid you'd have to compute 2x SHA256 of ~225 bytes. The first round of these two require: 2 blocks and 4 blocks respectively. So the speed of computing txid in this case is nearly twice the speed of computing wtxid for a simple tx. If the number of inputs were higher or if this was a longer witness (eg. spending a P2WSH of a 2of3 multisig scheme) the difference would be drastically more.
|
|
|
|
gmaxwell
Moderator
Legendary
Offline
Activity: 4242
Merit: 8647
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
October 05, 2019, 09:30:09 PM Last edit: October 08, 2019, 04:38:34 AM by gmaxwell |
|
Wtxids are not used anywhere (so it shouldn't be pre-computed already) and they are more expensive to compute,
Sure they are, they're required to tell two different transactions apart. When txs are only identified by txids I can take a valid transaction mutate its witness to make it invalid (or just too low a feerate), and it'll have the same txid, so if you fetch by txid you can't avoid fetching the same junk multiple times-- you can't negative cache the invalid values. wtxid works much better in that respect. They need to be computed in any case because otherwise block propagation latency would be much higher, and the computational cost of hashing a single transaction is pretty negligible in any case (and as mentioned, it's already being done). Switching to wtxid for relay has also been in the works for over three years.
|
|
|
|
Carlton Banks (OP)
Legendary
Offline
Activity: 3430
Merit: 3080
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
October 05, 2019, 10:28:55 PM |
|
It doens't affect block announcement latency, but could increase block orphaning rates at the margins.
Simulation seems to show that it improves block propagation delay in fact. Erlay's latency increase for propagation is almost entirely an increase in latency of the first hop. Overall with the recommended parameters in simulation the result has transactions take a little longer to propagate but there is less variance in their overall propagation time. This improves the block propagation speed because more often if a miner includes a txn everyone already has it. ... at least in simulation. hmm, so less variance in overall propagation leads to less variance in respective mempools contents over time, and so of course compact block success rate improves. That makes sense, nice consequential effect.
|
Vires in numeris
|
|
|
Carlton Banks (OP)
Legendary
Offline
Activity: 3430
Merit: 3080
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
October 07, 2019, 11:40:45 AM |
|
just quickly pointing out the thread moderation policy: because bitaps replied to a known troll account, their recent reply was wholly removed.
|
Vires in numeris
|
|
|
|