Segwit IS the compromise: everybody is equally unhappy with it, including me

[Carlton Banks]

"You know how you can tell it's a compromise? Because all parties are equally unhappy!!!!" - Larry David in the "Car Pool Lane" episode of Curb Your Enthusiasm


Larry David is famously pessimistic, but this particular truism applies very aptly to the Segwit proposal, and I will explain why.


4MB is too big. Very quickly after it activates, the transaction spammers could take advantage of the extra space, and stuff all 4MB with spam.




Here's how -

Single sig:
A typical transaction is around 250 Bytes in size, comprising 1 input and 2 outputs (1 output goes to the receiving party, the other goes back to the sender as their change). Only one person controls the spending keys for the input address, so only one signature is required.

Multi sig:
Multi sig transactions are different. They can have a wide range of sizes, while still comprising only 1 input and 2 outputs. The number people needed to sign the transaction increases the size of the transaction, like this:

• 500 Bytes for addresses controlled by 2 parties
• 750 Bytes for adressess controlled by 3 parties
• 1,000 Bytes for addresses controlled by 4 parties
• 1,250 Bytes for addresses controlled by 5 parties
• 1,500 Bytes for addresses controlled by 6 parties

etc.....





Visually:

Single Sig

<  250 bytes   >

|    -    | ||    -    ||
tx data    sig 1



2 party Multi-sig

<        500 bytes       >

|    -    | ||    -    ||    -    ||
tx data    sig 1    sig 2



3 party Multi-sig


<             750 bytes             >

|    -    | ||    -    ||    -    ||    -    ||
tx data   sig 1    sig 2   sig 3



4 party Multi-sig


<                 1000 bytes                  >

|    -    | ||    -    ||    -    ||    -    ||    -    ||
tx data   sig 1    sig 2   sig 3    sig 4








So, now that we can see clearly how increasing the number of people need to sign the transaction increases the size of the transaction, here's how it can be used to spam the 4MB size



1 Regular full 1MB block (i.e. HOW IT WORKS NOW)

|||      1 MB limit       |||

 <          1 MB          >

 |      -      | ||      -      ||
   tx data         sigs




1 Segwit block filled with 100% single sig transactions


|||                                                 4 MB limit                                                    |||

 <          1 MB          > <          1 MB          >

 |               -              | ||              -              ||
            tx data                         sigs



1 Segwit block filled with 100% 2 party Multisig transactions


|||                                                 4 MB limit                                                    |||

 <          1 MB          > <                        2 MB                         >

 |               -              | ||                            -                             ||
            tx data                                       sigs



1 Segwit block filled with 100% 3 party Multisig transactions


|||                                                 4 MB limit                                                    |||

 <          1 MB          > <                                     3 MB                                       >

 |               -              | ||                                         -                                          ||
            tx data                                                    sigs



1 Segwit block filled with 100% 4 party Multisig transactions


|||                                                 4 MB limit                                                    |||

 <       0.8 MB       > <                                       3.2 MB                                     >

 |             -             | ||                                           -                                           ||
           tx data                                                    sigs



1 Segwit block filled with 100% 5 party Multisig transactions


|||                                                 4 MB limit                                                    |||

 <    0.667 MB   > <                                       3.333 MB                                     >

 |           -           | ||                                             -                                             ||
        tx data                                                     sigs



1 Segwit block filled with 100% 6 party Multisig transactions


|||                                                 4 MB limit                                                    |||

 <   0.571 MB  > <                                        3.428 MB                                      >

 |          -          | ||                                              -                                              ||
       tx data                                                     sigs






So, we can see from this exactly how Segwit's new structure and new blocksize limit can be abused by the spammers: simply create the same mega-flood volumes of transactions as now, using the maximum Multi-sig signing parties they can (the protocol defines 15 multi-sig signing parties as the maximum)


That could potentially produce a maximum of 0.25 MB block of transaction data and 3.75 MB witness block of signature data, cutting tx/s rate to 0.75/s. Not good.





What I hope this demonstrates is that Segwit indeed is not perfect, it's a compromise.

THESE ARE ONLY SCENARIOS. In reality, real users of the Bitcoin network will outcompete the spam using higher fees.


And so, what does increasing the blocksize really achieve? The spam potential scales up at the same ratio the transaction rate does. We'd have more space for transactions, sure, but the TX FEES WILL BE ESSENTIALLY THE SAME. And the blockchain will grow x4 as fast as it does now.





So, I'm not saying Segwit's a bad idea. IT'S A COMPROMISE, WITH DRAWBACKS, AND IMPROVEMENTS. The sooner people genuinely understand the trade-offs, and why small-block people such as myself prefer to keep an absolute 1MB limit, and use 2nd layer networks like Lightning, Sprite etc to achieve TRUE TX SCALING, the better.

[Carlton Banks]

Apologies for the super massive OP. It's necessary though.

[AngryDwarf]

Apologies for the super massive OP. It's necessary though.

No need to apolgise, it is a very clear and graphic representation of the issue. Thank you very much for this.

[Carlton Banks]

Take a bow Dwarf, it's your post that inspired me to make this thread

[Carlton Banks]

Even a dwarf will bow to a virtuous Elf! 

I'm a dwarf too, in case you weren't aware

[7788bitcoin]

It will take some time before every one understands the pros and cons of each BIP. Your post is super massive but I do not find it difficult to read. It is one of the best presentations on the benefits of Segwit.

[manselr]

There are tradeoffs for everything and segwit is indeed the right compromise. I don't understand how some people still see the idea of Jihan BU being the ruler of bitcoin a better idea. Sooner or later things will happen and the balance will go towards segwit. Nodes and merchants want it, miners should follow.

[Vaccinus]

you said that 4mb is too much, while that is true, couldn't we have only what is need at any given time? if we need 1MB for 1 minute we have 1 mega if we need 4MB for 2 minutes we have 4 mega and so on, isn't this the best solution? there is something preventy this possibility pmaybe?

[calkob]

Thanks for the break down, i am under the impression also that there is nothing else in the pipeline either, its either Segwit or not.  like others i feel that the community will sort this out sooner or later.  hopefully sooner,

thanks again.

[Qartersa]

I would say that this thread was an eye opener. All this time I was thinking that segwit was the solution to our problem with slow transactions and high fees. Now, come to think of it, segwit does not really solve the problem. The problem will still be there, if we are talking about the spammers on the blockchain. I think a good solution probably, though I don't know how effective and if it will be supported by anyone, is to put a default transaction fee if that is possible ofcourse. Just put a small fee, like 100 satoshi. So that spammers don't do their shit for free. They will be forced to pay huge amounts of money because they are spamming thousands of transactions. 100 Satoshi is nothing for people who legitimately need to do a transaction.

[coins101]

Nicely explained.

You should consider adding some notes on any speed up improvements from moving signing data out of the validation process.

[gmaxwell]

Segwit doesn't give any spammers more power than they have had before, in fact it makes them much weaker.


It improves the system by changing the limited quantity from "size" to weight because weight better reflects the true costs to the system than size.   Size is totally irrelevant for pruning nodes, with fast-block-relay-protocol, BIP152, and Fibre size is almost totally irrelevant.  It matters for non-pruned nodes, but at the current size and growth rates those are already effectively pushed out of using cheap fast storage-- e.g. you're not likely to run a node on a 256GB SSD anymore.

e.g. see the stats from Matt's public fibre network:

-- with the exception of small or effectively empty blocks that fit into a single packet, the size of a block has no real effect on its propagation time.  (Time figures are in millisecond delay over the speed-of-light time)

Though size is decreasingly important, UTXO impact remains important as we previously ignored in the past limits. Because size is blind to UTOX impact:  A spammer could create transactions that bloated the UTXO set and paid _less_ per byte than ordinary transactions.

One possible solution is multiple distinct limits but a problem there is that with multiple limits in play it is impossible for wallets to figure out appropriate fees, because the right fees would depend on the exact mixture of transactions at play when the block is created. Multiple effective limits also make creating an income optimizing block very computationally difficult-- but with one limit you simply take transaction in order of fee per limit-used.  So to avoid that the weight limit combines both witness and non-witness (utxo impacting) limits into a single limit.   Whenever this is done, improving one factor (e.g. amount of worst case UTXO bloat per byte to typical) will make another worse (e.g. worst case amount of size.).  Segwit's parameters were set to massively decrease the amount of UTXO bloat possible while only increasing the worst cast size modestly:


Green line is how many times beyond typical a UTXO Bloat block can be, red line is how many times larger than typical a "size" bloat block can be.   X is the segwit factor, which is 0.25 in the actual segwit proposal.  The pre-segwit behavior is effectively an X of 1.

Since UTXO costs are much more concerning than size on the modern network it might be arguably to set the segwit factor even lower, but the size bloat goes up hyperbolically with a lower factor.  So even though size is less important the trade-off becomes less attractive with values much lower.  The value chosen for the proposal also happens to be just about what is required to get virtually all the capacity gain from this particular restructuring, owing to the typical ration of witness to non-witness data in transactions.

All that said,

Segwit is indeed a compromise: Increasing the networks' load and capacity is a non-trivial risk that may ultimately cause harm.  Segwit does a lot to mitigate that risk, but it remains.  It offers effective the same capacity that that the hardforksers were pushing for w/ Classic & BIP109 before they moved the goalposts again after a way was found to achieve that without splitting the network.

It just isn't a compromise because malicious miners could make blocks with 4MB size-- they could, but who cares... blocks with 4x more UTXO bloat would be a _much_ greater concern. And under the constraint that there be a single limit-- needed to make block creation and fee selection tractable-- improving the one bloat attack means that that the worst case for the other, less important, one will be worse.

[Carlton Banks]

It will take some time before every one understands the pros and cons of each BIP. Your post is super massive but I do not find it difficult to read. It is one of the best presentations on the benefits of Segwit.

Everyone including me, already 2 people have highlighted several implications of Segwit of which I was previously unaware

you said that 4mb is too much, while that is true, couldn't we have only what is need at any given time? if we need 1MB for 1 minute we have 1 mega if we need 4MB for 2 minutes we have 4 mega and so on, isn't this the best solution? there is something preventy this possibility pmaybe?

I understand why algorithmic/dynamic resizing is compelling, I initially thought such a system would be the eventual solution.

But the more I've thought and read about the proposals, it seems like opening a fresh can of worms that in fact changes the outcome of blocksize solutions very little (in practice, it may be worse than a static limit, or a static limit with step changes)

Thanks for the break down, i am under the impression also that there is nothing else in the pipeline either, its either Segwit or not.  like others i feel that the community will sort this out sooner or later.  hopefully sooner,

thanks again.

I'm trying to look at the significance of Segwit as objectively as I can. It seems to me that people are beginning to simplify the meaning of the actual word "Segwit" such that it means only "the team of neckbeards who have political affiliation to the Blockstream corporation, and also so happen to want 4MB blocks because they hate everyone and everything else".

The real definition of Segwit literally is the Segwit code, and the effects that the acitvated Segwit code paths confer to the Bitcoin network, and in turn to the overall Bitcoin ecosystem. In other words, Segwit is objectively the right change at the right point in Bitcoin's development, it's aggregate effects (and it's necessary method of deployment) are precisely what prove and define it, it is objectively the change that will meet the objectives of Bitcoin's continued evolution as a free-market, censorship resistant, ubiquitous money.

There is, and cannot be by definition, "something else in the pipeline". I feel it's a little like saying "this round thing is the wrong shape for a wheel, surely we can refine it so it rolls even better before we fit it to the axles"

I would say that this thread was an eye opener. All this time I was thinking that segwit was the solution to our problem with slow transactions and high fees. Now, come to think of it, segwit does not really solve the problem. The problem will still be there, if we are talking about the spammers on the blockchain. I think a good solution probably, though I don't know how effective and if it will be supported by anyone, is to put a default transaction fee if that is possible ofcourse. Just put a small fee, like 100 satoshi. So that spammers don't do their shit for free. They will be forced to pay huge amounts of money because they are spamming thousands of transactions. 100 Satoshi is nothing for people who legitimately need to do a transaction.

Yes, you're seeing the awkward reality of on-chain scaling; the additional blockspace resource gives the user equally increased ability to both co-operate, or disrupt. The power to create and the power to destroy.

As regards setting a transaction fee limit, this exists in several manifestations already. If you run a full Bitcoin node, you can configure which transactions your node relays to others depending on the fee each transaction pays. Many don't configure this option. I often raise my minimum relay fee above the default value, although not now I'm using Bitcoin 0.14, which increased the previous default to a higher value than my preferred settings (I have accepted the default value in 0.14 for now)

Nicely explained.

You should consider adding some notes on any speed up improvements from moving signing data out of the validation process.

I wasn't aware of this benefit, or at least I didn't think I was. Do you mean that the signing process was encumbered with the block validation process, due to the way the original consensus code was defined? Enlighten us



@gmaxwell

I would include you in this post if you hadn't spammed my thread with your overwrought & unnecessarily esoteric math & logic (JOKE ALERT). I'll respond a little later, I may just read it all a 3rd time before I do.

Accept my apologies if I seem alarmist in the OP and/or thread title; in hindsight, I could be a little clearer in representing what I'm actually thinking in contrast to the impression one may get from the OP, and the thread title was designed to be attention grabbing and/or rhetorical.

[dfd1]

If you really care so much about SegWit adoption you probably should find the way to help all the shitty altcons implement it. Hashing power of BU supporters will be divided into 100-1000 coins, not only to bitcoin, litecoin and viacoin. In order to derail voting BU supporters should simultaneously vote against SegWit on multiple blockchains with extremely huge power.

[coins101]

It seems to me that part of the job of verification is for your node to check signatures are valid.

If you move the signing data out of the primary block after a transaction has been accepted, then that process need not be done again, with such forensic analysis. Why would this not increase the speed of the verification process that each node performs?

The signing data moving out elsewhere should create the opportunity for a signatures db merkle tree, where instead of checking signatures for each past transaction, each node can verify if the signatures db has changed in anyway.

[Carlton Banks]

It seems to me that part of the job of verification is for your node to check signatures are valid.

If you move the signing data out of the primary block after a transaction has been accepted, then that process need not be done again, with such forensic analysis. Why would this not increase the speed of the verification process that each node performs?

Is this not already (or planned ) as a part of the Compact Blocks design, or is that wrong? Sounds interesting either way

[Carlton Banks]

Segwit doesn't give any spammers more power than they have had before, in fact it makes them much weaker.

yes, that's not exactly what I'm trying to illustrate (and concede that I didn't do very well and make it clear)



The problem in my eyes is the widespread perception that Segwit will only be good (in practice) for a ~2.1MB blocksize.

I'm not saying that Multi-sig spam will be more of a threat in it's own right, I did state that the ratio of spam to transactions would simply remain the same. My point is that the spammers would see the opportunity that 1.9 MB of extra space presents to them, with today's single purpose blocks, that cannot be achieved, only a DoS style attack to stuff the blocks with signatures (which would be unambiguously a spam attack). The point about every spam attack we've suffered in single purpose blocks is the same issue with the trolling of discourse on Bitcoin discussion forums: the smartest way to spam is to make it both effective flooding and ambiguous as to whether or not it really is spam. That way, trolls have an easy job of suggesting that those crying foul are actually just paranoid pessimists, as a means to aid prosecuting the "big blocks to handle all this genuine not-spam economic activity" perspective.

Filling 1MB blocks with 15 signer multi-sigs would not be sufficiently ambiguous to achieve that social engineering goal, but getting just enough multi-sig spam to fill unused space in the witness block would constitute the right balance of ambiguity to argue it's not malicious over-burdening of the blockchain; get out of here with your tin-foil hats etc.

It improves the system by changing the limited quantity from "size" to weight because weight better reflects the true costs to the system than size.   Size is totally irrelevant for pruning nodes, with fast-block-relay-protocol, BIP152, and Fibre size is almost totally irrelevant.  It matters for non-pruned nodes, but at the current size and growth rates those are already effectively pushed out of using cheap fast storage-- e.g. you're not likely to run a node on a 256GB SSD anymore.

e.g. see the stats from Matt's public fibre network:

-- with the exception of small or effectively empty blocks that fit into a single packet, the size of a block has no real effect on its propagation time.  (Time figures are in millisecond delay over the speed-of-light time)

Though size is decreasingly important, UTXO impact remains important as we previously ignored in the past limits. Because size is blind to UTOX impact:  A spammer could create transactions that bloated the UTXO set and paid _less_ per byte than ordinary transactions.

One possible solution is multiple distinct limits but a problem there is that with multiple limits in play it is impossible for wallets to figure out appropriate fees, because the right fees would depend on the exact mixture of transactions at play when the block is created. Multiple effective limits also make creating an income optimizing block very computationally difficult-- but with one limit you simply take transaction in order of fee per limit-used.  So to avoid that the weight limit combines both witness and non-witness (utxo impacting) limits into a single limit.   Whenever this is done, improving one factor (e.g. amount of worst case UTXO bloat per byte to typical) will make another worse (e.g. worst case amount of size.).  Segwit's parameters were set to massively decrease the amount of UTXO bloat possible while only increasing the worst cast size modestly:


Green line is how many times beyond typical a UTXO Bloat block can be, red line is how many times larger than typical a "size" bloat block can be.   X is the segwit factor, which is 0.25 in the actual segwit proposal.  The pre-segwit behavior is effectively an X of 1.

Since UTXO costs are much more concerning than size on the modern network it might be arguably to set the segwit factor even lower, but the size bloat goes up hyperbolically with a lower factor.  So even though size is less important the trade-off becomes less attractive with values much lower.  The value chosen for the proposal also happens to be just about what is required to get virtually all the capacity gain from this particular restructuring, owing to the typical ration of witness to non-witness data in transactions.

Re: bloated blocks aren't an issue; maybe for today's internet. I worry that everyone is assuming that global internet infrastructure can only improve, based on a Pavlovian observation that it has only ever improved. Specific political tensions are beginning to come together that threaten the trend we know and love, it may pass off withut major incident, but I believe that's dangerously optimistic planning.



I don't entirely understand your point about UTXO set bloating. Surely the UTXO set is defined by confirmed transactions, and so a direct relationship between block size and UTXO set does exist? Are you saying that because the UTXO set is only tx data and not signature data, that a different attack exists, whereby transactions with a high txdata:sigdata needs to mitigated?

I can see how limiting the power to bloat UTXO data would be the solution, but the way that the block weight designation mitigates that is not totally clear....

Also (a broader point) why is UTXO bloating such an issue, you mention that it's more serious than block bloating attacks, but I'm not grasping why. Does limiting tx data to a hard 1MB limit prevent an unmanageable amount of tx data-heavy from DoS'ing available RAM in Bitcoin nodes?