Malleated Transactions

[AliceWonderMiscreations]

As I understand it, both FlexTrans and SegWit are designed to fix the Transaction Malleability problem.

FlexTrans would require a hardfork.
SegWit does not.

With that difference acknowledged, can anyone explain why SegWit is a superior solution to FlexTrans?

[1714943548]

1714943548

[1714943548]

1714943548

[AliceWonderMiscreations]

Okay - I'll share one of the reasons why I am leaning towards FlexTrans over SegWit but I want to hear why people think SegWit is better.

I've been looking at both past few days and it seems to me that FlexTrans is a lot simpler.

That has many advantages.

When you are coding an application that needs some cryptography and you can do what you need to do with libsodium, is not better to use the libsodium API than the OpenSSL API?

Of course it is, because libsodium is much simpler and therefore it is far less likely that you will make a mistake in the implementation of it.

Wouldn't the same principle hold true with SegWit vs FlexTrans when coding software that deals with people's financial value? I certainly think it would.

That's why I want to know if there is something SegWit really does something that is needed that FlexTrans doesn't do, because looking at them both, I'm quite puzzled as to why SegWit was chosen over FlexTrans.

[achow101]

First of all, Segwit was proposed, implemented, and tested for a very long time before FlexTrans was proposed. By the time FlexTrans was proposed, segwit was practically ready for release. I don't think that those who worked on segwit were just going to suddenly drop segwit and switch to FlexTrans as they would have then wasted months, possibly more than a year, of effort making segwit as good as possible.

Secondly Segwit does more things than FlexTrans does. It allows for script versioning, so future changes to scripts such as the introduction of new OP_CODES and other script types can be easily introduced. This would allow for future improvements like Schnorr signatures which can further reduce the transaction size.

Lastly, I don't think FlexTrans reduces complexity. I think it in fact makes things more complex. With Segwit and standard transactions, there is a fixed order of where everything is and that is fairly easy to code for and deserialize a transaction because you know exactly where everything must be. With FlexTrans, you now have to check for tags which could be in any order, or possibly non-existant. That seems to me to be much more complex to handle as you don't know where everything is.

[AliceWonderMiscreations]

I don't think that those who worked on segwit were just going to suddenly drop segwit and switch to FlexTrans as they would have then wasted months, possibly more than a year, of effort making segwit as good as possible.

I thought the point though was to make Bitcoin as good as possible.

The time invested they can't get back, but the better solution is what should be used.

I'm not claiming FlexTrans is the better solution, just pointing out that it is easy to compromise the quality of your product when you include something because you spent more time on it. Simple elegance often alludes us and it is unwise to embrace the complex just because we didn't see the simpler solution first.

[Carlton Banks]

Isn't this the one where the author says in the BIP

"encoding transactions using XML with super short field names is THE most efficient data encoding possible, I mean, you can't make ANY data smaller than 1 character in text file, am-I-right?!"


I literally pissed myself laughing

[-ck]

Isn't this the one where the author says in the BIP

"encoding transactions using XML with super short field names is THE most efficient data encoding possible, I mean, you can't make ANY data smaller than 1 character in text file, am-I-right?!"


I literally pissed myself laughing

Bah, he wasn't ambitious enough; he should have been aiming for 1 bit.

[AliceWonderMiscreations]

It's called the Internet Compression Algorithm - you can compress the entire Internet into a single bit.

Seriously - I think what happened is when SegWit was developed, it was developed with the design constrain that it must be doable with a soft fork and that is why it became overly complex.

Maybe they should have put some more thought into whether or not that design constraint was justified.

Sometimes instead of only asking if you can do something, you need to first determine whether or not you should do that something. Simpler solutions make it easier for future developers to properly implement, and with a crypto-currency where money can literally vanish because of coding mistake - that is important.

[tomtomtom7]

To prevent a false dichotomy, I would like to note that Christian Decker's BIP 140 (https://github.com/bitcoin/bips/blob/master/bip-0140.mediawiki) also fixes maleability.

In my opinion in better way then both FlexTrans and SegWit.

[achow101]

Seriously - I think what happened is when SegWit was developed, it was developed with the design constrain that it must be doable with a soft fork and that is why it became overly complex.

Segwit was originally developed as a hard fork. It only became a soft fork after luke-jr figured out that the way it was as a hard fork could be fairly trivially done as a soft fork.

Can you please explain how Segwit is more complex than FlexTrans (with specific details about the proposal, not just hand wavy analogies)? I don't think it is more complex, in fact, I think it is simpler than FlexTrans as everything in the transaction is in a known location, you don't need to go searching for tags.

To prevent a false dichotomy, I would like to note that Christian Decker's BIP 140 (https://github.com/bitcoin/bips/blob/master/bip-0140.mediawiki) also fixes maleability.

In my opinion in better way then both FlexTrans and SegWit.

BIP140 does not do as much as Segwit or FlexTrans does.

[AliceWonderMiscreations]

Seriously - I think what happened is when SegWit was developed, it was developed with the design constrain that it must be doable with a soft fork and that is why it became overly complex.

Segwit was originally developed as a hard fork. It only became a soft fork after luke-jr figured out that the way it was as a hard fork could be fairly trivially done as a soft fork.

Can you please explain how Segwit is more complex than FlexTrans (with specific details about the proposal, not just hand wavy analogies)? I don't think it is more complex, in fact, I think it is simpler than FlexTrans as everything in the transaction is in a known location, you don't need to go searching for tags.

To prevent a false dichotomy, I would like to note that Christian Decker's BIP 140 (https://github.com/bitcoin/bips/blob/master/bip-0140.mediawiki) also fixes maleability.

In my opinion in better way then both FlexTrans and SegWit.

BIP140 does not do as much as Segwit or FlexTrans does.

SegWit splits the TX into two parts, to try and keep compatibility with old transactions instead of just using a new version number. It does this by subtly changing the meaning of the fields so that older clients think they are validating the transaction but they really are not, rather than just knowing that they aren't capable of properly validating that transaction version. Specifying a new version number for the transaction lets you be honest with old clients and you can't possibly make the mistake of misinterpreting the context (SegWit or not) resulting in application of the wrong interpretation to the field. Miners who don't upgrade can validate a transaction that is not actually valid because of this segregation. This can result in orphans because a transaction that would be valid if SegWit simply didn't exist is not valid in the context of SegWit, so bugs in detection of the context can result in confirmations that shouldn't confirm yet follow the old rules.

By using a tag based system, FlexTrans allows you to skip fields that are not used in the transaction, simplifying most transactions as you do not have to parse and interpret fields that that are not there. This also makes the transactions smaller (and thus cheaper) than SegWit transactions. A prime example of this is script version. FlexTrans provides for script version, but you don't actually have to use it until there is a new version and you need to specify that new version. SegWit requires every transaction to have a byte specifying the script version even though there presently is only one version. Currently every SegWit transaction wastes a byte.

The tagged structure of FlexTrans makes it easier to extend in the future, just add new tags, which will work just fine with soft forks because old clients will simply ignore any tags they don't support, just like we can use html tags and attributes that some browsers don't support. Just like we often do with JSON APIs - adding new fields that older clients can just ignore so they don't break because we added some new feature.

SegWit works by moving data into the coinbase which really isn't what the coinbase is for. That's a hack, that's not simplicity, it always complexity when you re-appropriate like that.

A tag-based transaction, btw, gets rid of the need to modify the meaning of fields or do hacks like the signatures in coinbase hack. You simply add new fields. Simple Elegance.

Additionally, because of the coinbase hack, SegWit requires an additional merkle-tree, FlexTrans does not.

[gmaxwell]

It does this by subtly changing the meaning of the fields so that older clients think they are validating the transaction but they really are not, rather than just knowing that they aren't capable of properly validating that transaction version. Specifying a new version number for the transaction lets you be honest with old clients and you can't possibly make the mistake of misinterpreting the context (SegWit or not) resulting in application of the wrong interpretation to the field.

This is disinformation that you're peddling here.  Segwit uses an _explicit_ _intentional_ forward compatibility mechanism.  Old nodes _know_ that the segwit transaction is making use of rules they don't understand, which is why they will not relay, mine, or display unconfirmed txn using it in their wallets.

By using a tag based system, FlexTrans allows you to skip fields that are not used in the transaction, simplifying most transactions as you do not have to parse and interpret fields that that are not there.

Go look at the history of ASN1 and XML-- 'tag' based formats, they're freeking disasters in the context of cryptosystems. They result in malleability bugs and all sorts of other disaster cases-- Bitcoin itself has had several vulnerabilities related to ASN1 just being used inside signatures. The extreme complexity of FT is also revealed by the fact that it's authors reference implementation had at least three buffer overflow vulnerabilities...

The design also doesn't actually create the flexibility it claims, since changing the fields requires a hardfork. ... and a hardfork could already have done anything. So there is no reason to have an overly generic structure in advance, it simply leads to vulnerabilities and inefficiency.

This also makes the transactions smaller (and thus cheaper) than SegWit transactions.

Actually, the FT format actually increases the entropy of transactions because the ordering of fields is signature normative by not canonically mandated.  This means that it actually takes more data to make an efficient representation of FT transactions than ordinary and segwit transactions.

SegWit works by moving data into the coinbase which really isn't what the coinbase is for.

Segwit doesn't move any data into the coinbase. Perhaps you mean that it includes a *commitment* (not data) in the coinbase transaction (though not in the coinbase itself)? This is specifically an intended use of the coinbase transaction, which it has been used for since ~2011... and it's a use that Satoshi recommended-- in fact.


Kinda funny how you asked a question but then followed up with a mountain of pretty extreme misinformation.  It's especially amusing to hear you in one breath attack an idea (fields that nodes explicitly ignore) while in the next extol the same concept.  It makes it hard for me to believe that you're really asking this stuff earnestly.

[Rude Boy]

Using a tagged format for a transaction is a one-time hard fork to upgrade the protocol and allow many more changes to be made with much lower impact on the system in the future.

Where SegWit tries to adjust a static memory-format by re-purposing existing fields, Flexible transactions presents a coherent simple design that removes lots of conflicting concepts.

Most importantly, years after Flexible Transactions has been introduced, we can continue to benefit from the tagged system to extend and fix issues we find then we haven't thought of today - using the same, consistent concepts.

The basic idea is to change the transaction to be much more like modern systems like JSON, HTML and XML. It's a 'tag'-based format and has various advantages over the closed binary-blob format.

For instance if you add a new field, much like tags in HTML, your old browser will just ignore that field making it backwards compatible and friendly to future upgrades.

Further advantages:

    Solving the malleability problem becomes trivial.

    We solve the quadratic hashing issue.

    Tag-based systems allow you to skip writing of unused or default values.

    Since we are changing things anyway, we can default to use only var-int encoded data instead of having 3 different types in transactions.

    Adding a new tag later, (for instance ScriptVersion) is easy and doesn't require further changes to the transaction data structure. All old clients can still make sense of all the known data.

    The actual transaction turns out to be about 3% shorter average (calculated over 200K transactions)

    Where SegWit adds a huge amount of technical debt, Flexible Transactions proposal instead amortizes a good chunk of technical debt.

[Manfred Macx]

Using a tagged format for a transaction is a one-time hard fork to upgrade the protocol and allow many more changes to be made with much lower impact on the system in the future.

Where SegWit tries to adjust a static memory-format by re-purposing existing fields, Flexible transactions presents a coherent simple design that removes lots of conflicting concepts.

Most importantly, years after Flexible Transactions has been introduced, we can continue to benefit from the tagged system to extend and fix issues we find then we haven't thought of today - using the same, consistent concepts.

The basic idea is to change the transaction to be much more like modern systems like JSON, HTML and XML. It's a 'tag'-based format and has various advantages over the closed binary-blob format.

For instance if you add a new field, much like tags in HTML, your old browser will just ignore that field making it backwards compatible and friendly to future upgrades.

Further advantages:

    Solving the malleability problem becomes trivial.

    We solve the quadratic hashing issue.

    Tag-based systems allow you to skip writing of unused or default values.

    Since we are changing things anyway, we can default to use only var-int encoded data instead of having 3 different types in transactions.

    Adding a new tag later, (for instance ScriptVersion) is easy and doesn't require further changes to the transaction data structure. All old clients can still make sense of all the known data.

    The actual transaction turns out to be about 3% shorter average (calculated over 200K transactions)

    Where SegWit adds a huge amount of technical debt, Flexible Transactions proposal instead amortizes a good chunk of technical debt.

Did you just copy Alice's reply without addressing any of the issues gmaxwell brought up?

I've tried reading FlexTrans whitepaper but I found it confusing. Where does it fix quadratic dependency issue with signature hashing?

[Carlton Banks]

Did you just copy Alice's reply without addressing any of the issues gmaxwell brought up?

I've tried reading FlexTrans whitepaper but I found it confusing. Where does it fix quadratic dependency issue with signature hashing?

This is entirely their strategy

1. Invent falsehood
2. Spread the falsehood
3. Repeat it, relentlessly
4. Don't respond directly to criticisms that expose the falsehood, change the subject instead
5. Repeat falsehood often enough that it drowns out exposition of the falsehood

Oh, and I forgot

6. If anyone becomes understandably irate with their behaviour after N repetitions of 3-5, point the finger: "See? See how unreasonable they are, they know they've lost the argument when they lose their temper"

It should be "You know you've lost the argument when complex lies and tricks are all you can do to 'win' "

[gmaxwell]

Using a tagged format for a transaction is a one-time hard fork to upgrade the protocol and allow many more changes to be made with much lower impact on the system in the future

Nope, changes to the tags require another hardfork. So much for avoiding impact.

Where SegWit tries to adjust a static memory-format by re-purposing existing fields,

Sewit doesn't repurpose any fields, it adds fields. The transaction format isn't static.

Tagged binary formats are a repeated source of vulnerabilities, as I mentioned the one that was being used in Bitcoin-- ASN1 has caused problems even for our very narrow use of it.

I won't elaborate more because I said much of it above and all you did was simply repeat the same claims without responding to any of it.

   Where SegWit adds a huge amount of technical debt, Flexible Transactions proposal instead amortizes a good chunk of technical debt.

This is just an outright lie-- go ask any actual Bitcoin developer: Segwit makes Bitcoin much easier to use. While FT adds a pile of complexity that results in vulnerabilities (including cryptographic compromises), and has _already_ done so in Zander's own codebase.

6. If anyone becomes understandably irate with their behaviour after N repetitions of 3-5, point the finger: "See? See how unreasonable they are, they know they've lost the argument when they lose their temper"
It should be "You know you've lost the argument when complex lies and tricks are all you can do to 'win' "

Yea, it's the same crud over and over again over a ever shifting sea of nyms-- some of them probably socks, other people just weaponized innocents.  The end result is that the people that get to deal with the disinformation and are working to defend Bitcoin from it learn to recognize the repeated arguments at their first sentence and cut it off curtly -- and then you get a pile of out-of-context links  "Oh look how mean they are! that was unjustified." -- yea, unjustified unless you also saw the same lies being repeated over and over again for months.

[buddhamangler]

So your argument against a flexible format is because it requires a hardfork to change existing fields?  Well sure, if you change a fields meaning completely then I'm sure it would be a hard fork, that has no relation whatsoever to the benefits of a flexible format.  Flex Trans benefit was it is easy to ADD fields as soft forks and OMIT them if they are not necessary.  I mean hell, why don't we switch the www to hard coded binary formats, i mean we can just switch to the new format when we need to!  No forward thinking required!

The design also doesn't actually create the flexibility it claims, since changing the fields requires a hardfork. ... and a hardfork could already have done anything. So there is no reason to have an overly generic structure in advance, it simply leads to vulnerabilities and inefficiency.

[gmaxwell]

So your argument against a flexible format is because it requires a hardfork to change existing fields?

You've been scammed. "flextrans" _ISNT FLEXIBLE_.   Bitcoin can add fields fine, even without a hardfork: thats exactly what segwit does, for example-- it adds witness fields to every transaction!

Flextrans is less flexible and less efficient, doesn't completely fix malleability, and incorporates security error prone design features (like non-canonical encodings).  The only thing 'flexible' about is is the ethics of the person who gave it a deceptive name.

why don't we switch the www to hard coded binary formats,

The web is not a global consensus flooding network, among other reasons.  And in most fields XML is considered to be a massive failure.  Talking about making Bitcoin transactions in the network XML (with bonus proprietary binary packing method like EBML) would have been a great april first joke if classic hadn't poe's lawed this.

Also, classic is now a Bcash client... so I think this thread might now be more ontopic for the altcoins subforum.