Question about SegWit Prunability and Compatibility with Older Bitcoin Versions

[competitormachine]

Hi everyone,

This is my first post in this community.

I've been diving into the Bitcoin SegWit standard and the topic of Ordinals prunability, and I have a couple of questions.

First, I hear that the Witness structure can be pruned, which got me thinking. Even though the Witness data isn’t part of the txid, wouldn’t there be issues when a new node downloads a block with pruned transactions and tries to verify that those transactions are still valid? Wouldn’t that new node reject the block because it doesn’t meet SegWit consensus rules/contract?

Secondly, I’ve been wondering what would happen if people started using old versions of Bitcoin Core that don’t fully support SegWit. For example, these older nodes might accept transactions with invalid SegWit data by default (which means invalid transactions), since they just accept this transactions as ANY can Spend. Could this be a security risk? I know it’s unlikely to happen now, but I’m curious about how this issue was handled when SegWit was first being deployed.

[Findingnemo]

1. The block header still contains the necessary data to verify the block is still valid or not even without the witness data.

2. Older versions see the TXs are anyone can spend but not as invalid and they still follow the consensus rule of the longest valid chain with block headers which doesn't let these nodes to propagate any invalid blocks thereby posing no security to the network.

[competitormachine]

1. The block header still contains the necessary data to verify the block is still valid or not even without the witness data.

2. Older versions see the TXs are anyone can spend but not as invalid and they still follow the consensus rule of the longest valid chain with block headers which doesn't let these nodes to propagate any invalid blocks thereby posing no security to the network.

Thanks Findingnemo but  how it is possible to verify that the transactions within the block still valid.

I agree that once the block is buried in several blocks pointing to it as a root it would mean that validations by multiple honest parties should have been done and due to the proof of work, forgery is not possible because the associated costs

Still if witness data is pruned I would lose the possibility of verify the transactions with the signature and/or check if the signatures has been modified (because this renders the wtxid hash useless) right?

About the second point I suppose that there was a moment where non-segwit nodes/miners where the majority if a invalid segwit transaction was broadcast which one was the mechanism that prevented that transaction to end up in a confirmed block?

[pooya87]

SegWit is probably the only soft-fork in Bitcoin that was introduced terribly in the social media that is still causing confusion to this day.

First, I hear that the Witness structure can be pruned, which got me thinking. Even though the Witness data isn’t part of the txid, wouldn’t there be issues when a new node downloads a block with pruned transactions and tries to verify that those transactions are still valid? Wouldn’t that new node reject the block because it doesn’t meet SegWit consensus rules/contract?

In order for a true Full Node to verify everything, they will need every part of the transaction. You can NOT prune (common term is strip) witness from a transaction and still verify it. And Full Nodes do NOT strip witness from transactions since witness is part and parcel of it.

What you are referring to here is what Full Nodes do for other older no-longer-full-node clients. In order to keep backward compatibility and help old clients (owners are too lazy to upgrade) still sync and catch up, they give these old nodes a stripped block to partially verify. Because these old clients are no longer performing a full verification on blocks they can not be called a Full [Verifying] Node anymore.

Secondly, I’ve been wondering what would happen if people started using old versions of Bitcoin Core that don’t fully support SegWit. For example, these older nodes might accept transactions with invalid SegWit data by default (which means invalid transactions), since they just accept this transactions as ANY can Spend. Could this be a security risk? I know it’s unlikely to happen now, but I’m curious about how this issue was handled when SegWit was first being deployed.

This attack can only happen in a very specific way targeting a very specific victim and it can be easily avoided.
If an attacker spends any SegWit output (without having the key and only using the pre-SegWit rules), they will also have to mine that transaction and create a block that would be invalid.
What happens is that this block will be rejected by all full nodes and only accepted by old (pre-SegWit) nodes.

Here is two problems with this attack:
1. It is extremely costly because the attacker has mined an invalid block meaning they've wasted mining power and missed out on the block reward (that's about $200k) since supermajority of the network rejects that block.
2. The old client accepts this as a valid block but it will soon become a stale chain that can not grow as fast as the actual chain. In Bitcoin all nodes (from day one) always follow the chain with the most work. Meaning if the victim of this attack simply waits for more than one confirmation, they won't get scammed since the malicious block will be rejected by their old node as well, as the actual chain grows and they switch to that valid chain as the chain with the most work.

[NotATether]

First, I hear that the Witness structure can be pruned, which got me thinking. Even though the Witness data isn’t part of the txid, wouldn’t there be issues when a new node downloads a block with pruned transactions and tries to verify that those transactions are still valid? Wouldn’t that new node reject the block because it doesn’t meet SegWit consensus rules/contract?

A Bitcoin Core node from pre-segwit times will consider segwit transactions to be invalid, because of the fact the flag at the beginning of the transaction implies segwit support happens to be at the same place as the number-of-inputs field for a non-segwit transaction, so when the flag is set, it makes the input count zero to a non-segwit node. The reason being, is that Bitcoin Core nodes without segwit interpret transactions slightly differently than newer nodes because there are a bunch of missing fields.

It is a clever design by bitcoin developers for old nodes to ignore all segwit transactions without halting the initial block download. Since the witness data is towards the end of a segwit transaction, the older nodes never get to that part before it marks the transaction as invalid. So it doesn't actually matter whether it's stripped or not.

[vjudeu]

A Bitcoin Core node from pre-segwit times will consider segwit transactions to be invalid

Non-standard? Yes. Invalid? No. It will be stripped, and converted into a non-segwit transaction. For example: https://mempool.space/tx/000000000fdf0c619cd8e0d512c7e2c0da5a5808e60f12f1e0d01522d2986a51

Full transaction data:

Code:

02000000000101c481eef584a17c0fd36a3802f5a7ab972ad06f628ffb57144b865850f9a7fc400200000000feffffff0301000000000000002a6a28f09f98bc2053616372696669636520746f204c61756461212028746f7069633d3532383239313129a0860100000000001600145ab6db66fb0dd08ecb238f07141ffea56189552deeca0600000000001600142a81628ca874ff8ecb9f879a81ae2fa5689850dc02473044022053310e70c23aa279b05cad3527bac28727a4615b9872be4e748cd91a78e5d76c02207d740b8a6904c0d391296249c7773515d947ce8535fc20637c3eb8020eb79f1801210227eb9f073f99f26c8ce69b17aec1bcee7b414ff4bae5b1b3ab0a710b63da62cf48f80900

Segwit hash: 2c7592d7e939b04d07d29de839877d3e50be7813589200ac918cf35ce49c7e9c

Stripped transaction data:

Code:

0200000001c481eef584a17c0fd36a3802f5a7ab972ad06f628ffb57144b865850f9a7fc400200000000feffffff0301000000000000002a6a28f09f98bc2053616372696669636520746f204c61756461212028746f7069633d3532383239313129a0860100000000001600145ab6db66fb0dd08ecb238f07141ffea56189552deeca0600000000001600142a81628ca874ff8ecb9f879a81ae2fa5689850dc48f80900

Transaction hash: 000000000fdf0c619cd8e0d512c7e2c0da5a5808e60f12f1e0d01522d2986a51

As you can see, pre-Segwit node can still refer to 000000000fdf0c619cd8e0d512c7e2c0da5a5808e60f12f1e0d01522d2986a51, and it didn't change since then. Both nodes will use this ID as their "txid". However, double SHA-256 of the full data, including witness, will be included in a coinbase commitment, inside OP_RETURN. Obviously, pre-Segwit node will not verify it. However, both nodes will use the same transaction ID, to refer to this particular transaction.

And also, here you can see, why Segwit fixes malleability: you can add any witness data you want, and transaction ID will stay the same, because it is calculated from stripped version.

So it doesn't actually matter whether it's stripped or not.

It matters, because you can only pass stripped version to the old node. And the code for stripping is still needed, because transaction IDs are still calculated in the old way, so even Segwit nodes refer to transactions by using transaction IDs, and not Segwit hashes.

[competitormachine]

Thank you guys,

Some of the points are now more or less clear.

It is clear that you cannot just strip the witness data and still being able to validate the the segwit txns. In fat if you just remove the data for one of them I understand that the witness root hash commitment will be broken leading all segwit transaction in that block unverifiable under the full consensus.

The main point is that I am studding bitcoin to understand some of the services under layer 2, specifically: counterparty, BRC20 SRC20 and Stamps and I found some comments that some of the txn info could be stripped/prunned, I'm still unsure how the data in the witness or scripts, which is protected by hashing in the Merkle tree (or witness tree) could be pruned without mining all the upper blocks or a hard fork. Unless they mean pruning applies only to new transactions, preventing the creation or transfer of new NFTs

----------

Regarding the compatibility with the old nodes it was just another non related question that jumped in to my mind. The thing is that for a while the majority of the nodes where old nodes that could propagate invalid segwit transactions because they where marked as any could spend and the witness was ignored. I suppose that the protection for this scenario must be what NotATether mention since the inputs for that block is 0 they will not match the commitment that the sum input amounts must be >= sum of outputs amounts. To be honest I manage to find a lot of info about how good or band segwit was but this point is very poorly explained at least in what I can find on internet.

[NotATether]

The main point is that I am studding bitcoin to understand some of the services under layer 2, specifically: counterparty, BRC20 SRC20 and Stamps and I found some comments that some of the txn info could be stripped, I'm still unsure how the data in the witness or scripts, which is protected by hashing in the Merkle tree (or witness tree) could be pruned. Unless they mean pruning applies only to new transactions, preventing the creation or transfer of new NFTs

Stripping apparently is only in the case of segwit transactions where a node removes all of the new segwit fields of a transaction so that its transaction hash becomes the same as the one an old node would see.

There is also a WTXID (witness transaction ID) that is basically the transaction hash including all of the segwit fields. And a separate Merkle tree to go along with it too, that is mainly used by miners when creating a block.

[Reatim]

First, I hear that the Witness structure can be pruned, which got me thinking. Even though the Witness data isn’t part of the txid, wouldn’t there be issues when a new node downloads a block with pruned transactions and tries to verify that those transactions are still valid?

The transaction data is separated from the witness data which allows it to still be validated even with pruned data. It will still try to verify the existing data it has and if it complies to the requirements then it can still be valid

Wouldn’t that new node reject the block because it doesn’t meet SegWit consensus rules/contract?

Consensus rules checks a lot of things such as block structure, appropriate signatures, input-output structure and etc. but it does not include witness data. So as long as the existing data adheres to the consensus no need for the pruned data just to get it validated.

[pooya87]

The main point is that I am studding bitcoin to understand some of the services under layer 2, specifically: counterparty, BRC20 SRC20 and Stamps and I found some comments that some of the txn info could be stripped/prunned, I'm still unsure how the data in the witness or scripts, which is protected by hashing in the Merkle tree (or witness tree) could be pruned without mining all the upper blocks or a hard fork. Unless they mean pruning applies only to new transactions, preventing the creation or transfer of new NFTs

What happens elsewhere is not related to Bitcoin. In Bitcoin the blockchain is immutable meaning anything that was included in it can not change without making it invalid. That is everything, the blocks, the transactions inside blocks, each part of those transactions from version to the locktime.

The thing is that for a while the majority of the nodes where old nodes that could propagate invalid segwit transactions because they where marked as any could spend and the witness was ignored.

Technically they don't relay such transactions by default since they are considered non-standard.

I suppose that the protection for this scenario must be what NotATether mention since the inputs for that block is 0 they will not match the commitment that the sum input amounts must be >= sum of outputs amounts.

What @NotATether explained is entirely different. It is also not about the satoshi amount being transferred, it is about input count.

You see in a transaction containing a witness, there needs to be a 2-byte "flag" (0x0001) right after the version. Basically in SegWit soft-fork this old transaction structure:

Code:

[version][input count][array of inputs][output count][array of outputs][locktime]

turned into this:

Code:

[version][0x0001][input count][array of inputs][output count][array of outputs][array of witnesses][locktime]

So when an old node that doesn't support SegWit tries to decode a transaction containing witness, it reads the 4-byte version and then looks for the input count. The first byte is 0x00 so they consider it as a transaction with 0 inputs. Such a tx is considered invalid and is rejected.
BUT as I said, old nodes don't receive such transactions in first place. When nodes connect to each other, the perform a handshake and in that they share a protocol version. With that the new nodes can know to strip the transaction before sending it to the old node. In the stripped tx, one of the things that gets stripped is this 2-byte flag (the other is i the witness).

[Mia Chloe]

~snip

I think op is  correct on the part about prunning Witness structure however this doesn't affect block verification. When a new node downloads a block with pruned transactions, it can still verify the block's validity using the transaction identifiers (txids) and the Merkle tree. The SEGWIT protocol doesn't actually dispose the witness data rather it only separates them to reduce the space occupied by the transaction in that block.

However when verifying individual transactions the Witness data is necessary. To address this nodes can request the missing Witness data from other nodes on the network using the `getwitnesstx` message. this  ensures that nodes can still verify transactions even if the Witness data is pruned.

Also op should be right that older nodes (that have not implemented SegWit) might accept invalid SegWit transactions as "ANYONE CAN SPEND" transactions. This is kinda known as the "SegWit backward compatibility" issue. To get rid of this risk, the SegWit deployment used a "soft fork" approach which ensured that old nodes would accept SegWit blocks as valid even if they didn't understand the Witness data and new nodes will  enforce SegWit consensus rules and reject invalid transactions.

The buttom Line is that the SIGWIT protocol is backward compatible to avoid any issues from old nodes and miners.

[pooya87]

I think op is  correct on the part about prunning Witness structure however this doesn't affect block verification. When a new node downloads a block with pruned transactions, it can still verify the block's validity using the transaction identifiers (txids) and the Merkle tree.

That is what SPV client can do, it is not longer a full verification. It is also not verifying the "validity" of the block, using merkle tree the client will just make sure the transactions exist in the block (header) they received.

The SEGWIT protocol doesn't actually dispose the witness data rather it only separates them to reduce the space occupied by the transaction in that block.

"Separate" is a misleading word to use. Witness is not separated from the transaction, it is part of it. Look at the code tag I posted above.

using the `getwitnesstx` message.

Where did you get that message from? I can't find in the P2P protocol core uses.

[Mia Chloe]

The SEGWIT protocol doesn't actually dispose the witness data rather it only separates them to reduce the space occupied by the transaction in that block.

"Separate" is a misleading word to use. Witness is not separated from the transaction, it is part of it. Look at the code tag I posted above.

using the `getwitnesstx` message.

Where did you get that message from? I can't find in the P2P protocol core uses.

Thanks for the correction. I that case I believe the best word to make use of is "segregated" just like the meaning of SEGWIT suggests (segregated witness). Anyways, majority of the things I wrote was more like a combination of things I read sometime ago while I was still a newbie here. I can't really remember the exact place I got the 'getwitnesstx' code from but I can still remember that they suggested it was in implemted in an older version of bitcoin core I think 1.13.0 when SEGWIT was first implemented.

Anyways here are some sources where I read from back then :

• https://github.com/bitcoin/bips/blob/master/bip-0144.mediawiki
• https://en.bitcoin.it/wiki/BIP_0144