Data structure of blocks showing segwit and taproot integration

[deepskydiver]

Hi everyone,

I'm continuing to do more self education because I'm uncomfortable not having an understanding. I feel I've got a good understanding of the data structure of blocks and the transactions within them.
But I'd like to add to that by by documenting for myself how those data structures are changed by both segwit and taproot.
For me, understanding the data is the core of wider understanding of the processes around it.

As an example here are my notes on the transaction data structure.



Does anyone know of something similar containing the updates for segwit and taproot please?

Thank you!

[achow101]

Segwit changed the data structure, taproot does not. This change is specified in BIP 144.

[deepskydiver]

Thanks, though it's not helpful unfortunately. I believe signatures are stored outside the inputs and outputs and the high level diagram on github doesn't explain that or the makeup of the witness data.

[NotATether]

The witness data simply contains the signature and public key + OP_CHECKSIG (usually). Despite looking like one, it is not a script.

As for the input and output structures, they are not modified except that the scriptPubKey inside the inputs no longer contains a script, it now contains an OP_0 indicating witness version 0, followed by an OP_PUSH of the witness program bytes which then must match the public key HASH160 that follows the witness bytes.

It is best described in BIP141 (and less so, but still somewhat) in BIP143.

[pooya87]

I believe signatures are stored outside the inputs and outputs and the high level diagram on github doesn't explain that or the makeup of the witness data.

Witness has nothing to do with outputs, it is like a replacement for signature script which is part of each input and it is stored in a new "field" called witness (the red part in image below). As a result for each input there has to be a witness (just as for each input we have a signature script). For example if we have 3 inputs then we will have 3 witnesses (that is assuming at least one input was spending a SegWit output, otherwise there must not be any witnesses).

The witness data simply contains the signature and public key + OP_CHECKSIG (usually).

Actually OP_CHECKSIG may only be found if you are spending a P2WSH outputs (or the new P2TR outputs) and that is very rare thing to happen since for using a single signature we simply use P2WPKH which only needs a signature and a public key in its witness when being spent; and more complex scripts (eg. If(checksig) else(checkmultisig)) in P2WSH are rare.

the scriptPubKey inside the inputs no longer contains a script,

OP_0 ... is a script itself!

[NotATether]

the scriptPubKey inside the inputs no longer contains a script,

OP_0 ... is a script itself!

Right. I meant to say that the Witness data is not to be interpreted as a script (because that's what BIP141 said too), but that's stored after the txouts, not inside the scriptPubKey, which is just pushing the witness hash on to the stack.

[deepskydiver]

Witness has nothing to do with outputs, it is like a replacement for signature script which is part of each input and it is stored in a new "field" called witness ..

I misunderstood - I thought the output scriptpubkey - most usually a public key was stored in the witness data..?

[pooya87]

I misunderstood - I thought the output scriptpubkey - most usually a public key was stored in the witness data..?

The output can be anything and the scripts in outputs are not even evaluated in a transaction (it is only evaluated when you try to spend that output in another transaction).
The inputs are evaluated and contain the "unlocking mechanism". To spend legacy outputs the "unlocking mechanism" would be placed in signature script whereas to spend SegWit outputs we move the "unlocking mechanism" to a different part of the transaction called witness.
What the "unlocking mechanism" is depends on the type of the pubkey script in the UTXO we are spending. In its simplest form it is a [signature + public key].

[deepskydiver]

The output can be anything and the scripts in outputs are not even evaluated in a transaction (it is only evaluated when you try to spend that output in another transaction).

This is something I wanted to better understand if that's ok.
1. The simplest case is evaluated though isn't it - to establish the address from which the output is available to be spent?
2. In the case of a more complex script surely even then at least the output address(es) must be known or does the recipient need the transaction ID to find it?

[pooya87]

This is something I wanted to better understand if that's ok.
1. The simplest case is evaluated though isn't it - to establish the address from which the output is available to be spent?
2. In the case of a more complex script surely even then at least the output address(es) must be known or does the recipient need the transaction ID to find it?

I feel like there is a little gap in your understanding of Bitcoin.
Bitcoin doesn't work based on balances or spend from addresses. Instead bitcoin works based on what we call UTXOs (unspent transaction outputs).

Each transaction has version, locktime and 2 arrays: an array of inputs and an array of outputs.
The inputs are the UTXOs that you are spending. The outputs are the new UTXOs you are creating.
In other words each time you create a transaction you consume one or more UTXOs and create one or more new UTXOs.

The way we figure what UTXO is being spent in each transaction is by looking at the first 36 bytes of each input which represent a txid and an index.
For example take a look at the first bitcoin transfer. It has 1 input and the first 36 bytes of that input tell us to look up transaction with ID=0437cd7f8525ceed2324359c2d0ba26006d92d856a9c20fa0241106ee5a597c9 and take the first output (at index 0) of it which is this coinbase transaction.
You see so far, there is no scripts or script evaluation yet and it didn't matter what type of scripts were used in this transaction or the UTXO it is spending.

Now that we have the output we can extract the amount and pubkey script then start the evaluation.

You should think of scripts as the "lock and key" duo. The script inside each output create a lock (like the one here) and the script inside each input provides the key to unlock that lock (like the one here). The only thing SegWit does here is to move the "key" to a new place where it is not used in calculation of transaction ID.

I hope this image makes it clearer:

[deepskydiver]

I might have mis-phrased my question. I had the basic understanding of UTXOs, but my question is around interpreting the output.
Specifically: the address(es) holding the UTXOs.
In reading about Taproot and privacy, I don't see how there can be any privacy where the input and output addresses must be explicit for the ledger to be accurate.
So is it ever the case that the address of the an UXTO is unknown, even for a short time? And if not, why is Taproot more private?
Do you know of a reference I can use to better understand this?

Thanks again!

[achow101]

In reading about Taproot and privacy, I don't see how there can be any privacy where the input and output addresses must be explicit for the ledger to be accurate.
So is it ever the case that the address of the an UXTO is unknown, even for a short time? And if not, why is Taproot more private?

No, Taproot does not hide the amounts nor the Taproot output key itself. It is still obviously a Taproot output and the address can be known.

What Taproot does allow for is to hide any scripts. Taproot conveys privacy in that exact scripts do not have to be revealed. Furthermore, Taproot introduces the idea of the cooperative case where all parties involved in a script agree with what to do. In that case, it is possible for a spend of the Taproot output to appear exactly the same as a single key Taproot spend.

A simple example of this is multisigs. Currently, if you have a multisig, when it is spent, it is obviously a multisig. All of the public keys involved can be seen by everyone. The signatures of those that signed call also been publicly viewed. But with Taproot, if a key aggregation scheme like MuSig were used the resulting spend would look just like any single key Taproot spend. This is where privacy is conveyed.

[deepskydiver]

A simple example of this is multisigs. Currently, if you have a multisig, when it is spent, it is obviously a multisig. All of the public keys involved can be seen by everyone. The signatures of those that signed call also been publicly viewed. But with Taproot, if a key aggregation scheme like MuSig were used the resulting spend would look just like any single key Taproot spend. This is where privacy is conveyed.

This is where I'm confused: what advantage is there in hiding the signatures when the addresses have to be visible? Aren't the signatures those corresponding to the input addresses involved?

Edit:
So back to my question a few posts up: I believe from some more reading now that it is the case that a transaction may not have a destination address. Which I didn't expect!
It can have the output parked - in limbo, not at any address but essentially assigned to to a formula involving addresses not visible.
And to get it you must know the TXID. This is something if true that I feel is not well explained in information I could find.

[DannyHamilton]

I believe from some more reading now that it is the case that a transaction may not have a destination address. Which I didn't expect!

Something I notice a lot of people don't expect is that:
There are no addresses in the blockchain.  There are no addresses in transactions. Not with TapRoot, not with SegWit, not with P2SH, not with P2PKH, not in any bitcoin transaction ever. Addresses are something that wallet software does for us humans to make it easier to talk about the transfer of control over value. It's not something that actually exists in the bitcoin protocol.

All bitcoin transactions use output scripts that set up conditions that must be satisfied in order for the output to be spent. Humans and wallets use the abstract concept of an "address" as a shorthand way to describe a specific script.

Without the concept of an "address", if you wanted someone to create a transaction for you, you'd have to say something like:
"Please create an output that can only be spent when a signature is provided which can be validated with the public key that hashes to f54a5851e9372b87810a8e60cdd2e7cfd80b6e31 when first hashed with SHA256 and the result is then hashed with RIPEMD160."

Instead, we say:
"Please send bitcoins to the following address 1PMycacnJaSqwwJqjawXBErnLsZ7RkXUAs"

Those 2 statements each mean the exact same thing.  The first one is what is actually stored in transactions and the blockchain (encoded using the bitcoin script language instead of english.  Wallet software has been standardized to know that when the second statement is made, it needs to create a transaction that is described in the first statement.  Wallet software (and block explorers) have also been standardized to recognize whenever they see any transaction output that matches the description in the first statement, they should display the "address" in the second statement.

If a transaction is created using ANY script that doesn't have a standardized way of converting that script into a single value, then that transaction doesn't have a "destination address". That's because there is no way for most wallets to recognize the script and know what single value to display to any human, and there's no way for any human to provide a single value to the wallet and for it to know exactly how to create that non-standardized script.

On the other hand, ANY script that becomes popular could become a new address type if the majority of wallet software maintainers all come to an agreement on how to represent that script as a single value.

I might have mis-phrased my question. I had the basic understanding of UTXOs, but my question is around interpreting the output.
Specifically: the address(es) holding the UTXOs.

This is where your confusion probably started.

Addresses do not "hold UTXO". Addresses don't even exist in the transactions or the blockchain.
Addresses are just a script re-written in a form designed for easier human consumption.

A UTXO is an unspent transaction output.  Until it is spent, all you need to know about it is its value and how to find it.

To find it, you just need to know that transaction ID, and (since transactions can have more than one output) which output in that transaction it is.

Later, when someone wants to spend that output, they describe it using the transaction ID and an indication of which output from the transaction is being spent (the TXID_INDEX in your attachment at the top of this thread), and the script needed to satisfy the conditions of the original output script.  This is when the original output script needs to be evaluated.  The information from the input script is applied to the information from the output script. If the result of evaluating the two scripts together is satisfactory (according to the rules of the bitcoin protocol), then the output can be spent and the transaction attempting to spend it is valid.  If the result of evaluating the two scripts together is not satisfactory (according to the rules of the bitcoin protocol), then the output cannot be spent and the transaction attempting to spend it is invalid.

In reading about Taproot and privacy, I don't see how there can be any privacy where the input and output addresses must be explicit for the ledger to be accurate.

Again, there are no addresses.  There are only scripts.

One type of script is a P2SH script.  In that type of output, the requestor of the payment is essentially asking:
"Please create an output that can only be spent when a SCRIPT is provided which can hashed to the following value (some hash value), AND that script is considered satisfied and valid under the rules of the bitcoin protocol."

Now, anyone that looks at the output will find that all of the signature requirements are hidden.  The only way to see the signature requirements (if you aren't involved in creating the signature requirements) is then to wait for the output to be spent.  When it is spent, the signature requirements (along with the necessary signatures) will be included in the input script. At that time you can see ALL of the signature requirements, and which ones were met. Sometimes only a subset of signatures are requires (for example a 2-of-3 multisig).  There are 3 different signature requirements, but only 2 of those signature requirements need to be met for the transaction to be valid.  Without TapRoot, all 3 signature requirements need to be included in the input script, otherwise the script won't hash to the value provided in the output that is being spent. Therefore, there is a loss of privacy for the person that did NOT sign, since the script will contain the information that he was a part of the original transaction as one of the potential signers.

With TapRoot, You can structure the scripts such that you ONLY need to reveal information about those that are providing the proof of validity (for example the signatures), and everyone else that was a party to the original transaction, but that aren't participating in the spending of it, remains hidden and unrevealed.

[deepskydiver]

Thanks, that was very helpful.

While I tend to use the term address and public key interchangably, I had indeed seen one reference which misled me by saying literally that it was the address not the public key hash.

My key realisations from reading your explanation and some other sources are:
- bitcoin transactions transfer money not to an address but to a hash with conditions that may be of an public key (as part of a script) or something much more complicated
- I presume that wallet software generates the amount at your address by looking for the hash of your address' public key in P2PKH transaction outputs
- some transactions are completely uninterpretable to anyone reading the blockchain
- with taproot, until the output is spent the keys are not visible at all and after being spend only the keys used are revealed
- if someone paid me using a script with conditions I didn't know about - I would have no way of knowing without the TXID and the conditions

One quick follow up question:

For UTXO awaiting multisig or unmet complex script conditions, is the amount in that TX still considered to be at the output it's being spent from? Or at the script hash?

Thanks again!