Bitcoin Forum

Hello,

I'm trying to understand the workings of bitcoin on the technical side, and I have a few questions which are probably beginners questions as they must have been extensively treated "back then".   So I put this in the "beginners" forum.  I appologize if this is not the right place, I hope a moderator will tell me where such things should be discussed then.

The first is the softforks regarding bip30 and bip34.
I understand that the coinbase transaction, given that it doesn't need any real inputs, can (could) always be exactly the same if the miner decided so (with the same output address, and the same sigscript), and so would result in the same hash, hence the same TXID.  However, I don't understand the argument that this would allow for double spends.  My idea was rather that the second coinbase transaction with the same TXID would rather be LOST if ever there was a transaction using this TXID as an input, because it would be considered "spent".  I thought an output was considered "spent" if its TXID appeared somewhere in a future input of a transaction.  What was the mechanism of "double spending" that was corrected with bip30 and bip34 ?

TXID is all zeros but there is a height of block in each Coinbase transaction. So two Coinbase transactions for two different blocks are different with each other.

Learn more about Coinbase transactions.
https://learnmeabitcoin.com/technical/mining/coinbase-transaction/
https://github.com/focusgrowth/Mastering-Bitcoin-/blob/develop/ch06_transactions.adoc#coinbase-transactions

Having to include the height of the block is exactly what bip34 was about IN ORDER to have the TXID of the coinbase transactions different.  It is argued that bip30 and bip34 were introduced to avoid the possibility of a double spend of the coinbase transaction OUTPUT.  But my question is: how was that possible, even before bip30 and bip34 ?  I'm of course not talking about the empty TXID of the coinbase transaction INPUT itself, but of the TXID of the coinbase transaction as a whole, that will be used to spend its output.  It is THIS txid (of the coinbase transaction as a whole) that could be the same for different coinbase transactions in different blocks by the same miner before bip34, and it was claimed that the fact that there were identical outputs with same TXID could lead to double spend.  THIS claim is what I don't understand because in my understanding, it would rather have led to LOSS of coins, not of double-spend.

In other words: how could the fact that there were transactions with identical TXID lead to double spends, and not, as I understand it, to loss of one of them ?  Because in my understanding, once a TXID is used as an input, it is considered spent, so if you spend the first of your identical outputs, you cannot spend your second one, because its (identical) TXID has already been put in an input and should hence be considered spent even if it isn't for real.


I used that site amongst others, thanks :-)

I don't think it's appropriate to use the term "double-spend" here.

According to BIP30, blocks can't include a transaction with the same hash as a not-fully-spent transaction.
Before implementation of BIP30, blocks could include a transaction with the same ID as an exsiting transaction, even if not all outputs of that existing transaction had been spent. This bug made the coins generated by blocks 91722 and 91812 unspendable because in the UTXO set, they were replaced by the coins generated by blocks 91880 and 91842 respectively.

According to BIP34 (which was introduced some time after BIP30), the scriptsig of coinbase transactions must contain the block height. This prevents coinbase transactions from being identical to each other and therefore in practice it's no longer possible that two coinbase transactions have the same transaction ID.

Ah, OK, this is also how I understood it: these identical TXID render some "honest" UTXO unspendable, there's no way to turn this into a double spend vulnerability, as I thought.

I had read this here:

https://www.learnbitcoin.com/glossary/bip-30


I wondered what could have been these double spend attacks...

Thanks for the clarification !

---

Ok, next question:

a hard fork is defined as a change in protocol so that what is allowed under the new protocol is not always allowed under the old (in other words, that the new protocol will sometimes produce "illegal" things when seen under the old protocol).

a soft fork is defined as a more restrictive change in protocol, that is to say, what was allowed under the old protocol will not anymore be allowed in all circumstances under the new protocol, but everything that is allowed under the new protocol is also allowed under the old.  In other words, things have become illegal under the new protocol.

This is how I understand these terms.

Now it is claimed that Seggregated Witness is a soft fork, but as far as I understand, before, there was a hard limit on the physical block size of 1 MB.  After segwit, there was a virtual limit of 4 MvB, where old-type bytes have a weight of 4, and new-type bytes a weight of 1, which makes that blocks entirely consisting of old-type bytes would still be limited to 1 MB of physical bytes, but theoretically, new-type byte blocks could go to 4 MB physical.

I don't understand how this is a soft fork.  Any block with old and new bytes would be potentially bigger than 1 MB physical (and they are, today), so how could this still be accepted by the old protocol (as a soft fork demands) ?

How can new blocks, being bigger than 1 MB physical, still be acceptable by the old protocol ?

So where am I wrong ?

SegWit node would strip all witness data before sending TX or block to non SegWit node. After it's stripped, the block size is never higher than 1 MB (not virtual MB).


The actual limit is 4 million weight unit, which is equal to 1 virtual MB.


Wrong, witness and non-witness data used to calculate in final size. See https://learnmeabitcoin.com/technical/upgrades/segregated-witness/#transaction-size-calculation (https://learnmeabitcoin.com/technical/upgrades/segregated-witness/#transaction-size-calculation).

I do understand that if you leave out the new bytes, the total amount of old bytes will indeed physically be below the old 1 MB physical limit.  But I thought that leaving out stuff from a block screws up the hashes (exactly because one wants every byte to be locked in forever in the block chain, that was the fundamental idea). 

So if I understand this correctly, the segwit data are not part of what is hashed into the merkle tree then ?  You can leave all the segwit data out of the block and obtain the same block header hash ?  Segwit data is then in fact "out of the block auxilliary and up to a point, optional data" for the old protocol ?

Short answer: You're right. Merkle root use TXID, where witness data isn't included to get the hash/TXID. But AFAIK node, wallet and other software that doesn't support SegWit never receive witness data.

Thanks for the explanation !

How would an old node see an output that is meant to be unlocked with a segwit script ?  I guess to the old node, this comes down to a scriptpubkey script that is unsolvable as it just pushes data on the stack at the end and will never finish with a single OP_1 on the stack.  So I guess that segwit-meant outputs are, to old nodes, essentially "lost coins forever" ?

However, how would an old node react to the spending of a segwit output as an input ?  As it doesn't have the segwit data nor understands the segwit signature, this input with reference to an "unspendable" output (see before) would always be considered invalid, no ?  And hence, a block containing such a transaction would be seen as invalid ?

How does an old node validate a block with a segwit transaction ?  Well, I mean, it can't validate the "segwit transaction" because it doesn't have the data or understand the protocol, but how can what to a new node is a segwit transaction, be validated somehow as a legit piece of transaction data in a block by the old node ?

Actually BIP30 and BIP34 fixed a bug where a miner could recreate a coinbase with the same TXID as an older one, which would overwrite the original transaction in the UTXO set what this actually meant is that people could accept and try to spend the old output but the miner could later replace it with a new one they control.

From an angle you can see it as effectively letting them reclaim or double spend that TXID. BIP30 banned creating a TXID that already exists with unspent outputs and BIP34 made coinbase TXIDs unique by actually forcing the block height into the coinbase.

Ah, it was hence actually a problem with the TXID database and not with the blockchain.
But let us assume that the new transaction "invalidates" the old one.
Now let me try to understand the motivation for that.  If it were just to "reclaim" the old coins in the former (coinbase) transaction child, there's no point, because the new coinbase transaction would provide you with exactly the same amount (unless after a halving eventually).  If you're going to "trick" the system in undoing an old transaction because you spent, say, 25 BTC, and you want to undo the spending by sacrifying a new coinbase transaction that would otherwise, if you did it correctly, bring in 25 extra BTC, there's no gain.  The only thing you can obtain is to piss off the receiver of the old transaction at best.
That said, in fact there's no reason to accept the MOST RECENT transaction as being associated with that TXID, and to consider that a once valid transaction, mined into the block chain, would suddenly become invalid (and hence make the entire block chain from that point onward invalid if we're strict because there's a transaction in it that is considered erroneous).  In my opinion, that was not a protocol error, but rather a database construction error, database which doesn't really take part in the protocol, because it is simply a tool to apply the bitcoin protocol faster (by keeping track of the unspent transaction ID and not having to run over the entire blockchain each time).  That said, I can understand that it was problematic to have outputs that are indistinguishable.

So it was rather a bug in the code, and not a weakness in the protocol if I understand it well.  The old transaction should prevail over the new one, and not vice versa, because the old one 1) was valid at a point and 2) was integrated in the block chain.  Its status shouldn't be changed afterwards.

SegWit have so many small or odd technical details, so i may forget to mention some detail.

But newly created SegWit UTXO would be converted into anyone-can-spend UTXO before it's send to old node. When a TX spend SegWit UTXO, old node only see someone spend converted into anyone-can-spend UTXO. It means old node doesn't fully verify TX or block that contain SegWit.

Thanks.  Actually I'm indeed essentially trying to wrap my head around segwit (and taproot next).  I was interested in the technical side of bitcoin around 2012-2013, then left it for what it was (had other priorities in life).  And now I come back and want to update myself with the changes to bitcoin and also to get deeper into what I was maybe not understanding fully back then.  Hence my "obsession" to understand how an old node would see things, and then understand what changed, given that bitcoin is supposed to never have undergone a hard fork (at least since it is famous).

I also thought that the obvious thing to do was to have it seen as a spend-to-all output, but the fact that there is an OP-0 followed by a push of the key hash makes me think that this is rather seen as an unspendable coin, and this cannot be modified for an old node, because - at least as far as I understand - the OUTPUT pub key script is not in the segwit field but in the "old" data field, that is part of the TXID that in itself is part of the Merkle tree.
Now - as far as I understand - there is no signature script that you can put before this OP_0 + pushbytes <key hash> that ends with a stack containing just "1", which is what you need to unlock an output.  
So in my idea, an old node would consider this output as unspendable.  This is indeed weird, because it would have made more sense to make it spendable-to-all (given that the more severe segwit softfork will not allow this).  I don't understand this but I don't understand how the OP_0 <key hash> can be interpreted as spendable-to-all by an old node either, or how it can be left out (a null pubkey script is of course always spendable to all: it is sufficient for the sigscript to simply push "1" on the stack).  All this depends of course on what I understood, that the scriptpubkeys of the outputs, also with segwit, is part of the "old" transaction fields and not of the segwit field.

At least that's how I understand https://learnmeabitcoin.com/technical/script/p2wpkh/

It's true in a degree since old nodes would see it as "non-standard".
There are "consensus" and "standard" rules; the former is required to follow by all nodes while the latter is per-client preference (the majority's preference is the standard).

As you know it, since it doesn't have a OP_CHECKSIG like a typical P2PKH ScriptSig it will just see a script that doesn't require a signature to spend which doesn't follow the standards but consensus valid.
So a typical non-upgraded client may be able create a transaction that spends it, but it wont be able to relay it to other nodes that follow the standards (both pre-SegWit and SegWit nodes for different reasons),
It has to be included to a block since it's still valid for pre-Segwit Nodes, just non-standard.
But since the Bitcoin network has more SegWit nodes, upgraded nodes wont accept that block since a transaction in it doesn't pass the "special witness validation" that is demonstrated in your link.
And it wont propagate even if he select those few non-SegWit nodes as his peers.

Ah, we might be hitting something I fundamentally misunderstood in bitcoin here.  I'd like to understand because clearly there's an inconsistency in how I understand things, and I can't put my finger on it, so this might be the thing I'm missing (and is somehow why I opened this thread in the first place).

By non-standard, I *thought* one meant a "basement-made" script that isn't one of the standard scripts like P2PKH or the like, but I *thought* that the universal mechanism was always the same:
the output has an output script which is called the "scriptpubkey", and the input that wants to spend it has a script which we call "scriptsig".  For the standard transaction methods, like P2PKH and the like, the form of these scripts is, well, standard (and for Segwit transactions, they are even absent because Segwit doesn't use explicit scripting any more) ; but of course one is "free" to write the scriptpubkey and the scriptsig one wants, and that is "non standard".
(this is how I understand "non standard").

Now I understand that different nodes can propagate different transactions, as long as they are not MINED.  But I *thought* that the blockchain was UNIQUE, and that once a transaction made it into the blockchain, there's no talking anymore about whether a node transmits it or not, because by the nature itself of the blockchain, there cannot be alternatives.  It is the entire concept of the blockchain that it is a unique ledger of which no alternative versions can circulate long time.  So I'm not talking about the transactions in the mem pool or those that are transmitted on the network, but about the stuff that has been mined into the block chain.

Now, I also understood (thanks to this thread) that since segwit, there's an "optional appendix" to the blockchain, which are the witness fields, and that old nodes actually might have the blockchain pruned of these extra fields which they don't regard.  So there are in fact 2 "versions" of the block chain, one with, and one without witness "fat", but it is nevertheless the same backbone block chain, with the same blocks, and the same transactions, only without the witness fields.  

So to come back to this non-standard: I *thought* that whatever standard or non-standard scripts, in ALL cases, a transaction can only be valid if all of its inputs satisfy that:
1) the referred output in a given input exists on the blockchain before it (unless it is a coinbase transaction)
2) the referred output has not yet been referred to by a previous input in a previous transaction on the blockchain

==> in other words, the input refers to an UTXO

and, here it comes:

3) if you put the input <script sig> and concatenate it with the referred output <script pub key>  you obtain a full script, that *when you execute it, you obtain at the end a stack simply containing "1" and nothing else*.

==> in other words, the input has the valid autorisation to spend the output.

4) a few supplementary rules such as eventual lockout times and so on if applicable

I *thought* that in order to accept a block on the block chain, for all nodes (old and new), it needed:

1) to have a valid header (that is, it refers to a previous valid block to which it is the successor, it has the right Merkle root of all transactions, it has a right hash, and this hash reflects the difficulty level of that block, and it has an acceptable time stamp)


2) every transaction in the block is validated as mentionned above (with special rules for the coinbase transaction).

And if I put all this of my fundamental (mis?)understandings together, I have a problem which I mentionned before:

In my understanding, if you concatenate the <script sig> of a segwit transaction INPUT which is EMPTY (because it resides in special form in the witness field), with the <script pub key> of the previous segwit transaction output (which essentially pushes the key hash on the stack), you DO NOT END UP with a stack simply containing "1" and nothing else, so in my understanding, this transaction should be considered INVALID.  And hence a block containing such a transaction should be rejected, and hence the entire block chain after this block.  

To me, the OPCHECKSIG was simply a way to obtain that "1" on the stack because it is an operator that takes (in reverse polish notation like the old HP calculators) 2 elements off the stack, namely a public key and the signature, and checks whether that signature was produced with the given public key: if so, it pushes a "1" on the stack, if not, it pushes a "0" on the stack.  So this is, in my understanding, just the way to obtain our stack with a final "1" in it and nothing else.

This is where I get stuck.

Yes, old nodes will not be able to process it into "1" but it's not empty either.
Since it's not zero, not negative nor empty, it'll still consider it as true; and since it wont ask for a signature, it's considered as anyone-can-spend.
Unless in a very unlikely event that the scriptSig somehow resulted with "0".
This is what SegWit utilized to be able to implement is as a Soft Fork as described in BIP141: /bips/blob/master/bip-0141.mediawiki (https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki#new-script-system)

BTW, the rule regarding the stack should be "clean" after execution is a "policy" (standard) not "consensus" rule.

Aaah !  It is the C-type "true", that is "not zero".
So a scriptpubkey that simply pushes something that is not zero on the stack is an anyone-can-spend output.

Right.  THAT was the missing piece in my understanding.  Thanks a lot.


Right !  I thought it was a hard consensus rule.