My Segwit questions - a Q&A for Achow and the rest

[Wind_FURY]

I finally found a video that explains it in a way a non-technical person like me can understand. https://www.youtube.com/watch?v=QYZv92F2kCw

But that brings me to a question. Where are extended blocks stored and can we call the information from within it on command? What if I want to see the signature of the sender? Are all the data pulled by Segwit still retrievable?

[1714654786]

1714654786

[1714654786]

1714654786

[1714654786]

1714654786

[1714654786]

1714654786

[1714654786]

1714654786

[1714654786]

1714654786

[achow101]

There are no "extended blocks". Blocks are the same as before; they only contain the block header and the transaction. What changed was the transactions. The transactions have two different formats: legacy, and segwit. When a segwit capable nodes receives a block, it will receive the block with transactions in the segwit format. When a non-segwit node receives a block, it will receive the transactions in the legacy format.

When you run a segwit capable node, using the getrawtransaction or gettransaction commands will give you all of the details of the transaction, including the witnesses.

[Wind_FURY]

Thanks for the reply. But where are the input signatures "stored" and hashed, and how does that "free" space from the blocks to allow more transactions to fit in?

The more I try to learn the more questions I have.

[achow101]

Thanks for the reply. But where are the input signatures "stored" and hashed, and how does that "free" space from the blocks to allow more transactions to fit in?

The more I try to learn the more questions I have.

The signatures are stored as part of the transaction. Because legacy nodes will not see the signatures, there is effectively more free space in the block for more data that those legacy nodes can see (e.g. more transactions).

What actually happened is that the block size limit was redefined to something called block weight. There is now a block weight limit, which is 4000000 weight units. Each byte of non-witness data (everything that a legacy node will see) is worth 4 weight units. Each byte of segwit data (everything that a legacy node won't see) is worth 1 weight unit. So if you use segwit, more bytes of your transaction will be in witness data than a comparable non-segwit transaction. So the weight of your transaction is lower and thus there will be more weight for other transactions in the block. That is where the capacity increase comes from.

[Wind_FURY]

Ok, sorry for all the dumb questions. I believe it is time for me to go back and read more about it. As you have seen, I am not a technical person so the concept is a little hard for me to grasp.

[Wind_FURY]

Hi Achow, did Segwit make the blocks more compact by removing the signatures from the transactions and placing them in an "extended block", and at the same time, maintaining the block size to 1mb?

Is that what it really did?

Another question. Where is the "extended block" stored exactly?

[nullius]

Hi Achow, did Segwit make the blocks more compact by removing the signatures from the transactions and placing them in an "extended block", and at the same time, maintaining the block size to 1mb?

Is that what it really did?

Another question. Where is the "extended block" stored exactly?

achow101 already answered your questions.  Fully.  Did you read his replies to you?

There are no "extended blocks". Blocks are the same as before; [...]

The signatures are stored as part of the transaction.

In other words, there is no “removing” of signatures as you say.  Now to elaborate on this, which you should read carefully:

What actually happened is that the block size limit was redefined to something called block weight. There is now a block weight limit, which is 4000000 weight units. Each byte of non-witness data (everything that a legacy node will see) is worth 4 weight units. Each byte of segwit data (everything that a legacy node won't see) is worth 1 weight unit. So if you use segwit, more bytes of your transaction will be in witness data than a comparable non-segwit transaction. So the weight of your transaction is lower and thus there will be more weight for other transactions in the block. That is where the capacity increase comes from.

Stop thinking in terms of “block size”.  The whole concept of a “block size” is obsolete.  With Segwit, there is no longer a block size limit at all!  Instead, as achow101 already explained to you, there is a block weight limit of 4000000 bytes:

Code:

/** The maximum allowed weight for a block, see BIP 141 (network rule) */
static const unsigned int MAX_BLOCK_WEIGHT = 4000000;

The above-mentioned “weight” calculation is specified by BIP 141:

Blocks are currently limited to 1,000,000 bytes (1MB) total size. We change this restriction as follows:

Block weight is defined as Base size * 3 + Total size. (rationale^[3])

Base size is the block size in bytes with the original transaction serialization without any witness-related data, as seen by a non-upgraded node.

Total size is the block size in bytes with transactions serialized as described in BIP144, including base data and witness data.

The new rule is block weight ≤ 4,000,000.

Aside:

Ok, sorry for all the dumb questions. As you have seen, I am not a technical person so the concept is a little hard for me to grasp.

There is no need for you to grasp it.  If you’re “not a technical person”, don’t even try.  And if you do want to try, then you should start by increasing your basic technical skills—then reading the specs.  Then, you would never find yourself apologizing for “dumb questions”.

There is some sort of absurd postmodern myth that everybody needs to understand everything.  No, you don’t—and you won’t; you never will.  You use technology every day which you have not the slightest hope of ever understanding.  I guarantee that you don’t understand GPS, either; but I’d wager that you trust it to guide you where you want to go.

Segwit is highly technical.  It’s a hack; but it’s a brilliant, elegant hack which makes everything Just Work without sacrificing backwards compatibility (= no hardfork).  The whole Internet is built on such hacks, most of them far less neat and orderly.  Do you want to start brushing up on IPv6 and DNSSEC?  Or for that matter, the long chain of weird kludges applied to SMTP since the protocol was first defined in 1982?

Segwit does not remove signatures; that’s a ridiculous canard peddled by smear campaigners for cheap propaganda purposes.  The engineers did a fine job on this one.  That’s all you really need to know.  If you can’t or won’t read all the specs—if you lack the ability and/or the will to work through the process of gaining significant expertise—then don’t ask questions, especially if you won’t read the answers anyway.  Just sit back, use, and enjoy.

(Note:  I am not affiliated with Core, though I hope to perhaps contribute someday.  I like specs and Unix kernel C code for bedtime reading.)

[DooMAD]

I finally found a video that explains it in a way a non-technical person like me can understand. https://www.youtube.com/watch?v=QYZv92F2kCw

It can't be that great a video if it explains it in a way that gave you the wrong impression.  I'll take a crack at it in the hope it might make more sense. 

There is only one set of blocks.  It's just that the blocks can be read in one of two ways.  Legacy nodes (any non-SegWit ones) can only ever see a maximum of 1MB of the block, however large it actually happens to be.  SegWit nodes, however, can see the total size of the block in its entirety.  SegWit transactions store witness data in the extra space (up to a maximum of 3MB added on top of the base 1MB) that all the legacy nodes can't read, meaning there's slightly more space for transactions in the 1MB base portion.

As an example, you can see with your own eyes plenty of blocks now taking up more than 1MB of space, but for all the nodes that don't support SegWit, they won't be able to recognise or understand the witness portions that take those blocks over 1MB.  They simply don't see it, so all of those blocks still appear to be smaller than 1MB to them, meaning they'll simply ignore everything in the witness space (because they don't even know it's there).

Technically it isn't any smaller, but it still means more transactions can fit into the base 1MB portion of the block, because the legacy nodes can ignore all the SegWit signatures.

[Wind_FURY]

Aside:

Ok, sorry for all the dumb questions. As you have seen, I am not a technical person so the concept is a little hard for me to grasp.

There is no need for you to grasp it.  If you’re “not a technical person”, don’t even try.  And if you do want to try, then you should start by increasing your basic technical skills—then reading the specs.  Then, you would never find yourself apologizing for “dumb questions”.

There is some sort of absurd postmodern myth that everybody needs to understand everything.  No, you don’t—and you won’t; you never will.  You use technology every day which you have not the slightest hope of ever understanding.  I guarantee that you don’t understand GPS, either; but I’d wager that you trust it to guide you where you want to go.

Segwit is highly technical.  It’s a hack; but it’s a brilliant, elegant hack which makes everything Just Work without sacrificing backwards compatibility (= no hardfork).  The whole Internet is built on such hacks, most of them far less neat and orderly.  Do you want to start brushing up on IPv6 and DNSSEC?  Or for that matter, the long chain of weird kludges applied to SMTP since the protocol was first defined in 1982?

Segwit does not remove signatures; that’s a ridiculous canard peddled by smear campaigners for cheap propaganda purposes.  The engineers did a fine job on this one.  That’s all you really need to know.  If you can’t or won’t read all the specs—if you lack the ability and/or the will to work through the process of gaining significant expertise—then don’t ask questions, especially if you won’t read the answers anyway.  Just sit back, use, and enjoy.

(Note:  I am not affiliated with Core, though I hope to perhaps contribute someday.  I like specs and Unix kernel C code for bedtime reading.)

Sorry. I know I am technically slow to you but the least you can do is to encourage the learning process. I also know I do not have to understand what goes on "under the hood". But what can I do? There interest is there.

[Valle]

The original bitcoin was plain and simple - there are blocks, there are transactions in the blocks, that's it. Now there is SegWit and looks like no one understands what does it mean. If all TX info is still in blocks, what was the point of segwit if transactions are still there. If the TX information is outside of the blocks from now on where is the rest?

"You are dumb and don't understand and it's okay" is not an answer - it means that you have no idea what's actually going on.

[nullius]

Sorry. I know I am technically slow to you but the least you can do is to encourage the learning process. I also know I do not have to understand what goes on "under the hood". But what can I do? There interest is there.

I wouldn’t have come down so hard on you, except that you asked again exactly what achow101 had already answered in detail.  Why did you bother asking the first time, if you did not intend to read the answers?

DooMAD’s high-level explanation is well done, if you just want to get the general feel of how Segwit works.

However, speaking only for myself, I’m not interested in “[encouraging] the learning process”.  People who really want to learn, will start by studying everything they can—before they ask questions.  And people who really want to learn, will not be stopped by anything; they don’t need to be encouraged.  I am reminded of an anecdote[1] about a professor who once told a hapless freshman, “You’re not smart enough and you’re not working hard enough.”  I think that is a suitable answer for many questions.  Deserving students will take the hint; and those who don’t have it in them will be saved much trouble.

1. From Philip Greenspun?  I can’t find it now.

The original bitcoin was plain and simple - there are blocks, there are transactions in the blocks, that's it. Now there is SegWit and looks like no one understands what does it mean. If all TX info is still in blocks, what was the point of segwit if transactions are still there. If the TX information is outside of the blocks from now on where is the rest?

"You are dumb and don't understand and it's okay" is not an answer - it means that you have no idea what's actually going on.

That’s an idiotic non-argument.  Tell me, Valle, if you understand the following technologies which you use every day; I guarantee that you will get zero for seven here:

0. The transistor, in its present-day integrated-circuit implementations measured in nanometers

1. Lasers

2. The Global Positioning System (GPS)

3. The secp256k1 elliptic curve cryptography algorithm used in the “original Bitcoin”, and used now

4. The past two decades of research in TCP congestion control algorithms

5. The metallurgy and materials science used to produce modern steels (hint: much different from steels produced 50 years ago!)

6. The virtual memory management code in your operating system’s kernel

The technologies I named above are much more difficult to understand than Segwit (!).  The technologies above are things which I don’t fully understand—I mean, not really, not according to my own standard of “understanding”.  But it is okay for me to not understand them.  It would not be okay for me to pass an opinion about their technical merits, without understanding them.

The fact that I do not understand them, does not mean that scientists and engineers who work with them daily “have no idea what’s actually going on.”

Bitcoin engineering is hard.  Scaling anything is hard.  “Plain and simple” is not a technical argument.  If you are too dumb to understand the technologies involved, then no, you are not entitled to an opinion.  Ill-informed opinions are worse than worthless, since they are nothing more than hot air which wastes the valuable time of people whose opinions actually mean something.

[nullius]

P.S.:

The original bitcoin was plain and simple - there are blocks, there are transactions in the blocks, that's it. Now there is SegWit and looks like no one understands what does it mean. If all TX info is still in blocks, what was the point of segwit if transactions are still there. If the TX information is outside of the blocks from now on where is the rest?

"You are dumb and don't understand and it's okay" is not an answer - it means that you have no idea what's actually going on.

Valle, since the “original bitcoin was plain and simple”, why don’t you write me an “ELI5” on the mathematics of the SHA-256 hash’s “avalanche effect”, and explain why that’s always been so important to Bitcoin’s network security.  That’s much harder to understand than Segwit.  Or if you can’t understand it, does that mean that Adam Back had “no idea what’s actually going on” when he invented HashCash?  And Satoshi was just stumbling blindly when he repurposed the HashCash concept to create a Byzantine fault-tolerant decentralized database?  Oh, puh-leaze tell me that you know what the Byzantine Generals Problem is!  “The original bitcoin was plain and simple”, right?

Bitcoin was never simple.  The fact that you say it was means that you, personally, never had any idea what was actually going on.

P.P.S., Satoshi was a genius.  The shame is that more oft than not, his great opus is as pearls before swine.

[Valle]

The original bitcoin was plain and simple - there are blocks, there are transactions in the blocks, that's it. Now there is SegWit and looks like no one understands what does it mean. If all TX info is still in blocks, what was the point of segwit if transactions are still there. If the TX information is outside of the blocks from now on where is the rest?

"You are dumb and don't understand and it's okay" is not an answer - it means that you have no idea what's actually going on.

That’s an idiotic non-argument.  Tell me, Valle, if you understand the following technologies which you use every day; I guarantee that you will get zero for seven here:

0. The transistor, in its present-day integrated-circuit implementations measured in nanometers
What exactly you don't understand in transistors? It's a school basics, isn't it? Nanometer size transistors are basically same as larger ones but takes less current.

1. Lasers
Physics, like schools grade 9. Simple quantum physics/chemistry what can be explained easily in terms or resonance and basic electron levels.

2. The Global Positioning System (GPS)
This is bit trickier to understand because of relativistic effectsm however most people supposed to understand it in high school physics.

3. The secp256k1 elliptic curve cryptography algorithm used in the “original Bitcoin”, and used now
I'm author of alternative bitcoin client (android paper wallet). I do understand EC.

4. The past two decades of research in TCP congestion control algorithms
Yes... I have a my own TCP-IP implementation to understand it.

5. The metallurgy and materials science used to produce modern steels (hint: much different from steels produced 50 years ago!)
I've been studying in Bauman state university for few years in metallurgy major (dropped it, too boring)

6. The virtual memory management code in your operating system’s kernel
Approximately.

The technologies I named above are much more difficult to understand than Segwit (!).

Cool. It means it's very easy to understand where extended blocks data is stored, right?

[Valle]

nullius, it is very silly of you to think that you are the smartest and if you cannot understand some things then nobody can.

[bakerlisa510]

As per average how many transactions can a 1 MB block hold?

[Valle]

P.S.:

Valle, since the “original bitcoin was plain and simple”, why don’t you write me an “ELI5” on the mathematics of the SHA-256 hash’s “avalanche effect”, and explain why that’s always been so important to Bitcoin’s network security. 

Actually bitcoin security is based 99% on EC, not on hashes. Mining does depend on good hashing to ensure it's hard (and good hashing by definition has the avalanche effect). Hashing in bitcoin protocols is mostly to protect public key and to build chain, but there are a lot of addresses with known public key and it is not such a big problem, see "address reuse".

[nullius]

As per average how many transactions can a 1 MB block hold?

Blocks are not limited to 1MB.  There is no longer any such concept as a block size limit.  Read upthread.

The average number of transactions a block can hold will depend on the transactions’ average weight—an empirical question, depending on what transactions people are actually making on the network.  You may expect the average weight to drop as more people send Segwit transactions.  You may expect it to plummet after Schnorr signatures become available—especially if/when that gets uptake from senders with very large numbers of inputs/outputs, such as exchanges.  Lower average weight is good:  That means more transactions can fit in a block, according to the simple equation from BIP 141 quoted above.


Valle, it was obvious that I meant “understand” on the level of neither high school physics, nor pop-science explanations—nor courses you allegedly dropped out of.  “Understand” in this context means understanding on the level of intelligently discussing implementations.  You’re a fool if based on your ninth-grade physics class, you would try to pass an opinion on engineering questions involved with maintaining and improving the GPS system, or how best to make light lase given a certain particular set of operational requirements.  Oh—and if you can write virtual memory code or a non-toy TCP implementation, I’ll eat my hat.  Serious engineering goes into such things.

But put all that aside.  You have a much more basic problem, one of first-grade reading comprehension:

Cool. It means it's very easy to understand where extended blocks data is stored, right?

Upthread (boldface added):

There are no "extended blocks". Blocks are the same as before; they only contain the block header and the transaction. What changed was the transactions. The transactions have two different formats: legacy, and segwit. When a segwit capable nodes receives a block, it will receive the block with transactions in the segwit format. When a non-segwit node receives a block, it will receive the transactions in the legacy format.

Why do Core people even bother trying to explain in public forums, if this is the response?

Actually bitcoin security is based 99% on EC, not on hashes. Mining does depend on good hashing to ensure it's hard (and good hashing by definition has the avalanche effect). Hashing in bitcoin protocols is mostly to protect public key and to build chain, but there are a lot of addresses with known public key and it is not such a big problem, see "address reuse".

No discussion of Byzantine generals and their security significance, I see.  In Bitcoin, exactly 0% of that problem is solved by EC.  Nor did you catch the subtle relation between avalanche and HashCash.  (I threw that in because it’s a rather abstruse observation, so you couldn’t crib the answer by checking Wikipedia.)  I could hear a whooshing noise as the whole point of my post went over your head.

Really, all problems for a digital currency except for Byzantine generals could be considered solved as of the 1990s.  Try reading the ACM Queue article I linked above, if you want a good overview of what complexity went into your ostensibly “plain and simple” Bitcoin.

[pebwindkraft]

{ethical behavior}
you both do not provide any value to the discussion.
Instead you try to challenge each other with a level of knowledge, which is highly irrelevant to the community, and (if at best) only proves a certain immaturity, developing an arrogance towards "ordinary people", which prevents you to respect others and thus adding value to the discussion.
And insulting language ("being dumb" or "must not interest you") is certainly out of scope of a technical forum like here.
{/ethical behavior}

[nullius]

{ethical behavior}
you both do not provide any value to the discussion.
Instead you try to challenge each other with a level of knowledge, which is highly irrelevant to the community, and (if at best) only proves a certain immaturity, developing an arrogance towards "ordinary people", which prevents you to respect others and thus adding value to the discussion.
And insulting language ("being dumb" or "must not interest you") is certainly out of scope of a technical forum like here.
{/ethical behavior}

pebwindkraft, I respect your desire for an environment of technical discussion.  I wish for the same thing.

Perhaps my patience is worn just a bit thin from people who have an axe to grind against Segwit, and do so solely with arguments from ignorance—in essence, “it’s too complicated because I don’t understand it.”  (I have never actually seen a technical argument against Segwit.  Not one!)  I also have zero respect for people who don’t read the technical answers provided—upthread, on the same page, with citations to specs and source code.  And why yes, I am an elitist who gives short shrift to ill-informed opinions.  I don’t apologize for any of that; and I do think it’s a benefit that I have no reason to be diplomatic toward anybody.

But I’ll bow out of this thread if it’s fanning the flames too much here.

[Valle]

pebwindkraft is right, it became too personal, I'm sorry about that.

However, there is still a common confusion about SegWit - since blocks are the same and still store all the info necessary to validate transactions, then why does it help to make blocks smaller/fit more transactions? Obviously it's related to the witness data in TX which seems to give ability to reuse some information somehow but as the thread author mentioned it's hard to see how does it do that.