My Segwit questions - a Q&A for Achow and the rest

[nullius]

However, there is still a common confusion about SegWit - since blocks are the same and still store all the info necessary to validate transactions, then why does it help to make blocks smaller/fit more transactions? Obviously it's related to the witness data in TX which seems to give ability to reuse some information somehow but as the thread author mentioned it's hard to see how does it do that.

The principal benefit of that particular aspect of Segwit is that it avoids a hardfork.  Hardforks are anathema to a currency which holds huge amounts of real-world value, although there is Bitcoin hardfork research ongoing to plan how to do one with absolutely no mistakes if/when it becomes necessary in the relatively far future.  Increasing the blocksize limit would have required a hardfork—where all nodes must be upgraded on deadline, and any which didn’t upgrade would suddenly break, badly.  But with Segwit, old nodes will continue to see blocks with a 1000000 byte block size limit, whereas new nodes will see blocks with a 4000000 byte block weight limit.  Old nodes can continue to send transactions, their old way; they simply will not know how to validate transactions sent the new way.  It’s ingenious!

Note that Segwit does not make blocks smaller at all.  It raises the block capacity, up toward a theoretical limit of 4000000 bytes.  In practice, best estimates are that with full Segwit adoption, we will get blocks a bit more than 2MB in actual size.  It was aimed to be an approximate doubling of block capacity, which so many people had been requesting.  Research indicated that such an actual size of blocks would be safe for the network and for nodes, but that much larger sizes would not be.

There are other important fixes built into Segwit—most importantly, a fix for tx malleability; but the foregoing summarizes the reasoning behind the part of Segwit which magically raises block capacity without instantly breaking anybody’s use of Bitcoin.

[1714619980]

1714619980

[Valle]

Now segwit makes sense for me, thank you! One more minor question - do I understand correctly that you can fit only more transactions what use segwit addresses, and in case all clients would keep using addresses starting with 1 then the block size limit stays effectively same as it was before softfork?

[Wind_FURY]

The original bitcoin was plain and simple - there are blocks, there are transactions in the blocks, that's it. Now there is SegWit and looks like no one understands what does it mean. If all TX info is still in blocks, what was the point of segwit if transactions are still there. If the TX information is outside of the blocks from now on where is the rest?

"You are dumb and don't understand and it's okay" is not an answer - it means that you have no idea what's actually going on.

That’s an idiotic non-argument.  Tell me, Valle, if you understand the following technologies which you use every day; I guarantee that you will get zero for seven here:

0. The transistor, in its present-day integrated-circuit implementations measured in nanometers
What exactly you don't understand in transistors? It's a school basics, isn't it? Nanometer size transistors are basically same as larger ones but takes less current.

1. Lasers
Physics, like schools grade 9. Simple quantum physics/chemistry what can be explained easily in terms or resonance and basic electron levels.

2. The Global Positioning System (GPS)
This is bit trickier to understand because of relativistic effectsm however most people supposed to understand it in high school physics.

3. The secp256k1 elliptic curve cryptography algorithm used in the “original Bitcoin”, and used now
I'm author of alternative bitcoin client (android paper wallet). I do understand EC.

4. The past two decades of research in TCP congestion control algorithms
Yes... I have a my own TCP-IP implementation to understand it.

5. The metallurgy and materials science used to produce modern steels (hint: much different from steels produced 50 years ago!)
I've been studying in Bauman state university for few years in metallurgy major (dropped it, too boring)

6. The virtual memory management code in your operating system’s kernel
Approximately.

The technologies I named above are much more difficult to understand than Segwit (!).

Cool. It means it's very easy to understand where extended blocks data is stored, right?

I am getting confused here. Are you arguing against yourself? Hahaha.

But thanks for the advice. I will set it upon myself to learn more now and then ask questions later. With that in mind I hope to learn from the both of you, nullius and Valle.

[nullius]

Now segwit makes sense for me, thank you! One more minor question - do I understand correctly that you can fit only more transactions what use segwit addresses, and in case all clients would keep using addresses starting with 1 then the block size limit stays effectively same as it was before softfork?

You’re welcome!  To your question:  Yes, you understand correctly on that point.  Segwit’s benefit to block capacity is proportionate to adoption; and if nobody were to use Segwit addresses and tranasctions, then Segwit would have no effect at all on block capacity.  But fortunately, to the flipside, the Segwit design implicitly incentivizes Segwit use.  Anybody who uses an address starting with a “1” is overpaying (approximately[1]) fourfold on fees, because their transactions “weigh” so much more than Segwit transactions.

Looking at it from the opposite perspective, switching to a Segwit address will bring you (approximately[1]) an instant 75% discount on fees.  I have spent a bit of time on the forum (futilely) trying to inform people who complain about fees that they can get themselves an instant 75% discount, in bold letters.  (Perhaps I should try adding emoji—ick! :-( )  I myself have been using Segwit addresses for months, largely for this reason.  Well, I do like showing off in my signature; and I am a Segwit supporter.  Still, I’ll admit that for actually making the change, the fee difference was an awfully nice incentive!

But it’s not there to be “nice”; and it must be emphasized, what I am here calling a “Segwit discount” is not some arbitrary rule made to push Segwit.  Rather, as I said, it is implicit in the design.  If you use an old-style address, then your bytes are consuming about 4× the block weight; that is, you are selfishly consuming about 4× the amount of a globally shared resource.  This pushes the network toward the position you just described, of still being effectually limited to 1MB blocks.  Whereas if you use Segwit, then byte for byte, you consume about 75% less block weight.  That frees up more space for use by others, increasing the network’s capacity and relieving fee pressure.  Thus it is a natural, logical consequence that you will pay about 75% less.  Fair is fair; and as with all else in Bitcoin, the system is carefully designed to align people’s selfish interests with the common good.

(This leaves two practical issues:  Choice of Segwit address type (nested-in-P2SH vs. Bech32), and actually using Segwit addresses with currently available software.  Earlier, I started writing an exceedingly long post about these issues.  It didn’t get done; and for discussing wallets and usage instructions, this is the wrong forum, anyway.  Generally, here or elsewhere, I would be happy to answer questions in so far as I am able.)

1. The fourfold increase/75% discount is approximate.  As quoted in my previous post, the precise equation for block weight from BIP 141 is “Base size * 3 + Total size”, where “Base size is the block size in bytes with the original transaction serialization without any witness-related data, as seen by a non-upgraded node”, and “Total size is the block size in bytes with transactions serialized as described in BIP144, including base data and witness data.”  That is the equation which miners now must use when selecting transactions and, it is to be hoped, trying to fit as many as they can into a block.  The witness data are by far the biggest part of any ordinary transaction; and when the size thereof doesn’t get multiplied, the difference swamps everything else.  Thus in practice, calling Segwit a “75% discount” will suffice for a rule of thumb.

[Valle]

Awesome! I'll try to implement SegWit in my client.

[Wind_FURY]

Looking at it from the opposite perspective, switching to a Segwit address will bring you (approximately[1]) an instant 75% discount on fees.  I have spent a bit of time on the forum (futilely) trying to inform people who complain about fees that they can get themselves an instant 75% discount, in bold letters.  (Perhaps I should try adding emoji—ick! :-( )  I myself have been using Segwit addresses for months, largely for this reason.  Well, I do like showing off in my signature; and I am a Segwit supporter.  Still, I’ll admit that for actually making the change, the fee difference was an awfully nice incentive!

But it’s not there to be “nice”; and it must be emphasized, what I am here calling a “Segwit discount” is not some arbitrary rule made to push Segwit.  Rather, as I said, it is implicit in the design.  If you use an old-style address, then your bytes are consuming about 4× the block weight; that is, you are selfishly consuming about 4× the amount of a globally shared resource.  This pushes the network toward the position you just described, of still being effectually limited to 1MB blocks.  Whereas if you use Segwit, then byte for byte, you consume about 75% less block weight.  That frees up more space for use by others, increasing the network’s capacity and relieving fee pressure.  Thus it is a natural, logical consequence that you will pay about 75% less.  Fair is fair; and as with all else in Bitcoin, the system is carefully designed to align people’s selfish interests with the common good.

(This leaves two practical issues:  Choice of Segwit address type (nested-in-P2SH vs. Bech32), and actually using Segwit addresses with currently available software.  Earlier, I started writing an exceedingly long post about these issues.  It didn’t get done; and for discussing wallets and usage instructions, this is the wrong forum, anyway.  Generally, here or elsewhere, I would be happy to answer questions in so far as I am able.)

1. The fourfold increase/75% discount is approximate.  As quoted in my previous post, the precise equation for block weight from BIP 141 is “Base size * 3 + Total size”, where “Base size is the block size in bytes with the original transaction serialization without any witness-related data, as seen by a non-upgraded node”, and “Total size is the block size in bytes with transactions serialized as described in BIP144, including base data and witness data.”  That is the equation which miners now must use when selecting transactions and, it is to be hoped, trying to fit as many as they can into a block.  The witness data are by far the biggest part of any ordinary transaction; and when the size thereof doesn’t get multiplied, the difference swamps everything else.  Thus in practice, calling Segwit a “75% discount” will suffice for a rule of thumb.

With all that in mind, is there a desktop wallet that can create Segwit addresses by default? Electrum is sadly a no. I also cannot believe that the largest online wallet provider Blockchain.info has not enabled Segwit. Were they not one of the companies that were complaining that their customers are paying high fees?

I see the hypocrisy here because Blockchain is now enabling Bitcoin Cash in their wallets. But why not Segwit?

Plus I believe we should start campaigning for Segwit usage in the forum. But first I need to understand how it works. I hope you will me patient with me.

[nullius]

Awesome! I'll try to implement SegWit in my client.

Awesome right back at you!  Core provides a concise technical overview of what is needed for implementing different levels of Segwit support, with links to the pertinent BIPs.

When you’re done, if desired, you may submit a GH pull request to add your software project or company to the list of those which have adopted Segwit.

That will be good for your users, and good for the network.  It will also prepare you to add important future upgrades such as Schnorr signatures and MAST, which will depend on support for Segwit’s script versioning system.  Segwit was not only a capacity upgrade; it was a bugfix (tx malleability, etc.) and a means to add forward compatibility.

With all that in mind, is there a desktop wallet that can create Segwit addresses by default? Electrum is sadly a no.

As of version 3.0, released last month, Electrum does support Segwit!  You need to generate a new Electrum wallet for this; and if you select “Segwit wallet”, it will only give you Bech32 addresses.  If you want Bech32 addresses, then your problem is solved.  Bech32 addresses are technically superior to old-style addresses; but they are not backward-compatible, so only people with Segwit support will be able to send you money.  I myself hope to switch to Bech32 in perhaps 6–12 months.  Future viewers of this post will see my signature showing an address which starts with “bc”.

To make Electrum give you a wallet of backward-compatible P2WPKH-nested-in-P2SH “3” addresses like the one you see now in my signature, use the following procedure, which I have tested:  (0) Generate a BIP39 seed by some other means.  I wrote a short C program for this, which I should polish up and post online; BIP 39 itself contains a list of implementations, including the Python reference implementation.  (1) In the new wallet dialogue, tell Electrum that you already have a seed.  (2) In the seed entry dialogue, click the button for more options and tell Electrum you’re using a BIP39 seed.  (3) In the next screen, change the derivation path to m/49'/0'/0'.  The only part which needs to be changed is the “49”.  You may refer to BIP 49 (as to the 49, the “purpose”) and BIP 44 (for the two zeroes, “coin type” and “account”) to confirm that I am not giving you bad instructions which will send your money off into the weeds; I mention that because I’ve seen Internet posts get this part wrong.

I also cannot believe that the largest online wallet provider Blockchain.info has not enabled Segwit. Were they not one of the companies that were complaining that their customers are paying high fees?

I see the hypocrisy here because Blockchain is now enabling Bitcoin Cash in their wallets. But why not Segwit?

Though perhaps some of the major players just don’t yet have their acts together, there is much political motivation behind some of these decisions.  I will take it as conclusive evidence when a company adds forkcoin support without letting their users use Segwit addresses.

Plus I believe we should start campaigning for Segwit usage in the forum. But first I need to understand how it works. I hope you will me patient with me.

Perhaps you can tell, I agree about that campaigning!

I hope that my posts have helped with understanding.

If you check my post history, you may notice that I am ruthless and downright nasty to anybody who seems to have ulterior motives.  Especially anybody who openly promotes a fork coin (n.b. that I just tossed that link because it’s handy; here lies a very deep rabbit hole, one where all the rabbits were killed and eaten by poisonous snakes).  But for decent people who want to use Bitcoin and desire to better understand it, I am happy to help.

[Wind_FURY]

I tried generating a wallet in Electrum 3.0.3 but it does not have an option to choose a "Segwit wallet". All the choices it has is the same as the previous builds.

Plus, you are right. The Bech32 addresses which start with "bc1" are not supported and, I believe, will not be in the near future.

I will also refrain from doing anything to technical to find a Segwit enabled wallet for now. I will exhaust all the easier options first. Thanks for your help, nullius.

[achow101]

I tried generating a wallet in Electrum 3.0.3 but it does not have an option to choose a "Segwit wallet". All the choices it has is the same as the previous builds.

Then you're doing it wrong. When you create the wallet, choose "create a new seed". Then in the next window, you will choose between "Standard" and "Segwit".

Plus, you are right. The Bech32 addresses which start with "bc1" are not supported and, I believe, will not be in the near future.

Electrum supports them, but few other clients do currently.

[Matias]

How do the old nodes see Segwit transactions? If I've understood correctly, they see Segwit transactions as something called "anyone can spend transactions"? What about the witness data? How the old nodes interpreted it? It is written somewhere, that the old nodes don't see the witness data at all, but how is that possible, when the data still is broadcasted in the network?

[DooMAD]

How do the old nodes see Segwit transactions? If I've understood correctly, they see Segwit transactions as something called "anyone can spend transactions"? What about the witness data? How the old nodes interpreted it? It is written somewhere, that the old nodes don't see the witness data at all, but how is that possible, when the data still is broadcasted in the network?

As per my previous post, any nodes not supporting SegWit will only see the "base" weight of the block.  That means they see the transaction details and signatures for non-SegWit transactions, just as they always have.  But they won't see the signatures from all the SegWit transactions, as that's stored in the new "witness" portion of the blocks which they ignore completely.  They don't try to "interpret" what they don't understand.  SegWit nodes see and validate all of it.  And there are enough SegWit supporting nodes for that to work and for the network to function correctly.

[Valle]

As far as I understand older clients see SegWit transactions as zero input/one output transactions due TX format.

[achow101]

As far as I understand older clients see SegWit transactions as zero input/one output transactions due TX format.

No, that is completely incorrect. Older clients will simply not see the witness data (which includes the 0x00 and 0x01 marker and flag bytes) as that data will not be transmitted to them. They will still see that inputs are being spend and outputs are created so that they can update their UTXO sets accordingly.

[Wind_FURY]

Is it possible for the community to do another version of the USAF by declaring that their nodes will stop accepting legacy blocks at a certain date?

Maybe this should happen and put Bitcoin's "oligarchy" on high alert to start using Segwit.

[DooMAD]

Is it possible for the community to do another version of the USAF by declaring that their nodes will stop accepting legacy blocks at a certain date?

It's always possible, I just wouldn't say it's advisable.  Whether it's miners trying to split off without support from nodes and devs, or nodes trying to split off without support from miners and devs (which is exactly what UASF was), it's still going to be a contentious fork.  The previous attempt at UASF was flagrantly duplicitous when most of its support came from people who claimed they want to avoid contentious forks and then suddenly propose a contentious fork.  Plus, while they might call it a "user activated softfork", it invariably results in a hardfork anyway, because there are two incompatible codebases.

If a proposed fork naturally ends up becoming contentious because people simply can't agree, then that can't be avoided.  But we shouldn't be aiming to cause division and controversy from the offset by trying to achieve consensus at the point of a gun, which is UASF's biggest drawback.  It's an act of force and coercion.  We need a healthy mix of support from nodes, miners and devs.  Ideally, all three groups will voluntarily come together and use the codebase that provides the strongest incentives to secure a chain and build upon it.  

People should follow a fork because they want to, not because someone is trying to make them.

[Carlton Banks]

Is it possible for the community to do another version of the USAF by declaring that their nodes will stop accepting legacy blocks at a certain date?

Maybe this should happen and put Bitcoin's "oligarchy" on high alert to start using Segwit.

No, it's impossible to soft fork by changing pre-existing consensus rules. Refusing to accept blocks with non-witness transaction could only be a hard fork, and would need very high (90-95%) user support.


Hard forks so far (Bcash and S2X) have shown that hashrate follows userbase, not the other way around. But it's not impossible to have a cryptocurrency with little discernible userbase (and hence no real economy), Bcash and Ripple being the most significant examples.


For the above reason though, it's possible to use flag-day activation for forks that are soft-forks. This is how many soft-forks were activated in the past, and also how Segwit was activated using BIP91 (non-supporting blocks were orphaned, and so miners had to upgrade to continue getting their blocks accepted).

[zoloq33]

Does the work needed for the POW scale with the new block size or it has nothing to do with it?

[achow101]

Does the work needed for the POW scale with the new block size or it has nothing to do with it?

The PoW and block size are completely independent and have nothing to do with each other.

[zoloq33]

Does the work needed for the POW scale with the new block size or it has nothing to do with it?

The PoW and block size are completely independent and have nothing to do with each other.

So, if I haven't got it all wrong, there is a great incentive for miners to mine Segwit blocks instead of old BTC blocks. There will be many more transactions inside each block, hence the total amount of transaction fees from each block will be much higher. If that's the case, the network would move to Segwit quickly, as people will have to follow the large pools to get their transactions confirmed in a reasonable time or pay substantially high fees.

[DooMAD]

So, if I haven't got it all wrong, there is a great incentive for miners to mine Segwit blocks instead of old BTC blocks. There will be many more transactions inside each block, hence the total amount of transaction fees from each block will be much higher. If that's the case, the network would move to Segwit quickly, as people will have to follow the large pools to get their transactions confirmed in a reasonable time or pay substantially high fees.

The vast majority of miners have already moved to SegWit.  That's how it got activated in the first place.  Nearly all the blocks are over the old 1MB limit now.  The issue is getting more users to utilise SegWit, but support for the new address format has been weaker than anticipated.  One would have thought the potential for lower fees would be enough of an incentive, but adoption is still slow.