SegWit transaction speeds

[crypto_curious]

Hello,

I have this transaction:
edited, not required anymore

It's Bech32 to Bech32, from Electrum 3.3.6 (with SegWit wallet) to Bitcoin Core. It was expensive like normal transaction, minimum fee I could pay was 48.3 sat/byte.

I thought, SegWit transctions will fit in their own dedicated space, without waiting for space in normal block space, and supposed to be cheap.
Yet this transaction have just skipped one block without confirming (block has 1,248,049 bytes), electrum is saying "Position in mempool: 1.93MB 2.82MB from tip". (EDIT: it's falling in the queue due to low fee I think, certainly this transaction wants to squeeze into normal block space, I don't know why)

I thought it will be sent on first block, as there is plenty of SegWit space?

What I did wrong, and what I misunderstand here, please?

[pooya87]

there is no such thing as "SegWit space" or a "separate/dedicated space" for SegWit transactions. there is only one block that has a limited 4 MB weight that can contain all types of transactions in it.

the reason why SegWit transactions are "cheaper" than legacy ones is because fees (ever since August 2017 when the Segregated Witness fork happened) are no longer calculated based on size but instead based on weight. and a SegWit tx has a lower weight compared to a legacy transaction so you pay less fee.

[crypto_curious]

there is no such thing as "SegWit space" or a "separate/dedicated space" for SegWit transactions. there is only one block that has a limited 4 MB weight that can contain all types of transactions in it.

the reason why SegWit transactions are "cheaper" than legacy ones is because fees (ever since August 2017 when the Segregated Witness fork happened) are no longer calculated based on size but instead based on weight. and a SegWit tx has a lower weight compared to a legacy transaction so you pay less fee.

Thanks. I am reading now trying to figure out how it works exactly. I understand I will be paying ~41% smaller fee on average. But I thought these transactions would be confirmed faster or something. My error I guess.

[mocacinno]

I agree with pooya87 but wanted to give a less technical explanation. It's less correct, and it contains some analogies, i sacrified technical correctness for simplicity.

The blocksize used to be limited to 1Mb since a very long time, with the introduction of segwit the blocksize suddenly became 4 Mb... But, since segwit is a soft fork, older nodes should still be able to receive 1 Mb blocks (strange stuff, isn't it ).

How is this solved? The "actual" transaction data contained in 1 block is still limited to 1Mb, so all the "important" data of both segwit and non-segwit transactions should still fit in this 1 Mb space. However, segwit introduced a smart "trick" (for lack of a better wording). It removed the witness data from the transaction. This witness data can be stored in the "extra" 3 Mb space, making the rest of the transaction that has to fit in the 1Mb block a lot smaller. Since the fee is calculated in satoshi/byte, it's logical that the smaller the portion of the transaction that has to fit into the 1Mb block, the lower the fee should be.

But, like pooya87 already said: there is no "segwit space", a big chunk of the segwit transaction still has to compete with both segwit and non segwit transactions to find a space in the 1Mb block... A segwit transaction just takes up less space in the 1Mb block because part of the transaction can actually be placed outside the 1Mb block.

"Newer" nodes receive the full 4Mb block (usually smaller), "legacy" nodes only receive the first 1 Mb containing all the important information of every transaction. A legacy node will receive less information about the segwit transaction, but since the rest of the network's non-legacy nodes received the full 4Mb block, the witness data is still known to the biggest part of the network.

[crypto_curious]

Thank you! That's amazing explanation and I understood it straight away! 

So, older nodes will not be able to process and "see" SegWit transactions correctly? As they only see a fraction of these transactions, only part which fitted in first 1MB? They just ignore them?
Is there any plan to retire legacy nodes and implement some form of new data organization in the blocks? Or it's going to stay like that for foreseeable future?

[crypto_curious]

Tx is still unconfirmed, my electrum is saying:

Code:

Status: Unconfirmed
Position in mempool: 2.86 MB from tip
Size: 209 bytes
Fee: 0.00010095 BTC  ( 48.3 sat/byte ) 

Can I determine somehow, how much of that 209 bytes is normal data, and how much is Witness data (which fits above 1MB?)

[mocacinno]

Thank you! That's amazing explanation and I understood it straight away!  

So, older nodes will not be able to process and "see" SegWit transactions correctly? As they only see a fraction of these transactions, only part which fitted in first 1MB? They just ignore them?
Is there any plan to retire legacy nodes and implement some form of new data organization in the blocks? Or it's going to stay like that for foreseeable future?

Well... I'm going to simplify even further here, so don't start looking into the actual theory and then come back to this thread to point out any inconsistencies i made

Basically, from the view of a legacy node, a segwit transaction is using an unspent output that can be spent by everybody as input. Since everybody can spend the unspent output used as an input for the tx, the witness data (containing the signature) isn't needed, but the transaction still has an input and an output section, and for a legacy node, the transaction is valid (it uses unspent outputs as an input and creates new outputs).

A "newer" node also receives the witness data (stored outside the first 1Mb), and knows that the unspent outputs used as an input for the transaction can't be spent by anybody... The "newer" nodes have the witness data, so they're able to actually check the signatures annd verify if the transaction is actually valid.

I can't really answer the last part of your question... Segwit is a soft fork, so normally there would be no pressure on old nodes to upgrade... However, nobody knows if there'll be a hard fork in the future that actually builds on (some of) the building blocks introduced by the segwit softfork... If such a hard fork would ever occur, the "legacy" nodes would probably have to upgrade if they wanted to stay part of the main chain.

Tx is still unconfirmed, my electrum is saying:

Code:

Status: Unconfirmed
Position in mempool: 2.86 MB from tip
Size: 209 bytes
Fee: 0.00010095 BTC  ( 48.3 sat/byte ) 

Can I determine somehow, how much of that 209 bytes is normal data, and how much is Witness data (which fits above 1MB?)

Yes, you can... You can save the transaction from electrum, open it with a text editor and copy the hex data... Then use one of the many online tools to decode the transaction... You'll be shown the size and vsize... vsize = virtual size, it takes the segwit data into consideration.
Each byte that should find a place in the first 1 Mb counts for 4 vbytes, each byte outside the first 1Mb count for 1 vbyte

[crypto_curious]

Thank you! That's amazing explanation and I understood it straight away!  

So, older nodes will not be able to process and "see" SegWit transactions correctly? As they only see a fraction of these transactions, only part which fitted in first 1MB? They just ignore them?
Is there any plan to retire legacy nodes and implement some form of new data organization in the blocks? Or it's going to stay like that for foreseeable future?

Well... I'm going to simplify even further here, so don't start looking into the actual theory and then come back to this thread to point out any inconsistencies i made

Basically, from the view of a legacy node, a segwit transaction is using an unspent output that can be spent by everybody as input. Since everybody can spend the unspent output used as an input for the tx, the witness data (containing the signature) isn't needed, but the transaction still has an input and an output section, and for a legacy node, the transaction is valid (it uses unspent outputs as an input and creates new outputs).

A "newer" node also receives the witness data (stored outside the first 1Mb), and knows that the unspent outputs used as an input for the transaction can't be spent by anybody... The "newer" nodes have the witness data, so they're able to actually check the signatures annd verify if the transaction is actually valid.

I can't really answer the last part of your question... Segwit is a soft fork, so normally there would be no pressure on old nodes to upgrade... However, nobody knows if there'll be a hard fork in the future that actually builds on (some of) the building blocks introduced by the segwit softfork... If such a hard fork would ever occur, the "legacy" nodes would probably have to upgrade if they wanted to stay part of the main chain.

Tx is still unconfirmed, my electrum is saying:

Code:

Status: Unconfirmed
Position in mempool: 2.86 MB from tip
Size: 209 bytes
Fee: 0.00010095 BTC  ( 48.3 sat/byte ) 

Can I determine somehow, how much of that 209 bytes is normal data, and how much is Witness data (which fits above 1MB?)

Thanks! I hope there will be hard fork in the near future, to cut out few years old not-upgraded nodes. If someone is maintaining a online 24/7 server, they may as well upgrade Bitcoin software on it from time to time! Hard fork would enable more complete solutions to be implemented, I am sure

Yes, you can... You can save the transaction from electrum, open it with a text editor and copy the hex data... Then use one of the many online tools to decode the transaction... You'll be shown the size and vsize... vsize = virtual size, it takes the segwit data into consideration.
Each byte that should find a place in the first 1 Mb counts for 4 vbytes, each byte outside the first 1Mb count for 1 vbyte

Did that, thank you:) However, online decoder says:
"size": 154,

Electrum says 209. How to decipher how much is 1Mb data, and how much is Witness data?

[mocacinno]

Did that, thank you:) However, online decoder says:
"size": 154,

Electrum says 209. How to decipher how much is 1Mb data, and how much is Witness data?

The main problem is that i don't know the transaction you're talking about, so it's not that easy to actually give you some advice... There's also the possibility you used a decoder that messed up.

Can you try http://chainquery.com/bitcoin-api/decoderawtransaction to decode the raw transaction?

In reality, it isn't even that important to know the size of the witness data... It's just important to get the vsize...

Since i don't know your actual transaction, i browsed my mempool and found a segwit transaction (not mine)

Code:

010000000001063ef3aedf4622845f1b9138b2f2e459239f2ecada686dc9eb2b801f209cbff0380000000023220020cfdb8b26d66884db0be226fa3b418cd6e614c67bd87a7203e7b07cc691f34016ffffffff356f58a927a053f15e996ec736eb5f99e113d72c71362557bda57212c96bd2f70000000023220020f3759e191a004f05f71078b9dec914b8fd40c35ab96ed699712733e8e4c34fb0ffffffff929191fe42e3bb1c7a43eee0cb5018593fccf62a9eae0fe3a4545da9e02ea71f0100000023220020643df8945cb7da3dd3c0673262a7c52221558d6989e186a4f06d3910bb9467d8ffffffff4749ba556b0f81b7320d1f65472f416ed75b31f417986675979d04a9872ff0e80100000023220020f530158f438efead03f070e538123d8454a78fdd700e1f31addb66a5ff9d0885ffffffffb0ee1ea23730983e2a663b32a02fa29a2410b90d91d150b8fd84cf9a18be79700000000023220020e04757184c40edd10e0beeebd8b3b7b0f1b376968c44c0d89664bd1aebb856cbffffffff94833307c3fe06c921a7dc8fa21b168b44fcdb2b37f3351a7fb1caf12e195dc005000000232200202eaf83bfd83fa25519c7f6b2d1bf052af79b1c8ed6eeae551a173fc8e181c65dffffffff0297c2a3000000000017a91424af774ddf8d24b53e46c244cf9e71d6c93cb9738702274b000000000017a914cc885ade74b93dd67c49828a499f03542fb7317487040047304402207861503af581693093ef62ef91315583fa641a29223ba596b7de2d5219e161db02201a4838ef850fb94ae01c9adc63ce10e92b717d7972a9ad19a23076cbf21c228a0147304402200a03d725d6b5d29d81fb06a3e436b2d8d97d1ae1156b8c153fb3b1f37424d4380220283bfafb1e62047bc15b0fbded7e1cc06faca44eed8155f0eba0715465b01b0901475221026559bb0e892738edfe959effb42f11dbb25fb618c8174c3e61a7e8209b2248dc2102f080b51bee43ea23b94e4d9b2359fa202856608ef2026c8c9a6bcbb9f4999b6852ae04004730440220296ba5d8c228ff1bbbaff2fd0bee4538e9395e93fe3a002a7b0f9f2df0778ccd02200f11105c30fc6b0d95eac781594f095cf855baa664868f0b33e18300d3effec801483045022100a9ac572bcf91bc7754fb01a1d3d2a199f0471f0c1cb22484fd5b5f85b0ea33e8022072631664f963e9cfd39421b1ca8dc8ca6feca3ff5410d04e8151bebbcd5cedfa01475221038104139bfcbdea8b7e9cbbf5278b10f7194a5354a8adc594c26d2185962422cb2102f080b51bee43ea23b94e4d9b2359fa202856608ef2026c8c9a6bcbb9f4999b6852ae040047304402207b8eb66653a93af3b04a8795422d3442bb96fb0fdc7b86b61e74d40cbcd66755022030992a9f50e89c2b99d07b34f60ff63a7443ccb472e6cb8e6b8f92e538285a8d01483045022100fb70e29b23887d5c4f7e30e47c6f36544bacf3738a591a0c46c9fa52a7013f58022064ef43741c5456ce36a5b74228c6f179b5c906e7a9e9d02e59920d226b9296f601475221031517d24554b915d9127595b83aa0a4ccac084fae76075da2f043869e5b1fa4d42102f080b51bee43ea23b94e4d9b2359fa202856608ef2026c8c9a6bcbb9f4999b6852ae04004730440220335fec8e04327a28ecc3b6368bdc323920b72029ecac3e3c039f8892af947700022061db9440f9a0bfaf7b6ebf6a91d574f0f2682f8615c7e47c243939f93e18d6b4014730440220293532c08a99d4a344a3b0dbdaf39284af3f9a8a4331b910321f8f5e5d51c35702205b8519f6760a9e3650b68d6e09cf0da0a49518e86e0c771acfebb58fa4bdcc090147522103655698a94c39150747b9c9a6eea89cc8278b9034b144807c957a5fc08b2565f32102f080b51bee43ea23b94e4d9b2359fa202856608ef2026c8c9a6bcbb9f4999b6852ae04004730440220600341c5f5288acd14855e887184d3d3147b46b914329b3b4ed1740666f8d45e02206ddace9ea9887f9f74cb19c5e508541be90ea4452ce6fbf0cce3102b1aadceeb0147304402206d2d9d4b266fe7c79e90467b5f1affc787179826631b89708d10539d83bd408602205e0579ad6a407389b3e9712a7cc2559b02a4f8de1de6f200bde04a95cbea52c101475221020eb7a23ad42e70d62d652b61ed32bffc2241e30cbe8a6cae60abfc643b8309e22102f080b51bee43ea23b94e4d9b2359fa202856608ef2026c8c9a6bcbb9f4999b6852ae0400483045022100ab3159efc518f32f4d35188ada40be31115704f5972f4b87372129a4e35eb223022033fbf7f36e4140f2b07b4d9baa4603641a7e546cb568ecadd77b928a0666e0ed014830450221008379ef5ba282111676ffee9b275c8b68a72ff77ae70f7a61cd0fd222655d97b00220621cdd0a7ab0fd76ed615ae79d2426515cdf16921af0aa70d377be85cb9cca5c0147522102b155838d1799de9aa24fdf52f091b3e096c66cc9422c969165f66089f2c4f0002102f080b51bee43ea23b94e4d9b2359fa202856608ef2026c8c9a6bcbb9f4999b6852ae00000000

If you use chainquery to decode this transaction, you'll actually be able to see the witness data, the "actual" size and the vsize...

[crypto_curious]

Yes, this one is better, showing the following:

"size": 372,
"vsize": 209,

[mocacinno]

Yes, this one is better, showing the following:

"size": 372,
"vsize": 209,

Well... using this info you should be able to calculate the size of the witness data (but like i said before, knowing this size isn't actually important, as long as you know the vsize.

source:https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki#Transaction_size_calculations

Transaction weight is defined as Base transaction size * 3 + Total transaction size (ie. the same method as calculating Block weight from Base size and Total size).

Virtual transaction size is defined as Transaction weight / 4 (rounded up to the next integer).

Base transaction size is the size of the transaction serialised with the witness data stripped.

Total transaction size is the transaction size in bytes serialized as described in BIP144, including base data and witness data.

source:https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki#Transaction_size_calculations

total transaction size = 372
virtual transaction size = 209

So... transaction weight : 209 * 4

(209 * 4) = (size of tx without witness data) * 3 + 372
(size of the witness data) = 372 -  (size of tx without witness data)

If i didn't make any errors... this should wean your base transaction would be ~154 bytes, the witness data would be ~218 bytes (core does round the vsize to the nearest integer, so the "actual" vsize is likely not exactly 209, so if you calculate the sizes you don't end up on a round integer)...

[Artemis3]

Hello,

I have this transaction:
edited, not required anymore

It's Bech32 to Bech32, from Electrum 3.3.6 (with SegWit wallet) to Bitcoin Core. It was expensive like normal transaction, minimum fee I could pay was 48.3 sat/byte.

I thought, SegWit transctions will fit in their own dedicated space, without waiting for space in normal block space, and supposed to be cheap.
Yet this transaction have just skipped one block without confirming (block has 1,248,049 bytes), electrum is saying "Position in mempool: 1.93MB 2.82MB from tip". (EDIT: it's falling in the queue due to low fee I think, certainly this transaction wants to squeeze into normal block space, I don't know why)

I thought it will be sent on first block, as there is plenty of SegWit space?

What I did wrong, and what I misunderstand here, please?

If all the variables are the same, native segwit (bech32) is the cheapest. First variable you didn't change, was the transaction fee. Electrum allows you to manually set that, regardless of network. You can force it to 1 Sat/B, if you don't mind waiting a few days to complete under current conditions.

There is also the number of inputs, if you have your coins spread it its more expensive than if they are all in a single address. Of course that also goes against privacy. Electrum has a couple of options that you can disable to tweak that as well.

Those "2MB from the TIP" is how further you are from getting processed, 2MB is rather low, i did a transaction last week and it was 40MB away, but it got processed anyway after a couple of days. Only if you are in a hurry and don't care spending more to have it quick, should you let either the wallet choose for you or you manually use a higher fee. You can also start with a low fee and then "speed it up" later, as long as you have more funds willing to spare into the transaction fee.

[crypto_curious]

If all the variables are the same, native segwit (bech32) is the cheapest. First variable you didn't change, was the transaction fee. Electrum allows you to manually set that, regardless of network. You can force it to 1 Sat/B, if you don't mind waiting a few days to complete under current conditions.

Thanks for your reply.
I did changed Electrum transaction fee. I choose lowest possible value, slide max to the left. Please have a look at Electrum interface:
https://imgur.com/a/AGa1orL
Option to choose my own value is not there, I have only a slider. Current lowest possible fee is 36.1 sat/byte, I can't go any lower. At the time of transaction in question, lowest possible fee for confirmation within 25 blocks was 48.3 sat/byte.
As a example, next block fee is 184 sat/byte and within 2 blocks is 123 sat/byte.
This is all from Electrum 3.3.6 with SegWit wallet.

There is also the number of inputs, if you have your coins spread it its more expensive than if they are all in a single address. Of course that also goes against privacy. Electrum has a couple of options that you can disable to tweak that as well.

There are 3 privacy options in Electrum:
"Use change address" this is a must, let's not go into extreme, this has to stay
"Use multiple change address" and "Enable output value rounding" were both disabled for this transaction.

Those "2MB from the TIP" is how further you are from getting processed, 2MB is rather low, i did a transaction last week and it was 40MB away, but it got processed anyway after a couple of days. Only if you are in a hurry and don't care spending more to have it quick, should you let either the wallet choose for you or you manually use a higher fee. You can also start with a low fee and then "speed it up" later, as long as you have more funds willing to spare into the transaction fee.

Could you share, how can I manually enter my own fee or at least go a little bit lower from what Electrum lets me? I'd love to send a transaction with "within 240 blocks" option, that would be 24 hours and it's fine for me.

[mocacinno]

Could you share, how can I manually enter my own fee or at least go a little bit lower from what Electrum lets me? I'd love to send a transaction with "within 240 blocks" option, that would be 24 hours and it's fine for me.

In 3.3.6, under preferences, under the tab "fees" there should be an option "edit fees manually". If you enable this option, you should be able to go as low as 1 sat/byte... However, you do have to realise electrum does not guarantee your tx will find it's way into a block within 240 blocks if you pay a x sat/byte fee. It's only an estimation based on the last couple of blocks, the fee (in sat/vbyte) and the size of the mempool.
If, by the time 240 more blocks have been found, the mempool gets flooded with loads of transactions paying higher fees (in sat/vbyte) than you, you'll have to wait longer... The shorter the estimated time, the higher the odds of your tx getting into a block within the estimated timeframe.

One last tip: if you start messing with the fees, it might be wise to enable rbf by default...

[crypto_curious]

Could you share, how can I manually enter my own fee or at least go a little bit lower from what Electrum lets me? I'd love to send a transaction with "within 240 blocks" option, that would be 24 hours and it's fine for me.

In 3.3.6, under preferences, under the tab "fees" there should be an option "edit fees manually". If you enable this option, you should be able to go as low as 1 sat/byte... However, you do have to realise electrum does not guarantee your tx will find it's way into a block within 240 blocks if you pay a x sat/byte fee. It's only an estimation based on the last couple of blocks, the fee (in sat/vbyte) and the size of the mempool.
If, by the time 240 more blocks have been found, the mempool gets flooded with loads of transactions paying higher fees (in sat/vbyte) than you, you'll have to wait longer... The shorter the estimated time, the higher the odds of your tx getting into a block within the estimated timeframe.

One last tip: if you start messing with the fees, it might be wise to enable rbf by default...

Thank you, that's it! I enabled this field and now I can manually modify the fee. I know this is only estimation based on past blocks, and I am happy with that, as RBF is always enabled. I just wish Electrum would add more estimated fee numbers to chose from, right now there is only 1 block, 2, 5, 10, 25. They should add also 50, 100, 200, 240, 1680, why not!