Increase block size (5%) by using CompactSize instead of UInt

[Coding Enthusiast]

TL;DR. Changing all Numbers to CompactSize can effectively clean up 3.5%-5.5% (up to 50 KB) of wasted block sizes and open up room for additional transactions.

Ever since I saw this irrelevant question I have been working on this theory that transactions are wasting some space the way they are.

For instance, the first field is Version which is wasting 3 bytes where it only needs 1 byte. Ever since the first transaction in 2009 version was 1 and any change will not shoot it up to 253 (the first value which needs 3 bytes in CompactSize)
The same goes for the following:

• Version: for the past 7 years version was 1
• lock_time: Almost all transactions are using 0
• index (in Outpoint): Most transactions are 0 and are less than 253
• Block version: Blocks start with 2

And the thing is that these numbers can be defined as a CompactSize Unsigned Integers freeing up those extra bytes.

I ran some statistical analysis on some random blocks between 440300 and 441300 heights and medium results are as follows:
Note: These numbers of average of about 20 random blocks.
Tx Count:      2,298
Average TxSize:   541 (bytes)
Clean Up Size:   46501 (bytes)
Clean Up %:   4.16%
Additional Tx that can be added after clean up:   94

NOTE: This is not limited to current block size but it is applicable to now and the possible future and the bigger the block size the bigger the effect of this change.
DOWNSIDE: I believe it requires a hardfork which will also be not-backforward-compatible.

p.s. I have created this topic to get some feedback on this opinion before I go ahead and increase the number of blocks I analyze. Each block is 2 MB raw data
p.p.s. I can also share the code I used for this whole Analysis and Changing.
Coded in C♯
https://github.com/Coding-Enthusiast/BitcoinBlockSizeChange

[1714010104]

1714010104

[1714010104]

1714010104

[Coding Enthusiast]

♯
Have you coded a change that can be listed on git? That's a better place for this discussion. That will allow your idea to be scrutinized by people that understand what your talking about (as opposed to the fools here that just want to argue without understanding). Maybe someone on git can help you develop a proposal that leads to a pull request. At the very least, move this to the dev and tech section of the forum.

I am not an expert and I have no C++ knowledge to do that, I did this in C# and this is more of an idea open for discussion and I want to know the experts opinion on pros and possible cons of this kind of change in transaction structure.

[Velkro]

That would be some nice optimization u got there but... need of hardfork completly ruins your idea even if it will be confirmed to actually give this kind of improvement. Its too low gain (5%) to hassle, amount of work needed and danger associated with hard fork.
So its something for future, if hard fork will be needed in future, maybe this will be deployed then.

[achow101]

I believe that BIP 134: https://github.com/bitcoin/bips/blob/master/bip-0134.mediawiki is very similar to this, but taking things to the next level (so to speak). It is a bit more complex but in theory it shrinks transactions quite a bit by removing a lot of unnecessary bytes.

Note that any such change would be a hard fork. Furthermore, AFAIK, BIP 134 has not received a lot of review and testing yet from multiple developers.

[theymos]

Doing this isn't even a softfork. You can basically just treat it as a form of over-the-wire and on-disk compression. All you need to do is restore it to its original form before hashing, or when transmitting to an old node.

[gmaxwell]

Reasonable thinking but you are far too unambitious.

The understanding you need is that a transaction is like an idea. It can be expressed in many languages.  We happen to use  a particular one today to send them over the network and store on disk which is not as efficient as it could be.  We
actually use another one to handle transactions in memory that is about half as efficient as the current format you know-- but faster to work with.

Nodes could use a different language to store the transactions they use on disk or send over the P2P wire; as theymos notes-- it's not even a softfork-- This has been recommended many times in the past when people show up trying to use zlib compression on transactions (which doesn't work very well, mostly just exploiting key reuse.)-- and it influenced my thinking about how weight is computed in segwit.

There is a more concrete design that was published a bit back http://people.xiph.org/~greg/compacted_txn.txt

And Sipa has taken that design, improved it, made it more generic and has been implementing it so we can benchmark it. Hopefully there will be a patch up soon.

The result is 20% or more savings on basically all transactions, 30% on some more inefficient kinds. ... and it applies to the whole history, back to day one.

[Coding Enthusiast]

Thanks for the replies, it is good to see a better version of my idea is already available.

[ArcCsch]

In binary code, what part of the transactions actually takes up the most space?
The signatures?
The output scripts?
the input hash pointers?
The amounts?
A multisig scheme can combine many signatures compactly, but, as far as I know, the signatures must all be written on the combination of transactions, is there any multisig scheme that can combine completed signatures? If this is impossible, would it be practical to sign combinations of transactions after broadcasting?
How much space is wasted by writing OP_DUP OP_HASH160 and OP_EQUALVERIFY OP_CHECKSIG?
This should be the default script, and omitted from the block chain.