Blocks are [not] full. What's the plan?

[bfire]

What's wrong with the minimum block size idea?  If dummy data doesn't work then make it require real transactions.

Miners will only fill blocks with dummy transactions if they don't have enough available transactions in their mempool.  Otherwise they'll fill it with transactions that earn them fees.  If we can expect a certain transaction rate on the bitcoin network then the min block size could be set accordingly and so we shouldn't see much use of dummy transactions.

No, it won't work. Miners will simply create an extra output in the reward transaction, sending 0 bitcoin to OP_TRUE. They will then use it as an input for another OP_TRUE output, and repeat this process. Since the process is totally deterministic, miners won't need to broadcast these dummy transactions. Other miners will recreate the full block locally. So we go back to the current system: with less (real) transactions, the orphan rate is lower.

Excellent insight.  What if we required a minimum number of transactions, rather than a minimum block size?    There wouldn't be a way to work around that.   As an added benefit, fast blocks would tend to clean up low priority transactions.

It seems like we need to be looking at EVERYTHING possible to increase the scalability of the system, and this would fix one dimension.  

The new minimum transaction requirement could be eased in over time...starting at say 1/5 the median number of transactions.  Once the code was there, it would be easier to upgrade to 1/3 or even 1/2 in the future.  

There is no value to the network in allowing near-empty blocks, and huge potential drawbacks.

[1715198874]

1715198874

[1715198874]

1715198874

[1715198874]

1715198874

[DeathAndTaxes]

So what happens when there aren't enough tx to fill the min requirement?  The network just halts until there is enough?  Of course all this is academic.  You do understand that Bitcoin works on a consensus system and there will never be a consensus to change a core element of Bitcoin (baring possibly a change in crypto due to a cryptographic break). 

Also as the block subsidy declines and networks get faster and cheaper there is less of an artificial subsidy in the network.  Miners which exclude paying tx will simply go bankrupt.  If a orphan cost is less than the tx fee there is no economical reason to not include a given tx.  The miner simply gets less revenue for the same amount of work.  Low barriers to entry will push margin on miners so low that leaving profit on the table will mean operating at a negative margin (mining to lose money).

Up thread I already proposed a non-core change that would cut orphan costs by 90% or more.  That combined with the block subsidy decline will make this a non-issue.

[jl2012]

What if we required a minimum number of transactions

Don't you realize that my suggested workaround will also maximize the number of transactions?

[jl2012]

What's wrong with the minimum block size idea?  If dummy data doesn't work then make it require real transactions.

Miners will only fill blocks with dummy transactions if they don't have enough available transactions in their mempool.  Otherwise they'll fill it with transactions that earn them fees.  If we can expect a certain transaction rate on the bitcoin network then the min block size could be set accordingly and so we shouldn't see much use of dummy transactions.

No, it won't work. Miners will simply create an extra output in the reward transaction, sending 0 bitcoin to OP_TRUE. They will then use it as an input for another OP_TRUE output, and repeat this process. Since the process is totally deterministic, miners won't need to broadcast these dummy transactions. Other miners will recreate the full block locally. So we go back to the current system: with less (real) transactions, the orphan rate is lower.

EDIT: The workaround suggest above does not work due to the 100-block maturity rule. A trivial fix is to put the dummy transaction as the second transaction in a block, and the rest are derived in the deterministic way.

[niniyo]

The block subsidy is not going to halve as quickly as the bitcoin price will double (that is, if bitcoin is successfull).  Fees will shrink in BTC terms as the bitcoin price rises (because the supply/demand for blockspace will be priced in accordance with its real value).  Unfortunately this means that the orphan cost is likely to get higher and higher compared to transaction fees.  IMO it could take many decades before a reasonable fee will outweigh the orphan cost.

[anth0ny]

The block subsidy is not going to halve as quickly as the bitcoin price will double (that is, if bitcoin is successfull).  Fees will shrink in BTC terms as the bitcoin price rises (because the supply/demand for blockspace will be priced in accordance with its real value).  Unfortunately this means that the orphan cost is likely to get higher and higher compared to transaction fees.  IMO it could take many decades before a reasonable fee will outweigh the orphan cost.

We could use some numbers on exactly what the orphan cost is. Make sure to take into account that only 4 miners account for more than 75% of the hashing power. Also make sure to consider the benefits that netsplits have on mining income (lower difficulties).

As long as you have a fast connection to GHash.IO, your orphan cost should be little or nothing. Within the next few decades connection speeds should get even faster, and most likely mining operations will move closer to each other network-wise. But most importantly, within the next few decades the need to transfer the entire block after finding a solution should be eliminated - certainly for the larger miners, anyway.

Instead of mining pools, within the next few decades it's likely we'll see propagation pools. The mining will be kept separate, but the generation of the parts of the block other than the coinbase transaction will be pooled. Thus there won't be anything to propagate other than the coinbase transaction and the block header. The rest of the block gets propagated as the transactions come in. And only the previous block hash has to get propagated all the way to the actual physical mining equipment. The equipment which verifies the block can be kept at a location with a really fast Internet connection, while the ASICs can sit in some remote cabin in the tundra or whatever (they'll need a connection with reasonably low latency, but don't need much bandwidth).

All of this without any need for a hard fork, too.

Hard forks should be an absolute last resort.

[Cryddit]

I think that because the problem is mainly about orphan risk, it needs to be addressed specifically via orphan risk.

consider https://en.bitcoin.it/wiki/Proof_of_blockchain_fair_sharing as a basis for a solution.

The premise is that the longer a transaction has been in existence, the more important it becomes to the acceptance of the next block.

Although the nodes may not all be looking at the exact same transaction list, if the numbers seen are similar, they should be computing a very similar 'credibility score' for the next block.  And if the 'credibility score' is too low, they ignore that block, mining on the previous block instead. 

This creates an orphan risk for *failing* to include transactions (specifically the oldest outstanding transactions) that balances the orphan risk for *including* transactions (and using up block space to do it). 

BDD = Number of bitcoin days destroyed
BD1 = Number of bitcoin days in fee-paid, nonconflicting tx 20 minutes or more old and not yet included in a block
BD2 = Number of bitcoin days in fee-paid, nonconflicting  tx 40 minutes old and not yet included in a block
BD3 = Number of bitcoin days in fee-paid, nonconflicting tx 60 minutes old and not yet included in a block
BD4 = Number of bitcoin days in fee-paid, nonconflicting tx 80 minutes old and not yet included in a block
etc.

So let
Credibility = BDD/ (BDD+ BD1 + 2^(BD2 + 2^(BD3 + 2^(BD4 + ... ))))

And if we find a chain is too 'incredible' (less than 0.000001 or so) we just ignore it and mine on a more credible chain (even if that  more credible chain is just the current 'incredible' chain minus the last block).  Because miners are looking at different tx pools, or may have learned about the same tx at different times, they may calculate different 'credibility' thus that some accept a new block and some don't.  But this shouldn't matter much because if 40% of the miners reject a block then that block gets a 40% orphan risk, which has a tangible cost to the miner producing it and gives the miners a strong motivation to avoid creating the kind of blocks for which that might be an issue.

This appropriately prioritizes fee-paid nonconflicting transactions that have been waiting for the longest time, and has the benefit of allowing the blockchain to resist a more-than-51% attack at least in terms of making sure that no one can keep a valid but unfavored tx *out* of the blockchain forever.

Does it cause a problem that initially rejected blocks can get brought into the chain via a reorg, when they become part of a chain that is, thanks to a later block that includes the tx it missed, more 'credible?' 

[anth0ny]

I think that because the problem is mainly about orphan risk, it needs to be addressed specifically via orphan risk.

consider https://en.bitcoin.it/wiki/Proof_of_blockchain_fair_sharing as a basis for a solution.

The premise is that the longer a transaction has been in existence, the more important it becomes to the acceptance of the next block.

Hmm... What happens when someone tries to split the network by flooding different sets of transactions to different parts of the network?

For that matter, what happens with nonstandard transactions?

This creates an orphan risk for *failing* to include transactions (specifically the oldest outstanding transactions) that balances the orphan risk for *including* transactions (and using up block space to do it).

Hmm... Along those lines, would we possibly see ourselves in a situation where miners pay for high value transactions?

That part would be nice. But then, it'd only increase the means by which someone can buy up...the private keys to old coins, for example.

[anth0ny]

Would it be possible to replace the MaxBlockSize by a MinBlockSize in the protocol,

that would adapt, depending on queue size

Sure. About the hardest part would be coming up with the name for this new altcoin.

[Altoidnerd]

Sure. About the hardest part would be coming up with the name for this new altcoin.

How about Bitcoin II ?