Would this be a reasonable compromise between Core and XT?

[RustyNomad]

Not too long ago, 1 TB harddisks were unimaginable. A bit longer ago, but still quite recently, 1 GB was already *huge*.

Read last night that that Samsung has announced their new 16TB SSD so storage space will not be a problem in the near future - link: http://www.zdnet.com/article/samsung-announces-16tb-ssd/

Reading through this thread it seems like the major debate now is no longer on whether the block size has to be increased or not but rather by how much and how often. I'm no expert when it comes to this but think that it has to be dynamic in nature meaning that it has to be based on actual events/growth and not just based on our best guesses of how fast and how big blocks will grow.

[tspacepilot]

What I don't get with this whole topic about block sizes is why there's not a solution out there to dynamically increase or decrease block size based on usage.  That's pretty much what we do with everything else in the network.  Difficulty goes up and down based on how hash power there is to keep the block time constant on average.  Why wouldn't we do the same thing with the block size?  Imagine, if blocks are >95% full for 2 weeks, block size limit goes up; if blocks are <95% full on average in another two weeks, block size limit goes back down.

It seems to me that such a solution would move the discussion to what's really important, which is how full we want the blocks to be.  I know that some want fuller blocks as a way to prompt higher fees, others say we don't want long confirmation times.  Wallets are adapting to these spam attacks already to allow people to more easily set a higher or lower fee on a given transaction (even my mobile wallet has this option now).  As wallets adapt to allow users to dynamically set their fee, the other variable is the block size, we don't want the blocks to be completely full all the time (right?), so how full do we want them?  I don't know if 95% is right, but it seems like we might find more consensus around a proposal that wasn't going to arbitrarily explode block sizes if the extra space isn't really crucial.  Allowing block size limit to expand and contract with usage means we might actually have a lower block size limit for a while.

I don't know, I know that others have proposed something like this, but it seems to be the least popular of the options.  Most people seem to be either trying to send the block size limit off on some exponential increase or they are arguing for keeping the status quo.  I think a dymanic block size limit (once which can decrease as well as increase) would be the more reasonable compromise.

[Kazimir]

What I don't get with this whole topic about block sizes is why there's not a solution out there to dynamically increase or decrease block size based on usage.

Note that the conditions I mentioned (which were just an example to illustrate this line of thought) actually do provide in a way to increase the limit based on usage. Not just "increase X% per Y time".

Why wouldn't we do the same thing with the block size?

It's not directly about the block size, it's about the block size limit. I doubt if it's very useful to decrease the limit, nothing prevents people from using less block size if the need is not there (no need to decrease the limit for that). Especially if you take careful, moderate increase conditions, things shouldn't run out of hand.
Then again, it couldn't hurt either, provided you also take some lower bound restrictions in mind, to avoid mining pools deliberately keeping blocks empty to get lower and lower block size limits, thus making block size scarce, thus increasing incentive to pay higher fees.

I think we agree that a more dyamic, adaptive approach is typically better, as it can auto-correct any temporary swings.

By the way, a problem with this:

if blocks are >95% full for 2 weeks

is that a mining pool can deliberately hold off the block size increase by just inserting one small or almost empty block once every 2 weeks.

Instead I'd rather say: the next increase (or decrease, let's call it the next limit adjustment) kicks in when there have been X almost full blocks since the last adjustment.

But in the light of this:

It seems to me that such a solution would move the discussion to what's really important, which is how full we want the blocks to be.

Perhaps it would be even better if a limit adjustment takes place every 2 weeks, just like the difficulty, based on the total size of all blocks of the past two weeks. We then adjust the limit so that, assuming the past two weeks volume, we would maintain a desired average block size saturation.

I'm really curious to hear how Gavin, Mike and Gregory think about this kind of approach. I most certainly hope they'll somehow find a compromise and avoid the risk of Bitcoin getting divided in Core vs XT.

[mmeijeri]

What I don't get with this whole topic about block sizes is why there's not a solution out there to dynamically increase or decrease block size based on usage.

What is and isn't safe technically is determined in large part by the sort of connectivity people all around the world have in their homes. What is and isn't safe from an economic point of view depends in large part on how many people want to use Bitcoin. Neither of these things can be measured directly from the blockchain.

[tspacepilot]

But in the light of this:

It seems to me that such a solution would move the discussion to what's really important, which is how full we want the blocks to be.

Perhaps it would be even better if a limit adjustment takes place every 2 weeks, just like the difficulty, based on the total size of all blocks of the past two weeks. We then adjust the limit so that, assuming the past two weeks volume, we would maintain a desired average block size saturation.

I'm really curious to hear how Gavin, Mike and Gregory think about this kind of approach. I most certainly hope they'll somehow find a compromise and avoid the risk of Bitcoin getting divided in Core vs XT.

I do believe that we agree.  Except that maybe I'm a little less worried about the fragmentation than you are.   I guess I tend to think that when people get very passionate about doing things one particular way, the great thing about FOSS is that they have the freedom to go off and do it and see what happens.  If Bitcoin (or any project) is truly resilient and robust then it will survive.  If it doesn't survive (in its present form), it wasn't meant to be, I guess.  Bitcoin has already given the world a huge gift of innovation and disruption and new ways to think about money and exchange of value and trustlessness.  I think bitcoin has more to give, but only time will tell.

[SebastianJu]

Why not dropping the blocksizelimit completely since we know now that it won't lead to endless spamming?

If the spamming showed anything then that it's expensive. It only makes sense when you have a different agenda you want to prove. It might be worth as a business when you cut down costs by randomly spam, so to spread fear that a low fee might lead to being caught in a next spam attack. If some big miner tip for that cause then it might be worth it when not done constantly and with 1 MB max block size.

So we can say that with no block size limit it would not be worth it doing it, theoretically, for the reason of raising fees.

And the past has shown that advertising spam, or similar things, didn't happen on a scale that the blocks are full constantly.

So i think there is no reason anymore to fear when dropping the max blocksize limit completely now. Ok, spamming would become a bit cheaper but advertising spam would cost, because of the low amounts send. And blocksize spam would be cheaper, yes. So spamming for malicious reasons. But they would need to create much larger transactions and they would not reach anything with it. So they can drop it because it would cause costs without sense.

But even that could be confined when adding output related fees like litecoin implemented. I don't see a reason, and didn't get an answer why it is needed, that bitcoin is the only currency that charges one fee for transacting to different targets in one transaction and paying way less then with single transactions.

Maybe i'm wrong. Then educate me.

[tspacepilot]

Why not dropping the blocksizelimit completely since we know now that it won't lead to endless spamming?

If the spamming showed anything then that it's expensive. It only makes sense when you have a different agenda you want to prove. It might be worth as a business when you cut down costs by randomly spam, so to spread fear that a low fee might lead to being caught in a next spam attack. If some big miner tip for that cause then it might be worth it when not done constantly and with 1 MB max block size.

So we can say that with no block size limit it would not be worth it doing it, theoretically, for the reason of raising fees.

And the past has shown that advertising spam, or similar things, didn't happen on a scale that the blocks are full constantly.

So i think there is no reason anymore to fear when dropping the max blocksize limit completely now. Ok, spamming would become a bit cheaper but advertising spam would cost, because of the low amounts send. And blocksize spam would be cheaper, yes. So spamming for malicious reasons. But they would need to create much larger transactions and they would not reach anything with it. So they can drop it because it would cause costs without sense.

But even that could be confined when adding output related fees like litecoin implemented. I don't see a reason, and didn't get an answer why it is needed, that bitcoin is the only currency that charges one fee for transacting to different targets in one transaction and paying way less then with single transactions.

Maybe i'm wrong. Then educate me.

I think a lot of people are worried that without a blocksize limit then there's no way to predict the size of blockchain storage for full nodes.  Additionally, zero-fee transaction may be included by miners just to change the  hash of the block.  I think there are a lot of reasons why block-size limits are there.  I'm not an expert tho.

[SebastianJu]

Why not dropping the blocksizelimit completely since we know now that it won't lead to endless spamming?

If the spamming showed anything then that it's expensive. It only makes sense when you have a different agenda you want to prove. It might be worth as a business when you cut down costs by randomly spam, so to spread fear that a low fee might lead to being caught in a next spam attack. If some big miner tip for that cause then it might be worth it when not done constantly and with 1 MB max block size.

So we can say that with no block size limit it would not be worth it doing it, theoretically, for the reason of raising fees.

And the past has shown that advertising spam, or similar things, didn't happen on a scale that the blocks are full constantly.

So i think there is no reason anymore to fear when dropping the max blocksize limit completely now. Ok, spamming would become a bit cheaper but advertising spam would cost, because of the low amounts send. And blocksize spam would be cheaper, yes. So spamming for malicious reasons. But they would need to create much larger transactions and they would not reach anything with it. So they can drop it because it would cause costs without sense.

But even that could be confined when adding output related fees like litecoin implemented. I don't see a reason, and didn't get an answer why it is needed, that bitcoin is the only currency that charges one fee for transacting to different targets in one transaction and paying way less then with single transactions.

Maybe i'm wrong. Then educate me.

I think a lot of people are worried that without a blocksize limit then there's no way to predict the size of blockchain storage for full nodes.  Additionally, zero-fee transaction may be included by miners just to change the  hash of the block.  I think there are a lot of reasons why block-size limits are there.  I'm not an expert tho.

Why include zero fee transactions to change the hash? Is there an advantage from doing so?

I don't see any of these risks being valid when i see that, without spamming, the blocks aren't fully filled. So all the fears should be baseless, i think.

[tspacepilot]

Why include zero fee transactions to change the hash? Is there an advantage from doing so?

I don't see any of these risks being valid when i see that, without spamming, the blocks aren't fully filled. So all the fears should be baseless, i think.

You "mine" a block when you find a block hash that is lower than the difficulty threshold.  Hash functions are made to be such that a small change in the string you're hashing leads to an unpredictable change in the hash.  Miners are trying lots and lots of hashes (by changing the nonce) to try to find the next block faster.  Along with the nonce, you can also change other things in the block you're trying to mine such as which transactions are included.  I'm not a miner myself, so someone else here will have to tell you more.  But at least in principle, there is a motivation for miners to include various sets of transactions in a block in order to try to win the race to find the next block.  As long as the coinbase (block reward) far outstrips any transaction fees (as is currently the case), any miner would be happy to mine a block with no transactions at all simply to claim the block rewards.  There are many such empty blocks in the blockchain.

[Delek]

The compromise is to have BIP100/BIP101/BIPxxx-where-xxx-is-a-good-scalable-solution ON CORE SATOSHI CLIENT.

[Kazimir]

The compromise is to have BIP100/BIP101/BIPxxx-where-xxx-is-a-good-scalable-solution ON CORE SATOSHI CLIENT.

Eh, yes. I didn't mention this explicitly so perhaps it wasn't clear, but indeed I meant integrating a compromise in the core client, so XT can be shutdown and all Bitcoin users are on the same, one and only Bitcoin protocol again.

[SebastianJu]

Why include zero fee transactions to change the hash? Is there an advantage from doing so?

I don't see any of these risks being valid when i see that, without spamming, the blocks aren't fully filled. So all the fears should be baseless, i think.

You "mine" a block when you find a block hash that is lower than the difficulty threshold.  Hash functions are made to be such that a small change in the string you're hashing leads to an unpredictable change in the hash.  Miners are trying lots and lots of hashes (by changing the nonce) to try to find the next block faster.  Along with the nonce, you can also change other things in the block you're trying to mine such as which transactions are included.  I'm not a miner myself, so someone else here will have to tell you more.  But at least in principle, there is a motivation for miners to include various sets of transactions in a block in order to try to win the race to find the next block.  As long as the coinbase (block reward) far outstrips any transaction fees (as is currently the case), any miner would be happy to mine a block with no transactions at all simply to claim the block rewards.  There are many such empty blocks in the blockchain.

I know that it is a factor the hash is based on. But when you say that is the reason then this does not make sense to me. I mean the ASICS are calculating an enourmous amount of hashvariations by already changing one of the factors. And, like you say, the outcome is random. Changing the amount of transactions could not be done in the asic, which means it would be done by the host computer and it would be incredibly slow. It would simply be the same like trying to mine with the host computer.

So i still don't see what it would be worth for.

[tspacepilot]

Why include zero fee transactions to change the hash? Is there an advantage from doing so?

I don't see any of these risks being valid when i see that, without spamming, the blocks aren't fully filled. So all the fears should be baseless, i think.

You "mine" a block when you find a block hash that is lower than the difficulty threshold.  Hash functions are made to be such that a small change in the string you're hashing leads to an unpredictable change in the hash.  Miners are trying lots and lots of hashes (by changing the nonce) to try to find the next block faster.  Along with the nonce, you can also change other things in the block you're trying to mine such as which transactions are included.  I'm not a miner myself, so someone else here will have to tell you more.  But at least in principle, there is a motivation for miners to include various sets of transactions in a block in order to try to win the race to find the next block.  As long as the coinbase (block reward) far outstrips any transaction fees (as is currently the case), any miner would be happy to mine a block with no transactions at all simply to claim the block rewards.  There are many such empty blocks in the blockchain.

I know that it is a factor the hash is based on. But when you say that is the reason then this does not make sense to me. I mean the ASICS are calculating an enourmous amount of hashvariations by already changing one of the factors. And, like you say, the outcome is random. Changing the amount of transactions could not be done in the asic, which means it would be done by the host computer and it would be incredibly slow. It would simply be the same like trying to mine with the host computer.

So i still don't see what it would be worth for.

But in fact miners do do this.  I don't have the specific reference for you at the moment, but once a miner has iterated through all of the nonce values for a given set of transactions they start changing other stuff: the time stamp, the list of transactions to include.  I don't think we should go too much farther into it as it seems afield of the main topic here.  But I believe it's quite true that miners manipulate the number of transactions included in order to win the race to find the next block.  Even if you didn't want to look into the technical details, I think the empty blocks on the blockchain are enough to show you this.

[danielW]

You need some sort of limit at some point because you need restricted space so fee market can develop.

A fee market will be needed at some point, even tho its not necessary now.

Also, dynamically adjusting block size, is almost like having no block size limit. It can be gamed, and will increase with the effect of driving fees to zero.

edit: unless another way can be found of paying miners, such as discarding 21 million limit and having a small constant reward.

(also unlimited block size might lead to huge blocks and problems for network)

[tspacepilot]

You need some sort of limit at some point because you need restricted space so fee market can develop.

A fee market will be needed at some point, even tho its not necessary now.

Also, dynamically adjusting block size, is almost like having no block size limit. It can be gamed, and will increase with the effect of driving fees to zero.

edit: unless another way can be found of paying miners, such as discarding 21 million limit and having a small constant reward.

(also unlimited block size might lead to huge blocks and problems for network)

It looks like BIP100 is gaining some traction.  BIP100 has a dynamic block size limit that's readjusted based on miner vote every certain number of weeks.  I think it's a pretty good solution and I'm hoping that everyone will start to rally around it.  It doesn't have the exponential growth problem of BIP101 either.

@danielW: changing the hard limit of 21 million bitcoins isn't going to happen, if it does, a lot of people's confidence would be immediately shattered.  That's considered one of those hard-and-fast promises.

[SebastianJu]

Why include zero fee transactions to change the hash? Is there an advantage from doing so?

I don't see any of these risks being valid when i see that, without spamming, the blocks aren't fully filled. So all the fears should be baseless, i think.

You "mine" a block when you find a block hash that is lower than the difficulty threshold.  Hash functions are made to be such that a small change in the string you're hashing leads to an unpredictable change in the hash.  Miners are trying lots and lots of hashes (by changing the nonce) to try to find the next block faster.  Along with the nonce, you can also change other things in the block you're trying to mine such as which transactions are included.  I'm not a miner myself, so someone else here will have to tell you more.  But at least in principle, there is a motivation for miners to include various sets of transactions in a block in order to try to win the race to find the next block.  As long as the coinbase (block reward) far outstrips any transaction fees (as is currently the case), any miner would be happy to mine a block with no transactions at all simply to claim the block rewards.  There are many such empty blocks in the blockchain.

I know that it is a factor the hash is based on. But when you say that is the reason then this does not make sense to me. I mean the ASICS are calculating an enourmous amount of hashvariations by already changing one of the factors. And, like you say, the outcome is random. Changing the amount of transactions could not be done in the asic, which means it would be done by the host computer and it would be incredibly slow. It would simply be the same like trying to mine with the host computer.

So i still don't see what it would be worth for.

But in fact miners do do this.  I don't have the specific reference for you at the moment, but once a miner has iterated through all of the nonce values for a given set of transactions they start changing other stuff: the time stamp, the list of transactions to include.  I don't think we should go too much farther into it as it seems afield of the main topic here.  But I believe it's quite true that miners manipulate the number of transactions included in order to win the race to find the next block.  Even if you didn't want to look into the technical details, I think the empty blocks on the blockchain are enough to show you this.

Interesting. I never thought that the amount of possible nounces is so limited that miners have to go use these ways. Yes, then this makes sense for sure. The asics go through all nounce iterations and in the meanwhile the host computer is preparing another hash for the blocks transactions less than a transaction. So the number of nounces to check can be raised. But i really did not think that this is needed at all.

If that is really needed then it's probably because of the high speed of asics nowadays. So wouldn't we run into problems someday then when we checked all possible iterations of transactions? Though when i think about it... the transactions only needed to be sorted a bit different and you already have a new hash. So most probably there will never be a problem. This workaround will work all the time.

But you already say it too, the time stamp is a factor too, yes, i checked these things out some years ago. I guess the timestamp should be enough to make it possible running a neverending repetition of nounces. Maybe the changing of transactions is only a sideeffect of trying to get included more valuable transactions that came to the network after the first round of hashing?

But i don't think the empty blocks are proof for this practice. They are proof of the believe that empty blocks will propagate much faster than full blocks. Having an empty block finding a sufficient hash is way too random when it should come from shuffling transactions. There are way too much empty blocks to that being the reason.

[SebastianJu]

You need some sort of limit at some point because you need restricted space so fee market can develop.

A fee market will be needed at some point, even tho its not necessary now.

Also, dynamically adjusting block size, is almost like having no block size limit. It can be gamed, and will increase with the effect of driving fees to zero.

edit: unless another way can be found of paying miners, such as discarding 21 million limit and having a small constant reward.

(also unlimited block size might lead to huge blocks and problems for network)

I often read that people believe that without a block size limit we would not have high fees. But we clearly saw in the past that this is not correct. The blocks were not full all the time, not even meeting soft limit often. And still people pay a good fee. It's not that bitcoiners are a bunch of cheapskates.

Then let's double the amount of transactions by rising adoption of bitcoin and you have a solution that works.

I don't see an artificial limit needed. There is already proof from past experiences.

[danielW]

You need some sort of limit at some point because you need restricted space so fee market can develop.

A fee market will be needed at some point, even tho its not necessary now.

Also, dynamically adjusting block size, is almost like having no block size limit. It can be gamed, and will increase with the effect of driving fees to zero.

edit: unless another way can be found of paying miners, such as discarding 21 million limit and having a small constant reward.

(also unlimited block size might lead to huge blocks and problems for network)

I often read that people believe that without a block size limit we would not have high fees. But we clearly saw in the past that this is not correct. The blocks were not full all the time, not even meeting soft limit often. And still people pay a good fee. It's not that bitcoiners are a bunch of cheapskates.

Then let's double the amount of transactions by rising adoption of bitcoin and you have a solution that works.

I don't see an artificial limit needed. There is already proof from past experiences.

How much was paid in fees in the past? It was not enough to pay for securing the network if mining reward were to become negligible.

The fees were statically set, to such a small amount that most people simply could not be bothered to turn it off and were happy to donate the tiny amount.

Bitcoins security is based on proof of work, and that can not be sustained by donations. It was NEVER sustained by donations. It is and was sustained by coinbase mining reward, thats why it can function without a block size limit now.

An economic incentive will be needed in the future, and that can only come about if block size is limited.

[Was]

You need some sort of limit at some point because you need restricted space so fee market can develop.

A fee market will be needed at some point, even tho its not necessary now.

Also, dynamically adjusting block size, is almost like having no block size limit. It can be gamed, and will increase with the effect of driving fees to zero.

edit: unless another way can be found of paying miners, such as discarding 21 million limit and having a small constant reward.

(also unlimited block size might lead to huge blocks and problems for network)

Tx fees will be enough to properly incentivize the miners to secure the network in 2140. Rest assured. Perhaps instead of discarding 21 Million limit, you could add a 9th decimal place? 1/10 of a satoshi? A dime?

was

[SebastianJu]

You need some sort of limit at some point because you need restricted space so fee market can develop.

A fee market will be needed at some point, even tho its not necessary now.

Also, dynamically adjusting block size, is almost like having no block size limit. It can be gamed, and will increase with the effect of driving fees to zero.

edit: unless another way can be found of paying miners, such as discarding 21 million limit and having a small constant reward.

(also unlimited block size might lead to huge blocks and problems for network)

I often read that people believe that without a block size limit we would not have high fees. But we clearly saw in the past that this is not correct. The blocks were not full all the time, not even meeting soft limit often. And still people pay a good fee. It's not that bitcoiners are a bunch of cheapskates.

Then let's double the amount of transactions by rising adoption of bitcoin and you have a solution that works.

I don't see an artificial limit needed. There is already proof from past experiences.

How much was paid in fees in the past? It was not enough to pay for securing the network if mining reward were to become negligible.

The fees were statically set, to such a small amount that most people simply could not be bothered to turn it off and were happy to donate the tiny amount.

Bitcoins security is based on proof of work, and that can not be sustained by donations. It was NEVER sustained by donations. It is and was sustained by coinbase mining reward, thats why it can function without a block size limit now.

An economic incentive will be needed in the future, and that can only come about if block size is limited.

The thing is that satoshi wanted bitcoin to replace fiat to get people out of the hands of the banks. You can't do this with high fees.

And he created the fee system and the block halving. He said that the fees will replace the block reward step by step. And it will. But miners can't await that the bitcoin network can survive when they await that we raise the fee with each block halving. It would be instant death to bitcoin.

Saying that... it means that miners will inevitable earn less. UNTIL adoption comes into game. The more transactions the more fee. That was always the plan. And we will kill that plan when we start to think like "Miners don't earn enough, we need to raise fees." It should not matter what miners want since of course they are in it for earning money. When half of the miners drop dead because they can't be run profitable anymore then be it. Be insured that the rewards will still be way enough to secure the network. But we can't feed all miners without end. That won't work.

Unfortunately miners are the one who decide the development of bitcoin nowadays. And that might mean higher fees will come. Raising the fees with an artificial limit is somewhat unliberal. It's like a bank consortium deciding on how high the fees has to be in order to run lucrative miners. It should not work that way. Miners has to be switched off. That happened all the time. And it has to happen in the future. Imagine all the old miners still working having them held lucrative artificially... bitcoin would be dead long ago.

The question is... will greed kill bitcoin adoption or will the far outlook win and adoption will bring the fees needed in the future?

But the bitcoin network will always be secure. You should not fear that.