A Scalability Roadmap

[jonny1000]

Sorry to be blunt, but your supply and demand curves are nonsense.

Miners will not automatically create the largest possible block they can regardless of their revenue.

The supply curve has no relationship with the maximum block size allowed by the protocol - it's determined by the costs involved with producing larger blocks.

Dear justusranvier

Thank you for you comments about the supply and demand curves.  They may well be nonsense as this is an emergent field and I may not have strong knowledge in this area, the curves are also just theoretical and in practice the situation may be different.  I merely propose they could provide a useful framework to analyse the dynamic between transaction fees and the blocksize limit.

I agree that the supply curve is "determined by the costs involved with producing larger blocks", although of course the limit still matters in the sense that supply cannot go above the limit.  The point I was trying to make is the costs "involved with producing larger blocks" are very low.  I think of it in terms of the marginal cost to a miner of including one more transaction in the block. There are only two costs I can think of:
1. The very small amount of processing required to verify the transaction and include a hash of the transaction in the block header,
2. The cost associated with the higher blocksize increasing the probability of orphan.

The supply curve I made was assuming successful IBLT O(1) block propagation, which should reduce the marginal increase in orphan risk to alomst zero, this will leave only the small amount of computer processing as the marginal cost of including a transaction in a block.  This small cost is why the supply curve looks the way it does.  Miners include as many transactions as they can because there cost of doing so is limited and therefore the market driven transaction fee will be very low.  Then if demand increases such that there is insufficient space in blocks, supply is constrained by the limit and the price increases.  This is the area on the curve the network may need to be at to ensure mining revenue is sufficient.
 
Many thanks

[Gavin Andresen]

An extremely large block size would mess up the economics of mining eventually.

I'm working on a follow-up blog post that talks about economics of the block size, but want to get it reviewed by some real economists to make sure my thinking is reasonably correct. But I'm curious: why do you think an extremely large block size will mess up the economics of mining?  What do you think would happen?

RE: geometric growth cannot go on forever:  true, but Moore's law has been going steady for 40 years now. The most pessimistic prediction I could find said it would last at least another 10-20 years; the most optimistic, 600 years.

I'd be happy with "increase block size 40% per year (double every two years) for 20 years, then stop."

Because if Bitcoin is going gangbusters 15 years from now, and CPU and bandwidth growth is still going strong, then either the "X%" or the "then stop date" can be changed to continue growing.

I did some research, and the average "good" broadband Internet connection in the US is 10Mbps speed. But ISPs are putting caps on home users' total bandwidth usage per month, typically 250 or 300GB/month. If I recall correctly, 300GB per month was the limit for my ISP in Australia, too.

Do the math, and 40% of a 250GB connection works out to 21MB dedicated to Bitcoin every ten minutes. Leave a generous megabyte for overhead, that would work out to a starting point of maximum-size-20MB blocks.

(somebody check my math, I'm really good at dropping zeroes)

[justusranvier]

The supply curve I made was assuming successful IBLT O(1) block propagation, which should reduce the marginal increase in orphan risk to alomst zero, this will leave only the small amount of computer processing as the marginal cost of including a transaction in a block.  This small cost is why the supply curve looks the way it does.  Miners include as many transactions as they can because there cost of doing so is limited and therefore the market driven transaction fee will be very low.  Then if demand increases such that there is insufficient space in blocks, supply is constrained by the limit and the price increases.  This is the area on the curve the network may need to be at to ensure mining revenue is sufficient.

Markets don't need quotas in order to prevent producers from operating at a loss.

If the cost of processing transactions drops due to improvements in technology this is a good thing - it means that Bitcoin users get more transactions at a lower price.

You don't have to worry about miners going out of business because they aren't going to add transactions to a block if doing so results in a net loss for them.

By the way: the curves you are looking for are here:

http://econpage.com/301/practice/mt1-s.htm

Right now, the supply curve for Bitcoin transaction processing is like S2 on that graph, except that at the moment the equilibrium point is to the left of the quota, so the quota isn't yet affecting the price.

Without the block size limit, the curve would look like S1.

Technologies which lower the cost of producing and transmitting blocks moves the supply curve to the right (except the quota does not move).

[odolvlobo]

There are only two costs I can think of:
1. The very small amount of processing required to verify the transaction and include a hash of the transaction in the block header,
2. The cost associated with the higher blocksize increasing the probability of orphan.

I believe that these per transaction costs are real and they will eventually determine the minimum transaction fee, assuming there is no artificial limit to the block size. Even if these costs are very low they are not 0, and the increased block size will allow more transactions and more income from fees.

[jonny1000]

I agree the current supply curve may look like S2 on http://econpage.com/301/practice/mt1-s.htm.

However the perhaps the reason fees are not that low now is the following:
1.   Mining is not currently perfectly competitive and thus miners are not lowering their offer prices of their services to the marginal cost
2.   The orphan risk is currently a large risk and the marginal cost of including a transaction is actually high

The problems I am talking about do not apply now when there is a large block reward, but could occur in the future when the block reward is low and miners need to be incentivised by the transaction fee.

In a competitive market price should equal marginal cost, in that environment mining profit should fall to zero and there will be limited incentives to mine.  I propose that the system needs arbitrary limits on the blocksize to “manipulate” the transaction fee market, so that there are high enough mining incentives.  

In the current network a corollary situation occurs with respect to the difficulty and the Bitcoin price.  For example, at any given Bitcoin price, miners enter or leave the market and profit margins tend to zero and the system self-adjusts.  This is not a problem, if the Bitcoin price falls, miners exit, the difficulty falls, mining profits increase and we are fine.

Let’s consider the situation when the block reward is low and transaction fees compensate miners, then the dynamics are different.  If supply is not constrained, transaction fees fall to the marginal cost, mining profit falls and then miners exit and the difficulty falls.  The remaining miners can then find blocks more easily, but they don’t necessarily get compensated more for this, because the fees would still be low.

If the cost of processing transactions drops due to improvements in technology this is a good thing - it means that Bitcoin users get more transactions at a lower price.

Yes in a normal market this is true.  Bitcoin users getting more transactions at a lower price seems good.  However the market for Bitcoin transactions is not a normal market.  The system requires a large number of miners to be around to provide a large hashrate for security purposes.

For example, let’s consider the music industry, improvements in technology are reducing the cost of distributing music and this is a good thing, it means more people get more music at a lower price.  If the marginal cost of distributing music is zero then in a competitive market the price should be zero and the consumer wins.  The record industry or CD production industry may suffer and shrink, which many people may argue is fine, that’s just market forces at work.  With respect to miners, it is different, users need miners to be around.  Improvements in technology which reduced the cost of processing transactions to zero, could damage network security.

[teukon]

An extremely large block size would mess up the economics of mining eventually.

I'm working on a follow-up blog post that talks about economics of the block size, but want to get it reviewed by some real economists to make sure my thinking is reasonably correct. But I'm curious: why do you think an extremely large block size will mess up the economics of mining?  What do you think would happen?

RE: geometric growth cannot go on forever:  true, but Moore's law has been going steady for 40 years now. The most pessimistic prediction I could find said it would last at least another 10-20 years; the most optimistic, 600 years.

I'd be happy with "increase block size 40% per year (double every two years) for 20 years, then stop."

Because if Bitcoin is going gangbusters 15 years from now, and CPU and bandwidth growth is still going strong, then either the "X%" or the "then stop date" can be changed to continue growing.

I did some research, and the average "good" broadband Internet connection in the US is 10Mbps speed. But ISPs are putting caps on home users' total bandwidth usage per month, typically 250 or 300GB/month. If I recall correctly, 300GB per month was the limit for my ISP in Australia, too.

Do the math, and 40% of a 250GB connection works out to 21MB dedicated to Bitcoin every ten minutes. Leave a generous megabyte for overhead, that would work out to a starting point of maximum-size-20MB blocks.

(somebody check my math, I'm really good at dropping zeroes)

Yeah, 40% of a 250 GB connection works out to about 23 MB depending on how you define month.  May I ask what would happen regarding TOR?

If 1 MB blocks give us, say, 3 transactions per second, then 20 years of "double every 2 years" growth starting from 20 MB would leave us with about 60 million transactions per second.  That's about 25 transaction per hour per human (assuming a world population of 8.5 billion in 20 years time).

This sounds a bit excessive to me but then again I've not thought seriously about how such a volume of transactions could be utilised.  https://en.bitcoin.it/wiki/Scalability doesn't speculate beyond a few hundred thousand transactions per second.  I'd certainly appreciate a link if a discussion on the utility of millions of transactions per second exists.

Edit: Oops.  I miscalculated "double every year for 20 years".  Starting from 1 MB blocks worth, say, 3 transactions per second, then 20 years of "double every 2 years" growth starting from 20 MB will yield about 60 000 transactions per second.  That's about 4 transactions per week per human (assuming a world population of 8.5 billion in 20 years time).

Looking forward to the block-size economics blog post.

[justusranvier]

I agree the current supply curve may look like S2 on http://econpage.com/301/practice/mt1-s.htm.

However the perhaps the reason fees are not that low now is the following:
1.   Mining is not currently perfectly competitive and thus miners are not lowering their offer prices of their services to the marginal cost
2.   The orphan risk is currently a large risk and the marginal cost of including a transaction is actually high

The problems I am talking about do not apply now when there is a large block reward, but could occur in the future when the block reward is low and miners need to be incentivised by the transaction fee.

In a competitive market price should equal marginal cost, in that environment mining profit should fall to zero and there will be limited incentives to mine.  I propose that the system needs arbitrary limits on the blocksize to “manipulate” the transaction fee market, so that there are high enough mining incentives.  

In the current network a corollary situation occurs with respect to the difficulty and the Bitcoin price.  For example, at any given Bitcoin price, miners enter or leave the market and profit margins tend to zero and the system self-adjusts.  This is not a problem, if the Bitcoin price falls, miners exit, the difficulty falls, mining profits increase and we are fine.

Let’s consider the situation when the block reward is low and transaction fees compensate miners, then the dynamics are different.

1. Miners are not pricing transaction very well because of the huge block subsidy. Like all subsidies, it adversely affects the market. Our only recourse is to wait until it diminishes.
2. Orphan risk affects the slope of the curve, not its basic shape. That adding transactions to a block creates costs for the miner in the form of orphan risk is an argument for why no quota is necessary to keep the block size reasonable.

Price discovery for transaction inclusion will get better with there is no subsidy and all miner revenue is derived from via transaction fees. At that point there's nothing "special" about that industry compared to every other service industry which does a fine job of using price discovery to match supply and demand.

If supply is not constrained, transaction fees fall to the marginal cost, mining profit falls and then miners exit and the difficulty falls.  The remaining miners can then find blocks more easily, but they don’t necessarily get compensated more for this, because the fees would still be low.

If there are fewer miners competing for the same amount of transaction fees, then each miner's revenue has increased. The process you describe will continue until the oversupply of miners is corrected and equilibrium is restored.

For example, let’s consider the music industry, improvements in technology are reducing the cost of distributing music and this is a good thing, it means more people get more music at a lower price.  If the marginal cost of distributing music is zero then in a competitive market the price should be zero and the consumer wins.  The record industry or CD production industry may suffer and shrink, which many people may argue is fine, that’s just market forces at work.  With respect to miners, it is different, users need miners to be around.  Improvements in technology which reduced the cost of processing transactions to zero, could damage network security.

Markets that never could have existed in the first place except for government-granted monopoly rights are not appropriate as models of what we should do.

[Gavin Andresen]

Yeah, 40% of a 250 GB connection works out to about 23 MB depending on how you define month.  May I ask what would happen regarding TOR?

Thanks for checking my math!  I used 31-day months, since I assume that is how ISPs do the bandwidth cap.

RE: what happens with Tor:

Run a full node (or better, several full nodes) that is connected to the network directly-- not via Tor.

But to keep your transactions private, you broadcast them through a Tor-connected SPV (not full) node. If you are mining, broadcast new blocks the same way.

That gives you fully-validating-node security plus transaction/block privacy. You could run both the full node and the SPV-Tor-connected node on a machine at home; to the rest of the network your home IP address would look like a relay node that never generated any transactions or blocks.

If you live in a country where even just connecting to the Bitcoin network is illegal (or would draw unwelcome attention to yourself), then you'd need to pay for a server somewhere else and administer it via Tor.

[teukon]

Yeah, 40% of a 250 GB connection works out to about 23 MB depending on how you define month.  May I ask what would happen regarding TOR?

Thanks for checking my math!  I used 31-day months, since I assume that is how ISPs do the bandwidth cap.

Ah, that makes sense.  No problem.

RE: what happens with Tor:

Run a full node (or better, several full nodes) that is connected to the network directly-- not via Tor.

But to keep your transactions private, you broadcast them through a Tor-connected SPV (not full) node. If you are mining, broadcast new blocks the same way.

That gives you fully-validating-node security plus transaction/block privacy. You could run both the full node and the SPV-Tor-connected node on a machine at home; to the rest of the network your home IP address would look like a relay node that never generated any transactions or blocks.

If you live in a country where even just connecting to the Bitcoin network is illegal (or would draw unwelcome attention to yourself), then you'd need to pay for a server somewhere else and administer it via Tor.

Thank you, this clears things up for me.  All I don't understand here is the notion of broadcasting newly generated blocks through a Tor-connected SPV node but I imagine I can look this up.

[tucenaber]

No, it does not mean that.  It means this:

2015: 1.5MB (block reward = 25)
2016: 2.25MB (block reward = 25)
2017: 2.8125MB (block reward = 12.5)
2018: 3.52...  (block reward = 12.5)
2018: 4.39...  (block reward = 12.5)
2019: 5.49...  (block reward = 12.5)
2020: 6.18...  (block reward = 6.25)
2021: 6.95...  (block reward = 6.25)
2021: 7.82...  (block reward = 6.25)

and so on.  That seems like a schedule that won't kill off too many nodes.

I see. I misunderstood you. But it still amounts to basically the same thing. It will level off to a fixed limit eventually. Besides, it is completely ad hoc. Gavin's formula as I understand it is basically to raise the limit as fast as possible without endangering decentralization too much, which makes sense. Yours seems just taken out of thin air.

An extremely large block size would mess up the economics of mining eventually.

No it won't.

[Syke]

If 1 MB blocks give us, say, 3 transactions per second, then 20 years of "double every 2 years" growth starting from 20 MB would leave us with about 60 million transactions per second.  That's about 25 transaction per hour per human (assuming a world population of 8.5 billion in 20 years time).

This sounds a bit excessive to me but then again I've not thought seriously about how such a volume of transactions could be utilised.  https://en.bitcoin.it/wiki/Scalability doesn't speculate beyond a few hundred thousand transactions per second.  I'd certainly appreciate a link if a discussion on the utility of millions of transactions per second exists.

I like this line of thinking. What TPS are we shooting for and when? That's what will determine what size blocks we need and how to grow to that target.

Simple growth rates like "50% increase per year" are guaranteed to end up with blocks that are too large, which will require another hard fork. Hard forks are bad, mkay?

[hello_good_sir]

But I'm curious: why do you think an extremely large block size will mess up the economics of mining?  What do you think would happen?

Limiting block size creates an inefficiency in the bitcoin system.  Inefficiency = profit.  This is a basic law of economics, though it is usually phrased in such a way as to justify profits by pointing out that they eliminate inefficiencies.  I am taking the other position, that if we want mining to be profitable then there needs to be some artificial inefficiency in the system, to support marginal producers.  Of course that profit will attract more hashing power thus reducing/eliminating the profit, but at a higher equilibrium.  However, I am not too worried about this aspect of large block sizes.  It is a fairly minor problem and one that is a century away.

RE: geometric growth cannot go on forever:  true, but Moore's law has been going steady for 40 years now. The most pessimistic prediction I could find said it would last at least another 10-20 years; the most optimistic, 600 years.

Well I found several predictions saying that it was only going to continue for about 7 more years.  However, that was about 12 or so years ago, so obviously those predictions didn't come true.

The problem with Moore's Law predictions is that they don't take into account game theory.  They assume that nearly everyone is either working to make better chips, buying better chips, enjoying better chips, or simply having nothing to do with better chips.

We need to imagine a world where the miners, bankers, and governments work to suppress computing technology.  Not because they want to destroy bitcoin, but because they want to be the dominant players.  If bitcoin is wildly successful, Moore's law will have an opponent.

I'd be happy with "increase block size 40% per year (double every two years) for 20 years, then stop."

That would probably work pretty well.  It would end (hopefully) before bitcoin is too big of a deal.

The goal should be to get to a situation where it is simply socially unacceptable to suggest changes to the bitcoin protocol.  This needs to be the situation before mass acceptance.  Once mass acceptance happens conversations like the one we are having now will be held behind private doors by central banks (central banks will always be with us).

[2112]

Limiting block size creates an inefficiency in the bitcoin system.  Inefficiency = profit.  This is a basic law of economics, though it is usually phrased in such a way as to justify profits by pointing out that they eliminate inefficiencies.  I am taking the other position, that if we want mining to be profitable then there needs to be some artificial inefficiency in the system, to support marginal producers.  Of course that profit will attract more hashing power thus reducing/eliminating the profit, but at a higher equilibrium.  However, I am not too worried about this aspect of large block sizes.  It is a fairly minor problem and one that is a century away.

This is fairly common misconception that the only way to pay for the space in a mined Bitcoin block is with fees denominated in bitcoins. But this is not true when a miner is integrated with an exchange, because an exchange can shave commissions on both sides of the transactions.

Imagine for a moment that Bitfury branches out into Consolidated Furies and spawns Hryvnafury, Roublefury, Eurofury, DollarFury, etc.; all of them being exchanges. It can then easily outcompete pure Bitcoin miners because it can directly funnel fiat commissions into electric utility bills without having to go twice through the fiat<->coin exchange.

Edit: In fact opportunities for integration are not limited to mining + coin exchange. Imagine for example Marijuanafury which does two things demanding lots of electricity: Bitcoin mining and indoor marijuana growing. If only somebody could come up with new optical ASIC technology that is pumped with energy via photons at the same wavelength that stimulate photosynthesis...

[painlord2k]

Just exposing some ideas:

Gavin plan appear to me to be VERY conservative (maybe too much).

To be able to process the same number of transaction of VISA, Bitcoin should grow x2,000.
The size of blocks should go up ~1.000x at least to accommodate so many transactions.
And we will not just want to take VISA burden, we want, also, offer a service to the currently unbanked (being humans or DACs).
In the block size increase 50% every year, it will take 20 years to take over VISA alone; never mind the unbanked and DAcs.

With a 100% increase every year, it will need 11 years to take over VISA. This will bring us to 2025, when the inflation rate will be around 1% and the coins mined will be 3.125 BTC/block.

If we suppose the income of the miners will be the same as now, when the block reward will become irrelevant and Bitcoin will have the same number of transactions of VISA, we need  0.01 $ value per transaction (200 K transactions x 0.01 cent = 2M$/day- like today).

We must keep in mind, the low cost of Bitcoin transactions will cause a greater use of them, not a lower use of them.
If now it is uneconomic to bill people daily, because VISA cost 20 cents+2% per transaction, but tomorrow, with a deployed BTC infrastructure, you could have people paying daily for a lot of things today they pay weekly or monthly.

We must also keep in mind BTC transactions will be what keep the network working. We could compensate with larger fees (10 cent per transaction?), but this will weed out marginal transactions and push them in some other coin.

I would suggest some schedule to go from 1 to 32 MB in the next 5 years at most, better in two years and plan to increase exponentially from there until 1 GByte block (at least).

Because as the merchants and adopters reach a critical mass, there will be an explosion of transactions.
Today, the great majority of BTC holders have little chance to spend their bitcoins or being paid in bitcoins in their daily lives.
But, as BTC become popular and adopted, they will start to use it more frequently and the number of transactions will explode. We have see nothing until now. The growth was pretty linear in the last two years.


The big problem with Gavin plan is exponential growth is tricky to manage.
Too slow and in grow too slowly initially to keep up with the demand.
Too fast and will not keep up with the resources available.

[teukon]

Gavin plan appear to me to be VERY conservative (maybe too much).

To be able to process the same number of transaction of VISA, Bitcoin should grow x2,000.
The size of blocks should go up ~1.000x at least to accommodate so many transactions.
And we will not just want to take VISA burden, we want, also, offer a service to the currently unbanked (being humans or DACs).
In the block size increase 50% every year, it will take 20 years to take over VISA alone; never mind the unbanked and DAcs.

Thank you.  You awakened me to a calculation error I made earlier.

I believe the proposal involves an initial jump in block size followed by temporary exponential growth with fixed parameters.  If the blocksize were increased to say 20 MB and then grown at 50% per year we'd be up 2 GB blocks in 11-12 years.  At 40% (double every 2 years) it would take 13-14 years.

[justusranvier]

Can people stop talking about increasing the block size?

It's the block size limit that needs to increase or be abolished.

Higher limits does not imply a suddenly larger blocks, just the possibility for larger blocks to be created when the need exists and the price is right.

[TierNolan]

Higher limits does not imply a suddenly larger blocks, just the possibility for larger blocks to be created when the need exists and the price is right.

Right.  The majority of the miners, working together, will always have the ability to set a lower block limit.

The block limit embedded in the reference client is there to prevent massive blocks that cannot be handled in a distributed way.  Large enough blocks mean that miners who don't have a high speed connection can't keep up.

As long as the average VPS can handle the block size, then centralisation risk is low.

Miners might still decide to keep the block size small to push up fees.  There is an inherent cost for larger blocks, since they take longer to distribute (though that depends on low latency optimisations).

[teukon]

If 1 MB blocks give us, say, 3 transactions per second, then 20 years of "double every 2 years" growth starting from 20 MB would leave us with about 60 million transactions per second.  That's about 25 transaction per hour per human (assuming a world population of 8.5 billion in 20 years time).

This sounds a bit excessive to me but then again I've not thought seriously about how such a volume of transactions could be utilised.  https://en.bitcoin.it/wiki/Scalability doesn't speculate beyond a few hundred thousand transactions per second.  I'd certainly appreciate a link if a discussion on the utility of millions of transactions per second exists.

I like this line of thinking. What TPS are we shooting for and when? That's what will determine what size blocks we need and how to grow to that target.

Simple growth rates like "50% increase per year" are guaranteed to end up with blocks that are too large, which will require another hard fork. Hard forks are bad, mkay?

Apologies.  I miscalculated, the figure should be 60 000 transactions per second, not 60 million (so about 4 transactions per human per week).

More meaningfully, the maximum block size would rise to a final value of 20 GB each.

[justusranvier]

Right.  The majority of the miners, working together, will always have the ability to set a lower block limit.

The block limit embedded in the reference client is there to prevent massive blocks that cannot be handled in a distributed way.  Large enough blocks mean that miners who don't have a high speed connection can't keep up.

As long as the average VPS can handle the block size, then centralisation risk is low.

Miners might still decide to keep the block size small to push up fees.  There is an inherent cost for larger blocks, since they take longer to distribute (though that depends on low latency optimisations).

Limits are still the wrong word to use.

Each miners will decide to include which transactions into their block. The point at which they decide to stop adding transactions to a block will depend on their own best guess of where it is no longer to profitable to do so.

This is not a limit - it's an equilibrium.

The "centralization risk" everybody keeps talking about is an artifact of the the P2P network lacking price discovery and operating entirely via donated bandwidth. Fix that problem and we'd never need to have these debates ever again.

[gtraah]

Great Idea Gav, I know its is ridiculously hard to choose appropriate protocol changes when there are MANY different people thinking differently, I like that you guys took your time discussed it with each other and came to the conclusion that the pros outweigh the cons.

I think BTC is growing and this will encourage innovation and more on-chain transactions and take btc to the next level