A Scalability Roadmap

[Gavin Andresen]

I propose the following rule to determine the block size limit, once the block reward is low
The block size limit would increase (or decrease), by X%, if total transaction fees in the last N blocks is Y Bitcoin or more (or less).  

......

I am aware miners could also manipulate fees by including transactions with large fees and not broadcasting these to the network.  However why would miners in this scenario want to manipulate the limit upwards?

The fear is that a cartel of big, centralized, have-huge-data-pipes miners would drive out smaller miners by forcing up the block size high enough so the smaller miners have to drop out.

[NewLiberty]

This blocksize increase effort is to support the interests of the merchant service companies, Circle et. al.  I sympathize with their plight, but Bitcoin is not made for these first.  Bitcoin is for everyone.  There are parts of the planet (some of which have the greatest need for Bitcoin) that have very limited bandwidth today and can be expected to not see rapid improvement.

We do need a path forward.  We need a way to scale up.  What I can't abide is the notion of picking a number based on historical data, extrapolating, and applying it to the future.  Whatever we guess, we are guaranteed to be wrong.  Its wrong now, (and since we are not facing any imminent existential crisis) unless we can do better than still being wrong, we aren't ready to contemplate hard forks.

Isn't it worth it to the future generations of Bitcoiners to get this right?  At the moment we have the luxury of time, and we have other developments that will further mitigate this issue are coming to give us even more time.

So... Either let the large(ish) companies that are pushing for this (through TBF) make the best use of this time to give us a path forward that will be a lasting one... or wait until the decentralized brains come up with something more future proof than a guess based on historical data.

Essentially... good work Gavin, for raising the issue and making a proposal, but more research is needed.  I have faith that you'll be able to win me over on this (as well as the others opposing it in its current form).  Its just not there yet.  I don't know the answer, and I don't think anyone else does yet, but with all of us working toward it (again thanks to you for raising the issue), we may find it.

We need to be better than the Central Bankers who get together with their economic advisers and pick numbers arbitrarily.  We need automated future-proof solutions written into open protocols that will still be working when we are long dead.  It is our responsibility being alive and here now at the beginning, to see it done right.  To do less than our best is shameful.

[justusranvier]

The fear is that a cartel of big, centralized, have-huge-data-pipes miners would drive out smaller miners by forcing up the block size high enough so the smaller miners have to drop out.

Price discovery of bandwidth is the solution.

Users want their transaction to be relayed to miners.
Miners want the transaction to reach them so they can earn the fees associated with those transactions.
Miners want other miners to receive their blocks to have their reward recognized by the network.
Users want to receive the block the miners produces to they can know the state of the network.

Relay nodes provide a service which everybody wants. Build a competitive open marketplace for relay nodes to offer connectivity to users on both sides of the network and then price discovery can occur. The relay nodes will get compensated for the resources which they are providing and the price signal will automatically make sure we have the right amount of relay node.

Then we never need to have these unresolvable debates again. When block sizes and tx rates increase, there will automatically be a mechanism in place to make sure the relay nodes receive additional income which they can use to defray their rising expenses.

[cbeast]

I propose the following rule to determine the block size limit, once the block reward is low
The block size limit would increase (or decrease), by X%, if total transaction fees in the last N blocks is Y Bitcoin or more (or less).  

......

I am aware miners could also manipulate fees by including transactions with large fees and not broadcasting these to the network.  However why would miners in this scenario want to manipulate the limit upwards?

The fear is that a cartel of big, centralized, have-huge-data-pipes miners would drive out smaller miners by forcing up the block size high enough so the smaller miners have to drop out.

Elements to make mining competitive are cheap power, good connectivity, cheap heat management, and technology development. Cartels can form that take advantage of either of these elements. As long as all of these elements are not overly abundant in only a few geographical regions, mining can stay decentralized.

[acoindr]

I sympathize with their plight, but Bitcoin is not made for these first.  Bitcoin is for everyone.  There are parts of the planet (some of which have the greatest need for Bitcoin) that have very limited bandwidth today and can be expected to not see rapid improvement.

You know I'm starting to think it doesn't matter. We win either way.

In the worst case, say we overshoot and Bitcoin becomes completely centralized by powerful miners which then emulate the current SWIFT system, blocking and regulating transactions. What would happen next? Would we curse and shout CRAP! We were this close. If only we'd ratcheted down our numbers a tiny bit. Well everyone go home. Nothing more to see here.

LOL of course not. We'd move to the next alt-coin not co-opted and continue on, having learned from our mistakes. In a post I wrote long ago which seems to have come true I talked about how alt-coins gave a value to our community Bitcoin never could by providing the one thing it alone never could: alternative.

The people who still say there can be only one will always be wrong. Alt-coins are not going anywhere. Most will have low market caps or blow up and deservedly die horrible deaths, but Bitcoin won't ever be all by itself. Won't happen. And if the free market demands a coin with fixed or less-than-bitcoin block size limit then that's what it will get, and value and usage will flow there.

The converse is also true. Say we are unable to gain consensus for raising the size limit, causing a collapse in price as people perceive Bitcoin as unable to serve the base they thought it would; or we proceed with a messy hard fork creating a rift in the community and price crash as people become confused about the future of Bitcoin and what to do next. Cryptocurrency would still go on, eventually, because that cat is out of the bag and people will continue working on it. Of course, I'd rather see the first scenario (a need to adopt an alt-coin) than second as I'm less certain about recovering well from the second since cryptocurrency ultimately has no backing other than overall confidence in its viability.

Either way I see Bitcoin as providing the world with education. It's teaching the world the possibilities of decentralization with currency and that's where the real value is, because Bitcoin isn't the only thing which can work in that model.

[teukon]

The fear is that a cartel of big, centralized, have-huge-data-pipes miners would drive out smaller miners by forcing up the block size high enough so the smaller miners have to drop out.

Price discovery of bandwidth is the solution.

A bandwidth market can lead to an efficient use of bandwidth but may do nothing to address the potential tragedy of the commons concerning decentralisation.

Users want their transaction to be relayed to miners.
Miners want the transaction to reach them so they can earn the fees associated with those transactions.
Miners want other miners to receive their blocks to have their reward recognized by the network.
Users want to receive the block the miners produces to they can know the state of the network.

I submit that pursuit of just these policies would actually encourage centralisation.  A small number of large miners will consume fewer resources than a decentralised mass.  A single trusted data centre could be even more efficient.

Elements to make mining competitive are cheap power, good connectivity, cheap heat management, and technology development. Cartels can form that take advantage of either of these elements. As long as all of these elements are not overly abundant in only a few geographical regions, mining can stay decentralized.

A market entity is not restricted to a single geographical location.  McDonald's have locations all over the world.

[justusranvier]

The fear is that a cartel of big, centralized, have-huge-data-pipes miners would drive out smaller miners by forcing up the block size high enough so the smaller miners have to drop out.

Price discovery of bandwidth is the solution.

A bandwidth market can lead to an efficient use of bandwidth but may do nothing to address the potential tragedy of the commons concerning decentralisation.

Users want their transaction to be relayed to miners.
Miners want the transaction to reach them so they can earn the fees associated with those transactions.
Miners want other miners to receive their blocks to have their reward recognized by the network.
Users want to receive the block the miners produces to they can know the state of the network.

I submit that pursuit of just these policies would actually encourage centralisation.  A small number of large miners will consume fewer resources than a decentralised mass.  A single trusted data centre could be even more efficient.

Can you define decentralization/centralization in this context?

[jonny1000]

I propose the following rule to determine the block size limit, once the block reward is low
The block size limit would increase (or decrease), by X%, if total transaction fees in the last N blocks is Y Bitcoin or more (or less).  

......

I am aware miners could also manipulate fees by including transactions with large fees and not broadcasting these to the network.  However why would miners in this scenario want to manipulate the limit upwards?

The fear is that a cartel of big, centralized, have-huge-data-pipes miners would drive out smaller miners by forcing up the block size high enough so the smaller miners have to drop out.

With the current mining dynamics my proposal would not be suitable for the reasons you suggest.  I merely suggest it as an eventual objective for when the block reward becomes low and hopefully mining becomes more decentralised, competitive and fee driven.  If mining doesn’t develop this way then Bitcoin may not be sustainable in the long run anyway.  We could still keep another maximum of maximums block size limit based on bandwidth considerations and then this transaction fee targeting based limit system could operate within this.

Whatever happens to the hash rate total mining revenue represents the “economic value” of network security.  For example currently the security of the network can now be considered as 25 bitcoin per block, regardless of the large hash rate increases as in theory 25 bitcoin per 10 minutes is the cost of mining.  In the future the value of the total transaction fees will represent the network security and therefore the dynamics which determine the fees will be vital.  Having “supply” potentially grow exponentially forever may not be appropriate.

The above proposal could be a good framework for a discussion on how the dynamics for the transaction fees could be determined in the future.  The system is kind of an aggregate transaction fee targeting scheme.  For example a target of 1 bitcoin per block is around 50,000 bitcoin per annum or 0.24% of the eventual total supply per annum.  Deciding if this is a suitable level would be difficult.  Is 0.24% high enough to secure the network or should it be 1%?  What if the number is too high, we create an arbitrarily high amount or environmental damage?

[cbeast]

Elements to make mining competitive are cheap power, good connectivity, cheap heat management, and technology development. Cartels can form that take advantage of either of these elements. As long as all of these elements are not overly abundant in only a few geographical regions, mining can stay decentralized.

A market entity is not restricted to a single geographical location.  McDonald's have locations all over the world.

That's not the point. McDonald's is also not a cartel. Geography plays a part in where you can have beef or pork sandwiches as well. My point is that nobody has every competitive edge in everything. Anyone that can gain a competitive edge over certain resources will attempt to exploit them. This merely addresses profitability. If you want to realize an actual profit, you will need customers, and customers need incentives.

[teukon]

Users want their transaction to be relayed to miners.
Miners want the transaction to reach them so they can earn the fees associated with those transactions.
Miners want other miners to receive their blocks to have their reward recognized by the network.
Users want to receive the block the miners produces to they can know the state of the network.

I submit that pursuit of just these policies would actually encourage centralisation.  A small number of large miners will consume fewer resources than a decentralised mass.  A single trusted data centre could be even more efficient.

Can you define decentralization/centralization in this context?

Sure.  By centralisation here I'm referring to the gradual reduction in the number of block-generating entities.  To be clear, I claim: absent a block-size limit, this centralisation process would occur naturally and that a good relay bandwidth market would accelerate this process.

Always happy to be proven wrong; just want to give you something more concrete to work with.

[justusranvier]

Sure.  By centralisation here I'm referring to the gradual reduction in the number of block-generating entities.  To be clear, I claim: absent a block-size limit, this centralisation process would occur naturally and that a good relay bandwidth market would accelerate this process.

Always happy to be proven wrong; just want to give you something more concrete to work with.

The number of individuals who control hashing equipment has been increasing since 2008, during the time in which the block size limit is effectively non-existent (because tx volume is too low to be affected by the limit).

Why are you predicting that this trend would reverse instead of continue?

[teukon]

Elements to make mining competitive are cheap power, good connectivity, cheap heat management, and technology development. Cartels can form that take advantage of either of these elements. As long as all of these elements are not overly abundant in only a few geographical regions, mining can stay decentralized.

A market entity is not restricted to a single geographical location.  McDonald's have locations all over the world.

That's not the point. McDonald's is also not a cartel. Geography plays a part in where you can have beef or pork sandwiches as well. My point is that nobody has every competitive edge in everything. Anyone that can gain a competitive edge over certain resources will attempt to exploit them. This merely addresses profitability. If you want to realize an actual profit, you will need customers, and customers need incentives.

Certainly, McDonald's is not a monopoly; it is a single player in a competitive market.  I merely wished to highlight that geographical diversity of the factors of production alone is not sufficient to ensure decentralisation of mining.

The point that "nobody has every competitive edge in everything" is what I'm worried about.  To be clear, I'm not saying that free-market forces cause an industry to converge to monopoly, far from it.  I'm suggesting that comparing Bitcoin mining with a typical industry requires care because the miners have some influence over the very nature of Bitcoin itself.

[teukon]

The number of individuals who control hashing equipment has been increasing since 2008, during the time in which the block size limit is effectively non-existent (because tx volume is too low to be affected by the limit).

Why are you predicting that this trend would reverse instead of continue?

Because I believe these trends are dependent on the negligible cost of handling blocks, costs which will become significant soon enough if the block-size growth trend continues unabated.

[justusranvier]

Because I believe these trends are dependent on the negligible cost of handling blocks, costs which will become significant soon enough if the block-size growth trend continues unabated.

Blocks don't just magically grow in size for no reason.

Blocks get larger because of an increased demand for Bitcoin transactions.

A larger number of bitcoin transactions means more aggregate fee revenue, which means as the block size grows the miners are competing for an expanding market.

Expanding markets attract new entrants into the market, and in the case of Bitcoin mining there is no way for incumbents to exclude competitors who produce valid proof of work.

[painlord2k]

...
To be able to process the same number of transaction of VISA, Bitcoin should grow x2,000.
The size of blocks should go up ~1.000x at least to accommodate so many transactions.
And we will not just want to take VISA burden, we want, also, offer a service to the currently unbanked (being humans or DACs).
In the block size increase 50% every year, it will take 20 years to take over VISA alone; never mind the unbanked and DAcs.

You're not thinking about safety.  Yes it would be nice for bitcoin to be able to handle 2000 as many transactions as it can now, however that is not as important as keeping bitcoin decentralized.  Let's keep in mind why bitcoin was created: to create a digital gold standard so that people could protect their assets from central banks.  If bitcoin also becomes a ubiquitous payment system that would be great, but not if it comes at the expense of decentralization.

Mining will need to be self sufficient using just fees and without block rewards in the long run.
So, if we take the capability of VISA network as a goal, we need at least 0.01 $ fee per transaction to give the miners the same amount of money they get today. And the minimum block size to be able to do so is very large to accommodate at least 4.000 transactions/second (and VISA process 8.000 transactions per second during high traffic days).

Decentralization is the main goal, because it make the network censor resistant.
But what make possible to maintain decentralization is openness. Anyone, no matter who, can connect and join the network if he has the resources to do so.

Miners need to distribute their block faster first to other miners and then to other nodes. If they distribute their block to other miners first, these miners will start to mine the next block earlier. If a miner was able to flash his blocks to the core nodes of network instantly (E.G. broadcast by DVB-T), but not to the (hash rate) majority of the miners, his ability to communicate would not count a lot, because the core nodes would be exposed to a longer competing chain shortly after. The core nodes just need to download the new blocks from the miners and/or offer them to other core nodes to be able to check the blocks, they don't mine, so they don't need the new blocks in a matter of seconds after they are discovered.
As with many different P2P networks, they work better if different nodes organize themselves in different ways/strata.

Just now, miners (usually miners pools) make large investments in mining rigs and related HW, SW, labor, power and real estate. They can afford to buy enough bandwidth to disperse blocks between themselves in a matter of seconds. More so if there is a Reverse lookup table  and they just need to exchange mainly references to the Tx included and not the actual data.


The rationale to grow faster now with the block size exist because when the block reward will be halved, the revenues of the miners will halve if the price do not at least double.
The single fees can not grow x100 to compensate the reduction of the reward in just few months. They must be able to grow in number to compensate the diminishing reward, not become more costly for the consumer.
At least there must be room to allow this grow to happen.

Now a transaction cost 18$ (miners income = reward + fees). It is obviously impossible to sustain this costs if it came out directly from the users.
https://blockchain.info/it/charts/cost-per-transaction

The network must be able to manage the 1000 times the current transactions to bring the cost down to 1.8 cents from 18 $. Currently we have around 300 KB blocks, so the block size should increase to 300 MB (at least). Make it 600 MB to bring it down to 1 cent and 1 GB to compete with VISA.

The network will grow proportionally faster now than in the future, when it will have taken larger part of the market of payments. The curve will be something resembling an asymptotic curve to some level (E.G. 10x VISA) or linear increase (VISA + x transactions/year). But the steepest part will happen now and in the near future (4-6 years and no more).

[teukon]

Expanding markets attract new entrants into the market, and in the case of Bitcoin mining there is no way for incumbents to exclude competitors who produce valid proof of work.

Expanding markets do attract new entrants in general, but in the case of Bitcoin, it also increases the barrier to entry, because each participant must be capable of handling all Bitcoin transactions.  The cost to the network of processing a transaction grows at least as quickly as the number of miners (assuming constant technology for simplicity).

Also, you're right that in a reasonably decentralised environment, large mining companies cannot exclude competitors through voluntary means alone.  However, with Bitcoin, there is the 50% threshold to worry about too, something absent in most markets.  In a typical free-market, if an entity accumulates more than 50% of the business then they'll keep that monopoly if and only if they continue to outperform their competition.  This entity will begin to lose market-share if they even cease to innovate, let alone try to abuse their position.  With Bitcoin, there are different incentives involved and a monopolist may well stand to gain from excluding small miners by dropping their blocks.

[painlord2k]

Expanding markets attract new entrants into the market, and in the case of Bitcoin mining there is no way for incumbents to exclude competitors who produce valid proof of work.

Expanding markets do attract new entrants in general, but in the case of Bitcoin, it also increases the barrier to entry, because each participant must be capable of handling all Bitcoin transactions.  The cost to the network of processing a transaction grows at least as quickly as the number of miners (assuming constant technology for simplicity).

Also, you're right that in a reasonably decentralised environment, large mining companies cannot exclude competitors through voluntary means alone.  However, with Bitcoin, there is the 50% threshold to worry about too, something absent in most markets.  In a typical free-market, if an entity accumulates more than 50% of the business then they'll keep that monopoly if and only if they continue to outperform their competition.  This entity will begin to lose market-share if they even cease to innovate, let alone try to abuse their position.  With Bitcoin, there are different incentives involved and a monopolist may well stand to gain from excluding small miners by dropping their blocks.

The costs of entering the market of mining grow with the number of transaction and the hashing power of the network, but there are ways to collaborate that lower the cost of entering in the mining sector (pools).
The pool hub is the only in need to pump out the blocks, the miners do not need a large bandwidth to start mining.

It is true a 51% pool could drop other miners blocks, but it would lose a lot of support from the associated miners if it did so, because it would be detrimental to the health of the network long term.
A large miner could invest heavily and gain the 51% and make an attack (just dropping others blocks), but he would jeopardize its investment in doing so, because it would become the controller and the central point of failure of the network. Essentially it would paint a big target on its back for law enforcement and assorted nasties. I bet, when someone make an investment on the order of $180M (plus assorted ancillary costs) a need a 150 MW power line he consult lawyers about its business plan.

[Syke]

The fear is that a cartel of big, centralized, have-huge-data-pipes miners would drive out smaller miners by forcing up the block size high enough so the smaller miners have to drop out.

Sounds like an irrational fear to me. Big centralized miners couldn't care less about little miners. Little miners are insignificant. And what are these big centralized miners going to use to fill up these huge blocks? Fake transactions? That makes no sense. Real transactions? Then there's a real need for such huge block sizes.

[cbeast]

The fear is that a cartel of big, centralized, have-huge-data-pipes miners would drive out smaller miners by forcing up the block size high enough so the smaller miners have to drop out.

Sounds like an irrational fear to me. Big centralized miners couldn't care less about little miners. Little miners are insignificant. And what are these big centralized miners going to use to fill up these huge blocks? Fake transactions? That makes no sense. Real transactions? Then there's a real need for such huge block sizes.

Yes they can make fake transactions to fill blocks. That would partially shut some miners with small bandwidth. The occasional very large blocks might cause the smaller blocks to not propagate and thus get orphaned. The low bandwidth miners would need to make up for it by increasing their hashrate if they have other advantages like cheap power or government subsidies. It's just another competition vector.

[jonny1000]

Gavin has now written part 2, Blocksize economics
https://bitcoinfoundation.org/2014/10/blocksize-economics/

New thread
https://bitcointalk.org/index.php?topic=825601.new#new