The economics of the block size

[amincd]

http://amincd.tumblr.com/post/100736367493/the-economics-of-the-block-size

I wrote this over a year ago, and I thought now would be a good time to share it. I didn't have time to make it shorter, so I hope it's not too long-winded to follow.

A block size limit or unlimited block sizes?

Among the opinions advocating eliminating the 1 MB block size limit, the first point of divergence is whether to:

1) Replace the 1 MB block size limit with something else to control the growth in the size of blocks

or

2) Replace it with nothing and let the ‘free market’ (put in quotation marks not to disparage the free market, which I am a huge advocate of, but because the shared costs related to Bitcoin mining make it very much unlike a free market) handle block size

Bitcoin would be better off if an option in category 1) was adopted and the 1 MB block size limit was replaced with another protocol feature that controls the growth of the block size.

To explain why, we need to first understand the problem with the Bitcoin protocol:

Block generating nodes do not pay the full network cost of the bandwidth required to download/upload - and the disk space required to store - txs that they include in blocks.

This is a fundamental shortcoming of the Bitcoin protocol

This makes it economically rewarding or only only slightly costly for individual block-generating nodes to include some txs that are economically very costly for all block-generating nodes to have in the blockchain.

Examples

A couple of examples can better demonstrate this problem. Let’s use a simplifying assumption in our examples: let’s imagine that there is an advanced micropayment channel network between all nodes that enables and incentivizes them to only relay those unconfirmed txs that meet the fee criteria of the receiving node.

For instance, if a node only accepts txs with a fee of at least 0.00001 BTC, its peer nodes will know not to forward to it txs with fees less than this amount.

This means that nodes do not need to use up bandwidth for any tx they do not intend to include in a block, since the txs forwarded to them are pre-vetted to ensure they meet their requirements for block inclusion.

In this setting, let’s suppose that there is one block generating node (N1) that has a hash rate equal to 20% of the network hash rate, meaning it finds 1 out of 5 blocks.

Now let’s examine how the network is affected by different behavior from N1.

Worst case scenario

The worst case scenario is if N1 accepts every tx, including those with no fees.

We can imagine that N1’s fee policy is driven by a misguided belief that the more txs the Bitcoin network processes, the better it is for the whole Bitcoin economy. It doesn’t really matter what the motivation for N1’s behavior is, just that exhibition of such a behavior is not outside the realm of possibility.

Now, despite the protocol not imposing any limits on number of txs per block, there is a practical limit to how many txs N1 can accept into a single block. This is because, due to the time it takes to download very large blocks, nodes each have their own limits on what sized blocks they will build upon.

Blocks above a certain size won’t be accepted and built upon by enough block generating nodes to get over 50% of the network hash rate building on them, leading to those blocks eventually being orphaned, which would eventually create an incentive for all block generating nodes to not build upon them. This is the ‘informal block size limit’.

Let’s suppose that 8 GB is the informal block size limit while the average block size of blocks other than those generated by N1 is 100 MB.

N1 always includes 8 GB worth of txs, the maximum, in its blocks. 1 GB of those txs have a fee, while 7 GB are txs with zero fees. The informal block size limit prevents  N1 from including another 2 GB of zero fee txs, while any tx with a fee is guaranteed a place in  N1’s blocks.

Ideally, the incentives of block generation would result in block generating nodes only accepting those txs that have a fee that covers their cost for downloading, and if they find a block, propagating those txs.

Due to N1’s fee policy though, other block generating nodes would have no incentive to reject txs that have insufficient fees, because if they reject the txs, they will still have to use bandwidth to download them, when N1 finds a 8 GB block and they download it.

In this scenario, it’s in their best interest to get whatever fees they can, since they will be paying the bandwidth for the tx anyway.

The result is that all of the block generating nodes change their fee policy to accept txs with very low fees. Users are thereby incentivized to also lower the fees they pay, since almost any fee is enough to get their tx included in a block in a timely manner.

The fees being provided in a network with these characteristics would eventually become totally insufficient to pay for the cost of bandwidth, and the informal block size limit, along with the average block size, would steadily rise as smaller nodes with less bandwidth drop out of the network, leaving only the giants.

More realistic scenario

A more realistic scenario is if N1 has a fee policy of only accepting txs that have a large enough fee that it is not a net-loss for N1 to include them in the blocks it works on.

Let’s assume the bandwidth cost for all nodes to download and propagate 1 GB of transaction data is $1, and that txs are on average 500 bytes. Since N1 has a hash rate equal to 20% of the network, it will need an average fee on txs that equals $0.0000025 (1/0.20 * $1/GB * 500 bytes) to break even.

Let’s further assume that N1 generates block that on average are 500 MB in size, while the average block size of blocks generated by nodes other than N1 is 100 MB, because most nodes have a lower hash rate than N1, and therefore need to have a stricter fee policy that rejects more txs.

As in the last example though, eventually, the other nodes see that it is in their interest to accept txs that N1 is accepting, because if they don’t, they will miss out on a chance to earn the fee attached to the txs, and will have to use bandwidth to download the txs anyway when N1 finds a 500 MB block that includes them.

Also as in the last example, as more nodes start accepting lower-fee txs, users will reduce the fees they pay, because they know that any tx meeting N1’s fee policy will be included into a block in a timely manner.

The result would be that the average block size would trend toward 500 MB, with tx fees trending towards $0.0000025, or a total $2.50 of per block ($0.0000025/500 bytes * 500 MB).

The bandwidth cost for N1 per block would be $0.50 ($1/GB * 500 MB), and the amount it earns in fees would also equal $0.50 ($2.50 earned per block found * ⅕ blocks found), meaning it would break even.

N1 is not a typical node though. Most block generating nodes have a hash rate significantly lower than 20% of the network hash rate.

For instance, N2 has a hash rate equal to 1% of the network’s, so it will earn an average of $0.025 per block ($2.50 per block found * 1/100 blocks found). Bandwidth costs for N2 will be the same as for N1 however, at $0.50 per block, meaning that N2 will suffer a loss of $0.475 per block.

Repercussions for Network

The total block reward for the network (R) will equal the average fees per GB of tx data (F) * the average block size (B) + the block subsidy (S).

R = F(B) + S

The total bandwidth cost for the network per block (T) will equal the number of block generating nodes in the network (N) * the average cost of bandwidth per node per GB of tx data (C) * the average block size (B).

T = N(C)(B)

The amount of resources allocated to producing proof of work (H) will equal the total block reward for the network (R) - the total bandwidth cost for the network per block (T) - the equilibrium profit margin for block generation (P) * the total block reward for the network (R).

H = R - T - P(R)

or

H = F(B) + S - N(C)(B) - P(F(B)+S)

In less formal terms: block generating nodes, as a result of competitive incentives, will be driven to spend any block reward they earn, net their bandwidth costs and the amount of profit they take, on increasing the proof of work they generate. The greater the block reward relative to the bandwidth cost, the more proof of work the network will produce, and therefore, the more secure it will be.

The following graph shows the relationship between average block size (B) and the the amount of resources allocated to producing proof of work (H). It assumes:

the number of block generating nodes in the network (N) = 1,000,

the block subsidy (S) = $2,000,

the average cost of bandwidth per node per GB of tx data (C) = $1,

the equilibrium profit margin for block generation (P) = 1%,

the average fees per GB of tx data (F) = $5.00



As can be seen, with these constants, increases in the block size lead to rapidly diminishing resources being used to produce proof of work.
We can make more optimistic assumptions about the values of the constants and the situation still looks dire.

For example, if we assume that the troublemaking node, N1, requires tx fees that are 5 times larger than the cost of the bandwidth it needs to use to download and propagate the tx, rather than tx fees that merely allow it to break even on bandwidth costs, then its required fee on a 500 byte tx would rise to $0.0000125 (5 * 1/0.20 * $1/GB * 500 bytes), and the average fees per GB of tx data (F) would rise to $25 ($0.0000125/500 bytes * 1 GB).

The graph below shows the relationship between block size and resources dedicated to producing proof of work with this value for F:



The total block reward for the network (R) increases by only $35 as the block size increases from 100 MB to 1.5 GB, because transaction fees average only $25 per GB of tx data. Meanwhile, the total bandwidth cost for the network (T) increases by $1,400 as the block size grows 1.4 GB, due to network bandwidth costs of $1,000 per GB of tx data.

Even with more optimistic assumptions about the values of the constants, the situation still remains dire.

For example, if we assume the number of block generating nodes (N) is reduced from 1,000 to 100, and make the same optimistic assumption about the average fees per GB of tx data (F) being $25, this graph is the result:



There is no noticeable increase in the resources dedicated to producing proof of work, despite the much more optimistic assumption about the behaviour of the highest hash rate node and the total number of block generating nodes in operation.

In a situation where the very plausible starting assumptions of this analysis turn out to be true, the absence of a block size limit will mean reduced network security as transaction volumes increase.

[DumbFruit]

Over the past couple of days I've been trying to come up with a solution to this that would involve the free market. The following is an excerpt, since the introduction doesn't seem necessary here. I suspect that it may be technically impossible, and I'm sure a change as substantial as this would never make it into bitcoin proper, but at least it's a solution away from the direction of "How Good Am I At Predicting The Future?".

..If developers manage to solve the problem, and create an algorithm that raises the block size limit approaching equilibrium, then network hash-power would atrophy as inflation stops (The block reward). The closer they get to a perfect solution, the more dangerous it is to network health.
The goal of the following solution is to separate the currency of the cryptocurrency from the capital, so that there is a market between service users and service providers, allowing the economy to find an appropriate market clearing price for transactions that can include the costs of hashing.
The first thing to do is to add a secondary unit to the cryptocurrency (hereafter called "Fuel") which is used whenever a transaction takes place and is equivalent to the amount of bytes used in transactions. (10 bytes of transactions = 10 units of fuel, for example.) Fuel is created by proving to other miners that you have space for a StorageBlob. If the StorageBlob is approved by the next X miners, then fuel is divided among them equal to that storage space that was proved to exist by the other miners, and the blocksize increased proportionally.

The goal of the following algorithm is to prove to the rest of the network that you have certain available space beyond the blockchain you're storing and that you want to raise the block size limit to reasonably use that space without filling that space too fast. If there were such a thing as a maximum blockchain length (Or a sliding blockchain), it could be configured to never raise the blocksize limit beyond the capacity of the current miners.

Size of StorageBlob(hashtree) = (PerBlockBytesRequested-CurrentBlockSizeLimit)*MaximumBlockChainLength

Miner1: Hash1 (StorageBlob + POWHash)
Miner2: Check Hash1 = Hash(StorageBlob + POWHash)
if Yep Hash2(StorageBlob + Salt1+POWHash)
...
MinerX: Check Hash[X-1] = (StorageBlob + Salt[X-2]+POWHash)
if Yep IncreaseBlockSizeLimitTo(PerBlockBytesRequested)
TakeSomeFuel/X
Release fuel to X previous miners.

Any miner along the chain can drop the storage increase request at their discretion or introduce a smaller one.

Fuel is bought from miners in order to do any transactions. Using fuel does not burn it in the transaction. It is transferred back to the block miner.

Since Bitcoin does not have a maximum blockchain length, a magic number can be substituted that would reasonably mitigate excessive optimism about future storage space availability or in other words; allow smaller players to be able to handle the entire blockchain. Or in even more other words; Lowers the barrier to entry.

This solution turns the network resources into an economic good that is owned by the miners, and makes artificial redundancy an explicit variable in the protocol (X). The higher the X value, the more likely the network will resist an increase in block size as any increase demands consensus and proof of available space across X consecutive blocks.

While fuel can be traded, it would not be used as the primary medium of exchange because it is inflationary by design and idle/lost fuel would need to be reclaimed by miners by some mechanism(s).

This solution could be approximated by implementing the StorageBlob concept without fuel. This is inadvisable because it doesn't address the two fundamental problems that caused the problem to begin with; Tragedy of the Commons, and the Economic Calculation Problem.  There would be a constant upward pressure on block sizes and a constant downward pressure on hashing power.

The introduction of "fuel" creates the situation in which a miner can accumulate hash-power which in turn accumulates fuel, which can then be used to put an upward pressure on fuel prices. Successful competition in the market necessarily transfers a proportional “ownership” of the blockspace via the control of transactions in this way circumventing both the problem of the Tragedy of the Commons, and the Economic Calculation Problem. There would be an upward pressure on hash-power and block sizes that would be held back by whether or not users actually want to pay for the fuel at the given hash-power and block sizes (storage costs).

Naturally, fuel prices would stratify between high fuel prices, high hash power, and high availability, and comparatively low fuel prices, low hash power, and lower availability.

Although "X" is a magic number like the current blocksize limit in Bitcoin, it more closely matches the intentional arbitrary node redundancy in the network. The "MaximumBlockchainLength" would necessarily exist for the same reason.
This idea is inspired somewhat by GMaxwell's "Proof of Storage" and Ripple's XRP.

[amincd]

So you're proposing introducing essentially a new currency into the Bitcoin network, 'fuel'. I believe this would be inflationary, as it would replace bitcoin as a means of paying transaction fees, which is the original use case of bitcoin the currency.

Also, storage is not the practical limiting factor for block size, bandwidth is. Bandwidth is far costlier than storage, and that will be even more the case once pruning becomes standard. Creating market-like forces to limit storage growth is important, but more important is creating economic incentives for participants in the Bitcoin network to provide accurate market information on the cost of bandwidth.

[justusranvier]

because the shared costs related to Bitcoin mining make it very much unlike a free market

This isn't difficult to fix.

All it takes is a price discovery mechanism for bandwidth.

[amincd]

How?

The best I could come up with is that we choose a magic number for how many mining nodes we think a decentralized currency should have at minimum, like say, 1,000, and then that would leave us to discover the cost of bandwidth, which we could get from having miners 'vote', through blocks, on what the market price of bandwidth is. We wouldn't need to use any fiat currencies as a reference. The price of bandwidth could be measured in BTC: e.g. 2,857 bits per GB of bandwidth (their client could have them input their local cost of bandwidth, and the price of bitcoin in their currency, and these two values could be used to calculate the BTC value of bandwidth in their region).

With the magic number and the cost of bandwidth, we could calculate the network cost incurred for each GB of block data, and charge miners for it. The penalty paid by miners could be redistributed to miners of future blocks, ensuring that fees are sufficiently high to pay for the total amount expended by the mining network on bandwidth, so that the hashrate doesn't suffer.

But then the voting element makes Bitcoin highly gameable by miners, even more so than it is now.

[justusranvier]

How?

The best I could come up with is that we choose a magic number for how many mining nodes we think a decentralized should have at minimum, like say, 1,000, and then that would leave us to discover the cost of bandwidth, which we could get from having miners 'vote', through blocks, on what the market price of bandwidth is. We wouldn't need to use any fiat currencies as a reference. The price of bandwidth could be measured in BTC: e.g. 0.002,857 bits per GB of bandwidth (their client could have them input their local cost of bandwidth, and the price of bitcoin in their currency, and that could be used to calculate the BTC value of bandwidth in their region).

But then this makes Bitcoin highly gameable by miners, even more so than it is now.

I'll let Daniel Krawisz explain, since it's an algorithm he developed, but it's far simpler than that.

There is no voting required, nor any centralized coordination.

Every node in the network independently negotiates prices with its immediate peers - that is all that is required.

[amincd]

Link?

[justusranvier]

Link?

He hasn't published yet. But soon...

[justusranvier]

I'll just summarize.

The mechanism is micropayment channels. Any two between nodes can set up a micropayment channel between them for handling the actual payments.

All that's left is for the nodes to figure who gets paid and how much. So, some kind an accounting system.

What nodes want to obtain is relevancy, as defined by access to the most recent transactions.

They keep records of which transactions they've sent to each peer first (before the peer sent the same transaction to them) and vice versa. Only transactions that eventually make it into a block count.

Based on this accounting, each node knows if it is more relevant, less relevant, or equally relevant compared to each of its peers.

It uses this information to decide whether to pay a peers, or to charge a peer, or if no net payment is appropriate.

Then it's a matter of each node haggling with its peers, attempting get the best price for its relevancy from the nodes it is charging, while minimizing the amount it spends on the nodes which it pays.

Nodes would also want to improve their position in the network by competing for low-relevancy nodes (source of revenue) while attempting to find equal relevancy peers to lower their costs.

The net flow of funds from the bandwidth-consuming nodes to the bandwidth-supplying nodes means that you can achieve price discovery for bandwidth and compensate the providers for the costs they incur.

[bigbadwolf111]

Link?

He hasn't published yet. But soon...

He should publish it. This one will surely help us bitcoiners learn something new.

[amincd]

<snip>

The net flow of funds from the bandwidth-consuming nodes to the bandwidth-supplying nodes means that you can achieve price discovery for bandwidth and compensate the providers for the costs they incur.

This is an excellent idea and I hope it is implemented. However, this doesn't address how to create a consensus on the price of bandwidth, so that block generating nodes that generate large blocks can be charged by the protocol for the bandwidth they force all other block generating nodes to pay.

[justusranvier]

However, this doesn't address how to create a consensus on the price of bandwidth, so that block generating nodes that generate large blocks can be charged by the protocol for the bandwidth they force all other block generating nodes to pay.

The point is there is no need for global consensus.

[opossum]

<snip>

The net flow of funds from the bandwidth-consuming nodes to the bandwidth-supplying nodes means that you can achieve price discovery for bandwidth and compensate the providers for the costs they incur.

This is an excellent idea and I hope it is implemented. However, this doesn't address how to create a consensus on the price of bandwidth, so that block generating nodes that generate large blocks can be charged by the protocol for the bandwidth they force all other block generating nodes to pay.

This would require that nodes prove they are in fact running a node (which is surprisingly difficult, especially when payments would instinctive people to fake running a node). This would also mean that Bitcoin would no longer mine via PoW (at least not exclusively) but rather would be mined via PoN (proof of node) or a combination of PoN/PoW. It is said that if Bitcoin were to be forked so that it no longer uses PoW (among other things) then it would no longer be considered Bitcoin

[justusranvier]

This would require that nodes prove they are in fact running a node (which is surprisingly difficult, especially when payments would instinctive people to fake running a node). This would also mean that Bitcoin would no longer mine via PoW (at least not exclusively) but rather would be mined via PoN (proof of node) or a combination of PoN/PoW. It is said that if Bitcoin were to be forked so that it no longer uses PoW (among other things) then it would no longer be considered Bitcoin

Everything in this post is incorrect.

[amincd]

The point is there is no need for global consensus.

The micropayment channel scheme would address the tendency toward centralization, by rewarding non-block-generating nodes that propagate data, through compensation schedules negotiated on a one-to-one basis. The problem I'm pointing out is one of block generating nodes indiscriminately including zero/low-fee txs into blocks they generate, and thus increasing the bandwidth costs of the network as a whole. Paid data propagation via the micropayment channel scheme wouldn't address this.

The solution I'm suggesting is for the protocol to charge block generating nodes for the blocks they create, based on how much data is contained in the block, and distribute fees collected to other block generating nodes (proportional to their share of the network hashrate). This would not solve the centralization problem, but it would solve the problem of txs becoming a net cost to the block generating nodes as a whole. For this proposed solution to work, we need to pick a magic number to represent the minimum number of block generating number we believe the network needs to be sufficiently decentralized, and the protocol needs a way to reach consensus on the cost of bandwidth in relation to bitcoin. Multiplying these two numbers together would give us an estimation of the cost of bandwidth for the network per byte of data included in a block.

[justusranvier]

The solution I'm suggesting is for the protocol to charge block generating nodes for the blocks they create, based on how much data is contained in the block, and distribute fees collected to other block generating nodes (proportional to their share of the network hashrate). This would not solve the centralization problem, but it would solve the problem of txs becoming a net cost to the block generating nodes as a whole. For this proposed solution to work, we need to pick a magic number to represent the minimum number of block generating number we believe the network needs to be sufficiently decentralized, and the protocol needs a way to reach consensus on the cost of bandwidth in relation to bitcoin. Multiplying these two numbers together would give us an estimation of the cost of bandwidth for the network per byte of data included in a block.

There's no need for magic numbers, which are wrong by definition.

The blocks miners produce are meaningless unless they reach an economic majority of the network. Therefore miners should be willing to pay relay nodes to ensure this happens.

Other users of the network require timely access to the most recent blocks in order to know the state of the ledger. Therefore users of the network should be willing to pay relay nodes to deliver the blocks to them.

Users want their transactions to reach the miners. Therefore they should be willing to pay relay nodes to deliver them.

Miners want to receive transactions, because the fees embedded within those transactions is a source of revenue for them. Therefore they should be willing to pay relay nodes to deliver those transactions to them.


The only thing needed to make a market for block/transaction relaying work is a mechanism for price discovery and payments.

Price discovery will ensure that relay nodes are sufficiently compensated by an amount that covers their costs, and will also regulate the number of relay nodes to meet the demand of the users, including their demand for decentralization (a user who wants to buy more decentralization does so by connecting to more relay nodes than they would otherwise need).

No magic numbers needed, no central planning needed.

The fact that Bitcoin has a magic number acting as a production quota for transaction processing is a flaw that was supposed to be temporary. Replacing one magic number with a different magic number isn't a long term solution.

[bounst]

tl;dr

I didn't have time to make it shorter,

anyway, could you take a summary?

[Rewap]

This would require that nodes prove they are in fact running a node (which is surprisingly difficult, especially when payments would instinctive people to fake running a node). This would also mean that Bitcoin would no longer mine via PoW (at least not exclusively) but rather would be mined via PoN (proof of node) or a combination of PoN/PoW. It is said that if Bitcoin were to be forked so that it no longer uses PoW (among other things) then it would no longer be considered Bitcoin

Everything in this post is incorrect.

Why do you say so?

[cbeast]

because the shared costs related to Bitcoin mining make it very much unlike a free market

This isn't difficult to fix.

All it takes is a price discovery mechanism for bandwidth.

That's the same argument they gave to deregulate electricity in the 1930s. It caused power companies to refuse service to farmers because of the line maintenance costs. It hurt rural development and farming. If there isn't regulated bandwidth then only densely populated regions will have adequate bandwidth for bitcoin. We don't even have good enough bandwidth for bitcoin nodes in most of rural America let alone poorer countries.
http://www.pbs.org/wgbh/pages/frontline/shows/blackout/regulation/timeline.html

[justusranvier]

That's the same argument they gave to deregulate electricity in the 1930s. It caused power companies to refuse service to farmers because of the line maintenance costs. It hurt rural development and farming. If there isn't regulated bandwidth then only densely populated regions will have adequate bandwidth for bitcoin. We don't even have good enough bandwidth for bitcoin nodes in most of rural America let alone poorer countries.
http://www.pbs.org/wgbh/pages/frontline/shows/blackout/regulation/timeline.html

If you're going to source history, then at least make sure you're being remotely accurate.

All the just so stories that supposedly prove the natural monopoly fallacy are fairy tales that fall apart with the slightest bit of investigation:

http://mises.org/daily/5266/