How a floating blocksize limit inevitably leads towards centralization

[jl2012]

With full nodes building on trusted computing platform, miners with low bandwidth can mine without being full nodes themselves.

In addition to working as a normal full node, a trusted computing full node will accept encrypted queries and reply with encryption, so the operator is unable to censor. It will also prepare a list of all unconfirmed valid transactions, including only the txid, size, fee (and optionally tx priority). The list will propagate in a P2P manner. Miners will construct blocks by choosing the transactions they want to include, in addition to coinbase and any other valid tx not provided by the trusted full node.

To prevent the node from cheating (not very possible due to trusted computing), individual miner will fully validate some blocks regularly, depending on their resources. For a miner with only 50kB/s connection (i.e. 30MB/10min) while maxblocksize is 300MB, he may validate only 1 in 10 blocks.

The trusted full nodes will be supported by donation and/or subscription fee. People with many bitcoins will support/offer these nodes to protect their wealth.

[Peter Todd]

However, I have extracted the transaction counts and they average 1190 each. Looking at a bunch of blocks maxing out at 250Kb they are in the region of 600 transactions each, which is to be expected. Obviously, there is a block header overhead in all of them. But this does mean that the 1Mb blocks will be saturated when they carry about 2400 transactions. This ignores the fact that some blocks are virtually empty as a few miners seem not to care about including many transactions.

So 2400 transactions per block * 144 blocks per day = 345,600 transactions per day or Bitcoin's maximum sustained throughput is just 4 transactions per second.
This is even more anemic than the oft-quoted 7 tps!

Excellent. Someone finally quoted some real numbers instead of theoretical maximums, the picture is a bit clearer now, thanks!

Those numbers are bogus and show very little understanding on the part of solex.

The problem is apparent if you look at transaction 3e4116059a0edb1134126047d9e5ebfa1619b6180153cdc8390e6e36c375a179 from block #259156, one of the blocks in solex's analysis. That transaction has one input, and 41 outputs. Now for the purposes of determining how many transactions Bitcoin can perform per second, are you going to count it as one transaction, or more than one? solex is counting it as one.

As transactions become more expensive per byte people are going to use all sorts of techniques to make transaction size smaller. For instance you can combine transactions together with other parties; each transaction includes a 10 byte header. If you get together with 20 other people, you've saved 200 bytes and you improve your privacy because you've mixed your funds with those 20 other people.

The absolute minimum transaction size(1) is for single input single-output transactions. They are 192 bytes each, 182 if transaction combining is aggressively used. 1MiB/10minutes * 182bytes = 9.6tx/s

Big eWallets services like instawallet will be able to get much closer to that theoretical limit than other services simply because they'll have a wider range of txouts to chose from. We may even see agreements between services to use each others wallets just to optimize fees, with something like ripple used to settle imbalances periodically; if off-chain transactions become supported, it's quite possible even day-to-day users will be running software that does stuff like that automatically.

1) Minimum signed transaction that is. The technical minimum is 60 bytes, but such transactions are spendable by anyone and thus offer no security.

[hazek]

retep thanks for making the picture even clearer. It's good that there's talk about concrete examples instead of theoretical maximums or minimums.

[Peter Todd]

With full nodes building on trusted computing platform, miners with low bandwidth can mine without being full nodes themselves.

In addition to working as a normal full node, a trusted computing full node will accept encrypted queries and reply with encryption, so the operator is unable to censor. It will also prepare a list of all unconfirmed valid transactions, including only the txid, size, fee (and optionally tx priority). The list will propagate in a P2P manner. Miners will construct blocks by choosing the transactions they want to include, in addition to coinbase and any other valid tx not provided by the trusted full node.

To prevent the node from cheating (not very possible due to trusted computing), individual miner will fully validate some blocks regularly, depending on their resources. For a miner with only 50kB/s connection (i.e. 30MB/10min) while maxblocksize is 300MB, he may validate only 1 in 10 blocks.

The trusted full nodes will be supported by donation and/or subscription fee. People with many bitcoins will support/offer these nodes to protect their wealth.

Technically speaking, that's a very clever idea.

Socially speaking though, it'll be an utter failure. Miners using pool have absolutely no incentive at all to verify the blocks they produce other than some vague desire to protect Bitcoin. This is why currently pools other than P2Pool aren't verified at all - Eligius supports getblocktemplate, but mining software that uses a full validating node to verify the blocks is hardware ever used.

Pools just aren't going to buy a bunch of expensive trusted computing hardware and switch their operations to use fragile trusted computing software just to please the 1% of miners who seem to care about this issue.

[jl2012]

With full nodes building on trusted computing platform, miners with low bandwidth can mine without being full nodes themselves.

In addition to working as a normal full node, a trusted computing full node will accept encrypted queries and reply with encryption, so the operator is unable to censor. It will also prepare a list of all unconfirmed valid transactions, including only the txid, size, fee (and optionally tx priority). The list will propagate in a P2P manner. Miners will construct blocks by choosing the transactions they want to include, in addition to coinbase and any other valid tx not provided by the trusted full node.

To prevent the node from cheating (not very possible due to trusted computing), individual miner will fully validate some blocks regularly, depending on their resources. For a miner with only 50kB/s connection (i.e. 30MB/10min) while maxblocksize is 300MB, he may validate only 1 in 10 blocks.

The trusted full nodes will be supported by donation and/or subscription fee. People with many bitcoins will support/offer these nodes to protect their wealth.

Technically speaking, that's a very clever idea.

Socially speaking though, it'll be an utter failure. Miners using pool have absolutely no incentive at all to verify the blocks they produce other than some vague desire to protect Bitcoin. This is why currently pools other than P2Pool aren't verified at all - Eligius supports getblocktemplate, but mining software that uses a full validating node to verify the blocks is hardware ever used.

Pools just aren't going to buy a bunch of expensive trusted computing hardware and switch their operations to use fragile trusted computing software just to please the 1% of miners who seem to care about this issue.

Lazy miners are always lazy. They won't verify no matter the block size is 1kB, 1MB or 1GB. So your reply is irrelevant to the OP.

With trusted full node, people with insufficient resources will still be able to run partial nodes, which will also protect the integrity of the blockchain. I don't think these will be provided by pools. They could be supported by donation or big exchanges. Also, it could be run by trusted Chaum banks (https://bitcointalk.org/index.php?topic=146307.0), as they have to run full nodes on trusted platforms anyway.

[markm]

Maybe having the world's most difficult proof of work also leads to centralisation, as my primary motivation for running a full *bitcoin* node is to use bitcoin, via p2pool, to secure other chains.

Is there a way of merged mining without having to "centralise" the chains to be merged-mined all at one site where the possibly to be massively bloated bitcoin chain is also to be found?

Maybe even think ahead that if bitcoin is to become some bloated monstrosity maybe some of the merged mined chains might too, so if the bitcoin chain can be made use of (for merged mining) without having to have a full node of it actually present (piped into the building in all its massive glory so to speak) could one extend that to have maybe some space between each massively bloated chain of a merge and the nice compact collection of however many much smaller chains all of which can all easily be handled at once at one Joe Sixpack's desk?

-MarkM-

[TierNolan]

Is there a way of merged mining without having to "centralise" the chains to be merged-mined all at one site where the possibly to be massively bloated bitcoin chain is also to be found?

You could have a standard that all chains comply with.  This would allow interoperability.  Atm, that means compatibility with the bitcoin chain.

However, the merge mining against the bitcoin chain means embedding information in the coinbase transactions.

A header made specifically for merge mining would be much more efficient for the alt chains.

For example, you could have a hash-header of the form

var_int: nonce
byte[32]: merkle root

This isn't even really a chain.  You create a header of that form and the roots of the merkle tree are what you are trying to stamp.

The alt chain's header would then be

byte[32]: previous block
byte[X]: message
var_int: difficulty
var_int: nonce
byte[32]: merkle root
byte[N][32]: path from root to hash(previous block + message)

Having said that merged mining means that you are generating more proof of work than you actually performed.  It is feels like forged proof of work.

An alt chain could end up being overwhelmed by "cost-free" proof of work from other chains.  It makes it possible to "stamp" alternative block headers for no margin cost.

[tvbcof]

...

Having said that merged mining means that you are generating more proof of work than you actually performed.  It is feels like forged proof of work.

An alt chain could end up being overwhelmed by "cost-free" proof of work from other chains.  It makes it possible to "stamp" alternative block headers for no margin cost.

At the end of the day, the basis for proof-of-work is harden the ledger so the concept of 'cost-free' is not really that big a deal (to me.)  In some ways it is pretty compelling that all of the mining effort in the world benefits by working for all block chains.  Kind of a win-win.

On the other hand, if various block chains became compatible in size to one another, or could form coalitions and display an adversarial posture, there will likely be attacks.  It makes economic sense and certainly it (resource conflicts) are more commonly observed in nature than not once a carrying capacity is approached.

I suspect that in the end a solution which relies exclusively on simple size dominance will be overcome.  But there are a lot of other possibilities I bet.

[markm]

Alt-Ripples!

Consensus ledger without (proof of) work!

-MarkM-

[solex]

The absolute minimum transaction size(1) is for single input single-output transactions. They are 192 bytes each, 182 if transaction combining is aggressively used. 1MiB/10minutes * 182bytes = 9.6tx/s

Fine, here you have the definition of a transaction unit. Of course I understand that there are multiple inputs/outputs possible, which makes bitcoin transfers much more efficient than in conventional payment systems. I was just using terminology common on sites such as blockchain.info. This makes 4 tx/s a valid description of the real-world situation.

It is no good having theoretical efficiency available if it is not being physically used! You say it could be used, and this is true, however optimum transaction efficiency cannot be achieved by a wide user-base overnight. My concern all along is that bitcoin usage is growing exponentially, at a much faster rate than the adoption of transaction efficiency, or complimentary off-chain solutions are appearing.

There are a lot of smart people on this forum who fully understand exponential growth curves. Yet it seems some of these people think that a hard limit can seamlessly force adoption of efficiencies and complementary solutions which are developing linearly in comparison. 

There can be no seamless transition in that scenario, which is why an adaptive/algorithmic block size limit is needed sooner than later.

[markm]

Okay, but is jumping directly to ten megabytes in one fell swoop the best way to go, instead of , for example, a series of doublings?

-MarkM-

[solex]

Okay, but is jumping directly to ten megabytes in one fell swoop the best way to go, insead of , for example, a series of doublings?

-MarkM-

No, no, no. I am the last person to advocate jumping straight to 10Mb.

What I think we need is a flexible limit along the lines of the many suggestions which attempt to take account of what we know to be important, especially encouraging fees and eliminating free micro-transactions. Perhaps it should also adapt to block propagation times, constraining the limit if network performance degrades too much.  I appreciate that it is impossible to get this perfect on the first attempt and agreement on the detail may take time.

So a two step plan seems best:

1. In the next version the limit constant becomes a variable that increases 20% if the average block within the previous difficulty period is >80% of the limit (or a simple variation of this).

2. In a subsequent version activate the best adaptive/algorithmic block limit strategy which has found a consensus.

The advantage of step 1 is that it gets an early solution into the field achieving huge risk mitigation. It will have more time to be adopted by the greatest number of nodes. Step 2 is the attempt at a permanent solution  which may happen even before step 1 has its first automatic increase, or it could be delayed for a year through analysis paralysis - it won't matter as much either way.

[MoonShadow]

As transactions become more expensive per byte people are going to use all sorts of techniques to make transaction size smaller. For instance you can combine transactions together with other parties; each transaction includes a 10 byte header. If you get together with 20 other people, you've saved 200 bytes and you improve your privacy because you've mixed your funds with those 20 other people.

That's quite literally the intent of the send-to-many transaction type, although it's much more likely that they'll be used to send to pay many different vendors from one single payer than multiple payers to multiple payees.  The best example is that of weekly payroll, as anyone getting payed wages in bitcoin, working for the same company or entity, can be paid their weekly wages in the transaction as everyone else.  Regular users could do the same thing using bitcoin aware bill payment programs, than can gather up all the re-occuring and one time bills that a person has received, and pay their water bill, electric bill and cable bill, etc. in a single action; so long as they have the total value in inputs that would be required.

So while a direct deposit payroll event for any significantly sized company would involved hundreds to thousands of electronic transactions per week, these same companies could do the entire event in a single send-to-many transaction that weighs in at a couple kilobytes, and currently should cost less than a quarter.  Even if the transaction cost rise that such a large transaction costs $10 at a time, that's chump change compared to the costs of simply printing cheques, much less mailing them.  In the event that small value transactions wherein the customer sends the vendor a set of low value keypairs (as opposed to transactions), the vendor would have a vested interest in flushing those keypairs in a timely manner, so as to limit the risk of a double spending fraud against them.  In this way, collecting those many inputs and pumping them back out to employees with the weekly payroll send-to-many transaction does double duty.

[WiW]

Yes, I'm still catching up on this thread (lots to read!), but I just have to point something out

Everybody needs to stop worrying about miner's profits already. Even if the block size is increased infinitely, there will always be profits for miners. If, as many here suggest, the txn fees drop to 0, then miners will simply stop mining. When nobody is mining, how much would people pay to get transactions moving again? I can assure you it will not be 0.

Infinite block size does not mean infinite supply. Somebody still needs to mine those infinitely-sized blocks, and they won't do it for free.

As with every non-innovative competitive market, the prices drop to cost of operation (in our case, electricity + bandwidth + equipment) plus competitive operating income. Look at the PC market for instance. All the OEMs use the same parts from the same factories using the same manufacturing process and people are willing to pay less and less for computers. OEMs left and right are losing money with many looking for a way out. Yet, computers are being produced by the millions. The moment one OEM drops out and supply falls, another OEM will pick up the slack for all the pent up demand. People are paying less and less for computers. They'd rather pay nothing, but they still want computers, dammit. even if it costs them whatever is the cost of operation plus operating income.

Seriously, nobody has to worry that miners will simply stop mining. It's not going to happen.

[hazek]

Yes, I'm still catching up on this thread (lots to read!), but I just have to point something out

Everybody needs to stop worrying about miner's profits already. Even if the block size is increased infinitely, there will always be profits for miners. If, as many here suggest, the txn fees drop to 0, then miners will simply stop mining. When nobody is mining, how much would people pay to get transactions moving again? I can assure you it will not be 0.

Infinite block size does not mean infinite supply. Somebody still needs to mine those infinitely-sized blocks, and they won't do it for free.

As with every non-innovative competitive market, the prices drop to cost of operation (in our case, electricity + bandwidth + equipment) plus competitive operating income. Look at the PC market for instance. All the OEMs use the same parts from the same factories using the same manufacturing process and people are willing to pay less and less for computers. OEMs left and right are losing money with many looking for a way out. Yet, computers are being produced by the millions. The moment one OEM drops out and supply falls, another OEM will pick up the slack for all the pent up demand. People are paying less and less for computers. They'd rather pay nothing, but they still want computers, dammit. even if it costs them whatever is the cost of operation plus operating income.

Seriously, nobody has to worry that miners will simply stop mining. It's not going to happen.

The outcome of this is what we don't want at all.

You'd have fewer miners, only those with the best margins would survive a constantly decreasing fees (as clearly outlined with your computers analogy), transaction fees would get lower and lower (as clearly outlined with your computers analogy) and there be an ever growing demand for transactions (as clearly outlined.. well you get it by now.).

This outcome that your analogy outlines couldn't be worse for a money system. It makes no sense to want fewer most competitive miners or to want them to earn as little in fees as possible. This money system, with it's growth, is bound to become more and more lucrative to attack therefore it makes sense to spend more and more on security which means more and more miners with better and better profits. Miner profits is exactly what we need to be worried about. Too few profits for miners leaves the entire money system vulnerable, and that's a tragedy of the commons scenario where no one will want to pay for it if they can pay as few fees as possible to get their transaction included.


I think we're paying around 12,3% per annum in monetary inflation for security right now, we were paying a hell of a lot more when the subsidy was 50BTC, as Bitcoin grows in user base and consequently with it the value of bitcoins, doesn't it make sense to pay more for security or at least the same in terms of BTC and not less? And yet less is exactly the course we are on atm, especially if we had unlimited block space.

It seriously doesn't make any sense to want the security of a money system to fall victim to the tragedy of the commons.

[jl2012]

I think we're paying around 12,3% per annum in monetary inflation for security right now, we were paying a hell of a lot more when the subsidy was 50BTC, as Bitcoin grows in user base and consequently with it the value of bitcoins, doesn't it make sense to pay more for security or at least the same in terms of BTC and not less? And yet less is exactly the course we are on atm, especially if we had unlimited block space.

This is misleading. The current 12.3% is paid by every bitcoin holders, no matter they do transactions or not. When there is no block subsidy, the burden is totally loaded on people who do transactions, but not people who hoard. Asking for the same percentage of fee in term of total money supply is too much

[markm]

This is misleading. The current 12.3% is paid by every bitcoin holders, no matter they do transactions or not. When there is no block subsidy, the burden is totally loaded on people who do transactions, but not people who hoard. Asking for the same percentage of fee in term of total money supply is too much

Do the hoarders want their hoard to be secure or not?

If transactors revolt, resulting in too few transactions to keep difficulty high, difficulty will start to fall and hoarders can either decide to lower the cap at stake by moving capital out, or pay (e.g. by fee-only transactions) to keep difficulty as high as they feel their hoard needs it to be.

This might actually be a good way to motivate people to continue to look for even cheaper methods of storing value long term, maybe someone will come up with something even cheaper per X amount of value to protect than bitcoin is.

-MarkM-

[hazek]

I think we're paying around 12,3% per annum in monetary inflation for security right now, we were paying a hell of a lot more when the subsidy was 50BTC, as Bitcoin grows in user base and consequently with it the value of bitcoins, doesn't it make sense to pay more for security or at least the same in terms of BTC and not less? And yet less is exactly the course we are on atm, especially if we had unlimited block space.

This is misleading. The current 12.3% is paid by every bitcoin holders, no matter they do transactions or not. When there is no block subsidy, the burden is totally loaded on people who do transactions, but not people who hoard. Asking for the same percentage of fee in term of total money supply is too much

It doesn't matter who pays for it. I'm telling you how much the current security costs. And if we want to keep the current security and even have more once it becomes more lucrative to attack Bitcoin, someone will have to pay for it or Bitcoin wont be secure.

[markm]

I wonder how much it costs to protect the Crown Jewels?

The Mona Lisa?

That red stripe painting in some Canadian museum (a puny 5 million of value to protect, was it?)

Fort Knox yearly operating budget?

The space shuttle?

The pentagon?

The twin towers... oops too late.

Etc... lets look at some real life numbers...

-MarkM-

[WiW]

It seriously doesn't make any sense to want the security of a money system to fall victim to the tragedy of the commons.

I'm not sure I understand what the motives are for someone to execute such a costly operation to destroy bitcoin. What am I missing? Today, I can understand. Central banks and the intertwined governments would love to keep business as usual and bitcoin is a huge threat. But if and when bitcoin takes hold as a universal currency, why would anyone want to destroy it?

The best I can think of is if someone is invested in a competing currency. But if someone invested in a competing currency can carry out such a costly operation, I get the impression that either that competition is strong enough in and of itself, or that it wouldn't catch on anyway...

The outcome of this is what we don't want at all.

You'd have fewer miners, only those with the best margins would survive a constantly decreasing fees (as clearly outlined with your computers analogy), transaction fees would get lower and lower (as clearly outlined with your computers analogy) and there be an ever growing demand for transactions (as clearly outlined.. well you get it by now.).

This outcome that your analogy outlines couldn't be worse for a money system. It makes no sense to want fewer most competitive miners or to want them to earn as little in fees as possible. This money system, with it's growth, is bound to become more and more lucrative to attack therefore it makes sense to spend more and more on security which means more and more miners with better and better profits. Miner profits is exactly what we need to be worried about. Too few profits for miners leaves the entire money system vulnerable, and that's a tragedy of the commons scenario where no one will want to pay for it if they can pay as few fees as possible to get their transaction included.

I'm not sure throwing more money at the issue is what will make it better. I think throwing more money at the issue will make it more prone to abuse. I really like the idea of applying the way the PC industry works on a stable currency system, especially because the PC industry is so stagnant (which is exactly how to not make a monopoly). The OEMs are totally handicapped but they do their job. I don't want miners to be Apple, taking premium profits and locking in services taking whatever course they wish. I want miners to be HP and Dell, entirely constrained by the tiny space between supply and demand. Is there any example of an extremely profitable market (which are usually artificially profitable) that are not monopolized? Every example I can think of is either an emerging market with a clear head start (a la every new market in high tech) or a state-regulated hugely-profitable market with a clear monopoly (a la drug cartels).

Honestly, I'm not quite sure how well the analogy applies because like you say, the network really relies on this security. So, I'm still not taking sides yet. However, I am worried about centralization and I don't want miners to be filthy rich. I just want them to do their job, and I will pay whatever the electricity costs + operating income is. Like many have said here (which I can't follow any more), bandwidth is just another factor in the competition. Just like electricity - we don't have a solution for an aggressive miner with free/cheap electricity.

I tend to believe that these things always end up centralizing. Especially where lots of money is involved. So where does that leave us?