I think these metrics should specifically exclude the two pools above as they are the ones who were SPV mining and not switching to using a validated block-header as soon as possible.

Don't get put-off from posting by Greg's feedback as it is easy to be silenced by someone-who-knows-best and let them run the show.
My big takeaway from his comment is how you got him into arguing from a position that big blocks are OK (handclap and kudos to you). In fact, I have learnt now that even 7.5GB blocks are today theoretically tolerable in at least one respect (validation where tx are pre-validated), although l suspect not in quite a number of other crucial respects.
Wasn't Gavin's doubling end-point 8GB in 2036? Effectively the same end-point!

I am reminded of Mike Hearn's words of wisdom:

One less bottleneck to expect with larger blocks.

Great work Peter, but do we have any empirical evidence for the 30 seconds? Seems surprisingly high and I would have guessed just a few seconds.

Yeah, tired of waiting and has to move it up?

Exactly. From a starting point of an average block size over one week (ABS) at 400KB they created a lot of impact. Consider that ABS will unlikely ever exceed 80% of 1MB because miners will often create small or empty blocks (even if thousands of legitimate fee-paying tx are backing up). So, it does not take a math guru to see that when the ABS is 70% then a bunch of redditards could spam the network 4x more effectively than today for the same effort. When the ABS is 80% of 1MB then any disturbance, whether 1Sochi, 1Enjoy, Dice site bots, Greek collapse news, could ramp volumes making Bitcoin a joke for thousands of users and the world's press. We are sleep-walking into a nightmare.

And the fastest way to get a dog-leg-down move in the chart of full node counts? Execute a fast hard-fork in a few weeks under emergency conditions.
Just brilliant! /sarc

Yep. I will be running my node as XT, but waiting for the patch before changing.
I still hope that Core Dev just adds the patch to BitcoinCore and they get on with normal dev work, rather than sulking about it.

Another great chart Peter.

One suggestion. Is it worth mentioning IBLT because by the time 16MB blocks are being mined they will likely be taking up just 500KB of bandwidth overhead to propagate. Obviously the real-time unconfirmed transaction overhead does remain. and is large long-term.

Just as during the 1MB debate so many people kept confusing the block size limit as the average block size, so is the case with Gavin's doubling schedule. These big block sizes are disk block sizes, the bandwidth-using blocks will be far smaller.

Full blocks will only need to be sent between nodes on resync and bootstrapping, not a normal state for most nodes. Even then, future Bitsat-type blockchain download services should mean that nodes catching up have a faster option than relying on terrestrial broadband.

Peter Todd's promotion of replace-by-fee scorched-earth is an attempt to keep the 1MB by the "back door", an attempt to realize the "Bitcoin as digital gold only for high-value transactions" dream. The effect of this change is to eliminate small purchases e.g. for cups of coffee, which require a workable zero-conf environment. Buying a car where the parties can wait for block confirmations would still be fine. He says that "zero-conf never worked" yet BitPay's business model repeatedly proves this false, every hour of every day.

I summarize my feelings here with:
Bitcoin as digital gold only for high-value transactions is a castle built upon quicksand. Bitcoin as digital gold for all transactions, where fiat is normally used, is a castle built upon solid rock.

Edit: this is interesting wrt tx vols:
http://www.reddit.com/r/Bitcoin/comments/3ag3jr/number_of_bitcoin_transactions_follows_cubic_trend/

My reply on an interesting thread in Dev & Tech:

Those of us who want to see an increased block size limit are at a disadvantage because "doing nothing" achieves the same result as a consensus decision to keep the 1MB and seeing what happens to the Bitcoin ecosystem when confirmation times blow out from the 10 minute average, which everyone expects today.

Yet, Wladimir commented last month that he was "weakly against" making this change.

So, since then, we know there is a clear majority for this change on all user polls, lists of businesses and wallet providers, and now mining opinion.

Mike and Gavin didn't follow consensus procedures? Boo hoo, too bad. They knew that there was a lot of entrenched opinion against changing the 1MB (whether misguided about Satoshi's original vision or not), so they probably tried very long and very hard to obtain consensus among the github commit access developers. They failed, so rightly took all the arguments public, where they found overwhelming support for the change.

Core Dev need to ask themselves "If the 1MB limit did not exist, would it get any ACK to put it in place in an upcoming change (e.g. v0.11)?"
This is not a rhetorical question, this is a valid question to ask. I bet that this type of change, a blunt hard limit with unknown consequences would get zero support in Bitcoin Dev. There would be all sorts of objections about how it's a naive attempt to vaguely "increase fees", "stop spam", "slow the decline in full nodes" could be done far more effectively in far more elegant ways. Implementing the 1MB today would get an unanimous NACK.

Core Dev also need to make a decision. they either:

a) delay the v0.11 until it gets a scaling improvement which replaces the max block size constant (whether by a patch reflecting Jeffs' BIP 100, or a functionally comparable patch from Gavin). After all, Gavin gave notice 2 months ago that he wanted to submit a patch for v0.11

b) release v0.11 without the above, which is effectively a declaration that they are prepared to allow the 1MB limit to be maxed out (noticeably affecting user confirmation times) before considering to release a patch for it. This might not be "1MB 4EVR" but it is practically equivalent as far as the rest of us are concerned who want to see the limit raised/modified/improved/removed before the inevitable PR disaster from inaction.

We have heard Gregory's opinion loud and clear on Bitcointalk and Reddit, so what does Wladimir think today?

Only by the addressable memory on each node, which varies. It would take a phenomenal number of unconfirmed tx to break nodes, but maybe enough of them would happen if wallets keep sending tx for thousands of frustrated BTC holders who are trying to get coins onto the exchanges as the price falls way below any TA support.


Oleganza is nuts if he thinks that $100 tx fees would ever be normal. How could Bitcoin keep any market share in that situation if another crypto-coin had 10x the volume and fees of about $1? How could Bitcoin keep any semblance of "digital gold" in that environment? (leaving aside the definition of it).

There is another reason why Bitcoin should not be allowed to hit a network-wide hard limit at any time, and ironically, for a trust-less system, it is trust in the system as a whole.
All users of Bitcoin have to trust that the trust-less network works and is viable. It has proved this over 6.5 years. That trust is hard-earned. Think about a relationship or a business partner, it takes years to earn trust, but that trust can be squandered over one serious event. If a partner cheats after years of loyalty, then how many years does it take to regain trust in that person again? Never?
Same with bitcoin, because: money.
A loss of trust just once is a scar for life.

Gavin has been seriously distracted from this  , just as the SC guys have been seriously distracted from their own work, by their blowback on the 1MB.

Yeah, you may well be right about SC/LN timing, I am still trying to give them the benefit of the doubt, FWIW (at least until Wladimir breaks ranks).

Gavin's solution is very good, and the 2-week grace period a smart move. The reason for this is that it gets the most people on-board with the >1MB change in a controlled manner. It is axiomatic that 100% of miners cannot be in the first 75% to upgrade even if they all wanted to. The 2-week period focuses minds on a known deadline and will likely see a "hockey-stick" uptick in the adoption curve. I would expect 85-90% mining power behind the change before the first >1MB block is mined. If 90% was the original target then Bitcoin's ecosystem growth, supported by the majority, could be stymied by a mere 11% of hold-out miners. The grace period also allows for non-mining nodes to upgrade who are not paying close attention to the whole process.

I like Jeff's BIP 100 with the miner voting, but not sure about the 90% he mentions because of the concern above.
Also, I really don't like a fixed date for everyone to upgrade (although this is far better than no change planned at all). Not using block versioning to change the 1MB means applying a sub-standard software solution for politically correct reasons, i.e. just so people can say "Aha, the users control Bitcoin and are telling the miners what to do." We already know the user consensus on a dozen btc and reddit polls, ecosystem business feedback, and now at least 50% of mining power opinion. So the change should be done as safely as possible. Gavin is perusing this path.

Probably the biggest reason that Core Dev is prepared to let the 1MB limit be constantly hit is that they think they can manage the situation. Perhaps they need to look up the word "hubris" in the dictionary.

A big bazooka in Core Dev's (fairly small) armory needed to "manage" a blocks-limit-full situation is prior roll-out of a change to purge unconfirmed tx from node mempools. This would in theory blunt some of the nasty effects in Mike's Crash Landing scenario.

So, no surprise that a pull request exists to do this:
https://github.com/bitcoin/bitcoin/pull/6281


(my emphasis)
Hmm, DoS issues like ordinary users legitimately trying to use Bitcoin after tx are backing up when free block-space has vanished?

However, what appears to be a bed of roses rapidly turns into a bed of thorns...


Their bazooka has a barrel bent into a U-shape.

Another big concern about mempool limiting is the implications for IBLT, which is Bitcoin's best hope for challenging the likes of Visa and MC in the medium, not long-term.
Randomly ejecting unconfirmed tx, or allowing a user-configurable max-mempool-size, creates non-uniform mempools. This seriously degrades the potential for IBLT, especially when miners want to clear most of the unconfirmed tx, to maximize revenue when the block reward is lower.

Maybe not that bad, and if general technology companies are considered separate from Bitcoin-focused businesses, then perhaps another 155 could be added for them, as per the MSCI World Index (IT).
https://www.msci.com/resources/factsheets/index_fact_sheet/msci-world-information-technology-index-usd-net.pdf

It is easy to imagine that even non-Bitcoin tech companies would want to run at least one node to have a hook into a real-world blockchain instance..

Thanks Peter, this helps.
We can also think of it as O(2n) because of the re-broadcasting, which IBLT would do a lot to mitigate, especially as the re-broadcast is the time critical component, and the capacity of the network in a true O(n) state is already more like 300 tps than 3 tps.

I don't think the full node count would scale as far as 12m in the growth example above, but payment channels for node services would at least reverse the decline even when it becomes necessary to support that loading.


Absolutely!

reddit post

(my emphasis)

Greg is giving me serious cognitive dissonance with these posts.
He is very concerned about the trend to centralization and yet can countenance a quick fork if necessary.

The most harmless hard fork is one which is prepared so far in advance that all software instances are upgraded before it takes effect. Something which was almost achievable if it had been scheduled in early 2013 with the target date about 2 years ahead. That is what Bitcoin needed.

Today, the time for this luxury is gone and about 6000 nodes exist (which accept connections). The worst scenario after a quick fork is 3000 left on each side, with the best scenario somewhere between 3000 and 6000 on the majority side, say 4500 nodes upgrading. The remainder either switching off or flailing away on a chain with a ridiculous difficulty. What a hammer-blow towards the furnace of centralization!

In my opinion, the odds of a dire situation after the 1MB is hit, needing a quick fork, is about 99%. This not an unreasonable interpretation of the separate findings of David Hudson and Statoshi. Even if Greg thinks the odds of needing a fast fork is more like 10% or 20% (otherwise why describe it in the manner above) then this is surely an unacceptable risk.

Meanwhile, Adam has just given us a Hallelujah moment:

Progress at last, buying time to smoothly scale the MC, as well as buying time for SC and LN. FFS! How can this be contentious to SC and LN supporters?

Lastly.
Perhaps Peter R can put this one to bed:

Adam claims that the network overhead in scaling Bitcoin is O(n^2), where n=transactions
Mike counters that it is O(nm), where m=nodes

The latter makes more sense to me, and further, that m is not as significant as it appears because it is highly parallelized.

Thanks for the distilled wisdom!

I have updated my sig.

Agreed.

I am really puzzled as to why Adam is pushing the extension block proposal for the 1MB. It is obviously the wrong place for it, but is exactly what is needed to overcome the original 33.5MB message-size limit which still remains. Tier Nolan came up with the extension block idea about 2 years ago IIRC and it is very good, but too soon now. Also it should be coupled with IBLT as the two together will future-proof Bitcoin as far as block sizes are concerned (within Moore's Law) allowing on-chain scaling and elimination of a lot of the out-of-band block-stuffing miner spam.

I am also puzzled as to why Greg keeps wasting mind-boggingly large amounts of time spamming reddit with dead pro-1MB arguments. Surely his time is more valuable than that.

This is fantastic news. Jeff is the one person who can push the 1MBers in Core Dev into a reasonable compromise.
Together with the Greek slow-motion Inception-style trainwreck finally off the rails...
...BITCOIN on the launchpad at $230.
UP!

Actually, I think that your suggestion is what is needed eventually, perhaps when the block reward gets below < 1BTC.

There is a long-term threat to the PoW model when the reward gets very small, "re-mining", where miners deliberately ignore recent blocks en-masse in order to grab the transaction fees for themselves. The incentive structure dissipates. However, IBLT should help a lot in this situation as it gives non-mining nodes some influence over the miners to only take tx from the mempools instead of orphaning. Might not be enough of a force and a PoS element could help.


Always a good idea to observe where the smart money is headed.