On Chain Scaling

[Witrebel]

For bitcoin to be peer to peer cash, as we all know, it will need to scale with regards to transaction capacity per second.  Obviously this can be addressed through off chain solutions, but I am curious about on chain scaling.  

As I understand it, there are only a few things that could be done to increase on chain capacity without drastically altering the protocol.  

1) You can increase block size.  The major argument against this seems to be the burden placed on running a node, which would increase centralization by making it harder/more expensive to run a full node.

2) Segwit, which makes more efficient use of a given blocksize, and is already implemented.  (I am not well versed in the technical arguments against segwit)

3) Alter the protocol to increase the rate of block production, and decrease the block reward, such that money supply is the same, but you end up with more transactions per second for a given block size.  This would appear to have all the drawbacks of 1), and also an increase in the orphan rate. [/li]

Are there other ON chain solutions being considered?

[1714023247]

1714023247

[mda]

https://bitcointalk.org/index.php?topic=2381234.0

[Witrebel]

https://bitcointalk.org/index.php?topic=2381234.0

This is essentially a mix of 1) and 3), you are increasing effective block size and doubling the rate of production. It sounds rather kludgey.

I'm looking for technical answers regarding the future of on chain scaling.  What options do we have? 

[BluOrb]

blockstream is controlled by bankers - it will never happen. Bitcoin Cash is our best hope. check out reddit.com/r/BTC for more info.

[Witrebel]

blockstream is controlled by bankers - it will never happen. Bitcoin Cash is our best hope. check out reddit.com/r/BTC for more info.

Bitcoin cash is mainly just an implementation of option 1).  And I'd like to leave all the political issues aside, looking to keep this a strictly technical conversation.

[HeRetiK]

Option 1) and 3) are different solutions that both lead to the same problem -- a faster growing blockchain resulting in harder to run nodes. Which means that eg. halving block times would increase node load just as much as doubling the block size.

Option 2) SegWit already is a block size increase, albeit in itself not a very efficient one -- blocks containing purely SegWit transactions are expected to effectively reach as much as 4MB while handling the equivalent of 2MB worth of legacy transactions. Its main intend is to pave the way for decentralized off-chain scaling.

As far as pure on-chain scaling is concerned, I'm afraid that's all there is to it.

You could increase blocksize and solely rely on pruned nodes, but that would weaken Bitcoin security and only solve part of the node resource problem. You could look into sidechains, but then you are by definition off-chain again. You could also move away from blockchains and look into alternatives such as directed acyclic graphs, but those protocols come with issues of their own.

[nullius]

For bitcoin to be peer to peer cash, as we all know, it will need to scale with regards to transaction capacity per second.  Obviously this can be addressed through off chain solutions, but I am curious about on chain scaling.  

As I understand it, there are only a few things that could be done to increase on chain capacity without drastically altering the protocol.  

1) You can increase block size.  The major argument against this seems to be the burden placed on running a node, which would increase centralization by making it harder/more expensive to run a full node.

2) Segwit, which makes more efficient use of a given blocksize, and is already implemented.  (I am not well versed in the technical arguments against segwit)

3) Alter the protocol to increase the rate of block production, and decrease the block reward, such that money supply is the same, but you end up with more transactions per second for a given block size.  This would appear to have all the drawbacks of 1), and also an increase in the orphan rate.


Are there other ON chain solutions being considered?

I should preface this by emphasizing:  On-chain scaling is and always will be a very limited proposition.  This is an engineering reality, not a matter of “opinion” in the sense most people mean that word.  At best, we will see incremental, linear improvements to on-chain capacity.  That’s not enough, and never will be.  I think big.  I want for Bitcoin to compete with Visa; and that will require a transactions-per-second capacity increase of about four orders of magnitude.  Adding layers vertically (Lightning Network) and horizontally (sidechains) will truly turn Bitcoin into the foundation of a global payment internetwork which will leave Visa in the dust.

But for on-chain capacity, now that we have Segwit, following is a description of two improvements which Core has in the pipeline.  There are and will be other ideas, too; I list here the things I reliably expect to happen in the not-so-distant future.  Note that these improvements depend on Segwit, and were blocked by the blocking of Segwit.  Segwit was not only a capacity upgrade:  In addition to fixing bugs such as tx malleability, Segwit also introduced script versioning.  The script versioning system will provide a smooth upgrade path for adding these technologies without any forking drama or headaches.

• Schnorr signatures with signature aggregation will improve block space usage (thus, scalability) and help making spam attacks economically prohibitive, as well as improving privacy.
• Merkelized Abstract Syntax Trees (MAST) will permit payment to large, multi-conditional scripts, while only revealing the pertinent part of the script when the output is spent.  Those who use such scripts will thus drastically reduce their use of block space.  MAST will also improve smart contract flexibility and privacy.
  

I think those are the two big things to watch right now.  Capacity-wise, I think Schnorr signatures will have a much bigger impact; with the mix of transactions on the real-world blockchain, it will probably give about a 25–30% increase in capacity.  Incremental, as I said.

Option 2) SegWit already is a block size increase, albeit in itself not a very efficient one -- blocks containing purely SegWit transactions are expected to effectively reach as much as 4MB while handling the equivalent of 2MB worth of legacy transactions. Its main intend is to pave the way for decentralized off-chain scaling.

Those numbers not correct.  Segwit lets capacity approach a theoretical limit of 4MB; best estimates I’ve seen are that we will eventually see blocks a bit over 2MB.  As recently discussed in another thread, if a block contains only old-format transactions, then it is still effectually limited to 1MB.  See BIP 141’s block weight calculation.  (You are absolutely correct about Segwit’s main purpose.)

You could increase blocksize and solely rely on pruned nodes, but that would weaken Bitcoin security and only solve part of the node resource problem.

I must emphasize that point about “the node resource problem”.  Pruning reduces disk capacity needs for the long-term storage of data at rest; but that is the smallest problem with big blocks!  Too bad it’s what everybody talks about.

If that were the only issue, then I would advocate 32+MB blocks right now.  Why not?  Buy another few terabytes of disk and snap it in.  Or prune.  Yawn.

Much bigger problems are latency in block delivery (=orphan rate), network bandwidth, CPU and IOPS for validation, and most of all:  UTXO set size growth.  Pruning nodes does not help with these problems, not at all.  On a forkchain with huge blocks, if the blocks were used to capacity (in which case fees would skyrocket just as in Bitcoin), even if you prune, you will still need a beefy computer with a fast SSD and lots of RAM plugged into the sort of connection you find in datacenters.  Or don’t run a node.  I think that’s part of the point—the other part being that mining would be even worse centralized, due to the orphan rate issue.

Parenthetically, I note that unlike some wannabe competitors, Bitcoin is sufficiently popular and valuable that big blocks would be promptly full.  That’s not only a matter of whatever mempool backlog you see, but also of people suddenly making transactions they currently choose to avoid due to fees.  So, big blocks would not even “kick the can down the road”, as some people say.

Back of the envelope, 8MB blocks would get us to somewhere around 1/40 of Paypal’s average TPS, and less than thousandth of Visa’s peak TPS.  Gargantuan 32GB blocks would get a few supernodes in datacenters processing transactions at maybe a tenth of Paypal’s average rate.  For my part, in the long term, I will accept no less than a ten thousandfold increase in Bitcoin’s TPS capacity on all layers.  Blocksize increase is not a scaling solution.

Of course, you are also right about weakening security.

blockstream is controlled by bankers - it will never happen. Bitcoin Cash is our best hope. check out reddit.com/r/BTC for more info.

Shoo, shoo.  Go back to Roger Ver’s padded playpen where you belong, and chat there about your funny scamcoin all you want.  This is a forum for discussing Bitcoin.

[jdoing]

I'm sure this idea has probably already been brought up and shot down so please forgive my noob idea.  Would it be possible archive the blockchain every few years? By this I mean take a snap shot of all wallet balances removing the transaction history and greatly compressing the data, this would become the starting point of transaction verification for the next few years and the archive (which would contain the full transaction history over the past few years) could be distributed by archive nodes.  The archive nodes would not be necessary to validate new transactions so someone could run a "full" node without the archive. You wouldn't need as many archive nodes as full nodes and even if the archives become several TB in size there would still be people willing to provide the storage necessary to secure bitcoin.

[Witrebel]

As far as pure on-chain scaling is concerned, I'm afraid that's all there is to it.

You could increase blocksize and solely rely on pruned nodes, but that would weaken Bitcoin security and only solve part of the node resource problem. You could look into sidechains, but then you are by definition off-chain again. You could also move away from blockchains and look into alternatives such as directed acyclic graphs, but those protocols come with issues of their own.

With regard to the bolded, that is pretty much what I suspected.  As for DAG's, that is a tangential conversation but I am very interested in that technology and see it as being a potential scaling solution in the crypto space at large.

I should preface this by emphasizing:  On-chain scaling is and always will be a very limited proposition.  This is an engineering reality, not a matter of “opinion” in the sense most people mean that word.  At best, we will see incremental, linear improvements to on-chain capacity.  That’s not enough, and never will be. I think big.  I want for Bitcoin to compete with Visa; and that will require a transactions-per-second capacity increase of about four orders of magnitude.  Adding layers vertically (Lightning Network) and horizontally (sidechains) will truly turn Bitcoin into the foundation of a global payment internetwork which will leave Visa in the dust.

Great post, I appreciate the response.  Again, regarding the bolded, this is what worries me.  The white paper refers to bitcoin as peer to peer cash, we have all believed in bitcoin as a currency from the get go.  If the only way to utilize bitcoin to buy a cup of coffee is to go off-chain, then would you really be using bitcoin?  

I really want to keep this discussion as technically oriented as possible, but I feel like it will be impossible to ignore the branding implications of technical obstacles.  For example I see a couple scenarios that lead to widespread bitcoin adoption as a consumer payment solution.

A) Off-chain scaling solutions are implemented and branded under the umbrella of the bitcoin name.  The consumer is unable to distinguish the layer they are transacting on, and off-chain/on-chain boundaries are blurred from the perspective of the masses.

B) Massive protocol alteration, where some paradigm shift is implemented that allows an exponentially increased on-chain capacity.  This has a host of branding issues due to violating the "code is law" mentality.  

I'm sure this idea has probably already been brought up and shot down so please forgive my noob idea.  Would it be possible archive the blockchain every few years? By this I mean take a snap shot of all wallet balances removing the transaction history and greatly compressing the data, this would become the starting point of transaction verification for the next few years and the archive (which would contain the full transaction history over the past few years) could be distributed by archive nodes.  The archive nodes would not be necessary to validate new transactions so someone could run a "full" node without the archive. You wouldn't need as many archive nodes as full nodes and even if the archives become several TB in size there would still be people willing to provide the storage necessary to secure bitcoin.

The issue, as noted by nullius above, is FAR less about the size of the static blockchain, and more about its rate of growth.  Any time you increase its rate of growth, either through more frequent blocks of the same size, or through the existing block production rate but larger blocks, you are placing a burden on the nodes by requiring greater bandwidth, and increase in the RAM and CPU requirements.  It also has negative affects on the mining process in the form of an increased orphan rate.

[ir.hn]

The inherent fact is the current rate of data transfer across the world limits the scalability of any given coin.  The fact is that a 1 mb block takes over 12 seconds to propagate across most of the network.  This means big miners get an advantage, their chances of winning a block right after they won the last one are higher since most miners won't get started on the next block 12 seconds after the first miner starts.

The only ways to mitigate this is to have small blocks that can be transmitted over the network faster or have longer blocktime's so that the propagation time is negligible.

The only fair way to scale bitcoin is to clone it:
https://bitcointalk.org/index.php?topic=2381234.0

[Witrebel]

The inherent fact is the current rate of data transfer across the world limits the scalability of any given coin.  The fact is that a 1 mb block takes over 12 seconds to propagate across most of the network.  This means big miners get an advantage, their chances of winning a block right after they won the last one are higher since most miners won't get started on the next block 12 seconds after the first miner starts.

The only ways to mitigate this is to have small blocks that can be transmitted over the network faster or have longer blocktime's so that the propagation time is negligible.

The only fair way to scale bitcoin is to clone it:
https://bitcointalk.org/index.php?topic=2381234.0

You do realize that "cloning bitcoin" is no different than lowering inter block time to 5 minutes or going to 2mb blocks right?

Much bigger problems are latency in block delivery (=orphan rate), network bandwidth, CPU and IOPS for validation, and most of all:  UTXO set size growth.

Been thinking about that last part for awhile now.  Literally you are saying "we can't scale on chain capacity because it will increase Bitcoin usage"
Fundamentally you are saying we can not have widespread consumer use of on chain bitcoin transactions?

[ir.hn]

Cloning bitcoin wouldn't be like halving the block time or doubling the mb.  It would be like having thousands of mb blocks.  A clone army.  BC1 BC2 BC3 etc etc.  It's ridiculous I know.  You could have an inter-clone exchange.  Its crazy but it would be cool and it would work to multiply the transactions that can happen at any given time.

Already people do this with altcoins.  In their exchange they convert their bitcoin to altcoins to send coins to avoid fees then that person converts the altcoins to bitcoins on his exchange.  The "spread" in buy/sell prices on exchanges will give you a hit but it is instant coin sending.  If instead of using altcoins we were using bitcoin clones I think there would be several advantages including consistency, lower spreads (since all the clones would have to compete for customers) and avoiding pump and dumps.  If one of the clones starts getting pumped people will just move to another clone.  There are no differences in the coin to warrant speculation or differing valuations.   You could even possibly mime all the transactions from one clone to another this way you always have the same amount of funds on every clone.  Probably would be a challenge to do but I think something like that is possible if all the clones trust eachothers transaction verifications.

[gulliblehive]

The question is, if on-chain-scaling is only a mediocre improvement at the best given current capabilities, then why are we pursuing it at all? a 100-500% increase probably won't make much of a difference since it still won't be in the ball-park needed to be used for small & instant transactions anyways.

I'm not saying that improving the on-chain scaling protocol isn't a good idea, only that unless someone introduces a radical improvement (nothing of the sort that has been suggested up until now), the focus should lie on developing other aspects of the protocol (instead of wasting time & resources on mediocrity).

Make improvement on things that can yield significant improvements, not mediocre improvements. Value prioritization.

[Witrebel]

Cloning bitcoin wouldn't be like halving the block time or doubling the mb.  It would be like having thousands of mb blocks.  A clone army.  BC1 BC2 BC3 etc etc.  It's ridiculous I know.  You could have an inter-clone exchange.  Its crazy but it would be cool and it would work to multiply the transactions that can happen at any given time.

You said it yourself.  It would be like having thousands of mb blocks.  Not to be rude but you should take some time to learn about the technical side of things a little more.

The question is, if on-chain-scaling is only a mediocre improvement at the best given current capabilities, then why are we pursuing it at all? a 100-500% increase probably won't make much of a difference since it still won't be in the ball-park needed to be used for small & instant transactions anyways.

I'm not saying that improving the on-chain scaling protocol isn't a good idea, only that unless someone introduces a radical improvement (nothing of the sort that has been suggested up until now), the focus should lie on developing other aspects of the protocol (instead of wasting time & resources on mediocrity).

Make improvement on things that can yield significant improvements, not mediocre improvements. Value prioritization.

This is where my own lack of technical knowledge starts to show through.  Specifically, I do not know enough about off-chain scaling solutions to have a technical opinion on whether those solutions are actually Bitcoin scaling, or essentially alt coins.  I.E. the reason for on chain scaling is that it IS Bitcoin. 

[nullius]

Great post, I appreciate the response.

Thanks for the kind words.  It can take me awhile to write an in-depth reply; and I keep adding to this in pieces, as new replies accrue.  Apologies that the result came out somewhat disjointed and rambling.

By way of preface:

The white paper refers to bitcoin as peer to peer cash, we have all believed in bitcoin as a currency from the get go.  If the only way to utilize bitcoin to buy a cup of coffee is to go off-chain, then would you really be using bitcoin?  

This is where my own lack of technical knowledge starts to show through.  Specifically, I do not know enough about off-chain scaling solutions to have a technical opinion on whether those solutions are actually Bitcoin scaling, or essentially alt coins.  I.E. the reason for on chain scaling is that it IS Bitcoin.  

For a very rough conceptual analogy, imagine that gold is still used for everyday currency.  But gold has some problems:  It’s cumbersome; it’s heavy; it’s difficult to carry around.  These problems are intrinsic to the nature of gold; they can’t be solved by changing gold itself.  So somebody invents paper notes which are redeemable on-demand for gold.  These can be used for convenience when purchasing cups of coffee.

However, this is where the analogy departs from the reality of anything yet seen.  Unlike real-world paper gold notes, these paper notes are magical.  They are not issued by any central authority; you yourself create them by hooking your gold storage wallet into a “payment channel”.  They require no trust, and impose no counterparty risk.  They are mathematically verifiable to represent actual gold; they are not promises to pay gold; they can be magically transformed back into gold at any time, whenever you wish to do so.

That would be an approximate description of gold’s version of the Lightning Network.  Now, tell me:  In your opinion, “would you really be using [gold]” with such a system?

Also relevant:  Many users (mostly n00bs) are already using off-chain transactions, but in an abysmally inferior way.  There are major exchanges which act exactly as an obsolete bank:  They give you an account, and they keep all the actual money for you.  All accounts’ coins are typically kept in one pool, with on-chain transactions only used to send/receive outside funds.  Transfers between users of the exchange are treated as intrabank transfers, made by changing some entries in an internal ledger.  This has all the centralization and counterparty risk which Lightning avoids.  If the bank fails, you have Mt. Gox.  Also, the bank can arbitrarily deny you access to your funds.

In your opinion, are people who use these exchanges right now “really [using] Bitcoin”?  My opinion is, not really—not in the fullest sense; though they can still transact in actual Bitcoin with people who really use Bitcoin per the motto, “be your own bank”.  At least they can, if the bank deigns to so permit.  I hear that Coinbase closes the accounts of people who send to or receive from addresses disliked by Coinbase.

Much bigger problems are latency in block delivery (=orphan rate), network bandwidth, CPU and IOPS for validation, and most of all:  UTXO set size growth.

Been thinking about that last part for awhile now.  Literally you are saying "we can't scale on chain capacity because it will increase Bitcoin usage"
Fundamentally you are saying we can not have widespread consumer use of on chain bitcoin transactions?

Sort of.  Literally, I am saying, “We can’t scale on-chain capacity because that would increase on-chain Bitcoin usage (to the point of overloading nodes).”

But that is a non-problem.  Why is it desirable to have “widespread consumer use of on chain bitcoin transactions”?  Emphasis added.

I realize here that, as you stated, we are moving from the technical discussion to its inseparable nontechnical counterpart.  But first briefly, on the technical side, it must be understood what an awe-inspiring achievement was Satoshi’s creation of a Byzantine fault-tolerant decentralized database, and his application thereof to solve the double-spend problem for a new form of money.  Very smart people had been breaking their heads over these problems, for about two decades.  Now, very smart people have been trying to outdo Satoshi for the past nine years.  Perhaps someday, an ingenious new invention will obsolete Bitcoin’s way of doing things.  But thus far, nobody has proposed a replacement which improves performance and capacity without compromising on decentralization and trustlessness.

I find myself linking this ACM Queue article quite oft of late.  It provides a good bird’s-eye view of not only the technical problems solved by Bitcoin, but also the academic history of those problems.  That will provide much necessary context for this discussion.

Satoshi’s BFT database has a few problems; principal among the ones hereto relevant:  Its performance is awful, and its capacity is very limited.  Fortunately, we can appeal to the Wheeler-Lampson Fundamental Theorem of Software Engineering, “All problems in computer science can be solved by another level of indirection.”  (See also RFC 1925 Section 2, (6), (6a), and also (8).  Also, (1) and (2a).)  Enter Lightning Network and sidechains.

If you could propose a sound reason for the importance of on-chain scaling, I would be curious to see that.  I’ve never seen such a reason given, other than handwaving and emotionalist argumentation.  For my part, to answer your question, yes!  I would love to use Bitcoin off-chain.

Bitcoin has a serious existing flaw in my eyes, lack of transactional privacy (unlinkability).  I myself lost a substantial proportion of my Bitcoin by trading it for a privacy-oriented altcoin, for purposes of private long-term value storage.  I dearly wish that instead, I had been able to store genuine Bitcoin on an appropriate Bitcoin sidechain.  Sidechain Bitcoin would be Bitcoin, redeemable on-chain at a 1:1 ratio.

Moreover, the inherent slowness of Bitcoin transactions is unacceptable for exactly the kinds of “cup of coffee” consumer transactions as you mention.  Lightning Network is and will be incomparably more secure than zero-conf on-chain transactions; in my overall assessment, proper use of Lightning should be even more secure than single-conf on-chain transactions.  (Small reorgs do happen!)  When all “best practices” are observed, it’s not even feasible for one person to buy a cup of coffee on chain—even if that were the only transaction occurring at that time worldwide.  As it stands, on-chain, either the merchant takes a risk, or the customer waits somewhere between 10 and 60 minutes.  With Lightning, both the wait and the risk are eliminated.  This is besides the matter of scale.  Why wouldn’t I wish to do such transactions off-chain, where I can use real Bitcoin (instead of toy money such as Visa or Ripple) and the experience will be superior?

Aside, I note that if you’re selling 0.0001 BTC cups of coffee with Lightning, then you’re not creating a permanent, globally archived historical record of each and every purchase of coffee from you.  Besides not bloating a global shared resource (“tragedy of the commons”), this also enhances privacy for both customers and merchants.  Instead of forever recording each and every purpose, Bitcoin’s global public ledger will keep a permanent record of some occasional bulk settlements.

Thus as you can see, I don’t deem on-chain scaling to be important.  What is important is:

0. To keep on-chain transaction processing decentralized, outside the control of any person, entity, corporation, cartel, clique, or government.  This in turn requires that any person of modest means must be able to run a full node.  It is unimportant that the “little people” be able to use on-chain transactions to buy cups of coffee.  It is an imperative absolute rule of Bitcoin that the “little people” must be empowered to run full nodes.  If a full node were to require more resources than can be provided with an ordinary PC and a residential Internet connection, then that would not be Bitcoin anymore.

1. To enable decentralized off-chain consumer transactions.  (0) enables this.  (0) is a prerequisite for this.  If and only if we have a decentralized settlement layer, we can use that as a foundation to add new layers vertically (Lightning Network) and horizontally (sidechains).

Conclusion:

“Bitcoin” as a currency unit means a currency which can be settled and redeemed on-chain, anytime and on-demand.  “Bitcoin” as a technology is now growing to encompass a new ecosystem of Bitcoin applications which use the main chain as their foundation.

I'm sure this idea has probably already been brought up and shot down so please forgive my noob idea.  Would it be possible archive the blockchain every few years? By this I mean take a snap shot of all wallet balances removing the transaction history and greatly compressing the data, this would become the starting point of transaction verification for the next few years and the archive (which would contain the full transaction history over the past few years) could be distributed by archive nodes.  [...]

This exact idea and many of its variations are regularly raised on bitcoin-dev, almost as if on a schedule; and it’s just as regularly shot down, yes.  A pointless discussion can here be obviated by asking, what would be the advantage over running pruned nodes?  Pruned nodes already do internally more or less what you describe, on the level of an individual node rather than of the whole network.  The only meaningful difference I can see is with initial sync, which only needs to be done once.

People who can’t afford to keep a few hundred gigabytes of data stored in perpetuity should enable pruning; and the vanishingly small number of people who are unable to make a one-time download of the blockchain should instead use an SPV client.  But really, almost anybody can afford to run a full node without pruning.  This is a solution in search of a problem.


Edit to add:

The question is, if on-chain-scaling is only a mediocre improvement at the best given current capabilities, then why are we pursuing it at all? a 100-500% increase probably won't make much of a difference since it still won't be in the ball-park needed to be used for small & instant transactions anyways.

I'm not saying that improving the on-chain scaling protocol isn't a good idea, only that unless someone introduces a radical improvement (nothing of the sort that has been suggested up until now), the focus should lie on developing other aspects of the protocol (instead of wasting time & resources on mediocrity).

Make improvement on things that can yield significant improvements, not mediocre improvements. Value prioritization.

Well said, generally.  That’s the right mindset for approaching these problems.  But please also observe that off-chain layers act as a multiplier of something.  If Lightning Network winds up averaging x off-chain transactions per on-chain transaction, for some impressively large x, then doubling on-chain capacity could add up to x capacity to the Bitcoin ecosystem.

That said, I do think that Segwit, Schnorr signatures, and MAST will suffice for on-chain capacity improvements for the near to medium future.  After Lightning begins to mature and sidechains (somehow) become a reality, then we will better know what more is needed on-chain.

[Witrebel]

Thanks for the kind words.  It can take me awhile to write an in-depth reply; and I keep adding to this in pieces, as new replies accrue.  Apologies that the result came out somewhat disjointed and rambling.

I would do well to take more time with formatting and review, taking example from your lead. 

you yourself create them by hooking your gold storage wallet into a “payment channel”.  
They require no trust, and impose no counterparty risk.  
They are mathematically verifiable to represent actual gold; they are not promises to pay gold;
they can be magically transformed back into gold at any time, whenever you wish to do so.

These claims are the exact issue I need to research to gain technical understanding.  Specifically, trustlessness and mathematically verifiable. 

That would be an approximate description of gold’s version of the Lightning Network.  Now, tell me:  In your opinion, “would you really be using [gold]” with such a system?

"Any sufficiently advanced technology is indistinguishable from magic." -Arthur C Clarke
As I noted earlier, I think if the difference between the gold and the paper can be made sufficiently indecipherable, this is a very attractive solution.  With the caveats that little to none of golds intrinsic properties are risked in the process, or overstated on the paper version.

In your opinion, are people who use these exchanges right now “really [using] Bitcoin”?  My opinion is, not really—not in the fullest sense; though they can still transact in actual Bitcoin with people who really use Bitcoin per the motto, “be your own bank”.  At least they can, if the bank deigns to so permit.  I hear that Coinbase closes the accounts of people who send to or receive from addresses disliked by Coinbase.

We agree that these people are not REALLY using bitcoin.   Are they doing so because they are noobs and not up to speed on best practices in crypto?  Are they aware of Mt. Gox but simply trust coinbase because they are FDIC insured on the USD side?  Maybe they are just in it for the speculative gains and not for the underlying tech.  Or perhaps they are sensitive to the friction of on chain transaction and require more liquidity.  Either way, they will need a good alternative.  Perhaps the alternative is off-chain scaling.

Sort of.  Literally, I am saying, “We can’t scale on-chain capacity because that would increase on-chain Bitcoin usage (to the point of overloading nodes).”

But that is a non-problem.  Why is it desirable to have “widespread consumer use of on chain bitcoin transactions”?  Emphasis added.

The general intent of this thread is to find out.  Looks like I need to do a deep dive into off-chain scaling solutions. 

I realize here that, as you stated, we are moving from the technical discussion to its inseparable nontechnical counterpart.  

But first briefly, on the technical side, it must be understood what an awe-inspiring achievement was Satoshi’s creation of a Byzantine fault-tolerant decentralized database, and his application thereof to solve the double-spend problem for a new form of money.  Very smart people had been breaking their heads over these problems, for about two decades.  Now, very smart people have been trying to outdo Satoshi for the past nine years.  Perhaps someday, an ingenious new invention will obsolete Bitcoin’s way of doing things.  But thus far, nobody has proposed a replacement which improves performance and capacity without compromising on decentralization and trustlessness.

I find myself linking this ACM Queue article quite oft of late.  It provides a good bird’s-eye view of not only the technical problems solved by Bitcoin, but also the academic history of those problems.  That will provide much necessary context for this discussion.

In fairness this is an underlying goal of this discussion.  Once the technical realities are objectively stated, the branding/non-technical decisions can be discussed. 
Specifically, the notion that the original white paper was a concept.  A brilliant concept that requires iteration.  The store of value vs peer to peer cash identity issue is largely a matter of branding.  If we continue to brand Bitcoin as both, we are forced to deliver a solution to the cup of coffee problem that exhibits virtually all characteristics of on chain transacting.  If we establish that Bitcoin as it was designed, is truly a settlement layer, we can brand the new layers accordingly.   

If you could propose a sound reason for the importance of on-chain scaling, I would be curious to see that.  I’ve never seen such a reason given, other than handwaving and emotionalist argumentation.  For my part, to answer your question, yes!  I would love to use Bitcoin off-chain.

I don't yet have enough information to answer this accurately.  From an un-informed perspective, its hard to wrap my brain around the idea of "using bitcoins off the Bitcoin blockchain" because the security, utility,  and novelty of the token comes from the chain itself.   The concept of transacting off-chain seems like you might as well brand it as another layer, or symbiotic alt-coin.  Largely this stems from lack of technical understanding.

0. To keep on-chain transaction processing decentralized, outside the control of any person, entity, corporation, cartel, clique, or government.  This in turn requires that any person of modest means must be able to run a full node.  It is unimportant that the “little people” be able to use on-chain transactions to buy cups of coffee.  It is an imperative absolute rule of Bitcoin that the “little people” must be empowered to run full nodes.  If a full node were to require more resources than can be provided with an ordinary PC and a residential Internet connection, then that would not be Bitcoin anymore.

This is another issue I have yet to fully understand.  How much security does a full node truly offer with no hashing power behind it? 

[ir.hn]

Lets look at that gold analogy some more.

The situation is you have one person hand delivering gold from Rome to Naples.  This is bitcoin.

On chain scaling:  Person makes the gold into jewelry so they can carry more, they can wear a bunch of chains and wear a bunch of rings etc.

Off chain scaling:  The person carries gold IOU's so the gold doesn't actually have to move at all.

Clone scaling:  You get 15 people to carry the gold instead of one person.

[nullius]

you yourself create them by hooking your gold storage wallet into a “payment channel”.  
They require no trust, and impose no counterparty risk.  
They are mathematically verifiable to represent actual gold; they are not promises to pay gold;
they can be magically transformed back into gold at any time, whenever you wish to do so.

These claims are the exact issue I need to research to gain technical understanding.  Specifically, trustlessness and mathematically verifiable.  

For a good overview, please see the Lightning Network whitepaper (PDF).

The security of Lightning relies on the property that in the event of any disagreement between transacting parties, either party can slam the payment channel closed by committing its current state to the blockchain.  The blockchain has the final word; parties always have instantly available recourse to the blockchain; thus, the system is trustlessness.  While the channel is open, it allows the parties to cooperatively keep a private ledger of their own transactions with each other (including transactions being routed on another party’s behalf, from another channel).

Imagine that you and I kept a pen-and-paper accounting ledger of our Bitcoin-denominated transactions with each other—but the ledger has the magic property that after every mutually agreed update thereof, each of us has the right bits to claim our coins on the blockchain.  That’s Lightning, in a nutshell.

In your opinion, are people who use these exchanges right now “really [using] Bitcoin”?  My opinion is, not really—not in the fullest sense; though they can still transact in actual Bitcoin with people who really use Bitcoin per the motto, “be your own bank”.  At least they can, if the bank deigns to so permit.  I hear that Coinbase closes the accounts of people who send to or receive from addresses disliked by Coinbase.

We agree that these people are not REALLY using bitcoin.   Are they doing so because they are noobs and not up to speed on best practices in crypto?  Are they aware of Mt. Gox but simply trust coinbase because they are FDIC insured on the USD side?  Maybe they are just in it for the speculative gains and not for the underlying tech.  Or perhaps they are sensitive to the friction of on chain transaction and require more liquidity.  Either way, they will need a good alternative.  Perhaps the alternative is off-chain scaling.

I hate to break it to you, but the vast majority of ostensible humans are actually mewling ovine anthropoids who couldn’t care less about freedom, privacy, security, self-empowerment, or “being your own bank”.  Their very noblest aspiration is to be kept warm, well-fed, and above all, entertained.  N.b. use of the passive, to be kept.

To illustrate, I distill here the essence of a conversation I once had with a person of far above-average intelligence—actually, a man with a science Ph.D.:

“Bitcoin gives you ultimate power over yourself, and ultimate responsibility for yourself.  Of course, these two principles are inseparable.  You need no one’s permission to use the system.  Nobody can freeze or seize your coins.  If you mess up and lose your private keys, there is no higher authority with an override switch.”

(I pause, awaiting the response of being impressed.)

“So if I lose it, it’s gone?”

“Yes.  Bitcoin has no mercy.  It is for smart, responsible people.  But care of your coins is not rocket science; you can keep yourself safe with a userfriendly hardware wallet, plus a paper backup of your seed phrase locked in a safe and written into your last will and testament.”

“If I lose it, it’s just gone forever?  I don’t like that.”

(Crickets chirping.)

Out of respect for the sacred memory of our prophet Satoshi Nakamoto, I did not mention Coinbase.

Specifically, the notion that the original white paper was a concept.  A brilliant concept that requires iteration.  The store of value vs peer to peer cash identity issue is largely a matter of branding.  If we continue to brand Bitcoin as both, we are forced to deliver a solution to the cup of coffee problem that exhibits virtually all characteristics of on chain transacting.  If we establish that Bitcoin as it was designed, is truly a settlement layer, we can brand the new layers accordingly.  

As a matter of my own original thinking (not anything I read anywhere), I myself have taken to thinking in terms of a Bitcoin ecosystem:

• Bitcoin, the Satoshi network, is the foundation.
• Bitcoin, the abstract currency unit, is created ex nihilo from within that foundation, secured by it, and ultimately controlled by it.
• Bitcoin, the abstract currency unit, can/will also power what I think of as “Bitcoin applications” on additional layers which build up and outward from the foundation horizontally (sidechains) and vertically (Lightning).

Distinguishing these concepts helps think clearly about the matter.  That was off-the-cuff; I should try to draw up a more formalized, more rigorous taxonomy. /* XXX TODO */

0. To keep on-chain transaction processing decentralized, outside the control of any person, entity, corporation, cartel, clique, or government.  This in turn requires that any person of modest means must be able to run a full node.  It is unimportant that the “little people” be able to use on-chain transactions to buy cups of coffee.  It is an imperative absolute rule of Bitcoin that the “little people” must be empowered to run full nodes.  If a full node were to require more resources than can be provided with an ordinary PC and a residential Internet connection, then that would not be Bitcoin anymore.

This is another issue I have yet to fully understand.  How much security does a full node truly offer with no hashing power behind it?  

There is a common misconception that miners provide the security of the network.

Miners have only and exactly one job:  To determine the ordering of transactions in a Byzantine fault-tolerant manner, which in turn solves the double-spend problem.  That is a very important job, and also quite resource-intensive; that is why miners get paid for it.  Nevertheless, it is only one component in a machine with many moving parts.

You may often hear that “miners validate transactions”.  This is a trivial half-truth:  Of course, miners must independently validate transactions so as to reliably produce valid blocks.  But miners are not the power responsible for enforcing the validity of transactions across the network.  Full nodes do that.

Each and every full node is responsible for validation:  Validation of transactions, and also validation of consensus rules.  A node’s owner gains additional individual security from independently validating the blockchain; and the network as a whole gains the security of all nodes independently validating the blockchain.  Observe how here as everywhere else, Bitcoin aligns the individual’s selfish interest with the common interest.  A node owner protects the whole network by protecting his own security, first and foremost.

The power of nodes is the reason why many ignorantly-proposed attacks by miners simply would not work.  For example, a persistent mendacious bit of FUD against Segwit is the allegation that miners could steal all the funds from Segwit transactions.  Well, I guess they could try, by creating a block which says “pay me all the money”.  For that matter, a miner could create a block which pays himself 21 million (or 21 billion) bitcoins ex nihilo.  Likewise, a miner could also mine a block filled with gibberish from /dev/random; and any miner could produce a block 8MB or 1GB in size.  But these blocks would be invalid, and would thus be rejected by all full nodes.  To nodes, it would be as if such blocks do not exist; as far as Bitcoin is concerned, they don’t.  Miners who tried such nonsense “attacks” would only be throwing away their hashpower (plus probably getting their IP addresses widely blacklisted).

Contra popular superstition, full nodes do not follow “the longest chain”.  They follow the valid chain with the highest total POW.

SPV clients do almost blindly follow “the longest chain”, which is why they are vulnerable to being misled onto forkchains (among other nasty attacks).

Lets look at that gold analogy some more.

The situation is you have one person hand delivering gold from Rome to Naples.  This is bitcoin.

On chain scaling:  Person makes the gold into jewelry so they can carry more, they can wear a bunch of chains and wear a bunch of rings etc.

Off chain scaling:  The person carries gold IOU's so the gold doesn't actually have to move at all.

Clone scaling:  You get 15 people to carry the gold instead of one person.

On-chain transactions are most analogous to physical exchange of bullion (coins and bars), not jewellery.  Yes, off-chain transactions are more like gold notes—but magical, trustless notes, as I said!

There is no such thing as “clone scaling”.  That’s more akin to eschewing gold, and using silver instead.  Or in this case, brass.  I would rather keep and trade my wealth in car wash tokens than in so-called “clones”; at least, I could get my car clean that way.

Anybody advocating forks, or this “clone” euphemism therefor, is in need of a dire warning:

I emphasize this, for good reason.  I seem to be throwing around that link several times per day, of late.  Bitcoin derives its monetary value from social factors, which are enabled but not created by its technical infrastructure.  The social part requires that there can be only one Bitcoin, unique and undiluted.

This notion of so-called “clone scaling” is absolutely wrong from a social and monetary perspective; but from a technical perspective, it’s one of those ideas which is “not right, and not even wrong”.  It does not make sense at all.  That is why I seem to have ignored it above.

[Colorblind]

For bitcoin to be peer to peer cash, as we all know, it will need to scale with regards to transaction capacity per second.  Obviously this can be addressed through off chain solutions, but I am curious about on chain scaling.  

As I understand it, there are only a few things that could be done to increase on chain capacity without drastically altering the protocol.  

1) You can increase block size.  The major argument against this seems to be the burden placed on running a node, which would increase centralization by making it harder/more expensive to run a full node.

2) Segwit, which makes more efficient use of a given blocksize, and is already implemented.  (I am not well versed in the technical arguments against segwit)

3) Alter the protocol to increase the rate of block production, and decrease the block reward, such that money supply is the same, but you end up with more transactions per second for a given block size.  This would appear to have all the drawbacks of 1), and also an increase in the orphan rate. [/li]

Neither of proposed on-chain solution gives long term solution. It'a all about how fast block chain size approaches infinity with number of transactions/time approaches infinity and the ratio between those two derivatives.

I see coins aging could be a viable on-chain scaling solution (scenario where coins that was not been moved for full halving period will automatically return to coinbase). If this will be the case - full nodes will only require to store blockchain for the past halving period since anything that remained unspent will transfer to the coinbase. This is drastic and, perhaps, cruel measure to take but that will also solve satoshi't billions, burned and lost coins problem.

[DooMAD]

I see coins aging could be a viable on-chain scaling solution (scenario where coins that was not been moved for full halving period will automatically return to coinbase). If this will be the case - full nodes will only require to store blockchain for the past halving period since anything that remained unspent will transfer to the coinbase. This is drastic and, perhaps, cruel measure to take but that will also solve satoshi't billions, burned and lost coins problem.

Firstly, not everyone sees that as a problem and I think you would struggle to find consensus for stealing peoples' property just because you believe they aren't using it when it's been there for a while.  And secondly, the very act of moving idle coins around and reintroducing them into circulation would result in more transactions, which places more burden on the network.  I have coins I haven't moved in four years and they're not going anywhere, thank you.  This sounds very much like altcoin territory as it likely decimates Bitcoin's predictable supply ethos when "new" money would come flooding back into the economy every halving.  

Not to mention the miners that grabbed the first block after that got approved could completely fuck the economy if the coinbase reward included Satoshi's stash along with all the other coins presumed lost.  It would literally be the biggest block reward since crypto was invented.  Potentially worth trillions of dollars depending on what the price is at the time.  Those miners would be newly anointed Gods on Earth and everyone would lose confidence as it all comes crashing down.  Think it through to conclusion, please.