Scalability - because it's good to have stretch goals

[justusranvier]

I think the wiki article for Scalability is not ambitious enough; Bitcoin should be capable of processing 1 million transactions per second by 2030. This allows a population of 10 billion to each initiate 100 transactions per day.

Can it be done on hardware expected to be available to average users in 2030? What changes would need to be implemented to allow Bitcoin to scale to this level?

Using the ratio from the Bitcoin wiki, 1 million transactions per second requires about 4 gigabits per second. If Nielsen's Law of Internet Bandwidth holds until 2030 this amount of bandwidth will be start to become available to home users by 2022, and should be a small fraction of the average 2030 user's connection. Broadcasting 1 million tps through the network would not appear to be a problem by 2030.

Transmitting blocks as they are currently constructed would be a problem, because right now they include a complete copy of every transaction so require a burst bandwidth much higher than the average in order to distribute blocks in a timely fashion. There's no reason this must remain the case, however. Assuming that all nodes have a method of retrieving transactions which are not in the memory pool, the block could consist of a header and a list of hashes. This reduces the size of a block by a factor of 16 (512 byte transaction / 256 bit hash). A miner could reduce the requirement for burst bandwidth further by pre-announcing the transactions which will occur in their block so that only the nonce and final hash would need to be broadcast when they solve a block.

CPU: If a common 2012 CPU can process 4000 tps, then a 2030 CPU should be able to process over 100 million tps by Moore's Law.

Storage: Requiring each node to store a complete copy of the entire blockchain back to the genesis block is excessive. A distributed, redundant, content-addressable data store could serve the function of storing history and broadcasting transactions. We already have an example of such a datastore in Freenet. Freenet's datastore is useful for this application because nodes automatically specialize without any explicit configuration, and the inter node routing adjusts to demand in real time in order to converge on an optimal configuration. Commonly-requested data is intelligently cached and the store has a provision for pruning unneeded keys when a node runs out of storage space. The P2P aspect of Freenet is slow due to it being optimized for privacy, but without that requirement other performance enhancements are possible.

[1713965407]

1713965407

[1713965407]

1713965407

[1713965407]

1713965407

[TangibleCryptography]

Other than needing storage space which grows by 35 TB per day you are all set.  While storage capacity likely will grow I doubt anyone is going to have a 12.6 EB drive anytime soon.

On edit: you updated storage ... not storing locally simply trades local storage for even more bandwidth. I say we get to 100 tps then worry about 10,000 tps.  Wondering if Bitcoin can scale to three or four magnitudes larger than the largest global payment network is just silly.

[justusranvier]

Other than needing storage space which grows by 35 TB per day you are all set.

1) It's not necessary for every node to keep a copy of every transaction forever.

2) It probably isn't necessary for anyone to keep all the prunable transactions forever.

3) A 12.6 EB drive should cost less than $1000 by 2030.

[justusranvier]

On edit: you updated storage

I accidentally hit the Enter key with the focus in the wrong part of the screen and submitted the post before I was ready.

[justusranvier]

Wondering if Bitcoin can scale to three or four magnitudes larger than the largest global payment network is just silly.

As Bitcoin scales it becomes increasingly more difficult to introduce changes like new block versions that would support that kind of growth. It's much easier to put algorithms in place now that are sufficiently efficient to scale to those magnitudes well before they are needed.

Avoiding this discussion in the present is a good way to help people who want to prevent Bitcoin from ever succeeding to that degree.

[drawingthesun]

Dedicated persons can run dedicated nodes, but most of us can be apart of a distributed node where we share the storage and bandwidth with other users to make up a full node.

Some might have enough clout to become apart of a 100 computer node, others might join in on a million computer node.

[acoindr]

Wondering if Bitcoin can scale to three or four magnitudes larger than the largest global payment network is just silly.

As Bitcoin scales it becomes increasingly more difficult to introduce changes like new block versions that would support that kind of growth. It's much easier to put algorithms in place now that are sufficiently efficient to scale to those magnitudes well before they are needed.

Avoiding this discussion in the present is a good way to help people who want to prevent Bitcoin from ever succeeding to that degree.

I agree discussion on block size issues are better earlier than later. Actually, I came to this forum section to post a thread asking for current thoughts on the issue. So I'll ask here instead. My belief is most people are letting the information digest as is often helpful with hard problems. Would that be accurate?

[markm]

To me the ASIC situation is a big concern, until we can order off the shelf consumer ASIC units any thoughts about scaling up are moot.

For all we yet know a few ASICs produced to finance original development might arrive in consumers hands and thereafter ASIC manufacturers simply go private with their own farms, making hashing a centralised industry thus rendering moot whether mining will be centralised or not.

Look at ASICminer for example, hoarding the ASICs to themselves, denying the use of their hashing power for merged mining, not only monopolising hashing to themselves but also to just one chain of potentially many merged chains. They even commented in some thread or other that they don't even see why anyone would even want ASIC at home, well hey guys I want to mine all my chains and you shitheads are failing to mine most chains, heck you dont even mine namecoin you fuckups! A real company might not even be allowed to do that with shareholders money, they are leaving earnings on the table, thus denying potential dividends to their shareholders, by wasting all that power on just one chain!

-MarkM-

[acoindr]

To me the ASIC situation is a big concern, until we can order off the shelf consumer ASIC units any thoughts about scaling up are moot.

For all we yet know a few ASICs produced to finance original development might arrive in consumers hands and thereafter ASIC manufacturers simply go private with their own farms, making hashing a centralised industry thus rendering moot whether mining will be centralised or not.

Look at ASICminer for example, hoarding the ASICs to themselves, denying the use of their hashing power for merged mining, not only monopolising hashing to themselves but also to just one chain of potentially many merged chains. They even commented in some thread or other that they don't even see why anyone would even want ASIC at home, well hey guys I want to mine all my chains and you shitheads are failing to mine most chains, heck you dont even mine namecoin you fuckups! A real company might not even be allowed to do that with shareholders money, they are leaving earnings on the table, thus denying potential dividends to their shareholders, by wasting all that power on just one chain!

-MarkM-

Bitcoin faces some challenges, I think, but we've always known that. I think we imagine most any impediment can be overcome by the creative determined efforts of the community.

We take problems as they come. Look at the apparent legal victory, for example, we've recently had with the FinCEN guidance.

The difference between ASICS concern and the block size issue is that ASICS, if ever a problem, could be overcome various ways. By contrast, the block size is built into the community itself. If block size is ever a problem, due to the way it's structured, then that problem remains so long as there is no workable, acceptable alternative.

[cjp]

I think the wiki article for Scalability is not ambitious enough; Bitcoin should be capable of processing 1 million transactions per second by 2030. This allows a population of 10 billion to each initiate 100 transactions per day.

I think 2 transactions per person per day is good enough; make it a bit more so that people don't immediately hit a scarcity wall during rush hours.

I also think that most transactions can be handled off-blockchain; I'd estimate that about 1 in 1000 transactions need to be registered in the block chain.

There is a thread about how to make trust-free nodes that only store unspent transactions. If this is going to work, block chain storage doesn't need to grow continuously. I once estimated that without off-blockchain handling there would be 30000 unspent transactions per person; with off-blockchain handling that might be about 30 per person, but I'm not really sure about that. It doesn't really sound like a bad estimate though.

Please don't trust too much on Moore's law and similar exponential growth laws. Every exponential growth curve is going to hit some physical limit, and since exponential growth goes so fast, it's going to end in decades rather than centuries. For CPU clock frequency it has already ended; for hard disk capacity we're near the limit (but storage space might continue to rise if we start using different technology); for component density in chips we still have a factor 1000 or so before the limit.

The good news is that even without hardware improvements there are still software improvements that can be a game changer when it comes to Bitcoin scalability.

[justusranvier]

I also think that most transactions can be handled off-blockchain

Off-blockchain transactions defeat the purpose of using Bitcoins in the first place as they reintroduce the possibility of prior restraint and do not have the same security and irreversibility guarantees as the main chain. Off-blockchain transactions don't help users in any way - they just make the lives of thieves and control freaks easier.

[acoindr]

I also think that most transactions can be handled off-blockchain

Off-blockchain transactions defeat the purpose of using Bitcoins in the first place as they reintroduce the possibility of prior restraint and do not have the same security and irreversibility guarantees as the main chain. Off-blockchain transactions don't help users in any way - they just make the lives of thieves and control freaks easier.

Wow. I'm probably the biggest believer in off-chain transactions.

In fact, even before I knew the details surrounding the block size I knew network transactions couldn't scale to global capacity. Still, I had high confidence in Bitcoin knowing off-chain transactions could play a major role.

I think it's key any plans for Bitcoin development take into consideration off-chain transactions, which don't defeat the purpose of Bitcoin any more than Casascius coins would (which obviously omit using the chain), because the core network remains an option. Different use cases fit different transaction channels. The core network, as I've said repeatedly, is not ideal for making everyday transactions.

[justusranvier]

In fact, even before I knew the details surrounding the block size I knew network transactions couldn't scale to global capacity. Still, I had high confidence in Bitcoin knowing off-chain transactions could play a major role.

How do you know this? I'm open to this possibility but I'm not going to accept it as a mere assertion. Show your work if you want me to believe you.

I think it's key any plans for Bitcoin development take into consideration off-chain transactions, which don't defeat the purpose of Bitcoin any more than Casascius coins would (which obviously omit using the chain), because the core network remains an option.

As of right now, with no future changes to the protocol, this is false. The core network will not remain an option because once the blocks reach 1 MB no one else will be permitted to use the core network no matter how much they are willing to pay.

If alternate chains are such a benefit then there is no problem removing the artificial transaction cap on the main chain. Miners won't process more transactions than it is profitable to mine on the main chain, and since the off-chain methods are beneficial in their own right then people will naturally start using them. They are beneficial to the users based on their own merits, right?

[cjp]

Off-blockchain transactions defeat the purpose of using Bitcoins in the first place as they reintroduce the possibility of prior restraint and do not have the same security and irreversibility guarantees as the main chain. Off-blockchain transactions don't help users in any way - they just make the lives of thieves and control freaks easier.

You underestimate what is possible with off-blockchain transactions. Basically, the way to make them secure is to always keep the possibility to move them into the block chain, to enforce good behavior.

To make an odd comparison: it's a bit similar to how a small police force can keep a large population under control, as long as everybody can stop/revert a crime by calling the police, and the police is responsive, most people will stay honest even without police intervention(*).

(*) OK maybe most people will stay honest even in a complete anarchy, but you get the idea.

[cjp]

BTW, if you combine my estimates, you'd only need 32MiB/block. That sounds like it's even doable with today's hardware, except maybe for storage (about 12TiB for 10 years of block chain; even that isn't so bad, since it's only needed 10 years from now). If the block size limit is put on 128MiB we'd still have some room in case things turn out different than expected.

[justusranvier]

You underestimate what is possible with off-blockchain transactions. Basically, the way to make them secure is to always keep the possibility to move them into the block chain, to enforce good behavior.

I am extremely skeptical that an off blockchain system will be able to secure itself against the kind of computing power that miners on the main chain are accumulating. Ripple claims to be able to do this but it remains to be proved. Any alternate transaction system which require fewer resources to break the network as compared to the main chain is less secure by definition. As the hashing power of the Bitcoin network grows, so also will the resources available to potential attackers.

[nwbitcoin]

Other than needing storage space which grows by 35 TB per day you are all set.  While storage capacity likely will grow I doubt anyone is going to have a 12.6 EB drive anytime soon.

On edit: you updated storage ... not storing locally simply trades local storage for even more bandwidth. I say we get to 100 tps then worry about 10,000 tps.  Wondering if Bitcoin can scale to three or four magnitudes larger than the largest global payment network is just silly.

I think its extremely important to think about being able to scale to this size because all the evidence from other forms of digital adaptation has shown a far higher rate of operation, once its become an option.

e.g. the number of ebooks that now exist since ebook readers became popular, think how much more people download as soon as their internet speeds up.  

We are also forgetting the current off network transactions.

If you are spending cash, its an offline transaction which most likely isn't recorded anywhere meaningful.
Give people the option to have a digital wallet and then its only possible to do online transactions - the number of online transactions is going to grow exponentially - especially when movements from one wallet to another is also considered a transaction.  
We shouldn't consider forecasting the future using only what we know today and adding time.  We have to put in some extra thinking!

100tps is a good target!

[acoindr]

In fact, even before I knew the details surrounding the block size I knew network transactions couldn't scale to global capacity. Still, I had high confidence in Bitcoin knowing off-chain transactions could play a major role.

How do you know this? I'm open to this possibility but I'm not going to accept it as a mere assertion. Show your work if you want me to believe you.

You're right. I should have phrased that as: "I knew network transactions couldn't [easily] scale to global capacity. "

I think it's key any plans for Bitcoin development take into consideration off-chain transactions, which don't defeat the purpose of Bitcoin any more than Casascius coins would (which obviously omit using the chain), because the core network remains an option.

As of right now, with no future changes to the protocol, this is false. The core network will not remain an option because once the blocks reach 1 MB no one else will be permitted to use the core network no matter how much they are willing to pay.

For the moment completely disregard off-chain transactions. If the network becomes inaccessible due to the 1 MB limit then it is no longer a good option regardless of off-chain transactions. I meant off-chain transactions don't make using the core network less of an option, if it's an option at all.

If alternate chains are such a benefit then there is no problem removing the artificial transaction cap on the main chain.

I personally believe that, yes. The problem is I can't speak for the entire community. So the potential problem of altering the cap seems to remain.

Miners won't process more transactions than it is profitable to mine on the main chain, and since the off-chain methods are beneficial in their own right then people will naturally start using them. They are beneficial to the users based on their own merits, right?

Yes, I believe that's right.

[acoindr]

I am extremely skeptical that an off blockchain system will be able to secure itself against the kind of computing power that miners on the main chain are accumulating.

Why? You don't think PayPal's accounts, for example, can be as secure? Off-chain systems means more security options, something not possible with the decentralized nature of Bitcoin.

I think its extremely important to think about being able to scale to this size because all the evidence from other forms of digital adaptation has shown a far higher rate of operation, once its become an option.

I completely agree. People gravitate to what's most effective/convenient. There has already been a massive shift away from physical money to digital money in the form of credit/debit cards etc. because of this, with all the fraud, fees and other problems that entails. Imagine Bitcoin adoption as a superior form of e-payment ability.

[justusranvier]

You don't think PayPal's accounts, for example, can be as secure? Off-chain systems means more security options, something not possible with the decentralized nature of Bitcoin.

PayPal has zero security - they can steal from their users at any time. This is a common feature of all centralized systems, which is exactly why I said that off-chain transactions only benefit thieves and control freaks.