Why bitcoin cannot grow past 4 million users

[SgtSpike]

I started this thread on another forum, but thought I'd post it here as well...  I believe that bitcoin, in its current form, cannot grow past about 4 million users, give or take a few million.

http://bitcoinforums.net/index.php?threads/the-blockchain-a-blessing-or-a-flaw.68/

[Gavin Andresen]

https://en.bitcoin.it/wiki/Scalability

[BinaryMage]

I started this thread on another forum, but thought I'd post it here as well...  I believe that bitcoin, in its current form, cannot grow past about 4 million users, give or take a few million.

http://bitcoinforums.net/index.php?threads/the-blockchain-a-blessing-or-a-flaw.68/

Wow, that's... scary.

I think the bottleneck here is download speeds. Computer storage capacities will increase at at least close to the pace of Bitcoin, and I don't mind dedicating some hard drive space. But internet speeds are dependent on big money-hungry companies, for the most part, and that's what worries me. What we really need is a fast Bitcoin-funded internet service; hopefully that will happen before network congestion becomes a major problem.

[SgtSpike]

https://en.bitcoin.it/wiki/Scalability

A couple things from this wiki that I'd like to respond to...

At very high transaction rates each block can be over a gigabyte in size. These blocks must be stored somewhere. Whilst for speed it'd be ideal to store the block chain entirely in RAM, for cheapness storing only the hot parts in RAM and the rest on disk is the way to go. A 3 terabyte hard disk costs less than $200 today and will be cheaper still in future, so you'd need one such disk for every 21 days of operation (at 1gb per block).

Spending $200 on a hard drive every 21 days is obviously impractical for the average user.  While it could be done by server-class nodes, ramping up transaction volume would server to centralize what is meant to be a decentralized currency.

But you only have to transmit a solved block to your connected peers. If we assume these big futuristic supernodes have something like 40 or 50 peered connections, that means in the worst case scenario where you solve a block OR you receive a block but none of your peers have it yet (unlikely), you have to send ~57 gigabytes of data (call it 60).
Shifting 60 gigabytes of data in, say, 60 seconds means an average rate of 1 gigabyte per second, or 8 gigabits per second.

It's going to be a while before anyone but a datacenter will have these kinds of internet speeds available.  Again, upping the number of transactions only serves to centralize what is meant to be a decentralized currency.

Right now, every client and user of bitcoin can be confident that the blockchain they are relying on for information is 100% accurate and reliable (it is on their computer and verified by thousands of other computers).  But as the blockchain is moved to server-class hardware, because that is the only hardware capable of running it, and people using bitcoins are forced to use a "lite" client version that connects to one of these server clusters, we would then be relying on these servers to provide accurate transaction data to us.  I don't like this, and neither should all the other bitcoin users out there...

[rb2k]

I guess the important part you should respond to:

Satoshi's paper explains how transactions with fully spent outputs can be pruned from long term storage due to how they are arranged in a Merkle tree.

[SgtSpike]

I guess the important part you should respond to:

Satoshi's paper explains how transactions with fully spent outputs can be pruned from long term storage due to how they are arranged in a Merkle tree.

I do not have the knowledge to craft a response to that, but I would love to hear someone who does have said knowledge explain how much space it could save.

We'd still have the gbps limitation though... my internet is only 10mbps upload and 50mbps download - it wouldn't handle a proposed load of 8 gpbs.  And we're a long way off the point at which a residential connection generally would support that kind of load.

[jimbo77]

I guess the important part you should respond to:

Satoshi's paper explains how transactions with fully spent outputs can be pruned from long term storage due to how they are arranged in a Merkle tree.

I do not have the knowledge to craft a response to that, but I would love to hear someone who does have said knowledge explain how much space it could save.

We'd still have the gbps limitation though... my internet is only 10mbps upload and 50mbps download - it wouldn't handle a proposed load of 8 gpbs.  And we're a long way off the point at which a residential connection generally would support that kind of load.

73 %
http://forum.bitcoin.org/index.php?topic=9461.msg137059#msg137059

[casascius]

I guess the important part you should respond to:

Satoshi's paper explains how transactions with fully spent outputs can be pruned from long term storage due to how they are arranged in a Merkle tree.

I do not have the knowledge to craft a response to that, but I would love to hear someone who does have said knowledge explain how much space it could save.

We'd still have the gbps limitation though... my internet is only 10mbps upload and 50mbps download - it wouldn't handle a proposed load of 8 gpbs.  And we're a long way off the point at which a residential connection generally would support that kind of load.

Here is a fundamental problem with pruning the tree to keep in mind.

Yes, it's great that the tree can be pruned.  But the person receiving the pruned data is put in a position where it must trust the source of that data.

Here is why:  The way a client knows that coins are unspent is by looking for a transaction that spends them.  If it's possible to receive a block for which the most recent spend transaction of a given coin has been pruned, that coin will look unspent.  That defeats the purpose of having a block chain copy in the first place, complete or otherwise.

It makes sense to prune off transactions that spent a coin that has been spent at least once more... but a client receiving the pruned block has, by definition, no idea what was pruned.  Pruning is a great way to save disk space once a client knows it can prune the data, but I don't see any way pruning could work without a trusted node scenario.  And if you have to trust a node, why not just trust it for everything.

[Rob P.]

Wow, that's... scary.

I think the bottleneck here is download speeds. Computer storage capacities will increase at at least close to the pace of Bitcoin, and I don't mind dedicating some hard drive space. But internet speeds are dependent on big money-hungry companies, for the most part, and that's what worries me. What we really need is a fast Bitcoin-funded internet service; hopefully that will happen before network congestion becomes a major problem.

This topic should be:  Why bitcoin cannot grow past 4 million MINERS (and even that's not true)

The bandwidth issues are only really applicable to miners, as they are the only ones that have to communicate found blocks in a timely fashion.  By the time there are 4 million miners, the difficulty rating will be such that only people WITH server class hardware will be mining, because it will be impractical for the "average user" to mine.

They will have no problem paying for larger hard drives and faster connections, as they are reaping the benefit of 2000+ transaction fees on each block discovery.  

This is really only an issue as transactions per second increases, today you rarely see a block with 100 transactions, and you often see blocks with 1.

We have a long way to go, which is why the devs would rather focus on the core code and making sure it's clean, rather than worrying about scalability at a 2000 tps level.  The protocol and clients can scale fine.  If the block chain does grow at that level you can begin to implement pruning which no one is actually worried about at this point.

[SgtSpike]

Rob P - do the clients not need to download the entire blockchain to verify that the transactions are legitimate?  If not, then wouldn't we be centralizing the authority on what is a valid transaction and what is not?  Isn't that a bad thing?

[ShadowOfHarbringer]

Rob P - do the clients not need to download the entire blockchain to verify that the transactions are legitimate?  If not, then wouldn't we be centralizing the authority on what is a valid transaction and what is not?  Isn't that a bad thing?

You are completely forgetting about mining pools.

Anybody can pay for few months hosting and start a mining pool... There can be very many pools and people can switch from pool to pool if they find a pool is unfair and/or damaging the network.

This should fix the centralization problem.

[jaime]

Rob P - do the clients not need to download the entire blockchain to verify that the transactions are legitimate?  If not, then wouldn't we be centralizing the authority on what is a valid transaction and what is not?  Isn't that a bad thing?

Afaik all clients are already "delegating trust" in the blockchain builders (mayority of minners). The clients only need the previous block to verify that new blocks coming over can be trusted. Besides that, clients will only need to keep the blocks related to their key pairs, and not the full blockchain.... but I might be wrong.

[SgtSpike]

Rob P - do the clients not need to download the entire blockchain to verify that the transactions are legitimate?  If not, then wouldn't we be centralizing the authority on what is a valid transaction and what is not?  Isn't that a bad thing?

You are completely forgetting about mining pools.

Anybody can pay for few months hosting and start a mining pool... There can be very many pools and people can switch from pool to pool if they find a pool is unfair and/or damaging the network.

This should fix the centralization problem.

Hmmm... that's a good point, and definitely helps.

So the pool server has to be able to have the bandwidth to send a new block out several times over, or to receive a new block and continue propagating it.  The pool server also has to have the hard drive space to store all of the blockchain information.

Would the pools then become entities that are trusted to hand out transaction information from "lite" clients?

I still don't like the idea of users not being able to store the blockchain on their own computer for their own verification.  But, perhaps I do not quite understand the verification process.  As jaime said, perhaps the client only needs to store the bits of the blockchain related to their own transactions.

[Rob P.]

Rob P - do the clients not need to download the entire blockchain to verify that the transactions are legitimate?  If not, then wouldn't we be centralizing the authority on what is a valid transaction and what is not?  Isn't that a bad thing?

Yes, the clients need a full copy of the block chain under the current network topology.  Not because they have to verify all of the transactions (though they can), but because we want the network to be resilient.  Having the entire block chain on every client means no one can usurp the chain, because everyone has a copy.

That said, as a client, you don't have to download the blocks "in a specific amount of time" which is the only thing driving the network speeds required.  Miners have to send them out, because they want a newly found block to be confirmed as rapidly as possible.  So, it behooves them to have fast connections and lots of peers.

As a user (not a miner) you can take your time downloading the block chain, and new blocks.  Unless those blocks have specific transactions impacting you, there's no rush to have it all down in a specific amount of time.  When you perform a transaction, you broadcast the transaction and ask that it be added to the current block.  Again, not an issue as a client, as you're only broadcasting the transaction, not the whole block.

[SgtSpike]

But Rob, we're still going to run in to hard drive space limitations if every client has to have a copy of the blockchain, even if it is only an eventual copy of it.

So the question is, does a client need a copy of the entire blockchain to run with 100% confidence and security in its balance?  What would a blockchainless client look like, and how would it verify its account balance with accuracy?

I still stand by my argument that, as currently implemented, bitcoins have a theoretical user maximum of around 4 million people.  But now I see there may be some fixes to that problem.

[Rob P.]

Afaik all clients are already "delegating trust" in the blockchain builders (mayority of minners). The clients only need the previous block to verify that new blocks coming over can be trusted. Besides that, clients will only need to keep the blocks related to their key pairs, and not the full blockchain.... but I might be wrong.

Yep.  However, as long as no single entity controls more than 50% of the total hashing power of the network, you are probably safe.  However, once a single group DOES control more than 50% of the hashing power, they can create and verify "bad blocks" and fork the chain.

However, this also doesn't address collusion which could also happen in the mining pools, where two pools act in consort to usurp the chain and create and verify "bad blocks".  However, this could be discovered by the client (if programmed properly).  Additionally, the pools currently compete against one another to find blocks, as it is "first come, first served" for a discovered block.  So we have some protections in the fact that it is a competitive market.

All that said, it would be safer for the entire network if mining pools didn't exist.  Unfortunately, with the difficulty ratings we have today, that means smaller miners would not be able to make any BTC on their mining efforts.  Pools solve this by paying out shares, so the smaller miners can still earn on their smaller rigs.  Hence, we have pools.  

[FreeMoney]

Decentralization and freedom do not mean that everyone has a sandwich shop. It means that anyone willing and able can have one and for the lost cost they can get it for with no artificial permissions or licenses. In the monetary realm it doesn't mean that everyone runs a node, but only that anyone who wants to pay the real relevant costs can do it and needs no permission.

That said I think it will be possible for small timers to mine (already don't need the chain) and even do it independently with a chain lookup service.

Small tx don't need to go in the chain individually either. Bitcoin will eventually make the perfect unit of account and settling in the chain can be done less frequently. The incentives exist to make people find ways to make this work well. If being a node is hard tx will be expensive and there will be more profit in finding out of the chain solutions.

And tech improves quickly.

[Rob P.]

But Rob, we're still going to run in to hard drive space limitations if every client has to have a copy of the blockchain, even if it is only an eventual copy of it.

Your argument is that a block could reach 1GB in size at high transactional volumes (2000 tps).  My argument is that you only have to download that block, not the whole chain.  You probably stream movies today, those are close to 1 GB in size, today (not in 10 years where we may hit the transactional volumes you're quoting).  They take an hour or two to come down at a very even rate.  You could probably transfer them, today, in 10-15 min.  

However, block creation is currently limited by difficulty factor.  So, we solve 1 block every 10 minutes or so.  I'm guessing your client would keep up fine pulling new blocks for the block chain over the existing P2P network.

So, you're stating that at 6GB / hour of new "blocks" coming down, you'd run out of hard drive space for the chain.  So, with a 1TB disk, you'd run out of space every 7 days for just the new blocks.

Okay, under the current software, current block chain, and current network design, I agree with you.  I'm not sure about the 4M user number, but I can concede there's an upper limit somewhere.

Does that mean we should figure out and solve that problem right now?  Because some day we may have 2000 tps and our hard drive capacities and network capacities won't have grown to keep up?  Or that Bitcoin is doomed, because this issue wasn't already addressed?  I think, currently, there are more important priorities.  Like getting to 100 tps.

All of these threads like "Why bitcoin cannot grow past 4 million users" or "Why the maximum of 21.000.000 bitcoins cannot be enforced" are just naysaying threads.  They're all predicated on the fact that Bitcoin as implemented TODAY cannot be modified to accommodate these issues in the future.  I just don't agree with that, as I believe the underlying concepts are sound and can be adapted in code to address the issues people are discussing.  

It's kind of like saying to Gottlieb Daimler or Karl Benz in 1885:  "the gas powered engine will never be successful because we'll need to travel 100 MPH in the future and your engine cannot put out that amount of power today".  And yet, today, the gas-powered engine is basically the exact same components it was 100 years ago, there are certainly improvements (fuel injection), but the basics are all still there (cylinders, pistons, and carburetors (in some)).  

Bitcoin in 1 year will not look like Bitcoin today.  But, it will still be Bitcoin.

[SgtSpike]

Sure, you can argue that technology will grow fast enough to allow the average user to continue keeping a copy of the block chain on their home computer, but is that a safe assumption to make?  What if bitcoin continues to grow exponentially?  What if we hit a 1 TB blockchain in the next two years?  Shouldn't we start thinking about and planning for these scenarios, rather than waiting until they are upon us?  What about the fact that most residential users in the US are capped to 250GB of download per month?  Or that some users in Australia or other countries are capped at 40GB or even less?

The basis of my argument is on hard drive space limitations, and the practicality of downloading gigabytes of data for a blockchain when some users in some countries only get 10, 20, 30 GB of data download per month.  If the client doesn't need to download the entire blockchain, and there is a safe and fast alternative to confirming and verifying transactions, then that argument is negated.  Pools can take care of the rest.

My question now then, is what does a blockchainless client look like?  How does it know what its account balance is?  What would stop people from creating a "false node" for blockchainless clients to connect to that looks like the real deal, and accepts incoming connections, but feeds out false information?

[MoonShadow]

But Rob, we're still going to run in to hard drive space limitations if every client has to have a copy of the blockchain, even if it is only an eventual copy of it.

So the question is, does a client need a copy of the entire blockchain to run with 100% confidence and security in its balance? 

No, a thin client can operate without a trusted peer by keeping a full copy of just the block headers.  At 80 bytes per block, that about 4 megabytes of space per year.  Add to that, the thin client must be able to request and verify the blocks that contain the history of the coins that it possesses, but only has to keep what parts of each of those blocks that represent the transactions that are important to it's own coins and the merkle tree branch that proves that transaction belongs there.  The thin client doesn't need to have a trusted peer to count upon, because it can randomly choose a couple of peers that it can see on the network and compare their independent responses to it's queries.  If an attacker can surround the thin client, then a spoof could happen, but if the client is just an average Joe's wallet on his smartphone, what's the point?  There is no way to identify a particular client on the network (via the bitcoin network anyway) to find a high value target, so the best that this could do is trick a random user to accepting an invalid transaction, not steal his coins.

 What would a blockchainless client look like, and how would it verify its account balance with accuracy?

I still stand by my argument that, as currently implemented, bitcoins have a theoretical user maximum of around 4 million people.  But now I see there may be some fixes to that problem.

In addition to the headers only client, a blockchainless client can also exist by choosing your trusted peer.  The nature of Bitcoin also means that if you choose to trust a peer, you are not limited in your choices of peers to choose from.