Can the Block Chain get too big and make Bitcoin unworkable?

[pyedpyper]

Hi,

I've just started investigating Bitcoin, and am very excited about the overall concept. I've only read a small amount of the tech, but as I understand it, the Block Chain contains a history of all Bitcoin transactions, and this Block Chain is replicated and stored on every user's computer (is that correct??). And when I installed Bitcoin for the first time, it took many hours to download the Block Chain - which seems to corroborate with how I understand it...

The immediate question which comes to mind is: As Bitcoin grows and many people start using it then could this file not become so big that the whole system becomes totally unworkable? If a billion people are using Bitcoin as their primary trading currency, with multiple billion transactions happening every single day, and every node is constantly being updated with the new information from all of these transactions then it seems this could be a very real problem. Not just for Bitcoin but in terms of overall Internet bandwidth usage!

And pity the poor soul who has to download the Block Chain for the first time! It seems this will be an absolute limit on Bitcoin growth beyond the relatively small scale it is now... Or do I have it all wrong??

Another thing - if someone wanted to fuck with Bitcoin, especially once it becomes widely used, it seems all one would need to do is set up two accounts to automatically send one Bitcoin back and forth multiple times a second (perhaps hundreds or thousands of times a second) - if the software can be so set up, and if that transaction needs to update all one billion block chains at every node every time, then it seems to me it would all just grind to a dysfunctional halt.

I'm wanting to invest heavily in developing Bitcoin and perhaps even setting up an exchange in South Africa (average bandwidth about 1mbps), but if my concerns are real then it seems that Bitcoin might eventually become impractical to actually use once it gets to a certain size and level of adoption, and that in turn would cause the value of the currency to totally devalue. Seems potentially dangerous...

I would appreciate some solid and in depth responses to this.

Thanks for your help...

[Kazimir]

Not a problem. The block chain could be compressed / collapsed at dedicated reference points in history.

Actually, clients could already do so right now, as this is just a 'local' issue and would not require any change in the protocol whatsoever.

[Mushoz]

Hi,

I've just started investigating Bitcoin, and am very excited about the overall concept. I've only read a small amount of the tech, but as I understand it, the Block Chain contains a history of all Bitcoin transactions, and this Block Chain is replicated and stored on every user's computer (is that correct??). And when I installed Bitcoin for the first time, it took many hours to download the Block Chain - which seems to corroborate with how I understand it...

The immediate question which comes to mind is: As Bitcoin grows and many people start using it then could this file not become so big that the whole system becomes totally unworkable? If a billion people are using Bitcoin as their primary trading currency, with multiple billion transactions happening every single day, and every node is constantly being updated with the new information from all of these transactions then it seems this could be a very real problem. Not just for Bitcoin but in terms of overall Internet bandwidth usage!

And pity the poor soul who has to download the Block Chain for the first time! It seems this will be an absolute limit on Bitcoin growth beyond the relatively small scale it is now... Or do I have it all wrong??

Another thing - if someone wanted to fuck with Bitcoin, especially once it becomes widely used, it seems all one would need to do is set up two accounts to automatically send one Bitcoin back and forth multiple times a second (perhaps hundreds or thousands of times a second) - if the software can be so set up, and if that transaction needs to update all one billion block chains at every node every time, then it seems to me it would all just grind to a dysfunctional halt.

I'm wanting to invest heavily in developing Bitcoin and perhaps even setting up an exchange in South Africa (average bandwidth about 1mbps), but if my concerns are real then it seems that Bitcoin might eventually become impractical to actually use once it gets to a certain size and level of adoption, and that in turn would cause the value of the currency to totally devalue. Seems potentially dangerous...

I would appreciate some solid and in depth responses to this.

Thanks for your help...

Multiple solutions have been discussed to address this problem. The first solution involves pruning old transactions from the blockchain, so that the blockchain only contains coins that haven't been spend yet. A second solution involves a meta-chain that only tracks coins that haven't been spend yet, which will coexist with the full chain. This way, most people and lightweight clients can rely mostly on the meta-chain together with a little part of the full chain, namely normal blocks that were created after the latest meta-chain block. People that prefer to keep the entire chain can also do so, so this solution will give people a choice what they personally want. This meta-chain, if implemented, will be merged-mined with a higher difficulty, so that a meta-chain block won't be created every 10 minutes but every X minutes (still to be decided.). For a discussion on this solution please check this thread: https://bitcointalk.org/index.php?topic=88208.0

Also, there's a wiki article addressing some of the scaling complaints. The summary is basically that with pruning and some other compression techniques, together with Moore's Law (exponentially increasing computer power and storage), scaling won't be a problem, even with transaction volumes matching that of Visa (2000 transactions per second!). Here's the link to the wiki article: https://en.bitcoin.it/wiki/Scalability

[nimda]

This problem is so well-known that Satoshi described a fix in his original whitepaper
Now we just need it implemented...

[Kluge]

this Block Chain is replicated and stored on every user's computer (is that correct??). And when I installed Bitcoin for the first time, it took many hours to download the Block Chain - which seems to corroborate with how I understand it...

There are many Bitcoin clients. The "official" Bitcoin client (Bitcoin-qt) is not really official, and there are clients out there which already provide comparable features without requiring you download the blockchain - so no, not everyone needs to download the blockchain to use Bitcoin. It's not necessary to wait hours or days to be able use Bitcoin after installing, and these thin/light clients require only an insignificant amount of computer resources. A fix may be unnecessary (though it would be ideal) while servers can host the blockchain.

Aside from e-wallets (blockchain.info is usually fairly highly-regarded), there are clients such as Electrum for your desktop computer, and clients such as BitcoinJ and Bitcoin Spinner for Android. There are more if you look, but these are the services/clients I hear the most about. With the exception of e-wallets (in all cases I'm aware of), all these lite clients do not store your private keys on a central server, so you are at no risk of losing coins if one of these services are hacked.

Cheers!

[pyedpyper]

Thanks everyone for the solid answers and follow-up links. I see this matter has had a lot of discussion!

Seems that trimming the block chain is a viable solution if properly implemented.

Is it correct that every transaction that happens is written to the block chain of every user?

If so this would appear to present its own bandwidth problem.

In order to be viable as a long term sustainable currency, Bitcoin ultimately needs to be able to scale to billions of transactions per day (thousands per second). Is this possible if each transaction needs to update every single block chain stored on every user's computer? (Moore's Law aside...)

[DannyHamilton]

. . . if someone wanted to fuck with Bitcoin . . . it seems all one would need to do is set up two accounts to automatically send one Bitcoin back and forth multiple times a second (perhaps hundreds or thousands of times a second) . . .

This is currently handled with enforced transaction fees.
https://en.bitcoin.it/wiki/Transaction_fees

[wildgift]

I thought the solution was to take a bunch of old transactions, hash them, and replace those transactions with the hash.  Though it's a one-way process, if someone has the original transactions, it's verifiable.

[Meni Rosenfeld]

Thanks everyone for the solid answers and follow-up links. I see this matter has had a lot of discussion!

Seems that trimming the block chain is a viable solution if properly implemented.

Is it correct that every transaction that happens is written to the block chain of every user?

If so this would appear to present its own bandwidth problem.

In order to be viable as a long term sustainable currency, Bitcoin ultimately needs to be able to scale to billions of transactions per day (thousands per second). Is this possible if each transaction needs to update every single block chain stored on every user's computer? (Moore's Law aside...)

As mentioned, end users will not need the entire blockchain. Just the headers and transactions that involve them specifically, which is a tiny part.

[MoonShadow]

Hi,

I've just started investigating Bitcoin, and am very excited about the overall concept. I've only read a small amount of the tech, but as I understand it, the Block Chain contains a history of all Bitcoin transactions, and this Block Chain is replicated and stored on every user's computer (is that correct??). And when I installed Bitcoin for the first time, it took many hours to download the Block Chain - which seems to corroborate with how I understand it...

The immediate question which comes to mind is: As Bitcoin grows and many people start using it then could this file not become so big that the whole system becomes totally unworkable? If a billion people are using Bitcoin as their primary trading currency, with multiple billion transactions happening every single day, and every node is constantly being updated with the new information from all of these transactions then it seems this could be a very real problem. Not just for Bitcoin but in terms of overall Internet bandwidth usage!

Not all clients need to download the blockchain, just full clients for users who don't want to have to trust anyone as well as miners.  Also, even full clients can prune long spent transactions from their local copy of the blockchain, this is one of the functions of the merkle tree structure within a block.  Certain types of full clients can't do this for practical reasons, but any end-user full client could do it and still be able to verify every transaction that comes it's way.

Another thing - if someone wanted to fuck with Bitcoin, especially once it becomes widely used, it seems all one would need to do is set up two accounts to automatically send one Bitcoin back and forth multiple times a second (perhaps hundreds or thousands of times a second) - if the software can be so set up, and if that transaction needs to update all one billion block chains at every node every time, then it seems to me it would all just grind to a dysfunctional halt.

Try it.  See what happens.

EDIT: To end your suspense, it won't have any noticable effect upon the network as a whole, as this has been considered and already adapted for.  You will, however, succeed in losing your entire balance due to tiny fees required for 'low priority' transactions.  This will still take some time, however, since the network will largely ignore spammy transactions, due to a built in 'point scale' system that discounts bitcoins that have recently been spent.  This doesn't prevent any valid transaction from occurring, but it does delay those that they system consiers spammy.

I'm wanting to invest heavily in developing Bitcoin and perhaps even setting up an exchange in South Africa (average bandwidth about 1mbps), but if my concerns are real then it seems that Bitcoin might eventually become impractical to actually use once it gets to a certain size and level of adoption, and that in turn would cause the value of the currency to totally devalue. Seems potentially dangerous...

I would appreciate some solid and in depth responses to this.

Thanks for your help...

there has been much discussion on the topic of end users in Africa, mostly as bitcoin as a replacement for M-Pesa.  Do more research, you will learn much of value to your position.

[MoonShadow]

I thought the solution was to take a bunch of old transactions, hash them, and replace those transactions with the hash.  Though it's a one-way process, if someone has the original transactions, it's verifiable.

This is functionally what the merkle tree does.  Each transaction has it's own hash, which is then included into a binary hash tree with one other transaction, and then each of those hashes are paired and hashed again to form the full merkle tree.  The last hash (merkle root) is included into the block header, which is then hashed again when the next block includes it into it's own header, thus forms the 'chain' part of the blockchain.  That merkle hash root inside the block header is intended to represent that entire block (in the sequence of the lifespan of the blockchain) once every transaction included into that block has been spent.  Presumedly in the future a great many blocks would be reduced to only the 80 byte, fixed length, headers after a few years time.  There are already several blocks that could be pruned down this way to only those headers, but the pruning functions have not yet been implimented into any client that I am aware.  Mostly because it's not consider a pressing need at the moment.

[wildgift]

I thought the solution was to take a bunch of old transactions, hash them, and replace those transactions with the hash.  Though it's a one-way process, if someone has the original transactions, it's verifiable.

This is functionally what the merkle tree does.  Each transaction has it's own hash, which is then included into a binary hash tree with one other transaction, and then each of those hashes are paired and hashed again to form the full merkle tree.  The last hash (merkle root) is included into the block header, which is then hashed again when the next block includes it into it's own header, thus forms the 'chain' part of the blockchain.  That merkle hash root inside the block header is intended to represent that entire block (in the sequence of the lifespan of the blockchain) once every transaction included into that block has been spent.  Presumedly in the future a great many blocks would be reduced to only the 80 byte, fixed length, headers after a few years time.  There are already several blocks that could be pruned down this way to only those headers, but the pruning functions have not yet been implimented into any client that I am aware.  Mostly because it's not consider a pressing need at the moment.

Thanks for the explanation

[pyedpyper]

Thanks everyone for the solid answers and follow-up links. I see this matter has had a lot of discussion!

Seems that trimming the block chain is a viable solution if properly implemented.

Is it correct that every transaction that happens is written to the block chain of every user?

If so this would appear to present its own bandwidth problem.

In order to be viable as a long term sustainable currency, Bitcoin ultimately needs to be able to scale to billions of transactions per day (thousands per second). Is this possible if each transaction needs to update every single block chain stored on every user's computer? (Moore's Law aside...)

As mentioned, end users will not need the entire blockchain. Just the headers and transactions that involve them specifically, which is a tiny part.

OK, so it's not the case that every transaction (between anyone) needs to be written to every copy of the block chain??

[nimda]

The transactions can be pruned from it later.

[yogi]

The current implementation of bitcoin does have upper limits. Primarily relating to bandwidth and processing requirements of transactions.

If the adoption of bitcoin outpaces moore's law, which I believe it will, then it will become unworkable unless a solution is found.

I think a solution will be found which will likely involve multiple chains or clients that operate in clusters.

[Alwaysmining]

Blockchains can be compressed, so i dont think so

[Meni Rosenfeld]

Thanks everyone for the solid answers and follow-up links. I see this matter has had a lot of discussion!

Seems that trimming the block chain is a viable solution if properly implemented.

Is it correct that every transaction that happens is written to the block chain of every user?

If so this would appear to present its own bandwidth problem.

In order to be viable as a long term sustainable currency, Bitcoin ultimately needs to be able to scale to billions of transactions per day (thousands per second). Is this possible if each transaction needs to update every single block chain stored on every user's computer? (Moore's Law aside...)

As mentioned, end users will not need the entire blockchain. Just the headers and transactions that involve them specifically, which is a tiny part.

OK, so it's not the case that every transaction (between anyone) needs to be written to every copy of the block chain??

It's more accurate to say that every transaction is written to every copy of the block chain, but not everyone has a copy of the block chain.

[pyedpyper]

Thanks everyone for the solid answers and follow-up links. I see this matter has had a lot of discussion!

Seems that trimming the block chain is a viable solution if properly implemented.

Is it correct that every transaction that happens is written to the block chain of every user?

If so this would appear to present its own bandwidth problem.

In order to be viable as a long term sustainable currency, Bitcoin ultimately needs to be able to scale to billions of transactions per day (thousands per second). Is this possible if each transaction needs to update every single block chain stored on every user's computer? (Moore's Law aside...)

As mentioned, end users will not need the entire blockchain. Just the headers and transactions that involve them specifically, which is a tiny part.

OK, so it's not the case that every transaction (between anyone) needs to be written to every copy of the block chain??

It's more accurate to say that every transaction is written to every copy of the block chain, but not everyone has a copy of the block chain.

OK thanks for that clarification. To help me understand better: Say there are 1,000,000 people with Bitcoin accounts then how many people would have a copy of the block chain (based on present stats)?

I'm definitely getting the sense from the various answer there actually is a problem in the Bitcoin architecture i.r.o. scalability which has not yet been resolved. And this is clearly an issue. Relying on Moore's Law or hoping that a solution will "emerge" in the future when it is needed, is a dangerous way to proceed IMHO. Has anyone articulated a detailed solution yet whereby Bitcoin could scale to say 1 billion users (each making 1 transaction per day)?

Does anyone know how many Bitcoin users there presently are? Or what the current average transactions per day are?

[yogi]

IMO scalability is the most pressing issue facing bitcoin at the moment.

For stats goto http://www.blockchain.info/

[Meni Rosenfeld]

OK thanks for that clarification. To help me understand better: Say there are 1,000,000 people with Bitcoin accounts then how many people would have a copy of the block chain (based on present stats)?

As many as are required to keep the network decentralized. I'd say 3,000 is enough, but it's possible to make do with less.

I'm definitely getting the sense from the various answer there actually is a problem in the Bitcoin architecture i.r.o. scalability which has not yet been resolved. And this is clearly an issue. Relying on Moore's Law or hoping that a solution will "emerge" in the future when it is needed, is a dangerous way to proceed IMHO. Has anyone articulated a detailed solution yet whereby Bitcoin could scale to say 1 billion users (each making 1 transaction per day)?

Did you read the page in the wiki about scalability that was linked in a previous comment?

The solutions don't need to emerge, the solutions are known and just need to be implemented.

Does anyone know how many Bitcoin users there presently are? Or what the current average transactions per day are?

How many users is anyone's guess, depends also on whom you call a user. In the past 24 hours there were 20338 transactions.