Bitcoin Forum

...under it's own weight.

The current blockchain is already over 13 GB, and the yearly growth is about 3x. And this while the transaction speed is limited, to delay this excessive growth.
If bitcoin will see a larger adoption, then a minimum of 3x blockchain growth per year is reasonable, but possibly more.
By 2018, 4 years from now, it may reach 13 * 3 ^ 4 = 1TB (one terabyte), which is going to be too much IMO, and still growing fast.
Possible solutions for Average Joe would be storing the blockchain on an external HDD, that's going to be purchased with the latest version of the blockchain,
because downloading without recoverable errors something this large is going to be tricky and take a long time (about 30 hours at 100 Mbps).
Pruning the blockchain might open the door for double-spending, and I haven't yet seen a serious attempt at this. It might also require some centralization.

Uber-bulls reading this will dismiss it as another bear attempt to scare the market, but I think it's a serious problem and should be addressed asap.

I am new so this may sound like a stupid question ( and who knows it maybe :)) but since bitcoin is not centralized who would make such a correction?

core dev team.  i am concerned because they don't appreciate the emergency of the situation.  suppose there is a global fiat crisis.  the blockchain could become a bottleneck when it is needed most.  people could die. 

anyhow, SPV clients, web clients are the ticket.  scalability of transaction volume is not solved, however.

No.

But in 4 years how "big" is 1 TB??  Look at the average hard drive just 4 years ago compared to today.  I think the standard will be 4-5TB+ hard drives.  I see the problem being the transaction speed more than the actual blockchain growth.  How much faster can the internet be to the average Joe to process the hopefully increased transactions?

Storage likely isn't going to be the issue as that traditionally has trended larger and larger.  Bandwidth on the other hand is more likely to be the real issue.  Most Internet connections haven't jumped up in speed over the past few years and unless some miracle happens for more people it will probably be the same speed in 4 years.  On top of that, more and more ISPs are instituting data caps on a monthly basis with insane overage costs.  Many of us who have been running a client all along won't feel the pain, but any new individual or business that is serious about setting something up will have to navigate the challenge of downloading the blockchain.

I like the idea of purchasing a multi-terabyte hard drive with a recent copy of the blockchain on it.  Plug it in, run a MD5 hash program on it to verify that it is legit and then only have to do a few months of catch up instead of 9 years...

Its possible to purge the blockchain of obsolete data. This is outlined in the whitepaper and the necessary data structures are in place (Merkle tree). The code is not yet there, but I see no reason why this would be even controversial.

5 years from now, 1gbps fiber will be what the average consumer's internet speeds are. And 1tb hard drives are already being used as default on most new computers.

The only thing to change is the block size limit, which will be done in a hard fork as Gavin said in one of the videos of a conference.

Most people use lite wallets though don't they?  I never update my client.

I'll be filthy rich by then probably anyway :)

I will add you to the list of people who did not read the satoshi white paper, we had the same discussion a couple of days ago :

7. Reclaiming Disk Space

Once the latest transaction in a coin is buried under enough blocks, the spent transactions before it can be discarded to save disk space. To facilitate this without breaking the block's hash, transactions are hashed in a Merkle Tree [7][2][5], with only the root included in the block's hash. Old blocks can then be compacted by stubbing off branches of the tree. The interior hashes do not need to be stored...


- Otherwise you do not have to worry about disk space, in a decade we saw technology adapting crazy fast, disk capacity went crazy, I still remember my PC having only 20GB disk in 1999, the one before had only 1GB, today I have 4TB, and 1TB is the basic that you get with your computer and an upgrade of 3TB is also affordable.

what most worries me is the time it takes to download the whole blockchain, it gets bigger and slower, but this is not a problem, Bitcoin is a programmable money that can be adapted when needed....

 

If this is an attempt at serious discussion, and not just a bear trying to scare the market, then why a new thread in the Speculation subforum when there are easily 10 existing threads between "Bitcoin Discussion" and "Development and Technical Discussion"?  Wouldn't it be more prudent to discuss this on the parts of the forum that developers are more likely to frequent?  If you were to read some of those threads, perhaps you would see that the developers agree that the current state of the code does have scalability issues, and you could bring yourself up to date on the efforts to address the various challenges.  Code is being written, but these things take time.  If progress is too slow for your tastes, roll up your sleeves and help out.

I really wish this was implemented in the default client.  The size of the blockchain is annoying. I don't think it will cause bitcoin to collapse, but it will slow down adoption at least a little.

I gave my Grandmothers paper wallets for Christmas, which they loved because of how techy it seems.  I'm not going to install the client though for now, because I don't want to trigger data overage charges, their computers are out of date, and I don't want to explain why their internet is so slow for the first two days.  For now, their bitcoins are in filing cabinets.  "Why does bitcoin make my computer slow?"  "Because it is downloading and processing a data file the size of your entire DVD collection."

Sure, there are solutions to the blockchain problem, but the next step for bitcoin is to make those solutions trivial.

edit:  Thanks notme, I will seek out those threads.

This is not a problem and never was. OP, go read the white paper and the developments in the dev subforum. This is what happens when n00bs spend too much time in Speculation and not enough time learning about the bitcoin protocol itself.

We get a thread whining about scalability at least every month.
1) You do not need the entire blockchain
2) Blockchain compression
3) Big hard drives. When I was a kid people had 32 megabyte hard drives. And again, you don't need the entire blockchain!!!
4) Lite clients that require 0 trust using some devious merkel tree tricks
5) Read the fucking available material on the problem already, smart people have already solved it.

yep. These are not new ideas. Indeed they have been thought about since before the Genesis block. The only question is which solution we use.

In terms of averting disaster:
It doesn't matter if solutions are known.
It doesn't matter if a single solution has a consensus.
It matters whether a solution has been fully deployed.
Only then can it be relied upon when SHTF.

I saw a 1 terabyte flash drive at CES this year. Not really worried about this.

Since when was the core dev team responsible for saving the world in this crazy person Y2K-style scenario?

Thank god, the blockchain is taking up sooooo many floppies.

What impending disaster do you forsee that can't be immediately dealt with via existing codebases?

The only thing I see that could be an issue is the 7tps limit, but in a true global economic crisis, I'm sure we could get support to replace the MAX_BLOCK_SIZE = 1 with a MAX_BLOCK_SIZE = 100 or some other hack to keep things running.  Ideally, we solve this problem carefully and methodically so we don't have to keep changing a constant, but the block size can adjust while maintaining proper incentives for broadcasting, processing, and storing transactions.

Otherwise, transaction volume can be dealt with via existing pruning and bloom filtering solutions.

About point 1.
I assume this is correct but imo there is no simple way to determine what can be deleted.
They cannot purge transactions older than X years since this might kill peoples offline wallets.
You can only remove transactions to accounts that now hold 0 BTC but that might give the spender free money again.
The only way you could pull it off is by making a 5year checkpoint every 1st of January, in which all the balances of blockchain accounts ever used are stored and only transactions from that date forward are processed.
I see a lot of problems with your point 1.

P.S. haven't read the smart people solution but a link would be appreciated.

There are multiple levels available with existing software.

Bitcoin-qt stores the entire blockchain, but supports bloom filters on the network side.  Bloom filters allow clients connected to the network to specify a small hash value that represents the addresses they are interested in.  Mobile/thin clients can use these filters to only receive transactions for the addresses they care about (and some false positives as a privacy feature).  They still get the full block header, which is small, but allows them to verify the transaction hash is indeed confirmed.  In the future, the official client will likely move to a model where some nodes store the full chain, but most nodes only store a GB or so.  This GB can be randomly selected (other than the transactions the user clearly cares about) and this will enable wide coverage of the blockchain spread across many nodes.  Miners only need the unspent transaction output set, or UTXO.  This is still only a few hundred megabytes.

Thank you for the insight!

I agree this could work as long as there are enough full-blockchain nodes available or enough partial nodes to "seed" all the addresses as false positives so you still get full coverage.
It's a good thing to know that the size of the blockchain won't become a problem :)

About the same answer was given 6 months ago in another thread. In theory, the problem can be solved, but it's not so simple,
or the devs would have solved it already. It requires some effort to code and test, and IMO the early adopters should be putting
some money into solving it before it becomes more than a nuisance.

Except work is being put into it.  There are several projects, but one is listed right on the bitcoin wiki's scalability page under "Related works".  You really haven't put much effort into researching this.  The problem really only exists with the reference client, and most new users aren't using that anyway.  They use a mobile wallet or a web wallet.

let's cross that bridge when we get there.