Blockchain size, exponential growth?

[Revalin]

I doubt 1gb will be that expensive in the future to store, storage capacity is still growing faster than blockchain size.

Hard drives double about every two years.  If the block chain is doubling every year then it will become a problem.

Transaction fees are currently very low to help adoption, but I think it's inevitable that they will have to be much higher in the future.

[1713409644]

1713409644

[Realpra]

I doubt 1gb will be that expensive in the future to store, storage capacity is still growing faster than blockchain size.

Hard drives double about every two years.  If the block chain is doubling every year then it will become a problem.

Transaction fees are currently very low to help adoption, but I think it's inevitable that they will have to be much higher in the future.

Transaction fees are already ~2.5$/trans to the miners. That's already too high in my view.

And yes I am including the block reward as that IS essentially a fee all BTC holders pay to miners.

[Syke]

Yes, at first sight the $10,000 buys a 1 GB of blockchain. But this will be transmitted, stored, re-transmitted, downloaded, etc, eventually forever. I think more of some moral harm to the project or loss of momentum, e.g. if such spammers grow the blockchain to 10 GB this year then many full nodes may just decide to stop.

You're worried that spammers will spend $100,000 to grow the blockchain by 10 GB? That costs a miner $1 worth of disk space. That's a very ineffective attack! That's not going to stop any miners.

[Meni Rosenfeld]

I doubt 1gb will be that expensive in the future to store, storage capacity is still growing faster than blockchain size.

Hard drives double about every two years.  If the block chain is doubling every year then it will become a problem.

Transaction fees are currently very low to help adoption, but I think it's inevitable that they will have to be much higher in the future.

Transaction fees are already ~2.5$/trans to the miners. That's already too high in my view.

And yes I am including the block reward as that IS essentially a fee all BTC holders pay to miners.

As you said yourself it's not a transaction fee, it's a holding fee. It's not the marginal price of making a transaction, you don't have to pay $2500 if you want to broadcast 1000 transactions.

Anyone who holds any bitcoins has signed up for the inflation schedule.

[johnyj]

I don't understand the technical details, but ...

Generally predictions that begin with "I don't understand this BUT" are worthless.

IF you don't understand the technical details how can you make predictions as to the scope of the problem or potential solutions.   

The reality is that only unspent tx need to remain in the blockchain and over time that will represent a smaller and smaller % of total transactions.  Storage requirements are a non-issue long term.  Casual users are best served by lite clients, a development that Satoshi covers in the white paper written over three years ago.

It's NOT a prediction, just some thing is happening now. During the past year, it growed from couple of megabytes into couple of gigabytes, I have not factored that growth rate into my prediction yet:p

[gmaxwell]

Generally predictions that begin with "I don't understand this BUT" are worthless.

It's NOT a prediction, just some thing is happening now. During the past year, it growed from couple of megabytes into couple of gigabytes, I have not factored that growth rate into my prediction yet:p

It's not a prediction, its not something happening, it's nonsense hysteria.  The blockchain size has a fairly modest linear maximum, which can not change without a hard fork of the protocol.  Can this stupid thread die now please?

[commonancestor]

It's not a prediction, its not something happening, it's nonsense hysteria.  The blockchain size has a fairly modest linear maximum, which can not change without a hard fork of the protocol.  Can this stupid thread die now please?

Perhaps not a hysteria, but pointing out the unusual fact that users need to download 2 GB of data to use the software nowadays. Also there is a space for a bigger growth rate within the existing protocol.
From what I've heard so far, this big data is here to stay, and the software will have to deal with it some day in future - full nodes acting as servers to lightweight wallets, pruning, etc.

[DeathAndTaxes]

USER DON'T NEED TO DOWNLOAD 2GB.

If they want to be a full node they will however no matter what is done the initial startup time for a new FULL node will be high.

If Bitcoin becomes 20x as popular the load on full nodes will rise.   Yes higher fees (no not stupidly increasing the low priority mandatory fees I am talking about competition among users raising the effective fees), blockchain pruning, openCL acceleration, more efficient indexing, multi-peer parallel downloads, will make the blockchain/nodes more efficient but just the overall increase in the size of Bitcoin "universe" means the load on full nodes will be HIGHER not LOWER in the future. 

So you have two scenarios:
a) Bitcoin growth stagnates at which point it doesn't really matter how large or inefficient the blockchain is.  The project has fizzled out anyways
b) Bitcoin growth continues and no matter what is done the load on full nodes will RISE.  Full nodes will be more efficient so the network being 20x as large won't require 20x the resources but the load will be higher than today.

NOT EVERYONE NEEDS TO RUN A FULL NODE.  A couple thousand full nodes is more than sufficient to ensure the network is secure, stable, and decentralized.  I don't see major businesses, exchanges, miners, mining pools, service providers, ewallet providers, and bitcoin enthusiasts seeing 2GB or 20GB being an impossible hurdle.  I mean people torrent hundreds of GBs a month.  Will 99.999999999% of users opt for the cost and complexity of a full node?  No and absolutely nothing you can propose will change that.  The good news is they don't have to; if someday there are 20 million bitcoin users and only in in 10,000 runs a full node well that would be awesome.

[johnyj]

That block size chart started from August 2011. BTC is created in 2009, and up to 2011 the block size did not grow over a couple of megabytes, so it was not a linear growth at all

[capsqrl]

USER DON'T NEED TO DOWNLOAD 2GB.

True, but most newbies will go to bitcoin.org and just get whatever's there, which is currently a full-node client. Their first impressions will be accordingly. Either the reference client needs an SPV mode, or bitcoin.org needs to list more alternative clients and explain the difference.

[JTurner]

I don't see major businesses, exchanges, miners, mining pools, service providers, ewallet providers, and bitcoin enthusiasts seeing 2GB or 20GB being an impossible hurdle.  I mean people torrent hundreds of GBs a month.

Maybe so, but they keep a few GB torrents running on a configured area of their hard drive and store the rest on some external hard drive which can be unplugged. The BTC node, on the contrary, keeps it all on the system partition and can't just be split and stored on some external switchable hard drive. Granted 2GB or, to a lesser extent, 20GB aren't too much of a problem, but 200GB would definitely be...

Pruning spent coins sounds great though, can't wait to see it implemented.

USER DON'T NEED TO DOWNLOAD 2GB.

True, but most newbies will go to bitcoin.org and just get whatever's there, which is currently a full-node client. Their first impressions will be accordingly. Either the reference client needs an SPV mode, or bitcoin.org needs to list more alternative clients and explain the difference.

+1. As long as the reference client can only be a full node, users are pretty much encouraged to actually run a full node...

[DeathAndTaxes]

http://bitcoin.org/clients.html

Yup there are lite clients available.

The reference client should never be lite.  I mean it is an oxymoron.  It is the reference implementation of how a node interacts with its peers using the Bitcoin protocol.  It by its very definition must be full and complete.  Users are not required to run a full node.  Satoshi never intended for most users to run full nodes.  It just happened that for the first couple years the amount of resources required to run a full node were trivial and lite nodes simply didn't exist. 

I do think decoupling the "satoshi client" from bitcoin.org eventually is useful.  The Bitcoin.org website should be client agnostic.  The client list page is a step in the right direction. 

[fornit]

when the block reward halves, the amount earned from transaction fees will become more relevant and spamming will become more expensive. with more users the blocks will become fuller and fuller and transaction fees will increase to the point were a hard fork is needed.
before that, there is an upper limit to blockchain growth that isnt very threatening. and considering how low transactions fees are right now, i predict we got some time before scalability becomes a problem even without any technological improvements.

unless bitcoin gets a big jump in users. which, in my opinion, is likely to also increase the speed of development since more people mean more potential developers, investors, sponsors etc.

[gmaxwell]

That block size chart started from August 2011. BTC is created in 2009, and up to 2011 the block size did not grow over a couple of megabytes, so it was not a linear growth at all

Good thing I did not say linear growth then, I said there is a linear maximum. I have a laptop with enough storage for something like 20 years of growth at the maximum rate.  There are certainly things to be concerned about, like the loss of decentralization that comes from more people lazying out of the initial startup time... but there is no danger from "exponential growth" in terms of size.

[johnyj]

Relax, I'm not saying it is stalling my hard drive, but as some one pointed out earlier, increased transaction number will also cause bandwidth and processor power usage surge sooner or later

I do not think this is a technical problem, since this is the first time there is a change has to be made into the system, and there is no existing directive for way of working

The decision making in a GIT world, as I observed for some years, is hierarchical where the core programmer stay in the top. A core programmer maybe very good at programming but have less experience in financial world, so he might not be seeing what is coming. I think someone from the bank/electronic exchange sector back office personal should provide some insight about how they handle the huge amount of transactions today

[mrvision]

Hello,
I don't know what solutions are coming for this real issue. But i find there IS a problem with the exponential growth of the blockchain.

In my opinion, the "There is no such problem" position is ignoring the fact that if nothing is changed, then the blockchain will probably be centralizated in a future. So, why didn't satoshi just said "Ok, for this bitcoin thing we will need a server that will store the blockchain"??. Indeed he didn't wanted to have the database centralizated in one place, he wanted the database to be distributed in the whole world probably to prevent bitcoin to be shut down by goverments.

Anyway, my point is that is better to have the blockchain distributed than centralized. So I was wondering... what if the blockchain was fragmented and the client just sends all the fragments to (like) 10.000 computers in the world so the blockchain could be reintegrated on demand.

To give a metaphor, if we were talking about michael jackson's discography  people are saying that you can download all of it, or stream it from a supernode. Why don't we just keep "some" songs, and be sure that all the 'songs' are available?

This option would be ADDED to the other two that are right now available, so you could choose to:
a) be a sopernode
b) connect to a supernode (and not to have the blockchain)
c) have some parts of the blockchain downloaded and 'lend' some of your hard drive space to the community.

What do you think?

[knight22]

I like this idea

[HeavyMetal]

There are also eWallets such as blockchain.info's MyWallet which do not require trust.

There is no such thing as an ewallet that does not require trust. Even if they propose client-side encryption of the keys, the user is not immune to code poisoning by the server: https://bitcointalk.org/index.php?topic=86278.20

Supernodes and their ewallets are part of the solution for the growing size of the blockchain. I recommend paytunia to new users.

I use a firefox plugin that warns me if javascript has changed.

[HeavyMetal]

As someone who regularly download hundreds of gigs of movies I can say that 2gigs is not very much.

It seemed to me that parsing the block chain was mostly IO bound, I think it is not very efficient in how it handles things in RAM. Neither my bandwidth or CPU was pegged, but my IO was. I think the blockchain could be parsed much faster if the parser was redesigned.

[Pieter Wuille]

Parsing the block chain is trivial, it is updating the transaction database while keeping it consistent that makes it slow.

That said, yes there are serious improvements possible, and we're working on it.