Blockchain size, exponential growth?

[johnyj]

I don't understand why the blockchain has grown exponentially in the latest months, with that huge size of the database on hard drive, it's getting difficult to convince new users to start to use it: They have to wait a whole day to download the block chain.

Is there any light client which download only latest part of blockchain and verify the rest part using only a checksum?

[CIYAM]

You might find this useful: https://bitcointalk.org/index.php?topic=94102.0

(the other choice is to use an alternative client)

[johnyj]

I don't understand the technical details, but look at the blockchain size chart:
https://blockchain.info/charts/blocks-size

if all the transactions are recorded in blockchain, it will surely grow out of the capacity of any hard disk in one year, after another year, it will grow out of the capacity of any storage server farm I know. This has to be solved as soon as possible

Any client should only interested in those transactions related to his own account, for other transactions he can use block explorer to search, maybe someone from a bank can explain how bank handle huge amount of transaction records?

[CIYAM]

Indeed the growing size is of concern (and there have been numerous threads about this not that long after Satoshi Dice was launched).

From these other threads I think that the momentum to implement "pruning" of the blockchain (and other scalability solutions) is increasing at an even more rapid rate.

(am certain that this will be sorted out before it becomes such a problem - but as stated before there are other clients that don't require the blockchain)

[Meni Rosenfeld]

The blockchain size is directly proportional to the number of transactions in Bitcoin's history. The blockchain is larger now because there are more transactions. This is in large part due to SatoshiDice, but also due to general growth.

There are indeed lightweight clients (they don't work in the way you suggested though) and this is an active development subject. There are also eWallets such as blockchain.info's MyWallet which do not require trust.

[johnyj]

I believe from client side there will be a solution, but how about blockchain itself?

After some study I can see that with today's 400 transaction/10minute rate, the block size is already 250K, if BTC gained popularity quickly, then there will be thousands of transactions per minute

[Meni Rosenfeld]

I believe from client side there will be a solution, but how about blockchain itself?

After some study I can see that with today's 400 transaction/10minute rate, the block size is already 250K, if BTC gained popularity quickly, then there will be thousands of transactions per minute

It's basically a solved problem. There's pruning and there are ways to do transfers outside of the blockchain. Also, not everyone needs the blockchain so it can afford to be a bit large.

[totaleclipseofthebank]

But has pruning been implemented?

This is a problem that is dismissed far too quickly by the same old arguments. There is no way that people are going to use bitcoin in the intended way if they need to download 4gb (and then another couple 100 every week) just in order to make payments.

It is counterproductive to simply dismiss this problem as solved, because it isn't actually solved yet

[Meni Rosenfeld]

But has pruning been implemented?

No. But it's being worked on.

There is no way that people are going to use bitcoin in the intended way if they need to download 4gb (and then another couple 100 every week) just in order to make payments.

It was never intended that every Bitcoin user will download the entire blockchain.

This is a problem that is dismissed far too quickly by the same old arguments.
...
It is counterproductive to simply dismiss this problem as solved, because it isn't actually solved yet

I see it the other way around - the same old arguments about the size of the blockchain are dismissed because they bring nothing new to the discussion and ignore existing work on the subject.

Yes, the solutions still need to be implemented. But complaining about it isn't helping.

[mp420]

I'd say the problem is not solved until there's a reference implementation of what's proposed in this thread. Even with pruning the blockchain is going to be too large before long, if Bitcoin is going to be ever more widely adopted.

Also, after the block size limit is lifted (this must be done at some point) network bandwidth is going to be the bottleneck that limits who can run a full-fledged client.

[totaleclipseofthebank]

Thanks for clarifying Meni.

I understand that complaining leads nowhere--my point was just that there is still work to be done here!

[Realpra]

But has pruning been implemented?

No and it would not be enough by itself soon anyway due to bandwidth and CPU usage also rising.

You can search "swarm client" on this forum to find a possible solution to BTCs algorithmic problem.

Until that is developed I suggest using electrum or some online wallet for new users.

[johnyj]

Now I understand why banks have to hire so many people to take care of the account related activities

[Technomage]

I don't think Bitcoin will ever again be so that it's feasible for everyone in the world to run a full node. I don't know if it ever was, light clients are simply much faster and more convenient and even more so recently. The scaling work is basically needed so at least some people can run the full client. Not everyone needs to run it.

[Boussac]

There are also eWallets such as blockchain.info's MyWallet which do not require trust.

There is no such thing as an ewallet that does not require trust. Even if they propose client-side encryption of the keys, the user is not immune to code poisoning by the server: https://bitcointalk.org/index.php?topic=86278.20

Supernodes and their ewallets are part of the solution for the growing size of the blockchain. I recommend paytunia to new users.

[commonancestor]

Hello, my opinion on the subject is

The blockchain growth may not be exponential but it grows quickly. (http://blockchain.info/charts/blocks-size?timespan=all&showDataPoints=false&daysAverageString=1&show_header=true&scale=0&address=) From May'12 it grows about +1 GB every 3 months! Plus here are my thoughts that it may possible for an attacker to spend $ 10,000 and do the same in 14 days. (https://bitcointalk.org/index.php?topic=94619.0#msg1046793)

On the other hand, I see this can be helped.
Raising fees. I think that fees are extremely low and should be considered at more realistic level, say 0.05 USD/tx. Also miners would welcome this 1000 times .
Light wallets, incentivised servers. I see there is a number of alternative clients but the reference client is simply a full node.
Pruning. Spent transactions will be possible to have pruned one day. This would be a relief from Satoshi Dice, however, I can still imagine millions of spam addresses with unspent micro-amounts.

I think if not addressed then it will be embarrassing for new users to download some 4 GB of blochchain next year.

[DeathAndTaxes]

I don't understand the technical details, but ...

Generally predictions that begin with "I don't understand this BUT" are worthless.

IF you don't understand the technical details how can you make predictions as to the scope of the problem or potential solutions.   

The reality is that only unspent tx need to remain in the blockchain and over time that will represent a smaller and smaller % of total transactions.  Storage requirements are a non-issue long term.  Casual users are best served by lite clients, a development that Satoshi covers in the white paper written over three years ago.

[DeathAndTaxes]

Raising fees. I think that fees are extremely low and should be considered at more realistic level, say 0.05 USD/tx. Also miners would welcome this 1000 times .

The protocol can't enforce fees.  The reference rules to prevent DOS attacks impose a required fee on LOW PRIORITY TX only.   Miners are free to choose what transactions they include in a block.  If you run a mining node you are free to exclude ANY transaction you wan't (for any reason).   Raising the mandatory fee on LOW PRIORITY TXs would be foolish and not have the desired effect you think it would as most transaction are HIGH PRIORITY and thus aren't affected by tx fee rules.  

Pruning. Spent transactions will be possible to have pruned one day. This would be a relief from Satoshi Dice, however, I can still imagine millions of spam addresses with unspent micro-amounts.

Anti-spam rules make this prohibitively expensive.  The attacker would end up paying 10x to 100x the cost incurred by full nodes (and light nodes would incur no cost).  Trying to outspend your enemy when there are more of them and it costs you $100 for every $1 it costs you is kinda dumb.  Sort of like designing weapons which shoot money at the enemy and trying to kill them by burying them in giant piles of cash.  While technically possible there are far more effective methods of attack.

I think if not addressed then it will be embarrassing for new users to download some 4 GB of blochchain next year.

Why?  There is no need for users today to run a full node.  If they wish to run a full node then the 4GB "cost" is something they accept.  A long term project would be one that OpenCL accelerates the blockchain verification.   In theory bootstrapping a new node could be done significantly faster.

[commonancestor]

Raising fees. I think that fees are extremely low and should be considered at more realistic level, say 0.05 USD/tx. Also miners would welcome this 1000 times .

The protocol can't enforce fees.  The reference rules to prevent DOS attacks impose a required fee on LOW PRIORITY TX only.   Miners are free to choose what transactions they include in a block.  If you run a mining node you are free to exclude ANY transaction you wan't (for any reason).   Raising the mandatory fee on LOW PRIORITY TXs would be foolish and not have the desired effect you think it would as most transaction are HIGH PRIORITY and thus aren't affected by tx fee rules.  

Pruning. Spent transactions will be possible to have pruned one day. This would be a relief from Satoshi Dice, however, I can still imagine millions of spam addresses with unspent micro-amounts.

Anti-spam rules make this prohibitively expensive.  The attacker would end up paying 10x to 100x the cost incurred by full nodes (and light nodes would incur no cost).   It is like trying to outspend your enemy when you enemy is thousands of users and it cost you $100 for every $1 it costs them.  Futile and stupid.

I think if not addressed then it will be embarrassing for new users to download some 4 GB of blochchain next year.

Why?  There is no need for users today to run a full node.  If they wish to run a full node then the 4GB "cost" is something they accept.

I hope I am not wrong here but something about fees may be encoded in the software, like not to transmit low priority transactions with fees under 0.0001 BTC. Anyway, I understand that the fees level to include low priority transactions in blocks is some general agreement among users. Indeed, I'm suggesting to raise this level, because I understand, that both Satoshi Dice and any possible spam fall under low priority tx, and it would help to keep them at more acceptable level.

Yes, at first sight the $10,000 buys a 1 GB of blockchain. But this will be transmitted, stored, re-transmitted, downloaded, etc, eventually forever. I think more of some moral harm to the project or loss of momentum, e.g. if such spammers grow the blockchain to 10 GB this year then many full nodes may just decide to stop. But that's just a matter of opinion. I still think there can be guys who may find the network endangering them, and go out and outspend it.

Indeed, non-technical type of users are not supposed to be running the reference client (something in the sense "Bitcoin-Qt - cool; 2 GB of data - wtf"). Somehow I find this unfortunate for the project, but again it's just a matter of opinion.

[racerguy]

I hope I am not wrong here but something about fees may be encoded in the software, like not to transmit low priority transactions with fees under 0.0001 BTC. Anyway, I understand that the fees level to include low priority transactions in blocks is some general agreement among users. Indeed, I'm suggesting to raise this level, because I understand, that both Satoshi Dice and any possible spam fall under low priority tx, and it would help to keep them at more acceptable level.

Yes, at first sight the $10,000 buys a 1 GB of blockchain. But this will be transmitted, stored, re-transmitted, downloaded, etc, eventually forever. I think more of some moral harm to the project or loss of momentum, e.g. if such spammers grow the blockchain to 10 GB this year then many full nodes may just decide to stop. But that's just a matter of opinion. I still think there can be guys who may find the network endangering them, and go out and outspend it.

Indeed, non-technical type of users are not supposed to be running the reference client (something in the sense "Bitcoin-Qt - cool; 2 GB of data - wtf"). Somehow I find this unfortunate for the project, but again it's just a matter of opinion.

I doubt 1gb will be that expensive in the future to store, storage capacity is still growing faster than blockchain size.