Blockchain growth is the biggest roadblock: use keyblocks

[Bitsky]

I took a look at the current blockchain growth and used the data from an older backup to do some calculations. The backup is at block 120238, which was created on 2011-04-26. The files blk0001.dat and blkindex.dat together occupy 211MB. Currently, the chain is at block 133998 and here those files need 463MB. So for 13760 blocks, Bitcoin stored another 252MB on disk. That's just for the transactions of roughly two months. If we assume that Bitcoin will not be growing anymore, the chain will increase by about 1.5GB/year. But since Bitcoin does grow, it will be more, perhaps 2GB, perhaps 10GB, depending on how fast it catches on.

So, if someone wants to join Bitcoin in about a year, he is expected to let his computer download and analyze 2-10GB of data? Or about 20GB in two years? At times where even major ISPs consider bandwith caps. For someone on a common DSL line, 10GB can only be downloaded in 24 hours, assuming a constant rate of 120kB/s. Does anybody seriously assume that your everyday John Doe will put up with this? He won't. Users want to make use of a payment system instantly. Telling them to sit and wait will basically push them to alternatives. Of course they can just use an online wallet, but there's always the problem with hacks (see MtGox). Imagine the shitstorm if eg Paypal would be unuseable for days/weeks after a hack. Putting Bitcoin on your smartphone? Forget it. If everybody switches to online wallets, the p2p network gets smaller again, and that will make it easier for a 3rd party to enforce an alternative blockchain. Bitcoins stability relies on the amount of active nodes; when becoming a node isn't simple and quick, the number of nodes will diminish until only the online wallet services run a few because they have the storage and bandwith to deal with the blockchain.

Bitcoin really needs a way to keep the blockchain and traffic at a reasonable size. One step is to purge old transactions and addresses from the chain. To keep bandwith low, something like keyframes (like in videostreams) could be added. The blockchain would contain a keyblock every x blocks which basically is a status-quo snapshot. Then a client would only need to download the latest keyblock and the blocks after that one to be up to date. Those keyblocks would, like transactions, only be valid with x confirmations of the p2p network.

[mouse]

this topic has been addressed many times.

Lookup Merkle Tree. Or bitcoin lite. I believe bitcoinj is already implementing this.

Finally, as a last resort, users can just rely on online bitcoin wallet providers.

[XIU]

this topic has been addressed many times.

Lookup Merkle Tree. Or bitcoin lite. I believe bitcoinj is already implementing this.

Finally, as a last resort, users can just rely on online bitcoin wallet providers.

In short, the official client already does it

[FreeMoney]

You only need your private keys to spend your coins. You only need to make a request to a trusted source of the block chain to learn about payments to you. You only need the full chain if you are solo mining.

[TierNolan]

Bitcoin really needs a way to keep the blockchain and traffic at a reasonable size. One step is to purge old transactions and addresses from the chain. To keep bandwith low, something like keyframes (like in videostreams) could be added. The blockchain would contain a keyblock every x blocks which basically is a status-quo snapshot. Then a client would only need to download the latest keyblock and the blocks after that one to be up to date. Those keyblocks would, like transactions, only be valid with x confirmations of the p2p network.

The original paper already covers this.

You only need 80 bytes per block to confirm a block is part of the chain.  These are called the block headers.

Once you have the headers for the entire chain, you can then ask for the blocks starting with the last block and working back.

To verify a transaction, all you need to know is the block that the coin comes from and confirm that it is still an unspent coin.

To do that, you need all blocks between the head of the chain and the transaction that created the coin.

[Bitsky]

Lookup Merkle Tree. Or bitcoin lite. I believe bitcoinj is already implementing this.

A quick search for Bitcoin Lite did not list a download. A faq I found mentioned "in the future".

In short, the official client already does it

I'm using the official client and it downloaded the entire chain. That's what I based my original posting on. Is there any "keep it small" option I missed?

You only need your private keys to spend your coins. You only need to make a request to a trusted source of the block chain to learn about payments to you. You only need the full chain if you are solo mining.

But who provides those trusted sources. I wouldn't trust a source that's not e.g. validated by the bitcoin.org website.

Once you have the headers for the entire chain, you can then ask for the blocks starting with the last block and working back.

How do I keep headers only and discard/reduce the chain itself? Ideally, I'd like to keep by bitcoin folder <100MB.


I want to point out that I'm using the 0.3.20.0 client, because the newer one fails to run on my system (https://forum.bitcoin.org/index.php?topic=24266.msg302002#msg302002). So if the above options are only available in the current client, please let me know.

[XIU]

It does it automatically, the size is not linear, it will only keep the header for old blocks. IIRC it will only grow by a few MBs per year (but it will still keep growing...)

[TierNolan]

Once you have the headers for the entire chain, you can then ask for the blocks starting with the last block and working back.

How do I keep headers only and discard/reduce the chain itself? Ideally, I'd like to keep by bitcoin folder <100MB.

By "you can", I mean that it is possible to do in principle.

The current client does not do it this way (as you saw).

Bitcoinj only downloads the headers and assumes that it is being used to connect to a trusted node.

You only need your private keys to spend your coins. You only need to make a request to a trusted source of the block chain to learn about payments to you. You only need the full chain if you are solo mining.

But who provides those trusted sources. I wouldn't trust a source that's not e.g. validated by the bitcoin.org website.

If you are connected to 100 nodes and 90 of them are dishonest/corrupted, you can verify that the chain that the 10 honest nodes sent is the correct chain (since it is longer than any of the chains sent by the dishonest ones).

This gets you the main chain, even in the presence of mostly dishonest other nodes.

If a node sends you a block, you can verify that it is part of the main chain by checking the hashes.

This means that if someone sends you the block that contains a payment to you, you can confirm that the block is part of the chain and that it contains a payment to you.  After an hour, it will be 6 blocks deep and this is considered confirmed.

If the block hasn't been incorporated into the chain, then you need to make a call on how much you trust the other nodes.

Payments by you can also be verified to have gone through by checking the chain.  If the payment is in the chain, then it has been accepted.

The difficulty is combining fast payments with double spending protection.  Nodes are supposed to reject transactions which double spend.  If this works, once one of the 2 transactions will propagate.  If most nodes are dishonest, this could be corrupted and both transactions could be propagated. 

However, this will be corrected within the hour as only 1 transactions will become part of the main chain.

[FreeMoney]

But who provides those trusted sources. I wouldn't trust a source that's not e.g. validated by the bitcoin.org website.
 
How do I keep headers only and discard/reduce the chain itself? Ideally, I'd like to keep by bitcoin folder <100MB.

You can make sure the blocks you are given are in the chain from the header.

It doesn't exist yet afaik, but it will before you need it.

This is not the roadblock you are looking for.

[phillipsjk]

This issue is why I think "Real" Internet access and IPv6 support is more important to me than bitcoin at the moment.

Most residential and commercial ISPs not not impose ridiculous bandwidth caps, they also prohibit server hosting. The ISPs (who are in the business of selling TV and phone subscriptions) do not want their customers generating "content". You are supposed to be a good little consumer and get all of you entertainment from the big media companies.