what are they going to do when the bitcoin chain becomes prohibitively large?

[bitfreak!]

Proposal: a distributed database, Each Node keeps track of random blocks of the blockchain
Each node should probably keep at least 0,1% of the blockchain to prevent data corruption, data loss and double spending.

There are multiple problems with this idea. First of all it doesn't solve anything in the long term because eventually people will be asking "what happens when 0.1% of the blockchain becomes prohibitively large". Secondly, nodes require most of the blockchain in order to verify transactions and calculate address balances. If they only held a small portion of the blockchain then every time they needed to verify a transaction or make a transaction they would need to search around for other nodes which had the data they needed, and that would dramatically increase the amount of network traffic. It's just a totally impracticable way of going about things.

[gollum]

Proposal: a distributed database, Each Node keeps track of random blocks of the blockchain
Each node should probably keep at least 0,1% of the blockchain to prevent data corruption, data loss and double spending.

There are multiple problems with this idea. First of all it doesn't solve anything in the long term because eventually people will be asking "what happens when 0.1% of the blockchain becomes prohibitively large". Secondly, nodes require most of the blockchain in order to verify transactions and calculate address balances. If they only held a small portion of the blockchain then every time they needed to verify a transaction or make a transaction they would need to search around for other nodes which had the data they needed, and that would dramatically increase the amount of network traffic. It's just a totally impracticable way of going about things.

Don't worry it will be many years before 0,1% of the blockchain is too big.

Facebook uses distributed databases on hadoop (hdfs) which is 21 PB (PetaByte) that's 1 million times bigger than the size of our beloved blockchain.
http://hadoopblog.blogspot.se/2010/05/facebook-has-worlds-largest-hadoop.html

It's of course two completely different use cases, but as you see such architectures can handle huge datasets.
http://hbase.apache.org/

Here is another thread suggesting a similar solution:
Bitcoin addon: Distributed block chain storage
https://bitcointalk.org/index.php?topic=197810.0

[MysteryMiner]

As I'm writing his message I am running full bitcoin node on 12 years old Pentium4 computer that was made in year 2002. It might not be fast but it works. I think that any Core2Duo or AMD FX based desktop computer with 1TB+ hard drive will be able to run full node for at least next 10 years. Long after these machines will become useless for running latest computer games or watching hyper-HD videos.

If not, use SPV clients like Electrum or Multibit. And at some point Bitcoin-Qt will have pruning. The basics to do this already are in the design and code. The blockchain size is not a problem as I see it.

[bitfreak!]

Don't worry it will be many years before 0,1% of the blockchain is too big

Lol... how long it takes is irrelevant, unless it's longer than the lifetime of the sun.

Facebook uses distributed databases on hadoop (hdfs) which is 21 PB (PetaByte) that's 1 million times bigger than the size of our beloved blockchain.

Typically the best way to store large amounts of data is on a distributed file system where each system stores a small part of the whole thing. But that is not true for bitcoin because of the reasons I already mentioned, mainly that it would require a prohibitive amount of network bandwidth to transmit the same data over and over again. It is also a bad idea because it opens the possibility for crucial data to be lost. Unless every node has a copy of the same data then all that needs to happen to lose parts of the blockchain is that those nodes holding the data leave the network and never return. That is not an acceptable situation because it could wipe out the balances of coin holders and break the network.

[MysteryMiner]

Bitcoin will become specialized. Miners (pool operators) need to have a full blockchain. Miners already have specialized hardware that is expensive. Large storage devices to complement the mining are cheap compared to ASICs. I still see no problem.

[infoman]

I was wondering about an other angle actually, the download speed is not going to get significantly faster to catch up with the size growth.
Average Joes will not bother to download something say 200-300Gb, even if it gets bigger. Even mining enthusiast will give up (not join mining any more)

so, at the end we could end up with a few centralized servers running the blockchain (when it is the size of 10Tb+ ) and this could cause all sort of other issues.

what is your view on this?

[tvbcof]

The blockchain will never become to large...for Google, Facebook, etc.  They can 'peer' with one another so we still have the much hyped 'p2p' system.

At that point we can just hook up with our clients as planned.  The cool thing is that transaction fees will likely be out the window since the value of the user data harvested will far exceed the cost of operation.

Not only that, but the these companies can handle all the laborious work of keeping track of real-world identities, bio-metric authentication, honoring blacklists, etc.  Hell, they'll probably get things so nailed down that they'll fund charge-backs out of their own profits as does VISA when my credit card gets mysteriously used to bye gasoline all over Northern California.  No more losing BTC by being an idiot about security!

What's not to love?!?

[infoman]

I can only say amen to this, (and I am not even religious)

[gollum]

Downloading all blockchain means less decentralization as it grows bigger which is against the ideal of bitcoin.
Besides that it's an ugly architecture. Statistically you would be fine with a fraction of the data.

It's convenient to not touch the current design of the blockchain, but I'm sure there are better technical solutions possible:

Distributed Hash Table, example: freenet, bittorrent
http://en.wikipedia.org/wiki/Distributed_hash_table

Graph database, example: neo4j, titan
http://en.wikipedia.org/wiki/Graph_database

Distributed data store, example: Voldemort, cassandra
http://en.wikipedia.org/wiki/Distributed_data_store

[Amitabh S]

Oh no.. I did some research and we are all screwed! 

[bitfreak!]

but I'm sure there are better technical solutions possible

The concepts you presented are not solutions necessarily applicable to cryptocurrency. You need to understand the way the blockchain works before you can begin suggesting alternative storage mechanisms. If you want to read a viable proposal then read my post on the first page of this thread and follow the link.

[franky1]

I was wondering about an other angle actually, the download speed is not going to get significantly faster to catch up with the size growth.
Average Joes will not bother to download something say 200-300Gb, even if it gets bigger. Even mining enthusiast will give up (not join mining any more)

so, at the end we could end up with a few centralized servers running the blockchain (when it is the size of 10Tb+ ) and this could cause all sort of other issues.

what is your view on this?

you say download speeds when blockchain is 200-300GB..
ok rebuttle time.
at the moment the transactions are no where near the 52gb yearly potential so a 4-6 year timescale is a minimum, im thinking maybe 8 years before the blockchain gets to that size.

so 8 years ago alot of people were on dial-up, and only just getting into ADSL at maybe half a mb/s... now we are a majority on ADSL ranging 2mb in the country, 8-12mb in most industrial towns and cities and upto 30mb in singapore..

so lets look 8 years into the future. .... hmmm it looks pretty fast to me, enough to cope with 300gb of data.

but with that said the majority of people will use phone apps attached to places like blockchain.info and other clones. but those serious about bitcoin will have no problem downloading the blockchain.

short story: don't worry about something in years to come, it may never happen

[infoman]

all valid and good points, thanks franky1!

[Sonny]

What about this idea?  
"Ultimate blockchain compression w/ trust-free lite nodes"
https://bitcointalk.org/index.php?topic=88208.0

[cbeast]

I will have my blockchain gold-plated and wear it on my bronzed bare chest.