Bitcoin Forum
December 04, 2016, 08:27:07 PM *
News: Latest stable version of Bitcoin Core: 0.13.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: [1]
  Print  
Author Topic: Blockchain growth is the biggest roadblock: use keyblocks  (Read 1162 times)
Bitsky
Hero Member
*****
Offline Offline

Activity: 542


View Profile
June 30, 2011, 10:48:08 AM
 #1

I took a look at the current blockchain growth and used the data from an older backup to do some calculations. The backup is at block 120238, which was created on 2011-04-26. The files blk0001.dat and blkindex.dat together occupy 211MB. Currently, the chain is at block 133998 and here those files need 463MB. So for 13760 blocks, Bitcoin stored another 252MB on disk. That's just for the transactions of roughly two months. If we assume that Bitcoin will not be growing anymore, the chain will increase by about 1.5GB/year. But since Bitcoin does grow, it will be more, perhaps 2GB, perhaps 10GB, depending on how fast it catches on.

So, if someone wants to join Bitcoin in about a year, he is expected to let his computer download and analyze 2-10GB of data? Or about 20GB in two years? At times where even major ISPs consider bandwith caps. For someone on a common DSL line, 10GB can only be downloaded in 24 hours, assuming a constant rate of 120kB/s. Does anybody seriously assume that your everyday John Doe will put up with this? He won't. Users want to make use of a payment system instantly. Telling them to sit and wait will basically push them to alternatives. Of course they can just use an online wallet, but there's always the problem with hacks (see MtGox). Imagine the shitstorm if eg Paypal would be unuseable for days/weeks after a hack. Putting Bitcoin on your smartphone? Forget it. If everybody switches to online wallets, the p2p network gets smaller again, and that will make it easier for a 3rd party to enforce an alternative blockchain. Bitcoins stability relies on the amount of active nodes; when becoming a node isn't simple and quick, the number of nodes will diminish until only the online wallet services run a few because they have the storage and bandwith to deal with the blockchain.

Bitcoin really needs a way to keep the blockchain and traffic at a reasonable size. One step is to purge old transactions and addresses from the chain. To keep bandwith low, something like keyframes (like in videostreams) could be added. The blockchain would contain a keyblock every x blocks which basically is a status-quo snapshot. Then a client would only need to download the latest keyblock and the blocks after that one to be up to date. Those keyblocks would, like transactions, only be valid with x confirmations of the p2p network.

Bounty: Earn up to 68.7 BTC
Like my post? Feel free to drop a tip to 1BitskyZbfR4irjyXDaGAM2wYKQknwX36Y
1480883227
Hero Member
*
Offline Offline

Posts: 1480883227

View Profile Personal Message (Offline)

Ignore
1480883227
Reply with quote  #2

1480883227
Report to moderator
1480883227
Hero Member
*
Offline Offline

Posts: 1480883227

View Profile Personal Message (Offline)

Ignore
1480883227
Reply with quote  #2

1480883227
Report to moderator
There are several different types of Bitcoin clients. Server-assisted clients like blockchain.info rely on centralized servers to do their network verification for them. Although the server can't steal the client's bitcoins directly, it can easily execute double-spending-style attacks against the client.
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1480883227
Hero Member
*
Offline Offline

Posts: 1480883227

View Profile Personal Message (Offline)

Ignore
1480883227
Reply with quote  #2

1480883227
Report to moderator
1480883227
Hero Member
*
Offline Offline

Posts: 1480883227

View Profile Personal Message (Offline)

Ignore
1480883227
Reply with quote  #2

1480883227
Report to moderator
1480883227
Hero Member
*
Offline Offline

Posts: 1480883227

View Profile Personal Message (Offline)

Ignore
1480883227
Reply with quote  #2

1480883227
Report to moderator
mouse
Jr. Member
*
Offline Offline

Activity: 56



View Profile
June 30, 2011, 10:58:21 AM
 #2

this topic has been addressed many times.

Lookup Merkle Tree. Or bitcoin lite. I believe bitcoinj is already implementing this.

Finally, as a last resort, users can just rely on online bitcoin wallet providers.

Any intelligent fool can make things bigger, more complex and more violent. It takes a touch of genius and a lot of courage to move in the opposite direction.
XIU
Member
**
Offline Offline

Activity: 84


View Profile
June 30, 2011, 11:01:38 AM
 #3

this topic has been addressed many times.

Lookup Merkle Tree. Or bitcoin lite. I believe bitcoinj is already implementing this.

Finally, as a last resort, users can just rely on online bitcoin wallet providers.


In short, the official client already does it Wink

1xiuHwHk81j4TRnLuLBMvH2ctqtTsubT6
FreeMoney
Legendary
*
Offline Offline

Activity: 1246


Strength in numbers


View Profile WWW
June 30, 2011, 11:05:14 AM
 #4

You only need your private keys to spend your coins. You only need to make a request to a trusted source of the block chain to learn about payments to you. You only need the full chain if you are solo mining.

Play Bitcoin Poker at sealswithclubs.eu. We're active and open to everyone.
TierNolan
Legendary
*
Offline Offline

Activity: 1036


View Profile
June 30, 2011, 11:14:38 AM
 #5

Bitcoin really needs a way to keep the blockchain and traffic at a reasonable size. One step is to purge old transactions and addresses from the chain. To keep bandwith low, something like keyframes (like in videostreams) could be added. The blockchain would contain a keyblock every x blocks which basically is a status-quo snapshot. Then a client would only need to download the latest keyblock and the blocks after that one to be up to date. Those keyblocks would, like transactions, only be valid with x confirmations of the p2p network.

The original paper already covers this.

You only need 80 bytes per block to confirm a block is part of the chain.  These are called the block headers.

Once you have the headers for the entire chain, you can then ask for the blocks starting with the last block and working back.

To verify a transaction, all you need to know is the block that the coin comes from and confirm that it is still an unspent coin.

To do that, you need all blocks between the head of the chain and the transaction that created the coin.

1LxbG5cKXzTwZg9mjL3gaRE835uNQEteWF
Bitsky
Hero Member
*****
Offline Offline

Activity: 542


View Profile
June 30, 2011, 11:40:31 AM
 #6

Lookup Merkle Tree. Or bitcoin lite. I believe bitcoinj is already implementing this.
A quick search for Bitcoin Lite did not list a download. A faq I found mentioned "in the future".

In short, the official client already does it Wink
I'm using the official client and it downloaded the entire chain. That's what I based my original posting on. Is there any "keep it small" option I missed?

You only need your private keys to spend your coins. You only need to make a request to a trusted source of the block chain to learn about payments to you. You only need the full chain if you are solo mining.
But who provides those trusted sources. I wouldn't trust a source that's not e.g. validated by the bitcoin.org website.

Once you have the headers for the entire chain, you can then ask for the blocks starting with the last block and working back.
How do I keep headers only and discard/reduce the chain itself? Ideally, I'd like to keep by bitcoin folder <100MB.


I want to point out that I'm using the 0.3.20.0 client, because the newer one fails to run on my system (https://forum.bitcoin.org/index.php?topic=24266.msg302002#msg302002). So if the above options are only available in the current client, please let me know.

Bounty: Earn up to 68.7 BTC
Like my post? Feel free to drop a tip to 1BitskyZbfR4irjyXDaGAM2wYKQknwX36Y
XIU
Member
**
Offline Offline

Activity: 84


View Profile
June 30, 2011, 11:54:10 AM
 #7

It does it automatically, the size is not linear, it will only keep the header for old blocks. IIRC it will only grow by a few MBs per year (but it will still keep growing...)

1xiuHwHk81j4TRnLuLBMvH2ctqtTsubT6
TierNolan
Legendary
*
Offline Offline

Activity: 1036


View Profile
June 30, 2011, 11:55:16 AM
 #8

Once you have the headers for the entire chain, you can then ask for the blocks starting with the last block and working back.
How do I keep headers only and discard/reduce the chain itself? Ideally, I'd like to keep by bitcoin folder <100MB.

By "you can", I mean that it is possible to do in principle.

The current client does not do it this way (as you saw).

Bitcoinj only downloads the headers and assumes that it is being used to connect to a trusted node.

You only need your private keys to spend your coins. You only need to make a request to a trusted source of the block chain to learn about payments to you. You only need the full chain if you are solo mining.
But who provides those trusted sources. I wouldn't trust a source that's not e.g. validated by the bitcoin.org website.

If you are connected to 100 nodes and 90 of them are dishonest/corrupted, you can verify that the chain that the 10 honest nodes sent is the correct chain (since it is longer than any of the chains sent by the dishonest ones).

This gets you the main chain, even in the presence of mostly dishonest other nodes.

If a node sends you a block, you can verify that it is part of the main chain by checking the hashes.

This means that if someone sends you the block that contains a payment to you, you can confirm that the block is part of the chain and that it contains a payment to you.  After an hour, it will be 6 blocks deep and this is considered confirmed.

If the block hasn't been incorporated into the chain, then you need to make a call on how much you trust the other nodes.

Payments by you can also be verified to have gone through by checking the chain.  If the payment is in the chain, then it has been accepted.

The difficulty is combining fast payments with double spending protection.  Nodes are supposed to reject transactions which double spend.  If this works, once one of the 2 transactions will propagate.  If most nodes are dishonest, this could be corrupted and both transactions could be propagated. 

However, this will be corrected within the hour as only 1 transactions will become part of the main chain.

1LxbG5cKXzTwZg9mjL3gaRE835uNQEteWF
FreeMoney
Legendary
*
Offline Offline

Activity: 1246


Strength in numbers


View Profile WWW
June 30, 2011, 12:55:10 PM
 #9


But who provides those trusted sources. I wouldn't trust a source that's not e.g. validated by the bitcoin.org website.
 
How do I keep headers only and discard/reduce the chain itself? Ideally, I'd like to keep by bitcoin folder <100MB.

You can make sure the blocks you are given are in the chain from the header.

It doesn't exist yet afaik, but it will before you need it.

This is not the roadblock you are looking for.

Play Bitcoin Poker at sealswithclubs.eu. We're active and open to everyone.
phillipsjk
Legendary
*
Offline Offline

Activity: 1008

Let the chips fall where they may.


View Profile WWW
June 30, 2011, 03:33:46 PM
 #10

This issue is why I think "Real" Internet access and IPv6 support is more important to me than bitcoin at the moment.

Most residential and commercial ISPs not not impose ridiculous bandwidth caps, they also prohibit server hosting. The ISPs (who are in the business of selling TV and phone subscriptions) do not want their customers generating "content". You are supposed to be a good little consumer and get all of you entertainment from the big media companies.

James' OpenPGP public key fingerprint: EB14 9E5B F80C 1F2D 3EBE  0A2F B3DE 81FF 7B9D 5160
Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!