Bitcoin Forum

Bitcoin => Development & Technical Discussion => Topic started by: bitcoinex on December 10, 2010, 09:02:55 PM



Title: infinite increasing of the chain size
Post by: bitcoinex on December 10, 2010, 09:02:55 PM
Tell me, me one why bothers infinite (!) increasing of the chain size?


Title: Re: infinite increasing of the chain size
Post by: MoonShadow on December 10, 2010, 09:33:47 PM
Tell me, me one why bothers infinite (!) increasing of the chain size?

This sentence doesn't parse in English.


Title: Re: infinite increasing of the chain size
Post by: davout on December 10, 2010, 09:35:29 PM
Tell me, me one why bothers infinite (!) increasing of the chain size?

This sentence doesn't parse in English.

It does parse, the AST just looks a little funny


Title: Re: infinite increasing of the chain size
Post by: MoonShadow on December 10, 2010, 09:37:45 PM
Tell me, me one why bothers infinite (!) increasing of the chain size?

This sentence doesn't parse in English.

It does parse, the AST just looks a little funny

I think my brain had a kernal panic attack.


Title: Re: infinite increasing of the chain size
Post by: davout on December 10, 2010, 09:53:56 PM
Tell me, me one why bothers infinite (!) increasing of the chain size?

This sentence doesn't parse in English.

It does parse, the AST just looks a little funny

I think my brain had a kernal panic attack.

Consider yourself lucky not to be rolling on the floor slobbering with a blue screen in your head.


Title: Re: infinite increasing of the chain size
Post by: bitcoinex on December 10, 2010, 09:54:06 PM
Paraphrase: who else worried about infinite increasing of the chain?


Title: Re: infinite increasing of the chain size
Post by: MoonShadow on December 10, 2010, 10:19:00 PM
I see.

Why would you be worried about the growth of the blockchain?  It can be pruned of long referenced transactions, it's just not right now.  We wouldn't want it to be right now anyway, because that might hamper the new user 'bootstrapping' issue in the near term.


Title: Re: infinite increasing of the chain size
Post by: davout on December 10, 2010, 10:35:20 PM
Paraphrase: who else worried about infinite increasing of the chain?

Should be fine, I remember reading something about what creighto mentions about compacting merkle trees in satoshi's PDF


Title: Re: infinite increasing of the chain size
Post by: FreeMoney on December 11, 2010, 02:43:22 AM
Paraphrase: who else worried about infinite increasing of the chain?

Should be fine, I remember reading something about what creighto mentions about compacting merkle trees in satoshi's PDF

This will make it grow more slowly. Is there some notion of how much more slowly?


Title: Re: infinite increasing of the chain size
Post by: MoonShadow on December 11, 2010, 03:10:32 AM
Paraphrase: who else worried about infinite increasing of the chain?

Should be fine, I remember reading something about what creighto mentions about compacting merkle trees in satoshi's PDF

This will make it grow more slowly. Is there some notion of how much more slowly?

(80 bytes per block * blockchain total length) + (Average transaction size * current bitcoins in circulation * maximum allowable division * number of transactions deep kept by the pruning algorithim) = realistic top limit on blockchain size at any given time.


Title: Re: infinite increasing of the chain size
Post by: lfm on December 11, 2010, 09:21:35 AM
Paraphrase: who else worried about infinite increasing of the chain?

Well, it IS NOT infinite! It is continuously increasing. It is, and will continue to be, quite finite and seems manageable for the foreseeable future as far as I can tell. Casandras to the contrary.

If you don't think you can handle the load of the chain on your disk space or think the network load is too much for your system you can always fall back to some service like mybitcoin.com or a mtgox account to store your BTC for you.


Title: Re: infinite increasing of the chain size
Post by: bitcoinex on December 11, 2010, 10:40:09 AM
Paraphrase: who else worried about infinite increasing of the chain?

Well, it IS NOT infinite! It is continuously increasing. It is, and will continue to be, quite finite and seems manageable for the foreseeable future as far as I can tell. Casandras to the contrary.

If you don't think you can handle the load of the chain on your disk space or think the network load is too much for your system you can always fall back to some service like mybitcoin.com or a mtgox account to store your BTC for you.


This is demagogy.


Title: Re: infinite increasing of the chain size
Post by: FreeMoney on December 11, 2010, 12:16:09 PM
Paraphrase: who else worried about infinite increasing of the chain?

Well, it IS NOT infinite! It is continuously increasing. It is, and will continue to be, quite finite and seems manageable for the foreseeable future as far as I can tell. Casandras to the contrary.

If you don't think you can handle the load of the chain on your disk space or think the network load is too much for your system you can always fall back to some service like mybitcoin.com or a mtgox account to store your BTC for you.


This is demagogy.

There might be a language issue here. No amount of data will ever be infinite, maybe you mean it will grow indefinitely.

Lfm is not wrong.


Title: Re: infinite increasing of the chain size
Post by: FreeMoney on December 11, 2010, 12:17:53 PM
Paraphrase: who else worried about infinite increasing of the chain?

Should be fine, I remember reading something about what creighto mentions about compacting merkle trees in satoshi's PDF

This will make it grow more slowly. Is there some notion of how much more slowly?

(80 bytes per block * blockchain total length) + (Average transaction size * current bitcoins in circulation * maximum allowable division * number of transactions deep kept by the pruning algorithim) = realistic top limit on blockchain size at any given time.

Yeah, I was just curious about the size of the benefit of the pruning algorithm. It's probably complicated, I know.


Title: Re: infinite increasing of the chain size
Post by: bitcoinex on December 11, 2010, 03:11:37 PM
Paraphrase: who else worried about infinite increasing of the chain?

Well, it IS NOT infinite! It is continuously increasing. It is, and will continue to be, quite finite and seems manageable for the foreseeable future as far as I can tell. Casandras to the contrary.

If you don't think you can handle the load of the chain on your disk space or think the network load is too much for your system you can always fall back to some service like mybitcoin.com or a mtgox account to store your BTC for you.


This is demagogy.

There might be a language issue here. No amount of data will ever be infinite, maybe you mean it will grow indefinitely.

Yeah, maybe I have a problem with the language.

I am concerned about the growth of wallet.dat


Title: Re: infinite increasing of the chain size
Post by: FreeMoney on December 11, 2010, 07:54:00 PM
Paraphrase: who else worried about infinite increasing of the chain?

Well, it IS NOT infinite! It is continuously increasing. It is, and will continue to be, quite finite and seems manageable for the foreseeable future as far as I can tell. Casandras to the contrary.

If you don't think you can handle the load of the chain on your disk space or think the network load is too much for your system you can always fall back to some service like mybitcoin.com or a mtgox account to store your BTC for you.


This is demagogy.

There might be a language issue here. No amount of data will ever be infinite, maybe you mean it will grow indefinitely.

Yeah, maybe I have a problem with the language.

I am concerned about the growth of wallet.dat

Oh, I really hadn't guessed that. I thought you were meaning the block chain. There is no automation for cleaning the wallet for it now, but it's fairly trivial. Send all coins to another computer, your own, mtgox, whatever then delete your wallet, open Bitcoin and it will make a new wallet. You will lose access to coins sent to your old addresses in the future. I imagine this is why bitcoinmarket made that announcement about deposit addresses changing a while back.


Title: Re: infinite increasing of the chain size
Post by: davout on December 11, 2010, 08:11:36 PM
yea, just zip your discarded wallet and check it back in ten years, see if something arrived by chance =)


Title: Re: infinite increasing of the chain size
Post by: MoonShadow on December 11, 2010, 11:48:53 PM
I am concerned about the growth of wallet.dat

I'm sorry, I thought you were speaking of the blockchain.  My wallet.dat is less than 100Kb, and I have it backed up to a (relatively tiny) 128Mb USB thumbdrive.  If your wallet.dat has grown too large, you could just start a new client with a fresh wallet.dat, and keep a backup or two of the old wallet.dat in case someone ends up sending coins to your old addresses.



Title: Re: infinite increasing of the chain size
Post by: bitcoinex on December 12, 2010, 10:48:00 AM
wallet.dat?! :) why I wrote this?

I meant the file that stores the chain of blocks, of course.

It is unclear how we will look into the eyes of users in a year


Title: Re: infinite increasing of the chain size
Post by: theymos on December 12, 2010, 11:19:53 AM
Only generators need to store the entire block chain. Non-generators only need to store the tiny block headers. This is already mostly implemented (see fClient in the source).


Title: Re: infinite increasing of the chain size
Post by: ShadowOfHarbringer on December 13, 2010, 02:43:00 AM
So how much data does the chain contain right now ?


Title: Re: infinite increasing of the chain size
Post by: theymos on December 13, 2010, 03:04:14 AM
So how much data does the chain contain right now ?

- 97,289 blocks
- 56,806,614 bytes
- 206,775 transactions
- 248,937 outputs
- 278,362 inputs
- 162,871 addresses


Title: Re: infinite increasing of the chain size
Post by: ShadowOfHarbringer on December 13, 2010, 03:50:33 AM
So how much data does the chain contain right now ?

- 97,289 blocks
- 56,806,614 bytes
- 206,775 transactions
- 248,937 outputs
- 278,362 inputs
- 162,871 addresses

The "bytes" were what i had in mind, thank you.


Title: Re: infinite increasing of the chain size
Post by: ribuck on December 13, 2010, 11:55:43 AM
The average of 275 bytes per transaction is interesting, I think.


Title: Re: infinite increasing of the chain size
Post by: lfm on December 13, 2010, 12:24:53 PM
So how much data does the chain contain right now ?

- 97,289 blocks
- 56,806,614 bytes
- 206,775 transactions
- 248,937 outputs
- 278,362 inputs
- 162,871 addresses

When I can buy a million megabytes (1tb) for $50 (on sale), I fail to see why I should worry about 56 megabytes or twice that much or 10 times that much. It is a non issue.


Title: Re: infinite increasing of the chain size
Post by: Timo Y on December 13, 2010, 01:13:47 PM
There are a lot of examples of "infinitely increasing" databases where nothing ever gets erased.

One is the US constitution, where new amendments are appended to the body of the text without removing what already exists.  Yet, the US constitution hasn't "run out of paper" in over 200 years.

Another is Wikipedia, where every edit is recorded for all eternity.

"Infitinitely increasing" is clearly not an issue. It's the rate of increase that matters.


Title: Re: infinite increasing of the chain size
Post by: FreeMoney on December 13, 2010, 01:36:46 PM
There are a lot of examples of "infinitely increasing" databases where nothing ever gets erased.

One is the US constitution, where new amendments are appended to the body of the text without removing what already exists.  Yet, the US constitution hasn't "run out of paper" in over 200 years.

Another is Wikipedia, where every edit is recorded for all eternity.

"Infitinitely increasing" is clearly not an issue. It's the rate of increase that matters.

Haha, I like this.


Title: Re: infinite increasing of the chain size
Post by: Anonymous on December 13, 2010, 10:51:21 PM
Its not the size that matters?

 :-X


Title: Re: infinite increasing of the chain size
Post by: Anonymous on December 13, 2010, 10:52:13 PM
That's what she said.  :-X


Title: Re: infinite increasing of the chain size
Post by: ribuck on December 14, 2010, 11:00:31 AM
Its not the size that matters?

Yeah. It's the rate of increase that counts. That's what she said anyway.


Title: Re: infinite increasing of the chain size
Post by: casascius on December 14, 2010, 04:17:14 PM
Is there any reason why every certain number of blocks, that everyone's balance could not just be calculated and written as a huge new "genesis block" heap that simply equates Bitcoin addresses with their balances, and then all the detail from all prior blocks could be truncated?  Just because we can compare Bitcoin growth to the US Constitution doesn't mean all of its adopters will have the resources or the desire to store an ever-exploding block chain.  The way I understand it, you need the whole block chain not just to be a generator, but to independently verify the validity of your bitcoins cryptographically, and that can be tedious if a cell phone in 2030 must trace each cent through countless micropayments back to the originating block.

Example, after a million blocks, block 1000001 could be a block that replaces all blocks 2 thru 1000000, so one may only need a block chain of 1 -> 1000001 -> 1000002 etc... until two million:  1 -> 1000001 -> 2000001 -> 2000002 etc...

Not only would this save space for people not wanting to dedicate an ever-increasing amount of resources at a difficult-to-sustain rate, but it would also reduce the computational resources needed to validate balances, which would probably grow in an O(n^2) manner relative to the block chain itself.

If this is not sensible or beneficial due to my misunderstanding of the way it works, I'd welcome being corrected.


Title: Re: infinite increasing of the chain size
Post by: MoonShadow on December 14, 2010, 07:33:41 PM
Is there any reason why every certain number of blocks, that everyone's balance could not just be calculated and written as a huge new "genesis block" heap that simply equates Bitcoin addresses with their balances, and then all the detail from all prior blocks could be truncated?


Yes, there is a reason that is not done in the standard clients, and running balance sheets have been proposed, but it's not neccessary in order to keep the local blockchain footprint down.  The blockchain can be 'pruned' of old transaction records that have been already referenced (spent) in "lightweight" clients such as cell phones, or even full clients once set up.  There will likely always be a complete copy of the blockchain available somewhere on the Internet, but only 'paranoid' clients need the entire thing and then only upon initial startup.  Once they have verified the entire blockchain, even the most paranoid client can prune years of records off the chain and need only keep the 80 byte long headers.

Quote

The way I understand it, you need the whole block chain not just to be a generator, but to independently verify the validity of your bitcoins cryptographically, and that can be tedious if a cell phone in 2030 must trace each cent through countless micropayments back to the originating block.


You understand it incorrectly.  Don't take this personally, as it's a very complicated system, and takes a while for even the geekiest among us to wrap their heads around.  I was here for weeks before I really understood it.

And a 'lightweight' client can verify transactions independently without the full chain, but it's more than possible to verify transactions without a local copy of the chain at all if your client is connected to the Internet at the time.  There is a number of ways that this can be done, but the current Android client functions as  a frontend to a remote bitcoind running on a server.  This bitcoind server is functioning as a "trusted node", in this case because both the server and client are owned by the same user; but it is reasonable that "trusted nodes" will be a common thing in a Bitcoin future, functioning as a type of bank and/or credit card company as a service.  The trusted nodes keep the blockchain, and verify all user transactions.

Quote

Example, after a million blocks, block 1000001 could be a block that replaces all blocks 2 thru 1000000, so one may only need a block chain of 1 -> 1000001 -> 1000002 etc... until two million:  1 -> 1000001 -> 2000001 -> 2000002 etc...


Feel free to write such a client.  If there are technical issues with such a shortcut, I'm sure that you will figure it out.

Quote
Not only would this save space for people not wanting to dedicate an ever-increasing amount of resources at a difficult-to-sustain rate, but it would also reduce the computational resources needed to validate balances, which would probably grow in an O(n^2) manner relative to the block chain itself.


The resources of verification do not grow exponentially, or even linerily, with the size of the blockchain.

Quote

If this is not sensible or beneficial due to my misunderstanding of the way it works, I'd welcome being corrected.

It might be both, but it's not immediately neccessary to be able to compact the blockchain for any reason; and provisions exist in the protocol to be able to prune the blockchain to significant degrees.  This system is still young, and alternative clients will start to pop up with other ways of doing things when the time is right.


Title: Re: infinite increasing of the chain size
Post by: casascius on December 15, 2010, 07:23:02 PM

Quote
Quote
Example, after a million blocks, block 1000001 could be a block that replaces all blocks 2 thru 1000000, so one may only need a block chain of 1 -> 1000001 -> 1000002 etc... until two million:  1 -> 1000001 -> 2000001 -> 2000002 etc...


Feel free to write such a client.  If there are technical issues with such a shortcut, I'm sure that you will figure it out.

By looking at what's left, I suppose that leads me to figure that keeping the headers (the proof of work for the size of the blockchain, including the hashes) is pretty vital to establishing which is the correct (longest in terms of PoW) blockchain.


Title: Re: infinite increasing of the chain size
Post by: MoonShadow on December 15, 2010, 07:28:04 PM

Quote
Quote
Example, after a million blocks, block 1000001 could be a block that replaces all blocks 2 thru 1000000, so one may only need a block chain of 1 -> 1000001 -> 1000002 etc... until two million:  1 -> 1000001 -> 2000001 -> 2000002 etc...


Feel free to write such a client.  If there are technical issues with such a shortcut, I'm sure that you will figure it out.

By looking at what's left, I suppose that leads me to figure that keeping the headers (the proof of work for the size of the blockchain, including the hashes) is pretty vital to establishing which is the correct (longest in terms of PoW) blockchain.

Yes.  The current system cannot function without the blockchain headers, at a minimum.  That shouldn't discourage you from seeking another solution.