Gabi
Legendary
Offline
Activity: 1148
Merit: 1008
If you want to walk on water, get out of the boat
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
September 01, 2011, 04:21:53 PM |
|
Remember that, due to the decentralized nature of Bitcoin, we need as many people as possible with the FULL blockchain, to relay all the blocks and transactions to others
1GB is almost nothing, you can buy 3TB hard disks for cheap... so as long as it doesn't become a real problem, it's better we keep the full blockchain
|
|
|
|
NothinG
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
September 01, 2011, 04:30:08 PM |
|
Remember that, due to the decentralized nature of Bitcoin, we need as many people as possible with the FULL blockchain, to relay all the blocks and transactions to others
1GB is almost nothing, you can buy 3TB hard disks for cheap... so as long as it doesn't become a real problem, it's better we keep the full blockchain
I see your point, and I raise you space detection. ![Smiley](https://bitcointalk.org/Smileys/default/smiley.gif) If their HDD (the one that has Bitcoin Client on it) has enough space that will allow at LEAST 10% left on it, to go ahead and download the full blockchain. Another option would be to ask the user if they would be so kind and help grow the Bitcoin economy by downloading the full blockchain. How about payment monthly for having the full blockchain with a decent upload speed? Nothing HUGE, but something to make them WANT to have all the blocks.
|
|
|
|
MoonShadow
Legendary
Offline
Activity: 1708
Merit: 1010
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
September 01, 2011, 06:35:22 PM |
|
Why not this:
If the client is ran for the first time, only grab the most recent block. And, from there on out...grab the latest blocks. -There is no need for new clients to go back and sift through all the blocks to see if they have a transactions.
BitcoinJ clients do this, because once they are started for the first time, they create their own addresses and have no logical need to assume that such addresses have existed prior to themselves. They download the most recent blocks, and then keep up with the chain, but then I think that they only keep the block headers and discard the data in all blocks that do not relate to coins sent to themselves. It's certainly possible, there just isn't a real need for a regular client that can do this just yet.
|
"The powers of financial capitalism had another far-reaching aim, nothing less than to create a world system of financial control in private hands able to dominate the political system of each country and the economy of the world as a whole. This system was to be controlled in a feudalist fashion by the central banks of the world acting in concert, by secret agreements arrived at in frequent meetings and conferences. The apex of the systems was to be the Bank for International Settlements in Basel, Switzerland, a private bank owned and controlled by the world's central banks which were themselves private corporations. Each central bank...sought to dominate its government by its ability to control Treasury loans, to manipulate foreign exchanges, to influence the level of economic activity in the country, and to influence cooperative politicians by subsequent economic rewards in the business world."
- Carroll Quigley, CFR member, mentor to Bill Clinton, from 'Tragedy And Hope'
|
|
|
MoonShadow
Legendary
Offline
Activity: 1708
Merit: 1010
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
September 01, 2011, 06:45:07 PM |
|
How about payment monthly for having the full blockchain with a decent upload speed? Nothing HUGE, but something to make them WANT to have all the blocks.
There isn't any real need for a large number of nodes that keep a full copy of the blockchain. There probably isn't any real need for any node to keep a full copy of the blockchain unpruned at all. Pruning of transaction data that 1) is older than a certain period of time, say three months or15K blocks or so and 2) has been referenced (spent) and the referencing transaction has been referenced (i.e. the transaction is at least two transactions long spent) will eventually result in a fairly stable blockchain size that mostly varies by transaction volumes over those three months. Some clients won't keep spent transactions unpruned at all, and will thus have a much smaller data footprint, growing only by the size of the block headers; which amounts to about 4 megs per year. That said, some nodes will keep full copies of the blockchain, if only for archival reasons. It's not neccessary that these nodes have high bandwidth or super powerful machines either. I have a VPS that can keep up with the blockchain just fine, that I direct my other client(s) to bootstrap and update from, since I don't keep clients running either on my home machine nor my android phone and I don't want either to announce to the network or to the bootstrapping IRC channel that they exist.
|
"The powers of financial capitalism had another far-reaching aim, nothing less than to create a world system of financial control in private hands able to dominate the political system of each country and the economy of the world as a whole. This system was to be controlled in a feudalist fashion by the central banks of the world acting in concert, by secret agreements arrived at in frequent meetings and conferences. The apex of the systems was to be the Bank for International Settlements in Basel, Switzerland, a private bank owned and controlled by the world's central banks which were themselves private corporations. Each central bank...sought to dominate its government by its ability to control Treasury loans, to manipulate foreign exchanges, to influence the level of economic activity in the country, and to influence cooperative politicians by subsequent economic rewards in the business world."
- Carroll Quigley, CFR member, mentor to Bill Clinton, from 'Tragedy And Hope'
|
|
|
wareen
Millionaire
Legendary
Offline
Activity: 910
Merit: 1001
Revolutionizing Brokerage of Personal Data
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
September 01, 2011, 07:23:56 PM |
|
Data in blkchain isn't very compressible: here's a bzip2 run on it: blk0001.dat: 1.268:1, 6.310 bits/byte, 21.13% saved, 554632957 in, 437434096 out. blkindex.dat: 1.598:1, 5.007 bits/byte, 37.41% saved, 236568576 in, 148074565 out.
For reference -here's the result of an lzma run: blk0001.dat: 32.17% saved blkindex.dat: 57.47% saved
And that's just a "dumb" entropy compressor, so I'd guess in combination with some low-level optimization of the data format we could easily get the size down to about 40-50%.
|
|
|
|
wareen
Millionaire
Legendary
Offline
Activity: 910
Merit: 1001
Revolutionizing Brokerage of Personal Data
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
September 01, 2011, 08:37:39 PM |
|
Yeah ... Not even an order of magnitude ... probably not worth the trouble.
I agree, but then again - with the total blockchain size growing steadily, saving 60% might one day be worth the trouble. Correct me if I'm wrong but I believe that pruning the blockchain of fully spent transactions is "only" expected to yield about 70% savings as well. There is a reason that every node keeps a copy of the blockchain and while we probably can get away with a lot of light clients, I'd rather not see the full history only be kept by large centralized institutions. Having more efficient network protocols and storage formats would be great but there are other more important issues right now IMHO.
|
|
|
|
MoonShadow
Legendary
Offline
Activity: 1708
Merit: 1010
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
September 01, 2011, 09:15:06 PM |
|
Correct me if I'm wrong but I believe that pruning the blockchain of fully spent transactions is "only" expected to yield about 70% savings as well.
That would be true about now, but that is also one reason pruning isn't high on the to-do list. In a future, wherein Bitcoin processes transactions on the scale of Paypal or Visa, the pruning of spent transactions will be a very useful thing. There is a reason that every node keeps a copy of the blockchain and while we probably can get away with a lot of light clients, I'd rather not see the full history only be kept by large centralized institutions.
Yes, there is a reason; and because of that reason, and the sentiment that you have expressed above, not every node is going to prune. Thus, the entire blockchain will continue to exist somewhere on the 'net forever. However, this isn't particularly useful for a node processing transactions in a live production environment. If you truely feel this way, then you can feel free to commit some of your own personal resources to make certain that a node with the complete and unabridged blockchain continues to exist. Having more efficient network protocols and storage formats would be great but there are other more important issues right now IMHO.
Agreed.
|
"The powers of financial capitalism had another far-reaching aim, nothing less than to create a world system of financial control in private hands able to dominate the political system of each country and the economy of the world as a whole. This system was to be controlled in a feudalist fashion by the central banks of the world acting in concert, by secret agreements arrived at in frequent meetings and conferences. The apex of the systems was to be the Bank for International Settlements in Basel, Switzerland, a private bank owned and controlled by the world's central banks which were themselves private corporations. Each central bank...sought to dominate its government by its ability to control Treasury loans, to manipulate foreign exchanges, to influence the level of economic activity in the country, and to influence cooperative politicians by subsequent economic rewards in the business world."
- Carroll Quigley, CFR member, mentor to Bill Clinton, from 'Tragedy And Hope'
|
|
|
kjj
Legendary
Offline
Activity: 1302
Merit: 1026
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
September 02, 2011, 03:45:27 PM |
|
There isn't any real need for a large number of nodes that keep a full copy of the blockchain. There probably isn't any real need for any node to keep a full copy of the blockchain unpruned at all. Pruning of transaction data that 1) is older than a certain period of time, say three months or15K blocks or so and 2) has been referenced (spent) and the referencing transaction has been referenced (i.e. the transaction is at least two transactions long spent) will eventually result in a fairly stable blockchain size that mostly varies by transaction volumes over those three months. Some clients won't keep spent transactions unpruned at all, and will thus have a much smaller data footprint, growing only by the size of the block headers; which amounts to about 4 megs per year.
That said, some nodes will keep full copies of the blockchain, if only for archival reasons. It's not neccessary that these nodes have high bandwidth or super powerful machines either. I have a VPS that can keep up with the blockchain just fine, that I direct my other client(s) to bootstrap and update from, since I don't keep clients running either on my home machine nor my android phone and I don't want either to announce to the network or to the bootstrapping IRC channel that they exist.
The hash of a pruned block will not match the hash in the header, and thus a pruned block cannot be verified. Checkpoints will help this a bit, in that you'll have a verification chain (by a signature on the source code or binary) that says that someone "trustworthy" claims to have verified up to a certain point. But you won't be able to prove it unless you have, or can get, the full chain, but you don't need to have all of it at once.
|
17Np17BSrpnHCZ2pgtiMNnhjnsWJ2TMqq8 I routinely ignore posters with paid advertising in their sigs. You should too.
|
|
|
MoonShadow
Legendary
Offline
Activity: 1708
Merit: 1010
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
September 02, 2011, 05:38:27 PM |
|
There isn't any real need for a large number of nodes that keep a full copy of the blockchain. There probably isn't any real need for any node to keep a full copy of the blockchain unpruned at all. Pruning of transaction data that 1) is older than a certain period of time, say three months or15K blocks or so and 2) has been referenced (spent) and the referencing transaction has been referenced (i.e. the transaction is at least two transactions long spent) will eventually result in a fairly stable blockchain size that mostly varies by transaction volumes over those three months. Some clients won't keep spent transactions unpruned at all, and will thus have a much smaller data footprint, growing only by the size of the block headers; which amounts to about 4 megs per year.
That said, some nodes will keep full copies of the blockchain, if only for archival reasons. It's not neccessary that these nodes have high bandwidth or super powerful machines either. I have a VPS that can keep up with the blockchain just fine, that I direct my other client(s) to bootstrap and update from, since I don't keep clients running either on my home machine nor my android phone and I don't want either to announce to the network or to the bootstrapping IRC channel that they exist.
The hash of a pruned block will not match the hash in the header, and thus a pruned block cannot be verified. It doesn't need to be verified. The client downloads the entire block, verifies it, and then proceeds to prune it to it's own liking. That is what the merkle tree block structure is for.
|
"The powers of financial capitalism had another far-reaching aim, nothing less than to create a world system of financial control in private hands able to dominate the political system of each country and the economy of the world as a whole. This system was to be controlled in a feudalist fashion by the central banks of the world acting in concert, by secret agreements arrived at in frequent meetings and conferences. The apex of the systems was to be the Bank for International Settlements in Basel, Switzerland, a private bank owned and controlled by the world's central banks which were themselves private corporations. Each central bank...sought to dominate its government by its ability to control Treasury loans, to manipulate foreign exchanges, to influence the level of economic activity in the country, and to influence cooperative politicians by subsequent economic rewards in the business world."
- Carroll Quigley, CFR member, mentor to Bill Clinton, from 'Tragedy And Hope'
|
|
|
kjj
Legendary
Offline
Activity: 1302
Merit: 1026
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
September 02, 2011, 06:03:22 PM |
|
The hash of a pruned block will not match the hash in the header, and thus a pruned block cannot be verified. It doesn't need to be verified. The client downloads the entire block, verifies it, and then proceeds to prune it to it's own liking. That is what the merkle tree block structure is for. Right, but that means that the full block must exist somewhere so the node can download it and verify it. So while any node can prune, not every node can prune.
|
17Np17BSrpnHCZ2pgtiMNnhjnsWJ2TMqq8 I routinely ignore posters with paid advertising in their sigs. You should too.
|
|
|
MoonShadow
Legendary
Offline
Activity: 1708
Merit: 1010
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
September 02, 2011, 07:58:24 PM |
|
The hash of a pruned block will not match the hash in the header, and thus a pruned block cannot be verified. It doesn't need to be verified. The client downloads the entire block, verifies it, and then proceeds to prune it to it's own liking. That is what the merkle tree block structure is for. Right, but that means that the full block must exist somewhere so the node can download it and verify it. So while any node can prune, not every node can prune. And that is why it is reasonable to assume that there will always be at least one person willing to run a full node that does not prune. It's also why I said that most full clients won't prune anything recent. As already noted, light clients don't need the full chain anyway, and can start their chain when they create their first address. Most full clients won't really need a full chain either, and can prune quite extensively and/or start from the most recent trusted checkpoint hash. All clients do full chains currently, in part, because the network is still very small compared to the expectations, and the many-copies-keep-data-safe method is employed. The concept of placing a full "quiet" node in orbit, on Earth protected in a safe zone, and eventually on the Moon, as archival devices have been discussed already. For now, there is neither the need nor the resources to do any such things, but if Bitcoin is as successful as some of us predict, those nodes will be cheap insurance; largely inaccessable to most forms of destruction. (one or another, but not all, depending on the kind of destruction being considered)
|
"The powers of financial capitalism had another far-reaching aim, nothing less than to create a world system of financial control in private hands able to dominate the political system of each country and the economy of the world as a whole. This system was to be controlled in a feudalist fashion by the central banks of the world acting in concert, by secret agreements arrived at in frequent meetings and conferences. The apex of the systems was to be the Bank for International Settlements in Basel, Switzerland, a private bank owned and controlled by the world's central banks which were themselves private corporations. Each central bank...sought to dominate its government by its ability to control Treasury loans, to manipulate foreign exchanges, to influence the level of economic activity in the country, and to influence cooperative politicians by subsequent economic rewards in the business world."
- Carroll Quigley, CFR member, mentor to Bill Clinton, from 'Tragedy And Hope'
|
|
|
AssemblY (OP)
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
September 04, 2011, 12:59:47 PM |
|
(...)For now, there is neither the need nor the resources to do any such things, but if Bitcoin is as successful as some of us predict, those nodes will be cheap insurance; largely inaccessable to most forms of destruction. (one or another, but not all, depending on the kind of destruction being considered)
I understand now. I thank everyone for their cooperation and commitment to this topic. Added much information to my knowledge. ![Smiley](https://bitcointalk.org/Smileys/default/smiley.gif)
|
|
|
|
Meatpile
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
September 04, 2011, 03:12:37 PM |
|
As far as I know there are hardcoded checkpoints, in the future they would just make sure you have data after said checkpoint.
However this leaves the system open to a possible conspiracy situation perhaps?
|
|
|
|
phillipsjk
Legendary
Offline
Activity: 1008
Merit: 1001
Let the chips fall where they may.
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
September 04, 2011, 08:47:42 PM |
|
There may exist coins minted/created in (split/merging) transactions since before the checkpoint. You can only prune coin obliterated by a split/merge.
|
James' OpenPGP public key fingerprint: EB14 9E5B F80C 1F2D 3EBE 0A2F B3DE 81FF 7B9D 5160
|
|
|
|