A rolling root to solve Bitcoin's scalability problem?

[Raize]

If you are a major exchange and are making $1B a month in profit (and you would if Bitcoin is so huge it has higher tx volume them VISA) do you think it might be worth it to pay $1,000 a year in additional storage cost to add some drives to the SAN?  I think it might be.

With that much data, what do you think would be a reasonable estimate of the disk I/O?

[DeathAndTaxes]

If you are a major exchange and are making $1B a month in profit (and you would if Bitcoin is so huge it has higher tx volume them VISA) do you think it might be worth it to pay $1,000 a year in additional storage cost to add some drives to the SAN?  I think it might be.

With that much data, what do you think would be a reasonable estimate of the disk I/O?

True VISA scale (i.e 5,000 tps on blockchain)?

Figure 5,000 transactions per second.  Average tx is 400 bytes that is maybe 2MB/s (16 Mbps) sustained to write to the disk.  Average block would be 600 * 5,000 * 400 / 1024^3 = 1.2 GB.  So pretty much nothing.  Now to validate txs requires reading the outputs from the UXTO so maybe double that in terms of random reads.  Still we are talking a tiny fraction of the IO that even high end disks (to say nothing of SANs) can handle today.

Everyone thinks of disk first because it is the most obvious but in my analysis of the resource requirements I see it as the least critical bottleneck for full nodes.  If I had to rank them it would be node bandwidth (especially for residential connections), then memory, then computing power, and way back in last place would be disk capacity and speed.

To give you an example of why memory would be more of an issue, is that latency for UXTO lookups will start to become excessive.  It may be less of an issue for non-miners (who can take 5 to 10 seconds to validate a block) but miners will want to validate potential blocks very rapidly.  To avoid the slow expensive pulls from disk, if possible one would want the entire UXTO and memory pool (list of unconfirmed transactions) in memory.  The only thing being written to disk would be the blocks.  The memory pool shouldn't be a problem even if it grew to 30x the size of the average block we are talking <64GB (which sounds like a lot but consider the doubling effect of Moores law for a decade or two).  The UXTO would be larger however the good news is the UXTO grows more linearly than the blockchain or tx volume.  

However eventually it may not be possible to have the entire UXTO in memory.  What I would consider an interesting project would be to design a probabilistic model which determines how likely an output would be involved in a transaction in the near future.   For example spam outputs which are uneconomical to spend would be less likely to be included and could be dropped from the UXTO in-memory cache (if they are used it would just need pulled from disk).  Other examples of low probability outputs would be "coins" with very old age, outputs to addresses which have a large number of other older outputs but few spends (probably paper or storage wallet).  Some true statistical modeling of the blockchain could provide better insight those were just some examples.  The client could score outputs on the likelihood it would be spent soon and the highest tx kept in the in-memory cache.

I am not convinced Bitcoin (at least on blockchain) will ever become that large but if it does clients will adapt to meet the needs of full nodes.  The protocol currently is rather inefficient in terms of resources (other than disk).  Tx are double verified, block messages include the full transaction (which most nodes already have) there will be a lot of room for optimization as transaction volume grows.

[sugarpuff]

True VISA scale (i.e 5,000 tps on blockchain)?

Figure 5,000 transactions per second.  Average tx is 400 bytes that is maybe 2MB/s (16 Mbps) sustained to write to the disk.  Average block would be 600 * 5,000 * 400 / 1024^3 = 1.2 GB.  So pretty much nothing.  Now to validate txs requires reading the outputs from the UXTO so maybe double that in terms of random reads.  Still we are talking a tiny fraction of the IO that even high end disks (to say nothing of SANs) can handle today.

Everyone thinks of disk first because it is the most obvious but in my analysis of the resource requirements I see it as the least critical bottleneck for full nodes.  If I had to rank them it would be node bandwidth (especially for residential connections), then memory, then computing power, and way back in last place would be disk capacity and speed.

Thanks for that analysis DeathAndTaxes! Very informative.

Also, just an FYI, I edited the OP to add:

Edit2 March 5, 2014: Something not mentioned here is that bitcoind can (and might already in some ways do) prune its locally stored blockchain, but only after it downloaded the entire thing once. Thus, this issue is more of a problem (I think) for new full-nodes, which currently must download the entire blockchain before they can prune the entire thing.

Based on a convo I just had about this topic over on #bitcoin-dev, I'm realizing that I have some knowledge gaps that I need to fill to better understand some of the alternative proposals, but I think others here might find this interesting so I'm sharing it here (with sipa's permission):

1:31:31 PM sipa: sugarpuff: now, about dealing with ways to improve performance, IF you are giving up zero trust
1:31:35 PM sipa: sugarpuff: see it this ways
1:31:47 PM sipa: sugarpuff: internally, every full node maintains a database of all unspent transaction outputs
1:32:13 PM sipa: sugarpuff: this is a high-efficiency, compact database with just outputs (not signatures for example) and not those that have already been spent
1:32:31 PM sipa: sugarpuff: every block you can see as sort of an authenticated patch against that database
1:32:45 PM sipa: (add outputs of trenasactions in the block, remove the outputs spent by transactions in it)
1:33:08 PM sipa: if we can somehow agree on a snapshot of that database at some point in time, and you trust it, we can just copy the database
1:33:13 PM sipa: instead of copying all blocks before it
1:34:01 PM sugarpuff: sipa: right, and that sounds like going into checkpoint territory. the rolling-root proposal is similar but not entirely the same
1:34:27 PM sugarpuff: but old blocks should have almost unanimous agreement in the network
1:34:36 PM sipa: sugarpuff: not done yet
1:35:07 PM sipa: sugarpuff: there are proposals to make such a snapshot of that database essentially for every block, and put a hash of that snapshot in the block itself
1:35:14 PM sipa: sugarpuff: and have this validated by miners too
1:35:43 PM sugarpuff: sipa: ah, yeah, that's a good idea. is that what UXTO is about?
1:35:56 PM sipa: the UTXO set is just what we call that database
1:36:10 PM sugarpuff: ah, lol, i just got the acronym
1:36:11 PM sipa: (Unspent TransaXtion Outputs)
1:36:14 PM sugarpuff:
1:36:29 PM sipa: now, this has a very significant impact on the network
1:36:41 PM sipa: as validating and maintaining these database snapshots is not cheap
1:37:16 PM sipa: (there are ways to make it pretty efficient, by using a merkle tree for that entire database, but the impact is not neglectable)
1:37:48 PM sipa: now, you can have a node that just asks a peer, "give me your current UTXO set", and it can validate that at least the majority of hashing power agrees with it
1:38:02 PM sipa: without needing to downloading any old blocks
1:38:36 PM sugarpuff: sipa: just by asking enough nodes for it and assuming the majority is right?
1:38:56 PM sipa: sugarpuff: no no, not the majority of your peers
1:39:06 PM sipa: sugarpuff: you can actually just validate the entire old blockchain, but only the headers
1:39:14 PM sipa: sugarpuff: which is very cheap (80 bytes per block)
1:39:18 PM sipa: sugarpuff: like SPV nodes do now
1:39:31 PM sipa: sugarpuff: but the headers would be committing to the hash of these UTXO set snapshots
1:39:42 PM sipa: so you can verify that the chain agrees with the snapshot
1:39:50 PM sipa: which implies the majority of miners do
1:40:31 PM sugarpuff: sipa: i'd like to do more reading on this and not use up more of your time, do you have some recommended links about this topic? Think SPV + UTXO thread is a good start?
1:40:54 PM sipa: several mailing list posts, forum posts, ...
1:41:00 PM sugarpuff: too broad...
1:41:05 PM sugarpuff: i could spend a year doing that
1:41:18 PM sugarpuff: i love the bitcoin paper, i've read it like 3.5 times now
1:41:18 PM sipa: there's no nice centralized knowledge repository of all these experimental ideas
1:41:28 PM sugarpuff: ah that's unfortunate
1:42:15 PM sugarpuff: sipa: ok, well thanks, i think what you said here is enough to get me started. i'll go do some more research on it. right now there are too many gaps in my knowledge to give you useful replies on this
1:42:23 PM sipa:
1:42:23 PM sipa: yw

[maaku]

Unless I'm misunderstanding you, I think I addressed that in an earlier reply:

I thought this was obvious, but perhaps not:

New nodes could ask others for the current root then. If nodes lie about it, there are one of two possible scenarios:

- They give a root that is too old. This can be mitigated by asking multiple nodes and picking the majority answer, combined with simply downloading the rest of the blockchain. If it got unlucky and the majority lied, it will eventually catch up and discard the outdated root on its own anyway because of the rolling window that's hard coded.

Downloading the rest of the block chain requires someone storing the entire block chain, no? You can't download something which everyone has deleted.

But besides that you have an even bigger problem: it costs me, or any network attacker approximately nothing to make sure that every single node you connect to is a compromised node I control. This is the problem which bitcoin seeks to solve! And the answer which Satoshi came up with, and the only answer which we have at this time is to validate the entire chain from start to finish and trust the longest valid fork.

- They give a root that's too young. This can be mitigated, again, by asking multiple nodes. If it also gets unlucky with its chosen majority, it should be able to figure this out because it's assumed that not all nodes are evil, and it will eventually stumble across a trustworthy one, which would make it have a longer blockchain than the ones that the lying nodes gave it, and it would automatically adopt that one.

This is an invalid assumption. Your ISP could simply block packets from honest nodes, for example.

[sugarpuff]

Unless I'm misunderstanding you, I think I addressed that in an earlier reply:

I thought this was obvious, but perhaps not:

New nodes could ask others for the current root then. If nodes lie about it, there are one of two possible scenarios:

- They give a root that is too old. This can be mitigated by asking multiple nodes and picking the majority answer, combined with simply downloading the rest of the blockchain. If it got unlucky and the majority lied, it will eventually catch up and discard the outdated root on its own anyway because of the rolling window that's hard coded.

Downloading the rest of the block chain requires someone storing the entire block chain, no? You can't download something which everyone has deleted.

I do not believe that is correct.

The relevant bits of "the entire block chain" are always carried forward by the network and stored.

This proposal is addressing the irrelevant bits, which can include some or all of the following:

- Lost coins (a coin can be declared "lost" by some metric, like coin age > T, where T = 2 years.)
- The histories of coins that were spent and are now living farther down in the blockchain (within the rolling window).
- Any other outdated information.

There's no reason to store the history of a "coin" (in quotes because we're really talking about balances) all the way back to its creation. It can simply have a new, verified "origination point" in the blockchain.

But besides that you have an even bigger problem: it costs me, or any network attacker approximately nothing to make sure that every single node you connect to is a compromised node I control. This is the problem which bitcoin seeks to solve! And the answer which Satoshi came up with, and the only answer which we have at this time is to validate the entire chain from start to finish and trust the longest valid fork.

That answer is, as far as I can tell, identical to what I am proposing. All the implications remain the same. You still end up needing to validate "the longest fork", it's just that forks are trimmed down to a reasonable size.

- They give a root that's too young. This can be mitigated, again, by asking multiple nodes. If it also gets unlucky with its chosen majority, it should be able to figure this out because it's assumed that not all nodes are evil, and it will eventually stumble across a trustworthy one, which would make it have a longer blockchain than the ones that the lying nodes gave it, and it would automatically adopt that one.

This is an invalid assumption. Your ISP could simply block packets from honest nodes, for example.

Yes, and that's a problem that Bitcoin also faces. There's no difference that I can see here.

[kjj]

1:31:31 PM sipa: sugarpuff: now, about dealing with ways to improve performance, IF you are giving up zero trust

I just want to point this out, to make sure that no one overlooked it.

Lots of problems get really, really easy once you give up security.  Trusted systems have wildly different properties than trustless systems.  If you don't mind moving from a trustless to a trusted system, then what you are proposing can work*.

P.S.  Nothing in this thread is new.  Even the posts telling you it won't work are recycled.

* For some values of work.  If bitcoin had been designed as a trusted system (even a distributed-trust system like you are proposing), I'm not sure that they would have worked in the sense of being worth trading 6 cents for, much less $600+.

[sugarpuff]

1:31:31 PM sipa: sugarpuff: now, about dealing with ways to improve performance, IF you are giving up zero trust

I just want to point this out, to make sure that no one overlooked it.

Lots of problems get really, really easy once you give up security.

That is nonsense. Just a false characterization of what is being proposed.

Security is not being given up.

Trusted systems have wildly different properties than trustless systems.  If you don't mind moving from a trustless to a trusted system, then what you are proposing can work*.

And people on the internet often make "wildly" thoughtless comments, like this one here.

This is not a black and white proposal like you are trying to make it out to be.

It is not a suggestion in the slightest to "give up security."

The fact is that ancient blocks, at any given point in time, are already highly trusted by the network. If they weren't, Bitcoin would simply not work.

There is already "trust" in the Bitcoin network, for these ancient blocks.

For established nodes, there would be zero difference.

And for brand new nodes:

1. They would actually exist.

If the "doomsday scenario" pans out, and 14TB is needed every 3 months, there would be virtually no new nodes. Your trustless system would be mostly useless to most people, including, probably, you (unless you have a datacenter in your back yard).

2. It would be not entirely trustless, but "approximately trustless" (like 99.9% trustless, once the entire window has been consumed). Read the rest of that IRC chat. New full-nodes would have security greater than the already very-secure light clients that we have today like Electrum.

3. There would be nothing stopping those nodes, if they really want, from downloading an archive of the no-longer-useful data from the discarded blockchain tail.

Even the posts telling you it won't work are recycled.

So far not a single post has shown that this "won't work". Every objection was answered.

[gmaxwell]

The fact is that ancient blocks, at any given point in time, are already highly trusted by the network. If they weren't, Bitcoin would simply not work.
There is already "trust" in the Bitcoin network, for these ancient block

Every node verifies the history, this is how you can be sure that claims like "satoshi premined a billion bitcoins and is secretly waiting to spend them" cannot be true. We do not trust, and cannot trust— because it would be trust founded on nothing. Without verification there is no reason for unidentifiable, anonymous parties not to lie in favor of their own common interests. What actually makes the Bitcoin state trust worthy is because people verify it instead of trusting it.

I think you should stop using the word trustless while you have multiple bitcoin developers telling you that you do not understand it.

[sugarpuff]

The fact is that ancient blocks, at any given point in time, are already highly trusted by the network. If they weren't, Bitcoin would simply not work.
There is already "trust" in the Bitcoin network, for these ancient block

Every node verifies the history, this is how you can be sure that claims like "satoshi premined a billion bitcoins and is secretly waiting to spend them" cannot be true. We do not trust, and cannot trust— because it would be trust founded on nothing.

Except your own Wiki says the opposite:

Only a small number of archival nodes need to store the full chain going back to the genesis block. These nodes can be used to bootstrap new fully validating nodes from scratch but are otherwise unnecessary.

From: https://en.bitcoin.it/wiki/Scalability#Storage

Notice how nowhere in that section does it say anything close to "Every node verifies the history.", and instead refers to "archival nodes". It also says "Only a small number" of them are needed.

I think you should stop using the word trustless while you have multiple bitcoin developers telling you that you do not understand it.

The way my mind works is that I do not accept BS from anyone. It doesn't matter how large an angry mob gets if what it is saying is complete nonsense.

I've addressed every objection to this proposal, and have received confirmation on it now from two people who seem to know something about Bitcoin: Realpra, and a friend of mine whom I know is intelligent (works on SocialVPN). And while not a vote of recognition, one of the Bitcoin is Vulnerable authors didn't say that the idea wouldn't work when I shared it with him, but said "i'd feel nostalgic if i had to part with the genesis block."

Edit: I called it 99.9% trustless, not 100%.

[gmaxwell]

Except your own Wiki says the opposite:

The wiki is a public document that anyone can edit. The scalability page is almost entirely the work of a single person who has removed views conflicting with his own onto other pages (including my own, for that matter).  You should be a sophisticated consumer of wiki content and not believe everything you read.

I think you should stop using the word trustless while you have multiple bitcoin developers telling you that you do not understand it.

The way my mind works is that I do not accept BS from anyone. It doesn't matter how large an angry mob gets if what it is saying is complete nonsense.
I've addressed every objection to this proposal, and have received confirmation on it now from two people who seem to know something about Bitcoin: Realpra, and a friend of mine whom I know is intelligent (works on SocialVPN).

You haven't really addressed _any_ of the objections here, because apparently you do not understand them. I spent quite a bit of time on IRC trying to help you understand but I have failed you. I'm sorry that I'm unable to help further.

[DannyHamilton]

One of the more talented trolls I've seen in a while.  I actually thought he was trying to learn and understand at first.  It took almost 24 hours before I realized that it was a ruse to bait knowledgeable people into boosting his ego.  Looks like my kill file is getting another entry.

[sugarpuff]

Except your own Wiki says the opposite:

The wiki is a public document that anyone can edit. The scalability page is almost entirely the work of a single person who has removed views conflicting with his own onto other pages (including my own, for that matter).  You should be a sophisticated consumer of wiki content and not believe everything you read.

Yes, I try to be. I also try to be a "sophisticated consumer" of public forum comments.

You haven't really addressed _any_ of the objections here, because apparently you do not understand them. I spent quite a bit of time on IRC trying to help you understand but I have failed you. I'm sorry that I'm unable to help further.

On IRC I agreed that it's not 100% trustless, and frankly you weren't very helpful, just like you're not being helpful here. sipa was the one who actually gave meaningful help and explanations. (None of which, btw, showed that this proposal "lacks security" or anything of the sort.)

So far, this is what has happened here (as far as I can tell):

Me: blah blah blah
Someone: that won't work because of X
Me: That's incorrect because of Y
Realpra: Sugarpuff, your proposal had this other name (indication of agreement).
gmaxwell: "You haven't really addressed _any_ of the objections here"
DannyHamilton: ad hominem attack directed at sugarpuff

OK, if this is how you guys discuss ideas, no thank you, I've had enough. I'll reply to constructive comments that actually address the merits of the proposal only.

[DeathAndTaxes]

The fact is that ancient blocks, at any given point in time, are already highly trusted by the network. If they weren't, Bitcoin would simply not work.
There is already "trust" in the Bitcoin network, for these ancient block

Every node verifies the history, this is how you can be sure that claims like "satoshi premined a billion bitcoins and is secretly waiting to spend them" cannot be true. We do not trust, and cannot trust— because it would be trust founded on nothing.

Except your own Wiki says the opposite:

Only a small number of archival nodes need to store the full chain going back to the genesis block. These nodes can be used to bootstrap new fully validating nodes from scratch but are otherwise unnecessary.

From: https://en.bitcoin.it/wiki/Scalability#Storage

Notice how nowhere in that section does it say anything close to "Every node verifies the history.", and instead refers to "archival nodes". It also says "Only a small number" of them are needed.

I think you should stop using the word trustless while you have multiple bitcoin developers telling you that you do not understand it.

The way my mind works is that I do not accept BS from anyone. It doesn't matter how large an angry mob gets if what it is saying is complete nonsense.

I've addressed every objection to this proposal, and have received confirmation on it now from two people who seem to know something about Bitcoin: Realpra, and a friend of mine whom I know is intelligent (works on SocialVPN). And while not a vote of recognition, one of the Bitcoin is Vulnerable authors didn't say that the idea wouldn't work when I shared it with him, but said "i'd feel nostalgic if i had to part with the genesis block."

Edit: I called it 99.9% trustless, not 100%.

No you just lack the knowledge to understand what you are reading.

As pointed out the wiki is simply public docs.  It is one possible way to deal with scalability.  Anything in the wiki which goes beyond the actual current protocol should be treated as an opinon.

However even if the cited section was implemented it would still be trustless.   Full nodes would still maintain a pruned copy of the blockchain going all the way back to the genesis block.  An unpruned copy (the archive node) is not required to validation transactions or create new blocks.   Nobody needs to trust the archive nodes.  

Your "solution" (and I use the term lightly) would break that trustless relationship between nodes.  

So either you are trolling for trolling sake or you are genuinely interested but lack the basic knowledge.  It would be like someone without any higher mathematics trying to understand how public key cryptography works and coming up with "enhancements".   If you aren't trolling, or just looking to debate to feel important ... then stop trying to fix Bitcoin before you undertstand Bitcoin.  Take a couple weeks starting with the whitepaper, the wiki and then the source code.  

Nobody is saying a ledger type system is impossible but it a non-trivial problem.  If you think you found a trivial solution to a non-trivial problem in a few minutes ... you haven't.

OK, if this is how you guys discuss ideas, no thank you, I've had enough. I'll reply to constructive comments that actually address the merits of the proposal only.

You haven't provided a proposal.

[kjj]

You haven't really addressed _any_ of the objections here, because apparently you do not understand them. I spent quite a bit of time on IRC trying to help you understand but I have failed you. I'm sorry that I'm unable to help further.

On IRC I agreed that it's not 100% trustless, and frankly you weren't very helpful, just like you're not being helpful here. sipa was the one who actually gave meaningful help and explanations. (None of which, btw, showed that this proposal "lacks security" or anything of the sort.)

So far, this is what has happened here (as far as I can tell):

Me: blah blah blah
Someone: that won't work because of X
Me: That's incorrect because of Y
Realpra: Sugarpuff, your proposal had this other name (indication of agreement).
gmaxwell: "You haven't really addressed _any_ of the objections here"
DannyHamilton: ad hominem attack directed at sugarpuff

OK, if this is how you guys discuss ideas, no thank you, I've had enough. I'll reply to constructive comments that actually address the merits of the proposal only.

We don't need to show shit.  You need to show that your proposal has merit.  We've tried to suggest that you'd do well to find the extensive literature on this exact idea, but you've refused to do that basic research*.

As I said on IRC:

I really don't understand why you are discussing this.  Clearly you've come up with an idea that no one ever has before.  Now you just need to code it up and the whole world will switch to your fork.

Option 1 is that you had a new and amazing insight that no one has ever had before*, and the world is waiting to beat a path to your door.
Option 2 is that other people have also thought of it, and yet it hasn't happened, quite possibly because it has flaws that you should acquaint yourself with*.

* This has come up many times in the last 3 years (I've personally seen it).  It probably came up a few times in the two years before then.  I bet that if you look closely, you'll find the basic outline of the general problem come up on crypto mailing lists back into the 90s.

[sugarpuff]

DeathAndTaxes, replies to your constructive comments are inlined below:

However even if the cited section was implemented it would still be trustless.   Full nodes would still maintain a pruned copy of the blockchain going all the way back to the genesis block.  An unpruned copy (the archive node) is not required to validation transactions or create new blocks.   Nobody needs to trust the archive nodes.  

Thank you, I understand that but it's helpful to emphasize that.

Your "solution" (and I use the term lightly) would break that trustless relationship between nodes.

How?

You can state that as much as you want, but without a "because ..." it's a "just-so comment" without a leg to stand on.

The relationship indeed becomes no longer 100% trustless as I've acknowledged multiple times.

But for the sake of a tiny increase in trust, you get a network where new full nodes can be brought online without having to download several hundred terabytes of data. EDIT: that still retain their ability to verify all the way back to the original root, on their own time.

If Bitcoin gets to that point, you will come back to this proposal, and you will see it in a different light all of a sudden.

You see, as I've mentioned, I have read the Bitcoin paper. Several times. I understand it well.

The Bitcoin paper points out this interesting property of the blockchain: that the longer it gets, the harder it gets to fake. Impossibly harder.

You don't need to download a blockchain that is 200TB in size to become pretty darn sure that what you're seeing is the Real McCoy. 1TB will probably do just fine (depending on the volume of transactions of course).

So, if you're going to reply again (and you want a response from me), please reply to one of the points that I've made, and avoid doing any of the following:

- Calling me a troll (because I am not trolling) or resorting to other forms of ad hominem
- Making "just-so" statements like I "don't understand", without showing what I am misunderstanding
- Telling me to go read the paper that I have read several times.
- Telling me to go read the wiki. I have done that. I've even quoted it here recently.
- Lying.

[sugarpuff]

you've refused to do that basic research*.

False.

[DeathAndTaxes]

But for the sake of a tiny increase in trust, you get a network where new full nodes can be brought online without having to download several hundred terabytes of data. EDIT: that still retain their ability to verify all the way back to the root, on their own time.

Satoshi already created a 100% trustless solution to this called SPV.  It allows lite nodes to be instantly online and download from peers only the portions of the blockchain they need in real time. Install client and you are instantly ready to start transacting.  Your "solution" doesn't work but even if it did would be inferior to SPV which is something Satoshi came up with before the blockhain even started.

You see, as I've mentioned, I have read the Bitcoin paper. Several times. I understand it well.

No you don't or you wouldn't make mistakes like saying "transactions which are spent" or the one you just repeated below.  You also would have realized the SPV clients exist.

The Bitcoin paper points out this interesting property of the blockchain: that the longer it gets, the harder it gets to fake. Impossibly harder.

This isn't in a vacuum that property only applies because for a given tx, the tx, and everything that is needed to validate that tx is also deeper in the blockchain.  You can't throw that away and then assume the same dynamic is in play.

The relationship indeed becomes no longer 100% trustless as I've acknowledged multiple times.

Any system which is not 100% trustless is inferior to the system we have now.   Why would you want to make the protocol worse.  The only way space ever becomes a concern is if Bitcoin becomes many magnitudes larger which means an economy valued in the hundreds of billions if not trillions.   Any amount of trust when the payoff is that large will be at risk of compromise.

If Bitcoin gets to that point, you will come back to this proposal, and you will see it in a different light all of a sudden.

If Bitcoin "gets to that point" true proposals for a ledger system will be given more weight but true proposals actually address the risks rather than just pretend they don't exist.   You haven't proposed anything other than drop all method of verifiy a transaction, throw it into a new block at the head of the chain and hope nobody abuses that.  That isn't a proposal, that is nonsense.

Pruning is very likely to keep the storage requirements modest.  It is likely Bitcoin will never need a ledger snapshot system but if it did it would require more thought that what you propose.  Still the true bottlenecks to transaction volume are node bandwidth and memory.  You are trying to "fix" the bottleneck which isn't a bottleneck while consuming resources on the actual bottleneck.

[kjj]

you've refused to do that basic research*.

False.

Saying "searching is hard" is not the same as doing research.

[sugarpuff]

No you don't or you wouldn't make mistakes like saying "transactions which are spent" or the one you just repeated below.

Sometimes I'm not super sharp with vocabulary that I don't use on a daily basis, sorry.

Just like people who say "ATM machine" or "HTTP protocol" though, it doesn't mean that I don't know what these things are.

You also would have realized the SPV clients exist.

Your "solution" doesn't work but even if it did would be inferior to SPV which is something Satoshi came up with before the blockhain even started.

Although you did not explain how this proposal is inferior to SPV, I'll try my best to show why it provides stronger security than SPV:

SPV clients can't do block-height verification, whereas rolling-root clients do. Because of this, SPV clients can be fooled by a MITM double-spending attack:

This is a concern in a situation where an SPV client is subjected to a double-spend attack by somebody who controls its network connection. For example, suppose you are at a wi-fi cafe and are paying for something using your smartphone -- the cafe owner controls your network connection. Satoshi acknowledges this implicitly when he writes that "the verification is reliable as long as honest nodes control the network" -- to be completely pedantic, this means that the verification is reliable as long as honest nodes control the part of the network that the SPV client is able to communicate with. In an attack-by-ISP scenario this may not be a sufficiently strong security property. The attacker would not need to overpower "the rest of the network" because the client is unable to communicate with it.

Rolling-root clients that have downloaded the blockchain of height W (the window size) cannot be fooled by this.

The Bitcoin paper points out this interesting property of the blockchain: that the longer it gets, the harder it gets to fake. Impossibly harder.

This isn't in a vacuum that property only applies because for a given tx, the tx, and everything that is needed to validate that tx is also deeper in the blockchain.  You can't throw that away and then assume the same dynamic is in play.

Sorry, I'm not sure I understand you, what are you accusing me of throwing away? I'm pretty sure I'm not throwing away block-depth verification if that's what you mean.

The relationship indeed becomes no longer 100% trustless as I've acknowledged multiple times.

Any system which is not 100% trustless is inferior to the system we have now.  Why would you want to make the protocol worse.

Don't think that's true, as I've pointed out multiple times.

A protocol that requires a new node to potentially download hundreds of terabytes seems like it already has problems significant enough to threaten its future existence.

If that absurd requirement can be removed at the tiny expensive of decreasing the "trustlessness" from 100% to ~99%, then it seems like an improvement to the protocol.

There is nothing stopping some nodes from serving as archival nodes (and potentially competing with one another). There are probably a bunch of ways to restore the trustlessness of the system (or notify nodes that they need to look farther back in time) if needed.

For example, say the window size is 1TB, and the total legit blockchain is 100TB. Rolling-root nodes as well as archive nodes could help with the verification of the current rolling root. They could include in the current rolling root a hash of some data that is representative of the rest of the 99TB. If the rolling-root clients discover a discrepancy in this hash, they'll know immediately that something is up. At that point they could initiate additional verification until they resolve the conflict.

You haven't proposed anything other than drop all method of verifiy a transaction

That is not true.

, throw it into a new block at the head of the chain and hope nobody abuses that.

Nobody could abuse it. The ancient unspent coins are signed by a private key that only their owner (hopefully) has. The balance isn't being modified. All that's happening is that balances are being carried forward so that they stay within the window.

There are two things that can be done with old coins:

1. If the idea of a "lost coin" is to be supported as a method of data-compression, then nodes would be responsible for ensuring that old coins are moved forward periodically. They could send a special transaction to do this and wait for it to be confirmed by the network.

2. If that sounds scary, coins can be kept forever regardless of age. The transactions containing them can simply be moved automatically according to some strict rules that I don't care enough to come up with (but am pretty sure exist). The private key is not needed to do this. The history of that transaction will be verified by the rolling-root clients up to the last known root that they have.

Also, the rolling-root clients can locally store (via pruning) the blockchain past the blockchain window size (for as long as they've been running).

Pruning is very likely to keep the storage requirements modest.  It is likely Bitcoin will never need a ledger snapshot system but if it did it would require more thought that what you propose.  Still the true bottlenecks to transaction volume are node bandwidth and memory.

Well this conversation is also about addressing the bandwidth requirement for new nodes to come online, and not specifically local storage requirements (pruning works for that, as you point out here, and as I pointed out earlier and in the edit to the OP).

[DeathAndTaxes]

There really is no more point in discussing it because you simply lack the proper understand of how bitcoin works.  It doesn't work the way you think it does and you can't fix something you don't understand.

I will leave you wish
a) SPV clients have copy of all block headers

b) there is no such thing as "balances" at an address.  Bitcoin works on the concept of inputs and outputs.

c) you say you will "carry forward" the balance into a new block but what happens when an attacker simply publishes a chain which contains unspent outputs that nobody has the historical records to validate anymore.  can you say unlimited minting?  With the current trust model an attacker with >51% hashrate is limited in what they can do.  Under your system they could simply extend the blockchain far enough that most nodes no longer have the historical records to validate a "old" tx placed in a new block ever existed.  The attacker could destroy Bitcoin through unauthorized minting.  

d) "The transactions containing them can simply be moved automatically according to some strict rules that I don't care enough to come up with"  That is why you have no proposal.  "Um yeah we can do this stuff but it will require some stuff but I don't really fill like saying the stuff but if you don't agree with me you are jerks and just trolling.  We can fill in the stuff later and stuff."

I won't be responding again, you can't provide a proposal when you can't even speak the language.   At appeared at one time you were interested in actually learning how Bitcoin really works but that is obviously not the case.  If you have no interesting in actually learning how the system you are trying to "fix" works then don't be surprised when nobody takes you seriously.