Proposal to solve bitcoin scaling problem

[bji]

Hi,

This topic was already discussed back in May. There's no need for payment schemes or other complexity.

   http://forum.bitcoin.org/index.php?topic=7972.0

I will add something to the scalability page about this.

From my reading, the proposal in that thread was more or less the same as mine, and an argument against it were that nobody has to download the entire block chain because they can just rely on the compiled-in "checkpoints".  Of course, 'supernodes', i.e. miners, would always need the entire chain to be able to validate transactions.

But even if my client has a checkpoint, if the size of blocks is 1 MB or more (which it will be eventually), it's still not practical to expect clients to ever have to download full blocks on an ongoing manner; end-user network bandwidth does not support that.  They will need elided (pre-pruned) blocks.

Of course, it's possible that there never will be enough transactions on the bitcoin network to make blocks this big; I strongly suspect that bitcoin will never gain foothold except as a currency for pseudoanonymously buying illegal goods or making untraceable payments for legal goods that the end-user doesn't want associated with them (it is the only monetary niche that I can see that bitcoin has advantages over other forms of payment), and maybe the number of transactions per second in that niche market will never exceed 10 or so ...

[1713583970]

1713583970

[1713583970]

1713583970

[Mike Hearn]

Checkpoints are orthogonal, I don't think anyone argued that they're enough on their own. I'm not sure what part you thought was an argument against it - AFAIK using transactions linked via branches to the root is uncontroversial.

[prof7bit]

I have proposed 2 new messages and one new inventory type in the wiki on the protocol specification talk page:
https://en.bitcoin.it/wiki/Talk:Protocol_specification#Proposing_additional_protocol_messages

(further explanation in the wiki)

Messages:

• setfilter
• getblocksfiltered

inventory types for getdata:

• 0x03 MSG_BLOCK_PRUNED

These messages need to be built into the standard client to allow light clients to be written. Additionally this should be announced in the services field of the version message with the following new flags:

• 0x02 NODE_CAN_FILTER
• 0x04 NODE_CAN_PRUNE

and the light client would advertise that it does not have a copy of the block chain and does not do verification on its own, and that it also won't relay messages to others.

• 0x08 NODE_NO_VERIFY
• 0x10 NODE_NO_BLOCKS
• 0x20 NODE_NO_RELAY

[AntiVigilante]

The issue is a real problem, but you lost me at "the only niche I see for bitcoin". You must have all your ducks in a row. Not in the path of a single shit distributing fan. I wish I could envy your hallucination.

[bji]

The issue is a real problem, but you lost me at "the only niche I see for bitcoin". You must have all your ducks in a row. Not in the path of a single shit distributing fan. I wish I could envy your hallucination.

If you have something useful to add to the discussion, please do so.

[Mike Hearn]

It might make more sense to think about this as a case of a new block structure version.

For pruning it's not enough to just provide the block header and then a list of transactions. You need to provide the merkle branches as well.

If you're serious about implementing this, I suggest posting to the bitcoin-development mailing list about it (maybe with a patch). We can discuss it further there.

[prof7bit]

For pruning it's not enough to just provide the block header and then a list of transactions. You need to provide the merkle branches as well.

I think this is exactly was what the OP wanted from post #1 on. Nobody wanted to leave away the merkle tree, where do you have this idea from? This thread is about simplified verification strictly accordig to the paper. A properly pruned block is also what I am suggesting should be answered to a getdata with the (still hypothetical) inventory type MSG_BLOCK_PRUNED.

If you're serious about implementing this, I suggest posting to the bitcoin-development mailing list about it (maybe with a patch). We can discuss it further there.

It would have been nice to mention in a sticky post at the top of this forum that this forum is (despite its name) not meant to discuss development questions or this forum should be renamed or deleted to avoid people putting their energy into writing posts that basically go to /dev/null.

I am still hoping the bitcoin community is seeing the need for completing the mising half of the protocol because I would like to start implementing my client ASAP, but this would only be possible if the protocol were extended with the missing functionality. I am basically in waiting position, ready to fire up my IDE any moment and start coding, as soon as I see the protocol allows for such clients.

[Mike Hearn]

This forum isn't /dev/null, the maintainers do read it. But for discussion of specific changes rather than higher level discussions the mailing list is better. And yes there should be notification that there is a mailing list. It's still very new.

You need to be precise with your terms though - a merkle tree is not the same thing as a merkle branch.

What I meant is you haven't defined the format of the pruned block. I think it makes more sense to think about this in terms of transactions rather than blocks. The protocols existing design means you can request thousands of headers in one go. You then want to find out about transactions that apply to you, along with the merkle branches associated with those transactions.

[prof7bit]

You then want to find out about transactions that apply to you, along with the merkle branches associated with those transactions.

Yes. This is exactly what I meant to accomplish with this proposed addition to the inventory types that can be requested with getdata: only the relevant branches and relevant transactions instead of the full tree and all transactions.

I meant to indirectly impliy this by mentioning "according to the paper", so i did not go into the detail. After all its just a proposal and a lot of other combinations of new protocol messages might be possible to serve the exact same purpose, important is that there is at least *some* way to do it and that it is designed to really be able to remove all traffic that is not absolutely needed.