Scaling Bitcoin with Subchains

[onelineproof]

tl; dr: This is a soft-fork way to allow Bitcoin to have an arbitrarily high transaction rate, without increasing the block size, and improving decentralisation. Yes it adds some complexity, but if you understand it carefully, you will see that it significantly improves the protection against censorship attacks and mining centralization. I'm not saying this is the best way to do things, and exact details can be tweaked, but I haven't seen a better proposal (Lightning alone has limitations), and it is definitely better than simply increasing the block size. As I have calculated, even 10 MB blocks is too much centralisation for many years to come, and the limit can instead be regulated with a softfork that forces miners to mine at a higher difficulty if they want bigger blocks.

I posted this to the Bitcoin-development mailing list a few weeks ago: http://sourceforge.net/p/bitcoin/mailman/message/34127318/. Please read that post. Here I want to clarify some things and add some more details.

It is similar to the tree chains idea by Peter Todd, but I think it solves some problems that those had. It is kind of like sidechains, but the chains are not independent. You can call also call it a form of "blockchain extensions", something that Adam Back (inventor of hashcash) likes: http://sourceforge.net/p/bitcoin/mailman/message/34157058/. So I just chose to call it “subchains” to distinguish it from the other similar concepts.

The idea is that you add ten blockchains, each with a 1 MB block size, that are validated by the main chain (the one we use now). The main chain can validate them by recording a hash of each of the chains’ block headers inside its own block header; so the 10 chains are synchronized with the main chain. As you know from the sidechains paper  (https://www.blockstream.com/sidechains.pdf) you can transfer Bitcoin between chains, and this is made easier now since the chains are synchronized, and the child chain always trusts the parent chain.

So now you have the equivalent of 11 MB blocks, but spread out on 11 chains. The benefit is that now you can choose what chain to store/validate. If you have only one chain, then you need to store all blocks (since a long term period) in order to see and validate all the transactions that occurred on that chain. The key word is “see”. An SPV node can validate whether a transaction occurred or not, but it is not certain whether it is looking at ALL the transactions (some can be hidden by full nodes) and, as I explained in the mailing list post, a heavily pruned node is BAD for transparency (keeping powerful entities such as governments and corporations in check). You cannot just store a subset of transactions from a chain that are important to you, since the validity of each new block depends on the validity of the previous blocks.

Due to the relationships between the chains and the sizes of the blocks, you only need to store two chains. You (and everyone else) stores the main chain. But you only need to store one of the 10 subchains. Just make sure to keep all of your transactions on that one chain that you choose. Transactions that include two different subchains will be recorded on each chain (so there is some duplication) Thus, you get the power of an 1+10 MB / 2 = 6 MB block size while only storing/validating two chains each with 1 MB blocks, which is equivalent in terms of storage size / computing power of you storing/validating a blockchain with a 2 MB block size. If you want to keep track of transactions that are in another subchain (such as the public purchases being made by your government MP), you can store and validate that one too, so you need the equivalent of 3 MB per block in that case. You can also have an alternate identity that uses a different blockchain... In general, the amount you store/validate is proportional to the amount of “stuff” you want to keep track of. With this kind of partitioning scheme you have the freedom to finely tune what you store/validate, instead of being forced to follow a “one size fits all” policy.

You can keep going like this by adding 10 children chains for each of the 10 subchains, resulting in 100 more chains (now we have the equivalent of a 1+10/2+100/2 = 56 MB block size with each participant required to only store/validate 3 MB. For 4 levels, you have the equivalent of 1+10/2+100/2+1000/2 = 556 MB block size with each participant only needing 4 MB. In general, if you have n levels of blockchains, the network has O(5^(n-1)) transactions, while each participant only stores O(n) of them. In other words, each participant stores/validates ~ log(N) transactions where N is the number of transactions in the network. This is far better than the ~ N^2 requirement on each participant if every transaction just happens on one chain. Note, even if you want to store/validate transactions on a chain, you don’t have to fully store/validate the parent of that chain. You just need the block headers of the parent chains in order to validate each block in the chain. Also, miners on a chain will only need the headers of the 10 child chains in order to validate the transactions going on in the child chains. The idea is that higher valued transactions will be performed on the chains that are higher up in the hierarchy, while lower valued transactions on the chains that are lower in the heirarchy.

For any chain that a miner mines on, it will be merged mined with its parent chain, so a miner has a chance of getting the rewards from the parent chain and also strengthening the hash power of the parent chain. Big miners will tend to stick to the upper level chains, since they will have the highest transaction fees (as well as a coinbase reward for the main chain). Note: Initially I thought that miners should merge-mine with the main chain, but then you wouldn’t have good mining decentralization, right?

Other things: Chains that are too empty have an incentive to be filled up with transactions since the transaction fees are lower there. For instant transactions, contracts, we can use Lightning channels attached to whatever chains you want. Privacy would be enhanced since it would be hard for one node to store all transactions when the number of transactions get very large.

To illustrate the point, I made a diagram:



Initially, I wanted to draw a classical tree-like structure, but it was awkward to fit all the nodes, so I decided to go with the circular structure. With just one blockchain, we are sticking all transactions in the center black dot (centralisation). With more blockchains, we can spread things out, let everyone contribute, and actually scale for a higher transaction rate than possible with the centralised model (no more O(N^2) or even O(N)).

One more thing: Recently, I have been thinking of ways to add flexibility to the block size, so that it is relevant even with better computing technology. As you should know, miners hash block headers. How about adding a soft-fork rule, that blocks must be 1 MB with block header hashing or more if the whole block is hashed? It doesn’t have to be the whole block, it can also be the hashes of the transactions inside the block, or anything proportional to the block size. That way, we have a robust proof-of-computers-are-now-fast-enough-to-have-this-block-size. For instance, in the future, computers (or ASICs) can be so fast that 10 MB blocks can be hashed at the same speed as headers are hashed today. Then the soft fork rule will allow 10 MB blocks. This needs more research.

Mike Hearn will probably tell you that this is a Rube-Goldberg contraption, but I believe that this is an elegant solution, and probably the simplest I can think of. It is more of a generalisation of Bitcoin rather than a change of Bitcoin, and only requires a soft-fork. It is like in physics, where you have the special theory of relativity, and then you zoom out and realize that it is actually a special case of the general theory of relativity.

And yes, I do have a strong academic background in the fields related to Bitcoin, so I am familiar with the mainstream arguments in these fields. I also have good practical experience with Bitcoin related projects. But it is possible that I made an error in my logic somewhere, so it would be good if some people could review this. In particular, I would appreciate it if I could hear comments from the following people: Gavin Andreson, Peter Todd, Adam Back, Pieter Wuille, Andrew Poelstra.

Thanks

Edit Jun 19: As I mentioned in the mailing list, the other main idea is that addresses will be constrained to a path of subchains. An address can be represented as a number, and therefore you can perform modular division on it. (Let "%" mean "mod") The rule can be like this: For any address A, if A % 1 = 0, then A goes to the top chain C0 (so every address); if A % 11 = 0, to subchain C00, else if A % 10 = 0, to subchain C01, else if A % 9 = 0, to subchain C02, ...., else if A % 2 = 0, to subchain C09; if A % 111 = 0, to subchain C000, else if A % 100 = 0, to subchain C001, ..., etc.. Note Cij is a child of Ci, and Cijk is a child of Cij, and Cijkl is a child of Cijk, etc... And to make it so that and address A satsfies A mod m = 0, just multiply the address by m. This way you can ensure that your wallet follows only one path of chains that you track, and that you way you can keep track of the UTXO database related to your wallet (or the public wallet of your government representative) in a O(log n) way without using UTXO commitments, which I don't think is as good for decentralization purposes and shortness of proof.

Also, I take back the merge-mining scheme, I think it should be done without merge mining at all. Parent chains will instead collect fees from child chains to put the hashes of the child chains as special transactions in the parent chain blocks. These fees will be of the form of UTXOs that are only valid at the child chains (not the parent chains). That will give incentive for the parent chain to be careful with what child block headers it commits to (thus properly validate the child chains).

Edit Jun 24: Forgot to add that at most 2 sibling chains can be used in a transaction, so that the duplication of transactions is limited. You can always create a more complicated transaction by combining multiple transactions that include at most 2 chains.

Also, to limit "deep" forks, we should make a rule that once an output moves from a child chain to its parent (or grandparent) chain, any later discovery of a flaw in the output's value in the child chain cannot change its value, so this limits the effect that errors in the validation of deep child chains has on the parent chains.

[ebliever]

This is interesting stuff. I just read your Sourceforge post and responses and am digesting this now. I will likely have some questions and maybe even helpful comments once I grasp it all better. (I'm brainstorming away on a little project that, by some back-of-the-envelope calculations, could ultimately scale up to need tens of thousands of blockchains. So I'm learning all I can about these possibilities.)

[ebliever]

OK... some questions.

1. What governs the establishment of new blockchains, at any given tier below tier 1? Is it hardcoded from the start, or can new blockchains be initiated on an ongoing basis? If so, what initiates them? (And if they can be generated at will, how would one prevent abuse of this capability by a malicious party generating endless chains either to insta-mine them or clog up the whole scheme? For example, one might require a payment to a higher level chain of a moderate amount to register the new sub-chain, though I'm not convinced that would be a full solution.)
2. If the number of chains at each tier is established at rollout, what would be involved in updating (hardforking?) the scheme to add more chains, add a new layer of chains, or potentially even delete a chain? How disruptive would it be?
3. If new blockchains can be initiated without a hardfork (requiring consensus) or central authority, how could one ensure that the new chains have the same feature set such as block time and block reward amount?
4. Are there any inherent limitations on how mining rewards are distributed between the various chains? Could the chains all be synced to provide an equal reward (for example, at the time of a halvening)?

Thanks for your work in putting this proposal together. I hope it bears fruit.

[tromp]

The idea is that you add ten blockchains, each with a 1 MB block size, that are validated by the main chain (the one we use now). The main chain can validate them by recording a hash of each of the chains’ block headers inside its own block header; so the 10 chains are synchronized with the main chain.

There may be a chicken-and-egg problem where most miners do not see sufficient traffic on
subchains to make collecting their tx fees worthwhile, and just keep copying the previous
10 hashes, which in turn makes use of the subchains unattractive...

[onelineproof]

OK... some questions.

1. What governs the establishment of new blockchains, at any given tier below tier 1? Is it hardcoded from the start, or can new blockchains be initiated on an ongoing basis? If so, what initiates them? (And if they can be generated at will, how would one prevent abuse of this capability by a malicious party generating endless chains either to insta-mine them or clog up the whole scheme? For example, one might require a payment to a higher level chain of a moderate amount to register the new sub-chain, though I'm not convinced that would be a full solution.)
2. If the number of chains at each tier is established at rollout, what would be involved in updating (hardforking?) the scheme to add more chains, add a new layer of chains, or potentially even delete a chain? How disruptive would it be?
3. If new blockchains can be initiated without a hardfork (requiring consensus) or central authority, how could one ensure that the new chains have the same feature set such as block time and block reward amount?
4. Are there any inherent limitations on how mining rewards are distributed between the various chains? Could the chains all be synced to provide an equal reward (for example, at the time of a halvening)?

Thanks for your work in putting this proposal together. I hope it bears fruit.

(1 and 2) Well first people would have to agree on the first 10 chains (level 2). The use of these chains can be softforked into the protocol (add it into the latest version of bitcoind for example). They can be identified by the hash of their headers in the main chain. Once enough hashpower agrees to the softfork, the hashes will start appearing in the main blocks. These hashes will basically give the state of the bitcoin that went to the subchains. If another subchain comes along, and miners start hashing its header instead of subchain # whatever, then it needs to be consistent with the previous transactions of the previous subchain # whatever; so it basically needs to be the same subchain.

Now one problem I can see is transferring from subchain to parent chain. What if the miners on the parent chain refuse to accept bitcoin that was received on a subchain? Well one way to ease this would be to add an additional rule into the softfork protocol that states that subchain transactions that were hashed into the main chain should always be recognized even if in the future we create some other softfork that does not anymore create these hashes into the main chain (so allow for any bitcoin that is tied up in sidechains a chance to come back to the main chain, but don't let it flow to the sidechains anymore, so that we can limit the amount of useless transaction traffic due to the old softfork, and still honour the old softfork). Of course, any spendable output can be refused to be recognized by a miner (personal blacklists or whatever), so this sidechains stuff doesn't really create these kinds of attacks as they are currently existent without them. But, we know that miners have an incentive to be cooperative, as they get fees for performing transactions (Satoshi mentioned this in his paper). Also, note that the child chains will be merge-mined with the parent chain, so miners on the main chain will get some fees that are recorded on the child chains, so there is an incentive for them to keep the child chains legitimate, as they have some fee rewards sitting there. Also, child chains can end up as altcoin chains in the worst case scenario (so still have some value even if not recognized by the parent chain). But yes, the child chains will generally have less security than the corresponding parent chain (less hashpower, chance of not being recognized by parent chain), but that's the price you pay for having lower transaction fees.

(3) The chains cannot just be randomly initiated. They have to be recognized by the corresponding parent chain through the hashes in the parent's header. And, no there will be no block rewards in the child chains (no inflation), at least in my vision, only fees.

(4) As I said, I think there should be a rule that anyone mining on a child chain will be merge mining the corresponding parent chain (only one level of parent). This basically means that a solution to the mining problem on the parent chain will be valid in the child chain (just with lower difficulty). See for example how Namecoin does the merge-mining with Bitcoin. So generally, miners who mine on the parent chain or children chain (or both), will get paid on both the parent chain and the child chains.

By the way, I'm not doing anything very new here. These are mostly things that people have talked about before. We just need to make it more precise, code it up, and test it out. It's just strange to me that no one really talks about something like this as being a good solution to scaling Bitcoin without increasing the block size. Yes Lightning can help, though I don't think it can help for arbitrarily high transaction volumes like this can.

The idea is that you add ten blockchains, each with a 1 MB block size, that are validated by the main chain (the one we use now). The main chain can validate them by recording a hash of each of the chains’ block headers inside its own block header; so the 10 chains are synchronized with the main chain.

There may be a chicken-and-egg problem where most miners do not see sufficient traffic on
subchains to make collecting their tx fees worthwhile, and just keep copying the previous
10 hashes, which in turn makes use of the subchains unattractive...

Well once transaction fees get too high on the main chain, people will have an incentive to pay quite high fees on the subchains, so that will give incentive to the miners I think.

[ebliever]

Onelineproof, thanks for the answers. Looks like you've put quite a bit of thought into it. I'm sure it's not easy, or someone would have solved the scalability issues with a solution like this already.

The one thing I'm still struggling to grasp is how a subchain gets created. You write:

(3) The chains cannot just be randomly initiated. They have to be recognized by the corresponding parent chain through the hashes in the parent's header. And, no there will be no block rewards in the child chains (no inflation), at least in my vision, only fees.

How does a new sub-chain's hash get inserted into a parent chains' header? Maybe this is wrong, but I'm thinking something along these lines: A miner mines a new parent chain block, and inserts a new hash for a new sub-chain that has just been created. Then the block is broadcast; either there was a pre-existing consensus for this addition and it is accepted by the other nodes, or it was the action of a rogue miner and it is rejected. Is that roughly accurate?

If so, how would the miners know and agree to a new hash being added? Would this be through either a soft-fork or a hard-fork? What I'm really trying to grasp is who would have the authority to initiate a new subchain, and whom would have to agree to it to make it happen.

Thanks,
ebliever

[johoe]

One problem I see with sidechains is how do we freeze the money in the main chain that is sent to the sidechain so that we can get it out again in a secure way.  The sidechains paper suggests two ways

1. Adding an OP_SIDECHAINVERIFY command; the money is freezed on the main chain and protected by this opcode.
2. Freezing the money on the main chain by sending it to a m of n multisig address that is controlled by the inventors of the side chain.

The first option only works if more than 50 % of the miners implement this opcode and check the sidechain.  Otherwise, someone can easily steal the frozen money by mining a new block and the majority would just accept it (forking off the minority that mines the sidechain).  So in the end a main chain miner needs to mine all sidechains where the first option is used.

The second option trusts a few people to keep their private keys secure (they are a tempting target for any hacker) and to not collude and run with all the money on the sidechain.  The fact that they are functionaries only means that fraud can be detected but it cannot be prevented.

[onelineproof]

Onelineproof, thanks for the answers. Looks like you've put quite a bit of thought into it. I'm sure it's not easy, or someone would have solved the scalability issues with a solution like this already.

The one thing I'm still struggling to grasp is how a subchain gets created. You write:

(3) The chains cannot just be randomly initiated. They have to be recognized by the corresponding parent chain through the hashes in the parent's header. And, no there will be no block rewards in the child chains (no inflation), at least in my vision, only fees.

How does a new sub-chain's hash get inserted into a parent chains' header? Maybe this is wrong, but I'm thinking something along these lines: A miner mines a new parent chain block, and inserts a new hash for a new sub-chain that has just been created. Then the block is broadcast; either there was a pre-existing consensus for this addition and it is accepted by the other nodes, or it was the action of a rogue miner and it is rejected. Is that roughly accurate?

That's a good question. A Bitcoin header is 80 bytes and doesn't really have room for more data, but perhaps the child chain hashes can be stored as transactions. Just put one transaction for each child that includes the hash of the child's header. The transactions/hashes can be verified as normal using the merkle root of the header (like what SPV nodes do), so all you need is the header plus the relevant transactions to verify. To give incentive for such transactions to be created, fees can be paid by miners on the child chains. This may even be necessary, as without fees, DOS attacks can be performed. One can also use transactions to include hashes of extended headers or other additional information, but I think this way is simplest.

If so, how would the miners know and agree to a new hash being added? Would this be through either a soft-fork or a hard-fork? What I'm really trying to grasp is who would have the authority to initiate a new subchain, and whom would have to agree to it to make it happen.

Due to the merge mining going on between the direct parent-child chains, I think miners on any given chain, would decide whether to create transactions on child chains, depending on whether people are willing to pay enough fees to make it happen. So if Alice wants to send some coins to Bob, she can say "I will pay 50 cents to send it on chain C or 5 cents to send it on a child of chain C". The miner will decide what is more worth it for him.

One problem I see with sidechains is how do we freeze the money in the main chain that is sent to the sidechain so that we can get it out again in a secure way.  The sidechains paper suggests two ways

1. Adding an OP_SIDECHAINVERIFY command; the money is freezed on the main chain and protected by this opcode.
2. Freezing the money on the main chain by sending it to a m of n multisig address that is controlled by the inventors of the side chain.

The first option only works if more than 50 % of the miners implement this opcode and check the sidechain.  Otherwise, someone can easily steal the frozen money by mining a new block and the majority would just accept it (forking off the minority that mines the sidechain).  So in the end a main chain miner needs to mine all sidechains where the first option is used.

The second option trusts a few people to keep their private keys secure (they are a tempting target for any hacker) and to not collude and run with all the money on the sidechain.  The fact that they are functionaries only means that fraud can be detected but it cannot be prevented.

The sidechain paper envisions sidechains as independent chains, so you need some kind of mechanism for one chain to prove to another chain that everything is fine, and reorganizations can occur, and it complicates things. My idea is to have dependent chains where the child chain trusts the parent chain and the parent chain miners do some work on the child chain verification. So it is more a form of "block extensions" rather than sidechains. Or you can just think of it as synchronized sidechains. You can visualize it using the diagram I created. If you think of the blockchains as living on top of a wheel, then each time the wheel turns 36 degrees (one tenth of a full revolution), a block is created, and all the chains move together and are on the same page as to which block is the correct block on all levels of chains. Makes sense?

[ebliever]

Thanks for explaining a bit more. I was struggling to understand about creating sidechains because of a mismatch between how you (and everyone else!) view and plan to employ sidechains, versus my own ideas. I've been reading the sidechain whitepaper and think I grasp things a bit better now.

Regarding the question of how to prevent funds from being double-spent or lost, I'd like to share a simplified form of a model I have. I'd like to have the opinion of those with more experience as to whether this approach is workable and what the risks or obstacles to use would be.
_____________

Imagine an arrangement with a master chain and multiple parallel sidechains. The master chain carries the hashes of the sidechains as in other sidechain proposals. But it also functions as a registry of public keys, assigning them to sidechains in some orderly, balanced manner. (Perhaps even authorizing/initiating new sidechains when certain criteria are reached to maintain capacity).

Whenever a transaction is broadcast, the master blockchain is scanned for the public keys in question, and a copy of the transaction is sent to each sidechain affected. For previously existing keys there would be no further activity on the master blockchain. If the transaction involves sending to a new (to the entire multi-chain) public key, the master chain records a transaction (registry entry) that lists the public key and the sidechain assigned to it forevermore.

This approach assumes a fairly high level of re-use of public keys, although that's not a requirement.

Is this workable, or is there a glaring problem or something intractable about it?

_____________

Postscript: If it is workable, what about the following modification...

In this scheme, the registration of public keys is done on the master blockchain by the owner of the key, who also has the opportunity to pick any available (not yet reserved) Alias. By paying a fee the Alias would be reserved and tied to that public key for a set duration (for example, a fee of 1 satoshi per block would work out to roughly $1/year at current prices for a bitcoin-like crypto). The registration entry would thus consist of the Public Key, Alias, Fee, and Block # Expiration (calculated from the fee and the block # it was logged at), and the Sidechain assigned to this key. (One would assume that subsequent fees for the same Alias could be made by the same public key holder to extend the duration.) The fee would go to miners presumably. Once the alias expired one would presumably still be able to send to the public key, just not the alias.

Anyway, in this scheme a client would not broadcast a transaction until it first scanned the master blockchain to ensure that the recipient(s) of the funds are properly registered. And the sender would not need the public keys - just the Alias. Their client would handle adding the public key(s) of the recipients automatically. So if I wanted to send funds to an early adopter who picked "Peter" as their alias, it would be as simple as typing their name to address the funds - no 1HF84fskj33Fg... rigamarole.

Thanks (to anyone) for any insight,
ebliever

[onelineproof]

Thanks for explaining a bit more. I was struggling to understand about creating sidechains because of a mismatch between how you (and everyone else!) view and plan to employ sidechains, versus my own ideas. I've been reading the sidechain whitepaper and think I grasp things a bit better now.

Regarding the question of how to prevent funds from being double-spent or lost, I'd like to share a simplified form of a model I have. I'd like to have the opinion of those with more experience as to whether this approach is workable and what the risks or obstacles to use would be.
_____________

Imagine an arrangement with a master chain and multiple parallel sidechains. The master chain carries the hashes of the sidechains as in other sidechain proposals. But it also functions as a registry of public keys, assigning them to sidechains in some orderly, balanced manner. (Perhaps even authorizing/initiating new sidechains when certain criteria are reached to maintain capacity).

Whenever a transaction is broadcast, the master blockchain is scanned for the public keys in question, and a copy of the transaction is sent to each sidechain affected. For previously existing keys there would be no further activity on the master blockchain. If the transaction involves sending to a new (to the entire multi-chain) public key, the master chain records a transaction (registry entry) that lists the public key and the sidechain assigned to it forevermore.

This approach assumes a fairly high level of re-use of public keys, although that's not a requirement.

Is this workable, or is there a glaring problem or something intractable about it?

_____________

Postscript: If it is workable, what about the following modification...

In this scheme, the registration of public keys is done on the master blockchain by the owner of the key, who also has the opportunity to pick any available (not yet reserved) Alias. By paying a fee the Alias would be reserved and tied to that public key for a set duration (for example, a fee of 1 satoshi per block would work out to roughly $1/year at current prices for a bitcoin-like crypto). The registration entry would thus consist of the Public Key, Alias, Fee, and Block # Expiration (calculated from the fee and the block # it was logged at), and the Sidechain assigned to this key. (One would assume that subsequent fees for the same Alias could be made by the same public key holder to extend the duration.) The fee would go to miners presumably. Once the alias expired one would presumably still be able to send to the public key, just not the alias.

Anyway, in this scheme a client would not broadcast a transaction until it first scanned the master blockchain to ensure that the recipient(s) of the funds are properly registered. And the sender would not need the public keys - just the Alias. Their client would handle adding the public key(s) of the recipients automatically. So if I wanted to send funds to an early adopter who picked "Peter" as their alias, it would be as simple as typing their name to address the funds - no 1HF84fskj33Fg... rigamarole.

Thanks (to anyone) for any insight,
ebliever

So you just want some kind of alias associated with a person's bitcoin addresses? I guess Namecoin does this, so you can extend Bitcoin to have this Namecoin functionality and it wouldn't really require sidechains.

By the way, I had some more replies to my original mailing list post. Some people think that what I am doing can be achieved with bigger block sizes and with SPV nodes, but this is not true. One of the main advantages of this tree structure of subchains is that you can constrain your wallet to just one path of subchains, so you can download all the full blocks on each of these chains and be sure that you have the status of all the UTXOs corresponding to your wallet. In simpler words, it lets you be sure of your balance in a scalable (O(log n)) way. Same with the balance of others you wish to track (such as your government representatives). I read a bit about so-called "UTXO commitments" but I don't think they are as reliable and decentralized as this method, since you still need supernodes to feed you the merkle-tree proofs. Can someone answer this?

Also, Peter Todd kind of replied and said that tree-chain like structures can possibly be used for scaling, but no one has yet made a solid proposal for it (I will make a more complete write-up soon). Also, he said merge-mined chains should not be used if we want miner decentralisation. Before I said that only direct child-parent merge mining would occur, but now I take that back (I see a problem with that). Actually, my scheme described above does not need merge-mined chains. As I mentioned, parent chains would get fees from child chain miners to put the hashes in, and (not sure if I said this) the fees can be in the form of outputs that are registered as spendable on the child chains only, so this would give the parent chain miners an incentive to be careful and validate as much as possible the transactions of the child chains.

[ebliever]

So you just want some kind of alias associated with a person's bitcoin addresses? I guess Namecoin does this, so you can extend Bitcoin to have this Namecoin functionality and it wouldn't really require sidechains.

(snip)

Also, Peter Todd kind of replied and said that tree-chain like structures can possibly be used for scaling, but no one has yet made a solid proposal for it (I will make a more complete write-up soon). Also, he said merge-mined chains should not be used if we want miner decentralisation. Before I said that only direct child-parent merge mining would occur, but now I take that back (I see a problem with that). Actually, my scheme described above does not need merge-mined chains. As I mentioned, parent chains would get fees from child chain miners to put the hashes in, and (not sure if I said this) the fees can be in the form of outputs that are registered as spendable on the child chains only, so this would give the parent chain miners an incentive to be careful and validate as much as possible the transactions of the child chains.

My interest in sidechains is scalability. I calculate that if my little idea is as good as I think it is, I will ultimately need a transaction capacity on the order of 1 million TX/minute for my little scheme. Given the tremendous debate over Bitcoin's capacity of 7 TX/second, I think I'm gonna need more blockchains. 

I like your ideas about pushing the mining fees down to the child chains to incentivize the miners to maintain them. That aligns with my own thinking as well. Actually, one of my big concerns right now is how on earth I could arrange for the security of tens of thousands (I estimate) of blockchains sharing a single cryptocurrency. Of course, if global adoption of my project did happen the amount of mining hashpower in support of it would be tremendous (many times larger than the entire cryptocurrency scene today), but we'd need to enforce major decentralization to spread it more or less equally. That may mean sticking with GPU/CPU mining and avoid ASICs. Perhaps by relying on an announced policy of periodic algorithm changes, using a multitude of varying algorithms. Or allowing ASICs coupled with something like 21 Inc.'s project to embed them in a multitude of appliances.

Still, in principle it would be a small matter for someone to lease and concentrate hashpower on one among thousands of chains and launch a 51% attack. So I also like your idea of a hierarchy of sidechains where lower chains handle smaller transactions, thus there is less incentive to waste time on a double-spending attack on a chain limited to micropayments.

[earonesty]

In a way, this is like the dotted-quad notation of IPv4, where each "level" of subchains is another dotted quad.   It might be more convenient to use 16 (0-9A-F) as the number of subchains per level.   

I would add that the difficulty of subchains should be vetted in the main chain, so that someone can choose an address that targets a level of difficulty they are comfortable with.   

It seems that subchains can safely be weaker than the main chain, because of the vetting done by the main chain, which acts, effectively, as a series of checkpoints.

Also, some addresses should be reserved for future use.   For example, an alternate hashing scheme could be added to address position 11 in a future where this is seen as desirable.  Again, since this is synched with the main chain, it benefits a great deal from the security the main chain provides, with the possiblity of a long-fork mitigated.   


0 - 10 full block subchains
11 - 15 reserved for future use

[hv_]

I d say we need all types of scaling as long as they are really secure!

https://medium.com/@solex1/time-for-bitcoin-1-x-17b54eed2c4a#.q7x1x44r0

[LucioTan]

I briefly mentioned something about this on the bitcoin-dev IRC room. In
general, it seems experts are against using
sidechains as a way of scaling. As I only have a high level understanding
of the Bitcoin protocol, I cannot be sure if what I want to do is actually
defined as a side chain, but let me just propose it, and please let me know
whether it can work, and if not why not (I'm not scared of digging into
more technical resources in order to fully understand). I do have a good
academic/practical background for Bitcoin, and I'm ready to contribute code
if needed (one of my contributions includes a paper wallet creator written
in C).

The main problem I see with increasing the block size limit is that it
increases the amount of storage required to fully validate all transactions
for say 100 years (a person's life). With 1 MB blocks, you can store all
lifetime transactions on a 5 TB drive, which basically any regular user can
do. With 10 MB blocks, you need a 50 TB drive, not accessible for regular
users. Yes, it's possible that in the future hard drive technology will get
cheaper and smaller, but this still didn't happen, so we can't just say "it
should be doable at the rate of Moore's law etc...", we need to know that
it is accessible for everyone, now. Also, don't forget that human life
expectancy can increase with time as well. I know, it sounds silly to use a
human lifetime as a measurement of how far back each user should be able to
store transactions for, but what is a better measurement? This is a
technology made for people i.e. humans, right, and the important part is
that it is for regular people and not just well privileged people. You can
search my last four emails for some more calculations.

What sipa told me on the IRC channel is that Bitcoin Core does not care
about old transactions. It only looks at the current blocks. Yes, that
makes sense, but how do you know that your machine wasn't compromised when
validating the previous blocks? And what if you want to check some old
transactions (assuming you didn't index everything)? What if some of your
old transaction data was lost or corrupted? I think it is clear that it is
useful to be able to validate all blocks (since 100 years) rather than just
a pruned part. It empowers people to have as much information about Bitcoin
transactions as do large data centers; transactions that may include
government or corporate corruption. This is the key to how Bitcoin enables
transparency for those who should be transparent (individual users with
private addresses can still remain anonymous). Also, 5 TB takes about 20
days to sync starting fresh, on a regular computer, so it allows easy entry
into the system.

So assuming we agree that people should be able to store ~ a lifetime of
transactions, then we need 1 MB blocks. But of course, this leads to huge
transaction costs, and small purchases will be out of limits. So to fix
this, I propose adding a 10 1 MB chains below the main chain (sorry on the
IRC room I said 10 10 MB chains by mistake), so effectively, you have a new
10 MB chain that is partitioned into 10 parts. You can also add a third
level: 100 1 MB chains, and keep going like that. The idea is that when you
make a large transaction, you put it through the top chain; when you make a
medium sized transaction, you put it through one of the middle chains,
which will be verified (mined) by the middle chain, and the top chain will
verify the aggregate transactions of the middle chain. If you have a small
sized transaction, you put it through one of the bottom chains, the bottom
chain verifies it, the middle chain verifies the aggregate transactions of
the bottom chain, and the top chain verifies the aggregate transactions of
the middle chain. By aggregate transaction, I mean the net result of
multiple transactions, and I suppose it can be 20 transactions belonging
only to one "sibling" chain for level 2, or 200 transactions for level 3,
etc...

Now, how does the system decide to which of the 10 chains the middle sized
transaction goes to? I propose just taking some simple function of the
input addresses mod 10, so that you can just keep randomly generating a
wallet until you get one with only addresses that map to only one of the 10
chains (even distribution), so that someone can choose one of the 10
chains, and store only the transactions that belong to that chain. They
should also choose a chain from level 3, etc... So in effect, they will be
storing a chain with block size O(n) where n is the number of levels. They
may store multiple sibling chains at one level, if they want to track of
other people's transactions, such as those of their government MP, or
perhaps, they want to have a separate identity that would be more anonymous
with a separate series of sibling chains. This will increase the storage
size, but the increase will be proportional to the number of things you
want to keep track of (at least this kind of system gives you the ability
to fine tune your storage needs to the level of "things" you want to keep
track of). Also, note that there may likely be duplication of transactions,
since transactions can include addresses that are associated with different
silbling chains, but this effect shouldn't make a big difference, and again
will depend on the complexity of the transactions you want to keep track of.

So how can this work? I propose that we keep the current chain as the top
chain, and then create 10 level 2 chains that also store Bitcoin and the
Bitcoin can be transferred between chains (I think this is the idea of
sidechains). How can we incentivize people to keep mining on the level 1
chain? Perhaps force it into the (soft fork) protocol that anyone mining on
level 2, has to also mine on level 1, and in general, anyone mining on
level n+1 has to also mine on levels n,n-1,...,1. Also, level 1 will have
the best decentralization, so there should be enough people paying fees to
get transactions there for their large transactions that require a high
level of security and trust. Even if people stop using level 1, any bitcoin
you own in the level 1 chain, can be transferred to level 2, and still you
have 1 MB blocks due to the partitioning scheme. How to prevent
transactions from clustering on one or a few sibling chains in a
particular level? Well the more empty chains should have lower fees, so
that should incentivize people... Note: This system also allows for the
fine tuning of the transaction size to security ratio. Yes the lower chains
will be less secure, but a lower sized transaction does not need as much
security.

For instant transactions, there can also be Lightning channels linked to
whatever level of chain you want.

OK so it seems to me that this can work and would only require a soft fork.
But, as I said, I only have a high level understanding of the Bitcoin
protocol, so it feels kind of too good to be true, and I am ready for heavy
criticism. I am only writing this because I care about Bitcoin and I want
it to remain decentralized and become the best that it can be. I think it
is important to do the right thing rather than (as most people naturally
do) the most convenient thing.

[DumbFruit]

An important property of money is fungibility, which means each unit is equivalent to any other. When you break up a chain into sidechains, subchains, "subchains", treechains, mini-blockchains or whatever, you end up introducing differences among different units of the currency. That's the goal with colored coins, but less obvious is that fungibility is damaged even when you aren't marking the coins explicitly.
Those differences carry a different market value, and so they would by traded as different commodities, which is at least one reason it doesn't work very well as a scaling solution.
Put differently, coins on a sidechain, subchain, "subchain", mini-blockchain, treechain, or whatever represent a different financial asset than coins on Bitcoin proper, even if redeemable in bitcoins.

If, on the other hand, you design a system in which each chain really is equivalently secure on the blockchain, and protected from double spends, then you've really just obfuscated a scheme of faster, or bigger, blocks, one way or another, and the standard caveats concerning that applies. That appeared to be what Peter R's "subchains" did, but what do I know? Maybe I misunderstood him.
https://bitcointalk.org/index.php?topic=1320095.msg13492452#msg13492452

There are various ways people have tried to get around this issue, but I haven't seen any that are all that compelling. For instance, Peter Todd says that in TreeChain's you can say that a transaction is safe once the work done on it exceeds the value of the transaction. At the end of the day it's indisputable that not all branches were created equal, and you're assuming different levels of security depending on where your transactions end up on the tree, thereby damaging fungibility. In the real world this would look like; "Oh you want to sell me a bitcoin? That's cool, but I'd like to wait for your branch to get more work on it before I feel comfortable doing that."

So what I'm trying to get at here is if you can somehow break up Bitcoin's transaction history while also preserving it's fungibility, that would be pretty neat.

[onelineproof]

Good to see the old post has a reply

I still want to work on this, just keep getting distracted with other tasks, but I am slowly dipping my hands deeper into the code, so maybe this year I can make some more progress on this.

I don't see any major problems with fungibility as what level of chain you use depends on how much fees you are willing to pay, and this also is directly dependedent on how big in value the transaction is. So if I want to sell you a bitcoin, you just tell me what chain "path" you are on, and I send it to whatever level on that path you want. The higher level, the more fees, just like with a bank or with postal mail, you pay more fees for more security for the transfer.

Sorry for the diagram, I will try to get the link back up. And also, it is annoying that Peter R used the same term "subchains" as me.

[DumbFruit]

I don't see any major problems with fungibility as what level of chain you use depends on how much fees you are willing to pay, and this also is directly dependedent on how big in value the transaction is. So if I want to sell you a bitcoin, you just tell me what chain "path" you are on, and I send it to whatever level on that path you want. The higher level, the more fees, just like with a bank or with postal mail, you pay more fees for more security for the transfer.

Sure, but is that complication, reduction in security, and damage to fungibility, worth the increased throughput? I'm not so sure it is.

"Checkpoints" could be added after a certain block depth, which would sort of "tie up" all the lose chains, but that seems like just increasing block sizes, except messier.

And also, it is annoying that Peter R used the same term "subchains" as me.

Not the first or last annoying thing he's ever done...

[zimmah]

An important property of money is fungibility, which means each unit is equivalent to any other. When you break up a chain into sidechains, subchains, "subchains", treechains, mini-blockchains or whatever, you end up introducing differences among different units of the currency. That's the goal with colored coins, but less obvious is that fungibility is damaged even when you aren't marking the coins explicitly.
Those differences carry a different market value, and so they would by traded as different commodities, which is at least one reason it doesn't work very well as a scaling solution.
Put differently, coins on a sidechain, subchain, "subchain", mini-blockchain, treechain, or whatever represent a different financial asset than coins on Bitcoin proper, even if redeemable in bitcoins.

If, on the other hand, you design a system in which each chain really is equivalently secure on the blockchain, and protected from double spends, then you've really just obfuscated a scheme of faster, or bigger, blocks, one way or another, and the standard caveats concerning that applies. That appeared to be what Peter R's "subchains" did, but what do I know? Maybe I misunderstood him.
https://bitcointalk.org/index.php?topic=1320095.msg13492452#msg13492452

There are various ways people have tried to get around this issue, but I haven't seen any that are all that compelling. For instance, Peter Todd says that in TreeChain's you can say that a transaction is safe once the work done on it exceeds the value of the transaction. At the end of the day it's indisputable that not all branches were created equal, and you're assuming different levels of security depending on where your transactions end up on the tree, thereby damaging fungibility. In the real world this would look like; "Oh you want to sell me a bitcoin? That's cool, but I'd like to wait for your branch to get more work on it before I feel comfortable doing that."

So what I'm trying to get at here is if you can somehow break up Bitcoin's transaction history while also preserving it's fungibility, that would be pretty neat.

well, that might be a potential issue, but is that unsolvable?

I think this idea has a lot of potential to add scaling, and I'd like to see it explored and possibly implemented.