|
Title: Distributed blockchain scaling solution. Example for Bitcoin. Post by: SanaHito on April 26, 2018, 04:48:25 PM Information disclosed in this post is valid as of the time of writing the post.
A few facts to start:
The idea is that instead of each node having the full blockchain, each node stores a fraction of the blockchain. If the current blockchain were split among all of the running nodes, each node would only need to hold 18.54MB of data (194.88GB / 10511). Instead of relying on block explorer services an DNS like protocol would be implemented: Quote 1.The client relays a transaction status query to any node. 2.1. (Case 1) The node has the block body which contains the transaction and relays it back to the client. 2.2.1 (Case 2) The node does not have the block body which contains the transaction. 2.2.2. The node relays the same transaction status query to two other nodes. 2.2.3. Repeat the previous step until the block with the transaction is found. 2.2.4. The client receives the block body and checks if it is valid by comparing the Merkle Root of the block header and Merkle Root of the received block body. There is a big problem with this idea. In reality, each node having a copy of a fraction of the blockchain would be exploitable. Once a node decides to stop being a node, a fraction of the blockchain is lost and transactions cannot be confirmed anymore. An example solution to that problem is that multiple nodes e.g. 100 nodes store the same blockchain fraction. (Chances of all of the 100 nodes leaving the blockchain at the same time is low) but this increases the previously stated blockchain fraction size for each node from 18.54MB to 1854MB/1.854GB which is a big problem. The solution to this is to have as many nodes as possible to reduce fraction of the blockchain each node has to store, but how? Nodes should be paid in newly generated Bitcoins for keeping the network running. (The number of nodes running would increase exponentially) And the final problem with this idea is how to reward nodes for keeping the network running? PoS [1] https://bitinfocharts.com/bitcoin/#tdid23 [2] The size of a Bitcoin header is 80B [a], the current number of blocks is 520000. 520000 * 80B is 41600000B which is 41.6MB [3] https://bitnodes.earn.com/ [a] https://bitcoin.org/en/glossary/block-header Title: Re: Distributed blockchain scaling solution. Example for Bitcoin. Post by: Slava79 on April 27, 2018, 08:58:04 AM Interesting.
What if you have 2 nodes N1 and N2. And 2 users u1 and u2. All the previous transactions of u1 are on node N1 and all the previous transactions of user u2 are on node N2. User u1 creates a transaction to transfer amount A to user u2. Questions: 1. Where to put this transaction, on which node? 2. How these nodes check the UTXO of users u1 and u2 so that there is no double spending, taking into a count both nodes don't trust each other? PS. You may found the following links worth reading: https://en.bitcoin.it/wiki/Scalability https://github.com/ethereum/wiki/wiki/Sharding-FAQ https://cryptonite.info/files/mbc-scheme-rev3.pdf Title: Re: Distributed blockchain scaling solution. Example for Bitcoin. Post by: SanaHito on April 28, 2018, 10:35:35 AM Quote What if you have 2 nodes N1 and N2. And 2 users u1 and u2. Let's say the size of the blockchain is 10K blocks. For simplicity reasons, let's say that N1 stores 50% of the blockchain block body and N2 also stores 50% of the blockchain block body. Besides these nodes storing a fraction of the blockchain blocks' body, they will also store the full blockchain block headers with which they can verify if the blockchain block body they received is valid. Quote All the previous transactions of u1 are on node N1 and all the previous transactions of user u2 are on node N2. User u1 creates a transaction to transfer amount A to user u2. 1. Where to put this transaction, on which node? 2. How these nodes check the UTXO of users u1 and u2 so that there is no double spending, taking into a count both nodes don't trust each other? 1. In a peer-to-peer network like Bitcoin, it is only needed to send the transaction to one node who will broadcast it to the rest of the network. 2. In principle, N1 could, by request of 2 or any other user say that U1 sent 10x their actual amount but this can be detected. Since all nodes have a copy of the blockchain block header which does not take up much space, they can verify if the block body is valid or if it was changed by comparing the Merkle Root on the block header with the received block body's Merkle Root. In this kind of system, full blockchain nodes would also be possible and they would speed up the system. A normal user has no reason to run a full blockchain node but big companies such as Coinbase would need to have these nodes running because it would speed up the time it requires to check if a payment is valid and it would use less bandwidth on their system. I would love to create a proof of concept of this, I believe it would spark a lot more interest but I lack free time and coding skills. This may be something I could release in the future. Title: Re: Distributed blockchain scaling solution. Example for Bitcoin. Post by: SanaHito on April 28, 2018, 07:39:57 PM TBH, it's bad idea and it needs lots of effort to make this possible. There are few difficult things such as : 1. If nodes only have fraction of block, they couldn't verify whether the input is still unspent. 2. If nodes connect to too many peers, it would affect nodes performance. 3. How to pay each nodes with fair method while keep decentralization? At best, i could only think similar method with PoS. Also, i think this would require lots of internet bandwitch since nodes need to broadcast transaction many times while they can't even verify whether the transaction is valid/not, it can be abused to attack the nodes with invalid transaction and this will make lots of delay or latency in network. I think simply keep current method (full nodes with whole blockchain copy, prune mode with list of UXTO/partical blockchain copy and SPV only with tx data user need) is better. Unfortunatelly it's hard to find people who would support this/help develop. It is just an idea and has lots of holes. I personally believe that it is possible to develop such a network while still maintaining scalability, speed, latency and most importantly, security. I believe if this would be developed it would be a huge leap in blockchain technology. I hope I will be able to make a paper on this in the future and maybe even a PoC code, something like when Bitcoin was created. And yes, PoS is the way to go. I am full of ideas in the PoS architecture and hope to develop a PoC currency in the near future while in college. Thank you for your posts. I really appreciate everyone's thoughts about this. Title: Re: Distributed blockchain scaling solution. Example for Bitcoin. Post by: streamplus on April 28, 2018, 07:40:49 PM So, it's good starting but this actual idea go nowhere.
Did you learned about Bloom Filter ? It compress the blockchain to have approximativally 300Mo (This is a good solution for the problem). Title: Re: Distributed blockchain scaling solution. Example for Bitcoin. Post by: Slava79 on April 28, 2018, 09:23:33 PM So, it's good starting but this actual idea go nowhere. Did you learned about Bloom Filter ? It compress the blockchain to have approximativally 300Mo (This is a good solution for the problem). Could you please provide more details on how bloom filter can compress blockchain? Title: Re: Distributed blockchain scaling solution. Example for Bitcoin. Post by: streamplus on April 28, 2018, 10:43:01 PM So, it's good starting but this actual idea go nowhere. Did you learned about Bloom Filter ? It compress the blockchain to have approximativally 300Mo (This is a good solution for the problem). Could you please provide more details on how bloom filter can compress blockchain? You can find more infos here: https://ethereumclassic.github.io/blog/2017-02-10-bloom-filters/ Title: Re: Distributed blockchain scaling solution. Example for Bitcoin. Post by: d5000 on April 28, 2018, 11:07:01 PM The idea is that instead of each node having the full blockchain, each node stores a fraction of the blockchain. This idea is not new. In fact, even Bitcoin implemented a "pruning" solution where the nodes can opt to store only a part of the blockchain - even if in Bitcoin's case the stored fraction is the same one.If the current blockchain were split among all of the running nodes, each node would only need to hold 18.54MB of data (194.88GB / 10511). In Ethereum, a "sharding" mechanism is developed which works a bit like the idea you propose. However, the problems you mention are not trivial to solve, and redundancy must be big enough to ensure that no transaction (more general: no relevant portion of the "state" of the blockchain) is lost. The way Ethereum wants to do it can be read in their Sharding FAQ (https://github.com/ethereum/wiki/wiki/Sharding-FAQ). Afaik they are, however, far away from a release (even if they don't admit it ;) ). The advantage Ethereum has is that it is account-based, so it may be easier to part the blockchain into "shards" than Bitcoin as account balances are also stored, not only transactions and block headers. There was also a sharding proposal for Bitcoin called the Merklix Tree (http://www.deadalnix.me/2016/11/06/using-merklix-tree-to-shard-block-validation). |