What is sharding, and will sharding scale Bitcoin?

[aliashraf]

But instead of "micro-payment shards", why not a micro-payment, off-chain layer developed, and built on top of the Bitcoin blockchain?

I believe there is already one, called the Lightning Network.

Because it is not bitcoin, and bitcoin is the only monetary system that is truly decentralized and censorship resistant. It is very easy to have 2nd layer applications that do excellent job on top of bitcoin but they involve either trust-->centralization or liquidity-->centralization just like LN.

It is more than obvious that an onchain solution is superior to any 2nd layer one no matter how fancy it would be. It is no magic, LN, you deposit some coins and bitcoin acts as an escrow thanks to its HTLC feature, the LN node does the same and you both now have a channel. When it comes to arbitrary trades the LN node needs to deposit even more with other nodes to act as a router and it escalates liquidity problem and puts people with more deposits in a central hub position eventually, it resembles traditional banking systems rather than cryptocurrency.

I understand LN now looks like an adventure but it is not bitcoin and it is very important to keep investigating onchain scalability solutions instead of saying:
"hey, LN solves everything why should we bother even thinking about improvements? Isn't it just more sensible to keep bitcoin safe and uncompromised and focus on LN?"
It is what they say, by the way, and it is where they are diverging from bitcoin without even being aware of their divergence! And it is sad. A decade of experiments and discussions and debates is just going to be denied because some guys are not good in strategy in spite of their expertise in math or coding.

[1714973444]

1714973444

[Wind_FURY]

But instead of "micro-payment shards", why not a micro-payment, off-chain layer developed, and built on top of the Bitcoin blockchain?

I believe there is already one, called the Lightning Network.

Because it is not bitcoin,

Stop. Why is it "not Bitcoin", in your opinion? Because Bitcoins in Lightning channels are Bitcoins under multi-sig escrow, with transactions stored locally, that can be broadcasted on-chain at a later date, or time.

[HeRetiK]

[...]

I understand LN now looks like an adventure but it is not bitcoin and it is very important to keep investigating onchain scalability solutions instead of saying:
"hey, LN solves everything why should we bother even thinking about improvements? Isn't it just more sensible to keep bitcoin safe and uncompromised and focus on LN?"

[...]

I think the nice thing about LN is though that it improves scalability without making any sacrifices on the lower layer.

That is, it does leave room for alternative scalability approaches as well. So while there's a lot of focus on LN right now I don't think that in practice it detracts that much attention from possible alternatives. There will be dev teams dedicated to other approaches just like dev teams are dedicated to LN right now. Maybe these approaches will end up as part of Bitcoin, maybe they won't. Point being, while many people may see LN as the scaling solution to end all scaling problems, many people don't. And those will still keep working on other ways to improve crypto.

[Wind_FURY]

[...]

I understand LN now looks like an adventure but it is not bitcoin and it is very important to keep investigating onchain scalability solutions instead of saying:
"hey, LN solves everything why should we bother even thinking about improvements? Isn't it just more sensible to keep bitcoin safe and uncompromised and focus on LN?"

[...]

I think the nice thing about LN is though that it improves scalability without making any sacrifices on the lower layer.

That is, it does leave room for alternative scalability approaches as well. So while there's a lot of focus on LN right now I don't think that in practice it detracts that much attention from possible alternatives. There will be dev teams dedicated to other approaches just like dev teams are dedicated to LN right now. Maybe these approaches will end up as part of Bitcoin, maybe they won't. Point being, while many people may see LN as the scaling solution to end all scaling problems, many people don't. And those will still keep working on other ways to improve crypto.

Thank you for sharing your invaluable opinion. But what's your opinion on aliashraf's proposal of a low-security shard for Bitcoin micro-payments?

Plus in the context of that, wouldn't an off-chain layer for micro-payments be a more feasible solution?

[spartacusrex]

The way Sharding is being 'attempted' on ETH only works for POS.  It does not work for POW.

The Reason :

They use sub-sampling of the POS stake holders to limit which accounts can mine which shard. It is impossible for more than the the allocated POS miners to mine a shard than the percentage allocated.

So 10% of the pos holders can mine each 10% shard. This changes every block. So you are randomly allocated a different shard to work on.

You can't do this in POW as there is no way of preventing a POW miner from adding his hash-rate to the shard of his choice. Subjective vs Objective miners.

I think this is a clever trick! But it does not fix the numerous other outstanding issues.

Clearly you MUST be allowed to perform transactions cross shard - or you destroy fungibilty. I have looked at the cross sharding communication algo.. err.. LOL! It's very complicated and messy. I still need to be shown _proof_  that this will in fact reduce traffic and not increase it.

The Security of the Shard-POS chain running the original ETH algo can be copied _exactly_. (As with ANY POS chain) In fact I can increase the security by simply increasing the distribution of stake. I can also charge less for gas.  Why would a new user choose a more expensive less secure chain ?  (You can't do this with POW)

Anyone who thinks this Hierarchical Tree Structure doesn't centralise faster than Usain Bolt - Is missing something.

I wish them Luck! I love seeing cool teck _attempted_

..

(I'm way way down the rabbit hole myself at the moment..)

[aliashraf]

Clearly you MUST be allowed to perform transactions cross shard - or you destroy fungibility.

I don't think so. On the contrary cross shard transaction is exactly what you should prohibit to avoid complexities.
In my scheme, we keep any segment of the state self-contained and to resist state take-over attacks we have higher level shards in charge.
Keeping coins on same shard and avoiding cross-shard transactions would be easy by applying the two rules I've proposed before:
1- Transactions should use inputs from same shard.
2- Using a light nonce, wallets should guarantee the transactions they generate falling in the same shard with their inputs (which follow rule #1 any way).

Belonging to different shards do not make two utxos any different because they all belong to and are maintained by the root shard eventually. You can think of the root shard as a very slow, yet heavy blockchain with gigabyte blocks (like 4 GB) and long blocktimes (like 1-2 hours). Bitcoin using such a scheme scales up to 1500 tps and more.

Anyone who thinks this Hierarchical Tree Structure doesn't centralise faster than Usain Bolt - Is missing something.

Although the name hierarchical resembles such thing, it is not what it looks like in the first glance:
It is absolutely possible to make it hard enough for upper shards (maintained by fewer full nodes because of the costs) to do anything bad against the network by applying a new technique: collaborative proof of work. In my model, upper level shards are free to accumulate works done in their subshards as part of their proof.

To commit any sort of misbehavior ( intentional chain rewrites for double-spending, censorship attempts, ... ) they need to collide with their subshards or ignore the work they have encapsulated in their blocks.
Any miner in any shard has to provide a block with a total T work. To do this, the miner needs a nonce for the block header with T/2 difficulty and the Merkle root of the header should refer to two blocks from its left and right subshards with a total difficulty of at least T/2.
It is a bit more complicated and yes, it is really, really going down the rabbit hole. For instance it is not just about a subshard's single block difficulty, it is about a chain of blocks with unknown length that are being committed to an upper level block, and it is obviously recursive process, ...

Sharding is not simple problem but the solution should be kept as simple as possible it is why I don't like what they are doing in Ethereum, for now I'm becoming convinced that tree chain and hierarchical sharding is the most go direction.

[bittraffic]

Increasing the TPS means can be a security threat and we see sharding can be a solution but will sharding also be a security threat to a blockchain?

I think it was vitalik that said blockchain is meant to be slow like 300K transaction. If a blockchain can process more than 10K per second, it may have some security issues.

[spartacusrex]

Hello Ali!

Clearly you MUST be allowed to perform transactions cross shard - or you destroy fungibility.

I don't think so. On the contrary cross shard transaction is exactly what you should prohibit to avoid complexities.

This is already done using altcoins. And for POS chains the security is stronger the wider the distribution of coins. I see no Security benefit to pooling POS coins. (POW chains are a different matter)

If you can't communicate cross chain, I would say it is preferable to have them separate - they can already communicate using smart contracts / HTLC / Layer2 etc.

[aliashraf]

Hello Ali!

Hi bro, we miss you here, why shouldn't you visit more frequently?

Clearly you MUST be allowed to perform transactions cross shard - or you destroy fungibility.

I don't think so. On the contrary cross shard transaction is exactly what you should prohibit to avoid complexities.

This is already done using altcoins. And for POS chains the security is stronger the wider the distribution of coins. I see no Security benefit to pooling POS coins. (POW chains are a different matter)

If you can't communicate cross chain, I would say it is preferable to have them separate - they can already communicate using smart contracts / HTLC / Layer2 etc.

I know, thinking of altcoins as shards with no cross-shard transactions is so common but I think it is not just like that. Altcoins (and I'm talking about PoW coins) are different coins and as you've reminded above they are not fungible like a huge ultra coin. Fungibility is not a technical challenge to be solved by any means e.g. Atomic Swaps, it is about different values.

A PoW coin's value is basically determined by the average social work necessary to produce it (the costs of mining each unit). Labour Theory of Value, proposed by Karl Marx more than a century and half ago, clearly describes why and how a Pow coin is valuable (and a PoS coin is not by the way  ) and fungibility essentially is about coins to be of same value,

The beauty of a hierarchical scheme is its potential to take care of difficulty and fair generation of coins hence keeping the value of coins in any shard absolutely the same at any given time and making it irrelevant and a matter of protocol and not users' concern that a utxo belongs to which shard. What a user wants is the mere transfer of funds.

[HeRetiK]

I think the nice thing about LN is though that it improves scalability without making any sacrifices on the lower layer.

That is, it does leave room for alternative scalability approaches as well. So while there's a lot of focus on LN right now I don't think that in practice it detracts that much attention from possible alternatives. There will be dev teams dedicated to other approaches just like dev teams are dedicated to LN right now. Maybe these approaches will end up as part of Bitcoin, maybe they won't. Point being, while many people may see LN as the scaling solution to end all scaling problems, many people don't. And those will still keep working on other ways to improve crypto.

Thank you for sharing your invaluable opinion. But what's your opinion on aliashraf's proposal of a low-security shard for Bitcoin micro-payments?

Plus in the context of that, wouldn't an off-chain layer for micro-payments be a more feasible solution?

To be honest I haven't looked deep enough into blockchain sharding yet to give an educated opinion on aliashraf's proposal or any other sharding proposal for that matter. Additionally given the conservative nature of Bitcoin's development I have my doubts that Bitcoin will ever see a sharding implementation to begin with (for better or worse; except for hard forks by alternative dev teams, I suppose). In that regard I guess the closest that Bitcoin will ever come to a sharding-like approach is via sidechains.

In general I believe that 2nd layer solutions stand to scale way better than sharding. Main reason being that everything else being equal sharding only scales linearly with the amount of shards (ie. to increase transaction throughput tenfold you need 10 shards, to increase transaction throughput by n you need n shards) while in theory LN is only limited by the respective nodes' underlying internet connection and the laws of physics.

That being said, I also believe that on-chain transactions offer a better user experience than LN transactions due to the almost-always-online requirement of the latter. Whether on-chain transaction throughput can be sufficiently increased via sharding without significant sacrifices is a different matter however (and one where I still have a lot of reading to do).

[Wind_FURY]

I think the nice thing about LN is though that it improves scalability without making any sacrifices on the lower layer.

That is, it does leave room for alternative scalability approaches as well. So while there's a lot of focus on LN right now I don't think that in practice it detracts that much attention from possible alternatives. There will be dev teams dedicated to other approaches just like dev teams are dedicated to LN right now. Maybe these approaches will end up as part of Bitcoin, maybe they won't. Point being, while many people may see LN as the scaling solution to end all scaling problems, many people don't. And those will still keep working on other ways to improve crypto.

Thank you for sharing your invaluable opinion. But what's your opinion on aliashraf's proposal of a low-security shard for Bitcoin micro-payments?

Plus in the context of that, wouldn't an off-chain layer for micro-payments be a more feasible solution?

To be honest I haven't looked deep enough into blockchain sharding yet to give an educated opinion on aliashraf's proposal or any other sharding proposal for that matter. Additionally given the conservative nature of Bitcoin's development I have my doubts that Bitcoin will ever see a sharding implementation to begin with (for better or worse; except for hard forks by alternative dev teams, I suppose). In that regard I guess the closest that Bitcoin will ever come to a sharding-like approach is via sidechains.

Bitcoin will never see any sharding proposal implemented. Not especially a proposal that allows a part of the blockchain to be less secure for micro-transactions.

I made this topic more as an inquiry to learn more about "sharding", and to memefy sharding in Bitcoin once we truly understand what it is.

[HeRetiK]

Bitcoin will never see any sharding proposal implemented. Not especially a proposal that allows a part of the blockchain to be less secure for micro-transactions.

Regardless of that and my own personal doubts about sharding as a scaling approach I'm looking forward to see it in the wild. It seems to be quite an enormous task though, seeing how Ethereum's roadmap is currently placing sharding support at 2021 [1], but that might be mostly due to the challenges of upgrading a running system. That is, it might be different for a currency starting with sharding from scratch.

Anyone aware of any alts that use sharding?


[1] https://www.mangoresearch.co/ethereum-roadmap-update/

[buwaytress]

Bitcoin will never see any sharding proposal implemented. Not especially a proposal that allows a part of the blockchain to be less secure for micro-transactions.

Regardless of that and my own personal doubts about sharding as a scaling approach I'm looking forward to see it in the wild. It seems to be quite an enormous task though, seeing how Ethereum's roadmap is currently placing sharding support at 2021 [1], but that might be mostly due to the challenges of upgrading a running system. That is, it might be different for a currency starting with sharding from scratch.

Anyone aware of any alts that use sharding?


[1] https://www.mangoresearch.co/ethereum-roadmap-update/

I only know of Ziliqa's sharding efforts already on mainnet. I've a soft spot for projects in my region, even if only to keep up with them on a cursory level. In fact, it was by some work circumstance I'd heard of it even before I'd heard about Ethereum's plans or even plasma... whether or not either implementation will ever be comparable to what may ever be considered for Bitcoin, I couldn't say.

Warning: that article's a bit of a shill but I suppose this is as good as it gets for a mature-ish implementation. Ziliqa's all about payments (almost all hot projects out of Singapore are now!) so let's see.

[Wind_FURY]

Bitcoin will never see any sharding proposal implemented. Not especially a proposal that allows a part of the blockchain to be less secure for micro-transactions.

Regardless of that and my own personal doubts about sharding as a scaling approach I'm looking forward to see it in the wild. It seems to be quite an enormous task though, seeing how Ethereum's roadmap is currently placing sharding support at 2021 [1], but that might be mostly due to the challenges of upgrading a running system. That is, it might be different for a currency starting with sharding from scratch.

Anyone aware of any alts that use sharding?


[1] https://www.mangoresearch.co/ethereum-roadmap-update/

Me too, and especially in Ethereum. They have been promoting it as "scaling out", but with Ethereum's block size depending on gas limits, it might actually be scaling in.

I believe sharding is only a temporary solution for the inevitability of centralization of blockchains that utilizes big blocks.