Block lattice project

[clemahieu]

I’m hoping people would have some feedback on a project I’ve been working on.  The issue I was trying to solve was scalability: transactions per second and confirmation speed.  What I ended up with is a system similar to side-chains except it’s taken to the extreme where each account manages its own chain and all chains are replicated to all nodes.  What happens with this system is there are virtually no distributed agreements because each account has authoritative control over its own chain.  With the account owner having authoritative control over their own chain, transactions don’t need to be mined for validity so confirmation time drops to near zero and transactions speed is as fast as they can be published.

I was looking for some comments, good, bad, indifferent on what people think.

For gritty details I did a writeup here https://github.com/clemahieu/raiblocks/wiki/Block-lattice which includes information about double-spending

[1715319976]

1715319976

[1715319976]

1715319976

[1715319976]

1715319976

[1715319976]

1715319976

[1715319976]

1715319976

[1715319976]

1715319976

[DrGrid]

From what I have understood so far, this is not really a blockchain, or even crypto related concept, in fact it sounds to me like you have proposed a new standalone transaction system. I don't really see why you think that this concept is similar to sidechains, it rather sounds like a disitributed transaction server, with no need of a native currency. I saw you don't have comments in the source code, so I won't be spending much time for now to figure out what goes where.
How is this different from any off chain transactions?
You claim this is double spending proof, how do you collect proof form what you call an account, that he is overseeing his chain in a way the network deems ethical?
 

[clemahieu]

I called parts of it a block chain since the send/receive blocks contain a hash of the previous block and it has a one-follows-another order.  It's different than other cryptos in that one block is one transaction, a send or a receive, where others contain many transactions.

Thanks for spending the time you did looking at it, sorry about the lack of comments that's admittedly in poor style.

These differ from off-chain transactions in that each chain is replicated to all peers so all account actions are observed and validated by everyone.  Rather than packing all transactions in to a single chain, all transactions for a single account are in a single chain.

On the double spending front, since all blocks for each chain are replicated to everyone, nodes can ask and publish their view of signed Y-follows-X blocks.  If anyone instead sees a signed Z-follows-X block this is a fork and needs resolution.  Unlike monolithic blockchains, a fork in this system only affects a single account instead of everyone, again working toward scalability where we don't stall everyone because one section has an issue.

Resolution is done through a weighted vote system where losing blocks and dependents are rolled back.  Senders can be sure they're not receiving a double spend by observing votes and tallying them, if they observe no forks or a resolved fork they can safely accept the transaction in to their chain. Notice though that resolution is never needed in the normal, well-behaved case.  Forks only happen with signed, conflicting blocks; only accounts that are misbehaving by intentionally creating forks will every be put up for resolution.

Let me know if any of what I wrote doesn't make sense.

[DrGrid]

I called parts of it a block chain since the send/receive blocks contain a hash of the previous block and it has a one-follows-another order.  It's different than other cryptos in that one block is one transaction, a send or a receive, where others contain many transactions.

Thanks for spending the time you did looking at it, sorry about the lack of comments that's admittedly in poor style.

These differ from off-chain transactions in that each chain is replicated to all peers so all account actions are observed and validated by everyone.  Rather than packing all transactions in to a single chain, all transactions for a single account are in a single chain.

On the double spending front, since all blocks for each chain are replicated to everyone, nodes can ask and publish their view of signed Y-follows-X blocks.  If anyone instead sees a signed Z-follows-X block this is a fork and needs resolution.  Unlike monolithic blockchains, a fork in this system only affects a single account instead of everyone, again working toward scalability where we don't stall everyone because one section has an issue.

Resolution is done through a weighted vote system where losing blocks and dependents are rolled back.  Senders can be sure they're not receiving a double spend by observing votes and tallying them, if they observe no forks or a resolved fork they can safely accept the transaction in to their chain. Notice though that resolution is never needed in the normal, well-behaved case.  Forks only happen with signed, conflicting blocks; only accounts that are misbehaving by intentionally creating forks will every be put up for resolution.

Let me know if any of what I wrote doesn't make sense.

So a block is taken as a single tx id with timestamp? If so this seems to fall into the category of Transactions as Proof of Work. I still see a  number of challenges with this system. First up of course consensus; at what point can a transaction block be viewed as verified and have an acceptable level of cryptographic confirmations inside a network with an unknown number of peers. I actually see hard-coded rounds of confirmations as a possibility for this. Secondly, how do you make the attack you have just described as costly as possible. Since the Bitcoin community is always very passionate about strong security, that is so far unmatched by anything else, I am actually particularly intrigued by this point. Of course we can look at typical solutions like including a transaction fee, expanding the proofhash to multiple layers and reward good actors for their work.

[Fuserleer]

I looked at this yesterday, and I'm glad to finally see a real thinking, stepping away from damn blockchains and wasteful Proof of Work!   So well done, and immediate respect!

Your "Block Lattice" is similar to our channeled ledger aka Transaction Forest, except you have many chains of transactions, we have many trees of transactions.  The name you chose is confusing though, as if each transaction acts in the same manner as in ours, there are no need for blocks at all.  If so, drop the blocks reference, step away from the crowd, it's confusing

At that point the similarities stop, perhaps because its a young project, or perhaps because you haven't hit a number of issues yet that you WILL need to solve.  

First and foremost, you NEED a consensus algorithm!  Nodes that are claiming the delta, will be susceptible to bad actors at each end of the network attempting to claim the same spend twice, and unless you have managed to find a way around network latencies et all and ensure that ALL nodes forward transactions honestly, you will have 2 transactions in the network attempting to claim the same spend twice.  

Secondly, you NEED a consensus algorithm!  Nodes that are spending, will be susceptible to bad actors at each end of the network attempting to spend past a zero balance, and unless you have managed to find a way around network latencies et all and ensure that ALL nodes forward transactions honestly, you will have 2 transactions in the network attempting to spend past zero.

Unless I missed something when I looked at the code (admittedly, I skimmed through), this is something you haven't solved IMO, you mention voting but this isn't really explained, and you can not rely on the transactions creating X->Y links themselves to be the authority when resolving forks/double claims and similar attacks.  Also, who signs these block and how are they selected?

That said, very commendable!  Anyone that thinks out of the box (like me ) I have a lot of respect for, and I'd love to chat to you more in a private manner.  eMunie is going to need some additional smart thinkers soon, maybe there is a place for you on the team.

[clemahieu]

Thanks for taking a second look guys, I took your questions to heart and thought the best way to explain was to create a couple graphics of how things work.



The transaction animation shows the typical, non-attack scenario how transfer of balances occurs.  The contents of these blocks are almost exactly how it's done in the code, minus signatures.

The animation is:
1) Create send block linking previous head block and with a new account balance < old balance
2) Create receive block linking previous head and linking source block ensuring source hasn't been received already.



The fork animation shows what most seem to be interested in, consensus of bad-actors.
1) One or more signed blocks pointing to the same previous blocks are created and distributed to different nodes in the network
2) Nodes redistribute blocks they've just seen and soon everyone in the network has observed a fork
3) Nodes initiate voting by broadcasting their vote and tallying the votes of others
4) Nodes who are voting for a block with <50% of the tally change their vote
5) Consensus is achieved after a few rounds of voting

Going with the theme of efficiency and scalability we tried to making resolution as easy and efficient as possible.  We notice people marked as a receiver can keep their own chain separate from a bad actor trying to send forks to them, this way only the bad actor is stalled and the receiver can wait out the resolution until it's definitive.  This goes to the scalability where we stall only the people doing things the wrong way.

It's important to note the time window for a fork block getting anything other than a 0% vote is less than one network propagation interval.  Nodes in RaiBlocks rebroadcast their transactions so once everyone has received a copy, any new fork will get no votes.

I added the animations to the main description pages in the wiki
https://github.com/clemahieu/raiblocks/wiki/Block-lattice
https://github.com/clemahieu/raiblocks/wiki/Double-spending-and-confirmation

[Fuserleer]

Thanks for the images, it clears up and confirms what I was thinking.

I understand totally how the transfer of balances occur, as I said before, its very similar to our method.

You consensus method is also how I expected, and I don't believe your method will work for 2 reasons:

1.  Sybil attacks:  It seems very trivial for me to create a large quantity of nodes that can vote on whatever I like, and there doesn't seem to be any mechanism to prevent it, every node seems to have a vote, period.  ( I notice REP in your animation tho, what is this, node reputation?  If so where does that reputation come from?)   So I pay someone at one end of the network, it gets accepted, I also pay someone at the other end of the network, which *may* gets accepted if I'm quick enough.  Voting happens, all of my nodes then vote for the 2nd transaction and that is the one that is honored.  In real world, Ive may have just spent them twice, as the first payment was initially accepted and the merchant/receiver/whatever gave me what I paid for ( assume I was at a store paying for some goods and left immediately).

2.  Efficiency:  Perhaps I misunderstood, but when voting happens, these votes are broadcast around the network?  How can you be sure that everyone receives (and honors) the votes they receive, and broadcasts them on to the nodes they are connected to so that they can also do the tally?  This seems like a mammoth task orchestrating all those votes over and over, and will certainly push bandwidth use up if that really is the case.

Thanks again

[Fuserleer]

Also this line:

4) Nodes who are voting for a block with <50% of the tally change their vote

What if they don't?  And why are they forced to do so?  What if the block they are voting for is the correct block, and everyone else voting up the incorrect one are dishonest?

If the incorrect block makes it through naturally then so be it, but I dont think you should force nodes to change their vote just because they aren't batting for the majority block.  Plus you can't even be sure that nodes are changing their vote, unless you have every node review all votes of all nodes in the system.  Then you get into the realms of O(n) work/bandwidth loads.

[Fuserleer]

Good to have you back Fuserleer  My personal opinion, you're doing something better than Satoshi Nakamoto and I know that's heresy around some parts but hey what can I say 

Can't wait for the launch.

Thanks for the kindness, much appreciated  

But lets not derail this thread from the good work this chap is also doing 

[bitcoinpaul]

I looked at this yesterday, and I'm glad to finally see a real thinking, stepping away from damn blockchains and wasteful Proof of Work!   So well done, and immediate respect!

Hey hey, don't forget simcoin.

[clemahieu]

To prevent a Sybil attack, votes are weighted based on account balance, the only way to have a larger vote is to have more value stored in the the protocol. People voting against the protocol are voting to destroy the value they hold in the network I.e. Burning your own money.

Rep in this context is representative. The concept is to address your good second point, votes flying around the network. Each account can designate another account as their representative and the rep can vote with but not spend with their balance weight. Also note that voting is not initiated unless a fork is actually observed.

Nodes not flipping their votes is analogous to Bitcoin nodes not taking the longest branch, it's not really Bitcoin anymore. Nodes trying to publish blocks from orphaned forks will be rejected because the previous hash value is not a valid block and receivers won't accept these blocks because then their chain would be invalid.

[Fuserleer]

To prevent a Sybil attack, votes are weighted based on account balance, the only way to have a larger vote is to have more value stored in the the protocol. People voting against the protocol are voting to destroy the value they hold in the network I.e. Burning your own money.

Rep in this context is representative. The concept is to address your good second point, votes flying around the network. Each account can designate another account as their representative and the rep can vote with but not spend with their balance weight.

Nodes not flipping their votes is analogous to Bitcoin nodes not taking the longest branch, it's not really Bitcoin anymore. Nodes trying to publish blocks from orphaned forks will be rejected because the previous hash value is not a valid block and receivers won't accept these blocks because then their chain would be invalid.

Ok, so the weight of a vote is as per the stake of the voter, makes sense, that will work to a degree.

I'm not getting the representative part (or the point of it):

Alice has a balance of 10
Bob has a balance of 20
Tom has a balance of 100

Do Alice, Bob and Tom pick one of the other to be the representative of their vote?  So for example, Alice picks Tom, and Tom picks Alice, then Alice's vote holds more weight than Toms?  What if everyone picks Tom?  Alice and Bob could be dishonest, pick Tom as their rep, they control the network, even though Tom is the honest one but cant vote with his weight.

Perhaps I'm wrong in my understanding of this representative voting due to lack of detail.

Also note that voting is not initiated unless a fork is actually observed.

You'll get forks, and plenty of them, there will always be some asshole trying to cheat

[clemahieu]

I think you're describing it accurately.   Any account can be a representative and accounts can change their rep at any time.   If accounts named each other as you described they would have vote weight as you described.

*edit* I think the description of representation was backwards. If Alice and bob name Tom as their rep, Tom has all the vote strength and Alice and bob have none even though Alice and bob have a balance. Representation is giving weight equal to your balance.

One thing to note is that voting only happens if there is a signed conflict so the easiest way for your account to not be susceptible to collusive votes is to not sign forks; only cheaters are voted on. Collusion of voting is analogous to collusion of miners in Bitcoin. The difference is how much does it cost. With mining the cost of getting >50% strength is the capital investment in hardware to mine that fast. With voting collusion the cost of getting >50% control is the market cap of the entire currency.  Market cap of currency > capital investment of mining hardware and it isn't susceptible to technology advances e.g. Gpu -> asic so I feel this is a stronger guarantee than mining.

[NxtChg]

Read most of your stuff, this is indeed very similar to Simcoin, down to UDP, Ed25519, C++ code and even some function names, like app_path() Spooky.

Anyway, this part got me confused:

"Unlike existing cryptocurrency systems, this rollback only affects the account involved in the fork instead of affecting all transactions during a time period."

If Alice creates a block to send to Bob. And then Bob creates a block to accept it, you now can't rollback Alice's block without also rolling back Bob's.

Now, what if Bob already sent money to Charlie? Does this mean that his second transaction must be rolled back too, simply because his account chain is now broken?

[clemahieu]

It's a good observation. The "accounts involved" in the context you described would roll back the send to Charlie in bob's chain as well as the receive from Alice in bob's chain. And finally the losing fork in Alice's chain.

Two things to notice here is that all the thousands of other accounts uninvolved aren't affected at all, this is the advantage of the individual chains.

The other thing to notice is bob could guard against having his chain rolled back by allowing time for him to observe possible forks. How long does he have to wait?  About one network propagation period because by then, someone somewhere would have rebroadcast the block and if they haven't, the fork will have a very low vote percentage because everyone's already agreed on the first block.

The worst case scenario for forks is creating n forks and distributing it to n representatives with close to equal vote strength simultaneously. Then bob should wait until the vote total is statistically definitive in his direction, >95% or a heuristic before accepting it.

Bob can still continue accepting from non-forks and sending to others even if he is named as the recipient of a forked block. Bob could also choose to never accept the forked block if he wanted.

[NxtChg]

It's a good observation. The "accounts involved" in the context you described would roll back the send to Charlie in bob's chain as well as the receive from Alice in bob's chain. And finally the losing fork in Alice's chain.

Yeah, all the other stuff was already clear

It's just that that particular statement should probably be rephrased - the rollback is not limited to a single account in a strict sense.

I will think more about all this, but for now one thing seems clear - a system like this has a very low tolerance for problems that cause rollbacks. I understand all your points about network propagation, I've been working on a similar system for 9 months after all

What you need to account for is attacks and network problems. What if half of your representative nodes got DDoSed? What if a large part of the network gets disconnected?

Under perfect conditions it's easy to make things work, even Ripple thought they had it. And now they admit their consensus is broken.

Decentralized consensus is hard

[Fuserleer]

It's a good observation. The "accounts involved" in the context you described would roll back the send to Charlie in bob's chain as well as the receive from Alice in bob's chain. And finally the losing fork in Alice's chain.

Decentralized consensus is hard

Here here, we've been through 3 different algorithms in the past 18 months before we found the right one that meets our needs and is solid!

[clemahieu]

These are great points. One thing nodes will be able to do is track the total weight of votes that have been received for a forked block. If this is noticeably lower than the total supply indicating disconnection of representatives, nodes can leave forked sends unreceived until they're satisfied the block has enough integrity for them to accept it.  This would be analogous to observing hash rate significantly dropping indicating miners being disconnected. If representatives are being DDOSed they can choose to not touch forks so well behaved accounts have no problems and accounts generating forks are unable to transact.  

If the network gets partitioned I think the problem would be equivalent to if the Bitcoin network somehow got partitioned. The partitioned segments would be continuing on their own sections until the networks merge and one side gets evicted. One thing to notice though is all it takes is one node connected to both partitions for them to begin to merge.

It would be incredibly hard though hypothetically feasible for the network to be perfectly disjointly partitioned and one fork to be placed in exactly one partition. In this case we'd have to fall back on observing vote totals dropping indicating massive disconnection.

Heuristic suggestions are welcomed!

It's recommended that people pick reliable and diverse representatives, universities, enthusiast groups, corporations, placed on platforms designed to deal with all the problems any popular network service needs to deal with.

[NxtChg]

...solid...

You didn't forget to cross your fingers while saying it, did you?

--

@clemahieu, as much as I doubt Fuserleer's sanity, one thing he was spot on is that the core of your system is not how exactly you arrange your ledger or how accounts will interact with each other.

The core is consensus - it's an age-old and difficult problem, that's why everybody praises Satoshi so much - he was able to find an original solution for that problem.

We, the 2nd and dare I say 3rd generation coins, are trying to solve it via PoS and some clever trickery.

Right now you don't seem to understand that consensus is the main problem and think of it more as a secondary feature to help you solve some border cases.

It isn't. It's a thing that will drive you mad in 3 months.

--

"It's recommended that people pick reliable and diverse representatives, universities, enthusiast groups, corporations"

Right... Now you're reinventing Ripple with their Unique Node Lists. We've already been there

Sorry to kinda rain on your parade, but the sooner you realize what your main enemy is, the sooner you will be able to focus on it and start looking for solutions.

[Fuserleer]

...solid...

You didn't forget to cross your fingers while saying it, did you?

--

@clemahieu, as much as I doubt Fuserleer's sanity, one thing he was spot on is that the core of your system is not how exactly you arrange your ledger or how accounts will interact with each other.

The core is consensus - it's an age-old and difficult problem, that's why everybody praises Satoshi so much - he was able to find an original solution for that problem.

We, the 2nd and dare I say 3rd generation coins, are trying to solve it via PoS and some clever trickery.

Right now you don't seem to understand that consensus is the main problem and think of it more as a secondary feature to help you solve some border cases.

It isn't. It's a thing that will drive you mad in 3 months.

--

"It's recommended that people pick reliable and diverse representatives, universities, enthusiast groups, corporations"

Right... Now you're reinventing Ripple with their Unique Node Lists. We've already been there

Sorry to kinda rain on your parade, but the sooner you realize what your main enemy is, the sooner you will be able to focus on it and start looking for solutions.

Hey, leave my sanity (or lack of) out of this will ya :p

I do believe that our consensus is (now) solid, we've been through it many times (as a group), smart members of our community have spent extended periods of time with me at my home and we've ripped it to pieces trying to find a hole, we've even had some external input and theory review from a number of sources.  Paper will be available publicly soon that gets into most of that, so you can forge your own conclusion

But yes, I also agree with NxtChg, creating a novel transaction system is hard, and you've gotta be smart to do it.  Creating a novel consensus, that holds up under ALL attack vectors is quite another beast unto itself.

It isn't. It's a thing that will drive you mad in 3 months.

Ahh......perhaps I am insane after all..... 

I like where you are going with this RaiBlocks, but please please please, don't do what the rest do and rush it....take your time and review and refactor as many times as you need to get it RIGHT!