Proof-of-Approval: A Better Blockchain Consensus Protocol

[shunsaitakahashi]

Hello,

Looking for some feedback on a protocol that I have been working on.

Article describing it https://medium.com/@shunsai.takahashi/proof-of-approval-a-better-blockchain-consensus-protocol-b19a55dc331b

Full technical paper - https://github.com/Takanium/doc/blob/bf8f7934bac20d4ce15d8b611cc3525e18618651/research/proof-of-approval.pdf

Comparison with other protocols:


Other comparisons
1. With 2-hop chain Post#25 of this thread https://bitcointalk.org/index.php?topic=3125137.msg33082148#msg33082148
2. With Tendermint Post #35 of this thread https://bitcointalk.org/index.php?topic=3125137.msg33559358#msg33559358


Thanks,
Shunsai

[1714861909]

1714861909

[1714861909]

1714861909

[1714861909]

1714861909

[1714861909]

1714861909

[monsterer2]

Hi Shunsai,

Firstly, congratulations on producing a genuine white paper; there is a lot of rubbish that gets posted here, this is a cut above most.

I've only skimmed the paper, but a few things caught my attention:

* In your description of the ideal blockchain (from the article) you've omitted a very important attribute, 'permissionlessness' (excuse the non-word), which describes the ability of a party interested in participating in the block creation process to do so. Because your protocol is PoS, any interested party requires stake in order to participate in block production, making it permissioned.

* The consensus mechanism is based around timestamps, but there is no objective way to verify a historical timestamp. Doesn't this present a number of problems, e.g bootstrap poisoning?

* You are rewarding newly minted blocks with stake, yet to produce blocks, you require stake. How is this resolved?

* Doesn't the Pareto distribution of stake over time present centralisation concerns?

* How does your protocol deal with the keys-from-the-very-recent-past attack, whereby an attacker purchases newly emptied private keys of very large stake value, then creates a fork where he steals all the stake back for himself?

Cheers, Paul.

[Anti-Cen]

I think the gizmo term "Proof of something" gets taken too far in an effort to gain acceptance
in the crypto world but it is a good document that you have produced and forward booking space
on the block-chain will help things to scale.

You still need to distribute the block-chain and not just replicate it across hundreds of nodes
by moving in two directions instead of just left to right using a linked list as is the case with Bitcoin
but maybe it helps to think about it as a two dimensional arrays instead of one.

Ledgers can have check-sums too and can be used in the second dimension with the chain itself being
held in the top dimension so blocks contain a number of new wallet addresses and a block ledger (second dimension) is
used to represent the wallets transactions.

Each transaction could then reference where the credit came from by including the original wallet address along with the row
number of the other side of the transaction along with the signatures and of course the ledgers will need to be replicated across dozens of
nodes which is not happening in the case of lightning and happens too much (20,000 times) in Bitcoin main block-chain.

[cellard]

No such thing as "better blockchain consensus protocol" until your so called better blockchain consensus protocol has been through hell and back, namely, attacked by all angles on the technical aspect and attacked by all angles in the social engineering aspect (fork attempts by governments, social media attacks such as the twitter handle taken over by a fork, exchanges against you...) this is what Bitcoin has survived.

So far there is no PoW replacement. PoS has been proven to fail with NXT in the past, it wasn't pretty. We've seen IOTA fail big time as well, so DAG doesn't cut it. Hybrid attempts such as Decred.. no one hasn't even bothered to attack it for real yet.

I don't see realistic reasons for anyone to sell their BTC to get anywhere else.

[shunsaitakahashi]

Hello Paul,

I appreciate your kind words and your taking time to review this work. Please see rest of my comments inline.

* In your description of the ideal blockchain (from the article) you've omitted a very important attribute, 'permissionlessness' (excuse the non-word), which describes the ability of a party interested in participating in the block creation process to do so. Because your protocol is PoS, any interested party requires stake in order to participate in block production, making it permissioned.

'Permissionless' is an extremely important attribute, especially for the growth aspects of a blockchain and Proof-of-Approval is in fact permissionless. Proof-of-Approval does not require any stake to create blocks, but it does require approvals from parties holding a quorum (>50%) stake. If a party has 0 stake, it's block needs approval from others totaling a quorum stake (>50%). On the other hand, if a party has 10% stake, it's block needs approval from others totaling a quorum stake minus its own stake (>40%). PoS applies for approvals and not for block creation or winning rewards.

* The consensus mechanism is based around timestamps, but there is no objective way to verify a historical timestamp. Doesn't this present a number of problems, e.g bootstrap poisoning?

Consensus mechanism is based on signed approval messages that are stored in the blockchain to deter long-range attacks. Also note that the approving stake is at the current timeslot, not something in the past. Let's consider a couple of scenarios here.

1. An adversary attempts to create a block n blocks in the past. In order to successfully fork from n block in the past, the adversary needs signed approval, in that timestamp, from >50% stakeholders for every block since the n-blocks in past (blocks -n, -n+1, -n+2...).

2. Past >50% stakeholders (n blocks ago) sell off all their stake and turn adversarial (but current >50% stakeholders are still honest). Again, in order to successfully fork from n blocks ago, they need signed approvals for every block in the past (blocks -n, -n+1, -n+2...). While they may be able to approve some blocks when they did hold > 50% stake, as their stake goes down, they need other parties to approve their more recent blocks. In order to get the newest block approved, they would need adversarial stake > 50% in the present.

* You are rewarding newly minted blocks with stake, yet to produce blocks, you require stake. How is this resolved?

As stated above, a party with no stake requires approvals with other parties that can form a quorum. While it is harder to achieve than another party holding a large amount of stake but is definitely achievable.

* Doesn't the Pareto distribution of stake over time present centralisation concerns?

Yes, Pareto distribution does present centralization concerns just like mining pools for bitcoins. There does not seem to be a known solution for it yet.

* How does your protocol deal with the keys-from-the-very-recent-past attack, whereby an attacker purchases newly emptied private keys of very large stake value, then creates a fork where he steals all the stake back for himself?

I believe it is the same as the scenario 2 described above.

Thanks again.
Shunsai

[shunsaitakahashi]

Hello Anti-Cen,

Thank you for taking time to review this work and providing very valuable feedback and suggestions. I do see scaling to be the key challenge for blockchains and similar technologies.

My day job relates to databases and microservices. Many of the concerns of databases also apply to blockchains. In order for blockchains to be successful, they not only have to store a lot of data fast but also be able to read any stored data (like a needle in a haystack) very fast. Blockchain data structures are not naturally conducive to fast readability.

Proof-of-Approval, like many others, needs to define

1. Data structures that support fast write and read performances without sacrificing security
2. Parallelize the node tasks in order make computational and storage needs manageable
3. Achieve the above 2 and have the code be well designed for correctness and maintainability

“If you aren't in over your head, how do you know how tall you are?”
― T.S. Eliot

Thanks,
Shunsai

[Adeyemi]

I read part of the post and it lead me on a journey to reading other articles that are enlightening. There's so much knowledge hidden in your article which is not something to be rushed. I'm still relatively new in the cryptosphere but am learning at a fast pace. I should come back to make a more meaningful comment when I'm done reading your post and fully understand your protocol. Thank you for sharing and adding to my knowledge. All the best.

[Anti-Cen]

So far there is no PoW replacement. PoS has been proven to fail with NXT in the past, it wasn't pretty. We've seen IOTA fail big time as well, so DAG doesn't cut it. Hybrid attempts such as Decred.. no one hasn't even bothered to attack it for real yet.

I don't see realistic reasons for anyone to sell their BTC to get anywhere else.

From what I read DAG does cut it and IOTA has been sorted out and the facts are that Bitcoin does not scale
and that ignoring transactions that get lost or stuck.

I agree about standing up to attack so far but Lightning hubs can and will get attacked and being a single point of
failure I am sure they will be taken down without having to resort to feedback loops if the fees don't prevent these
types of attack.

"realistic reasons" could also be financial or based on people just not liking banks !

[Anti-Cen]

My day job relates to databases ..........

Yes, worked on many distributed query's myself and they could make life easier and using digits 3-6 from the public
key to sub-divide the data across clustered nodes (replication servers) to provide redundancy.

dear oh dear how did we ever write scaleable distributed network apps (DNA) without having to invent CPU-Wars
I will never know and now apparently the answer is to go "off-block" so keep doing what your doing and keep an
eye on network chatter becoming too big because that's another killer if you don't watch it.

[shunsaitakahashi]

Hello Cellard,

While "better" is just tongue-in-cheek designed to attract attention, the consensus protocol itself is serious.  

Like many others, I am a huge fan of Satoshi Nakamoto's work. But that doesn't mean it's the final word in that category. In science, it rarely is. Isaac Newton's wonderful work paved way to Albert Einstein's work and others are sure to follow.

Also totally agree that protocol's have made many claims in past, most turning out to be false. While some of it may have been ill-intentioned, I do believe that the majority has been just the scientific process - theory followed by either proof or disproof.

As you said, a protocol does need to be "through hell and back" because any success will invite attacks from all angles. My purpose here is just the first step--peer review. I am looking for flaws in the protocol that I haven't been able to think of. And I do want readers to be skeptic - it's better to find flaws sooner than later. If it does turn out that the protocol has merits, the next steps of implementation and validation will follow.

Thanks,
Shunsai

[-DEVX-]

Like many others, I am a huge fan of Satoshi Nakamoto's work. But that doesn't mean it's the final word in that category. In science, it rarely is. Isaac Newton's wonderful work paved way to Albert Einstein's work and others are sure to follow.

I really liked this one Shunsai. Checking your paper all night today. Of course i have many questions in my mind right now but I'm really impressed with it for the first 11 pages.

[shunsaitakahashi]

Devx,

Whole night! You are working harder than I did! Really appreciate it!

Shunsai

[monsterer2]

Hello Paul,

I appreciate your kind words and your taking time to review this work. Please see rest of my comments inline.

* In your description of the ideal blockchain (from the article) you've omitted a very important attribute, 'permissionlessness' (excuse the non-word), which describes the ability of a party interested in participating in the block creation process to do so. Because your protocol is PoS, any interested party requires stake in order to participate in block production, making it permissioned.

'Permissionless' is an extremely important attribute, especially for the growth aspects of a blockchain and Proof-of-Approval is in fact permissionless. Proof-of-Approval does not require any stake to create blocks, but it does require approvals from parties holding a quorum (>50%) stake. If a party has 0 stake, it's block needs approval from others totaling a quorum stake (>50%). On the other hand, if a party has 10% stake, it's block needs approval from others totaling a quorum stake minus its own stake (>40%). PoS applies for approvals and not for block creation or winning rewards.

That's a chicken and egg problem though. You cant approve any blocks if you don't have any stake... What happens at launch, when there is no stake?

Consensus mechanism is based on signed approval messages that are stored in the blockchain to deter long-range attacks. Also note that the approving stake is at the current timeslot, not something in the past. Let's consider a couple of scenarios here.

1. An adversary attempts to create a block n blocks in the past. In order to successfully fork from n block in the past, the adversary needs signed approval, in that timestamp, from >50% stakeholders for every block since the n-blocks in past (blocks -n, -n+1, -n+2...).

2. Past >50% stakeholders (n blocks ago) sell off all their stake and turn adversarial (but current >50% stakeholders are still honest). Again, in order to successfully fork from n blocks ago, they need signed approvals for every block in the past (blocks -n, -n+1, -n+2...). While they may be able to approve some blocks when they did hold > 50% stake, as their stake goes down, they need other parties to approve their more recent blocks. In order to get the newest block approved, they would need adversarial stake > 50% in the present.

In case 2, what if the forked blocks that the historical stake holders approve, steal their stake back again?

* You are rewarding newly minted blocks with stake, yet to produce blocks, you require stake. How is this resolved?

As stated above, a party with no stake requires approvals with other parties that can form a quorum. While it is harder to achieve than another party holding a large amount of stake but is definitely achievable.

I fail to see why any party with stake would sacrifice their chance to claim the minted coins by approving a non-stake holder's blocks? That's not rational behaviour; they surely would just approve their own blocks?

Cheers, Paul.

[shunsaitakahashi]

Hello Paul,

That's a chicken and egg problem though. You cant approve any blocks if you don't have any stake... What happens at launch, when there is no stake?

Yes, it does require that genesis block allocates stake to one or more parties.

In case 2, what if the forked blocks that the historical stake holders approve, steal their stake back again?

Stakes (or stake changes) in any block affect decision process in forks containing that block only, not sibling forks.

A - B0 - C0 - D0
  \ B1 - C1 

Assuming B0/C0/D0 is the "honest" fork and B1/C1 is an adversarial attempt to a long-range revision attack, any stake changes in B1 and C1 do not affect decision process on the fork B0/C0 etc. Stakes for the decision process are calculated just before each block for that specific fork. More details are in Section 3.3.1 of the paper.

I fail to see why any party with stake would sacrifice their chance to claim the minted coins by approving a non-stake holder's blocks? That's not rational behaviour; they surely would just approve their own blocks?

By "approval" I meant "approval-message" (Section 2.2.14) where stakeholders rank valid blocks in order of their own preference. Each stakeholder always ranks his/her own block on the top. The question then boils down to whether each stakeholder would approve only his/her block and no other block (approval-message contains a single block header). In that scenario (assuming no single stakeholder has a quorum stake), the blockchain fails resulting in a negative impact on the stakeholders.

In another scenario, some stakeholders may only approve their own blocks in an attempt to tilt the winning chances in their own favor. This is a real possibility. A possible mechanism to deter such an "attack" is to simply "blacklist" such stakeholders temporarily. An honest stakeholder can tell within a few blocks with certainty if another stakeholder is not approving his/her valid blocks. Blacklisting them temporarily for 2x the blocks they failed to approve may work. But they should not be blacklisted permanently since this is likely to result in all stakeholders being blacklisted permanently by other stakeholders.

This deterrence algorithm along with "approval scoring function" (2.2.15) need to be defined for a blockchain implementation.

Regards,
Shunsai

[CarltonWyatt]

I think we agree proof of work is extremely energy inefficient if secured through mining so a new proof will eventually probably over take proof of the work I think. Only as the industry develops though but this is yet another good new idea that I look forward to seeing it progress

[monsterer2]

Stakes (or stake changes) in any block affect decision process in forks containing that block only, not sibling forks.

A - B0 - C0 - D0
  \ B1 - C1 

Assuming B0/C0/D0 is the "honest" fork and B1/C1 is an adversarial attempt to a long-range revision attack, any stake changes in B1 and C1 do not affect decision process on the fork B0/C0 etc. Stakes for the decision process are calculated just before each block for that specific fork. More details are in Section 3.3.1 of the paper.

Hi Shunsai,

In the following example, say I had a majority of stake at block A. Then at B0, C0 all the way up to head block H0, I sell off my majority of stake, understanding that there is an upper limit on stake transfer.

Code:

A - B0 - C0 - D0 - H0
  \ B1 - C1 - D1 - H1

Then, I create a double spend of my stake using the historical private keys, which I still own, thus sending it to myself in a fork continuing fork B1 - C1 - D1 that I create and approve myself (majority stakeholder).

This is not a 50% attack, because I don't own the value held in the private keys anymore, the cost for doing this is nothing.

Why doesn't the network just accept this fork as canon, given that its the same length (or longer if I chose) than the genuine fork?
  

In another scenario, some stakeholders may only approve their own blocks in an attempt to tilt the winning chances in their own favor. This is a real possibility. A possible mechanism to deter such an "attack" is to simply "blacklist" such stakeholders temporarily. An honest stakeholder can tell within a few blocks with certainty if another stakeholder is not approving his/her valid blocks.

My question is, what does the stakeholder have to gain by approving anyone's blocks but his own? If he has an edge by not doing this, there is a very real problem.

[fresheneesz]

Am I right that the goals of this protocol are primarily to reduce the finality time in comparison to Bitcoin as well as reduce the cost of running the system?

The image of how blocks are set up (https://cdn-images-1.medium.com/max/1000/1*fPsB4-t6J_1u1UpF37dYHw.png) reminds me strongly of the way blocks are chained in the "two-hop blockchain" (https://eprint.iacr.org/2016/716.pdf). I would like to see more in-depth comparisons to other consensus protocols, especially DPoS which seems particularly close in mechanism to Proof-of-Approval.

Your comparison table claims Proof-of-Approval is fully finalized after a single block,  however, you also talk about how there may be multiple top-level blocks where the "preferred" chain is unresolved. Doesn't this imply that your transaction being inside a valid block doesn't mean its been fully finalized? How do you reconcile that?

Since any node can produce a candidate block, this seems like it would potentially cause unbounded network traffic with everyone and their mother suggesting blocks to vote on. How is this resolved in the case of, say, an adversarial ddos attack? But I have the same question as monsterer2, why would anyone with stake approve the block of anyone without stake? In fact, why would anyone with stake approve *anyone* else's blocks? It seems there's no incentive to mine on top of someone else's blocks. This could manifest in more subtle attacks where you have stakeholders who refuse to mint on top of a competitor. It could also manifest in deadlock as nodes try to maximize their probability of winning the minted block.

I have a hard time seeing the purpose of having nodes pay attention to timestamps and comparing them to their receive-time. At its core, the mechanism is just voting, right? I don't see why the protocol doesn't just let nodes vote for whatever blocks they want. I see the appeal of a simple stake-voting process for block creation (similar to DPoS), but the Proof of Approval method seems like rational profit-seeking minters would make things devolve into the top 50% stakeholders passing around permission to mine the blocks, completely excluding the other 50%.

Also, could you provide a glossary of variables you use in this paper? Its hard for me to keep track of which variables mean what and find their definition in the prose when I need to refer to what they mean later.

@monsterer2 Are you suggesting that greater permissionlessness is desriable? Also, I've come to the conclusion that bootstrap poisoning can be entirely solved by blockchain checkpoints hardcoded into whatever software you use - since the software should be audited, and malicious software can make your client choose whatever chain it wants, hardcoded checkpoints don't reduce security in any meaningful way and yet completely remove the possibility of long range attacks including bootstrap poisoning.

Also @monsterer2 I would actually call your most recent proposed attack a 51% attack, since you *did* own a majority stake, and are using that fact to attack the network. I'd say its just an interesting twist on a 51% attack.

[aliashraf]

Just read the article, nice work but, as one of the guys have mentioned earlier, the 'proof of something' discourse is exhaustively saturated and it is hard to attract enough attention from the community these days for a new one.

I'm not satisfied yet with the protocol itself as it seems not to be secure against the bootstrap poisoning attack and I have not found strong enough game theory and incentive analysis (why people should approve  each other's blocks) and I do not like the idea of zero cost block generation (we have enough trouble with transaction malleability and DDoS attack already) ... BUT

But first of all I have not read the technical paper yet and, more importantly, I don't think  the consensus problem has a solution at all. I mean other than being ignored!

The way I see the situation in crypto world, we will be stuck in wasting resources and jeopardizing performance and scalability  as long as we are seeking a solution for this problem: consensus.

I know, I know it looks kinda weird or ridiculous, speaking about public blockchain or public ledger while one is denying the importance of the holly consensus , it is kinda blasphemy, isn't it?

Common sense tells us to remain a believer but I don't give a sh*t to common sense, actually I love questioning the most obvious predicates issued by common sense, and it is one of them: to have a secure, public, permissionless blockchain you should have a protocol for the participants to reach an agreement upon the state of the blockchain, a consensus protocol, got it you crazy fool?

And I'm like

Thank you sir for the enlightening, but NO! I don't get it, you know why? Because it costs too much and hell it is not scalable and I can't imagine a crypto dominated globe with millions of machines all agreed upon every single transaction.


If I was such a reasonable person to bend knees to common sense I wouldn't participate in the crazy saga for a decentralized paradigm in the heights of capitalism fully controlled by corporates and banks.


Yes, I'm crazy enough to don't pay attention to anything other than game theory and pure mathematics and to this point, I have not found a proof for the necessity of consensus, actually there even exists  no unproven theorem about it. It is just an 'obvious' assumption, something like 'The earth is the center of the cosmos and the sun along with other stars are rotating around us', nothing more.

So, my advice, just forget about consensus protocols, they either do not exist or they are bad protocols.

[shunsaitakahashi]

Hello Monsterer2,

In the following example, say I had a majority of stake at block A. Then at B0, C0 all the way up to head block H0, I sell off my majority of stake, understanding that there is an upper limit on stake transfer.

Code:

A - B0 - C0 - D0 - H0
  \ B1 - C1 - D1 - H1

Then, I create a double spend of my stake using the historical private keys, which I still own, thus sending it to myself in a fork continuing fork B1 - C1 - D1 that I create and approve myself (majority stakeholder).

This is not a 50% attack, because I don't own the value held in the private keys anymore, the cost for doing this is nothing.

Why doesn't the network just accept this fork as canon, given that its the same length (or longer if I chose) than the genuine fork?

Any single person owning (or colluding with) a quorum (majority) stake is a problem because that person (or colluders) can rewrite the chain as they choose. This situation is essentially the same as an adversary owning a quorum stake which is the limit of the protocol.

In another scenario, some stakeholders may only approve their own blocks in an attempt to tilt the winning chances in their own favor. This is a real possibility. A possible mechanism to deter such an "attack" is to simply "blacklist" such stakeholders temporarily. An honest stakeholder can tell within a few blocks with certainty if another stakeholder is not approving his/her valid blocks.

My question is, what does the stakeholder have to gain by approving anyone's blocks but his own? If he has an edge by not doing this, there is a very real problem.

My assumption, which could be wrong, is
1. Stakeholders will approve (through approval message) more than one blocks or risk blockchain failure or being blacklisted by other node members.
2. Stakeholders will try to rank more candidate blocks created by other higher stakeholders lower in order to gain advantage in block creation.

Of course, these are just assumptions and need to be proven in real life. Perhaps some coins should be given for approving a winning block in order of their ranking of the block. I am open to suggestions here. In the meantime I will analyze the if incentivizing block approval does have the desired effect.

Thanks for pointing it out - this is exactly what I am looking for from this forum.
Shunsai

[fresheneesz]

Stakeholders will approve (through approval message) more than one blocks or risk .. being blacklisted by other node members.

How would other nodes detect that they're intentionally not approving blocks in order to blacklist them? Isn't anyone allowed to not approve blocks if they don't want to?

Stakeholders will try to rank more candidate blocks created by other higher stakeholders lower in order to gain advantage in block creation.

But isn't only including your block as the only approved block the best way of gaining an advantage? This is known as the "bullet-voting" problem in approval voting, ranked voting, and range voting circles. https://en.wikipedia.org/wiki/Bullet_voting . While in normal voting, this shouldn't usually be much of a problem, it seems like in Proof-of-Approval there is a big incentive to bullet vote.