A World of Trust – eMunie Consensus Primer

[Fuserleer]

Hello everyone,

Before hand I would like to express that any and all information you think you already know about how eMunie functions should be forgotten immediately!  Almost all of that old information is defunct and out of date and no longer relevant in anyway.  Thank you

It is with great pleasure that I publish the first article regarding eMunie technology focusing on our consensus algorithms based around trust.

The first of many upcoming articles covering all areas of the eMunie project, I wanted to start with consensus as it is probably the most important of all.

This first document focuses on the topic in a high level and is meant for a wide audience. There is lots of detail to dig into in future articles, even though it gets quite in depth, but I want to also provide documentation for those not so technically minded which is written in every day language and is easy to understand.

To enable a full and complete understanding of the project, what it offers and how it functions will require many articles and documentation totaling 100s of pages. I'll do my best to make sure that important topics that rely on content within other documents are explained as best I can so that the topics being covered in the material being read can be understood clearly.

I'm happy to answer any questions where I can, but please understand that there may be some items that will be difficult to explain without these yet unfinished additional documents.

The article can be viewed at http://blog.emunie.com

Best Regards to all

- Dan/Fuserleer

PS: I've set self-moderation on this topic as a precaution and will only be removing posts that are blatant attacks and severe trolling.

[1714983655]

1714983655

[1714983655]

1714983655

[1714983655]

1714983655

[1714983655]

1714983655

[1714983655]

1714983655

[r0ach]

Well, whether this turns out to be the best or worst distributed ledger, once you factor in the consensus mechanism + economics system (if it's still the same one as before), it will probably win the award for most complicated one.  I was surprised that I couldn't find any Satoshi post regarding inputs/outputs vs balance sheet, only a Gavin post saying basically it won't work, and a Theymos post saying headers first and merkle root solves the same problem.  It does solve space constraints, creating SPV clients, but does nothing to address the issue of lower number of consensus nodes in general from exponentially increasing overhead in the input/output blockchain model.

I tried to speed read through this post and didn't really see what motivation existed for creating a large number of unique, non-sybil nodes.  I'm not sure if incentives were adequately addressed or not.  Overhead is the main variable.  If the overhead is low, then you have a larger number of sybil nodes, and possibly a small number of real nodes since who wants to be one out of a billion transaction processors making one cent a month?  If the overhead is high, then you have a lower amount of every type of node, until the moment someone decides to attack it, since it's almost impossible to guard against people who willingly operate at a loss.  You have a no win situation in that regard in terms of trying to design overhead into the system.  Quite a paradox...

This is related to the Satoshi "no free lunch" principle and general game theory, where in order to trust output from an entity, somebody somewhere has to be making a revenue stream, where them substituting false information is going to end that revenue stream.

[Fuserleer]

Thanks for taking the time to read.

Firstly, I should of made it clear in the OP to dis-regard anything you may have read about eMunie in the past, as all of it is out of date and likely incorrect by now.  Ive modified to the OP to state that.

Regarding balances, the decision to use them was based on an lot of research into which of the two method could achieve the core goals that we wanted.  Lots of time and effort was put into applying different scenarios, use and edge cases which were evaluated against the strengths and weaknesses of both approaches.  In the end, balances won out due to the long term efficiency and flexibility they can provide as will be clear soon.

They are more complicated to implement than input/output, but the declarations that it does not work are simply wrong, it can work, and it does work.  Its just a hard task to achieve in an efficient and reliable manner.

I really didn't want to get into economics within the article, as the eMunie ecosystem is very fluid, dynamic and diverse.  It will likely require quite a few detailed documents to cover all aspects of it, and I didn't want to "brain dump" too much unrelated information, distract from the main topic of the article and risk sending readers into a confusion induced coma.

The details following really should be left for the associated economic documentation to explain them fully, but your question about incentive is an important one so I'll briefly cover some of it.  

Revenue for the work performed by the nodes in the network can quickly become quite significant when you apply all factors of the economics.  Not only that, the distribution of this revenue is very broad, due to work being recorded at a much granular level all over the network, instead of it being a huge "race condition" such as with Bitcoin. Nodes that perform even the smallest amount of work get rewarded for doing so, and the execution of that work is much more efficient than simply testing billions of hashes per second in what might as well be a lottery.

In terms of income there are multiple streams:  

Firstly, fees that are generated by transactional and peripheral content actions are allocated to the relevant service providers, and this income is periodically distributed between them as per the amount of work they have done.  We've performed a number of calculations with regard to income and even in the most frugal of cases, the cost required to operate the services providers is much less than the fees recouped, even if that node only performs a couple of tasks per day.

Bitcoin is perhaps the lowest income provider of any currency in relation to the amount of work done.  Megawatts of electricity is currently burned just to enable miners to make a few cents of profit over that cost, not to mention hardware replacement costs, maintenance, disposal of old hardware, pollution from both the creation and disposal of said hardware and the additional pollution of generating all that electricity.  Most of that is a hidden cost, but at some point it has to be paid for in some manner.

With eMunie the efficiency of performing work is many magnitudes more efficient than Bitcoin.  For example a Pi could be used to provide some of the lighter network services, consuming just a handful of watts per day yet seeing an income of a few dollars or more per month from minimal workload.  When there is no work for it to do, it can sit idle and power consumption is literally 1-2 watts.

Secondly, eMunie's new currency supply is dynamic and the total amount of currency can increase or decrease depending on the demand.  Assuming that eMunie is successful and sees even moderate success compared to Bitcoin, then the network will nearly always be in growth, and an increase in supply will consistently be required.  Of course, at some point in time there comes a plateau but I'm digging too much into economics already, so I'm going to skip that.  

New supply is distributed in 2 ways, the first being in the same manner as fees.  Nodes that have done work are also eligible for a slice of the new supply proportionate to the amount of work they have done since the last distribution.  The second mechanism is via interest on assets that are held in balances within the network, with holders of balance receiving a slice of new supply proportional to the amount of EMU they hold v's total supply, and how long that EMU has been sitting in that balance.

For service nodes that are active, not only do they receive a portion of fees for that service and a portion of new supply for doing work, if they were to hold those earned EMU and not move them, they would also receive a 2nd portion of new supply as interest.

The question isn't should I run a service node, the question should be why on earth aren't I?

[Come-from-Beyond]

Before hand I would like to express that any and all information you think you already know about how eMunie functions should be forgotten immediately!

What has made you abandon the idea of block-trees?

[monsterer]

For the sybil attack to be unprofitable, the amount of funds spent to acquire the needed trust to pull it off must be <= the amount you can potentially double spend with such an attack. Is this the case?

[Come-from-Beyond]

For the sybil attack to be unprofitable, the amount of funds spent to acquire the needed trust to pull it off must be <= the amount you can potentially double spend with such an attack. Is this the case?

You didn't take into account indirect profits of double-spending attacks. Also, there always can be an irrational attacker.

[Fuserleer]

Before hand I would like to express that any and all information you think you already know about how eMunie functions should be forgotten immediately!

What has made you abandon the idea of block-trees?

It simply didn't scale enough.

Transaction throughput was high, but on networks with sustained high load, even with pruning, it would alienate a lot of commodity hardware.

eMunie has to be able to exceed Visa scale, but also allow everyone an opporutinity to take part in securing the network efficiently, and block trees couldn't provide that.

The new ledger does, you could run a couple Pis, make a income and participant fairly significantly to the operation and security of the network.

[Fuserleer]

For the sybil attack to be unprofitable, the amount of funds spent to acquire the needed trust to pull it off must be <= the amount you can potentially double spend with such an attack. Is this the case?

This is true and Sybil attacks are expensive even at low network loads.

Let me give some example numbers:

The base fee is currently defined as the equivalent to $0.01.  We'll ignore the load fee for now.

Lets assume that the network is processing 1 tx/s (similar to Bitcoin) which is a total transaction load of 86400 tx per day.  A attacker using a Sybil ideally wants 40%+ as stated in the article so that he is able to influence the outcome of the TCW process enough to direct which transactions are selected.  Thus he has to produce ~35k transactions per day, the cost of which is 35k * 0.01 = $345 per day.  Not a great deal, but not insignificant either.

Any profits from double spends or Finney variants needs to be greater than $345 per day for it to be worth while, but also remember that there is maturity time and decay of trust.  The attacker can't counter-sign any transactions until the trust he has produced is mature, which is 1 snapshot period (currently set to a week).  So the total cost before he can make any profit is 7*£345 = $2415 + $345 per day there after.

If we then apply the load fee the costs become greater, at < 1 or tx/s the load fee is very high, 0.10c and greater.  The load fee doesn't diminish until > 25 tx/s.

Applying the load fee to the above results in the following 35k * ($0.01 + $0.10) = $3,850 per day, with an initial cost before maturity of $26,950

Plus, I would imagine that anyone transacting where values are exceeding a few thousand dollars or more, would at least be prudent and wait a minute or so before releasing whatever goods have been purchased as even with a successful Sybil, the window of opportunity is still very short, <30s or so from the time of the 3rd party receiving the initial transaction.

Micro transactions just aren't worth the effort, and for larger transactions its trivial to instruct the user to "wait for confirmation" like with BTC and others.  Except in this case, only 1 "confirmation" is required and you get it within a short period of time.

As the network grows and hopefully is a success, it becomes self-securing and the load fee isn't needed at all due to the massive costs of successfully attempting a Sybil.   Lets assume success is achieved and the network is processing Paypal type loads of 150 tx/s

150 * 86400 = 12.9M tx/day
Attacker requires 40%+ so 5.1M tx/day need to be produced.
5.1M * 0.01 = $51,840 per day with a cost of $362,880 to achieve maturity

Just as in the early days of BTC's adoption, 51% attacks, Finney attacks were much easier, but over time the security of the network has increased to a point where its nigh impossible nor worth it to pull these off.  The same can be said for eMunie, a larger network is a safer network.

[patmast3r]

Could explain how trust is formed on a network level and not on a node level ?
As far as I understood nodes send challegens and if those are answered correctly the node on the other end will gain a little trust. But that trust would only be for the challenging node as he sent the challenge right ? How is that experience brought to a network leve ? Does the simply broadcrast to the network that he received a valid result for a challegen from node x ?

[TPTB_need_war]

Evaluation of eMunie

CN = consensus node = holds local information regarding the state of the ledger, and likely peripheral repository content.
IN = issuing node = challenges CN to insure they hold the correct data
SN = service nodes = only they can earn trust, and only if they respond to services timely. Thus CN and IN must be SNs.
TXN = transaction creation node = seed the trust earning process for each txn for a chosen service. Unclear how the network reaches consensus on trust awarded.

To start, one nitpick is there is no reliable concept of time in a consensus network, so instead you must base decay on blocks elapsed. If you entrust consensus on time, it adds another complex attack vcctor.

If I understand the general conflict resolution concept (ignoring for the moment possible attack vectors created by the bandwidth optimization that need more study), transactions are propagated over the network and counter signatures (CS) weighted by trust of the SNIDs propogate telling each node which of each set of transactions (amongst each double spend set) are the "first seen" transaction.

The attack vectors are DoS (especially on network propagation), Sybil attacks (spreading trust around to infinite nodes in order to control the transaction propagation order legally in the protocol), and collusion of nodes (legally within the protocol).

You address the Sybil attack due to trying to impart trust to self by creating spam transactions, but you don't address the Sybil attack designed to control propagation order. Additionally your proposed solution to transaction spam is flawed because if the transaction fees are greater than what is earned by any node(s), then eventually the money supply must go to zero (and that is true even if you create new money supply in other ways since the game theory will be holistically connected).

Also what is the financial incentive for these nodes to participate? Game theory can't be analyzed without that information.

You will have a very difficult challenge to beat me in terms of designing a better consensus network. Seriously. Good luck.

P.S. you have decent coin name. Perhaps you may consider abandoning your (what appears to me to be a) faulty design and joining me.

No personal offense intended, and I will not harp on this. Again good luck.

[Fuserleer]

Could explain how trust is formed on a network level and not on a node level ?
As far as I understood nodes send challegens and if those are answered correctly the node on the other end will gain a little trust. But that trust would only be for the challenging node as he sent the challenge right ? How is that experience brought to a network leve ? Does the simply broadcrast to the network that he received a valid result for a challegen from node x ?

Sure, maybe a simple example will help.

Lets assume there are only 3 nodes in the network, A is you, B is me, and C is someone else and is currently offline.

A & B are connected in the network, and have been sending challenges to each other periodically.  
B wants to send a transaction to somebody, and as it is only connected to A, it endorses A for any work done and for passing the challenges, and sends the transaction.
This endorsement is recorded in the transaction directly, remember that.

C now comes online, synchronizes and connects to A.  
C is curious if A is trusted, so scans the transactions for any endorsements that reference A's SNID.  
C will see the recent endorsement of B -> A and any others that have been made in the past, and can easily calculate A's trust level from the endorsement information in transactions.  
(Note: C will not be able to discover who endorsed A, that information is not recorded)

Transactions are stored in a public ledger just as in BTC and other cryptos, and endorsements are not encrypted or masked in any way.  Therefore everyone can see at any moment, who has been endorsed and calculate their trust level and compare it should they wish to any other trust level.

Does that help to understand it?

[patmast3r]

Could explain how trust is formed on a network level and not on a node level ?
As far as I understood nodes send challegens and if those are answered correctly the node on the other end will gain a little trust. But that trust would only be for the challenging node as he sent the challenge right ? How is that experience brought to a network leve ? Does the simply broadcrast to the network that he received a valid result for a challegen from node x ?

Sure, maybe a simple example will help.

Lets assume there are only 3 nodes in the network, A is you, B is me, and C is someone else and is currently offline.

A & B are connected in the network, and have been sending challenges to each other periodically.  
B wants to send a transaction to somebody, and as it is only connected to A, it endorses A for any work done and for passing the challenges, and sends the transaction.
This endorsement is recorded in the transaction directly, remember that.

C now comes online, synchronizes and connects to A.  
C is curious if A is trusted, so scans the transactions for any endorsements that reference A's SNID.  
C will see the recent endorsement of B -> A and any others that have been made in the past, and can easily calculate A's trust level from the endorsement information in transactions.  
(Note: C will not be able to discover who endorsed A, that information is not recorded)

Transactions are stored in a public ledger just as in BTC and other cryptos, and endorsements are not encrypted or masked in any way.  Therefore everyone can see at any moment, who has been endorsed and calculate their trust level and compare it should they wish to any other trust level.

Does that help to understand it?

So basically trust is "credited" on the ledger and there for everyone to see. Thanks for clearing that up.

[Fuserleer]

Could explain how trust is formed on a network level and not on a node level ?
As far as I understood nodes send challegens and if those are answered correctly the node on the other end will gain a little trust. But that trust would only be for the challenging node as he sent the challenge right ? How is that experience brought to a network leve ? Does the simply broadcrast to the network that he received a valid result for a challegen from node x ?

Sure, maybe a simple example will help.

Lets assume there are only 3 nodes in the network, A is you, B is me, and C is someone else and is currently offline.

A & B are connected in the network, and have been sending challenges to each other periodically.  
B wants to send a transaction to somebody, and as it is only connected to A, it endorses A for any work done and for passing the challenges, and sends the transaction.
This endorsement is recorded in the transaction directly, remember that.

C now comes online, synchronizes and connects to A.  
C is curious if A is trusted, so scans the transactions for any endorsements that reference A's SNID.  
C will see the recent endorsement of B -> A and any others that have been made in the past, and can easily calculate A's trust level from the endorsement information in transactions.  
(Note: C will not be able to discover who endorsed A, that information is not recorded)

Transactions are stored in a public ledger just as in BTC and other cryptos, and endorsements are not encrypted or masked in any way.  Therefore everyone can see at any moment, who has been endorsed and calculate their trust level and compare it should they wish to any other trust level.

Does that help to understand it?

So basically trust is "credited" on the ledger and there for everyone to see. Thanks for clearing that up.

Yeah exactly, everyone else can see it and make of it what they will.

[patmast3r]

So let's see if I understand this.

Nodes build trust. With that trust they can - simply put - vote on transactions by countersigning them which gives the transactions consensus value. When nodes sync with each other they don't pcik the longest chain like in bitcoin but they pick the chain with the highest concesus value i.e. the chain that most trusted nodes signed.

Is that about right ?

[Fuserleer]

So let's see if I understand this.

Nodes build trust. With that trust they can - simply put - vote on transactions by countersigning them which gives the transactions consensus value. When nodes sync with each other they don't pcik the longest chain like in bitcoin but they pick the chain with the highest concesus value i.e. the chain that most trusted nodes signed.

Is that about right ?

For the purpose of understanding trust consensus you do not need information about the ledger or how it is constructed.

There is no chain as specified in the document, there are no blocks either.  Transaction are validated and committed individually in real time.  The ledger is 100% different from Bitcoin and everything else.

If me and you both submit a transaction 10 seconds apart, then mine will be committed and final around 10 seconds after yours.

[Fuserleer]

Evaluation of eMunie

CN = consensus node = holds local information regarding the state of the ledger, and likely peripheral repository content.
IN = issuing node = challenges CN to insure they hold the correct data
SN = service nodes = only they can earn trust, and only if they respond to services timely. Thus CN and IN must be SNs.
TXN = transaction creation node = seed the trust earning process for each txn for a chosen service. Unclear how the network reaches consensus on trust awarded.

To start, one nitpick is there is no reliable concept of time in a consensus network, so instead you must base decay on blocks elapsed. If you entrust consensus on time, it adds another complex attack vcctor.

First, there are no blocks!!

Because the commital of transaction is fluid and real time, and providing you have a majority of honest nodes where no Sybil attacker has achieved 50%+ of the trust, you can safely and easily approximate the current time to within about a minute.  Time management is also another document, but that fact should be apparent if you "read between the lines" a little when digesting the consensus.

If I understand the general conflict resolution concept (ignoring for the moment possible attack vectors created by the bandwidth optimization that need more study), transactions are propagated over the network and counter signatures (CS) weighted by trust of the SNIDs propagate telling each node which of each set of transactions (amongst each double spend set) are the "first seen" transaction.

Your understanding is slightly wrong, its not explicitly the "first seen" transaction, as the first seen may not validate for that node at the time.  Because transactions are fluid, nodes may be missing dependent transactions and so can not validate the first seen.  By the time the second has come along, those nodes may have the dependent transactions required and so counter-sign on it.

This is part of the reason why an attacker that causes a conflict without a successful Sybil attack has a best a 50/50 chance, as explained in the document, of the second transaction being accepted

The attack vectors are DoS (especially on network propagation), Sybil attacks (spreading trust around to infinite nodes in order to control the transaction propagation order legally in the protocol), and collusion of nodes (legally within the protocol).

You address the Sybil attack due to trying to impart trust to self by creating spam transactions, but you don't address the Sybil attack designed to control propagation order. Additionally your proposed solution to transaction spam is flawed because if the transaction fees are greater than what is earned by any node(s), then eventually the money supply must go to zero (and that is true even if you create new money supply in other ways since the game theory will be holistically connected).

Again your understanding breaks down here.  A successful Sybil attack can prevent and hinder the propagation of counter-signature information to some degree by not forwarding on counter-signatures, and it can also isolate some nodes from broadcasting counter-signatures at all.  But you don't need every node in the network to present a counter-signature to form a consensus on a transaction, only a majority.  Thus to completely prevent any counter-signatures and prevent any transaction committals at all, the attacker would have to isolate ALL nodes in the network from ALL other nodes other than his to be sure of disruption, and that is no easy task to do.

The most an attack of this type could do, would be to slow down the finalization of transactions.

In Bitcoin you can target a miner and isolate him much easier, provide him with bad transactions to mine, and not broadcast on any of his blocks.  Isolating ALL miners its a monumental undertaking.

Also what is the financial incentive for these nodes to participate? Game theory can't be analyzed without that information.

Explained in a follow up post further up, it should really be left to a document of its own on economics, but the information I've provided should be adequate.

You will have a very difficult challenge to beat me in terms of designing a better consensus network. Seriously. Good luck.

Isn't this the case with all developers?

P.S. you have decent coin name. Perhaps you may consider abandoning your (what appears to me to be a) faulty design and joining me.

No personal offense intended, and I will not harp on this. Again good luck.

No offense intended either, but I'm too vested into this to even consider jumping ship thanks

[patmast3r]

So let's see if I understand this.

Nodes build trust. With that trust they can - simply put - vote on transactions by countersigning them which gives the transactions consensus value. When nodes sync with each other they don't pcik the longest chain like in bitcoin but they pick the chain with the highest concesus value i.e. the chain that most trusted nodes signed.

Is that about right ?

For the purpose of understanding trust consensus you do not need information about the ledger or how it is constructed.

There is no chain as specified in the document, there are no blocks either.  Transaction are validated and committed individually in real time.  The ledger is 100% different from Bitcoin and everything else.

If me and you both submit a transaction 10 seconds apart, then mine will be committed and final around 10 seconds after yours.

There is a record of all transactions though right ? And nodes do agree on that record right ?

[Fuserleer]

So let's see if I understand this.

Nodes build trust. With that trust they can - simply put - vote on transactions by countersigning them which gives the transactions consensus value. When nodes sync with each other they don't pcik the longest chain like in bitcoin but they pick the chain with the highest concesus value i.e. the chain that most trusted nodes signed.

Is that about right ?

For the purpose of understanding trust consensus you do not need information about the ledger or how it is constructed.

There is no chain as specified in the document, there are no blocks either.  Transaction are validated and committed individually in real time.  The ledger is 100% different from Bitcoin and everything else.

If me and you both submit a transaction 10 seconds apart, then mine will be committed and final around 10 seconds after yours.

There is a record of all transactions though right ? And nodes do agree on that record right ?

Yes of course, the trust consensus is to ensure that all nodes agree on the valid set of transactions that make up the ledger.

I'd like to just clarify that all transaction information is public, other than who the sender is and who the receiver is.  Any person can audit the ledger to ensure that balances hold enough to make the transactions, audit the fee and supply distributions, audit new supply etc etc.  All that is hidden is the participants.

You don't need to know this information to understand consensus, and all of the above will be covered in ledger documentation.

[monsterer]

Lets assume that the network is processing 1 tx/s (similar to Bitcoin) which is a total transaction load of 86400 tx per day.  A attacker using a Sybil ideally wants 40%+ as stated in the article so that he is able to influence the outcome of the TCW process enough to direct which transactions are selected.  Thus he has to produce ~35k transactions per day, the cost of which is 35k * 0.01 = $345 per day.  Not a great deal, but not insignificant either.

Why does he *have* to produce transactions himself (and therefore pay the fee) in order to gain trust? Can't he just act like 2000 normal nodes for a while until he has the necessary trust and then perform his attack?

[monsterer]

I'd like to just clarify that all transaction information is public, other than who the sender is and who the receiver is.  Any person can audit the ledger to ensure that balances hold enough to make the transactions, audit the fee and supply distributions, audit new supply etc etc.  All that is hidden is the participants.

You don't need to know this information to understand consensus, and all of the above will be covered in ledger documentation.

You kind of do, because otherwise you can't analyse things like the long range attack. I'd appreciate a brief description of what the ledger looks like