How fast can eMunie go? Heres how fast....

[Fuserleer]

Thought I would post this here too, as some may be interested.

Today we ran the first real test of the new channeled ledger aka Transaction Forest, and despite most of the code being very untested, incomplete and in need of optimizing, the result was pretty good!

Before I get to exactly how fast, let me first set a few points you should keep in mind:

    The test was ran in a small network of around 10 nodes
    Due to the small network size, only around 20 channels were in use at any one moment
    Live network will support potentially unlimited channels
    Most nodes were also running all other services as well as transaction verification (hatching)

After setting up and performing some ramp up we set about applying load to the network, this is simply done by cross spamming each other with transactions, with each node producing 2-3 transactions per second to the network for processing.

With all nodes participating, there should be between 20-30 total transactions per second being presented to the network, however it is difficult to determine exactly the transaction quantity each node is presenting at any one time, and if all are indeed operating at the expected pace. Therefore it can (and probably does) vary throughout the test by perhaps +/- 5 tx/s at any one moment.

Our peak processing load during the test was 110 tx/s, with many peaks hovering around the 50tx/s mark, see screen shot below:



As you can see from the graph, there are definite regular peaks of transaction load, with a baseline transaction throughput around the 15tx/s mark.

What you can see here is a saturation of available resources, most specifically, bandwidth! Most of the nodes in the test network are behind ADSL connections, and while having good downstream bandwidth, many of which have only 5-10Mb/s upload. As I'm sure many of you know with ADSL, the more of the upload bandwidth you use, the download availability and ping rates suffer dramatically.

All of the nodes in the test were hatching, and connected to a few other nodes, which results in a significant amount of upstream being used for broadcasting verified and countersigned transactions, along with additional upstream bandwidth being used for data delivery for other services, most nodes were hitting upload bandwidth limits.

The graph reflects this almost perfectly, you can see there is a burst of transaction verification activity from all nodes, which quickly saturates most nodes upload bandwidth, preventing them from presenting transactions quite as fast as they would like. The lesser loading after the peaks, are those slower frequency transactions making it out to the network, along with the 2 nodes that were not upload limited as they were sitting on 100Mb/100Mb lines. Shortly after, everyone catches up, upload saturation drops, and we have another burst of transactions around the network. Rinse and repeat.

If we average the graph, we arrive at an average transaction throughput of ~30-35 tx/s, which is as we would expect with 10 nodes performing as explained.

This effect is exaggerated in our test, as all nodes are doing verification, all nodes have a route to all other nodes, and there is low channel utilization. Therefore, all nodes need all new data for all channels immediately, as all nodes are verifying transactions for all channels. In a production environment, where verification is global, and spread over many many more nodes, these nodes will only need a very small portion of all channel data to service the transactions they are processing, thus upload saturation of nodes within any interconnected group will be much reduced.

With that in mind, the global transaction processing capability of the eMunie network will be very high indeed, as 10 nodes can process 30-35 tx/s average, and the scalar as the network grows should prove to be near linear, 1000 nodes could in theory process in excess of 20,000 tx/s globally.

That said, the peaks are real processing happening in the network as the graph is generated by countersigning time and not when the node received the transaction (take 2 graphs, from 2 nodes and they are identical).

So, as to the question, how fast is eMunie, there are 2 valid answers....in a 10 node test, 30-35tx/s if you take the average, 110tx/s if you take the peak. Which you choose is up to you, but I think you'll agree, either of those numbers, on commodity/consumer hardware, is mighty impressive

[Fuserleer]

Might also be worth pointing out that while BTC (or your favorite alt) could match this test, they can't do so efficiently.

Transactions in the eMunie network are verified within 10-15s, and in a test such as the above, mature within another 20s or so.  Network latency & propagation is low, so consensus reached quickly, thus maturity is low.   In a production environment, network latency/propagation will be higher, therefore so will maturity, perhaps in the order of about 60s.  I read somewhere a while ago that BTC network propagation of a block can take between 30s and 2 mins, to achieve peaks of 110tx/s BTC needs Gavins 20MB block hardfork, which in turn pushes network propagation up, and could result in more orphans in slower parts of the network.

The point to remember is that once mature, an eMunie transaction is as good as 6 BTC confirmations at whatever hash rate you like to pick (adjust accordingly for your favorite alt), and doesn't require either special hardware, or the burning of 70 tons of coal for each block.

[freequant]

Sounds interesting.
Where can I find the source  code?

[Fuserleer]

Sounds interesting.
Where can I find the source  code?

...about 6 months after we have launched.

Yes yes I know, and no no I'm not having another open v's closed source debate

[freequant]

Sounds interesting.
Where can I find the source  code?

about 6 months after we have launched.
Yes yes I know, and no no I'm not having another open v's closed source debate

No worries, it's perfectly fine that you don't want to be plagarized before you even had the time to launch.
What's the go-live ETA btw?

[Fuserleer]

Sounds interesting.
Where can I find the source  code?

about 6 months after we have launched.
Yes yes I know, and no no I'm not having another open v's closed source debate

No worries, it's perfectly fine that you don't want to be plagarized before you even had the time to launch.
What's the go-live ETA btw?

Yeah thats exactly the point and a downside of open-source from the get go.

Go-live is still tentative end Q1/start Q2, lots to do before then though, papers are finally being worked on now that most of the tech is prototyped and nailed down, first of which should be publicly available in a month or so.  After that we want to have a decent open beta, so that people can test drive it, play with it, whatever...and we can iron out any final issues before going live.

[13Darko]

... the global transaction processing capability of the eMunie network will be very high indeed, as 10 nodes can process 30-35 tx/s average, and the scalar as the network grows should prove to be near linear, 1000 nodes could in theory process in excess of 20,000 tx/s globally.

That's fantastic! Bitcoin is only 7 tx/s and 20,000 tx/s looks like a VISA network So.. can I say eMunie is a Bitcoin KILLER and direct competitor of VISA?

[One Six]

When is emunie coming out?

[DrGrid]

The results are really impressive, thanks for the detailed explanation. Would be interesting to see what would happen to the transaction bursts, when the latency of the global network starts to kick in. Have you conducted a simulation yet, that would prove it to scale linear?

[billotronic]

only 10 nodes? BOOOOO

Come on man, thats not a sexy test at all! You have 10 nodes in your bathroom for fucks sakes.

[tokeweed]

... the global transaction processing capability of the eMunie network will be very high indeed, as 10 nodes can process 30-35 tx/s average, and the scalar as the network grows should prove to be near linear, 1000 nodes could in theory process in excess of 20,000 tx/s globally.

That's fantastic! Bitcoin is only 7 tx/s and 20,000 tx/s looks like a VISA network So.. can I say eMunie is a Bitcoin KILLER and direct competitor of VISA?

yes you can.

but we'll see when it officially goes live.

[Fuserleer]

... the global transaction processing capability of the eMunie network will be very high indeed, as 10 nodes can process 30-35 tx/s average, and the scalar as the network grows should prove to be near linear, 1000 nodes could in theory process in excess of 20,000 tx/s globally.

That's fantastic! Bitcoin is only 7 tx/s and 20,000 tx/s looks like a VISA network So.. can I say eMunie is a Bitcoin KILLER and direct competitor of VISA?

Perhaps, time will tell.  The 20k is all based on theory, solid theory but theory nonetheless...and we all know how theory can out in the real world

When is emunie coming out?

Soon, I mention the timeline in a early post in this thread.

The results are really impressive, thanks for the detailed explanation. Would be interesting to see what would happen to the transaction bursts, when the latency of the global network starts to kick in. Have you conducted a simulation yet, that would prove it to scale linear?

Those "bursts" should not happen, or at least, be a lot less severe, due to the network topology.  In these test the topology is nothing like what production would be, tight cluster cross-spamming large transaction loads just isn't going to happen on a global scale.

We have a number of additional tests planned where we will scale this up, so we'll see how close to linear it is.  I'll report back when we do them

only 10 nodes? BOOOOO

Come on man, thats not a sexy test at all! You have 10 nodes in your bathroom for fucks sakes.

LOL I do, but this wasn't a local test, this was a true WAN test over the internet.   LAN tests yield much higher transaction throughputs,

[Ix]

From the point of view of the bitcoin world, bandwidth saturation being the tx/s limitation is not surprising - the block size limit is completely artificial after all, and modern commodity CPUs can verify thousands of tx/s. I know from what you've said in the past that transaction verification is more complex in eMunie, but you've never actually given any technical detail that I've seen so it's hard to gauge why 30-35tx/s is impressive. A bitcoin node running in a similar test would be using somewhere around 100kB/s in upload bandwidth in the worst case scenario.

This effect is exaggerated in our test, as all nodes are doing verification, all nodes have a route to all other nodes, and there is low channel utilization.

To a technical person, some of what you say sounds like lines out of Star Trek describing fictional devices. What a channel is is not defined except by some PR in other posts. If your whitepaper describing this will be out soon, why not wait to post this sort of thing until after? Otherwise, no one is able to draw any of their own conclusions from the data or assert the validity of yours.

[Fuserleer]

From the point of view of the bitcoin world, bandwidth saturation being the tx/s limitation is not surprising - the block size limit is completely artificial after all, and modern commodity CPUs can verify thousands of tx/s. I know from what you've said in the past that transaction verification is more complex in eMunie, but you've never actually given any technical detail that I've seen so it's hard to gauge why 30-35tx/s is impressive. A bitcoin node running in a similar test would be using somewhere around 100kB/s in upload bandwidth in the worst case scenario.

This effect is exaggerated in our test, as all nodes are doing verification, all nodes have a route to all other nodes, and there is low channel utilization.

To a technical person, some of what you say sounds like lines out of Star Trek describing fictional devices. What a channel is is not defined except by some PR in other posts. If your whitepaper describing this will be out soon, why not wait to post this sort of thing until after? Otherwise, no one is able to draw any of their own conclusions from the data or assert the validity of yours.

Actually a commodity CPU would struggle to verify even 1000 tx/s in Bitcoin with a single signature check per transaction. ECDSA verification is not trivial, and additional to that there are various other tasks to perform to validate (touching the DB being the other expensive task). If that wasn't the case, then how come one of my 32 core, SSD servers on a 100Mb line still takes half a day to full sync?  By my calculation with a 60M blockchain, that is around 1,300 tx/s......hardly 1000's upon 1000's, yet a 100Mb line running at 80-90% of load, with a ton of BTC nodes, and BTC transactions averaging about 600b should be pulling in about 16,000 transactions per sec....hmm...

Maybe someone in the BTC community should do a real report on syncing the chain, because there are no real tests been done of performance, and everything everyone writes (including me) seems to be speculation on educated guesses

The processing ability of eMunie in raw speed isn't the problem, in a sync state (node syncing from 0 transactions), verification throughput is anywhere from ~150-300 tx/s on commodity stuff depending what the transaction is doing.  You'll see in a post just after my OP, I do say that Bitcoin could do the same thing to a point, but what is "impressive" is the efficiency of which this is done in eMunie and the time it takes for a transaction to be confirmed. 

Also, Bitcoin doing 35tx/s isn't really Bitcoin doing 35tx/s is it, it would be 21,000 transactions every ~10 minutes (if you wanted to base that on 1 confirm).  You are correct on the bandwidth usage of both though, a Bitcoin node would be using similar upload resources to do the same thing due to the fluid nature of the UTXO and broadcasting it.

Hopefully the next few tests will leave you more impressed  

[starik69]

With that in mind, the global transaction processing capability of the eMunie network will be very high indeed, as 10 nodes can process 30-35 tx/s average, and the scalar as the network grows should prove to be near linear, 1000 nodes could in theory process in excess of 20,000 tx/s globally.

Hm... Linear?  
It is only hope or has some math to prove?
BTW. About release. I said many times - DECEMBER!

[Crestington]

Quite interesting indeed!

What are you looking at for scalability in terms of disk space size? The major problem with Bitcoin is the size of the Blockchain and the time it takes to Sync to the highest Chain. You can have the ability to confirm thousands of tx's but if after 5 years it requires terabytes of space to run a node, that's a big problem.

What kind of reward mechanism do you have? is it a type of Proof of Work or Proof of Stake? What is the incentive to use and hold onto eMunie beyond just the speed of transactions?

[Sharky444]

It's neither POW nor POS. You are still thinking "blockchain". It's so 2012.

[Crestington]

It's neither POW nor POS. You are still thinking "blockchain". It's so 2012.

Thanks captain obvious, care to answer my question on the reward mechanism then?

[okiefromokc]

The rewards/earnings are based on how much support your node provides to the network.  How many transactions you verify, and/or how much network time your node is providing various services to the network. If you are just running a bare client without providing any other services to the network then your earnings will reflect that.

Also, there will be interest earnings from emu balances on a periodic basis.