I'm glad to hear that the DIGI solution is being implemented.  But let me say this- this isn't a win for me.  My team and I just took the time to test the solution we thought would work, and it worked.

The win here belongs to the community.  While everyone might not have the same ideas, we came together and made a decision as a whole.  Sure, I really pressed hard for the decision to favor my team's findings, but we made a decision none the less.

Actions speak louder than words.  Today we spoke.


That being said, I would love people to look over the code to make sure I didn't miss anything.  /GJ will be looking at it, but a second or third set of eyes would be good to make sure we're solid going into the transition.  I know I missed adding a checkpoint in my git pull, but I figured I'd let the devs make the checkpoint additions.  The code, the way I submitted it, was the code used to get the graph results I presented, minus the change from 20% to 25%.  Our results with 25% were very aggressive, but still effective.  I don't have charts for the 25% tests, but I can say that they were similar in movement.  Either way, we're not going to overshoot the sweet spot, and we're not going to drop to a false low.

-Fuse

Honestly doesn't surprise me.  There's a possibility that he's doing this with NLG as well.  Using pool miners to mine the coin, then hoarding a considerable amount instead of selling to pay the miners.

I'll let Terk speak for himself though...............................

-Fuse

I can't speak to the testing done on DGW3.  I don't know what happened with DGW3.  Right off the bat, we experienced some shakiness.  I remember the posts about waiting to give it time to level out so that the swings weren't so dramatic.  But the swings stayed pretty dramatic.  My only guess is that the target timespan vs actual timespan is causing a more exaggerated swing in difficulty with the longer block time.  Kilo proposed this to me in Criptoe chat, and it's why we brought up shorter block times.  It's my only thought as to why it didn't work.

-Fuse

You will undoubtedly lose miners.  I for one wouldn't CPU mine.  I did it once with YAC and I hated it.  The stress it put on my computer was horrible.  Imagine the person who doesn't know any better and melts their Macbook pro trying to CPU mine while they're asleep.  No bueno.

You assume that we are going to gain miners because everyone has a CPU.  That's like saying if you have a car you can race in the Grand Prix.  The typical non-experienced miner is not going to CPU mine.  They will pay $20 to buy cloud miners though.  Clever proves that with it's available hashrate.

-Fuse

You don't just change from GPU/ASIC based mining to CPU mining after 9 months of mining.  I have a hard time believing you don't have a reason for suggesting it other than the fact that it would eliminate the MPs.  But you forget that it will eliminate miners in general.

What about alienating the already established mining population?  Dedicated miners who have held the chain together for months now.  Are you ready to tell them that they have to abandon their gear and adapt to a completely different way of mining?

It's nonsensical at best.  I said it when you first recommended it, and I'm saying it again.

-Fuse

Exactly this ^

Absolutely no need to change from scrypt.  Doing so would alienate the majority of dedicated miners.  We can't make a change like this because you made the mistake of buying rackmount blade servers instead of ASICs.

-Fuse

The charts I provided were for 20%.  25% is really aggressive, but it works as well.  That is the only real change made besides the retarget time variables.



No, forks aren't caused by the algo, but a fork instantly alters the network hashrate when it splits and then rejoins.  With enough hashrate you can fork anything.  That's something the simulator won't account for that a testnet can.



Correction... the algorithm is aware of the percentage of time elapsed between blocks.  That's how DIGI calculates the next difficulty.  Go over 150 seconds, the difficulty shifts down.  Go under, and the difficulty shoots up.  All proportionate to the percentage of time.  Time is a huge factor in our issue here.  One that has been discussed to the point of completely ignoring blocks under a certain time.  By reducing the total block time, you allow for smoother adjustments that happen more often, and DIGI eats that up like roast beast at a Whoville feast.



By scaling, we mean being able to increase in hashrate without issue.  Whether we're at 10GH total network hashrate or 10PH.  However, once we get past a certain point, the event horizon of sorts, you no longer need a readjustment algo like DIGI or DGW3.  You just implement the stock algo and let it ride out until the end.

As far as examples go- any coin, pre-hashlets, that implemented DIGI.  I'm sure I could track down a list, but I know POT for instance did it.  They shot themselves in the foot with a flawed block halving scheme, but their network hashrate increased substantially after their DIGI implementation, and the multipools dropped off almost completely.  There are others that had similar results... just search the forums.



The code you highlighted is not what I based our code on.  This is the DIGI function, as it was on our testnet:


You need this:


And you change this to set the max change:


That's for 20%.  Replace the 120 with 125 for 25%.

It's pretty straight forward.  Compared to the original algo for LTC, and the GetNextWorkRequired_original function, you can see why I would say it's only 1-2 lines of additional code.


DIGI accounts for time, so it scales with the block time.  The difficulty doesn't change because you halve the block time.  DIGI says this is what the time should be, and what percentage of that time was the last block.  If you go over the time, the difficulty goes down, if you go under the difficulty goes up.  In our current code, you would be correct with this.  I'm not talking about our current code though.  I'm talking about the code that will fix our situation.  Halving the block time isn't a necessity.  Keep it where it is and use a higher max adjustment.  The selling point here is that with a shorter block time, you can reduce the max change, and get more adjustments per period of time.  Again though, not a necessity.  Just an added step to making things better.



Please don't let the port to the simulator hold up the talks.  While I am in agreement that the simulator should be used, I need to know if it is ready to take in to account everything needed for real-life results.  If it isn't ready to simulate every possible variable, then we need to put it aside for these discussions.  If that's not an option, we need to know when additional features will be implemented so we can press forward.

Additionally, the simulator needs to be verified against a chain.  I would seriously suggest creating a post in the development sub-forum here to get it out to the public and let them develop and test it for you.  There is no reason why it shouldn't be made public.  It will strengthen development of the simulator and it will bring publicity to NLG.

I'll reiterate this point though-

You can't predict the future of mining, so you can't future-proof the algorithm.  I've been in crypto for a little over 2 years now, active here for 90% of that time.  Things change dramatically over small timeframes in crypto.  People need to realize that no algorithm or solution will ever be the final answer.  Coins need to adapt to the changes.  It's evolution at a code level.  It you don't adapt and overcome, you can't survive.  But you also can't adapt to something that hasn't happened yet because the world can take the opposite turn, and then you'll be left exactly where we are right now.

Don't be afraid to make future changes.  It's not a sign of weak coin planning, but rather a sign of active adaptation to an ever changing landscape.

-Fuse

Not FUD if it's true.  Vesper and I tried to warn everyone, but the opinions on Reddit outweighed those here.  The POT block halving debacle of 2014, as it shall now be called, was the downfall of this coin.  The coin has never rebounded since, with the exception of the marijuana pump and dumps we witnessed about 2 months ago.

Here's two months of FUD for you:



-Fuse

This.

/GJ, don't take my post as an attack, or what I'm going to say right now.

You spent that entire post trying to debunk my team's findings, and shoot down the possibility of using DIGI.  However, you didn't mention a single plan of action other than to state that a more complex algo needs to be created.

A plan that has been needed for months now.  Give us direction.  You're the rudder.  Don't let us steer ourselves into the rocks.

-Fuse

You change the max difficulty adjustment.  Other than that it's standard DIGI.



Can it simulate a fork caused by an isntant, massive hashrate increase?  You also talk about luck with a 100MH miner... well the 600GH miner is going to have luck too.  Computer simulations, while very accurate(when proven so) are only as good as the code.  There's a reason why robot AI isn't to a point of consciousness yet.



I think you're missing the point here.  The reason Kilo brings this up is VERY valid.  If you have more iterations of retargeting in a specific period, the more change can happen.  Lets say the NLG codebase was only allowed to be altered every 6 months.  You would have to make a lot of major changes to account for everything that happened in that defined time period.  Maybe something happened 5 months ago, but now you're waiting to catch up.  If you say you can change the codebase every 3 months, now you're only 2 months behind instead of 5.  Mind you, I know the blocks carry the changes with each block, but when they are targeted for a longer time, they reaction is more severe.  I'll get back to this.



Averaging difficulties to calculate the newer difficulty is more complex in ways.  It's allows more room for error and skewed calculations.  It's pretty obvious DGW3 doesn't scale well.  You can't argue that because the chain shows it.  I think it has for a long time now.

I'll take a stab at what Kilo meant by elegant.  DIGI is simple.  It's a 1 to 2 line addition to the base LTC difficulty algorithm.  That's it.  It's elegant in that it's simple and it works.  Additionally, Kilo and I have witnessed DIGI implementations over the last year that scaled very well.  When a chain's hashrate grows 5X it's size in a day and stays healthy, I would say it scales pretty darn good.



Completely related if you reduce the block time to accommodate for an algo that is meant for shorter block times.  A 150 second block time works for LTC because LTC uses the standard algo with a shit ton of hashing power, think petahash, behind it consistently.  NLG doesn't have that.  Most other altcoins that have healthy chains use a block time closer to 60 seconds but not less.  90 seconds would be optimal, but Kilo's 75 seconds is probably a very safe number for NLG.  Less than 60 seconds and you run the risk of timewarp attacks.  Kilo has successfully done this to a few coins just to prove the maths.  But that aside, shortening the block time is a solid idea.  It reduces the magnification of the retarget changes and allows for faster reactions.  If you shorten the block time, you have to reduce the reward, unless you want to mine out faster.  Normally, I would be against reducing block rewards, but halving the block time and the block reward gives you the same coins mined daily, but with 100% more difficulty changes in that time.



I would say the 50% stuck time is conservative with DIGI.  I would guess you would reduce the amount of stuck time(blocks that take forever to mine) closer to 90-95%.  The chain would function smoother, and not make wild swings into the stratosphere like it does now with DGW3.  The 100% increase in difficulty adjustments is correct if you halve the block time.  The reduction in the amount of blocks that clever could mint is about right.  It's not hard to see in the charts I posted that if you threw 10X the network hashrate at the chain, based on profitability, you're profitability would be gone before you could grab 15 blocks in under a minute.  There's no averaging, so DIGI doesn't need to take in to account the false lows that DGW3 produces.  The reaction is instant instead of delayed because of a moving average.

The numbers, while not statistically absolute, are sound.  No need to dismiss them.



I don't take the remarks personally, at all.  I think you are 100% incorrect about DIGI not handling the hashrate spikes though.  We've proved it on a testnet.  It's funny... we presented actual data, but there's still people that deny it, like it was all made up lol.  DIGI works, the graphs back that up, and the solution will work until the entire altcoin environment makes it's next paradigm shift.  You can't predict the future of mining, so you can't create an algo for it.  You can however use what is effective in this era of mining.  DIGI is the answer.

-Fuse

Trust isn't necessarily the issue here.  I trust him completely.  I just want to see things move along and not sit stagnant.

I've spoken in the past with /GJ about development in PM, and he expressed his dedication to NLG development, but lack of time and funds to make it a dedicated job.  Let's face it... in most cases, devs do this in their free time because they need to pay the bills with a full-time job.  I'm not entirely sure what /GJ does for a living outside of crypto, but if using dev premine to pay him to allow him to dedicate more time to development helps, I'm all for it.  This is all up to /GJ though... we can't tell him to quit his job and only work on NLG.

As far as GO skills go, that's not my bag.  I just don't have the time to put into learning it right now.  I'm sure I could, but I've got enough on my plate.  However, if the dev team and community decide to push forward with DIGI, then I'm ready to create the code git pull if needed.  No dev time is needed on the dev team's end, minus the windows compile.  At least that could take a little pressure of the team.

Like others have said though, I'd like to hear from /GJ and see what he has to say.  I'm ready to act on any decision, I just need direction.




Love the story, and the analogy!

-Fuse

I'll start looking on Odesk and Elance.  Anyone with C++ and GO experience would be able to do the port.  I'd rather let /GJ decide on whether he wants to dev the rest.  It's his baby after all.  Has any post been made to announce the simulator to the crypto community?  There is a dev section of the forums that is filled with very competent devs who eat up this kind of development.  I would guess that announcing it there would lead to an "outsourcing" of community development.  Might just be easier to go that route.

What I say next might seem argumentative, but it's really just for my clarification.  Don't take it the wrong way.

With respect to the testing, how is real world mining not actual maths?  It is how the algorithm is going to behave on a chain, rather than how a simulator says it will behave.  I would think if you wanted true test data you would truly test it, would you not?  Like I said, I'm for testing against the simulator, but we need to know that the simulator(which simulates actual mining, not actually mining) is giving us substantiated results.  We can't say the simulator is the key to testing algorithms without testing the simulator against an actual testnet first.

For example, you would need to mine on a testnet.  You would simulate large wave, small wave, etc. and record all the data.  Then you would need to replicate that chain with the simulator, and compare the results.  If the simulator was off by a predetermined margin, you wouldn't be able to say the simulator was accurate.  After all the testnet chain would be the definitive data... it's the actual mining.

So to say that testnet data is an educated guess is a little off base.  It is the actual, substantiated data collected from mining.

-Fuse

I'd love to see the simulator results as well, but like /GJ said himself, there is still work to be done to properly test large wave simulations, and of course there's the matter of porting DIGI.  I'm all for testing with the simulator if those features can be implemented in a timely fashion.  Whether that's with outside, paid help, or through /GJ, paid via dev premine, or unpaid... I don't care.  Let's just get roadmap, action plan, whatever figured out and in the works.

-Fuse

Any update would essentially require a fork, because you are in fact forking away from the minority(unupdated users) with the majority(updated users).  As long as a majority of people upgrade, the fork isn't noticed.  But BioMike is correct in that a change can happen at a specific block.  That's why I mentioned that if I had a desired "go-live" block number, I could post the git pull if that's what the dev team and community wanted.

Again, I'm 100% behind whatever the dev team wants to do.  I just want to present this plan of action because my team feels it's a smart choice.  If /GJ wants to see other options, I'm all for it.  Keeping DGW3 and shortening the block time is an option, DIGI with a shorter block time is an option, a new weighted average algo is an option.  There's a lot of different ways this can go.

However, 20% DIGI with our current block time works, and it's an easy fix.  If someone has a better solution, I want to hear it because I want this to succeed.

As far as paying /GJ for development, I'm all for using the premine for it.  I also mentioned using a portion of the physical coin profit.  I'm not going to make millions off the coin sales, but maybe I can help with a server bill to offset dev costs for a month or two or something.  Or maybe a case of [insert caffeinated beverage here] to keep the fire stoked lol.  IDK... just tell me how to help.  Give me a plan of action.

-Fuse

Edit:


I would like to hear from /GJ as well, and I stand behind a vote.  However, if /GJ wants to test DIGI against the simulator, I would want to see a roadmap of how we're going to get to that point with all the needed variables(large wave, DIGI code, etc).

Additionally, if a vote is created in the official forums, can you do it in English as well?  I feel lost trying to get info there, as most of the current discussion happens in Dutch.  I know, I know... it's a Dutch coin  .  I just want to keep abreast of what's going on.

Click on the pictures for a larger view.

Overall testnet chain difficulty chart at 20% step limit:



The first 2 jumps are from baseline to 5 times the network hashrate.  After that, the jumps are 10X increases, and the highest peak is when we toggled a 15X increase for giggles.  In the following pictures you will see the difficulty values and the steps up and down instead of instant spikes and crashes.  Notice that each time the hashrate was removed to go back to baseline, it never dives below the baseline.  It tapers back, and when the block is found, it stabilizes instantly.  This is something DGW3 isn't doing properly.

5X increase in hashrate:



15X increase in hashrate:



10X increase in hashrate:




It's getting late here, and the wife wants to watch The Interview, so I'm taking off.  I'll be back in the morning to answer questions and provide any needed info.

-Fuse

While I can understand the skepticism, I will assure you that my team is dedicated to helping this coin however we can.  So we decided that regardless of what was happening, we would test DIGI anyway on a testnet.  I've got a block explorer with the data on it, so I'm putting it together now.  I might install my data analysis software from work here so I can do a bit more than excel can.  So it might take me a little bit, so hang tight.

As far as Terk liking DIGI, I'm not sure he "likes" it per-say.  In my PM convo with him, I feel like he was at least a little sincere when he said it was the right move.  At that point, I don't think he was the full-bore asshat he ended up being now.  My initial thoughts were for DIGI, way before we even knew it was clever that was messing us up.  I've seen proper DIGI implementations with coins that had 2-3 times the hashrate we have had thus far.  The instant the DIGI went live, MPs dropped off and mining was stable.  Sure, clever will still mine NLG, but they won't pull 20 blocks and leave us with a sky-high block that takes an hour to solve.  Maybe they make off with 1 or 2 before they are left unprofitable.  At that point, it might not even be worth it to them to mine, as the daily profit is significantly less.

All that being said though, we haven't proved the reliability of the simulator results.  While I am 100% behind it, how do we know that the results we'll see with it will be the same as with a testnet?  It is in fact a first version that has really only been tested by a handful of people with a small subset of variable conditions.  If we treat this like any other scientific/mathematical research, it needs to be peer-reviewed, tested and substantiated.  I'm not saying that we absolutely have to do that.  I'm just saying we need to know it's results are as reliable as an actual testnet.  To fully stand behind it as the sole decision making tool is just as much of a leap of blind faith as trusting me.  We still need large wave simulation and we need to verify that data that we find is similar to what we would find on a testnet/live chain.  It is a simulator, not actual mining... there could be variables that just won't present themselves with it.  Again though, I'm all about it's development and couldn't be happier that it was made.  I just don't think we can wait to get it sorted before we work on the algo.

AGAIN... not knocking the simulator.  I just want to make sure we are using the right decision making tools, and using them in a timely manner.

Just my 2 cents.

-Fuse

No, you're on point, mate.  I never released our results publicly.  And I will be completely transparent in saying that our tests were done on a smaller scale, but with proportionate hashrates.  We got baselines, then threw 10x the hashrate at the chain.

Reaction times were very quick, and extremely effective.  Difficulty ramped up fast, and dropped off in a controlled fashion back to baselines, instead of plummeting into insta-mine territory.

Instead of the traditional 10% limit DIGI uses for per block difficulty jumps, we upped it to 15, 20, and 25%, testing each time.  The increase was to account for the fact that DIGI is usually implemented with coins that have a block time closer to 1 minute.  That means that there are 2.5 times more retargets per hour/day/etc with a 1 minute block.  Our block time plays against us here, and I'm thinking it might be part of the reason DGW3 isn't working as intended.  So we found the magic number to be either 20% or 25% for DIGI.  You could shorten the block time(and rewards to match), but that's a major change.  I think the change would help, but I'm not going to push for it at this time.  If you don't increase the number of retargets(shorter block time), you need to increase the difficulty limits.  So the results I will post in a bit is for the 20% limit.  25% was great, but it might be too aggressive IMO.

As far as another change down the road... well we can't predict what will happen down the road.  What's to say that quantum computing doesn't hit next month and we are in a world of hurt regardless of what we do.  I will tell you that coins that implemented DIGI well over 4 to 5 to 6 months ago are still solid with it.  It the algo Terk himself recommended, and the one I pushed for for almost 4 months now.  There's a reason behind that- it works.

I'm holding the baby right now, so this reply took me quite a long time to write.  I would post the stats right now, but this is proving difficult lol.  Give me a little bit and I will post charts and some explanations.

-Fuse

Easiest action is implementing DIGI.  However, /GJ made it clear that he wants to see a GO port of the DIGI algo for the simulator.  But besides the DIGI algo, we need the additional wave simulations that /GJ needs to implement for that testing to be completed.  Testnet work was done by my team, and we're confident it would be the right move.  But we're not the dev team, and our opinions aren't the only ones that matter.  That's why I've been asking for the dev team to come and discuss an action plan for the algo change.  I don't need an ETA for when the change will happen, I just need to know that we are working towards a change.

I think you're 100% correct though in people losing interest/patience because of the delay with the algo.  A member of my team pulled his hashrate from the network because of it.  It's not a substantial amount, but it's not little either.  Another is considering it.  You have to think that there are a handful of people like them that have done so as well.  I'm not saying we need to rush, but we need to not be sitting here waiting for dedicated miners to jump ship and CM to take 80% of the network.  We need to formulate a plan, and the devs need to be involved in it.

Just to let you know though, if I was given a block number when the change would go live, I could post the modified DIGI code git pull request right now.  It's not a difficult code change.

-Fuse

Any thoughts on formulating a algo action plan?  I'll give you a few reasons why we should:



-Fuse

pm sent

-Fuse