https://dev.gulden.com/august-development-updateAugust Development Update
We are now quite a bit past the initial release estimate for PoW² and while we are almost ready for release we are still not quite there yet.
Understandably some are quite anxious to see the release happen, as we ourselves are too, however good development cannot be rushed so we must be patient for a bit longer.
As we understand that it can be more difficult for those outside the project without the knowledge that we have on the inside of what is going on, and that in the absence of this information some start to assume the worst, we thought we would give a bit more information to those that are waiting.
Sequence of events
I) SEGWIT – OCT 2016
As some of you may know ‘SegWit’ has been a long ongoing argument in the Bitcoin community for quite some time now.
It first came onto the Gulden radar when we released 1.6.0 last year, with quite a lot of people questioning whether we would activate it etc. – at the time we updated our codebase to contain SegWit but stopped short of activating it.
I don’t want to get into a massive discussion on this but at the time our reasoning for not activating SegWit was as follows:
1) There was no immediate pressing need for us to do so.
2) It was unclear whether it would ever activate for Bitcoin themselves in its current form or whether they would eventually give up and implement it differently.
3) The current implementation of SegWit as done by the Bitcoin team was non-optimal – for our use case.
While we can understand why the Bitcoin team did SegWit the way they did with a focus on backwards compatibility at all costs (even though there is argument to be had that perhaps they should have done it differently).
We don’t believe it makes sense for us (or really any other currency that isn’t Bitcoin) to copy it, we don’t have the same backwards compatible requirements that they do so temporarily having larger transactions just to be backwards compatible doesn’t make sense. Likewise it doesn’t make sense to us to not take the opportunity to also implement other pending improvements to the transaction format.
We did not want to activate something that was not optimal, but we also did not want to spend time creating a ‘better’ SegWit only to find that the Bitcoin people do the same and we could have just used their work for ‘free’, as a small team we need to try and work efficiently as possible and be strategic about what we spend our work time on.
So while we internally had a list of things we wanted to see if/when we activated SegWit we adopted a “wait and see” attitude to see what the next move from Bitcoin would be.
II) POW² – MAY 2017
After a huge amount of work to develop and prove the concept possible in May 2017 we released the whitepaper for PoW², the idea was to finalise development and head straight to releasing PoW² as a hardfork – i.e. all older clients would no longer be able to partake in the network after a certain fixed block height, as we have done for previous updates with delta.
Part of the PoW² concept requires the creation of a new ‘address type’, and while this can technically be ‘shoe horned’ into the existing transaction format, doing so is not the best way to do it, and we want to try and avoid introducing ugly hacks like this into the codebase in favour of keeping a clean codebase.
III) SEGSIG – JUNE 2017
While working on the optimal way to implement the new address type, it became obvious that many parts of the codebase that need to change to accommodate this are the same parts that are affected by SegWit.
At around about the same time after watching SegWit for some time, it become clear that though the Bitcoin team had still failed to activate SegWit there was now a clear determination to do so and that one way or another SegWit in its current form -will- eventually be activated, we therefore had to at this point give up hope that the Bitcoin team might develop a better SegWit which makes the sort of changes we were hoping for and instead accept that if we wanted this it would have to be something we do ourselves.
And so the concept of SegSig was born, we were faced with two possible paths forward:
1) Release PoW² hardfork as originally planned. (including extensive testing)
Go through the work of having everyone including exchanges and mining pools upgrade to it.
Produce SegSig as a hardfork, which involves changing a lot of the same code as PoW²
Put our testers through extensive testing once again.
Go through the work of having everyone upgrade for a second time in a short time period.
2) Combine PoW² and SegSig together into one release, despite possible delays and release it all at once.
This was not an easy decision, I am not a fan of trying to do too many things in one release as this has the potential to get out of control and normally I would prefer to try and seperate things into multiple small releases. As a developer it would have been easier for me to take option 1.
However there were multiple special factors here:
1) Both releases change similar parts of the code.
2) Both releases are major and require extensive testing – with major overlap in what needs to be tested.
3) Both releases break backwards compatibility and as such require coordinated efforts to get all services updated correctly.
Some of the main points on which this decision hinged:
1) I felt that having our testers test the same area of code twice in a row in quick proximity was likely to cause “tester fatigue” and that bugs were more likely to creep into both releases as a result, that by instead doing one release we would likely have much better quality testing.
2) Each release puts significant strain on us as the developers as well as our “support community” in order to handle the large volume of update related enquiries – I felt that a single release would ease the burden here compared to two subsequent releases.
3) Multiple mandatory releases in a row has the potential for “upsetting” third parties like exchanges, and I felt it would be beneficial to avoid putting them through this if possible.
4) A single release would be overall better in terms of use of my time and therefore in the long term better for the project.
A final thing that came up in our internal discussion was that our user base has completely exploded over the last year, and the release of the PoW² caused an even more overwhelmingly positive response than we anticipated, with installs absolutely skyrocketing.
Comparing how many users we have now to our previous releases and how much work it was to get people to upgrade even when we were smaller it started to become obvious to us that our previous plan of “PoW² as a mandatory hardfork from block xxx – everyone must upgrade before that” was no longer good enough, would not be a pleasant experience for all these users and that at this point we are large enough that we need to do better.
Taking these into consideration, and based on experiences from previous releases I came to the decision that, despite it being harder for me and despite the possibility of delays, a single release combining the two was ultimately the right thing to do.
Further it was decided that instead of a simple hardfork the upgrade would take place in multiple stages, allowing the best of both worlds – 1) People to start participating in PoW² as soon as possible 2) A period of backwards compatibility so that there is time for people to upgrade naturally (at this point we were considering also that the status of our iOS acceptance was unknown and wanted to allow as much upgrade time for iOS users as possible) and it doesn’t all have to be forced before a certain block.
IV) THE NATURE OF DELAYS ON LARGE PROJECTS
Unfortunately I must admit I underestimated the amount of work involved in doing SegSig correctly, this happened on two fronts.
1) I have been aware of some small inefficiencies in the bitcoin transaction format for quite some time and this was always on our internal roadmap and what SegSig was intended to address (other than the transaction malleability issue).
I had never spent massive amounts of time looking into the transaction format. Once I started the work to correct these small inefficiencies some other less obvious inefficiencies became apparent to me, there was a lot more scope for improvement than I thought, and as I looked into those even more appeared, and so there was a bit of feature creep.
2) Despite the changes to the transaction format itself being “relatively small” it has knock on effects for an extremely large part of the codebase, the codebase has an extensive amount of parts that make assumptions about or make use of the transaction format code and need to be adjusted for the changes, While individually these changes are easy to make, I underestimated a bit just how many of them there would be..
In terms of the multi stage upgrade, I also slightly underestimated the work involved. PoW² is already a very complex beast to implement and having the backwards compatible change complicates it a bit more. I was prepared for this but there were some unexpected ‘incompatibilities’ with the existing Bitcoin codebase that complicates this, theoretically this complication shouldn’t have existed but it’s just an implementation detail of how Bitcoin handles things – and this is unfortunately the nature of Software Development that these things happen.
V) WRAPPING UP
And so here we are, a part of me regrets not sticking to doing PoW² as a solo release, it certainly would have been easier, in the short term it would have made a lot of people happier and instead of feeling bad about “letting people down” right now I could instead be soaking up all the praise.
However a larger part of me knows that even now with the delays this would have been the wrong thing to do, in the long term picture these delays are a very short period of time and when we are enjoying the benefits that they bring in the long term I think everyone will come to appreciate the long term game we are playing.
And so, while I am sorry there are delays and while I apologise to anyone who is disappointed by these delays I am not sorry that I have taken this decision.
While we have not released yet, the finish line is in sight and we will be there soon, stay tuned for more news to come.
