Bitcoin Forum

Hey guys,

I wanted to give my proposal for an official Mastercoin daemon reference implementation. I think enough valid arguments have already been raised why this is needed but just to confirm.

• It would enable rapid development of products based on Mastercoin without having to know all the ins and outs of the protocol.
• It would serve as a guideline for other implementations on how interpret the rules.
• It would make it easier to get Mastercoin trading on exchanges.

Now when creating a reference implementation the following features are of major importance.

• Cross-platform
• The least amount of dependencies
• Optimised for speed and security
• Readable code and documentation
• Testable (TDD/BDD)
• High level API on top

I've discussed this a lot over the last few weeks with a dear friend of mine, who hopefully be involved with the project soon too, and we came up with the following suggestions.

We would develop a reference implementation in Go using Leveldb as data storage. The application would exist of multiple levels and be developer in the following steps

• Reader module; the reader module would be responsible for reading and listening for messages from different sources (this doesn't have to be Bitcoin perse but would be so initially).
• Parser; retrieves and parses data specifically for Mastercoin and save it to a key value storage (leveldb).
• API Library; the higher level API library which serves as a wrapper around the reader, parser and database interface (so it should be possible to develop upon the specification, rather than the JSON RPC)
• JSON-RPC; A JSON interface wrapper around the library API

The lowest level would be the reader module. This module would read messages from different sources. Initially this would be the Bitcoin block chain. The reader module would be responsible for listing in on a network, gathering messages and then saving these messages to be parsed by the second module. The second module would be the parser itself. It would go through all unparsed messages and retrieve the Mastercoin data. It would then go through all the logic dictated in the spec and save the data in a presentable form in the leveldb database. The last module would be a JSON-RPC interface that gives easy access to all the data saved in the database. There will be no hard dependency on any Bitcoin specific application like Bitcoind. We want it to be as much as stand-alone project as possible carefully weighing development against the dependency cost.

We are considering Go for the following reasons.

• Cross-platform portability and cross-platform compilable
• Easy multithreading
• Secure out of the box. I.e. buffer overflows can't lead to arbitrary code execution
• Build in garbage collection
• Easy dependency management
• Easy syntax, compared to languages like c

Of course there are also downsides to Go.

• Although production-ready it's a relatively young language
• Slower to develop than Python / Ruby

This would be no small project and would probably not make any quick gains. However we believe this might be what is needed for the the future stability of Mastercoin. The idea is that I would keep my curren Ruby libraries up to date with the new features while simultaneously working on the Go implementation. However my friend would have the main role and would be doing the most work on it.

Please voice your opinions and concerns so we can come to a final solution for this problem.

+1 for Go. Definitely the best language for this purpose. Portable and almost zero dependency. Reasonable speed both in running and developing.

That's an excellent proposal, Tachikoma. You have my full support in this.

I am less familiar with Go, but I trust your judgement in that respect, and do truly believe that this is, overall, a good direction for the Mastercoin project to head in.

This sounds like a solid proposition. At first I was against using some rather obscure language like Go, but I did some research on it and came away impressed.

It's not clear from your proposal, but I would strongly recommend against us writing our own bitcoin block chain parsing logic. Instead, I would recommend we use SWiG to integrate up against libbitcoin (which already can write out to a leveldb backend):

• Go and linking to C++ libraries: http://www.swig.org/Doc2.0/Go.html
• Libbitcoin: https://bitcointalk.org/index.php?topic=30646.0

Writing our own block chain implementation would be a big hassle to both do and maintain, when there's already an excellent library for this. The only hang up here is I don't believe it's been compiled under Windows yet...but that would be a good use of a bounty possibly (i.e. get the dev to spend some time to get a build under windows going...would enhance libbitcoin AND Mastercoin).

Failing that, I would just initially integrate up against bitcoind, to minimize time-to-market as much as possible. Once this thing is out and running up against bitcoind, time can be taken to develop the necessary functionality to remove that dependency. (This guy has some Go code that interacts with bitcoind on Github, which may be a good start for bootstrapping: https://github.com/GeertJohan/go.bitcoin)

I'm encouraged we're finally getting to this point in the conversation. We needed a mastercoind out yesterday.

Actually we will be using libraries wherever possible, as long as they don't bring in hard depdencies, like Bitcoind. I've already talked with Amir about using Libbotcoin for Mastercoin purposes. An other option we came up with is using parts of BTCD (https://github.com/conformal/btcd) which is a pure Go implementation of the Bitcoin reference implementation. It is still alpha but it's already passing block acceptance tests so it looks like we can cut development time by using parts of their libs. The only problem is we haven't had time to really dive into their code and see how useful it would be for us.

We have considered using Bitcoind as an interim solution however we rather cut that out since it would be wasting a lot of resources. We only need a small part of the Blockchain it makes no sense to waste all those resources saving data on transactions we will never use.

Ok, that's good that the focus is using other people's solid code whereever possible.

Given this, would you be willing to risk a guess as to how long it may take to get v1.0 of this out, given that you have your friend and maybe yourself working full-time on it? Being a developer by trade myself, I know how off these kinds of high level estimates can sometimes be, so if you are not sure, that's fine too.

(I just look at Nxt coming up along side, as well as the success Protoshares is having on the exchanges. Things are ripe for this.)

I can't say anything about that yet. Even if I would take the job I would need a grace period at my old job since with us leaving the whole company might break down. I would probably need at least three months to make sure they stay afloat. I can already start working before that time but it won't be fulltime.

I don't think we need to hurry this thing though. Right now we mostly need new core messages supported. You can't build that much on top of just simple sends. The more value we add to the protocol the more a need for a Mastercoin reference implementation there will be. I rather do this one right then rushed.

I will see if I can do some discovery work to make a estimate about the development time for an early alpha version.

Just wanted to jump in and say Hi!

I'm the 'friend' Tachikoma is referring to :-) If everything goes well I'll be dedicating my time developing the Go implementation.

Regarding the time frame for 1.0; it would be too early to tell. I'm sure that once we start plotting out a proper course and thinker in finer detail we could give a rough estimate, for now it's just too early.

Also I don't think we should focus on a quick and small gain and believe that much more support can be found in a proper, solid implementation. I know from experience that building on a poor foundation leads to nothing but trouble and unhappy developers ;)

Anyways, just thought I'd share my thoughts :)

Hey guys,

Well if I were smart financially I should come out against this and be pro rbdrbd's proposal (since he's using my library & engine = better result for me in the contest).

But... I have to get behind what I think is best for Mastercoin as a whole. Rbdrbd was taking my disparate components (eg lib & engine) and wrapping to come up with a daemon, where as Tachikoma is proposing a ground up build where design intent is a daemon from the get go.

As such fwiw I'm behind this.  Two primary reasons - firstly I completely agree with the benefits a reference client would bring (hence my support of rbdrbd's efforts), secondly Tachikoma's abilities are a 'known commodity' - ie we already know he has the capability to deliver.

Thanks!
Zathras

EDIT: Not to downplay rbdrbd's efforts at all, I'm still of course happy to support you if you end up continuing with your masterdaemon project. :)

I'm down for what's best for the community, given that we can bring something to market quickly and intelligently without sacrificing quality (which I was never advocating doing, BTW :)). This is why I put forward the idea of me completing masterdaemon before I actually went off and did it...I do have a decent amount of time already sunk into it, but didn't want to finish it up without getting everyone's feedback for exactly this reason -- namely that something else would pop up.

I put forward masterdaemon and the general concept initially because we needed *something*. As this team becomes more established and the idea gains traction, it makes sense that we would have a more formalized product materialize. Perhaps some of the documentation can end up being picked over, or something (or I can get a few MSC for my troubles, hah!). :)

Let's definitely have a thought out design and "do it right", but especially if we have 1 or more people working on it full-time and we make maximal use of established libraries where it makes sense, I would think we should be able to put our heads together and come up with something relatively quickly.

I'm excited about the prospects of us all working together to assemble a talented and experienced full-time team to work on this and other core products.

The plan so far is me fulltime go while Tachikoma continues his work on his ruby implementation and help out on go when he feels like.

The planning as I see it would be in several stages and will be set up in a modular fashion. Meaning that the daemon would actually exist of more components that can be reused where ever possible. This to ship out individual parts quickly and gaining momentum in the development process. It's easier to hack on individual parts/modules (for 'outsiders') rather than one large project and should (hopefully) gain some support and feedback from other developers.

And because the project would exist of 'small' components planning should be more accurate (though I would never sacrifice quality over making a deadline, if there ever are such things).

Go will be used to listen to the exodus address (and probably other addresses)
it parses and saves the transactions (block number, date, reference address, transaction type currency etc ) in LevelDb.

All  developers can then access Go via api or json to get the transactions.

Is this correct?

That could be seen as a summary; yes.

Ok.  It's a Go :)


 z0mbie, looking forward to see your work in Go. 

z0mbie, thanks for stepping up. Do you have a resume or CV you could send to the board and zbx, who has been helping out with hiring for these full time devs? Just to give a bit of a better idea of your background?

(apologies if you've already done this...just wasn't sure)

One of my 'demands' was that if I would be a hire; z0mbie would be as well. I already talked this over with Ron and he had no problems with this. I don't think supplying a resume would be a problem though :)

Ok, I wasn't aware of that. Ron had also tasked zbx to help out/take care of the dev hiring as a Tech Lead candidate. I think there is a little bit of confusion from that all, then. (unless zbx was aware of this, then it's just me being out of the loop :))

Not trying to be difficult, I'd just like to get a better idea of his experience and background.

I will let him talk about his background, I already have a pm-conversation going with zbx. Let's try to stay on-topic :)

You proposed a technological solution. There didn't seem to be much objection to that (or what there was was cleared up). The next step is determining who will work on it full time (as this is the core product for the full-time team). I asked a question to that extent. How is that not on topic?

Actually, nevermind. If you and zbx are talking that's fine. I'll get the info through him.

In my talk with Ron I already said this is what I wanted to build it with my team, z0mbie. As far as I knew the question 'who is going to build it' was already answered. I think that's why I was confused by your questions.

Ok, that makes sense. I think that's why we're all confused. Ron didn't make any of that known to anyone else (zbx, or myself, unless I'm off here). So to me it just looked like a unilateral, ad-hoc decision.

Hopefully, that's just because zbx hadn't been brought on yet as the Tech Lead candidate. Otherwise, I think the board needs to give some more thought to it's communication workflow.

(I have to re-iterate that I'm not trying to be difficult here. But I got involved initially because I saw a distinct lack of organization which was going to hinder Mastercoin from staying competitive. I'm not talking about bureaucracy or anything, just having the right people making the right kind of decisions, and having best practices in place --- e.g. viewing a resume/doing an interview to get a better idea of who is coming on to a team ... there's nothing odd about that).

That's one of the things I brought up yesterday with Ron as well. I'm no idea who all these random people are and what they are suppose to be doing. My dev email is being flooded with all kinds of emails from people I never heard about before a week ago.

What is Zbx's role precisely?

As I understand it at least (and I may not be 100% correct), zbx was a candidate for Tech Lead. He had talked to Ron and Ron was down with him working through the dev hires.

I think now there's some organization taking form, which is why you are seeing more people. But I think there needs to be a clear organizational structure laid out to everyone, otherwise it just ends up looking like (and working like) a Chinese firedrill. The board's hiring strategy is still not clear to me, but to their credit, I sent them over an organizational diagram and we have been working through what the final structure will look like. Hopefully we can get something finalized shortly that will lend some clarity to everyone.

The problem is, essentially, to grow, roles need to be delegated. If I bring on someone to do thing X, then I still take care of thing X and don't tell them anything about the decisions I'm making, that can be a problem. :) I'd like to think that Ron had this conversation with you before he had the conversation with zbx (things have been moving quickly lately I guess).

Zbx mentioned he had a call with Ron and that's why he contacted me. I'm still confused now as to what his role exactly is going to be but perhaps he or Ron will shed some light on it :)

Guys,

I can say with some confidence that Ron is not watching bitcointalk threads, so if he needs to clear up questions about who is doing what, you'll need to email him directly, preferably use info (at) mastercoin (dot) org, since several people will see your questions who might want to weigh in.

He is taking the initiative in hiring devs, and he's having lots of conversations with lots of people. Things are bound to be chaotic while we get this all figured out.

Thanks!

For my part, I've made some direct inquiries, and I put up a job offer on Coinality today. As of right now, there has been very little progress made in the search for full-time developers, and Ron's whole approach to the hiring process is completely wrong. For one thing, the project should not be trying to hire two teams of three developers each all at once, to work on 'whatever they want' (as Ron would have it). I'll continue to search for developers outside of the Mastercoin community, and recommend some of them to the board to be hired.

Ron did not say to me that Tachikoma and z0mbie were [effectively] already hired, but I do expect that he (and a number of others) would be happy to engage them full-time.

I'm sorry there's some confusing amongst you guys as to what my role is and to who I am. I assumed I had been introduced and that everyone had been notified as what I would like to do.

First and foremost, I haven't been hired (yet). Tachikoma has made aware of his situation and that he wouldn't leave his day job if I couldn't/wouldn't tag along. But since this project is something that fits my profile and I'd like to be part of the 'greater good' I said to him; "Fuck yes, of course". So here I am getting in to your good graces ;-) (Hah)

As for my background/resume, I'm afraid I would have to disappoint you to some degree. I don't do resumes. I believe they hardly say anything about a person, I mean; everyone can write down some nonsense/buzzwords as to what they supposedly can do. I just don't believe in them so I won't give you any :-P

What I can do however is give some sort of introduction about my self and point you in to some direction as to where my code lives that I've open sourced. I think I'm somewhat of a controversial developer and have a strong i-can-do-better-than-that mentality. I'm currently working on a OS JavaScript framework that has been in development for about 2 months now (https://github.com/stygeo/frame) I have developed (and designed) two interpreted languages, one which can be found on my github profile (https://github.com/stygeo/rad). Unfortunately I completed screwed over 'Rad' (the language) and neglected it for too long. At the moment I'm designing a new language in my super secret coding hideout (aka my 2x3 home office). And why? For no particular reason really. I just love languages and designing languages. These are just two project but if you like just have a look at my github profile :-) (https://github.com/stygeo) I'm sure it says more than a piece of paper which says "resume: BUZZWORDS" :-P

Besides coding I believe I'm good at designing/structurizing development projects. I have a feel for the overal scope of a process and know 'exactly' how to get from A to B (I know there is never such a thing as exactly). It's hard to explain but I see the connected dots in my head and know how to implement them and communicate them to others. All this sounds a little cocky maybe, but since Tachikoma knows me so well and we've been working for so long together he can give you a honest opinion about my skills.

If there's anything unclear about me (or my hopefully new role within the msc community) let me know. I'd be more than happy to answer any of your questions.

Jeff,

Thanks for sharing your Github track record, very interesting stuff.

I'm glad you are engaging the community and I do hope you dive into the development effort.

Tachikoma has been doing great stuff and he has highly recommended you and so I'm excited to see your future contributions to Mastercoin.

David Johnston - Mastercoin Foundation

Hi guys!

I'm not watching the main Mastercoin thread on bitcointalk, but that doesn't mean I closed my bitcointalk account!

(I'm not monitoring every thread, nor logging in every day, but I'll be keeping an eye on important threads like this one)

I'd like to clear up a few things:

1. We have not decided on who the two lead devs will be.
2. Anybody and everybody is asked to help us find & hire them. In a phone call zbx waas concerned that hiring was going to be paused for two weeks because I'm going to a few conferences - I replied that he's welcome to help out. That does not grant him any decision power in the process, it was just a call to him to criticize less, and help out more.
3. Nobody has decided "who will implement mastercoind". Anybody is welcome to do it. We can do this as a bounty (tell us how much you need to get this done, preferably email info@mastercoin.org), or we can do this as part of our Role Based Bounty program once we secure a hire or two. Also we haven't decided on a technology, although Go does seem like a good choice.
4. If there are any other questions I missed, I'm available to answer them! You can post questions here and skype me the link (ripper234), that would be the quickest way to get them answered. You can also use my email ron@mastercoin.org, although things tend to get buried in my inbox.

Now, if you'll excuse me, I'm off to the Future of Money conference, where I'll be in a panel with Vitalik from Colored Coins, Chris Larsen from Ripple, and Dave Sterry from Litecoin. ttyl

If its a bounty, I will be interested to join. (Go looks challenging)



Good luck in the conference.  Very interested to hear what everyone has to say.  But I hope you win  :)

I wouldn't want to be this to be a bounty. I think this should be the main product the Mastercoin foundation sets forth. This isn't something you could 'set and forget', bounties are better tailored to such solutions. If I were to be employed by the foundation, this is what I would want to build.

Just to offer a quick update here - Maran and Jeffrey, our first 2 developer Role Based Bounties, will be focusing on the reference implementation on January (written in Go). Jeffrey is going to lead that initiative.

Sadly progress will be a little less quick than we want, because Maran and Jeffrey both still have commitments at their existing workplace. They are coming online on a 20% position on Jan 1st, and will be upgrading to 60% position on Feb, to be increased in future months.

Jeffrey plans to start by creating a roadmap on Trello for the reference implementation.

We've started drafting some general overview document (https://docs.google.com/document/d/1g1Mwjmw8LdwgLgKmdEs76BqdFhEm4zTsma_KR3G3NFY) and filling the Trello board. (https://trello.com/b/Tungw5f3/mastercoind) It's not that exciting yet but if you have any comments please let us know.

One of the additional best reasons for going with Go is an existence of almost fully functional bitcoin fully validating node in Go language: https://github.com/conformal/btcd   

Watch a video about it here:  http://www.youtube.com/watch?v=d1IYvJs5GGw

If you read the doc you would have seen we are indeed using that :)