mastercoind proposal to speed development: Please read, vote, and comment

[rbdrbd]

And actually, I'm going to stop my work on this until I have a clearer direction of which library/libraries I should be using, and if Mastercoin even wants this. I was a bit premature, sorry. Just let me know.

[zathras]

Hey rbdrbd and welcome to Mastercoin development

I thought I'd chime in on a couple of points - I have to agree with Tachikoma on this one, whether or not you state it's the reference implementation, the choice of name (mastercoind) will lead some members of the community to perhaps think it is.  I would love to see any reference implementation written by JR too.

Simply put, the project will likely have a reference mastercoin daemon at some point and I think it would be courteous to leave the 'mastercoind' moniker for that purpose - you could always rename your output binary later if Grazcoin/your work ended up becoming the reference implementation.

Good luck with your wrapper!

P.S. (sorry for plugging my own stuff in your thread ) but masterchest_engine & masterchest_library have been around for a while to offer devs a low barrier to entry - the library already does decoding/encoding/parsing/bitcoind integration etc on behalf of the dev and the engine processes all transactions to arrive at a known/current state so for example web devs can query balances/transactions/exchange state etc without knowing anything other than a bit of SQL.  They're of course works in progress but since your post kind of implies before your wrapper devs had no choice but to start from scratch, I wanted to clear up that misconception.

Thanks!

Good point on the "mastercoind". I'm good with whatever JR says here. My goal with this thing was to get it to a point and turn it over anyhow.

The optimal library for this purpose would be python, would interface with bitcoind via JSON RPC (so it had good cross-platform compatibility), and would take care of encoding/decoding of transactions, and maybe even processing to a embedded leveldb database (if it didn't do the latter, I could add that easily enough).

We don't have this optimal case today, so I have a few options:

* Zathras' libraries -- Windows-only it appears, but sound like it's the right focus. I just skimmed your code. If I found a way to interface the DLLs well to Python (I think it's possible), I could use them under Windows. Looks like I could use masterchest-library alone if I wanted to use something different than SQL for the serialization (e.g. write something higher level that used that to parse the blocks and serialize to leveldb, for instance), or combine in using masterchest-engine and a SQL backend. I really do like how you're interfacing around bitcoind here.

* grazcoin's stuff looks good for Linux, at least. However, it's still a bit rough around the edges for working with as a 3rd party library... mostly around how it's coupled to the web site interface he has, and how he serializes things. I have a few messages to him, still undecided here (the good news out of this is that I made some setup and init.d scripts that will be included in 'sx' though).

* A clean port of masterchest-library to python is another option here, too, since that interfaces with bitcoind, it could be used on both platforms.

* I looked at Tachikoma's stuff too.... ruby though (same situation).

Optimally, I'd want to use the same library and serialization strategy under both Windows and linux, at least for starters.

It may seem like this is a piece of cake, but my goal is something that is very easy to set up and works across multiple platforms, like bitcoind. With that, things become a lot more complicated - code/script wise. I already have a good amount of work into this (and will share my code once I get a basic version of it up), but if this is something one of you thinks you could do up to this level, and make available for both windows and Linux, just let me know and I'll stop my work on it. I don't want to waste your time or my time, I just want to see this done, which is why I started working on it.

Basically, guys, I'm asking for your advice. I'm willing to write up against something I can use for both Windows and Linux (or possibly take two different approaches). I'm also willing to package this up and make it easy to deploy on both platforms. Just let me know how I can best work with you all to get this done, and what you think the best library choice is for me here, given my parameters.

If this starts becoming political, or if I've ruffled feathers with this, then I'll just dump it on a github repo for folks to potentially pick over and move on. I'm not looking to establish a name or identity in any of this, I just want to see it done in one form or another, to help Mastercoin out, as well as my investment in it. That's all.

Please don't be discouraged by any of the posts (mine included) - I'm all for having more people & development on board

Re the name, that's simply a label but seems to have been adopted by the community at large as the naming for a given projects' daemon - hence my desire to keep it free for when a reference is officially nominated - for my own personal opinion I'd actually like to see 'mastercoind/qt' as a reference client in it's sincerest form - ie without any external dependencies (websites or bitcoind etc).  Mastercoind/qt should be a standalone reference client that connects to the bitcoin network and performs all the necessary bitcoin functions as well as Mastercoin functions without needing anything else.  I see a fork of bitcoind/qt as the simplest (not necessarily best, but simplest) path to this, but that's just my two cents.

Re the libraries, none of them is 'the' correct library at present - for one I can tell you my stuff alone is not ready for stable - the functions do what they're supposed to but we're always coming up with new fringe cases and bugs that can knock things off track.  We currently develop to a consensus where by we discuss & agree areas of the spec that are open to interpretation (or just outright suggested changes/additions) and the multiple different implementations are critical to using RAD (rapid application development) techniques (basically everything can change fluidly with no change control) in these early stages.  

For example only yesterday when I was about to release the new version of my library (and thus broadcast a few example transactions) did I find out my implementation was dropping a byte in the minfee as we crossed packets and not prefixing a leading 0 for the timelimit int if the hex was single digit.  Using my own implementation I would never have known (as I was encoding/decoding the same way), but being able to test against Tachikoma's meant I was able to pick up the error and fix the bug.  This happens A LOT  For example Mastercoin-explorer & Masterchest may differ on a transaction so Tachikoma & I will investigate - sometimes he's got a bug, sometimes I have - but it's those differing implementations that actually help us to identify those bugs & correct them while everything is so green.  Tachikoma even has a verification API to automate these checks for differences coming I believe.  EDIT: Sorry, not to downplay Grazcoins & Bitoys work either, I check my errors against their code too

TL:DR; contribute what you think will be of value.  If Grazcoin's libraries give you a good foundation for something you want to build & you think it will benefit the community, then go right ahead  Just try to keep in mind we're young and there are still multiple ways of doing things - we're not at the stage of saying any one implementation is the 'right' one yet and need to be careful about community confusion.

P.S.  Re Linux support I have actually tested (and rewritten in places) my wallet to work under wine on Linux.  I'm only testing with a couple of distros while I remove the kinks but to date I've worked around most bugs and have it working successfully (just with the odd out of place/badly colored label etc).  I haven't done this with the engine yet but since I'm not using as many of the cryptographic providers or doing any GUI work it should be reasonably easy to get going if that became a requirement.

Thanks!

[rbdrbd]

Zathras, thanks for your reply.

I gave some thought to things...to support both Windows and Linux (and even Mac OS X in the future) in as unified manner as possible, I'm thinking the best approach may be go to with your masterchest-engine/masterchest-library. I could use mono (under Linux and Mac OS X) to run it.

You mentioned testing under wine, but have you ever tested the engine and library under Mono? Beyond testing under mono (which I can do if you haven't), one enhancement I was wondering if you would be interested in making to masterchest-engine in order to make it possible for mastercoind (or whatever we end up calling it) to use your stuff cross-platform would be to allow it to write to a Sqllite database (instead of just SQLServer).

You could use System.Data.SQLite for this (http://system.data.sqlite.org/index.html/doc/trunk/www/index.wiki) ...probably just have a command line option to specify the engine to use (SQLServer or SQLite) and the actual SQL API commands wouldn't hopefully have to change much... you have very basic SQL statements and logic in there, thankfully.

My app could then periodically launch masterchest-engine.exe (either natively or via mono) to update the SQLite DB file, and my app would then just be reading the data from that DB as-needed. This would allow a single approach across all platforms, without requiring SQLServer on Windows...or any external database server. Sqlite supports a concurrent writer and reader no problem, with a bit of locking during the write itself.

This would make "mastercoind" be able to run well on Windows, Linux, and Mac OS X in the future....and I could come out with a build for Windows and Linux right off the bat. And as an advantage, folks on Windows that wanted to use Masterchest-engine directly could do so without having to install and run SqlServer, which is often overkill for smaller/test installations.

What do you think?

[zathras]

Zathras, thanks for your reply.

I gave some thought to things...to support both Windows and Linux (and even Mac OS X in the future) in as unified manner as possible, I'm thinking the best approach may be go to with your masterchest-engine/masterchest-library. I could use mono (under Linux and Mac OS X) to run it.

You mentioned testing under wine, but have you ever tested the engine and library under Mono? Beyond testing under mono (which I can do if you haven't), one enhancement I was wondering if you would be interested in making to masterchest-engine in order to make it possible for mastercoind (or whatever we end up calling it) to use your stuff cross-platform would be to allow it to write to a Sqllite database (instead of just SQLServer).

You could use System.Data.SQLite for this (http://system.data.sqlite.org/index.html/doc/trunk/www/index.wiki) ...probably just have a command line option to specify the engine to use (SQLServer or SQLite) and the actual SQL API commands wouldn't hopefully have to change much... you have very basic SQL statements and logic in there, thankfully.

My app could then periodically launch masterchest-engine.exe (either natively or via mono) to update the SQLite DB file, and my app would then just be reading the data from that DB as-needed. This would allow a single approach across all platforms, without requiring SQLServer on Windows...or any external database server. Sqlite supports a concurrent writer and reader no problem, with a bit of locking during the write itself.

This would make "mastercoind" be able to run well on Windows, Linux, and Mac OS X in the future....and I could come out with a build for Windows and Linux right off the bat. And as an advantage, folks on Windows that wanted to use Masterchest-engine directly could do so without having to install and run SqlServer, which is often overkill for smaller/test installations.

What do you think?

Firstly I'll just quickly say it doesn't have to be my library - you won't be stepping on any toes if you run with one of the other guys libs   With that said, if you think masterchest library & engine would be a good fit for you, have at it!

Just to clarify, I've only tested & fixed up the wallet for wine, not the engine.  Since they share a common codebase (and the same library) though I don't see too much of an issue.  I removed roadblock incompatibilities with the wallet & wine in about half a day so even if there are a few quirks with the engine, it's not masses of work to get it running.  Not tried Mono with either wallet or engine, but I'm happy to spin up a quick test if you like (will just connect to SQL over network for now).

I don't see any major headaches offering an SQLite option, I'd have to check it out but should be fine - let me come back to you on this.

FYI state processing for DEx is still very alpha in the engine & the matching isn't quite there yet.  State processing is fine for things like simple sends.

[ofer]

Hi,
I think that writing mastercoind  is an excellent idea, and I have an idea for improvement:

I think that the best way to implement this is by making mastercoind use bitcoind as a backend.
Just like HTTP implementations are using Berkeley sockets and not rewriting them.

The benefits we get by choosing this architecture are simplicity and more important security.
We can now assume that bitcoind is a very secured piece of software.

Few days ago I saw a lecture by Gavin Anderssen where he said that the bitcoin-qt had a lot of security holes in it's early days.

We can't afford security holes in mastercoind, it will not be under the radar like bitcoin-qt in it's early days.

[rbdrbd]

For transparency, here's my PM exchange with grazcoin regarding potentially using his library (before I had checked out zathras stuff). grazcoin gave me permission to post:

Hey grazcoin,

I've been working on using your mastercoin-tools with mastercoind that I'm developing and have a few questions:

* Have you considered using an embedded database like leveldb instead of creating a bunch of files on the filesystem? I can think of several advantages of using an embedded db to the raw file level storage, especially as the # and volume of transactions grows.

I considered, but the reasons that kept me in this file based methodology are:
* security: serving static files is much more secure.
* simple + cross platform: each user can take a snapshot and continue parsing while using only python and a connection to an obelisk server. It keeps the system really distributed.
* easy to verify: we humans read text better than leveldb format

* Do you have any plans to split the parsing and storage part of mastercoin-tools from the website logic which uses it anytime soon? This would definitely help on the mastercoind front.... the parsing process could store the data into the embedded database (leveldb, sqlite, etc) and your website could just read off of it. likewise, mastercoind would just read off of that same database

I tried to keep it separated. Tools should be only tools. The front end shows that it could look also good.

* I finally got sx semi-working ... in order to get msc-parse.py to really do anything though, do I have to wait for the entire blockchain to download?

unfortunately you have to wait.

Oh yeah... I felt I should mention that after looking at your code more and thinking things through, I am now attempting not to modify your code or monkey patch it at all, trying to just run it as you do, and use the outputs created as-is. This will reduce duplication and headaches in the future, assuming the integration points are set. That's why I had the interest in using leveldb (which bitcoind uses, as well as obelisk)....I could just run your cron-script on a timer and dip into the database to get what I needed.

I think it will be easier for you to take the json results, since my code keep getting developed. Otherwise you will have to constantly merge. I am open for changes/requests in the json format though.
You could also read directly the ready json which is web-exported by any running instance of mastercoin-tools (see http://masterchain.info/API.html )
This introduces a trust issue, but it would be wise for you to include this option for people that do not want to run the python code locally. The trust issue could be partially solved by the code signing the parsing results + ssl (will come soon).

And to clarify a point in my earlier message, by 'splitting up the parsing and website code', I meant having them as separate github repos... there is somewhat of a segmentation now, to a point, but a separate repo for each would be great... just an idea to help you make the library more useful to 3rd party utils (and, have it used by anyone that is using mastercoind, which I hope will be a lot of people).

For now it is simpler for me to maintain a single repo.
Maybe it will change later.

[rbdrbd]

Hi,
I think that writing mastercoind  is an excellent idea, and I have an idea for improvement:

I think that the best way to implement this is by making mastercoind use bitcoind as a backend.
Just like HTTP implementations are using Berkeley sockets and not rewriting them.

The benefits we get by choosing this architecture are simplicity and more important security.
We can now assume that bitcoind is a very secured piece of software.

Few days ago I saw a lecture by Gavin Anderssen where he said that the bitcoin-qt had a lot of security holes in it's early days.

We can't afford security holes in mastercoind, it will not be under the radar like bitcoin-qt in it's early days.

Good point, I'm aiming for the level of security to at least match bitcoind. From a remote exploit prevention, mastercoind is written in python using cherrypy. This fact makes certain types of exploits less likely (e.g. buffer overflows, and other things that could plague writers in lower level languages like C/C++ if they aren't careful with their string length checking). Other vulnerabilities, such as SQL injection, are still possible, and from there it's good coding practices, and doing extensive cleanup/sanity checking on user-provided input (the lack of which is always what leads to these kinds of attacks, essentially).

Then you have things like stored data security. For now at least, RPC username and password are stored in a plaintext settings file, similar to bitcoind. We can look at changing this in the future, however. If something very sensitive is required (such as private wallet key for signing requests), then I plan to collect that as part of the request and not store it, if at all possible.

[Ola]

Forgive my ignorance but what does this differently compared to the current implementations? Are you not just creating Yet An Other Mastercoin implementation?

I'm actually not creating another Mastercoin implementation at all, as I'm utilizing grazcoin's mastercoin-tools package for the heavy-lifting (the integration will initially be a bit ghetto, but hopefully that will smooth out with time).

As I wrote in the initial post, the goal is to lower the knowledge requirements for mastercoin application development by making a program similar to bitcoind, where future mastercoin developers can just query using a simple API, instead of having to figure out blockchain parsing. This is much like how bitcoin developers today don't need to know about the guts of bitcoin's workings to create a simple bitcoin-related app, they just need to know how to interact with bitcoind's JSON RPC API.

Ok thanks for clarifying you are not actually creating a client but a RPC wrapper on top of a client.

I still think J.R. should be the one who writes the reference client since he can make decisions on how the spec should be interpreted. No offence intended, but who gave you the right to decide your implementation should be the reference one. It's not so much how the clients handles, you can strap an RPC client on top of any implementation, but the decisions you make in the underlying layers.

I think an RPC accessible client on top of any library is a excellent idea, I just think that the word reference implementation should be removed.  

I agree with that. My initial post was a bit confusing up top, sorry about that, but I wasn't thinking that I was making the "reference library" here.....only that mastercoin needed one.

In my mind, mastercoind is simply a gateway/frontend to whatever the reference library is. At some point, once/if it makes sense, I would like mastercoind to possibly become the "reference client"/"reference front-end" that uses a separate backend reference library, but I have no say in that decision itself. As I said, it can be done if it makes sense, and people start building apps around this thing. Plus, I have no desire or intention of writing any mastercoin parsing library, or having to deal with the low-level details of one (this was the purpose I wanted a mastercoind so badly :p).

As I said earlier, right now I'm using grazcoin's MSC library, which is more utilities for his mastercoin site than a formal library ready for 3rd party use. This requires me to hack it a bit to get it to "fit in" to mastercoind. However, this is 110% better than me writing something from scratch. I see the backend library as taking care of those difficult details, and mastercoind wraps it up real nice and makes it very easy to use (as well as allowing it to be used by a client written in any language, or on a different server, etc ... list the list of advantages I have in my 2nd or 3rd post in this thread).

Maybe a future goal for mastercoind is to allow it to plug into 2 or more backend libraries...i.e. the "reference" one (whatever that is) and the main alternative. That way, you can get the similar front-end API functionality, but swap out the backend library if you need/want (e.g. for easy compatibility testing, or maybe one doesn't compile under windows and another one does, etc). The swap should be as easy as modifying a config file in the program, and making sure the library is installed on the system.

The end goal with this thing is to have something that's as easy to use, or easier, than bitcoind is today.

+1000 If you could design the backend to plug in into more than one of the main libraries that will be super awesome...That way all code from the main and new mastercoin devs will keep being used. A modifiable config file is also definitely the way to go in determining what library to choose...

I urge you to keep on the development path, there is noting stopping you from dong this. More-so it is all open source and WILL DEFINITELY open mastercoin up to main stream developers.  I will be available to aid in testing out the features if you need too because I am seriously waiting on this, I am sure a lot of regular web devs are waiting on this also. I thought this feature would be a year or more out, but you changed all that by taking on this initiative and this could be a reality within a couple of months.


I hope JR/Foundation blesses this initiative so that it is looked upon as a credible point to get involved with mastercoin apps development. There  should be nothing political about this..If the name "Mastercoind" is a problem, which is simple a convention, maybe that can be fixed later by JR or any of the executives of the foundation..Please push on rbdrbd

[rbdrbd]

A few more things:

* This thing is going to be slick, I have plans for packaging the windows version as an exe and using a windows installer to install (using pyinstaller and NSIS or something similar), similar to bitcoind on windows.

* I'm open to naming this "masterdaemon" or something that's not "mastercoind" for the reasons zathras and Tachikoma brought up (mastercoind maybe being a "reserved" name). If JR/Ron would like this, just let me know and I'll change the name.

* Regarding the library, after spending an extensive amount of time looking and working with sx (and even improving its build process, see https://bitcointalk.org/index.php?topic=259999.msg3700259#msg3700259), it's clear to me that Windows support will be a big effort...since it totally lacks support for Windows now, and its not only about compiling sx on Windows, but about getting sx, libbitcoin, obliesk, and leveldb working on windows, along with all other dependencies!

A big goal for mastercoind is to be as multi-platform as possible. If I had to do Linux only, I'd go with grazcoin's stuff (which is written to use sx/libbitcoin for all transaction processing). However, given our multiplatform goal with this, I'm thinking at this point, zathras library sounds like a more straightforward implementation to be the primary/initial library that mastercoind uses. The gotchas are getting it working under mono, and getting it working with sqlite, both of which I think can get done without too much hassle. I'll get into working with it some and have a better idea....(hey, at least we have choices here! )

[ofer]

Hi,
I think that writing mastercoind  is an excellent idea, and I have an idea for improvement:

I think that the best way to implement this is by making mastercoind use bitcoind as a backend.
Just like HTTP implementations are using Berkeley sockets and not rewriting them.

The benefits we get by choosing this architecture are simplicity and more important security.
We can now assume that bitcoind is a very secured piece of software.

Few days ago I saw a lecture by Gavin Anderssen where he said that the bitcoin-qt had a lot of security holes in it's early days.

We can't afford security holes in mastercoind, it will not be under the radar like bitcoin-qt in it's early days.

Good point, I'm aiming for the level of security to at least match bitcoind. From a remote exploit prevention, mastercoind is written in python using cherrypy. This fact makes certain types of exploits less likely (e.g. buffer overflows, and other things that could plague writers in lower level languages like C/C++ if they aren't careful with their string length checking). Other vulnerabilities, such as SQL injection, are still possible, and from there it's good coding practices, and doing extensive cleanup/sanity checking on user-provided input (the lack of which is always what leads to these kinds of attacks, essentially).

Then you have things like stored data security. For now at least, RPC username and password are stored in a plaintext settings file, similar to bitcoind. We can look at changing this in the future, however. If something very sensitive is required (such as private wallet key for signing requests), then I plan to collect that as part of the request and not store it, if at all possible.

What I was trying to say is that mastercoind should call the API of bitcoind to make the actual low-level bitcoin/mastercoin transactions, this way we reduce the security risk even more.
This is because we can assume that bitcoind is very secure.

Also, good to hear that security is one of your main concerns. Programmers often ignore security issues. In such application that deals with money security is a top priority concern.

[rbdrbd]

Hi,
I think that writing mastercoind  is an excellent idea, and I have an idea for improvement:

I think that the best way to implement this is by making mastercoind use bitcoind as a backend.
Just like HTTP implementations are using Berkeley sockets and not rewriting them.

The benefits we get by choosing this architecture are simplicity and more important security.
We can now assume that bitcoind is a very secured piece of software.

Few days ago I saw a lecture by Gavin Anderssen where he said that the bitcoin-qt had a lot of security holes in it's early days.

We can't afford security holes in mastercoind, it will not be under the radar like bitcoin-qt in it's early days.

Good point, I'm aiming for the level of security to at least match bitcoind. From a remote exploit prevention, mastercoind is written in python using cherrypy. This fact makes certain types of exploits less likely (e.g. buffer overflows, and other things that could plague writers in lower level languages like C/C++ if they aren't careful with their string length checking). Other vulnerabilities, such as SQL injection, are still possible, and from there it's good coding practices, and doing extensive cleanup/sanity checking on user-provided input (the lack of which is always what leads to these kinds of attacks, essentially).

Then you have things like stored data security. For now at least, RPC username and password are stored in a plaintext settings file, similar to bitcoind. We can look at changing this in the future, however. If something very sensitive is required (such as private wallet key for signing requests), then I plan to collect that as part of the request and not store it, if at all possible.

What I was trying to say is that mastercoind should call the API of bitcoind to make the actual low-level bitcoin/mastercoin transactions, this way we reduce the security risk even more.
This is because we can assume that bitcoind is very secure.

Also, good to hear that security is one of your main concerns. Programmers often ignore security issues. In such application that deals with money security is a top priority concern.

Regarding your first point, I think that eventually, I'd like to see a fully integrated mastercoind. For now, however, I'm planning on probably going with Zathras masterchest-engine to reduce code duplication and greatly speed development time. My hope is that a "reference library" emerges from the Mastercoin community that is written in python (or has python bindings) and can be used by mastercoind directly. Even in that event, I'd still leave in the masterchest-engine backend, and allow switching between them for the reasons I've raised earlier.

Grabbing of masterchest-engine will be done directly from github in the automated setup script, to reduce the chance of error/security. It will also be possible for people to create their own mastercoind windows binaries, if they so desired, or didn't trust the pre-made binaries offered.

[rbdrbd]

Zathras, thanks for your reply.

I gave some thought to things...to support both Windows and Linux (and even Mac OS X in the future) in as unified manner as possible, I'm thinking the best approach may be go to with your masterchest-engine/masterchest-library. I could use mono (under Linux and Mac OS X) to run it.

You mentioned testing under wine, but have you ever tested the engine and library under Mono? Beyond testing under mono (which I can do if you haven't), one enhancement I was wondering if you would be interested in making to masterchest-engine in order to make it possible for mastercoind (or whatever we end up calling it) to use your stuff cross-platform would be to allow it to write to a Sqllite database (instead of just SQLServer).

You could use System.Data.SQLite for this (http://system.data.sqlite.org/index.html/doc/trunk/www/index.wiki) ...probably just have a command line option to specify the engine to use (SQLServer or SQLite) and the actual SQL API commands wouldn't hopefully have to change much... you have very basic SQL statements and logic in there, thankfully.

My app could then periodically launch masterchest-engine.exe (either natively or via mono) to update the SQLite DB file, and my app would then just be reading the data from that DB as-needed. This would allow a single approach across all platforms, without requiring SQLServer on Windows...or any external database server. Sqlite supports a concurrent writer and reader no problem, with a bit of locking during the write itself.

This would make "mastercoind" be able to run well on Windows, Linux, and Mac OS X in the future....and I could come out with a build for Windows and Linux right off the bat. And as an advantage, folks on Windows that wanted to use Masterchest-engine directly could do so without having to install and run SqlServer, which is often overkill for smaller/test installations.

What do you think?

Firstly I'll just quickly say it doesn't have to be my library - you won't be stepping on any toes if you run with one of the other guys libs   With that said, if you think masterchest library & engine would be a good fit for you, have at it!

Just to clarify, I've only tested & fixed up the wallet for wine, not the engine.  Since they share a common codebase (and the same library) though I don't see too much of an issue.  I removed roadblock incompatibilities with the wallet & wine in about half a day so even if there are a few quirks with the engine, it's not masses of work to get it running.  Not tried Mono with either wallet or engine, but I'm happy to spin up a quick test if you like (will just connect to SQL over network for now).

I don't see any major headaches offering an SQLite option, I'd have to check it out but should be fine - let me come back to you on this.

FYI state processing for DEx is still very alpha in the engine & the matching isn't quite there yet.  State processing is fine for things like simple sends.

Yeah, I'm thinking your stuff is the best fit considering the multiplatform component. Tachikoma's stuff may work as a backup, but it looks like his whole data storage layer is inside the mastercoin-explorer project (as ruby on rails). Also, I did research calling over to a ruby library from python... didn't find anything conclusive with the short amount of time I spent on it.

masterchest-engine I should be able to us as-is (once it has the sqlite backend support and any necessary mono fixes), without having to touch/fork any of your code.

I'm installing SQL Server Express to be able to test masterchest-engine. The plan is to test it under windows first, then under mono (pushing to a SQL Server instance in Windows still). Once you're able to add the sqlite integration, I can test it all under mono in Linux.

Do you have any .sql files to create the tables and indexes for your schema or should I write my own?

[dacoinminster]

I think it would be cool to have a mastercoin daemon (probably just name it "MasterDaemon" or something -mastercoind sounds too official).

We seem to be doing pretty well with no reference implementation so far. The various devs implement a piece of the spec, find differences and corner cases, discuss them, resolve them, and then repeat.  If I were to contribute code, it would probably just be pull requests against one or more of the existing code-bases.

[zathras]

Zathras, thanks for your reply.

I gave some thought to things...to support both Windows and Linux (and even Mac OS X in the future) in as unified manner as possible, I'm thinking the best approach may be go to with your masterchest-engine/masterchest-library. I could use mono (under Linux and Mac OS X) to run it.

You mentioned testing under wine, but have you ever tested the engine and library under Mono? Beyond testing under mono (which I can do if you haven't), one enhancement I was wondering if you would be interested in making to masterchest-engine in order to make it possible for mastercoind (or whatever we end up calling it) to use your stuff cross-platform would be to allow it to write to a Sqllite database (instead of just SQLServer).

You could use System.Data.SQLite for this (http://system.data.sqlite.org/index.html/doc/trunk/www/index.wiki) ...probably just have a command line option to specify the engine to use (SQLServer or SQLite) and the actual SQL API commands wouldn't hopefully have to change much... you have very basic SQL statements and logic in there, thankfully.

My app could then periodically launch masterchest-engine.exe (either natively or via mono) to update the SQLite DB file, and my app would then just be reading the data from that DB as-needed. This would allow a single approach across all platforms, without requiring SQLServer on Windows...or any external database server. Sqlite supports a concurrent writer and reader no problem, with a bit of locking during the write itself.

This would make "mastercoind" be able to run well on Windows, Linux, and Mac OS X in the future....and I could come out with a build for Windows and Linux right off the bat. And as an advantage, folks on Windows that wanted to use Masterchest-engine directly could do so without having to install and run SqlServer, which is often overkill for smaller/test installations.

What do you think?

Firstly I'll just quickly say it doesn't have to be my library - you won't be stepping on any toes if you run with one of the other guys libs   With that said, if you think masterchest library & engine would be a good fit for you, have at it!

Just to clarify, I've only tested & fixed up the wallet for wine, not the engine.  Since they share a common codebase (and the same library) though I don't see too much of an issue.  I removed roadblock incompatibilities with the wallet & wine in about half a day so even if there are a few quirks with the engine, it's not masses of work to get it running.  Not tried Mono with either wallet or engine, but I'm happy to spin up a quick test if you like (will just connect to SQL over network for now).

I don't see any major headaches offering an SQLite option, I'd have to check it out but should be fine - let me come back to you on this.

FYI state processing for DEx is still very alpha in the engine & the matching isn't quite there yet.  State processing is fine for things like simple sends.

Yeah, I'm thinking your stuff is the best fit considering the multiplatform component. Tachikoma's stuff may work as a backup, but it looks like his whole data storage layer is inside the mastercoin-explorer project (as ruby on rails). Also, I did research calling over to a ruby library from python... didn't find anything conclusive with the short amount of time I spent on it.

masterchest-engine I should be able to us as-is (once it has the sqlite backend support and any necessary mono fixes), without having to touch/fork any of your code.

I'm installing SQL Server Express to be able to test masterchest-engine. The plan is to test it under windows first, then under mono (pushing to a SQL Server instance in Windows still). Once you're able to add the sqlite integration, I can test it all under mono in Linux.

Do you have any .sql files to create the tables and indexes for your schema or should I write my own?

It's on the to do list (auto creating tables) just hasn't been prioritized.

I'm just using express for masterchest.info - if you like I'm happy to shoot you a backup of its DB (the engine runs that site) which you can then just restore and test Mono off the bat without needing to manually do the tables?

From a cursory glance SQLite looks an easy enough addition.  Let me know how you get on with Mono (& what distro/kernel/mono versions you're testing with so I can duplicate).

Thanks

[rbdrbd]

Zathras, thanks for your reply.

I gave some thought to things...to support both Windows and Linux (and even Mac OS X in the future) in as unified manner as possible, I'm thinking the best approach may be go to with your masterchest-engine/masterchest-library. I could use mono (under Linux and Mac OS X) to run it.

You mentioned testing under wine, but have you ever tested the engine and library under Mono? Beyond testing under mono (which I can do if you haven't), one enhancement I was wondering if you would be interested in making to masterchest-engine in order to make it possible for mastercoind (or whatever we end up calling it) to use your stuff cross-platform would be to allow it to write to a Sqllite database (instead of just SQLServer).

You could use System.Data.SQLite for this (http://system.data.sqlite.org/index.html/doc/trunk/www/index.wiki) ...probably just have a command line option to specify the engine to use (SQLServer or SQLite) and the actual SQL API commands wouldn't hopefully have to change much... you have very basic SQL statements and logic in there, thankfully.

My app could then periodically launch masterchest-engine.exe (either natively or via mono) to update the SQLite DB file, and my app would then just be reading the data from that DB as-needed. This would allow a single approach across all platforms, without requiring SQLServer on Windows...or any external database server. Sqlite supports a concurrent writer and reader no problem, with a bit of locking during the write itself.

This would make "mastercoind" be able to run well on Windows, Linux, and Mac OS X in the future....and I could come out with a build for Windows and Linux right off the bat. And as an advantage, folks on Windows that wanted to use Masterchest-engine directly could do so without having to install and run SqlServer, which is often overkill for smaller/test installations.

What do you think?

Firstly I'll just quickly say it doesn't have to be my library - you won't be stepping on any toes if you run with one of the other guys libs  With that said, if you think masterchest library & engine would be a good fit for you, have at it!

Just to clarify, I've only tested & fixed up the wallet for wine, not the engine.  Since they share a common codebase (and the same library) though I don't see too much of an issue.  I removed roadblock incompatibilities with the wallet & wine in about half a day so even if there are a few quirks with the engine, it's not masses of work to get it running.  Not tried Mono with either wallet or engine, but I'm happy to spin up a quick test if you like (will just connect to SQL over network for now).

I don't see any major headaches offering an SQLite option, I'd have to check it out but should be fine - let me come back to you on this.

FYI state processing for DEx is still very alpha in the engine & the matching isn't quite there yet.  State processing is fine for things like simple sends.

Yeah, I'm thinking your stuff is the best fit considering the multiplatform component. Tachikoma's stuff may work as a backup, but it looks like his whole data storage layer is inside the mastercoin-explorer project (as ruby on rails). Also, I did research calling over to a ruby library from python... didn't find anything conclusive with the short amount of time I spent on it.

masterchest-engine I should be able to us as-is (once it has the sqlite backend support and any necessary mono fixes), without having to touch/fork any of your code.

I'm installing SQL Server Express to be able to test masterchest-engine. The plan is to test it under windows first, then under mono (pushing to a SQL Server instance in Windows still). Once you're able to add the sqlite integration, I can test it all under mono in Linux.

Do you have any .sql files to create the tables and indexes for your schema or should I write my own?

It's on the to do list (auto creating tables) just hasn't been prioritized.

I'm just using express for masterchest.info - if you like I'm happy to shoot you a backup of its DB (the engine runs that site) which you can then just restore and test Mono off the bat without needing to manually do the tables?

From a cursory glance SQLite looks an easy enough addition.  Let me know how you get on with Mono (& what distro/kernel/mono versions you're testing with so I can duplicate).

Thanks

A sql schema dump would be great.... I just want to make sure I'm creating the schema to be like you are (a backup is fine too...just throw it up on dropbox or something and PM the link to me).

I'm testing on Ubuntu 13.10 x64, so Mono looks like 2.10, not 3.2.4:

"mono -V" output: "Mono JIT compiler version 2.10.8.1 (Debian 2.10.8.1-5ubuntu2)"
"uname -a" output: "Linux d01b 3.11.0-12-generic #19-Ubuntu SMP Wed Oct 9 16:20:46 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux"

Mono-wise, I've installed the "mono-runtime" package. If I need to install any additional packages (e.g. mono-complete, or other specific mono libs) just let me know.

I'm ready to test around sqlite as soon as you can add it, mono too. Thanks for your help on this!!

[rbdrbd]

Thanks JR. I'll rename the project to be "masterdaemon" at your advice.

[zathras]

Zathras, thanks for your reply.

I gave some thought to things...to support both Windows and Linux (and even Mac OS X in the future) in as unified manner as possible, I'm thinking the best approach may be go to with your masterchest-engine/masterchest-library. I could use mono (under Linux and Mac OS X) to run it.

You mentioned testing under wine, but have you ever tested the engine and library under Mono? Beyond testing under mono (which I can do if you haven't), one enhancement I was wondering if you would be interested in making to masterchest-engine in order to make it possible for mastercoind (or whatever we end up calling it) to use your stuff cross-platform would be to allow it to write to a Sqllite database (instead of just SQLServer).

You could use System.Data.SQLite for this (http://system.data.sqlite.org/index.html/doc/trunk/www/index.wiki) ...probably just have a command line option to specify the engine to use (SQLServer or SQLite) and the actual SQL API commands wouldn't hopefully have to change much... you have very basic SQL statements and logic in there, thankfully.

My app could then periodically launch masterchest-engine.exe (either natively or via mono) to update the SQLite DB file, and my app would then just be reading the data from that DB as-needed. This would allow a single approach across all platforms, without requiring SQLServer on Windows...or any external database server. Sqlite supports a concurrent writer and reader no problem, with a bit of locking during the write itself.

This would make "mastercoind" be able to run well on Windows, Linux, and Mac OS X in the future....and I could come out with a build for Windows and Linux right off the bat. And as an advantage, folks on Windows that wanted to use Masterchest-engine directly could do so without having to install and run SqlServer, which is often overkill for smaller/test installations.

What do you think?

Firstly I'll just quickly say it doesn't have to be my library - you won't be stepping on any toes if you run with one of the other guys libs  With that said, if you think masterchest library & engine would be a good fit for you, have at it!

Just to clarify, I've only tested & fixed up the wallet for wine, not the engine.  Since they share a common codebase (and the same library) though I don't see too much of an issue.  I removed roadblock incompatibilities with the wallet & wine in about half a day so even if there are a few quirks with the engine, it's not masses of work to get it running.  Not tried Mono with either wallet or engine, but I'm happy to spin up a quick test if you like (will just connect to SQL over network for now).

I don't see any major headaches offering an SQLite option, I'd have to check it out but should be fine - let me come back to you on this.

FYI state processing for DEx is still very alpha in the engine & the matching isn't quite there yet.  State processing is fine for things like simple sends.

Yeah, I'm thinking your stuff is the best fit considering the multiplatform component. Tachikoma's stuff may work as a backup, but it looks like his whole data storage layer is inside the mastercoin-explorer project (as ruby on rails). Also, I did research calling over to a ruby library from python... didn't find anything conclusive with the short amount of time I spent on it.

masterchest-engine I should be able to us as-is (once it has the sqlite backend support and any necessary mono fixes), without having to touch/fork any of your code.

I'm installing SQL Server Express to be able to test masterchest-engine. The plan is to test it under windows first, then under mono (pushing to a SQL Server instance in Windows still). Once you're able to add the sqlite integration, I can test it all under mono in Linux.

Do you have any .sql files to create the tables and indexes for your schema or should I write my own?

It's on the to do list (auto creating tables) just hasn't been prioritized.

I'm just using express for masterchest.info - if you like I'm happy to shoot you a backup of its DB (the engine runs that site) which you can then just restore and test Mono off the bat without needing to manually do the tables?

From a cursory glance SQLite looks an easy enough addition.  Let me know how you get on with Mono (& what distro/kernel/mono versions you're testing with so I can duplicate).

Thanks

A sql schema dump would be great.... I just want to make sure I'm creating the schema to be like you are (a backup is fine too...just throw it up on dropbox or something and PM the link to me).

I'm testing on Ubuntu 13.10 x64, so Mono looks like 2.10, not 3.2.4:

"mono -V" output: "Mono JIT compiler version 2.10.8.1 (Debian 2.10.8.1-5ubuntu2)"
"uname -a" output: "Linux d01b 3.11.0-12-generic #19-Ubuntu SMP Wed Oct 9 16:20:46 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux"

Mono-wise, I've installed the "mono-runtime" package. If I need to install any additional packages (e.g. mono-complete, or other specific mono libs) just let me know.

I'm ready to test around sqlite as soon as you can add it, mono too. Thanks for your help on this!!

No probs Here you go:

Code:

USE [master]
GO
/****** Object:  Database [chestmaster]    Script Date: 11/26/2013 04:27:36 ******/
CREATE DATABASE [chestmaster] ON  PRIMARY 
( NAME = N'chestmaster', FILENAME = N'c:\Program Files\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL\DATA\chestmaster.mdf' , SIZE = 5120KB , MAXSIZE = UNLIMITED, FILEGROWTH = 1024KB )
 LOG ON 
( NAME = N'chestmaster_log', FILENAME = N'c:\Program Files\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL\DATA\chestmaster.ldf' , SIZE = 10176KB , MAXSIZE = 2048GB , FILEGROWTH = 10%)
GO
ALTER DATABASE [chestmaster] SET COMPATIBILITY_LEVEL = 100
GO
IF (1 = FULLTEXTSERVICEPROPERTY('IsFullTextInstalled'))
begin
EXEC [chestmaster].[dbo].[sp_fulltext_database] @action = 'enable'
end
GO
ALTER DATABASE [chestmaster] SET ANSI_NULL_DEFAULT OFF
GO
ALTER DATABASE [chestmaster] SET ANSI_NULLS OFF
GO
ALTER DATABASE [chestmaster] SET ANSI_PADDING OFF
GO
ALTER DATABASE [chestmaster] SET ANSI_WARNINGS OFF
GO
ALTER DATABASE [chestmaster] SET ARITHABORT OFF
GO
ALTER DATABASE [chestmaster] SET AUTO_CLOSE OFF
GO
ALTER DATABASE [chestmaster] SET AUTO_CREATE_STATISTICS ON
GO
ALTER DATABASE [chestmaster] SET AUTO_SHRINK OFF
GO
ALTER DATABASE [chestmaster] SET AUTO_UPDATE_STATISTICS ON
GO
ALTER DATABASE [chestmaster] SET CURSOR_CLOSE_ON_COMMIT OFF
GO
ALTER DATABASE [chestmaster] SET CURSOR_DEFAULT  GLOBAL
GO
ALTER DATABASE [chestmaster] SET CONCAT_NULL_YIELDS_NULL OFF
GO
ALTER DATABASE [chestmaster] SET NUMERIC_ROUNDABORT OFF
GO
ALTER DATABASE [chestmaster] SET QUOTED_IDENTIFIER OFF
GO
ALTER DATABASE [chestmaster] SET RECURSIVE_TRIGGERS OFF
GO
ALTER DATABASE [chestmaster] SET  DISABLE_BROKER
GO
ALTER DATABASE [chestmaster] SET AUTO_UPDATE_STATISTICS_ASYNC OFF
GO
ALTER DATABASE [chestmaster] SET DATE_CORRELATION_OPTIMIZATION OFF
GO
ALTER DATABASE [chestmaster] SET TRUSTWORTHY OFF
GO
ALTER DATABASE [chestmaster] SET ALLOW_SNAPSHOT_ISOLATION OFF
GO
ALTER DATABASE [chestmaster] SET PARAMETERIZATION SIMPLE
GO
ALTER DATABASE [chestmaster] SET READ_COMMITTED_SNAPSHOT OFF
GO
ALTER DATABASE [chestmaster] SET HONOR_BROKER_PRIORITY OFF
GO
ALTER DATABASE [chestmaster] SET  READ_WRITE
GO
ALTER DATABASE [chestmaster] SET RECOVERY SIMPLE
GO
ALTER DATABASE [chestmaster] SET  MULTI_USER
GO
ALTER DATABASE [chestmaster] SET PAGE_VERIFY CHECKSUM
GO
ALTER DATABASE [chestmaster] SET DB_CHAINING OFF
GO
USE [chestmaster]
GO
/****** Object:  Table [dbo].[transactions_processed]    Script Date: 11/26/2013 04:27:40 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [dbo].[transactions_processed](
	[TXID] [nvarchar](100) NOT NULL,
	[FROMADD] [nvarchar](100) NOT NULL,
	[TOADD] [nvarchar](100) NULL,
	[VALUE] [bigint] NULL,
	[TYPE] [nvarchar](100) NOT NULL,
	[BLOCKTIME] [nvarchar](100) NOT NULL,
	[BLOCKNUM] [nvarchar](100) NOT NULL,
	[VALID] [bit] NOT NULL,
	[CURTYPE] [int] NOT NULL,
	[ID] [int] IDENTITY(1,1) NOT NULL,
	[saleamount] [bigint] NULL,
	[offeramount] [bigint] NULL,
	[minfee] [bigint] NULL,
	[timelimit] [int] NULL,
	[purchaseamount] [bigint] NULL,
	[matchingtx] [nvarchar](100) NULL
) ON [PRIMARY]
GO
/****** Object:  Table [dbo].[transactions]    Script Date: 11/26/2013 04:27:40 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [dbo].[transactions](
	[TXID] [nvarchar](100) NOT NULL,
	[FROMADD] [nvarchar](100) NOT NULL,
	[TOADD] [nvarchar](100) NULL,
	[VALUE] [bigint] NULL,
	[TYPE] [nvarchar](100) NOT NULL,
	[BLOCKTIME] [bigint] NOT NULL,
	[BLOCKNUM] [int] NOT NULL,
	[VALID] [bit] NOT NULL,
	[CURTYPE] [int] NOT NULL,
	[ID] [int] IDENTITY(1,1) NOT NULL,
	[saleamount] [bigint] NULL,
	[offeramount] [bigint] NULL,
	[minfee] [bigint] NULL,
	[timelimit] [int] NULL,
	[purchaseamount] [bigint] NULL
) ON [PRIMARY]
GO
/****** Object:  Table [dbo].[processedblocks]    Script Date: 11/26/2013 04:27:40 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [dbo].[processedblocks](
	[BLOCKNUM] [int] NOT NULL,
	[BLOCKTIME] [bigint] NOT NULL
) ON [PRIMARY]
GO
/****** Object:  Table [dbo].[exotransactions]    Script Date: 11/26/2013 04:27:40 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [dbo].[exotransactions](
	[txid] [nvarchar](100) NOT NULL,
	[blocktime] [bigint] NOT NULL,
	[blocknum] [int] NOT NULL,
	[vouts] [nvarchar](2048) NOT NULL,
	[id] [int] IDENTITY(1,1) NOT NULL
) ON [PRIMARY]
GO
/****** Object:  Table [dbo].[exchange]    Script Date: 11/26/2013 04:27:40 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [dbo].[exchange](
	[txid] [nvarchar](100) NOT NULL,
	[fromadd] [nvarchar](100) NOT NULL,
	[type] [nvarchar](100) NOT NULL,
	[blocktime] [bigint] NOT NULL,
	[blocknum] [int] NOT NULL,
	[valid] [bit] NOT NULL,
	[curtype] [int] NOT NULL,
	[id] [int] IDENTITY(1,1) NOT NULL,
	[saleamount] [bigint] NULL,
	[offeramount] [bigint] NULL,
	[minfee] [bigint] NULL,
	[timelimit] [int] NULL,
	[purchaseamount] [bigint] NULL,
	[unitprice] [bigint] NOT NULL
) ON [PRIMARY]
GO
/****** Object:  Table [dbo].[balances]    Script Date: 11/26/2013 04:27:40 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [dbo].[balances](
	[ADDRESS] [nvarchar](100) NOT NULL,
	[CBALANCE] [bigint] NOT NULL,
	[CBALANCET] [bigint] NOT NULL,
	[ubalance] [bigint] NOT NULL,
	[ubalancet] [bigint] NOT NULL
) ON [PRIMARY]
GO
/****** Object:  Default [DF__transacti__VALID__0519C6AF]    Script Date: 11/26/2013 04:27:40 ******/
ALTER TABLE [dbo].[transactions] ADD  DEFAULT ('0') FOR [VALID]
GO
/****** Object:  Default [DF__transacti__CURTY__060DEAE8]    Script Date: 11/26/2013 04:27:40 ******/
ALTER TABLE [dbo].[transactions] ADD  DEFAULT ('0') FOR [CURTYPE]
GO
/****** Object:  Default [DF__balances__CBALAN__08EA5793]    Script Date: 11/26/2013 04:27:40 ******/
ALTER TABLE [dbo].[balances] ADD  DEFAULT ((0)) FOR [CBALANCET]
GO
/****** Object:  Default [DF__balances__ubalan__09DE7BCC]    Script Date: 11/26/2013 04:27:40 ******/
ALTER TABLE [dbo].[balances] ADD  DEFAULT ((0)) FOR [ubalance]
GO
/****** Object:  Default [DF__balances__ubalan__0AD2A005]    Script Date: 11/26/2013 04:27:40 ******/
ALTER TABLE [dbo].[balances] ADD  DEFAULT ((0)) FOR [ubalancet]
GO

[rbdrbd]

Great, thanks zathras! I'll start fiddling with masterchest-engine this evening.

One other thing: I looked through your library code and didn't see a way for it to handle simple sends (or DEx user-initiated operations).

In your wallet, I do see some of this functionality though: https://github.com/zathras-crypto/masterchest-wallet/blob/master/Form1.vb#L1142

Optimally, might it be possible for you to abstract this stuff (i.e. functions for simple send and similar DEx user-initiated functions) from the GUI logic and move it to your library or engine?

Then your wallet code could call this directly still. For masterdaemon, this functionality would just need to be wrapped up into a command-line utility, like grazcoin has for simple sends:

https://github.com/grazcoin/mastercoin-tools/blob/master/msc_send.py

If you abstract the functions over from your wallet to the library, I may be able to write this command line wrapper in .NET if you don't have the time (I'm rusty with it).

Let me know what you think about this.

[zathras]

Great, thanks zathras! I'll start fiddling with masterchest-engine this evening.

One other thing: I looked through your library code and didn't see a way for it to handle simple sends (or DEx user-initiated operations).

In your wallet, I do see some of this functionality though: https://github.com/zathras-crypto/masterchest-wallet/blob/master/Form1.vb#L1142

Optimally, might it be possible for you to abstract this stuff (i.e. functions for simple send and similar DEx user-initiated functions) from the GUI logic and move it to your library or engine?

Then your wallet code could call this directly still. For masterdaemon, this functionality would just need to be wrapped up into a command-line utility, like grazcoin has for simple sends:

https://github.com/grazcoin/mastercoin-tools/blob/master/msc_send.py

If you abstract the functions over from your wallet to the library, I may be able to write this command line wrapper in .NET if you don't have the time (I'm rusty with it).

Let me know what you think about this.

Always happy to help

The functions you speak of (creating send & DEx transactions) are already in the library (encodetx, encodeselltx, encodeaccepttx etc).  They're just not used in the engine as its original design intent was just parsing transactions and processing them to arrive at a known state rather than creating new transactions (that's left up to my wallet).

It should be quite easy to spin up a CLI binary to expose these functions though.  I'll throw you something together (it'll be quick & dirty, my primary focus is fixing DEx state processing in the engine & wallet at the mo).

This may also help for some background here

[zathras]

DEVELOPERS ONLY.  THIS IS NOT READY FOR END USERS.  THANKS!

Hey rbdrbd,

Here's the cli wrapper you asked for.  Just provides a binary to expose the library functions for encoding transactions.  Note there is no state or validation here, we're just creating the transaction.  The result can be signed & broadcast if you're happy with the output.

Please use with care, it's completely untested (there could be bad numeric conversions or other bugs for example) so only test against a wallet with fractional amounts

Let me know how you get on with it under Mono.  I'll take a look at putting sqlite support into the engine over the weekend, but for the next few days I really need to hunt down these bugs that are messing up my DEx states.

Thanks!

Built against .NET4.  Pick it up from here.  Note, use the -type=type switch to select between send, sell & accept.

Simple send:
Use the bitcoinrpc switches from the engine and -am (amount) -to (to address) -fr (from address) and -cu (currency type) switches.  For example:

Code:

MasterchestCLI.exe -bitcoinrpcserv=127.0.0.1 -bitcoinrpcport=8332 -bitcoinrpcuser=ooo -bitcoinrpcpass=ooo -type=send -fr=1KUu56RMsafvgwxkiGHmptrLLmBgQ16uQG -to=1MCHESTptvd2LnNp7wmr2sGTpRomteAkq8 -cu=MSC -am=1.234

Sell offer:
Use the bitcoinrpc switches from the engine and -am (sale amount) -fr (from address) -cu (currency type) -of (offer amount) -tl (timelimit) -mf (minfee) switches.  For example:

Code:

-bitcoinrpcserv=127.0.0.1 -bitcoinrpcport=8332 -bitcoinrpcuser=ooo -bitcoinrpcpass=ooo -type=sell -fr=1KUu56RMsafvgwxkiGHmptrLLmBgQ16uQG -cu=MSC -am=1.234 -of=0.5 -tl=6 -mf=0.0001

Accept offer:
Use the bitcoinrpc switches from the engine and -am (purchase amount) -fr (from address) -to (to address) -cu (currency type) switches.  For example:

Code:

-bitcoinrpcserv=127.0.0.1 -bitcoinrpcport=8332 -bitcoinrpcuser=ooo -bitcoinrpcpass=ooo -type=accept -fr=1KUu56RMsafvgwxkiGHmptrLLmBgQ16uQG -to=1MCHESTptvd2LnNp7wmr2sGTpRomteAkq8 -cu=MSC -am=1.234