300 BTC Coding Contest: Distributed Exchange (MasterCoin Developer Thread)

[grazcoin]

Hi guys, quick question on the distributed exchange: as Mastercoin is becoming much more valuable now, is there a way to create buy and sell offers from a totally "offline" machine?

I was planning on moving to an offline armory-based setup (with cheapo laptop with no internet connection) for BTC, and wanted to use this same kind of setup with MSC as well...i.e. comprise the trade on the offline machine (that has wallet.dat), then sneakernet/email some trade code/key/trade token to the connected machine to actually input into the MSC network (that does not have wallet.dat). It seems that currently one can not do simple sends with an offline setup like this (from my own experimentation), but I wanted to check on the distributed exchange features.

If this is not possible, then any ideas when someone may get around to it? If MSC raises another 10-30x, you'll have several MSC millionaires in the bunch. If I was one of those guys, I'd be very concerned with doing any trades out of a wallet connected to the internet.

(oh and Tachikoma...btw, the top50 page is broken again ...it displays, but is not showing the full list like it was before...but if you're having parse errors/invalids, then that would make sense why http://mastercoin-explorer.com/addresses)

mastercoin-tools / masterchain.info will support offline wallet in a user-friendly way.
I expect it to take another 1-2 weeks until it is ready.

[1715449812]

1715449812

[1715449812]

1715449812

[Bitoy]

Since there are only three 0.00006 outputs, one of which is the Exodus we know the recipient address is the one that does not contain the mastercoin data. I believe we decided this would be valid. 

Ok thanks.    transaction falls in the "peek and decode".

[ripper234]

mastercoin-tools / masterchain.info will support offline wallet in a user-friendly way.
I expect it to take another 1-2 weeks until it is ready.

The speed at which speed is accelerating speed is just accelerating with a speedy acceleration.

Crazy world.

[zathras]

Mastercoin web-services verification API proposal.

The Mastercoin validation rules are getting harder the more transaction types are being added. We need a way to easily determine when one of the implementations differs from the others. In order to faciliate this I would recommend every webbased Mastercoin service implements an API that repsonds to a few API calls in a specific format. This data can be used by a third-party application/webapplication to compare implementations and give notifications when there is no consensus.

I would like to start out simple with just two API calls which return valid JSON files.

GET /mastercoin-verify/addresses

Code:

[{address: 1KZmDQGzGJWYmPP9X3b7TA9dY91KBXgaG4, balance: 20.1}, {address: 1Q1sFqsi8S5DxV5hz6sWLamGBp9To93iG7, balance: 3.1}, etc..]

This could be used to do a quick and dirty comparison between address balances. If the amount differs between implementations the following API call be used to find the offending transaction.

GET /mastercoin-verify/transactions/<ADDRESS>

Code:

{address: 1KZmDQGzGJWYmPP9X3b7TA9dY91KBXgaG4, transactions: [{tx_hash: 5f01def181b761f1d03bcd20590c5729a47b11c68955b364add9253d7aec5eb9, valid: true}, {tx_hash: 130c5175d4f3e9add03bd1d115a87b26e613293fbe3815b970f8fc830f018ebc, valid: false}....etc]}

Using these methods we can find the offending transaction and report it. Either by sending an email to signed up developers, simply displaying the transaction on a site or some other way. End users can even use it themselves to verify their own transactions and make sure they are valid everywhere.

We could make the urls customisable and create a robots.txt like file that tells us where to find the API calls per service but I couldn't think of a reason why we can't hardcode the paths.

I would like to make this an addendum to the spec, although I'm not 100% sure it belongs there. Opinions?

Completely agree this would be a good idea.  My views on APIs from the beginning have been that we should have compatible APIs available for end users so thin clients don't have to rely on a single 'source of truth' (ie they can aggregate data from multiple sources for verification). 

As to the specific approach I'd need to have a think about this (as currently for my implementations you would use GET ......./whatever.aspx?parameter=value rather than GET ...../parameter/value).

[zathras]

Since there are only three 0.00006 outputs, one of which is the Exodus we know the recipient address is the one that does not contain the mastercoin data. I believe we decided this would be valid. 

Ok thanks.    transaction falls in the "peek and decode".

Yep; peek and decode because the sequence numbers are wrong.  The open question is whether 'peek and decode' is acceptable.  Tachikoma & I after some discussion agreed that we'd allow for peek & decode if the sequence numbers weren't done right, but JR has to make the final call as to whether that makes it into the spec.

From the suggested amendment:

A last resort 'peek and decode' method may be used to identify the data packet in the event of ambiguity following the above rules. This involves decoding each packet and looking for the correct bytes for a simple send (the majority of bytes in a Class A simple send do not change).
These byte checks are defined as:
o Bytes two to eight must equal 00
o Byte nine must equal 01 or 02
Should there still be packet ambiguity or 'peek and decode' reveals more than one packet (simple sends are always one packet) the transaction is considered invalid.

FYI fwiw as it stands now Masterchest does not currently implement 'peek & decode' in code.  It's a principal we've suggested and I'm waiting to see if it's accepted before cutting code.

Another example is ecb77ee990de29745de949462e1f6e44584c310a0da12c9fbdf86dbe6ffabcfc, we discussed this one a few pages back.  The sequence numbers are in the wrong order so Masterchest throws it out.  'Peek and decode' could have validated this transaction.

[djohnston]

I just wanted to drop a note here congratulating Ron Gross (ripper234) on volunteering to commit his full time efforts as the Executive Director for the Mastercoin Foundation (unanimously confirmed by the Board).

As many of you know and J.R. has stated here before, the Mastercoin Foundation has been searching for someone to take the Executive Director role for the last 70 days since the Exodus Address period finished.

This emerged organically as Ron began spending more and more of his time on the Mastercoin Project, after having participated in the Exodus Address and being very active in the community. I've witnessed first hand Ron passionately spread the word to many early adopters, host the very first Mastercoin meetup, actively recruit additional developers, advocate for improvements to the protocol including the Proof of Stake system / Contracts For A Difference, and get the Mastercoin protocol officially up on Github.

Thanks Ron for committing your time, energy and talents to this extremely important project for all of us in the Bitcoin / Mastercoin community.

David A. Johnston - Executive Director of BitAngels.co and Board Member of the Mastercoin Foundation

P.S. All the developers on this thread are awesome! I check in about once a day and I'm constantly blown away by the progress and how fast things are moving forward. Its not said often enough, so let me just state for the record, "You guys are rock stars".



  

[Tachikoma]

As to the specific approach I'd need to have a think about this (as currently for my implementations you would use GET ......./whatever.aspx?parameter=value rather than GET ...../parameter/value).

If creating custom urls is a problem for you we can just do the discovery method. I'm guessing creating a mastercoin.txt file in your root is possible? If so we can have a json file there that has two keys.

Code:

{addresses_index: {method: "get", url: "/mastercoin-verify/addresses"}, transaction_lookup: {method: "get", url: "/mastercoin-verify/transactions/", parameter: "id"}}

This would tell the API where and how to query the data. The transaction lookup would become /mastercoin-verify/transactions?id=addresssuppliedhere.

Would this work for everybody?

I just wanted to drop a note here congratulating Ron Gross (ripper234) on volunteering to commit his full time efforts as the Executive Director for the Mastercoin Foundation (unanimously confirmed by the Board).

As many of you know and J.R. has stated here before, the Mastercoin Foundation has been searching for someone to take the Executive Director role for the last 70 days since the Exodus Address period finished.

This emerged organically as Ron began spending more and more of his time on the Mastercoin Project, after having participated in the Exodus Address and being very active in the community. I've witnessed first hand Ron passionately spread the word to many early adopters, host the very first Mastercoin meetup, actively recruit additional developers, advocate for improvements to the protocol including the Proof of Stake system / Contracts For A Difference, and get the Mastercoin protocol officially up on Github.

Thanks Ron for committing your time, energy and talents to this extremely important project for all of us in the Bitcoin / Mastercoin community.

David A. Johnston - Executive Director of BitAngels.co and Board Member of the Mastercoin Foundation

P.S. All the developers on this thread are awesome! I check in about once a day and I'm constantly blown away by the progress and how fast things are moving forward. Its not said often enough, so let me just state for the record, "You guys are rock stars".   

Agreed, it's awesome Ros is taking the lead. The only problem is that he has way too many ideas and the developers can't keep up with the growing feature-list

[ripper234]

Mastercoin web-services verification API proposal.

The Mastercoin validation rules are getting harder the more transaction types are being added. We need a way to easily determine when one of the implementations differs from the others. In order to faciliate this I would recommend every webbased Mastercoin service implements an API that repsonds to a few API calls in a specific format. This data can be used by a third-party application/webapplication to compare implementations and give notifications when there is no consensus.

I would like to start out simple with just two API calls which return valid JSON files.

Terrific proposal Tachikoma!
I haven't delved into the technicals, but I love how we distribute and standardize even our QA processes.
This should be in the spec. Submit a pull request?

[ripper234]

Agreed, it's awesome Ros is taking the lead. The only problem is that he has way too many ideas and the developers can't keep up with the growing feature-list

Sorry for making your lives so difficult

If it's comforting to you, I have too many ideas for myself as well ... I can't sleep in the last week because of all the adrenaline.
I know I'll have to wind down a bit (health is top priority), but I'm still pushing myself just a bit more because I can't let this go.

Don't worry all, I'm in perfect health ... there's just so much going on these days with Bitcoin, Mastercoin and everything, it's mind boggling.

[zathras]

I just wanted to drop a note here congratulating Ron Gross (ripper234) on volunteering to commit his full time efforts as the Executive Director for the Mastercoin Foundation (unanimously confirmed by the Board).

As many of you know and J.R. has stated here before, the Mastercoin Foundation has been searching for someone to take the Executive Director role for the last 70 days since the Exodus Address period finished.

This emerged organically as Ron began spending more and more of his time on the Mastercoin Project, after having participated in the Exodus Address and being very active in the community. I've witnessed first hand Ron passionately spread the word to many early adopters, host the very first Mastercoin meetup, actively recruit additional developers, advocate for improvements to the protocol including the Proof of Stake system / Contracts For A Difference, and get the Mastercoin protocol officially up on Github.

Thanks Ron for committing your time, energy and talents to this extremely important project for all of us in the Bitcoin / Mastercoin community.

David A. Johnston - Executive Director of BitAngels.co and Board Member of the Mastercoin Foundation

P.S. All the developers on this thread are awesome! I check in about once a day and I'm constantly blown away by the progress and how fast things are moving forward. Its not said often enough, so let me just state for the record, "You guys are rock stars".

Fantastic   Congrats & Thanks Ron.

As to the specific approach I'd need to have a think about this (as currently for my implementations you would use GET ......./whatever.aspx?parameter=value rather than GET ...../parameter/value).

If creating custom urls is a problem for you we can just do the discovery method. I'm guessing creating a mastercoin.txt file in your root is possible? If so we can have a json file there that has two keys.

Code:

{addresses_index: {method: "get", url: "/mastercoin-verify/addresses"}, transaction_lookup: {method: "get", url: "/mastercoin-verify/transactions/", parameter: "id"}}

This would tell the API where and how to query the data. The transaction lookup would become /mastercoin-verify/transactions?id=addresssuppliedhere.

Would this work for everybody?

Actually taking a second glance at this I can just use a URL rewrite mod so shouldn't be a problem

[ripper234]

Fantastic   Congrats & Thanks Ron.

I think I just noticed your profile photo for the first time.
WE ALL LOVE ZATHRAS!!!

[Tachikoma]

Mastercoin web-services verification API proposal.

The Mastercoin validation rules are getting harder the more transaction types are being added. We need a way to easily determine when one of the implementations differs from the others. In order to faciliate this I would recommend every webbased Mastercoin service implements an API that repsonds to a few API calls in a specific format. This data can be used by a third-party application/webapplication to compare implementations and give notifications when there is no consensus.

I would like to start out simple with just two API calls which return valid JSON files.

Terrific proposal Tachikoma!
I haven't delved into the technicals, but I love how we distribute and standardize even our QA processes.
This should be in the spec. Submit a pull request?

Actually taking a second glance at this I can just use a URL rewrite mod so shouldn't be a problem

Awesome. I will submit a pull request later today.

[ripper234]

I just wanted to drop a note here congratulating Ron Gross (ripper234) on volunteering to commit his full time efforts as the Executive Director for the Mastercoin Foundation (unanimously confirmed by the Board).

As many of you know and J.R. has stated here before, the Mastercoin Foundation has been searching for someone to take the Executive Director role for the last 70 days since the Exodus Address period finished.

This emerged organically as Ron began spending more and more of his time on the Mastercoin Project, after having participated in the Exodus Address and being very active in the community. I've witnessed first hand Ron passionately spread the word to many early adopters, host the very first Mastercoin meetup, actively recruit additional developers, advocate for improvements to the protocol including the Proof of Stake system / Contracts For A Difference, and get the Mastercoin protocol officially up on Github.

Thanks Ron for committing your time, energy and talents to this extremely important project for all of us in the Bitcoin / Mastercoin community.

David A. Johnston - Executive Director of BitAngels.co and Board Member of the Mastercoin Foundation

P.S. All the developers on this thread are awesome! I check in about once a day and I'm constantly blown away by the progress and how fast things are moving forward. Its not said often enough, so let me just state for the record, "You guys are rock stars".

Thanks a lot David!

My reply

http://pastebin.com/V1D8f2Hi

[Bitoy]

Mastercoin web-services verification API proposal.

GET /mastercoin-verify/addresses

Code:

[{address: 1KZmDQGzGJWYmPP9X3b7TA9dY91KBXgaG4, balance: 20.1}, {address: 1Q1sFqsi8S5DxV5hz6sWLamGBp9To93iG7, balance: 3.1}, etc..]

Hi Tachikoma,

Will this json work?
http://mymastercoins.com/MSCJSON.aspx

[Tachikoma]

Yes, I believe so. I haven't started building it myself so I can't be 100% sure but it looks valid enough!

Can you also use a mod-rewrite like solution to get a proper url?

[Tachikoma]

I've just send a pull request on Github for the verification API. I've added one more thing and that's supplying the currency_id you want to see the data on. Easiest way to read the text is not via the PR but via this direct link. As always please discuss the specification in the pull request itself to keep the discussion centralised.

[rbdrbd]

I've just send a pull request on Github for the verification API. I've added one more thing and that's supplying the currency_id you want to see the data on. Easiest way to read the text is not via the PR but via this direct link. As always please discuss the specification in the pull request itself to keep the discussion centralised.

An idea: this API should be a great start for web-based MSC packages, but as at least I see the base MSC parsing libraries (such as mastercoin-tools maybe) being the base of the implementation, wouldn't it be good to add a unit test suite to them? Or are these various web tools currently using their own MSC parser implementations at this time?

I'm wondering what everyone's thought is to these multiple implementations of the MSC spec... eventually a good idea may be to code a reference implementation up in C/C++, which can then be brought up into higher level language libraries (in Ruby/Python/Java/C#, etc) using tools like swig/FFI/Cython, etc to prevent having to duplicate the "guts" of the protocol for each separate language. Maybe this was the idea behind mastercoin-tools and pymastercoin, etc eventually...??

Even the above would not be necessary if there was a standard "test suite" list of test scenarios published, which each MSC parser library implementation would build a scripted unit test suite against. Then, as various "gotchas" were caught that fell through the cracks of the test suite, it could be enhanced to cover those, as well as giving additional test scenarios for new features as they are built in and implemented across the various libraries. The pass rate on this standard list of test scenarios would denote the overall quality of the library, and keep functionality consistent across languages, sites, etc. "Certified status" could even be awarded by the Mastercoin foundation to implementations that consistently have a pass rate > X%.

I think this is in the spirit of what Tachikoma is trying to do, but not every MSC tool is web-based, with a web-based API.

Just a thought...

[Tachikoma]

I'm already writing an rspec test-suite for my Rails implementation. It won't have 100% coverage but it will give you a good explanation of what to test for and how to implement it.

My implementation is based on Bitcoin-ruby, Zathras uses Bitcoin and Grazcoin Obelisk, loads of different backends

[rbdrbd]

I'm already writing an rspec test-suite for my Rails implementation. It won't have 100% coverage but it will give you a good explanation of what to test for and how to implement it.

My implementation is based on Bitcoin-ruby, Zathras uses Bitcoin and Grazcoin Obelisk, loads of different backends

Sounds great. Maybe it may make sense for your test suite cases to be standardized at some level eventually (and possibly maintained independently), so that other library implementations can use those same test cases. That would make debugging differences between implementations (and their bugs) more apples-to-apples, and give a good "baseline" point for new library/backend implementations.

[Bitoy]

MyMastercoins.com

Here are the JSON files based on Tachikoma's API
http://mymastercoins.com/jaddress.aspx
http://mymastercoins.com/jtransactions.aspx?Address=$AddressHere


The default CurrencyID=1.   You can place &CurrencyID=2 for Test MSC.

PS.  I'm still working on the URL rewrite.