300 BTC Coding Contest: Distributed Exchange (MasterCoin Developer Thread)

[dacoinminster]

J.R can comment his perspective on this. Here is mine:

We have a bit of a mixup here. On the one hand, I do believe that the "High bar for usability" goal has not been met, and doubt it will be met this month.
On the other hand, we moved to paying out monthly bounties in February, and .

I hope it is clear that the DEx bounties paid in Feb, and the March monthly bounty, count towards the 300 BTC bounty (out of which 150 BTC has been previously paid).
So after March I don't think that a lot of money will remain in the original 300 BTC money pool (J.R needs to reply with some numbers).
I expect the bulk of the money remaining from the DEx to be divided in March's monthly payout.

Perhaps if there is only a tiny amount left after that, we will decide to increase March's payout and pay out the full remainder of the 300 BTC bounty this month, in order to avoid the accounting/judging headache. We will wait for J.R's number regarding February's payout (due any day now), and see what makes sense.

Yup. February numbers are nearly ready. They are my top priority once I catch up on urgent emails.

edit: I think we are close on usability. If we spend the next couple weeks fixing things that make the various wallets hard to use and trade with, I don't see any barrier to finishing the payout at the end of March.

[1714489720]

1714489720

[1714489720]

1714489720

[1714489720]

1714489720

[1714489720]

1714489720

[dacoinminster]

Guys,

Here are the addresses I plan to use for February, based on what I have from each of you:

Who	BTC Address	MSC Address
Faiz	1JJ6e9iWjGgEnCQRivKvdfNjRTgYVaKtWy	1MMF42zk8f4TqoV1QFeia18bb14rBHXzAU
Colbert Lau	1MnPcdUFDYnTz3jz9n3erfWbd7JZfPunxL	??
Grimentz	128MGmQKtKUPf66w2w9eaGc7YBqLrcnLx5	1rM1oMEFMfhPBo4kRaf2CMXf9XQCHyYWi
Adam Chamely	18hFjy7CWvX7FgZEXEW3bgz35KpYAzFR1c	18hFjy7CWvX7FgZEXEW3bgz35KpYAzFR1c
Bitoy	14cn7BiyzUeMcddC3RUFk77Z2k8NcP66yf	14cn7BiyzUeMcddC3RUFk77Z2k8NcP66yf
jeroenn13	15EMewc1nH8Cqzsx8hrFNNdbQBjVTEWahp	15EMewc1nH8Cqzsx8hrFNNdbQBjVTEWahp
Marvshneider	1ACsaXeqiPwZVzEuWXa2vwe9etxaet6QWg	1PKKRfHgvfZi2ToZDKkKrCUFpvuVKHz9WV
Ryan Keenan	1PyQbHr3ywRdbkcq3sZKpkkJNBJUXbGXJH	13NzZ7nwzEzLfuCEtfC1iQv5mJzs4PsFoW
grazcoin	1CZ5WRYj48vWV7cYRRdheyhiodMVP3EMi3	1CZ5WRYj48vWV7cYRRdheyhiodMVP3EMi3
Tesca	16BurP5f7n5RqjWGQ44Lj5NX6GzxaoAaqU	16BurP5f7n5RqjWGQ44Lj5NX6GzxaoAaqU
Craig	17xr7sbehYY4YSZX9yuJe6gK9rrdRrZx26	??
MasterXchange	1zobh1SwsVdk3uMjEHeGV86hjNZeJ5CwC	1zobh1SwsVdk3uMjEHeGV86hjNZeJ5CwC
Curtis	19TwBziBeCb4VnrZgJasJukwRKNSVvDp7Y	13pm7cmA5vVpKkDLJCvqh26kcp6V6PJ1Aq
dexX	11NiSNoGYomEA3ZHEN4R9ne4kwDjECsAy	11NiSNoGYomEA3ZHEN4R9ne4kwDjECsAy
Marin(Bebopzzz)	15AnM1gg5HAAnRPPBUXFX1N6mbTt4cLyZJ	1GmsjoERUACUwAsjKSLhSekH8EeXLRWKku
Shannon Code / Genecyber	1N6vMzy46RNGny7u5BYMFGpPin7uW3cTKK	1DMH2jezmNha2ShBhq3ogsXGviPBRppmwX
jakecnn	1HXsrKJHbJCdWThFQL14tQ2ThuUhKUzmsf	1HXsrKJHbJCdWThFQL14tQ2ThuUhKUzmsf
Patrick	1EdtK7EkErgeU17qRuifxASA2yMoxibxYv	1AQBeWsRfbHst7NojYdt42MZThcR4jy8fk
Zathras	1DrYXscsbMxhQMvvCmK7aVppbJGhr3RRSN	1Px5ruEuWdtvkG3AJ1sJ8Q7ckV8B1f3us5

Please check your address to make sure it is correct. Speak now, or forever hold your peace!!

[LOL]

I started a discussion with dexX7 over in the mastercoin announcement thread. He invited me over here due to the technical nature.

I've seen a bit of mastercoin history - I participated in the fundraiser, I attempted a few decentralized trades back in mid November here on this forum, and I was one of the first to complete a trade on the 15th. I consider myself pretty gung ho about mastercoin, however a few items have left me a bit disenfranchised recently. So, if you'll excuse some potentially blunt sentences I'd like to open up discussion on some issues that I feel fairly strongly about.

I'm going to preface this with a quick distinction. As demonstrated below, a private key produces two distinct key pairs. Both key pairs have the same private key, however as they are implemented in a wallet, the private key WIF of key pair #1 cannot spend outputs sent to address #2, and the compressed private key WIF of key pair #2 cannot spend outputs to address #1.

Code:

Private Key: d0d772f23d11b78aefdecc9c91715518381dffd995818772996168cc397b8254
Private Key WIF: 5KQGADurfnHLPnwRQ2myYztUYabnCLvErWAgEXy9rD3Z5d3xRbv
Private Key WIF Hex Encoded: d0d772f23d11b78aefdecc9c91715518381dffd995818772996168cc397b8254
Public Key: 0490871c4239585c8b879bdbb9f50c7f49258db450ac010a6c352ca60d674c968e5135abe7eabfb1cd11782fb54b4dde5f66c5ce4d670c693aa66bc250dbe23236
Address: 1F7Bp8NNPyGY8i1p3nw1vmTpUePQhScP72

Private Key: d0d772f23d11b78aefdecc9c91715518381dffd995818772996168cc397b8254
Compressed Private Key WIF: L4DfuhzJEAs6bbmCgeqWcEWCaPJWV4uz9VjiAjgr4CAGb59Yrn3K
Compressed Private Key WIF Hex Encoded: d0d772f23d11b78aefdecc9c91715518381dffd995818772996168cc397b825401
Public Key: 0290871c4239585c8b879bdbb9f50c7f49258db450ac010a6c352ca60d674c968e
Address: 1Pbme9p5J73P2FcBtADsBTaKhd8Gct6D19

Keep that in mind while I continue.

• Currently different mastercoin implementations generate Class B transactions differently. One includes the sender's uncompressed public key in the P2SH, while another includes the sender's compressed public key in the P2SH.
• I acknowledge that the outputs produced by both implementations are spendable, however I pose that it's not currently reasonable to expect a user to spend them.
• There is not a mastercoin implementation that will always generate a P2SH output that can be spent by key pair associated with the sending address.
• There is a likelihood that the P2SH output will not be able to be spent by a key pair in the sending addresses wallet.
• P2SH outputs from Class B transactions are not recognized, and cannot be spent by any bitcoin or mastercoin wallet currently available(please correct if wrong)
• If mastercoin clients send P2SH outputs to both compressed and uncompressed public keys originating from a single private key, the implication is that mastercoin recognizes a "unit of identity" as being defined by a private key, and thus two key pairs. This isn't compatible with any other client that I'm aware of, can be a privacy leak, and security risk if one of the WIF private keys is compromised. Further, I'd be inclined to think that mastercoin implementations would always need to display and identify both key pairs associated with a private key.
• Class B transactions technically do not create unspendable outputs, however it's a cop out to say that Class B transactions address the concern of blockchain bloat due data storage. If outputs of a transaction are not recognized, cannot be spent, or are otherwise not used in transactions when reasonable, they will not be spent and are subject to loss due to perceived lack of value or understanding.

What I'm trying to point out is that the absence of standardization between mastercoin implementations creates a weird scenario in which it's significantly harder to spend P2SH outputs than it has to be. If a wallet could recognize and spend those P2SH outputs, another weird situation occurs in which the P2SH outputs cannot all be spent without the wallet generating the other key pair associated with a private key. IMO, that's funky and there's no graceful or secure way to handle it.

The next thing I want to bring up is the mastercoin specification. It's not my intent to be a stickler, however I believe it's critical to meticulously maintain the specification, because without the specification there is no mastercoin and there is no value. Yes, the disappearance of the mastercoin specification is an extreme case. So let me provide a lesser example. The mastercoin specification states that a Class B transaction must satisfy the following:

"Has one or more 1-of-n OP_CHECKMULTISIG outputs each containing at least two compressed public keys, one of which must be the senders address."

Logical conclusions that follow:

• Only a compressed key pair can broadcast valid Class B transactions.
• There is a mastercoin client that is actively broadcasting invalid Class B transactions.
• Since no mastercoin implementation recognizes a Class B transaction with a P2SH output to an uncompressed public key as invalid, there is a growing chain of invalid transactions that is publicly and unanimously identified as valid.

That which is built to a specification should model that specification as closely as possible with a minimum of interpretation. It appears to me that the the specification is modeled after multiple different mastercoin implementations, is up to interpretation, and does not serve the purpose of defining mastercoin. As such, one could argue that a developer's implementation defines mastercoin and mastercoin is therefore subject to his whim. I am not stating this as my point of view. I say it to demonstrate the importance of the mastercoin specification. The value of mastercoin and the mastercoin protocol rely directly on a specification which defines mastercoin transactions and mastercoin implementation. Yes, it's a slippery slope - if the user can't rely on the specification to define x, when why should it be relied on to define y?

edit: I think we are close on usability. If we spend the next couple weeks fixing things that make the various wallets hard to use and trade with, I don't see any barrier to finishing the payout at the end of March.

Like I said previously, I've been enthusiastic about mastercoin. However, recently it feels esoteric, loose, and out of touch. I recognize that decentralized trading is groundbreaking, and I recognize that trades are being made.

But I do not think that it is by any means reasonable to think that a client is close to a "high bar for usability" if it doesn't recognize the outputs of a transaction it created, signed and broadcast, cannot spend unspent outputs which it is (cryptographically) capable, and creates multisig outputs that cannot be spent by a key pair in the wallet. Further, there are only native Windows clients, and other OSes have to use a web client and manually handle WIF private keys.

Also, the depth of knowledge required to spend P2SH outputs as they are currently generated is unreasonable to expect in a user. Like I said previously,

If outputs of a transaction are not recognized, cannot be spent, or are otherwise not used in transactions when reasonable, they will not be spent and are subject to loss due to perceived lack of value or understanding.

If a mastercoin client can't accommodate and expect this perceived lack of value and understanding in a user, it is irresponsible to taut the client as one that has a "high bar for usability."

Mastercoin,

• uses irresponsible implementations of Class B transactions.
• creates outputs that aren't spendable by any wallet.
• greatly privileges Windows users.
• and doesn't have a specification that describes current implementations.

I feel strongly enough about these items that I'd rather see some sort of resolution than sleep like a normal person. I don't feel like it's my place to offer unsolicited solutions to the problems I'm posing, however would be happy to discuss my ideas on the subjects.

And if consensus is that the items I bring up are not problems, take this as feedback from a "long time mastercoin user" that the project feels out of touch, lacks on the PR front, and has direction at the expense of overall quality.

[ninjaboon]

Guys,

Here are the addresses I plan to use for February, based on what I have from each of you:

Who	BTC Address	MSC Address
Faiz	1JJ6e9iWjGgEnCQRivKvdfNjRTgYVaKtWy	1MMF42zk8f4TqoV1QFeia18bb14rBHXzAU
Colbert Lau	1MnPcdUFDYnTz3jz9n3erfWbd7JZfPunxL	??
Grimentz	128MGmQKtKUPf66w2w9eaGc7YBqLrcnLx5	1rM1oMEFMfhPBo4kRaf2CMXf9XQCHyYWi
Adam Chamely	18hFjy7CWvX7FgZEXEW3bgz35KpYAzFR1c	18hFjy7CWvX7FgZEXEW3bgz35KpYAzFR1c
Bitoy	14cn7BiyzUeMcddC3RUFk77Z2k8NcP66yf	14cn7BiyzUeMcddC3RUFk77Z2k8NcP66yf
jeroenn13	15EMewc1nH8Cqzsx8hrFNNdbQBjVTEWahp	15EMewc1nH8Cqzsx8hrFNNdbQBjVTEWahp
Marvshneider	1ACsaXeqiPwZVzEuWXa2vwe9etxaet6QWg	1PKKRfHgvfZi2ToZDKkKrCUFpvuVKHz9WV
Ryan Keenan	1PyQbHr3ywRdbkcq3sZKpkkJNBJUXbGXJH	13NzZ7nwzEzLfuCEtfC1iQv5mJzs4PsFoW
grazcoin	1CZ5WRYj48vWV7cYRRdheyhiodMVP3EMi3	1CZ5WRYj48vWV7cYRRdheyhiodMVP3EMi3
Tesca	16BurP5f7n5RqjWGQ44Lj5NX6GzxaoAaqU	16BurP5f7n5RqjWGQ44Lj5NX6GzxaoAaqU
Craig	17xr7sbehYY4YSZX9yuJe6gK9rrdRrZx26	??
MasterXchange	1zobh1SwsVdk3uMjEHeGV86hjNZeJ5CwC	1zobh1SwsVdk3uMjEHeGV86hjNZeJ5CwC
Curtis	19TwBziBeCb4VnrZgJasJukwRKNSVvDp7Y	13pm7cmA5vVpKkDLJCvqh26kcp6V6PJ1Aq
dexX	11NiSNoGYomEA3ZHEN4R9ne4kwDjECsAy	11NiSNoGYomEA3ZHEN4R9ne4kwDjECsAy
Marin(Bebopzzz)	15AnM1gg5HAAnRPPBUXFX1N6mbTt4cLyZJ	1GmsjoERUACUwAsjKSLhSekH8EeXLRWKku
Shannon Code / Genecyber	1N6vMzy46RNGny7u5BYMFGpPin7uW3cTKK	1DMH2jezmNha2ShBhq3ogsXGviPBRppmwX
jakecnn	1HXsrKJHbJCdWThFQL14tQ2ThuUhKUzmsf	1HXsrKJHbJCdWThFQL14tQ2ThuUhKUzmsf
Patrick	1EdtK7EkErgeU17qRuifxASA2yMoxibxYv	1AQBeWsRfbHst7NojYdt42MZThcR4jy8fk
Zathras	1DrYXscsbMxhQMvvCmK7aVppbJGhr3RRSN	1Px5ruEuWdtvkG3AJ1sJ8Q7ckV8B1f3us5

Please check your address to make sure it is correct. Speak now, or forever hold your peace!!

JR.

My Mastercoin Address:
1KSim7sKQ9bjsMd2RyUbe9HxU73yX6CwuJ

will drop an email too.

regards
Colbert Lau

[TKeenan]

[omitted]...And if consensus is that the items I bring up are not problems, take this as feedback from a "long time mastercoin user" that the project feels out of touch, lacks on the PR front, and has direction at the expense of overall quality.

Wow.  That was a mouthful.  You probably have a fair point on some of that stuff.  For sure the PR is lacking.  No offense to Dom who works hard.  But I think the priority has been to work diligently on the technical aspects.  Once those are working, a good PR scheme will introduce them properly to the community. 

On the other hand, it seems like a knowledgeable longtime mastercoiner like you has loads to contribute.  I sure hope you find your deal with the team and are welcomed into the effort to perfect that which will soon become a very powerful Mastercoin protocol. 

[dexX7]

...

Say the protocol would adapt a more strict output policy (e.g. only a valid pub key of the sender + data-packages are allowed), how would you deal with historical transactions that may be affected by such a change?

[dacoinminster]

<snip>

Mastercoin,

• uses irresponsible implementations of Class B transactions.
• creates outputs that aren't spendable by any wallet.
• greatly privileges Windows users.
• and doesn't have a specification that describes current implementations.

I feel strongly enough about these items that I'd rather see some sort of resolution than sleep like a normal person. I don't feel like it's my place to offer unsolicited solutions to the problems I'm posing, however would be happy to discuss my ideas on the subjects.

And if consensus is that the items I bring up are not problems, take this as feedback from a "long time mastercoin user" that the project feels out of touch, lacks on the PR front, and has direction at the expense of overall quality.

I'll let one of the devs weigh in on the class B stuff. We do plan to have better options for non-windows users soon. Omniwallet should be a good solution, and for those who want to run a full node, we want to have a full desktop client too (possibly a port of Masterchest using mono). We hope to greatly improve usability on all wallets which we sponsor.

I think the spec is accurate right now, but if you've found something that is out of date, we welcome pull requests

Thanks for weighing in.

[LOL]

...

Say the protocol would adapt a more strict output policy (e.g. only a valid pub key of the sender + data-packages are allowed), how would you deal with historical transactions that may be affected by such a change?

Easy.

First, define the purpose of the mastercoin specification. Revisions as necessary.

Second, add a section to the specification that details a system of revision control, and how a specification revision affects past and future mastercoin transactions. I'd start with the following bulletpoints:

A specification revision...

• Does not change the validity of any mastercoin transaction prior to the public publication.
• Will not impose changes that cannot or do not take place immediately upon public publication. e.g. define Class B transactions as only valid if OP_RETURN is used for data storage, because no implementation is prepared to make this transition.
• Is considered to define mastercoin, the mastercoin protocol (etc.) until the point in time at which another valid revision is publicly published.
• Is defined as valid when cryptographically signed by the entity that is granted the right to revise the specification.
• Is defined as publicly published when the following has taken place: 1.) The revision and signature is announced and made publicly available, and 2.) The hash of the revision is stored in the blockchain and is signed by the same entity that signed the revision.

And thus established is a method by which an individual can independently prove the validity of a mastercoin transaction at any point in time. The only tricky revision is the first one included in the blockchain, which must establish which transactions up to that point are valid. I say, publicly publish the revision, and then take a "snapshot" of the balances of all mastercoin address greater than 0 as of the block height at which the revision hash transaction was first confirmed. Take a hash of that dataset, store the hash in the blockchain, and define that as proof of a transactions validity up to the point at which the first revision was confirmed.

[LOL]

I think the spec is accurate right now, but if you've found something that is out of date, we welcome pull requests

Thanks for weighing in.

I propose that if the specification is accurate then only transactions which are not invalidated (or otherwise specifically validated) by the specification are valid.

I'm going to use the same example as my previous post to demonstrate the effect of this. The specification states, "A single transaction must be constructed satisfying the following requirements:"

Has one or more 1-of-n OP_CHECKMULTISIG outputs each containing at least two compressed public keys, one of which must be the senders address

Excuse me for quoting myself, but this is what follows:

• Only a compressed key pair can broadcast valid Class B transactions.
• There is a mastercoin client that is actively broadcasting invalid Class B transactions.
• Since no mastercoin implementation recognizes a Class B transaction with a P2SH output to an uncompressed public key as invalid, there is a growing chain of invalid transactions that is publicly and unanimously identified as valid.

If the specification is accurate it also follows that, because there is no implementation that recognizes Class B transactions with P2SH outputs to uncompressed public keys as invalid, no implementation can be trusted to display the correct mastercoin balance of an address.

Further, a significant sum of bitcoin has been unknowingly traded with parties which may not even know how much mastercoin they have. Given the possibility that this party has a mastercoin balance to cover the trade, there's no guarantee that it was actually sent to the buyer.

It's really tragic for mastercoin if the spec is accurate, because the loss of bitcoin in invalid mastercoin transactions cannot be recovered, and every single wallet/implementation is useless as a method of sending, receiving, trading, or otherwise interacting with the mastercoin network.

I would suggest against maintaining that the specification is accurate.

EDIT: Concerning the invitation to submit a pull request, it's not a matter of the spec simply being outdated. If it's reasonable to consider that the specification can be out of date, the public release of an updated specification can change the validity of mastercoin transactions confirmed in the blockchain prior to the time at which the revision is published. Bitcoin sets a very high standard on the ability to verify the validity of a transaction, and there's no reason that any entity should be able to retroactively change mastercoin balances.

I recognize the invitation as a suggestion to get involved and address items I may have issues with in a structured and accepted manner, however this isn't about redefining a valid Class B transaction. It's about the ability to know an addresses mastercoin balance, to trust that an addresses mastercoin are secure, to feel safe moving or trading mastercoin, and finally to minimize the impact of data storage on the blockchain.

But for the sake of argument, if I did submit a pull request and it was merged, this would just exacerbate the issue I have with my ability to trust the specification as a document that defines mastercoin, and my ability to trust mastercoin as valuable. If the specification does not or cannot define mastercoin (and consequently the security of mastercoin that one owns), I believe it follows that mastercoin is not valuable.

EDIT 2: I expanded my first edit, and clarified a poorly worded sentence.

[LOL]

Say the protocol would adapt a more strict output policy (e.g. only a valid pub key of the sender + data-packages are allowed)

Sorry for another post.

A few different things going on here. This is in response to Class B "output policy."

First of all, it's unclear to me which Class B transactions are valid because the mastercoin implementations do not follow the requirements of the specification. Second, it doesn't matter what the policy is regarding the validity of a Class B transaction, it's only necessary that implementations have consensus, and that consensus is modeled after the specification. Third, despite the overarching importance of that which is described in my previous point, policy regarding validity of Class B transactions should be specified in a manner which is beneficial to the user or minimizes the detrimental impact on the user.

Fourth, the flexibility of Class B transactions directly affects the ability to spend a P2SH output, and the likelihood that it will be spent. Fifth, policy regarding valid Class B transactions should be based on the balance struck between the Mastercoin Foundation's willingness to recognize transactions that are, or may be, directly detrimental to the blockchain, and whatever benefit is found in flexibility. Finally, however valid Class B transactions may be defined, a wallet should always create P2SH outputs that will be spent by the wallet from which it originated.

tl;dr Flexibility is okay. Wallets that create outputs that it cannot or will not spend are not okay.

[dexX7]

I was thinking about all this a bit more. Some of this was suggested earlier:

1. Snapshots

After a specification change, a time span, a number of transactions or whatsoever the current state of the Mastercoin ecosystem should be stored in an agreed upon format. This should include all balances and may include open orders, if a snapshots is going to allow to restore a state. The correctness of a snapshot must be confirmed by several parties and finally sealed by some higher authority. The data provided by a sealed snapshot shall never be challenged and has the highest power of ruling. No user shall ever fear that a change affect the past. (note: this is always high priority and nothing new!)

2. Checkpoints

Snapshots should be explicitly used to verify an application.

3. Reference documentation

The specification is great, but I'm not yet sure how it could be used as some kind of reference for different versions of specific transaction types and similar. Commit numbers may be used, but I think in the best case there were some additional isolated and very short formal definitions which precisely define the different transactions at a given version and the changes made to previous ones. If I'd like to lookup something of the past, I'd probably need to invest some time for extracting all details from different places. Pseudo code or isolated reference implementation snippets may be used. Use-case and goal: "I want to lookup the policies applied prior snapshot 123.. ah, input is defined as.., only Exodus output allowed, otherwise invalid, 1-of-3 multisig, second and third multisig output is encoded data, ...".



Some more lower priority stuff:

4. Define how changes are pushed

Process.md describes the process of making changes to the Mastercoin specification. This document may be expanded to a more detailed "step-by-step" process guide similar to Bitcoin's release-process.md but related to sealing snapshots and pushing updates.

5. Test cases

With clear implementation guidelines it may be easier to run wide scale tests. I'm not sure, if this is done at the moment, but I'm thinking about things like "scan the chain for all transactions with unordered data-packages to get an overview about what the implications were, if an order would be enforced". (I'm not good at making up examples - ) Rephrased: if a change is pushed, evaluate all consequences. Run different settings, create test-snapshots etc.

6. Transaction archival

Instead of storing only a current state, historical transactions may be stored and sealed similar to snapshots, e.g.:

Code:

[{
  "basetx": "98419d8b3056ce50aa63d1f464555e038ae517996ac6ec13bfd8c26689f5bece",
  "input": "1NqkqJkGCA7HJkBQqJrRvJCDCLAnQndw7",
  "reference": "1MaStErt4XsYHPwfrN9TpgdURLhHTdMenH",
  "datahex": "0000000000000001000000012a05f200"
}, {
  "basetx": "cc70f686b1402d092da400f817133b07d774b3d9791564dc6efee8dbd0a54a1d",
  "input": "1Foh78nqSDzrAncvTCCqtFSzz143CsGdcz",
  "datahex": "00010014000000010000000023c346000000000005e69ec0020000000000003a9801"
}]

[LOL]

1. Snapshots

After a specification change, a time span, a number of transactions or whatsoever the current state of the Mastercoin ecosystem should be stored in an agreed upon format. This should include all balances and may include open orders, if a snapshots is going to allow to restore a state. The correctness of a snapshot must be confirmed by several parties and finally sealed by some higher authority. The data provided by a sealed snapshot shall never be challenged and has the highest power of ruling. No user shall ever fear that a change affect the past. (note: this is always high priority and nothing new!)

Can you point me to discussions had on this subject? I'm interested to know if any discussed ideas allow a snapshot to be referenced without developer involvement.
   
Ideally a system could be implemented that only requires a single snapshot at the implementation of a specification revision control system, to validate transactions occurring before that point in time. Of course, more may be necessary if a disaster situation ever occurs in which consensus is violated, or a revision has unintended consequences etc. Regardless, I'd imagine that snapshots are far less manageable than a set of rules that could define validity of transactions in prior revisions, and I fear that snapshots may pose scalability issues.

2. Checkpoints

Snapshots should be explicitly used to verify an application.

I feel like this may be challenging in terms of upkeep, and scalability.

3. Reference documentation

The specification is great, but I'm not yet sure how it could be used as some kind of reference for different versions of specific transaction types and similar. Commit numbers may be used, but I think in the best case there were some additional isolated and very short formal definitions which precisely define the different transactions at a given version and the changes made to previous ones. If I'd like to lookup something of the past, I'd probably need to invest some time for extracting all details from different places. Pseudo code or isolated reference implementation snippets may be used. Use-case and goal: "I want to lookup the policies applied prior snapshot 123.. ah, input is defined as.., only Exodus output allowed, otherwise invalid, 1-of-3 multisig, second and third multisig output is encoded data, ...".

I was thinking about something similar as I was submitting my pull request. Maybe it's not necessary to revise the entire specification if only one or two small changes are made. If different sections are broken out one can submit pull requests for items that don't warrant an entire specification revision, and changes can be discussed in a context more relevant to the issue at hand.
   
If we consider the specification a dataset instead of a specific document, it may be reasonable to precisely define a type of transaction as a data point and discrete unit, and consequently revise that definition independently of any other definition. This also allows all changes to a specific definition to be queried, and a "timeline" built of changes relevant to a specific transaction type.
   
Maybe a crazy idea, but why not use Master Protocol itself to store the specification? For the sake of opening up a discussion on the subject, consider if a transaction definition could be stored in the blockchain. Not only is it universally publicly accessible and immutable, but a standardized dictionary could theoretically allow a client to programatically and dynamically verify the validity of a transaction based on the relevant definition at the time. I have experience designing and implementing relational databases and would be thrilled to contribute if others think that this may be worth pursuing. I'm not under the impression that it's an easy task, but then again, neither was a distributed exchange.

4. Define how changes are pushed

Process.md describes the process of making changes to the Mastercoin specification. This document may be expanded to a more detailed "step-by-step" process guide similar to Bitcoin's release-process.md but related to sealing snapshots and pushing updates.

I agree. The specification feels too easy to manipulate right now, IMO. The revision procedure should detail the steps taken to ensure that the implications of a change are known, and that there are no unforeseen consequences that could compromise funds or security. This would be a great place to discuss results of test cases which you discuss below.

5. Test cases

With clear implementation guidelines it may be easier to run wide scale tests. I'm not sure, if this is done at the moment, but I'm thinking about things like "scan the chain for all transactions with unordered data-packages to get an overview about what the implications were, if an order would be enforced". (I'm not good at making up examples - ) Rephrased: if a change is pushed, evaluate all consequences. Run different settings, create test-snapshots etc.

I think there's a lot of value in a program/script that could scan the blockchain and output a list of x based on parameters y, and z. For example, transactions with outputs to the Exodus address, or more complex, transactions with outputs to Exodus address with a multisignature output that is not definitively spendable by the sending address or 2nd key pair created from associated (hex) private key. A tool of this nature would allow one to determine how "ready" various implementations are to transition to a new transaction definition, for example.
   
If a functional and usable language could be used to write a query, a tool of this nature could be revolutionary for bitcoin protocol layers, let alone for blockchain data analysis.

6. Transaction archival

Instead of storing only a current state, historical transactions may be stored and sealed similar to snapshots, e.g.:

Code:

[{
  "basetx": "98419d8b3056ce50aa63d1f464555e038ae517996ac6ec13bfd8c26689f5bece",
  "input": "1NqkqJkGCA7HJkBQqJrRvJCDCLAnQndw7",
  "reference": "1MaStErt4XsYHPwfrN9TpgdURLhHTdMenH",
  "datahex": "0000000000000001000000012a05f200"
}, {
  "basetx": "cc70f686b1402d092da400f817133b07d774b3d9791564dc6efee8dbd0a54a1d",
  "input": "1Foh78nqSDzrAncvTCCqtFSzz143CsGdcz",
  "datahex": "00010014000000010000000023c346000000000005e69ec0020000000000003a9801"
}]

IMO, this is unnecessary. I don't see any reason why a properly formed transaction definition shouldn't be able to determine validity of all applicable transactions. Archival could imply more limitations than the benefit gained by ease of gathering examples. Maybe your concern here could be appeased with the ability to prove a transaction types definition at block x?

[jakecnn]

So did something go wrong with the dev-mastercoin send? Because all bounty addresses seem to be out of consenus.

@masterchain.info: When trying to add another address to my wallet, my wallet got deleted. I think the UUID is still the same(815369f9-6558-4a30-b29e-04e413bd64f5), but no addresses are shown anymore. 'Sync with server' doesn't seem to do anything.

[Fandekasp]

Hello,

I have been recommended by Ron and Dominik to post here and suggest my contribution to the monthly bounty contest for March.

I'm one of the developers behind CoinsManager, a free and Open-source Portfolio. It's very easy to use, and sexy.
All you need is to add your public address, and it will automatically detect everything else: The related coin, the coin balance, and its value in your favorite cryptocurrency. Because Mastercoin shares the same address format as Bitcoin, if you try adding a Mastercoin address, it will ask you to choose the correct coin, here Bitcoin or Mastercoin.

Read more about the project in the press releases.

The beta version is already very usable! Here is an example of a user dashboard:




We started this project a few months ago, because it was hard for us to manage our assets (how much my coins were worth), and we wanted a place to list all our public addresses, easy place to find and copy them instead of re-opening our wallets everytime. After looking up to existing online portfolio, I was very disappointed that they required us a lot of manual input (adding the transactions, or manually adding the number of coins you possess), when it could all be automatized by reading the blockchain.


We have a lot of plans for the futures, you can look into our trello board for all the ideas/features we plan to develop (eg gamification elements). The most ambitious idea is to host our own ledgers for all implemented coins, and build a reactive api on top of those ledgers for a blazingly fast application (basically, the server will tell clients when to update their values, there won't be anymore need for the client to query the server at regular intervals to ask for changes).

Because we like free and open-source, and do not like ads, we completely rely on donations to pay for the infrastructure and support the developers, and contributions (cf our online doc and youtube screencasts).
A bounty from the Mastercoin foundation would be very welcome. You can send coins to 1CoinsMPAy5Mz5SVzeAmU6qNmUThiLXYv1 and give extra visibility to Mastercoin in our application.

Thank you for your interest and support!

[Bitoy]

this transaction is not yet confirmed in blockchain.org (0 confirmation)
https://blockchain.info/tx/4db98b7d610ed1b7170c7c8e3cd165298801e098e12314004b4d1f25676cc06f


But is already in MasterChain (and MasterChest)
https://masterchain.info/btcpayment.html?tx=4db98b7d610ed1b7170c7c8e3cd165298801e098e12314004b4d1f25676cc06f&currency=MSC

[jeroenn13]

this transaction is not yet confirmed in blockchain.org (0 confirmation)
https://blockchain.info/tx/4db98b7d610ed1b7170c7c8e3cd165298801e098e12314004b4d1f25676cc06f


But is already in MasterChain (and MasterChest)
https://masterchain.info/btcpayment.html?tx=4db98b7d610ed1b7170c7c8e3cd165298801e098e12314004b4d1f25676cc06f&currency=MSC

It seems blockchain.info is wrong, http://blockr.io/tx/info/4db98b7d610ed1b7170c7c8e3cd165298801e098e12314004b4d1f25676cc06f
TX is confirmed.

[grazcoin]

this transaction is not yet confirmed in blockchain.org (0 confirmation)
https://blockchain.info/tx/4db98b7d610ed1b7170c7c8e3cd165298801e098e12314004b4d1f25676cc06f


But is already in MasterChain (and MasterChest)
https://masterchain.info/btcpayment.html?tx=4db98b7d610ed1b7170c7c8e3cd165298801e098e12314004b4d1f25676cc06f&currency=MSC

It seems blockchain.info is wrong, http://blockr.io/tx/info/4db98b7d610ed1b7170c7c8e3cd165298801e098e12314004b4d1f25676cc06f
TX is confirmed.

The tx is indeed on the blockchain. Also on blockchain's.info, it says that the tx got included on block 291918, but the page claims zero confirmations.
I must say that in the last week or so, I see many problems on blockchain.info, and often they are few blocks behind the real state.

[dexX7]

blockchain.info is currently 6 blocks behind.

This is most likely not a reason to worry, because blockchain.info had multiple problems similar to this one in the last weeks.


@LOL: I'll appreciate your feedback very much and will respond soon!

[LOL]

Hi again.

It's clear to me that there is significantly more emphasis placed on development of features and progress towards functional goals than there is on the fundamentals of the Mastercoin system (is Master Protocol applicable here?). What I mean by this is the development of, for example, the distributed exchange and smart property has been prioritized over providing 1.) comprehensive and accurate documentation, 2.) a method of proof of transaction validity, and 3.) a structure for responsible Master Protocol implementation. I feel like this is primarily a matter of responsibility of the Mastercoin Foundation rather than any particular issue in specific. The funds held by the Mastercoin Foundation place it in an interesting position of responsibility. By creating financial incentive, the Mastercoin Foundation acts like an authority, in certain ways, regarding development and direction. The ability to create (significant) financial incentive in an open source project can have huge benefits, however also appears to carry the implication that potentially critical items, for which incentive is not provided, may not be developed at a satisfactory pace. Thus, a state of fundamental financial security in Mastercoin is deferred to an indefinite point in time because the project has been driven by development, because the specification could not be used as a specification.

This lack of preparedness is now framed by the Mastercoin infrastructure, and all bitcoin used in Mastercoin transactions. The bitcoin developers have been said to be patching the bitcoin-plane in mid flight, if you'll allow me to paraphrase Andreas Antonopoulos. I don't think the state of Mastercoin is far off from this description - in fact may be far worse considering that there is no provable method of transaction validation. The application has far outpaced the fundamentals. As the Shepard to Mastercoin development, it should be the primary goal of the Mastercoin Foundation to establish a structure from which the various implementations and features can flourish. Instead, it seems as if the development of the current implementations was sponsored at the unintentional expense of the core framework.

I've made offers to contribute significant amounts of time towards core work on a structure that defines Mastercoin in a manner that is "hard enforceable." The offers have gone unrecognized, my requests for links to discussion unanswered, and my concerns largely blown off. I recognize that I'm not talking to any single entity that represents Mastercoin, however it stands that the community has largely not responded to any of my concerns in any meaningful fashion. I maintain that I would like to work on a structure that will enable the Mastercoin specification to be "hard enforceable." This is an open offer to enter that conversation with anybody who's interested, and to coordinate with any developers/the Mastercoin Foundation if they find that it may be beneficial.

I've put together the following list to address some items that I believe are currently misrepresenting the state of the project.

• Purpose of the specification The purpose of the specification is not currently clear. There are sections that clearly identify supported/recognized functions, and sections that specify transaction types and requirements. There are also sections that are much more "fluffy," with the apparent intent of appealing to readers, or otherwise painting a picture outside the direct goal of defining the Mastercoin ecosystem. Though it may be nice to have an accessible "overview" of Mastercoin, the specification should be largely technical, with verbiage kept to a minimum so as to maintain a focus on the protocol instead of the opinions of the authors.
• State of the specification With implementation largely driven by the results of coordinated development, the specification does not currently serve the purpose of defining either the various implementations, or determining the validity of a Mastercoin transaction. Despite this, the specification is maintained to a certain degree, however the arbiter appears to be consensus. It should be clearly noted what the intent of the specification is, and what purpose the specification currently serves (i.e. documentation trailing development; not to be confused with the previous bullet).
• The term "specification" The word "specification" does not accurately describe the function of the specification. The document should be titled to fill the purpose it is defined as.
• Definition of consensus Until the specification can fill the role of a specification, the factors that determine the validity of a Mastercoin transaction should be identified and detailed. As far as I understand, this is currently consensus between various implementations. Given the decision to discontinue support for the MyMastercoins Wallet, there may soon be one less entity to participate in this consensus. It may prove valuable to define consensus (including any circumstances in which the Mastercoin Foundation does not recognize that which has achieved consensus), and detail a plan to act on in the event of loss of consensus.
• Specification vs. whitepaper Any item that is not recognized as "enabled" or will not be considered valid should be explored in a document more similar to a whitepaper. The specification should define the Mastercoin ecosystem, though that not currently being the case, should act as reference for that which is recognized as valid.
• Risk given flux in specification Holding or transacting in or with Mastercoin can pose a financial risk, due to the fluid nature of that which is considered "valid." A statement should be made or included in documentation regarding the possibility that a transaction will retroactively change states of validity. This statement should also acknowledge the fact that if validity is determined by consensus, there may be circumstances in which the Mastercoin Foundation (or any other party) may not be able to "fix" actions which represent a financial loss in either that of Bitcoin or Mastercoin.

Thoughts and input are welcome.

EDIT:

@LOL: I'll appreciate your feedback very much and will respond soon!

Thanks dexX7. I appreciate your discussion and ideas.

[DGulari]

Hi again.
It's clear to me that there is significantly more emphasis placed on development of features and progress towards functional goals than there is on the fundamentals of the Mastercoin system (is Master Protocol applicable here?).

Johnston - could you please set this guy straight?  David has all the answers for you. 

It is easy to sit around and point to all the shortcomings.  Big deal.  I can detail all the places I'd like MSC protocol to be better too.  But, with the resources they have, they are too busy building the thing. 

Imagine a Ferrari - a high performance serious piece of gear.  They are busy making the drive train balanced.  Then some dick comes up and says: "Hey guys, the paint isn't very nice!"  Dude?  What are you thinking?  "Documentation"?  Who's got time for that now?  They are all very freaking busy staying up all night to get the thing balanced.  They don't care about the paint.  That is why Masterchest is so ugly.  If you want nice paint, go to NXT.  That is all they have.