300 BTC Coding Contest: Distributed Exchange (MasterCoin Developer Thread)

[Bitoy]

What use case scenarios are we enabling here?

To date we have always applied the logic that the sender is the address with the largest input - I'm trying to see where the value is in deviating from that.

Thanks!

It is a Payment transaction
5697c55a0fb5ee61f6e9c78cb9c84cef23e11c84c68cd6e2821b8869489b808c


1BKpa19m5Xy9SvSzC5djPWtCfbuynSDwmb (0.00006 BTC - Output) n=2
1BKpa19m5Xy9SvSzC5djPWtCfbuynSDwmb (0.00006 BTC - Output) n=2
1NZC9Pk4Rt1mVe5dL8w2fShXJppyJU7Vq8 (0.00009 BTC - Output) n=1
1BKpa19m5Xy9SvSzC5djPWtCfbuynSDwmb (0.00006 BTC - Output) n=2

If its the largest input, 1NZC9Pk4Rt1mVe5dL8w2fShXJppyJU7Vq8  (0.00009) is the reference.
Is it is the largest total input, 1BKpa19m5Xy9SvSzC5djPWtCfbuynSDwmb (0.00018) is the reference.


The  sender is really 1NZC9Pk4Rt1mVe5dL8w2fShXJppyJU7Vq8.  It is payment for a purchase offer. But using the sendmany method it created the input transaction above.

[1715653995]

1715653995

[1715653995]

1715653995

[1715653995]

1715653995

[1715653995]

1715653995

[1715653995]

1715653995

[zathras]

What use case scenarios are we enabling here?

To date we have always applied the logic that the sender is the address with the largest input - I'm trying to see where the value is in deviating from that.

Thanks!

It is a Payment transaction
5697c55a0fb5ee61f6e9c78cb9c84cef23e11c84c68cd6e2821b8869489b808c


1BKpa19m5Xy9SvSzC5djPWtCfbuynSDwmb (0.00006 BTC - Output) n=2
1BKpa19m5Xy9SvSzC5djPWtCfbuynSDwmb (0.00006 BTC - Output) n=2
1NZC9Pk4Rt1mVe5dL8w2fShXJppyJU7Vq8 (0.00009 BTC - Output) n=1
1BKpa19m5Xy9SvSzC5djPWtCfbuynSDwmb (0.00006 BTC - Output) n=2

If its the largest input, 1NZC9Pk4Rt1mVe5dL8w2fShXJppyJU7Vq8  (0.00009) is the reference.
Is it is the largest total input, 1BKpa19m5Xy9SvSzC5djPWtCfbuynSDwmb (0.00018) is the reference.


The  sender is really 1NZC9Pk4Rt1mVe5dL8w2fShXJppyJU7Vq8.  It is payment for a purchase offer. But using the sendmany method it created the input transaction above.

Hey bud,

My implementation would look at this in the literal sense of the spec.  So 1NZC9Pk4Rt1mVe5dL8w2fShXJppyJU7Vq8 would be the sender since it was the largest single input.

It could be argued that this is not a mastercoin transaction (no data packets) so doesn't need to abide by the same rules, but I'm not sure we want to open that proverbial door!

Has a single or the largest input signed by the sending address.

Sendmany is not a great way to do this as we have no control over the inputs used (and would have to work around bitcoind 'account' handling (a concept internal to bitcoind/qt, not the protocol).

Would it help if I built an encodedexpayment function into the lib that handles this?

Thanks!

EDIT: Sorry, I see you already asked for an encodedexpayment-style function a few days ago - must have missed that somehow.  Yep no problems, I'll let you know when it's ready

[dacoinminster]

Pull request 6 merged:

Mastercoin messages that also have a reference output to the seller address, for instance if the buyer wants to change his offer, should not be counted towards the actual purchase of Mastercoins.

Pull request 7 merged:

## Unlocking features

Not all features described in this document are active by default. Each feature will be unlocked on a certain block once it's deemed stable. Only Test Mastercoin transactions will be allowed if a feature is not unlocked yet. All other messages will be invalidated. The only exception to this rule is the Simple Send message, this has been enabled since Exodus.

I consider both of these changes quite uncontroversial. We agreed to #6 awhile back, and it is super-obvious to me that we must take the approach outlined in #7. If this is controversial to anybody, please let me know.

[Faraday]

I don't suppose there is any chance of an update as to how long till the Distributed exchange may be up and running? I know it's nearly impossible to provide an exact date or anything, but a rough idea would be nice, days, weeks, months?

I don't mean to nag you guys, and I know it's not easy! The reason that I ask is that a new system for the current buying and selling order book is being discussed, and it would be good to have a rough idea of how much longer we may actually be using it before the guys running it spend their time setting up something a bit more advanced! Cheers  

[Tachikoma]

I hope to start testing before the end of the year. I think most implementations are nearing completion.

[vokain]

I hope to start testing before the end of the year. I think most implementations are nearing completion.

I've only been 10-15% away from full blockchain synch for the past two weeks... hopefully progress will both converge by the end of this month

[Tachikoma]

I think Zathras should comment since I can't detected why their are being invalidated on his end. Once I know the reason I can compare it to my validation checking to see which of the implementations has the correct interpretation of the spec.

[DGulari]

I think Zathras should comment since I can't detected why their are being invalidated on his end. Once I know the reason I can compare it to my validation checking to see which of the implementations has the correct interpretation of the spec.

Thanks Tachi -
I'll stand by until Zathras has a chance to look it over.  Meanwhile, keep up the great work. 

[Bitoy]

Hey bud,

My implementation would look at this in the literal sense of the spec.  So 1NZC9Pk4Rt1mVe5dL8w2fShXJppyJU7Vq8 would be the sender since it was the largest single input.

It could be argued that this is not a mastercoin transaction (no data packets) so doesn't need to abide by the same rules, but I'm not sure we want to open that proverbial door!

Has a single or the largest input signed by the sending address.

Sendmany is not a great way to do this as we have no control over the inputs used (and would have to work around bitcoind 'account' handling (a concept internal to bitcoind/qt, not the protocol).

Would it help if I built an encodedexpayment function into the lib that handles this?

Thanks!

EDIT: Sorry, I see you already asked for an encodedexpayment-style function a few days ago - must have missed that somehow.  Yep no problems, I'll let you know when it's ready

Zathras,

My code is missing "the largest single input is the reference".   I will update it and reparse everything.   

I'm also trying to build encodedexpayment so that i can remove the "sendmany" from the wallet.

Thanks again!

 

[zathras]

Hey bud,

My implementation would look at this in the literal sense of the spec.  So 1NZC9Pk4Rt1mVe5dL8w2fShXJppyJU7Vq8 would be the sender since it was the largest single input.

It could be argued that this is not a mastercoin transaction (no data packets) so doesn't need to abide by the same rules, but I'm not sure we want to open that proverbial door!

Has a single or the largest input signed by the sending address.

Sendmany is not a great way to do this as we have no control over the inputs used (and would have to work around bitcoind 'account' handling (a concept internal to bitcoind/qt, not the protocol).

Would it help if I built an encodedexpayment function into the lib that handles this?

Thanks!

EDIT: Sorry, I see you already asked for an encodedexpayment-style function a few days ago - must have missed that somehow.  Yep no problems, I'll let you know when it's ready

Zathras,

My code is missing "the largest single input is the reference".   I will update it and reparse everything.    

I'm also trying to build encodedexpayment so that i can remove the "sendmany" from the wallet.

Thanks again!

No worries  I'm currently working on putting encodedexpayment into the library - it may take a day or two to translate to the library though as I'm currently doing this directly in my wallet - the other library encode functions are a poor base for this (those functions take a shortcut because it's only a tiny fraction of a bitcoin and just look for a single input to cover the fees (to be precise an input >=0.0005) - encodedexpayment is a different beast (hence why I originally did it wallet-side) & needs to handle this differently - ie be capable of putting together multiple inputs to build enough to cover the payment).

Thanks!

[zathras]

I think Zathras should comment since I can't detected why their are being invalidated on his end. Once I know the reason I can compare it to my validation checking to see which of the implementations has the correct interpretation of the spec.

Thanks Tachi -
I'll stand by until Zathras has a chance to look it over.  Meanwhile, keep up the great work. 

Looking into this now.

[zathras]

I think Zathras should comment since I can't detected why their are being invalidated on his end. Once I know the reason I can compare it to my validation checking to see which of the implementations has the correct interpretation of the spec.

Thanks Tachi -
I'll stand by until Zathras has a chance to look it over.  Meanwhile, keep up the great work.  

Looking into this now.

OK transactions:
f0862dda475d17383e7c3c63ad5a015e8dfd77d627ca45ec2e1ba2579c29e0f9
8175fcd0221ee5cda1808f48861bef5c2d5e8b9acf91a7b0fade5768d35f6665

Are not considered Mastercoin transactions by my implementation.

This is because my implementation enforces a literal interpretation of the spec, being that only one additional output is allowed for change in Class A.  Both those transactions are attemping Class A but have two additional change outputs.

Technically speaking I can allow those through, but I'm trying to take myself out of the equation as I really don't think it's right for an individual developer to decide whether someone gets their money or not.  This is why I'm very literal with the spec.

We can perhaps do a pull that changes the spec for Class A to allow multiple change addresses as follows:

An additional output is permitted for the remainder of the input (the 'change' address)

to

Multiple additional outputs are permitted for the remainder of the input (the 'change' addresses)

But before saying yay/nay to that I'd want to do some due diligence to see if that has any other impact I'm not seeing at first glance (ie like all of a sudden validating other past transactions that may previously have been seen as invalid and thus resent etc).

Thanks!

[Bitoy]

OK transactions:
f0862dda475d17383e7c3c63ad5a015e8dfd77d627ca45ec2e1ba2579c29e0f9
8175fcd0221ee5cda1808f48861bef5c2d5e8b9acf91a7b0fade5768d35f6665

Are not considered Mastercoin transactions by my implementation.

This is because my implementation enforces a literal interpretation of the spec, being that only one additional output is allowed for change in Class A.  Both those transactions are attemping Class A but have two additional change outputs.

Technically speaking I can allow those through, but I'm trying to take myself out of the equation as I really don't think it's right for an individual developer to decide whether someone gets their money or not.  This is why I'm very literal with the spec.

We can perhaps do a pull that changes the spec for Class A to allow multiple change addresses as follows:

An additional output is permitted for the remainder of the input (the 'change' address)

to

Multiple additional outputs are permitted for the remainder of the input (the 'change' addresses)

But before saying yay/nay to that I'd want to do some due diligence to see if that has any other impact I'm not seeing at first glance (ie like all of a sudden validating other past transactions that may previously have been seen as invalid and thus resent etc).

Thanks!

Isn't the transaction covered by "peek and decode"?

[zathras]

OK transactions:
f0862dda475d17383e7c3c63ad5a015e8dfd77d627ca45ec2e1ba2579c29e0f9
8175fcd0221ee5cda1808f48861bef5c2d5e8b9acf91a7b0fade5768d35f6665

Are not considered Mastercoin transactions by my implementation.

This is because my implementation enforces a literal interpretation of the spec, being that only one additional output is allowed for change in Class A.  Both those transactions are attemping Class A but have two additional change outputs.

Technically speaking I can allow those through, but I'm trying to take myself out of the equation as I really don't think it's right for an individual developer to decide whether someone gets their money or not.  This is why I'm very literal with the spec.

We can perhaps do a pull that changes the spec for Class A to allow multiple change addresses as follows:

An additional output is permitted for the remainder of the input (the 'change' address)

to

Multiple additional outputs are permitted for the remainder of the input (the 'change' addresses)

But before saying yay/nay to that I'd want to do some due diligence to see if that has any other impact I'm not seeing at first glance (ie like all of a sudden validating other past transactions that may previously have been seen as invalid and thus resent etc).

Thanks!

Isn't the transaction covered by "peek and decode"?

Potentially - peek and decode is there to avoid ambiguity following rules for sequence numbers (eg to prevent users who used MastercoinAdviser to send from having a random chance of a sequence number collision (6 in 255 if change amount = other output amounts)).

Considering the wording, it's open to interpretation I think.  "Following the above rules" was intended to mean the rules covering sequence numbers and outputs being the same (ie the rules under "The following conditions must also be satisfied for the transaction to be considered decode-able:").  

It wasn't intended to cover the general rules (eg stating an output to exodus is required and an output for change is permitted etc) as if we applied that last resort peek & decode statement to all rules then technically even a Mastercoin transaction without an output to Exodus would be valid as long as it could be decoded via peek & decode.

But I do agree it's open to interpretation, so we should agree consensus between us all and I'll submit a pull to remove any ambiguity.

What do you guys think?

Thanks!

EDIT: TL:DR for an abundance of clarity as this won't be clear for those not familiar with the spec.  

Long story short, peek and decode was not intended to allow the rules in the preceding paragraph to be optional.  I do agree for the need for a clarification, whichever way we go however.  Hope this helps make my particular perspective a bit clearer.  

To allow multiple change outputs (still not ready to comment on whether we should allow this) we would either:
     a) Change the wording to state multiple change outputs are allowed as I mentioned a couple of posts back, or
     b) Move the change output statement into the next paragraph where peek & decode is applicable

From the spec, I've highlighted in navy what my intention was with regard to the purple-highlighted text:

The transaction data is encoded into said fake Bitcoin address which is then used as an output in a single Bitcoin transaction satisfying the following requirements:

• Has a single or the largest input signed by the sending address
• Has an output for the recipient address (the 'reference' address)
• Has an output for the exodus address
• Has an output for the encoded fake address (the 'data' address)
• Has all output values equal & above the 'dust' threshold (currently 0.00005430 BTC)
• An additional output is permitted for the remainder of the input (the 'change' address)

The following conditions must also be satisfied for the transaction to be considered decode-able:

• The reference address sequence number must be the data address sequence number + 1
• Ideally, all outputs should be the same (except the change). In fringe cases where the change output value is equal to the other output values the change address can be identified by sequence number, which must not equal or be +/-1 of the sequence number for either the reference address or the data address
• A last resort 'peek and decode' method may be used to identify the data packet in the event of ambiguity following the above rules. This involves decoding each packet and looking for the correct bytes for a simple send (the majority of bytes in a Class A simple send do not change). These byte checks are defined as:
  
  • Bytes two to eight must equal 00
  • Byte nine must equal 01 or 02
• Should there still be packet ambiguity or 'peek and decode' reveals more than one packet (simple sends are always one packet) the transaction is considered invalid.

[Bitoy]

I see. In order for the transaction to work

Instead of

An additional output is permitted for the remainder of the input (the 'change' address)

To

Additional outputs are permitted for the remainder of the input (the 'change' address)


Good work on the strict implementation.  However the "peek and decode" gave "class A" a lot of flexibility.  If we are for flexibility then allowing  multiple change output should also be included.

[zathras]

I see. In order for the transaction to work

Instead of

An additional output is permitted for the remainder of the input (the 'change' address)

To

Additional outputs are permitted for the remainder of the input (the 'change' address)


Good work on the strict implementation.  However the "peek and decode" gave "class A" a lot of flexibility.  If we are for flexibility then allowing  multiple change output should also be included.

Agreed and I'm supportive of peek and decode for anything that isn't a core rule - I know my pedantic nature seems harsh sometimes but I think in the end the community will want us to follow the spec to the letter in our implementations - that's why the spec is evolving so we can change things by consensus if they're too strict

For clarity I'm not necessarily against multiple change outputs, I just haven't done enough investigation to recommend either way at the mo.

[DGulari]

But before saying yay/nay to that I'd want to do some due diligence to see if that has any other impact I'm not seeing at first glance (ie like all of a sudden validating other past transactions that may previously have been seen as invalid and thus resent etc).

Thanks for looking into this.  It is good to get clarification at an early date.  These unusual deviations all need to be cleared up to prevent the chaos which comes without good concordance between systems. 

Will you let me know (post here) when a decision is made?  I don't want to trouble you with inquiries as I am anxious for you to finish the DEx like everyone else.  But, I also don't want this to be dropped without finality.  Anything I need to do?  Thanks again.

[Tachikoma]

I am swamped at the moment but I will try to do some thinking on the whole multiple change addresses output and try to discover what kind of impact that will have on other transactions. We can't take this lightly, I do want to ask the owners of those addresses to freeze their coins if at all possible.
We should try to see what is the best for the protocol regardless of if somebody already considered this transaction valid, how painful that might be. To be continued.

Edit: I think the reason this worked for me was indeed the peek & decode code. I use it as fallback and it managed to find the correct details for this transaction since the output that are non-change are correct. I believe changing the spec to your suggestion would be safe since it would only allow more flexibility without invalidating current transactions.

Zathras, would you do the honors for the pull request, it was your suggestion after all

[grazcoin]

I have started a thread for the purpose of syncing the different mastercoin implementations:

https://bitcointalk.org/index.php?topic=362883.msg3882308

This is great that we have multiple implementations for the mastercoin protocol, but they should all agree with each other.
I think the sync action between the different implementations deserves its own thread.

Currently I compare:

• https://masterchain.info/mastercoin_verify/addresses/0
• http://mastercoin-explorer.com/mastercoin_verify/addresses

and generate a diff using:
https://github.com/grazcoin/mastercoin-tools/commit/0fe108a0e234441d533414efc6ef4e7e4d1ac90d

The diff is generated every round hour (since it seems like a heavy task on mastercoin-explorer.com):
https://masterchain.info/general/difference.json

Let's discuss the differences and try to get a sync.

Grazcoin

This way, we don't pollute the contest thread.

[zathras]

I am swamped at the moment but I will try to do some thinking on the whole multiple change addresses output and try to discover what kind of impact that will have on other transactions. We can't take this lightly, I do want to ask the owners of those addresses to freeze their coins if at all possible.
We should try to see what is the best for the protocol regardless of if somebody already considered this transaction valid, how painful that might be. To be continued.

Edit: I think the reason this worked for me was indeed the peek & decode code. I use it as fallback and it managed to find the correct details for this transaction since the output that are non-change are correct. I believe changing the spec to your suggestion would be safe since it would only allow more flexibility without invalidating current transactions.

Zathras, would you do the honors for the pull request, it was your suggestion after all

I'm going to make the code change in Dev, export balances before and after then diff - as long as it's just the address in question (ie doesn't affect any other transactions) then I'll look at doing a pull.  I do want to think this through a little more but from what I've looked at so far I haven't found any showstoppers to allowing multiple change outputs