300 BTC Coding Contest: Distributed Exchange (MasterCoin Developer Thread)

[dacoinminster]

I use OpenSSL to check ECDSA validity. I believe Zathras has an implementation you could probably use since he is also using something Microsoft(y) to create his code, excuse my ignorance I'm not sure what language you are using Zathras.

Zathras is using Microsoft  .net.   Zathras, can I  borrow your  ecdsa validity check and multi sig sending module

I'll speak to this: feel free to use anybody's code. Just please be clear that you are doing so and don't try to take credit for it when payout time comes. If other people build on Zathras' code, the contest rules state that his payout gets bigger (how much it gets bigger is subjective, based on our collective feelings about how much his code sharing helped achieve the contest goals)

Redundancy is helpful too - if you write your own version of the code, the differences will help find bugs. The payouts in the last contest reflected this, with four people getting a share of the pot with a lot of overlap between their work.

I agree with vokain about the work you guys are doing - I couldn't be happier.

[1713997205]

1713997205

[1713997205]

1713997205

[1713997205]

1713997205

[1713997205]

1713997205

[jadair10]

This thread is devoted to the current MasterCoin contest (this thread was originally for contest #1).

JR,

Can you add a link in the OP to where the discussion begins for *this* contest so people can jump right to the new discussion? https://bitcointalk.org/index.php?topic=292628.msg3381794#msg3381794

[dacoinminster]

JR,

Can you add a link in the OP to where the discussion begins for *this* contest so people can jump right to the new discussion? https://bitcointalk.org/index.php?topic=292628.msg3381794#msg3381794

Good idea. Done!

[grazcoin]

I was trying to collect all the great ideas from you all, and implement them in mastercoin-tools send and parse.
It includes:
1. using compressed pubkeys.
2. using only valid ecdsa points.
3. obfuscating the data.
4. ignore all previous multisig experiments

All code available on the head on https://github.com/grazcoin/mastercoin-tools

The first transaction was generated with debug on:

Code:

$ python msc_send.py -m multisig -c 1 -a 0.12345678 -x 0.0001 -r 17RVTF3vJzsuaGh7a94DFkg4msJ7FcBYgX -f 182osbPxCo88oaSX4ReJwUr9uAcchmJVaL -k -d
[I] main: Using settings: {'broadcast': False, 'recipient_address': '17RVTF3vJzsuaGh7a94DFkg4msJ7FcBYgX', 'fee': '0.0001', 'from_address': '182osbPxCo88oaSX4ReJwUr9uAcchmJVaL', 'key_prompt': True, 'host_port': None, 'currency_id': 1, 'amount': '0.12345678', 'debug_mode': True, 'priv_key': None, 'tx_method': 'multisig'}
Enter your private key:
[I] main: Private key was entered
[D] main: plain dataHex: --0100000000000000010000000000bc614e0000000000000000000000000000--
[D] main: obfus dataHex: 021bf733f7aab3932560cd8e8a3ec11b45ee47f0694a0b61c86ab48e63bba57cd4
[I] get_nearby_valid_pubkey: trying 021bf733f7aab3932560cd8e8a3ec11b45ee47f0694a0b61c86ab48e63bba57cd4
[I] get_nearby_valid_pubkey: trying 021bf733f7aab3932560cd8e8a3ec11b45ee47f0694a0b61c86ab48e63bba57cd5
[I] get_nearby_valid_pubkey: trying 021bf733f7aab3932560cd8e8a3ec11b45ee47f0694a0b61c86ab48e63bba57cd6
[I] get_nearby_valid_pubkey: valid  021bf733f7aab3932560cd8e8a3ec11b45ee47f0694a0b61c86ab48e63bba57cd7
[D] main: valid dataHex: 021bf733f7aab3932560cd8e8a3ec11b45ee47f0694a0b61c86ab48e63bba57cd7
[D] main: change address is 182osbPxCo88oaSX4ReJwUr9uAcchmJVaL
[D] main: receipent is 17RVTF3vJzsuaGh7a94DFkg4msJ7FcBYgX
[D] main: total inputs value is 34000
[D] main: fee is 10000
[D] main: dust limit is 5430
[D] main: BIP11 script is 1 [ 031f204911ec19cb5b7b10dd87ccf6a52552466d14356212e881288512eeff8e20 ] [ 021bf733f7aab3932560cd8e8a3ec11b45ee47f0694a0b61c86ab48e63bba57cd7 ] 2 checkmultisig
Added input 886710238086cdc020d2e74c9c1773648beb9c55aac9c154d4ca5b8b9fedccd3:1
Added output sending 5430 Satoshis to 1EXoDusjGwvnjZUyKkxZ4UHEf77z6A5S4P.
Added output sending 5430 Satoshis to 17RVTF3vJzsuaGh7a94DFkg4msJ7FcBYgX.
Added output sending 10860 Satoshis to 1 [ 031f204911ec19cb5b7b10dd87ccf6a52552466d14356212e881288512eeff8e20 ] [ 021bf733f7aab3932560cd8e8a3ec11b45ee47f0694a0b61c86ab48e63bba57cd7 ] 2 checkmultisig.
[D] main: inputs_outputs are /dev/stdout -i 886710238086cdc020d2e74c9c1773648beb9c55aac9c154d4ca5b8b9fedccd3:1 -o 1EXoDusjGwvnjZUyKkxZ4UHEf77z6A5S4P:5430 -o 17RVTF3vJzsuaGh7a94DFkg4msJ7FcBYgX:5430 -o 5121031f204911ec19cb5b7b10dd87ccf6a52552466d14356212e881288512eeff8e2021021bf733f7aab3932560cd8e8a3ec11b45ee47f0694a0b61c86ab48e63bba57cd752ae:10860
[D] main: parsed tx is {'inputs': [{'previous_output': '886710238086cdc020d2e74c9c1773648beb9c55aac9c154d4ca5b8b9fedccd3:1', 'sequence': 4294967295, 'address': None, 'script': ''}], 'locktime': 0, 'version': 1, 'hash': '797ace6b846b99c2182ca4ab31734de4ae96fa180b25d870e8b813cd66687fd6', 'outputs': [{'address': '1EXoDusjGwvnjZUyKkxZ4UHEf77z6A5S4P', 'value': 5430, 'script': 'dup hash160 [ 946cb2e08075bcbaf157e47bcb67eb2b2339d242 ] equalverify checksig'}, {'address': '17RVTF3vJzsuaGh7a94DFkg4msJ7FcBYgX', 'value': 5430, 'script': 'dup hash160 [ 46727d1b3d6847f9ed344561a315f54b801edf63 ] equalverify checksig'}, {'address': None, 'value': 10860, 'script': '1 [ 031f204911ec19cb5b7b10dd87ccf6a52552466d14356212e881288512eeff8e20 ] [ 021bf733f7aab3932560cd8e8a3ec11b45ee47f0694a0b61c86ab48e63bba57cd7 ] 2 checkmultisig'}]}
[I] sign: signing tx
[I] main: validating tx: Status: Success
[I] main: SIGNED tx (multisig) of 0.12345678 Mastercoin to 17RVTF3vJzsuaGh7a94DFkg4msJ7FcBYgX signed by 182osbPxCo88oaSX4ReJwUr9uAcchmJVaL
0100000001d3cced9f8b5bcad454c1c9aa559ceb8b6473179c4ce7d220c0cd868023106788010000008b483045022100b42b3facdafdd9033e5572e745dbc2704efb985b0bc0002d2629b00b90d42d6e02205fede2b0a26e3f296dc0c223341176426ca958a0b09be17b8d51c6c4f39fcb210141041f204911ec19cb5b7b10dd87ccf6a52552466d14356212e881288512eeff8e2084ddff9997fdfb22fae6b09a255e3937a7890491ab5106ce7912bc253e430887ffffffff0336150000000000001976a914946cb2e08075bcbaf157e47bcb67eb2b2339d24288ac36150000000000001976a91446727d1b3d6847f9ed344561a315f54b801edf6388ac6c2a000000000000475121031f204911ec19cb5b7b10dd87ccf6a52552466d14356212e881288512eeff8e2021021bf733f7aab3932560cd8e8a3ec11b45ee47f0694a0b61c86ab48e63bba57cd752ae00000000
[I] parse_test: {'tx_hash': 'unknown', 'tx_type_str': 'Simple send', 'from_address': '182osbPxCo88oaSX4ReJwUr9uAcchmJVaL', 'currencyId': '00000001', 'padding': '000000', 'tx_method_str': 'multisig', 'amount': '0000000000bc614e', 'currency_str': 'Mastercoin', 'formatted_amount': '0.12345678', 'to_address': '17RVTF3vJzsuaGh7a94DFkg4msJ7FcBYgX', 'baseCoin': '00', 'dataSequenceNum': '01', 'transactionType': '00000000'}
[I] main: please send using "sx broadcast-tx signed_tx.tx"

A commentary for to the debug log:

• The padded dataHex is 0100000000000000010000000000bc614e0000000000000000000000000000
• After obfuscation (using sha256 of the string '17RVTF3vJzsuaGh7a94DFkg4msJ7FcBYgX') and adding 02 at the beginning and a random tail, it becomes 021bf733f7aab3932560cd8e8a3ec11b45ee47f0694a0b61c86ab48e63bba57cd4
• Within 4 iterations of searching for a valid pubkey (each time adding 1), a valid one is found.
• A transaction is created and signed.
• A parsing test shows the same values.

On blockchain.info, the transaction looks this way. Note that the input used for the transaction was too small to include also change (the change was less than the dust limit), so all the under-dust change got added to the fee. The fee then increased to 0.0001228 instead of the requested 0.0001.

[Tachikoma]

Cool, glad we are getting up-to-date! I still need to rewrite mastercoin-explorer with the new obfuscation code. So much to do, so little time

I will take a few minutes to verify your key.

Edit: Confirmed, looks good!

[zathras]

Posting this here too as I think it's a critical issue we need to clear up as it affects a transaction in the wild:

I've checked it out and fixed it, your transaction shows up on mastercoin-explorer now.

It will probably show up on Masterchest as well since I can't find anything wrong with it.

Please understand that we are currently working hard on a new encoding standard which requires us to reparse the data a lot, during these times our sites might not miss some transactions or perhaps not display them correctly. This is all very cutting edge software so things might go wonky from time to time.

Sorry guys have to be quick - back to back meetings today.

Long story short according to the current rules that transaction is not valid due to ambiguous sequence numbers, this is why masterchest.info throws the transaction out.

19b5BiXWZERFoCNhVKiYDf9i829P1W1wiE has a sequence number of 94
19UjqqjXmQxyyn4xStA7mWXVkzXwVDgu7Z has a sequence number of 93
19feAR37pguLwDyEMc8oiW2WT4esrgR5z6 has a sequence number of 95

If there is a broken sequence (i.e. 3,4,8), then the odd-man-out is the change address (8 in this example)
If there is an ambiguous sequence (i.e. 3,4,4), then the transaction is invalid!
If there is a perfect sequence (i.e. 3,4,5), then the transaction is invalid!

You have a perfect sequence (93, 94, 95) so there is no clear way to identify recipient vs change.  

Tachikoma, we (or just I?) removed the requirement for the outputs to be the same and only used sequence numbers for address identification instead.  This was discussed a while back when you were designing your original multisig and had to use a different amount for the multisig output due to the higher dust threshold with a bigger size of the output (multisig).  I raised the issue of outputs amounts no longer being the same as required by the spec and suggested an amendment that multisig output amounts were specified as exodus output amount*2, but that never got included as it was stated that outputs actually did not need to be the same, it was just a convenience:

Having all the outputs be the same amount is merely a convenience for identifying the change address. I'm fine with a less strict implementation as long as it is still possible to identify the change address.

So I removed the requirement for outputs to be the same amount to allow for multisig outputs no longer meeting said requirement.  So as it stands now I no longer evaluate what the value is of each vout, only the sequence numbers matter.

I think we need some further discussion on backwards compatibility for these Class A transactions.  Perhaps we need to re-introduce the 'same output amount' rule as a requirement just for Class A transactions as we can then use the amounts to help identify change as per the original spec - at the moment there seems to be some ambiguity around Class A transaction validity between our implementations and we need to clear this up.  You can probably understand now why I'm putting so much effort into having these rules explicitly defined and documented  I do concur that random chance of sequence number collisions should not be a factor in transaction validity.

None of this is an issue in multisig as we require change to be from the sender as a method of removing address ambiguity.

Could I please suggest the parties involved in this transaction just hold for a little, while the development team have a discussion around Class A transaction validity.

Thanks!

EDIT: for clarity

[zathras]

JR,

Can you add a link in the OP to where the discussion begins for *this* contest so people can jump right to the new discussion? https://bitcointalk.org/index.php?topic=292628.msg3381794#msg3381794

Good idea. Done!

Is the contest actually running now btw?  FYI I'm (and think Tachikoma is also) still spending my time to properly achieve the goals of the last contest (storing data in the blockchain) as I don't feel that's actually finished yet (though we're close).

Thanks

[zathras]

I use OpenSSL to check ECDSA validity. I believe Zathras has an implementation you could probably use since he is also using something Microsoft(y) to create his code, excuse my ignorance I'm not sure what language you are using Zathras.

Zathras is using Microsoft  .net.   Zathras, can I  borrow your  ecdsa validity check and multi sig sending module

.NET yep.  The most recent changes haven't made their way into the masterchest library yet as they're in a separate codebase (you can still use the encodetx function to build the cleartext mastercoin packet but that function also builds the entire transaction, not just the packet so you'll have to pull the relevant data out of the final raw transaction hex).  I'll have a modified version up with new functions for encoding/decoding reflecting the suggested amendments over the weekend.

For ECDSA validity (I'll also put a check function into the masterchest library) I took my inspiration from the casascius bitcoin address utility - have a read through some of the pubkey and ECDSA stuff to point you in the right direction.

Thanks!

[Bitoy]

I was able to decode the 1st packet

Code:

Data to Parse: 02d52c390e46f1110410078a9db1482ba4cf924666fb1b41689be9cc2e2ecde3e5
OBFUSCATED MASTERCOIN PACKET: d52c390e46f1110410078a9db1482ba4cf924666fb1b41689be9cc2e2ecde3
SHA256 HASH: d42c390e52f1110412078a9db148e7a306924666fb10aaaa9bffcc2e2ecde344
REFERENCE ADDRESS: 1J2svn2GxYx9LPrpCLFikmzn9kkrXBrk8B
CLEARTEXT MASTERCOIN PACKET: 0100000014000000020000000000cc07c9000000000bebc200160000000000

Clear text looks ok


01: 1 (01)
Trans Type: 20 (00000014)
Currency ID: 2 (00000002)
Amount for Sale: 13371337 (0000000000cc07c9)
BTC Desired:  200000000 (000000000bebc200)
Time Limit:  1 (1)

But have a problem on the 2nd

Code:

Data to Parse: 026c17b960d1aa810b6f736760a03166dec0ecc617de661915e06981d5d88f28b5
OBFUSCATED MASTERCOIN PACKET: 6c17b960d1aa810b6f736760a03166dec0ecc617de661915e06981d5d88f28
SHA256 HASH: 4266395eef8a3a62fb74ed5ff4d6201573fd51318fce9eaf452eecb3ab9a8ba9
REFERENCE ADDRESS: 1J2svn2GxYx9LPrpCLFikmzn9kkrXBrk8B
CLEARTEXT MASTERCOIN PACKET: 2e71803e3e20bb6994078a3f54e746cbb311972651a887baa5476d667315a3

Should the 2nd packet be 2x sha?

Ok! With all this new found knowledge let's try decoding this 'Selling Mastercoins for Bitcoins' (SMFB from now on) message again.

Code:

XOR Reference: 1J2svn2GxYx9LPrpCLFikmzn9kkrXBrk8B
Clear text Mastercoin message: <REMOVED FOR NOW :) >
Result: 02d52c390e46f1110410078a9db1482ba4cf924666fb1b41689be9cc2e2ecde3e5, 026c17b960d1aa810b6f736760a03166dec0ecc617de661915e06981d5d88f28b5

Some notes. When I SHA the reference I only take the first 62 bytes since this is the exact amount we need for the obfuscation. This will change the the SHA of the next iteration of hashes that follow so I'm open to discuss this.

Once we agree on the output of these keys I think it's safe to try and broadcast a message

Edit: Updated keys since I forgot to increment the amount of hashes.

[Bitoy]

Hi Grazcoin

You are the first to post to the the blockchain using the new multi sig.  Nice work!   ( I was able to parse your transaction  =)   

Data to Parse: 021bf733f7aab3932560cd8e8a3ec11b45ee47f0694a0b61c86ab48e63bba57cd4
OBFUSCATED MASTERCOIN PACKET: 1bf733f7aab3932560cd8e8a3ec11b45ee47f0694a0b61c86ab48e63bba57c
SHA256 HASH: 1af733f7aab3932561cd8e8a3ec1a724a047f0694a0b61c86ab48e63bba57cd0
REFERENCE ADDRESS: 17RVTF3vJzsuaGh7a94DFkg4msJ7FcBYgX
CLEARTEXT MASTERCOIN PACKET: 0100000000000000010000000000bc614e0000000000000000000000000000

01: 1 (01)
Trans Type: 0 (00000000)
Currency ID: 1 (00000001)
Amount for Sale: 12345678 (0000000000bc614e)

• The padded dataHex is 0100000000000000010000000000bc614e0000000000000000000000000000
• After obfuscation (using sha256 of the string '17RVTF3vJzsuaGh7a94DFkg4msJ7FcBYgX') and adding 02 at the beginning and a random tail, it becomes 021bf733f7aab3932560cd8e8a3ec11b45ee47f0694a0b61c86ab48e63bba57cd4
• Within 4 iterations of searching for a valid pubkey (each time adding 1), a valid one is found.
• A transaction is created and signed.
• A parsing test shows the same values.

On blockchain.info, the transaction looks this way. Note that the input used for the transaction was too small to include also change (the change was less than the dust limit), so all the under-dust change got added to the fee. The fee then increased to 0.0001228 instead of the requested 0.0001.

[zathras]

Some notes. When I SHA the reference I only take the first 62 bytes since this is the exact amount we need for the obfuscation. This will change the the SHA of the next iteration of hashes that follow so I'm open to discuss this.

Wasn't quite sure what you meant by this so I thought I'd just note that there is no need to drop or add bytes to/from the inputs of our SHA256 hashing if that's what you mean?  SHA256 hashing will always produce a 256 bit (32 byte) hash regardless of input length.  To clarify my take on things:

With each packet:
   * For sequence number 1 we SHA256 the entire length of the address (which could be anywhere from 27 to 34 bytes), result = 32 byte hash.  
   * For sequence numbers 2 onwards, we take the previous 32 byte hash and SHA256 it again (and again), result = 32 byte hash.  
   * We then take the resulting 32 byte hash, grab the first 31 bytes and XOR with the cleartext Mastercoin packet.
Rinse & repeat.

Perhaps that's what you meant, sorry if I'm getting confused or repeating stuff - there's been so much thought & discussion on this stuff it's all kind of a blur!

So for your address of 1J2svn2GxYx9LPrpCLFikmzn9kkrXBrk8B, the first 5 packets should have hashes of (in between { } is what you would XOR with):

Code:

SEQNUM=1   {D42C390E52F1110412078A9DB148E7A306924666FB10AAAA9BFFCC2E2ECDE3}44
SEQNUM=2   {000EC2C68806819E67A030E82A6AF98376DAC1065D7FE533DAF251D43AA836}3B
SEQNUM=3   {999722F745CC7EA5559D871285A697513D6D1F69294A472AB71499C280CFDA}72
SEQNUM=4   {23C4AC723733621964260EC4639D9DF3469E983E677B083457F325C6F56FA5}D0
SEQNUM=5   {A2989BBA3E4BF3B2995A8573E19450381C94CDE10F95A157756148217B0E37}1B

Thoughts?

We haven't discussed what we will use to XOR data for a 'Selling MasterCoins for Bitcoins' package. I want to propose using the sending address whenever a Mastercoin message does not contain a recipient address.

Agreed.  Though I actually think we should make it the sender address for everything because as you note, not all transactions will have a reference address.  We may as well stick with an address we know will always be there.  Unless you guys know of a reason for not using the sender address let's lock that in as our initial source for the SHA256 hashing & I'll update the amendment accordingly.

EDIT: for clarity

[Tachikoma]

Sorry guys have to be quick - back to back meetings today.

Long story short according to the current rules that transaction is not valid due to ambiguous sequence numbers, this is why masterchest.info throws the transaction out.

19b5BiXWZERFoCNhVKiYDf9i829P1W1wiE has a sequence number of 94
19UjqqjXmQxyyn4xStA7mWXVkzXwVDgu7Z has a sequence number of 93
19feAR37pguLwDyEMc8oiW2WT4esrgR5z6 has a sequence number of 95

If there is a broken sequence (i.e. 3,4,8), then the odd-man-out is the change address (8 in this example)
If there is an ambiguous sequence (i.e. 3,4,4), then the transaction is invalid!
If there is a perfect sequence (i.e. 3,4,5), then the transaction is invalid!

You have a perfect sequence (93, 94, 95) so there is no clear way to identify recipient vs change.  

Tachikoma, we (or just I?) removed the requirement for the outputs to be the same and only used sequence numbers for address identification instead.  This was discussed a while back when you were designing your original multisig and had to use a different amount for the multisig output due to the higher dust threshold with a bigger size of the output (multisig).  I raised the issue of outputs amounts no longer being the same as required by the spec and suggested an amendment that multisig output amounts were specified as exodus output amount*2, but that never got included as it was stated that outputs actually did not need to be the same, it was just a convenience:

Having all the outputs be the same amount is merely a convenience for identifying the change address. I'm fine with a less strict implementation as long as it is still possible to identify the change address.

So I removed the requirement for outputs to be the same amount to allow for multisig outputs no longer meeting said requirement.  So as it stands now I no longer evaluate what the value is of each vout, only the sequence numbers matter.

I think we need some further discussion on backwards compatibility for these Class A transactions.  Perhaps we need to re-introduce the 'same output amount' rule as a requirement just for Class A transactions as we can then use the amounts to help identify change as per the original spec - at the moment there seems to be some ambiguity around Class A transaction validity between our implementations and we need to clear this up.  You can probably understand now why I'm putting so much effort into having these rules explicitly defined and documented  I do concur that random chance of sequence number collisions should not be a factor in transaction validity.

None of this is an issue in multisig as we require change to be from the sender as a method of removing address ambiguity.

This is why I am not a fan of flagging a transaction invalid based sequence. This guy was just trying to create a transaction and he had no way that random chance could make it invalid.

As long as we can safely identify the data address then a perfect sequence number is not a problem. I say we simply peak into each address and see if it contains a known Mastercoin message. Currently only SimpleSends are supported so we can just peak into an address, decode it and see if the transaction type is 0 and the and currency_id is within limits and if the sequence number makes sense. If we can say with a certain certainty that this is most likely a simple send then we have our data address and know which sequence the target address should be. This will work for a broken sequence as well. The only problem is the ambiguous sequence. In this case there is simply no way of finding out the correct address and the transaction should be made invalid. Since the random factor can't be solved here we might need to solve it differently and go back a step.

How about we do the following.

• Only allow Simple Sends to be encoded as addresses. All new messages should use pulic keys.
• All outputs that contain Mastercoin data for Class A transaction should have the same output amount, but it doesn't matter how much this amount is.
• Probe each address to see if it's a Mastercoin encoded address using the checks outlined above.

Checks to see if it's a Mastercoin encoded address

• Transaction type is 0
• Currency ID is an existing Currency ID, for now only 1 and 2 are created but this might chance in the future

If we follow these rules then there should always be three outputs. One you can rule out based on the fact that it's Exodus. One of those is the data package and one the target address. In most cases you can probably know which is which even without the sequence number.

I will make some time today to see if this change would affect any existing transactions but I highly doubt it. 

[Tachikoma]

Some notes. When I SHA the reference I only take the first 62 bytes since this is the exact amount we need for the obfuscation. This will change the the SHA of the next iteration of hashes that follow so I'm open to discuss this.

Wasn't quite sure what you meant by this so I thought I'd just note that there is no need to drop or add bytes to/from the inputs of our SHA256 hashing if that's what you mean?  SHA256 hashing will always produce a 256 bit (32 byte) hash regardless of input length.  To clarify my take on things:

With each packet:
   * For sequence number 1 we SHA256 the entire length of the address (which could be anywhere from 27 to 34 bytes), result = 32 byte hash.  
   * For sequence numbers 2 onwards, we take the previous 32 byte hash and SHA256 it again (and again), result = 32 byte hash.  
   * We then take the resulting 32 byte hash, grab the first 31 bytes and XOR with the cleartext Mastercoin packet.
Rinse & repeat.

Perhaps that's what you meant, sorry if I'm getting confused or repeating stuff - there's been so much thought & discussion on this stuff it's all kind of a blur!

So for your address of 1J2svn2GxYx9LPrpCLFikmzn9kkrXBrk8B, the first 5 packets should have hashes of (in between { } is what you would XOR with):

Code:

SEQNUM=1   {D42C390E52F1110412078A9DB148E7A306924666FB10AAAA9BFFCC2E2ECDE3}44
SEQNUM=2   {000EC2C68806819E67A030E82A6AF98376DAC1065D7FE533DAF251D43AA836}3B
SEQNUM=3   {999722F745CC7EA5559D871285A697513D6D1F69294A472AB71499C280CFDA}72
SEQNUM=4   {23C4AC723733621964260EC4639D9DF3469E983E677B083457F325C6F56FA5}D0
SEQNUM=5   {A2989BBA3E4BF3B2995A8573E19450381C94CDE10F95A157756148217B0E37}1B

Thoughts?

You were right about using the bytes, I coded that up wrong. It was late last night (and it's early now..

I'm still not getting the same sequence as you are however. I think we might be doing this differently.

Code:

d42c390e52f1110412078a9db148e7a306924666fb10aaaa9bffcc2e2ecde344
370ee8c285babbf857761796c0ab5c652db7da17fdd2fea8657809ca8428bd2f
e3ab016c159270d4649ab732d41c7895ad3e05f5b94d611f7a5e19780c6502d2
...etc...

Just so I know we are doing the same thing.

Code:

sequence1 = Digest::SHA256.hexdigest("1J2svn2GxYx9LPrpCLFikmzn9kkrXBrk8B")
sequence2 = Digest::SHA256.hexdigest(sequence1)
sequence3 = Digest::SHA256.hexdigest(sequence2)
..etc..

We haven't discussed what we will use to XOR data for a 'Selling MasterCoins for Bitcoins' package. I want to propose using the sending address whenever a Mastercoin message does not contain a recipient address.

Agreed.  Though I actually think we should make it the sender address for everything because as you note, not all transactions will have a reference address.  We may as well stick with an address we know will always be there.  Unless you guys know of a reason for not using the sender address let's lock that in as our initial source for the SHA256 hashing & I'll update the amendment accordingly.

Agreed, let's just use the sending address for all packages. This should go into the spec.

[zathras]

Sorry guys have to be quick - back to back meetings today.

Long story short according to the current rules that transaction is not valid due to ambiguous sequence numbers, this is why masterchest.info throws the transaction out.

19b5BiXWZERFoCNhVKiYDf9i829P1W1wiE has a sequence number of 94
19UjqqjXmQxyyn4xStA7mWXVkzXwVDgu7Z has a sequence number of 93
19feAR37pguLwDyEMc8oiW2WT4esrgR5z6 has a sequence number of 95

If there is a broken sequence (i.e. 3,4,8), then the odd-man-out is the change address (8 in this example)
If there is an ambiguous sequence (i.e. 3,4,4), then the transaction is invalid!
If there is a perfect sequence (i.e. 3,4,5), then the transaction is invalid!

You have a perfect sequence (93, 94, 95) so there is no clear way to identify recipient vs change.  

Tachikoma, we (or just I?) removed the requirement for the outputs to be the same and only used sequence numbers for address identification instead.  This was discussed a while back when you were designing your original multisig and had to use a different amount for the multisig output due to the higher dust threshold with a bigger size of the output (multisig).  I raised the issue of outputs amounts no longer being the same as required by the spec and suggested an amendment that multisig output amounts were specified as exodus output amount*2, but that never got included as it was stated that outputs actually did not need to be the same, it was just a convenience:

Having all the outputs be the same amount is merely a convenience for identifying the change address. I'm fine with a less strict implementation as long as it is still possible to identify the change address.

So I removed the requirement for outputs to be the same amount to allow for multisig outputs no longer meeting said requirement.  So as it stands now I no longer evaluate what the value is of each vout, only the sequence numbers matter.

I think we need some further discussion on backwards compatibility for these Class A transactions.  Perhaps we need to re-introduce the 'same output amount' rule as a requirement just for Class A transactions as we can then use the amounts to help identify change as per the original spec - at the moment there seems to be some ambiguity around Class A transaction validity between our implementations and we need to clear this up.  You can probably understand now why I'm putting so much effort into having these rules explicitly defined and documented  I do concur that random chance of sequence number collisions should not be a factor in transaction validity.

None of this is an issue in multisig as we require change to be from the sender as a method of removing address ambiguity.

This is why I am not a fan of flagging a transaction invalid based sequence. This guy was just trying to create a transaction and he had no way that random chance could make it invalid.

Completely agree the guy was not at fault at all, just unlucky.  As rough as it sounds though it has to be the rules of the protocol that define transaction validity regardless of whether it's right or wrong from a particular point of view.  I'm with definitely on board that this should be a valid transaction though and I'm confident if we present the right set of changes to include fringe cases that JR will be open to adopting them and dropping things like the perfect sequence invalidation (which would then make this transaction valid).

As long as we can safely identify the data address then a perfect sequence number is not a problem. I say we simply peak into each address and see if it contains a known Mastercoin message. Currently only SimpleSends are supported so we can just peak into an address, decode it and see if the transaction type is 0 and the and currency_id is within limits and if the sequence number makes sense. If we can say with a certain certainty that this is most likely a simple send then we have our data address and know which sequence the target address should be. This will work for a broken sequence as well. The only problem is the ambiguous sequence. In this case there is simply no way of finding out the correct address and the transaction should be made invalid. Since the random factor can't be solved here we might need to solve it differently and go back a step.

How about we do the following.

• Only allow Simple Sends to be encoded as addresses. All new messages should use pulic keys.
• All outputs that contain Mastercoin data for Class A transaction should have the same output amount, but it doesn't matter how much this amount is.
• Probe each address to see if it's a Mastercoin encoded address using the checks outlined above.

Checks to see if it's a Mastercoin encoded address

• Transaction type is 0
• Currency ID is an existing Currency ID, for now only 1 and 2 are created but this might chance in the future

If we follow these rules then there should always be three outputs. One you can rule out based on the fact that it's Exodus. One of those is the data package and one the target address. In most cases you can probably know which is which even without the sequence number.

I will make some time today to see if this change would affect any existing transactions but I highly doubt it. 

I definitely support only allowing simple sends, I have this in the amendment:

NOTE: Class A transactions are restricted to the ‘simple send’ transaction type only. All other Mastercoin transaction types are supported by Class B transactions only. Client implementations should utilize Class B for all transaction types, including ‘simple send’.

Also agree with re-instating the requirement for outputs to be for the same amount, I actually really liked this   This requirement would only be for Class A transactions as you note.

If someone gets really unlucky and creates a transaction where the change amount equals the rest of the output, and then also gets ambiguous sequence numbers, we could use the decode & peek method - I'm just trying to consider ways this could be abused but I can't think of any (since we would throw the transaction out if we found 2 decode-able packets in a Class A transaction).

[zathras]

After reading a lot of old posts I think I know where we are.

That quote

If there is a broken sequence (i.e. 3,4,8), then the odd-man-out is the change address (8 in this example)
If there is an ambiguous sequence (i.e. 3,4,4), then the transaction is invalid!
If there is a perfect sequence (i.e. 3,4,5), then the transaction is invalid!

Is from a larger quote which begins:

...

• All protocol transactions should have the same output amount. If one output is different, that is the change address.
• If all outputs are the same, then look at sequence numbers:
  • If there is a broken sequence (i.e. 3,4,8), then the odd-man-out is the change address (8 in this example)
  • If there is an ambiguous sequence (i.e. 3,4,4), then the transaction is invalid!
  • If there is a perfect sequence (i.e. 3,4,5), then the transaction is invalid!

...

It was originally stated that all outputs should be for the same amount & the odd one out was change.  Then we needed bigger outputs and larger fees, so that requirement was made more of a convenience and we started to use outputs with different amounts, making checking for the odd one out as change no longer reliable and change was identified with sequence numbers instead. 

It's all in the interpretation - just because the 'odd one out is change' method was no longer reliable as the outputs no longer had to be the same amount, that doesn't mean I can't use it to decode and validate a transaction in some cases.  In other words, my implementation is being too strict/literal.  I thus consider this transaction valid and will amend my code to reflect this

I definitely agree we should make the outputs be the same amount a requirement for Class A.  If you guys & JR adopt the amendment we'll now have (logically) two classes of transaction, so we can make this really clear for others since we can apply rules to one class but not the other.  We can make matching output values a requirement for Class A ('original') transactions but not Class B ('multisig'). We can apply the change-goes-to-sender rule to Class B and not Class A.  Add peek & decode to Class A etc.  Helps me keep things clear anyway

[Bitoy]

We haven't discussed what we will use to XOR data for a 'Selling MasterCoins for Bitcoins' package. I want to propose using the sending address whenever a Mastercoin message does not contain a recipient address.

Agreed.  Though I actually think we should make it the sender address for everything because as you note, not all transactions will have a reference address.  We may as well stick with an address we know will always be there.  Unless you guys know of a reason for not using the sender address let's lock that in as our initial source for the SHA256 hashing & I'll update the amendment accordingly.

Agreed, let's just use the sending address for all packages. This should go into the spec.

Excellent, using the sender address for all transactions will simplify the coding (and lessen the bugs).

[zathras]

I've updated the amendment to include the changes we've discussed.  If viewing in browser you may need to refresh.

Anything we're missing, anything unclear?

Thanks!

[Tachikoma]

I've updated the amendment to include the changes we've discussed.  If viewing in browser you may need to refresh.

Anything we're missing, anything unclear?

Thanks!

Should we perhaps specify the rules what 'peek and decode' means? Other than that it looks good

[zathras]

You were right about using the bytes, I coded that up wrong. It was late last night (and it's early now..

I'm still not getting the same sequence as you are however. I think we might be doing this differently.

Code:

d42c390e52f1110412078a9db148e7a306924666fb10aaaa9bffcc2e2ecde344
370ee8c285babbf857761796c0ab5c652db7da17fdd2fea8657809ca8428bd2f
e3ab016c159270d4649ab732d41c7895ad3e05f5b94d611f7a5e19780c6502d2
...etc...

Just so I know we are doing the same thing.

Code:

sequence1 = Digest::SHA256.hexdigest("1J2svn2GxYx9LPrpCLFikmzn9kkrXBrk8B")
sequence2 = Digest::SHA256.hexdigest(sequence1)
sequence3 = Digest::SHA256.hexdigest(sequence2)
..etc..

Haha that took a little figuring out - you're hashing lowercase hex, I'm hashing uppercase hex

[Tachikoma]

hahahahaahaha, duh. Is one way better then the other, or the default?