300 BTC Coding Contest: Distributed Exchange (MasterCoin Developer Thread)

[zathras]

Brings up another interesting question that needs to be answered - can a buyer have multiple 'accept offers' to the same seller open at the same time?  There (iirc) is nothing in the spec that says you can, or that you can't.

I actually haven't coded for that scenario at all, I'll have a look through your test case to see how Masterchest has handled it.  At first thought I think it adds unncessesary complexity (allowing multiple accepts from the same buyer at the same time - for example we then need to explicitly define in what order/how bitcoin payments are applied to which accept) - but I haven't looked at it deeply.

J.R. - can we get a ruling?  Multiple 'accept offers' from the same buyer to the same seller open simultaneously.  Yay or nay?

FYI:
* Graz (assuming from the above) allows multiple accept offers from the same buyer, unsure how applies payments
* Bitoy allows multiple accept offers from the same buyer, and applies payment to the first accept until paid, then to the next accept until paid and so on.  But only for whole payments, so if the first payment did not fully pay the first accept, the second payment would be applied to the first accept and any remaining would be an overpayment and not applied to the next accept offer.
* Masterchest hasn't factored for multiple accept offers one way or the other - will need to evaluate behaviour

Ooooo. Interesting. You're right - I never considered this situation.

I think the most intuitive handling of two offers from the same buyer to the same seller is to append them. That is, if the buyer offered to purchase 2 MSC, then 2 more MSC, then their total amount they can buy is actually 4 MSC.

I don't think the UI needs to necessarily offer the user the ability to do this, but I don't think it makes sense to prevent it at the protocol level either. Zathras, I hope when you say it would be a big change for you, that you mean it would be a big change to allow this in the UI. I don't think you need to do that as long as Masterchest parses it correctly.

We go live with real MSC in 5 days! I think/hope this is the last major issue to iron out?

Actually, unfortunately this is not simply a UI issue for me.  It goes to the core of the logic I currently apply.  Let's say there is an accept in block 3 for 50 MSC.  I then search block 3 to block 3+timelimit for payments to the seller.  For example:
1) Accept 50 MSC for 5 BTC in block 3
2) Check blocks 3 to 9 for payment, yep found 5 BTC payment to seller in block 8
3) Accept 50 MSC for 5 BTC in block 6
4) Check blocks 6 to 12 for payment, yep found 5 BTC payment to seller in block 8

Oops, same payment has now been applied to two accepts.  So as we can see logic was written with the design assumption of a single accept per address per sell.

Also we can't simply merge the two accepts, what happens if the seller updated the sell with a different price in between the two accepts?  They have to be handled as seperate entities and thus we have to define an order of precedent for applying payments and so on.

I'll put my thinking hat on and go review my code logic, but there will be some complexities to allowing this.

[dacoinminster]

Actually, unfortunately this is not simply a UI issue for me.  It goes to the core of the logic I currently apply.  Let's say there is an accept in block 3 for 50 MSC.  I then search block 3 to block 3+timelimit for payments to the seller.  For example:
1) Accept 50 MSC for 5 BTC in block 3
2) Check blocks 3 to 9 for payment, yep found 5 BTC payment to seller in block 8
3) Accept 50 MSC for 5 BTC in block 6
4) Check blocks 6 to 12 for payment, yep found 5 BTC payment to seller in block 8

Oops, same payment has now been applied to two accepts.  So as we can see logic was written with the design assumption of a single accept per address per sell.

Also we can't simply merge the two accepts, what happens if the seller updated the sell with a different price in between the two accepts?  They have to be handled as seperate entities and thus we have to define an order of precedent for applying payments and so on.

I'll put my thinking hat on and go review my code logic, but there will be some complexities to allowing this.

Ugh. Good point. I'm leaning towards just one accept allowed per unique buyer/seller at a time, because it is more complicated than I at first thought, and I like simplicity whenever I can get it.

Bitoy and Grazcoin, how bad does that sound to you?

[grazcoin]

Actually, unfortunately this is not simply a UI issue for me.  It goes to the core of the logic I currently apply.  Let's say there is an accept in block 3 for 50 MSC.  I then search block 3 to block 3+timelimit for payments to the seller.  For example:
1) Accept 50 MSC for 5 BTC in block 3
2) Check blocks 3 to 9 for payment, yep found 5 BTC payment to seller in block 8
3) Accept 50 MSC for 5 BTC in block 6
4) Check blocks 6 to 12 for payment, yep found 5 BTC payment to seller in block 8

Oops, same payment has now been applied to two accepts.  So as we can see logic was written with the design assumption of a single accept per address per sell.

Also we can't simply merge the two accepts, what happens if the seller updated the sell with a different price in between the two accepts?  They have to be handled as seperate entities and thus we have to define an order of precedent for applying payments and so on.

I'll put my thinking hat on and go review my code logic, but there will be some complexities to allowing this.

Ugh. Good point. I'm leaning towards just one accept allowed per unique buyer/seller at a time, because it is more complicated than I at first thought, and I like simplicity whenever I can get it.

Bitoy and Grazcoin, how bad does that sound to you?

We have 5 days before we go live, and originally we wanted 10-14 days of consensus before that. We will not have that, but we really can't wait for the last moment.
Two implementations have consensus on the current code base now, and if we make another change, we may lose that.

I was asking to drop the whole multiple payments saying that it contradicts the basic design of my code, and it hides some complexities that we didn't think of, but you guys insisted.
It was 2 full working days with a lot of testing, and a huge patch.

At the current stage, I prefer either to keep the consensus without limiting user freedom for multiple accepts and ask Zathras to join the consensus, or roll back the whole multiple payments feature.

[dexX7]

Here is one more thing to discuss, I think.

1. Flawed public key compression:

There are a few transactions which use an uncompressed public key as input and then compress the key, but use a prefix of "02" where "03" would be used. Those transaction outputs are not spendabe by the user nor do they represent the initial address in any way, if base58 encoded.

Example:

Code:

Input: uncompressed public key
04e6da9c60084b43d28266243c636bcdaf4d8f17b5954e078d2dece7d4659e0dee3419a40b939c24ac813c692a323ca5207a6fb387ffe28e48f706c95dbf46648f

Base58 encoded:
13NRX88EZbS5q81x6XFrTECzrciPREo821

Output: compressed with "02" instead of "03":
1JzhWo3tuReu8rsbhVPrBLxxGTcve6aaD

18 transactions (+ 3 invalid) are affected:

Code:

Transaction id                                                   Original input address
5885b1aa8938f234da483a5190084c3c425ab03363810e68b0214d358a8144c6 1MCHESTptvd2LnNp7wmr2sGTpRomteAkq8 INVALID
5fe5e0a2c7a4c9ae92708631b0559c32ebad33aa65d12d9d13b23869a1a68cbf 1MCHESTptvd2LnNp7wmr2sGTpRomteAkq8 INVALID
4842af86dac35a2c294f56511c7d08362330e82bd5fcae32c07119844e019b0a 14hm8rTdknVCDpqXGY5nFqVWruU9UBeuHd (mine)
ba63aa51cbf233a63db56b2dfb731c4f609b728125d1154fbc5453ec841c07cc 1MCHESTxYkPSLoJ57WBQot7vz3xkNahkcb
3b08b5219e9dc8474b54d9d22fcdc7ccc03e916de313d306a5135ff607480cb6 1MCHESTxYkPSLoJ57WBQot7vz3xkNahkcb INVALID
bc12d67c9403128a7effd9ff1da30dbcb3a47bc82207b2e93ea62de7ae91a4c9 1MCHESTptvd2LnNp7wmr2sGTpRomteAkq8
e36099be9139bfe6a3a0a3f2b29a83b4cfe0c56d4fe58fd4e672a4bc9d4adae4 1MCHESTptvd2LnNp7wmr2sGTpRomteAkq8
f6bb04e866fcf7a267f1af5659a7350a87b2c132ee1253757846c5be0b455a42 1MCHESTxYkPSLoJ57WBQot7vz3xkNahkcb
8a8c2e7fe253fe8635d34ecc71510955d3a344546410db6961b35cd51809d3ea 1MCHESTptvd2LnNp7wmr2sGTpRomteAkq8
f91f80606f96c0b0e5e30f2929e04a099b92c2dd26971180165d45d9b06a363e 1MCHESTxYkPSLoJ57WBQot7vz3xkNahkcb
eef12413271bdda3446aac2209f26f01c35add0a18fcffce143cf298225a2375 1MCHESTptvd2LnNp7wmr2sGTpRomteAkq8
f1897e2fa5b495b3e5dfd29fb30de63725d58e925ea7e6ef250066b5abde1bb8 1MCHESTptvd2LnNp7wmr2sGTpRomteAkq8
5058febb5a048e021114534728a1aff7907b504ac47406067de81cd8384d8a07 1MCHESTptvd2LnNp7wmr2sGTpRomteAkq8
e412453bde7abb42b3610beff5fad0fc3db3c4317a503914ab1e452e5fc326a5 13NRX88EZbS5q81x6XFrTECzrciPREo821
f8f166210911b28ef1cd720b57a94b43cd551203116c3abed84490d14200e191 13NRX88EZbS5q81x6XFrTECzrciPREo821
a3e4b7728835008b141c5f813e76416bd0876a50197bacafec8b039380ed36f1 13NRX88EZbS5q81x6XFrTECzrciPREo821
a47ddb72ebed729257bf29ca4907381551f6eb772b913f3895e5697673addeeb 13NRX88EZbS5q81x6XFrTECzrciPREo821
f8e53b52fa18e478fca931946bdf388ffcbecd9b053db3672f5289696e552739 13NRX88EZbS5q81x6XFrTECzrciPREo821
78735dfd07c20789a249063ab18b43516c5ac744c1b79612f9e5895773e707c4 13NRX88EZbS5q81x6XFrTECzrciPREo821
708d30405a58e1be0301e9475d0c9a23a9090d85e1fdf52808f90a8fe72f8a49 13NRX88EZbS5q81x6XFrTECzrciPREo821
8b0c75191564a937c4996702c425f9399c8d70742b44e4d9e306016d87bbc708 13NRX88EZbS5q81x6XFrTECzrciPREo821

2. No input reference in the outputs:

There are 10 transactions (+ 3 invalid) with an output with a public key that is not the input and where the output is no data packet:

Code:

Transaction id                                                   Original input address
4805d3fba156e622cfdf47b12e68380b181cc96e9e6ef21076d9bc7e6704e336 14hm8rTdknVCDpqXGY5nFqVWruU9UBeuHd (mine)
55791fff99bbf40b8aeb5d0aac40e15bf0120bc5cce4b06c00c6626329c90fed 1donutMH4L7kdRqh4xvSvesfd3KFu3UNm
98314ebb0d96a98b5b52c891c8aac982e77420429450661b078a795051389d75 1donutMH4L7kdRqh4xvSvesfd3KFu3UNm
6784b73048de3ab36c08ee06897be92ccb20f9b288345f96b0681cbc5b014ad4 1gYs2kC58J7aRBRJ7CYqysBpYoT7b5i8H INVALID according to Masterchain
a7120ed05e1b42f48fec59d0a3b56202d15a0bb8272653f995ac584f16e67c09 14Q2NNiF5YzDZCjo7vrBuHdbZcmHRJyW3j
f4d5e7a5af93b5091a2652719943c3a19f56cf6933d071a87f2bf87cba9a7f0a 14Q2NNiF5YzDZCjo7vrBuHdbZcmHRJyW3j
31620c7e58f31a1833808f52eb240e4c4510994765f4d269bb2740ecfa136f9a 14Q2NNiF5YzDZCjo7vrBuHdbZcmHRJyW3j
2ae17011888e9a2e6c752999774cd6d88feb7c0751abe72a602be89858e44f6a 14Q2NNiF5YzDZCjo7vrBuHdbZcmHRJyW3j
b9e6b1f6f428441d7e91e7715ed4524de942b8d755367828e1d85ca850a9dc43 14Q2NNiF5YzDZCjo7vrBuHdbZcmHRJyW3j
8c9027bfbee656c591ec87d233f0e338e623452519fbf8d76961e1afa480949c 14Q2NNiF5YzDZCjo7vrBuHdbZcmHRJyW3j INVALID according to Masterchain
38c37aec242e1054a366bbbe79fac2b01c392fe6c9c1732d25f54f0dbcc317dd 14Q2NNiF5YzDZCjo7vrBuHdbZcmHRJyW3j INVALID according to Masterchain
100e35e3fba087407a67e69ac60436f2b1f097823d083464ad26940d9fa06f6e 1Pa6zyqnhL6LDJtrkCMi9XmEDNHJ23ffEr (mine)
e308a5c9f7f035e3c1ed1abafdda85d9b790dbf8243f90484a9b25d6a7f8c8e7 1Pa6zyqnhL6LDJtrkCMi9XmEDNHJ23ffEr (mine)

The spec says:

- Has a single or the largest input signed by the sending address.
- Has one or more 1-of-n OP_CHECKMULTISIG outputs each containing at least two compressed public keys, one of which must be the senders address.

The above mentioned transactions are in conflict with those requirements.

There are multiple transactions with more than one input, but in almost all cases the same input address is used. There are currently 5 simple send transactions with more than one different input addresses. Not a single valid multi sig transaction.

I personally dislike the allowence of different inputs (many with the same pubKey are fine), because such transactions requires to fetch all input transactions to extract the value to determine the highest one which would otherwise be not necessary.

I have no solution how data packages can be safely identified and ordered and no implementation is currently able to parse a transaction with unordered data packages.

But if a given order is strictly required it would make the whole concept of sequence numbers more or less obsolete. I don't want to believe this though.

Here is my take:

1. Class A transactions are seperated and can be treated completey different. It's obsolete anyway, so workarounds or inefficiencies are not an issue.
2. I suggest to disallow multiple inputs with different public keys. This does not invalidate any transaction up until now, it is much more painless and furthermore the identification of the input reference is solely possible via the input itself. Including the input reference as part of a multisig still makes a lot of sense to redeem the output, but that's not mutually exclusive.
3. Fixing the prefix issue of compressed public keys is advised.
4. It would be very, very, very awesome, if there is a) a way to distinguish between data packages and non-data packages and b) if this is also possible, if data packages are not ordered. If this is the case, then including the pubKey as output would not require a special note nor would any malformated or additional pubKey in the outputs raise a conflict.

So the questions:

What do you think about dropping the support for multiple inputs with different pubKeys (= different wallets) for class B transactions?

How should multi sig outputs be treated which are neither a valid input reference (e.g. due to flawed compression, compression or because it's simply junk) nor a valid data package?

Currently all this mostly affects transactions that were issued by devs and those conflicts did not yet lead to notable problems, but once it gets real, there might be no turning back anymore.

That's my biggest "fear".

[dacoinminster]

We have 5 days before we go live, and originally we wanted 10-14 days of consensus before that. We will not have that, but we really can't wait for the last moment.
Two implementations have consensus on the current code base now, and if we make another change, we may lose that.

I was asking to drop the whole multiple payments saying that it contradicts the basic design of my code, and it hides some complexities that we didn't think of, but you guys insisted.
It was 2 full working days with a lot of testing, and a huge patch.

At the current stage, I prefer either to keep the consensus without limiting user freedom for multiple accepts and ask Zathras to join the consensus, or roll back the whole multiple payments feature.

Ah. I didn't realize you two were in consensus already on that. That does help. I don't particularly like any option here, but we have to take the path of least resistance at this point.

I had hoped that it would be trivial to invalidate additional accept offers after the first. Is that not the case?

Bitoy, I hope you'll weigh in on this too.

Zathras, I know it sucks, but we may need to change yours if it is a large change for the other two implementations to disable multiple accepts.

[zathras]

Actually, unfortunately this is not simply a UI issue for me.  It goes to the core of the logic I currently apply.  Let's say there is an accept in block 3 for 50 MSC.  I then search block 3 to block 3+timelimit for payments to the seller.  For example:
1) Accept 50 MSC for 5 BTC in block 3
2) Check blocks 3 to 9 for payment, yep found 5 BTC payment to seller in block 8
3) Accept 50 MSC for 5 BTC in block 6
4) Check blocks 6 to 12 for payment, yep found 5 BTC payment to seller in block 8

Oops, same payment has now been applied to two accepts.  So as we can see logic was written with the design assumption of a single accept per address per sell.

Also we can't simply merge the two accepts, what happens if the seller updated the sell with a different price in between the two accepts?  They have to be handled as seperate entities and thus we have to define an order of precedent for applying payments and so on.

I'll put my thinking hat on and go review my code logic, but there will be some complexities to allowing this.

Ugh. Good point. I'm leaning towards just one accept allowed per unique buyer/seller at a time, because it is more complicated than I at first thought, and I like simplicity whenever I can get it.

Bitoy and Grazcoin, how bad does that sound to you?

We have 5 days before we go live, and originally we wanted 10-14 days of consensus before that. We will not have that, but we really can't wait for the last moment.
Two implementations have consensus on the current code base now, and if we make another change, we may lose that.

I was asking to drop the whole multiple payments saying that it contradicts the basic design of my code, and it hides some complexities that we didn't think of, but you guys insisted.
It was 2 full working days with a lot of testing, and a huge patch.

At the current stage, I prefer either to keep the consensus without limiting user freedom for multiple accepts and ask Zathras to join the consensus, or roll back the whole multiple payments feature.

Graz,

Don't want to undo any work you've done mate, but multiple payments != multiple accepts from the same buyer.

I'm not saying we should invalidate multiple payments, just multiple accepts from the same buyer simultaneously.  A buyer could accept 100 MSC and send 5 bitcoin payments to cover the buy - that's still OK.  A buyer could not accept 100 MSC and then accept another 100 MSC in a second offer (possibly at a different price), then make a bunch of payments.  Not without something in the spec that defines (explicitly) how bitcoin payments would be applied to these accepts (in what order, whether overpayments are rolled to another accept offer and so on).

What payment logic for multiple accepts are you using now if I may ask?

Hope that makes more sense

Thanks
Zathras

[zathras]

Here is one more thing to discuss, I think.

1. Flawed public key compression:

There are a few transactions which use an uncompressed public key as input and then compress the key, but use a prefix of "02" where "03" would be used. Those transaction outputs are not spendabe by the user nor do they represent the initial address in any way, if base58 encoded.

Example:

Code:

Input: uncompressed public key
04e6da9c60084b43d28266243c636bcdaf4d8f17b5954e078d2dece7d4659e0dee3419a40b939c24ac813c692a323ca5207a6fb387ffe28e48f706c95dbf46648f

Base58 encoded:
13NRX88EZbS5q81x6XFrTECzrciPREo821

Output: compressed with "02" instead of "03":
1JzhWo3tuReu8rsbhVPrBLxxGTcve6aaD

18 transactions (+ 3 invalid) are affected:

Code:

Transaction id                                                   Original input address
5885b1aa8938f234da483a5190084c3c425ab03363810e68b0214d358a8144c6 1MCHESTptvd2LnNp7wmr2sGTpRomteAkq8 INVALID
5fe5e0a2c7a4c9ae92708631b0559c32ebad33aa65d12d9d13b23869a1a68cbf 1MCHESTptvd2LnNp7wmr2sGTpRomteAkq8 INVALID
4842af86dac35a2c294f56511c7d08362330e82bd5fcae32c07119844e019b0a 14hm8rTdknVCDpqXGY5nFqVWruU9UBeuHd (mine)
ba63aa51cbf233a63db56b2dfb731c4f609b728125d1154fbc5453ec841c07cc 1MCHESTxYkPSLoJ57WBQot7vz3xkNahkcb
3b08b5219e9dc8474b54d9d22fcdc7ccc03e916de313d306a5135ff607480cb6 1MCHESTxYkPSLoJ57WBQot7vz3xkNahkcb INVALID
bc12d67c9403128a7effd9ff1da30dbcb3a47bc82207b2e93ea62de7ae91a4c9 1MCHESTptvd2LnNp7wmr2sGTpRomteAkq8
e36099be9139bfe6a3a0a3f2b29a83b4cfe0c56d4fe58fd4e672a4bc9d4adae4 1MCHESTptvd2LnNp7wmr2sGTpRomteAkq8
f6bb04e866fcf7a267f1af5659a7350a87b2c132ee1253757846c5be0b455a42 1MCHESTxYkPSLoJ57WBQot7vz3xkNahkcb
8a8c2e7fe253fe8635d34ecc71510955d3a344546410db6961b35cd51809d3ea 1MCHESTptvd2LnNp7wmr2sGTpRomteAkq8
f91f80606f96c0b0e5e30f2929e04a099b92c2dd26971180165d45d9b06a363e 1MCHESTxYkPSLoJ57WBQot7vz3xkNahkcb
eef12413271bdda3446aac2209f26f01c35add0a18fcffce143cf298225a2375 1MCHESTptvd2LnNp7wmr2sGTpRomteAkq8
f1897e2fa5b495b3e5dfd29fb30de63725d58e925ea7e6ef250066b5abde1bb8 1MCHESTptvd2LnNp7wmr2sGTpRomteAkq8
5058febb5a048e021114534728a1aff7907b504ac47406067de81cd8384d8a07 1MCHESTptvd2LnNp7wmr2sGTpRomteAkq8
e412453bde7abb42b3610beff5fad0fc3db3c4317a503914ab1e452e5fc326a5 13NRX88EZbS5q81x6XFrTECzrciPREo821
f8f166210911b28ef1cd720b57a94b43cd551203116c3abed84490d14200e191 13NRX88EZbS5q81x6XFrTECzrciPREo821
a3e4b7728835008b141c5f813e76416bd0876a50197bacafec8b039380ed36f1 13NRX88EZbS5q81x6XFrTECzrciPREo821
a47ddb72ebed729257bf29ca4907381551f6eb772b913f3895e5697673addeeb 13NRX88EZbS5q81x6XFrTECzrciPREo821
f8e53b52fa18e478fca931946bdf388ffcbecd9b053db3672f5289696e552739 13NRX88EZbS5q81x6XFrTECzrciPREo821
78735dfd07c20789a249063ab18b43516c5ac744c1b79612f9e5895773e707c4 13NRX88EZbS5q81x6XFrTECzrciPREo821
708d30405a58e1be0301e9475d0c9a23a9090d85e1fdf52808f90a8fe72f8a49 13NRX88EZbS5q81x6XFrTECzrciPREo821
8b0c75191564a937c4996702c425f9399c8d70742b44e4d9e306016d87bbc708 13NRX88EZbS5q81x6XFrTECzrciPREo821

2. No input reference in the outputs:

There are 10 transactions (+ 3 invalid) with an output with a public key that is not the input and where the output is no data packet:

Code:

Transaction id                                                   Original input address
4805d3fba156e622cfdf47b12e68380b181cc96e9e6ef21076d9bc7e6704e336 14hm8rTdknVCDpqXGY5nFqVWruU9UBeuHd (mine)
55791fff99bbf40b8aeb5d0aac40e15bf0120bc5cce4b06c00c6626329c90fed 1donutMH4L7kdRqh4xvSvesfd3KFu3UNm
98314ebb0d96a98b5b52c891c8aac982e77420429450661b078a795051389d75 1donutMH4L7kdRqh4xvSvesfd3KFu3UNm
6784b73048de3ab36c08ee06897be92ccb20f9b288345f96b0681cbc5b014ad4 1gYs2kC58J7aRBRJ7CYqysBpYoT7b5i8H INVALID according to Masterchain
a7120ed05e1b42f48fec59d0a3b56202d15a0bb8272653f995ac584f16e67c09 14Q2NNiF5YzDZCjo7vrBuHdbZcmHRJyW3j
f4d5e7a5af93b5091a2652719943c3a19f56cf6933d071a87f2bf87cba9a7f0a 14Q2NNiF5YzDZCjo7vrBuHdbZcmHRJyW3j
31620c7e58f31a1833808f52eb240e4c4510994765f4d269bb2740ecfa136f9a 14Q2NNiF5YzDZCjo7vrBuHdbZcmHRJyW3j
2ae17011888e9a2e6c752999774cd6d88feb7c0751abe72a602be89858e44f6a 14Q2NNiF5YzDZCjo7vrBuHdbZcmHRJyW3j
b9e6b1f6f428441d7e91e7715ed4524de942b8d755367828e1d85ca850a9dc43 14Q2NNiF5YzDZCjo7vrBuHdbZcmHRJyW3j
8c9027bfbee656c591ec87d233f0e338e623452519fbf8d76961e1afa480949c 14Q2NNiF5YzDZCjo7vrBuHdbZcmHRJyW3j INVALID according to Masterchain
38c37aec242e1054a366bbbe79fac2b01c392fe6c9c1732d25f54f0dbcc317dd 14Q2NNiF5YzDZCjo7vrBuHdbZcmHRJyW3j INVALID according to Masterchain
100e35e3fba087407a67e69ac60436f2b1f097823d083464ad26940d9fa06f6e 1Pa6zyqnhL6LDJtrkCMi9XmEDNHJ23ffEr (mine)
e308a5c9f7f035e3c1ed1abafdda85d9b790dbf8243f90484a9b25d6a7f8c8e7 1Pa6zyqnhL6LDJtrkCMi9XmEDNHJ23ffEr (mine)

The spec says:

- Has a single or the largest input signed by the sending address.
- Has one or more 1-of-n OP_CHECKMULTISIG outputs each containing at least two compressed public keys, one of which must be the senders address.

The above mentioned transactions are in conflict with those requirements.

There are multiple transactions with more than one input, but in almost all cases the same input address is used. There are currently 5 simple send transactions with more than one different input addresses. Not a single valid multi sig transaction.

I personally dislike the allowence of different inputs (many with the same pubKey are fine), because such transactions requires to fetch all input transactions to extract the value to determine the highest one which would otherwise be not necessary.

I have no solution how data packages can be safely identified and ordered and no implementation is currently able to parse a transaction with unordered data packages.

But if a given order is strictly required it would make the whole concept of sequence numbers more or less obsolete. I don't want to believe this though.

Here is my take:

1. Class A transactions are seperated and can be treated completey different. It's obsolete anyway, so workarounds or inefficiencies are not an issue.
2. I suggest to disallow multiple inputs with different public keys. This does not invalidate any transaction up until now, it is much more painless and furthermore the identification of the input reference is solely possible via the input itself. Including the input reference as part of a multisig still makes a lot of sense to redeem the output, but that's not mutually exclusive.
3. Fixing the prefix issue of compressed public keys is advised.
4. It would be very, very, very awesome, if there is a) a way to distinguish between data packages and non-data packages and b) if this is also possible, if data packages are not ordered. If this is the case, then including the pubKey as output would not require a special note nor would any malformated or additional pubKey in the outputs raise a conflict.

So the questions:

What do you think about dropping the support for multiple inputs with different pubKeys (= different wallets) for class B transactions?

How should multi sig outputs be treated which are neither a valid input reference (e.g. due to flawed compression, compression or because it's simply junk) nor a valid data package?

Currently all this mostly affects transactions that were issued by devs and those conflicts did not yet lead to notable problems, but once it gets real, there might be no turning back anymore.

That's my biggest "fear".

Hey DexX,

Apologies I don't have time to give your post all the attention it warrants, but a few quick notes:

* Re keys - this is just Masterchest sender key compression & I will be dropping this.  It does this to save transaction space but causes confusion when the key is uncompressed in the wallet but compressed in the multisig.  Both perfectly still redeemable by the same private key but the addresses differ, hence confusion.  Thus sender keys will soon be sent in the same format they are stored in the local bitcoin wallet.  Graz doesn't compress sender keys, and Bitoy does (because he uses my library).  Technically the spec states compressing is correct:

Has one or more 1-of-n OP_CHECKMULTISIG outputs each containing at least two compressed public keys, one of which must be the senders address

However given the amount of confusion over this (the debate went on back in the day) I think it's just easier to say the sender key can be either compressed or uncompressed, such that clients can broadcast it in whatever format it's stored locally (and thus have addresses match).
* There is an amendment to the spec that needs to be made - it's actually not largest input, it's largest input by sum - so the input address is selected from the address that contributed the highest sum input value.
* Sequence numbers are there for packet ordering, though there may be some shortcuts/assumptions in Masterchest code that assume a given packet order in the multisig (but that's not a spec problem, that's just code)

Thanks
Zathras

[grazcoin]

Graz,

Don't want to undo any work you've done mate, but multiple payments != multiple accepts from the same buyer.

I'm not saying we should invalidate multiple payments, just multiple accepts from the same buyer simultaneously.  A buyer could accept 100 MSC and send 5 bitcoin payments to cover the buy - that's still OK.  A buyer could not accept 100 MSC and then accept another 100 MSC in a second offer (possibly at a different price), then make a bunch of payments.  Not without something in the spec that defines (explicitly) how bitcoin payments would be applied to these accepts (in what order, whether overpayments are rolled to another accept offer and so on).

Hope that makes more sense

Thanks
Zathras

Zathras,

Multiple accepts were allowed on my code base since ever. For me multiple accept are way more reasonable (and required) than multiple payments, and since it is so clear, I didn't even think it should be mentioned in the spec.
A user can accept an offer and then decides that he wants to buy more, so he accepts another portion.
I don't see a reason for a user to accept an offer and then make a partial payment, and right after, within the allowed timeframe, to make another partial payment.

I have 2 reasonably tested code states:
1. Without multiple payments.
2. With multiple payments.

On the second there was (is) already consensus between masterchain and mymastercoins.

A version of multiple payment with a limit of a single accept per address could be implemented, but this requires a very detailed check if it break something in some wierd scenario, it requires again full testing on my dev setup, and then deployment on the production.
There can be many reasons to lose the current consensus as part of this change, and hit bugs which I don't currently expect.

The approach taken with multiple payment (and again - I am ready to drop that), was to associate the payments with the first accept until it is at least 100% paid, and on the next payment to associate with the next accept.
We all do not expect such behaviour to be often used, but we have to close those corners in the implementation.

It can't be so difficult for you to modify the code and enable the user to do something that basic, is it?

[zathras]

Graz,

Don't want to undo any work you've done mate, but multiple payments != multiple accepts from the same buyer.

I'm not saying we should invalidate multiple payments, just multiple accepts from the same buyer simultaneously.  A buyer could accept 100 MSC and send 5 bitcoin payments to cover the buy - that's still OK.  A buyer could not accept 100 MSC and then accept another 100 MSC in a second offer (possibly at a different price), then make a bunch of payments.  Not without something in the spec that defines (explicitly) how bitcoin payments would be applied to these accepts (in what order, whether overpayments are rolled to another accept offer and so on).

Hope that makes more sense

Thanks
Zathras

Zathras,

Multiple accepts were allowed on my code base since ever. For me multiple accept are way more reasonable (and required) than multiple payments, and since it is so clear, I didn't even think it should be mentioned in the spec.
A user can accept an offer and then decides that he wants to buy more, so he accepts another portion.
I don't see a reason for a user to accept an offer and then make a partial payment, and right after, within the allowed timeframe, to make another partial payment.

I have 2 reasonably tested code states:
1. Without multiple payments.
2. With multiple payments.

On the second there was (is) already consensus between masterchain and mymastercoins.

A version of multiple payment with a limit of a single accept per address could be implemented, but this requires a very detailed check if it break something in some wierd scenario, it requires again full testing on my dev setup, and then deployment on the production.
There can be many reasons to lose the current consensus as part of this change, and hit bugs which I don't currently expect.

The approach taken with multiple payment (and again - I am ready to drop that), was to associate the payments with the first accept until it is at least 100% paid, and on the next payment to associate with the next accept.
We all do not expect such behaviour to be often used, but we have to close those corners in the implementation.

It can't be so difficult for you to modify the code and enable the user to do something that basic, is it?

Appreciate that but actually yeah it is quite difficult to modify the code to support multiple accepts - I understand from your PoV it should be straight forward, but you treat bitcoin payments as the transactions that bought the TMSC, where as I treat the accept offer as the transaction that bought the TMSC - it's a different way of coming at it.

I'd like to be very clear on what it is I'd need to implement before I look at gap analysis, from talking to Bitoy he said he implements payments as follows:

1) User A accepts 50 MSC at 0.01 each (accept 1)
2) User A accepts 50 MSC at 0.01 each (accept 2)
3) User A sends a payment 0.2BTC (20MSC) - this is applied to accept 1
4) User A sends another payment 0.6BTC (60 MSC) - this is applied to accept 1.  The first 0.3BTC pays accept 1 fully (the remaining 30 MSC).  The remaining 0.3BTC is an overpayment and ignored.
5) User A sends another payment 0.2BTC (20 MSC) - this is applied to accept 2 as accept 1 is now fully paid.

So in this scenario the buyer has sent 1BTC for 100 MSC but actually only receives 70 MSC and the seller gets an overpayment.  This is how Bitoy has described it to me - if you guys have consensus are you doing the same thing then?

Thanks
Zathras

[zathras]

Also while I have you guys - I'm now enforcing the minimum fee requirements - you both validate this transaction 61d922eb409fcd932798cb519f97ba65de18099c501b324ae86b4658530da92b but it's invalid because the accept offer didn't have sufficient fees (0.00011 vs the required 0.0005).

Good though, as it led me to a bug in my wallet where accepts only sent with fee at 0.00011.

Thanks
Zathras

[grazcoin]

Appreciate that but actually yeah it is quite difficult to modify the code to support multiple accepts - I understand from your PoV it should be straight forward, but you treat bitcoin payments as the transactions that bought the TMSC, where as I treat the accept offer as the transaction that bought the TMSC - it's a different way of coming at it.

I'd like to be very clear on what it is I'd need to implement before I look at gap analysis, from talking to Bitoy he said he implements payments as follows:

1) User A accepts 50 MSC at 0.01 each (accept 1)
2) User A accepts 50 MSC at 0.01 each (accept 2)
3) User A sends a payment 0.2BTC (20MSC) - this is applied to accept 1
4) User A sends another payment 0.6BTC (60 MSC) - this is applied to accept 1.  The first 0.3BTC pays accept 1 fully (the remaining 30 MSC).  The remaining 0.3BTC is an overpayment and ignored.
5) User A sends another payment 0.2BTC (20 MSC) - this is applied to accept 2 as accept 1 is now fully paid.

So in this scenario the buyer has sent 1BTC for 100 MSC but actually only receives 70 MSC and the seller gets an overpayment.  This is how Bitoy has described it to me - if you guys have consensus are you doing the same thing then?

Thanks
Zathras

That's exactly like mastercoin-tools works, so we have consensus also on the algorithm.

[zathras]

Appreciate that but actually yeah it is quite difficult to modify the code to support multiple accepts - I understand from your PoV it should be straight forward, but you treat bitcoin payments as the transactions that bought the TMSC, where as I treat the accept offer as the transaction that bought the TMSC - it's a different way of coming at it.

I'd like to be very clear on what it is I'd need to implement before I look at gap analysis, from talking to Bitoy he said he implements payments as follows:

1) User A accepts 50 MSC at 0.01 each (accept 1)
2) User A accepts 50 MSC at 0.01 each (accept 2)
3) User A sends a payment 0.2BTC (20MSC) - this is applied to accept 1
4) User A sends another payment 0.6BTC (60 MSC) - this is applied to accept 1.  The first 0.3BTC pays accept 1 fully (the remaining 30 MSC).  The remaining 0.3BTC is an overpayment and ignored.
5) User A sends another payment 0.2BTC (20 MSC) - this is applied to accept 2 as accept 1 is now fully paid.

So in this scenario the buyer has sent 1BTC for 100 MSC but actually only receives 70 MSC and the seller gets an overpayment.  This is how Bitoy has described it to me - if you guys have consensus are you doing the same thing then?

Thanks
Zathras

That's exactly like mastercoin-tools works, so we have consensus also on the algorithm.

OK great - it's good that you two work the same way, but do you guys not see this as a massive flaw?  User A has accepted 100 MSC and made full payment within the allotted timeframe, yet only receives 70 MSC?

[dexX7]

Hey zathras,

thanks for the answers.

Both perfectly still redeemable by the same private key but the addresses differ, hence confusion.

No objection for compression at all, but I think the "02" and "03" is in the wrong order. At least according to brainwallet.org.

Code:

pubKeyUncompressed % 2 == 0 -> 0x02 + ...
pubKeyUncompressed % 2 == 1 -> 0x03 + ...

My question is how to deal with those and other non-data-package-and-not-the-input-pubKeys in the future.

There is an amendment to the spec that needs to be made - it's actually not largest input, it's largest input by sum - so the input address is selected from the address that contributed the highest sum input value.

Ah, I see. Well, not much a difference. To get the sum you'd need to query the blockchain for each input to determine which one is used as input. But is this really needed on such a deep protocol level? In my opinion keeping things simple and more atomic may be preferred.

Sequence numbers are there for packet ordering, though there may be some shortcuts/assumptions in Masterchest code that assume a given packet order in the multisig (but that's not a spec problem, that's just code)

Yes, it's used for ordering, but the trouble starts, if you don't know the order before trying to extract the sqeuence numbers. Say you have 4 packages, one of them is your input pubKey, so we can remove that one, the other three are unknown. You may now create three SHA512 hashes of the input pubKey according to the spec and then try to deobfuscate each package with each hash - that's six combinations. If you find a combination which results in a sequence of 01, 02, 03 it would be perfect, but what if one of the pubKeys was actually not a properly and valid encoded data package, but something else? That's why I posted the transactions with unknown or potentially prefix-swapped pubKeys. This could lead to collisions and it's no longer clear, if it's indeed a data-package with sequence number n or simply junk that appeared to result in a conflict. Well, in the later case of compressed pubKeys with swapped prefix it wouldn't be a problem, because the first byte is stripped, but the broader issue remains and I'm trying to see this as general as possible. Maybe I miss something essential the whole time?

OK great - it's good that you two work the same way, but do you guys not see this as a massive flaw?  User A has accepted 100 MSC and made full payment within the allotted timeframe, yet only receives 70 MSC?

That's indeed counterintuitive.

[grazcoin]

Appreciate that but actually yeah it is quite difficult to modify the code to support multiple accepts - I understand from your PoV it should be straight forward, but you treat bitcoin payments as the transactions that bought the TMSC, where as I treat the accept offer as the transaction that bought the TMSC - it's a different way of coming at it.

I'd like to be very clear on what it is I'd need to implement before I look at gap analysis, from talking to Bitoy he said he implements payments as follows:

1) User A accepts 50 MSC at 0.01 each (accept 1)
2) User A accepts 50 MSC at 0.01 each (accept 2)
3) User A sends a payment 0.2BTC (20MSC) - this is applied to accept 1
4) User A sends another payment 0.6BTC (60 MSC) - this is applied to accept 1.  The first 0.3BTC pays accept 1 fully (the remaining 30 MSC).  The remaining 0.3BTC is an overpayment and ignored.
5) User A sends another payment 0.2BTC (20 MSC) - this is applied to accept 2 as accept 1 is now fully paid.

So in this scenario the buyer has sent 1BTC for 100 MSC but actually only receives 70 MSC and the seller gets an overpayment.  This is how Bitoy has described it to me - if you guys have consensus are you doing the same thing then?

Thanks
Zathras

That's exactly like mastercoin-tools works, so we have consensus also on the algorithm.

OK great - it's good that you two work the same way, but do you guys not see this as a massive flaw?  User A has accepted 100 MSC and made full payment within the allotted timeframe, yet only receives 70 MSC?

A more accurate calculation would increase the complexity.
I already said that this is not the most common user behaviour, but I don't want to ban users from using multiple accepts.
As long as they pay the exact payment for each accept, everything works well.
If they want to split their payments in a funny way (partial payments and then over payments), they will have to check the exact rules.
That's a reasonable compromise.

[zathras]

Appreciate that but actually yeah it is quite difficult to modify the code to support multiple accepts - I understand from your PoV it should be straight forward, but you treat bitcoin payments as the transactions that bought the TMSC, where as I treat the accept offer as the transaction that bought the TMSC - it's a different way of coming at it.

I'd like to be very clear on what it is I'd need to implement before I look at gap analysis, from talking to Bitoy he said he implements payments as follows:

1) User A accepts 50 MSC at 0.01 each (accept 1)
2) User A accepts 50 MSC at 0.01 each (accept 2)
3) User A sends a payment 0.2BTC (20MSC) - this is applied to accept 1
4) User A sends another payment 0.6BTC (60 MSC) - this is applied to accept 1.  The first 0.3BTC pays accept 1 fully (the remaining 30 MSC).  The remaining 0.3BTC is an overpayment and ignored.
5) User A sends another payment 0.2BTC (20 MSC) - this is applied to accept 2 as accept 1 is now fully paid.

So in this scenario the buyer has sent 1BTC for 100 MSC but actually only receives 70 MSC and the seller gets an overpayment.  This is how Bitoy has described it to me - if you guys have consensus are you doing the same thing then?

Thanks
Zathras

That's exactly like mastercoin-tools works, so we have consensus also on the algorithm.

OK great - it's good that you two work the same way, but do you guys not see this as a massive flaw?  User A has accepted 100 MSC and made full payment within the allotted timeframe, yet only receives 70 MSC?

A more accurate calculation would increase the complexity.
I already said that this is not the most common user behaviour, but I don't want to ban users from using multiple accepts.
As long as they pay the exact payment for each accept, everything works well.
If they want to split their payments in a funny way (partial payments and then over payments), they will have to check the exact rules.
That's a reasonable compromise.

To be honest perhaps I'm misunderstanding your point, but if a user accepts 100 MSC across these multiple accepts and then fully pays for them within the time limit (including across multiple bitcoin payments) there is no reasonable compromise where the user only receives 70 MSC.

I do understand the complexity argument, but surely if we're talking about multiple accepts we should either do it right or not at all - even if doing it right increases complexity.  As I've mentioned before multiple accepts would be a problem for me - and I see no real prudence to implementing them in the structure described.

If we are to do multiple accepts, surely something like the sum of bitcoin payments within the various timelimits would need to be taken and applied to the accept offers (in blockchain order) incrementally at each respective unit price as each one is fully paid or something along those lines - sorry this is a new concept for me so haven't thought it through thoroughly.  My own view is that if I somehow need to code for multiple accepts, it has to be in a way that if a user accepts X and pays for X within the timelimit, then they receive X.

Thanks
Zathras

[zathras]

Hey zathras,

thanks for the answers.

Both perfectly still redeemable by the same private key but the addresses differ, hence confusion.

No objection for compression at all, but I think the "02" and "03" is in the wrong order. At least according to brainwallet.org.

Code:

pubKeyUncompressed % 2 == 0 -> 0x02 + ...
pubKeyUncompressed % 2 == 1 -> 0x03 + ...

My question is how to deal with those and other non-data-package-and-not-the-input-pubKeys in the future.

There is an amendment to the spec that needs to be made - it's actually not largest input, it's largest input by sum - so the input address is selected from the address that contributed the highest sum input value.

Ah, I see. Well, not much a difference. To get the sum you'd need to query the blockchain for each input to determine which one is used as input. But is this really needed on such a deep protocol level? In my opinion keeping things simple and more atomic may be preferred.

Sequence numbers are there for packet ordering, though there may be some shortcuts/assumptions in Masterchest code that assume a given packet order in the multisig (but that's not a spec problem, that's just code)

Yes, it's used for ordering, but the trouble starts, if you don't know the order before trying to extract the sqeuence numbers. Say you have 4 packages, one of them is your input pubKey, so we can remove that one, the other three are unknown. You may now create three SHA512 hashes of the input pubKey according to the spec and then try to deobfuscate each package with each hash - that's six combinations. If you find a combination which results in a sequence of 01, 02, 03 it would be perfect, but what if one of the pubKeys was actually not a properly and valid encoded data package, but something else? That's why I posted the transactions with unknown or potentially prefix-swapped pubKeys. This could lead to collisions and it's no longer clear, if it's indeed a data-package with sequence number n or simply junk that appeared to result in a conflict. Well, in the later case of compressed pubKeys with swapped prefix it wouldn't be a problem, because the first byte is stripped, but the broader issue remains and I'm trying to see this as general as possible. Maybe I miss something essential the whole time?

OK great - it's good that you two work the same way, but do you guys not see this as a massive flaw?  User A has accepted 100 MSC and made full payment within the allotted timeframe, yet only receives 70 MSC?

That's indeed counterintuitive.

Actually dexx that's a REALLY good point about packet ordering - I've just checked my code and out of order packets would FAIL (ie if key 2 was seqnum 2 and key 3 was seqnum 1).  I'll look at getting a test transaction on the network to probe this further - I think that's an immediate issue.

Also my key compression is as follows:

Code:

                frompubkey = validate.result.pubkey
                If validate.result.iscompressed = False Then
                    'compress public key
                    frompubkey = frompubkey.Substring(2, 128)
                    If Val(Right(frompubkey, 1)) Mod 2 Then
                        frompubkey = "03" & Left(frompubkey, 64)
                    Else
                        frompubkey = "02" & Left(frompubkey, 64)
                    End If
                End If

[Bitoy]

To be honest perhaps I'm misunderstanding your point, but if a user accepts 100 MSC across these multiple accepts and then fully pays for them within the time limit (including across multiple bitcoin payments) there is no reasonable compromise where the user only receives 70 MSC.

I do understand the complexity argument, but surely if we're talking about multiple accepts we should either do it right or not at all - even if doing it right increases complexity.  As I've mentioned before multiple accepts would be a problem for me - and I see no real prudence to implementing them in the structure described.

If we are to do multiple accepts, surely something like the sum of bitcoin payments within the various timelimits would need to be taken and applied to the accept offers (in blockchain order) incrementally at each respective unit price as each one is fully paid or something along those lines - sorry this is a new concept for me so haven't thought it through thoroughly.  My own view is that if I somehow need to code for multiple accepts, it has to be in a way that if a user accepts X and pays for X within the timelimit, then they receive X.

Thanks
Zathras

I've always thought that the system allows multiple transaction accepts.

Ex.  
Alice sell offer 100 tmsc for 1 btc
Bob sell offer 50 tmsc for .6 btc
Charles accepts 40  from Alice
Deb accepts 100 from Alice but gets only 60 (bec. Charles got it first )
Deb accepts 40 from Bob (because she wants 100 and got only 60)


So Deb has 2 transaction accepts.  

In grazcoins example it just happens that the buyer accepted 2 transaction from the same sell offer.   But we should treat it as regular multiple accept.

[dexX7]

I think you need to exchange "02" and "03". Here is some test data:

Code:

Uncompressed: 0478d430274f8c5ec1321338151e9f27f4c676a008bdf8638d07c0b6be9ab35c71a1518063243acd4dfe96b66e3f2ec8013c8e072cd09b3834a19f81f659cc3455
Compressed:
0378d430274f8c5ec1321338151e9f27f4c676a008bdf8638d07c0b6be9ab35c71

Uncompressed: 0451a2a7382944132a9ba4429fd12180d63e763ddedca57bdd21a4fb9faef20915280545aef177da1e7c90aec32800b1b313a410e196c916fbfa5b206b7619dfe4
Compressed:
0251a2a7382944132a9ba4429fd12180d63e763ddedca57bdd21a4fb9faef20915

 

[zathras]

To be honest perhaps I'm misunderstanding your point, but if a user accepts 100 MSC across these multiple accepts and then fully pays for them within the time limit (including across multiple bitcoin payments) there is no reasonable compromise where the user only receives 70 MSC.

I do understand the complexity argument, but surely if we're talking about multiple accepts we should either do it right or not at all - even if doing it right increases complexity.  As I've mentioned before multiple accepts would be a problem for me - and I see no real prudence to implementing them in the structure described.

If we are to do multiple accepts, surely something like the sum of bitcoin payments within the various timelimits would need to be taken and applied to the accept offers (in blockchain order) incrementally at each respective unit price as each one is fully paid or something along those lines - sorry this is a new concept for me so haven't thought it through thoroughly.  My own view is that if I somehow need to code for multiple accepts, it has to be in a way that if a user accepts X and pays for X within the timelimit, then they receive X.

Thanks
Zathras

I've always thought that the system allows multiple transaction accepts.

Ex.  
Alice sell offer 100 tmsc for 1 btc
Bob sell offer 50 tmsc for .6 btc
Charles accepts 40  from Alice
Deb accepts 100 from Alice but gets only 60 (bec. Charles got it first )
Deb accepts 40 from Bob (because she wants 100 and got only 60)


So Deb has 2 transaction accepts.  

In grazcoins example it just happens that the buyer accepted 2 transaction from the same sell offer.   But we should treat it as regular multiple accept.

The system does allow multiple transaction accepts - what we're talking about is multiple transaction accepts at the same time, from the same buyer.  That's where the complexity comes from.

[zathras]

I think you need to exchange "02" and "03". Here is some test data:

Code:

Uncompressed: 0478d430274f8c5ec1321338151e9f27f4c676a008bdf8638d07c0b6be9ab35c71a1518063243acd4dfe96b66e3f2ec8013c8e072cd09b3834a19f81f659cc3455
Compressed:
0378d430274f8c5ec1321338151e9f27f4c676a008bdf8638d07c0b6be9ab35c71

Uncompressed: 0451a2a7382944132a9ba4429fd12180d63e763ddedca57bdd21a4fb9faef20915280545aef177da1e7c90aec32800b1b313a410e196c916fbfa5b206b7619dfe4
Compressed:
0251a2a7382944132a9ba4429fd12180d63e763ddedca57bdd21a4fb9faef20915

 

OK thanks, will check into that