300 BTC Coding Contest: Distributed Exchange (MasterCoin Developer Thread)

[Tachikoma]

Question to all devs working on creating a distributed exchange: in your opinion, what's a realistic timeframe for a decentralized MSC <> BTC exchange? 1 week? 2 weeks? 1 month?

Together with Pouncer I'm running the Mastercoin order book. Adding and editing entries is done manually and takes quite a lot of time.
So I was thinking about setting up a self-service order book where people could add, edit and remove orders on their own. This would be done by signing messages with their Mastercoin addresses so there would be no fake orders (balance of addresses can be easily checked using the Mastercoin-explorer API).

Creating such an improved order book would take some time and effort and I'm wondering if it makes sense to do this at this point in time. If we have a working decentralized exchange within a week or two it might be better to continue with the current order book. But if we're talking about a month then I guess it would make sense to invest time in an improved order book. After all, Mastercoin trades are really booming right now.

So what's your estimates, when will we see a decentralized MSC <> BTC exchange?

I think I need to implement two more validation cases, mainly around reserved balances, and then it should be done on my part. However this is where the fun starts. We will need to do a whole lot of testing to make sure we got all the edge cases. I think the best way is to start trading for real and see what happens. We should have three sites online very soon that will be able to parse the transactions so we can check if we all have the same idea.

I think the ultimate question will actually be; when will the Mastercoin users trust the MSC <> BTC exchange protocol enough to actively choose this over doing everything by hand. I'm afraid this might be still a long way out. Technically you could use it right now.

Actually, no. 8932.02057119

I've run some test cases and I can get your number - same as before mate, you're using 9 decimal places in your dev MSC total.

In this part of your formula

Code:

* 0.1 * 563162.23576222

you are effectively multiplying by 56316.223576222 - note the additional decimal.  You can fix your code by changing:

Code:

(1-(0.5**time_difference)) * 0.1 * 563_162.23576222

to either of:

Code:

(1-(0.5**time_difference)) * 56316.22357622

Code:

(1-(0.5**time_difference)) * (0.1 * 563_162.23576222).round(8)

I've been hammered with the day job this last week but I can see there is some stuff I need to catch up on the main thread (eg Luke's patch).  I'm determined to get the Masterchest update out today, at least to the website - once I've got that finished I'll catch up on what's been happening.

Thanks!

See this is what I currently hate about doing all the MSC stuff. I'm currently switching between projects (wallet, site, validation site, library) and I often forget I was doing something else before switching energy. It's just too much stuff to do. I modified it and I now get the proper amount. Although if we want to do it totally correct we should round the last digit up so in theory it should be 8932.02250395 instead of 8932.02250394. If we can agree on this I can finally consider vesting to be perfect

Speaking of which - Tachikoma, I'm keen to identify why our orderbook states don't match.  I've taken a quick look at Bitoy's site and his orderbook state looks like mine - I'll look into each

I need to do a fresh import & validation on the Purchase Offer code. These transactions were verified using older code an I need to recheck them. It is probably fixed by now but I need to look into it when I can find enough time to do something useful. I'm still having huge issues getting proper block numbers from Bitcoin-ruby which is really a time sink at the moment, once I fix that I can move on

[Tachikoma]

Developers; I've been thinking about invalidating Purchase Offers which want to buy from a Selling Offer when the Selling Offer is exhausted. Normally you adjust the amount bough to the total amount for sale. But it's not stated what to do when the amount is actually zero. Technically it doesn't matter anyway because both transactions have zero impact on the balance but just to let people know what's going on, and make sure they don't end up paying I would still like to invalidate these type of transactions.

Just an example to make sure everybody understands me.

User A sells 10 MSC
User B buys 8 MSC and gets them
User C buys 2 MSC and gets them
User D sends his request for 5MSC just a little after C and gets added behind him in the block.

Right now user D's transaction would come through but the accepted amount would be 0 MSC. I would propose to invalid this transaction so it's clear this offer is sold out.

[Bitoy]

Speaking of which - Tachikoma, I'm keen to identify why our orderbook states don't match.  I've taken a quick look at Bitoy's site and his orderbook state looks like mine - I'll look into each transaction to see if I can figure out where we differ.  Is your orderbook on mastercoin-explorer your latest? 

Good to know our order book matches

.  Oh and on the subject of the dev MSC calculations you can use https://masterchest.info/sneakpeek/lookupadd.aspx?address=1EXoDusjGwvnjZUyKkxZ4UHEf77z6A5S4P to check your dev MSC numbers against mine (as at time of latest block).

I only recalculate  and validate dev mastercoins only if the exodus address sends msc.

[Bitoy]

Developers; I've been thinking about invalidating Purchase Offers which want to buy from a Selling Offer when the Selling Offer is exhausted. Normally you adjust the amount bough to the total amount for sale. But it's not stated what to do when the amount is actually zero. Technically it doesn't matter anyway because both transactions have zero impact on the balance but just to let people know what's going on, and make sure they don't end up paying I would still like to invalidate these type of transactions.

Just an example to make sure everybody understands me.

User A sells 10 MSC
User B buys 8 MSC and gets them
User C buys 2 MSC and gets them
User D sends his request for 5MSC just a little after C and gets added behind him in the block.

Right now user D's transaction would come through but the accepted amount would be 0 MSC. I would propose to invalid this transaction so it's clear this offer is sold out.

+1
User D's transaction should be invalid.

[Bitoy]

Another scenario.

User A sells 10 msc
User B and User C buys 10 at the same (block) time.
Who gets the 10 msc?

I'm thinking it should be order by time, then address.

[vokain]

My guess is full development of the protocol could take 1-3 years. We (the Mastercoin community) are all doing something to accelerate this, whether it be clearing up misconceptions in other fora, participating in the giveaway or actively volunteering themselves to run a manual orderbook, to developing incentivized and structured bounty systems and evaluating the huge talent pool that have been asking how they could help, or of course, encouraging the developers (best accomplished by using and buy testing their work!).

Play your part

[Tachikoma]

Another scenario.

User A sells 10 msc
User B and User C buys 10 at the same (block) time.
Who gets the 10 msc?

I'm thinking it should be order by time, then address.

The person with the highest position in the block. This already has been decided a long time ago and goes for all transactions types.

[Tachikoma]

Zathras, it seems I am lacking some validations on my end on Purchase Offers that you do have. Could you take a look at this post and tell me why Masterchest rejects that offer?

Edit: I'm guessing this is because there is no selling order to match to? You consider that an invalid transaction? I think this is a good way to go, shall I add this to the spec?

[Bitoy]

Another scenario.

User A sells 10 msc
User B and User C buys 10 at the same (block) time.
Who gets the 10 msc?

I'm thinking it should be order by time, then address.

The person with the highest position in the block. This already has been decided a long time ago and goes for all transactions types.

Ok thanks again Tachikoma.

[Bitoy]

Edit: I'm guessing this is because there is no selling order to match to? You consider that an invalid transaction? I think this is a good way to go, shall I add this to the spec?

Same here.  No selling order to match the transaction, it is invalid.

[Bitoy]

Confirmed payment for first distributed exchange msc transaction.  (I think this is from tachikoma's wallet

http://mymastercoins.com/Orders.aspx?CurrencyID=1


Buying MSC
Waiting for Payment   Payment Confirmed   Expired
Date   Amount to Purchase   Purchased   Unit Price   Total BTC   Transfer Fee   Tx ID   CFS ID   Buyer ID   Seller ID   Max Block Time       Confirm TxID
11/17/2013 4:03:10 PM   .101   .101   .2   .0202   .001   3919   162   9046   9044   270161   Payment Confirmed   3921

[zathras]

I modified it and I now get the proper amount. Although if we want to do it totally correct we should round the last digit up so in theory it should be 8932.02250395 instead of 8932.02250394. If we can agree on this I can finally consider vesting to be perfect

My figure for your test scenario is 8932.02250394725, so yep that should indeed be 8932.02250395, not 8932.02250394.  I must not have had my Weetbix that morning

Developers; I've been thinking about invalidating Purchase Offers which want to buy from a Selling Offer when the Selling Offer is exhausted. Normally you adjust the amount bough to the total amount for sale. But it's not stated what to do when the amount is actually zero. Technically it doesn't matter anyway because both transactions have zero impact on the balance but just to let people know what's going on, and make sure they don't end up paying I would still like to invalidate these type of transactions.

Just an example to make sure everybody understands me.

User A sells 10 MSC
User B buys 8 MSC and gets them
User C buys 2 MSC and gets them
User D sends his request for 5MSC just a little after C and gets added behind him in the block.

Right now user D's transaction would come through but the accepted amount would be 0 MSC. I would propose to invalid this transaction so it's clear this offer is sold out.

In my implementation user D's transaction would be considered invalid.  Once the transaction from User C was processed the sell offer from User A would be 'closed' (as fulfilled) thus when we then go to process user D's accept offer it would be unmatched and thus rejected.

Zathras, it seems I am lacking some validations on my end on Purchase Offers that you do have. Could you take a look at this post and tell me why Masterchest rejects that offer?

Edit: I'm guessing this is because there is no selling order to match to? You consider that an invalid transaction? I think this is a good way to go, shall I add this to the spec?

If an accept offer is sent to an address without an 'open' sell offer then yep I consider it invalid.

I only recalculate  and validate dev mastercoins only if the exodus address sends msc.

IMO we should calculate dev MSC in each block (and this is what Masterchest does).  Consumers should be able to check the MSC balance of the Exodus address at any time just like any other.

Confirmed payment for first distributed exchange msc transaction.  (I think this is from tachikoma's wallet)

Up until yesterday there was only one complete distributed exchange purchase that was actually valid.  There are now five

Code:

select * from transactions_processed where TYPE='purchase'

c7103de53a8ea3a20cec9a74543132f98302f75ab1868092e911dd5516f322a5	16rAwebBXhJAM9ALf3fLFbaHKz24r2o3UN	18xEZx3po1iJWP5H2aM3Do11dCGQyaebnT	NULL	purchase	1383386099	267480	1	2	35721301	NULL	323200		NULL	NULL	20000000	9087a1b72536f3b2909943553ba6d4e320565eeea1e42379dd4337b532f09fd8
baba8972de24528bd56ec4ab3ce5fcead28eb45ac418eb87aace08d3da062a9b	1EAuHj8Z6rTCHPxXfaGzzPsZevC2mg1XAj	1AMfFzbrhhizKDpqebYVYFGaTwdtSt5ux2	NULL	purchase	1384704190	270151	1	1	35721741	NULL	2020000		NULL	NULL	10100000	35822ef50d1c5639c80ded37b6580396ecc163bdbad34b3865b9e67b7f0787b3
ac7593ff4fb4a9bc6b053fc12af10e4fc04db73e0fb467d1dbc2466c86c5936b	1EAuHj8Z6rTCHPxXfaGzzPsZevC2mg1XAj	1F73UPD5xBKgTSRd8q6QhuncVmDnJAHxYV	NULL	purchase	1384708399	270160	1	1	35721744	NULL	10000000	NULL	NULL	50000000	35822ef50d1c5639c80ded37b6580396ecc163bdbad34b3865b9e67b7f0787b3
f67cc2760446b1458c4012bbe7d5f6129badcf4ac6896f82a4de7ec238103216	1EAuHj8Z6rTCHPxXfaGzzPsZevC2mg1XAj	13tKKPNwDZGKhBK8gAHns7bXe2wtqhvzDb	NULL	purchase	1384699499	270144	1	2	35721727	NULL	20000		NULL	NULL	100000		35822ef50d1c5639c80ded37b6580396ecc163bdbad34b3865b9e67b7f0787b3
87018464d016a067e5c8fedc07551c027bd011f43f6d3a02e9a1e252b551ba82	1EAuHj8Z6rTCHPxXfaGzzPsZevC2mg1XAj	1F73UPD5xBKgTSRd8q6QhuncVmDnJAHxYV	NULL	purchase	1384699499	270144	1	2	35721728	NULL	2000000		NULL	NULL	10000000	35822ef50d1c5639c80ded37b6580396ecc163bdbad34b3865b9e67b7f0787b3

I currently test by simulating blockchain transactions (as I just drop dummy transactions into my transaction list), but I might start pushing some of mine actually out onto the blockchain since it looks like we're all starting to converge in terms of implementation compatibility

Thanks!

[Bitoy]

Code:

select * from transactions_processed where TYPE='purchase'

c7103de53a8ea3a20cec9a74543132f98302f75ab1868092e911dd5516f322a5	16rAwebBXhJAM9ALf3fLFbaHKz24r2o3UN	18xEZx3po1iJWP5H2aM3Do11dCGQyaebnT	NULL	purchase	1383386099	267480	1	2	35721301	NULL	323200		NULL	NULL	20000000	9087a1b72536f3b2909943553ba6d4e320565eeea1e42379dd4337b532f09fd8
baba8972de24528bd56ec4ab3ce5fcead28eb45ac418eb87aace08d3da062a9b	1EAuHj8Z6rTCHPxXfaGzzPsZevC2mg1XAj	1AMfFzbrhhizKDpqebYVYFGaTwdtSt5ux2	NULL	purchase	1384704190	270151	1	1	35721741	NULL	2020000		NULL	NULL	10100000	35822ef50d1c5639c80ded37b6580396ecc163bdbad34b3865b9e67b7f0787b3
ac7593ff4fb4a9bc6b053fc12af10e4fc04db73e0fb467d1dbc2466c86c5936b	1EAuHj8Z6rTCHPxXfaGzzPsZevC2mg1XAj	1F73UPD5xBKgTSRd8q6QhuncVmDnJAHxYV	NULL	purchase	1384708399	270160	1	1	35721744	NULL	10000000	NULL	NULL	50000000	35822ef50d1c5639c80ded37b6580396ecc163bdbad34b3865b9e67b7f0787b3
f67cc2760446b1458c4012bbe7d5f6129badcf4ac6896f82a4de7ec238103216	1EAuHj8Z6rTCHPxXfaGzzPsZevC2mg1XAj	13tKKPNwDZGKhBK8gAHns7bXe2wtqhvzDb	NULL	purchase	1384699499	270144	1	2	35721727	NULL	20000		NULL	NULL	100000		35822ef50d1c5639c80ded37b6580396ecc163bdbad34b3865b9e67b7f0787b3
87018464d016a067e5c8fedc07551c027bd011f43f6d3a02e9a1e252b551ba82	1EAuHj8Z6rTCHPxXfaGzzPsZevC2mg1XAj	1F73UPD5xBKgTSRd8q6QhuncVmDnJAHxYV	NULL	purchase	1384699499	270144	1	2	35721728	NULL	2000000		NULL	NULL	10000000	35822ef50d1c5639c80ded37b6580396ecc163bdbad34b3865b9e67b7f0787b3

I'm only getting the first 2 as valid.  The rest invalid.  I'll check my codes.

[zathras]

Code:

select * from transactions_processed where TYPE='purchase'

c7103de53a8ea3a20cec9a74543132f98302f75ab1868092e911dd5516f322a5	16rAwebBXhJAM9ALf3fLFbaHKz24r2o3UN	18xEZx3po1iJWP5H2aM3Do11dCGQyaebnT	NULL	purchase	1383386099	267480	1	2	35721301	NULL	323200		NULL	NULL	20000000	9087a1b72536f3b2909943553ba6d4e320565eeea1e42379dd4337b532f09fd8
baba8972de24528bd56ec4ab3ce5fcead28eb45ac418eb87aace08d3da062a9b	1EAuHj8Z6rTCHPxXfaGzzPsZevC2mg1XAj	1AMfFzbrhhizKDpqebYVYFGaTwdtSt5ux2	NULL	purchase	1384704190	270151	1	1	35721741	NULL	2020000		NULL	NULL	10100000	35822ef50d1c5639c80ded37b6580396ecc163bdbad34b3865b9e67b7f0787b3
ac7593ff4fb4a9bc6b053fc12af10e4fc04db73e0fb467d1dbc2466c86c5936b	1EAuHj8Z6rTCHPxXfaGzzPsZevC2mg1XAj	1F73UPD5xBKgTSRd8q6QhuncVmDnJAHxYV	NULL	purchase	1384708399	270160	1	1	35721744	NULL	10000000	NULL	NULL	50000000	35822ef50d1c5639c80ded37b6580396ecc163bdbad34b3865b9e67b7f0787b3
f67cc2760446b1458c4012bbe7d5f6129badcf4ac6896f82a4de7ec238103216	1EAuHj8Z6rTCHPxXfaGzzPsZevC2mg1XAj	13tKKPNwDZGKhBK8gAHns7bXe2wtqhvzDb	NULL	purchase	1384699499	270144	1	2	35721727	NULL	20000		NULL	NULL	100000		35822ef50d1c5639c80ded37b6580396ecc163bdbad34b3865b9e67b7f0787b3
87018464d016a067e5c8fedc07551c027bd011f43f6d3a02e9a1e252b551ba82	1EAuHj8Z6rTCHPxXfaGzzPsZevC2mg1XAj	1F73UPD5xBKgTSRd8q6QhuncVmDnJAHxYV	NULL	purchase	1384699499	270144	1	2	35721728	NULL	2000000		NULL	NULL	10000000	35822ef50d1c5639c80ded37b6580396ecc163bdbad34b3865b9e67b7f0787b3

I'm only getting the first 2 as valid.  The rest invalid.  I'll check my codes.

Thanks - I'll double check my parsing (given how close those payments are together and that they're to the same seller it's certainly possible I've got a bug somewhere counting the same payment in multiple trades).

[Bitoy]

Code:

ac7593ff4fb4a9bc6b053fc12af10e4fc04db73e0fb467d1dbc2466c86c5936b
f67cc2760446b1458c4012bbe7d5f6129badcf4ac6896f82a4de7ec238103216
87018464d016a067e5c8fedc07551c027bd011f43f6d3a02e9a1e252b551ba82

The transactions above are invalid because
No Offer to Sell from Seller 1EAuHj8Z6rTCHPxXfaGzzPsZevC2mg1XAj for Currency "Test MSC"

[Tachikoma]

I've added a PR to group the selling/purchase offer clarifications. Developers, please read through them and see if you agree so far.

I also found out I missed a line in the spec that said that the actual purchase offer should be adjusted based on the amount of funds received in the actual payment. I missed this and will need to add it to mastercoin-explorer.

Now how should we deal with this? Should we only count the highest payment to an address? Should we collect all payments send in the timeframe and add them together? Allowing to be paid in parts? Should we change the spec to say we only allow the exact payment amount?

I personally would like to collect all payments received during the block time limit and group them together and then adjust the Purchase Offer according to how many coins were received in total, although the amount can never be more then the Purchase Offer's initial amount.

Example 1

User A creates a Selling Offer for 10 MSC @ 0.1 a coin with a time limit of 6 in block 20.
User B sends a Purchase Offer to buy 5 MSC in block 22.
User B sends a payment to User A of 0.3 BTC in block 23.
Nothing more happens.

User B Purchase Offer gets adjusted to buy 3 MSC and gets validated.

Example 2

User A creates a Selling Offer for 10 MSC @ 0.1 a coin with a time limit of 6 in block 20.
User B sends a Purchase Offer to buy 5 MSC in block 22.
User B sends a payment to User A of 0.3 BTC in block 23.
User B sends a payment to User A of 0.8 BTC in block 24.
User B paid 0.11BTC in total, enough to completely satisfy his order.

[Bitoy]

I've added a PR to group the selling/purchase offer clarifications. Developers, please read through them and see if you agree so far.



Example 1

User A creates a Selling Offer for 10 MSC @ 0.1 a coin with a time limit of 6 in block 20.
User B sends a Purchase Offer to buy 5 MSC in block 22.
User B sends a payment to User A of 0.3 BTC in block 23.
Nothing more happens.

User B Purchase Offer gets adjusted to buy 3 MSC and gets validated.

Example 2

User A creates a Selling Offer for 10 MSC @ 0.1 a coin with a time limit of 6 in block 20.
User B sends a Purchase Offer to buy 5 MSC in block 22.
User B sends a payment to User A of 0.3 BTC in block 23.
User B sends a payment to User A of 0.8 BTC in block 24.
User B paid 0.11BTC in total, enough to completely satisfy his order.

Another option is the seller has to pay the full btc amount.  No partial payment allowed.
Ex.
User A creates a Selling Offer for 10 MSC @ 0.1 a coin with a time limit of 6 in block 20.
User B sends a Purchase Offer to buy 5 MSC in block 22.
User B sends a payment to User A of 0.5 BTC in block 24.

If user B sends a partial payment, it is ignored.  System will wait only for the exact amount.

[Tachikoma]

Yeah that's one of the other suggestions that I offered. It would make parsing easier but reduce the flexibility of the system.

[dacoinminster]

The pull request looks good, but it does not appear to address the issue of what happens when the user sends a different payment than what they reserved. I think the right thing to do is to honor the payment as a purchase if at all possible.

For instance, if they send less payment before the time limit expires, the amount purchased is adjusted downward appropriately. If they pay MORE than what they reserved and there are still unreserved coins for sale, we adjust the amount purchased upwards, even though they didn't reserve that much. Obviously if another buyer reserves the remaining coins before the over-payment in the block-chain or the seller decides to cancel, the first one recorded in a block wins.

Another example is if the buyer foolishly sends payment without reserving anything at all. I don't think we can count that as a purchase, since it's effectively a bitcoin payment which happens to have an output to the Exodus Address. We can't be certain that they were trying to buy MSC with that payment.

[Tachikoma]

That still doesn't answer the main question though.

Should multiple transaction be accumulated and should the sum be used to match the purchase order
 or
Should the highest transaction be counted and matched to a purchase order