300 BTC Coding Contest: Distributed Exchange (MasterCoin Developer Thread)

[ripper234]

So there is no way to cancel my selling order?

Hmm, I was about to point you towards masterchain for that, but I can't find the Cancel function on it at all (I opened a bug on this as well).

Let's see if we can at least get an explanation of how to do this using the mastercoin-tools python script.

[jakecnn]

I just encountered the same problem, can't cancel my order.

Also, the message 'ping?' isn't really helpful at masterchain.info when trying to create a sell offer while another one is active. When I try to cancel the offer(that is using 0 as amount), I also get 'ping?' after a long delay.

[ripper234]

I just encountered the same problem, can't cancel my order.

Also, the message 'ping?' isn't really helpful at masterchain.info when trying to create a sell offer while another one is active. When I try to cancel the offer(that is using 0 as amount), I also get 'ping?' after a long delay.

Understood - please open a bug - https://github.com/grazcoin/mastercoin-tools/issues

[jakecnn]

Another weird thing is the timelimit: I made an offer today and someone accepted and payed 15 minutes later(issued the tx at least). Now the timelimit was set to 20 blocks and I think it took about 24 blocks until the payment was confirmed... So basically I got free bitcoin while the other person payed in a timely manner from his point of view(15 minutes), but due to the nature of complexer transactions, fees and the mercy of miners his transaction got delayed. It was just a tiny amount(0.00012), but if people are unaware of this, it can cause quite some financial damage to them.
This system appears unreliable to me. There should at least be warnings in the wallets/clients if users want to issue/accept an offer with an unreasonable short time limit.

[zathras]

Craig just pinged me to say there was a critical bug with cancelling sell orders - sorry about that guys - committed a fix (here).  4AM & need to sleep so heading offline again

Thanks
Zathras

[jakecnn]

I just encountered the same problem, can't cancel my order.

Also, the message 'ping?' isn't really helpful at masterchain.info when trying to create a sell offer while another one is active. When I try to cancel the offer(that is using 0 as amount), I also get 'ping?' after a long delay.

Understood - please open a bug - https://github.com/grazcoin/mastercoin-tools/issues

I think something is wrong with the server, trying to accept an offer returns the same message.

[ripper234]

So there is no way to cancel my selling order?

Craig just pinged me to say there was a critical bug with cancelling sell orders - sorry about that guys - committed a fix (here).  4AM & need to sleep so heading offline again

Thanks
Zathras

I did tell Craig to wait until it's 8 AM your time before waking you

I confirmed it now and the fix seems to be working.

jeroenn13 - can you verify on your end?

[jeroenn13]

So there is no way to cancel my selling order?

Craig just pinged me to say there was a critical bug with cancelling sell orders - sorry about that guys - committed a fix (here).  4AM & need to sleep so heading offline again

Thanks
Zathras

I did tell Craig to wait until it's 8 AM your time before waking you

I confirmed it now and the fix seems to be working.

jeroenn13 - can you verify on your end?

I can confirm it works now.
https://masterchest.info/lookuptx.aspx?txid=1260b28bb77ce470d31725294a65b3c5f920f8ebef3fbd0449f48c9ecd6e5956

Thank you

[grazcoin]

So there is no way to cancel my selling order?

Hmm, I was about to point you towards masterchain for that, but I can't find the Cancel function on it at all (I opened a bug on this as well).

Let's see if we can at least get an explanation of how to do this using the mastercoin-tools python script.

I just answered on this issue on github:
cancel is not available on masterchain.info, but it is available on demo.masterchain.info.
simply visit the same sell offer on demo, and you'll see the button.

[prophetx]

Another weird thing is the timelimit: I made an offer today and someone accepted and payed 15 minutes later(issued the tx at least). Now the timelimit was set to 20 blocks and I think it took about 24 blocks until the payment was confirmed... So basically I got free bitcoin while the other person payed in a timely manner from his point of view(15 minutes), but due to the nature of complexer transactions, fees and the mercy of miners his transaction got delayed. It was just a tiny amount(0.00012), but if people are unaware of this, it can cause quite some financial damage to them.
This system appears unreliable to me. There should at least be warnings in the wallets/clients if users want to issue/accept an offer with an unreasonable short time limit.

the tx had no mining fee right?

[jakecnn]

The payment in question is https://blockchain.info/tx/b9151c3371d46853e59ee6c9124c2a534f41363b29f372d914a19a83a5917665
and according to blockchain.info a tx-fee was paid.

[zathras]

The payment in question is https://blockchain.info/tx/b9151c3371d46853e59ee6c9124c2a534f41363b29f372d914a19a83a5917665
and according to blockchain.info a tx-fee was paid.

This is not so much a problem of the Master Protocol, but more one of how bitcoin works.  If I may take a moment to explain

Miners 'prioritize' transactions.  This means when your transaction is relayed to a given mining client it adds the transaction to its memory pool and assigns a priority and fee.  If the priority/fee is high enough, hopefully the transaction will be included in the next block.

Now - the interesting bit is in how this transaction priority is calculated by bitcoin core, as follows:

priority = sum(input_value_in_base_units * input_age)/size_in_bytes

Thus we can see the smaller and younger the inputs, the lower the transaction priority becomes.  

If we specifically relate this to your transaction, we can see that the inputs were all very small.  Additionally because there are a number of them, the size is actually over 1KB.  Thus we have a small input value and a (relatively) large byte size for the transaction.  If you pop those numbers into the formula above you'll see this provides for a very low priority transaction.

The good news (and this is bitcoin-wide) is that in laymans terms - the more you send, the higher your transaction priority.  Thus this may be an issue for low value fractional transactions and I'll do some further investigation and discuss with the team - but essentially as the value of the DEx purchase increases, the risk of the issue you describe occurring decreases.

I have also been doing some thinking on input selection, but that would apply only to the Masterchest implementation when I get there.  Currently I select smallest input for risk minimization (avoid losing a big input if something goes wrong creating the transaction).  On the todo list is adaptive transaction encoding in the library, so rather than 'dumb' input selection we look at input values, ages and eventual transaction size to determine priority before it's broadcast & if this would result in a LP transaction that is likely to take a while to get mined, re-encode it if possible for a higher priority, else send it with a higher fee.  That's a complex undertaking though so will take a little time (we always have to be careful when coding input selection and amounts as that's where the biggest potential to lose bitcoins sits IMO).

Thanks!
Zathras

EDIT: Also realized I wasn't explicit in stating it's of course also feasible to simply discourage (or disallow) fractional purchases of for example 0.000X BTC after a specific block height in the protocol itself too - the Master Protocol DEx has never been intended for high frequency or micro trading (and these micro trades cost more in fees than the trade itself).

[dexX7]

Additionally because there are a number of them, the size is actually over 1KB.  Thus we have a small input value and a (relatively) large byte size for the transaction.

This is the most important part related to this transaction. Over 1 KB size = 0.0002 fee required to be treated as standard transaction. If this was issued via one of the wallets, the wallet creator needs to adjust to take transaction size into consideration asap. Delayed transactions due to insufficient fees can be a game breaker, if MSC are bought via DEX and the time window closes before the transaction confirms.

[prophetx]

Additionally because there are a number of them, the size is actually over 1KB.  Thus we have a small input value and a (relatively) large byte size for the transaction.

This is the most important part related to this transaction. Over 1 KB size = 0.0002 fee required to be treated as standard transaction. If this was issued via one of the wallets, the wallet creator needs to adjust to take transaction size into consideration asap. Delayed transactions due to insufficient fees can be a game breaker, if MSC are bought via DEX and the time window closes before the transaction confirms.

+1

and thanks Zathras for the explanation

what should be the process for this?  someone going to file an issue? i guess technically its not a master protocol thing, but it would need to be at least discussed in a special considerations area of the spec?

[zathras]

Additionally because there are a number of them, the size is actually over 1KB.  Thus we have a small input value and a (relatively) large byte size for the transaction.

This is the most important part related to this transaction. Over 1 KB size = 0.0002 fee required to be treated as standard transaction. If this was issued via one of the wallets, the wallet creator needs to adjust to take transaction size into consideration asap. Delayed transactions due to insufficient fees can be a game breaker, if MSC are bought via DEX and the time window closes before the transaction confirms.

That's actually a really interesting problem - especially in relation to these fractional transactions.  We build the transaction before we can see its total size, but by then we've already selected the inputs.  As we're talking such tiny amounts (often inputs of 0.00006) increasing the fee by 0.0001 likely means we have to add another/more inputs to cover the increased fee, but that has now changed the size of our transaction again so we may need to yet again go back and add more inputs & more fees and so on.  Fun game

In all seriousness though I do agree with DexX - changes to payments make me nervous but it should be safe.  I've pushed up a shortcut fix which grows the fee during transaction build using simple (very conservative 200 bytes per input) guestimation based on total number of inputs at that point in the transaction build.  Should take care of any potential risk for now until I can be a bit more classy with input selection

Code:

                    'sanity check input count is not >24
                    If inputcount > 24 Then
                        MsgBox("ERROR: Input count >24, temporary restriction applied - library currently will not send transactions with over 24 inputs.  Aborting")
                        Exit Function
                    End If
                    'quick fix for avoiding large transactions with small fees - come back & revisit this
                    If inputcount <= 4 Then totaltxfee = 17000
                    If inputcount > 4 Then totaltxfee = 27000
                    If inputcount > 9 Then totaltxfee = 37000
                    If inputcount > 14 Then totaltxfee = 47000
                    If inputcount > 19 Then totaltxfee = 57000
                    'recalculate totalout
                    totalout = totaltxfee + paymentamount
                    'do we have enough yet?
                    If inputsum >= totalout + 6000 Then Exit For

Note this only applies to DEx payment transactions, all other transaction types Masterchest selects a single input only to cover fees.

Thanks
Zathras

[zathras]

Additionally because there are a number of them, the size is actually over 1KB.  Thus we have a small input value and a (relatively) large byte size for the transaction.

This is the most important part related to this transaction. Over 1 KB size = 0.0002 fee required to be treated as standard transaction. If this was issued via one of the wallets, the wallet creator needs to adjust to take transaction size into consideration asap. Delayed transactions due to insufficient fees can be a game breaker, if MSC are bought via DEX and the time window closes before the transaction confirms.

+1

and thanks Zathras for the explanation

what should be the process for this?  someone going to file an issue? i guess technically its not a master protocol thing, but it would need to be at least discussed in a special considerations area of the spec?

It's an interesting topic - receive lots of MP messages and you'll have lots of small ~0.00006 inputs - redeemimg them at the same time means a bigger transaction, thus requires more fees.

I actually think it's mostly about us devs being smart & creative about the way we craft our transactions to make sure we're spending these 0.00006 inputs where possible together with inputs from other transactions but at the same time not throwing around silly transactions with 50 tiny inputs.

[dexX7]

I think there are two ways:

You can calculate the transaction size by taking each subitem into consideration (e.g. public key, signature, outputs etc.) and determine the fee directly. All those numbers should be fixed, but I think this is probably a pain. Or you create the transaction, sign it, check the length and adjust the fee, if necessary, and sign again.

I could create a lookup-table for all included parts, if that would help? E.g. one uncompressed public key = 65 byte etc.

[zathras]

Well the thing is bitcoin uses heuristics to intelligently select coins to use intentionally to minimize the transaction weight on the network etc.  Masterchest does none of this and just follows the rules whether that results in an optimized transaction or not.  

Like a car from 50 years ago, it still gets from A to B but it could be more efficient.

I think the best way to come at it is to just go straight for proper adaptive transaction crafting and make Masterchest smart enough to handle some context - eg a couple examples from one of my git comments:

Can I craft this transaction in a way that leaves unspent inputs available for further Master Protocol transactions from this address same block?
Can I craft this transaction in a way that minimizes the fees payable by the user?
Can I craft this transaction in a way that consolidates some of the small reference outputs (.00006 etc)?

And so on...  After that we move from MP to Bitcoin considerations:

Can I craft this transaction in a way that minimizes transaction size on the network?

Etc Etc

Appreciate the offer but I think I'll be OK - after looking at this a bit more I actually don't think it's that cumbersome to look at increasing fee after initial transaction creation & signing - even if the fee does need to be increased (and the total input doesn't cover it), that's at worst two extra >dust inputs which would be about ~300 bytes so not enough to trigger another fee increase - that nullifies my concerns that we'd get into a loop adding inputs to cover fees which grows the tx to require more inputs thus more fees, which grows the tx and rinse repeat.  Worst case is we have to go back and change things once and resign, which isn't so bad.

Thanks
Zathras

[grazcoin]

Acceptance criteria:

• Minimum one PC wallet (for both Linux and Windows) which can generate simple sends and the buy/sell messages required for the distributed exchange, using agree-upon multisig format
• Minimum two websites parsing such messages, and the resulting balance transfers
• Minimum one website showing BTC/MSC price charts derived from these messages
• Minimum 10 days of real-world usage with no major problems
• High bar for usability. (Current heavy traders like maxmint, lishbtc, and buymastercoin should be happy with the final product, if at all possible)

As far as I see it - on 2014-03-25 04:54:22 GMT it will be exactly 10 days and the DEx prize can be distributed. Right?
It is still not fully clear how the Feb bounty and the DEx bounty are coordinated.
A short explanation would be great.

[ripper234]

Acceptance criteria:

• Minimum one PC wallet (for both Linux and Windows) which can generate simple sends and the buy/sell messages required for the distributed exchange, using agree-upon multisig format
• Minimum two websites parsing such messages, and the resulting balance transfers
• Minimum one website showing BTC/MSC price charts derived from these messages
• Minimum 10 days of real-world usage with no major problems
• High bar for usability. (Current heavy traders like maxmint, lishbtc, and buymastercoin should be happy with the final product, if at all possible)

As far as I see it - on 2014-03-25 04:54:22 GMT it will be exactly 10 days and the DEx prize can be distributed. Right?
It is still not fully clear how the Feb bounty and the DEx bounty are coordinated.
A short explanation would be great.

J.R can comment his perspective on this. Here is mine:

We have a bit of a mixup here. On the one hand, I do believe that the "High bar for usability" goal has not been met, and doubt it will be met this month.
On the other hand, we moved to paying out monthly bounties in February, and .

I hope it is clear that the DEx bounties paid in Feb, and the March monthly bounty, count towards the 300 BTC bounty (out of which 150 BTC has been previously paid).
So after March I don't think that a lot of money will remain in the original 300 BTC money pool (J.R needs to reply with some numbers).
I expect the bulk of the money remaining from the DEx to be divided in March's monthly payout.

Perhaps if there is only a tiny amount left after that, we will decide to increase March's payout and pay out the full remainder of the 300 BTC bounty this month, in order to avoid the accounting/judging headache. We will wait for J.R's number regarding February's payout (due any day now), and see what makes sense.