MasterCoin: New Protocol Layer Starting From “The Exodus Address”

[Peter Todd]

Note that http://eligius.st/~wizkid057/newstats/pushtxn.php will push a non-standard, fee-bearing transaction into Eligius-mined blocks.

Not anymore: Eligius won't mine anything that looks like it's storing data these days. I'd have to check with Luke, but I think the rules are currently that only IsStandard() scriptPubKey's will be mined, although P2SH scriptSigs can be non-standard.

Thus there is no requirement to fit the mold of a "standard" transaction that is relayed by most.

Typically the development process looks like

• Design the best transaction format
• Write software, prove it works on testnet
• Test on mainnet through manual miner submission, like the Eligius URL above
• Now you have a proven use case, and time has passed proving that your concept remains interesting to some user base somewhere
• Submit a patch to bitcoin/bitcoin.git, adding that transaction as a standard transaction

Some of these steps may be done in parallel, of course.

Frankly I think that's awful advice in the case of Mastercoin and similar systems, precisely because they can get away with using standard transaction to achieve their goals.

Why risk having the reference client maintainers reject your transaction standard if you don't have to?

[ASICSRUS]

we're paying fees just like any other user of the blockchain - we'd like a miner to include our transaction and we thus pay them a fee to do so.

Exactly.  I am baffled as to why the bitcoin foundation wants to limit use.  Who cares if Mastercoin ends up doing a million transactions per second - we pay the same as anyone else.  Are they admitting that the blockchain cannot support high volume use? - then bitcoin is doomed.  

hijacking the blockchain ~green light from Gavin?lol hmmm i wonder how many copy cats are going to jump at the chance to be the next mastercoin?  

[jgarzik]

Frankly I think that's awful advice in the case of Mastercoin and similar systems, precisely because they can get away with using standard transaction to achieve their goals.

Why risk having the reference client maintainers reject your transaction standard if you don't have to?

That presumes additional checks on pubkeys are not coming down the pipe, which is also awful presumption / advice.

[Peter Todd]

Frankly I think that's awful advice in the case of Mastercoin and similar systems, precisely because they can get away with using standard transaction to achieve their goals.

Why risk having the reference client maintainers reject your transaction standard if you don't have to?

That presumes additional checks on pubkeys are not coming down the pipe, which is also awful presumption / advice.

I was explaining earlier in this thread how to turn arbitrary data into into valid pubkeys to get around that issue.

In addition Mastercoin would be very smart to retain (encrypted) data embedding in pay-to-(script|pubkey)-hash outputs as a backup, and in addition to that, make it possible to embed them in P2SH scriptSigs.

[prophetx]

Frankly I think that's awful advice in the case of Mastercoin and similar systems, precisely because they can get away with using standard transaction to achieve their goals.

Why risk having the reference client maintainers reject your transaction standard if you don't have to?

That presumes additional checks on pubkeys are not coming down the pipe, which is also awful presumption / advice.

Basically what this means, to me, is that the Mastercoin Foundation would need to ensure that there is at least one developer interfacing with the core bitcoin development process.  Sort of like a lobbyist...it should not be left to chance encounters on this forum.

Anyway like salesforce has appexchange, facebook has fb apps, and apple has appstore, bitcoin will need to be friendly to its "partner ecosystem" at least if it is to mushroom... not sure that is in the vision, or who is leading the vision on that.

Mastercoin is to bitcoin, as zynga is to facebook, as angrybirds is to iphone, etc. Btc holders and miners should be happy about vendor lock in lol.

[maxmint]

We have a small discrepancy between Masterchest and Mastercoin-explorer regarding this transaction:
https://blockchain.info/tx/32962c6254a8c48459ca9c729509272f02e291a326c43a106cfb108f00cd5570

http://mastercoin-explorer.com/transactions/32962c6254a8c48459ca9c729509272f02e291a326c43a106cfb108f00cd5570
https://masterchest.info/lookuptx.aspx?txid=32962c6254a8c48459ca9c729509272f02e291a326c43a106cfb108f00cd5570

It looks like the "To" address on Mastercoin-explorer (1N4R6K9MHmZQewYRdATUT849Gm2XSc2fZ2) actually is the data address.
The correct receiving address should be 1NADJn7sackXPKxS94DiB9geu3mS5Me1Uw – any idea what's going on here?

[Tachikoma]

We have a small discrepancy between Masterchest and Mastercoin-explorer regarding this transaction:
https://blockchain.info/tx/32962c6254a8c48459ca9c729509272f02e291a326c43a106cfb108f00cd5570

http://mastercoin-explorer.com/transactions/32962c6254a8c48459ca9c729509272f02e291a326c43a106cfb108f00cd5570
https://masterchest.info/lookuptx.aspx?txid=32962c6254a8c48459ca9c729509272f02e291a326c43a106cfb108f00cd5570

It looks like the "To" address on Mastercoin-explorer (1N4R6K9MHmZQewYRdATUT849Gm2XSc2fZ2) actually is the data address.
The correct receiving address should be 1NADJn7sackXPKxS94DiB9geu3mS5Me1Uw – any idea what's going on here?

Sorry I'm working on checking this out right now.

[mishax1]

So is this still a legit transfer ? Did the target address got the Mastercoins ?

[Tachikoma]

I think I'm doing something wrong with sequences so I would appreciate a second pair of eyes on this.

This transaction: https://blockchain.info/tx/32962c6254a8c48459ca9c729509272f02e291a326c43a106cfb108f00cd5570

According to Maximt the target address is 1NADJn7sackXPKxS94DiB9geu3mS5Me1Uw however this address has a sequence of 232 according to my code. The data address, 1N4R6K9MHmZQewYRdATUT849Gm2XSc2fZ2, has a sequence of 231 instead of 233.

In order to distinguish data packets from the reference address, the data packet sequence numbers start at n+1 where n is the sequence number of the reference packet if it were treated as a data packet.

According to this logic 1N4R6K9MHmZQewYRdATUT849Gm2XSc2fZ2 is the target and 1NADJn7sackXPKxS94DiB9geu3mS5Me1Uw the data packet. What am I doing wrong or is this an invalid transaction. How was this transaction created?

Edit: Looks like this is a problem with mastercoin-explorer, in all likely hood the transaction is correct.

[grazcoin]

I think I'm doing something wrong with sequences so I would appreciate a second pair of eyes on this.

This transaction: https://blockchain.info/tx/32962c6254a8c48459ca9c729509272f02e291a326c43a106cfb108f00cd5570

According to Maximt the target address is 1NADJn7sackXPKxS94DiB9geu3mS5Me1Uw however this address has a sequence of 232 according to my code. The data address, 1N4R6K9MHmZQewYRdATUT849Gm2XSc2fZ2, has a sequence of 231 instead of 233.

In order to distinguish data packets from the reference address, the data packet sequence numbers start at n+1 where n is the sequence number of the reference packet if it were treated as a data packet.

According to this logic 1N4R6K9MHmZQewYRdATUT849Gm2XSc2fZ2 is the target and 1NADJn7sackXPKxS94DiB9geu3mS5Me1Uw the data packet. What am I doing wrong or is this an invalid transaction. How was this transaction created?

In order to be in sync with the project (which is obviously required for the next contest), I have added parsing using Tachikoma's multisig and disabled mine.
You can use my API to help debugging, e.g.:
http://dev.masterchain.info/tx/28ffa93e3d3091f7c501585fd741327d3c58ef3f45bcd94f72166889798283cf.json will tell you that this transaction is invalid since there is ambiguous sequence [119, 119, 120]

[grazcoin]

I think I'm doing something wrong with sequences so I would appreciate a second pair of eyes on this.

This transaction: https://blockchain.info/tx/32962c6254a8c48459ca9c729509272f02e291a326c43a106cfb108f00cd5570

According to Maximt the target address is 1NADJn7sackXPKxS94DiB9geu3mS5Me1Uw however this address has a sequence of 232 according to my code. The data address, 1N4R6K9MHmZQewYRdATUT849Gm2XSc2fZ2, has a sequence of 231 instead of 233.

In order to distinguish data packets from the reference address, the data packet sequence numbers start at n+1 where n is the sequence number of the reference packet if it were treated as a data packet.

According to this logic 1N4R6K9MHmZQewYRdATUT849Gm2XSc2fZ2 is the target and 1NADJn7sackXPKxS94DiB9geu3mS5Me1Uw the data packet. What am I doing wrong or is this an invalid transaction. How was this transaction created?

In order to be in sync with the project (which is obviously required for the next contest), I have added parsing using Tachikoma's multisig and disabled mine.
You can use my API to help debugging, e.g.:
http://dev.masterchain.info/tx/28ffa93e3d3091f7c501585fd741327d3c58ef3f45bcd94f72166889798283cf.json will tell you that this transaction is invalid since there is ambiguous sequence [119, 119, 120]

and the tx you asked for is just fine:
http://masterchain.info/Transaction.html?tx=32962c6254a8c48459ca9c729509272f02e291a326c43a106cfb108f00cd5570&currency=MSC

[Tachikoma]

Found the problem, the official spec is still not updated. This makes it a lot harder for new developers (and developers who forget stuff...)

J.R. Could you please update the spec with all the new data packet order stuff and the new validation rules? (broken sequence, ambiguous sequence etc.)

Rolling out fix in the next few minutes.

Edit:

Transaction should be fixed.

I somehow missed part of the sequence discussion.

If there is a broken sequence (i.e. 3,4,8), then the odd-man-out is the change address (8 in this example)
If there is an ambiguous sequence (i.e. 3,4,4), then the transaction is invalid!
If there is a perfect sequence (i.e. 3,4,5), then the transaction is invalid!

Is this really necessary? We have no control over the sequence number for a change address, is it fair to disregard a transaction based on random luck? I would like to propose to do a sanity check before invalidating a transaction based on sequence. You could try to decode the address and see if the transaction_type and the other options based on that transaction_type make sense. If one of the addresses with the same sequence for instance does that, but an other does not you know which one is the data address and which one the target. If that also fails it seems safe to flag it as invalid but I think we should try harder before falling back to invalid.

[mishax1]

Great, thanks! 

[maxmint]

Rolling out fix in the next few minutes.

Edit:

Transaction should be fixed.

Thanks for the quick fix. It seems there are 2 other transactions showing wrong receiving addresses:

http://mastercoin-explorer.com/transactions/28ffa93e3d3091f7c501585fd741327d3c58ef3f45bcd94f72166889798283cf
https://masterchest.info/lookuptx.aspx?txid=28ffa93e3d3091f7c501585fd741327d3c58ef3f45bcd94f72166889798283cf

and

http://mastercoin-explorer.com/transactions/20832232e40cd39178b96b0196655ea056686ac5b898680531db417b7035b5cf
https://masterchest.info/lookuptx.aspx?txid=20832232e40cd39178b96b0196655ea056686ac5b898680531db417b7035b5cf

[Tachikoma]

Rolling out fix in the next few minutes.

Edit:

Transaction should be fixed.

Thanks for the quick fix. It seems there are 2 other transactions showing wrong receiving addresses:

http://mastercoin-explorer.com/transactions/28ffa93e3d3091f7c501585fd741327d3c58ef3f45bcd94f72166889798283cf
https://masterchest.info/lookuptx.aspx?txid=28ffa93e3d3091f7c501585fd741327d3c58ef3f45bcd94f72166889798283cf

and

http://mastercoin-explorer.com/transactions/20832232e40cd39178b96b0196655ea056686ac5b898680531db417b7035b5cf
https://masterchest.info/lookuptx.aspx?txid=20832232e40cd39178b96b0196655ea056686ac5b898680531db417b7035b5cf

Thanks! Re-indexed. I will probably do a full re-index soon but I am currently scattering my attention between the library, the site, the wallet and the theory so it's all spread very thin. Sorry about these mixups.

I ran up 20,000 compressed pubkeys from random, then used the last byte rotation method to try and make them valid ECDSA points.  

Seems to have worked for all 20,000 - but my ECDSA point validity testing is being done with code from the Casascius Bitcoin Address Utility and there is a line in there that is making me wonder whether the validatePoint() and other check functions are giving me the whole picture on whether the keys are valid:

Code:

// todo: ensure X and Y are on the curve

You can find the results here.  Tachikoma, when you're back and have a bit of time would you mind taking the '*RAW' file and running some of those pubkeys through your Mastercoin::Util.valid_ecdsa_point? function and see what results you get?  They are the corrected keys with the last byte rotated and all 20,000 are supposed to be valid.

I've ran double the amount just to be sure but the longest sequence number I needed was 15 iterations in order to make it work. One more thing I want to check is not use random looking hex keys but random looking Mastercoin data since that data looks different the results might be different as well.

[grazcoin]

Tachikoma:
Just a minor order issue on http://mastercoin-explorer.com/
http://mastercoin-explorer.com/transactions/69a67d4840cb5e9ea75c8bc4ea6aeb0985d5dc1d710c8de7b0771360b6968f44 has position 176 and
http://mastercoin-explorer.com/transactions/766646269feb962c95dd38e163158b658aad0520cb5bcb4e0d2d01a868a5c2b6 has position 178
but you show them in the wrong order.

[grazcoin]

I've ran double the amount just to be sure but the longest sequence number I needed was 15 iterations in order to make it work. One more thing I want to check is not use random looking hex keys but random looking Mastercoin data since that data looks different the results might be different as well.

If we already run brute force iterations to make the pubkey valid, we could consider doing the same for the non-compressed pubkeys (MIP1) and have 2 outputs instead of 4, plus extra place for future required data. I didn't check how much more iteration that would take, but it would add much less contamination to the blockchain.

[Tachikoma]

I've ran double the amount just to be sure but the longest sequence number I needed was 15 iterations in order to make it work. One more thing I want to check is not use random looking hex keys but random looking Mastercoin data since that data looks different the results might be different as well.

If we already run brute force iterations to make the pubkey valid, we could consider doing the same for the non-compressed pubkeys (MIP1) and have 2 outputs instead of 4, plus extra place for future required data. I didn't check how much more iteration that would take, but it would add much less contamination to the blockchain.

Uncompressed public keys have an added problem, the x and y coords need to be correct in order to be on the same curve.

Tachikoma:
Just a minor order issue on http://mastercoin-explorer.com/
http://mastercoin-explorer.com/transactions/69a67d4840cb5e9ea75c8bc4ea6aeb0985d5dc1d710c8de7b0771360b6968f44 has position 176 and
http://mastercoin-explorer.com/transactions/766646269feb962c95dd38e163158b658aad0520cb5bcb4e0d2d01a868a5c2b6 has position 178
but you show them in the wrong order.

In the wrong order where exactly?

[dacoinminster]

I've updated our official contest thread with the rules for the next contest: https://bitcointalk.org/index.php?topic=292628.0

My intention is that thread should become our hub for development-related discussion, while this thread will remain for general MasterCoin discussion.

For instance, the discussion with dillpicklechips about MasterCoin's value would stay here, while just about anything by tachikoma, zathras, grazcoin, retep, and other devs would go in the other thread.

I won't be a jerk about it on this thread (I'll just give gentle reminders if needed), but I may be more of a jerk about it on the development thread, as I want to keep it focused on development.

I see a couple dev-related things (especially the stuff related to multi-sig) which I need to reply to. I'll do that shortly on the other thread.

Thanks!

[grazcoin]

Tachikoma:
Just a minor order issue on http://mastercoin-explorer.com/
http://mastercoin-explorer.com/transactions/69a67d4840cb5e9ea75c8bc4ea6aeb0985d5dc1d710c8de7b0771360b6968f44 has position 176 and
http://mastercoin-explorer.com/transactions/766646269feb962c95dd38e163158b658aad0520cb5bcb4e0d2d01a868a5c2b6 has position 178
but you show them in the wrong order.

In the wrong order where exactly?

On the main page http://mastercoin-explorer.com/  69a67d appears before 766646 but 69a67d is older.