Is Mastercoin bloating the blockchain and what we can do about it?

[Peter Todd]

Hey Peter! Thanks for checking out the project! (Peter and I met at the conference in San Jose, and had some interesting discussions there, and he is a tireless advocate for keeping bitcoin awesome, as you can see from his sig)

I completely agree. We seem to have a rather formidable war-chest

Being a good block-chain citizen is a matter of survival for MasterCoin. If we don't, we'll face competition from a clone that says "We're like MasterCoin, but with less impact on the blockchain"

With any luck you'll face competition from clones that realize the entire idea is just dumb. Are you planning to release the sourcecode to MasterCoin? Because if you don't, we'll all just say how it's probably got hidden backdoors that are going to steal all your (master)coins, and if you do release it, we can just take that code and remove the silly requirement for the exodus address/MasterCoins. Or just stick with, who was it, TierNolan and co's colored coins stuff?

Also, even if you want to use the colored coins paradigm, you actually don't need any data in the blockchain at all beyond what appear to be bog-standard pubkeys. Even better, is that you can design your protocol to really make it censorproof by allowing the colored coins to be included in standard CoinJoin transactions in such a way that anyone looking at the blockchain data will have no way of knowing what outputs were the colored coins.

Frankly the only clever thing you've done is you've taken the same colored coins/side-chains ideas everyone else has and found a way to get Bitcoin users to shovel money at you for it. Too bad really - I guess the rest of us couldn't imagine how you didn't even need to write any code to separate $300k worth of Bitcoins from the idiots who used to own them. (We forgot about The Pirate already!) Hopefully that's not actually true, and it's actually your own money and not many people are getting scammed here.

Anyway, I have better things to do than waste my time talking about ideas sufficiently stupid to be indistinguishable from scams for free, so don't bother replying unless you've got a paying contract. (1.5BTC/hr, escrow required for you)

I will say though, you're so close to doing something useful for me: showing the world how little protection we actually have against the UTXO set being bloated other than the blocksize limit. Do me a favor and keep your stupid protocol just the way it is.

[bpd]

Hey Peter! Thanks for checking out the project! (Peter and I met at the conference in San Jose, and had some interesting discussions there, and he is a tireless advocate for keeping bitcoin awesome, as you can see from his sig)

I completely agree. We seem to have a rather formidable war-chest

Being a good block-chain citizen is a matter of survival for MasterCoin. If we don't, we'll face competition from a clone that says "We're like MasterCoin, but with less impact on the blockchain"

With any luck you'll face competition from clones that realize the entire idea is just dumb. Are you planning to release the sourcecode to MasterCoin? Because if you don't, we'll all just say how it's probably got hidden backdoors that are going to steal all your (master)coins, and if you do release it, we can just take that code and remove the silly requirement for the exodus address/MasterCoins. Or just stick with, who was it, TierNolan and co's colored coins stuff?

Also, even if you want to use the colored coins paradigm, you actually don't need any data in the blockchain at all beyond what appear to be bog-standard pubkeys. Even better, is that you can design your protocol to really make it censorproof by allowing the colored coins to be included in standard CoinJoin transactions in such a way that anyone looking at the blockchain data will have no way of knowing what outputs were the colored coins.

Frankly the only clever thing you've done is you've taken the same colored coins/side-chains ideas everyone else has and found a way to get Bitcoin users to shovel money at you for it. Too bad really - I guess the rest of us couldn't imagine how you didn't even need to write any code to separate $300k worth of Bitcoins from the idiots who used to own them. (We forgot about The Pirate already!) Hopefully that's not actually true, and it's actually your own money and not many people are getting scammed here.

Anyway, I have better things to do than waste my time talking about ideas sufficiently stupid to be indistinguishable from scams for free, so don't bother replying unless you've got a paying contract. (1.5BTC/hr, escrow required for you)

I will say though, you're so close to doing something useful for me: showing the world how little protection we actually have against the UTXO set being bloated other than the blocksize limit. Do me a favor and keep your stupid protocol just the way it is.

Thanks for saying what many are thinking. Even if it isn't a scam, the system simply can't work. It shows complete lack of understanding about how markets and asset pricing actually work. Gold assets not backed by gold, but by MasterCoins (which will clearly appreciate faster than dumb old gold)... It's so obvious... Why didn't the people who started GLD think of that?  They'd be so much richer!

[lishbtc]

/sigh @ how many times supposedly intelligent people have thrown around the term 'scam' in this thread - get over yourselves.

Criticize the design to your hearts content, healthy debate is fine but don't go after a guy who has simply solicited investment for an idea - his personal identity is well known (I understand some of you have met him) and he has done nothing to warrant calling scam.

I wonder how economic development would have progressed historically if everyone thought onboarding investors was such a tangible evil.

[killerstorm]

With any luck you'll face competition from clones that realize the entire idea is just dumb. Are you planning to release the sourcecode to MasterCoin? Because if you don't, we'll all just say how it's probably got hidden backdoors that are going to steal all your (master)coins, and if you do release it, we can just take that code and remove the silly requirement for the exodus address/MasterCoins.

They say MasterCoin has an advantageover copycats; just like Bitcoin has advantages over "alt-coins".

Or just stick with, who was it, TierNolan and co's colored coins stuff?

Also, even if you want to use the colored coins paradigm, you actually don't need any data in the blockchain at all beyond what appear to be bog-standard pubkeys.

MasterCoin and "colored coins" use different approaches and implement different features.

Within 'colored coins' approach we use Bitcoin transaction graph, and state changes are localized in the sense that one transaction cannot affect another transaction which doesn't depend on it.

MasterCoin's approach is different: there is a global state, and MasterCoin transactions are processed sequentially, in the order they appear in the blockchain, and the modify the global state. This is absolutely necessary because many MasterCoin features absolutely require global state.

J.R.W. believes that these features are very important, and for this reason colored coins are not suitable for what he is trying to do.

These features, their importance and possibility to accomplish same goals though other means are out of scope of this discussion. (I'll just note that it is possible to implement arbitrarily complex rules through 'oracle' or 'operator' of some sort, then the only concern is trustworthiness of that oracle and possibility of recovery from malfunction.)

Anyway, the MasterCoin's paradigm is: there is a list of messages (commands) which are reliably timestamped (i.e. all actors agree on their order) and carry authorization information (are signed with corresponding private key); messages are processed in order and each one affects the global state.

It's quite obvious that the Bitcoin blockchain is used for two things here:

1. Message timestamping, i.e. blockchain ensures that everybody sees these messages in same order.
2. Message storage and distribution.

It is possible to decouple these things: use Bitcoin blockchain for timestapming, but keep messages in some other place.

In the idea case (if our goal is to minimize the bloat) it's possible to keep only root hash in the Bitcoin blockchain (one hash per block), and messages themselves can go either to some alt-chain, or to DHT, or whatever.

Obviously, such a change will affect many aspects, so it won't be exactly same as the current form of MasterCoin, but all important features will be preserved.

I really doubt J.R. is willing to organize it in such a way, as he wants to make it possible to use ordinary Bitcoin client to send MasterCoin commands.

There is a quick and dirty way to modify it: move to some merged-mined alt-coin, as is. Most Bitcoin clients can be adapted to work with an alt-coin.

However, J.R. believes that MasterCoin is beneficial for Bitcoin, so there goes an opportunity...

Even better, is that you can design your protocol to really make it censorproof by allowing the colored coins to be included in standard CoinJoin transactions in such a way that anyone looking at the blockchain data will have no way of knowing what outputs were the colored coins.

Eh? How do colored coin clients know which outputs were the colored coins then? They sort of need it to show balance etc.

[Infinum]

Being a good block-chain citizen is a matter of survival for MasterCoin. If we don't, we'll face competition from a clone that says "We're like MasterCoin, but with less impact on the blockchain"

With any luck you'll face competition from clones that realize the entire idea is just dumb. (...)

Frankly the only clever thing you've done is you've taken the same colored coins/side-chains ideas everyone else has and found a way to get Bitcoin users to shovel money at you for it. Too bad really - I guess the rest of us couldn't imagine how you didn't even need to write any code to separate $300k worth of Bitcoins from the idiots who used to own them. (We forgot about The Pirate already!) Hopefully that's not actually true, and it's actually your own money and not many people are getting scammed here.

Anyway, I have better things to do than waste my time talking about ideas sufficiently stupid to be indistinguishable from scams for free, so don't bother replying unless you've got a paying contract. (1.5BTC/hr, escrow required for you)

I will say though, you're so close to doing something useful for me: showing the world how little protection we actually have against the UTXO set being bloated other than the blocksize limit. Do me a favor and keep your stupid protocol just the way it is.

Thanks for saying what many are thinking. Even if it isn't a scam, the system simply can't work. It shows complete lack of understanding about how markets and asset pricing actually work. Gold assets not backed by gold, but by MasterCoins (which will clearly appreciate faster than dumb old gold)... It's so obvious... Why didn't the people who started GLD think of that?  They'd be so much richer!

I hear you. I have read the specs for this ponzi coin and it just does not add up. People who sent their BTC to the Exodus node address will not see those BTCs ever again (perfect name BTW) - this meta-coin is parasitic and scammy.

As a full blockchain node operator I really want to see some kind of solution to the blockchain bloat problem, preferably with some fees so I can turn this into a business rather than a non-profit. Miners verify transactions and get compensation for it, but nobody pays for the transaction's storage once it gets verified and this will eventually shrink full-nodes numbers and further strengthen the oligopoly of Bitcoin miners.

[TierNolan]

As a full blockchain node operator I really want to see some kind of solution to the blockchain bloat problem, preferably with some fees so I can turn this into a business rather than a non-profit. Miners verify transactions and get compensation for it, but nobody pays for the transaction's storage once it gets verified and this will eventually shrink full-nodes numbers and further strengthen the oligopoly of Bitcoin miners.

The advantage of storage is that you can verify more transactions.  However, that is not much good if other miners drop your block because they dropped the UTXO.

The only "market" solutions are something like UTXO "rental" or just add a hard limit for the rate at which the set can increase per block.

Rental could be implemented by having a system where unspent UTXOs are dropped from memory after a while.  Each block a UTXO could have its output drop by <some value>.  Once it drops below zero, it is dropped from the in memory set.  This wouldn't actually cause the output to be less valuable, as long as you spent it before it was dropped it would be ok.  Spending old transactions would require adding info for fast loading.  For example, the tx message would have to also include info to allow fast proving that the UTXO is valid.

This is something that have a merkle tree for the UTXO set would be helpful for.  You give the path to prove the UTXO is valid and then it doesn't need to be contained in memory.

[killerstorm]

The advantage of storage is that you can verify more transactions.  However, that is not much good if other miners drop your block because they dropped the UTXO.

If you have 1 MB block size limit and each input is approximately 250 bytes, you only have at most 4000 inputs per block.

Thus you only have to buy hardware which can do ~400 random reads per second if you want to spend no more than 10 seconds to verify a block.

I'm fairly certain you can do this with 4 ordinary HDDs if they can do reads in parallel.

Or you can buy a SSD, pretty much any device can do more than 10000 random reads per second.

As for inputs in transactions not included in blocks, you can just drop ones you cannot fetch in a reasonable time. There is nothing wrong with dropping transactions.

[TierNolan]

If you have 1 MB block size limit and each input is approximately 250 bytes, you only have at most 4000 inputs per block.

Well, the size limit (at least the 1MB version) is not likely to last indefinitely.

The block chain would have to be saved with a flag to indicate if the output was consumed.  This flag could be set when the tx is burred enough that a reversal is unlikely.

However, it does reduce the overall performance of the system.

As for inputs in transactions not included in blocks, you can just drop ones you cannot fetch in a reasonable time. There is nothing wrong with dropping transactions.

That is part of what I meant.  If a tx message could have meta-data attached, then at least you could prove that the tx output actually exists.  This acts as a way to prevent spam.

Keeping only recent UTXO set elements in RAM and the rest on disk would be the end result.

If it is harder to spend "old" transaction outputs, then that weakens the ability of the system to act as a long term store of value.

Bloom filtering could be used.  If every node had a different random seed for the hash function, then producing the false positive that propagates would be hard.

[killerstorm]

If you have 1 MB block size limit and each input is approximately 250 bytes, you only have at most 4000 inputs per block.

Well, the size limit (at least the 1MB version) is not likely to last indefinitely.

My point is that we can associate performance requirements with block size rather than with UTXO set size.

If you think about it, it makes more sense: block size limit is the part of the protocol. It is something which can be controlled, and is being controlled now. But UTXO set size might be as high as 21*10^14, and controlling it is rather problematic and non-trivial.

So it is possible to design software in such a way that block size limit will dictate hardware requirements for full nodes. If there is a need to increase throughput, block size limit will be increased and hardware requirements will be adjusted accordingly.

So you can just tell full nodes maintainers which hardware they need to keep up with blockchain growth. It will give them peace of mind. Isn't that nice?

However, it does reduce the overall performance of the system.
If it is harder to spend "old" transaction outputs, then that weakens the ability of the system to act as a long term store of value.

You can prioritize them by value. E.g. if size of 1 UTXO  in memory is 100 bytes, storing 21 million entries requires only 2.1 GB, and thus you can easily guarantee that all UTXOs which have at least 1 Bitcoin in them will never be flushed away.

[drawingthesun]

Maybe this has already been addressed I am not sure, what if we could prove that a output in the blockchain is unspendable (a transaction to a valid but unusable address) if we know for certain that the coin is unspendable could they not be removed in the future when we start compressing the blockchain?

It would make sense that coins that can not move and this is known to be true are simply removed and those addresses removed too. Because they are no longer spendable Bitcoins. You could even claim these coins after they have sat in their addresses for x amount of time and move them as tx fees into the next block to be mined.

Of course this can only be done for addresses that are proven to have no possible private key access. If mastercoin uses valid and usable addresses but deletes the private key then we can't use this technique because that would be theft.

[spiral_mind]

/sigh @ how many times supposedly intelligent people have thrown around the term 'scam' in this thread - get over yourselves.

Criticize the design to your hearts content, healthy debate is fine but don't go after a guy who has simply solicited investment for an idea - his personal identity is well known (I understand some of you have met him) and he has done nothing to warrant calling scam.

I wonder how economic development would have progressed historically if everyone thought onboarding investors was such a tangible evil.

How about the fact that his project is illegal in multiple ways?

He's directly issuing a virtual currency in exchange for real money. Sound familiar? http://www.theguardian.com/world/2013/may/28/liberty-reserve-arthur-budovsky-arrested-spain

There's a reason Satoshi Nakamoto didn't reveal his true identity.

If you take investors money for an asset that is actually illegal in the first place, will the SEC prosecute you for that too? Likely yes, especially if you've raised as much as Mr. Willet has.

So really how can you call this anything other than a scam? If he doesn't flee with the money to another country he's going to jail eventually.

[TierNolan]

Maybe this has already been addressed I am not sure, what if we could prove that a output in the blockchain is unspendable (a transaction to a valid but unusable address) if we know for certain that the coin is unspendable could they not be removed in the future when we start compressing the blockchain?

That is what OP_RETURN is for.  If you pay to an output containing OP_RETURN, then it cannot be spent. 

Clients can then drop that output from the set.

The problem here is that to keep things simple, they are using standard transactions.  They are indistinguishable from "real" outputs, by design.

It would make sense that coins that can not move and this is known to be true are simply removed and those addresses removed too. Because they are no longer spendable Bitcoins. You could even claim these coins after they have sat in their addresses for x amount of time and move them as tx fees into the next block to be mined.

If they are "destroyed", then the value is effectively paid to everyone.

Of course this can only be done for addresses that are proven to have no possible private key access.

They pay to the output of a hash function.  There is no way to figure out which outputs are valid and which aren't.

The standard script is effectively "Pay to the public key that hashes to this value <20 byte hash>". 

You spend it by saying "here is the public key and here is the signature using that public key".  The public key is hashed to make sure it is the right hash and then the signature is checked.

Instead of a hash, mastercoin replaces it with a "data" packet.

In theory, the reference client could detect mastercoin operations and delay the data packet transactions by more than 6 blocks from the initiation tx.  That would break the protocol, since they are only valid if all the packets are within 6 blocks of each other.

If there is a false positive, then it would delay the transaction by an hour.

Another option would be a flag in the various wallets that marks a payment address as invalid.  Outputs that pay to OP_RETURN <standard script> would also be considered standard.

[jl2012]

ATM, The UTXO set is 239,681,979 bytes out of 10,618,084,349 bytes, or 2.26% of the total blockchain

I have proposed it for many times: set a hard limit for the growth of UTXO (e.g. 1% of max block size, i.e. 10kB) per block, so the UTXO set will grow at most 525MB per year. This will encourage people to merge dust outputs, and punish any UTXO-bloating activities. At the same time, we could make OP_RETURN as standard output and not count it as UTXO growth.

[standardcrypto]

There are plenty of ways to accomplish that, the easiest of which that works today would be to stuff data into unused public keys of an OP_CHECKMULTISIG transaction.

In case it helps other readers of the thread... If I am understanding correctly:
  1)  the idea here is that you can use something that looks like a public key, but which you don't actually have a private key that hashes right for it, since you don't need that particular private key to spend as long as you have private keys for other addresses.
  2) any 64 bytes of entropy can be used as a public key, as long as you don't need the private key

Please correct me if I'm wrong.

https://en.bitcoin.it/wiki/Technical_background_of_Bitcoin_addresses

I'm still fuzzy on how this could actually be implemented.

How many public keys are currently usable for m of n signatures? Are OP_CHECKMULTISIG transactions "official" and generally relayed by miners, or are these still in a grey area? I did google a bit, but I couldn't figure out where to look to determine what transactions can generally expect to be supported. Obviously it's important to have a good grasp on this for people steering mastercoin.

[StarenseN]

* StarenseN following

[ofer]

I suggest that instead of sending a micro payment to the exodus address, we can enforce sending a bigger amount.
This extra money can be really significant if Mastercoin will gain popularity (if not the spam will be irrelevant).
The extra money can be used to support the Bitcoin foundation (like R&D for scaling up the block-chain, etc.).

[btcdrak]

I'm totally cool with stuffing the data in alternate places once we have our own client. I see using unspendable transactions as a "crutch" so that the protocol works until we get to that point. For instance, right now none of the existing bitcoin clients has functionality that will let an average user hide data in OP_CHECKMULTISIG.

It is inevitable that in the future, bit coin does prevent the permanent storage of arbitrary data in the blockchain. If you rely on such features now, there could be colossal consequences later. #bitcoin wont care about you or your bitcoin-bloat. BE WARE, YOU HAVE BEEN WARNED.

[dacoinminster]

I'm totally cool with stuffing the data in alternate places once we have our own client. I see using unspendable transactions as a "crutch" so that the protocol works until we get to that point. For instance, right now none of the existing bitcoin clients has functionality that will let an average user hide data in OP_CHECKMULTISIG.

It is inevitable that in the future, bit coin does prevent the permanent storage of arbitrary data in the blockchain. If you rely on such features now, there could be colossal consequences later. #bitcoin wont care about you or your bitcoin-bloat. BE WARE, YOU HAVE BEEN WARNED.

MasterCoin hopes to be symbiotic with bitcoin, not parasitic, and we plan to morph and change to whatever degree is needed as bitcoin morphs and changes. If our old data gets pruned out of the block-chain used by most bitcoin clients, our clients will have to be modified to keep different parts of the block chain, and use different pruning techniques. Nothing I can foresee leaves MasterCoin completely stranded or unusable.

[jl2012]

I'm totally cool with stuffing the data in alternate places once we have our own client. I see using unspendable transactions as a "crutch" so that the protocol works until we get to that point. For instance, right now none of the existing bitcoin clients has functionality that will let an average user hide data in OP_CHECKMULTISIG.

It is inevitable that in the future, bit coin does prevent the permanent storage of arbitrary data in the blockchain. If you rely on such features now, there could be colossal consequences later. #bitcoin wont care about you or your bitcoin-bloat. BE WARE, YOU HAVE BEEN WARNED.

MasterCoin hopes to be symbiotic with bitcoin, not parasitic, and we plan to morph and change to whatever degree is needed as bitcoin morphs and changes. If our old data gets pruned out of the block-chain used by most bitcoin clients, our clients will have to be modified to keep different parts of the block chain, and use different pruning techniques. Nothing I can foresee leaves MasterCoin completely stranded or unusable.

I won't be surprised. Parasites always morph and change as the host evolves.

[dacoinminster]

I won't be surprised. Parasites always morph and change as the host evolves.

Yes, that's something like what I have in mind, although long-term, we hope to be mitochondria, not tape worms