Is Mastercoin bloating the blockchain and what we can do about it?

[ripper234]

Let's create a healthy, constructive discussion here about the question whether Mastercoin "bloats" the blockchain, and what are some good technical solutions to that.

Please stop!

You are creating enormous amounts of  nonredeemable UTXO.  This kind of usage undermines the scalability of Bitcoin because you are pushing data storage into the UTXO set and not merely into the transaction history. The UTXO set must be rapidly available for validation and can not be pruned unlike the rest of the blockchain.  Currently pruning yields 50:1 compression (and improving) of the space required to run a full node. The addition of never-redeemable outputs undermines that.

Do you believe that "Bitcoin" cares about how people use it?
Bitcoin scalability is a definite challenge. We should tackle that challenge as early as possible, instead of nicely asking people to only use it in specific ways.

Willet is open to discuss better, more efficient ways to encode Mastercoin transactions into the blockchain.

gmaxwell, do you belong to the school of thinking that categorizes SatoshiDice as spam?

[1715233575]

1715233575

[1715233575]

1715233575

[1715233575]

1715233575

[Gavin Andresen]

It bloats the UTXO set, which is bad.

MasterCoin transactions should all be spendable or provably prune-able. There are plenty of ways to accomplish that, the easiest of which that works today would be to stuff data into unused public keys of an OP_CHECKMULTISIG transaction.

[maaku]

Or prefix the data output with OP_RETURN, which is guaranteed to be prunable. If the output is never meant to be spent, then it is irresponsible not to do this.

[gmaxwell]

Creating an altcoin system which stores its data in blockchain is parasitic and increases the cost for non-consenting Bitcoin uses— and I think that is not great, and, I think it's pretty objectively clear that it bloats the blockchain, but that is not what I was complaining about.

Nor do I think we needed _yet another thread_ to help pump the name of this latest altcoin. This will be my one and only post in this thread and I hope other people decline to help give free PR here by continuing to post in this thread and giving them a free controversy to spin.

The issue here is that that mastercoin stores data in Bitcoin by creating perpetually unredeemable transaction outputs. It doesn't just store data in the blockchain, it stores it in the UTXO set and the blockchain. The former is a more precious resource than the latter. This kind of behavior, if wide spread, changes the asymptotic fast storage requirements for a full validating node from something close to constant (just some growth due to forever lost Bitcoins which all users are paid for via deflation) to perpetually growing.  The ability to for a full node to forget old transaction data would currently let you run one with about 2% of the storage that the blockchain takes up, but these outputs cannot be spent so they cannot be forgotten.

instead of nicely asking people

I am reasonably confident that you would prefer I defend Bitcoin by nicely asking as opposed to the alternatives.

Do you believe that "Bitcoin" cares about how people use it?

No more than the door of your home cares when some thief walks through it when you've left it unlocked.  The technically possibility of something does not equal moral authority, nor does the possibility of doing something now guarantee that that possibility will remain in the future— especially if it causes harm to others, like taking up their resource for private use. Just because it's possible doesn't mean its right, nor does it mean that its wise.

I certainly believe the users of Bitcoin care and will take action to defend Bitcoin from use which is harmful to it. Some time back I'd made a minor technical proposal which would completely block data-storage txouts with no impact to normal Bitcoin usage.  Adopting it would require switching people to a new address style for new transactions, so it's not something that would be adopted quickly if the Bitcoin userbase were to decide to go down that route, but it certainly could be done.  I tend to think asking nicely is preferable, however.

Willet is open to discuss better, more efficient ways to encode Mastercoin transactions into the blockchain.

I have better things to do with my time than to provide free consulting for altcoins who've just raised hundreds of thousands of dollars worth of investments in their ill considered schemes. The concerns here are not new, the impacts of unspendable outputs are not mysterous to people who have even half a technical clue about the Bitcoin system, and the solutions are not subtle.

My preference is that you do what hundreds of other altcoins have done and not store your data inside Bitcoin (but feel free to merge-mine if you need a consensus system), don't non-consensually consume the resources of people who are disinterested— even opposed— to your system... but failing that, see what all the other people piling on are saying about keeping data out of the UTXO set as a bare minimum.

[Rampion]

Unfortunately not everybody will be willing to use the Bitcoin blockchain as we/the devs would like. If someone creates something that bloats the UTXO set, they might just do not care about what the devs think about it.

How we solve THAT problem?

[freedomno1]

How we solve THAT problem?

Off-Chain transactions comes to mind but may only be a part of the total solution
https://bitcointalk.org/index.php?topic=152334.0

https://bitcointalk.org/index.php?topic=274711.msg2949016#msg2949016

@gmaxwell
Sorry to add towards the posting just replying now while it is still fresh.

EDIT:
After reading the bitangels response on the Mastercoin thread and the devs reply I am on the fence with either implementation this needs more discussion since both sides seem to have validated points.
That said discussion has started, which is encouraging.

[murraypaul]

Unfortunately not everybody will be willing to use the Bitcoin blockchain as we/the devs would like. If someone creates something that bloats the UTXO set, they might just do not care about what the devs think about it.

How we solve THAT problem?

Identify those transactions and reject them as invalid.
Do not store or relay them.

[d'aniel]

@dacoinminster, if you're still intending to give me those 3 BTC for pointing out that problem with your proposal, could you please send them to 1GMaxweLLbo8mdXvnnC19Wt2wigiYUKgEB

@gmaxwell, thanks for your efforts.

[TierNolan]

How we solve THAT problem?

There was a suggestion to directly limit the UTXO set.  For example, a block is invalid if the number of outputs is more than <some number> greater than the total number of inputs.

An output that pays to OP_RETURN wouldn't count as an output.

What about a weaker rule, if there is a tie between 2 blocks, break the tie in favor of the one with lower (total output - total inputs).

If a node receives a block and it would result in a small UTXO set, it still forward it and mine against it, displacing the earlier received block.

This would only affect blocks when orphans would happen, but does create an incentive to favor transactions which reduce the UTXO set.

A block which combines lots of dust into a single output would have an advantage, so miners would include those for free.

[lishbtc]

Whilst getting consensus on this topic is unlikely, I would think it's clear JRW is not just spitting out another 'altcoin' and would look at any suggestions to mitigate blockchain bloat - Mastercoin is supposed to complement Bitcoin after all.

Given Mastercoin is an additional layer of functionality that sits on top of Bitcoin, it therefore needs Bitcoin to succeed and as such it would make no sense to make design choices that negatively impact Bitcoin.

Shame to see the hostility (I've always found collaborating to be much more rewarding) but I'm sure with the quarter million plus $ raised the project will be able to engage the right minds to find the best implementation.

[rbdrbd]

Given Mastercoin is an additional layer of functionality that sits on top of Bitcoin, it therefore needs Bitcoin to succeed and as such it would make no sense to make design choices that negatively impact Bitcoin.

Shame to see the hostility (I've always found collaborating to be much more rewarding) but I'm sure with the quarter million plus $ raised the project will be able to engage the right minds to find the best implementation.

+1. This was going to happen sooner or later. There's way too much brainpower on these forums to not have a rational storage implementation come out of this effort.

[dacoinminster]

I'm totally cool with stuffing the data in alternate places once we have our own client. I see using unspendable transactions as a "crutch" so that the protocol works until we get to that point. For instance, right now none of the existing bitcoin clients has functionality that will let an average user hide data in OP_CHECKMULTISIG.

I expect once we have alternate ways of storing the data, we'll support both methods for awhile, then switch to the more efficient one at some pre-determined block number.

MasterCoin wants to be a good block-chain citizen in the long run, but we are willing to put that off in the short run in favor of simplicity and immediate usability. Also, one of my design goals was to be "censorship resistant", meaning it would be really hard for anybody to tweak the bitcoin protocol such that MasterCoin no longer works. Any pruning scheme will need to be similarly robust.

Thanks for your input!

[dacoinminster]

@dacoinminster, if you're still intending to give me those 3 BTC for pointing out that problem with your proposal, could you please send them to 1GMaxweLLbo8mdXvnnC19Wt2wigiYUKgEB

@gmaxwell, thanks for your efforts.

Sure. I've got to run a small list of expenditures by the oversight board next week, which includes the bounty that you and bytemaster earned. Thanks!

[dacoinminster]

Relevant PM:

Hi JR, thanks for your work on Mastercoin so far. I am about to be investing ~50BTC...just finishing up my due diligence.

Regarding your post at: https://bitcointalk.org/index.php?topic=284178.msg3043044#msg3043044

So the plan is to fork a full client (I'd think bitcoin QT or Armory ...the latter you've forked from for your test implementation) and add all the Mastercoin functionality to that? How much effort is adding the ability to store the data in OP_CHECKMULTISIG ?? (Sorry, I'm a software developer by trade, but am not familiar with the bitcoin codebase).

If it's not a ton of work, why not plan to do this from the start? That would avoid issues with the bitcoin community at large around this (especially if you have folks like Gavin saying that it should be avoided). Having the bitcoin devs irked from this initiative from the start won't bode well... :/

-----

Frankly, I don't know how hard this will be yet. I believe Gavin when he says there are multiple ways to do it, but I don't know what all of them are or which way is best.

I do plan to use bounty money to encourage innovation in this area early on.

The most important thing to me is not to be dependent on the charity and goodwill of people running the higher protocol layer. They are great people, but I don't want to give them a lever to turn off MasterCoin, however well-intentioned they may be.

edit: I accidentally said "higher protocol layer" in this PM, but I meant bitcoin, the lower protocol layer.

[killerstorm]

Hi JR, thanks for your work on Mastercoin so far. I am about to be investing ~50BTC...just finishing up my due diligence.

Regarding your post at: https://bitcointalk.org/index.php?topic=284178.msg3043044#msg3043044

So the plan is to fork a full client (I'd think bitcoin QT or Armory ...the latter you've forked from for your test implementation) and add all the Mastercoin functionality to that? How much effort is adding the ability to store the data in OP_CHECKMULTISIG ?? (Sorry, I'm a software developer by trade, but am not familiar with the bitcoin codebase).

If it's not a ton of work, why not plan to do this from the start? That would avoid issues with the bitcoin community at large around this (especially if you have folks like Gavin saying that it should be avoided). Having the bitcoin devs irked from this initiative from the start won't bode well... :/

-----

Frankly, I don't know how hard this will be yet.

I believe no more than a couple of days of work, if you know what are you doing. Likely less than that for a person who is very familiar with the client codebase, data structures etc.

So, yes, it makes sense to do this from the start.

[dacoinminster]

I believe no more than a couple of days of work, if you know what are you doing. Likely less than that for a person who is very familiar with the client codebase, data structures etc.

So, yes, it makes sense to do this from the start.

Agree it should be easy on one client, but less so for people who are currently using blockchain.info for MasterCoin. I don't want to leave them hanging.

I look forward to seeing what people come up with for hiding this data in the chain in more efficient ways

[maaku]

JR, put OP_RETURN in front of the data script. That can't take more than a few seconds to change, and a few minutes to communicate. You may even be able to use dust/empty outputs in this case (you should, but not sure if the client supports it yet).

[dacoinminster]

JR, put OP_RETURN in front of the data script. That can't take more than a few seconds to change, and a few minutes to communicate. You may even be able to use dust/empty outputs in this case (you should, but not sure if the client supports it yet).

I think you are assuming that we have code which directly sends MasterCoin transactions, which we do not. Sends are currently accomplished using standard unmodified bitcoin clients (bitcoin-qt, blockchain.info, etc)

[Peter Todd]

JR, put OP_RETURN in front of the data script. That can't take more than a few seconds to change, and a few minutes to communicate. You may even be able to use dust/empty outputs in this case (you should, but not sure if the client supports it yet).

I think you are assuming that we have code which directly sends MasterCoin transactions, which we do not. Sends are currently accomplished using standard unmodified bitcoin clients (bitcoin-qt, blockchain.info, etc)

Given how much money you've raised it'd be quite easy to pay someone to add support for OP_RETURN to those clients, among other things.

[dacoinminster]

Given how much money you've raised it'd be quite easy to pay someone to add support for OP_RETURN to those clients, among other things.

Hey Peter! Thanks for checking out the project! (Peter and I met at the conference in San Jose, and had some interesting discussions there, and he is a tireless advocate for keeping bitcoin awesome, as you can see from his sig)

I completely agree. We seem to have a rather formidable war-chest

Being a good block-chain citizen is a matter of survival for MasterCoin. If we don't, we'll face competition from a clone that says "We're like MasterCoin, but with less impact on the blockchain"