Bitcoin Forum

Bitcoin => Project Development => Topic started by: ripper234 on August 30, 2013, 07:21:00 AM



Title: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: ripper234 on August 30, 2013, 07:21:00 AM
Let's create a healthy, constructive discussion here about the question whether Mastercoin "bloats" the blockchain, and what are some good technical solutions to that.

Please stop!

You are creating enormous amounts of  nonredeemable UTXO.  This kind of usage undermines the scalability of Bitcoin because you are pushing data storage into the UTXO set and not merely into the transaction history. The UTXO set must be rapidly available for validation and can not be pruned unlike the rest of the blockchain.  Currently pruning yields 50:1 compression (and improving) of the space required to run a full node. The addition of never-redeemable outputs undermines that.


Do you believe that "Bitcoin" cares about how people use it?
Bitcoin scalability is a definite challenge. We should tackle that challenge as early as possible, instead of nicely asking people to only use it in specific ways.

Willet is open to discuss better, more efficient ways to encode Mastercoin transactions into the blockchain.

gmaxwell, do you belong to the school of thinking that categorizes SatoshiDice as spam?


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: Gavin Andresen on August 30, 2013, 07:29:22 AM
It bloats the UTXO set, which is bad.

MasterCoin transactions should all be spendable or provably prune-able. There are plenty of ways to accomplish that, the easiest of which that works today would be to stuff data into unused public keys of an OP_CHECKMULTISIG transaction.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: maaku on August 30, 2013, 08:07:15 AM
Or prefix the data output with OP_RETURN, which is guaranteed to be prunable. If the output is never meant to be spent, then it is irresponsible not to do this.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: gmaxwell on August 30, 2013, 08:44:35 AM
Creating an altcoin system which stores its data in blockchain is parasitic and increases the cost for non-consenting Bitcoin uses— and I think that is not great, and, I think it's pretty objectively clear that it bloats the blockchain, but that is not what I was complaining about.

Nor do I think we needed _yet another thread_ to help pump the name of this latest altcoin. This will be my one and only post in this thread and I hope other people decline to help give free PR here by continuing to post in this thread and giving them a free controversy to spin.

The issue here is that that mastercoin stores data in Bitcoin by creating perpetually unredeemable transaction outputs. It doesn't just store data in the blockchain, it stores it in the UTXO set and the blockchain. The former is a more precious resource than the latter. This kind of behavior, if wide spread, changes the asymptotic fast storage requirements for a full validating node from something close to constant (just some growth due to forever lost Bitcoins which all users are paid for via deflation) to perpetually growing.  The ability to for a full node to forget old transaction data would currently let you run one with about 2% of the storage that the blockchain takes up, but these outputs cannot be spent so they cannot be forgotten.

instead of nicely asking people
I am reasonably confident that you would prefer I defend Bitcoin by nicely asking as opposed to the alternatives.

Do you believe that "Bitcoin" cares about how people use it?
No more than the door of your home cares when some thief walks through it when you've left it unlocked.  The technically possibility of something does not equal moral authority, nor does the possibility of doing something now guarantee that that possibility will remain in the future— especially if it causes harm to others, like taking up their resource for private use. Just because it's possible doesn't mean its right, nor does it mean that its wise.

I certainly believe the users of Bitcoin care and will take action to defend Bitcoin from use which is harmful to it. Some time back I'd made a minor technical proposal (http://sourceforge.net/mailarchive/message.php?msg_id=30705609) which would completely block data-storage txouts with no impact to normal Bitcoin usage.  Adopting it would require switching people to a new address style for new transactions, so it's not something that would be adopted quickly if the Bitcoin userbase were to decide to go down that route, but it certainly could be done.  I tend to think asking nicely is preferable, however.

Willet is open to discuss better, more efficient ways to encode Mastercoin transactions into the blockchain.
I have better things to do with my time than to provide free consulting for altcoins who've just raised hundreds of thousands of dollars worth of investments in their ill considered schemes. The concerns here are not new, the impacts of unspendable outputs are not mysterous to people who have even half a technical clue about the Bitcoin system, and the solutions are not subtle.

My preference is that you do what hundreds of other altcoins have done and not store your data inside Bitcoin (but feel free to merge-mine if you need a consensus system), don't non-consensually consume the resources of people who are disinterested— even opposed— to your system... but failing that, see what all the other people piling on are saying about keeping data out of the UTXO set as a bare minimum.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: Rampion on August 30, 2013, 08:48:08 AM
Unfortunately not everybody will be willing to use the Bitcoin blockchain as we/the devs would like. If someone creates something that bloats the UTXO set, they might just do not care about what the devs think about it.

How we solve THAT problem?


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: freedomno1 on August 30, 2013, 08:58:09 AM

How we solve THAT problem?

Off-Chain transactions comes to mind but may only be a part of the total solution
https://bitcointalk.org/index.php?topic=152334.0

https://bitcointalk.org/index.php?topic=274711.msg2949016#msg2949016

@gmaxwell
Sorry to add towards the posting just replying now while it is still fresh.

EDIT:
After reading the bitangels response on the Mastercoin thread and the devs reply I am on the fence with either implementation this needs more discussion since both sides seem to have validated points.
That said discussion has started, which is encouraging.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: murraypaul on August 30, 2013, 09:02:13 AM
Unfortunately not everybody will be willing to use the Bitcoin blockchain as we/the devs would like. If someone creates something that bloats the UTXO set, they might just do not care about what the devs think about it.

How we solve THAT problem?

Identify those transactions and reject them as invalid.
Do not store or relay them.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: d'aniel on August 30, 2013, 09:27:42 AM
@dacoinminster, if you're still intending to give me those 3 BTC for pointing out that problem with your proposal, could you please send them to 1GMaxweLLbo8mdXvnnC19Wt2wigiYUKgEB

@gmaxwell, thanks for your efforts.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: TierNolan on August 30, 2013, 10:49:27 AM
How we solve THAT problem?

There was a suggestion to directly limit the UTXO set.  For example, a block is invalid if the number of outputs is more than <some number> greater than the total number of inputs.

An output that pays to OP_RETURN wouldn't count as an output.

What about a weaker rule, if there is a tie between 2 blocks, break the tie in favor of the one with lower (total output - total inputs).

If a node receives a block and it would result in a small UTXO set, it still forward it and mine against it, displacing the earlier received block.

This would only affect blocks when orphans would happen, but does create an incentive to favor transactions which reduce the UTXO set.

A block which combines lots of dust into a single output would have an advantage, so miners would include those for free.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: lishbtc on August 30, 2013, 12:34:37 PM
Whilst getting consensus on this topic is unlikely, I would think it's clear JRW is not just spitting out another 'altcoin' and would look at any suggestions to mitigate blockchain bloat - Mastercoin is supposed to complement Bitcoin after all.

Given Mastercoin is an additional layer of functionality that sits on top of Bitcoin, it therefore needs Bitcoin to succeed and as such it would make no sense to make design choices that negatively impact Bitcoin.

Shame to see the hostility (I've always found collaborating to be much more rewarding) but I'm sure with the quarter million plus $ raised the project will be able to engage the right minds to find the best implementation.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: rbdrbd on August 30, 2013, 02:06:58 PM
Given Mastercoin is an additional layer of functionality that sits on top of Bitcoin, it therefore needs Bitcoin to succeed and as such it would make no sense to make design choices that negatively impact Bitcoin.

Shame to see the hostility (I've always found collaborating to be much more rewarding) but I'm sure with the quarter million plus $ raised the project will be able to engage the right minds to find the best implementation.

+1. This was going to happen sooner or later. There's way too much brainpower on these forums to not have a rational storage implementation come out of this effort.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: dacoinminster on August 30, 2013, 02:58:42 PM
I'm totally cool with stuffing the data in alternate places once we have our own client. I see using unspendable transactions as a "crutch" so that the protocol works until we get to that point. For instance, right now none of the existing bitcoin clients has functionality that will let an average user hide data in OP_CHECKMULTISIG.

I expect once we have alternate ways of storing the data, we'll support both methods for awhile, then switch to the more efficient one at some pre-determined block number.

MasterCoin wants to be a good block-chain citizen in the long run, but we are willing to put that off in the short run in favor of simplicity and immediate usability. Also, one of my design goals was to be "censorship resistant", meaning it would be really hard for anybody to tweak the bitcoin protocol such that MasterCoin no longer works. Any pruning scheme will need to be similarly robust.

Thanks for your input!


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: dacoinminster on August 30, 2013, 04:14:40 PM
@dacoinminster, if you're still intending to give me those 3 BTC for pointing out that problem with your proposal, could you please send them to 1GMaxweLLbo8mdXvnnC19Wt2wigiYUKgEB

@gmaxwell, thanks for your efforts.

Sure. I've got to run a small list of expenditures by the oversight board next week, which includes the bounty that you and bytemaster earned. Thanks!


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: dacoinminster on August 30, 2013, 04:28:24 PM
Relevant PM:

Quote
Hi JR, thanks for your work on Mastercoin so far. I am about to be investing ~50BTC...just finishing up my due diligence.

Regarding your post at: https://bitcointalk.org/index.php?topic=284178.msg3043044#msg3043044

So the plan is to fork a full client (I'd think bitcoin QT or Armory ...the latter you've forked from for your test implementation) and add all the Mastercoin functionality to that? How much effort is adding the ability to store the data in OP_CHECKMULTISIG ?? (Sorry, I'm a software developer by trade, but am not familiar with the bitcoin codebase).

If it's not a ton of work, why not plan to do this from the start? That would avoid issues with the bitcoin community at large around this (especially if you have folks like Gavin saying that it should be avoided). Having the bitcoin devs irked from this initiative from the start won't bode well... :/

-----

Frankly, I don't know how hard this will be yet. I believe Gavin when he says there are multiple ways to do it, but I don't know what all of them are or which way is best.

I do plan to use bounty money to encourage innovation in this area early on.

The most important thing to me is not to be dependent on the charity and goodwill of people running the higher protocol layer. They are great people, but I don't want to give them a lever to turn off MasterCoin, however well-intentioned they may be.

edit: I accidentally said "higher protocol layer" in this PM, but I meant bitcoin, the lower protocol layer.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: killerstorm on August 30, 2013, 04:48:20 PM
Quote
Hi JR, thanks for your work on Mastercoin so far. I am about to be investing ~50BTC...just finishing up my due diligence.

Regarding your post at: https://bitcointalk.org/index.php?topic=284178.msg3043044#msg3043044

So the plan is to fork a full client (I'd think bitcoin QT or Armory ...the latter you've forked from for your test implementation) and add all the Mastercoin functionality to that? How much effort is adding the ability to store the data in OP_CHECKMULTISIG ?? (Sorry, I'm a software developer by trade, but am not familiar with the bitcoin codebase).

If it's not a ton of work, why not plan to do this from the start? That would avoid issues with the bitcoin community at large around this (especially if you have folks like Gavin saying that it should be avoided). Having the bitcoin devs irked from this initiative from the start won't bode well... :/

-----

Frankly, I don't know how hard this will be yet.

I believe no more than a couple of days of work, if you know what are you doing. Likely less than that for a person who is very familiar with the client codebase, data structures etc.

So, yes, it makes sense to do this from the start.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: dacoinminster on August 30, 2013, 05:13:20 PM

I believe no more than a couple of days of work, if you know what are you doing. Likely less than that for a person who is very familiar with the client codebase, data structures etc.

So, yes, it makes sense to do this from the start.


Agree it should be easy on one client, but less so for people who are currently using blockchain.info for MasterCoin. I don't want to leave them hanging.

I look forward to seeing what people come up with for hiding this data in the chain in more efficient ways :)


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: maaku on August 30, 2013, 07:22:20 PM
JR, put OP_RETURN in front of the data script. That can't take more than a few seconds to change, and a few minutes to communicate. You may even be able to use dust/empty outputs in this case (you should, but not sure if the client supports it yet).


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: dacoinminster on August 30, 2013, 08:02:52 PM
JR, put OP_RETURN in front of the data script. That can't take more than a few seconds to change, and a few minutes to communicate. You may even be able to use dust/empty outputs in this case (you should, but not sure if the client supports it yet).

I think you are assuming that we have code which directly sends MasterCoin transactions, which we do not. Sends are currently accomplished using standard unmodified bitcoin clients (bitcoin-qt, blockchain.info, etc)


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: Peter Todd on August 30, 2013, 10:05:38 PM
JR, put OP_RETURN in front of the data script. That can't take more than a few seconds to change, and a few minutes to communicate. You may even be able to use dust/empty outputs in this case (you should, but not sure if the client supports it yet).

I think you are assuming that we have code which directly sends MasterCoin transactions, which we do not. Sends are currently accomplished using standard unmodified bitcoin clients (bitcoin-qt, blockchain.info, etc)

Given how much money you've raised it'd be quite easy to pay someone to add support for OP_RETURN to those clients, among other things.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: dacoinminster on August 30, 2013, 10:27:45 PM
Given how much money you've raised it'd be quite easy to pay someone to add support for OP_RETURN to those clients, among other things.

Hey Peter! Thanks for checking out the project! (Peter and I met at the conference in San Jose, and had some interesting discussions there, and he is a tireless advocate for keeping bitcoin awesome, as you can see from his sig)

I completely agree. We seem to have a rather formidable war-chest :)

Being a good block-chain citizen is a matter of survival for MasterCoin. If we don't, we'll face competition from a clone that says "We're like MasterCoin, but with less impact on the blockchain"


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: Peter Todd on August 31, 2013, 01:33:31 AM
Hey Peter! Thanks for checking out the project! (Peter and I met at the conference in San Jose, and had some interesting discussions there, and he is a tireless advocate for keeping bitcoin awesome, as you can see from his sig)

I completely agree. We seem to have a rather formidable war-chest :)

Being a good block-chain citizen is a matter of survival for MasterCoin. If we don't, we'll face competition from a clone that says "We're like MasterCoin, but with less impact on the blockchain"

With any luck you'll face competition from clones that realize the entire idea is just dumb. Are you planning to release the sourcecode to MasterCoin? Because if you don't, we'll all just say how it's probably got hidden backdoors that are going to steal all your (master)coins, and if you do release it, we can just take that code and remove the silly requirement for the exodus address/MasterCoins. Or just stick with, who was it, TierNolan and co's colored coins stuff?

Also, even if you want to use the colored coins paradigm, you actually don't need any data in the blockchain at all beyond what appear to be bog-standard pubkeys. Even better, is that you can design your protocol to really make it censorproof by allowing the colored coins to be included in standard CoinJoin transactions in such a way that anyone looking at the blockchain data will have no way of knowing what outputs were the colored coins.

Frankly the only clever thing you've done is you've taken the same colored coins/side-chains ideas everyone else has and found a way to get Bitcoin users to shovel money at you for it. Too bad really - I guess the rest of us couldn't imagine how you didn't even need to write any code to separate $300k worth of Bitcoins from the idiots who used to own them. (We forgot about The Pirate already!) Hopefully that's not actually true, and it's actually your own money and not many people are getting scammed here.

Anyway, I have better things to do than waste my time talking about ideas sufficiently stupid to be indistinguishable from scams for free, so don't bother replying unless you've got a paying contract. (1.5BTC/hr, escrow required for you)

I will say though, you're so close to doing something useful for me: showing the world how little protection we actually have against the UTXO set being bloated other than the blocksize limit. Do me a favor and keep your stupid protocol just the way it is.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: bpd on August 31, 2013, 03:03:03 AM
Hey Peter! Thanks for checking out the project! (Peter and I met at the conference in San Jose, and had some interesting discussions there, and he is a tireless advocate for keeping bitcoin awesome, as you can see from his sig)

I completely agree. We seem to have a rather formidable war-chest :)

Being a good block-chain citizen is a matter of survival for MasterCoin. If we don't, we'll face competition from a clone that says "We're like MasterCoin, but with less impact on the blockchain"

With any luck you'll face competition from clones that realize the entire idea is just dumb. Are you planning to release the sourcecode to MasterCoin? Because if you don't, we'll all just say how it's probably got hidden backdoors that are going to steal all your (master)coins, and if you do release it, we can just take that code and remove the silly requirement for the exodus address/MasterCoins. Or just stick with, who was it, TierNolan and co's colored coins stuff?

Also, even if you want to use the colored coins paradigm, you actually don't need any data in the blockchain at all beyond what appear to be bog-standard pubkeys. Even better, is that you can design your protocol to really make it censorproof by allowing the colored coins to be included in standard CoinJoin transactions in such a way that anyone looking at the blockchain data will have no way of knowing what outputs were the colored coins.

Frankly the only clever thing you've done is you've taken the same colored coins/side-chains ideas everyone else has and found a way to get Bitcoin users to shovel money at you for it. Too bad really - I guess the rest of us couldn't imagine how you didn't even need to write any code to separate $300k worth of Bitcoins from the idiots who used to own them. (We forgot about The Pirate already!) Hopefully that's not actually true, and it's actually your own money and not many people are getting scammed here.

Anyway, I have better things to do than waste my time talking about ideas sufficiently stupid to be indistinguishable from scams for free, so don't bother replying unless you've got a paying contract. (1.5BTC/hr, escrow required for you)

I will say though, you're so close to doing something useful for me: showing the world how little protection we actually have against the UTXO set being bloated other than the blocksize limit. Do me a favor and keep your stupid protocol just the way it is.

Thanks for saying what many are thinking. Even if it isn't a scam, the system simply can't work. It shows complete lack of understanding about how markets and asset pricing actually work. Gold assets not backed by gold, but by MasterCoins (which will clearly appreciate faster than dumb old gold)... It's so obvious... Why didn't the people who started GLD think of that?  They'd be so much richer!


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: lishbtc on August 31, 2013, 04:37:13 AM
/sigh @ how many times supposedly intelligent people have thrown around the term 'scam' in this thread - get over yourselves.

Criticize the design to your hearts content, healthy debate is fine but don't go after a guy who has simply solicited investment for an idea - his personal identity is well known (I understand some of you have met him) and he has done nothing to warrant calling scam.

I wonder how economic development would have progressed historically if everyone thought onboarding investors was such a tangible evil.




Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: killerstorm on August 31, 2013, 09:13:45 AM
With any luck you'll face competition from clones that realize the entire idea is just dumb. Are you planning to release the sourcecode to MasterCoin? Because if you don't, we'll all just say how it's probably got hidden backdoors that are going to steal all your (master)coins, and if you do release it, we can just take that code and remove the silly requirement for the exodus address/MasterCoins.

They say MasterCoin has an advantageover copycats; just like Bitcoin has advantages over "alt-coins".

Or just stick with, who was it, TierNolan and co's colored coins stuff?

Also, even if you want to use the colored coins paradigm, you actually don't need any data in the blockchain at all beyond what appear to be bog-standard pubkeys.

MasterCoin and "colored coins" use different approaches and implement different features.

Within 'colored coins' approach we use Bitcoin transaction graph, and state changes are localized in the sense that one transaction cannot affect another transaction which doesn't depend on it.

MasterCoin's approach is different: there is a global state, and MasterCoin transactions are processed sequentially, in the order they appear in the blockchain, and the modify the global state. This is absolutely necessary because many MasterCoin features absolutely require global state.

J.R.W. believes that these features are very important, and for this reason colored coins are not suitable for what he is trying to do.

These features, their importance and possibility to accomplish same goals though other means are out of scope of this discussion. (I'll just note that it is possible to implement arbitrarily complex rules through 'oracle' or 'operator' of some sort, then the only concern is trustworthiness of that oracle and possibility of recovery from malfunction.)

Anyway, the MasterCoin's paradigm is: there is a list of messages (commands) which are reliably timestamped (i.e. all actors agree on their order) and carry authorization information (are signed with corresponding private key); messages are processed in order and each one affects the global state.

It's quite obvious that the Bitcoin blockchain is used for two things here:

1. Message timestamping, i.e. blockchain ensures that everybody sees these messages in same order.
2. Message storage and distribution.

It is possible to decouple these things: use Bitcoin blockchain for timestapming, but keep messages in some other place.

In the idea case (if our goal is to minimize the bloat) it's possible to keep only root hash in the Bitcoin blockchain (one hash per block), and messages themselves can go either to some alt-chain, or to DHT, or whatever.

Obviously, such a change will affect many aspects, so it won't be exactly same as the current form of MasterCoin, but all important features will be preserved.

I really doubt J.R. is willing to organize it in such a way, as he wants to make it possible to use ordinary Bitcoin client to send MasterCoin commands.

There is a quick and dirty way to modify it: move to some merged-mined alt-coin, as is. Most Bitcoin clients can be adapted to work with an alt-coin.

However, J.R. believes that MasterCoin is beneficial for Bitcoin, so there goes an opportunity...

Even better, is that you can design your protocol to really make it censorproof by allowing the colored coins to be included in standard CoinJoin transactions in such a way that anyone looking at the blockchain data will have no way of knowing what outputs were the colored coins.

Eh? How do colored coin clients know which outputs were the colored coins then? They sort of need it to show balance etc.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: Infinum on August 31, 2013, 04:26:28 PM
Being a good block-chain citizen is a matter of survival for MasterCoin. If we don't, we'll face competition from a clone that says "We're like MasterCoin, but with less impact on the blockchain"

With any luck you'll face competition from clones that realize the entire idea is just dumb. (...)

Frankly the only clever thing you've done is you've taken the same colored coins/side-chains ideas everyone else has and found a way to get Bitcoin users to shovel money at you for it. Too bad really - I guess the rest of us couldn't imagine how you didn't even need to write any code to separate $300k worth of Bitcoins from the idiots who used to own them. (We forgot about The Pirate already!) Hopefully that's not actually true, and it's actually your own money and not many people are getting scammed here.

Anyway, I have better things to do than waste my time talking about ideas sufficiently stupid to be indistinguishable from scams for free, so don't bother replying unless you've got a paying contract. (1.5BTC/hr, escrow required for you)

I will say though, you're so close to doing something useful for me: showing the world how little protection we actually have against the UTXO set being bloated other than the blocksize limit. Do me a favor and keep your stupid protocol just the way it is.

Thanks for saying what many are thinking. Even if it isn't a scam, the system simply can't work. It shows complete lack of understanding about how markets and asset pricing actually work. Gold assets not backed by gold, but by MasterCoins (which will clearly appreciate faster than dumb old gold)... It's so obvious... Why didn't the people who started GLD think of that?  They'd be so much richer!

I hear you. I have read the specs for this ponzi coin and it just does not add up. People who sent their BTC to the Exodus node address will not see those BTCs ever again (perfect name BTW) - this meta-coin is parasitic and scammy.

As a full blockchain node operator I really want to see some kind of solution to the blockchain bloat problem, preferably with some fees so I can turn this into a business rather than a non-profit. Miners verify transactions and get compensation for it, but nobody pays for the transaction's storage once it gets verified and this will eventually shrink full-nodes numbers and further strengthen the oligopoly of Bitcoin miners.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: TierNolan on August 31, 2013, 05:01:32 PM
As a full blockchain node operator I really want to see some kind of solution to the blockchain bloat problem, preferably with some fees so I can turn this into a business rather than a non-profit. Miners verify transactions and get compensation for it, but nobody pays for the transaction's storage once it gets verified and this will eventually shrink full-nodes numbers and further strengthen the oligopoly of Bitcoin miners.

The advantage of storage is that you can verify more transactions.  However, that is not much good if other miners drop your block because they dropped the UTXO.

The only "market" solutions are something like UTXO "rental" or just add a hard limit for the rate at which the set can increase per block.

Rental could be implemented by having a system where unspent UTXOs are dropped from memory after a while.  Each block a UTXO could have its output drop by <some value>.  Once it drops below zero, it is dropped from the in memory set.  This wouldn't actually cause the output to be less valuable, as long as you spent it before it was dropped it would be ok.  Spending old transactions would require adding info for fast loading.  For example, the tx message would have to also include info to allow fast proving that the UTXO is valid.

This is something that have a merkle tree for the UTXO set would be helpful for.  You give the path to prove the UTXO is valid and then it doesn't need to be contained in memory.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: killerstorm on August 31, 2013, 05:15:06 PM
The advantage of storage is that you can verify more transactions.  However, that is not much good if other miners drop your block because they dropped the UTXO.

If you have 1 MB block size limit and each input is approximately 250 bytes, you only have at most 4000 inputs per block.

Thus you only have to buy hardware which can do ~400 random reads per second if you want to spend no more than 10 seconds to verify a block.

I'm fairly certain you can do this with 4 ordinary HDDs if they can do reads in parallel.

Or you can buy a SSD, pretty much any device can do more than 10000 random reads per second.

As for inputs in transactions not included in blocks, you can just drop ones you cannot fetch in a reasonable time. There is nothing wrong with dropping transactions.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: TierNolan on August 31, 2013, 07:34:36 PM
If you have 1 MB block size limit and each input is approximately 250 bytes, you only have at most 4000 inputs per block.

Well, the size limit (at least the 1MB version) is not likely to last indefinitely.

The block chain would have to be saved with a flag to indicate if the output was consumed.  This flag could be set when the tx is burred enough that a reversal is unlikely.

However, it does reduce the overall performance of the system.

Quote
As for inputs in transactions not included in blocks, you can just drop ones you cannot fetch in a reasonable time. There is nothing wrong with dropping transactions.

That is part of what I meant.  If a tx message could have meta-data attached, then at least you could prove that the tx output actually exists.  This acts as a way to prevent spam.

Keeping only recent UTXO set elements in RAM and the rest on disk would be the end result.

If it is harder to spend "old" transaction outputs, then that weakens the ability of the system to act as a long term store of value.

Bloom filtering could be used.  If every node had a different random seed for the hash function, then producing the false positive that propagates would be hard.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: killerstorm on August 31, 2013, 11:09:06 PM
If you have 1 MB block size limit and each input is approximately 250 bytes, you only have at most 4000 inputs per block.

Well, the size limit (at least the 1MB version) is not likely to last indefinitely.

My point is that we can associate performance requirements with block size rather than with UTXO set size.

If you think about it, it makes more sense: block size limit is the part of the protocol. It is something which can be controlled, and is being controlled now. But UTXO set size might be as high as 21*10^14, and controlling it is rather problematic and non-trivial.

So it is possible to design software in such a way that block size limit will dictate hardware requirements for full nodes. If there is a need to increase throughput, block size limit will be increased and hardware requirements will be adjusted accordingly.

So you can just tell full nodes maintainers which hardware they need to keep up with blockchain growth. It will give them peace of mind. Isn't that nice?

However, it does reduce the overall performance of the system.
If it is harder to spend "old" transaction outputs, then that weakens the ability of the system to act as a long term store of value.

You can prioritize them by value. E.g. if size of 1 UTXO  in memory is 100 bytes, storing 21 million entries requires only 2.1 GB, and thus you can easily guarantee that all UTXOs which have at least 1 Bitcoin in them will never be flushed away.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: drawingthesun on September 01, 2013, 04:12:50 AM
Maybe this has already been addressed I am not sure, what if we could prove that a output in the blockchain is unspendable (a transaction to a valid but unusable address) if we know for certain that the coin is unspendable could they not be removed in the future when we start compressing the blockchain?

It would make sense that coins that can not move and this is known to be true are simply removed and those addresses removed too. Because they are no longer spendable Bitcoins. You could even claim these coins after they have sat in their addresses for x amount of time and move them as tx fees into the next block to be mined.

Of course this can only be done for addresses that are proven to have no possible private key access. If mastercoin uses valid and usable addresses but deletes the private key then we can't use this technique because that would be theft.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: spiral_mind on September 01, 2013, 04:14:33 AM
/sigh @ how many times supposedly intelligent people have thrown around the term 'scam' in this thread - get over yourselves.

Criticize the design to your hearts content, healthy debate is fine but don't go after a guy who has simply solicited investment for an idea - his personal identity is well known (I understand some of you have met him) and he has done nothing to warrant calling scam.

I wonder how economic development would have progressed historically if everyone thought onboarding investors was such a tangible evil.




How about the fact that his project is illegal in multiple ways?

He's directly issuing a virtual currency in exchange for real money. Sound familiar? http://www.theguardian.com/world/2013/may/28/liberty-reserve-arthur-budovsky-arrested-spain

There's a reason Satoshi Nakamoto didn't reveal his true identity.

If you take investors money for an asset that is actually illegal in the first place, will the SEC prosecute you for that too? Likely yes, especially if you've raised as much as Mr. Willet has.

So really how can you call this anything other than a scam? If he doesn't flee with the money to another country he's going to jail eventually.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: TierNolan on September 01, 2013, 11:50:36 AM
Maybe this has already been addressed I am not sure, what if we could prove that a output in the blockchain is unspendable (a transaction to a valid but unusable address) if we know for certain that the coin is unspendable could they not be removed in the future when we start compressing the blockchain?

That is what OP_RETURN is for.  If you pay to an output containing OP_RETURN, then it cannot be spent. 

Clients can then drop that output from the set.

The problem here is that to keep things simple, they are using standard transactions.  They are indistinguishable from "real" outputs, by design.

Quote
It would make sense that coins that can not move and this is known to be true are simply removed and those addresses removed too. Because they are no longer spendable Bitcoins. You could even claim these coins after they have sat in their addresses for x amount of time and move them as tx fees into the next block to be mined.

If they are "destroyed", then the value is effectively paid to everyone.

Quote
Of course this can only be done for addresses that are proven to have no possible private key access.

They pay to the output of a hash function.  There is no way to figure out which outputs are valid and which aren't.

The standard script is effectively "Pay to the public key that hashes to this value <20 byte hash>". 

You spend it by saying "here is the public key and here is the signature using that public key".  The public key is hashed to make sure it is the right hash and then the signature is checked.

Instead of a hash, mastercoin replaces it with a "data" packet.

In theory, the reference client could detect mastercoin operations and delay the data packet transactions by more than 6 blocks from the initiation tx.  That would break the protocol, since they are only valid if all the packets are within 6 blocks of each other.

If there is a false positive, then it would delay the transaction by an hour.

Another option would be a flag in the various wallets that marks a payment address as invalid.  Outputs that pay to OP_RETURN <standard script> would also be considered standard.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: jl2012 on September 01, 2013, 04:43:32 PM
ATM, The UTXO set is 239,681,979 bytes out of 10,618,084,349 bytes, or 2.26% of the total blockchain

I have proposed it for many times: set a hard limit for the growth of UTXO (e.g. 1% of max block size, i.e. 10kB) per block, so the UTXO set will grow at most 525MB per year. This will encourage people to merge dust outputs, and punish any UTXO-bloating activities. At the same time, we could make OP_RETURN as standard output and not count it as UTXO growth.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: standardcrypto on September 01, 2013, 07:45:29 PM
There are plenty of ways to accomplish that, the easiest of which that works today would be to stuff data into unused public keys of an OP_CHECKMULTISIG transaction.

In case it helps other readers of the thread... If I am understanding correctly:
  1)  the idea here is that you can use something that looks like a public key, but which you don't actually have a private key that hashes right for it, since you don't need that particular private key to spend as long as you have private keys for other addresses.
  2) any 64 bytes of entropy can be used as a public key, as long as you don't need the private key

Please correct me if I'm wrong.

https://en.bitcoin.it/wiki/Technical_background_of_Bitcoin_addresses

I'm still fuzzy on how this could actually be implemented.

How many public keys are currently usable for m of n signatures? Are OP_CHECKMULTISIG transactions "official" and generally relayed by miners, or are these still in a grey area? I did google a bit, but I couldn't figure out where to look to determine what transactions can generally expect to be supported. Obviously it's important to have a good grasp on this for people steering mastercoin.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: StarenseN on September 07, 2013, 12:44:08 AM
/me following


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: ofer on September 09, 2013, 07:26:32 PM
I suggest that instead of sending a micro payment to the exodus address, we can enforce sending a bigger amount.
This extra money can be really significant if Mastercoin will gain popularity (if not the spam will be irrelevant).
The extra money can be used to support the Bitcoin foundation (like R&D for scaling up the block-chain, etc.).


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: btcdrak on September 12, 2013, 08:29:36 AM
I'm totally cool with stuffing the data in alternate places once we have our own client. I see using unspendable transactions as a "crutch" so that the protocol works until we get to that point. For instance, right now none of the existing bitcoin clients has functionality that will let an average user hide data in OP_CHECKMULTISIG.

It is inevitable that in the future, bit coin does prevent the permanent storage of arbitrary data in the blockchain. If you rely on such features now, there could be colossal consequences later. #bitcoin wont care about you or your bitcoin-bloat. BE WARE, YOU HAVE BEEN WARNED.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: dacoinminster on September 12, 2013, 04:19:22 PM
I'm totally cool with stuffing the data in alternate places once we have our own client. I see using unspendable transactions as a "crutch" so that the protocol works until we get to that point. For instance, right now none of the existing bitcoin clients has functionality that will let an average user hide data in OP_CHECKMULTISIG.

It is inevitable that in the future, bit coin does prevent the permanent storage of arbitrary data in the blockchain. If you rely on such features now, there could be colossal consequences later. #bitcoin wont care about you or your bitcoin-bloat. BE WARE, YOU HAVE BEEN WARNED.

MasterCoin hopes to be symbiotic with bitcoin, not parasitic, and we plan to morph and change to whatever degree is needed as bitcoin morphs and changes. If our old data gets pruned out of the block-chain used by most bitcoin clients, our clients will have to be modified to keep different parts of the block chain, and use different pruning techniques. Nothing I can foresee leaves MasterCoin completely stranded or unusable.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: jl2012 on September 12, 2013, 04:52:21 PM
I'm totally cool with stuffing the data in alternate places once we have our own client. I see using unspendable transactions as a "crutch" so that the protocol works until we get to that point. For instance, right now none of the existing bitcoin clients has functionality that will let an average user hide data in OP_CHECKMULTISIG.

It is inevitable that in the future, bit coin does prevent the permanent storage of arbitrary data in the blockchain. If you rely on such features now, there could be colossal consequences later. #bitcoin wont care about you or your bitcoin-bloat. BE WARE, YOU HAVE BEEN WARNED.

MasterCoin hopes to be symbiotic with bitcoin, not parasitic, and we plan to morph and change to whatever degree is needed as bitcoin morphs and changes. If our old data gets pruned out of the block-chain used by most bitcoin clients, our clients will have to be modified to keep different parts of the block chain, and use different pruning techniques. Nothing I can foresee leaves MasterCoin completely stranded or unusable.

I won't be surprised. Parasites always morph and change as the host evolves.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: dacoinminster on September 12, 2013, 05:36:34 PM
I won't be surprised. Parasites always morph and change as the host evolves.

Yes, that's something like what I have in mind, although long-term, we hope to be mitochondria, not tape worms :)


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: btcdrak on September 13, 2013, 12:48:39 PM
I won't be surprised. Parasites always morph and change as the host evolves.

Yes, that's something like what I have in mind, although long-term, we hope to be mitochondria, not tape worms :)

Willet, what I really don't understand about you is that CLEARLY, a separate block-chain would be best for your project, but you are just ADAMANT, just for the sake of it, to use the bitcoin blockchain. Why on earth is that? To be honest, the BEST way would be to do what Namecoin does, merged mining. It's the perfect solution. Why don't you want to do it? "Just because you are attached to Bitcoin". it's a stupid reasoning.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: dacoinminster on September 13, 2013, 05:37:40 PM
Willet, what I really don't understand about you is that CLEARLY, a separate block-chain would be best for your project, but you are just ADAMANT, just for the sake of it, to use the bitcoin blockchain. Why on earth is that? To be honest, the BEST way would be to do what Namecoin does, merged mining. It's the perfect solution. Why don't you want to do it? "Just because you are attached to Bitcoin". it's a stupid reasoning.

I describe my reasoning for this in the "Assumptions" section of my whitepaper (see the link in my sig):

Quote
Assumptions

Our claims are built on the following assumptions:

•   Alternate block chains compete with bitcoins financially, confuse our message to the world, and dilute our efforts. These barriers interfere with the adoption momentum of bitcoin and the adoption momentum of alternate currencies as well, regardless of how well-conceived their rules may be.
•   New protocol layers on top of the bitcoin protocol will increase bitcoin values, consolidate our message to the world, and concentrate our efforts, while still allowing individuals and groups to issue new currencies with experimental new rules. The success of any experimental currency protocol layer will enhance the value and success of the foundational bitcoin protocol.
•   Getting consensus and widespread adoption from the bitcoin community is not needed to add protocol layers, since no changes to the foundational bitcoin protocol are required.
•   Tiny bitcoin transactions can be encoded into the block chain to support and represent transactions in higher protocol layers.
•   A protocol can pay for its own software development, “bootstrapping” itself into existence, utilizing a trusted entity to hold funds and hire developers.
•   It is possible to create tools to allow end users to create currency protocol layers which have a stable value, pegged to an external currency or commodity. In this way, users of these currencies can own stabilized virtual currency tied to U.S. Dollars, Euros, ounces of gold, barrels of oil, etc.
•   It is possible for users of these new currencies to exchange between currencies with each other using simple rules and no central exchange.


One reason I didn't mention in the paper is that this method of accomplishing my goals is way, way easier and faster than building an alt coin. Building on bitcoin significantly reduces the technical risk of my project.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: btcdrak on September 13, 2013, 05:42:41 PM
One reason I didn't mention in the paper is that this method of accomplishing my goals is way, way easier and faster than building an alt coin. Building on bitcoin significantly reduces the technical risk of my project.

Not doing something like merged mining which relies on 100% Bitcoin is the biggest risk. The days of bitcoin bloatware are numbered. Your funeral, and the funeral of all the poor suckers who gave you money already when it finally happens. You're probably safe until Bitcoin 0.9, but by 1.0, it'll be game over.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: dacoinminster on September 13, 2013, 06:39:31 PM
One reason I didn't mention in the paper is that this method of accomplishing my goals is way, way easier and faster than building an alt coin. Building on bitcoin significantly reduces the technical risk of my project.

Not doing something like merged mining which relies on 100% Bitcoin is the biggest risk. The days of bitcoin bloatware are numbered. Your funeral, and the funeral of all the poor suckers who gave you money already when it finally happens. You're probably safe until Bitcoin 0.9, but by 1.0, it'll be game over.

I expect MasterCoin will have to change and implement it's own custom pruning solution at some point, but I'm not expecting any funerals :)


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: btcdrak on September 13, 2013, 07:06:01 PM
One reason I didn't mention in the paper is that this method of accomplishing my goals is way, way easier and faster than building an alt coin. Building on bitcoin significantly reduces the technical risk of my project.

Not doing something like merged mining which relies on 100% Bitcoin is the biggest risk. The days of bitcoin bloatware are numbered. Your funeral, and the funeral of all the poor suckers who gave you money already when it finally happens. You're probably safe until Bitcoin 0.9, but by 1.0, it'll be game over.

I expect MasterCoin will have to change and implement it's own custom pruning solution at some point, but I'm not expecting any funerals :)

Your arrogance can only be compared to corporate slime who adapt their tax-evasion with the passing of every new law, and gloat about it.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: dacoinminster on September 13, 2013, 08:47:24 PM
Your arrogance can only be compared to corporate slime who adapt their tax-evasion with the passing of every new law, and gloat about it.

I guess if my choices are between building yet another alt chain and "arrogance", I choose arrogance.

The possibility of building a new protocol layer on top of bitcoin has always been there, and someone was bound to do it sooner or later. It just happens that I'm the first one to actually take a crack at it.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: dacoinminster on September 13, 2013, 08:52:51 PM
People concerned about MasterCoin bloating the bitcoin block-chain will be pleased to hear that we now have proof-of-concept for at least one alternate way of storing our data without creating unspendable outputs:

I have a proposal for alternative Mastercoin encoding ready and I wanted to discuss the feasibility of my solution. We are going to use a OP_CHECKMULTISIG script with three (or more) public keys to encode the data.

Because we will be using public keys to encapsulate the data we now have 64 bytes to encode data (65 minus one byte for the 04 marker). In order to not polute the blockchain we will always supply a publickey that the sender owns as first signature this way the output can always be redeemed. The other signatures we supply will be data-addresses; these transactions won't be able to be converted to addresses since they are not technically ECDSA x/y coordinates but they should be relayed and mined just fine.

I've created a test transaction on testnet3 (http://test.webbtc.com/tx/4bbef8adce4009bf8e24ec90f652f78c5ed2f5863414362ad551f3f78f969d9e.json) (raw (http://test.webbtc.com/tx/4bbef8adce4009bf8e24ec90f652f78c5ed2f5863414362ad551f3f78f969d9e.json)) to test this functionality.

miKnddGDQfU6rRYpLp2dhRyttnxH1WUo21 is the exodus that I use on Testnet. I'm sending the coins to myself (miDyBpL3TeeJRbtwdFUsw9mUWUEhN6FyXf) in this example so I send both the change amount and 0.00006 transaction to this address. I also use this address as the first signature for the multisg output so I can redeem this later. If you check the raw transaction you will see that there are two more keys supplied. If you decode these using the encoding standard from the whitepaper (but instead pad with zeros until you have 65 bytes) you will see that I'm doing two transaction for 1 test Mastercoin each.

In theory you could do multiple Mastercoin transactions in one multisig output but I would like to propose to keep it simple and only one transaction per output. If you want to send multiple Mastercoin transactions in one Bitcoin transaction you can create multiple multisig outputs.

Since all current ideas of the whitepaper fit within 126 bytes (2 signatures) and signatures are in fixed order we might not need to add sequences to all signatures but for now let's continue doing it.

I know there are a lot of users with more knowledge of Bitcoin lurking so please let me know if this approach would work within the rules set out by the reference implementation. I'm especially interested in if we can use 1 out of 5 multisig transactions. I tried to make one; and it relayed; but I'm not sure it's actually a valid transaction type.

Please let me know your thoughts so we can continue building cool things on top of Mastercoin :)


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: gmaxwell on September 13, 2013, 08:57:58 PM
It just happens that I'm the first one to actually take a crack at it.
I believe Luke's "Tonal Bitcoin" from 2011 probably has claim at this. :P


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: TomHirsch on September 13, 2013, 09:32:24 PM
I have better things to do with my time than to provide free consulting for altcoins who've just raised hundreds of thousands of dollars

don't non-consensually consume the resources of people who are disinterested— even opposed— to your system...

...You send 1BTC to your grandmother - I am disinterested and it consumes blockchain resources - and I oppose you sending this kind of transaction.  

Look, J.R. is breaking his back trying to find a way to very lightly consume these resources and to be a good 'blockchain citizen'.  There is no point trying to bust his balls about the project.  

Your anger reeks of jealousy because you have raised any money to support any of your ideas.  J.R. is being very respectful and polite and trying to find the best way possible to keep bitcoin strong and healthy.  Then, you angry dicks come around cursing him down for his good efforts.  WTF?  

Contribute or go home pussy.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: TomHirsch on September 13, 2013, 09:42:20 PM
Hey Peter! Thanks for checking out the project!
...the entire idea is just dumb.

Anyway, I have better things to do than waste my time talking about ideas sufficiently stupid to be indistinguishable from scams for free, so don't bother replying unless you've got a paying contract. (1.5BTC/hr, escrow required for you)

Do me a favor and keep your stupid protocol just the way it is.
'
WHAT - A - DICK. 

The guy was being nice and saying 'hello in a friendly way'.  Why is there so much anger in the bitcoin upper echelon?  Did someone steal all your coins?



Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: dacoinminster on September 13, 2013, 10:05:49 PM
Peter is a nice guy in person. He's also very passionate about the health of the bitcoin network - more than anybody I know. In retrospect, maybe I shouldn't have expected a continuation of the friendly dialog we started at the conference, given MasterCoin's potential to rapidly expand the block chain (even more than it already is expanding). That's probably a sensitive issue.

Also, while I doubt anybody stole his coins, he may have just been having a really bad day. I try to give people a little grace and understanding in situations like this.

GMaxwell, I think I remember when tonal bitcoin came out. Gosh, that seems like a long time ago. I should probably just claim to be the first to create a new protocol layer that actually adds new functionality (as opposed to an alternate display)? Also, I hope you enjoy your 3 BTC I sent you, courtesy of d'aniel. I believe I have actually sent more money to detractors than boosters at this point! That should change soon :)


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: TomHirsch on September 13, 2013, 10:25:07 PM

GMaxwell, I hope you enjoy your 3 BTC I sent you, courtesy of d'aniel.

$450 to a dick?  We need to save the funds better than this.  I hope his contribution was valuable.  Just because the projects wallet is fat, doesn't mean you have to throw the money around unnecessarily.

Did you know that people consume about 20% more shampoo when their shampoo bottle is full?  No need sending BTC to dicks unless their help was a good improvement for the project.



Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: dacoinminster on September 13, 2013, 10:41:37 PM

GMaxwell, I hope you enjoy your 3 BTC I sent you, courtesy of d'aniel.

$450 to a dick?  We need to save the funds better than this.  I hope his contribution was valuable.  Just because the projects wallet is fat, doesn't mean you have to throw the money around unnecessarily.

Did you know that people consume about 20% more shampoo when their shampoo bottle is full?  No need sending BTC to dicks unless their help was a good improvement for the project.

d'aniel earned a bounty very early on for describing a potential attack vector on underfunded escrows. His contribution was very valuable, and led to some changes in the most recent rev of the spec. The fact that he is a detractor doesn't diminish his contribution. d'aniel asked me to send his bounty to GMaxwell, so I did. That bounty was approved by the MasterCoin Foundation board (which was two people, not including myself, at that time).

If we do everything we can to be good block-chain citizens, and if the protocol is successful, some detractors will eventually become advocates. People here can be very passionate and opinionated, but most of them can also be swayed by sound arguments, especially if the argument is in the form of code which is proving useful to lots of people and making bitcoin more valuable.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: tspacepilot on September 14, 2013, 06:45:18 PM
Where can I find some more background reading about these topics: utxos and pruning, etc.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: maaku on September 15, 2013, 12:31:33 AM
Where can I find some more background reading about these topics: utxos and pruning, etc.

https://bitcointalk.org/index.php?topic=88208.0


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: milkyman on September 16, 2013, 03:27:22 PM
Seems like what I wrote in the main mastercoin discussion thread fits better into this thread:

There was a lot of critics about the Mastercoin network, in particular due to the many generated un-prunable transactions that are necessary to transmit the data between Mastercoin users.

I have an idea how to solve this problem, but this would require some changes to the protocol. Basically, instead of sending Bitoins to a data address without known private key (which is the current Mastercoin implementation), data can also be transmitted by sending Bitcoins to addresses that other Bicoin users own. To do so, a 'directory' of addresses has to be generated first. Each address can store a few bytes of data - and by sending tiny amounts of Bitcoins to these addresses, the pieces of data can be connected (similar to the current Mastercoin implementation). In this presented implementation, each address can store about 3 bytes. Therefore, to have enough data for a Mastercoin transaction, about 6 - 12 Bitcoin transactions have to be sent.

However, there is an important advantage in this implementation: Users that 'own' these addresses can later send all their earned Bitcoins to a single address. As a result, the millions of addresses are fully spent, and the whole system becomes prunable.

In contrast, in the current Mastercoin system, all the data addresses will remain with 0.00006 BTC stored and have to be saved in the blockchain forever.


Here's the implementation:

First, a global list of length 58^4 = 11316496 and width = 30 is created, and each cell can contain a Bitcoin-address. Let us index these cells with (n,#). The size of the list is a few GB, which is well managable to keep in lots of copies among all Mastercoin users.

Now to the actual idea: The cells (1,#) must contain addresses with the last four characters being -1111. The cells (2,#) must contain addresses -1112, and so on. Finally, the cells (11316496,#) must contain addresses with the last 4 characters -zzzz.

Now, every Bitcoin user that finds a key pair can publish the corresponding address in this list. To this end, he has to prove that he owns the private key of this address, to prevent people 'scamming' the list. Assumed, he found an address ending with -1113, he publishes it and it will be written into cell (3,1). Now let's assume > 29 other users also find addresses ending with -1113 and publish them. Then, all these addresses are alphabetically ordered and written into cells (3,1), (3,2), ... (3,30). Addresses with index > 30 are discarded. Obviously, with time, it becomes harder and harder to 'win' a cell in the list, because an address with low alphabetical index has to be found to kick out another one.

Since it is attractive for users to 'own' cells in the list, the time will come the list is complete, meaning that every cell has an address written in it. Of course, after completing the list, new addresses with lower index can still be published, and the list will be continuously updated.

Last, how can a string of data be transmitted?
First, the data is converted into base58 and chopped into pieces with length 4. Example: 3HZ28xpWUhq4... -> 3HZ2, 8xpW, Uhq4... The addresses (taken from the global directory) to send bitcoins to must have the following properties:

1.) They have to end with -3HZ2, -8xpW, -Uhq4...
2.) The addresses have to be in alphabetic order: (...-3HZ2) < (...-8xpW) < (...-Uhq4) < ...

Point 2 is important to connect the data pieces in the right order. It should be possible to find such an alphabetic order, since there are 30 different available addresses for each piece of data.


So, what you think... Assumed that Bitcoin miners feel threatened by the current Mastercoin implementation and stop to confirm transactions that have an Exodus recipient, this implementation could be a solution, since it harms the Bitcoin network less.


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: dacoinminster on September 16, 2013, 05:11:41 PM
Seems like what I wrote in the main mastercoin discussion thread fits better into this thread:

There was a lot of critics about the Mastercoin network, in particular due to the many generated un-prunable transactions that are necessary to transmit the data between Mastercoin users.

I have an idea how to solve this problem, but this would require some changes to the protocol. Basically, instead of sending Bitoins to a data address without known private key (which is the current Mastercoin implementation), data can also be transmitted by sending Bitcoins to addresses that other Bicoin users own. To do so, a 'directory' of addresses has to be generated first. Each address can store a few bytes of data - and by sending tiny amounts of Bitcoins to these addresses, the pieces of data can be connected (similar to the current Mastercoin implementation). In this presented implementation, each address can store about 3 bytes. Therefore, to have enough data for a Mastercoin transaction, about 6 - 12 Bitcoin transactions have to be sent.

However, there is an important advantage in this implementation: Users that 'own' these addresses can later send all their earned Bitcoins to a single address. As a result, the millions of addresses are fully spent, and the whole system becomes prunable.

In contrast, in the current Mastercoin system, all the data addresses will remain with 0.00006 BTC stored and have to be saved in the blockchain forever.


Here's the implementation:

First, a global list of length 58^4 = 11316496 and width = 30 is created, and each cell can contain a Bitcoin-address. Let us index these cells with (n,#). The size of the list is a few GB, which is well managable to keep in lots of copies among all Mastercoin users.

Now to the actual idea: The cells (1,#) must contain addresses with the last four characters being -1111. The cells (2,#) must contain addresses -1112, and so on. Finally, the cells (11316496,#) must contain addresses with the last 4 characters -zzzz.

Now, every Bitcoin user that finds a key pair can publish the corresponding address in this list. To this end, he has to prove that he owns the private key of this address, to prevent people 'scamming' the list. Assumed, he found an address ending with -1113, he publishes it and it will be written into cell (3,1). Now let's assume > 29 other users also find addresses ending with -1113 and publish them. Then, all these addresses are alphabetically ordered and written into cells (3,1), (3,2), ... (3,30). Addresses with index > 30 are discarded. Obviously, with time, it becomes harder and harder to 'win' a cell in the list, because an address with low alphabetical index has to be found to kick out another one.

Since it is attractive for users to 'own' cells in the list, the time will come the list is complete, meaning that every cell has an address written in it. Of course, after completing the list, new addresses with lower index can still be published, and the list will be continuously updated.

Last, how can a string of data be transmitted?
First, the data is converted into base58 and chopped into pieces with length 4. Example: 3HZ28xpWUhq4... -> 3HZ2, 8xpW, Uhq4... The addresses (taken from the global directory) to send bitcoins to must have the following properties:

1.) They have to end with -3HZ2, -8xpW, -Uhq4...
2.) The addresses have to be in alphabetic order: (...-3HZ2) < (...-8xpW) < (...-Uhq4) < ...

Point 2 is important to connect the data pieces in the right order. It should be possible to find such an alphabetic order, since there are 30 different available addresses for each piece of data.


So, what you think... Assumed that Bitcoin miners feel threatened by the current Mastercoin implementation and stop to confirm transactions that have an Exodus recipient, this implementation could be a solution, since it harms the Bitcoin network less.

Thanks!

Here's my response from that thread:

I believe that something like this would work, and as you say, it wouldn't create unspendable transactions. However, it would take a LOT more bytes to store the same data in the block chain using this method.

As I've said before, I need to get smarter about the internal workings of bitcoin, but I have a lot of confidence that our developers will come up with something which works. Currently they are (if I understand correctly) attempting to use Gavin's suggestion (https://bitcointalk.org/index.php?topic=284178.msg3040840#msg3040840). If that doesn't work, they may try the OP_RETURN method suggested by maaku on the same thread. If that doesn't work, there may be more options which haven't been suggested yet.

Once this is nailed down, I'll be looking forward with great anticipation to see the first try at the distributed exchange using testnet and/or test MasterCoins. Given how dang fast these guys are, we may all be pleasantly surprised with how quickly that comes together :)


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: jgarzik on September 16, 2013, 07:09:56 PM
OP_RETURN is the current proposal that people have been using, for adding prune-able data to the blockchain.  Here is an example implementation for relaying such transactions https://github.com/bitcoin/bitcoin/pull/2738 and https://github.com/bitcoin/bitcoin/pull/2791 is the pruning piece.

alt-coins and similar schemes should at a minimum produce pruneable outputs or use inputs + P2SH.  The data remains available via blockchain, just not bloating the precise UTXO space.

CHECKMULTISIG schemes still bloat the UTXO space (unless they are P2SH).



Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: dacoinminster on September 16, 2013, 07:28:45 PM
OP_RETURN is the current proposal that people have been using, for adding prune-able data to the blockchain.  Here is an example implementation for relaying such transactions https://github.com/bitcoin/bitcoin/pull/2738 and https://github.com/bitcoin/bitcoin/pull/2791 is the pruning piece.

alt-coins and similar schemes should at a minimum produce pruneable outputs or use inputs + P2SH.  The data remains available via blockchain, just not bloating the precise UTXO space.

CHECKMULTISIG schemes still bloat the UTXO space (unless they are P2SH).



Thank you so much for the reply!

It appears that these changes haven't been merged yet? I'm a little worried about relying on features which haven't been officially approved. What will current bitcoin implementations do with transactions using OP_RETURN - hopefully not reject them?


Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: jgarzik on September 16, 2013, 07:39:12 PM
It appears that these changes haven't been merged yet?

The prune-unspendable is very likely to go in, and the general consensus is that OP_RETURN is the lesser of the various other more-bloat-producing solutions for timestamping data into the chain.  We did not want to put in OP_RETURN without having the prune-unspendable change in first.

Quote
I'm a little worried about relying on features which haven't been officially approved. What will current bitcoin implementations do with transactions using OP_RETURN - hopefully not reject them?

Most implementations today will not relay OP_RETURN transactions, meaning they will probably not be confirmed without a little extra legwork and patience.

All implementations will accept OP_RETURN in mined blocks, as it is a normal and supported opcode.

In practice, today, that means sending the transaction with appropriate fee attached to https://en.bitcoin.it/wiki/Free_transaction_relay_policy

After OP_RETURN is upstream, implementations will relay OP_RETURN transactions just like any other "standard" transaction.



Title: Re: Is Mastercoin bloating the blockchain and what we can do about it?
Post by: ripper234 on September 20, 2013, 06:25:32 AM
I'm glad my op started this healthy discussion, even if I'm not an active participant myself.
I just wanted to state my opinion here for the record.

I just started another thread (https://bitcointalk.org/index.php?topic=298331) that hopefully will be less technical and more high-level, copy pasting my post here:

There have been several well known bitcoiners that tried to convince J.R.Willet to move Mastercoin to an altchain, instead of using the bitcoin blockchain. Willet's opinion is that Mastercoin should stay on the blockchain, and I fully concur. Here is my reasoning:

It is easier to implement Mastercoin on the blockchain. The entire concept of mining "vanishes", or actually is satisfied by Bitcoin miners. This means the security of Mastercoin is on par with Bitcoin (up to potential bugs) ... there is no way to 51% attack it.

The same could be implemented using merged mining, but no matter how you look at it, merged mining is technically a more complicated solution, because it requires an entirely new blockchain one needs to reason about. You need to think about the hashrate of this separate blockchain and to encourage miners to merge-mine it, things that are just not needed when you directly encode things into the Bitcoin blockchain. As far as software development goes, I'm super lazy - I do not like to do something complicated if I can do something simple that serves the same purpose.

Now, it is true that this "burdens" the blockchain ... in the same way that Satoshi Dice does.
However, this burdening is something that Bitcoin users and developers will just have to live with. Nobody is the boss of Bitcoin, and any standard Bitcoin transaction (like Mastercoin transactions are) are legitimate transactions that Bitcoin simply has to learn to deal with. There is just no feasible way that I see that Bitcoin developers could ban such transactions (please enlighten me if I'm wrong).

Bitcoin is the future of finance, resistant to hacking, government censorship and regulation. Do you really want to base that future on asking other people to pretty please "not abuse it for you own purposes"? This is not the answer. Bitcoin has to be made scalable enough to meet its role as the one major, global, world currency.

Mastercoin developers have no intention of disrupting Bitcoin, and will gladly adapt to any friendly encoding that the brilliant minds of these forums will propose (https://bitcointalk.org/index.php?topic=284178.0), as long as it doesn't harm Mastercoin (using non-standard transactions that are relayed more slowly is not an option at the moment). The Bitcoin blockchain must and will remain neutral, and if Willet hadn't come up with Mastercoin encoding, someone else would have created something similar sooner or later.