Is Mastercoin bloating the blockchain and what we can do about it?

[btcdrak]

I won't be surprised. Parasites always morph and change as the host evolves.

Yes, that's something like what I have in mind, although long-term, we hope to be mitochondria, not tape worms

Willet, what I really don't understand about you is that CLEARLY, a separate block-chain would be best for your project, but you are just ADAMANT, just for the sake of it, to use the bitcoin blockchain. Why on earth is that? To be honest, the BEST way would be to do what Namecoin does, merged mining. It's the perfect solution. Why don't you want to do it? "Just because you are attached to Bitcoin". it's a stupid reasoning.

[dacoinminster]

Willet, what I really don't understand about you is that CLEARLY, a separate block-chain would be best for your project, but you are just ADAMANT, just for the sake of it, to use the bitcoin blockchain. Why on earth is that? To be honest, the BEST way would be to do what Namecoin does, merged mining. It's the perfect solution. Why don't you want to do it? "Just because you are attached to Bitcoin". it's a stupid reasoning.

I describe my reasoning for this in the "Assumptions" section of my whitepaper (see the link in my sig):

Assumptions

Our claims are built on the following assumptions:

•   Alternate block chains compete with bitcoins financially, confuse our message to the world, and dilute our efforts. These barriers interfere with the adoption momentum of bitcoin and the adoption momentum of alternate currencies as well, regardless of how well-conceived their rules may be.
•   New protocol layers on top of the bitcoin protocol will increase bitcoin values, consolidate our message to the world, and concentrate our efforts, while still allowing individuals and groups to issue new currencies with experimental new rules. The success of any experimental currency protocol layer will enhance the value and success of the foundational bitcoin protocol.
•   Getting consensus and widespread adoption from the bitcoin community is not needed to add protocol layers, since no changes to the foundational bitcoin protocol are required.
•   Tiny bitcoin transactions can be encoded into the block chain to support and represent transactions in higher protocol layers.
•   A protocol can pay for its own software development, “bootstrapping” itself into existence, utilizing a trusted entity to hold funds and hire developers.
•   It is possible to create tools to allow end users to create currency protocol layers which have a stable value, pegged to an external currency or commodity. In this way, users of these currencies can own stabilized virtual currency tied to U.S. Dollars, Euros, ounces of gold, barrels of oil, etc.
•   It is possible for users of these new currencies to exchange between currencies with each other using simple rules and no central exchange.

One reason I didn't mention in the paper is that this method of accomplishing my goals is way, way easier and faster than building an alt coin. Building on bitcoin significantly reduces the technical risk of my project.

[btcdrak]

One reason I didn't mention in the paper is that this method of accomplishing my goals is way, way easier and faster than building an alt coin. Building on bitcoin significantly reduces the technical risk of my project.

Not doing something like merged mining which relies on 100% Bitcoin is the biggest risk. The days of bitcoin bloatware are numbered. Your funeral, and the funeral of all the poor suckers who gave you money already when it finally happens. You're probably safe until Bitcoin 0.9, but by 1.0, it'll be game over.

[dacoinminster]

One reason I didn't mention in the paper is that this method of accomplishing my goals is way, way easier and faster than building an alt coin. Building on bitcoin significantly reduces the technical risk of my project.

Not doing something like merged mining which relies on 100% Bitcoin is the biggest risk. The days of bitcoin bloatware are numbered. Your funeral, and the funeral of all the poor suckers who gave you money already when it finally happens. You're probably safe until Bitcoin 0.9, but by 1.0, it'll be game over.

I expect MasterCoin will have to change and implement it's own custom pruning solution at some point, but I'm not expecting any funerals

[btcdrak]

One reason I didn't mention in the paper is that this method of accomplishing my goals is way, way easier and faster than building an alt coin. Building on bitcoin significantly reduces the technical risk of my project.

Not doing something like merged mining which relies on 100% Bitcoin is the biggest risk. The days of bitcoin bloatware are numbered. Your funeral, and the funeral of all the poor suckers who gave you money already when it finally happens. You're probably safe until Bitcoin 0.9, but by 1.0, it'll be game over.

I expect MasterCoin will have to change and implement it's own custom pruning solution at some point, but I'm not expecting any funerals

Your arrogance can only be compared to corporate slime who adapt their tax-evasion with the passing of every new law, and gloat about it.

[dacoinminster]

Your arrogance can only be compared to corporate slime who adapt their tax-evasion with the passing of every new law, and gloat about it.

I guess if my choices are between building yet another alt chain and "arrogance", I choose arrogance.

The possibility of building a new protocol layer on top of bitcoin has always been there, and someone was bound to do it sooner or later. It just happens that I'm the first one to actually take a crack at it.

[dacoinminster]

People concerned about MasterCoin bloating the bitcoin block-chain will be pleased to hear that we now have proof-of-concept for at least one alternate way of storing our data without creating unspendable outputs:

I have a proposal for alternative Mastercoin encoding ready and I wanted to discuss the feasibility of my solution. We are going to use a OP_CHECKMULTISIG script with three (or more) public keys to encode the data.

Because we will be using public keys to encapsulate the data we now have 64 bytes to encode data (65 minus one byte for the 04 marker). In order to not polute the blockchain we will always supply a publickey that the sender owns as first signature this way the output can always be redeemed. The other signatures we supply will be data-addresses; these transactions won't be able to be converted to addresses since they are not technically ECDSA x/y coordinates but they should be relayed and mined just fine.

I've created a test transaction on testnet3 (raw) to test this functionality.

miKnddGDQfU6rRYpLp2dhRyttnxH1WUo21 is the exodus that I use on Testnet. I'm sending the coins to myself (miDyBpL3TeeJRbtwdFUsw9mUWUEhN6FyXf) in this example so I send both the change amount and 0.00006 transaction to this address. I also use this address as the first signature for the multisg output so I can redeem this later. If you check the raw transaction you will see that there are two more keys supplied. If you decode these using the encoding standard from the whitepaper (but instead pad with zeros until you have 65 bytes) you will see that I'm doing two transaction for 1 test Mastercoin each.

In theory you could do multiple Mastercoin transactions in one multisig output but I would like to propose to keep it simple and only one transaction per output. If you want to send multiple Mastercoin transactions in one Bitcoin transaction you can create multiple multisig outputs.

Since all current ideas of the whitepaper fit within 126 bytes (2 signatures) and signatures are in fixed order we might not need to add sequences to all signatures but for now let's continue doing it.

I know there are a lot of users with more knowledge of Bitcoin lurking so please let me know if this approach would work within the rules set out by the reference implementation. I'm especially interested in if we can use 1 out of 5 multisig transactions. I tried to make one; and it relayed; but I'm not sure it's actually a valid transaction type.

Please let me know your thoughts so we can continue building cool things on top of Mastercoin

[gmaxwell]

It just happens that I'm the first one to actually take a crack at it.

I believe Luke's "Tonal Bitcoin" from 2011 probably has claim at this.

[TomHirsch]

I have better things to do with my time than to provide free consulting for altcoins who've just raised hundreds of thousands of dollars

don't non-consensually consume the resources of people who are disinterested— even opposed— to your system...

...You send 1BTC to your grandmother - I am disinterested and it consumes blockchain resources - and I oppose you sending this kind of transaction.  

Look, J.R. is breaking his back trying to find a way to very lightly consume these resources and to be a good 'blockchain citizen'.  There is no point trying to bust his balls about the project.  

Your anger reeks of jealousy because you have raised any money to support any of your ideas.  J.R. is being very respectful and polite and trying to find the best way possible to keep bitcoin strong and healthy.  Then, you angry dicks come around cursing him down for his good efforts.  WTF?  

Contribute or go home pussy.

[TomHirsch]

Hey Peter! Thanks for checking out the project!

...the entire idea is just dumb.

Anyway, I have better things to do than waste my time talking about ideas sufficiently stupid to be indistinguishable from scams for free, so don't bother replying unless you've got a paying contract. (1.5BTC/hr, escrow required for you)

Do me a favor and keep your stupid protocol just the way it is.

'
WHAT - A - DICK. 

The guy was being nice and saying 'hello in a friendly way'.  Why is there so much anger in the bitcoin upper echelon?  Did someone steal all your coins?

[dacoinminster]

Peter is a nice guy in person. He's also very passionate about the health of the bitcoin network - more than anybody I know. In retrospect, maybe I shouldn't have expected a continuation of the friendly dialog we started at the conference, given MasterCoin's potential to rapidly expand the block chain (even more than it already is expanding). That's probably a sensitive issue.

Also, while I doubt anybody stole his coins, he may have just been having a really bad day. I try to give people a little grace and understanding in situations like this.

GMaxwell, I think I remember when tonal bitcoin came out. Gosh, that seems like a long time ago. I should probably just claim to be the first to create a new protocol layer that actually adds new functionality (as opposed to an alternate display)? Also, I hope you enjoy your 3 BTC I sent you, courtesy of d'aniel. I believe I have actually sent more money to detractors than boosters at this point! That should change soon

[TomHirsch]

GMaxwell, I hope you enjoy your 3 BTC I sent you, courtesy of d'aniel.

$450 to a dick?  We need to save the funds better than this.  I hope his contribution was valuable.  Just because the projects wallet is fat, doesn't mean you have to throw the money around unnecessarily.

Did you know that people consume about 20% more shampoo when their shampoo bottle is full?  No need sending BTC to dicks unless their help was a good improvement for the project.

[dacoinminster]

GMaxwell, I hope you enjoy your 3 BTC I sent you, courtesy of d'aniel.

$450 to a dick?  We need to save the funds better than this.  I hope his contribution was valuable.  Just because the projects wallet is fat, doesn't mean you have to throw the money around unnecessarily.

Did you know that people consume about 20% more shampoo when their shampoo bottle is full?  No need sending BTC to dicks unless their help was a good improvement for the project.

d'aniel earned a bounty very early on for describing a potential attack vector on underfunded escrows. His contribution was very valuable, and led to some changes in the most recent rev of the spec. The fact that he is a detractor doesn't diminish his contribution. d'aniel asked me to send his bounty to GMaxwell, so I did. That bounty was approved by the MasterCoin Foundation board (which was two people, not including myself, at that time).

If we do everything we can to be good block-chain citizens, and if the protocol is successful, some detractors will eventually become advocates. People here can be very passionate and opinionated, but most of them can also be swayed by sound arguments, especially if the argument is in the form of code which is proving useful to lots of people and making bitcoin more valuable.

[tspacepilot]

Where can I find some more background reading about these topics: utxos and pruning, etc.

[maaku]

Where can I find some more background reading about these topics: utxos and pruning, etc.

https://bitcointalk.org/index.php?topic=88208.0

[milkyman]

Seems like what I wrote in the main mastercoin discussion thread fits better into this thread:

There was a lot of critics about the Mastercoin network, in particular due to the many generated un-prunable transactions that are necessary to transmit the data between Mastercoin users.

I have an idea how to solve this problem, but this would require some changes to the protocol. Basically, instead of sending Bitoins to a data address without known private key (which is the current Mastercoin implementation), data can also be transmitted by sending Bitcoins to addresses that other Bicoin users own. To do so, a 'directory' of addresses has to be generated first. Each address can store a few bytes of data - and by sending tiny amounts of Bitcoins to these addresses, the pieces of data can be connected (similar to the current Mastercoin implementation). In this presented implementation, each address can store about 3 bytes. Therefore, to have enough data for a Mastercoin transaction, about 6 - 12 Bitcoin transactions have to be sent.

However, there is an important advantage in this implementation: Users that 'own' these addresses can later send all their earned Bitcoins to a single address. As a result, the millions of addresses are fully spent, and the whole system becomes prunable.

In contrast, in the current Mastercoin system, all the data addresses will remain with 0.00006 BTC stored and have to be saved in the blockchain forever.


Here's the implementation:

First, a global list of length 58^4 = 11316496 and width = 30 is created, and each cell can contain a Bitcoin-address. Let us index these cells with (n,#). The size of the list is a few GB, which is well managable to keep in lots of copies among all Mastercoin users.

Now to the actual idea: The cells (1,#) must contain addresses with the last four characters being -1111. The cells (2,#) must contain addresses -1112, and so on. Finally, the cells (11316496,#) must contain addresses with the last 4 characters -zzzz.

Now, every Bitcoin user that finds a key pair can publish the corresponding address in this list. To this end, he has to prove that he owns the private key of this address, to prevent people 'scamming' the list. Assumed, he found an address ending with -1113, he publishes it and it will be written into cell (3,1). Now let's assume > 29 other users also find addresses ending with -1113 and publish them. Then, all these addresses are alphabetically ordered and written into cells (3,1), (3,2), ... (3,30). Addresses with index > 30 are discarded. Obviously, with time, it becomes harder and harder to 'win' a cell in the list, because an address with low alphabetical index has to be found to kick out another one.

Since it is attractive for users to 'own' cells in the list, the time will come the list is complete, meaning that every cell has an address written in it. Of course, after completing the list, new addresses with lower index can still be published, and the list will be continuously updated.

Last, how can a string of data be transmitted?
First, the data is converted into base58 and chopped into pieces with length 4. Example: 3HZ28xpWUhq4... -> 3HZ2, 8xpW, Uhq4... The addresses (taken from the global directory) to send bitcoins to must have the following properties:

1.) They have to end with -3HZ2, -8xpW, -Uhq4...
2.) The addresses have to be in alphabetic order: (...-3HZ2) < (...-8xpW) < (...-Uhq4) < ...

Point 2 is important to connect the data pieces in the right order. It should be possible to find such an alphabetic order, since there are 30 different available addresses for each piece of data.


So, what you think... Assumed that Bitcoin miners feel threatened by the current Mastercoin implementation and stop to confirm transactions that have an Exodus recipient, this implementation could be a solution, since it harms the Bitcoin network less.

[dacoinminster]

Seems like what I wrote in the main mastercoin discussion thread fits better into this thread:

There was a lot of critics about the Mastercoin network, in particular due to the many generated un-prunable transactions that are necessary to transmit the data between Mastercoin users.

I have an idea how to solve this problem, but this would require some changes to the protocol. Basically, instead of sending Bitoins to a data address without known private key (which is the current Mastercoin implementation), data can also be transmitted by sending Bitcoins to addresses that other Bicoin users own. To do so, a 'directory' of addresses has to be generated first. Each address can store a few bytes of data - and by sending tiny amounts of Bitcoins to these addresses, the pieces of data can be connected (similar to the current Mastercoin implementation). In this presented implementation, each address can store about 3 bytes. Therefore, to have enough data for a Mastercoin transaction, about 6 - 12 Bitcoin transactions have to be sent.

However, there is an important advantage in this implementation: Users that 'own' these addresses can later send all their earned Bitcoins to a single address. As a result, the millions of addresses are fully spent, and the whole system becomes prunable.

In contrast, in the current Mastercoin system, all the data addresses will remain with 0.00006 BTC stored and have to be saved in the blockchain forever.


Here's the implementation:

First, a global list of length 58^4 = 11316496 and width = 30 is created, and each cell can contain a Bitcoin-address. Let us index these cells with (n,#). The size of the list is a few GB, which is well managable to keep in lots of copies among all Mastercoin users.

Now to the actual idea: The cells (1,#) must contain addresses with the last four characters being -1111. The cells (2,#) must contain addresses -1112, and so on. Finally, the cells (11316496,#) must contain addresses with the last 4 characters -zzzz.

Now, every Bitcoin user that finds a key pair can publish the corresponding address in this list. To this end, he has to prove that he owns the private key of this address, to prevent people 'scamming' the list. Assumed, he found an address ending with -1113, he publishes it and it will be written into cell (3,1). Now let's assume > 29 other users also find addresses ending with -1113 and publish them. Then, all these addresses are alphabetically ordered and written into cells (3,1), (3,2), ... (3,30). Addresses with index > 30 are discarded. Obviously, with time, it becomes harder and harder to 'win' a cell in the list, because an address with low alphabetical index has to be found to kick out another one.

Since it is attractive for users to 'own' cells in the list, the time will come the list is complete, meaning that every cell has an address written in it. Of course, after completing the list, new addresses with lower index can still be published, and the list will be continuously updated.

Last, how can a string of data be transmitted?
First, the data is converted into base58 and chopped into pieces with length 4. Example: 3HZ28xpWUhq4... -> 3HZ2, 8xpW, Uhq4... The addresses (taken from the global directory) to send bitcoins to must have the following properties:

1.) They have to end with -3HZ2, -8xpW, -Uhq4...
2.) The addresses have to be in alphabetic order: (...-3HZ2) < (...-8xpW) < (...-Uhq4) < ...

Point 2 is important to connect the data pieces in the right order. It should be possible to find such an alphabetic order, since there are 30 different available addresses for each piece of data.


So, what you think... Assumed that Bitcoin miners feel threatened by the current Mastercoin implementation and stop to confirm transactions that have an Exodus recipient, this implementation could be a solution, since it harms the Bitcoin network less.

Thanks!

Here's my response from that thread:

I believe that something like this would work, and as you say, it wouldn't create unspendable transactions. However, it would take a LOT more bytes to store the same data in the block chain using this method.

As I've said before, I need to get smarter about the internal workings of bitcoin, but I have a lot of confidence that our developers will come up with something which works. Currently they are (if I understand correctly) attempting to use Gavin's suggestion (https://bitcointalk.org/index.php?topic=284178.msg3040840#msg3040840). If that doesn't work, they may try the OP_RETURN method suggested by maaku on the same thread. If that doesn't work, there may be more options which haven't been suggested yet.

Once this is nailed down, I'll be looking forward with great anticipation to see the first try at the distributed exchange using testnet and/or test MasterCoins. Given how dang fast these guys are, we may all be pleasantly surprised with how quickly that comes together

[jgarzik]

OP_RETURN is the current proposal that people have been using, for adding prune-able data to the blockchain.  Here is an example implementation for relaying such transactions https://github.com/bitcoin/bitcoin/pull/2738 and https://github.com/bitcoin/bitcoin/pull/2791 is the pruning piece.

alt-coins and similar schemes should at a minimum produce pruneable outputs or use inputs + P2SH.  The data remains available via blockchain, just not bloating the precise UTXO space.

CHECKMULTISIG schemes still bloat the UTXO space (unless they are P2SH).

[dacoinminster]

OP_RETURN is the current proposal that people have been using, for adding prune-able data to the blockchain.  Here is an example implementation for relaying such transactions https://github.com/bitcoin/bitcoin/pull/2738 and https://github.com/bitcoin/bitcoin/pull/2791 is the pruning piece.

alt-coins and similar schemes should at a minimum produce pruneable outputs or use inputs + P2SH.  The data remains available via blockchain, just not bloating the precise UTXO space.

CHECKMULTISIG schemes still bloat the UTXO space (unless they are P2SH).

Thank you so much for the reply!

It appears that these changes haven't been merged yet? I'm a little worried about relying on features which haven't been officially approved. What will current bitcoin implementations do with transactions using OP_RETURN - hopefully not reject them?

[jgarzik]

It appears that these changes haven't been merged yet?

The prune-unspendable is very likely to go in, and the general consensus is that OP_RETURN is the lesser of the various other more-bloat-producing solutions for timestamping data into the chain.  We did not want to put in OP_RETURN without having the prune-unspendable change in first.

I'm a little worried about relying on features which haven't been officially approved. What will current bitcoin implementations do with transactions using OP_RETURN - hopefully not reject them?

Most implementations today will not relay OP_RETURN transactions, meaning they will probably not be confirmed without a little extra legwork and patience.

All implementations will accept OP_RETURN in mined blocks, as it is a normal and supported opcode.

In practice, today, that means sending the transaction with appropriate fee attached to https://en.bitcoin.it/wiki/Free_transaction_relay_policy

After OP_RETURN is upstream, implementations will relay OP_RETURN transactions just like any other "standard" transaction.