Dust Collection

[justusranvier]

Without any changes to the protocol itself miners could make it economical to create transactions which reduce the UTXO set by changing their transaction rules to favor transactions which reduce dust. Once this happens, clients could be programmed to do that automatically.

One way it could be done: if all outputs are above the dust cutoff, and if N inputs are below the dust cutoff, prioritize the transaction as if it included N*the minimum fee in addition to fees which are actually present.

[etotheipi]

Without any changes to the protocol itself miners could make it economical to create transactions which reduce the UTXO set by changing their transaction rules to favor transactions which reduce dust. Once this happens, clients could be programmed to do that automatically.

One way it could be done: if all outputs are above the dust cutoff, and if N inputs are below the dust cutoff, prioritize the transaction as if it included N*the minimum fee in addition to fees which are actually present.

I don't think the BTC-size of the inputs should matter.  Simply look at whether it increases or reduces the global UTXO set all full nodes will have to track.  Even huge transactions should be free if they combine dozens of inputs into a one or two outputs.  As long as it is very expensive to go the other way.

In fact, you could look at the contraction efficiency of a transaction:  how many kB did it use to remove X UTXOs from the set?  Above a certain efficiency, those tx should be free.  Any neutral or negative tx will be charged a fee, with the highest fees going to those who dramatically expand the set.  But the rules should be balanced so that it's not cheaper to send 100 single-output transactions instead of a single 100-output tx.

[solex]

Excellent ideas!
Is it feasible to get such an improvement into an official version during 2013?

[justusranvier]

Is it feasible to get such an improvement into an official version during 2013?

Solo miners and pool operators could change how they prioritize transactions at any time.

To make dust collection automatic it would require changes in the various client programs. Any client that can do coin control could do this manually.

[etotheipi]

Is it feasible to get such an improvement into an official version during 2013?

Solo miners and pool operators could change how they prioritize transactions at any time.

To make dust collection automatic it would require changes in the various client programs. Any client that can do coin control could do this manually.

True, it doesn't require any protocol changes... but it does require miners to care.  At the moment they don't care, because they still use the full blockchain.  No one uses the UTXO set.  But once they start transitioning, I'm sure we'll see fee "schedules" get stirred up to reflect their new priorities.

Armory already does some degree of dust collection, but it doesn't do a very good job of it.  The logic has been tested, but I think it doesn't work well in practice, unless you recycle addresses a lot.  Part of it, is that I didn't want to reduce anonymity just to clean up dust, and frequently throwing in that dust would link more addresses on the input side, so I avoid it.  Only if you have a default coin-selection that includes addresses that also have dust inputs, then it will throw it in there.

Someone recently brought up the idea that I don't have to consider same addresses only.... I can consider dust from any address that is already linked to one of the addresses on the input side.  There's no privacy issues if the addresses were already linked.  But that's a much more complicated algorithm to figure that out.  Maybe one day it will reach priority for me...

[solex]

etotheipi, Just want to say: "Thanks very much. I use Armory and it is great."
edit: donation pinged your way now...

[justusranvier]

True, it doesn't require any protocol changes... but it does require miners to care.  At the moment they don't care, because they still use the full blockchain.  No one uses the UTXO set.

I was sure that I recently saw a pool operator vocally complaining about a certain service bloating the blockchain with unprunable outputs. Maybe I was just imagining it though.

[etotheipi]

True, it doesn't require any protocol changes... but it does require miners to care.  At the moment they don't care, because they still use the full blockchain.  No one uses the UTXO set.

I was sure that I recently saw a pool operator vocally complaining about a certain service bloating the blockchain with unprunable outputs. Maybe I was just imagining it though.

To say they "don't care" was perhaps too strong of a statement.  I think everyone acknowledges that it will be important in the future, and would like to see a pleasant blockchain waiting for them when we get there.  But I think UTXO-only nodes are not happening for a while, so their immediate priorities are different.

[justusranvier]

If dust collection is ever going to happen it should be started as soon as possible, because each day a certain percentage of bitcoin users will die, lose their private keys, stop using bitcoins, or otherwise stop making new transactions. The sooner dust collection starts the less of those outputs will stay in the UTXO set forever.

[etotheipi]

If dust collection is ever going to happen it should be started as soon as possible, because each day a certain percentage of bitcoin users will die, lose their private keys, stop using bitcoins, or otherwise stop making new transactions. The sooner dust collection starts the less of those outputs will stay in the UTXO set forever.

Well I do agree, I would feel more inspired to optimize my dust collection algo in Armory, if it helped me avoid requiring my users to pay a fee..

[niko]

If I use the reference client to spend the whole wallet to one new address in the same wallet, have I removed the dust from the blockchain?

[killerstorm]

At the moment they don't care, because they still use the full blockchain.  No one uses the UTXO set.

Well, if they use bitcoind 0.8, while they still full blockchain on disk, working set depends on UTXO set.

Performance difference is probably too small to be considered, but still, UTXO set has effect on performance.

[etotheipi]

In fact, I think miners should find a way to pay for transactions like this:  

http://blockchain.info/tx/8d46c863dbbbbe7c99ef8cb1a5b2c24cf45bbacda01a751f939d0b32b64e2a08

That is 410 inputs turned into two, for a net reduction of 408 UTXOs.  I feel skinnier already...

[cypherdoc]

In fact, I think miners should find a way to pay for transactions like this:  

http://blockchain.info/tx/8d46c863dbbbbe7c99ef8cb1a5b2c24cf45bbacda01a751f939d0b32b64e2a08

That is 410 inputs turned into two, for a net reduction of 408 UTXOs.  I feel skinnier already...

are you sure you gave us the right tx?  i only see 2 inputs.

[etotheipi]

are you sure you gave us the right tx?  i only see 2 inputs.

You have to click "Show scripts & coinbase".  There's only two input addresses, but there's 409 UTXOs being spent by the 12vNB address.

[d'aniel]

Without any changes to the protocol itself miners could make it economical to create transactions which reduce the UTXO set by changing their transaction rules to favor transactions which reduce dust. Once this happens, clients could be programmed to do that automatically.

One way it could be done: if all outputs are above the dust cutoff, and if N inputs are below the dust cutoff, prioritize the transaction as if it included N*the minimum fee in addition to fees which are actually present.

I don't think the BTC-size of the inputs should matter.  Simply look at whether it increases or reduces the global UTXO set all full nodes will have to track.  Even huge transactions should be free if they combine dozens of inputs into a one or two outputs.  As long as it is very expensive to go the other way.

In fact, you could look at the contraction efficiency of a transaction:  how many kB did it use to remove X UTXOs from the set?  Above a certain efficiency, those tx should be free.  Any neutral or negative tx will be charged a fee, with the highest fees going to those who dramatically expand the set.  But the rules should be balanced so that it's not cheaper to send 100 single-output transactions instead of a single 100-output tx.

One problem with ignoring coin value altogether is that we're more likely to end up with coins that are uneconomical to ever spend, and thus more likely to bloat the UTXO set indefinitely.

Your reasoning would make sense to me if UTXO set size was the only scarce resource being rationed, but block space needs to be rationed at the same time.  This means transaction priority can't be coin value-agnostic.

Edit: Additionally, the UTXO set size is naturally bounded if uneconomical coin values are discouraged.  Of course this brings up the problem of discouraging exotic uses of small valued coins.  I addressed this here though if you're interested: https://bitcointalk.org/index.php?topic=151177.0.

[cypherdoc]

Without any changes to the protocol itself miners could make it economical to create transactions which reduce the UTXO set by changing their transaction rules to favor transactions which reduce dust. Once this happens, clients could be programmed to do that automatically.

One way it could be done: if all outputs are above the dust cutoff, and if N inputs are below the dust cutoff, prioritize the transaction as if it included N*the minimum fee in addition to fees which are actually present.

I don't think the BTC-size of the inputs should matter.  Simply look at whether it increases or reduces the global UTXO set all full nodes will have to track.  Even huge transactions should be free if they combine dozens of inputs into a one or two outputs.  As long as it is very expensive to go the other way.

In fact, you could look at the contraction efficiency of a transaction:  how many kB did it use to remove X UTXOs from the set?  Above a certain efficiency, those tx should be free.  Any neutral or negative tx will be charged a fee, with the highest fees going to those who dramatically expand the set.  But the rules should be balanced so that it's not cheaper to send 100 single-output transactions instead of a single 100-output tx.

One problem with ignoring coin value altogether is that we're more likely to end up with coins that are uneconomical to ever spend, and thus more likely to bloat the UTXO set indefinitely.

Your reasoning would make sense to me if UTXO set size was the only scarce resource being rationed, but block space needs to be rationed at the same time.  This means transaction priority can't be coin value-agnostic.

Edit: Additionally, the UTXO set size is naturally bounded if uneconomical coin values are discouraged.  Of course this brings up the problem of discouraging exotic uses of small valued coins.  I addressed this here though if you're interested: https://bitcointalk.org/index.php?topic=151177.0.

doesn't this depend on how much Bitcoin appreciates assuming of course that as that appreciation occurs fees are reduced?

[d'aniel]

doesn't this depend on how much Bitcoin appreciates assuming of course that as that appreciation occurs fees are reduced?

Yes, absolutely.  This would mean the UTXO set size scales roughly with the number of users, as it should, ideally.

[justusranvier]

Your reasoning would make sense to me if UTXO set size was the only scarce resource being rationed, but block space needs to be rationed at the same time.

True at the moment, but in the long term block space does not need to be rationed. It's not even strictly necessary for anyone to keep every transaction all the way back to the Genesis block.

The only data that absolutely must persist forever for the network to function is the UTXO set.

[cypherdoc]

doesn't this depend on how much Bitcoin appreciates assuming of course that as that appreciation occurs fees are reduced?

Yes, absolutely.  This would mean the UTXO set size scales roughly with the number of users, as it should, ideally.

seems to me that as # users increases, Bitcoin price increases, fees decrease, and UTXO decreases.