Re: Gavin's blog, "Reworking Bitcoin Transaction Fees" -- some Qs and ideas

[blablahblah]

Hi, I've been looking at Gavin's blog updated a few days ago, hoping to learn more about the "fee model", so I've got some questions, critiquing, and a couple of ideas to run past everyone.

Goals

In rough order of priority:

    Create a self-regulating market for transaction fees between miners and merchants/users
    Smoothly transition from the rules we currently have in place to the new rules; transactions from users running old versions of Bitcoin should get confirmed reasonably quickly under the new rules.
    Make sure transaction spam is expensive for would-be spammers
    Replace compiled-in constants with dynamic, market-determined values

Pre-emptive intervention as a design goal? This may be nitpicking, but I would have thought that in a market where "supply meets demand", transaction spam would no longer be relevant. Miners would tend to simply sign "winning bids for transactions" in order of profitability and the problem would sort itself out. (Supply and demand might be chaotic but it's not Voodoo.)

Should the priority calculation be changed?

As the total number of transactions grows, transaction pruning schemes that "forget" old, completed transaction become more and more important. Pruning is possible when all of a transaction's outputs are spent, so a transaction that takes 10 inputs and combines them into 1 output is good for the network.

The transaction priority calculation could be modified to prefer transactions with a high ratio of inputs to outputs. However, we need to be careful and make sure that there is still an incentive to use 'sendmany' instead of creating dozens of single-in/single-out transactions; a 1-input-10-output transaction should not have one-tenth the priority of a 1-input-1-output transaction.

To give a mild preference to transactions with fewer/smaller outputs than inputs the priority formula could be modified to be:

priority = (sum(value*age)/size) * (1000 + size of previous txouts) / (1000 + size of this txn's txouts)

My question is: should priority calculation exist at all? Pruning generally seems like a sensible idea, but pre-emptively intervening and hard-coding the decision into the protocol seems like unnecessary bloat. Why not get rid of it altogether and provide miners with an economic incentive instead? E.g.: "reward miners who prune old blocks by giving them optional extra new blocks (without the coin reward of course)". Of course that might only work if block-space were already a reasonably scarce resource. Anyway, just an idea -- no idea if it's feasible.

maximum block size : The maximum size block that can be created. If not set, it will default to the same as the target block size. Miners can increase this to make room for more fee-paying transactions.

--The controversy never ends At the moment I'm favouring an algorithm where each time a block is discovered, the miner can reference the previous block's maximum size and vote it 'up', 'down', or 'no change'. The question is how much should the vote count? E.g.: a 2^(+-1/2016) change? (E.g.: doubling or halving every 2 weeks with that example). To me this seems like a relatively simple algorithm that alleviates 3 problems: 1) miners all having different opinions on exactly how much capacity blocks 'ought' to have, 2) minimising some potential problems with rogue miners/merchants/anyone DOSing transaction requests, 3) of course scaling.


Lastly, one thing I'm unable to find anywhere, and perhaps someone could point me to it -- where are "unconfirmed transactions" kept? Presumably this memory heap is itself a scarce resource, and ought to be treated as such? It seems that some merchants et al. are using this memory heap in lieu of a low-cost messaging system. Does Bitcoin already have a messaging scheme for data other than transactions?

[justusranvier]

Goals

In rough order of priority:

    Create a self-regulating market for transaction fees between miners and merchants/users
    Smoothly transition from the rules we currently have in place to the new rules; transactions from users running old versions of Bitcoin should get confirmed reasonably quickly under the new rules.
    Make sure transaction spam is expensive for would-be spammers
    Replace compiled-in constants with dynamic, market-determined values

Pre-emptive intervention as a design goal? This may be nitpicking, but I would have thought that in a market where "supply meets demand", transaction spam would no longer be relevant. Miners would tend to simply sign "winning bids for transactions" in order of profitability and the problem would sort itself out. (Supply and demand might be chaotic but it's not Voodoo.)

If we want the network to be successful then what we want is not a specific outcome (high fees or low fees, big blocks or small blocks), but rather an efficient mechanism for price discovery.

Nobody can calculate ahead of time how much hard drive space, bandwidth, and cpu time the miners, pool operators, and owners of full nodes are willing to devote to the network for a given amount of revenue. Likewise nobody can calculate the tradeoff between speed and fee percentage all the users of Bitcoin are willing to pay.

Instead of attempting to calculate the incalculable and impose a central plan on the network, the best strategy is to design a market for transaction processing that will find the right answer on its own.

[blablahblah]

Goals
...
    Make sure transaction spam is expensive for would-be spammers

...a market where "supply meets demand"...

If we want the network to be successful then what we want is not a specific outcome (high fees or low fees, big blocks or small blocks), but rather an efficient mechanism for price discovery.

Nobody can calculate ahead of time how much hard drive space, bandwidth, and cpu time the miners, pool operators, and owners of full nodes are willing to devote to the network for a given amount of revenue. Likewise nobody can calculate the tradeoff between speed and fee percentage all the users of Bitcoin are willing to pay.

Instead of attempting to calculate the incalculable and impose a central plan on the network, the best strategy is to design a market for transaction processing that will find the right answer on its own.

Thanks! Price discovery was the jargon I was looking for, but my mental block. So I stuck in "supply meets demand" hoping to convey similar thoughts. I guess you were also referring to the pruning? Point taken. In that case, perhaps a good way might be simply drop the priority/"old coins" incentive and allow miners to optionally prune according to rules that are kept as simple as people are able to make them. Even without artificial incentives, miners should still do it to save themselves disk space. This might also make bitcoins slightly more fungible.

[justusranvier]

In meme form:

[Zangelbert Bingledack]

Goals

In rough order of priority:

    Create a self-regulating market for transaction fees between miners and merchants/users
    Smoothly transition from the rules we currently have in place to the new rules; transactions from users running old versions of Bitcoin should get confirmed reasonably quickly under the new rules.
    Make sure transaction spam is expensive for would-be spammers
    Replace compiled-in constants with dynamic, market-determined values

Pre-emptive intervention as a design goal? This may be nitpicking, but I would have thought that in a market where "supply meets demand", transaction spam would no longer be relevant. Miners would tend to simply sign "winning bids for transactions" in order of profitability and the problem would sort itself out. (Supply and demand might be chaotic but it's not Voodoo.)

If we want the network to be successful then what we want is not a specific outcome (high fees or low fees, big blocks or small blocks), but rather an efficient mechanism for price discovery.

Nobody can calculate ahead of time how much hard drive space, bandwidth, and cpu time the miners, pool operators, and owners of full nodes are willing to devote to the network for a given amount of revenue. Likewise nobody can calculate the tradeoff between speed and fee percentage all the users of Bitcoin are willing to pay.

Instead of attempting to calculate the incalculable and impose a central plan on the network, the best strategy is to design a market for transaction processing that will find the right answer on its own.

+1

Interventionism will just add friction, brittleness, and headache.

[BitHits]

As I recently mentioned in another thread.

tx fee should be a VERY SMALL % (I suggested 2%) of the total Input of the transaction!

Having to pay 300% of the Total Input...to send the Input Value in question...Is ridiculous!

I shouldn't have to save up a whole months worth of transactions just to make the tx not redundant!

[Sukrim]

tx fee should be a VERY SMALL % (I suggested 2%) of the total Input of the transaction!

So I want to send 0.1 BTC, but only have a 100 BTC unspent output... I have to pay 2% (2 BTC) just to move 0.1 BTC?!
Also 2% is not "VERY SMALL" at all, I can receive credit card payments for less fees.

[BitHits]

tx fee should be a VERY SMALL % (I suggested 2%) of the total Input of the transaction!

So I want to send 0.1 BTC, but only have a 100 BTC unspent output... I have to pay 2% (2 BTC) just to move 0.1 BTC?!
Also 2% is not "VERY SMALL" at all, I can receive credit card payments for less fees.

Taking into account the source address. Not the transaction after it has been lumped together with many other transactions......

When a transaction is created client side and the fee is requested, It has not been added to the blockchain and sent along with other transactions at that point.

[Timo Y]

price discovery.

You are aware that price discovery does not work with public goods?

[Prattler]

price discovery.

You are aware that price discovery does not work with public goods?

This! Also see Tradegy of the commons.

What does the original poster suggest? That it should be cheap to spam the network?!

[blablahblah]

tx fee should be a VERY SMALL % (I suggested 2%) of the total Input of the transaction!

So I want to send 0.1 BTC, but only have a 100 BTC unspent output... I have to pay 2% (2 BTC) just to move 0.1 BTC?!
Also 2% is not "VERY SMALL" at all, I can receive credit card payments for less fees.

Taking into account the source address. Not the transaction after it has been lumped together with many other transactions......

When a transaction is created client side and the fee is requested, It has not been added to the blockchain and sent along with other transactions at that point.

If the cost of sending 5c by snail-mail is 45c, does that mean it shouldn't be allowed?

If you control prices by calling them 'fees', you limit miner income, which reduces competition and network security.
If you allow the market to determine prices, miner income is maximised, which maximises competition and network security.

...At least that's my theory. I would also prefer a 'bidding' process, but apparently that would be "multiple-spend attempts" and some vendors rely on transaction requests being as-good-as real signed transactions on the block-chain.

[markm]

Public goods, the 'pedia says, are non-rivalrous. That does not sound akin to block space at all at all.

Might have to give ya "commons" though...

-MarkM-

[blablahblah]

price discovery.

You are aware that price discovery does not work with public goods?

This! Also see Tradegy of the commons.

What does the original poster suggest? That it should be cheap to spam the network?!

You are mistaken. It's not a public good -- a 'block' is the private property of the miner/whatever entity who discovered it. Selling pieces of that block to interested parties (who want to make transactions) is the whole point.

[Timo Y]

Public goods, the 'pedia says, are non-rivalrous. That does not sound akin to block space at all at all.

Might have to give ya "commons" though...

-MarkM-

Ok, let me be more precise:

Hashrate is a public good
Block space is common good

Both goods are vital to bitcoin's survival and both are currently financed by the same fee.  Finding the "correct price" of those types of goods is not a trivial task.

[edit:] Even less trivial for bitcoin, because in the current implementation, both prices are coupled. What is the "correct price" for hashrate might not be the "correct price" for block space and vice versa.

[Timo Y]

You are mistaken. It's not a public good -- a 'block' is the private property of the miner/whatever entity who discovered it. Selling pieces of that block to interested parties (who want to make transactions) is the whole point.

That's like saying that a public park is private property of the construction company who was commissioned to build it.

A block does NOT behave like a private good.  It behaves like a common good.  It needs to be stored, at a cost, by ALL miners and ALL full nodes for all eternity. Not just by the original miner. Otherwise it is completely useless to the buyer.

[blablahblah]

You are mistaken. It's not a public good -- a 'block' is the private property of the miner/whatever entity who discovered it. Selling pieces of that block to interested parties (who want to make transactions) is the whole point.

That's like saying that public park is private property of the construction company who was commissioned to build it.

A block does NOT behave like a private good.  It behaves like a common good.  It needs to be stored, at a cost, by ALL miners and ALL full nodes for all eternety. Not just by the original miner.  Otherwise it is completely useless to the buyer.

OK, good point.

Related question:
Are "full nodes" really needed if they're not mining? I don't really understand their purpose.

Because if full nodes are really needed for something, maybe there should be a separate fee paying them to stay on-line?

Block reward vs block-chain... I'll have a think about that.

[Timo Y]

Related question:
Are "full nodes" really needed if they're not mining? I don't really understand their purpose.

Because if full nodes are really needed for something, maybe there should be a separate fee paying them to stay on-line?

"Full nodes" are not needed in the sense that Bitcoin could still function without them.  However, they are beneficial to all users of Bitcoin because they make the network more resilient against certain types of attack. 

[justusranvier]

You are aware that price discovery does not work with public goods?

If the theory of public goods was correct, the situation which motivated the creation of Bitcoin in the first place would never have arisen.

[whitenight639]

The discovered / created / solved block by miners is space to store transactions, transactions are simply information, there are already bitcoin like networks storing other types of data in their blockchain. The problem is miners and the community are setting prices today for future storage - potentially forever, so if SD has sent dust to some address it may have paid a tiny amount to store that transactional information in the blockchain forever, so as if some kind of self adjusting limit is not imposed like Gavin is suggesting what is to stop Me writing a program to back up all my Mp3 into the blockchain just for lulz (obviously I'd need a shit load of money to do it, but in theory that data would be there forever, might not be a bad investment).

Obviously we have Moores law but we do not have a law that says we will have cheap electricity or network connectivity forever, infact quite the opposite, I think Satoshi argued that miners will use economies of scale to find the cheapest source of electricity and make mining profitable for them etc etc but the fact is the whole world wants cheap energy and even with that enormous market pressure prices are not coming down for the forseeable future. So we need to prune spent transactions and use the market to set Tx pricing at a rate that makes spam traffic like SD Uneconomical, It has to be that the Bitcoin money system that we have created is most economical for transferring & storing money not my MP3s or SDs' cheaper alternative to email. Theres that old saying "Theres money in money"

Maybe when a Btc is worth $1000 and Tx's fees are around the same it will make the above uneconomical and the bitcoin will be best used for real transactions.

[whitenight639]

I knew I should have looked into some of these details earlier. There seem to be so many unresolved things, that in my mind it's hard to know where to start. E.g.:
Full nodes without mining -- how is this supposed to improve security if there's no economic incentive to run one unless you're a miner anyway?

Memory pool -- what is going on with this thing? I thought transactions were stored in the block-chain. Now it turns out the memory pool stores 'IOUs', which might never be confirmed. A software patch seems to be in progress so that additional fees can be appended without changing the IOUs that are already there.

Double spend -- I thought they were inconceivably difficult (51% attack and all that; no arguments there). However, some people are using this terminology to refer to the IOU nonsense in the memory pool. Why?!

Zero-conf transactions/IOUs -- why are developers writing new features to support this bad practice? Conflict of interest? Profitable side project? If this keeps going then Bitcoin eventually have to collapse. Somebody has to pay for upkeep of the memory pool as well as data transfer costs. I'm no-longer convinced that it's sustainable, unless I'm missing some key factor?

I'm really still a n00b trying to understand all the bit'n'pieces of Bitcoin and there implications, Maybe the mempool is due to the fact that because of the block size only 7tps can be processed which for a system growing like this is not enough, hence why the hard 1mb block size was removed in v0.8 if I am correct. I hope that the Dev team can thoroughly test 0.8.1v for bugs and backwards comparability and get this rolled out before bitcoin really crawls and hopefully it'll put and end to the above 0 confirm madness you mention.

Also I just read Satoshis white paper is it really only 9 pages long?? it just described hashes and merkle trees, I expected alot more for such a profound piece of software.