What will keep transaction fees up?

[da2ce7]

I agree with db on this. The issue can be put simply like this: the generator which accepts the lowest fees, makes the most profit. given any two generators with the same capacity.

I don't quite understand da2ce7's post, but he seems to be implying that it's profitable "in the long run" to accept higher fees only. This seems flawed, because in the short run, such a generator will be the first to go out of business.

Ok long post, I think that I’ll re-write it when I’m less tired.  Hopefully people can see what I’ve been saying, that the network as it is now will self-balance and maintain the transaction fees.

Edit: I have some parts of bitcoin's structure wrong , instead of 'keeping the block secret', the network will just 'wait until there is a block they like.'  Once a block is generated no more changes can be made to it inc. adding new transactions.


It is true that any generator that accepts the lowest fees will make the most profit/work done, in the short run.   This is half of the story tho.  I have done a very poor job of explain the other half: why any given generator will decide not to undercut the others (much).  It takes a little bit of market and game theory, but hey that stuff is fun isn’t it?!

Firstly, when the value gained from transactions is much larger than new coins, the divergence between the networks average hashing rate and maximum will be much larger.  The network will evolve into a big game of ‘transaction fee hash chicken’ where a few (or a single) generators will have found a valid hash before the time they announce it.  They will keep it secret until a certain amount of transaction fees has accumulated, at that point they will announce it.  Generally if a block is announced before any other it will be the winning bock.

Over the course of the 2016 blocks there will average close to 10min per winning block, stabilizing the difficulty.  When the transaction fees go up, the incentive to ‘jump in earlier’ will increase, thus increasing the difficulty.  Even on this case alone, even if every generator accepted ever transaction, this is enough to push the transaction fees up. (But that is a side point, and in reality will be a very small proportion of the reason that transaction fees will be maintained)

The earlier that a generator ‘jump in’ on average more of the other generators will have not found their block, whenever a generator finds a block before jumping in, it automatically moves to the next block, increasing the chance of winning two in a row.

After a generator ‘jumps in’ the others generators that have generated the block will announce their blocks straight after.

The rest of the generators will need to decide who they support for the next block.  They will generally go with the block that has the highest threshold for transaction fees (for new transactions, low fee older transactions will be always included).

All the generators, whom jump in together, will need to decide to give up and generate for the block that has the highest threshold for transaction fees, or gamble and hope their head start will beat the rest of the network to the next transaction.

If none of the blocks are attractive to move to (accept too low fee transactions), the rest of the generation will wait for a block that accepts only higher fees.

If the next block is generated and announced before anyone else, and it is reasonable, it is automatically accepted and everyone moves to the next block.  If many jump in, then the one that accepts the least low fees wins again. And so on.

One can see that the other generators won’t pick the generators that undercut them.  This is self-balancing, as if the difficulty goes too high, the generators will all accept as many transactions as they can, lowering the transaction fee, lowing the incentive to generate, lowering the difficulty.  Thus the incentive to wait and ‘jump in later’ will increase… and so on.

[theymos]

The network will evolve into a big game of ‘transaction fee hash chicken’ where a few (or a single) generators will have found a valid hash before the time they announce it.  They will keep it secret until a certain amount of transaction fees has accumulated, at that point they will announce it.

Adding transactions invalidates your block hash.

[MoonShadow]

Close to 100%. You will get few generators who selflessly sacrifice their own profits to keep prices up for their competitors.

I’ll try and explain it more formally. (The entire system is quite complex, but I’ll give it a go).
  

That's seems to be a long proof to show that future generators purely motivated by profit would modify their clients to jump in & out of the game based on the presence of unserviced transaction fees in the transaction queue, while ignoring transactions that don't meet their own minimum fee, once the block reward drops low enough that it is not enough on it's own.  This is pretty much by design, as that is when we will hit a balanced difficulty level.  Yet, even if there are profit seeking generators that would benefit by stopping and starting, it is unreasonable to assume that there will not be significant generation motivated by rational self interest other than profit.  For example, some future generators might be motivated to maintain a minimum level of security for defense of their own balances; or a future version of a bitcoin credit union might be motivated to process transactions for free.  Even if free transaction generators were selective in who they accept transactions from, they are likely to have agreements with other such institutions to process one anothers' mermbers' transactions.

And if I can think of ways around it now, I'm sure that there will be other forces influencing the pricing when the market actually comes to that point.  This is way to complex to predict.

[MoonShadow]

The network will evolve into a big game of ‘transaction fee hash chicken’ where a few (or a single) generators will have found a valid hash before the time they announce it.  They will keep it secret until a certain amount of transaction fees has accumulated, at that point they will announce it.

Adding transactions invalidates your block hash.

Are you sure?  I may be wrong, but my understanding was that the hashing of a block is the previous block's headers & transactions, not the transactions that you are putting into your block.  This way the transaction queue can pile up while you are trying to hash the previous set without interuptions.

Hmm, there might be problems with that as well, so I'm probably wrong about that, as that might permit a malicious node to strip the block owner's own address from the reward transaction and replace it with his own, in the hopes of spreading around his altered block faster than your's.

[theymos]

Are you sure?

Yes. The data covered by the hash includes the Merkle root, which is (basically) a hash of all of the transactions in your block. Adding a transaction changes the Merkle root, which changes the header, which makes you hash invalid.

People might very well hold their generation power until after some high-fee transactions have accumulated, but they'll have to start from scratch.

[da2ce7]

I was under a misunderstanding on how bitcoin worked , It still doesn't stop the network for ignoring blocks that include to-many new low fee transaction as a whole, just makes the process a little less elegant.

[FreeMoney]

I was under a misunderstanding on how bitcoin worked , It still doesn't stop the network for ignoring blocks that include to-many new low fee transaction as a whole, just makes the process a little less elegant.

I think your idea of wait will surely happen, but it won't be waiting to release, it will just be waiting to start working. It is actually more elegant the way it is as we get the most difficulty for a given amount of power that way.

[MrFlibble]

I'm interested in what will ensure clients & generator stay up-to-date (before discovering that transactions are not being confirmed or blocks are being ignored).  Software in general has a really terrible track record of staying recent, perhaps because people are involved, people forget to maintain things, and don't trust automatic updates.

[max block size change is] a backward-incompatible change. Everyone needs to change at once or we'll have network fragmentation.

Probably [...] set the larger size limit to take effect at some block number. If an overwhelming number of people accept this change, the generators will also have to change if they want their coins to remain valuable.

Do the block-generator or transaction-creator record client version in the block?  I see "ver": 1 fields in the raw json of BBE, but what is connected to this?  Is the range of values limited?  (Sorry, I will read the code One Day^TM.)

Making incompatible changes (size params, new hash alg, fee schedule...) at a specified blknum in the near future makes sense because everyone synchronises on the central timestamp.

Daemons & GUIs are sitting watching a stream of version number go past on blocks and transactions.  If they see something new, should they not enquire about likely consequences?  This could be centralised (IRC or project home page) or P2P (like the transactions?).

• e.g. I'm running ver:1 and I see ver:2.  I ask the neighbours and they don't know anything about it.  I consult the project homepage and it says "incompatible change of class <c> coming at <blknum>" and signed by (hmm, feature request for reputation management) so I inform the neighbours and start nagging the hoomun to do an upgrade before <date>, or face <consequences>.
• e.g. I'm running ver:50 and I see ver:57 go past.  I know (wired into client) that I can deal with anything between ver:46 and ver:60 so I accept the block and go looking for more info.
• e.g. I see ver:61 arrive.  That's out of bounds.  I could quit; I could refuse the block (and potentially fork the chain).  My options are limited, so it's not clear that the "acceptable version range" idea is very useful...
• e.g. I see ver:58 arrive.  I ask for the changelog, see it's signed by someone I trust and it says I have a security hole.  If I'm sitting on cash, or running on the user's workstation inside the firewall, I might immediately disconnect myself and run offline.  If I have no cash and was configured to know I'm in a DMZ, I carry on and let the sysad worry about security.

They're just ideas...  but without accurate client version stats (linked to the svn revision number?) it's hard to know the age of the transacting & generating populations until something goes bang.

[theymos]

Do the block-generator or transaction-creator record client version in the block?  I see "ver": 1 fields in the raw json of BBE, but what is connected to this?  Is the range of values limited?  (Sorry, I will read the code One Day^TM.)

Blocks and transactions have version numbers in them. These are distinct from the client version number, and have always (AFAIK) been 1. Client versions are not included in the chain, but they are communicated to network peers. Version is a signed integer, so the limit is about two million.

There is already a network alert facility. If Satoshi broadcasts a signed message to the network, text will display in all Bitcoin clients of the specified version. Alerts can also disable the JSON-RPC interface, shutting down all sites that rely on Bitcoin until the network becomes safe. A more decentralized scheme would be a good idea in the future, but it's not needed now.

[asdf]

In the future we will have generators that will _not generate_ until there is a certain amount of high fee blocks waiting, and then they will automaticity turn on their massive generation capacity and quickly generate the block.  This leaves the average difficulty relatively low, and makes a strong incentive to include larger fees.  These generators plainly won’t include any low fee blocks, low fee blocks will have to wait around for a charity block.

How will the fees build up if the smaller generators are constantly processing them. there isn't going to be a sudden influx of high fee transactions... The large generators will simply be leaving all the profit to competitors. And if the competitors are snapping up any transaction no matter how small the fee, there is little incentive to pay a high fee in the first place.

There is a lot of guessing going in this thread, for we seem to have all fallen victim to the idea that we can guess the nature of a market that doesn't yet exist.  As it is now, there is no incentive to pay a transaction fee at all, as the fee structure that currently exists is intended to limit spamming without prohibiting legitimately non-conforming transactions; not offer a profit motive to generators decades away.  The fee structure can be altered without much difficulty on a technical level, but the future political issues are as difficult to predict as the market is.  Yet many seem to get blinded by the profit motive as the only motive for generation, which can easily be shown to not be the case presently, and I have seen no compelling reason as to why it must become the only reason in the future. 

I just feel that any motive other than profit, will not be good enough. I agree that profit is not the ONLY motive for generation as you have pointed out, but other incentives are just too weak and will result in too few generators. The network will be too weak and will surely fail.

How high do you think the difficulty would be now, if there was not the 50BTC incentive to generate? I think it would be < 1000.

No one has commented on my idea of placing a rule on fee distribution on blocks to prevent the acceptance of arbitrarily low fees.

For example, for a block to be valid, no more than half of the transactions in the block can have fees below the average fee for that block. So, if 10 transactions have a fee of 1BTC and 10 more have a fee of 0.5BTC the average is 0.75. Now half the fees are already below this so an additional transaction with a 0.5BTC fee cannot be accepted, but a 0.8BTC fee is okay.

This forces clients to compete with each other for the current block, eliminates the need for a fixed block size (ie. fixes the spam problem) and scales to accommodate any number of transactions. If a spammer sends lots of small-to-zero fee transactions, only some of them, if any, will be accepted, because you can only include max 50% fees which are below the average. To include more smaller fees a generator will have to include more larger fees.

Think about it, clients will compete and a stable market rate for fees will be established. Tiny fees, too far below the market rate will not be accepted soon and fees above the market rate will be guaranteed to be accepted in the current block. It's a simple protocol rule for generators which should be computationally cheap.

Do you guys get this? Perhaps I haven't thought it through enough, it's just a proof of concept; the exact details can be refined. Does anyone have a reason that this can't work? Maybe it's just a really crap idea, but I would appreciate SOME feedback, please.

[da2ce7]

For example, for a block to be valid, no more than half of the transactions in the block can have fees below the average fee for that block. So, if 10 transactions have a fee of 1BTC and 10 more have a fee of 0.5BTC the average is 0.75. Now half the fees are already below this so an additional transaction with a 0.5BTC fee cannot be accepted, but a 0.8BTC fee is okay.

'No more than half less than the average' will not maintain the transaction fees, without other factors pushing up the fees.  The fee will progressively get lower as the low side will get attacked harder than the high side.  In the above example a transaction fee of 0.751 will increase the average very slightly, then the next transaction fee of 0 will lower the average (relatively) much more.

The current breed of stock generators, generate continuously, and only ignore a block if it is deemed illegal.

When transaction fees become the main incentive to generate a new breed of generators will be developed that will maximize the profit made.  One of the features of this bread of generator is that certain 'rules' will be voluntarily agreed upon, some will not accept blocks that have two many low fee transactions, and will wait for blocks that have only higher fee transactions.

The risk of a generator who accepts too many (new) low fee transactions is that the most of the network will ignore and orphan it's block.

[RHorning]

For example, for a block to be valid, no more than half of the transactions in the block can have fees below the average fee for that block. So, if 10 transactions have a fee of 1BTC and 10 more have a fee of 0.5BTC the average is 0.75. Now half the fees are already below this so an additional transaction with a 0.5BTC fee cannot be accepted, but a 0.8BTC fee is okay.

This forces clients to compete with each other for the current block, eliminates the need for a fixed block size (ie. fixes the spam problem) and scales to accommodate any number of transactions. If a spammer sends lots of small-to-zero fee transactions, only some of them, if any, will be accepted, because you can only include max 50% fees which are below the average. To include more smaller fees a generator will have to include more larger fees.

Think about it, clients will compete and a stable market rate for fees will be established. Tiny fees, too far below the market rate will not be accepted soon and fees above the market rate will be guaranteed to be accepted in the current block. It's a simple protocol rule for generators which should be computationally cheap.

Do you guys get this? Perhaps I haven't thought it through enough, it's just a proof of concept; the exact details can be refined. Does anyone have a reason that this can't work? Maybe it's just a really crap idea, but I would appreciate SOME feedback, please.

I'm very hesitant to support a protocol change that puts any sort of restriction at all on what a miner must put into a block or leave out.  That can and should be something which is decided by the miner itself.  Also, it is possible that the miner node itself can put in all sorts of bogus transactions to fiddle with this sort of arrangement anyway, and that just adds overhead as well as wasted data storage when miners have to deliberately make stuff up to get some fees.  A miner could include a whole bunch of transactions that would "lower" the average TX fee allowing the miner to include some additional "real" fees, if any.

Still, it is an interesting way to set up an algorithm that also puts some emphasis on transaction fees.  I'm sure people who have dedicated miners would love this.  You could also say that blocks of a certain size (current max size or perhaps a bit less) could include more "free" or "cheap" transactions or simply follow the current rules.  There is much to be said about this concept.  I wonder if there is a way to "fix" these problems?

[MoonShadow]

I just feel that any motive other than profit, will not be good enough. I agree that profit is not the ONLY motive for generation as you have pointed out, but other incentives are just too weak and will result in too few generators. The network will be too weak and will surely fail.

Not surely, but maybe.  I don't think so because of the preservation motive is nearly as strong as the profit motive, and because there will still be some who can generate at a negligible additional cost, so there is no point at which generation is unprofitable.  The confluence of these motivations will ultimately create a self regulating market with a fairly stable difficulty level.  The real question then becomes, is that difficulty level secure enough to defend against a brute force attack?  That is a question that may never be answerable.

How high do you think the difficulty would be now, if there was not the 50BTC incentive to generate? I think it would be < 1000.

That's the wrong question, because none of us pay our electric bills in bitcoin just yet.  The question should be, how high would the difficulty level be if the reward were worth less than $13?  The answer would be less, but the total value of breaking the blockchain would be less as well.  It's a self-balancing system, almost organic in how it acts; and any risk calculations that could ever be devised would be incomplete due to the human factor.

No one has commented on my idea of placing a rule on fee distribution on blocks to prevent the acceptance of arbitrarily low fees.

This forces clients to compete with each other for the current block, eliminates the need for a fixed block size (ie. fixes the spam problem) and scales to accommodate any number of transactions. If a spammer sends lots of small-to-zero fee transactions, only some of them, if any, will be accepted, because you can only include max 50% fees which are below the average. To include more smaller fees a generator will have to include more larger fees.

Think about it, clients will compete and a stable market rate for fees will be established. Tiny fees, too far below the market rate will not be accepted soon and fees above the market rate will be guaranteed to be accepted in the current block. It's a simple protocol rule for generators which should be computationally cheap.

I was still considering it, myself.  I think that the priority system does much of what you are trying to do except allow the blocksize to stretch; and I have concerns about the bounds of this rule.  Without the priority system, this rule would permit blocksize spamming anytime there were only free transactions in the queue, so there would have to be a minimum allowed average fee, even if that were just .00000001 bitcoins, but then that would prohibit any transactions from being processed at all in the absence of a fee paying transaction even if there was an altruistic generator who won the block.  So there would have to additional rules to permit altruistic generators to help clear transaction backlogs, because there is bound to be more free transactions to process than fee paying ones for the foreseeable future.

I offered a plan before you posted yours, and no one commented upon it either.

[asdf]

For example, for a block to be valid, no more than half of the transactions in the block can have fees below the average fee for that block. So, if 10 transactions have a fee of 1BTC and 10 more have a fee of 0.5BTC the average is 0.75. Now half the fees are already below this so an additional transaction with a 0.5BTC fee cannot be accepted, but a 0.8BTC fee is okay.

'No more than half less than the average' will not maintain the transaction fees, without other factors pushing up the fees.  The fee will progressively get lower as the low side will get attacked harder than the high side.  In the above example a transaction fee of 0.751 will increase the average very slightly, then the next transaction fee of 0 will lower the average (relatively) much more.

Well that's just fine. if there are no other fee paying transactions, the generator can accept the 0.751 and the 0 and still satisfy the rule, but he can't accept another 0 fee tx. Though, I now believe it is flawed for other reasons.

The current breed of stock generators, generate continuously, and only ignore a block if it is deemed illegal.

When transaction fees become the main incentive to generate a new breed of generators will be developed that will maximize the profit made.  One of the features of this bread of generator is that certain 'rules' will be voluntarily agreed upon, some will not accept blocks that have two many low fee transactions, and will wait for blocks that have only higher fee transactions.

The risk of a generator who accepts too many (new) low fee transactions is that the most of the network will ignore and orphan it's block.

This is an interesting, market based solution. Generators can stop competitors from undercutting them, by "voting" out blocks that don't adhere to their own customized rules. I'm not convinced that it is sustainable. It looks like a problem for game theory.

If generators get too greedy, accepting all transactions, They run the risk that most of their peers will reject their hard work generating a block. If they reject blocks that their peers have accepted, they'll waste time working on a smaller block chain.

If this sort of thing works, then miners should be doing it now! This will stop the spam problem: If miners start rejecting blocks that are too generous to spamers, we shouldn't need a block size limit. I suppose, though, that the block size limit IS an implementation of a voluntary rule. Though a rather primitive one, it's sufficient for now.

Perhaps miners will converge to using an optimal rule, similar to what I've described and I'm trying to preempt this development. This definitely looks like the problem of finding an "optimal strategy" in game theory.

[asdf]

I just feel that any motive other than profit, will not be good enough. I agree that profit is not the ONLY motive for generation as you have pointed out, but other incentives are just too weak and will result in too few generators. The network will be too weak and will surely fail.

Not surely, but maybe.  I don't think so because of the preservation motive is nearly as strong as the profit motive, and because there will still be some who can generate at a negligible additional cost, so there is no point at which generation is unprofitable.  The confluence of these motivations will ultimately create a self regulating market with a fairly stable difficulty level.  The real question then becomes, is that difficulty level secure enough to defend against a brute force attack?  That is a question that may never be answerable.

I don't disagree that there is a preservation motive to rival the profit motive, but I think that the profit motive is far more prevalent. Only some want to preserve the integrity of the chain, but everyone wants to make profit. As it is, contributing as an honest node is more profitable than attacking the network, but remove the incentive of fees and profit motivated generators become profit motivated attackers. Will the network be strong enough...

No one has commented on my idea of placing a rule on fee distribution on blocks to prevent the acceptance of arbitrarily low fees.

This forces clients to compete with each other for the current block, eliminates the need for a fixed block size (ie. fixes the spam problem) and scales to accommodate any number of transactions. If a spammer sends lots of small-to-zero fee transactions, only some of them, if any, will be accepted, because you can only include max 50% fees which are below the average. To include more smaller fees a generator will have to include more larger fees.

Think about it, clients will compete and a stable market rate for fees will be established. Tiny fees, too far below the market rate will not be accepted soon and fees above the market rate will be guaranteed to be accepted in the current block. It's a simple protocol rule for generators which should be computationally cheap.

I was still considering it, myself.  I think that the priority system does much of what you are trying to do except allow the blocksize to stretch; and I have concerns about the bounds of this rule.  Without the priority system, this rule would permit blocksize spamming anytime there were only free transactions in the queue, so there would have to be a minimum allowed average fee, even if that were just .00000001 bitcoins, but then that would prohibit any transactions from being processed at all in the absence of a fee paying transaction even if there was an altruistic generator who won the block.  So there would have to additional rules to permit altruistic generators to help clear transaction backlogs, because there is bound to be more free transactions to process than fee paying ones for the foreseeable future.

I offered a plan before you posted yours, and no one commented upon it either.

It would only take one non zero transaction to cut out the freeloading spammers. A generator could inject one himself, which would force clients to match it or come close. Anyway, da2ce7 has me half convinced that miners should be imposing their own rules on the blocks, forming a democracy. It's not like anyone can stop this anyway: A miner is always free to reject any block, for better or worse, for whatever reason it sees fit. So, I guess the conversation about block rules is moot and the question of how to keep fees up is in the hands of the mathematical (game theory) viability of the protocol. Has Satoshi already figured all of this out, I wonder?

[MoonShadow]

It would only take one non zero transaction to cut out the freeloading
spammers.

That was not my complaint.  One non zero transaction would also only allow for one zero transaction.

ASo, I guess the conversation about block rules is moot and the question of how to keep fees up is in the hands of the mathematical (game theory) viability of the protocol. Has Satoshi already figured all of this out, I wonder?

I would say so.

[MrFlibble]

The current [...] generators [...] only ignore a block if it is deemed illegal.

[In the next breed of generators] certain 'rules' will be voluntarily agreed upon, some will not accept blocks that have two many low fee transactions, and will wait for blocks that have only higher fee transactions.

The risk of a generator who accepts too many (new) low fee transactions is that the most of the network will ignore and orphan it's block.

This is an interesting, market based solution. Generators can stop competitors from undercutting them, by "voting" out blocks that don't adhere to their own customized rules. I'm not convinced that it is sustainable. It looks like a problem for game theory.

Yes!  This sounds cool.  When generators disagree there will be inefficiency, confusion and confirmation-delays.  It would be useful to mediate agreement among competitors.

If this sort of thing works, then miners should be doing it now! This will stop the spam problem: If miners start rejecting blocks that are too generous to spamers, we shouldn't need a block size limit. I suppose, though, that the block size limit IS an implementation of a voluntary rule. Though a rather primitive one, it's sufficient for now.

Perhaps miners will converge to using an optimal rule, similar to what I've described and I'm trying to preempt this development. This definitely looks like the problem of finding an "optimal strategy" in game theory.

Miners could do it now, but the software-capital cost and uncertainty look high enough to postpone it for a while.

Meanwhile, it should be possible to simulate the network.  No need to calculate hashes, just model a population of generators G[0 .. g] pulling hashrates HPS[0 .. g] and having their own rules & motivations, and some clients.

This needs three sets of parameters,

• one to describe generators (effective costs for electricity, existing hardware & incremental hardware purchase) and allowing for resistive heating, gamers' hardware habit as well as hard-nosed corporations etc.
• another to describe 'voluntary rules' used by generators for declaring a block of transactions valid.  We have ideas here, they just need a weighting parameter each s they can be combined.
• finally (the hardest?) for a model which generates transactions among a population of merchants, speculators, hoarders, spammers etc..  Assuming they all know what the 'rules' are at any time, but can only influence them by generating, spending or getting patches accepted and installed.

Might make an interesting student project (one semester?).  I don't have time, sorry.

I suspect the generator rules can vary with time, but to fend off chaos they should be a deterministic function of the content of previous block(s) and not depend on the transaction pool.  I also suspect that if the customers & merchants do not hold a significant fraction of the Ghash/sec, the 'rules' will tend to make a generator cartel with high fees.

[da2ce7]

I suspect the generator rules can vary with time, but to fend off chaos they should be a deterministic function of the content of previous block(s) and not depend on the transaction pool.  I also suspect that if the customers & merchants do not hold a significant fraction of the Ghash/sec, the 'rules' will tend to make a generator cartel with high fees.

Yes I believe the real potential issue is the 'cartel,'  I'm not worried about the transaction fees being too low, rather too high.  I think that studying this with game theory would be very fun, but, I don't have the time also.

[tyler]

Most transaction are free, but you can make it go faster by paying transaction fees.

Will there be a default fee amount included in future clients, or will be have to guess/check what the average miner is charging to process transactions?

[theymos]

Will there be a default fee amount included in future clients, or will be have to guess/check what the average miner is charging to process transactions?

Bitcoin will look at network conditions and make a guess for you.