Bitcoin Forum

When coin creation diminishes transaction fees are supposed to encourage people to keep generating blocks. But how does the transaction pricing work?

The cost of hashing does not increase from including another transaction in a block. A generator will always benefit from including a transaction no matter how small the fee. So the fees will approach zero (as, indeed, they are now) making block creation very unrewarding which will reduce the computing power providing Bitcoin its security to almost nothing.

The wiki only talks about (http://www.bitcoin.org/wiki/doku.php?id=transaction_fee) how limited block sizes keeps transactions scarce and prices up. Is that it? I see no market mechanism connecting the demand for transactions to the block size limits. How will everyone agree on maximum block sizes?

Am I missing something?

I'm not sure, but wouldn't it be the same way we agree on the reward for a generate? By refusing to accept blocks that are too big?

Sure, but the reward sizes are set in stone in the protocol and follow a predetermined path. The maximum block size will have to adjust to balance security and transaction demand.

I think the max block size is set so that the chain file will never be larger than [max block size]*[time since first block]/[10 minutes]

If a generator could just increase it they could force a huge block on everyone. To avoid that, we all reject oversize blocks. I believe it would be a breaking change, but there are ways to get it done anyway. If file storage becomes absurdly cheap and many blocks are full we will organize the change by issuing all new versions for a while to trigger the rule change at some future block.

This is my understanding, anyone please correct.

There is a small cost to adding a new transaction. You have to store the transaction until it is spent, validate the transaction, and recompute part of the Merkle tree for the block. There will also be big network, disk, and electricity costs for generating blocks.

Block size limitations indicate that the network is unanimously willing and able to download, upload, and store blocks of that size every 10 minutes. It will probably stay at the current 1MB until Bitcoin has a solid backbone of generators. Then it can increase rapidly.

Some generators will set fees to the minimum, and some will set fees to the highest profitable level in the hope of raising it further. Most will be set somewhere in the middle. The generators will be saying, "We want to be paid on average this much," and the users will be saying, "We will pay only this much on average." This creates a really nice effect: your transaction will almost always clear eventually, but more fee = more speed.

The "minimum fee" for a generator will usually be the fee that causes the most fee-transactions to be included in the block. Volunteers or people who feel that the generation reward is enough might have a minimum fee of 0. This will happen less and less in the future, though, because costs will rise, inherent rewards will be reduced, and you'll therefore want to create blocks with at least some non-free transactions.

If the network overcharges, competitors will come in that claim the leftover transactions. If the network undercharges (a high-fee transaction exists, but it is not included in a block in favor of a cheaper one), competitors will come in with proper fee-based prioritization.

When the coins generated per block is low, users may not be willing to pay enough for generators to ever be profitable. Transactions will be really slow in this case, and the users will hopefully change their minds. If they don't, the least efficient generators will leave, the difficulty will be reduced, and generation will be profitable again. The users will be saying, "I'm not paying for that much CPU power! Reduce it."

Very nice explanation, Theymos. Can you comment on "max block size" in the future? Is it likely to stay the same for all time? If not how will it be increased?

It's a backward-incompatible change. Everyone needs to change at once or we'll have network fragmentation.

Probably the increase will work like this: after it is determined with great certainty that the network actually can handle bigger blocks, Satoshi will set the larger size limit to take effect at some block number. If an overwhelming number of people accept this change, the generators will also have to change if they want their coins to remain valuable.

It might also work in reverse, where almost all generators decide to reduce (or raise) the max block size. If clients want to have any real protection from double-spending, they also need to switch.

The limit needs to increase at some point if Bitcoin is to become mainstream. 1MB is not an awful lot of transactions, so not increasing it would raise fees to unreasonable levels.

satoshi has already commented on that subject. (http://bitcointalk.org/index.php?topic=1347.msg15366#msg15366)

Most transaction are free, but you can make it go faster by paying transaction fees.

Yes, the transaction fees and block reward are the 'seen' incentives for generation, but you are overlooking the 'unseen' incentives.  If you were one of the very early generators of the system, and in another 2 years when the block reward steps down to 25 bitcoin each, how would you respond to the concern that difficulty (and therefore blockchain resistance to a brute force attack) would drop?  If you had 10K coins (not unrealistic for some) and the market value were, say $5 apiece, would you sell all that you had knowing that the market value may crash and you might not get half of the net worth out of them, or would you buy and/or run a generator of your own even at a loss?  If you have 50 bitcoins at $5 apiece, you probably wouldn't run a generator (unless you were one of these guys that has to heat their high latitude apartment with electric anyway, and therefore the cost of running a CPU/GPU at full tilt is negligible) but if you had a $50K net worth in Bitcoins you would have a strong personal incentive to protect that net worth.  In a future with a successful Bitcoin, most generation would likely be performed at a net loss by persons or institutions with the most interest in Bitcoins security.  After all, you might not think twice about $50 in your wallet on the nightstand with that one-night-stand, but what would you do if you had $10K in cash?  Would you deposit that into a savings account at a bank with a $50K safe?  If you do, you are contributing to the cost of that safe, even though your share may be so small as to ignore; but the bank still invests into it's own security at it's own loss.

Something I realized that is slightly related is that in the future the time between blocks will become more regular. Currently the reward is the same no matter how long it has been since the last block. When fees make up most of the reward they will increase linearly and as the pot builds more and more generators will come online. Of course some will not bother turning off after a block is found, but those with high electricity costs or with something else to compute will switch on and off.

No! Thats a tragedy of the commons (http://en.wikipedia.org/wiki/Tragedy_of_the_commons) situation. It just doesn't work. If Bitcoin will have to rely on that it's doomed.

Hmm, I can't think my way out of this problem.

It's not a TOC situation because of the fees, as blocks go unsolved, fees accumulate, increasing incentive to solve. People need not generate at a loss for bitcoin to work. Though maybe with some definition of 'loss' some people will anyway.

Blocks will always be solved. Sometimes way too easily.

There are two cases. One is when all transactions fit in the blocks. Then there is a tragedy of the commons. Transaction fees drop towards zero and there is hardly any incentive to generate. This is the case now. Transaction fees are zero and the only reason people generate is coin creation.

The other case is when all transactions don't fit in the blocks. Then transactions will compete to get into the blocks using transaction fees. The maximum block size must be continuously adjusted to keep the transaction prices stable. The only way to change the maximum block size is through a lengthy political process of debate, decree, network fragmentation and majority agreement.

It's not a tragedy of the commons because generators will have interest in seeing the network continue to function.

Correct me if I'm wrong here, but it is right now a very rare situation where blocks are even close to the maximum block size?  In fact, on average most blocks don't even have transactions included in them at all.  In other words, all they are doing at the moment is purely generating coins.

If there is a situation where there is a huge pile of transactions backing up, perhaps the ones with fees could get move to the front of the pack, so to say, but the rest would generally be cleared in subsequent blocks.

There certainly isn't anything stopping you from modifying your source code on Bitcoins and only accepting transactions with fees for at least blocks you are able to mine and create.  Essentially, if you are going through the effort to generate a block, you ought to be paid for any transactions in that block, at least if you think that way.  If enough people don't want to process transactions without getting paid, those without a fee simply won't get processed.  It really is that simple.

For myself, I'd love to see this as a option in the User Interface itself where you could set the rules up in some way for how you would personally want to process transactions, and make it more of a market-based approach.  There is room for multiple rules regarding how transactions are processed.

Just like the herders have an interest in having the common pasture continue to grow grass?

No, there will be instititutions with a high value to protect.  Costs of protection are a 'tax' of sorts, but they are unavoidable.  They are not a tragedy of the commons scenario, each is still looking out for his own interests, and his interests benefit others.  It's a postitive externality.

Such a priority ranking system is in the current release client.

I think db has a point. If the block reward isn't profitable, that does look like a tragedy of the commons. It's true that if it happens the least efficient miners will give up, what will decrease the difficulty for the most efficient... but a lower difficulty is bad for the network...

This is a bad idea. The generators will collude to keep the block size small; transactions scarce, gouging the market.

Which is why major institutions will still be willing to contribute clock-cycles at or just below a break even point.  Because there are more forms of economic motivation than just profit.  I'm really suprised that so many who seem so well educated on economic issues can't wrap their head around this simple concept.  If you have something valuable to protect, have you ever paid the rental fee on a safety deposit box?  The cost of the box rental is tiny compared to the value of the object within, but that's not a tragedy of the commons!  People do it all the time!  It's a cost of security, not a resource access issue!  The tragedy of the commons parable is a limited resource issue!

Some may try.  Keep in mind that generators have no sustainable monopolies on generation, not even as a group.  If the major generators collude to keep block sizes small amongst themselves; say by keeping their own max block sizes at 1 meg, but the regular users' clients all have a max block size limit of 3 megs, then the rising backlog of lower fee transactions will attract new players into generation.  Maybe forcing the colluding generators to change, maybe not, but a natural price balance will be maintained.  Perhaps the occasional blockchain split fight is neccessary.

I presume this has been used in the test network, but are there any blocks where that has been necessary in the regular client?  I'm not asking somebody to deliberately force enough messages to be processed to do that, but more as a general question, is it something which has been done?

Yes, I realize that it is in the current release.

I just thought of something. The time it takes to generate a hash if proportional to the number of transactions you're hashing, right? So, it'll take twice as long (on average) to generate a block with 1000 transactions as one with 500. You're not going to waste precious hasing time on small fee transactions, they'll just decrease your hash/s for negligible gain.

Say for example, there are 999 transactions with 0.1 fee to process and one transaction with a 1BTC fee. It's more profitable to just process the one transaction and ignore the rest, because your hash/s will be 1000 times faster!

Now if you have 10 transactions with a 1BTC fee, your best off processing all 10 of them, because there's a small overhead in processing a hash. But, including an 11th transactions for 0.9BTC fee probably won't be worth it. 0.99BTC fee, maybe. It depends on the size of the overhead.

This implies that generators will only process the transactions with the highest fee! I'm still thinking through the implications of this. Ideas anyone?

No, not really. It's related to the difficulty factor only.

Are you sure? Given any difficulty, you still have to crunch the numbers to solve a block. The more transactions, the more numbers to crunch, thus the longer it takes to compute a given hash.

I wouldn't assume that it's as straightforward as that, and you are probably overthinking it anyway.  Feel free to try it, though.

Block hashes are only hashes of the fixed-size 80 byte block header, which contains a hash of the transactions. Transactions only have a small one-time CPU cost for adding.

Well it's pretty simple really, unless I'm missing something, or hashes don't actually work like I think they do. The more data you have to hash, the longer it will take to compute the hash. Makes sense right? I'm pretty sure the computation time is linear with respect to data size, so double the number of transactions and you double the time to compute the hash.

This is a huge problem now, because why would anyone hash more than one transaction to generate a block. They get 50BTC either way, so you might as well just hash one transaction giving you the optimal hash/s.

I hope I'm wrong about this, I'd really like more feedback from you guys. Perhaps we need a minimum block size or something.

Ahhh, okay then. Thanks for the reassurance.

This is not like individual safety deposit boxes. This is like one big collective vault in which it is free for anyone to place their valuables and paying is optional.

how about making the max block size a function of the difficulty? If there are too few generators, then the max block size will decrease making transactions scarce. This will drive up the txfee and create incentive for new generators to enter the market. vice-versa.

Thats true on most calculations, but most certainly not so with data structures that self reference, and a conditional is a self reference.  I'm not sure if it would be true with hashing algorithims or not, but I wouldn't assume that the algorithim is particularly linear in nature.

I'm afraid you missed the analogy that I was trying to present, and your's is more than a bit flawed as well.

I meant:
"I'm pretty sure the computation time is linear with respect to data size, for hash functions"

Anyway, as theymos pointed out, you only have to hash the transaction once, not on every attempt. So, this isn't a problem.

Absolutely not.  The max block size would increase to the point that too much room was available for spamming, and that would be to much of a temptation for some. 

Now that's an example of the Tragedy of the Commons issue!  The amount of space available to free transactions in the block.

Okay, I'm starting to like this idea. It could be a bit messy with blockchain splits, but it seems like it will work.

Let's drop the analogies then and go straight at the problem.

The maximum block size is big, having room for all or most transactions.
Therefore, transaction fees are (close to) zero.
Therefore, total block transaction fees are also (close to) zero.
Therefore, all for-profit block generation ceases.
Therefore, difficulty drops.

If for-profit block generation was the only generation going on then difficulty would be very low and double spending very easy. The only thing that can save the system is people generating at a loss.

Now we have a lot of bitcoin holders that would lose greatly if payments become unreliable and confidence in the system drops. Will they contribute to their common good? If they do it will not be out of self interest. If you contribute to the system reliability you lose your contribution and benefit very little because the benefit is shared with everyone. If you use the system without contributing you benefit from everyone elses contributions anyway.

The usual sad result is that everyone tries to live off of everyone elses contributions, very little is actually contributed and everyone loses.

Classic tragedy of the commons.

Db is right. We should not expect people to generate on a net loss. I don't doubt that there will be people willing to do so. But I think we should aim more than that, otherwise the difficulty factor would be much lower than what it could be.

And this is a problem that could be partially solved by an automatic adjustment of the block size limit (http://bitcointalk.org/index.php?topic=1865.msg23226), as I argue on the other thread.

From my understanding of the technology and current system, this entire discussion about the block size somehow being related to the transaction fees is silly in the long run.

Quite plainly, who said that the generators have to fill the entire block up?  To me it is simple that generators will simply include: x amount of low fee, and y amount of medium fee, and remainder high fee.  If the space isn’t used, it isn’t used.

It is in the generators interest for to keep the network healthy, this includes not generating at a loss.

Artificially holding the price high by having a smaller block size is both silly and rather pointless, it creates an artificial tax for bitcoin.  The only purpose of having block size limits is for flood protection; otherwise it should be as large as it needs to be.

You must remember that the generators are acting in their self-interest, therefor, they will charge for their service, of course, eventually the low fee transactions will be accepted by some charitable person, but on the whole, the large generators will be selfish and not undercharge for their service, rather they will decide on fees that are profitable.

Finally, who ever said that generators will blindly accept transactions from anyone?   I quite possibly foresee that a bank generator will have low transactions for any of its members address, and high fees for the competition.  This would give the bank a competitive advantage.

@da2c7

Why would a generator refuse a transaction with any amount of fee if s/he can still add it to the block?

A bank doesn't need to mind with transactions the way you suggest. All transactions in between its clients don't need to be real bitcoin transactions, they can be just updates in the bank database. The bank would only need to send transactions to the chain when transferring from/to an "outsider" address.

Some will, that is charitable, but most aren’t that charitable (and this is a good thing).  I think that an analogy is in order.

A man runs a bakery; he bakes a certain amount of goods early each morning, and sells them throughout the day, when he has leftovers he may reduce the price to flog the bread off, or may take it home for his chooks to eat.

The fallacy that this entire discussion has been trapped into is the assumption that the ‘selfish’ thing is to accept every transaction, no matter how small the fee is, rather that is ‘altruistic.’ The real selfish thing is to accept whoever pays into the block.

See most that run bakeries know that most people buy the bread anyway, cheap or not.  They just wait until the end of the day when it is on sale. That is why it is (more) profitable to take the bread home for you chooks.

Both bakeries and generators work on the lovely formula:  Average Price = (Total Cost + Profit) / Average Sales.

On banks, banks will allow members free or cheap transactions for personal use, if they report the transaction to the bank.

Accepting every transaction is the selfish thing. Every batch from the oven yields an unlimited amount of bread. And there is a thousand people who offer to pay very little. Will the baker decline to take the money, throw away the bread and let the competitors make the sales instead?

But don't you need to accept a price?

How do you mean?

I think there is an important difference. When one generator is faced with deciding to take tiny fees and fill up the block or leave them will compare the immediate gain of fees to his future loss via the general reduction of fees. But unless he has significant market share this general influence will affect other generators more than himself.

<here I paused to think for several minutes and almost deleted my post>

I was thinking that he should ignore the influence on other generators and simply calculate how much on average he expects fees to rise by keeping the cheapest fees out of his blocks and see if it is worth it. But then I realized that he actually benefits from reducing fees because they will remove marginal generators and decrease difficulty resulting in him generating blocks more quickly. And he will keep these particular small fees from going to a competitor as well.

I still can't say for sure, but I think the best strategy will be to accept all fees.

The bread story is not so good because almost everyone prefers the morning bread and won't wait just for a discount.

This is the problem: there is NO incentive to NOT accept every fee paying transaction, unless the fee is extremely small (ie. it costs more for the ram + cpu time than the value of the fee). Fundamentally, I think there has to be a built in cost for generators to process a fee and this cost must scale with economic activity.

Adjusting the block size automatically is fine, if you can find a function that scales appropriately. This creates an artificial scarcity for transactions, so generators prioritize higher fees.

We need more debate on this issue. This is a serious economic flaw.

This is not a flaw, economicly or otherwise.

This point just confirms the bread analogy; people who like to buy bread in the morning are akin to the people who want to be included in the block chain quickly; they pay a price for this privilege.

The bakery analogy is very correct, including low fee transactions is like making bread at cost/low profit, one can do it, however selfish thing IN THE LONG RUN is to make only as much bread that people will buy at a price that MAXIMIZES profit.

In the future we will have generators that will _not generate_ until there is a certain amount of high fee blocks waiting, and then they will automaticity turn on their massive generation capacity and quickly generate the block.  This leaves the average difficulty relatively low, and makes a strong incentive to include larger fees.  These generators plainly won’t include any low fee blocks, low fee blocks will have to wait around for a charity block.

There will be bank generators that generate regularly, however charge high fees for ANY transaction from the competition. (Just like ATM fees).

The large generators that do this will have a larger percentage of the high fees; they will also not waste electricity on nothing.

The problem is that there isn’t an unlimited supply, in this case of time.  The generator that accepts every transaction is not going to be 100% of the blocks generated, therefore there will be transactions that don’t’ get accepted in every block (i.e. low fee ones).  What people are paying for isn’t ‘to be included’ (where they are included anyway in the long run), but rather to be included more reliably in the next block(s).

This should be moved to Economics.

Close to 100%. You will get few generators who selflessly sacrifice their own profits to keep prices up for their competitors.

How will the fees build up if the smaller generators are constantly processing them. there isn't going to be a sudden influx of high fee transactions... The large generators will simply be leaving all the profit to competitors. And if the competitors are snapping up any transaction no matter how small the fee, there is little incentive to pay a high fee in the first place.

There is a lot of guessing going in this thread, for we seem to have all fallen victim to the idea that we can guess the nature of a market that doesn't yet exist.  As it is now, there is no incentive to pay a transaction fee at all, as the fee structure that currently exists is intended to limit spamming without prohibiting legitimately non-conforming transactions; not offer a profit motive to generators decades away.  The fee structure can be altered without much difficulty on a technical level, but the future political issues are as difficult to predict as the market is.  Yet many seem to get blinded by the profit motive as the only motive for generation, which can easily be shown to not be the case presently, and I have seen no compelling reason as to why it must become the only reason in the future. 

Ironically, the spamming has created an incentive to pay at least a 0.01 TX fee (the Bitcoin Faucet does this now).

This is a good point.

While I wouldn't be real excited if, for example, transaction fees didn't even exist and the plan was that people would donate cycles since they wanted it to work, I can't even be sure that would fail. As the system is, I think the incentive scheme is good, but can't tell quite what the ultimate drivers of difficulty will be and to what extent, but one way or another I think it will be huge. People buy lottery tickets that are -EV all the time, millions of them daily. Well, when 100 million people are using bitcoin a 12.5+fees win might be pretty huge. People might buy that ticket way way more often than would be rational.

I’ll try and explain it more formally. (The entire system is quite complex, but I’ll give it a go).

Limited and Unlimited domains:
Demand: Number of transactions. (Limited)
Demand Restriction: Amount that the bitcoin community is willing to spend on transactions. (Limited)
Demand Modifier:  Priority to record transaction in block chain. (Choice)
Supply: Recording transactions in block chain. (Unlimited or limited, depending on block size, variable of interest.)
Supply Restriction: Cost of generating block. (Variable, Limited)
Supply Restriction, Restriction: Number of block generated / time. (Fixed, Limited)

Now, supply and demand would dictate that an unlimited resource (space in block chain) with a limited demand (Number of transactions) will tend towards no cost.  This is the entire basis of the argument to limit the size of the block, to create a ‘market’ to be included in such block.  I’ll explain later (another post, this is already getting long) that this effectively is an artificial tax on the entire bitcoin community, and will diminish the performance of the economy.

The main fallacy is that happening is that the real cost isn’t just resource cost (dictated by supply and demand):
 
Resource Cost = Demand (transactions) / Resource Supply (space in block chain)
Supply Cost = Supply Restriction (Generation cost) * Supply Amount (Number of Transactions)
Cost = Resource Cost + Supply Cost + Profit

Even if the resource cost is 0, there still is a supply cost.  This can is because of the restricted supply domain. (A restriction on supply is the inverse of supply)

But why doesn’t the Supply Cost tend to zero also, through competition? This is a slightly less obvious, it depends on the time Doman: (number of block generated / time) fixed at around 1/10 bock per minute.

There is an unlimited supply of block in the timeless domain; this would normally push the price down to 0, making no incentive to produce blocks.  However there is a restriction in the time domain: the number blocks generated per time.

Therefore the calculations can be summarized as the first partial derivative, in respect to time.

See Attachment eq1 and eq2.
http://bitcointalk.org/index.php?action=dlattach;topic=1847.0;attach=556
http://bitcointalk.org/index.php?action=dlattach;topic=1847.0;attach=557
Where c = cost, d = demand, s = supply, t = time.

you can see for any time period (t), the demand and supply can be non zero. (eq2) as s, and d are for the prescribed t.

So simply, a selfish generator will always be better off by charging the real cost of transactions.  (In the time domain).  That means that the selfish thing in the long run is to not accept low fee transactions.

Hope this heps.  It is a bit of an over simplification, but I think it gets the basic message across.

Edit: Note, this is a simplification of supply and demand that works in percentage changed, and therefore limits and trends can be gleaned from it... It is _NOT_ useful in predicting absolute price.  

I agree with db on this. The issue can be put simply like this: the generator which accepts the lowest fees, makes the most profit. given any two generators with the same capacity.

I don't quite understand da2ce7's post, but he seems to be implying that it's profitable "in the long run" to accept higher fees only. This seems flawed, because in the short run, such a generator will be the first to go out of business.

I want to start thinking about solutions.

One avenue might be to impose some sort of protocol restriction on the fee distribution of transactions in a block. In such a way that generators can't just include every little transaction in a block or else the network will reject it. For example, for a block to be valid, no more than half of the transactions in the block can have fees below the average fee for that block. So, if 10 transactions have a fee of 1BTC and 10 more have a fee of 0.5BTC the average is 0.75. Now half the fees are already below this so an additional transaction with a 0.5BTC fee cannot be accepted, but a 0.8BTC fee is okay. That's just an example, I'm not sure if it would actually work.

Ok long post, I think that I’ll re-write it when I’m less tired.  Hopefully people can see what I’ve been saying, that the network as it is now will self-balance and maintain the transaction fees.

Edit: I have some parts of bitcoin's structure wrong :(, instead of 'keeping the block secret', the network will just 'wait until there is a block they like.'  Once a block is generated no more changes can be made to it inc. adding new transactions.


It is true that any generator that accepts the lowest fees will make the most profit/work done, in the short run.   This is half of the story tho.  I have done a very poor job of explain the other half: why any given generator will decide not to undercut the others (much).  It takes a little bit of market and game theory, but hey that stuff is fun isn’t it?!

Firstly, when the value gained from transactions is much larger than new coins, the divergence between the networks average hashing rate and maximum will be much larger.  The network will evolve into a big game of ‘transaction fee hash chicken’ where a few (or a single) generators will have found a valid hash before the time they announce it.  They will keep it secret until a certain amount of transaction fees has accumulated, at that point they will announce it.  Generally if a block is announced before any other it will be the winning bock.

Over the course of the 2016 blocks there will average close to 10min per winning block, stabilizing the difficulty.  When the transaction fees go up, the incentive to ‘jump in earlier’ will increase, thus increasing the difficulty.  Even on this case alone, even if every generator accepted ever transaction, this is enough to push the transaction fees up. (But that is a side point, and in reality will be a very small proportion of the reason that transaction fees will be maintained)

The earlier that a generator ‘jump in’ on average more of the other generators will have not found their block, whenever a generator finds a block before jumping in, it automatically moves to the next block, increasing the chance of winning two in a row.

After a generator ‘jumps in’ the others generators that have generated the block will announce their blocks straight after.

The rest of the generators will need to decide who they support for the next block.  They will generally go with the block that has the highest threshold for transaction fees (for new transactions, low fee older transactions will be always included).

All the generators, whom jump in together, will need to decide to give up and generate for the block that has the highest threshold for transaction fees, or gamble and hope their head start will beat the rest of the network to the next transaction.

If none of the blocks are attractive to move to (accept too low fee transactions), the rest of the generation will wait for a block that accepts only higher fees.

If the next block is generated and announced before anyone else, and it is reasonable, it is automatically accepted and everyone moves to the next block.  If many jump in, then the one that accepts the least low fees wins again. And so on.

One can see that the other generators won’t pick the generators that undercut them.  This is self-balancing, as if the difficulty goes too high, the generators will all accept as many transactions as they can, lowering the transaction fee, lowing the incentive to generate, lowering the difficulty.  Thus the incentive to wait and ‘jump in later’ will increase… and so on.

Adding transactions invalidates your block hash.

That's seems to be a long proof to show that future generators purely motivated by profit would modify their clients to jump in & out of the game based on the presence of unserviced transaction fees in the transaction queue, while ignoring transactions that don't meet their own minimum fee, once the block reward drops low enough that it is not enough on it's own.  This is pretty much by design, as that is when we will hit a balanced difficulty level.  Yet, even if there are profit seeking generators that would benefit by stopping and starting, it is unreasonable to assume that there will not be significant generation motivated by rational self interest other than profit.  For example, some future generators might be motivated to maintain a minimum level of security for defense of their own balances; or a future version of a bitcoin credit union might be motivated to process transactions for free.  Even if free transaction generators were selective in who they accept transactions from, they are likely to have agreements with other such institutions to process one anothers' mermbers' transactions.

And if I can think of ways around it now, I'm sure that there will be other forces influencing the pricing when the market actually comes to that point.  This is way to complex to predict.

Are you sure?  I may be wrong, but my understanding was that the hashing of a block is the previous block's headers & transactions, not the transactions that you are putting into your block.  This way the transaction queue can pile up while you are trying to hash the previous set without interuptions.

Hmm, there might be problems with that as well, so I'm probably wrong about that, as that might permit a malicious node to strip the block owner's own address from the reward transaction and replace it with his own, in the hopes of spreading around his altered block faster than your's.

Yes. The data covered by the hash includes the Merkle root, which is (basically) a hash of all of the transactions in your block. Adding a transaction changes the Merkle root, which changes the header, which makes you hash invalid.

People might very well hold their generation power until after some high-fee transactions have accumulated, but they'll have to start from scratch.

I was under a misunderstanding on how bitcoin worked >:(, It still doesn't stop the network for ignoring blocks that include to-many new low fee transaction as a whole, just makes the process a little less elegant.

I think your idea of wait will surely happen, but it won't be waiting to release, it will just be waiting to start working. It is actually more elegant the way it is as we get the most difficulty for a given amount of power that way.

I'm interested in what will ensure clients & generator stay up-to-date (before discovering that transactions are not being confirmed or blocks are being ignored).  Software in general has a really terrible track record of staying recent, perhaps because people are involved, people forget to maintain things, and don't trust automatic updates.


Do the block-generator or transaction-creator record client version in the block?  I see "ver": 1 fields in the raw json of BBE, but what is connected to this?  Is the range of values limited?  (Sorry, I will read the code One Day^TM.)

Making incompatible changes (size params, new hash alg, fee schedule...) at a specified blknum in the near future makes sense because everyone synchronises on the central timestamp.

Daemons & GUIs are sitting watching a stream of version number go past on blocks and transactions.  If they see something new, should they not enquire about likely consequences?  This could be centralised (IRC or project home page) or P2P (like the transactions?).

• e.g. I'm running ver:1 and I see ver:2.  I ask the neighbours and they don't know anything about it.  I consult the project homepage and it says "incompatible change of class <c> coming at <blknum>" and signed by (hmm, feature request for reputation management) so I inform the neighbours and start nagging the hoomun to do an upgrade before <date>, or face <consequences>.
• e.g. I'm running ver:50 and I see ver:57 go past.  I know (wired into client) that I can deal with anything between ver:46 and ver:60 so I accept the block and go looking for more info.
• e.g. I see ver:61 arrive.  That's out of bounds.  I could quit; I could refuse the block (and potentially fork the chain).  My options are limited, so it's not clear that the "acceptable version range" idea is very useful...
• e.g. I see ver:58 arrive.  I ask for the changelog, see it's signed by someone I trust and it says I have a security hole.  If I'm sitting on cash, or running on the user's workstation inside the firewall, I might immediately disconnect myself and run offline.  If I have no cash and was configured to know I'm in a DMZ, I carry on and let the sysad worry about security.

They're just ideas...  but without accurate client version stats (linked to the svn revision number?) it's hard to know the age of the transacting & generating populations until something goes bang.

Blocks and transactions have version numbers in them. These are distinct from the client version number, and have always (AFAIK) been 1. Client versions are not included in the chain, but they are communicated to network peers. Version is a signed integer, so the limit is about two million.

There is already a network alert facility. If Satoshi broadcasts a signed message to the network, text will display in all Bitcoin clients of the specified version. Alerts can also disable the JSON-RPC interface, shutting down all sites that rely on Bitcoin until the network becomes safe. A more decentralized scheme would be a good idea in the future, but it's not needed now.

I just feel that any motive other than profit, will not be good enough. I agree that profit is not the ONLY motive for generation as you have pointed out, but other incentives are just too weak and will result in too few generators. The network will be too weak and will surely fail.

How high do you think the difficulty would be now, if there was not the 50BTC incentive to generate? I think it would be < 1000.

No one has commented on my idea of placing a rule on fee distribution on blocks to prevent the acceptance of arbitrarily low fees.


This forces clients to compete with each other for the current block, eliminates the need for a fixed block size (ie. fixes the spam problem) and scales to accommodate any number of transactions. If a spammer sends lots of small-to-zero fee transactions, only some of them, if any, will be accepted, because you can only include max 50% fees which are below the average. To include more smaller fees a generator will have to include more larger fees.

Think about it, clients will compete and a stable market rate for fees will be established. Tiny fees, too far below the market rate will not be accepted soon and fees above the market rate will be guaranteed to be accepted in the current block. It's a simple protocol rule for generators which should be computationally cheap.

Do you guys get this? Perhaps I haven't thought it through enough, it's just a proof of concept; the exact details can be refined. Does anyone have a reason that this can't work? Maybe it's just a really crap idea, but I would appreciate SOME feedback, please.

'No more than half less than the average' will not maintain the transaction fees, without other factors pushing up the fees.  The fee will progressively get lower as the low side will get attacked harder than the high side.  In the above example a transaction fee of 0.751 will increase the average very slightly, then the next transaction fee of 0 will lower the average (relatively) much more.

The current breed of stock generators, generate continuously, and only ignore a block if it is deemed illegal.

When transaction fees become the main incentive to generate a new breed of generators will be developed that will maximize the profit made.  One of the features of this bread of generator is that certain 'rules' will be voluntarily agreed upon, some will not accept blocks that have two many low fee transactions, and will wait for blocks that have only higher fee transactions.

The risk of a generator who accepts too many (new) low fee transactions is that the most of the network will ignore and orphan it's block.

I'm very hesitant to support a protocol change that puts any sort of restriction at all on what a miner must put into a block or leave out.  That can and should be something which is decided by the miner itself.  Also, it is possible that the miner node itself can put in all sorts of bogus transactions to fiddle with this sort of arrangement anyway, and that just adds overhead as well as wasted data storage when miners have to deliberately make stuff up to get some fees.  A miner could include a whole bunch of transactions that would "lower" the average TX fee allowing the miner to include some additional "real" fees, if any.

Still, it is an interesting way to set up an algorithm that also puts some emphasis on transaction fees.  I'm sure people who have dedicated miners would love this.  You could also say that blocks of a certain size (current max size or perhaps a bit less) could include more "free" or "cheap" transactions or simply follow the current rules.  There is much to be said about this concept.  I wonder if there is a way to "fix" these problems?

Not surely, but maybe.  I don't think so because of the preservation motive is nearly as strong as the profit motive, and because there will still be some who can generate at a negligible additional cost, so there is no point at which generation is unprofitable.  The confluence of these motivations will ultimately create a self regulating market with a fairly stable difficulty level.  The real question then becomes, is that difficulty level secure enough to defend against a brute force attack?  That is a question that may never be answerable.


That's the wrong question, because none of us pay our electric bills in bitcoin just yet.  The question should be, how high would the difficulty level be if the reward were worth less than $13?  The answer would be less, but the total value of breaking the blockchain would be less as well.  It's a self-balancing system, almost organic in how it acts; and any risk calculations that could ever be devised would be incomplete due to the human factor.


I was still considering it, myself.  I think that the priority system does much of what you are trying to do except allow the blocksize to stretch; and I have concerns about the bounds of this rule.  Without the priority system, this rule would permit blocksize spamming anytime there were only free transactions in the queue, so there would have to be a minimum allowed average fee, even if that were just .00000001 bitcoins, but then that would prohibit any transactions from being processed at all in the absence of a fee paying transaction even if there was an altruistic generator who won the block.  So there would have to additional rules to permit altruistic generators to help clear transaction backlogs, because there is bound to be more free transactions to process than fee paying ones for the foreseeable future.

I offered a plan before you posted yours, and no one commented upon it either.

Well that's just fine. if there are no other fee paying transactions, the generator can accept the 0.751 and the 0 and still satisfy the rule, but he can't accept another 0 fee tx. Though, I now believe it is flawed for other reasons.


This is an interesting, market based solution. Generators can stop competitors from undercutting them, by "voting" out blocks that don't adhere to their own customized rules. I'm not convinced that it is sustainable. It looks like a problem for game theory.

If generators get too greedy, accepting all transactions, They run the risk that most of their peers will reject their hard work generating a block. If they reject blocks that their peers have accepted, they'll waste time working on a smaller block chain.

If this sort of thing works, then miners should be doing it now! This will stop the spam problem: If miners start rejecting blocks that are too generous to spamers, we shouldn't need a block size limit. I suppose, though, that the block size limit IS an implementation of a voluntary rule. Though a rather primitive one, it's sufficient for now.

Perhaps miners will converge to using an optimal rule, similar to what I've described and I'm trying to preempt this development. This definitely looks like the problem of finding an "optimal strategy" in game theory.

I don't disagree that there is a preservation motive to rival the profit motive, but I think that the profit motive is far more prevalent. Only some want to preserve the integrity of the chain, but everyone wants to make profit. As it is, contributing as an honest node is more profitable than attacking the network, but remove the incentive of fees and profit motivated generators become profit motivated attackers. Will the network be strong enough...


It would only take one non zero transaction to cut out the freeloading spammers. A generator could inject one himself, which would force clients to match it or come close. Anyway, da2ce7 has me half convinced that miners should be imposing their own rules on the blocks, forming a democracy. It's not like anyone can stop this anyway: A miner is always free to reject any block, for better or worse, for whatever reason it sees fit. So, I guess the conversation about block rules is moot and the question of how to keep fees up is in the hands of the mathematical (game theory) viability of the protocol. Has Satoshi already figured all of this out, I wonder?

That was not my complaint.  One non zero transaction would also only allow for one zero transaction.


I would say so.

Yes!  This sounds cool.  When generators disagree there will be inefficiency, confusion and confirmation-delays.  It would be useful to mediate agreement among competitors.


Miners could do it now, but the software-capital cost and uncertainty look high enough to postpone it for a while.

Meanwhile, it should be possible to simulate the network.  No need to calculate hashes, just model a population of generators G[0 .. g] pulling hashrates HPS[0 .. g] and having their own rules & motivations, and some clients.

This needs three sets of parameters,

• one to describe generators (effective costs for electricity, existing hardware & incremental hardware purchase) and allowing for resistive heating, gamers' hardware habit as well as hard-nosed corporations etc.
• another to describe 'voluntary rules' used by generators for declaring a block of transactions valid.  We have ideas here, they just need a weighting parameter each s they can be combined.
• finally (the hardest?) for a model which generates transactions among a population of merchants, speculators, hoarders, spammers etc..  Assuming they all know what the 'rules' are at any time, but can only influence them by generating, spending or getting patches accepted and installed.

Might make an interesting student project (one semester?).  I don't have time, sorry.

I suspect the generator rules can vary with time, but to fend off chaos they should be a deterministic function of the content of previous block(s) and not depend on the transaction pool.  I also suspect that if the customers & merchants do not hold a significant fraction of the Ghash/sec, the 'rules' will tend to make a generator cartel with high fees.

Yes I believe the real potential issue is the 'cartel,'  I'm not worried about the transaction fees being too low, rather too high.  I think that studying this with game theory would be very fun, but, I don't have the time also.

Will there be a default fee amount included in future clients, or will be have to guess/check what the average miner is charging to process transactions?

Bitcoin will look at network conditions and make a guess for you.

There can be no 'cartel' in Bitcoin, because there can be no monopoly.  The barriers to entry into the generation pool are simply too low, and there is no way that a government could enforce an artifical monopoly even if it wanted to.  If the transaction fees begin to creep up due to consolidation of the generation pool, others will begin to notice and they will choose to participate as the possibilites of a random reward become worth the costs.  This will create a price point that is ever fluctuating, but balanced, as this effect removes any pricing power of any group of generators in collusion.  I would predict that the transaction fees will be somewhere between 0% and 3% on average, but closer to 0%; because 3% is where the credit card companies and PayPal have set their prices, and they do have a degree of pricing power.  If a 'cartel' were to set up a datacenter that could dominate the network, but only include transactions with fees at say 2.9% (PayPal's regular rate), they would be ignoring an entire set of fee paying transactions that only non-cartel generators could capture.  The results being that the network would generally see that 2.9% is the premium price to get one's transaction recorded quickly, but if one is willing to wait until one of the non-cartel generators then there is a budget option (currently that option is free).  The cartel could not be certain that they could even capture the premium fees, so they would have a choice to make.  Either recognize that they don't have the kind of pricing power that they thought that they did, or ramp up their generation capabilities until they do.  The second choice creates an 'arms race', as for as long as the cartel increases their capacity, they must attempt to pass on that cost, but they can only do so temporarily before non-cartel generators see a profit opprotunity to do the same.  Eventually the cartel hits a point that it can no longer continue, and in the meantime the system has benefited from their attempts to corner a market that cannot be cornered.

Of course, anyone with the resources to do any of this is also going to hire people who know the system, who are likely to point out that they can't really do what it is that they want to do.

Some clients may have a default fee amount, but I doubt that those will survive in the marketplace.

@creighto, in general thank-you for sharing your excellent understanding of economics!  It is well appreciated, by myself, and I'm sure many others on this forum.

On cartels:

There will never be a problem with a cartel that has less than 50% of the generation power.  The problem lies with a cartel that has significantly more than 50%.  Any block that accepts transactions that the cartel rejects (i.e. low fees), would be orphaned by the cartel, and in a short time later, the cartel will have developed a block chain much longer than the chain developed by the so-called 'scabs.'

The only way I can foresee the free market correcting for the cartel, is for the people conducting the transactions (as a large group) to ignore the cartel's block chain when they behave badly, and only accept the (shorter) scab chain that isn't accepted by the cartel, until the cartel gives up and accepts the chain with small transaction fees.

The problem for bitcoin is that when this 'war' / standoff between the people doing the transactions and the cartel happens, there may be a good 20 blocks of uncertainty before a winner is decided.  It really is all about 'my gun is bigger than yours.'  This could mean that there would be a good 2h of time where transactions will not be secure from double spending.  However there is nothing stopping a smart client from checking both chains.

Oh, thank you!  And your quite welcome.


Although there is nothing wrong with anything you said in this post, bear in mind that the act of securing 50% of the generation is no small task even for a cartel with nation-state resources, and maintaining that majority is vastly more difficult.  I won't say that it is impossible, but I would say that the odds that any single nation or private group acting in concert could maintain such an escalating 'generator war' against not only the existing and/or future bitcoin community, but also the generating capacities of nations or other private/political groups who have something to gain by opposing the desires of the cartel/nation attempting the takeover would be somewhere close to the odds of a verifiable bitcoin address collision within my lifespan.

Which is to say, slim to none.

This was posted when I was away from Bitcoin for a while. I've just been referred here by another thread.

A miner or group of miners with >40% of the hashing power can mine more than their fair share of blocks under very reasonable expectations. Someone with 46% of the hashing power can win 51% of the blocks. I now call this strategy "hostile mining".

Recent post here http://bitcointalk.org/index.php?topic=6284.msg99312#msg99312

ByteCoin

I just saw a blog that stated that the tenX wallet dont charge any transaction fees for BTC transactions.

After i tried it, i can confirm there are no fees. After transfering 1USD my friend recieved 1USD.  

How ist that possible?