Fee discovery

[piotr_n]

Fee discovery, the way for you to figure out how much fee you need to pay so your transaction would be mined withing the next N hours - this is going to be a useful thing in a future.
We don't know how far future, but probably we will get to see it.

I heard that they work on something like this in the bitcoin core, but obviously their solution isn't going to work.
Because they don't use the bitcoin community to solve the problem.

I was thinking how to solve it and I think the most efficient way is to ask miners (mining pools) to include their transaction queue statistics into the blocks they mine.
So nodes could figure out how much they have to pay to be mined, just by looking into the headers of past blocks.

I think this is the easiest solution, and putting such info inside the coinbase's payload, makes it totally backward compatible.

BTW, I am really surprised that miners don't yet use the extra space, inside the coinbases, as advertising space.
for all we know its going to last longer than piramids

[piotr_n]

I heard that they work on something like this in the bitcoin core, but obviously their solution isn't going to work.
Because they don't use the bitcoin community to solve the problem.

There is no need to use the "community" when you have the blockchain.  The blockchain provides sufficient information to make a solid estimate of the parameters used by miners (max block size, min tx fee, and kb reserved for "free" high priority txs).  This lets you know what parameters miners are using and thus what future blocks are likely to look like.  The memory pool lets you know what tx are awaiting and where you will be in the last list when ranked by fee and/or priority.

Regardless of what miners say they will do, the blockchain provides proof of what they did.  

Yes, but the trick is to predict the future, not to measure the past.
Otherwise you will always lag behind, whenever there is like a rush hour peek.
Miners can say what fee is needed to include a transaction of a certain size in the next block - literally.
And that's all you need.
If that works, then they will even agree to lift the block size limit, because it would let them to keep up to their promises. Which is important in the fee business

[piotr_n]

If you are reading the coinbase to determine what fee requirements are then you are still reading the past.

No - you are reading a promise.
Miners have no reason whatsoever to deliver a false promise.

Miners do set their own hard limits to block size (many use 250KB and some use 700KB) these aren't limits imposed by the network but instead self imposed limits so if a miner requires 0.1 mBTC per KB and there are more than 700KB of tx waiting then paying 0.1 mBTC per KB is not a guarantee you will be included in the next block.

Of course.
Is there anyone who says otherwise?

But the mining business is going to develop, and eventually the fees will be the only thing that keeps it alive - the only thing that protects a security of our money.

Miners will develop new business models around fees.
The idea of them announcing bids on what level of fee they plan to include in the next block - this seems not only a good business for miners, but also a nice fee discovery solution for the community.
And the best part is: we don't need to change Bitcon Core, in order to achieve that. I mean miner do a bit, but most of them use their own mods anyway.

[piotr_n]

I don't know why you assume that miners would be tricking the network.
They care about the consistency of this network and its well being just as much as we do - maybe even more.

If a miner isn't going to deliver his promise, nobody is going to hang him, but why would he do that?
Had you been mining blocks, would you be putting a fake information, just to mislead people like this?
It would be really bad for your business, also unethical, so such mining pool would likely soon get abandoned by their miners.

What better idea do you have for the fee discovery?

[piotr_n]

If a miner isn't going to deliver his promise, nobody is going to hang him, but why would he do that?

I already told you.  To raise the average fee paid.  A miner can actually require a minimum of 0.1 mBTC to include a tx in the next block but advertise that it requires 0.2 mBTC

Fine - so what?
Then just do what you do now: assume that he's bluffing and pay as little as you want.
But then you wont be sure that he will mine it, in the next block.

If you want to play poker with miners - I don't see anything wrong with that.

No but any system should be built on the assumption that if a system can be gamed it will be gamed.   Promises without consequence are worthless in a trustless network.

But the promise is only as for the fee that guarantees a certain service - that's a service they sell.
If you don't want to buy the guarantee to be mined withing in the next block, use your system; set you fee to ten times less and pray...

[DeathAndTaxes]

If you dot want to buy the guarantee to be mined withing in the next block, use your system; set you fee to ten times less and pray...

I think we are done here and clearly posting was a waste of time.

[piotr_n]

I just don't understand why are you so much against miners giving promises on for which fee they guarantee to include a transaction in the next block.

How would such an information hurt anyone? And IMHO it would help a lot - enough to build a nice business model around it,

EDIT:
@DeathAndTaxes, I don't understand why you deleted all your previous post in this topic.

[NanoAkron]

Because it will trigger a positive feedback loop of ever-rising fees.

[piotr_n]

Because it will trigger a positive feedback loop of ever-rising fees.

Only if the miners create a cartel,
But if they do, with of without this feature, they can rise fees all they want.

I think you are too much paranoid about all these your evil miners.
Miners are not evil - they do very important job. And they profit in bicoins so they are actually most interested in keeping a value of this currency up.

[DannyHamilton]

Miners can already do this.

Miners already don't do this.

Do you think that by creating a post here at bitointalk, suddenly miners are going to say:

"Hey, piotr_n said we should do this.  We'll do what he says."

I'll watch the blocks for the next day or two, but I suspect your demands have been ignored by the mining pools.

Perhaps take this to the "Mining" sub-forum instead of the "Development & Technical Discussion" sub-forum and see what the miners have to say about it.

Note:  Bitcoin is a voluntary system.  We can't force the miners to implement your idea, all you can do is ask them, and then wait and see what they do.

[piotr_n]

Miners can already do this.

Miners already don't do this.

Do you think that by creating a post here at bitointalk, suddenly miners are going to say:

"Hey, piotr_n said we should do this.  We'll do what he says."

I'll watch the blocks for the next day or two, but I suspect your demands have been ignored by the mining pools.

Perhaps take this to the "Mining" sub-forum instead of the "Development & Technical Discussion" sub-forum and see what the miners have to say about it.

Note:  Bitcoin is a voluntary system.  We can't force the miners to implement your idea, all you can do is ask them, and then wait and see what they do.

I think I can certainly find a few people who find this idea interesting, some of them maybe mining pool operators already, others their friends.
And then, we can maybe work out some standard, add these options to the protocol and deliver a useful feature for the community.
Obviously not supported by the Bicoin Core - you guys are to proud, and have too powerful sponsors, to ask some dirty miners how much do they want to mine your so important transaction.

But other bitcoin clients/wallet? They are not as proud and all they care about is progress - to move this project forward.
Alternative clients would quickly adapt such a simple protocol.
Not even clients - you can just as well provide this information to the community as simple web page.
And then even people using the bitcon core would finally know what fee they should put, since they need to send it now.

[Cryddit]

I think the way to do fee discovery is to use data mining.  

I mean, given a set of assumptions and conditions, we want to predict the correlation of fees and delays?  We should be monitoring what the relationship of fees and delays is in correlation with those conditions.  IE, instead of getting miners to promise a particular behavior, we should be monitoring what miners actually do.

Important conditions are probably

• number and size of tx currently not in a block
• recent hash rate to difficulty ratio (block rate)
• the priority (bitcoin days destroyed) of the current transaction
• time of day (I bet strong repeating cycles show up on a 24-hour frequency)
• time of week (I bet strong repeating cycles show up on a weekly frequency too)

Anyway, set up a statistical regression on those variables and I bet that with you'll get a good solid basis for statistical prediction.  

[DannyHamilton]

add these options to the protocol

This doesn't require changes to the protocol. Any miner that wants to do this already can.  That was your point, remember?

And the best part is: we don't need to change Bitcon Core, in order to achieve that.

Obviously not supported by the Bicoin Core - you guys are to proud, and have too powerful sponsors, to ask some dirty miners how much do they want to mine your so important transaction.

Other than inflating your ego, I don't understand the point of this comment.  There's a lot of opinion, insults, and assumption in there, but not a lot of information.

Alternative clients would quickly adapt such a simple protocol.

Unlikely, but you are welcome to give it a try and see what happens.

Not even clients - you can just as well provide this information to the community as simple web page.

I've always wondered why mining pools don't already report their fee accepting rules on websites.  I've also wondered why they don't create a page where users can submit a request to confirm an unconfirmed transaction by paying an out-of-band fee (for receivers that are willing to pay a fee to get their transaction confirmed), and why they haven't implemented "child pays for parent" yet.  All three of those things seem likely to increase the total revenue for the mining pool.

[DannyHamilton]

I think the way to do fee discovery is to use data mining.

You, DeathAndTaxes, and I are all in agreement on this matter.

piotr_n seems to disagree.

More interestingly, he seems to believe that anyone that disagrees with him is not "the bitcoin community".

He keeps going on about "the Bicoin Core - you guys", and how "the bitcoin core, . . . they don't use the bitcoin community to solve the problem."

[Cryddit]

how "the bitcoin core, . . . they don't use the bitcoin community to solve the problem."

How is solving a problem by watching to see what the community does, not "using the community to solve the problem?"

[Peter Todd]

See my replace-by-fee-tools, specifically bump-fee.py for an alternative based on increasing your bid. Not always convenient by itself - fee estimation is still useful - but being able to increase fees after the fact lets you be much more aggressive in your fee estimates.

[piotr_n]

I think the way to do fee discovery is to use data mining.

You, DeathAndTaxes, and I are all in agreement on this matter.

piotr_n seems to disagree.

More interestingly, he seems to believe that anyone that disagrees with him is not "the bitcoin community".

He keeps going on about "the Bicoin Core - you guys", and how "the bitcoin core, . . . they don't use the bitcoin community to solve the problem."

It is not so much that I disagree.
I am just saying that your solution isn't going to work, after you spend another two years on developing it.
What I proposed is simple, very easy to implement and it achieves the goal.
I would have said that it achieves the same goal, if not for the fact that your data mining solution isn't going to achieve any goal.

[piotr_n]

See my replace-by-fee-tools, specifically bump-fee.py for an alternative based on increasing your bid. Not always convenient by itself - fee estimation is still useful - but being able to increase fees after the fact lets you be much more aggressive in your fee estimates.

Yeah, but this isn't trivial to implement and also creates additional traffic.
Sure, adjustable fees is a solution, but there is a long way for it.
Besides, one approach to the problem does not exclude another.

[Peter Todd]

See my replace-by-fee-tools, specifically bump-fee.py for an alternative based on increasing your bid. Not always convenient by itself - fee estimation is still useful - but being able to increase fees after the fact lets you be much more aggressive in your fee estimates.

Yeah, but this isn't trivial to implement and also creates additional traffic.
Sure, adjustable fees is a solution, but there is a long way for it.
Besides, one approach to the problem does not exclude another.

Just to be clear, the above is a complete, working, implementation of the idea. Any wallet software that handles double-spends and mutability correctly can implement it without much difficulty.

[DannyHamilton]

What I proposed is simple, very easy to implement and it achieves the goal.

Great!

Implement it.  I look forward to seeing it in action.

Thanks for all your hard work.