estimate tx fee in script

[t.altcoin]

how to estimate tx fee in java, python and so on?

[1715160006]

1715160006

[1715160006]

1715160006

[1715160006]

1715160006

[ReggieBush]

There's programs and websites that will do that for you. Block.io to name one

[ABCbits]

You could either use API which return fee estimation or use RPC (if you run Bitcoin Core) and use command estimatesmartfee, example :

Code:

estimatesmartfee 1

Code:

estimatesmartfee 6 "ECONOMICAL"

[NotATether]

First you need to have the transaction size. Then you can either calculate the fee based on the mempool depth, how deep in the mempool you want it to be in, or you calculate based on ETA of how many blocks have to pass before your transaction is included. estimatesmartfee uses block inclusion ETA as its calculation. The first parameter is the number of blocks from now you want the transaction to be included in. It does not let you pass the transaction size, but you can divide the return value by 1000 (it returns sats/kB so this converts it to sats/byte), and multiply it by your transaction size to get your transaction fee.

You need a full node that is configured with an rpc username and password. You could also initialize one on localhost for your personal use. Then you use one of the code snippets at https://en.bitcoin.it/wiki/API_reference_(JSON-RPC)#Languages to call estimatesmartfee.

The documentation for estimatesmartfee: https://bitcoincore.org/en/doc/0.20.0/rpc/util/estimatesmartfee/

[bitmover]

There's programs and websites that will do that for you. Block.io to name one

All of the programs, api, etc that estimate fees and suggest one overestimate them.
Why? Because it is impossible to predict the future. You dont know if a bunch of new transactions are coming paying a higher fee and competing to get into the next block.

So those websites just suggest you a much higher fee that will mostly warranty to get into the next block
 I dont think anyone should use them.

The best thing is to look at the mempool and see what is the mempool size and what is the lower one that will probably get confirmed.

I do not know any good recommendation fee script.

[PrimeNumber7]

The best thing is to look at the mempool and see what is the mempool size and what is the lower one that will probably get confirmed.

This is exactly what any program or script will do.

The difficulty in estimating the required fee is you don't know when the next blocks will be found. Assuming the last block was found 1 minute ago, the next block may be found in one second, 9 minutes, or in over an hour. If the next block is found in one second, the required fee to get a transaction included in that block will be lower than if the next block is found in an hour because transactions would be included in the interim.

For transactions you need confirmed quickly, you need to overestimate the fee because of the risk there are long gaps between when the next two blocks will be found. If your program assumes 10 minutes between now and when each of the next two blocks will be found, you will underestimate the required fee to get a transaction included 1/2 the time. If your program assumes 20 minutes between now and when each of the next two blocks will be found, you will more frequently get your transaction confirmed in the next two blocks, but will also overpay most of the time.

[HCP]

To be honest... It's more art than science.

There are so many variables involved that it is pretty much impossible to say with complete certainty what a "good" fee is... do you need block confirmation? can you wait a day (or more)? Is the network busy or are there only a few hundred unconfirmed transactions? Is it about to take 2 hrs to find the next block? Is BitMEX about to dump it's daily transactions onto the network? Is <insert "random" event here> about to happen that pumps/dumps the price and causes a flurry of activity on the network? Is today a day ending in the letter "y"?

Attempting to code a "catch all" fee estimation tool is going to be pretty difficult... like PrimeNumber7 says, you can either shoot "low" but end up with the potential to miss your time/block target... or aim "high" to try and meet your time/block target more often than not, but will likely end up "overpaying"...

I have personally been caught out a number of times by "cheaping out" on fees... trying to be clever and hit the sweet spot of "minimal fee with reasonable confirmation time"... a block isn't found for an hour or 2... the transaction backlog builds up, fees go up... transaction gets "stuck" for 12-24 hrs...

You get what you pay for, I guess

[bitmover]

The difficulty in estimating the required fee is you don't know when the next blocks will be found. Assuming the last block was found 1 minute ago, the next block may be found in one second, 9 minutes, or in over an hour. If the next block is found in one second, the required fee to get a transaction included in that block will be lower than if the next block is found in an hour because transactions would be included in the interim.

For transactions you need confirmed quickly, you need to overestimate the fee because of the risk there are long gaps between when the next two blocks will be found. If your program assumes 10 minutes between now and when each of the next two blocks will be found, you will underestimate the required fee to get a transaction included 1/2 the time. If your program assumes 20 minutes between now and when each of the next two blocks will be found, you will more frequently get your transaction confirmed in the next two blocks, but will also overpay most of the time.

To be honest... It's more art than science.

I made a very simple script which will just look at the mempool and suggest the lower fee within the 1 MB blocksize.
The transaction may not be confirmed within the next block.

What do you guys think? Is  it useless, or should I publish in my mempool page? I thought about creating an alert like this "Optimum fee: X sat/vbyte. Warning: Your transaction may not be confirmed within 30 minutes, but probably will." (or soemthing like that)

I don't think any of those fee estimators do any good. Few days ago the mempool was cleared (1 sat byte would get a confirmation) and electrum was suggesting 200 sat/byte .

[ranochigo]

I made a very simple script which will just look at the mempool and suggest the lower fee within the 1 MB blocksize.
The transaction may not be confirmed within the next block.

What do you guys think? Is  it useless, or should I publish in my mempool page? I thought about creating an alert like this "Optimum fee: X sat/vbyte. Warning: Your transaction may not be confirmed within 30 minutes, but probably will." (or soemthing like that)

I don't think any of those fee estimators do any good. Few days ago the mempool was cleared (1 sat byte would get a confirmation) and electrum was suggesting 200 sat/byte .

Hmm, how different is it from pre-existing ones? There's no harm sharing with us.

I think the main aspect that most estimators are missing is likely the issue that the fees are sticky downwards and tend to be less responsive to the smaller mempool. I find it hard to estimate fee given the variables and uncertainty. It'll be great if it could display the fee stats of a few hours prior and a visualisation of the current mempool.

Anyways, when I feel that Electrum is overpaying with their ETA fee estimation, I'll just switch to mempool and hope for the best. Don't find anything wrong with it, their ETA has to be more cautious since mempool could be smaller due to a bunch of blocks being mined in succession and resulting in a perceived smaller mempool size.

[bitmover]

Hmm, how different is it from pre-existing ones? There's no harm sharing with us.

It is different because all of them overestimate fees. This is the only one to underestimate them. 

Take a look here


I am not sure if I will keep it... I would like to know what people think about it.

I just suggested the lowest fee rate inside the top 1MB of the mempool


Edit:
My script for fee estimation is just this:

Code:

for (let i = 0; i < cumulative.length; i++) {
	if (parseInt(mempool_size - cumulative_size[i]) / 4000000) == 0) {
			  break;
			}
	else {}
}}

How does it work?

mempool_size = total mempool size in Weight Units.
i=fee rate (which I will loop through all of them)
cumulative_size(i) = Sum of the size of all transactions paying a more than or equal to that fee rate i.

I divide everything by 4000000, so I will deal with vBytes instead of  weight units.

 I loop through the all fee rates currently in the mempool, from 1 to the higher mempool feerate, calculating how much is the mempool size above it. Then i convert everything to integer (to ignore decimals).
The first fee rates which returns less than 1vMB will break the loop, returning zero (as I convert them to integers). This is the "recommended " fee.

Very simple. I do not try to predict the future.

Limitations:
 Sudden spikes in the mempool will make your transaction get delayed. Fee is underestimated in those cases.
But i think this is better than suggesting 200 vbytes while you could use less than 10.

[HCP]

I am not sure if I will keep it... I would like to know what people think about it.

I guess it depends if you think that underpaying (and potentially getting a transaction stuck) is "worse" than overpaying (and potentially spending a few hundred/thousand satoshi's more than you needed to).

Rather than choosing to always under or overpay... I tend to go with whatever is my priority at the time. Sometimes, I'm just consolidating funds and don't anticipate needing to spend anything in the immediate future, so I can afford to wait hours (or days) and a 1 sat/vbyte fee is just fine. Other times, I need the money moved ASAP, so if I end up having to use 20 sats/vbyte instead of 10 sats/vbyte to "guarantee" it in the next 30mins-1hr (and then wait 2 hours for a block! then so be it...

Unless you've got large transaction weight, the difference in the fee rate isn't going to make much of a difference to the total fee anyway... 168 sats vs 3000 sats? that's ~3c vs ~50c... chump change really.

Once you get into transactions that are like 1000bytes+ and/or you start using fee rates 100+ sats/vbyte, then things start to get a bit more "expensive"...


Still, it's nice to see that people are thinking about this stuff and taking a slightly different approach.

[ranochigo]

Limitations:
 Sudden spikes in the mempool will make your transaction get delayed. Fee is underestimated in those cases.
But i think this is better than suggesting 200 vbytes while you could use less than 10.

Pros and Cons to both approach I guess. I don't think your code would compensate for the sudden drop in fees right after a block is mined right? Fees after a block is mined tend to be lower (of varying degrees depending on the fee structure). Giving users the visualisation of the fees is great but I think users tends to complain more when their transactions get stuck than when they over pay for the fees, from what I've observed anyways.

I actually stumbled on your site a few days ago and I liked the graph on the right a lot. Please do keep it

[bitmover]

I guess it depends if you think that underpaying (and potentially getting a transaction stuck) is "worse" than overpaying (and potentially spending a few hundred/thousand satoshi's more than you needed to).

Yes, this is why I warned "If you need to get into the next block use your wallet recommended fee".

BUt I think this is a good cost/benefit fee.

Still, it's nice to see that people are thinking about this stuff and taking a slightly different approach.

Thanks.

I don't think your code would compensate for the sudden drop in fees right after a block is mined right? Fees after a block is mined tend to be lower (of varying degrees depending on the fee structure).

If a block is just found when the user open the  tool, he see a lower fee than he would see a minute before.

My tool is always suggesting the lowest fee if a block is found now... There is no compensation (like suggesting a slightly higher fee because the last block was found one minute ago). This could be done... i will think about it. I like to keep things simple.

Giving users the visualisation of the fees is great but I think users tends to complain more when their transactions get stuck than when they over pay for the fees, from what I've observed anyways.

They are already a lot of people complaining that they are paying high fees (even when mempool is empty)

For example, take a look here:
Here is why btc will die as a crypto

By the time he made that post, the mempool was empty, but he used a 36 sat/byte fee. (which is not very high, but not low)

I made a check in my website that will make a warning if you are using a very high fee.

I actually stumbled on your site a few days ago and I liked the graph on the right a lot. Please do keep it

Thank you

[PrimeNumber7]

I made a very simple script which will just look at the mempool and suggest the lower fee within the 1 MB blocksize.
The transaction may not be confirmed within the next block.

What do you guys think?

I don't think looking at the current state of the mempool is sufficient. What if for example, 4 or 5 blocks were recent found in a span of 3 minutes? A fee consistent with transactions 1 MB 'deep' in the mempool might not confirm for hours/days. On the other hand, if there have been 2 blocks found in the last 3 hours, a transaction 1 MB 'deep' might confirm in just a few blocks. In all cases, a transaction 1 MB 'deep' is unlikely to be confirmed in the next block.

IMO, any fee estimator needs to have an adjustable parameter that assumes the average block time for the next n blocks. Over the long term, blocks will be found on average, once every approximately every 10 minutes, but at 2:00 on a Saturday afternoon, the time until the next 6 blocks are found could be a wider range of times.

edit:
maybe a better analogy would be the last block was found 10 seconds ago versus the last block was found 90 minutes ago.

[ranochigo]

I don't think looking at the current state of the mempool is sufficient. What if for example, 4 or 5 blocks were recent found in a span of 3 minutes? A fee consistent with transactions 1 MB 'deep' in the mempool might not confirm for hours/days. On the other hand, if there have been 2 blocks found in the last 3 hours, a transaction 1 MB 'deep' might confirm in just a few blocks. In all cases, a transaction 1 MB 'deep' is unlikely to be confirmed in the next block.

IMO, any fee estimator needs to have an adjustable parameter that assumes the average block time for the next n blocks. Over the long term, blocks will be found on average, once every approximately every 10 minutes, but at 2:00 on a Saturday afternoon, the time until the next 6 blocks are found could be a wider range of times.

edit:
maybe a better analogy would be the last block was found 10 seconds ago versus the last block was found 90 minutes ago.

Definitely isn't. There isn't a reliable way to predict all the factors and what they can do is to give the best estimation. If you want to assume such scenarios, then there lies the risk of overcompensating which is what some fees prediction formulas accounts for and could grossly over-estimate the fees.

I don't think there would ever be a way to accurately predict the fees given the huge uncertainty of the various factors and giving the users a choice to see which of the fee mechanism they want to follow would be the best. Anyways, I don't think having such excessive fluctuations are common, the varience wouldn't be that big without a huge cut in the hashrate.

[bitmover]

I don't think looking at the current state of the mempool is sufficient. What if for example, 4 or 5 blocks were recent found in a span of 3 minutes? A fee consistent with transactions 1 MB 'deep' in the mempool might not confirm for hours/days. On the other hand, if there have been 2 blocks found in the last 3 hours, a transaction 1 MB 'deep' might confirm in just a few blocks. In all cases, a transaction 1 MB 'deep' is unlikely to be confirmed in the next block.

Thanks for your suggestions.

I added 1mb deep fee +  5 sat. I think this will not overestimate, but will get a closer confirmation.

I have been staring the mempool for long times while I was creating this tool, and I noticed that a 1mb deep fee+5 really can get a confirmation in the next block (or in the next 2)

The website updates every minute, and I can see that the suggested fee will be going up everyminute from 0 to 2 sats/minute. Then it drop again to the same value as 10 minutes before when the block is mined.

As I added a 5 extra sat, and I don't know when user opened the website (in the first minute the block was mined or the last minute before it is mined again), the tx will be very close to the lower confirmed in that block.


For now, I kept the website opened for 20 minutes, and the suggested fee range was from 77-80 all the time. Blocks were mined in that period and some 72,75,76 fee transactions were confirmed.

You can check the block (from 1 minute ago) here:

https://www.blockchain.com/btc/block/000000000000000000070ecb6abb85eed06c6be4c31519ad8ee8d15c62de709d?page=5

[NotATether]

The way the mempool works, is that there will always be unlucky transactions tailed at the end. Even if someone manages to make the most sophisticated fee estimator, it's still a queue at the end of the day. You could have all the transactions using "smart" fees but some will still end up waiting for several hours at the back, or overpaying at the front, because the others had just a little smarter fees than the rest. It's a never-ending arms race against the network.

[BrewMaster]

The way the mempool works, is that there will always be unlucky transactions tailed at the end. Even if someone manages to make the most sophisticated fee estimator, it's still a queue at the end of the day.

it is not a "queue" but a "pool". a queue is for when objects enter the group they keep their order and the first one in is the first one out (FIFO). a pool however has no order.

You could have all the transactions using "smart" fees but some will still end up waiting for several hours at the back, or overpaying at the front, because the others had just a little smarter fees than the rest. It's a never-ending arms race against the network.

i wouldn't call overpaying fees "smarter". paying the exact amount needed to have high priority is "smart".
as for the arms race, you are correct but those over-paying or under-paying are small percentage and rarely affect anything.

[HCP]

i wouldn't call overpaying fees "smarter". paying the exact amount needed to have high priority is "smart".

And there is the rub... what exactly is "the exact amount needed to have a high priority"? This can not be answered with any real certainty as there are just so many factors at play, some of which are just not predictable.

The exact amount "now" may not be the "exact amount" required in even 1 seconds time depending on things like new transactions being broadcast, a block being found etc. It's all guesswork, and likely always will be.

[bitmover]

The exact amount "now" may not be the "exact amount" required in even 1 seconds time depending on things like new transactions being broadcast, a block being found etc. It's all guesswork, and likely always will be.

All this uncertainty created this "culture" of overestimating fees all around.  Even when the mempool is clear you can see 100 sat/byte transactions in the mempool ( I have been checking it daily while I was creating this tool). Those overpayed fees are probably from newbies that just followed wallets recommendations.

I believe I made a nice estimation of the minimum fee to get into the next block. It is not 100% to get into the next block, but you will get a confirmation in few blocks and you will pay a reasonable fee.