Algorithm for selecting a low-fee transaction from the mempool

[bomberb17]

I broadcasted a low fee (11sat/vbyte) transaction last week, just at the time when they fees went up.
I knew that the fee was low and that the transaction would take a few days to confirm (according to https://twitter.com/CoreFeeHelper , it would take around 3 days) but it's been a week and my transaction is still in the mempool. Sometimes my wallet (electrum) shows it with a blue "unbroadcasted" icon and I have to rebroadcast it again. I also sometimes use "free" tx accelerator services which basically rebroadcast the transaction everywhere.

My question is what is the algorithm that a miner picks a transaction from the mempool to include it to a block?
Is it a greedy algorithm which just picks the ones with highest fees? Is the "age" of a transaction in the mempool also a factor? Or is it just a combination of fee and a random choice?

[hosseinimr93]

My question is what is the algorithm that a miner picks a transaction from the mempool to include it to a block?

Miners prioritize transactions based on their fee rate. The more fee rate you use for your transaction, the faster it can be included in the blockchain.

Is it a greedy algorithm which just picks the ones with highest fees? Is the "age" of a transaction in the mempool also a factor? Or is it just a combination of fee and a random choice?

It doesn't matter when you have broadcasted your transaction. It all depends on the fee rate.

[bomberb17]

My question is what is the algorithm that a miner picks a transaction from the mempool to include it to a block?

Miners prioritize transactions based on their fee rate. The more fee rate you use for your transaction, the faster it can be included in the blockchain.

Is it a greedy algorithm which just picks the ones with highest fees? Is the "age" of a transaction in the mempool also a factor? Or is it just a combination of fee and a random choice?

It doesn't matter when you have broadcasted your transaction. It all depends on the fee rate.

So you say it is purely a greedy algorithm?

[Nwada001]

Since you are using the electrum wallet and you have already seen that your transaction is still unconfirmed, why not try and speed up the transaction by using the CPFP option so that you just have to replace the old transaction with a newer one with a higher transaction fee, which will speed up the transaction if you actually need to execute the transaction as soon as possible? I have tried most of these accelerator services, but they are not much different from where my transaction was before.

[hosseinimr93]

So you say it is purely a greedy algorithm?

It's not that there's a certain algorithm which miners have to follow that.
Miners are free to include any valid transaction they want. They can even include a transaction with zero fee, but they include transactions with highest fee rates to maximize their profit.

[bomberb17]

So you say it is purely a greedy algorithm?

It's not that there's a certain algorithm which miners have to follow that.
Miners are free to include any valid transaction they want. They can even include a transaction with zero fee, but they include transactions with highest fee rates to maximize their profit.

So to be precise, I'm not saying what algorithm miners should follow. Of course miners are free to choose any transaction they want (or even censor specific ones, despite if this means a lower reward for them).

What I am asking is what algorithm miners/mining pools typically follow.

[Yogee]

"Typically" would be filling up their mined block with the highest fee transactions as much as possible before the next one is mined so that falls under your "greedy algorithm". They can maybe accommodate some requests in including TXs with lower fees but that's definitely not their priority. They are here for the money after all - let's keep it real.

[BitMaxz]

What I am asking is what algorithm miners/mining pools typically follow.

Not miners who can decide what transaction they want to include on the block but the mining pool operator.
I think the algo they use is fee-rate or fee-per-byte where they can sort all transactions from low to high fees.

Only mining pool have their own way how to pick transactions but most of the pools always pick high transaction fees to make more profit.

[serjent05]

Since you are using the electrum wallet and you have already seen that your transaction is still unconfirmed, why not try and speed up the transaction by using the CPFP option so that you just have to replace the old transaction with a newer one with a higher transaction fee, which will speed up the transaction if you actually need to execute the transaction as soon as possible? I have tried most of these accelerator services, but they are not much different from where my transaction was before.

I think you are talking about RBF or replace-by-fee, while CPFP is the receiving end making a transaction with higher fees so when the child transaction is included on the block, the parent transaction will also be included.

The speed of the Bitcoin transaction to be confirmed is dependent on the fee.  I do not think there is any algorithm that can bypass the transaction fee hierarchy except if a huge mining farm operator decides to include low fees when they mined a block which rarely happens since most mining farm operator has the default option of picking the highest transaction fee before the lower ones.

[buwaytress]

"Typically" would be filling up their mined block with the highest fee transactions as much as possible before the next one is mined so that falls under your "greedy algorithm". They can maybe accommodate some requests in including TXs with lower fees but that's definitely not their priority. They are here for the money after all - let's keep it real.

Yup. Highest total fees after filling up the blocks is probably how they go, rather than just blind highest fees (e.g. if you can fit 11 txs with average fee of 10.1 sat/byte rather than just 10 txs with 11 sat/byte, that's what they'd go).

You can't really blame "greedy" algorithm, OP. It's a cost-revenue thing, quite literally.

Obviously, you can still see low fee txs, I suspect that's paid for in different ways (e.g. viabtc's paid accelerator... as opposed to broadcasting, which doesn't... accelerate).

[Aanuoluwatofunmi]

I broadcasted a low fee (11sat/vbyte) transaction last week, just at the time when they fees went up.
I knew that the fee was low and that the transaction would take a few days to confirm

Even when the fee was high, there are some that make use of some features like RBF to pump their transaction ad make it faster, they would have also reduce their transactions fee rate by using the lowest priority,  while using bitcoin mining pool that has the best offers in confirming their transactions with yours when the next block get confirmed.

My question is what is the algorithm that a miner picks a transaction from the mempool to include it to a block?

Those transactions with the higher fees get confirmed first with higher priority over those with lower transaction fee, but for the benefit of those that uses a mining pool to boost their transactions, their transactions is being added to their own block transaction to get confirmed altogether.

 

Is it a greedy algorithm which just picks the ones with highest fees?

They aren't greedy, just that the miners get higher opportunities in this for making more profits than before and when the whole blocks had being completely mined, they still have to depend on the transaction fee to earn their income

[stompix]

So you say it is purely a greedy algorithm?

It's a free market algorithm.
There are more people who want a confirmed tx than space available and since there is no other way to select tx that won't be abused by spammers and make thinks far worse this is it!

I'm pretty sure you would also sell your coins to the one offering $22.5 k when you have offers for 26.5k just because the other guy has been waiting for days for a customer!Right?

What I am asking is what algorithm miners/mining pools typically follow.

They all follow it, one exception to the rule is viabtc which allows up to 100 tx/h submitted on their website in the next block, but with their own rules, under 0.5 kb and over 10sat/b.

[dzungmobile]

My question is what is the algorithm that a miner picks a transaction from the mempool to include it to a block?
Is it a greedy algorithm which just picks the ones with highest fees?

If it is not their own transactions, they will be greed and prioritize transactions with highest fee rates from tip of mempool first. In other words, they can proceed their own transactions with zero or very low fees.

Is the "age" of a transaction in the mempool also a factor? Or is it just a combination of fee and a random choice?

There are many mempools, not only one mempool. Each mempool has its own capacity and own settings. They can drop pending transactions from mempools after 1 day, 3 days (72 hours) or 14 days. It depends on their random settings.

Setup guides
Drop pending transactions with low fee after 72 hours, max mempool is 300 MB.

Code:

mempoolexpiry=72
maxmempool=300

[Rruchi man]

My question is what is the algorithm that a miner picks a transaction from the mempool to include it to a block?

Miners prioritize transactions based on their fee rate. The more fee rate you use for your transaction, the faster it can be included in the blockchain.

@OP, if you are in a hurry to make a transaction, increasing the fee rate will ensure that you are given priority and you will not have to wait so much. In the mempool every transaction has a priority depending on the fee used if you check, there is a low priority fee for people not in a hurry, medium priority and high priority depending on how quickly you want your transaction. Miners respond to a transaction depending on the fee.

[yhiaali3]

You can't say it's greed. If you were them, would you take the low-fee transactions first? Or will you rush to take higher-fee transactions?

This is the logic that miners work with. If two people offer you to do the same work: one at 1$ and the other at 10$, which job will you accept?
This is the principle by which all human beings work.

[teosanru]

I broadcasted a low fee (11sat/vbyte) transaction last week, just at the time when they fees went up.
I knew that the fee was low and that the transaction would take a few days to confirm (according to https://twitter.com/CoreFeeHelper , it would take around 3 days) but it's been a week and my transaction is still in the mempool. Sometimes my wallet (electrum) shows it with a blue "unbroadcasted" icon and I have to rebroadcast it again. I also sometimes use "free" tx accelerator services which basically rebroadcast the transaction everywhere.

My question is what is the algorithm that a miner picks a transaction from the mempool to include it to a block?
Is it a greedy algorithm which just picks the ones with highest fees? Is the "age" of a transaction in the mempool also a factor? Or is it just a combination of fee and a random choice?

Yes obviously it's this also only which gives them highest reward or basically which has highest sat/ byte fees. I won't call the algorithm greedy rather I'll say it rational. If you see in a real world this is only the case. You'll always pick up highest paying person when you are given an option to do just a small task for someone. Can't blame miners when they do this.

Talking about rebroadcast. So rebroadcast does nothing but brings your transaction in the top of the pool again for miners to choose. But eventually if the fees is low and the current rate is still higher there are very less chances any miner will still pick it. If you really want to do the transaction your best bet will be to make it to redo the transaction with the higher fees the old one will get declined automatic.

[entebah]

I also sometimes use "free" tx accelerator services which basically rebroadcast the transaction everywhere.

which website that use for your tx accelerator? because i usually use tx accelerator with a medium fee and it got confirmed in the next block they mine.

My question is what is the algorithm that a miner picks a transaction from the mempool to include it to a block?
Is it a greedy algorithm which just picks the ones with highest fees? Is the "age" of a transaction in the mempool also a factor? Or is it just a combination of fee and a random choice?

i think the miner includes a transaction with the highest fees first because they got rewarded from the transaction fees that are included in the blocks they mine.
so before you send your transaction make sure you don't use a low fee because it will take days to get confirmed.

[tiCeR]

So you say it is purely a greedy algorithm?

It's not that there's a certain algorithm which miners have to follow that.
Miners are free to include any valid transaction they want. They can even include a transaction with zero fee, but they include transactions with highest fee rates to maximize their profit.

So to be precise, I'm not saying what algorithm miners should follow. Of course miners are free to choose any transaction they want (or even censor specific ones, despite if this means a lower reward for them).

What I am asking is what algorithm miners/mining pools typically follow.

I have asked a very similar question here and o_e_l_e_o kindly answered in detail. If you read that post I think everything should be answered. Miners will process transactions paying the highest fees and they are constantly building a "candidate" block which is adjusted on a highest fee first principle until the next block gets mined. Even if you are part of the candidate block for a second, your tx will only confirm once it is part of the candidate block that then also gets mined. Until your tx could be dropped and another one paying a higher fee be included instead. This is why I thought broadcasting a tx when fees are low in a certain moment will also ensure that your tx gets processed, but that was wrong because it could still get dropped again if the block hasn't been mined and a higher fee tx gets preferred over yours.

[Mate2237]

This had happened to me as well, I waited for one week plus and lastly when I checked back the transaction I saw remove transaction or canceled at first I clicked removed and the coins went back to my wallet straight, then after 24 hours they still remine it again and asked me to broadcast it which I did. And after 3 days the same thing to remove from transaction. Hey guys!!! I gat to cancel the transaction and happy peace of mind. People really suffered in these days of the high fee transaction.

[bomberb17]

"Typically" would be filling up their mined block with the highest fee transactions as much as possible before the next one is mined so that falls under your "greedy algorithm". They can maybe accommodate some requests in including TXs with lower fees but that's definitely not their priority. They are here for the money after all - let's keep it real.

"Typically" would be filling up their mined block with the highest fee transactions as much as possible before the next one is mined so that falls under your "greedy algorithm". They can maybe accommodate some requests in including TXs with lower fees but that's definitely not their priority. They are here for the money after all - let's keep it real.

Yup. Highest total fees after filling up the blocks is probably how they go, rather than just blind highest fees (e.g. if you can fit 11 txs with average fee of 10.1 sat/byte rather than just 10 txs with 11 sat/byte, that's what they'd go).

You can't really blame "greedy" algorithm, OP. It's a cost-revenue thing, quite literally.

Obviously, you can still see low fee txs, I suspect that's paid for in different ways (e.g. viabtc's paid accelerator... as opposed to broadcasting, which doesn't... accelerate).

The 10tx-11sat/byte vs. 11tx-10.1sat/byte makes some sense.

I didn't "blame" anyone, I am asking what is the algorithm they use to create a block.

They aren't greedy, just that the miners get higher opportunities in this for making more profits than before and when the whole blocks had being completely mined, they still have to depend on the transaction fee to earn their income

You can't say it's greed. If you were them, would you take the low-fee transactions first? Or will you rush to take higher-fee transactions?

This is the logic that miners work with. If two people offer you to do the same work: one at 1$ and the other at 10$, which job will you accept?
This is the principle by which all human beings work.

I didn't say that the miners are "greedy". I was asking if they are adopting a "greedy algorithm" to choose transcations.

Please all read first https://en.wikipedia.org/wiki/Greedy_algorithm