Why does it take so long for an unconfirmed transaction to be dropped?

[headingnorth]

This morning was the first time in 8 years of using bitcoin that a transaction sent from my hardware wallet
is still shown as pending and unconfirmed more than an hour (almost three hours).

I usually check mempool.space and adjust the fee high enough to avoid that problem but this time forgot to do it.
But problem with that is the default fee shown in your wallet app is often inaccurate and much lower than the actual amount.

So I just went with the default setting for the fee in the app, which is an easy thing to forget to do.
I only had a small amount of bitcoin in this wallet so just sent the whole amount to Kraken.
But if you send the whole amount from you wallet you can't use the RBF (replace-by-fee) feature or cancel it if has been
stuck pending too long. Why?

Because RBF uses the change UTXO amount sent back to your wallet to fund the higher replacement fee,
so if there is no change left you have nothing to pay the RBF with. Which is another thing I wonder about because
why can't the replacement fee just be taken out of the pending UTXO (the main amount being sent)?
That would make sense to do but I assume there is a technical reason why it's not possible.

I wonder why bitcoin isn't simply programmed to automatically cancel an unconfirmed transaction within a
more reasonable amount of time? It seems that should be a relatively simple thing for the devs to do. AFAIK it's not
uncommon for bitcoin transactions to get stuck for hours or even days before it is dropped or cancelled by the network
and the pending amount returned to the sender. I believe the current maximum time is two weeks before automatically
getting dropped, which seems an absurdly long time to wait. Even ACH transfers are faster.

It shouldn't be more than an hour or two maximum IMO. If the TX doesn't go through after one or two hours then it should
be automatically dropped from the system and returned to sender so they can resend it with a higher fee.
To me one that is a much more reasonable time to wait. Two weeks is crazy.

It's things like this that makes bitcoin a turnoff to use for many, and I believe remains a huge barrier to greater mass adoption.
The user-friendliness is still quite poor. It seems bitcoin is designed to appeal to the tech savvy not the masses, but even tech savvy
people make mistakes so human error should be taken into account as much as possible. Even if someone is very tech savvy doesn't
mean they have hundreds of hours to spend learning to become proficient at using bitcoin. The learning curve is still very steep.

Fortunately for me the amount of my bitcoin still in limbo is only a couple hundred dollars' worth, but imagine if the price
shoots up 20% with in a couple days and you want to sell a large amount to lock in profits, but you can't do it
because your damn TX is stuck pending for hours or days stuck in the mempool. And by the time it goes through the price
has tanked 20% or more. Oh oh you are screwed.

[Charles-Tim]

Because RBF uses the change UTXO amount sent back to your wallet to fund the higher replacement fee,

This is not true, you can use part of the coins that you sent for the replace-by-fee which solves the problem. I have sent coins like this to exchanges before and the exchanges still credited me even when I used RBF. Although, not advisable if you are sending to a custodial platform, but the exchanges know that the coins are coming from you.

[headingnorth]

Another improvement could be to prompt or remind the sender with a pop-up notice if the TX fee is too low
and recommend it to be raised before finalizing the TX.

That way you don't have to worry about forgetting to adjust the fee and avoid the hassle of trying to correct it after the fact.

Because RBF uses the change UTXO amount sent back to your wallet to fund the higher replacement fee,

This is not true, you can use part of the coins that you sent for the replace-by-fee which solves the problem. I have sent coins like this to exchanges before and the exchanges still credited me even when I used RBF. Although, not advisable if you are sending to a custodial platform, but the exchanges know that the coins are coming from you.

But it's not letting me do that even though Blockchain Explorer says RBF is turned on.  

[Charles-Tim]

But it's not letting me do that even though Blockchain Explorer says RBF is turned on. 

That is the problem with the hardware wallet that you are using and not how bitcoin wallets are desiged to use RBF. On Electrum, you can set it to decrease payment or preserve payment.

Preserve payment will be chosen if you do not want part of your coin in the transaction to be used to increase the fee
Decrease payment will be chosen if you want part of your coin in the transaction to be used for the RBF.

If you can connect the wallet with Electrum, it will be better so that you can make use of the feature.

[headingnorth]

But it's not letting me do that even though Blockchain Explorer says RBF is turned on.  

That is the problem with the hardware wallet that you are using and not how bitcoin wallets are desiged to use RBF. On Electrum, you can set it to decrease payment or preserve payment.

Preserve payment will be chosen if you do not want part of your coin in the transaction to be used to increase the fee
Decrease payment will be chosen if you want part of your coin in the transaction to be used for the RBF.

If you can connect the wallet with Electrum, it will be better so that you can make use of the feature.

If that is true then RBF probably would have worked better for me using the Sparrow wallet app instead of Trezor suite.

But I normally use Sparrow only for sending larger amounts and Trezor suite for smaller amounts.

[NeuroticFish]

I wonder why bitcoin isn't simply programmed to automatically cancel an unconfirmed transaction within a
more reasonable amount of time?

Because many bitcoiners know about patience and it's not uncommon one sends a tx he doesn't care if it will be confirmed in one hour or 10 days.
You have unrealistic request imho. Banks usually return double payments or card payments not redeemed by the shops in about 30 days.

Plus, if you cannot - for some reason - do RBF, ask the recipient to do CPFP if he's in a hurry.

[Ambatman]

It's things like this that makes bitcoin a turnoff to use for many

You sound like someone that haven't had a transfer issue with a Bank.
Maybe it's my country but One of my biggest fear, not that it is stressful and eats your time
It takes long before they are rectified.

Another improvement could be to prompt or remind the sender with a pop-up notice if the TX fee is too low
and recommend it to be raised before finalizing the TX.

Electrum warns if network are below a certain threshold.
I sometimes check mempool for fee before making any transactions.

[headingnorth]

I wonder why bitcoin isn't simply programmed to automatically cancel an unconfirmed transaction within a
more reasonable amount of time?

Because many bitcoiners know about patience and it's not uncommon one sends a tx he doesn't care if it will be confirmed in one hour or 10 days.
You have unrealistic request imho. Banks usually return double payments or card payments not redeemed by the shops in about 30 days.

As I mentioned an ACH bank transfer takes 4-5 business days at most. Bitcoin is normally a lot faster than that with settlement
speed less than an hour or so on average. Unless there is network congestion or you forgot to pay a high enough fee which
is a common problem, then the wait could be much longer than a bank ACH or wire transfer.

That is painfully slow, and unacceptable for many, especially when you have lightning-fast competitors like solana,
arbitrum, polygon, etc. that settle within 5 minutes or less. I don't think bitcoin has to be that fast to compete,
but a settlement of many hours or days is just not acceptable. Lightning was supposed to be a fix but the problem is
most major exchanges still don't support lightning TXs. And lightning can be difficult and technical to use.

Plus, if you cannot - for some reason - do RBF, ask the recipient to do CPFP if he's in a hurry.

I thought CPFP requires you to have change in your wallet. So if you sent the full amount there is nothing
to draw the fee from. But that could depend on the wallet being used similar to RBF, as someone mentioned.

[nc50lc]

I thought CPFP requires you to have change in your wallet. So if you sent the full amount there is nothing
to draw the fee from. But that could depend on the wallet being used similar to RBF, as someone mentioned.

Bitcoin is capable of doing that, RBF doesn't restrict the replacement transaction from changing any of output, amount or entirely.

Refer to these rules to check its limitations: BIP-0125 - Implementation Details
On a side note: last rule is talking about a complication for when an unconfirmed transaction with unconfirmed children got dropped or replaced.

Next to CPFP:

I thought CPFP requires you to have change in your wallet. So if you sent the full amount there is nothing
to draw the fee from. But that could depend on the wallet being used similar to RBF, as someone mentioned.

You are correct in this one.
Child-pays-for-parent is simply another transaction that spends an unconfirmed transaction's output set with a high transaction fee.
Without another output (change) that you can spend, only the recipient can do CPFP to it.

The goal is to make its txfee high enough that the miners couldn't resist including it to their block.
However, since its parent transaction is unconfirmed and it can't be confirmed without it, they'll have to include both transactions.

[NeuroticFish]

Plus, if you cannot - for some reason - do RBF, ask the recipient to do CPFP if he's in a hurry.

I thought CPFP requires you to have change in your wallet. So if you sent the full amount there is nothing to draw the fee from.

You thought correctly, just you didn't pay attention to what I wrote.
While in your situation you indeed cannot do CPFP, you can ask the recipient do so, if he's in a hurry.
Of course, this works when the recipient is a person you can talk with.

Unless there is network congestion or you forgot to pay a high enough fee which is a common problem

Interestingly it happened to me only 1-2 times in more than 10 years to underpay a tx when I was in a hurry.
It was more often I underpaid knowingly that I'll have to wait for longer.

That is painfully slow, and unacceptable for many, especially when you have lightning-fast competitors like solana,
arbitrum, polygon, etc. that settle within 5 minutes or less.

Well, they have LN for that. Of course, they have to fill their LN bucket, which means planning.
Plus, I am under the impression that most people expecting such fast settlements do rather small value payments/transfers. It's known that bitcoin is not that good equipped for that and the current solution for that is LN.

I don't think bitcoin has to be that fast to compete, [...]. Lightning was supposed to be a fix but the problem is
most major exchanges still don't support lightning TXs. And lightning can be difficult and technical to use.

I agree with the fact that it's a problem that most businesses don't accept yet LN.
I disagree that it's difficult to use, although I understand you, I faced a similar "barrier" until I actually started using it.

As I mentioned an ACH bank transfer takes 4-5 business days at most

Now let's use your kind of words.
Comparing bitcoin to an system that works only inside a county is unacceptable and void.
Comparing bitcoin to the fastest local settlement is unacceptable and void.
Comparing bitcoin to something that is not trustless is unacceptable and void.

You are insisting with the "not acceptable" wording. That's too harsh imho and those who cannot accept the reality can turn to other networks, it's a free world.
On the same topic, while I do agree that improvements can be done here and there, dropping any transaction so fast is not a viable option, since, as I mentioned, people may underpay on purpose.
If I try to think in a constructive manner, maybe in the future a setting/flag could be set up to mark transactions as "confirm in in x blocks or drop it". I don't know how difficult would that be (more technical people will tell if it's a viable option), but it could end up as an under-used feature, with people still make mistakes and complain. Plus, you probably know that "x blocks" is not the best time-wise estimation (but you won't get any better).

[Charles-Tim]

While in your situation you indeed cannot do CPFP, you can ask the recipient do so, if he's in a hurry.

He sent the coin to an exchange, Kraken. I think he is the owner of the exchange account but you know that CPFP can not be done on the exchange.

I posted this because he included this in the OP:

I only had a small amount of bitcoin in this wallet so just sent the whole amount to Kraken.

[NeuroticFish]

He sent the coin to an exchange, Kraken. I think he is the owner of the exchange account but you know that CPFP can not be done on the exchange.

I did miss that at start, but I was writing for generic cases, not his specific situation.
And meanwhile (after seen that it's about an exchange) I've added:

You thought correctly, just you didn't pay attention to what I wrote.
While in your situation you indeed cannot do CPFP, you can ask the recipient do so, if he's in a hurry.
Of course, this works when the recipient is a person you can talk with.

Maybe it doesn't help him specifically (because, indeed, I see no reason they'd be in a hurry to get those coins)... oh well, too bad...

[NotATether]

Transactions stay in the mempool until the default mempool size grows too big to accommodate all transactions, in which case the low-fee transactions are automatically dropped from nodes.

With the current mempool fee rate and sub 1 sat/vbyte transactions, it is unlikely that the mempool will grow large enough to overflow any time soon.

The other way to get a transaction is in the block immediately is to ask a miner, but you'll need to provide something in return.

[Synchronice]

I usually check mempool.space and adjust the fee high enough to avoid that problem but this time forgot to do it.
But problem with that is the default fee shown in your wallet app is often inaccurate and much lower than the actual amount.

Sparrow has a very accurate fee suggestion. The same can't be said about Electrum but if you check mempool.space, then you can really make a great assumption about fees. Look at how many blocks ahead the mempool currently is, also look at when where the last 6 or more blocks mined, especially when the last block was mined. If 50 minutes have passed since last mined block, then be assured that the next block will be mined soon. Check the fees of upcoming block and also check the size of transactions in that block. This way you'll understand what your fee should be.

I wonder why bitcoin isn't simply programmed to automatically cancel an unconfirmed transaction within a
more reasonable amount of time? It seems that should be a relatively simple thing for the devs to do. AFAIK it's not
uncommon for bitcoin transactions to get stuck for hours or even days before it is dropped or cancelled by the network
and the pending amount returned to the sender. I believe the current maximum time is two weeks before automatically
getting dropped, which seems an absurdly long time to wait. Even ACH transfers are faster.

Bitcoin is basically the same that it was since the day it was created, there are only minor changes. So, my point is that nothing can be perfect from the very beginning. We have the code that we can modify if there is a need, willingness and acceptance from the devs and the community.

[hosemary]

Transactions stay in the mempool until the default mempool size grows too big to accommodate all transactions, in which case the low-fee transactions are automatically dropped from nodes.

It may worth mentioning that nodes usually drop transactions from their mempool after a certain time, even if their mempool isn't full.
Nodes using default settings keep transactions in their mempool up to 14 days and unless the transaction is rebroascasted, it would be dropped from the mempool.