Of course I do set the fee manually for my transactions.

But if there is a bad algorithm for fee estimation and everyone uses it, for the network, it doesn't really matter what one or 2 users do.

The algorithm used by bitcoin core is likely used by many big players that are not going to set the estimate manually, they have to rely on an algorithm and they trust bitcoin core.

To be more clear:
Right now I'm not advocating for any central planning or stuff like that.
I'm just trying to understand why the vast majority of the network does fee estimation poorly.
It's something everyone can see by looking at the last few days and hours.

Is there any issue in bitcoin core fee estimation algorithm right now?
Looking at the mempool:

While the mempool looks empty, I get +110 sat/b fee estimation from bitcoin core to get into the next block and +90 to get into the next 6 blocks.
Am I the only one seeing these numbers?

I couldn't have said it better myself.

Im not even sure that’s the case.
I believe people using light wallets like “blockstream green wallet” or all the other light wallets that get their fee estimation from bitcoin core will be unaffected by this change

No, it is actually quite different from my idea.
I think it is necessary to punish wallets that do a bad fee estimation because they damage the user experience and ultimately bitcoin in general.
One of the ways to incentivise them is relaying their transactions only if they don't overpay.
For example if the mempool is basically empty there is zero reason to relay a transaction of 20 sat/b.

Having a parameter that you can define in your bitcoin node that specifies how much clients are allowed to overpay to get their transaction included in the nodes mempool, and so relayed, would help to improve things imho.
it is like cold war weapon.
Of course people could just ignore it, but by just implementing it and having it available would be a signal for the network:
- hey be careful that if you overpay you risk to not have an optimal propagation of your transaction.
A block explorer could tag those transactions with a warning saying: "this transaction payed too many fees, this could mean it is not relayed optimally by the network and therefore could not be confirmed quickly"
Please RBF it to a lower fee to make sure it pays a reasonable amount

With your idea you don’t really optimize, you only make things cheaper and that may not be good because eventually bitcoin will have to pay for its security with just mining fees.
What I am proposing is not to change the rules to make cheap tx.
What I want is to give the option to nodes to set a parameter of maximum allowed overpaying.
For example I would set this parameter to 1 sat/b while you may set it to 5 sat/b and someone else might leave it unlimited like it is now.
Eventually I think that having this option would lead to people using it and force wallets to improve their fee estimation to be sure everyone relays their tx

That's why you want to incentivize them to double check what they are doing.
If they want to pay too much they should risk to have their transactions being not relayed by all the nodes.
My node simply doesn't understand why another node wants to overpay when they could get into the next block just paying 1 extra sat/b.
Since for my node that overpaying is an attack trying to drive up fees for everyone than it does not include it in its mempool and doesn't relay that tx until it makes sense to pay that much.

Of course users can just pay 201 instead of 200.
What I propose here is to apply a MAX amount of how much they should be able to pay more if they want to make it into my node's mempool.
Like if there are 2 tx of 200 bytes each and paying 200 sat fee -> you can't pay more than 400 sat for a tx of 200 bytes if you want to make it into my node's mempool.
If instead there are 10k transactions that pay up to 200 sats and other 10k transactions that pay up to 400 sat -> you can start paying up to 600 sat to get into my node's mempool

I totally agree with you that it is hard to solve the problem.
But maybe we could find a way to point the other implementations in the right direction by not allowing users to overpay for transactions.
We all know that miners would hate this kind of implementation because it would cause less income for them in the short term.
But I think in the long therm it would make bitcoin fees much more predictable.
Btw there would be no reason to remove this code because as I see it this would just be a default setting that can be turned off.
Hopefully though, just like segwit, people may realize that if enough people use this, there are benefits like paying less fees and the mempool fee levels increasing in a linear way.

Right now it costs very low amout of money to spam, just a little bit, the mempools so that users naturally start to overpay to get in the next block.
People are forced to pay from 20x to 100x the amount they would have to pay.
spam is an actual economic gain for miners with the current system if done right because even if you burn some money in fees to other mining pools, you also earn much more fees from the blocks you mine, from the honest users that have to pay 50 sat/b instead of 10 sat/b.

More importantly I don't really see any big drawback because this would just be a default setting to help users not overpaying, but not being something mandatory, it should not break anything.
Worst case is that people don't really use this, but I think in the long run this will be something that will be implemented in all wallets.

The reason we don't pay like this is that transactions that occupy more space on the blockchain should pay more.

Hey! 👋
Thanks for your reply, I’m not an expert so please excuse me if I say something stupid.
1. I’m totally ok if some transactions pass even if they overpay. It would be great if this was a default option on the most used wallets so the vast majority of the users would find themselves using it. That alone would optimize things a lot imho.
2.  People using bad spv wallets upgrading to better, safer wallets is something positive imho. Spv wallets should be able to request fee levels from the node they connect to anyways
3. I think some transactions are not relayed by bitcoin nodes if they pay less than 1 sat/b. I don’t think that is censorship.
4. I actually think , but I might be wrong, that with this setting it would be much worse for spammers. Right now all they have to do is wait until there are enough transactions from users and then post their spam to force the users to pay more and more and more in a exponential increase.
With this proposal all they could do is force the users to bump their tx with rbf, but the increase would only be linear: basically 1 sat/b for every spam attack

I'd like to discuss a possible change to the bitcoin core wallet that does not impact consensus rules.
Right now bitcoin core nodes are still the vast majority and over 60% are recent versions of bitcoin core (>16.3)
Say the mempool is like this:


What we can see from this visualisation of the mempool is that there are 5.3 MB of transactions that span from 1 sat per byte to 40 satoshi per byte (we will leave extremely expensive transactions (red color) aside for a second).
That means that we could compress all those transactions into just 6 fee levels:
5 MB with fees from 1 to 5 satoshi per byte
0.3 MB with fees of 6 satoshi per byte.
Instead of 40 satoshi per byte!!

Edit:
What I want is to give the option to nodes to set a parameter of maximum allowed overpaying.
For example I would set this parameter to 1 sat/b while you may set it to 5 sat/b and someone else might leave it unlimited like it is now.
Eventually I think that having this option would lead to people using it and force wallets to improve their fee estimation to be sure everyone relays their tx
It is like a cold-war weapon. You don't have to use it, just having it ready does change stuff.
Of course people could just ignore it, but by just implementing it and having it available would be a signal for the network:
- hey be careful that if you overpay you risk to not have an optimal propagation of your transaction.
A block explorer could tag those transactions with a warning saying: "this transaction payed too many fees, this could mean it is not relayed optimally by the network and therefore could not be confirmed quickly"
Please RBF it to a lower fee to make sure it pays a reasonable amount

We can achieve this by having bitcoin full nodes not creating or including in their mempool:
- transactions that pay more than (insert here your selected option, mine is: 1) 1 extra sat/b when compared to the other transactions in the node mempool unless there are already enough tx to fill at least 1block (4000 KWU)

When including transactions in their mempool the nodes would basically start from those paying 1 sat per byte and include those, then move to those paying 2 sat per byte.
After that the nodes would check if there are enough transactions that pay 2 sat/b to fill 1 block (4000KWU). If true then it starts to include the transactions that pay 3 sat/b, if not it stop until there are.
In general I think this would make sure that the mempool grows in a gradual way, without any holes in it like those you can see in the image at 8 sat per bytes where there are very very few transactions.
One possible downside of this is if you have to create an expensive cpfp.

The power of a default setting and the incentive that, if the majority of wallets implement this “best practice”, the prices of the mining fees become rational and optimized should hopefully enough to push people to implement something like this in any wallet.
I know and understand that mining fees need to exist and I think that it is right to develop a fee market. But at the same time we should encourage the users of the wallets to not overpay for their tx to get fast confirmations because, not only do they not gain anything personally by doing that, but they also could create a spiral in the price of the mining fees that can damage Bitcoin.
In some ways it is similar to batching. No one can stop a big exchange if they want to send millions of single  transactions to each of their customers when they buy or sell. Coinbase did that for quite some time.
But eventually it should become a best practice to batch transactions to minimize the footprint on the bitcoin ecosystem.
Exactly the same for this proposal.
No one can stop you to use your own wallet that let you create a tx that pays double the fees of everyone else and get into the next block. But you could’ve been included anyways by paying just 1 more sat/b than the highest fee level. And by doing that you also don’t make it more expensive for everyone else.

hi, is there any way to connect to nicehash if i'm behind a firewall that blocks port 3333?
I guess the answer is no but you never know!

Ok cool so I have to set up an electrum server if I want to connect my phone to my own full node.
hopefully they will release electrum wallet for  iOS soon!
Thanks!

Hello,
Open source mobile app spv wallets that give me the option to connect to my own full node at home?
Thanks