Currently I run a program to estimate the lowest fee rate likely to get into the next block. It connects to my local Bitcoin core bode via RPC and runs getrawmempool. It sorts the transactions by fee rate, then takes off the top 1 MB worth of transactions. The fee rate of the last transaction to get in is, in my mind at least a good estimate of the required fee to get into the next block.
I am aware that this is a fairly naive approach, and I would like to improve it. Currently, it does not account for the SegWit discount and will incorrectly decide how many transactions will fit in the block. Also, I am not sure if looking at the mempool is the best option, maybe a rolling average of the lowest fee included in the past 3 blocks would be better for instance.
If these kinds of fee estimation questions have been asked before, feel free to link me but I wasn't able to find what I was looking for.
tl;dr How can I improve my simple fee estimation to be smarter?
When you look at latest blocks, you should check low-fee transactions for CPFP - they might be included in a block because their children compensated for low fee. Also, mining pools might include in blocks withdrawals of their members with lower fees, or they might include low fee transactions because they were paid through some other channels. So, you probably shouldn't expect that the lowest fee included in a recent block is safe enough to also get included in next few blocks - I'd raise that value by 10-20% to get some safer estimation.
