To illustrate this clearly with arbitrary numbers. Assume 1BTC is worth a bit less than $1000 for the sake of simplicity.
I am a miner - I would, disregarding risk, be willing to spend around $25000 to mine a 25BTC block.
For a 'tx fee' block, let's say it contains 0.25BTC - I would therefore be willing to spend around $250. I think this should be apparent.
This is true, but there will be many more participants willing to compete for these $250 blocks, which means transaction fees and centralisation decrease. Also, while the big miner is busy 'wasting' cycles on this $250 block, another big miner could be claiming the $25000 block from under his feet.