So when it wasn't you that found the block, you pay a 0.5% fee to the block finder? That will just increase your variance, not give you higher payouts overall.
Not quite. The 0.5% is "taken off the top", not donated by other miners. The first 0.5% is rewarded to the node/miner that found the block, then the rest of the 99.5% is evenly distributed amongst the rest of the miners.
So if you didn't find the block, you pay some of your income to the one who did.
Psychologically it may feel great to have a block bonus, but what it means is that you earn less all the rest of the time (when you didn't find the block). Your average earnings are the same, but the variance is increased.
It's a bit more like a kitty or community lottery than a pool fee, though. The person getting it doesn't do it for providing the pool, they get it to discourage block withholding. While you are literally earning 0.5% less if you don't find the block, and it does increase your ROI if you're "unlucky" and never find a block, it doesn't go to anyone in particular for managing nodes or p2pool authors or anyone like that, only fellow miners. You don't have to like that, but as you say, it's still pretty cheap for what it is.