The difficulty in replacing the app on your pc with a malicious version itself is moderate.
This would technically require 1) physical access to your pc or 2) admin privileges (which is doable, especially on windows).
But this wouln't lead to a loss of your funds if you double check the addresses you type in.
On your Nano S, Blue you have to confirm your payment address on the screen of the device.
On the older versions (nano and HW..) you need to verify the payment via security card. This second layer protects you from this kind of attacks.
On my ledger nano s there's no address on the screen. Maybe ledger blue has that.
Actually the website could just show up the addresses generated by the ledger no? then you check that you are indeed sending your coins to one of your addresses and not to the hacker. I don't know why i wrote this thing about signing something. Ha yes, i know: you copy the address from the app and paste it into the website. The website tells you if it comes from your ledger (your ledger signs something and the website checks it). This is faster and the website doesn't have to show addresses that you have to check one by one.