I disagree. The purpose of this device is to protect the key from being copied and to sign requests. Providing the correct address is a concern of the recipient of the funds, if the POS terminal is infected and the store does not get its money then it has nothing to do with the card.
I think this smartcard is solving a very specific problem and that other concerns must be addressed elsewhere.
Well, I'm targeting traditional PC-s with this solution and the concern for trojans, sitting between the screen and keyboard and the key-container device is a real concern. Meaning that if Bob wants to send funds to Alice and the trojan controlling Bob's computer replaces Alice's bitcoin address with the one beling to the adversary, things go bad.
One solution would be to create a challenge-response type protocol so that Alice (not Bob) could be sure that the right address (and right amount) are sent to Bob, but that's only part of the solution.
But there are countless other ways you can be fscked and what you should be concerned of if your host is compromised, anyway.