Cause if you only get back a generic success message, that doesn't seem that secure because attacker with access to your machine could modify your code and bypass the key.
Correct. YubiKey does
authentication, not
encryption, which is what you really need to protect your wallet.
The way YubiKey (and similar 2FA systems) work is, the server runs software that generates one-time codes from a seed, and the YubiKey also generates one-time codes from the same seed, and sends it to the server. If the codes match, the server allows you to log in. If the don't match, your login is refused. This works exactly the same as if you had logged in with a password, except that the one-time code changes every time you log in, so old codes cannot be re-used, thwarting keyloggers.
The important thing to note here is that
both the seed and the software to generate one-time codes are stored on the server, which is obviously not secure at all if an attacker is able to access files on the server (and if they
can't access your files, your wallet is safe anyway - hence the usefulness of cold storage). To protect files that an attacker could potentially access, you need encryption, and YubiKey cannot help you with that.