How can I know that there is no malware inside my USB which copy itself automatically as a hidden file to my offline
To actually be sure about that, you need to make sure that the hardware (USB flash drive) has not been tampered with and that your online pc is not infected with a malware which spread itself this way onto your USB.
The latter is quite tricky considering that the setup should still be secure even if used with an compromised online PC.
One of the easiest way would be to not use USB flash drives, but webcams and QR codes.
Even tho a possibility exists that a sophisticated malware might exploit a potential vulnerability in the encoding of the QR reading software, that probability is close to zero. At least if you aren't the most looked for criminal in this century.
Using QR's is considered a pretty neat and secure approach to that.
If you definitely want to use USB flash drives, you might as well use a 3rd system as an intermediary which is only used to get the necessary data (unsigned transaction) from the usb device, then formats it, and stores it again on the flash drive.
If you use a virtual machine for that, you can even restore it after each time. But this is is slowly heading towards the level of paranoia at this point.
The easiest and most secure ways are these, where you can actually control the data you are transmitting. Even if your online pc is compromised.
QR codes and webcams are extremely good for this. You could even encode the transaction and simply type it into your offline device. That's secure too, but takes time and isn't very practicable.
Maybe I am paranoid but Armory has been here for quite a while, isn't it possible that this sort of malware exist?
It is possible, but quite unlikely i'd say.
At least, there aren't any known cases which do exactly this.
Malware to exfiltrate data from air-gapped devices, does exist. There is malware to exfiltrate data using the sound of hard drives, their led's, the powerline, etc...
But there hasn't been any known case someone losing cryptocurrencies from an air-gapped wallet through something like this.
You won't ever achieve 100% security. It is all about probabilities. And the probability to get your coins stolen from an air-gapped wallet with a few security measurements is relatively low.
At least as long as you don't make it a challenge and ask to get infiltrated.