This contradicts the entire purpose of hardware wallets.
The point is that the private keys should never leave the device.
It's not even needed to use VeraCrypt as in your example, a simple, encrypted wallet.dat would work the same if not better.
Malware can steal your private key while it's used on your machine.
Again, the point of using a hardware wallet is isolating your private key from the outside.
The hardware wallet can only sign your transactions when asked to (and would ask for your confirmation for it).
A simple USB stick is not a hardware wallet.
It's exactly the same as any of the hardware wallets with funky marketing names.
It provides storage for data, it provides software for encryption, it is usually offline and it has to be connected to a computer to get decrypted and data become available for use. No private key has to leave the device. I can not even see any reason whatfor this would be required.
Those "hardware wallets" are in my opinion overrated, you depend on their services, which may be at any time become obsolete, and you may not be able to use it with other data/wallets other than they provide for you, for a small fee perhaps,
The VeraCrypt solution covers all security considerations and is open to any wallet and way beyond that. It just needs a little bit more of effort to set it up.