Even with Hierarchical Deterministic wallets?
If you don't remember the path, then you don't have access to your funds.
I see.
I guess it is this way because during the initial "discovery" a wallet does with one's public keys to find the funds, the software does not care to search beyond the first tens of unused paths. Right?
If this is the case then indeed, the problem would be more difficult than I initially assumed.
If you want your vanity addresses' private keys to be stored on a hardware wallet, then yes, this is the only way to do it since hardware wallets will not allow you to import external private keys for obvious reasons.
Trezor actually allows their users to import an already existing seed.
This was implemented to securely replace one hardware wallet for another one in case of loss/ destruction of the device but having the seed available.
Anyways, thanks for your replies. I already have a better grasp of the problem and the complexity behind it.
If there is no further activity, I may lock the topic soon.