As I understand a hardware wallet does "only" store private keys...
It stores the seeds itself and also signs the transaction on it.
But what happend if I am going to loose such a hardware wallet: Can the Bitcoin funds be
restored with the 12 word seed?
Your funds won't be restored. The private keys are generated from your seed. You can derive the private keys using the seeds alone. As such, you can spend the funds using your private keys.
Certain devices has inherent seed extraction vulnerability (Trezor), so if it gets lost/stolen I'll recommend you to transfer your funds to an address generated from a new seed ASAP.