I would like to connect my Electrum wallet to a BitBox02 to be safer.
If I lose my BitBox in the future and the seeds from the BitBox as well, are my coins always gone on the Electrum Wallet?
Or can I just restore my Electrum wallet with the seeds and just connect a new BitBox to it?
With BitBox device you can connect their native Bitbox app, Electrum wallet, Sparrow wallet, etc. but keys (seed) are never moved from secure element chip that is in BitBox device, and they are always offline.
If you lost your BitBox device, and you lost your seed backup that is written on paper or metal (that should always be kept separated), than you can't restore your coins anymore.
Be careful experimenting with keys and stuff like multisig setup.
The coins are still on the seed of the BitBox because Electrum only serves as an interface
Correct.
Unless you leak it yourself, send it to some suspicious website, or keep it in digital form on your computer... you should never do this.