The answer depends upon your level of paranoia. Personally, if I ever had my SEED from a hardware wallet online I would re-initialize my hardware wallet, which creates a new wallet. Then send my coins to the new hardware wallet. NEVER place the hardware wallet SEED anywhere on ANY computer - not ever. The advantage of a hardware wallet is that NO computer will ever see the SEED at any time, not even an offline computer. I understand what you did perfectly. When I started using my Trezors I too wanted to be certain that I could do a full restore without a Trezor present. I was using Electrum and restored everything without any hitches. The recovery is simple in fact, for anyone reading along here. After confirming the process for myself on a small insignificant wallet amount, I then re-initialized my Trezors offline. Never again will any SEED for one of my Trezors ever be placed on any computer. Unless of course I need a restore without a Trezor. This is my opinion. Why would you use a hardware wallet with coins on it that has had the SEED on even a computer where you suspect its totally clean. To me it defeats the one super big advantage of NO SEED EVER on a computer. My .02!
I agree with the above. A hardware wallet's benefit is that the seed only remains on the hardware device and not on any PCs. By restoring the seed into Electrum, this is no longer true. Your best bet would be to create a new seed on the hardware wallet and send the coins to the new one to ensure the seed only remains on the hardware device.