When you try to restore with your seed... are you restoring to a "Standard" wallet... or a "2FA wallet"?
To restore, you need to restore to a "2FA wallet", as apparently it uses a specific type of seed:
http://docs.electrum.org/en/latest/2fa.html#restoring-from-seedImportant to take notice of the note at the bottom:
Note: The “restore” option should be used only if you no longer want to use TrustedCoin, or if there is a problem with the service. Once you have restored your wallet in this way, two of three factors are on your machine, negating the special protection of this type of wallet.
So, if you still wish to use 2FA, you will need to create a new 2FA wallet and send your coins there.