Yes, an attacker would either need your seed or your password if said attacker were to somehow get ahold of your electrum file. This attacker would however have access to your xpubkey which includes all your addresses, past and present. With your xpubkey, and one of your private keys, an attacker could potentially calculate all your private keys.
It is also possible, even probable for an attacker to attempt to brute force your password if they gained access to your wallet file, so I would recommend that you move any Bitcoin held in your wallet if you discover that your wallet file might have been compromised.
Just wanted to say that even if the hacker gets hold of your electrum files its very unlikely for him to brute force your password if you have a strong one like this:
*((Xtream2000thousanddates11!))* with a password like that it takes more than 546 years with brute force attack to find it. Still I agree with the idea to move funds to another wallet which is safer but electrum for me is the most safe of wallets.