Yes, that's all you'd need, but, here's the thing that catches most people because they don't realize how Bitcoins work, first of all, a tl;dr, "When you spend any amount of money, the money will probably no longer be in the same address".
Longer version:-
It's impossible to spend less than the amount you received in a previous transaction, but, obviously, the majority of the time you won't want to spend the exact amount of a TXIN (or multiple ones combined), so, what do you do? You send the amount you want to spend to the receiver, then, you send the remainder ('change') back to yourself,
normally, on a different address, meaning the private key for the last address is now worthless.
Personally, I recommend Armory (Or any other client that uses a pre-saved seed to generate keys), as, it fights this by using a seed which you print off to generate all addresses, this means even if you receive money on an address after paying, or, send change to a new address, that printed wallet two-line long string still saves all your funds.
EDIT:- I'd just like to point out I've never used Electrum, so, if Electrum also used a pre-saved seed, just back that up (And any other required info) and you're good. This is what the Armory .pdf (you print) looks like:-
