There is no big difference.
Standalone = You only need the .exe file, so you can open the wallet on any flash drive.
Installer = Will install the software on your C:\Program Files (x86) folder.
Both of them opens the wallet in %appdata%/Electrum by default. Even though you can open Electrum using just the executable, you will still need the wallet file along with you. The difference is just one installs itself while one doesn't.
Right. But you can easily move your wallet file to your flash drive plus the standalone executable: Open your wallet wherever you want!
In case OP don't know where is his wallet file by default when creating a new one: %AppData%\Electrum\wallets