That's just Windows being Windows... %AppData% = Application Data = AppData\Roaming
You won't get any errors if you remove the files from the "AppData" wallets folder(s)... Although, you'll probably find they're actually the same, just Windows has a few different ways of displaying them
If you remove all the wallet files, you'll probably find that Electrum will just prompt you to create a new wallet when you start it up the first time... if you then click the "choose" button and select the file from the new location, the next time you start Electrum, it *might* remember where you opened it from... if it doesn't, it'll just prompt you to create a new wallet, but you can just click "choose"...
NOTE: I've never tested this functionality, I always just use the default location.