I'll let someone else explain the steps since I believe the new version added a "cancel" button and I haven't tested it. My alternative method would be complicated.
For future reference:
0. Update your Electrum to at least 4.0.4 if you haven't already because the Double Spend feature was added in that version.
1. Go to the History tab and right-click on the stuck transaction
2. Select "Cancel (Double Spend)
3. This will open the familiar "Transaction" dialog that you see when you create a transaction. In this case, adjust the fee slider at the bottom left to be at least 1.5x higher than the current fee.
4. Then follow the normal procedure for making a new transaction: Click on "Sign" and enter your password if prompted, then click "Broadcast" to replace the higher-fee transaction with the older one.