i believe electrum recognizes the lower amount as your change address so it deducts the additional fee from (what it thinks) is the change.
i'm afraid if the GUI doesn't offer selecting the output to reduce (i don't have any RBF tx to check) and if what i said above is true, the only way is to manually edit the transaction and sign it again.
Is this really the case? If true, then enabling the GUI to choose which output to decrease while increasing the fees would be helpful.
The software shouldn't presume which output is the change and which output is not.