Last week I tried to transfer money using Electrum.
Since the status didn't change to confirmed in a day I checked the info and as it turned out I used one of my old addresses that isn't in use anymore.
However now the status has changed to confirmed.
How's that possible? How can I get my money back?
If the transaction is confirmed then there is nothing you can do. Bitcoin transactions are irreversible. It is like digital cash.
Also if you still have access to the wallet with the old address then you can get access to your bitcoins. Some users have the misconception that addresses expire over time but that is not the case.