That's right, one of the addresses.
And it's probably just the number of transactions. How to solve such a problem?
Electrum servers have a default limit for the history of transactions; most public servers have the default settings, hence you may have not much luck there.
Some possible options would be:
1. Import the private key (safely) of only that one address into another wallet and try your luck - this may work because it will ask for history for only one address and not more.
2. Import the private key(s) into Bitcoin Core - but syncing means time and hardware (and a lot of disk space).
3. Find somebody who hosts an Electrum server that can be accessed from the internet (i.e. not for local use only) and ask him nicely to increase the limits until you do what you must, then move to a new wallet.