Does this mean that any installation of ElectrumX also has to have Bitcoin Core on the same machine in order to make an Electrum server? I want to be able to give it my full node's IP address.
a lot of nodes including Electrum nodes are written on top of bitcoin core. basically core is the heart of them responsible for network communication, downloading and verifying blocks and transactions and building the blockchain, and the application on top (ElectrumX in this case) is adding additional features on top (in this case indexing that blockchain so that it could be easily looked up based on addresses).