Not an overkill, but not something that couldn't handle hundreds of requests.
per second? per minute?
And also I would like to know how much better is it to run Electrum server as opposed to Memepool space?
One (or maybe more) mempool.space feature require Electrum server.
This step is optional. You can run Mempool without configuring an Electrum Server for it, but address lookups will be disabled.
And just like others, i recommend choosing Fulcrum due to it's performance.
For this reason, Fulcrum emerges as a clear winner in this benchmark. Although it takes slightly longer to index than Electrs, once that indexing is complete the benefits of a scalable and performant server will greatly outweigh this once-off cost, not only in query speed and reliability but in prolonging hardware lifespan.