if you open the debug.log ... you can see that bitcoin core must read all the block since the creation of your wallet.
not a problem.
when i have restored my january 2014 wallet, i must wait like 2 minutes.
all android wallet does the same think (with merkle tree summary).
---
decrease cache to 4Mb in bitcoin core help a lot in poor system (2go of RAM) ... but request more reading on disc in node mode.
I know that. The problem is that it was consuming that cpu and memory forever, even after calculate the final balance.