or a hacker will steal them from a distance, knowing your IP and taking advantage of a bug
What bug are you talking about?
At worst, a Russian mafia will appear at your house and make holes in your knees with a drill until you hand them all the bitcoin they know you have.
Yeah, I'm pretty sure the Russian Mafia will travel around the world based on IPs, miraculously locating you by that and then starting torturing you to get your bitcoins. Unless they find during their flight you have sent those coins away so they must cancel the plan and return home. Are they coming with power adaptors or do they buy their tools at the destination? Why would they think of this when they have way better sources of information already, called Facebook and Instagram. Forget IP, they can even find out the layout of your house, when your parents go to work, if you're on vacation and with what brand of dog food to bribe your dog.
-----
LE after LoyceV comment
I've removed my previous comment about the balance, and I'm just as curious as him about the server guessing it from your requests!
The client subscribes to its own addresses (nit: sha256 hashes of scriptPubKeys) so that it would be notified of new transactions touching them. It also synchronizes the existing history of its addresses. This means the client sacrifices some privacy to the server, as the server can now reasonably guess that all these addresses belong to the same entity.