Teamspeak 3 has encrypted voice and is free. I want to look more into how secure it really is though. It doesn't get you SIP, but there is an iPhone app (maybe other phones too) so you can still go mobile with it.
And tor will definitely cause latency issues. I was reading that i2p is better for faster services but the article didn't really go into why and that might have just been for resolving, not sure about latency. Maybe a different anonymous routing system could work, but the latency is kind of built into having your connection bounce around the world.