They got my email from BTC-e. The exchanged was compromised some years ago and they got hold of user emails.
It's interesting that I posted this in the yahoo help forums and my post got deleted. I suspect it's a Yahoo Mail bug because the hacker was able to add an alternate email to my account without requiring my password. It may be a bug with how the account security recovery code is sent when you are signed in to mobile devices that have not been updated for long.
Oh gosh, this case is very alarming. I guess Yahoo! Should look into this very well, or they compromise the security of accounts they are handling. It's a bit alarming in a sense that hackers can do this without an actual password.
Do you think hackers can penetrate bitcoin accounts too and transfer hard earned bitcoins to another account? Curious on this though. Thanks for your reply.