The problem of why we haven't met aliens, though, is that I'm guessing is has to do with the fact that space is so huge.

Yes, because I feel that it is highly unlikely that we are the only intelligent life in a universe with billions of stars, planets, and solar systems. With that many places for life to arise, statistical improbabilities become pretty high in terms of likelihood.