but contributing to a pool allows for a 51% attack
Not really. And to the extent that it does, it will be fixed in the future.
Also, you pay a fee to be a part of the pool.
Most pools take no fee or a fee negligible with respect to the benefits they provide.
Is it for variance reduction (i.e. smaller more frequent payments)?
Yes. And to a smaller extent because it's more convenient and gives monitoring features. Also some things like merged mining and long polling require extra work if you mine solo. (So in fact you'll earn more in a pool than with plain solo)
If that's the case, why has the entire community not switched over to the p2pool?
p2pool uses difficult shares and is thus not suitable for small miners. It's great for large miners and proxy pools though.
I don't get it. I'm not a miner though.
If you try mining solo you'll get it. I'm serious, you need to experience losing sleep wondering if you've found a block.