I just received the Ledger Nano S and to my surprise there is no Desktop app, is the only way to access the wallet via Chrome app?
Not really. Read my answer bellow.
Do I have to install chrome just so I can use the ledger? I run mac. Also is there a iOS app so I can have the coins with me on my phone?
If you want to use the official wallet from Ledger, you will need Chrome. But you can use Ledger with other supported wallets like:
Electrum[1] for Windows.
Mycelium[2] for Android/IOS.
And AFAIK there is no standalone wallets that support Ledger for MAC.
So the private key is like the key to my coins right? And the public key is how people can send my coins etc?
Yes. Send your public key (address) to people so they can send you coins and keep your private key safe.
If I buy coins off of Kraken, how do I then send them to my wallet (Ledger or Exodus)? Will I receive a private or public key with my purchase, I really don't understand. Or is the private and public key saved within the Wallet itself? And what do people mean by importing or exporting private keys?
Request a withdraw from Kraken and put your public key (address). Your coins will be sent to your wallet. The private key is the one that you can't show to anyone or they will be able to spend your coins. Exporting/Importing keys are used to use your private key that contains all your coins in other wallet softwares.
[1]
https://electrum.org/[2]
https://itunes.apple.com/us/app/mycelium-bitcoin-wallet/id943912290?mt=8