For mining you need a miner, about $100 on Ebay. Just pick up a cheap Antminer S3 to start and get your feet wet. Then point it to any pool and you'll make a dollar every few days. Yes, you'll go in the hole in electricity most likely, but you're just learning.
If you just want to learn the technicals of mining you need to read the mining forum's first 2 pages of threads. Start with oldest, work through to newest. You'll be an expert by the time you finish. This is for the tech minded folks.
IF you want to solo mine download "bitcoin core" from bitcoin.org, run it and let it sync with the network for 3 days or so. Make a config file then point your miner at that. You're not as likely to get coins that way quickly (it's like a lottery, think 3 months or more) but when you do you get 25+btc out of the deal.
For a fun solo mining experiment with an S3 check out my 1 year experiment here.
https://bitcointalk.org/index.php?topic=1132914.180That should get you started!