Check your local high schools/middle schools for old computers that they're going to throw away. You can harvest chips and set up a basic mining rig. Won't be anything to write home about (or even bitcointalk), but it'll be something. It might even be not so bad with Alt-coins
Yeah, old school systems would be worth the trouble for bitcoins. They almost definitely wouldn't have a dedicated GPU, if any at all.
I haven't looked into alt-coins too much, but they'd probably be better if you were going this route.