That sounds like the threads aren't properly bound to cores.
Try using cpuminer as that properly binds threads to cores.
https://github.com/jgarzik/cpuminerI strongly doubt you are breaking your CPU, but as a general rule, try to mine with a power of two number of cores (1, 2 or 4 in your case) to avoid things like that.
Matthew:out