if you take this viewpoint as valid, then the self-supporting hermit is evil because it is assumed that he took the resources from society (by the act of growing up and being 'educated' and 'socialized' and presumedly eating along the way) and has chosen a life that fails to return those resources to society.
This is an interesting viewpoint, though I wouldn't say it would be evil or even vaguely wrong to do this. If the hermit is indebted to society for his upbringing then
he is essentially a slave to that society, that there should only be one type of society and that there should be no experimentation with new ways of life, no matter how unjust the society you were raised in is.
But that's just the point. Haven't you understood by now the system is designed to enslave you. Your entire life is pretty much dictated in one way or another from the moment you were born. I never asked to go to school and "rob society of it's resources". In all actuality, I was forced to go to school regardless of my desires. And if my parents didn't get me a proper education they too would face punishment. We are crafted into robots designed to give our whole life (or the better part of it) away in return for the betterment of all of society. Humans are much like ants, anyone who can't perform as expected is considered entirely useless. All they know how to do build upon their empire and work as part of a larger organism that churns away mindlessly expanding it's domain. Advancement is good and all, but it shouldn't be treated as something that "must be done or die".