Based on your initial problem description I suspect a SW bug in the miner for solo mining. The protocol
sems to be correct if you're getting blocks, submitting hash and getting a reply.
The reject reason will provide more info. Invalid shares are usually user error (wrong algo) or a SW bug in the
hash code. There are other types of rejects that could point in another direction.
You could try another version of cgminer. I've never used it myself so I'm not familiar with it's issues and quirks.
But it's best to get more info before drawing any conclusions.
Interesting..
I am using CGminer 4.10 on windows.
I am not sure how to compile it on Linux, but I have a server that I could test it on instead.
And I dont know how to compile previous versions of CGminer on Windows.