Basically, I need the Desktop version? Darn, gotta reinstall. Any other input before I have to reinstall the desktop version?
Short answer yes.
Long answer, not really, but you need some sort of x-server running since its Xorg that loads the fglrx module. I too started with a minimal server install, spent time to figure out the xorg relation. If i had spent some more time i could perhaps get it working without full blown desktop install. Its on my todo list to have a minimal live USB build... since the ancient HDD on the mining box is showing signs of dieing.
You need to make sure that your user is auto-logged in. Its an option during installing, easiest to set it there. I did not do that initially, needed to hook in monitor on each reboot to login. Now that ive set it to autologin, i dont need a monitor at all.