Do the server logs say anything useful?
I failed debugging 101 somewhere. The error log did indeed give me useful information.
What happened:
The bitcoind client when run acts as an RPC client and connects to the already running instance of bitcoind to get its data(I did not know that). Also, the default apache configuration uses / as its home directory not the same directory the bitcoind client uses to store the bitcoin.conf file. When bitcoind was run with a command switch from the CGI it was looking in the home directory of the user it was run from(the webserver) for bitcoin.conf and not from the home directory of the user bitcoind was running as.
Fixes:
1) Have bitcoind and the webserver run as the same user so they share the same configuration directory
2) Create a link between the bitcoin.conf from the home directories of the users(webserver and bitcoind user)
3) Create a duplicate configuration file for both users.
Thanks for the help everyone!