For those of you who are wondering how much difference the right combination of OS, SDK, mining software, and overclocking can make, learn from my tale as illustrated by this graph:
Section 1:
This is where I began setting up my rigs. One 2x6990 running Ubuntu 11.04, SDK 2.3, Diablo Miner. Clocked at 950/300. Probably throttling a bit due to overheating.
Section 2:
Rig #2 comes online. It's a 3x5970 running Ubuntu 11.04 64-bit, SDK 2.1, and Diablo Miner. Stock settings in the first section. With each break you can see where I took the system offline to try retooling things. The software stayed the same, but I experimented with different core overclocks and memory underclocks. At the end I was running 900/175 on the 2x6990 and 850/300 on the 3x5970. Overheating is a pretty big problem for a couple of the 5970s, even though I'm using PCIe extenders. The 6990s are cool and stable, though only giving me a combined output of 1Gh/s.
Section 3:
A gap where I take the whole system down again to install SDK 2.4 on the 6990 machine. Clocked at 900/300. I also switch to the phatk kernel on phoenix. The part of this section after the gap is my 2x6990 rig all by itself.
Section 4:
I put a fresh install of Ubuntu 11.04 64-bit and SDK 2.1 on my 3x5970 box and start using the poclbm kernel on phoenix. Clocked at 825/175. Everything's running cool, with a peak temperature of ~79 degrees celcius on my 3x5970.
Full disclosure:
Both rigs are caseless. The 6990 rig runs in the upper 60s to mid 70s range with 3 fans. The 5970 runs in the upper 60s to 70s range with 2 fans. All fans used are Scythe Ultra Kazes. I also have a table fan blowing directly on the 5970. Both rigs sit beneath an open window. It gets to be around 49F/9C at night where I live, which keeps things nice and cool. So far I've noticed a slight difference between my daytime hashing power and my nighttime hashing power due to the temperature changes.