I finally got mining working on linux by following this guide (roughly), and I thought I'd share a few pitfalls I discovered, to help others to avoid the same fate.
I'm running Ubuntu 10.04.2 64-bit on a Phenom II X6, using a single 5970, Stream SDK 2.1, and poclbm.
Tip #1: Start with Ubuntu Desktop. If you start with Ubuntu Server you'll just make things harder for yourself. Running the "full stack" GUI is essential for mining to work at all. (I ended up "converting" Server using 'apt-get install ubuntu-desktop'.)
Tip #2: The drivers from ati.com (10.11 and 11.4) did not work at all for me for OpenCL using the GPUs. They would drive the display fine, but I was never able to make the GPUs available for OpenCL (OpenCL apps would show the CPU only. Using the drivers from the ppa:ubuntu-x-swat/x-updates repo solved that problem for me.
Tip #3: The whole business of setting LD_LIBRARY_PATH seems really clunky to me, and it didn't work either. Instead I added the path to /etc/ld.so.conf.d/OpenCL.conf (creating a new file), and then ran sudo ldconfig. After that everything was able to link dynamically to the Stream SDK just fine.
Tip #4: If you have limited internet bandwidth like me, there are a couple ways you can make the install easier. One is to get the Ubuntu 10.04.2 install DVD and add it to /etc/apt/sources.list. This will allow many of the needed packages to be installed from there instead of being downloaded. For a physical disc, you can use 'apt-cdrom' to access it. For the iso, you can mount it using 'mount -o loop ubuntu_dvd.iso /mnt/ubuntu-install-dvdrom', and then add a line to /etc/apt/sources.list like "deb file:/ubuntu-install-dvdrom/ lucid main restricted". Another bandwidth saver is to use squid to cache any downloaded packages and indexes. See http://itkia.com/using-squid-to-cache-apt-updates-for-debian-and-ubuntu/
Pitfall #1: If the GUI (Xorg) is not running, mining and any other OpenCL operations will fail.
Pitfall #2: If the GUI is running, BUT is not currently trying to display the GUI (i. e. you use ctrl-alt-F1 to switch to text mode display), any OpenCL programs such as CLInfo and poclbm will hang when you try to execute them.
Pitfall #3: If the GUI is running AND displaying, BUT the user is not logged into the GUI (It's sitting at the login screen), the user will not be able to use the GPUs with OpenCL apps (logging in via ssh). You must then use sudo in order for CLInfo or poclbm to be able to see the GPU(s).
Pitfall #4: If the GUI is running AND displaying AND the user is logged into a GUI session, then you will able to run OpenCL processes as the user and use the GPUS, HOWEVER you must export DISPLAY=:0 in order to see both GPUs. If you fail to do this, only one GPU will show up.
Pitfall #5: If the GUI is running AND attempting to display BUT a display is not connected to the video card, most functions will work, but you will be unable to use "aticonfig --odgt" to query the GPU temperature until you connect a display. The program will just hang instead. Using a dummy display adapter may help with this problem.
I hope this information is useful.