Here's what I did, and I know I'm not the first to do this but since I can only post here I thought I'd share:
1) Put your bitcoin data folder and client on a USB key. It doesn't really need to be encrypted, but you can encrypt the filesystem of the USB key if you're really paranoid.
u:\bitcoin - put client in here
u:\bitcoin\data - put your wallet data in here
2) Use an archiver like RAR to compress your local copy of the data folder into a backup file, and set a password on the archive for some extra protection. Keep this file on your computer in case you ever lose the USB key.
3) Create a simple script (for windows) to make your bitcoin client portable. Copy and paste this into a text file:
@echo off
start /B bitcoin.exe -datadir=./data
Save the text file as "start btc.cmd" and always use that to start bitcoin, so the client uses the directory on your USB key for data. Make sure that your system is not hiding file extensions for known types or you will end up with "start btc.cmd.txt". You may need to disable hiding of file extensions within "Folder Options" in WIndows Explorer.
That's all you need to do. You can now access your wallet on any computer, although I would avoid running this on unsecured public computers.
Don't run the bitcoin client unless you need to...i.e. to check your balance or to send money.
