You can sign, but because of the technology used you can't encrypt. In the current 0.6+ release of the reference bitcoin client, go to the "Receive Coins" tab, select an address, and click "Sign Message".
To verify, the recipient will need the exact message, the signature, and the bitcoin address that you signed from. Verification can currently only be done with bitcoind, the commandline version of the client.
Wow. You guys thought of everything.
Thanks, I may need this some time.