It probably depends what else you're using.
I use pyopencl which needs to be recompiled against the SDK when it's changed, but other than that I only change the path to the SDK.
Ah, I think you're on to something,
I tried to run configure.py on pyopencl-0.92 with ./configure.py --cl-inc-dir=/opt/AMD-APP-SDK-v2.4-lnx64/include/ --cl-lib-dir=/opt/AMD-APP-SDK-v2.4-lnx64/lib/x86_64
but it complained to me about already being configured, I assume from the previous version and I really don't know what to do... any suggestions for a noob?