I had this problem earlier selling 5BTC.
Actually I found the problem.
If I sell for 0.00000001 less then the amount of BTC in my account it works.
If I add a sell order for all of my BTC it duplicates it with one saying "active" and the other saying "insufficient funds", but neither actually works.
This is probably some rounding error at MtGox.
Yes I had this issue as well. I found it worked if i added a sell order for 1 less btc than my total then put the last 1btc separately