No.
I diagree. OP is withdrawing funds to one of his addresses, which mean that he has the corresponding private key. Coins sent to the adress are usable only if you're up to date, however transactions sent to that address are simply stored in the blockchain even if the client is not up to date.
I was confused as well at first, but his no was to the second question in the OP, do I have to download the entire blockchain first.