yes, it needs to be re-encrypted to remove access to particular key. If he don't have previous copy of file.
Maybe encrypting file with symmetric master key and then encrypting the master key with asymmetric key can save the decrypt-encrypt process of all data.
How about using multiple public keys? If I have multiple public keys for each person then I could just re-encrypt with the one person removed and they won't be able to see the data. The only drawback then would be that the data has to be decrypted twice to get it now? Would that work?