It means that the PoW will change from being practically optimization free,
as currently the case with near-optimal ASICs, to becoming extremely optimization prone,
with huge advantages available only to the most advanced and well-funded organizations
(like your favorite 3-letter agency).
That is, mining power will go from fairly decentralized to absolutely centralized.
A post-quantum bitcoin will need to move away from Hashcash to some asymmetric PoW.
Simply increasing the key space of the hash to say, 512 bits, would provide us the same security as 256 bits does today. Hashcash would not be dead, we would just need to have larger key spaces.
I don't think that the QC mining will increase the difficulty and let the blocks stay the same size. Because QC for now are reaaaaaaally rare and that's means that only few can afford to use them and few + bitcoin node are already a problem, so I don't think it will be accepted the worsening of the situation.
The difficulty and the block size have nothing to do with each other. Do not conflate the issues.