AFAIK, Quantum computing compromises the ECDSA part of Bitcoin.
There are known quantum weaknesses for the discrete logarithm problem due to Shor's algorithm, so ECDSA is known to become weaker under a sufficient quantum attack.
There may be weaknesses discovered in SHA256 and RIPEMD160, but at the moment there are no known significant weaknesses.
Public keys are sent only when a transaction is initiated. Hence, a transaction is needed from that address to compromise the private key if Quantum computing becomes a reality.
In the earliest transactions, the outputs were pay-to-pubkey and not pay-to-pubkey-hash. The standardization on pay-to-pubkey-hash occurred later. Therefore if someone could break sufficiently break ECDSA, then those very early transactions and block rewards would be available for them to steal, since the pubkey is stored in the blockchain in those transactions.
More modern transactions (using addresses) would still be safe (as long as addresses aren't re-used).