Or do these need to be reinstalled on the new phone?
Reinstalled. Which means going to each site then disabling OTP (which requires the OTP from the old device) and enabling 2FA using the new device.
Some people store the secret for each OTP service in something like LastPass or in a document protected with GPG encyryption, for instance. That way if the device is lost or reset to factory (e.g. before it gets sold) a new device can be prepped without having to go into each site to disable then re-enable the OTP.