I should have mentioned, anyone with access to your old device would be able to
view your account... but without the Ledger device (and PIN etc), they would not be able to
spend your coins. Essentially, your old device has a "watching only" version of the wallet. All the private keys are stored on the ledger itself, NOT on your phone.
If you want to remove all traces of the wallet from your old device, you can simply delete the cache/app data for Mycelium and then uninstall Mycelium from your old device. It won't affect your ability to link the Ledger to a new device and you won't lose your coins etc.
I would advise that you make sure that you can actually connect the Nano S to your new phone first... some devices do not support OTG