If the Nano S doesn't let you do that, you may have to look at getting or deriving the private keys from the 3 addresses used for inputs on the transaction.
The nano s is just a hardware device which lets you sign transactions by pushing buttons.
If
something doesn't let you do that, its because of the wallet client used (in most cases: ledgers chrome app).
I don't know any other way to cancel, I read that after about 2-3 weeks it disappears on its own, but it doesn't seem to be the case.
The reference value is 14 days. But every node decides how long he keeps transactions in the mempool.
If a different node (or maybe your wallet) is rebroadcasting your transaction it might never get dropped from the network.
Is there any way I can make it disappear on my own?
Unfortunately, no. You can't make other nodes forget about something.
There are 2 scenarios where transactions 'disappear':
1) When a conflicting transaction has been included into a block, the TX will be dropped
2) 'all nodes' (or better: the nodes which are 'important' to you) drop your TX for any reason (e.g. not confirmed 14+ days)
The easier way to solve your problem would be to get your TX through, instead of trying to abandon it.
I would suggest you to do a child-pays-for-parent. This works as long you control at least one output of your transaction (most likely your change from the TX).
You can use your nano s with electrum (
https://electrum.org/#download). Electrum lets you choose specific 'coins' to send from.
From your addresses tab you need to choose the change address from your unconfirmed TX and right click -> 'spend from'.
Afterwards send your coins to an address of yourself and include a fee which is high enoug to cover
both of your transactions fee.
To get an estimation you can visit
https://estimatefee.com/.
This way you don't need to import your seed phrase into another wallet (as mentioned earlier by others).
With 'just connecting' your nano s to electrum your seed still stays safe and you will have to confirm the TX on your device.
Afterwards you can uninstall electrum (if you wish) and you don't have to consider your seed as compromised because it never left your device.