I want to make sure my full node has this arbitrary transaction data in storage permanently.
Both Taproot and P2PKH ensure your data is stored permanently on all Bitcoin full node.
Accepting that I may pay more per kB, I assume I must use P2PKH?
Could you rephrase your question? I have hard time trying to understand your question. If you wanted to ask about storing more than 80 bytes (soft limitation of OP_RETURN) of arbitrary data, Taproot (through Ordinal wallet) probably is better option.