If a serious issue like this was ever found in the Bitcoin Core client again, would the offending txs be removed again?
I would assume so.
if so, then why doesn't this happen when bugs are found in other clients?
It would.
If a full node client were to accept something into the blockchain that the protocol doesn't allow, then those that implement that client program would have a choice. They could either roll back the the last block that the protocol allows and then re-sync with the rest of the network, or they could fork their own new blockchain and convince enough users that their bitcoin is the real bitcoin and that any other client that doesn't implement the protocol the way they did is an invalid fork.
Does Bitcoin Core have some kind of privilege in this sense?
Right now, bitcoin core is considered by most to be "the reference implementation". In other words, the exact specifications of the protocol are defined as the being "the protocol as implemented by Bitcoin Core". If someone else was to create their own full node implementation, and was to convince a significant majority of users that their implementation should be considered to be the "reference implementation", then Bitcoin Core would lose that privilege.