Perhaps part of the problem is the fraud protection that many/most cards have. That causes consumers to not be afraid to give out their card information willy-nilly, because if it's stolen and abused, they're not liable. On the other hand, as a card holder, I love fraud protection.
Fraud protection is great, but the system is set up to lead to fraud and a need for fraud protection.
The math has existed for a long long time to make it way way harder.