Only the people who can extract the private key without destroying the chip, which ideally is zero. You aren't really exchanging private keys, that was somewhat imprecise. You're exchanging chips that contain private keys.
Sorry, reading comprehension fail on my part.
Ok, I think I get the concept now. The chips are never spent in normal commerce, only used to transfer value or to have amounts added. What's the button press for then though?
So If I make a purchase of 8.0 BTC and have in my pocket two chips with 5.0 BTC each, I give them to the merchant. The merchant gives back to me a different chip that has exactly 2.0 BTC on it already, or the merchant takes a chip with a smaller amount on it and tops it up until there are 2.0 BTC.
So instead of a merchant just having to manage cash (ensuring there are enough coin and currency to give change and making deposits) the merchant will also need to manage a supply blank / small-value chips and redeem the ones received and not useful for change.
These chips will have a short lifespan between being blank and becoming too large to be useful.
Also, how do I as the customer remember which chip had what amount on it?
With the ability to buy a functional wi-fi enabled smartphone for $100 or less there's really little room for poker chips, mint chips and smartcard devices. A mobile app shows your balance, recent payments received and sent, etc. That's information that a poker chip or smartcard doesn't provide and with my money, yes -- whatever device that gives that information is the device I will use. There may be a number of ways this will play out (transfer value on QR code on receipt, or via NFC, etc., depending on the device's capabilities) but the platform (mobile smartphone) solves the problem (protecting a store of value) the best , to-date.