You're thinking too hard. Consider it like Subway sandwich stamps. Is there a high propensity to fraud or delays with those? They're not cash.
Actually, yes. We don't have the Subway stamps anymore, at least in the United States. Care to guess why? (the URL will tell you anyway)http://www.msnbc.msn.com/id/8075754/ns/business-retail/t/fraud-stamps-out-subway-sandwich-promo/
Generally, protection against fraud in this country when using cash isn't an implementation issue, it's a "going to send people with guns to your door" issue. With credit cards, the protection is subsidized by people who aren't paying their balances on time. With checks, there really isn't much protection at all. You've got to see their ID, and the retribution by a company consists of "going to put a drawing of you next to the register". Check fraud is so bad that most restaurants simply won't accept them.
Bitcoin essentially functions like a check that you have to buy hardware to accept. Confirmation time is not instantaneous, so there's no guarantee a modified handheld payment device hasn't shifted money around immediately before the purchase. You don't even check their (fake) ID! While the confirmation time is less than a physical check, it's still greater than the time it takes to saunter to your car and drive off.
Asking a restaurant to accept Bitcoins is like asking them to start accepting checks again.