Right, I was only thinking about the time span of one block. In the case of low-risk, low-value transactions (a coffee shop, for example, that wants to immediately accept zero-confirmation transactions): Is it fair to say that the merchant should always be listening for network transactions, and checking for double spends when accepting a payment?
It is highly unlikely that a merchant would ever see a double-spend until it's too late. The favorite double-spend example involves secretly mining a block containing the second transaction, TX #2, the double spend. The merchant sees TX #1, hands out the goodies with zero confirmations, and then the evil thief releases his block containing TX #2.
In the other double-spend case, where TX #1 races TX #2 across the network... Presumably the merchant is well connected to the P2P network, receiving and broadcasting transactions normally. Any double spend not sent in a block, but rather simply sent out to the network, would probably be rejected by all the merchant's peers. The merchant would likely never see TX #2. The moral of the story here is merchants should always be well-connected.
Zero-confirmation transactions are fraught with danger and possible mischief. We recommend against zero-conf transactions.
It has been shown that even 1-confirmation transactions are within the realm of a determined attacker, so I would never go below 2 confirmations for anything remotely valuable. But at that point... if you can wait 20 minutes, surely you can wait 60 minutes for a full 6 confirmations.
The merchant must simply ask themselves about their risk level: if the transaction is low value, then will an attack cost more than the price of the good being sold (coffee)? That is a business decision and not a technical decision. Can you afford the occasional double-spent coffee, in exchange for convenience of zero-confirmation?