I would remove the "withdraw" option from the client (still available if they both agree on it), and add a timer-based delivery.
Such that if the client dies, and can't "clear" the transfer, the escrow won't keep the money indefinitely while the seller is screwed. The client should also have the option to extend the timer manually.
Also in my opinion withdraw should be removed. Moreover, the expiration should be fixed to a specific block number. In case of the agreement will not reached, the deposit fund will be added to the block reward (or it would be splited among N blocks, in order to achieve more constant rewards).
So, the payment will be performed in 2 steps:
- payment-id = escrow(to-address, expiration-block, amount);