Sadly, if the funds aren't coming from a wallet or an address you control, you can't control the fees either.
Gambling sites will use minimal fees for withdrawals and it will have an impact on how fast you get these funds.
Appropriate fees depends on the transaction volume.
http://bitcoinexchangerate.org/fees to get estimates on how fast your transaction would confirm depending on the fee.