Bitcoin deposits could also be accepted and converted to USD by the poker site, but withdrawals are a much bigger issue as high volume players aren't depositing very often and recreational players are less likely to know about bitcoins and be willing to use them.
Conversion to USD is a thorny legal and ledger issue at the volumes that would be required to make this feasible. Look at the current issues with MtGox AML requests. There are hundreds or thousands of people currently locked out of their accounts, with some going on 3 weeks. MtGox is the largest exchange, and there's a lot of private chatter among bitcoin account holders that MtGox is actually playing the AML game to shore up liquidity. In other words, no Bitcoin exchange to date has established enough credibility to assure any new or existing Poker / Gambling sites that Bitcoin is a good alternative.
Obviously a poker site using exclusively bitcoins would be ideal, but there aren't anywhere near enough people willing to play on a bitcoin only site. Integrating bitcoin into an existing USD site would be a pretty big step towards mainstream acceptance of bitcoins, for gambling at least.
You aren't stating anything that hundreds, or perhaps thousands of people haven't already concluded. There's an obvious utility for Bitcoin for gambling sites, but until there's also a number of robust and trustworthy exchanges, or Bitcoin has a more direct utility for other goods and services without the need for exchange, this isn't a realistic possibility.
Would gox or other exchanges be willing to take the business of a US serving poker site? Seems that this would draw alot of attention from the US govt if bitcoin became the primary way of cashing out and/or depositing onto poker sites for americans.
They'd be happy to take anyone's bitcoin or USD. The issue is that they are even more willing to lock withdrawals and seize yours and others bitcoins and dollars for any reason whatsoever. AML requests have been going on for almost a month, and there are very few people who have actually regained access to their accounts.
Bitcoin is too young for widespread adoption. It's still an experiment.