The issue is that his server seed plus your client seed are combined using XOR into a 32-bit shuffle seed.
It doesn't matter who picks what, or whether you know his server seed or whether he knows your client seed. The issue is that there are only 4 billion possible shuffles.
If you can see five to seven cards, and you know the sequence in which those cards came off the deck, then you know which of those 4 billion shuffles it was. And therefore what all the other cards (the ones you "can't see") are too.
This is the fallout from 'Oh crap they can't do math.'
That is, either they *Really* can't do math, and you can rob them blind because you know what all the face-down cards are - or they're *pretending* they can't do math while they rob all the players blind because they know what all the face-down cards are.
In this case, they came up with a protocol that allows people to 'verify' that the shuffle was "fair" in terms of having both sides pick a seed and having both seeds used in the shuffle. But 'oh crap they can't do math' because the combined seed was only 32 bits long (4 billion possible shuffles) instead of ~250 bits (same number of possible shuffles as with a real card deck. The result is that the protocol isn't badly incorrect but because of the implementation it doesn't matter because the game is still unfair.