The signatures sign a hash of the whole transaction with blanks in place of the signatures (obviously, since the signatures aren't known before signing).
Changing a txout will change the hash and therefore require a different signature.
I knew it would be something simple and obvious, I just wasn't seeing it. Thanks.