I always thought that if I transfer BTC that it could/would cost me fees to pay the miners to include my transaction in the blockchain, correct? And that those fees could go up to $25 or even higher.
That is why the exchange charges a withdrawal fee, to pay the miners fee plus administration costs.
How do I know if both exchanges accept the withdrawal&deposit of a certain coin?
You could try looking at their withdrawal and deposit pages and seeing which coins are listed.
At the moment with high fees on the BTC network, many people are choosing to exchange for another coin to transfer from exchange to exchange as it works out much cheaper.