I go back into the client that is still using that same wallet.dat with 10BTC and I send out 5BTC to a friend's wallet.
Now if I replace that wallet.dat file with the encrypted one (after its been decrypted) what will the balance that is shown in the client say?
if you restore it properly (either download the block chain again, or -rescan) the client will show either 5 btc or a lesser number -- even 0 btc is possible.
why might it show 0? it depends on whether or not at the time you made your backup that in the key pool at the time was the address that your client ended up using as a change transaction when you spent the 5 btc.
because the key pool holds 100 addresses, if the only transaction was the 5 btc spend or there were under 100 new addresses needed, then your restore will have all your bitcoins and will show the correct amount.
http://en.bitcoin.it/wiki/Address http://en.bitcoin.it/wiki/Securing_your_wallet