Yup, the difference between inputs and outputs will go towards tx fee. So, please never ever forget to include the change.
This is important enough, that it's worth repeating.
The inputs provide value to the transaction. Add up all the value from all the inputs, and that is the total transaction value.
Then the outputs assign that value to one or more outputs.
Any remaining unassigned value is a transaction fee to the miners.
If the total of all your outputs is greater than the total value supplied by all your inputs, then you have an invalid transaction.
If the total of all your outputs is equal to the total value supplied by all your inputs, then you have a valid transaction that is not paying any fee.
If the total of all your outputs is less than the total value supplied by all your inputs, then
ALL REMAINING UNASSIGNED VALUE BECOMES TRANSACTION FEES FOR THE MINERS.