If you click exit in the GUI window of Bitcoin, it won't actually "exit", it'll just minimize to the systray.
As you've correctly observed, clicking exit from the systray, completely quits the program.
This is just a feature for allowing to easily hide Bitcoin in the systray, as to leave it operating silently in the background.
Bitcoin demands so few resources from your computer, that you might aswell leave it running in the systray whenever your machine is turned on. That way it can see new blocks and update your wallet data accordingly.