Security is a non-trivial problem and as such any solution will also be non-trivial.
There are a few potential issues that jump right out at me, as error mentioned you're using Windows which is probably not idea. While you have to worry about viruses and malware on any platform, they're a bit more of a concern with Windows than elsewhere. Also, deleting the wallet doesn't really get rid of anything other than an entry telling Windows where, physically, on the disk it's stored that data - the data is still there. Your best bet is probably to use something like a TrueCrypt encrypted volume to store the wallet.dat. You can change where bitcoin stores the wallet.dat with the -datadir flag, which can either be added to the shortcut you use to launch bitcoin or added to your bitcoin.conf. This way you don't have to delete anything, merely unmount the volume when done. You could even store the entire app in such a volume though I'm not sure what additional security it might provide.
If you're the kind of person who shuts down their computer from time to time or if we're talking about a portable system like a laptop you may also consider full-disk encryption. At the risk of sounding like a TrueCrypt salesman, it does this too
