I have heard if you remove fees it will take a very very long time.
When speed is not important and my transactions are smaller than 1kb I typically send them without fee. Most of them are confirmed within a few blocks. Once I had to wait one day until a medium priority transaction was confirmed.
If you exactly know what you do it's OK to send zero fee transactions. As a normal user your wallet should automatically calculate a suitable fee.