I feel your pain, OP. I'm fairly new to Bitcoin, and it took me about a year to understand why including higher fees are so important. I use Blockchain for my hot wallet, and their recommended fee structure has been outdated for about half a year now. As the volume of btc transactions increase, they all compete to be included in the next block mined for confirmations. Confirmation times will be negatively affected for you if two factors are considered: 1. The amount of unconfirmed transactions are high and 2. Your fee is set lower than the current average fee recommendation.
You can view more stats about each of those factors at the following links:
1.
https://blockchain.info/unconfirmed-transactions2.
https://bitcoinfees.21.coYou'll notice a correlation between the two factors: the more unconfirmed transactions there are, the higher the recommended fees there are.
Hope this brings a little more insight as to understanding how transactions work from a technical side.
One thing that has really helped me with confirmation times via Blockchain'a wallet service is to add 50-100% to the fee that they recommend. Hope this info helps!