If the price is going up, it helps miners keep being motivated to crunch. Difficulty to mine is rising each day that passes, so it makes sense to have them rewarded with higher prices. In this sense, the price really matters.
Well if the price goes down some miners will just turn off their rigs and difficulty will go down instead so it doesn't really matter for mining as a whole. Of course the ones with the lowest profit margins will be hit when that happens though.