you can see that the reward is actually decreasing with every reward, and not only every four years.
Not really. The rate of production is known in advance by everybody who participates. So this predictable inflation in the supply of BTC can be factored in to everybody's assessment of the value of the currency. The perceived value of the "earlier" coins already takes into account the expected future production.
Moreover, BTC is fungible: the 25BTC reward you get for your block is indistinguishable (for all intents and purposes) to the 25BTC that the miner of the previous block earned. If your BTC are somehow worth less, then so are his.