I buy a krugerrand for $1600. I have a piece of yellowish shiny metal.
Market price of krugerrands falls to $1400. I do nothing; I still have a piece of yellowish shiny metal.
I buy one BTC for $40. I have one BTC.
Market price of BTC falls to $20 per BTC. I do nothing; I still have one BTC.
Market price in terms of fiat bankster paper does matter to a point, but it is not the be all and end all. At last year's Bitcoin conference, investment guru Charles Vollum reminded us in the audience not to be heavily focused on fiat price, but the whole picture.
While i agree that the large picture is important, we just cannot simply remove fiat as the defacto reference for bitcoins value.
This cannot change unless bitcoin becomes bigger than the fiats it is referenced against.
And in any case, you cannot completely divorce these money systems when they act in our global economy.
The only situation you could consider bitcoin as not relating to other value representations is to have the bitcoin economy have nothing to do with them.