But what i have found is that when Bitcoin goes down, it drags down all the Alt-Coins with it.
No, not really. The idea that bitcoin falling drags altcoins down is an unproven hypothesis, it is something that traders have noticed, but due to the various flaws in human observation, we cannot say for certain it is true. There are several factors that could lead to this effect in the short term - but these are all speculative.
The reason for this i think is because we have to buy BTC in order to buy ALTs.
Wrong again. That mostly depends where and how you buy altcoins.
Question... is why ? so i guess my real question is, is there a way to buy/sell Alt-Coins directly from Fiat ?
Yes, there is. There are plenty of options out there when it comes to buying altcoins. For example, all the coins that are on the Korean exchanges, whether they are alt or bitcoin, can be bought in fiat currency.