Right. It's impossible to unambiguously track a single bitcoin (or whatever) through many transactions because there's no objective way to say which output it's in.
So, in that case how is concept of "colored coins" is going to work?
You define your own standard of matching inputs and outputs. Colored coins transactions need to be carefully constructed to conform to this standard and make such matching possible.
A transaction like you propose will uncolor the coins because it mixes inputs of several colors into the same output, and is this something to avoid.
Please see
https://bitcoil.co.il/BitcoinX.pdf for a worked example with order-based coloring.