First thing I thought was about Color Blind people (1 in 12 men are color blind, or about 8%) that simply don't see some colors, but even people with normal eyesight can interpret same color differently.
Seeing the colors is irrelevant. Even someone with perfect eyesight cannot pick out the difference between
this color or
this color or
this color or
this color, and yet all four have different hex codes and therefore would generate different seed phrases. The hex codes
must be backed up, which is more difficult, more time consuming, and more error prone than just backing up a seed phrase in the first place.
Remember seed phrase with words is more easily and more memorable than remember color codes.
You shouldn't be remembering anything, hex codes or seed phrases. Relying only on your memory is risky and unsafe. Just write down your seed phrase on paper and store it somewhere safe, like every good wallet tells you to do.