You are right, but problem is we don't know what color they prefer for a particular condition.

It is generally believed that darker color is the choice when you fish deep and lighter color for fish shallow. However I've seen so many times this theory doesn't work.
I also saw so many times one particular species prefers one color one day and prefer different color in different situations.
The best way is to disregard color selection or choose what you like. And if one color is working, stay with the color until fish don't respond to the color.