It is possible to define per template how variants should be grouped in listings.
Very similar to group by in SQL, if Color is defined in your template to group by then all items with the same color will be represented by the first item of that color only.
You can also configure how this grouping is done, see more info in Group variants on category page