Emergent color categorization in a neural network trained for object recognition
- Experimental Psychology, Giessen University, Germany
- Center for Vision Research, Department of Psychology, York University, Canada
Figures
Figure 1
Invariant border experiment.
(A) Six stimulus samples corresponding to the primary and secondary colors in the hue spectrum (red, green, blue, yellow, cyan and magenta, respectively). (B) Hue spectrum from HSV color space (at …
Figure 2
Border transitions in the color classifications.
(A) Summation of border transitions, calculated by counting the border transitions (as depicted in Figure 1E) for each point on the HSV hue spectrum (thin grey line). A smoothed signal (using a …
Figure 3
Left the classification of colors for 7 training bands being shifted over the hue spectrum as in Figure 1E.
Right the same analysis, but applied to a network trained to classify scenes (natural vs. artificial).
Figure 4
Human psychophysics.
(A) Example display of an iPad trial. The observer’s fingertip is placed in the central circle (white at the start of the trial) upon which it shrinks and disappears (over 150ms); subsequently it …
Figure 5
Evolutionary results.
The evolutionary algorithm is repeated 12 times and we calculate the frequency of borders in the top 10 border sets of each repetition. The resulting frequencies are plotted in blue. Border-location …
Figure 6
Multi-colored stimuli classification performance.
(A) 7 example stimuli, each sampled from a different color band. Each stimulus consists of three equally colored (target) words of which the color is determined by the selected class. Subsequently, …
Figure 7
Colored objects experiment.
(A) Samples of the google doodle dataset as colored by our simple coloring algorithm. (B) Proportion correct as a function of hue. The 14 individual plots correspond to the 14 training bands that …
Figure 8
A single set of 7 borders (indicated by vertical dashed lines; labeled B1 through B7).
Each space in between two adjacent borders represents a class. Colors for the training samples for, for example, Class 1 are randomly selected from one of the two bands, LB1 (Left Band for Class 1) …
Appendix 1—figure 1
Transition counts for five different ResNet instances.
(A) Transition count for the ResNet-18 from the Border Invariance Experiment in the main text. Transition counts are calculated by summing all transitions in the network’s color classifications …
Appendix 2—figure 1
Classification simulation.
We plot the color classification for the 3 cases from left to right (Shifting borders, data from the Invariant Border experiment and data from a categorically trained network, respectively). Each …
Appendix 3—figure 1
Luminance controlled stimuli.
(A) Stimulus examples drawn from 6 training bands aligning with the primary and secondary colors. (B) Raw transition count, that is, the number of times borders between colors are found in a …
Appendix 4—figure 1
HSV hue spectrum and RGB hue spectrum displayed relative to RGB color cube.
HSV hue spectrum at maximum brightness and saturation can be seen following the edges of the RGB color cube (subsection of RGB space in which values fall between 0 and 1). The RGB hue spectrum is …
Appendix 4—figure 2
Left: Results from rerunning the original experiment with stimuli sampled from the custom RGB spectrum.
Right: Results from rerunning the experiment with the stimuli as defined in Appendix 3, but sampling colors from the custom RGB spectrum.
Appendix 5—figure 1
Histogram of colors in ImageNet.
Colors from ImageNet have been selected to have brightness and saturation exceeding 99%. The bars are colored in seven different colors, corresponding to hues of the centers found by the k-means …
Appendix 6—figure 1
Color representations throughout the layers.
In the left column each panel shows classification of the network as 7 training bands are shifted through the hue space, as in Figure 1E of the main text. In the right column we show the cumulative …
Appendix 7—figure 1
Classification of input samples by hue, as extracted from the final layer upon which object classification is performed.
Left: The results from the original Invariant Border Experiment for reference. Center: The classification for the random hue network, where the colors in each input image are subjected to random hue …
Appendix 8—figure 1
Transition counts (left column) and classification visualization (right column) for six different CNNs: From top to bottom: ResNet-18, Alexnet, GoogLeNet, VGG-19, MobileNet V2 and DenseNet.
Transition counts are calculated by summing all transitions in the network’s color classifications (as obtained from retraining a new output layer for 4 through 9 training bands) and evaluating …
Appendix 9—figure 1
Classification plots for 4, 5, 6, 7, 8, and 9 training bands, respectively, ordered left-to-right, top-to-bottom.
Specifications of these plots can be found in the main text; Figure 1E.
Appendix 9—figure 2
Each row displays the evaluation of a separate output layer trained on a set of color bands.
These training bands are represented by darkened opaque vertical bands. Each layer is trained on 7 training bands, each falling within one of the categories found in the Invariant Border Experiment. …
