Color and orientation are jointly coded and spatially organized in primate primary visual cortex

See allHide authors and affiliations

Science  28 Jun 2019:
Vol. 364, Issue 6447, pp. 1275-1279
DOI: 10.1126/science.aaw5868

Rethinking primary visual cortex function

Understanding how color is coded in the brain is central to vision research. The presently dominant model suggests that color and orientation are separately extracted in the primate primary visual cortex. These characteristics are thought to be represented by neurons located in different cortical columns that project separately to higher visual areas for further processing. Working with macaques, Garg et al. recorded from thousands of neurons using two-photon calcium imaging with single-neuron resolution. Nearly half of sampled hue-selective neurons responded more strongly to equiluminant colored stimuli than to full-contrast achromatic stimuli. A majority of strongly color-preferring neurons were also orientation selective. Processing of orientation and color is thus combined at the earliest stages of visual processing, which challenges existing models.

Science, this issue p. 1275


Previous studies support the textbook model that shape and color are extracted by distinct neurons in primate primary visual cortex (V1). However, rigorous testing of this model requires sampling a larger stimulus space than previously possible. We used stable GCaMP6f expression and two-photon calcium imaging to probe a very large spatial and chromatic visual stimulus space and map functional microarchitecture of thousands of neurons with single-cell resolution. Notable proportions of V1 neurons strongly preferred equiluminant color over achromatic stimuli and were also orientation selective, indicating that orientation and color in V1 are mutually processed by overlapping circuits. Single neurons could precisely and unambiguously code for both color and orientation. Further analyses revealed systematic spatial relationships between color tuning, orientation selectivity, and cytochrome oxidase histology.

The primate primary visual cortex (V1) fundamentally transforms the inputs that it receives from retinal cone photoreceptors in the domains of both shape and color. Circularly symmetric inputs are combined to generate orientation selectivity (1), and cone-opponent inputs are transformed to create color selectivity (2, 3). Understanding how color is coded in the cortex remains a central question in vision research (4). A general model has been proposed in which color and orientation are separately extracted in V1 and are represented by neurons located in different cortical columns that project separately to higher visual areas for further processing (5). In this model, neurons in cytochrome oxidase (CO) blobs respond selectively to color or brightness but not orientation, whereas neurons in interblobs are selective for orientation but not the color of the stimulus. Subsequent studies have demonstrated that spatial segregation of these properties is not strictly modular (6, 7). Despite extensive investigation, previous studies have failed to identify neurons selective for both orientation and color that also respond much more strongly to their preferred color than to achromatic stimuli. Owing to the lack of definitive data to the contrary, the segregated model persists in modern neuroscience textbooks as either a dominant feature of V1’s functional organization or a major unresolved question regarding visual processing (810).

We recorded from thousands of neurons in primate V1 using two-photon calcium imaging with single-neuron resolution. The enhanced spatial resolution of this technique and the ability to record from hundreds of contiguous neurons simultaneously for many hours allowed us to reveal several notable findings regarding color and orientation processing and their spatial micro-organization in primate V1.

Methods used in previous studies of color selectivity in primate V1 include intrinsic signal imaging (ISI) (11, 12), which reveals functional organization but lacks spatial and temporal resolution, and single-cell electrophysiology, which has high temporal resolution but cannot reveal large-scale functional maps and suffers from sampling biases. Investigating color selectivity in large populations of neurons that are highly selective for several features such as orientation, ocular dominance (OD), and spatial frequency necessitates techniques that allow stable long-term recordings. We used a tetracycline-inducible expression system to express GCaMP6f in V1 of anesthetized Old World primates (see supplementary materials and methods). We first performed ISI to image the superficial blood vessel pattern (fig. S1, A to E) and locate OD columns (fig. S1, F to J). Next, we conducted two-photon calcium imaging at multiple depths within superficial cortical layers (170 to 350 μm from the pial surface). We recorded the calcium responses of 4351 visually responsive neurons from 14 imaging regions in four animals. After each experiment, we tangentially sectioned postmortem tissue from each imaging region and stained for CO to recover the location of CO blobs (fig. S1, K to O). We used the blood vessel pattern from superficial sections and the location of radial blood vessels at multiple depths to align ISI images and in vivo two-photon microscopy with CO histology (fig. S2).

During calcium imaging experiments, we presented equiluminant monocular visual stimuli of 12 specific hues selected to evenly sample CIE (Commission Internationale de l’Éclairage) color space (fig. S3A and table S1). All chromatic stimuli were presented using a calibrated cathode ray tube (CRT) monitor (see supplementary materials and methods) as equiluminant square-wave drifting gratings of a single chromaticity on an equiluminant gray background at eight directions and four spatial frequencies (Fig. 1A). We designed our colored stimuli to sample evenly in the CIE space (11, 13, 14). However, the different colors in these stimulus sets do not evenly activate the cone-opponent mechanisms that drive the lateral geniculate nucleus (LGN) inputs to V1 (15). This is most apparent when the stimuli used are plotted in a physiological color space such as the MacLeod-Boynton chromaticity diagram (fig. S3B). The red and blue colors in our stimulus set activate cone-opponent mechanisms much more strongly than the other colors do and might therefore be expected to also most strongly activate many V1 neurons. Along with colored stimuli, we also displayed full contrast (100%) achromatic sine-wave drifting gratings at the same spatial frequencies and directions.

Fig. 1 In vivo GCaMp6f two-photon calcium imaging in primate V1.

(A) Schematic of experimental setup (see supplementary materials and methods). (B) Average fluorescence of one imaging region after presentation of colored drifting gratings. Four cells are indicated and their corresponding traces are shown in (C). Scale bar: 200 μm. (C) Sample fluorescence traces, indicated by the color of the stimulus to which they responded most strongly. Colored bars indicate the hue of the stimulus displayed at each time point.

After segmentation of neurons and extraction of fluorescence traces for each neuron (movie S1 and Fig. 1, B and C), we computed the mean change in fluorescence from baseline (∆F/F) in response to three to five randomized trials of each stimulus condition. We recorded from 3164 hue-selective neurons across all imaging regions and an additional 1187 neurons that were visually responsive but not hue selective. Our imaging windows allowed us to visualize the spatial organization of neurons relative to CO histology (Fig. 2, A to H, and fig. S4). Neurons ranged from being highly orientation tuned to having no orientation selectivity (Fig. 2, C and G). We illustrate the tuning curves of five sample neurons to depict the diversity of tuning properties that we observed (Fig. 2, I to R). We discovered a large population of neurons that responded more strongly to colored stimuli than achromatic stimuli (Fig. 2, B, F, and I to L) and a variety of orientation-tuning properties (Fig. 2, N to R).

Fig. 2 Orientation-selective, hue-selective, and color-preferring neurons in primate V1.

(A) Aligned CO histology for two-photon imaging region in (B to D) (see fig. S2). Contours demonstrate normalized CO intensity. Scale bar: 200 μm. (B) The location of each visually responsive neuron is plotted. Neurons are colored according to their CPI. This figure contains two superimposed cortical imaging depths. Color scale (bottom) for CPI values. (C) Same region as (B), with the color of individual neurons plotted on the basis of each neuron’s OSI in response to its preferred stimulus. Color scale (bottom) for OSI values. (D) Neurons considered hue selective are labeled by their preferred hue, whereas neurons that were visually responsive but not hue selective are depicted in gray. Color bar (bottom) showing presented hues. (E to H) Same as (A to D), for a second imaging region. Cells 1 to 5 depicted in (I to R) are indicated with arrows. Scale bar: 200 μm. (I to M) Mean (± SEM) change in fluorescence to 12 hues at each neuron’s preferred orientation and spatial frequency. The response to achromatic gratings at each neuron’s preferred orientation and spatial frequency is plotted in black. (N to R) Average change in fluorescence to eight stimulus directions at each neuron’s preferred hue (or achromatic) and spatial frequency.

Early studies found a preponderance of unoriented, color-selective neurons in CO blobs, forming the basis of the modular model of V1 (1618). Subsequent studies (7, 19) found no correlation between responses to color and orientation but failed to assess whether neurons were unambiguously coding for color by comparing the responses to equiluminant colored versus luminance-modulated achromatic stimuli. The limited studies that have compared such responses generally concluded that the few neurons that respond more strongly to equiluminant colored stimuli (i.e., color-preferring neurons) are weakly tuned, at best, for orientation (2023). These neurons, later described as single-opponent cells (3), can code for specific colors but not orientation. Only neurons that responded strongly to achromatic stimuli (luminance cells) or approximately equally to equiluminant colored stimuli and achromatic stimuli (color-luminance cells) were described as being orientation selective. Although color-luminance cells, like retinal and LGN cells, carry information about both orientation and color, they do not unambiguously code for color; these findings are therefore consistent with the long-standing segregated model (5).

Our analyses reveal a wide range of tuning across color and orientation (Fig. 3A). To characterize whether neurons jointly and unambiguously code for orientation and color, we had to compare the responses of neurons to equiluminant colored versus achromatic stimuli. We therefore assessed the population of all visually responsive neurons on the basis of their color preference index (CPI), a measure of each neuron’s response to full-contrast achromatic versus equiluminant colored stimuli. Neurons with positive values respond more strongly to their preferred equiluminant colored stimulus than to a full-contrast achromatic stimulus, whereas those with negative values prefer achromatic stimuli. Values greater than 0.33 or less than −0.33 correspond to neurons that respond twice as strongly to colored or to achromatic stimuli, respectively.

Fig. 3 Population statistics demonstrating mutual representation of color and orientation.

(A) Relationship between CPI and OSI for all cells. Neurons above the horizontal dashed line responded at least twice as strongly to their preferred versus orthogonal orientation (OSI > 0.5). Vertical dashed lines represent CPI = −0.33 and 0.33 (neurons responded twice as strongly to achromatic or equiluminant colored stimuli, respectively). (B) Relationship between CO intensity and OSI. Trend line fit using least-squares linear regression. r, correlation coefficient. (C) Same as (C), for CO intensity versus CPI. (D) Histograms of neurons in each region of (A), based on CO intensity. Because of geometric considerations, the numbers of cells sampled are not equal in each bin. The actual sampling distributions are shown in fig. S2G.

There were nearly as many neurons that preferred color [CPI > 0; 2018 of 4351 cells (46.4%)] as there were neurons that preferred achromatic stimuli [CPI < 0; 2333 of 4351 cells (53.6%)] and also nearly as many that responded at least twice as strongly to their preferred color [CPI > 0.33; 828 of 4351 cells (19.0%)] as those that responded at least twice as strongly to achromatic stimuli [CPI < −0.33; 906 of 4351 cells (20.8%)]. Pairs of neurons that were located closer to each other tended to have more-similar CPI values (fig. S5), suggestive of micro-organization of color preference. In contrast to the trend for the orientation selectivity index (OSI) to be negatively correlated with CO staining intensity (Fig. 3B) (P < 0.001, one-sample t test), CPI was positively correlated (Fig. 3C) (P = 0.057, one-sample t test), which is consistent with previous findings that color responsiveness is greater in blobs (16). We found a significant negative correlation between the CPI and OSI of neurons (Fig. 3A) (P = 0.005, one-sample t test), in contrast with previous studies that reported no relationship between orientation and color tuning (7, 19). Despite the statistically significant trends, we did not find any evidence of strict segregation between these measures.

Are there neurons that strongly prefer color and are also strongly tuned for orientation? Neurons that respond at least twice as strongly to their preferred versus orthogonal orientation have an OSI of 0.5 or greater, indicated by the dashed horizontal line in Fig. 3A. Thus, neurons that are both strongly orientation tuned (OSI > 0.5) and strongly prefer equiluminant color over achromatic stimuli (CPI > 0.33) are found in the upper right sector of Fig. 3A. Such cells constitute 11.6% (503 of 4351) of the visually responsive sample. Not only is this a substantial proportion of the entire sample, but there are actually more neurons in this sector than there are strongly color-preferring neurons (CPI > 0.33) that have an OSI < 0.5 (Fig. 3A, bottom-right sector) [7.5% (325 of 4351 cells)], indicating that a majority of the color-preferring population is highly orientation selective. The overwhelming majority (97.2%) of neurons with a CPI > 0.33 were hue selective (805 of 828 cells), demonstrating that they responded selectively to one or more hues.

Our ability to study the responses of large, contiguous populations of V1 neurons to luminance-modulated achromatic and equiluminant colored stimuli while also recording their locations relative to CO staining yielded both expected and unexpected relationships between these parameters. In Fig. 3D, we provide histograms of the locations of neurons relative to CO intensity for each of the six sectors of Fig. 3A. Unoriented, color-preferring neurons (CPI > 0.33, OSI < 0.5) were predominantly located in regions of intense CO staining (Fig. 3D, bottom right). However, we also discovered a larger neuronal population of strongly color-preferring, orientation-tuned neurons (CPI > 0.33, OSI > 0.5) that were located in regions with significantly lower CO staining intensity (P = 0.009, two-sample t test). These trends can also be seen in the example imaging regions illustrated in Fig. 2. For example, the imaging region illustrated in Fig. 2, E to H, contains a population of neurons in a weakly CO-stained region (at the bottom right) that is orientation selective and prefers color over achromatic stimuli. In contrast, the color-preferring neurons in the more intensely CO-stained region at the middle of the imaging region were not orientation selective. Trends in the other four sectors (Fig. 3D) were largely as expected given the overall trends for lower OSI and higher CPI at more intensely stained regions (Fig. 3, B and C). The anatomical separation of oriented, color-selective neurons supports the notion that this is a distinct population of neurons that integrates color and orientation responses.

Previous studies have suggested that hue preferences are organized across the surface of V1 (11). The spatial organization of hue-selective neurons and their relationship to CO histology have not been analyzed with single-cell resolution. The vast majority of hue-selective neurons [2288 of 3164 (72.3%)] responded most strongly to red or blue hues (fig. S3C). It is crucial to reiterate (see above) that this is likely a reflection of the inherent biases in cone contrast of our stimulus set (fig. S3, A and B). Examination of the maps of the strongest color response of each neuron (e.g., Fig. 2, D and H, and fig. S3) shows that, despite the expected preponderance of red-selective and blue-selective cells, these two populations were largely separated from each other and surrounded by similarly tuned neurons. The few neurons that preferred green also appeared to be grouped together. We computed the Pearson correlation coefficient of the color-tuning curves of all pairs of color-selective neurons within the same imaging region (including multiple cortical depths for several regions). Neurons that were closer to each other (up to ~100 μm in the xy plane) tended to have more strongly correlated color tuning curves (Fig. 4A), suggesting micro-organization within color-selective regions. We imaged most regions of interest at multiple depths to demonstrate that this micro-organization is columnar (fig. S6), as the strength of this relationship is independent of neurons being recorded from the same depth.

Fig. 4 Spatial organization of hue selectivity and relationship to CO histology.

(A) Histogram of distance between simultaneously recorded cell pairs and correlation coefficient of their hue-tuning curves. Black circles indicate the average of all points in 25-μm bins and gray circles indicate the average of all points when shuffled. (B to E) Histograms of the number of green (B), blue (C), red (D), and non-hue-selective (E) neurons based on CO intensity. Dashed lines indicate the median of each distribution. (F) Mean (± SEM) OSIs for cells that prefer red, green, and blue hues, and neurons that are not hue selective but are visually responsive to achromatic stimuli (gray). *P = 0.002; ***P < 0.001. Populations of included red, green, blue, and non-hue-selective neurons are indicated with asterisks in fig. S3C.

There was a systematic relationship between CO staining intensity and the strongest color response of each neuron. The cells that responded most strongly to green hues tended to be located in the densest regions of CO staining, followed by blue- and red-selective neurons (Fig. 4, B to E). Neurons that responded most strongly to green were the least orientation-selective group of hue-selective neurons, with a mean OSI of 0.465 and just 19 of 126 (15.1%) neurons showing orientation selectivity (P < 0.01, analysis of variance). Of blue-selective and red-selective neurons, 675 of 1464 (46.1%) and 431 of 824 (52.3%) were classified as orientation selective and their mean OSIs were 0.557 and 0.617, respectively, demonstrating a highly organized relationship between hue preference, orientation selectivity, and CO histology. Of visually responsive but not hue-selective neurons, 540 of 1187 (45.5%) were classified as orientation selective, with a mean OSI of 0.648. The difference in OSI was statistically significant between green versus blue, blue versus red, and green versus red cells (P < 0.001 for all three comparisons, two-sample t test). The difference in OSI was also significant between each group (green, blue, and red) and visually responsive but not hue-selective neurons (P < 0.001, P < 0.001, and P = 0.002, respectively; two-sample t test) (Fig. 4F).

We found that shape and color are mutually and unambiguously extracted and represented in a substantial population of V1 neurons that are predominantly located in CO interblobs. Early studies proposed a strict separation of color and orientation processing (5, 24), whereas later, conflicting studies reported no relationship between these features (7, 19). We demonstrate that neither extreme is accurate. Rather, color and orientation are represented in spectrums, with populations of neurons representing either feature separately or both features simultaneously. Thus, textbook models of primate V1 must be updated. Stable expression of GCaMP6f in macaque V1 enabled us to sample a larger stimulus space than previously possible. Information in primate V1 is more sparsely coded than indicated by previous results. For example, our finding that nearly 20% of V1 cells strongly prefer a particular hue over achromatic stimuli (CPI > 0.33) indicates a more efficient representation than had been suspected on the basis of our previous observation that ~94% of cells respond reliably to achromatic stimuli (25). Although orientation-selective neurons such as those illustrated in Fig. 2 (e.g., cell 1 and cell 3) were responsive to an achromatic stimulus, they responded far more strongly to their preferred equiluminant color. Thus, they are not redundant with other neurons (e.g., cell 5) that are orientation selective and prefer achromatic stimuli. Because the color-preferring neurons were also selective for a range of hues and had varied orientation-tuning properties, they also lacked redundancy with each other.

Although most studies of color selectivity in the higher visual area V2 have focused on the CO thin stripes, the color- and orientation-selective neurons that we found in V1 interblobs are likely to connect more strongly to V2 pale stripes (16, 2629). Our findings therefore suggest that V2 pale stripes might play a specialized role in the joint processing of shape and color relative to the importance of thin stripes for color processing. Future studies should directly examine the functional properties of V1 neurons that project to different CO staining regions within V2 and should more carefully examine the visual responses and functional microarchitecture of V2 neurons in relation to CO staining using GCaMP imaging and large, multifeature stimulus sets. Conversely, the tuning properties and spatial organization of visual responses that we observed in V1 are likely to reflect the organization of local circuits and cone-opponent LGN inputs. Because our imaging methods sampled only the most superficial neurons in V1, and these represent the most processed output population that is most distant synaptically from the LGN input (30), it is also of great interest to identify intermediate stages in the transformation of shape and color that might be represented by neurons in deeper layers of V1. Systematic studies of these neurons and their relationship to the functional microarchitecture revealed by GCaMP imaging would therefore be informative.

Supplementary Materials

Materials and Methods

Supplementary Text

Figs. S1 to S8

Table S1

References (3136)

Movie S1

References and Notes

Acknowledgments: We thank T. Yamamori’s laboratory for providing AAV-TRE3-GCaMP6f and AAV-thy1s-tTa plasmids used to make the adeno-associated viruses used in this study. We thank all members of the Callaway lab; D. Ringach and A. Cheng for assistance with our two-photon microscope; B. Hansen for contributions to the project; and T. Sejnowski, J. Reynolds, G. Horwitz, and S. Chatterjee for a critical reading of the manuscript. Funding: This work was supported by NIH grants EY022577 (E.M.C.), NS105129 (E.M.C.), EY028084 (A.K.G.), the Gatsby Charitable Trust (E.M.C.), and the Pioneer Fund (P.L.). Author contributions: Design: P.L., A.K.G., and E.M.C.; experiments: P.L., A.K.G., M.S.R., and E.M.C.; analysis: A.K.G. and P.L.; writing: A.K.G. and E.M.C. Competing interests: The authors declare no competing interests. Data and materials availability: All data necessary to support the paper’s conclusions are present in the manuscript and supplementary materials.

Stay Connected to Science

Navigate This Article