Report

Microcircuits for Hierarchical Elaboration of Object Coding Across Primate Temporal Areas

See allHide authors and affiliations

Science  12 Jul 2013:
Vol. 341, Issue 6142, pp. 191-195
DOI: 10.1126/science.1236927

Hierarchy and Representation

Neuronal representations of objects are elaborated through the hierarchy of occipitotemporal cortical areas, and the recognition of a feature as “novel” is commonly thought to emerge and become prevalent at a cortical area because of local signal processing. Hirabayashi et al. (p. 191) tested another possibility—that a feature representation becomes prevalent in a given area because a microcircuit creates a small number of precursor representations in a prior area in the cortical hierarchy, and the representations then become prevalent through proliferation in the subsequent area. In support of this notion, critical microcircuits for object association were observed using multiple single-unit recordings in two areas of the macaque temporal cortex.

Abstract

In primates, neuronal representations of objects are processed hierarchically in occipitotemporal cortices. A “novel” feature of objects is thought to emerge and become prevalent at a cortical area because of processing in this area. We tested the possibility that a feature representation prevalent in a given area emerges in the microcircuit of a hierarchically prior area as a small number of prototypes and then becomes prevalent in the subsequent area. We recorded multiple single units in each of hierarchically sequential areas TE and 36 of macaque temporal cortex and found the predicted convergent microcircuit for object-object association in area TE. Associative codes were then built up over time in the microcircuit of area 36. These results suggest a computational principle underlying sequentially elaborated object representations.

Neuronal representations of objects are processed hierarchically in the primate occipitotemporal lobe (1, 2). Representations of a “novel” feature that constitutes a given object are thought to emerge and become prevalent in one of the areas in the cortical hierarchy (3, 4). Although this sudden emergence and prevalence of novel feature representations have been established for the transformation from a lateral geniculate nucleus to V1 in the visual processing pathway (5, 6), it is not known whether the same principle holds for the corticocortical hierarchy (7, 8) (see supplementary text for the background of the present study).

Previous studies have demonstrated that single neurons representing associations between two object images (pair-coding neurons) (911) become prevalent in area 36 of the perirhinal cortex, whereas the pair-coding neurons only constitute a small minority in area TE (10, 11), a hierarchically prior cortical area (Fig. 1A) (12, 13). It is not known whether the activities of the small number of neurons in area TE merely reflect a possible variability of response selectivity or whether they emerge as a result of specific computations in a convergent microcircuit. Therefore, we looked for microcircuits that generated pair-coding neurons in areas TE and 36 by simultaneously recording from multiple single neurons in each of these areas (Fig. 1, B and C) while monkeys performed a pair-association memory task (fig. S1) (911, 14, 15). Conventional cross-correlation analyses were conducted to detect functional connectivity between recorded neurons (1420) (see supplementary text for methodological issues on the functional connectivity). Figure 1, D to K, shows two example pairs of cells recorded in area TE. In both of these cell pairs, unit 1 exhibited a selective response to the optimal stimulus but did not respond to its paired associate (Fig. 1, F and J, top). Similar to unit 1, a selective response to the optimal stimulus was also observed in unit 2 (Fig. 1, F and J, bottom). Unlike unit 1, however, unit 2 showed a substantial response to the paired associate, too. We then calculated cross-correlograms by using spikes elicited by the optimal stimulus to estimate the functional connectivity between units 1 and 2. We observed a prominent displaced peak on the right side of the shift-predictor-subtracted cross-correlogram (SSCC) (14, 15, 17, 18, 20), indicating the directed functional coupling from unit 1 to 2 in both cell pairs (Fig. 1, G and K).

Fig. 1 Microcircuit analysis of object coding and example cell pairs.

(A) (Left) Schematic diagram depicting hierarchical elaboration of object association across primate temporal areas. (Right) Stimulus set for monkey 1. (B) Lateral (top) and coronal (bottom) views of a monkey brain. Scale bar indicates 10 mm; dashed line, anteroposterior level of the coronal view; rs, rhinal sulcus; amts, anterior middle temporal sulcus; sts, superior temporal sulcus. (C) Simultaneous recordings of multiple single units. (D to K) Two example cell pairs and their functional connectivity. (D and H) Waveforms and autocorrelograms. Horizontal scale bars, 0.5 ms; vertical, 50 μV. (E and I) Stimulus selectivity of responses for each unit. Each vertex corresponds to each stimulus, and the vertices facing each other depict a pair of stimuli. Top and bottom vertices, optimal cue of the cell pair and its paired associate, respectively. Gray, baseline firing rate. Maximum, 29 and 23 spikes/s (100 to 600 ms from cue onset) for units 1 and 2 in (E) and 10 and 11 spikes/s for units 1 and 2 in (I). (F and J) Peristimulus time histograms (PSTHs) for the optimal stimulus and its paired associate. (G and K) SSCCs between units 1 and 2 for the optimal stimulus. Horizontal gray lines, confidence limit (P < 0.05, corrected for multiple comparisons).

In total, 70 pairs of regular-spiking putative excitatory pyramidal neurons (see materials and methods in the supplementary materials) were recorded from area TE of two monkeys, in which both of the constituent neurons in each pair exhibited stimulus-selective responses and at least one of the constituent neurons responded to both the optimal stimulus and its paired associate. The functional connectivity of each cell pair was then examined by calculating the SSCC for the responses to the optimal stimulus. A significant displaced peak on the SSCC was detected for 29 cell pairs based on the peak position and an asymmetry index (1417, 20, 21), and the source and target units were determined for each pair on the basis of the coupling direction (fig. S2 and supplementary text). Source units represent leading neurons, and target units represent lagging neurons, in terms of relative spike timings on the SSCC (14, 22). As a population, both of the constituent neurons of these pairs showed prominent responses to their optimal stimuli [P < 0.001, versus baseline; one-way analysis of variance (ANOVA), followed by the least significant difference (LSD) test] (Fig. 2, A and B, fig. S3, and supplementary text). In contrast, responses to the paired associate of the optimal stimulus were observed only in the target units (P < 0.001) but not in the source units (P > 0.08), and the responses were significantly different between these units (P < 0.02, two-way ANOVA followed by LSD) (Fig. 2, A and C). These results support the view of functional convergence onto pair-coding neurons in area TE (Fig. 2G).

Fig. 2 Associative object representations through functional convergence in area TE.

(A) Population PSTHs of constituent neurons of pairs with displaced SSCC peak [n = 29 for (A) to (F)] for the optimal stimulus and its paired associate. Thick and thin lines, mean and mean ± SEM. Horizontal black bars, cue period. A ±25-ms window was slid in 10-ms steps. (B) Population responses of constituent neurons of pairs with displaced SSCC peak for the optimal stimulus and its paired associate during the cue period. Error bars in all figures represent SEM. (C) Population responses for the paired associate, normalized to those for the optimal stimulus. (D) Proportions of cell pairs showing directed functional connectivity toward the neurons with stronger (left) and weaker (right) associative coding. (E) Population coupling strength in directions toward the neurons with stronger (left) and weaker (right) associative coding. (F) Population averages (left) and individual data (right) of pair-coding indices. ++, comparison with 0. (G) Schematic diagram depicting the observed functional convergence toward associative object representations in area TE. Note that the schema reflected asymmetric representation of object association in a TE neuron.

The observed functional convergence was further supported by four different analyses. First, the majority of SSCCs with a displaced peak exhibited directed connectivity toward the neurons with stronger associative coding (22 of 29 cell pairs, 76%, P < 0.006, χ2 test) (Fig. 2D), and the coupling strength was greater in the direction toward the neurons with stronger associative coding (P < 0.03, paired t test) (Fig. 2E, figs. S4 to S6, and supplementary text). Directed couplings toward neurons with stronger associative coding were observed irrespective of whether target units responded more strongly to the optimal stimulus or its paired associate (13 and 9 of 22 pairs showing directed couplings toward neurons with stronger associative coding; P > 0.3, χ2 test). Second, in addition to analyzing neuronal responses to the optimal pair of stimuli, we further evaluated the strength of associative coding by calculating a pair-coding index (PCI) (911), the response correlation for all the learned pairs of stimuli. The resultant PCI values were greater for the target units than for the source units (P < 0.02, paired t test) (Fig. 2F), and only the target units showed significant associative coding (P < 0.006). A third one on the Granger causality (figs. S9 to S11) and a fourth one on the connectivity dynamics (fig. S12) are detailed in the supplementary text.

We next examined the functional connectivity between neurons in area 36, which hierarchically locates at the next stage of area TE. In area 36, we recorded from 86 pairs of regular-spiking neurons in which both the constituent neurons of each pair exhibited stimulus-selective responses, and at least one of the constituent neurons responded to both the optimal stimulus and its paired associate. Of these, 31 pairs of neurons exhibited a significant displaced peak on the SSCC in response to the optimal stimulus (fig. S13 and supplementary text). Both of the constituent neurons of these pairs showed prominent responses to their optimal stimuli (P < 0.001, one-way ANOVA followed by LSD) (Fig. 3A, fig. S14, and supplementary text) but not to their worst stimuli. In contrast to area TE, the paired associate of the optimal stimulus elicited responses from both of the constituent neurons in area 36 (P < 0.02 for source units, P < 0.004 for target units, two-way ANOVA followed by LSD) (Fig. 3B and fig. S14), without significant difference to each other (P > 0.8). SSCCs of area 36 neurons showed comparable probability of functional connectivity in directions toward neurons with stronger and weaker associative coding (55 and 45%, P > 0.5, χ2 test) (Fig. 3C), and the coupling strength was indistinguishable between these directions (P > 0.7, paired t test) (Fig. 3D) (see figs. S15 to S17 and supplementary text for further analyses). PCI analysis also revealed that both of the constituent neurons exhibited significant associative coding (P < 0.01 for source units, P < 0.001 for target units) (Fig. 3E) without difference to each other (P > 0.6). Applying a more stringent criterion for displaced SSCC peak replicated the above statistical results (fig. S18).

Fig. 3 Microcircuit for associative object representations in area 36.

(A) Population responses of constituent neurons of pairs with displaced SSCC peak [n = 31 for (A) to (E)] for the optimal stimulus and its paired associate during the cue period. (B) Population responses for the paired associate normalized to those for the optimal stimulus. (C) Proportions of cell pairs showing directed functional connectivity toward the neurons with stronger (left) and weaker (right) associative coding. (D) Population coupling strength toward the neurons with stronger (left) and weaker (right) associative coding. (E) Population averages (left) and individual data (right) of pair-coding indices. ++ and +, comparison with 0. (F to I) Dynamics of pair-coding indices in the constituent neurons of pairs with displaced SSCC peak (n = 28). (F) Population dynamics of pair-coding indices in the constituent neurons, aligned at the time of half-max value in the source units. Thick and thin lines, mean and mean ± SEM. A ±100-ms window was slid in 20-ms steps. (G) Individual data and (H) population average of the change in the pair-coding index during 300 ms after the half maximum in the source units. (I) Comparisons of the dynamics of the pair-coding index between the constituent neurons.

Because both the constituent neurons of a given pair exhibited associative representation in area 36, we next compared the dynamics of associative coding between these neurons. The dynamics of PCI values in the target units were aligned at the time when the PCI values of the corresponding source units reached the half maximum. In contrast to the rapid PCI increase in the source units, PCI values of the corresponding target units slowly increased after the peak in the source units (Fig. 3F). The response latency by itself was indistinguishable between these neurons (P > 0.1, paired t test). Whereas the PCI values of the source units decreased during the 300 ms after the half maximum (P < 0.001, paired t test) (Fig. 3G, left), PCI values of the corresponding target units increased during the same period (P < 0.02) (Fig. 3G, right). As a result, the source and target units exhibited distinct PCI dynamics from each other (P < 0.001) (Fig. 3H). During this period, the target units in a significant majority of cell pairs exhibited a larger increase in the pair-coding index as compared with that of the source units (82%, 23 of 28, P < 0.001, χ2 test) (Fig. 3I) (see supplementary text for discussion about the microcircuit operation in area 36 for object association).

The areal differences in the connectivity results were statistically supported by two two-way ANOVAs, each with a different dependent variable: Significant interactions were found both between factors of area (TE versus 36) and unit (source versus target) for the normalized response to the paired associate of the optimal stimulus (P < 0.02) and between factors of area and coupling direction (toward the neuron with stronger versus weaker associative coding) for the coupling strength (P < 0.04) (supplementary text). It should be noted that the observed areal differences in the connectivity results are not due to the relative predominance of pair-coding neurons in area 36 (supplementary text).

In Fig. 4, response properties of target units in area TE were directly compared with those of area 36 neurons to characterize their hierarchical relationships.

Fig. 4 Direct comparisons of response properties between target units in area TE and area 36 neurons.

(A) Cumulative distributions of response latencies of target units in area TE (n = 29) and area 36 neurons (n = 62). Arrows, median values. (B) Population average latencies with which TE target units (n = 29) and area 36 neurons (n = 62) represented the optimal pair of stimuli (i.e., responded to both stimuli). (C) Cumulative distributions of pair-coding indices for the source and target units in area TE (n = 29) and those for area 36 neurons (n = 62). Arrows, median values. (D) Schematic diagram depicting the results summary of the present study.

First, the response latencies of the target units in area TE were significantly shorter than those of area 36 neurons (P < 0.008, Kolmogorov-Smirnov test) (Fig. 4A and figs. S20 to S21) (11, 23). Furthermore, the target units in area TE represented the optimal pair of stimuli with shorter latencies than area 36 neurons (P < 0.03, paired t test) (Fig. 4B), consistent with feedforward transfer of the associative representations from area TE to 36 (Fig. 4D).

Second, the PCI values also supported the hierarchical relations. As shown in previous studies, the PCI values were significantly greater in area 36 than in area TE, and significant associative representations were observed only in area 36 but not in area TE as a population (fig. S22) (10, 11). However, PCI values of the target units, but not the source units, in area TE were indistinguishable from those of area 36 neurons (P < 0.008 for TE source versus TE target, TE source versus area 36, Kolmogorov-Smirnov test; P > 0.7 for TE target versus area 36) (Fig. 4C), further supporting the notion that the prototype of associative representations can be generated in area TE as a result of microcircuit processing and then transferred to area 36, where the representations become prevalent (Fig. 4D).

Previous studies have attempted to uncover the functional hierarchy of cortical processing by showing the areal differences in the response properties of single units and presumed that novel feature representations emerge and become prevalent within a single cortical area (10, 11) (see supplementary text for the background of the present study). In the present study, we instead demonstrated the areal differences in the computational principles by detecting the functional microcircuits from different cortical areas and suggest that the prevalence of elaborated object representations in a cortical area can be attained by emergence of the representations through the specific microcircuit computations in a prior area and subsequent building up over time in the microcircuit of the next area (see supplementary text for perspectives and limitations of the present microcircuit analyses). Because the present results were obtained with learned object associations, it remains elusive whether the present scheme can be applied for representations of more general object features (supplementary text).

The microcircuits for associative object representations found in the present study might act in parallel with other channels, including feedforward functional convergence from area TE (10, 11), top-down feedback from the prefrontal and/or medial temporal cortices (2427), thalamic contributions (28, 29), and across-laminar interactions (14, 22) (supplementary text). Elucidating the entire view of across-areal elaboration of object representations will be an important issue for future studies to manifest the computational principles of the hierarchical brain.

Supplementary Materials

www.sciencemag.org/cgi/content/full/341/6142/191/DC1

Materials and Methods

Supplementary Text

Figs. S1 to S23

References (3087)

References and Notes

  1. Acknowledgments: We thank G. Rangarajan for kindly providing the script to conduct the nonparametric Granger causality analysis, H. Kasahara and T. Watanabe for experimental assistance, and M. Takeda and K. W. Koyano for helpful discussions. This work was supported in part by Ministry of Education, Culture, Sports, Science, and Technology (MEXT)/JSPS KAKENHI grant nos. 19002010 and 24220008 to Y.M.; by CREST, Japan Science and Technology Agency to Y.M.; by a grant from Takeda Science Foundation to Y.M.; and a Grant-in-Aid for Young Scientists (B) from MEXT to T.H. (18700378).
View Abstract

Stay Connected to Science

Navigate This Article