Report

Organizing conceptual knowledge in humans with a gridlike code

See allHide authors and affiliations

Science  17 Jun 2016:
Vol. 352, Issue 6292, pp. 1464-1468
DOI: 10.1126/science.aaf0941

Coding abstract concepts in the brain

Grid cells are thought to provide the neuronal code that underlies spatial knowledge in the brain. Grid cells have mostly been studied in the context of path integration. However, recent theoretical studies have suggested that they may have a broader role in the organization of general knowledge. Constantinescu et al. investigated whether the neural representation of concepts follows a structure similar to the representation of space in the entorhinal cortex. Several brain regions, including the entorhinal cortex and the ventromedial prefrontal cortex, showed gridlike neural representation of conceptual space.

Science, this issue p. 1464

Abstract

It has been hypothesized that the brain organizes concepts into a mental map, allowing conceptual relationships to be navigated in a manner similar to that of space. Grid cells use a hexagonally symmetric code to organize spatial representations and are the likely source of a precise hexagonal symmetry in the functional magnetic resonance imaging signal. Humans navigating conceptual two-dimensional knowledge showed the same hexagonal signal in a set of brain regions markedly similar to those activated during spatial navigation. This gridlike signal is consistent across sessions acquired within an hour and more than a week apart. Our findings suggest that global relational codes may be used to organize nonspatial conceptual representations and that these codes may have a hexagonal gridlike pattern when conceptual knowledge is laid out in two continuous dimensions.

Humans have a remarkable capacity for generalizing experiences to novel situations (1, 2). It has been hypothesized that this capacity relies on a “cognitive map,” allowing conceptual relationships to be navigated in a manner similar to that of space (36). Grid cells use a hexagonally symmetric code to organize spatial representations (7). Here we ask whether conceptual knowledge may also be organized by gridlike codes.

Human grid cells have been identified during intraoperative recordings (8) and are the likely source of a precise sixfold (hexagonal) symmetry in the functional magnetic resonance imaging (fMRI) signal, as a function of movement direction during virtual navigation (911). This hexagonal signal varies depending on whether the direction of moving in space is aligned or misaligned with the orientation of the grid (Fig. 1, D and E). Such a signal is a precise and unusual prediction for fMRI (supplementary text): It is predicted in the bulk activity because grid cells share a common grid axis (1214), and conjunctive grid cells fire faster on average when movement is aligned to this axis (9). It pertains to the moving direction and cannot be explained by any characteristics of the currently experienced visual scene. It is hexagonal and therefore does not align on average to the cardinal directions. Its temporal waveform is different in every scan and every subject (fig. S4) and thus cannot be easily predicted by imaging artefacts (supplementary text and table S1).

Fig. 1 Experimental design for navigation in abstract space.

(A) Subjects were trained to associate stimuli (birds) with outcomes (Christmas symbols). (B) Example trajectory in abstract space. A location in this abstract space was represented by a bird stimulus. A trajectory was equivalent to visually morphing one bird into another (Fig. 1C). The direction θ of the trajectory depended on the ratio of the rates of change of the legs and the neck (movie S1). Subjects were not consciously aware that these associations could be organized in a continuous “bird space.” (C) Example trial corresponding to the trajectory with direction θ. (D) Trajectories can be categorized as aligned (red sectors) or misaligned (gray sectors) with the mean orientation ϕ of the hexagonal grid. Note that ϕφ is different for each participant [see (26) for details on how ϕ was calculated]. Here, the direction θ is aligned with the grid. (E) fMRI markers of grid cells showing hexagonal symmetry: The signal is bigger for trajectories aligned versus those misaligned with the grid. (F) (Top) Color-coded trajectory maps illustrating time spent in each part of the environment during the “explore” task in the first (quantile1) and last parts of training (quantile5). Yellow is maximum and dark blue is 0. (Bottom) Barplots showing the amount of time spent at the locations/stimuli paired with outcomes in each epoch relative to the total time spent navigating (“time at outcomes” quantile1 versus quantile5, t22 = –3.17, **P < 0.01). (G) In the “collect” task, participants made significant improvements in training day 2 compared to training day 1: The percentage of trials with an angle error < 15° (t33 = 2.37, *P < 0.05) and with only one transition increased (t33 = 2.55, *P < 0.05). In the “recall” task, participants made significantly more correct responses in day 2 compared to day 1 (t41 = 3.89, ***P < 0.001). (H) Example data from the most commonly used schedule: even distribution of trajectory angles across all trials (light gray), outcome trials (medium gray) and nonoutcome trials (dark gray). (I) Example data from the most commonly used schedule: We tested if the sin(6θ) and cos(6θ) regressors correlated with multiple confounding factors. These regressors did not correlate with the start neck, start legs, end neck, and end legs lengths, whether the subject responded accurately or whether the morph passed through an outcome (all coefficients of determination R2 averaged across all subjects < 0.02). [(D) and (E) are adapted by permission from Nature (9)]

This gridlike signal is not unique to the entorhinal cortex, but can be measured during spatial navigation in prescribed parts of the medial frontal, medial parietal, and lateral temporal cortices (9). Despite no report in rodents of grid cells outside the hippocampal formation, direct recordings during brain surgery in humans have confirmed gridlike firing patterns in some of these areas (8). This same network of brain regions, often referred to as the “default mode network” (15), is also regularly activated in nonspatial tasks that involve the manipulation of conceptual knowledge, such as memory (16), imagination (17), scene construction (18), valuation (19), and theory of mind (20), and in situations when subjects must generalize learned concepts to novel situations (1, 2, 21).

The ability to interact with knowledge in this flexible and generalizable way is the central advantage of maintaining an explicit cognitive map (3). Together with the regions’ established role in nonspatial conceptual generalization, the finding of gridlike activity in these brain regions during spatial navigation therefore raises the possibility of common neural coding mechanisms for storing spatial and conceptual representations. Indeed, this hypothesis is strengthened by the findings that hippocampal cells (analogous to rodent place cells) encode individual concepts in humans (22) and sound frequency in rodents (23), and that rodent grid cell coding may not be restricted to spatial dimensions, but also represent time (24).

We used fMRI to test if humans use a hexagonally symmetric code when navigating through abstract conceptual representations. We designed a task analogous to the one used for navigation in physical space (9), with the notable difference that our dimensions were organized in an abstract, rather than physical, space.

Twenty-eight healthy subjects performed a stimulus-outcome (S-O) learning task in which they learned that bird stimuli (25) were associated with different Christmas symbols (Fig. 1A). The study therefore resembles other S-O learning tasks except that here, the bird stimuli were not independent fractals or symbols but, instead, they varied according to two continuous dimensions: the lengths of the neck and legs. Each stimulus could therefore be described within a two-dimensional conceptual “bird space” (Fig. 1B). Even though the features of this bird space were lengths as in physical maps, here they had to be extracted from a one-dimensional (vertical) visual space where different stimuli did not form any angles (Fig. 1A) and transformed into a two-dimensional conceptual map (Fig. 1B).

Participants had extensive experience of the bird space the day before scanning (26). Briefly, we first trained them to morph birds with specific neck:legs ratios, using a nonspatial controller (fig. S1). Next, the participants learned which bird stimuli were paired with outcomes by freely morphing the neck and legs dimensions. The outcome symbols would appear on screen whenever the morph matched the associated bird (fig. S2). This ensured that subjects became familiar with the entire bird space and not just with the S-O pairings. Indeed, participants progressively refined the locations of the outcomes through training (Fig. 1F and fig. S6). We periodically tested their knowledge of the bird space by asking them to find specific outcomes from arbitrary start positions, by correctly choosing the appropriate neck:legs ratio. When asked to make such ballistic movements, subjects significantly increased their precision through training, and therefore learned to visualize the target bird (Fig. 1G and fig. S7). In the scanner, participants continued to improve performance each day, possibly because they received fresh training before each day’s scanning (Fig. 1G and fig. S8).

Each subject participated in two to four separate fMRI sessions, spanning two separate days, at least 1 week apart. In each trial during scanning, subjects watched a video of a bird morphing according to a predefined neck:legs ratio (Fig. 1C and movie S1). They were then instructed to imagine the outcome if the bird continued to morph with the same neck:legs ratio. In some trials, they had to choose one of three offered outcomes: two outcomes they were trained with and a “no outcome” option (black square). Participants reached a performance of 72.8 ± 1.0% accuracy in predicting outcomes, and 0 out of 28 reported conceiving of the relationships between birds or outcomes as lying in a spatial map (26).

To test whether the fMRI signal had hexagonal symmetry, as a proxy for grid cells, we ensured that the orientation of the trajectories for movement in bird space was dissociated from the properties of the visual scene. That is, trajectories with the same orientation were formed by different stimuli, and trajectories with different orientations could pass through the same stimulus. Indeed, the critical hexagonal symmetry regressors described below did not share more than 5% variance with any tested basic visual property of the stimuli, outcomes, or behavioral accuracy in any subject (Fig. 1I). The trajectories were sampled evenly across directions both in sum and when separated according to outcome (Fig. 1H and fig. S5).

Despite the absence of any hexagonal modulation of these confounding factors, we found a hexagonal modulation effect in the neural activity. We first identified hexagonally symmetric signals across the whole brain and then we focused on those regions where the effect was strongest. This approach allowed us to test in an unbiased way if this hexagonal symmetry had a consistent grid angle across two sessions acquired on the same day and more than a week apart.

To identify brain regions sensitive to hexagonal symmetry, we used a Z-transformed F-statistic to test for a significant modulation of the fMRI signal by any linear combination of sin(6θ) and cos(6θ), where θ is the trajectory angle in bird space (26) (fig. S3). We found hexagonal symmetry in a network of brain regions that overlapped anatomically with the network found during navigation in physical space (8, 9) and with the default-mode network (15) (Fig. 2A and fig. S10). However, although these brain regions all survived whole-brain cluster correction, this quadrature test could overestimate the Z scores (26). Thus, we used this test not for statistical inference per se, but rather to create orthogonal regions of interest that allowed us to test in an unbiased way if the grid angle was consistent across separate experimental sessions. This was possible because the quadrature test was independent from the phase of the periodic signal, that is, the grid angle. We focused on brain regions where grid cells have been recorded in humans during spatial tasks (8): that is, the anterior cingulate/medial prefrontal cortex (mPFC) and entorhinal cortex (ERH).

Fig. 2 Identifying hexagonally symmetric signals across the whole brain.

(A) Hexagonal modulation in a network of brain regions including the medial prefrontal cortex, with a peak in its ventral region (vmPFC; peak Montreal Neurological Institute coordinates –8/42/0, peak Z score = 4.09), the medial entorhinal (ERH; –18/0/–38; Z = 4.41), the orbitofrontal (OFC; 6/44/–10; Z = 4.27), the posterior cingulate (PCC; 0/–32/28; Z = 4.3), retrosplenial (RSC; 6/–52/24; Z = 4.73), and lateral parietal cortices (LPC; 30/–62/28; Z = 4.96) and the temporoparietal junction (TPJ; 52/–42/40; Z = 4.13). For visualization purposes, the maps are cluster corrected at a cluster threshold Z = 3.1 and P < 0.05 for all brain regions apart from the ERH, where we used a more lenient threshold of Z = 2.3 and P < 0.05. (B) Subjects who performed better at the task had significantly more hexagonal signal modulation in the vmPFC (correlation coefficient r = 0.432, P = 0.039).

Using the peak coordinate of the hexagonal modulation signal in the mPFC located ventrally (vmPFC), we found that subjects with greater hexagonal modulation had a more accurate performance at the task (Fig. 2B). This region has also been shown to correlate with the performance in memory and conceptual knowledge tests (1, 21). As previously described for spatial hexagonal symmetries (9, 10), we next asked whether the grid angle to which this hexagonal modulation was aligned was consistent between separate experimental sessions. We thus used the data from one session to estimate the grid angle for a given participant, using the beta coefficients for the sin(6θ) and cos(6θ) regressors (26) (fig. S3). We then took the data from a separate session and looked for differences in activation between trials in which the trajectories were aligned versus misaligned to this hexagonal grid. This was achieved with the regressor cos(6[θ(t) – ϕ]), where θ(t) is the trajectory orientation in trial t and ϕ is the mean grid orientation across the region of interest (9, 10). This “cross-validation” procedure was counterbalanced across sessions. We then performed a one-sample t test across the group on the resulting regression coefficients.

Using this approach, we tested for hexagonal consistency between separate sessions acquired half an hour apart. We found such an effect at the whole-brain level in the vmPFC (Fig. 3A, left). To test if this effect was a function of the presence of outcomes, we included in the design matrix a confound regressor that modeled out the effect of outcomes. We found that the map did not change (fig. S11). Again, the consistency effect correlated with behavior in subjects who performed better at the task (Fig. 3A, right). Thus, we replicated the finding that participants with more gridlike representations performed better at the task, using two independent analyses. The same neural signal was not predicted by the speed of learning of the task during training (26) (Fig. 1, F and G). Gridlike activity was therefore related to the current performance rather than the trajectory of learning.

Fig. 3 Grid angle consistency between separate sessions acquired within the same day.

(A) Left: Whole-brain level grid angle consistency in vmPFC (cluster corrected Z = 2.3 and P < 0.05; 16/54/–2; Z = 3.76, P < 0.0001). Right: participants with higher hexagonal consistency performed more accurately on the task (r = 0.431, P = 0.039). (B and C) Left panels: Sixfold modulation signals aligned to the same grid angle in the vmPFC (t26 = 2.61, *P < 0.05) and ERH (t27 = 2.36, *P < 0.05). The effect is plotted separately for all aligned (red) and misaligned (gray) trajectories. Right panels: This effect was specific for sixfold, but not any other control periodicities between four- and eightfold (all P > 0.15).

Next, to examine the pattern underlying this hexagonal effect, we separated all aligned (red) and misaligned (gray) trajectories (26) (Fig. 1D). The signal in the vmPFC was significantly higher for aligned than misaligned trajectories (Fig. 3B). The same pattern appeared in ERH (Fig. 3C). This effect was significant only for sixfold but not control four-, five-, seven- and eightfold symmetries (Fig. 3, B and C, right panels).

We also tested for hexagonal consistency between separate sessions acquired more than a week apart. We found a significant effect in the vmPFC (Fig. 4A). When grouping together within- and between-day data, the consistency effect was strongest in the vmPFC (Fig. 4B). Moreover, we found hexagonal consistency also between the ERH and vmPFC, suggesting that different brain regions may contain gridlike activity that is aligned to the same angle (Fig. 4C). Again, all these effects were significant only in hexagonal but not control symmetries (Fig. 4, right panels).

Fig. 4 Grid angle consistency between separate sessions acquired more than a week apart.

(A) Cross-day consistency of the grid angle in vmPFC (left panel t20 = 3.65, **P < 0.01; right panel all P > 0.18 for control periodicities). (B) Within- and cross-day consistency in vmPFC (left panel t20 = 3.41, **P < 0.01; right panel all control P > 0.15). (C) Cross-region consistency between the ERH and vmPFC (left panel t21 = 2.18, *P < 0.05; right panel all control P > 0.46).

Although the coarse nature of the fMRI signal urges caution in making conclusions at the level of neuronal codes, we have reported an unusually precise hexagonal modulation of the fMRI signal during nonspatial cognition. When subjects perform this abstract cognitive task, this signal exists in a set of brain regions markedly similar to those observed when subjects run in a virtual reality spatial environment (9) (fig. S9), despite profound differences in the cognitive and perceptual demands of the two tasks. The hexagonal grid is consistently oriented across sessions that are acquired both half an hour and more than a week apart. Together, this evidence suggests that gridlike codes that are known to underlie spatial navigation, and recently discovered in the temporal dimension (24), can also be used to organize abstract knowledge of the type that is difficult to study in nonhuman species. In the event that such conceptual grid cells can be recorded directly, it will be of interest to know whether they share relationships (such as relative phases) across spatial and conceptual tasks, suggesting that conceptual tasks can be solved by subconsciously mapping abstract dimensions onto preexisting spatial maps, or whether new organizations can emerge to represent conceptual problems. It will also be informative to study how such cells behave in conceptual problems that are not easily mapped onto continuous two-dimensional spaces (27, 28).

Supplementary Materials

www.sciencemag.org/content/352/6292/1464/suppl/DC1

Materials and Methods

Supplementary text

Figs. S1 to S11

Table S1

Movie S1

References (2935)

References and Notes

  1. Materials and methods are available as supplementary materials on Science Online.
  2. Acknowledgments: We thank N. Burgess, Z. Kurth-Nelson, H. Barron, L. Hunt, M. Garvert, and T. Muller for helpful discussions and comments on this manuscript, as well as the anonymous peer reviewers who greatly improved the manuscript. We thank S. Jbabdi for help with data collection and analysis, as well as A. Winkler and L. Griffanti for guidance. We also thank all participants for volunteering in this study. The study was supported by a Wellcome Trust 4-year Ph.D. studentship (099715/Z/12/Z) to A.O.C., a Medical Research Council Career Development Award (MR/L019639/1) to J.X.O.R., and a Wellcome Trust Senior Research Fellowship (WT104765MA) together with a James S. McDonnell Foundation Award (JSMF220020372) to T.E.J.B. The authors declare no conflicts of interest. All raw data are archived at the Oxford Centre for Functional MRI of the Brain.
View Abstract

Navigate This Article