Report

Greater Neural Pattern Similarity Across Repetitions Is Associated with Better Memory

See allHide authors and affiliations

Science  01 Oct 2010:
Vol. 330, Issue 6000, pp. 97-101
DOI: 10.1126/science.1193125

Abstract

Repeated study improves memory, but the underlying neural mechanisms of this improvement are not well understood. Using functional magnetic resonance imaging and representational similarity analysis of brain activity, we found that, compared with forgotten items, subsequently remembered faces and words showed greater similarity in neural activation across multiple study in many brain regions, including (but not limited to) the regions whose mean activities were correlated with subsequent memory. This result addresses a longstanding debate in the study of memory by showing that successful episodic memory encoding occurs when the same neural representations are more precisely reactivated across study episodes, rather than when patterns of activation are more variable across time.

Repeated study of the same materials can significantly strengthen memory representations and make them more resistant to forgetting (1), but not all repetitions are equal. One fundamental issue is how multiple study episodes add up to improve later memory. A widely accepted theory, often referred to as the encoding variability hypothesis (25), proposes that each study episode is encoded differently as a result of contextual drift with time, and that greater encoding variability leads to better memory. Alternatively, it has been claimed that each subsequent study episode serves as a retrieval cue to reactivate and strengthen the memory representation of the information stored during earlier study episodes (6). Evidence for this reactivation view comes from the finding in an AB-AC paradigm in which the presence of A during AC study reinstated AB and therefore also improved memory for B (7), an effect related to activity in the posterior medial temporal lobe (8). However, previous work has not yet established a link between the nature of neural representations during encoding and later memory.

In this study, we used functional magnetic resonance imaging (fMRI) and representational similarity analysis (9) to examine how the similarity in patterns of neural activity across multiple study presentations is related to subsequent memory. The encoding variability hypothesis predicts that better subsequent memory should be associated with greater dissimilarity between activity patterns across study presentations. To the contrary, we found, across three studies, that better subsequent recognition and recall is associated with greater similarity between neural activity patterns across repetitions, consistent with the hypothesis that practice improves memory by retrieving and strengthening a consistent representation.

In the first experiment, 24 subjects were scanned while memorizing 120 novel faces (Fig. 1A) (10). Each face was presented four times, with an interrepetition interval (ITI) ranging from 1 (i.e., consecutive) to 20 faces. One hour after the scan, subjects were given a recognition memory test, during which a total of 240 faces (half learned, half new) were randomly mixed together. For each stimulus, the subjects had to decide whether or not it had been presented before by responding on a 6-point confidence scale, from 1 (definitely new) to 6 (definitely old). Out of the 120 old faces, subjects on average recognized with high confidence (e.g., a 5 or 6 rating) 51.7 ± 18.6 items and forgot (a 1 or 2 rating) 37.3 ± 16.9 items (table S1). Using a subsequent memory paradigm (11, 12), we compared encoding-related brain activity for subsequently recognized faces with that for subsequently forgotten faces across four repetitions. Consistent with previous literature (1315), this comparison identified stronger activation for subsequently remembered faces than for subsequently forgotten faces in the left fusiform gyrus (Montreal Neurological Institute, MNI: –46, –60, –8, Z = 3.88) and right fusiform gyrus (MNI, 44, –60, –10, Z = 3.58), extending into the inferior temporal gyrus and lateral occipital cortex (fig. S1).

Fig. 1

(A) Experimental design of Experiment 1 and (B) schema of the cross-repetition pattern analysis. A total of 120 novel faces were studied over three scanning runs. (A) Each face was repeated four times. They were categorized post hoc, as remembered faces and forgotten faces, according to performance on the recognition memory test administered after a 1-hour delay. Each presentation of the remembered faces (R1 to R4) and forgotten faces (F1 to F4) was separately modeled. (B) Pattern analysis was based on independent structural ROIs (top) (10). Activation pattern in a given ROI was extracted for each presentation (middle) and then subjected to Pearson correlation analysis. The encoding variability hypothesis predicts that the degree of pattern similarity for subsequently remembered faces is lower than that for subsequently forgotten faces (bottom).

We then tested the core hypothesis that pattern similarity in this region is associated with subsequent memory. To do this, we reestimated the model with unsmoothed data.

We then extracted the signal for each individual voxel within anatomically defined regions of interest (ROIs) and used representational similarity analysis (9) to examine the degree of similarity in the fMRI activation patterns between repetitions (averaged over stimuli), using the Pearson correlation coefficient as the similarity metric. Because pattern similarity can be affected by the number of trials in each condition (fig. S2), our analyses were based on a model that matched the number of trials included in the regressors for remembered and forgotten items, as well as their repetition lags.

We focused our analyses on 20 independent anatomically defined regions in the dorsal and ventral visual stream, frontoparietal cortex, and middle and medial temporal cortex (table S3), all of which have been previously shown to be important for visual object perception and memory encoding. Nine of these regions showed significantly higher degrees of pattern similarity for subsequently remembered faces than subsequently forgotten faces (P < 0.05, two regions remained significant by Bonferroni correction), whereas no regions showed the opposite effect (Figs. 2 and 3, fig. S3, and table S4). Only four regions—the left inferior frontal gyrus, right inferior frontal gyrus, right fusiform gyrus, and right parahippocampal gyrus (LIFG, RIFG, RFUS, and RPHG, respectively)—showed stronger overall activation for subsequently remembered faces than subsequently forgotten faces (table S4). To ensure that the signficantly higher pattern similarity was not caused by differences in mean activity, we reran this analysis in the bilateral ventral visual stream—the lateral occipital lobe, fusiform gyrus, and inferior temporal gyrus (LOC, FUS, and ITG, respectively)—after removing voxels showing significant subsequent memory effects in mean activity under a liberal threshold (P < 0.05, uncorrected). Even after removing these mean-responsive voxels, there was a significantly higher degree of pattern similarity for remembered versus forgotten faces (F1,23 = 6.16, P = 0.02) (Fig. 3 and table S4).

Fig. 2

Neural pattern similarity in a sample region. (A) The location of the right dorsal lateral occipital cortex (RdLOC), which was anatomically defined according to the Harvard-Oxford probabilistic map, and overlaid onto the group-averaged anatomical map. (B) Neural pattern similarity from a single subject’s single-run data. Pattern similarity was calculated by computing the correlation between the parametric estimates (beta) for each voxel within the ROI across the two repetitions. The line reflects unit slope. (C) Neural pattern similarity averaged across all subjects (n = 24), separately for each pair of a repetition combination. (D) The mean neural pattern similarity as a function of subsequent memory. A repeated-measures analysis of variance (ANOVA) was used to examine the differences between conditions. Error bars represent within-subject error. REM, remembered; FORG, forgotten.

Fig. 3

Neural pattern similarity is associated with face memory. Greater pattern similarity for subsequently remembered faces than for subsequently forgotten faces was found in (A) the right inferior parietal lobule (RIPL), (B) the right ventral lateral occipital cortex (RvLOC), which were anatomically defined, and (C) the bilateral ventral visual cortex, which includes the bilateral fusiform gyrus, bilateral inferior temporal gyrus and bilateral ventral lateral occipital cortex, but excludes voxels showing significant subsequent memory effects in activation levels defined by a liberal threshold (P < 0.05, uncorrected). All ROIs were overlaid onto the group-averaged anatomic map. The bar graphs show the group-averaged (n = 24) mean correlation of all pairs of repetitions as a function of subsequent memory. Error bars represent within-subject error. R, remembered; F, forgotten; See table S4 and fig. S3 for detailed statistics and results for other regions.

The results from our first experiment suggest that the degree of pattern similarity between successive study episodes is associated with subsequent memory performance in a recognition test. Because free recall is more sensitive to contextual associations than recognition is, the encoding variability hypothesis thus predicts that subsequently recalled items might be associated with more divergent contexts than items that are not subsequently recalled (1618). To provide a further and more direct test of the encoding variability hypothesis, we conducted a second experiment to examine whether greater pattern similarity is also associated with better free-recall performance. Subjects (n = 22) were asked to perform a semantic (concrete versus abstract) judgment task on familiar words during the scan (10). Each word was repeated three times, with a repetition lag ranging from 1 to 18 trials. After the study session, participants were asked to return 6 hours later to perform two memory tests. In the first test, subjects were asked to recall the words they had studied in the scanner. They were then asked to perform a recognition test similar to that described in Experiment 1. Out of the 180 items, 44.7 ± 21.7 items were categorized as Recalled (items correctly recalled on the free-recall test), 81.2 ± 25.5 items as Recognized [items recognized with high confidence (score of 5 or 6) but not recalled], and 54.1± 27.4 items as Forgotten (items that were neither recalled nor recognized) (table S1).

The differences in behavioral performance and mean activation during the semantic task among recalled, recognized, and forgotten items are depicted in table S2 and fig. S4. From the findings from Experiment 1, we constructed a new model using equal numbers of trials from the recalled, recognized, and forgotten conditions and calculated the pattern similarity in the same 20 regions. We found that out of the 20 regions, 15 regions showed a higher level of pattern similarity across repetitions (again averaging over items) for subsequently recalled items than for subsequently recognized or forgotten items (one region remained significant by Bonferroni correction); no region showed the opposite effect (fig. S5 and table S5). Only four regions showed stronger mean activation for subsequent recalled words than for subsequently forgotten words (table S5). Again, after removing voxels showing subsequent memory effects for mean activation (P < 0.05, uncorrected), the remaining voxels in the left dorsal visual stream [dorsal lateral occipital lobe (dLOC) and inferior parietal lobe (IPL)] also showed greater pattern similarity across repetitions for subsequently recalled than subsequently recognized or forgotten words (F1,21 = 4.64, P = 0.015) (fig. S5 and table S5).

In both Experiments 1 and 2, the use of rapid event-related designs did not allow for reliable estimates of item-specific blood oxygen level–dependent (BOLD) activation patterns, and pattern similarity was calculated from the aggregated BOLD activation pattern across stimuli in each condition. There are thus two possible explanations that are consistent with these results. First, it is possible that the results reflect overlap in item-level encoding processes. Alternatively, it is possible that they reflect process-level overlap, such that better memory occurs when the same general processes (e.g., perceptual, attentional, or semantic processes) are engaged across repetitions. In order to more directly test the hypothesis of item-specific pattern overlap, we performed a third experiment using a slow event-related fMRI design (12 s for each trial), which enabled us to extract BOLD signal patterns associated with each single trial. In this experiment, 22 young adults were asked to perform a semantic (living versus nonliving) judgment task on 60 familiar words during the scan (10). To prevent further encoding of each item during the repetition lag, subjects performed a highly engaging self-paced visual orientation judgment task for 8 s after each semantic judgment task (lasting for 3 s), and the next trial started after a 1-s delay (fig. S6). Each item was repeated three times, with a repetition lag ranging from four to nine trials. Thirty minutes after the study session, participants were asked to freely recall the words they had studied in the scanner. Out of the 60 items, subjects, on average, recalled 13.5 ± 4.7 items (table S1).

Subjects’ response time on the semantic judgment task decreased across repetitions (F2,42 = 42.96, P < 0.001); accuracy was high (mean = 97.5% ± 2%) and did not change across repetitions (F2,42 = 1.68, P = 0.19). Accuracy (F1,21 = 1.79, P = 0.19) and response times (F1,21 = 0.15, P = 0.70) did not differ between subsequently recalled items and forgotten items (table S2). There were no significant interactions between repetition and subsequent memory for either accuracy (F2,42 = 0.14, P = 0.087) or response time (F2,42 = 1.69, P = 0.20). Functional imaging data revealed that, similarly to Experiment 2, there were significantly stronger activations for subsequently recalled items than for subsequently forgotten items in the left middle and inferior frontal gyrus (LMFG/LIFG) (MNI: –50, 14, 34; Z = 4.17), and the left dorsal lateral occipital lobe (LdLOC) and adjacent inferior parietal lobule (LIPL) (MNI: –40, –66, 46; Z = 3.48) (fig. S7).

We then examined whether the degree of pattern similarity in the 20 anatomically defined regions was associated with subsequent memory performance. We constructed a beta-series model (19) with one regressor for each trial and estimated the model using ridge regression (20). Consistent with the first two experiments, 7 of the 20 regions showed a significantly higher level of pattern similarity across repetitions for subsequently recalled items than for subsequently forgotten items (P < 0.05; one region remained significant by Bonferroni correction), whereas no region showed the opposite effect (Fig. 4, fig. S8, and table S6). Taking the LdLOC as an example, we found that the level of pattern similarity across repetitions showed a significant subsequent memory effect (F1,21 = 18.69, P = 0.0003) (Fig. 4D). Again, after removing voxels showing subsequent memory effects in terms of activation levels (P < 0.05, uncorrected), the remaining voxels in the left dorsal visual stream (dLOC and IPL) also showed stronger pattern similarity across repetitions for subsequently recalled than subsequently recognized or forgotten words (F1,21 = 7.97, P = 0.01) (Fig. 4F and table S6).

Fig. 4

Neural pattern similarity is associated with free recall of words. Greater pattern similarity for subsequently recalled words than for forgotten words was found in (A) the LIFG, (B) the left middle temporal gyrus (LMTG), (C) the right middle temporal gyrus (RMTG), (D) the LdLOC, (E) LIPL, and (F) the left dorsal visual stream that includes the anatomical region of LIPL and LdLOC, but excludes voxels showing significant subsequent memory effects in activation levels when we assume a liberal threshold (P < 0.05, uncorrected). All ROIs were overlaid onto the group-averaged anatomic map. The bar graphs show the group-averaged (n = 22) mean correlation as a function of subsequent memory. The within-item correlation was calculated for each individual item (averaged across all pairs of repetitions) and then averaged separately for recalled and forgotten items. The cross-items correlation was calculated between items within each memory status. Error bars represent within-subject error. See tables S6 and S7 and fig. S8 for detailed statistics and results for other regions.

Finally, if the degree of pattern similarity truly reflects item-specific reinstatement of activation patterns, we should expect a higher level of pattern similarity within items (i.e., cross-repetitions) than between items. To test this prediction, we calculated the averaged across-item pattern similarity of all possible pairings (except the within-item, cross-repetition pairings), separately for recalled items and forgotten items. The results showed that within-item correlation was higher than that for cross-item correlation, especially for recalled items (fig. S8 and table S7). Again taking the LdLOC as an example, we found that the degree of pattern similarity across repetitions in this region for recalled items was significantly higher than cross-item pattern similarity [t(21) = 3.13, P = 0.005], whereas the difference was not significant for forgotten items (P = 0.41) (Fig. 4D), which suggests that repeatedly studying the same material is not sufficient to introduce activation reinstatement at the item-specific level, and failure of pattern reinstatement is associated with forgetting.

We took a number of measures to ensure that our results were not due to the effects of repetition lag or repetition priming. In addition to matching the number of trials, we also carefully matched the repetition lags between remembered and forgotten items (Experiment 1: remembered versus forgotten: 6.11 versus 6.05; t = 1.02, P = 0.31; Experiment 2: recalled versus recognized versus forgotten: 6.03 versus 6.02 versus 5.63, F2,42 = 1.41, P = 0.26). There was also no difference in the repetition lag between recalled and forgotten trials in Experiment 3 (6.4 versus 6.3, t = 1.22, P = 0.23). In addition, we did not find a significant interaction between repetition priming and subsequent memory in most of the brain regions in any of the experiments (tables S4 to S6). Third, in Experiment 3, the reaction times and accuracy in the semantic judgment task and the orientation judgment task that followed subsequently remembered and forgotten items were not different (table S2). Finally, for Experiments 1 and 2, the degree of pattern similarity is not an artifact of design matrix orthogonality (figs. S9 and S10).

Taken together, our results suggest that episodic memory encoding is enhanced by reactivating the initial neural representation in each subsequent study episode. Using different study materials and different memory tests, our data suggest that pattern reinstatement can account for subsequent memory effects for both verbal and nonverbal materials and in both recall and recognition tests. Although the within- versus across-items analysis in Experiment 3 demonstrates a significant effect of pattern overlap at the level of individual items, the results of Experiments 1 and 2 are also consistent with an effect of process-level overlap; we propose that both of these are likely to be important in determining the effectiveness of repeated study. We suggest that repeated study episodes lead to more effective encoding when the same neural representation is reinstated, which is incompatible with the encoding variability hypothesis. Consistent with our results, it has recently been shown that more reproducible neural patterns are associated with more conscious cognitive processing (21), which suggests that consistency of pattern engagement may be a more general marker of effective cognitive processing (perhaps because of its effects on memory encoding).

Previous studies have shown that memory retrieval is associated with reactivation of some of the same sensory regions that were activated during perception of those items (2226). This category-specific or sequence-specific activation reinstatement precedes memory (27) and is associated with performance in free recall (28). The present study extends these findings and shows that during subsequent learning where no explicit retrieval was required, item-specific pattern reinstatement occurs, resulting in a stronger episodic encoding event that supports subsequent memory. Our results are also consistent with evidence showing that memory consolidation, whether during sleep or awake periods following learning, involves replay of neural activation patterns during learning (29, 30). Given the important role of memory retrieval on memory retention (31), these results suggest that reactivation of the same neural pattern during initial learning, whether during repeated practice, memory consolidation, and/or memory retrieval, can enhance memory.

Although most previous studies on the subsequent memory effect focused on one-shot learning, real learning in daily life often involves repeated practice. Our study suggests that, for repeated study events, pattern reinstatement is as sensitive as, if not more sensitive than, overall activation (11, 12, 32) as a predictor of subsequent memory. Our approach can readily be used to examine the neural mechanisms underlying other manipulations that affect memory encoding during repeated practice, such as the spacing effect and the variance effect (7), which would help to clarify the effects of encoding variability. However, fMRI data are a relatively coarse aggregate measure of the responses of large populations of neurons and, thus, may not necessarily capture all of the aspects of encoding variability that might be at play. Future studies need to further examine this issue by applying similar approaches using complementary neuroimaging techniques, such as electroencephalography and single-unit recording.

Supporting Online Material

www.sciencemag.org/cgi/content/full/science.1193125/DC1

Materials and Methods

Figs. S1 to S10

Tables S1 to S7

References

References and Notes

  1. Materials and methods are available as supporting material on Science Online.
  2. We wish to thank C. Stark for his suggestions in implementing Experiment 3; R. A. Bjork, D. M. Schnyer, and R. Raizada for their helpful comments on an earlier version of this report; and L. Mei, F. Xue, X. Lei, X. Xiao and M. Zhang for their help in data collection. This study was supported by the Program for New Century Excellent Talents in University, NSF (BCS 0823624 and BCS 0823495), NIH (HD057884-01A2), and the 111 Project of China (B07008).
View Abstract

Navigate This Article