Report

Preference by Association: How Memory Mechanisms in the Hippocampus Bias Decisions

See allHide authors and affiliations

Science  12 Oct 2012:
Vol. 338, Issue 6104, pp. 270-273
DOI: 10.1126/science.1223252

The Right Choice?

So-called irrational decisions made by humans are popular fodder for “believe it or not” stories. But what's actually happening when we make choices that do not seem to be justifiable on purely economic or logical grounds? Presumably, we are not simply making errors; instead, our choices may reflect an internal bias that we are not aware of. Wimmer and Shohamy (p. 270) show how the hippocampus can instill an unconscious bias in valuations, whereby an object that is not highly valued on its own, increases in value when it becomes implicitly associated with a truly high-value object. As a consequence, we then end up preferring the associated object over a neutral object of equal objective value while not really knowing why.

Abstract

Every day people make new choices between alternatives that they have never directly experienced. Yet, such decisions are often made rapidly and confidently. Here, we show that the hippocampus, traditionally known for its role in building long-term declarative memories, enables the spread of value across memories, thereby guiding decisions between new choice options. Using functional brain imaging in humans, we discovered that giving people monetary rewards led to activation of a preestablished network of memories, spreading the positive value of reward to nonrewarded items stored in memory. Later, people were biased to choose these nonrewarded items. This decision bias was predicted by activity in the hippocampus, reactivation of associated memories, and connectivity between memory and reward regions in the brain. These findings explain how choices among new alternatives emerge automatically from the associative mechanisms by which the brain builds memories. Further, our findings demonstrate a previously unknown role for the hippocampus in value-based decisions.

Decisions are sometimes guided by direct past experience: If a choice led to a good outcome in the past, people are likely to make that same choice again. This process is known to depend on reward learning mechanisms in the striatum (1, 2). But frequently in life, we have to decide between options we have never considered before. It has been suggested that such decisions could be guided by associative memory (35); however, surprisingly little is known about how this process happens.

We investigated the mechanism by which neural circuits for memory modulate value and guide decisions about new choice options. Our central hypothesis was that the hippocampus enables the positive value of reward to spread across associated memories, thereby increasing the value of items that were never rewarded. Specifically, we hypothesized that receiving reward can lead to two simultaneous and interactive processes: (i) the direct learning of stimulus-reward associations in the striatum and (ii) the spread of reward to associated items stored in memory via the hippocampus.

Our hypothesis is grounded in two essential features of how the hippocampus builds memories. First, the hippocampus encodes relationships between items and events that appear together, forming an associative link between them (68). Second, because of this associative link, when a person later encounters one item, the hippocampus can complete the pattern and automatically reactivate the neural representation of the other item (912), allowing the integration of old memories with new ones.

We reasoned that these features of memory formation in the hippocampus could provide a mechanism by which reward experiences can systematically change the value of items that were never rewarded: These items gain a positive value merely by association. If so, this mechanism predicts that later, when confronted with a decision, people will be biased to choose items that were never rewarded in the past (4, 1315). By emphasizing the associative nature of processes in the hippocampus, regardless of awareness (7, 8, 16, 17), this mechanism further predicts that the hippocampus might bias value even when associations are not explicitly remembered.

To test our prediction that reward will spread across associated memories, we used functional magnetic resonance imaging to measure brain responses during a learning and decision task designed to test how associative memory biases decisions between new choice options (Fig. 1) (13). First, we had participants (n = 28) build new associative memories by exposing them to regularities between pairs of neutral stimuli (denoted here as S1 and S2). These regularities were encoded incidentally while participants performed a cover task (Fig. 1A).

Fig. 1

The task consists of three phases: association learning, reward learning, and decision-making. (A) In the association phase, participants were exposed to a series of pairs of pictures (S1 and S2 stimuli) while performing a cover task to detect “target” upside-down pictures. S1 stimuli were either face, scene, or body part pictures; S2 stimuli were circle images. (B) In the reward phase, participants learned through classical conditioning that half of the S2 stimuli were followed by a monetary reward (S2+), whereas the other S2 stimuli were followed by a neutral outcome (no reward, S2–). S1 stimuli never appeared in this stage. (C) In the decision phase, participants were asked to decide between two stimuli (both S1 or both S2) for a possible monetary win. No feedback was provided, and all gains were awarded at the end of the experiment. Decision bias was operationalized as the tendency to choose S1+ over S1– stimuli in this phase.

Next, we associated value with some items by using a classical conditioning paradigm in which half of the S2 stimuli (S2+) were now followed by a monetary reward (Fig. 1B). This procedure is known to enhance the value of the directly rewarded items via well-described reward learning mechanisms in the striatum (2). Critically, we hypothesized that at the same time, associative memory processes in the hippocampus would activate the specific S1 items associated with the rewarded S2 items, resulting in a transfer of the reward to the S1 items as well, through hippocampal-striatal connectivity (18, 19). This would increase the value of S1 items that were linked to the rewarded S2 items (S1+), creating a bias toward choosing these items in the future, despite the fact that these items were never rewarded and were not even seen during reward learning.

To measure the effect of associative memory on value, in the final phase of our experiment, we asked participants to make a series of decisions in which they had to choose between two S1 items, selecting the “luckier” one for potential winnings, awarded at the end of the experiment (Fig. 1C, top). In the absence of any spread of reward, participants should be equally likely to choose any of these nonrewarded items, and brain activity during the prior reward phase should be unrelated to these decisions. However, if reward spreads and biases decisions, then participants should be biased toward choosing those nonrewarded S1 items that were previously associated with the S2 rewarded items. Thus, we operationalized “decision bias” as the tendency to chose S1+ items over S1– items, and we hypothesized that decision bias should be related to neural processes in the hippocampus during reward learning.

To test this hypothesis, we focused our analyses on brain activity during the reward phase and asked whether activity during this phase was related to later biases in decisions. We made three specific predictions about the neural mechanisms giving rise to the spread of value to new choice options: (i) If associative memory processes underlie shifts in value, then biased decisions should be predicted by the magnitude of activation in the hippocampus during reward learning. (ii) If reactivation of associations is the mechanism by which value spreads, then during reward learning there should be evidence of neural reactivation in visual areas that represent the associated S1 items. (iii) If decision bias stems from interactions between associative memory processes in the hippocampus and reward learning processes in the striatum, then decision bias should be related to functional connectivity between these two regions.

Behaviorally, participants tended to choose the S2+ items over the S2– items in the decision phase, indicating successful reward learning. Interestingly, decision bias in favor of S1+ items varied markedly both within and across individuals: Most participants displayed a bias in favor of S1+ items, but some did not (Fig. 2A). Within individuals, this measure of bias was strong for some associations, but weaker for others. This variability in behavior allowed us to ask: What are the neural mechanisms that support decision bias?

Fig. 2

Decision bias varies within and across participants and is related to activation in the hippocampus during the reward phase. (A) Decision-phase preferences for S2+ stimuli (dark gray) and S1+ stimuli (average, blue; within-participant mean, black) are shown. Error bars represent ±SEM. (B) Within-participant decision bias variability. Each participant was exposed to three different types of stimuli, allowing for differential bias across associations. Individual mean decision bias [shown in (A)] was separated, ranked by level of bias, and averaged across participants. (C) Hippocampal activation within participants during the reward phase (S2 stimulus presentation and outcome) predicted subsequent levels of decision bias (for associated S1 stimuli) [z score = 4.00 (26, –34, –12); P < 0.05, SVC; images thresholded at P < 0.005, uncorrected for display]. R, right.

To test our first prediction that decision bias is related to hippocampal activity, we used a general linear model to compare blood oxygen level–dependent (BOLD) activity during the reward phase between items that led to later behavioral decision bias versus those that did not, i.e., S1+ > S1–, within individuals (Fig. 2B). Activation in the posterior hippocampus during reward learning was greater for items that led to more decision bias (Fig. 2C). A similar pattern was found across individuals: Activation in the hippocampus correlated with the proportion of biases in subsequent decisions [Montreal Neurological Institute (MNI) space coordinates (R, A, S) 30, –6, –20; P < 0.05 small-volume corrected (SVC) for familywise error]. As would be expected if decision bias is driven by the spread of value via associative memory processes, neural predictors of bias were selective to the reward phase. Parallel analyses of BOLD activity during the association and decision phases showed that no areas of the brain within or across participants had activation correlated with decision bias.

To investigate whether hippocampal activity was related to explicit memory for the S1-S2 associations, after scanning we tested participants’ ability to correctly pair S1 and S2 items and asked about their choice strategy and awareness of task structure (20). Reflecting the automatic nature of the underlying associative memory processes (13, 16), we found no evidence for explicit memory of the associations. Moreover, there was no relation between measures of explicit S1-S2 memory and either decision bias or hippocampal activity (fig. S3) (20). Although it is difficult to conclusively determine implicitness of cue pairings (21), these findings suggest that the role of the hippocampus in decision bias does not seem to be driven by explicit or strategic effects.

To test our second prediction that decision bias is supported by reactivation of associations, we exploited the fact that the different categories of S1 stimuli (faces, scenes, and body parts) elicit activation in distinct areas of the visual cortex (22). In our design, these category-specific S1 items were associated with S2 items in the association phase; in the reward phase, however, only S2 items were presented. Thus, during reward learning, any selective activation in these visual cortical areas probably reflects associative reactivation, in memory, of these items. This allowed us to test whether biases in decisions, as measured behaviorally, were predicted by differential activation in category-specific areas during the reward phase.

We analyzed associative reactivation during the reward phase using activation masks defined by participants’ responses to S1 visual stimuli during the association phase (Fig. 3A) (20). These participant-specific masks were then applied to reward-phase responses evoked by S2 stimuli, relative to the alternative categories, resulting in a measure of reactivation for each participant for each association. We compared reactivation for associations that led to high versus low decision bias.

We found that reactivation in visual regions during reward learning related to later biases in decisions (Fig. 3), such that across all categories there was greater neural reactivation for high- versus low-bias decisions. Reactivation was significant in the first half of the reward phase. Reflecting the continual updating of memory representations, it was present but weaker over the full reward phase, as additional learning about the S2 items took place. Importantly, reactivation was selective to the associated category-specific regions and was not general to all visual regions of interest (20).

Fig. 3

Reactivation of category-specific visual areas during the first half of the reward phase is related to subsequent decision bias. (A) Example participant region of interest masks (derived from the association phase) for body, face, and scene S1 stimuli. Masks were applied to S2 presentations during the reward phase. (B) S2 presentation elicits activation in visual regions responsive to associated S1 stimuli when participants later exhibit decision bias [t(22) = 2.29, P < 0.05]. Error bars indicate ±SEM; a.u., arbitrary units.

Finally, we tested our third prediction that decision bias is related to functional interactions between the hippocampus and the striatum during the reward phase. Specifically, we reasoned that an interaction between associative reactivation of S1 stimuli via the hippocampus and reward learning in the striatum could provide a link between S1 stimuli and reward.

We investigated functional connectivity during the reward phase by conducting a psychophysiological interaction (PPI) analysis using the right hippocampus as a seed and decision bias (high versus low) as the behavioral (psychological) modulator. We found a functional relation between the hippocampus and the striatum; this relation was significantly greater for high-bias stimuli [z score = 3.23 (6, 6, 12); P < 0.05 SVC)] (Fig. 4) (20).

Fig. 4

Decision bias was related to functional connectivity between the hippocampus and the striatum during the reward phase. A PPI analysis revealed correlated activity between the hippocampus and the striatum during trials that led to high versus low decision bias. The same region in the striatum was also found to correlate with reward learning. This finding indicates that the hippocampus and the striatum may form a functional circuit to support shifts in value.

We then investigated whether the correlation between the hippocampus and the striatum for high-bias decisions is mediated by reactivation in visual cortical regions. To test this, we extracted trial-by-trial measures of evoked responses to high-bias stimuli and performed a formal mediation analysis on the path between the hippocampus and the striatum via the potential visual cortex mediator (20). Consistent with the PPI result, we found that high decision bias was related to correlated BOLD activity in the hippocampus and striatum (0.214 ± 0.054, P < 0.001). Additionally, we found a trend for a mediating effect of visual reactivation on this relationship (0.010 ± 0.006, P < 0.08) (20).

To test for the specificity of these connectivity results, we conducted a series of control analyses (20). In a PPI analysis of the association phase, we found no areas with differential connectivity that related to decision bias, and in the reward phase, we found no significant activation outside of the striatum. Furthermore, in the mediation analysis, a control analysis of regions that were activated by the reward phase did not show any connectivity with regions of interest for high-bias stimuli (20). Together, these results indicate that decision bias depends not only on hippocampal activation during reward learning, but additionally on functional connectivity between the hippocampus and the striatum, putatively mediated by reactivation in visual cortical regions.

These results indicate that reward can spread to bias the value of options that were themselves never directly rewarded (13, 15). This finding provides insight into how people are biased by past experience to make new decisions between options that were never previously rewarded: Networks of associations in memory, formed across many different experiences, can result in the spread of value across associations.

The idea that memory and decision-making are intertwined has deep roots in behavioral theories of both memory (3, 7, 18) and decision-making (2326), yet there has been very little evidence for an underlying mechanism. Understanding the mechanism by which value spreads among related memories in the brain may help to explain why people sometimes develop seemingly ungrounded preferences for or against particular things, places, or people. Although we highlight how transfer of past experience can guide behavior in a changing environment, this same mechanism may also lead to seemingly irrational choices, consistent with social and cognitive theories regarding the role of associative memory in decision-making heuristics (2327).

The finding that the hippocampus supports the spread of value provides several new insights into the neural bases of both memory and decision-making. First, our findings extend the role of the hippocampus beyond memory per se, demonstrating that the hippocampus contributes directly to value assignment and decision-making.

Second, although in humans the hippocampus is traditionally associated with explicit declarative memory (7), our results indicate that transfer of value by the hippocampus is not driven by conscious awareness. Thus, our results suggest that the hippocampus contributes to an automatic assessment of value, perhaps performing a function similar to Bayesian inference about value (28).

Finally, though it is known that memory can support decisions by retrieving relevant information at the time of decision-making (29, 30), our results demonstrate an alternative mechanism whereby the hippocampus dynamically modulates value representations during learning itself (5, 31, 32). This mechanism allows value to spread and bias decisions without effortful retrieval at the time of decision.

Understanding how associative memory biases decisions provides insight into critical open questions in decision-making research. Although reward learning models have been successfully applied to many aspects of behavior, these models cannot account for the full diversity of animal and human decision-making (2, 33, 34). The uncovering of a neural mechanism by which associative memory biases decision-making sheds light on how value generalizes across experience, with implications for both adaptive behaviors and maladaptive behaviors such as addiction.

Supplementary Materials

www.sciencemag.org/cgi/content/full/338/6104/270/DC1

Materials and Methods

Supplementary Text

Figs. S1 to S5

References (3554)

References and Notes

  1. Materials and methods are available as supplementary materials on Science Online.
  2. Acknowledgments: Funding was provided by a NSF Career Development Award and National Institutes of Drug Abuse grant R03-DA026957 to D.S. We thank E. K. Braun, L. Davachi, R. Hassin, and E. Weber for helpful comments on an earlier draft of the manuscript; L. Atlas and T. Wager for guidance with the mediation analyses; and N. Daw and R. Hassin for insightful conversations.
View Abstract

Navigate This Article