Preference by Association: How Memory Mechanisms in the Hippocampus Bias Decisions

Decisions are sometimes guided by direct past experience: If a choice led to a good outcome in the past, people are likely to make that same choice again. This process is known to depend on reward learning mechanisms in the striatum (1, 2). But frequently in life, we have to decide between options we have never considered before. It has been suggested that such decisions could be guided by associative memory (3–5); however, surprisingly little is known about how this process happens.

We investigated the mechanism by which neural circuits for memory modulate value and guide decisions about new choice options. Our central hypothesis was that the hippocampus enables the positive value of reward to spread across associated memories, thereby increasing the value of items that were never rewarded. Specifically, we hypothesized that receiving reward can lead to two simultaneous and interactive processes: (i) the direct learning of stimulus-reward associations in the striatum and (ii) the spread of reward to associated items stored in memory via the hippocampus.

Our hypothesis is grounded in two essential features of how the hippocampus builds memories. First, the hippocampus encodes relationships between items and events that appear together, forming an associative link between them (6–8). Second, because of this associative link, when a person later encounters one item, the hippocampus can complete the pattern and automatically reactivate the neural representation of the other item (9–12), allowing the integration of old memories with new ones.

We reasoned that these features of memory formation in the hippocampus could provide a mechanism by which reward experiences can systematically change the value of items that were never rewarded: These items gain a positive value merely by association. If so, this mechanism predicts that later, when confronted with a decision, people will be biased to choose items that were never rewarded in the past (4, 13–15). By emphasizing the associative nature of processes in the hippocampus, regardless of awareness (7, 8, 16, 17), this mechanism further predicts that the hippocampus might bias value even when associations are not explicitly remembered.

To test our prediction that reward will spread across associated memories, we used functional magnetic resonance imaging to measure brain responses during a learning and decision task designed to test how associative memory biases decisions between new choice options (Fig. 1) (13). First, we had participants (n = 28) build new associative memories by exposing them to regularities between pairs of neutral stimuli (denoted here as S₁ and S₂). These regularities were encoded incidentally while participants performed a cover task (Fig. 1A).

• Download high-res image
• Open in new tab
• Download Powerpoint

Next, we associated value with some items by using a classical conditioning paradigm in which half of the S₂ stimuli (S₂+) were now followed by a monetary reward (Fig. 1B). This procedure is known to enhance the value of the directly rewarded items via well-described reward learning mechanisms in the striatum (2). Critically, we hypothesized that at the same time, associative memory processes in the hippocampus would activate the specific S₁ items associated with the rewarded S₂ items, resulting in a transfer of the reward to the S₁ items as well, through hippocampal-striatal connectivity (18, 19). This would increase the value of S₁ items that were linked to the rewarded S₂ items (S₁+), creating a bias toward choosing these items in the future, despite the fact that these items were never rewarded and were not even seen during reward learning.

To measure the effect of associative memory on value, in the final phase of our experiment, we asked participants to make a series of decisions in which they had to choose between two S₁ items, selecting the “luckier” one for potential winnings, awarded at the end of the experiment (Fig. 1C, top). In the absence of any spread of reward, participants should be equally likely to choose any of these nonrewarded items, and brain activity during the prior reward phase should be unrelated to these decisions. However, if reward spreads and biases decisions, then participants should be biased toward choosing those nonrewarded S₁ items that were previously associated with the S₂ rewarded items. Thus, we operationalized “decision bias” as the tendency to chose S₁+ items over S₁– items, and we hypothesized that decision bias should be related to neural processes in the hippocampus during reward learning.

To test this hypothesis, we focused our analyses on brain activity during the reward phase and asked whether activity during this phase was related to later biases in decisions. We made three specific predictions about the neural mechanisms giving rise to the spread of value to new choice options: (i) If associative memory processes underlie shifts in value, then biased decisions should be predicted by the magnitude of activation in the hippocampus during reward learning. (ii) If reactivation of associations is the mechanism by which value spreads, then during reward learning there should be evidence of neural reactivation in visual areas that represent the associated S₁ items. (iii) If decision bias stems from interactions between associative memory processes in the hippocampus and reward learning processes in the striatum, then decision bias should be related to functional connectivity between these two regions.

Behaviorally, participants tended to choose the S₂+ items over the S₂– items in the decision phase, indicating successful reward learning. Interestingly, decision bias in favor of S₁+ items varied markedly both within and across individuals: Most participants displayed a bias in favor of S₁+ items, but some did not (Fig. 2A). Within individuals, this measure of bias was strong for some associations, but weaker for others. This variability in behavior allowed us to ask: What are the neural mechanisms that support decision bias?

• Download high-res image
• Open in new tab
• Download Powerpoint

To test our first prediction that decision bias is related to hippocampal activity, we used a general linear model to compare blood oxygen level–dependent (BOLD) activity during the reward phase between items that led to later behavioral decision bias versus those that did not, i.e., S₁+ > S₁–, within individuals (Fig. 2B). Activation in the posterior hippocampus during reward learning was greater for items that led to more decision bias (Fig. 2C). A similar pattern was found across individuals: Activation in the hippocampus correlated with the proportion of biases in subsequent decisions [Montreal Neurological Institute (MNI) space coordinates (R, A, S) 30, –6, –20; P < 0.05 small-volume corrected (SVC) for familywise error]. As would be expected if decision bias is driven by the spread of value via associative memory processes, neural predictors of bias were selective to the reward phase. Parallel analyses of BOLD activity during the association and decision phases showed that no areas of the brain within or across participants had activation correlated with decision bias.

To investigate whether hippocampal activity was related to explicit memory for the S₁-S₂ associations, after scanning we tested participants’ ability to correctly pair S₁ and S₂ items and asked about their choice strategy and awareness of task structure (20). Reflecting the automatic nature of the underlying associative memory processes (13, 16), we found no evidence for explicit memory of the associations. Moreover, there was no relation between measures of explicit S₁-S₂ memory and either decision bias or hippocampal activity (fig. S3) (20). Although it is difficult to conclusively determine implicitness of cue pairings (21), these findings suggest that the role of the hippocampus in decision bias does not seem to be driven by explicit or strategic effects.

To test our second prediction that decision bias is supported by reactivation of associations, we exploited the fact that the different categories of S₁ stimuli (faces, scenes, and body parts) elicit activation in distinct areas of the visual cortex (22). In our design, these category-specific S₁ items were associated with S₂ items in the association phase; in the reward phase, however, only S₂ items were presented. Thus, during reward learning, any selective activation in these visual cortical areas probably reflects associative reactivation, in memory, of these items. This allowed us to test whether biases in decisions, as measured behaviorally, were predicted by differential activation in category-specific areas during the reward phase.

We analyzed associative reactivation during the reward phase using activation masks defined by participants’ responses to S₁ visual stimuli during the association phase (Fig. 3A) (20). These participant-specific masks were then applied to reward-phase responses evoked by S₂ stimuli, relative to the alternative categories, resulting in a measure of reactivation for each participant for each association. We compared reactivation for associations that led to high versus low decision bias.

We found that reactivation in visual regions during reward learning related to later biases in decisions (Fig. 3), such that across all categories there was greater neural reactivation for high- versus low-bias decisions. Reactivation was significant in the first half of the reward phase. Reflecting the continual updating of memory representations, it was present but weaker over the full reward phase, as additional learning about the S₂ items took place. Importantly, reactivation was selective to the associated category-specific regions and was not general to all visual regions of interest (20).

• Download high-res image
• Open in new tab
• Download Powerpoint

Finally, we tested our third prediction that decision bias is related to functional interactions between the hippocampus and the striatum during the reward phase. Specifically, we reasoned that an interaction between associative reactivation of S₁ stimuli via the hippocampus and reward learning in the striatum could provide a link between S₁ stimuli and reward.

We investigated functional connectivity during the reward phase by conducting a psychophysiological interaction (PPI) analysis using the right hippocampus as a seed and decision bias (high versus low) as the behavioral (psychological) modulator. We found a functional relation between the hippocampus and the striatum; this relation was significantly greater for high-bias stimuli [z score = 3.23 (6, 6, 12); P < 0.05 SVC)] (Fig. 4) (20).

• Download high-res image
• Open in new tab
• Download Powerpoint

We then investigated whether the correlation between the hippocampus and the striatum for high-bias decisions is mediated by reactivation in visual cortical regions. To test this, we extracted trial-by-trial measures of evoked responses to high-bias stimuli and performed a formal mediation analysis on the path between the hippocampus and the striatum via the potential visual cortex mediator (20). Consistent with the PPI result, we found that high decision bias was related to correlated BOLD activity in the hippocampus and striatum (0.214 ± 0.054, P < 0.001). Additionally, we found a trend for a mediating effect of visual reactivation on this relationship (0.010 ± 0.006, P < 0.08) (20).

To test for the specificity of these connectivity results, we conducted a series of control analyses (20). In a PPI analysis of the association phase, we found no areas with differential connectivity that related to decision bias, and in the reward phase, we found no significant activation outside of the striatum. Furthermore, in the mediation analysis, a control analysis of regions that were activated by the reward phase did not show any connectivity with regions of interest for high-bias stimuli (20). Together, these results indicate that decision bias depends not only on hippocampal activation during reward learning, but additionally on functional connectivity between the hippocampus and the striatum, putatively mediated by reactivation in visual cortical regions.

These results indicate that reward can spread to bias the value of options that were themselves never directly rewarded (13, 15). This finding provides insight into how people are biased by past experience to make new decisions between options that were never previously rewarded: Networks of associations in memory, formed across many different experiences, can result in the spread of value across associations.

The idea that memory and decision-making are intertwined has deep roots in behavioral theories of both memory (3, 7, 18) and decision-making (23–26), yet there has been very little evidence for an underlying mechanism. Understanding the mechanism by which value spreads among related memories in the brain may help to explain why people sometimes develop seemingly ungrounded preferences for or against particular things, places, or people. Although we highlight how transfer of past experience can guide behavior in a changing environment, this same mechanism may also lead to seemingly irrational choices, consistent with social and cognitive theories regarding the role of associative memory in decision-making heuristics (23–27).

The finding that the hippocampus supports the spread of value provides several new insights into the neural bases of both memory and decision-making. First, our findings extend the role of the hippocampus beyond memory per se, demonstrating that the hippocampus contributes directly to value assignment and decision-making.

Second, although in humans the hippocampus is traditionally associated with explicit declarative memory (7), our results indicate that transfer of value by the hippocampus is not driven by conscious awareness. Thus, our results suggest that the hippocampus contributes to an automatic assessment of value, perhaps performing a function similar to Bayesian inference about value (28).

Finally, though it is known that memory can support decisions by retrieving relevant information at the time of decision-making (29, 30), our results demonstrate an alternative mechanism whereby the hippocampus dynamically modulates value representations during learning itself (5, 31, 32). This mechanism allows value to spread and bias decisions without effortful retrieval at the time of decision.

Understanding how associative memory biases decisions provides insight into critical open questions in decision-making research. Although reward learning models have been successfully applied to many aspects of behavior, these models cannot account for the full diversity of animal and human decision-making (2, 33, 34). The uncovering of a neural mechanism by which associative memory biases decision-making sheds light on how value generalizes across experience, with implications for both adaptive behaviors and maladaptive behaviors such as addiction.