Report

BOLD Responses Reflecting Dopaminergic Signals in the Human Ventral Tegmental Area

See allHide authors and affiliations

Science  29 Feb 2008:
Vol. 319, Issue 5867, pp. 1264-1267
DOI: 10.1126/science.1150605

Abstract

Current theories hypothesize that dopamine neuronal firing encodes reward prediction errors. Although studies in nonhuman species provide direct support for this theory, functional magnetic resonance imaging (fMRI) studies in humans have focused on brain areas targeted by dopamine neurons [ventral striatum (VStr)] rather than on brainstem dopaminergic nuclei [ventral tegmental area (VTA) and substantia nigra]. We used fMRI tailored to directly image the brainstem. When primary rewards were used in an experiment, the VTA blood oxygen level–dependent (BOLD) response reflected a positive reward prediction error, whereas the VStr encoded positive and negative reward prediction errors. When monetary gains and losses were used, VTA BOLD responses reflected positive reward prediction errors modulated by the probability of winning. We detected no significant VTA BOLD response to nonrewarding events.

Functional magnetic resonance imaging (fMRI) has become a prominent method for imaging brain activity in humans. Commonly used fMRI protocols acquire functional data with a spatial resolution on the order of several millimeters. These protocols are adequate for measuring blood oxygen level–dependent (BOLD) responses from relatively large neural structures such as the cortex and basal ganglia. However, they are not suitable for imaging brainstem structures that are of long-standing interest to neuroscientists. In particular, the brainstem nuclei of the dopamine, norepinephrine, and serotonin systems have long been known to play a critical role in the regulation of brain function, and disturbances of these systems have been implicated in most major psychiatric disorders. Recent theoretical advances have begun to identify specific functions for these brainstem systems. In particular, the reward prediction error theory of dopamine function (1, 2) proposes a role for this neuromodulator in reinforcement learning. This theory makes specific predictions that have been tested in direct neuronal recordings from brainstem dopaminergic nuclei in nonhuman species. However, imaging studies in humans have been restricted to measurements from projection areas of the dopamine system [such as the ventral striatum (VStr) and medial prefrontal cortex] that are larger and therefore more easily imaged with fMRI. Here, we report the use of a combination of recently developed neuroimaging techniques that address the difficulties inherent to brainstem imaging in order to directly image the VTA.

There are several methodological challenges to imaging brainstem nuclei. First among them is the small size of the nuclei. The VTA is ∼60 mm3 in volume (3), or roughly the size of 2 voxels at the resolution common in fMRI studies (418). To address this issue, we acquired high-resolution echo planar images that have been shown to be sufficient to discern individual subcortical nuclei and reduce partial volume effects [e.g. (19, 20)]. Second, brainstem structures are near large pulsatile blood vessels that create physiological movement artifacts and consequent magnetic field inhomogeneities. We therefore acquired functional data synchronized with the participants' cardiac cycles so that these effects were minimized (21). Third, image contrasts commonly used for anatomical localization (such as T1-weighted images) are poorly suited to brainstem studies because they do not provide contrast between different brainstem nuclei. For the midbrain, several alternate pulse sequences have been shown to produce desired contrast (22, 23). We used proton-density weighted images to visualize the substantia nigra (SN) and then used the SN as a landmark to locate the VTA (22) (Fig. 1). Finally, it has been shown that the brainstem does not reliably align to standard brain templates used for spatial normalization in the group analysis of data. We used a new normalization algorithm that significantly improves registration of the brainstem across participants (24).

Fig. 1.

Localization of midbrain dopamine nuclei with MRI. (A) The SN are located in the lateral portions of the midbrain, between the red nucleus and the cerebral peduncle. The VTA flanks the midline, medial to the SN (3, 50). (B) Midbrain dopamine nuclei are clearly visible on proton-density weighted images; the midbrain is outlined in the box and expanded in the rightmost image. An axial slice is shown. The SN are the hyperintense areas adjacent to the hypointense red nucleus and cerebral peduncle (22). The SN are outlined with a dashed line and the VTA with a solid line in the expanded view. (C) Slice placement for one participant. The midbrain was identified in the central sagittal slice of the T1-weighted structural image. An oblique slab comprising axial/coronal slices (each slice 1.9 mm thick) was centered on the VTA and tilted to include as much of the SN and VStr as possible (38). The number of slices used was determined by the participant's heart rate. (See supporting online material for details of image acquisition.)

We used these methods in two experiments modeled on previous nonhuman primate studies (25, 26) and motivated directly by the reward prediction error theory of dopamine function (1, 2). According to this theory, the firing rate of dopamine neurons directly encodes the difference between expected and received reward. This assumes that inputs to dopaminergic midbrain structures relay both the reward experienced at a given time and the reward expected at that time (27). Dopamine neurons are thought to calculate the difference between these two inputs to produce a reward prediction error signal (1, 2). Because the BOLD response is thought to correlate most strongly with synaptic current (28, 29), and therefore is likely to reflect afferent input, the BOLD response in midbrain dopaminergic structures could reflect several quantities: (i) current reward, (ii) expected reward, or (iii) a sum of the inputs equal to the reward prediction error (30). We hypothesized that BOLD responses measured in midbrain dopaminergic structures would reflect a reward prediction error. We positioned our slices to incorporate the VTA and as much of the SN (2, 31, 32) and VStr (3337) as possible (38).

In our first experiment, we used a classical conditioning procedure known to elicit responses to unpredictable rewards and cues predicting rewards in the nonhuman primate dopamine system (2). Thirsty human participants were trained to expect a liquid reward at a fixed interval after the display of a visual cue (35, 39, 40). After training, delivery of the liquid reward was delayed in a subset of trials. This design allowed for investigation of signals related to both positive and negative reward prediction errors. Previous studies indicate that the omission of reward at the expected time generates a negative reward prediction error (1, 25). Subsequently, when reward is delivered at the untrained, delayed, time, its delivery produces a positive reward prediction error (1, 2, 25). We estimated a general linear model (GLM) to look for deviations in the BOLD response at times when positive and negative reward prediction errors were expected (41).

The BOLD response in VTA was significantly related to positive (P < 0.05, two-sample t test corrected for multiple comparisons) but not negative reward prediction error signals (Fig. 2). The BOLD response in the VStr was significantly related to negative reward prediction error signals (P < 0.05, two-sample t test corrected for multiple comparisons) and also showed a nonsignificant trend toward a relation with positive reward prediction error signals (Fig. 3) (38). Post hoc analyses of the mean event-related time courses in the VStr indicated that BOLD responses were significantly enhanced by positive reward prediction errors (fig. S4).

Fig. 2.

The VTA BOLD response encoded positive reward prediction errors from delivery of primary rewards. (A) A random effects GLM analysis showed that VTA BOLD responses encoded positive reward prediction error signals (n = 18; P <0.05, two-sample t test corrected for multiple comparisons). The cluster volume is 39 mm3. Statistical maps are displayed on “brainstem-normalized” images (24): a proton-density weighted image in the axial view (left) and a T1-weighted image in the sagittal view (right). Coordinates shown are in brainstem-normalized space. (B) Plot of contrast estimates. Bars are average of regression coefficients (beta-weights from the GLM) for all voxels shown in (A). The VTA BOLD response to an unexpected reward was significantly larger than the BOLD response to an expected reward (*P <0.05, two-sample t test corrected for multiple comparisons). The VTA BOLD response to a negative reward prediction error was not significant (NS; P = 0.2671, two-sample t test).

Fig. 3.

The striatal BOLD response to positive and negative reward prediction errors from delivery of primary rewards. (A) A random effects GLM analysis revealed regions in the bilateral VStr [left: (14, 2, –7); right: (–18, 3, 8)] with BOLD responses reflecting negative reward prediction errors (n = 18; P < 0.05, two-sample t test corrected for multiple comparisons). Statistical maps are displayed on a T1-weighted image in Talairach space. The region of the VStr that we recorded from is delineated with bluelines (38). (B) Plot of contrast estimates. Bars are average of regression coefficients (beta-weights from the GLM) for all voxels shown in (A). VStr BOLD response reflects a negative reward prediction error (reward expected but not received; *P <0.05, two-sample t test corrected for multiple comparisons). To determine if the VStr region shown in (A) also responded to positive reward prediction errors, we examined the contrast estimates for unexpected and expected reward receipt. The VStr region in (A) showed a trend toward a stronger BOLD response to unexpected than expected rewards. (C) Correlation analysis between BOLD responses to a positive reward prediction error measured from VStr and the VTA.

If the BOLD response to positive reward prediction errors in the striatum resulted at least in part from activity in the VTA, then the responses in the two structures should correlate with one another. To test this prediction, we correlated the BOLD response in these two regions after the delivery of delayed, unexpected rewards. VTA and VStr BOLD responses were positively correlated (r = 0.51; Fig. 3C), suggesting a functional connection between the two areas (42).

We conducted a second experiment to verify that the BOLD response in the VTA encoded positive but not negative reward prediction errors. The experiment was also designed to assess the generality of these signals, by testing whether secondary (monetary) rewards were effective in producing BOLD responses in the VTA. Finally, we tested whether BOLD responses to reward prediction errors scale with reward probability, as predicted by theory. Participants were shown a number and indicated by button-press their guess as to whether a second number would be greater or less than the first number. Both numbers were randomly selected from the range of 0 to 10 and were never equal. Thus, the probability of guessing correctly was greatest when the first number was toward the extremes of the range. The outcome of the trial was displayed after a variable delay; participants won $1 for correct guesses and lost $1 for incorrect guesses. According to reinforcement learning models, the positive reward prediction error signal should decrease with increasing probability of winning (1, 43). Conversely, negative reward prediction error signals (resulting from losses) should increase in magnitude with the probability of winning (1, 43).

We estimated a model that included separate regressors for outcomes in which the participant won and lost. For outcomes in which the participant won, we found that BOLD responses in the VTA significantly scaled with the probability of winning (P < 0.05, two-sample t test corrected for multiple comparisons; Fig. 4). This replicated the finding from the first experiment that the VTA BOLD response reflects a positive reward prediction error. Though the two experiments used different reward modalities, the VTA regions we identified in both experiments overlapped (Fig. 4C).

Fig. 4.

VTA BOLD response to monetary gains and losses. (A) A random effects GLM analysis revealed that the VTA BOLD response was significantly increased for rewarding trial outcomes (gain $1) and was modulated by the probability of winning (n = 21; P <0.05, two-sample t test corrected for multiple comparisons). The cluster volume is 59 mm3. Statistical maps are overlaid on a proton density– weighted image (left) and T1-weighted image (right) in brainstem-normalized space (24). (B) Plot of contrast estimates for trial outcome. VTA BOLD responses reflected a positive reward prediction error (*P < 0.05, two-sample t test corrected for multiple comparisons). Because no brainstem regions were found to respond to negative reward prediction error signals (trial outcome when participant lost $1), the same voxels shown in (A) were used to generate this contrast estimate. (C) The VTA regions identified in both experiments overlap.

We found no midbrain regions showing BOLD responses that correlated with negative reward prediction errors. In the first experiment, negative reward prediction errors resulted from the omission of an expected reward, whereas in the second experiment they were elicited by the loss of money. Although theory predicts that the dopamine system should report a negative reward prediction error (1, 2), the underlying physiology of such a response remains unclear. Dopamine neurons have been shown, as predicted, to exhibit phasic decreases in firing for negative reward prediction errors (2). However, such reductions in dopamine firing rate are subtle (32) and their significance remains controversial (44). Furthermore, negative reward prediction errors resulting from aversive events (such as losses, as opposed to the omission of a reward) may produce no response at all in dopamine neurons (45, 46).

Negative reward prediction errors might also have been expected to elicit an increase in activity within the VTA, among a subpopulation of neurons that are known to respond to aversive stimuli (“tertiary cells”) (46, 47). However, tertiary cells constitute less than 30% of the neurons in the VTA (47). Therefore, while it is possible that tertiary cells did respond to monetary losses in our second experiment, it is likely that the corresponding positive BOLD response was too small to be resolved with the imaging protocols used. The absence of such BOLD responses suggests that the ones we did observe were sensitive selectively to dopamine-related signals.

We have shown that BOLD responses can be measured in the brainstem at a spatial resolution sufficient for investigating individual nuclei. Our findings are consistent with the reward prediction error theory of dopamine function in the human brain. Not only does this work add precision and accuracy to the study of dopamine function in humans, it also opens the door to the careful study of the role of dopamine in cognition at large (for example, its hypothesized role in updating working memory representations in prefrontal cortex) (48). It should now be possible to carry out similar investigations of other brainstem nuclei, such as norepinephrine and serotonin, that have been related to human cognition in recent theoretical work (49, 44).

Supporting Online Material

www.sciencemag.org/cgi/content/full/319/5867/1264/DC1

Materials and Methods

Figs. S1 to S5

References

References and Notes

View Abstract

Navigate This Article