Report

A critical time window for dopamine actions on the structural plasticity of dendritic spines

See allHide authors and affiliations

Science  26 Sep 2014:
Vol. 345, Issue 6204, pp. 1616-1620
DOI: 10.1126/science.1255514

Animal behavior follows rewards

Animal behavior is learned and reinforced by rewards. On a molecular level, the reward comes in the form of the neurotransmitter, dopamine, which modulates synapses. The exact timing and mechanism of this process remain unknown. Using optical stimulation, Yagishita et al. found that dopaminergic modulation involved dendritic spine enlargement only during an extremely narrow time window. Known as reinforcement plasticity, this cellular basis for learning could provide insight into psychiatric disorders involving dopaminergic regulation, such as depression, drug addiction, and schizophrenia.

Science, this issue p. 1616

Abstract

Animal behaviors are reinforced by subsequent rewards following within a narrow time window. Such reward signals are primarily coded by dopamine, which modulates the synaptic connections of medium spiny neurons in the striatum. The mechanisms of the narrow timing detection, however, remain unknown. Here, we optically stimulated dopaminergic and glutamatergic inputs separately and found that dopamine promoted spine enlargement only during a narrow time window (0.3 to 2 seconds) after the glutamatergic inputs. The temporal contingency was detected by rapid regulation of adenosine 3′,5′-cyclic monophosphate in thin distal dendrites, in which protein-kinase A was activated only within the time window because of a high phosphodiesterase activity. Thus, we describe a molecular basis of reinforcement plasticity at the level of single dendritic spines.

Animal behaviors are reinforced only when rewarded shortly after a motor or sensory event (1, 2). The neocortex, hippocampus, and amygdala process the sensorimotor signals and send glutamatergic synaptic output to the striatum (3), where connections can be modified by Hebbian learning mechanisms, such as spike-timing-dependent plasticity (STDP) (4). Animals learn to associate the sensorimotor signals with subsequent rewards through reinforcement of the neuronal circuits involving dopamine (57). Despite its importance, this narrow timing detection has never been demonstrated at the cellular level and might be ascribed to neural network properties (6, 8).

Dendritic spine morphology is correlated with spine function (9), and dendritic spines enlarge during long-term potentiation in the cortices (1012). We examined the effects of dopamine on the structural plasticity in striatal medium spiny neurons (MSNs). Results show that dopamine affected spine structural plasticity in a narrow time window consistent with behavioral conditioning (5). Functional imaging revealed the molecular interrelationships between the reinforcement and Hebbian plasticity.

We investigated dopamine actions on glutamatergic synapses on MSNs using optogenetics and two-photon uncaging. For optogenetic stimulation of dopaminergic fibers, a Cre-dependent adeno-associated virus (AAV) vector expressing channelrhodopsin-2 (ChR2) was injected into the ventral tegmental area (VTA) of DAT-Cre mice expressing Cre specific to dopaminergic neurons (Fig. 1A and fig. S1). The direct pathway–constituting MSNs, which mainly express dopamine 1 receptors (D1Rs) (13), were labeled by an AAV vector with a specific promoter for D1R-MSNs (Fig. 1A and fig. S1). In acute coronal slices, including the nucleus accumbens (NAc) core, whole-cell recordings were obtained from the identified D1R-MSNs. Dendritic spines were visualized by means of two-photon microscopy (980 nm) detecting fluorescence of Alexa488 loaded through a recording pipette (Fig. 1B and fig. S1C). Stimulation of a single spine by means of two-photon uncaging (720 nm) of CDNI-Glu (Fig. 1B) induced two-photon excitatory postsynaptic potentials (2pEPSPs) with amplitudes similar to miniature EPSPs (mEPSPs) (Fig. 1C and fig. S1D). The amplitudes of 2pEPSCs positively correlated with spine volumes, as they did with pyramidal neurons (fig. S1E) (9). The STDP protocol of repetitive uncaging of glutamate paired with action potentials (APs) (Glu + AP) (Fig. 1D) (14) induced robust spine enlargement that was selective for the stimulated spine (Fig. 1, E and F) when dopamine (100 μM) was puff applied, whereas only a weak enlargement occurred in the absence of dopamine (Fig. 1G), which is consistent with previous findings (14, 15).

Fig. 1 A temporal profile of dopamine actions on spine enlargement.

(A) Injection of AAV vectors for ChR2 and the D1R-MSN marker (PPTA-mCherry) in 3-week-old DAT-Cre mice. (B) Selective stimulation of dopaminergic and glutamatergic inputs by means of blue laser field irradiation to ChR2 and two-photon uncaging of caged-glutamate at a single spine, respectively, in acute slices of NAc obtained from 5- to 7-week-old mice. (C) An amperometric measurement of dopamine (top) by carbon-fiber electrode and whole-cell recording of glutamate-induced current (bottom, 2pEPSP) in identified D1R-MSNs. (D) An STDP protocol with dopamine puff application. (E) Images of the dendritic spine (red arrowhead) that received STDP stimulation in the presence of dopamine (100 μM). (F and G) Time courses of spine enlargement in the presence [(F), 13 spines, 4 dendrites] and absence of dopamine [(G), 58 spines, 14 dendrites]. (H) Amplitudes of spine enlargements with or without dopamine. **P = 0.0041 by Mann-Whitney U test. (I) STDP with repetitive activation of dopaminergic fibers containing ChR2 (blue lines) at 30 Hz, 10 times (DAopto). (J) Images of the dendritic spine (arrowhead) that received STDP + DAopto with a delay of 1 s. (K to M) Time courses of spine enlargement induced by STDP + DAopto at 1 s [(K), 48 spines, 14 dendrites], –1 s [(L), 20 spines, 5 dendrites] and 5 s [(M), 28 spines, 7 dendrites] after STDP onset. (N) Timings of DAopto application. (O) Increases in spine volumes by STDP + DAopto plotted versus DAopto delay (fig. S2, A to C). Data are presented as mean ± SEM. P = 4.2 × 10−6 with Kruskal-Wallis and **P = 0.0018 (0.6 s) and 0.0027 (1 s) by Steel test in comparison with STDP in the absence of DAopto. Scale bars, 1 μm.

A single pulse of blue laser stimulation of ChR2-expressing dopaminergic fibers induced a transient increase in the tissue dopamine concentration (Fig. 1C). We tested the actions of a physiologically relevant phasic release of dopamine (30 Hz, 10 times) induced by the optogenetic stimulation of dopaminergic fibers (DAopto) and found that DAopto just after STDP stimulation (DAopto delay = 1 s) (Fig. 1I) induced robust spine enlargement (Fig. 1, J and K). Stimulation of dopaminergic fibers 1 s before (DAopto delay = −1 s) or 5 s after (DAopto delay = 5 s) STDP (Fig. 1, L and M) resulted in only a slight enhancement. The actions of DAopto were examined with various timings (Fig. 1N), revealing that DAopto timing was critical to enhance plasticity, with maximal effects for a delay of 0.6 s (Fig. 1O and fig. S2, A to C), and decayed in a few seconds, which is consistent with behavioral study results (5, 16). The spine enlargement was accompanied by an increase in the 2pEPSC (fig. S3). A similar DAopto timing was observed when we induced STDP through electrical stimulation of presynaptic fibers (fig. S4).

Pharmacological studies revealed that D1R-MSN structural plasticity was dependent on N-methyl-d-aspartate receptors (NMDARs), Ca2+/calmodulin–dependent protein kinase II (CaMKII), and protein synthesis (Fig. 2A and fig. S5A), suggesting that the molecular mechanisms for D1R-MSN plasticity are similar to those underlying structural plasticity in hippocampal pyramidal neurons (1012). Plasticity also depended on D1R and protein kinase A (PKA), but not on D2R (Fig. 2B and fig. S5A). Spine enlargement was prevented by an inhibitory peptide blocking the interaction of dopamine- and adenosine 3′,5′-cyclic monophosphate (cAMP)–regulated phosphoprotein 32 kD (DARPP-32) with protein phosphatase 1 (PP-1) (17), but not its control peptide (Fig. 2, C and D). Moreover, spine enlargement was induced even in the absence of DAopto when PP-1 was inhibited by calyculin A (fig. S5, B and C). These results suggest that similar to hippocampal preparations (18), the phosphorylation of DARPP-32 by PKA would inhibit PP-1 and disinhibit CaMKII.

Fig. 2 Pharmacology of spine enlargement induced by STDP plus DAopto with a 0.6-s delay.

(A) Time courses of spine enlargement induced by STDP + DAopto with a 0.6-s delay in the absence (control, 24 spines, 7 dendrites) and presence of NMDAR antagonist (50 μM D-AP5, 22 spines, 6 dendrites), CaMKII inhibitor (3 μM KN62, 23 spines, 6 dendrites), or protein synthesis inhibitor (5 μM anisomycin, 25 spines, 6 dendrites). (B) Time courses of spine enlargement in the presence of D1R antagonist (3 μM SCH23390, 23 spines, 6 dendrites), D2R antagonist (10 μM sulpiride, 22 spines, 6 dendrites), or PKA inhibitor (10 μM PKI, in the pipette, 24 spines, 6 dendrites). (C) Time courses of spine enlargement in the presence of inhibitory (100 μM, in the pipette, 24 spines, 6 dendrites) or control peptide for DARPP-32 (100 μM, in the pipette, 24 spines, 6 dendrites). (D) Averaged volume changes in the absence and presence of the compounds. Data are presented as mean ± SEM. P = 3.4 × 10−6 with Kruskal-Wallis and *P = 0.023 (AP5), 0.023 (KN62), 0.037 (AIP) (fig. S5A), 0.023 (anisomycin), 0.035 (SCH23390), 0.023 (PKI), 0.037 (KT5720) (fig. S5A), and 0.023 (DARPP-32 inhibitory peptide) with Steel test.

To test whether changes in Ca2+ signaling account for DAopto timing (19), we imaged increases in cytosolic Ca2+ concentrations ([Ca2+]i) in spines using a low-affinity calcium indicator Fluo-4FF (KCa = 10 μM) to avoid saturation. The [Ca2+]i increases gradually built up and quickly waned after cessation of the STDP protocol (Fig. 3A). We found that DAopto did not affect Ca2+ transients (Fig. 3, B to D, and fig. S6, A to C), indicating that Ca2+ signaling modulation did not play a major role.

Fig. 3 DAopto effects on STDP stimulation-induced increases in [Ca2+]i and CaMKII activities.

(A and B) Increases in Fluo4-FF fluorescence, representing [Ca2+]i increases, within single spines in response to a train of STDP stimulation in the absence [(A), 20 spines, 15 dendrites] and presence of DAopto with a 0.6-s delay [(B), 15 spines, 8 dendrites]. Blue laser irradiation during DAopto is blanked by the blue bar. (C) STDP and DAopto protocols for Ca2+ and CaMKII imaging. Unlike plasticity induction (Fig. 1N), only one train was applied. (D) No effect of DAopto on the peak values of [Ca2+]i (fig. S6, A to C). P = 0.91 with Kruskal-Wallis test. (E and F) Ratiometric imaging with Camuiα-CR during STDP stimulation in the absence [(E), 33 spines, 14 dendrites] or presence of DAopto with a 0.6-s delay [(F), 42 spines, 14 dendrites]. Relative increases in the ratio are shown as pseudocolor coding in (E). (Bottom) Time courses of FRET ratios in the spines stimulated with glutamate uncaging or the neighboring spines. Scale bars, 1 μm. (G) Dependence of the peak Camuiα-CR FRET ratios on the DAopto delay (fig. S6, D to F). P = 1.3 × 10−5 with Kruskal-Wallis test and ***P = 8.4 × 10−5 with Steel test versus those without DAopto. (H) Normalized increases in Camuiα-CR ratios by STDP + DAopto with a 0.6-s delay in the stimulated (42 spines, 14 dendrites) and neighboring spines (42 spines, 14 dendrites), and in the presence of DARPP-32 inhibitory peptide in the stimulated spines (43 spines, 10 dendrites) (fig. S6G). Data are presented as mean ± SEM. P = 8.8 × 10−9 with Kruskal-Wallis test and *** P = 2.6 × 10−9 and 1.1 × 10−6 with Steel test.

We examined whether CaMKII activation was related to DAopto timing by using the Förster resonance energy transfer (FRET) indicator of CaMKII, Camuiα-CR (2022), which was virally transfected into D1R-MSNs. CaMKII activation was weak in the absence of DAopto (Fig. 3E) but was greatly potentiated by DAopto (Fig. 3F), with timing similar to DAopto timing for spine enlargement (Fig. 3G and fig. S6, D to F). This enhancement was specific to the stimulated spine (Fig. 3H), abolished by the inhibitory peptide for DARPP-32 (Fig. 3H and fig. S6G), and mimicked by PP-1 inhibitors, calyculin A and tautomycetin (fig. S6, H and I). These results support the hypothesis that PKA/DARPP-32 disinhibits CaMKII via PP-1 (fig. S11).

We addressed whether the DAopto timing was formed at the level of PKA activation by using a FRET probe of PKA, AKAR2-CR (22), which was virally delivered to the D1R-MSNs. Unlike structural plasticity or Camuiα-CR activation, PKA activation in response to stimulation of a dendritic spine by STDP and DAopto was not restricted to the stimulated spine; neighboring spines also exhibited significant PKA activation (fig. S7, A to D). Even without glutamate uncaging, PKA activation was observed in the spine and dendritic shaft (Fig. 4A) in an AP-dependent manner (Fig. 4B), suggesting that PKA activation is a cell-wide phenomenon. When DAopto was applied at various times relative to APs, we obtained a timing (Fig. 4C and fig. S7, E to I) similar to DAopto timing on spine enlargement and CaMKII activation (Fig. 1O). The extent of spine enlargement positively correlated with that of PKA activation (Fig. 4D). APs themselves were not sufficient to activate PKA (fig. S8, C to E). The contingency between APs and DAopto might be detected by Ca2+/calmodulin–dependent adenylyl cyclase 1 (AC1), which is synergistically activated by Ca2+/calmodulin and Gs (23). Consistent with this, AC1 blocker (NB001) eliminated AKAR2 activation, as well as structural plasticity (fig. S9, A to D).

Fig. 4 AP effects on DAopto-induced PKA activation in proximal and distal dendrites.

(A and B) Images and time courses of FRET ratios of AKAR2-CR in the spine and distal dendrites stimulated by APs + DAopto with a 0.6-s delay [(A), 153 spines, 25 dendrites] or DAopto only [(B), 158 spines, 26 dendrites]. The relative increases in the ratio were pseudocolor coded as shown in (A). Scale bar, 2 μm. (C) AKAR2-CR responses to APs + DAopto with various delays (fig. S7, E to I). P = 5.3 × 10−10 with Kruskal-Wallis test and *P = 0.012, ***P = 1.3 × 10−6 (0.6 s), and 4.0 × 10−4 (1 s) with Steel test versus DAopto only. (D) AKAR2-CR response (C) plotted against spine volume changes (Fig. 1O) for various DAopto timings. The Spearman’s correlation coefficient was 0.94, and P = 0.0048. The blue and gray bars indicate the dynamic ranges of volume changes predicted by the dynamic ranges of AKAR2 responses at dendritic spine (blue) and soma (gray). (E) AKAR2-CR responses at the soma and first, second, and distal dendrites in response to DAopto only and DAopto + APs with a 0.6-s delay (fig. S8, A and B).

In the soma and proximal (first branch) dendrites, however, DAopto alone was sufficient to activate PKA (Fig. 4E and fig S8, A and B) (24), and APs could only slightly enhance PKA. We predict that APs might have modulated spine enlargement to a small degree in these regions, if there had been spines (Fig. 4D). Thus, DAopto-induced PKA activation must be suppressed in distal dendrites in order to attain the large dynamic range for the timing detection. In fact, when phosphodiesterase 10A (PDE10A), the major phosphodiesterase in MSNs, was blocked by its inhibitor papaverine (25), PKA activations were similarly induced in distal dendrites as in the soma (fig. S9, E to G). Papaverine also disrupted the time window for structural plasticity (fig. S9, H and I). Why were PDE10A actions particularly potent in the distal dendrites? Subcellular differences in PDE10A expression might not account for this, considering that PDE10A is expressed at the plasma membrane and is uniformly distributed along the dendrites (26). Instead, we found a negative correlation between dendrite diameter and DAopto-induced PKA activity (fig. S10A, blue), which was lost when the phosphodiesterase was inhibited (fig. S10A, orange). Therefore, PDE10A might counteract the increases in cAMP more potently in the thin distal dendrites because of its high surface-to-volume ratios (fig. S10, B and C). Spines are only found in the distal thin dendrites of MSNs (fig. S10, D to E) (27), suggesting that spines are distributed to be efficiently modulated by dopamine timing in MSNs.

It has been enigmatic why dopamine reinforces preceding, but not subsequent, sensorimotor events. If dopamine always activates PKA, its effects should last long enough to reinforce the subsequent events over tens of seconds (Fig. 4A). However, dopamine did not activate PKA unless [Ca2+]i primed AC1 to outcompete the high phosphodiesterase activity in thin dendrites (fig. S11). Our data show that [Ca2+]i priming should occur strictly before dopamine delivery (0 s) (Figs. 1O, 3G, and 4C), which is reminiscent of serotonin’s action in the classical conditioning of the siphon withdrawal reflex in Aplysia (28), in which serotonin, carrying an aversive signal, was only effective when it was preceded by increases in [Ca2+]i for activation of calcium-dependent adenylyl cyclase (AC) and PKA in presynaptic terminals (29). The delay in [Ca2+]i priming of AC may compensate the time lag of monoaminergic signals after reward or punishment. Our data suggest that reinforcement plasticity occurs at the single spine level, even though PKA activation is a cell-wide phenomenon, in such a way that dopamine regulates the gain of NMDAR-dependent Hebbian plasticity via CaMKII activity (fig. S11). This interdependence between Hebbian and reinforcement plasticity has been implicitly assumed in the reinforcement learning theory, in which the Hebbian term is used for the credit assignment (6, 30), and the dopamine timing in our study corresponds to the eligibility trace that determines the time window for reward action (30). Thus, we have clarified a molecular and cellular basis of reinforcement plasticity at the level of single dendritic spines.

Supplementary Materials

www.sciencemag.org/content/345/6204/1616/suppl/DC1

Materials and Methods

Figs. S1 to S11

References (3139)

References and Notes

  1. Acknowledgments: We thank K. Kaibuchi, M. Mishina, T. Tanabe, K. Doya, J. Wickens, T. Toyoizumi, and K. Kasai for helpful discussion; K. Deisseroth for the channelrhodopsin-2 plasmids; M. Z. Lin for the Camuiα-CR and AKAR2-CR plasmids; K. Iwamoto and S. Jinde for confocal microscopy; M. Ikeda for technical assistance; and M. Ogasawara and M. Tamura for laboratory facility. This work was supported by the Strategic Research Program for Brain Sciences (“Bioinformatics for Brain Sciences” to H.K. and S.I.) and Grants-in-Aid (2000009 and 26221011 to H.K.) from the Ministry of Education, Culture, Sports, Science, and Technology and by the National Institutes of Health GM53395, DA035612, and NS069720 (G.C.R.E.-D.). This work was also supported by the Human Frontier Science Program (H.K.). G.C.R.E.-D. has a patent in the United States for the synthesis of dinitroindolinyl-caged neurotransmitters.
View Abstract

Navigate This Article