Report

Role for Cingulate Motor Area Cells in Voluntary Movement Selection Based on Reward

See allHide authors and affiliations

Science  13 Nov 1998:
Vol. 282, Issue 5392, pp. 1335-1338
DOI: 10.1126/science.282.5392.1335

Abstract

Most natural actions are chosen voluntarily from many possible choices. An action is often chosen based on the reward that it is expected to produce. What kind of cellular activity in which area of the cerebral cortex is involved in selecting an action according to the expected reward value? Results of an analysis in monkeys of cellular activity during the performance of reward-based motor selection and the effects of chemical inactivation are presented. We suggest that cells in the rostral cingulate motor area, one of the higher order motor areas in the cortex, play a part in processing the reward information for motor selection.

The cingulate motor areas (CMAs) of primates reside in the banks of the cingulate sulcus in the medial surface of the cerebral hemisphere and are subdivided into rostral and caudal parts (1). Anatomical studies have revealed prominent afferent input to the CMAs from the limbic structures and the prefrontal cortex, which can send information about motivation and the internal state of subjects, as well as cognitive evaluation of the environment (2,3). The CMAs send output to the primary and secondary motor areas and other motor structures in the brainstem and spinal cord (4). The CMAs are thought to be in a pivotal position to process the information necessary to select voluntary actions in accordance with the subject's internal and external requirements because of this anatomical connectivity (5–7). However, it is not yet known exactly how the CMAs are used or how individual cells behave in relation to the actual performance of motor tasks requiring motor selection based on reward evaluation. Therefore, we devised an experimental model of reward-based motor selection and analyzed cellular activity in the CMAs.

We trained three monkeys (Macaca fuscata) to perform two different arm movements, either pushing or turning a handle, in response to a visual trigger signal (8, 9). The essence of the motor task was that the animals voluntarily selected one of the two movements based on the amount of reward. During a series of constant-reward trials, they kept selecting a particular movement. If the reward was reduced, they chose to perform the alternate movement. We used conventional single-cell recording techniques to record cellular activity from the rostral and caudal cingulate motor areas [CMAr and CMAc; see (10)], as well as from the primary motor area. We confirmed the cortical recording sites on the basis of histological and physiological criteria (1, 7, 10). As reported previously, the CMAr and CMAc are found in the upper and lower banks, and their transitional areas, of the cingulate sulcus. The neurons described here were recorded at sites in the CMAs that (i) project to the primary motor cortex (11) and (ii), when stimulated, evoke limb movements (12).

We found that four types of cells in the CMAr exhibit changes in activity during the interval between the occurrence of the reduced reward and the initiation of a new selected movement. The first type of cell had phasic activity that began 200 to 600 ms after the occurrence of the reduced reward, and well before the monkey initiated the alternate movement for the next trial, as shown in Fig. 1A (left panel). The same cell did not respond to the reward as long as the reward was not reduced (right panel). The second type showed a long-lasting change in activity (>1000 ms following the reduced reward; Fig. 1B, left) that also built up rapidly, but decayed before initiation of the next movement. The third type had a rapid build-up in activity starting more than 200 ms after the reduced reward, with little decay before initiating a new movement on the basis of the reduced reward (Fig. 1C). The fourth type showed a more gradual increase in activity that peaked near the initiation of the movement selected after the reduced reward (Fig. 1D). Interestingly, none of these types of cellular activity was observed when the reward was reduced, but the monkey did not select the alternate movement (Fig. 1E). These four types of cells appear useful in relaying information about the reward reduction to the process of selecting the next movement, at least in the simplified model of motor selection we present.

Figure 1

(Left panels) activity of four types of CMAr cells (A through D) showing increased discharges after the reward was reduced and before initiation of a newly selected movement. Each raster corresponds to a single trial when the animal switched from Turn to Push (A and B) or Push to Turn (C and D), with 5 to 12 trials in between. (A) Short-lasting activity after the reward. (B) Long-lasting but decaying activity. (C) Continuous activity. (D) Activity that increases progressively before the next movement. (E) Discharges of the same CMAr cell as shown in (A) when the reward was reduced, but the monkey did not select the alternate movement. (Right panels) The same four cells were not as active under the constant reward condition when subjects could not select an alternative movement. In the raster displays, dots represent individual discharges of a single cell, and small crosses denote the onset of movement. In the histograms, discharges over 11 trials are aligned at the onset of the reward and summed, except for (D), which is aligned at onset of next movement. A step in the ordinate denotes 10 spikes per second. A horizontal bar in the top right of a raster display indicates the time range of occurrences of the next movement.

In the CMAr, cellular activity of the types described above (selective relation to reduced reward/motor selection) was found in 81 (37%) of 221 task-related cells in the CMAr (Table 1) (13). A prominent property of these cells was that the majority (n = 55, or 68%) of the activity depended on which of the two movements the subjects selected, and thus was differential [P < 0.01; (14)]. A typical example of the differential activity is shown in Fig. 2. The cell is active after the reduced reward (Fig. 2A), but the increase in activity is selective to trials when the monkey selected Turn as the next movement (right panels), changing from the previous movement of Push, but not when the change was the reverse (left panel) (15).

Figure 2

(A) Activity of a CMAr cell exhibiting an increase in discharge after the occurrence of the reduced reward, but only when the monkey selected the Turn movement after changing from the Push movement (right), and not when the next selection after the reduced reward was Turn (left), or (B) when the reward remained unchanged. (C) The same cell did not respond to the tone signal that instructs the animal to change movements.

Table 1

Distribution of cells in CMAr and CMAc related to different aspects of motor task. Prep., preparatory; Pre-, premovement; Post-, postmovement.

View this table:

To test the possibility that the cellular activity may be related nonspecifically to a signal requiring the animal to change future movements, rather than specifically to the reward information, we added a control task. For the control task, a tone signal (1 kHz, 300 ms) in the waiting period told the animal to change the future movements, thereby imposing animals to select an alternative movement involuntarily. The cell shown in Fig. 2C (same cell as in Fig. 2A) did not respond to the tone signal, even when the animal changed from Push to Turn (right). The same test was performed while recording from 46 CMAr cells that responded to the reduced reward. A great majority (n = 41) responded selectively to the reduced reward rather than to the tone signal, indicating a preferential relationship between the CMAr cells and the reward-based motor selection. In the remaining cells, however, the response was nonselective. These cells may be involved in the shift of future movements, much as cells reported in the presupplementary motor area (16).

In the CMAr, other types of cells showed activity related to movement initiation (n = 38), the period of preparation for the next movement or movements (n = 74), or the occurrence of the reward nonselectively (constant or reduced, n = 16). In contrast, the activity of cells in the CMAc was mostly related to movement initiation or motor preparation. Only four CMAc cells showed activity related to motor selection (Table 1). The proportion of cells that were active in motor selection was significantly less in the CMAc than in the CMAr (P < 0.001 by chi-square test). In the primary motor cortex, we analyzed the activity of 114 task-related cells; none exhibited the properties shown in Fig. 1 or 2 (17). The premovement preparatory activity was slightly dissimilar (P < 0.05) in only two cells, depending on whether the subject selected the alternate movement.

If the CMAr is crucially involved in the process of reward-based motor selection, then deactivation of this area is likely to impair the ability to select an appropriate movement. This was exactly what we observed by reversible inactivation of the CMAr with topical application of muscimol, a γ-aminobutyric acid agonist (18). When 3 to 4 μl of muscimol was injected bilaterally in the forelimb part of the CMAr, the monkey began to fail to select a correct movement 10 to 15 min after injection. Even if the reward was reduced considerably, the monkey kept selecting the previously performed movement and failed to select the alternate movement (Fig. 3). At other times, the animal made a mistake and prematurely selected the alternate movement before the reward was reduced. These effects, observed at six injection sites in the forelimb part of the CMAr, were dose-dependent and not observed with a concentration of less than 5 μg/μl. The effects were not observed when we injected muscimol bilaterally into the hindlimb representation area of the CMAr or into the forelimb part of the CMAc. Furthermore, it was confirmed in four additional injection experiments that the animal had no problems in selecting the alternative movements when the alteration was cued with the tone signal (19), despite the failure in the reward-based motor selection.

Figure 3

Effects of muscimol injection into the CMAr on the task performance visualized with a serial display (from top to bottom) of the time course of consecutive performance of either Push or Turn movement, with each row representing individual trials. Triangles, squares, and circles indicate the onsets of trigger signals, movements, and reward delivery, respectively. After completing each movement, the handle was quickly returned to the hold zone mechanically, where the monkey had to hold it until the next trigger signal appeared. After the muscimol injection (right), the animal often failed to select the alternate movement despite a considerable reward decrease (yellow symbols) or made a mistake of selecting the alternative movement too early, before the reward decrease (green symbols).

Previous brain-imaging studies in humans have shown that the region presumably corresponding to the CMAr is particularly active when a variety of motor tasks require subjects to voluntarily select movements (7, 20). Our findings determined that the CMAr is indeed crucially involved in voluntary motor selection. Our study also indicates that the CMAr contains cells active in relation to motor selection based on the amount of reward. Such cells seem to be profoundly involved in processing information about assessing the reward obtained by executing a current movement and selecting the next movement if the reward is not satisfactory. Anatomical studies have revealed connections from the amygdala and ventral striatum to the anterior cingulate cortex (area 32) and the cingulate gyrus (21), which in turn project to the CMAr (2, 3). These limbic projections provide ample information about reward values that are directly connected to the goals of motor acts. On the other hand, the direct or indirect pathways from the prefrontal cortex to the CMAr (2, 3) are able to transmit information concerning short-term memory about the occurrence of events during the performance of a motor task in the previous trial (22). Thus, the CMAr is ideally situated to combine information from the two sources, and our results suggest a way in which CMAr cells make use of the combined information to select an appropriate motor act.

  • * To whom correspondence should be addressed. E-mail: tanjij{at}mail.cc.tohoku.ac.jp

REFERENCES AND NOTES

View Abstract

Navigate This Article