Differential Representation of Perception and Action in the Frontal Cortex

See allHide authors and affiliations

Science  16 Jan 2004:
Vol. 303, Issue 5656, pp. 380-383
DOI: 10.1126/science.1087788


A motor illusion was created to separate human subjects' perception of arm movement from their actual movement during figure drawing. Trajectories constructed from cortical activity recorded in monkeys performing the same task showed that the actual movement was represented in the primary motor cortex, whereas the visualized, presumably perceived, trajectories were found in the ventral premotor cortex. Perception and action representations can be differentially recognized in the brain and may be contained in separate structures.

Voluntary movements often begin as a reaction to a visual stimulus and then are monitored visually. The process underlying these movements can be considered serial: Stimulus perception is transformed in stages during the behavior until the desired goal is attained (1). During object tracing, for example, there is tight interplay between visual feedback and hand movement. Perception of the hand's location leads to a movement along a preregistered path.

Perceiving movements correctly is a key element of volitional behavior. Normally, the perception and generation of movement cannot be readily distinguished, but it is not uncommon for them to become dissociated. For instance, a new pair of bifocals shifts the visual environment so that it no longer matches one's internal model (neural representation) of the world established from a previous history of conjoint visual, vestibular, and proprioceptive sensation. The mismatch between vision and the internal model often leads to disorientation. When first wearing prisms, experimental subjects initially reach to the apparent displaced target, and then rapidly compensate for the error (2, 3). Presumably, the subject's internal model is being updated in the process.

We designed a motor illusion to dissociate perception of the movement from movement execution. Subjects working in a virtual environment saw a three-dimensional (3D) representation of their hand displacement but had no actual vision of their arm or hand. An elliptical tube, oriented horizontally in the frontal plane, was projected stereographically from a computer monitor and appeared to be floating shoulder-high in space. Hand position was tracked and represented in the graphic display with a sphere. Subjects placed the cursor inside the tube and then moved it to trace the object. Successful trials were made by pushing a marker band around the figure five times while maintaining contact between the tube and cursor. During the trials in which the illusion took place, the horizontal gain of the cursor, relative to the hand's position, was increased in the third and fourth circuits, forcing the subject to move the hand in a circular trajectory to trace the virtual oval during the fifth circuit. The gain increase was gradual and undetected by the subjects. Even though their arm trajectories were circular during the last circuit, human subjects reported making only oval trajectories during the task (4). We consider this an illusion in which proprioceptive sensation and vision conflict.

Assuming that aspects of the transformation from visual percept to movement are serial, detecting stages of this process in different brain structures might be possible. Experiments with prism adaptation point to the ventral premotor cortex (PMv) as an important node in this putative processing chain. Transcranial magnetic stimulation of the premotor cortex interrupts learned prism adaptation (5). Positron emission tomography experiments have shown increased metabolism in the portion of the parietal cortex projecting to the premotor cortex during prism adaptation (6). In macaques, inactivation of the PMv with muscimol led to specific loss of adapted prism behavior, whereas the inactivation of the dorsal premotor cortex had no effect (7).

Cell activity in the primary motor cortex (M1) is related to the production of purposeful movement. Activity patterns of these cells, when combined, accurately predict the details of arm trajectory (810). Firing rates of individual cells are also modulated by imposed loads, joint angular velocity, arm position, acceleration, and movement amplitude (1115). However, M1 has nonmotor attributes as well because units recorded in this region are modulated by parameters in tasks such as mental rotation, remembered sequences, and maze-solving (1618).

We used the motor illusion paradigm to ask whether firing patterns in the monkey M1 or PMv were better related to the visualized or actual movement of the arm. The same behavioral paradigm was used in monkey and human experiments. Perceptual experiments require a report from the subjects. Although we have no such report from the monkeys, and therefore cannot make direct claims of what the animals perceived, we will still refer to the movement-vision mismatch condition in the monkey experiments as an illusion. This terminology is based on reports from humans that used the same paradigm (4), prism adaptation experiments showing analogous results in humans and monkeys, and the concept that the movements visualized in the mismatch condition are congruent with the perceived motion. Our finding that the perceived motion matches the visual display is consistent with experimental results showing that vision is dominant when mismatched with proprioception (19, 20).

Single-unit responses were recorded in the targeted cortical areas as four tasks were performed. Upon isolation of a unitary response, each animal performed the “3D center-out task.” Then, while isolation of the same unit was maintained, the animal drew five circuits of an oval, five circuits of a circle, and two circuits of an oval, with a gradual transition to circles in the last three circuits during the illusion condition (21). Each of these tasks (center-out, oval, circle, and illusion) was repeated five times in a block before the next task was performed.

Firing rates of M1 and PMv units were cyclically modulated as monkeys performed the illusion task [supporting online material (SOM) text, “Single-unit analysis”]. Movement direction is a large determinant of firing rate in these units, which tend to fire maximally in a single “preferred direction” and less so as the angle between movement and the preferred direction increases. The relation between direction and discharge rate was modeled with a cosine function. In this experiment, each unit's tuning function was derived from data collected during the center-out task. This model was then used during the drawing tasks to get a “predicted discharge rate” based on the instantaneous movement direction of the hand. Examples of these rates from an M1 and a PMv unit are shown in Fig. 1.

Fig. 1.

Actual (dashed) and predicted discharge rates for an M1 and a PMv unit during the illusion task.

The units showed five cycles of modulation corresponding to the five circuits of the figure. The actual discharge rate led the movement-derived predicted rate for the M1 unit but lagged the predicted rate for the PMv cell. The leads and lags between actual and predicted discharge rates for all units were measured by cross-correlating the two signals and finding the shift that optimized the correlation coefficient (Fig. 2). Although there is a large range of leads and lags with an extensive overlap for both cortical areas, the histogram for M1 is unimodal with the peak at about 75 ms, whereas the PMv distribution appears bimodal with peaks at about ±170 ms. The direction-related discharge in M1 tended to precede the corresponding movement. Based on the histogram, PMv activity fell into two groups of intervals, one that showed that cortical discharge preceded the hand movement (+) and the other that followed it (–).

Fig. 2.

Histograms of time intervals between predicted and actual discharge rates.

Trajectories were extracted with population vectors constructed with firing rates from cells of each cortical area recorded during each drawing task. The analysis was carried out on individual repetitions of the five circuits (cycles). Each circuit was separated from the continuous drawing of five cycles and divided into 100 bins with a population vector calculated for each bin. The population vectors were then added tip to tail to form a neural trajectory (SOM text, “Population analysis”).

The trajectories from the illusion task in Fig. 3 are arranged by circuit. Hand trajectory (blue) changed from an oval in circuits 1 and 2 to a circle in cycle 5 as the cursor gain changed in circuits 3 and 4. The green trace, showing the path of the cursor as visualized by the subject, matches the hand trajectory for the first two circuits and remains oval-shaped through the remaining three. The neural trajectories (red) from the two different cortical areas represented different aspects of the task. The M1 population output matches the hand path through all five circuits. Both the hand and neural trajectories were circular in circuit 5. In contrast, the neural trajectories calculated from PMv units matched the visualized cursor path. The PMv and cursor trajectories were oval-shaped in the fifth circuit when the illusion was fully in place.

Fig. 3.

Illusion task trajectories. Top row is five cycles from M1 units. Bottom row is from the PMv. The hand trajectory is blue, cursor trajectory is green, and neural trajectory is red. Each displayed trajectory is the mean across five repetitions.

These data were analyzed across repetitions. The correspondence to the illusion was measured by calculating the eccentricity (horizontal amplitude/vertical amplitude) of each trajectory (22). Oval templates presented to the subject had an eccentricity of 1.8. Circles had an eccentricity of 1.0. Because the animal only had to touch the outside of the cursor sphere to the outside of the projected template tube, the sum of template tube and cursor radius gave a tolerance of 2.5 cm. This allowed the animal to successfully complete the task with a range of hand eccentricities. In the control tasks, where only circles or ellipses were drawn, the mean eccentricities were 1.2 and 1.6 respectively. During the illusion task, the eccentricity of the hand path decreased from 1.6 to 1.1. In this task, the eccentricities of the motor cortical neural trajectories (Fig. 4, error bars show 95% confidence) matched those of the hand for the M1 units. In contrast, the eccentricities of the PMv trajectories matched those of the cursor path.

Fig. 4.

Eccentricities of illusion and control trajectories. Error bars are the 95% confidence intervals calculated across five repetitions of the task.

Eye movements tracked during these experiments showed that the eyes tended to saccade between the horizontal extremes of the figures. They did not move smoothly along the outline of the figure, in contrast to the hand movement (23). This makes it unlikely that our results are a direct function of either eye movement or gaze.

During the illusion task, the visualized arm path is represented in the discharge pattern of PMv cells at the same time that the actual trajectory is contained in the M1 population activity. Behavioral analyses suggest that two separate visual systems may be operational in volitional arm movement (24). The first, termed “vision for action,” is relatively immune to illusion and used to map visual targets to movement coordinates. The other system, “vision for perception,” leads to perception or conscious registration of visual objects. Our results show that the PMv activity is consistent with the behavioral category of vision for perception, whereas the M1 has responses congruent with vision for action. The PMv projects to area 12 (25) and to area 46 (26) in the prefrontal cortex as well as to the M1. Area 12 is considered to be near the top of the ventral stream hierarchy for object recognition, and area 46 has the same presumed position in the dorsal stream (spatial processing) (27). In this regard, the PMv can be considered a node that is common to two systems; it may be a link between object-based perception and the resulting object-dependent movement. Cell activity in the prefrontal cortex was associated with the topological features of drawn figures (28, 29). During drawing, the PMv-prefrontal pathway may be used for comparing dynamic, incremental information of the movement's progress against a more static topological construct. This corresponds to the idea that the prefrontal cortex is critical for making behavioral predictions or guesses based on remembered sensory information, analogous to an internal model (24).

The appearance of a bimodal lag distribution in the PMv (SOM text, “Timing”) suggests that two subpopulations may exist in this region: one group related to late visual feedback and an earlier group related to a feed-forward motor command signal. The dissociation of vision for action and vision for perception might be supported if the two PMv groups projected differentially, for instance to M1 and area 46. However, when the PMv population was divided into two parts, based on the histogram in Fig. 2 (21), the neural trajectories of both subpopulations matched the illusory hand path. Interestingly, when the same procedure was carried out on the M1 population, the “late” neural trajectory had an eccentricity that fell between those of the actual and illusory trajectories, whereas the “early” population output matched the actual hand path. Although these populations are small, the results suggest that there may be a subset of cells in the M1 that are influenced by the visual display or perception of the movement in a way that is similar to those in the PMv. This subset of M1 cells that encode the trajectory after it takes place may receive PMv input. If so, the representation provided by these cells appears to be subordinate to the overall M1 population. Although perception and action may be more or less represented in different neuronal populations and even separated by cortical area, the neural operations that take place to form a percept or that lead to a movement decision are problems we can now address with the combination of these behavioral methods and population-based neurophysiology.

Supporting Online Material

Materials and Methods

SOM Text

Figs. S1 to S6

Tables S1 and S2

References and Notes

View Abstract

Navigate This Article