Report

Coding the Location of the Arm by Sight

See allHide authors and affiliations

Science  01 Dec 2000:
Vol. 290, Issue 5497, pp. 1782-1786
DOI: 10.1126/science.290.5497.1782

Abstract

Area 5 in the parietal lobe of the primate brain is thought to be involved in monitoring the posture and movement of the body. In this study, neurons in monkey area 5 were found to encode the position of the monkey's arm while it was covered from view. The same neurons also responded to the position of a visible, realistic false arm. The neurons were not sensitive to the sight of unrealistic substitutes for the arm and were able to distinguish a right from a left arm. These neurons appear to combine visual and somatosensory signals in order to monitor the configuration of the limbs. They could form the basis of the complex body schema that we constantly use to adjust posture and guide movement.

Without an accurate sense of the position of the limbs, head, and torso, we would be unable to guide movement, process the spatial location of nearby objects, or distinguish our own body parts from external objects. People with damage to their parietal lobes can have difficulty in all of these dimensions (1, 2). Studies in normal humans show that the body schema is not simply a representation of joint angles, but a complex integration of vision, proprioception, touch, and motor feedback (3–6). Although a great deal is known about the processing of joint angle and muscle stretch in the somatosensory system (7), little is known about how different sensory modalities are combined by neurons in the parietal lobe or elsewhere to construct the body schema (8, 9).

The present set of studies focused on the coding of static arm position. The sense of arm position depends on many sources of information, including proprioception and vision (3–6,10–12). Here we show that neurons in parietal area 5 of the monkey brain, but not in the primary somatosensory cortex, respond in relation to the seen position of a false arm. They are also sensitive to somatosensory signals, responding in relation to the felt position of the monkey's actual arm. These somatosensory and visual signals are combined in individual neurons to provide a possible code for static limb position.

Responses of single neurons in area 5 were studied in two monkeys (13). The recording site in monkey 1 is shown in Fig. 1A, and the apparatus is shown in Fig. 1B. The arm contralateral to the recording electrode was outstretched, and the ipsilateral arm was held close to the body (not shown). The arms were covered with a black plastic plate. On top of the plate, a realistic false arm was placed in the monkey's view. This false arm was from a monkey of the same species and had been prepared by a taxidermist. The cut end was covered from view by a portion of the monkey's chair, and the arm extended from the region of the shoulder. We did not know if the false arm “fooled” the monkey, but we could study its influence on the behavior of neurons.

Figure 1

(A) Side view of the monkey brain showing the part of the superior parietal lobe studied (black area). In the horizontal cross section of the cortex, stripes show the recording site in area 5 of monkey 1, and stippling shows the anterior recording site presumed to overlap the primary somatosensory cortex (areas 1 and 2). (B) Diagram of the apparatus used for testing whether neurons are sensitive to the felt or seen position of the arm. The monkey's real arm was held in an adjustable arm holder covered from view while a realistic fake arm was in view. (C) The real arm and the visible fake arm (striped) were placed on the left or right, resulting in four experimental conditions. The monkey was trained to fixate on a central light-emitting diode.

As shown in Fig. 1C, two variables were manipulated: the monkey's real arm was placed on the left or right, and the visible false arm was placed on the left or right (14). The resulting four conditions were presented in an interleaved, pseudorandom order. Any effect of the position of the real arm on a neuron can be attributed to a somatosensory signal reaching the neuron; any effect of the position of the false arm can be attributed to a visual signal.

The result for one example neuron is shown in Fig. 2A. As for most cells, the activity of this neuron was unaffected by fixation. The mean activity over the 12.5-s trial is shown in Fig. 2B. The neuron was significantly affected by the position of the real arm, firing at a higher rate when the arm was on the left. This effect of static arm position has been described before and is common in area 5 (15–20). However, the neuron was also significantly affected by the position of the fake arm, firing at a higher rate when the monkey saw the fake arm on the left. This neuron therefore received both a somatosensory and a visual signal that matched in direction. The neuron combined the two cues about arm position in a simple fashion: the firing rate was highest when both the felt and seen positions were on the left and lowest when both the felt and seen positions were on the right.

Figure 2

Effect of the felt position of the real arm and the seen position of the fake arm on area 5 neurons. (A) Histograms showing the activity of one example neuron (12 trials per condition). The neuron fired more to the real arm on the left and also to the fake arm on the left. (B) The mean firing rate of the neuron shown in (A), averaged over the 12.5-s trial. The effect of the real arm position was significant (ANOVA, F = 207.53,P < 0.001), the effect of the fake arm position was significant (F = 11.53, P = 0.002), and the interaction was not significant (F = 0.609, P = 0.440). Error bars are the standard error. (C) Results from a neuron that preferred the real and fake arms on the right (real arm, F = 11.83, P < 0.001; fake arm, F = 15.98,P < 0.001; interaction, F = 0.02,P = 0.889). (D) Results from a neuron that showed an interaction between real and fake arm position (real arm,F = 225.10, P < 0.001; fake arm,F = 6.74, P = 0.013; interaction,F = 6.12, P = 0.018). (E) Results from a neuron tested with five positions of the real arm and two positions of the fake arm. Position 1 for the real arm is the same as the LEFT position in (A) through (D). Position 5 for the real arm is the same as the RIGHT position in (A) through (D). (F) The mean result for all 173 neurons tested. Before averaging, for each neuron, the data were expressed as a percentage of the maximum firing rate. For neurons that fired more to the real arm on the right, the data were left-right reversed. (Within-subjects ANOVA values are as follows: real arm, F = 277.31, P< 0.001; fake arm, F = 18.07, P < 0.001; and interaction, F = 12.01, P = 0.001.)

Data from another example neuron are shown in Fig. 2C. This cell fired at a higher rate to the placement of the real arm on the right and, correspondingly, to the sight of the false arm on the right (21). Data from a neuron for which the two signals interacted are shown in Fig. 2D; the visual effect of the false arm was present only when the real arm was on the right. This result indicates that the visual and somatosensory signals are not always additive but may be combined in a more complex fashion. Data from a neuron tested with five different positions of the real arm to obtain a tuning curve are shown in Fig. 2E.

Of 173 neurons tested, 29% showed a significant effect of the position of the fake arm (22). For the distribution of neuron types within area 5, see (23). The mean result for all 173 cells is shown in Fig. 2F. For those neurons that preferred the real arm on the right, the data were left-right reversed so that all neurons could be averaged together. Even though most of the neurons (71%), when tested individually, were not significantly affected by the position of the fake arm, the average response of the population of cells showed a significant effect. Across the sample, the preferred location for the fake arm matched the preferred location for the real arm (24).

In addition to a realistic false arm, some cells were tested with nonarm objects. One object was a rectangle of white paper the same length and width as the fake arm and clearly visible against the black background. The mean result for 20 neurons tested with the fake arm and paper rectangle is shown in Fig. 3A. The position of the fake arm had a large significant effect on the activity of the neurons, whereas the position of the paper rectangle had no significant effect.

Figure 3

Effect of different visual stimuli on area 5 neurons. (A) The mean result for 20 neurons tested with the fake arm and a white paper rectangle the same size as the fake arm. Error bars are the standard error. Before averaging, for each neuron, the data were expressed as a percentage of the maximum firing rate of the eight conditions. For neurons that fired more to the real arm on the right, the data were left-right reversed. The position of the fake arm significantly affected the activity of the neurons (real arm,F = 27.78, P < 0.001; fake arm,F = 22.54, P < 0.001; interaction,F = 1.89, P = 0.18). The position of the paper rectangle had no significant effect (real arm,F = 35.28, P < 0.001; paper rectangle,F = 1.78, P = 0.20; interaction,F = 0.039, P = 0.85). (B) The mean of 17 neurons tested with the fake arm and, on interleaved trials, an apple slice. The position of the fake arm significantly affected the activity of the neurons (real arm, F = 24.51, P < 0.001; fake arm, F = 5.53,P = 0.03; interaction, F = 4.80,P = 0.04). The position of the apple slice had no significant effect (real arm, F = 17.23,P < 0.001; apple slice, F = 0.20,P = 0.66; interaction, F = 0.17,P = 0.69). (C) The mean of 15 neurons tested with the fake arm and, on interleaved trials, the fake arm placed backward (hand toward shoulder and cut end extended outward). When the fake arm was oriented realistically, its position significantly modulated the activity of the neurons (real arm, F = 64.55, P < 0.001; fake arm, F = 20.12,P = 0.001; interaction, F = 5.1,P = 0.04). When the fake arm was backward, its position had no significant effect on the activity of the neurons (real arm,F = 16.41, P < 0.001; fake arm reversed, F = 1.40, P = 0.257; interaction, F = 0.52, P = 0.483). (D) The mean of 17 neurons tested on interleaved trials with the contralateral fake arm extending from the contralateral shoulder, the ipsilateral fake arm extending from the contralateral shoulder, and no visual stimulus. When the correct, contralateral fake arm was used, its position significantly modulated the activity of the neurons (real arm, F = 20.72,P < 0.001; fake arm, F = 12.73,P = 0.003; interaction, F = 3.50,P = 0.08). When the ipsilateral fake arm was used, its position had no significant effect on the activity of the neurons (real arm, F = 20.18, P < 0.001; mirror-reversed fake arm, F = 1.20, P = 0.290; interaction, F = 0.27, P = 0.608). (E) The mean of 20 neurons from monkey 2 tested on interleaved trials with the contralateral fake arm, palm up, extending from the contralateral shoulder and the ipsilateral fake arm, palm up, extending from the contralateral shoulder. When the correct, contralateral fake arm was used, its position significantly modulated the activity of the neurons (real arm, F = 34.06,P < 0.001; fake arm, F = 7.86,P = 0.011; interaction, F = 6.20,P = 0.022). When the ipsilateral fake arm was used, its position had no significant effect on the activity of the neurons (real arm, F = 35.97, P < 0.001; mirror-reversed fake arm, F = 0.06, P = 0.815; interaction, F = 0.391, P = 0.539). (F) The mean of data from (A) through (E). The mean for real arm left, realistic fake arm left is significantly higher than the mean for real arm left, unrealistic fake arm left (t= 3.78, df = 88, P < 0.001). The mean for real arm left, realistic fake arm right is significantly lower than the mean for real arm left, unrealistic fake arm right (t= –3.70, df = 88, P < 0.001).

A second group of cells was tested with a stimulus designed to attract the monkey's attention. Figure 3B shows the mean result for 17 cells tested on interleaved trials with the fake arm and a slice of apple placed at the location where the hand would have been. When the apple slice was present instead of the fake arm, the monkey made vocalizations and had difficulty performing the fixation task, tending to fixate on the apple. Despite the monkey's apparent interest in this stimulus, the position of the apple slice had no effect on the activity of the neurons, whereas the position of the fake arm had a significant effect.

A third group of cells was tested with the fake arm backward, such that the hand was near the shoulder and the cut end was extended outward. This stimulus therefore had the same color, texture, and size as the properly oriented fake arm. The result is shown in Fig. 3C. When the fake arm was in a realistic orientation, the neurons were significantly affected by its position. On interleaved trials, when the fake arm was backward, the neurons were not significantly affected by its position.

A fourth group of cells was tested with the ipsilateral fake arm extending from the contralateral shoulder. In this condition, the wrong hand appeared to be attached to the contralateral side of the body. This small visual difference, the mirror reversal of the hand, had a pronounced effect on the neurons, as shown in Fig. 3D. When the realistic, contralateral fake arm was used, the neurons were significantly affected by its position. When the unrealistic, ipsilateral fake arm was used, the neurons were not significantly affected by its position. Instead, the neurons behaved in essentially the same fashion as when no visual stimulus was present.

To further probe this ability of the neurons to distinguish the left arm from the right arm, we tested a fifth group of cells with the fake arm placed palm up. The real arm, out of view, remained palm down. As shown in Fig. 3E, the neurons were sensitive to the position of the contralateral fake arm extending from the contralateral shoulder in this realistic-looking palm-up posture. The neurons were not sensitive to the position of the ipsilateral fake arm extending unrealistically from the contralateral shoulder. This result indicates that the neurons are not merely sensitive to a hand for which the thumb points toward the right; instead, they can successfully distinguish the right from the left hand regardless of whether the hand is oriented palm down or palm up.

Taken together, these results suggest that neurons in area 5 encode the position of a visual stimulus that looks plausibly like the monkey's arm extending from the shoulder. Stimuli that do not match the normal body schema, even those that look like an arm, do not affect the neurons in the same way. Remarkably, the neurons are able to distinguish a left from a right arm on sight. A similar mechanism may exist in humans. When people view a picture of a hand and judge whether it is a left or right one, they appear to consult the configuration of their own hands (25), and the superior parietal lobe becomes active (26).

The data on the different fake-arm substitutes (paper rectangle, apple slice, backward fake arm, and ipsilateral fake arm) are combined in Fig. 3F. This graph shows that the neuronal activity was in some cases increased and in others decreased by the sight of the realistic fake arm extending from the shoulder. When the fake arm was on the neurons' preferred side, it increased the neuronal activity above the level obtained with an unrealistic arm (upward pointing arrow). In contrast, when the fake arm was on the neurons' nonpreferred side, it decreased the neuronal activity below the level obtained with an unrealistic arm (downward pointing arrow). This result suggests that both excitation and inhibition shape the neurons' visual tuning to arm position.

In humans, the visual sense of arm position is modifiable through experience. For example, if a person sees a rubber hand being stroked repeatedly with a brush and simultaneously feels his or her own hand being stroked, he or she reports the illusion that the rubber hand is his or hers and that the touch is located on the rubber hand (27). Data from an area 5 neuron tested in a similar fashion are shown in Fig. 4. In the first block of trials, shown in Fig. 4A, the cell showed an effect of the position of the real arm but not of the fake arm. We then tested the cell in a second block of trials. Between each trial, the experimenter used a paint brush to stroke the back of the fake hand in the monkey's view while the monkey's real hand was being stroked with another brush, out of view. The hands were stroked 10 times in succession before the next trial. As shown in Fig. 4B, the neuron became sensitive to the position of the fake arm. In a third block of trials, we stroked the real hand and the fake hand asynchronously between trials. As shown in Fig. 4C, the neuron was no longer sensitive to the position of the fake arm. We tested neurons with this procedure on only a few occasions to avoid permanently changing the effect of the fake arm on area 5 and thus interfering with the basic phenomenon of the study. However, of five cells tested, four showed a similar effect for stroking the fake and real hand. These results suggest that the visual sensitivity of area 5 neurons can be modified by experience in the same way that the body schema can be modified in humans.

Figure 4

The effect of tactile and visual experience on the responses of a neuron. (A) First block of testing, showing a significant effect for the position of the real arm but not for the fake arm (real arm, F = 54.29,P < 0.0001; fake arm, F = 0.002,P = 0.97; interaction, F < 0.001, P = 1.00). (B) Second block of testing. Between each trial, the real hand was stroked with a paint brush out of view and, synchronously, the fake hand was stroked in view, 10 times. As a result, the neuron became sensitive to the position of the fake arm (real arm, F = 48.49,P < 0.0001; fake arm, F = 6.65,P = 0.01; interaction, F = 2.98,P = 0.09). (C) Third block of testing. Between each trial, the real hand was stroked with a paint brush out of view, and asynchronously, the fake hand was stroked in view, 10 times. As a result, the neuron lost its sensitivity to the position of the fake arm (real arm, F = 32.99, P < 0.0001; fake arm, F = 0.06, P = 0.81; interaction, F < 0.001, P = 0.99).

We studied an additional 33 neurons in the anterior part of the superior parietal gyrus in monkey 1 (Fig. 1A). Although 22 (67%) of these neurons were significantly affected by the position of the real arm, none showed a significant effect from the position of the false arm. The mean result for the 33 neurons also showed a significant effect from the real arm only (28). These data suggest that, in the ascending somatosensory pathway from the periphery to area S1 and to area 5, the first stage at which visual information about arm position is integrated with somatosensory information is in area 5. Visual information about arm position does not appear to reach area S1, at least as measured by this procedure.

Visual processing in the primate cerebral cortex begins in area V1 in the occipital lobe and progresses through an array of higher order visual areas (29). A set of visual areas extending into the parietal lobe is thought to process location and movement of visual stimuli; another set of areas extending into the temporal lobe is thought to process color, texture, and shape (30). Recent evidence, however, suggests that these two types of processing are at least partially intermixed (31–34). The present study shows that neurons in parietal area 5 are not merely concerned with the location of a visual stimulus, but also with the identity of the stimulus. These visual properties may represent a mechanism for localizing the limbs in space. In the case of humans, similar properties of parietal neurons could in principal underlie the incorporation of external objects into the body schema, such as prosthetic limbs, tools, or the outer edges of a car.

  • * To whom correspondence should be addressed. E-mail: graziano{at}princeton.edu

REFERENCES AND NOTES

View Abstract

Navigate This Article