Modulation of Human Visual Cortex by Crossmodal Spatial Attention

See allHide authors and affiliations

Science  18 Aug 2000:
Vol. 289, Issue 5482, pp. 1206-1208
DOI: 10.1126/science.289.5482.1206


A sudden touch on one hand can improve vision near that hand, revealing crossmodal links in spatial attention. It is often assumed that such links involve only multimodal neural structures, but unimodal brain areas may also be affected. We tested the effect of simultaneous visuo-tactile stimulation on the activity of the human visual cortex. Tactile stimulation enhanced activity in the visual cortex, but only when it was on the same side as a visual target. Analysis of effective connectivity between brain areas suggests that touch influences unimodal visual cortex via back-projections from multimodal parietal areas. This provides a neural explanation for crossmodal links in spatial attention.

Spatial attention picks out particular locations for further sensory processing. Most studies of spatial attention have considered only a single sensory modality at a time (1, 2), but crossmodal links have now been demonstrated psychophysically. For instance, a tactile cue at one location can improve discrimination for visual stimuli at that location relative to others (3, 4). This crossmodal effect arises even if the tactile cues are task-irrelevant and do not predict the location of the visual targets, suggesting an exogenous (stimulus-driven) attentional mechanism. The neural basis of these crossmodal effects in humans remains unknown. It has generally been assumed that they affect only multimodal neural structures (5–7), but recent accounts suggest that back-projections from multimodal areas to unimodal areas may play a role (8–10).

We used event-related functional magnetic resonance imaging (fMRI) (11) to test whether tactile stimuli can spatially influence unimodal visual areas via back projections. On each trial, the participants (12) received visual targets in either the left or right hemifield in a manner that was unpredictable. On a randomly chosen half of the trials, this visual stimulus was coupled with concurrent tactile stimulation to the right hand (i.e., at the same external location as any right visual stimulation) (13). Four event types were organized in a 2 by 2 factorial design. One factor was the side of visual stimulation (right or left). The second factor was the occurrence of right tactile stimulation (present or absent). Our analysis first established the effect of lateralized visual stimulation on contralateral occipital areas. We then tested whether responses within these areas were modulated by the presence of tactile stimulation (14).

Figure 1 shows activations due to the side of the peripheral visual stimulation. As expected, activations were found in contralateral occipital areas. The main effect of left hemifield visual stimulation was a cluster of activation in the posterior part of the right lingual gyrus, contralateral to the visual targets. The maximum activation was at x, y,z = 30, –84, –14 (Z score = 4.8;P-corrected = 0.005). The reverse comparison showed that stimulation in the right hemifield also activated the posterior part of the lingual gyrus, now in the left hemisphere (x,y, z = –24, –74, –10; Z score = 5.1;P-corrected <0.001) (15).

Figure 1

Effect of side of the visual stimulation. The effect of peripheral visual stimulation was projected on a rendered view of the canonical MNI brain template. The cerebellum was removed and the brain was tilted to allow a direct view of the ventral surface of the occipital lobes. (A) Main effect of left versus right visual stimulation. Comparison of left versus right events (with and without tactile stimulation) revealed activation of the right lingual gyrus, contralateral to the visually stimulated side. The right middle and superior occipital gyri showed a similar trend. (B) Main effect of right versus left visual stimulation. This comparison also revealed activation of the lingual gyrus, now in the left hemisphere, contralateral to the stimulated hemifield. The cluster was symmetrical to the ventral activation observed in the right hemisphere for the opposite comparison. Additionally, significant contralateral responses were observed (15) in the left middle and superior occipital gyri (not visible in this projection).

These symmetrical activations in the lingual gyri define a brain response that is sensitive to the side of our visual stimulation. We then examined the effect of concurrent right tactile stimulation on these contralateral responses by testing for the interaction between the side of the visual stimulation and the presence of touch. The analysis revealed amplification of visual response to lights on the right side by touch on the same side. The left lingual gyrus showed a significant interaction, with the maximum at x,y, z = –18, –82, –6 (Z score = 3.2,P-corrected = 0.044) (Fig. 2). Thus, the left lingual gyrus not only responded to right visual stimulation, but did so more strongly with concurrent tactile stimulation on the right (compare red curve with blue curve in left graph of Fig. 2C). Such modulation was not simply due to this brain area responding directly to touch. The crossmodal amplification was spatially specific because bimodal stimulation that was spatially incongruent (i.e., adding right tactile to left visual stimulation) did not cause any signal increase in the same area (see magenta curve in right graph of Fig. 2C).

Figure 2

Crossmodal effects in the left lingual gyrus. An interaction between side of the visual stimulation and the presence of right somatosensory input was detected in the left lingual gyrus. We found signal amplification when the right visual stimulus was coupled with the right tactile stimulation. (A) Group results of activation rendered on the brain surface. (B) Size of the interaction effect for each subject (with SE). The search volume for each subject-specific maximum was restricted to the same region as for the group effect (i.e., areas showing a main effect of side of the visual stimulation). The plotted effect is the weighted sum of parameter estimates from the multiple regression. For the interaction shown, this weighting corresponds to participants receiving right visual stimuli with right tactile stimuli versus those without right touch, minus participants receiving left visual stimuli with right tactile stimuli versus those without. All effects are scaled to SE units. (C) Estimated signal in the cluster showing significant amplification of response to right visual stimuli by right touch. The shape of the curves reflects the function used to fit the data. The effect size (scaled to SE units) is the sum of the parameter estimates for the six participants for each of the four events. For each event type, the parameters were estimated by fitting the mean signal across the activated voxels.

By contrast, in the right lingual gyrus, responses to left visual stimuli tended to show a reverse pattern of suppression by right touch, though this was not significant. In a follow-up study, tactile stimulation was delivered to the left hand instead. This produced mirror-image results with significant crossmodal attentional effects now in the right lingual gyrus, which showed maximal response for left visual stimulation combined with left touch (16).

The lingual gyrus has long been considered a unimodal visual area, with no tactile afferent inputs. This agrees with the lack of any main effect from tactile stimulation in this region (Fig. 2C). A main effect of adding right touch (i.e. regardless of side of visual stimulation) was found only in the left superior postcentral gyrus and postcentral sulcus (x, y, z = –44, –40, 56; Z score = 4.8; P-corrected <0.001) and in the parietal operculum (x, y, z = –64, –22, 16; Z score = 5.6; P-corrected <0.001). All these tactile activations occur in somatosensory areas (17) contralateral to the stimulated hand; no such overall increase with touch was found anywhere in the occipital lobe.

What is the source for the crossmodal spatial effect in the lingual gyrus? We hypothesized that tactile information reaches the occipital lobe via back-projections from higher multimodal areas. We performed a second analysis, testing for condition-dependent changes in “effective connectivity” (18). This analysis highlights changes in the coupling between brain areas (i.e., the contribution of one area to the fMRI signal measured in a different area), which depend on the experimental condition. We anticipated that areas mediating the observed crossmodal influence should increase their effective connectivity with the left lingual gyrus only during the spatially congruent bimodal stimulation, for which the amplification in the left lingual gyrus had been observed.

We tested for any such increase of effective connectivity with the left lingual gyrus specific to the spatially congruent bimodal stimulation across the whole brain. Only a few circumscribed areas showed the predicted increase in effective connectivity, indicating a highly specific network. The most robust effect was in the right inferior parietal lobe (x, y, z = 52, –22, 34; Z score = 4.0), in the anterior part of the supramarginal gyrus (Fig. 3A). This area has been suggested to be the homolog of area 7b in monkeys (19), where single-unit electrophysiology has demonstrated the presence of visuo-tactile neurons with spatially corresponding bimodal receptive fields (20). The right-hemisphere lateralization of this activation agrees with previous proposals that, in the human parietal lobe, the right-hemisphere plays a dominant role in spatial representation and attention (21), consistent with studies of deficits in neurological patients (22).

Figure 3

Changes in effective connectivity with the left lingual gyrus. Such changes were revealed by testing for condition-specific changes in the contribution of any other brain areas to the fMRI signal in the left lingual gyrus (18). The right supramarginal gyrus in the inferior parietal lobe (A) demonstrated the most significant change in effective connectivity with the left lingual gyrus. This inferior parietal region showed higher coupling during bimodal, spatially congruent trials than during bimodal, spatially incongruent trials. A similar pattern was found in the left postcentral gyrus (B), which is the somatosensory area contralateral to the stimulated hand. For both brain areas, effects for individual participants are plotted on the right side of the figure (scaled to SE units).

The left postcentral gyrus (x, y,z = –30, –34, 56; Z score = 3.3), which is the somatosensory cortex contralateral to the stimulated right hand, also showed condition-specific increases in coupling with the visual area in the lingual gyrus (Fig. 3B) during spatially congruent bimodal stimulation. The only additional areas to show increased coupling were the right posterior parietal lobule (x, y,z = 16, –68, 52; Z score = 3.5) and left lateral occipital gyrus (x, y, z = –34, –82, 16; Z score = 3.7). Reciprocal connections between occipital cortex, posterior parietal cortex, and multimodal areas in the inferior parietal lobe might provide the anatomical substrate for the crossmodal interaction observed in the lingual gyrus. Although there are no known direct connections from the supramarginal gyrus (or area 7b in monkeys) to posterior ventral occipital areas, pathways may exist via posterior parietal areas (23, 24).Tactile information from the postcentral gyrus (i.e., contralateral somatosensory cortex) might thus be transferred to occipital areas (contralateral visual cortex) via back-projections from parietal areas. This process depends on whether the visual and tactile stimuli are on the same side.

Our findings allow several conclusions. First, they demonstrate that spatial cueing in one modality can affect activity in structures that only receive afferent input from a different modality. Second, such crossmodal influences of touch on the visual cortex depend on the spatial relation between visual and tactile sensory input. Finally, the analysis of effective connectivity suggests that tactile input to the somatosensory cortex may influence the visual cortex via back-projections through association areas in the parietal lobe. This accords with recent theoretical proposals (8) that such back-projections may play a crucial role in crossmodal links in spatial attention.

  • * To whom correspondence should be addressed. E-mail: e.macaluso{at}


View Abstract

Navigate This Article