A Neural Correlate of Working Memory in the Monkey Primary Visual Cortex

See allHide authors and affiliations

Science  06 Jul 2001:
Vol. 293, Issue 5527, pp. 120-124
DOI: 10.1126/science.1060496


The brain frequently needs to store information for short periods. In vision, this means that the perceptual correlate of a stimulus has to be maintained temporally once the stimulus has been removed from the visual scene. However, it is not known how the visual system transfers sensory information into a memory component. Here, we identify a neural correlate of working memory in the monkey primary visual cortex (V1). We propose that this component may link sensory activity with memory activity.

We trained monkeys (Macaca mulatta) to perform a delayed-response task in which the animals had to remember briefly the location of a figure after it had been removed from the visual scene (1). The animals fixated on a small central red dot on a computer screen (Fig. 1). After a 300-ms fixation, a motion-defined figure appeared very briefly (28 ms) at one of three locations (Fig. 1). After this stimulus had been presented, the animal had to continue fixating the central spot until it was switched off (Fig. 1A, “Cue time”). The removal of the fixation point indicated to the animal to make a saccade toward the position where the figure had been presented. The animal was rewarded only when fixation was maintained until the cue, and when the saccade was made to the correct position. The latency of the cue time was varied between 0 and 2000 ms after stimulus onset. Thus, while fixating, the animal had to remember the location of the briefly presented figure during a period of up to about 2 s. Detection of the figure was high and declined for longer delay periods (Fig. 2A), indicating that the task requires short-term memory processes.

Figure 1

Experimental setup and illustration of a figure-ground display. (A) Animals were trained to fixate a central red spot (FP) on a stimulus screen filled with random dots. At stimulus onset a motion-defined figure (B) appeared for 28 ms. The animals maintained fixation for 0 to 2000 ms. After the offset of the fixation point (cue time), animals had to make a saccade toward the location where the figure had been presented (arrow in upper right panel). (C) The small circle indicates the locations of the receptive fields (RF). When the figure was overlying the RF, responses to “figure” were recorded. In the two other figure locations, “ground” responses were obtained.

Figure 2

Behavioral performance and neural responses in a delayed-response task with random dot motion–defined (A to D) and texture orientation–defined (E) figure-ground stimuli. (A) The percentage of correct trials for the different delay periods. (B) Neuronal responses (population average of all trials) to figure (thick lines) and to ground (thin lines). Contextual modulation is indicated by the shaded area. Dotted lines above the figure responses indicate the SEM. (C) Average modulation strength for the first 250-ms period after stimulus onset for each individual electrode is plotted against the average modulation strength of the 250-ms period before cue time. A linear regression curve is fitted through the data points. (D) Slopes of the regression curves in (C) for the different delay periods. Data of the 0-ms-delay period cannot be analyzed this way. (E) Neuronal responses for static figure-ground–defined texture. The first peak corresponds to the figure onset, and the second peak to figure offset/mask onset. Shading as in (B).

During the delayed-response task, multiunit activity of V1 neurons was recorded in two monkeys (2). The display was filled with random dots. Stimulus onset thus evoked neural responses for “figure” [when the figure dots were overlying the V1 receptive fields (RFs)] as well as for “ground” motion (when the figure was presented elsewhere and background dots covered the RF) (Fig. 1C). We arranged the directions of motion such that, on average, the motion stimuli on the RF were identical for the “figure” and “ground” situations (3, 4).

The initial responses to figure and ground motion were identical up to about 70 ms after stimulus onset (Fig. 2B). At longer latencies, however, the response to figure motion was typically stronger than to background motion. This late enhancement of the sensory response—contextual modulation—correlates closely with the perception of the figure (4–6). Contextual modulation in V1 depends on feedback from higher visual areas (7–9), which implies that it is a specific correlate of recurrent processing. What happens to this modulation once the stimulus is no longer present, but has to be remembered?

During the delay period, the figure response remained stronger than the ground response (Fig. 2B) (P < 10−4 for all delay periods). Thus, contextual modulation continues after the figure is removed from the visual field. In a control experiment, we observed the same phenomenon when a static, rather than a moving, stimulus was used in the same delayed-response task. Here, a static texture with an orientation-defined figure (10) was presented for 100 ms and followed by a mask containing a different texture, where the figure was no longer visible. Also in this experiment, contextual modulation continued during the whole period (900 ms) that the animal had to remember the figure location (Fig. 2E). Thus, the persistence of contextual modulation is not due to any peculiarity of the motion stimulus.

We calculated the strength of contextual modulation (3) for the first 250 ms after stimulus onset as an indication of the initial segregation strength of the figure from ground, and for the last 250 ms before cue time as an indication of the signal strength available for responding in the memory task. In the first part of the response, the strength of contextual modulation was similar for all conditions. However, the strength of contextual modulation at subsequent stages decreased with increasing delay length. To quantify the decline in modulation strength with increasing delay length, we plotted the initial modulation strength of the first 250 ms after stimulus onset against the modulation strength of the last 250 ms before cue time for each individual recording unit (Fig. 2C). For the 1000- and 2000-ms delay periods, there was a significant decline in modulation (P < 10−7). We then fitted a linear regression line through these data points for each delayed-response period. The slope declined for longer delay periods (Fig. 2D). This indicates that the strength of contextual modulation is related to the delay period, where modulation becomes weaker for longer delays.

To establish a direct correlation between the persistence of modulation and memory performance, we analyzed the data according to the report of the animal, i.e., correct and incorrect responses. In correct trials, the animal made a saccade toward the figure location within 500 ms after cue time. In incorrect trials, the animal failed to respond correctly within this period. We then calculated the strength of contextual modulation for correct and incorrect responses separately. Contextual modulation was present during the first 250-ms period after stimulus onset for both the correct and incorrect trials (Fig. 3). Subsequently, the average strength of contextual modulation decreased in the later parts of the delay period. The decrease of modulation was stronger for incorrect trials, and contextual modulation for the incorrect trials disappeared completely in the late part of the delay period (Fig. 3, B and D), whereas modulation for the correct trials continued (Fig. 3, A and C). Analysis of individual recording sites (Fig. 3, E and F) showed that the strength of the figure-ground signal during the first 250 ms after stimulus onset was similar for the correct and incorrect trails (P > 0.1 for both delay periods), but that most electrodes showed stronger modulation at the final 250 ms before cue time for correct trials compared with incorrect trials (P < 10−5 for both delay periods). Dividing the average modulation strength of the last 250 ms before cue time by the modulation strength of the first 250-ms period after stimulus onset gives the relative degree of modulation persistence during the delay period (Fig. 3, G and H). This persistence is much weaker (and in fact negative) for incorrect than for correct trials (P < 0.005 for both delay periods).

Figure 3

Neurophysiological responses and modulation strength for the 1000- and 2000-ms-delay period sorted according to behavioral result. (A to D) The population average neural activity for figure and ground for correct [(A) and (C)] and incorrect trials [(B) and (D)]. Shading as in Fig. 2. (E and F) Modulation strength during the given intervals of each recording unit for correct trials versus incorrect trials. (G and H) Population average of modulation ratios (modulation strength during the first 250 ms divided by modulation during the last 250 ms) for correct and incorrect trials.

Thus, for correct trials as well as for incorrect trials, the figure is equally well segregated from the ground, as indicated by the presence of contextual modulation in the first 250 ms after stimulus onset. However, during the delay period, modulation continues for correct trials and disappears completely for incorrect trials.

Another important feature of working memory is that relevant information is actively stored for later use, i.e., the information is stored only if necessary (11). To test whether the sustained modulation that we observed was an active process, we presented a second stimulus during the delay period (Fig. 4, A and B). The second stimulus appeared 500 ms after the onset of the first stimulus and remained on the screen until the end of the trial. The second stimulus could be either relevant or irrelevant to the delayed-response task. In the figure = target condition, the animals had to saccade to the remembered position of the first stimulus (as before), and the second stimulus was irrelevant. In the figure = distractor condition, the animals had to remember the position of the second stimulus, and the figure was irrelevant (12).

Figure 4

Experimental setup and figure-ground responses for a 1000-ms delayed-response task where the figure was either target or distractor. (A and B) The delayed-response task was as described in Fig. 1 except that a second stimulus (circle) was presented 500 ms after the figure onset (square). After the offset of the fixation point, animals had to make a saccade toward the figure location in the figure = target condition (A) or toward the second stimulus (circle) in the figure = distractor condition (arrows in upper right panels). (C and D) The average figure-ground responses for figure = target condition (C) and figure = distractor condition (D). Shading as in Fig. 2. (E) Strength of modulation for the 250-ms period after figure onset versus strength of modulation for the 250-ms period before cue time of each recording unit for the figure = target condition (solid circles) and for the figure = distractor condition (open circles). (F) Population average of modulation ratios (seeFig. 3, G and H) for the figure = target and figure = distractor conditions.

In both conditions, contextual modulation was present before the onset of the second stimulus (Fig. 4, C to E) (difference in modulation strength P > 0.1). The appearance of the second stimulus had a strong nonspecific effect on the activity of the recorded neurons, where the average responses to both figure and ground decreased after the onset of the second stimulus (Fig. 4, C and D). However, a very task-specific effect was observed on the difference between figure and ground responses: Contextual modulation continued in trials when the figure was the target, whereas it decreased when the figure was the distractor (Fig. 4, C and D). Again, we quantified the initial (0 to 250 ms) and the final (750 to 1000 ms) amount of modulation for each individual electrode (Fig. 4E), and calculated the relative degree of persistence of modulation during the delay period (Fig. 4F). The remaining amount of modulation is much stronger when the figure is relevant to the memory task than when it is not (P < 0.0005). These results indicate that the continuation of modulation is not a passive process but is related to an active storage of information needed for the animal's goal.

In the primary visual cortex, neural activity related to figure-ground segregation thus continues during the delay period in a memory guided task (Fig. 2). The persistence of figure-ground modulation is stronger when the task is performed correctly (Fig. 3) and when the figure evoking the modulation is relevant to the task (Fig. 4). The continued figure-ground signal is accompanied by an overall reduction in activity (see Figs. 2 to 4). Thus, whereas one mechanism suppresses overall activity in V1 during the delay, another mechanism seems to be able to maintain the difference in figure versus ground signals. In that sense, the maintained activity is different from delay-period activity recorded in temporal (13) or prefrontal (14) cortex. This difference may also explain why in functional magnetic resonance imaging studies, very little delay activation is found in early visual areas (15).

It is highly unlikely that the maintained activity is a neural correlate of visual persistence, i.e., the lasting visibility of a stimulus after its physical disappearance. In humans, visual persistence for motion-defined stimuli has been shown to last 40 to 130 ms after stimulus offset (16, 17). However, our data show that contextual modulation is present for up to 2 s after the removal of the figure (Fig. 2B). Maintained modulation occurs even when the display is replaced with a different stimulus (Fig. 2E).

Instead, our results show a strong correlation between the active and successful storage of information about (the location of) the stimulus and the maintained figure-ground signal. Whereas the first part of this activity may thus be related to the perceptual experience of the figure segregating from ground [see also (5, 6)], the later part is more likely related to a memory trace of the stimulus (11,13). We therefore propose that contextual modulation in the primary visual cortex is a correlate of the process that forms a bridge between sensory activity and working memory.

  • * To whom correspondence should be addressed. E-mail: h.super{at}


View Abstract

Stay Connected to Science

Navigate This Article