Report

Temporal and Spatial Enumeration Processes in the Primate Parietal Cortex

See allHide authors and affiliations

Science  08 Sep 2006:
Vol. 313, Issue 5792, pp. 1431-1435
DOI: 10.1126/science.1130308

Abstract

Humans and animals can nonverbally enumerate visual items across time in a sequence or rapidly estimate the set size of spatial dot patterns at a single glance. We found that temporal and spatial enumeration processes engaged different populations of neurons in the intraparietal sulcus of behaving monkeys. Once the enumeration process was completed, however, another neuronal population represented the cardinality of a set irrespective of whether it had been cued in a spatial layout or across time. These data suggest distinct neural processing stages for different numerical formats, but also a final convergence of the segregated information to form most abstract quantity representations.

Humans and animals share an evolutionarily ancient quantification system that allows them to approximately estimate the size of a set (its numerosity) without verbal symbols (14). How numerical information can be extracted depends on the presentation format: whether the elements of a set are perceived simultaneously or sequentially. When presented simultaneously as in multiple-item patterns, numerosity can be estimated from a spatial arrangement at a single glance. Here, parallel processing mechanisms are engaged, as indicated by constant reaction times (57), equal numbers of scanning eye movements (7), and comparable neural response latencies (8) across absolute set size. In contrast, when the elements of a set are presented one by one, they need to be enumerated successively across time (911). This latter process constitutes a nonverbal precursor of real counting, which is a sequential enumeration process via number symbols.

Both human (1221) and monkey (22, 23) studies relate the processing of numerical quantity information to the posterior parietal cortex, including the intraparietal sulcus (IPS). However, none of these studies addresses the actual “counting” aspect, namely the abstract accumulation of sensory events one by one. Moreover, it remains unknown whether and how numerical information extracted from temporally and spatially arranged presentations is combined at the neuronal level.

To investigate the role of individual IPS neurons in representing simultaneously and sequentially presented visual quantity, we trained monkeys to discriminate small numerosities. The monkeys had to judge whether two successive task periods (first sample, then test) separated by a 1-s delay contained the same number of items (one to four) (Fig. 1, A and B). If so, the animals had to release a lever. In the sample period, the numerosity was presented randomly: either by multiple-dot patterns showing the number of items simultaneously (the simultaneous protocol, Fig. 1B) or by single dots appearing one by one to indicate the number of items in sequence (the sequential protocol, Fig. 1A) (24). We ensured that non-numerical parameters (visuospatial cues in multiple-dot patterns and temporal cues in the sequential presentation protocol) could not be used by the monkeys to solve the task. Controls in the simultaneous protocol included displays in which the circumference (and with it total area), density, and configuration (shapelike or linear) were controlled across different quantities. Controls in the sequential protocol eliminated temporal factors that may covary with increasing numbers of sequential items, such as the total duration of the sample period, the duration of individual items and pauses in between, the total visual energy (or total area across time, respectively), and the regularity (rhythm) of the item sequence (see Table 1 and fig. S1 for details). In the test period, numerosity was always cued by multiple-item displays.

Fig. 1.

Task protocols and behavioral performance. (A) Sequential delayed match-to-quantity task (for example, numerosity 3). A trial started when the monkey grasped a lever. The monkey had to release the lever if the sample period and test display contained the same number of items but continue holding it if they did not (probability of match/nonmatch condition = 0.5). The sample numerosity was cued by sequentially presented items temporally separated by pauses containing no items. The temporal succession and duration of individual items were varied within and across quantities (Table 1). The numerosity in the test period was always cued by multiple-item displays. (B) Simultaneous delayed match-to-quantity task. Task conditions were identical to those in the sequential protocol, but numerosity was cued by a single multiple-dot display in the sample period. The physical appearance of the displays varied widely for the same quantities (Table 1). For both the simultaneous and sequential task protocols, the nonmatch stimuli for sample numerosity 1 was 2; for sample 2 the nonmatch numerosities were 1 and 3 (probability = 0.25); for sample numerosity 3 and 4, nonmatches were one and two numbers up and down. (C and D) Behavioral performance of both monkeys during transfer tests in the sequential task. The monkeys performed a baseline discrimination of 2 versus 4 [green data for different stimulus protocols (I to III) are displayed as listed in Table 1] and 4 versus 2 (blue). Both monkeys [(C) and (D)] spontaneously discriminated sequential numerosity 3 from 2 and 4 in transfer tests without reinforcement (red). (E) Average performance of both monkeys in the simultaneous and sequential tasks (under standard and control conditions) during the recording sessions. Chance performance = 50% (also see fig. S2 for details).

Table 1.

Stimulus presentation protocols and variation of non-numerical parameters with quantity (w.q.). The timing of the sample period in the sequential protocol was as follows: Standard protocol: sample period duration for all numerosities, 2450 ms; single item/pause duration for numerosity 1, 409 to 1328 ms; numerosity 2, 372 to 1359 ms; numerosity 3, 229 to 1066 ms; numerosity 4, 212 to 607 ms. Equal sample duration protocol: sample period duration for all numerosities, 2450 ms; single item/pause duration for numerosity 1, 2450 ms; numerosity 2, 816 ms; numerosity 3, 490 ms; numerosity 4, 350 ms. Equal item/pause duration protocol: single item/pause duration for all numerosities, 350 ms; sample period duration for numerosity 1, 350 ms; numerosity 2, 1050 ms; numerosity 3, 1750 ms; numerosity 4, 2450 ms. For trials with a sample period duration of 3450 ms, values were correspondingly adjusted.

Simultaneous protocol
Stimulus type Spatial layout Surface area Circumference Density
Standard RandomizedView inline Increasing w.q. Increasing w.q. Increasing w.q.
Circumference RandomizedView inline Decreasing w.q. Equal Increasing w.q.
DensityView inline RandomizedView inline Increasing w.q. Increasing w.q. Equal
Configuration Linear Increasing w.q. Increasing w.q. Increasing w.q.
Sequential protocol
Stimulus type Sample period duration Individual item or pause duration Regularity (rhythm) Intensity over time
Standard Constant Decreasing w.q. Irregular Variable
Equal sample duration Constant Decreasing w.q. Regular Decreasing w.q.
Equal item/pause duration Increasing w.q. Constant Regular Increasing w.q.
  • View inline* Density was determined by calculating the average distance between the dots.

  • View inline High probability that three dots were arranged as a triangle, four dots as a quadrangle, and five dots as pentagon.

  • The monkeys were first trained on the simultaneous task alone and subsequently on the sequential numerosity discrimination. Initially, they only learned to discriminate sequential numerosity 2 from 4 (and vice versa). To investigate whether the monkeys would understand the concept of sequential numerosity without further training, we introduced transfer tests requiring the monkey to discriminate sequential numerosity 3 from 2 and 4 (see SOM for details). Because the animals were randomly rewarded for their responses in transfer tests, any learning of the “correct” response was impossible. Both monkeys immediately discriminated sequential numerosity 3, with a precision comparable to the baseline discrimination of 2 versus 4 (as indicated by the fact that the slopes for the match and nonmatch discrimination in the transfer and baseline tests are almost parallel) (Fig. 1, C and D). The numerical distance between match and nonmatch in the transfer tests is only one, which is more difficult to discriminate than the baseline discrimination with a numerical distance of two (25).

    After these transfer tests, monkeys had to perform both the simultaneous and the sequential task within a single session. The average performance of the two monkeys on both the sequential and simultaneous protocols was between 71 and 99% correct (Fig. 1E) and significantly better than chance for all tested quantities (binomial test, P < 0.001; see fig. S2 for detailed performance curves). Discrimination of the quantity of sequentially presented items was more difficult for both monkeys (binomial test, P < 0.01). The monkeys readily generalized to the control stimulus sets; performance was very similar across them in the sequential and simultaneous protocols (Fig. 1E and fig. S2).

    We recorded from 228 randomly selected neurons in the depth of the IPS (Fig. 2, A and B) of two monkeys performing the numerosity discrimination task. Random presentation of either the sequential or simultaneous protocol from trial to trial allowed us to investigate an individual neuron's responses to both presentation types in an unbiased way. A proportion of the tested neurons [27 out of 228 (27/228) or 12%, two-way analysis of variance (ANOVA), with factors (stimulus protocol) × (sample numerosity), P < 0.01] showed activity that varied significantly with the number of items during sample presentation in the simultaneous protocol, irrespective of the displays' visuospatial properties (23). Even more neurons (58/228 or 25%, two-factor ANOVA, P < 0.01) showed maximum discharge in response to a certain number of dots in the sequential protocol. To further test whether the neurons' discharges to the preferred sequential item were unaffected by temporal parameters, we computed a multiple regression analysis with the spike rate to the preferred sequential item as a dependent variable and the duration of the preferred item in a sequence, the duration of the pause preceding the preferred item, and the duration of the previous-to-preferred item as independent variables (24). Only 8 of the 58 sequential numerosity-selective neurons exhibited a significant correlation (P < 0.01) with temporal parameters and were thus excluded from the pool of numerosity-selective neurons (fig. S4). Based on the combined results from the two-factor ANOVA and the multiple regression analysis, 50 neurons (from a total of 228 cells; that is, 22%) were found to be selective to sequential quantity only. Two example neurons tuned to sequential numerosity are shown in Fig. 2. The tuning function of the neuron in Fig. 2, C to E, showed peak activity for the sequential quantity 2 and a systematic dropoff of activity as the number of items in the sample period varied from the preferred value (Fig. 2D); this was true even in trials with three or four consecutive dots and varied sequence timing (Fig. 2E). A neuron with preferred numerosity 4 is plotted in Fig. 2, F to H.

    Fig. 2.

    Recording sites and neural responses during sample presentation. (A) Lateral view of the left hemisphere of a monkey brain indicating the topographical relationships of cortical landmarks. ips, intraparietal sulcus; ls, lateral sulcus; sts, superior temporal sulcus. (B) Coronal section at the level of the dotted line in (A) reconstructed from a structural magnetic resonance imaging scan (Horsley-Clark coordinates 0 mm anterior/posterior). The colored area in the depth of the IPS marks the recording area. (C to E) Responses of an example neuron selective to the sequential quantity 2 (only the “equal item/pause duration” protocol is shown for clarity). The top panel illustrates the temporal succession of individual items (square pulses represent single items). The corresponding latency-corrected discharges for many repetitions of the protocol are plotted as dot-raster histograms (middle panels; each dot represents an action potential) and averaged spike-density functions (bottom panels; activity averaged and smoothed). The first 500 ms represent the fixation period. Colors correspond for the stimulation illustration and the plotting of the neural data. Gray shaded areas represent item presentation. (D) Rate functions indicate the mean activity of the neuron in (C) to the standard and equal item/pause duration protocols [error bars in (D), (E), (G), and (H) represent the standard error of the mean] for four sequential dots. In both protocols, the neuron was tuned to numerosity 2. (Responses to the first item in a sequence of one, two, three, or four items were not statistically different.) (E) The same neuron shown in (C) was significantly tuned to numerosity 2 irrespective of whether the sample period showed two, three, or four sequential items (standard and control protocols pooled). (F to H) Neuron tuned to sequential numerosity 4. (F) Neuronal responses for the control protocol [layout as in (C)]. (G) Rate functions show monotonic increase of discharges up to numerosity 4 for both protocols. (H) Comparable discharges of this neuron to the sequential items, irrespective of whether the items were presented in sequences of two, three, or four items.

    Similar response profiles were observed for all neurons tuned to numerosities 1, 2, 3, or 4 in the sequential (see population tuning curves in Fig. 3A) or simultaneous (Fig. 3B) protocol. Each cell showed peak activity for one of the visual quantities and a systematic dropoff of activity as the number of sample items varied from the preferred value (Fig. 3C). Numerosity 1 was the most frequent preferred numerosity in both protocols (Fig. 3D).

    Fig. 3.

    Normalized responses averaged for neurons preferring the same sample quantity for the sequential (A) and simultaneous (B) protocols, respectively. (C) All neurons showed a progressive dropoff of the response with increasing numerical distances from the preferred numerosity, resulting in averaged tuning functions that were comparable for neurons tuned to sequential or simultaneous presentation. In (A) to (C), error bars represent the standard error of the mean. (D) Frequency distributions of the preferred numerosities for the sequential and simultaneous protocols. (E) Venn diagram illustrating the distributions of numerosity-selective neurons in the sample period. Some of the neurons exhibiting numerosity selectivity in the simultaneous protocol were also tuned in the sequential protocol during item presentation and/or during the pauses (indicated by graphical overlap), albeit mostly with different preferred numerosity. Numbers indicate the number of neurons per set.

    Many of the tested neurons (43/228 or 19%) also showed activity that varied significantly during the pauses between item presentation in the sequential protocol (Fig. 1A), irrespective of the temporal arrangement [again tested with a combination of a two-factor ANOVA and a multiple linear regression analysis; see supporting online material (SOM) and fig. S5 for details]. More than half of the neurons that were significantly tuned during the pauses were also tuned during sequential item presentation (25/228 or 11%); thus, the firing rates during item presentations as well as during the pauses between them varied with the position in the sequence. The neuron in Fig. 2F, for example, exhibits a significant increase in discharge during successive pauses, with the highest discharge during the third pause in the sequence. At the same time, this neuron has the preferred item position “four” (Fig. 2, G and H). We found a significant correlation between the neurons' preferred serial position during the pauses and the preferred number of sequential items (r = 0.70, P < 0.001, n = 25 neurons), indicating that the activation of neurons between items could provide a neuronal storage mechanism to keep track of the actual items to enumerate. Such an accumulation of activation was successfully implemented in neural networks simulating numerosity detection (26).

    A comparison of neurons tuned to numerosity during the sample period in the sequential and simultaneous protocols showed a clear dissociation of neuronal populations. Only 10 neurons (4% of the total sample) were significantly tuned to numerosity in both protocols. Of those 10 cells, only 2 (1% of the total sample) were tuned to the identical numerosity in both the sequential and simultaneous protocols (see neuron in Fig. 2F and fig. S3), which corresponds to chance occurrence. This finding suggests that different populations of neurons are engaged in extracting numerosity in a parallel or serial fashion, respectively (Fig. 3E).

    Only after the enumeration process in the sample period was completed did the animals have full information about the cardinality of a set. They had to maintain the quantity in mind throughout a delay period and prepare to find the matching quantity in a test display. Many neurons (43/228 or 19%) were significantly tuned to numerosityonlyinthememory period, irrespective of the presentation protocol (only significant numerosity effect, two-factor ANOVA with numerosity and presentation protocols as factors, P < 0.01). For example, the neuron displayed in Fig. 4 showed remarkably similar delay activity in the sequential (Fig. 4A) and simultaneous (Fig. 4B) presentation protocol, with 3 as the preferred numerosity (Fig. 4C). The average response profiles of all numerosity-selective neurons during the delay period are shown in Fig. 4D. Few cells (9/228 or 4%) were tuned to numerical quantity but also differentiated between the simultaneous and sequential presentations (numerosity and presentation protocol effect or interaction, two-factor ANOVA, P < 0.01). Some neurons tuned to the set size in the delay period represented quantity in the sample period (simultaneous protocol: 7/43 or 16%; sequential protocol: 9/43 or 21%).

    Fig. 4.

    Neural responses during the delay period. (A to C) A single neuron showing remarkably similar delay activity in the sequential (A) versus simultaneous (B) presentation protocol, with 3 as the preferred numerosity. Top panels in (A) and (B) show color-coded dot-raster histograms; bottom panels are the corresponding spike-density histograms. The first 500 ms represent the fixation period. This neuron was not numerosity-selective in the sample period. (C) Tuning function of the displayed neuron based on averaged discharge rates calculated over the delay period. (D) Normalized responses averaged for all neurons preferring the same quantity in the delay period, irrespective of the presentation protocol (sequential and simultaneous). In (C) and (D), error bars represent the standard error of the mean.

    An examination of error trials suggested that the delay activity of IPS neurons was directly related to the monkeys' performance. When monkeys made judgment errors, neural delay activity for the preferred numerosity was significantly reduced to 83.6% of that observed on correct trials (i.e., 100%) (P = 0.01, Wilcoxon signed ranks test, two-tailed).

    These results argue for segregated processing of simultaneous and sequential numerical quantity. Different populations of neurons were involved in extracting numerosity across spatial or temporal arrangements during an ongoing quantification process. In contrast, the final and common result of the quantification process was coded by a third population of neurons, irrespective of whether numerosity was cued simultaneously or in sequence. Thus, the intermediate numerosity of an ongoing quantification process and the storage of the final cardinality are accomplished by different neuronal populations.

    In contrast to a direct, perceptual-like assessment of numerical information in multiple-item displays, sequential enumeration requires a more complex coding of numerical information. Our data point toward the pauses between individual successive items as a potential key mechanism for the coding of sequential numerosity. For many neurons, activation changes during the pauses were sometimes more prominent than during the presentation of successive items (Fig. 2F). In these neurons, activation to the item presentation seemed to ride on ever-increasing discharges during the pauses, which is consistent with the idea of an accumulator mechanism (9, 26).

    The enumeration of sequentially presented items requires an organism to keep track of the serial position of the previously presented item. Similar to our findings in nonhuman primates, neuropsychological (27) and electrophysiological (28) studies in humans suggest dissociated processes involved in judging cardinality (numerical quantity) as opposed to ordinality (serial position), but sometimes with a common activation in the parietal and prefrontal cortices (14, 28). Neurons in the lateral prefrontal cortex of the monkey are also selectively tuned to numerical rank (29) and numerical quantity (8), but typically later than IPS neurons (23). This suggests that neurons in the posterior parietal and prefrontal cortices are linked to form a single functional network for the representation of numerical information across space and time.

    Supporting Online Material

    www.sciencemag.org/cgi/content/full/313/5792/1431/DC1

    Methods

    Figs. S1 to S7

    References

    References and Notes

    View Abstract

    Navigate This Article