Technical Comments

Response to Comment on “Single-trial spike trains in parietal cortex reveal discrete steps during decision-making”

+ See all authors and affiliations

Science  25 Mar 2016:
Vol. 351, Issue 6280, pp. 1406
DOI: 10.1126/science.aad3596


Shadlen et al.’s Comment focuses on extrapolations of our results that were not implied or asserted in our Report. They discuss alternate analyses of average firing rates in other tasks, the relationship between neural activity and behavior, and possible extensions of the standard models we examined. Although interesting to contemplate, these points are not germane to the findings of our Report: that stepping dynamics provided a better statistical description of lateral intraparietal area spike trains than diffusion-to-bound dynamics for a majority of neurons.

We organize our Response to Shadlen et al. (1) around four topics: (i) comparisons to other experiments, (ii) the integration behavior of our animals, (iii) alternative formulations of the drift-diffusion model, and (iv) interpretation of data from Roitman and Shadlen (2), followed by (v) technical comments.

(i) Shadlen et al. assert that our main finding in Latimer et al. (3) is inconsistent with other experiments and analyses. Their figure 1A shows saccade-aligned peristimulus time histograms (PSTHs) from a reaction time (RT) motion discrimination task. First, a PSTH (an average over trials) cannot provide definitive evidence about the dynamics governing firing on single trials, a primary point of motivation for our Report. Second, their figure 1A simply shows that spike rate steps are not precisely aligned with saccade times. Our Report made no assumptions and drew no conclusions about the relationship between spike rates and decisions or saccades—in fact, both stepping and ramping models were fit without knowledge of the animal’s choices. Moreover, we obtained an identical stepping result with spike trains from this very data set (2), indicating that there is nothing inconsistent about a finding of stepping dynamics with ramping saccade-aligned PSTHs.

Shadlen et al. then argue that lateral intraparietal (LIP) area activity during a different decision-making task (4) conflicts with our results. We fail to see the conflict: We did not analyze data from this task, and it is entirely conceivable that LIP neurons exhibit different dynamics in distinct contexts. Regardless, Kira et al. (4) analyzed population-averaged responses over many trials, and this cannot provide evidence for or against stepping single-trial dynamics. Although figure 1B from Shadlen et al. overlays spikes with instantaneous decision evidence, this spike train appears to be hand-picked, and no single-trial spike trains were analyzed in that paper.

(ii) Shadlen et al. argue that our behavioral data (5) might reflect brief or variable evidence accumulation. However, a drift-diffusion model (DDM) fit confirms that the monkey’s behavior during electrophysiological recordings met or exceeded conventional periods of integration (Fig. 1A), and is nearly indistinguishable from that in Kiani et al. (6) when assessed with shorter viewing durations in sessions outside of electrophysiology (Fig. 1B). The slightly lower accuracy observed during electrophysiology sessions is not evidence that the monkey employed two different decision-making strategies and instead is likely a simple consequence of differences in stimulus geometry. Specifically, our neural recording sessions employed unique target and dot-motion aperture locations and motion directions tailored to each neuron under study, whereas the shorter-duration behavior (shown in Fig. 1B) had consistent cardinal stimulus geometry across sessions. This difference in stimulus geometry strikes us as a more plausible account than Shadlen et al.’s suggestion that this monkey only employed conventional temporal integration when we were not recording from LIP. [Additionally, unlike many LIP studies in which dot motion is presented at central fixation—e.g., (2, 69)—we presented the dots peripherally, better encouraging broad spatiotemporal integration and avoiding individual dot tracking that is possible in high-resolution central vision.]

Fig. 1 Our animal’s behavior reflects a period of evidence accumulation as long or longer than durations reported previously.

(A) Monkey’s accuracy as a function of stimulus duration during neurophysiological recording sessions (dots) from (5) overlaid with the theoretical curves (solid lines) obtained from a maximum likelihood fit of a DDM. The median durations of behavioral integration under this model are 408, 362, and 152 ms across the range of motion strengths shown. These integration times are in fact longer than those recently reported in (10). A modest lapse rate in the DDM accounts for the asymptotic performance slightly below 100% [e.g.,(7,8,15] The other monkey (not discussed by Shadlen et al.) exhibits similar signatures of substantial evidence accumulation. (B) This animal’s behavior is also very similar to that of previous studies. Dependency of accuracy on viewing duration and motion strength [same conventions as in (A)] from a purely behavioral data set collected from the same monkey (5) (grayscale dots) using a range of shorter durations that closely match those used in a previous study (6) (colored dots). The data from the two studies are very similar (i.e., they overlap and follow matching forms), demonstrating that our monkey achieves performance on the random dots task nearly identical to that reported in a study by some of the authors of Shadlen et al.

(iii) Shadlen et al. call for an alternative diffusion-to-bound model with variable per-trial integration start times. We constructed our model to be faithful to previous formulations in the LIP literature. Shadlen and Kiani (10) stated, “There’s a reproducible starting time ~200 ms after the onset of motion.” Applications of the DDM for behavior typically use a fixed start time [e.g., (4, 9, 11)]. Many LIP studies analyze spike counts averaged across trials and neurons, thereby assuming that integration begins at the same time on each trial and in each neuron [e.g., (4, 12)]. Without this assumption, average spike rates would reflect a mixture of temporally shifted ramping trajectories and preintegration activity, instead of a coherent ramping process (13). Moreover, the claim that the fixed start time “unfairly penalized” the ramping model is incorrect. The stepping model does not have a flexible start time. Both models describe spike trains in terms of a conditionally Poisson process beginning at a fixed time after motion onset on every trial and evolves according to discrete stepping or continuous ramping dynamics for the remainder of the trial. We did test a range of start times for both models for every cell, with no noticeable changes to our results [figure S18 in (3)].

(iv) Shadlen et al. claim that our supplementary analysis of data from Roitman and Shadlen (2) included cells inappropriate for studying evidence accumulation. We are perplexed by this argument because it appears to conflict with conclusions about LIP function made by Shadlen et al. from this data set (point i above). All the cells that we analyzed were taken from the same data set used to construct Shadlen et al.’s figure 1A. Additionally, Shadlen et al. posit that trials with integration times under 150 ms biased our results, but we only analyzed trials with RTs of at least 350 ms [see the supplementary materials for (3)]. Shadlen et al. assert that the average ΔDIC (deviance information criterion) is small (ΔDIC = 19), but this value is clearly in favor of stepping; ΔDIC ≥ 10 is conventionally regarded as “strong” statistical evidence (14).

(v) Shadlen et al. assert that our ramping model simulations produce evidence for stepping. The figure in question [figure S6 in (3)] shows that for ramping simulations (based on our fits and trial counts from real data), the distribution of ΔDICs strongly supports ramping. A few small positive values indicated that the two models are not always identifiable given limited data (3 of 40 individual-neuron simulations yielded negligible evidence for stepping; ΔDICs < 3), which does not undermine the consistency of our analyses. Finally, Shadlen et al. argue that our model comparison is biased because we cannot “identify latent firing rates” in the ramping model. Frankly, we do not understand this remark. Our Bayesian fitting methods integrate over all possible latent rates consistent with the data, for both models.

In summary, (i) hypotheses about how LIP dynamics relate to decision formation are intriguing and worthy of future investigation but not relevant to our statistical analyses; (ii) our monkey behavior and modeling assumptions match previous studies of LIP, although we certainly welcome future standardization and generalization of experimental and theoretical protocols; and (iii) our reanalysis of data from Roitman and Shadlen still supported the stepping model. We stand behind the conclusions of our Report and believe that considering alternative hypotheses to integration will continue to be illuminating. We of course agree with Shadlen et al. that extrapolations of our original study’s characterizations of single-neuron spike trains to that of population-level dynamics and/or to decision-making would be premature. These important issues require consideration of richer multineuron data sets, which we have recently collected and are currently analyzing. The ongoing introduction of powerful new tools and data sets will likely bring continued constructive debate, and we share Shadlen et al.’s enthusiasm for testing and generalizing theories that link brain and cognition.


View Abstract


Navigate This Article