How Good Are Neuron Models?

See allHide authors and affiliations

Science  16 Oct 2009:
Vol. 326, Issue 5951, pp. 379-380
DOI: 10.1126/science.1181936

Opinions strongly diverge on what constitutes a good model of a neuron (13). Two lines of thought on this have coexisted for a long time: detailed biophysical models (of the style proposed in 1952 by the physiologists Alan Hodgkin and Andrew Huxley) that describe ion channels on the tree-like spatial structure of the neuronal cell (4), and simple “integrate-and-fire” models based on the much older insight that pulsatile electrical activity (known as an action potential or spike) is a threshold process. Electrophysiologists generally prefer the biophysical models, familiar with the notion of ion channels that open and close (and hence, alter neuronal activity) depending on environmental conditions. Theoreticians, by contrast, typically prefer simple neuron models with few parameters that are amenable to mathematical analysis. Earlier this year, following previous attempts at model comparison on a smaller scale (5), the International Neuroinformatics Coordinating Facility (INCF) launched an international competition (6) that allowed a quantitative comparison of neuron models.

The idea behind the INCF competition is that a good model can predict neuronal activity based on data that were not used for parameter tuning (see the figure). The competition included three in vitro and one in vivo data set. The in vitro data sets were assembled from classical electrophysiological experiments in which random electrical current was injected through an electrode into a pyramidal cell and an interneuron. The task was to predict for 13 (or 9, respectively) repetitions of the same injected current waveform, the exact timing of spikes in neuronal electrical activity evoked during a 22-s time span, based on the activity observed during the first 38 s of data collection. The winning submission correctly predicted 59.6% (or 81.6%, respectively) of the spike times of the two neurons, using a simple integrate-and-fire model with a moving threshold (7).

Most threshold models are point neuron models—they neglect dendritic morphology and reduce the neuron to an extensionless mathematical construct. However, the INCF competition included as a third challenge a double-electrode experiment, in which current injection into the neuronal cell body (soma) was combined with current injection into the apical dendrite located about 600 to 700 µm from the soma, enabling an intricate interplay between somatic and dendritic spike activity (8). Surprisingly, the best performance was achieved by a variant of a threshold model, enriched with two equations for the dendrite.


The same fluctuating input current is injected into a live neuron and a model neuron. The first few seconds of voltage time course (dark blue) are used to optimize the parameters of the model. Performance of the model is measured as the percentage of correctly predicted spikes in the final part of the stimulation. A threshold model generates spikes (brown) whenever the simulated voltage (light green) hits a dynamic threshold (dashed line).


The potential value of the competition is best illustrated for the in vivo data set, which allowed a reevaluation of previously published data from a neuron in the lateral geniculate nucleus of the brain (9). The winning submission (a threshold model) predicted the timing of 90.5% of the spike activity of this neuron (knowing its input, which was caused by visual stimulation of the retina), and thereby surpassed the performance of the previous data analysis by an astonishing 11%.

How well did the detailed biophysical neuron models perform? We don't know, because no prediction based on a detailed model was submitted. The reason may be that it is simply too difficult to tune the parameters of a detailed neuron model to perfection. However, systematic methods for automatic parameter tuning are just becoming available (10, 11).

Among the lessons to be learned from the INCF competition is that every neuron is different and one should not think of “the” model of a pyramidal cell or interneuron. Rather, parameters need to be tuned on a neuron-by-neuron basis. Another lesson is that the quality of a neuron model has to be measured on new data that are not accessible during the phase of parameter tuning. These new data (test set) can be statistically of the same type, but must be different from the data in the training set. In addition, making data publicly available is most rewarding if the data set is combined with a well-formulated task. A final lesson is that, for tasks consisting of predicting spike activity times under single- or double-electrode current injection, simple neuron models of the threshold type that are augmented by adaptation (to describe neuronal fatigue) are sufficient in that they can predict all the predictable spikes. The good performance of threshold models is excellent news for studying properties of neural coding or dynamics of large neuronal networks using adaptive integrate-and-fire neurons. Stochastic versions of such threshold models, also called generalized linear models, have recently been successfully used to decode neural information in sensory (12) and motor (13) areas.

Threshold models give a phenomenological description of neural behavior, but provide only a weak link to the underlying biophysical causes of electrical activity. By construction, threshold models are rather limited in predicting the precise time course of the voltage during and after a spike, and cannot predict the influence of temperature dependence, changes in the chemical environment, or pharmacological manipulations of ion channels, whereas biophysical models of the Hodgkin-Huxley type can do all this. It may soon be possible to measure most parameters of biophysical models in a systematic fashion with a suitable combination of immunostaining methods to determine ion-channel distribution, calibrated measurements of ion-channel kinetics, and expression studies to identify tens of ion channels in individual cells. Automatic model construction along these lines is on its way (14). Moreover, intricate nonlinear spatiotemporal effects on the dendritic tree such as the interplay of back-propagating action potentials (those that travel into a dendrite) with shunting inhibition, or local spikes in the concentration of intracellular calcium that are triggered by multiple, spatially distributed, synaptic inputs, are beyond the scope of threshold models. Although these nonlinear spatiotemporal aspects were difficult to quantify with traditional experimental methods, new imaging techniques that measure the instantaneous voltage time course across the dendritic tree at high spatial resolution in combination with a controlled multisite stimulation (15)—either by glutamate uncaging (16) or optogenetic methods (17)—will open the door to an era of quantitatively predictive biophysical models.

Competitions and model comparisons are widespread in the community of machine learning, where new computer algorithms are tested on benchmark data under well-defined procedures. With a few exceptions, the idea of benchmarking neuron models on a publicly available set of data in a prediction task has not yet found its way into the standard repertoire of neuroscience, but the increased numbers of participants in the INCF competition compared to earlier years (up from 9 to 33 submissions) indicate a paradigm shift in that respect.


Navigate This Article