Report

Reproducibility Distinguishes Conscious from Nonconscious Neural Representations

See allHide authors and affiliations

Science  01 Jan 2010:
Vol. 327, Issue 5961, pp. 97-99
DOI: 10.1126/science.1180029

Abstract

What qualifies a neural representation for a role in subjective experience? Previous evidence suggests that the duration and intensity of the neural response to a sensory stimulus are factors. We introduce another attribute—the reproducibility of a pattern of neural activity across different episodes—that predicts specific and measurable differences between conscious and nonconscious neural representations indepedently of duration and intensity. We found that conscious neural activation patterns are relatively reproducible when compared with nonconscious neural activation patterns corresponding to the same perceptual content. This is not adequately explained by a difference in signal-to-noise ratio.

Though once controversial, it is now widely accepted that sensory-perceptual information can be processed by the brain, even at the semantic level, without that information “reaching” or “entering” awareness (13). But what does it mean for neural information to reach awareness? Once the information has been encoded in neural activity, what else has to happen for it to become part of one’s subjective reality? A growing body of evidence suggests that the intensity of activation in areas that encode the contents of perception (such as the ventral-temporal cortex) is one determinant of whether or not that information contributes directly to subjective experience (47). However, local enhancement of a cortical sensory signal is also associated with attention (8), which can be independent of awareness (911). Therefore, there may be additional features other than the intensity of neural activity that distinguish conscious from nonconscious neural information.

Kinsbourne (12) proposes three interacting properties that collectively determine whether or not a neural representation will contribute directly to subjective experience: (i) the duration and (ii) the intensity of a pattern of activity and (iii) the coherence of that pattern of activity with the dominant “configuration” of neural activity at the global level. Here, we propose that another attribute of neural activity patterns, reproducibility, characterizes conscious representations. We define reproducibility as the similarity of patterns of neural activity across different instances of the same percept. We focused specifically on reproducibility because it is measurable and therefore empirically testable. A corollary of our proposal that conscious representations are more reproducible is that unconscious representations are more variable, even as they may carry information within a given episode.

We used functional magnetic resonance imaging (fMRI) to measure brain activity while subjects performed a simple visual category-discrimination task (n = 12 subjects) (13). Stimuli were simple line drawings of faces and houses (12 of each), rendered in two opposing but isoluminant colors (Fig. 1) (13). Visibility of the stimuli was manipulated by using dichoptic color masking (DCM) (Fig. 1) (7). Subjects were asked to identify the category of the stimulus (face or house) on each trial, guessing if necessary, and to wager (“high” or “low” for monetary rewards) on the accuracy of each of their perceptual decisions (1416). Wagering was used as a collateral index of subjects’ awareness of the object.

Fig. 1

Dichoptic-color masking. This method of manipulating awareness, originally devised by (7), relies on the phenomenon of dichoptic color fusion. The “same color” mode corresponds to the visible condition, and the “opposite color” mode corresponds to the invisible condition. In order to achieve disappearance of the image in the opposite color mode, the two colors must be approximately isoluminant and the object boundaries slightly blurred. Before the experiment, subjects were trained to maintain steady fixation and were cued to do so during each trial with the appearance of the fixation point (500 ms before stimulus onset). Stimuli were presented stereoscopically in the fMRI scanner by using a cardboard divider and prism lenses (28).

For visible stimuli, performance was at or near 100% correct for all 12 subjects, and all wagers were high. For invisible stimuli, task performance was only marginally different from chance (54 ± 2.5[SEM]% correct; P < 0.06, one-tailed t test), and sensitivity of high wagers to correct responses [wagering d-prime, or d′ (13)] was not different from zero (mean d′ = 0.015 ± 0.11[SEM]; P = 0.45, one-tailed t test). For invisible stimuli, wagering d′ and overall willingness to place high wagers were not significantly correlated across subjects [correlation coefficient (r) = 0.33, P > 0.30, n = 12 subjects]. This reassures against the possibility that wagering d′ was artificially low because of an interaction with a wagering bias (16). The proportion of high wagers (for invisible stimuli) was similar for faces and houses (0.20 and 0.19, respectively).

Subjects were always aware of a visual event—a yellowish flickering square—and this provoked substantial activation in and of itself. What varied was subjects’ awareness of an object embedded in the square. We used multivariate pattern analysis to ascertain how the encoding of perceptual information differs depending on whether or not that information is present in subjective experience (17). Thus, in our analyses we focused specifically on the patterns of activation corresponding to the perceptual information of which the subject was or was not aware: the category of the object.

To verify the neural representation of category-specific information for both visible and invisible stimuli, we attempted to discriminate the category of the stimulus (faces versus houses) on the basis of the spatial pattern of neural activity in the temporal lobes [derived statistically from each run of functional data (13)]. We did this independently for the visible and invisible stimuli using a Gaussian naïve Bayes (GNB) classifier (18). We focused our analyses on the temporal lobes because these are widely viewed as being critical for high-level perceptual representation of visual information (19). Mean accuracy of the classifier (percent correct averaged across 12 subjects) was significantly different from chance (50%) for both visible [63% correct; Student’s t test, t = 3.82, P < 0.002] and invisible (58% correct; t = 2.53, P < 0.02) stimuli (Table 1). The difference in accuracy for visible versus invisible stimuli was not significant (P < 0.2, one-tailed paired-samples t test). It might be expected that as long as the classifier performed above chance on both types of stimuli, then it would also perform well when trained on one type and tested on the other (20). However, this was not the case for these stimuli (Table 1).

Table 1

Performance of a GNB classifier. The objective of the classifier was to discriminate the category of the stimulus based on the pattern of beta weights [a general linear model (GLM) was applied separately to each run of functional data (13)]. A voxel-wise analysis of variance and nested cross-validation (18) were used for dimensionality reduction on each round of training and testing. For within-condition classification (visible-visible and invisible-invisible) a leave-one-run-out cross-validation was performed. For between-condition classification, we trained on all the data from one condition and tested on the other and vice-versa. All t tests are one-tailed with df = 11.

View this table:

Each round of training and testing of the classifier involved a dimensionality-reduction step, in which we determined which voxels (features) varied most consistently as a function of stimulus category (feature selection) separately for visible and invisible stimuli (13). Training and testing of the classifier was then performed on these smaller feature spaces (“selections”). Our approach involved the examination of the patterns of activity within these selections of voxels on the assumption that these would reveal properties of information encoding under conditions of conscious and nonconscious perception.

Treating patterns of activation as vectors allows us to test hypotheses about the properties of neural information independently of specific loci and their level of activity. The angle between two activation vectors reflects differences in the contents of perception, whereas the norm of each vector corresponds to the intensity of the information being encoded. We can then define reproducibility as the similarity in the pattern of activity across different instances of the same stimulus category among voxels that carry relevant information. This can be measured by computing the trial-to-trial variability of the vector angle in the space of the voxels selected as informative for classification.

We predicted that activation vectors associated with conscious perception (visible stimuli) would exhibit less trial-to-trial variability in their angle than those associated with nonconscious perception (reflecting greater reproducibility) without necessarily any difference in the norm (that is, in intensity). To assess the reproducibility of representations, we measured the variability in the angle between pairs of vectors (both from the same run and same stimulus category), as well as the norm of each vector, separately for visible and invisible stimuli (13, 21). We repeated this in both the visible and the invisible selections (22). This resulted in four sets of data: responses to visible and invisible stimuli in the visible selection and responses to visible and invisible stimuli in the invisible selection. To avoid confounds that were likely to arise from comparing properties of vectors in different subsets of voxels (and hence different regions of cortex), we restricted our comparisons to vectors within the same selection (23). We used the mean within-category within-run angular deviation as an index of reproducibility.

Within the invisible selection, the variability of the vector angle (dVA) is significantly less for visible than for invisible stimuli (P < 0.01, paired-samples two-sided signed rank test) (Fig. 2B). There was no difference in dVA between visible and invisible stimuli in the visible selection (Fig. 2A), suggesting that the variability is found primarily in the subset of voxels that carry nonconscious information and that this subset is distinct from that within which conscious information is found (for this particular combination of stimuli and task). This is consistent with the failure of the classifier to generalize across the two levels of visibility. When dVA for the invisible selection was compared with the baseline level 4 s earlier (at the time of stimulus onset), there was a significant interaction (P = 0.021, two-sided signed rank test on the deviation from baseline): dVA was below baseline in response to visible stimuli and higher than baseline in response to invisible stimuli (Fig. 2B). There was no difference in the mean or variance of the vector norm for visible versus invisible stimuli, either in the visible or invisible selection (means, P > 0.35, paired-samples two-sided signed rank test; variances, P > 0.7, Levene’s test) (Fig. 2, C and D). Thus, a difference in signal-to-noise ratio is not sufficient to explain the effect.

Fig. 2

Variability in the angle of activation vectors in the visible and invisible selections (A and B) and mean vector norm (C and D). In both (A) and (B), t0 corresponds to the TR (2 s) on which the stimulus was presented before the hemodynamic response had begun to rise. t2 corresponds to 2 TRs (4 s) after the stimulus was presented at the (approximate) peak of the hemodynamic response (n = 12 subjects). This analysis was performed by using a leave-one-run-out procedure: Voxel selection was performed on data from n – 1 runs, and the norm and angular deviation were computed on data from the run that had been left out (13). Comparisons between the two selections [(A) versus (B) or (C) versus (D)] are not valid (23).

Because measurable category-specific information had been identified separately for both visible and invisible stimuli, we examined where in the brain the information tended to coalesce in each case (Fig. 3). For any given subject, reliably informative voxels could be found throughout the temporal lobes (Fig. 3A). Averaging across subjects (24) revealed two clusters in the right ventral temporal cortex, one for visible and the other for invisible stimuli, with minimal spatial overlap, which is consistent with the failure of the classifier trained on one type of stimulus to generalize to the other (Fig. 3, B and C). The anterior-posterior relationship of the two clusters (visible and invisible selections, respectively) coincides with previous observations (25).

Fig. 3

Spatial distribution of informative voxels. (A and B) Voxels that were selected as informative for classification (face versus house) on 6 or more (out of 12) runs for a subject with comparable classification accuracy (72% correct) for visible and invisible stimuli. (C and D) The mean across subjects (24) projected onto the AFNI TT_N27 template brain (right hemisphere) at a statistical threshold of P < 0.05 (corrected). The oblique white line serves as a visual landmark. The cluster in (C) encompasses a portion of the fusiform and parahippocampal gyri in the area of the fusiform face area (FFA) and parahippocampal place area (PPA). The cluster in (D) lies along the posterior fusiform gyrus.

Conscious and nonconscious neural activation patterns coexist within the cerebral cortex, side by side at the same time, but presumably they differ in several ways. Proposed differences include duration, intensity, and coherence. Here, we show that they also differ in their relative reproducibility across presentations of similar stimuli. Why might reproducibility distinguish conscious from nonconscious representations? One possibility is that conscious information is represented in a more discrete form (26), making it more durable and robust, but also more stereotypical (and therefore more reproducible). Another possibility is that conscious information manifests itself in relatively stable neural firing patterns, corresponding to the “settled” states of recurrent network interactions (27). There are a number of plausible theories regarding the neural correlates of consciousness but relatively little data concerning the nature of conscious versus nonconscious encoding. Further work is required to understand the difference (or differences) in the way perceptual information is encoded in the brain depending on whether or not that information is present in subjective experience. Such work is likely to have profound importance in a variety of arenas, including the assessment of consciousness under presumed anesthesia or coma and the investigation of brain function in conditions such as schizophrenia, autism, and dissociation disorders.

Supporting Online Material

www.sciencemag.org/cgi/content/full/1180029/DC1

Materials and Methods

Fig. S1

References

References and Notes

  1. Materials and methods are available as supporting material on Science Online.
  2. Post-decision wagering has been proposed as an independent measure of awareness under the assumption that if the subject is aware of the relevant sensory information, then there will be a correspondence between high wagers and correct responses. If no such correspondence is found, then under this assumption we deduce that the subject was not aware of the relevant sensory information (in this case, information sufficient to discern the category of the stimulus) (15).
  3. N. Persaud, P. McLeod, A. Cowey, Nat. Neurosci. 10, 257 (2007).
  4. While voxel selection was based on coefficients derived statistically from each functional run (13), the activation patterns among these voxels were taken trial by trial from the minimally processed fMRI signal data [at t0 + 2TR, where t0 is time of stimulus onset and 1 TR (time of repetition) = 2 s]. This was done in a leave-one-run-out fashion: The selection was chosen on the basis of data from n – 1 runs, and then the activity vectors from the left-out run (2 visible/invisible × face/house per run) were projected into that space (13).
  5. The visible selection comprises the voxels that were maximally informative as to the category of visible stimuli. Likewise, the invisible selection comprises the voxels that were maximally informative as to the category of invisible stimuli.
  6. Because the visible selection and the invisible selection occupy separate and largely non-overlapping regions of cortex, comparisons between their functional properties are confounded with differences between the hemodynamic and magnetic-field properties of the regions they inhabit.
  7. To produce spatial maps of reliably informative voxels, each voxel was coded with either a 1, if selected on a majority of runs, or a 0 otherwise (Fig. 3, A and B). In order to uncover regional tendencies in the average across subjects, maps for each subject were blurred by ~10 mm and then discretized again (ceiling). The probability distribution of the average map under the null hypothesis was estimated by using a permutation test (number of voxels held constant for each subject per selection, but locations randomized) and used to set a statistical threshold.
  8. A.S. was supported by a grant from the Mind Science Foundation and by a Ruth L. Kirschstein National Research Service Award from the National Institute of Mental Health (MH075342). Special thanks to S. Sher for helpful discussions, S. Dehaene and two anonymous reviewers for comments, M. Kim for help with behavioral testing, and L. Nystrom for advice and assistance with data analysis.
View Abstract

Stay Connected to Science

Navigate This Article