Perceptual Learning Incepted by Decoded fMRI Neurofeedback Without Stimulus Presentation

See allHide authors and affiliations

Science  09 Dec 2011:
Vol. 334, Issue 6061, pp. 1413-1415
DOI: 10.1126/science.1212003


It is controversial whether the adult primate early visual cortex is sufficiently plastic to cause visual perceptual learning (VPL). The controversy occurs partially because most VPL studies have examined correlations between behavioral and neural activity changes rather than cause-and-effect relationships. With an online-feedback method that uses decoded functional magnetic resonance imaging (fMRI) signals, we induced activity patterns only in early visual cortex corresponding to an orientation without stimulus presentation or participants’ awareness of what was to be learned. The induced activation caused VPL specific to the orientation. These results suggest that early visual areas are so plastic that mere inductions of activity patterns are sufficient to cause VPL. This technique can induce plasticity in a highly selective manner, potentially leading to powerful training and rehabilitative protocols.

Whether adult primate visual cortex has sufficient plasticity to allow for behavioral and/or sensitivity changes remains a point of great controversy. Many studies have examined how activity changes in the brain are correlated with performance improvements on a visual task resulting from repetitive training, known as visual perceptual learning (VPL). However, such a correlational approach has not conclusively settled the adult plasticity debate. Although some studies have found correlations between performance increase and changes in early visual areas (15), other studies found correlations in higher visual and/or decision areas (68). None of these studies directly addresses the question of whether early visual areas are sufficiently plastic to cause VPL. Changes in early visual areas observed in correlation with VPL do not exclude the possibility that the changes are, in reality, a reflection of the influences of changes in other brain areas. On the other hand, changes in higher brain areas in some conditions in correlation with some types of VPL do not rule out the possibility that early areas are sufficiently plastic to cause VPL in other conditions.

To addresses the question of whether early visual areas are that plastic, we developed a functional magnetic resonance imaging (fMRI) online-feedback method, by which activation patterns corresponding to the pattern evoked by the presentation of a real and specific target orientation stimulus were repeatedly induced without the participants’ knowledge of what is being learned and without external stimulus presentation [see supporting online materials (SOM) and methods]. The mere induction of the activation patterns resulted in significant behavioral performance improvement on the target stimulus orientation, but not on other orientations.

The complete experiment consisted of four stages: (i) pre-test (1 day), (ii) fMRI decoder construction (1 day), (iii) induction (decoded fMRI neurofeedback, 5 days for four participants and 10 days for six participants), and (iv) post-test (1 day) stages (Fig. 1A).

Fig. 1

Procedure of the main experiment. (A) Stages of the experiment. (B) Procedure of a trial of the test stages. (C) Gabor patches of the three orientations used in this experiment. (D) Example of a sequence of stimuli in one trial of the decoder construction stage. (E) Procedure of a trial in the induction stage.

In the pre- and post-test stages, we measured participants’ performance in an orientation discrimination task (10 participants). In each trial (Fig. 1B), participants were asked to report which of three orientations (10°, 70°, or 130°) (Fig. 1C) had been presented in a Gabor patch (see SOM).

Next, we tried to obtain fMRI activity patterns in V1/V2 that are induced by the presentation of each of the three tested orientations in Gabor patches for each participant (fMRI decoder construction stage). Participants were asked to perform a task designed to maintain their attention to the Gabor patches while fMRI signals in V1/V2 were measured (Fig. 1D and SOM). Based on the fMRI signals, we constructed a multinomial sparse logistic regression decoder (9) to classify a pattern of the measured fMRI signals into one of the three orientations (fig. S1).

Once the decoder was constructed, each participant took part in a 5- or 10-day induction stage during which he or she learned to induce activation patterns in V1/V2 that corresponded to the target orientation, one of the three orientations, that was randomly assigned to each participant. During each trial (Fig. 1E and SOM), participants were asked to “somehow regulate activity in the posterior part of the brain to make the solid green disc that was presented 6 s later as large as possible (the maximum possible size corresponds to the outer green circle).” The size of the disc presented in the feedback period corresponded to the decoder output for the target orientation, which represented the likelihood of the blood oxygen level–dependent signal pattern in V1/V2 obtained in the preceding induction period being classified into the target orientation for which the performance is aimed to be improved. It roughly represented how similar the activation pattern obtained in the induction period in the absence of visual stimulation is to the pattern evoked by the real Gabor stimulus of the target orientation presented during the decoder construction stage. We call the likelihood (similarity) target-orientation likelihood. However, participants were not informed of what the size represented. They were told that they would receive a payment bonus proportional to the mean size of the feedback disc, but all other information, including the target orientation, the purpose of the neurofeedback, and the meaning of the disc size, was withheld from the participants.

The main purpose of our study was to test whether early visual cortical areas are sufficiently plastic to cause VPL of a specific orientation as a result of mere repetitive inductions of activity patterns corresponding to that orientation. However, before testing this hypothesis, it was necessary to examine whether participants could learn to induce significantly high target-orientation likelihood; that is, a neural activity pattern in V1/V2 that is similar to an activity pattern evoked by the actual presentation of the target orientation. Thus, we first examined whether outputs of the decoder could be biased toward the selected target orientation compared with the other two orientations that were rotated ±60° from the target orientation. Figure 2 shows that the overall mean target-orientation likelihood in V1/V2 was significantly higher than chance across the participants, on average, during the induction stage [t(9) = 3.34, P < 10−2]. The mean (across the participants) target-orientation likelihood in V1/V2 for the first 30 trials of the first neurofeedback day was around chance level (fig. S2). Thus, there was no significant orientation bias for the target orientation before neurofeedback, and participants quickly learned to induce significantly high target-orientation likelihood, even during the first neurofeedback day (see performance for day 1 in Fig. 2). We also applied the same analysis to the overall activity pattern that is the mean across trials and confirmed the same tendency (fig. S3).

Fig. 2

Results of the induction stage. The mean (±SE; represented by error bars) likelihoods of three orientations assessed by a decoder based on V1/V2 activity patterns. For the first 5 days, combined data from 5-day (4 participants) and 10-day (6 participants) neurofeedback sessions is shown. For the last 5 days, data from 10-day neurofeedback session is shown.

Were participants aware of the purpose of the induction stage? After the post-test stage, participants were asked about what they thought the size of the feedback disc represented, but none of their responses was even remotely related to the true workings of the experiment (see “After post-test stage” section of SOM). After being told that the disc size represented the likelihood of one of three orientations, subjects were asked to report the orientation they thought they had been trained on by picking one of the three orientations. The percentage of the choice of the target orientation in fig. S4 was statistically undistinguishable from what would be expected from chance (χ2 = 0.20, P = 0.90).

The purpose of the induction stage was to have participants learn and then continue to induce activity patterns with significantly high target-orientation likelihood in V1/V2. This learning in the induction stage should not be confused with the plasticity or VPL that refers to improvements on visual tasks. The main purpose of the present study was to examine whether the mere repetitive induction of specific activation patterns in V1/V2 causes VPL reflected as performance improvement.

We compared participants’ performance in the pre- and post-test stages (Fig. 3, A to C). Three-way [test stage × orientation × signal-to-noise (S/N) ratio] analysis of variance with repeated measures indicated significant main effect of S/N ratio (F3,27 = 683.17, P < 10−4) and significant effect of interaction between test stage, orientation, and S/N ratio (F6,54 = 2.68, P = 0.02). Post-hoc t test between accuracies in pre- and post-tests revealed that discrimination performance for the target orientation significantly improved at the 6% S/N ratio [t(9) = 5.76, P < 10−2 with Bonferroni correction]. Sensitivity (d′) in the pre-test subtracted from that of the post-test was significantly greater than zero for the target orientation at the 6% S/N ratio [t(9) = 5.60, P < 10−3 with Bonferroni correction] (Fig. 3D).

Fig. 3

Results of the pre- and post-tests. (A to C) Discrimination accuracies for –60° (A), target orientation (B), and +60° (C). Error bars indicate SE. (D) Sensitivity values (d′) in the pre-test subtracted from those in the post-test for three orientations at 6% S/N ratio. Error bars indicate SE. (E) Correlation between the summation of target-orientation likelihoods in V1/V2 in the induction stage and sensitivity changes for the target orientation at 6% S/N ratio.

What is the relation between the target-orientation likelihood in V1/V2 and d′ changes? The sensitivity changes for the participants with 10 days training (induction) were larger than for those with 5 days training (fig. S5). This observation was consistent with the general tendency that the magnitude of VPL is larger with longer training until it reaches an asymptote. Thus, we computed the summation of the target-orientation likelihoods in V1/V2 for all trials for each participant and plotted the sensitivity change against the summation. The correlation was even stronger for the likelihood summation (r = 0.87, P = 10−3) (Fig. 3E) than for the average likelihood (r = 0.74, P = 0.01).

To test whether the VPL observed in the main experiment resulted simply from participants’ involvement in the test stages, we conducted a control experiment with six new participants in which only the pre- and post-test stages were conducted (SOM). No significant performance improvement was observed (fig. S6).

Participants can be trained to control the overall mean activation of an entire brain region or the activation in one region relative to that in another region (1013). One might wonder if the participants in the present study simply learned to regulate the overall activity of V1/V2. However, two lines of evidence argue against this possibility. First, the multinomial sparse logistic regression decoder used in this study computed the linear weighted sum of voxel activities, and weights of the decoder were almost symmetrically distributed around 0 (fig. S7). Second, some voxels in V1/V2 were activated positively and others negatively, rather than uniform positive or negative activation during the induction stage (fig. S8).

The neurofeedback provided to participants was based on activation patterns only in V1/V2. However, this procedure might have induced neural activities in areas other than V1/V2, which might also contribute to VPL. To test whether other regions quantitatively contributed to VPL, we conducted two offline tests with other areas (such as V3, V4, the intraparietal sulcus, and the lateral prefrontal cortex) that have been implicated in VPL (68).

If the orientation-specific activation patterns in V1/V2 during the induction stage induced similar orientation-specific brain activities in other areas, the activation patterns in those areas should predict the target-orientation likelihood in V1/V2 on a trial-by-trial basis. In the first offline test, we employed a sparse linear regression method (14) to predict the target-orientation likelihoods in V1/V2 from activation patterns in those higher areas in each trial during the induction stage (SOM). Goodness of prediction for the target-orientation likelihood in V1/V2 by other areas, or prediction accuracies of the sparse linear regression, was evaluated by coefficients of determination, all of which were less than 5% (fig. S9A).

We conducted a second offline test to examine the possibility that the decoder simply performed poorly in higher brain areas. We examined whether accurate orientation information can be read out from each brain area when real orientation stimuli are presented in the decoder construction stage. As was done for V1/V2 during the fMRI decoder construction stage, we built a multinomial sparse logistic regression decoder to classify activation patterns into each of the three orientations (SOM). Decoding accuracies were significantly higher than chance level in all of these areas (fig. S9B, also compare fig. S9, A and B). The results of these two offline tests indicate that influences of the neurofeedback on VPL were largely confined to early visual areas such as V1/V2.

Our results indicate that the adult early visual cortex is so plastic that mere repetition of the activity pattern corresponding to a specific feature in the cortex is sufficient to cause VPL of a specific orientation, even without stimulus presentation, conscious awareness of the meaning of the neural patterns that participants induced, or knowledge of the intention of the experiment. How is the present research on VPL distinguished from previous approaches? Unit recording and brain imaging studies have successfully revealed the correlation between VPL and neural activity changes (18). However, these correlation studies cannot clarify cause-and-effect relationships. The studies that examined the effect of a lesion (15) or transcranial magnetic stimulation (TMS) (16, 17) to a brain region on VPL have shown whether the examined region plays some role in VPL. However, these studies cannot clarify how particular activity patterns in the region are related to VPL. In contrast, the present decoded fMRI neurofeedback method allowed us to induce specific neural activity patterns in V1/V2, which caused VPL.

The present decoded fMRI neurofeedback method can be used to clarify cause-and-effect relationships in many functions in system neuroscience (18, 19). Although previous fMRI online-feedback training is a promising technique for influencing human behaviors (1013), as in lesion or TMS studies, it could at best reveal influences of the entire extent of an area/region on learning/memory, which is a certain limitation for neuroscientific research (20). In contrast, the present decoded fMRI neurofeedback method induces highly selective activity patterns within a brain region, thus allowing the investigator to influence specific functions. It can “incept” a person to acquire new learning, skills, or memory, or possibly to restore skills or knowledge that has been damaged through accident, disease, or aging, without a person’s awareness of what is learned or memorized.

Supporting Online Material

Materials and Methods

Figs. S1 to S9

References (2131)

References and Notes

  1. Acknowledgments: This work was conducted in “Brain Machine Interface Development” under the Strategic Research Program for Brain Sciences by the Ministry of Education, Culture, Sports, Science and Technology of Japan. T.W. was partially supported by NIH grants R01 AG031941 and R01 EY015980 and Y.S. by grants R01 MH091801 and NSF 0964776. We thank J. Dobres, M. Fukuda, G. Ganesh, H. Imamizu, A. R. Seitz, and K. Tanaka for their comments on a draft of this manuscript and M. Fukuda, Y. Furukawa, S. Hirose, M. Sato, and ATR BAIC for technical assistances.
View Abstract

Stay Connected to Science

Navigate This Article