Special Perspectives

Immersive Interfaces for Engagement and Learning

See allHide authors and affiliations

Science  02 Jan 2009:
Vol. 323, Issue 5910, pp. 66-69
DOI: 10.1126/science.1167311


Immersion is the subjective impression that one is participating in a comprehensive, realistic experience. Interactive media now enable various degrees of digital immersion. The more a virtual immersive experience is based on design strategies that combine actional, symbolic, and sensory factors, the greater the participant's suspension of disbelief that she or he is “inside” a digitally enhanced setting. Studies have shown that immersion in a digital environment can enhance education in at least three ways: by allowing multiple perspectives, situated learning, and transfer. Further studies are needed on the capabilities of immersive media for learning, on the instructional designs best suited to each type of immersive medium, and on the learning strengths and preferences these media develop in users.

As another article in this special issue discusses, the information technologies used by children during their formative years influence their learning strengths and preferences (1). An increasingly prevalent type of media, immersive interfaces, can aid in designing educational experiences that build on students' digital fluency to promote engagement, learning, and transfer from classroom to real-world settings.

Immersive Presence

Immersion is the subjective impression that one is participating in a comprehensive, realistic experience (2, 3). Immersion in a digital experience involves the willing suspension of disbelief, and the design of immersive learning experiences that induce this disbelief draws on sensory, actional, and symbolic factors (4). Sensory immersion replicates digitally the experience of location inside a three-dimensional space; total sensory interfaces utilize either head-mounted displays or immersive virtual reality rooms, stereoscopic sound, and—through haptic technologies that apply forces, vibrations, and motions to the user—the ability to touch virtual objects. As described below, interactive media now enable various degrees of sensory immersion.

Actional immersion involves empowering the participant in an experience to initiate actions impossible in the real world that have novel, intriguing consequences. For example, when a person playing an Internet game can make new discoveries by becoming a bird and flying around, the degree of concentration this activity creates is intense.

Inducing a participant's symbolic immersion involves triggering powerful semantic, psychological associations by means of the content of an experience. As an illustration, digitally fighting a terrifying, horrible virtual monster can build a mounting sense of fear, even though one's physical context is unchanging and rationally safe. Invoking digital versions of archetypical situations from one's culture deepens the immersive experience by drawing on the participant's beliefs, emotions, and values about the real world. The more a virtual immersive experience is based on design strategies that combine actional, symbolic, and sensory factors, the greater the participant's suspension of disbelief that she or he is “inside” a digitally enhanced setting.

Immersion Enhances Learning Through Multiple Perspectives

According to studies, immersion in a digital environment can enhance education in at least three ways: by enabling multiple perspectives, situated learning, and transfer. First, the ability to change one's perspective or frame of reference is a powerful means of understanding a complex phenomenon. Typically, this is done by shifting between an exocentric and an egocentric frame of reference. The exocentric frame of reference (Fig. 1A) provides a view of an object, space, or phenomenon from the outside; the egocentric frame of reference (Fig. 1B) provides a view from within the object, space, or phenomenon. With funding from the National Science Foundation (NSF), in the 1990s, our Project ScienceSpace research team conducted studies on sensory immersion in frames of reference and found that the exocentric and the egocentric perspectives have different strengths for learning (5) (SOM Text 1).

Fig. 1.

(A) Exocentric view of NewtonWorld. (B) Egocentric view inside a ball.

A major advantage of egocentric perspectives is that they enable participants' actional immersion and motivation through embodied, concrete learning, whereas exocentric perspectives foster more abstract, symbolic insights gained from distancing oneself from the context (seeing the forest rather than the trees). Bicentric experiences that alternate these views combine these strengths.

Immersion Enhances Learning Through Situated Experience

Immersive interfaces can foster educational experiences that draw on a powerful pedagogy: situated learning. Situated learning requires authentic contexts, activities, and assessment coupled with guidance from expert modeling, mentoring, and “legitimate peripheral participation” (6, 7). As an example of legitimate peripheral participation, physical science graduate students work within the laboratories of expert researchers, who model the practices of scholarship in field work and laboratory work. These students tacitly learn through watching experts in research, as well as by interacting with other team members who understand sophisticated scholarship to varying degrees. While in these settings, students gradually move from novice researchers to more advanced roles, with other team members' expectations for them evolving as their skills develop.

Potentially quite powerful, situated learning is seldom used in classroom instruction because arranging complementary, tacit, relatively unstructured learning in complex real-world settings is difficult. However, immersive interfaces can draw on the power of situated learning by enabling digital simulations of authentic problem-solving communities in which learners interact with other virtual entities (both participants and computer-based agents) who have varied levels of skills.

As discussed in another article in this special issue, scholars are studying the extent to which Internet games and virtual environments such as Second Life provide situated learning that leads to knowledge useful in the real world; their findings thus far are promising (8, 9). The research my colleagues and I are conducting on gamelike virtual simulations for educating young people about higher-order inquiry skills illustrates how immersion can aid engagement and educational achievement through situated learning.

The NSF-funded River City multiuser virtual environment is centered on skills of hypothesis formation and experimental design, as well as on content related to national standards and assessments in biology and epidemiology (10). Students learn to behave as scientists as they collaboratively identify problems through observation and inference, form and test hypotheses, and deduce evidence-based conclusions about underlying causes. Learners immerse themselves inside a simulated, historically accurate 19th-century city (Fig. 2A). Collaborating in teams of three or four participants, they try to figure out why people are getting sick and what actions can remove sources of illness. They talk to various residents in this simulated setting, such as children and adults who have fallen ill, hospital employees, merchants, and university scientists (Fig. 2B and SOM Text 2).

Fig. 2.

(A) River City. (B) Avatar talking to agent.

Our research results from River City show that a broader range of students gain substantial knowledge and skills in scientific inquiry through immersive simulation than through conventional instruction or equivalent learning experiences delivered via a board game. Our findings indicate that students are deeply engaged by this curriculum through actional and symbolic immersion and are developing sophisticated problem-finding skills (in a complex setting with many phenomena, problems must be identified and formulated before they can be solved). Compared with a similar, paper-based curriculum that included laboratory experiences, students overall (regardless of factors such as gender, ethnicity, or English language proficiency) were more engaged in the immersive interface and learned as much or more (11, 12).

Many academically low-performing students do as well as their high-performing peers in River City, especially on performance-based measures (such as a letter to River City's mayor describing an intervention to help reduce illness and providing evidence to support this claim). Digital immersion allows these students to build confidence in their academic abilities by stepping out of their real-world identity of poor performer academically, which shifts their frame of self-reference to successful scientist in the virtual context. This suggests that immersive media may have the potential to release trapped intelligence and engagement in many learners, if we can understand how best to design instruction using this type of immersive, simulated experience.

Other researchers who study educational multiuser virtual environments designed for young people, such as Quest Atlantis or Whyville (13, 14), also are finding that immersive digital settings enhance their participants' engagement and learning. Research indicates that active learning based on immersive situated experiences that include frequent opportunities for reflection via combining egocentric and exocentric perspectives (e.g., participant inside River City versus external observer of the town's overall dynamics) is both motivating and powerful for a broad spectrum of students. The success of immersive simulations in corporate and military training (15, 16) suggests that these positive findings also apply to learners considerably older than those we study.

Immersion may enhance transfer through simulation of the real world. Situated learning through immersive interfaces is important in part because of the crucial issue of transfer. Transfer is defined as the application of knowledge learned in one situation to another situation and is demonstrated if instruction on a learning task leads to improved performance on a transfer task, ideally a skilled performance in a real-world setting (17).

Researchers differentiate between two ways of measuring transfer: sequestered problem-solving and preparations for future learning (18). Sequestered problem-solving tends to focus on direct applications that do not provide an opportunity for students to utilize resources in their environment (as they would in the real world); standardized tests are an example of this. Giving students presentational instruction that demonstrates solving standard problems, then testing their ability to solve similar problems involves near-transfer: applying the knowledge learned in a situation to a similar context with somewhat different surface features.

When evaluation is based on the success of learning as a preparation for future learning, researchers measure transfer by focusing on extended performances where students “learn how to learn” in a rich environment and then solve related problems in real-world contexts. With conventional instruction and problem-solving, attaining preparation for future learning requires far-transfer: applying knowledge learned in a situation to a quite different context whose underlying semantics are associated, but distinct.

One of the major criticisms of instruction today is the low rate of far-transfer generated by presentational instruction. Even students who excel in educational settings often are unable to apply what they have learned to similar real-world contexts. The potential advantage of immersive interfaces for situated learning is that their simulation of real-world problems and contexts means that students must attain only near-transfer to achieve preparation for future learning. Flight and surgical simulators demonstrate near-transfer of psychomotor skills from digital simulations to real-world settings; a variety of studies are currently under way to assess whether other types of immersive learning show transfer to the real world to some degree.

Lesser Degrees of Immersion Can Still Provide Situated Learning

Our research team is currently studying augmented reality, in which users are immersed in a mixture of real and virtual settings. Participants in these immersive simulations use location-aware handheld computers [generally with global positioning system (GPS) technology], which allow users to physically move throughout a real-world location while collecting place-dependent simulated field data, interviewing virtual characters, and collaboratively investigating simulated scenarios (19). Today, augmented reality relies on coupling a handheld computing device to a GPS receiver; however, in the near future, sophisticated cellular phones will provide a ubiquitous infrastructure for this type of immersive learning.

The Handheld Augmented Reality Project, funded by U.S. Department of Education, is part of a collaborative effort by Harvard University, the University of Wisconsin, and the Massachusetts Institute of Technology to study the efficacy of augmented-reality technology for instruction in math and language arts at the middle-school level. Alien Contact! is a curriculum my research team designed to teach math and literacy skills to middle- and highschool students. (20) This narrative-driven, inquiry-based augmented-reality simulation is played on a Dell Axim X51 handheld computer and uses GPS technology to correlate the students' real world location to their virtual location in the simulation's digital world (Fig. 3A).

Fig. 3.

(A) Dell Axim and GPS receiver. (B) Students exploring school grounds.

As the students move around a physical location, such as their school playground or sports fields (Fig. 3B), a map on their handheld displays digital objects and virtual people who exist in a simulated world superimposed on real space (Fig. 4). When students come close to these digital artifacts, the augmented reality and GPS software triggers video-, audio-, and text files, which provide narrative, navigation and collaboration cues, as well as academic challenges to build math and literacy skills (SOM Text 3).

Fig. 4.

Handheld display of digital resources on school grounds.

Early research on the Alien Contact! curriculum found high levels of student engagement, as well as educational outcomes in literacy and math equivalent to students playing a similar, engaging board game as a control condition (21). Further design-based research is needed to determine the extent to which more powerful learning outcomes emerge from augmented-reality experiences. Such studies will aid in determining what degree of digital immersion is necessary for achieving various types of engagement, learning, and transfer.

Next steps in research on immersive interfaces for learning. Due to the growing ubiquity of sophisticated mobile phones and multiplayer Internet games, people of all ages increasingly will have life-style choices involving engaging forms of immersion in both virtual and augmented realities. Understanding the strengths and limits of these immersive media for education is important, particularly because situated learning seems a promising method for learning sophisticated cognitive skills, such as using inquiry to find and solve problems in complicated situations.

Further studies are needed on the affordances immersive media offer for learning, on the instructional designs best suited to each type of immersive medium, and on the learning strengths and preferences use of these media develops in users. Illustrative research questions include:

  1. To what extent does good instructional design for immersive environments vary depending on the subject matter taught or on the characteristics of the learner? For what types of curricular material is full sensory immersion important?

  2. To what extent can the successes of one's virtual identity in immersive environments induce greater self-efficacy and educational progress in the real world?

  3. To attain transfer, what is the optimal blend of situated learning in real, augmented, and virtual settings?

  4. What insights about bicentric frames of reference can generalize from immersive environments to pedagogical strategies in face-to-face settings?

Results from studies of immersive environments for learning seem sufficiently promising that further investment in this type of research is indicated.

Supporting Online Material


SOM Text

Figs. S1 to S7

References and Notes

View Abstract

Stay Connected to Science

Navigate This Article