Review

Undergraduate research experiences: Impacts and opportunities

See allHide authors and affiliations

Science  06 Feb 2015:
Vol. 347, Issue 6222, 1261757
DOI: 10.1126/science.1261757

Assessing the value of undergraduate research

Undergraduate research experiences often engender enthusiasm in the students involved, but how useful are they in terms of enhancing student learning? Linn et al. review studies that focus on the effectiveness of undergraduate research programs. Undergraduate research experiences in a class were distinguished from those involving individualized participation in a research program. Mentoring emerges as both an important component of a successful experience and a target for improvement.

Science, this issue 10.1126/science.1261757

Structured Abstract

For any undergraduate contemplating a career in scientific research, participating in authentic research seems like a good opportunity. But what are authentic research experiences? How do they benefit undergraduates? What forms of mentoring are successful? What needs improvement? And how can these experiences meet the needs of interested students while at the same time be cost-effective in large research universities?

We review the research tackling these questions and find few answers. While most undergraduates give high ratings to research experiences, specific benefits have not been documented. Of the 60 empirical studies published in the last 5 years, only 4 directly measured gains in research capabilities or conceptual understanding. Most studies draw conclusions from self-report surveys or interviews, notoriously poor methods for documenting impacts. These studies leave us with few insights into what works and little idea about how to make the experiences more effective.

BACKGROUND

Most colleges and universities offer Undergraduate Research Experiences (UREs) and/or Course-based Undergraduate Research Experiences (CUREs) (Fig. 1). Two large surveys, the 2004 Freshman Survey and the 2008 College Senior Survey, administered at over 200 institutions, generated data about the impact of undergraduate research experiences on persistence in science and intention to pursue graduate school. These studies document that students appreciate undergraduate research experiences. The surveys are unable, however, to distinguish between UREs and CUREs. In addition, the value that undergraduate research adds cannot be disentangled from precollege preparation, especially for students from groups that are underrepresented in science.

ADVANCES

Designers of UREs expect students to benefit from participating in a scientific laboratory but have not determined optimal ways to orient and guide participants. Students often expect the URE to mimic their college laboratory experiences with procedural guidance and planned outcomes. During the first year of a URE, students often report spending most of their effort on setting up and conducting experiments and limited effort on understanding the investigation or interpreting the results. Students would benefit from an orientation that integrates their beliefs and expectations with the realities of the research experience. The few studies that measure changes in understanding of scientific practices or relevant science concepts report little or no gains after 1 year in a URE. Students who spend over a year in a URE often learn new methodological techniques, collect their own data, interpret findings, and formulate new research questions. The slow enculturation into lab activities may make sense, especially when students join labs investigating questions that do not arise in undergraduate education. The time and resources needed, however, limit the scalability of UREs. Students encounter new ideas during their research experiences but often need guidance to integrate these ideas with their expectations. We discuss ways that designers of UREs can speed up enculturation and strengthen guidance.

Individual mentoring emerges as an effective way to guide students and improve learning from research experiences. Activities that could help students benefit from research experiences include discussion with mentors, participation in group meetings where current research is discussed, guided opportunities to explore relevant research literature, reflection on observations in weekly journals, and synthesis of their insights by creating research proposals, reports, or posters. We discuss ways to prepare mentors so that they can efficiently guide students.

OUTLOOK

Undergraduate research experiences absorb a lot of time, money, and effort. The costs and benefits of research experiences for building human capital, benefitting undergraduates, improving workforce diversity, and strengthening educational outcomes need better understanding. Making the best use of extramural funds and the (often voluntary) contributions of faculty to improve undergraduate research experiences requires a strong research base.

More rigorous research is needed, and the field could benefit by building on insights from the learning sciences. We use the knowledge integration framework to interpret the available findings and to identify gaps in the research base.

We discuss ways to develop validated, generalizable assessments such as methods for measuring ability to locate and interpret primary literature. We suggest techniques for developing criteria for evaluating mentoring interactions. We identify ways to strengthen mentoring and to ensure that research experiences meet the needs of diverse students.

URE in action.

Biofuel research engages a Berkeley undergraduate researcher during a summer internship with the Synthetic Biology Engineering Research Center (funded by NSF grant 1132670).

Abstract

Most undergraduates give high ratings to research experiences. Studies report that these experiences improve participation and persistence, often by strengthening students’ views of themselves as scientists. Yet, the evidence for these claims is weak. More than half the 60 studies reviewed rely on self-report surveys or interviews. Rather than introducing new images of science, research experiences may reinforce flawed images especially of research practices and conceptual understanding. The most convincing studies show benefits for mentoring and for communicating the nature of science, but the ideas that students learn are often isolated or fragmented rather than integrated and coherent. Rigorous research is needed to identify ways to design research experiences so that they promote integrated understanding. These studies need powerful and generalizable assessments that can document student progress, help distinguish effective and ineffective aspects of the experiences, and illustrate how students interpret the research experiences they encounter. To create research experiences that meet the needs of interested students and make effective use of scarce resources, we encourage systematic, iterative studies with multiple indicators of success.

Many claim that undergraduate research experiences improve preparation of the next generation of scientists and increase persistence in science (13). The limited evidence for the impact of undergraduate research experiences makes it difficult, however, to justify the substantial resources they require. Of the 60 empirical investigations published during the last 5 years, over half rely exclusively on self-report surveys or interviews to document outcomes, although such evidence has serious flaws (4) (Fig. 1). Fewer than 10% of the studies validate self-reports with analysis of research products (such as presentations or culminating reports), direct measures of content gains, longitudinal evidence of persistence, or observations of student activities. Although researchers often call for better assessments, valid measures have yet to be designed (510). In addition, undergraduate research programs often select students who already intend to persist in science and primarily document that they continue to major in science. More nuanced indicators of success such as improved use of scientific practices, increased ability to interpret original sources, or a better sense of possible flaws in research designs would strengthen the research base. We draw on the most convincing studies to identify impacts and opportunities for future investigations. We identify mentoring as essential for successful support of undergraduates considering careers in science. We call for studies that distinguish which types of undergraduate research experiences succeed for students with distinct interests, backgrounds, and preparation.

Fig. 1 Goals measured, methodologies used, and key findings.

We used electronic databases and Internet search engines such as ERIC, Web of Science, SciFinder Scholar, Science Citation Index, and EdFull to identify studies. We reviewed citations in relevant articles and examined individual journals (e.g., J. Chem. Ed.) to locate papers missed by the search. Key words included undergraduate research, research opportunities, and science, physics, chemistry, biology. Computer science and psychology were excluded to keep the research experiences as similar as possible. Consistent with journal policy, we included papers published in the last 5 years (2010 to 2014). This process yielded 253 documents published in peer-reviewed journals. Review of titles and abstracts yielded 60 empirical studies. We used an emergent categorization process to analyze and score the characteristics of each study. References (1, 11, 12, 2227, 30, 3235, 37, 3947, 5052, 5574) provided the basis of the analysis in Fig. 1.  See the reference list for full citations.

Designing research experiences to promote integrated understanding

Undergraduate research experiences provide a window on science in the making, allowing students to participate in scientific practices such as research planning, modeling of scientific observations, or analysis of data. The experiences are intended to enculturate students into scientific investigation. Faculty, postdoctoral scholars, and other members of the lab mentor students. Ideally, mentors guide students to interpret authentic images of scientific research and link their experiences to their own beliefs or expectations. Interview studies document the many inconsistent ideas about scientific research that undergraduates develop. Many expect scientific research to mimic their college laboratory experiments. Others are unprepared for the failure rate in independent research. For example, one student said, “I honestly expected it to be like my organic chemistry lab that I just finished last year [...] I’m used to ‘here is the procedure, now get to it,’ and I thought that was what the experience would be like” [(11), p.1084]. In a post–research experience interview another student reflected, “I think this experience helped me to really understand that it’s not, like, a magical experiment and you come up with magical data and some magical conclusion, and that it is frustrating, but you get through it, and you get over it, and you’ll run it again and if it’s just as frustrating, you’ll do it again” [(12), p.65].

To characterize the investigations of research experiences, analyze how they promote integrated understanding in science, and recommend improvements, we draw on research in the learning sciences. Specifically, we use the knowledge integration framework that synthesizes extensive research on inquiry science to identify gaps and conundrums in the research on undergraduate research experiences (1317). This framework calls for eliciting students’ initial ideas (consistent with hypothesizing) and encouraging students to test them against new ideas (18). To add new ideas, the framework documents the value of participating in personally relevant contexts, such as research experiences to make sense of science practices. The framework also highlights the value and importance of dynamic models of scientific phenomena that reveal insights into unseen processes such as molecular reactions (19). Perhaps most importantly, the framework emphasizes that new ideas can be isolated and forgotten and highlights the need to guide students to become adept at distinguishing among their initial ideas and those they encounter in courses or research experiences to build coherent understanding (17). Finally, the framework builds on research showing that learners benefit from reflecting on their investigations and observations to sort out and consolidate their ideas (20). This framework guides our analysis of the literature on research experiences and our recommendations for improving them (Fig. 2).

Fig. 2 Mentoring to promote knowledge integration.

Successful mentors elicit ideas to find out what students think, add relevant new ideas, encourage students to find evidence to distinguish among disparate ideas, and ask students to reflect and consolidate their experiences.

Distinguishing among research experiences

Research experiences include Undergraduate Research Experiences (UREs) and Course-based Undergraduate Research Experiences (CUREs) (21). UREs feature individual students in faculty research laboratories and provide the opportunity for one-on-one mentoring (Fig. 3). Typically, students spend one or more semesters in labs, although the type of activity and form of mentoring varies substantially. Selection for UREs is highly competitive because few students can be accommodated. Using grades, test scores, and essays, selection committees generally identify students who succeeded in high school and college, although a few studies use other criteria. Most students in UREs are already motivated to succeed in science. UREs may exclude students whose interests are not represented by the available research. In contrast, CUREs have a curriculum and are open to most students. CUREs put high demands on one or a few mentors to guide many students (22). Many studies demonstrate that duration for both UREs and CUREs affects outcomes (2327). UREs and CURES vary in selectivity, duration, setting, mentoring, and cost (Fig. 4).

Fig. 3 Characteristics of UREs and CUREs.

CUREs typically provide research experiences for 30 or more students guided by a course instructor and/or graduate student, and involve classes, credits, grades, and assignments. Students typically compete for URE placements, spend time in a research laboratory, and receive one-on-one mentoring from a postdoc, graduate student, or faculty member.

Fig. 4 Implementation site, duration, and research participants for UREs and CUREs.

About 70% of the research experiences were studied at 4-year institutions. Most studies of CUREs involved lower-division students and lasted one semester or less, whereas studies of UREs involved both upper- and lower-division students whose length of participation varied within the study.

Impacts and opportunities

We synthesize impacts and opportunities of undergraduate research experiences from the studies reviewed. We organize the studies in five themes: (i) mentoring participants, (ii) selecting participants and promoting identity, (iii) improving research practices, (iv) expanding conceptual understanding, and (v) communicating the nature of science.

Mentoring participants

Successful mentoring balances the dual goals of helping undergraduates deepen their understanding of science and guiding them to develop a scientific identity. Mentors ideally orient undergraduates to develop and integrate (i) conceptual knowledge and background information in the topic of the research experience; (ii) science practices such as developing an argument from evidence; and (iii) insights into the culture of the lab, including the requirements of the funding and the roles of the participants. Mentors guide students to form a scientific identity by helping them imagine roles they can play in the lab, recognize gaps in their knowledge that future courses will fill, and identify ways to contribute that also strengthen their current capabilities. Researchers have studied mentoring in varied professional and educational contexts using surveys and interviews (28, 29).

Typically, mentoring is shared among faculty, postdoctoral researchers, and graduate students with UREs offering an individual relationship with mentors and CUREs requiring students to share one or several mentors. Studies indicate that undergraduates interact most frequently with graduate students and postdocs, and less with faculty (26). Mentoring by graduate and postdoctoral researchers tends to focus on technical aspects of the projects, whereas faculty are likely to help students build a scientific identity by articulating their knowledge, reasoning, or problem-solving skills (30). Thus, graduate students and postdoctoral researchers may spend more time with their mentees than faculty, but rarely support development of the complex scientific reasoning skills and professional identity formation that could benefit undergraduates. Peers can also help orient and inform other undergraduates about research, especially in UREs where experienced students mentor newcomers (31).

Mentors have responsibility to orient students so they can see the connections among experiences with experimental design, data collection, interpretation of findings, and scientific communication; help students understand the science concepts and practices necessary for the research project; and guide students to develop resilience to inevitable failures (3234). Additionally, mentors need to provide professional socialization and emotional support so that students can integrate their ideas about their scientific identity with the setbacks and confusions they encounter in their interactions with the members of the lab. One study shows that mentors can challenge students to become aware of the tension between their own cultural norms and those of the scientific community (35). Other investigations found that students who feel supported by faculty are more likely to go to graduate school (22), that frequency of meetings with faculty mentors correlates with student confidence to perform science practices or pursue a research career (36), and that students who lack interactions with mentors and fail to get direction for research projects are likely to change career plans away from science (37). Studies suggest that mentors who function as career coaches, focusing on ways to remedy gaps in preparation, may be more effective than those who primarily emphasize social support (38). Overall, mentors play a crucial role in undergraduate research experiences, often with little preparation, support, or even rewards for their contributions.

Promoting persistence and identity

Longitudinal studies using the 2004 Freshman Survey and 2008 College Senior Survey tracked students who initially expressed an intention to pursue a STEM major (22, 39, 40). They found that students generally rated URE and CURE participation highly. Those who participated in UREs were 14 to 17% more likely to persist in science majors and more likely to retain their interest in graduate school than nonparticipants. However, the validity of this finding is undermined because intention to enroll in graduate school was inferred from a question asking for graduate school major, and those not planning to attend were instructed to skip the question (22). Although the analysis adjusts for missing data and low response rates, the surveys did not adequately differentiate between UREs and CUREs, adjust for difference in selection criteria across institutions, or control for duration (a known factor in impact), making interpretation worrisome (41, 42). Furthermore, these analyses do not establish the direction of causality since students may participate in a URE because of their desire to persist or their interest in building a relationship that could result in a letter of recommendation for graduate school, rather than persisting because of their experience in the URE.

To promote identity as a scientist, five studies augment self-report surveys about persistence with interviews (31, 4345) or journal reflections (33). In interviews, undergraduates from groups underrepresented in science reported that research experiences increased their confidence and expanded their images of science careers by allowing them to try out the roles of research scientists (31, 45). Other undergraduates reported that the research experience gave them the opportunity to broaden their academic and professional science networks (33), the chance to learn how to act like professionals in a research setting (45), a feeling of ownership of their research project (46), and the option of determining whether the work of a scientist could align with their personal values and goals (44). Both interviews and journal writing had the added value of supporting students to connect ideas from their research experience to their views of themselves as scientists.

Access, duration, and selectivity of research experiences influence their impact. In the studies reviewed, students are more likely to participate in directed research with a faculty member in a small liberal arts institution (37) than in a large research university (25). Sustained participation (three or more semesters) in a URE builds identity as a scientist, whereas intermittent URE participation can be a negative experience (37) and short UREs have little to no benefit (25). Selection practices could also limit the impact of research experiences: One study found that high school preparation mattered more than research experiences for a sample of students who are from groups underrepresented in science and who begin college with high grades and aspirations (39). These findings reinforce the importance of mentoring and illustrate the complexity of designing effective research experiences.

These results raise the question of whether recruiting underrepresented students to UREs and CUREs coupled with appropriate mentoring could increase diversity in science. One program, SURE at Emory University, reports increasing diversity by bringing second-year community college students to flagship universities for UREs (41). They select students based on math preparation, experience with science, and success in overcoming challenges. Preference is given to first-generation college attendees and students from underrepresented groups. Regression analyses of transcripts revealed that SURE participants took more science courses as seniors and earned higher grades in those courses than nonparticipants. A replication with randomized assignment to SURE would strengthen the findings.

Improving research practices

Although self-report surveys show that participants believe they learned science practices such as lab techniques, ability to analyze data, and skill in oral and written presentation, the studies could be strengthened by measuring progress directly and by determining whether students have a coherent view of science practices (23, 41, 4749). For example, students may develop data collection skills but lack ability to interpret results (2527, 30). One study that combined surveys, interviews, and shadowing of eight undergraduates as they interacted with a research team found that students primarily set up and conducted experiments, rather than understanding the rationale for design or the interpretation of results (40).

These studies do help explain why duration affects outcomes from research experiences (2325, 27, 37). During the first year, students, who are typically unfamiliar with the science concepts and techniques of the lab, need orientation to the specific research project and slowly acquire this knowledge. In 1 year or less, the duration of most UREs, students learn how to set up experiments specific to that lab and collect data but can rarely relate the analyses to a research question (30). Several studies found that adopting the traits, habits, and temperament (patience, perseverance, initiative) of scientific researchers only begins to emerge in the third semester of a URE (26, 27). Some second-year students learned new methodological techniques, collected their own data, interpreted findings, and formulated new research questions. The slow enculturation into scientific practices helps explain survey results showing that 1-year UREs generate little progress in understanding science practices.

Expanding science conceptual understanding

Understanding of the underlying theories and concepts is essential for students to benefit from authentic science experiences and is more successful in CUREs than in UREs (27). CUREs offer opportunities to develop conceptual understanding by integrating lectures and readings with investigations of an important research question (50, 51). Thus, CUREs build on learning strategies that students have used in other courses.

URE placements often require conceptual understanding that is beyond the student’s academic preparation, especially at research universities where students may join labs investigating questions that do not arise in undergraduate education. Although URE placements can motivate students to develop new insights, the limited evidence for gains in conceptual understanding suggests that this is rare. Typically, students need guidance to understand the rationale, research design, and contribution to the field in this new area (26, 27). Mentors may orient students to relevant literature, or students may seek resources on their own. After completing 1 year in the URE, undergraduates begin to benefit from reading literature, talking with senior scientists, and participating in lab meetings, activities likely to help them integrate their understanding of the underlying theories and concepts (30).

Studies have evaluated gains in conceptual understanding by using grade-point averages and patterns of future course selection, but these activities could result from multiple aspects of the experience (52). Promising assessments ask students to analyze new examples of primary literature, but are rarely used, probably due to the challenges of developing scoring rubrics (8, 50).

Communicating the nature of science

Both UREs and CUREs help students refine their appreciation of the process of scientific research (23, 27, 43). Students come to research experiences with inaccurate ideas about the nature of science (53, 54). For example, undergraduates expect research to be more efficient and to have fewer setbacks than they encounter. A student may describe a procedure as “finicky” or prone to errors rather than recognizing that trial and refinement are part of the nature of science (27). Students often develop conflicting ideas about the nature of science. For example, one student explained that the “scientific method” should be “stuck to like glue,” yet also reported that, “well it is alright to keep some things [the same] and change others” [(11), p. 1091].

Several studies use interviews and journals to show that students in research experiences make progress in understanding the nature of science. In one study, undergraduates gained ability to attribute a scientific purpose to an experiment, describe theories scientifically, and recognize creativity in research and teaching (12). In another study, students shifted from viewing science as a stepwise linear progression to seeing science as messy and involving iteration (11). Examination of the interviews or journals suggests that motivating students to articulate their views about the nature of science and talk or write about their experiences helps them reflect on their experiences, consistent with the knowledge integration framework (see Fig. 2).

Conclusions

Evidence for the benefit of research experiences from the 60 reviewed studies published in the last 5 years is limited. One challenge involves documenting the differences between undergraduate research experiences and typical labs. Studies report that UREs often engage undergraduates in following experimental protocols rather than interpreting results. CUREs use lectures and readings to impart conceptual understanding about an important research question, consistent with instructional strategies in other courses (30, 50, 51, 55). While introducing new images of science, research experiences may also reinforce incomplete or inaccurate images. As expected, duration of UREs and intensity of mentoring both strengthen impacts and differentiate research experiences from typical labs (25, 36, 37).

Overall, these findings suggest the need for greater emphasis on integration of research experiences with the beliefs and expectations of undergraduates (1317). When research experiences introduce new ideas, these ideas are often isolated and fragmented. Students need opportunities to integrate evidence from their research experiences to strengthen views of their identity as scientists, the range of science practices, ways to learn science concepts, and the nature of science. Interviews following UREs and CUREs document the value of asking students to reflect. These interviews sometimes engage undergraduates in reflecting on their experiences for the first time.

Using the knowledge integration framework to interpret the findings, we note that students need mentors who orient them to the practices and concepts of the lab so that they can fully benefit. In many UREs, it takes over a year for students to gain sufficient understanding to make sense of the science practices or concepts of the lab. The few studies that validate self-report findings with other evidence report that research experiences can expand students’ images of the roles available in science (31, 45). This expanded repertoire of opportunities in science has the potential to help undergraduates find ways to self-identify as scientists. However, mentoring to incorporate these images into the students’ identity is related to the amount of contact with a faculty mentor—a scarce resource in most programs. The most promising studies use interviews or journals to elicit ideas and to encourage students to distinguish among their initial ideas and their experiences. These studies suggest that students who are encouraged to articulate their ideas and reflect on their experiences also consolidate their ideas, consistent with the emphasis in the knowledge integration framework on distinguishing ideas and reflecting. Designers could use the features of knowledge integration as criteria for reviewing and improving undergraduate research experiences (see Fig. 2).

Studies of research experiences need the same rigorous designs and assessments as other scientific research. Studies comparing promising alternative approaches to mentoring, ways to illustrate science practices, or methods for orienting participants to a URE placement would advance understanding. Following the knowledge integration framework, studies could compare journals, peer support, structured interviews, and one-on-one mentoring as ways to help students consolidate their understanding.

Comparison studies need appropriate controls to advance our understanding. Thus, comparisons of UREs and CUREs need to account for the differences among participants. For example, most students who meet the selection criteria for UREs have already decided to persist in science; those in CUREs may still be deciding.

The field needs agreed-upon criteria for undergraduate research experiences and validated, generalizable assessments for these criteria. Research to identify measures of research practices that can be used in multiple studies and criteria for evaluating mentoring interactions, as well as methods for measuring ability to locate and interpret primary literature, could advance the field. Promising measures should be tested across investigations and refined.

Disaggregating the populations targeted by undergraduate research experiences could help decision-makers allocate scarce resources. In many studies, the value of undergraduate research cannot be disentangled from precollege preparation. Using good measures of prior understanding and expectations, studies that analyze benefits for subgroups of students could also help those designing research experiences address the unique interests and aspirations of individuals and groups. Such studies could experiment with variations in the selection criteria for research experiences to determine whether current approaches are ideal. In addition, findings about successful experiences need replication and extension, particularly for students from nondominant cultures.

Research suggests a conundrum between mentor availability and mentor impact. Graduate students and postdocs generally mentor undergraduates about scientific practices and research, whereas faculty mentor undergraduates to develop an image of themselves as scientists. Mentors rarely receive guidance about how best to mentor undergraduates. The field would benefit from research that identifies mentoring practices and incorporates them into professional development for mentors, including graduate and postdoctoral researchers. Professional development can help mentors (i) identify and negotiate expectations with their mentees; (ii) explore undergraduate assumptions about research experiences; (iii) monitor student progress; (iv) encourage reflection; and (v) support students emotionally as well as intellectually (51, 52). Methods are needed to allocate credit to faculty for mentoring students and developing effective UREs and CUREs.

Finally, the costs and benefits of research experiences for building human capital, benefiting undergraduates, improving workforce diversity, and strengthening educational outcomes need better understanding. Making the best use of extramural funds and the, often voluntary, contributions of faculty to allocate opportunities and improve undergraduate research experiences requires a strong research base.

References and Notes

  1. Acknowledgments: This material is based upon work supported by the National Science Foundation under grant nos. DRL-1119670, DRL-0918743, and DRL-0822388. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.
View Abstract

Subjects

Navigate This Article