Policy ForumEducation

Closing global achievement gaps in MOOCs

See allHide authors and affiliations

Science  20 Jan 2017:
Vol. 355, Issue 6322, pp. 251-252
DOI: 10.1126/science.aag2063

Advocates for free massive open online courses (MOOCs) have heralded them as vehicles for democratizing education and bridging divides within and across countries (1). More than 25 million people enrolled in MOOCs between 2012 and 2015, including 39% from less-developed countries (LDCs) (2). But the educated and affluent in all countries enroll in and complete MOOCs at relatively higher rates (3, 4). Judged by completion rates, MOOCs do not spread benefits equitably across global regions. Rather, they reflect prevailing educational disparities between nations (see the first chart) (5). Although the global achievement gap could be caused by barriers in LDCs, such as less broadband Internet access, formal education, and English proficiency, we explore another potential but underappreciated cause. Members of LDCs may suffer from the cognitive burden of wrestling with feeling unwelcome while trying to learn and, therefore, underperform. This can be exacerbated by social identity threat, which is the fear of being seen as less capable because of one's group (6). We discuss field experiments with interventions that targeted social identity threat and caused substantial improvements in MOOC persistence and completion rates among learners in LDCs, eliminating the global achievement gap.

Social identity threat can impair working memory, learning, and performance (7, 8) and contribute to academic achievement gaps based on students' race, gender, and social status (6). Brief psychological interventions have been shown to improve performance of members of identity-threatened groups in the United States, such as African Americans and women in male-dominated fields (911). How social identity threat unfolds at an international level has received little scholarly attention. An initial survey found that members of LDCs expressed greater concern about being seen negatively in MOOCs on the basis of their nationality than did members of more-developed countries (MDCs) (5).

Multiple aspects of MOOCs, most of which are offered in English by North American providers, may lead members of LDCs to experience social identity threat. Stereotypes that cast members of LDCs as less competent in school and work may raise stereotype threat, a specific form of social identity threat (6). Courses may be poorly tailored to non-Western cultures, with a preponderance of North American content, opinions, and pedagogical ideas (12). Other cues, such as university logos or photographs, can convey that one is seen as an outsider and raise social identity threat (13).

MOOC completion is higher on average in more-versus less-developed countries.

National average course completion rate, as a function of UN Human Development Index, of 1.8 million learners enrolled in 55 MOOCs created by Stanford University and offered on the Coursera platform between 2013 and 2015. Each point represents a country with size proportionate to the number of learners (points with n < 100 excluded). A weighted best-fit line is shown (r = 0.78, P < 0.001) (5).

GRAPHIC: G. GRULLÓN/SCIENCE

In both of our experiments, we randomly assigned some learners to complete activities known to lessen social identity threat, while others completed control activities (5). Learners were guided through a writing activity that was embedded in a course survey at the beginning of each MOOC. One activity encouraged learners to affirm cherished personal values, such as relationships with family. When people write about core values, their self-concept becomes more expansive, making a particular threat less consequential for their sense of self-integrity (14, 15). We adapted this intervention to help reduce attrition by having participants write about how taking the course reflects and serves their values (16). For example, one wrote, “I need this course mostly for changing my career, which will help me spend more time with family” (5).

A second activity assured learners that doubts about belonging in the course are normal, short-lived, and not unique to them or members of their group (11). Participants read ostensible testimonials from advanced learners about how they worried about whether they “belonged” in the course at first but with time became comfortable. Participants summarized the key themes in the testimonials and then, to help them internalize the message, they integrated the themes into a short letter of advice to future learners [see materials and selected responses in (5)].

The first experiment was conducted in a MOOC related to computer science and offered by Stanford University in 2014. The course was self-paced with no option to earn a certificate for completion. It offered a vast set of materials: 286 lectures and assessments in total. The outcome measure was the number of course materials that learners engaged with, which reflects course persistence. We replicated the experiment a year later in a second course featuring a different topic, U.S. public policy, and an alternative course format. This course, offered by Harvard University, followed a cohort-based format, with a strict start date and a 6-week schedule that encompassed eight sequential “chapters.” Course persistence was assessed as the number of chapters that learners engaged with. The format of the second course permitted measurement of an additional outcome, course completion, achieved by earning a sufficiently high grade on multiple assessments.

In both experiments, participants were randomly assigned to one of three conditions: the value relevance affirmation, the social-belonging intervention, or the control condition in which they read and wrote about study skills, an activity shown to have no impact on performance (17). Of the participants, 75% completed the intervention in fewer than 10 minutes (median = 4.5 minutes). We analyzed data of 2286 participants (16% in LDCs) in the first experiment and 1165 participants (7.4% in LDCs) in the replication experiment (5). Across both experiments, the most-represented LDCs were India, Pakistan, and Egypt, and the most-represented MDCs were the United States, Canada, and the United Kingdom. Country-level location information was determined by learners' IP address. LDC or MDC status was defined according to thresholds established by the United Nations along the Human Development Index (HDI), based on national income, education, and health levels (MDC: HDI > 0.7, “high” and “very high”; LDC: HDI < 0.7, “medium” and “low”). We compared the outcomes in LDCs and MDCs in the three experimental conditions using linear regression with robust standard errors. Results were robust to alternative analytic techniques (5).

The interventions had large effects consistent with predictions, eliminating the global achievement gap in both experiments (see the second chart). In the first experiment, both interventions doubled persistence for learners in LDCs (social belonging: z = 2.68, P = 0.007; affirmation: z = 2.82, P = 0.005), and as predicted, did not affect persistence for learners MDCs (z values < 0.36, P values > 0.71]. Thus, the significant global gap in persistence in the control condition [95% confidence interval (CI) = (2.29, 6.17), z = 4.28, P < 0.001] was closed both in the social-belonging condition [95% CI = (−4.22, 3.10), z = −0.30, P = 0.765] and in the affirmation condition [95% CI = (−0.94, 3.29), z = 1.09, P = 0.276].

In the replication experiment, the gap in persistence in the control condition [95% CI = (0.74, 2.33), z = 3.78, P < 0.001] was again closed in both the social-belonging condition [95% CI = (−1.29, 0.93), z = −0.32, P = 0.749] and the affirmation condition [95% CI = (−1.78, 0.22), z = −1.52, P = 0.128]. The social-belonging intervention increased persistence for LDC learners by 51% [z = 2.09, P = 0.036], without significantly affecting persistence for MDC learners [z = −1.67, P = 0.094]. Although the affirmation intervention reduced persistence for MDC learners by 12% [z = −2.66, P = 0.008], it increased persistence for LDC learners by 67% [z = 2.91, P = 0.004].

Interventions closed the average global gap between members of more- versus less-developed countries

Course persistence and completion outcomes of two randomized experiments in MOOCs (total N = 3,451). Error bars represent ±1 standard error (5).

GRAPHIC: G. GRULLÓN/SCIENCE

The global gap in course completion in the control condition was 15 percentage points (pp) [95% CI = (1, 30), z = 2.12, P = 0.034]. In the social-belonging condition, this gap was reduced to 2 pp [95% CI = (−17, 21), z = 0.19, P = 0.847], although the intervention's effect was not significant for LDC learners [z = 0.91, P = 0.364] or MDC learners [z = −0.84, P = 0.404]. In the affirmation condition, the global gap in completion was reversed to −18 pp [95% CI = (−35, −1), z = −2.04, P = 0.042]. Although the affirmation intervention reduced the completion rate in MDCs from 32% to 23% [z = −2.60, P = 0.009], it increased completion for LDC learners from 17% to 41% [z = 2.26, P = 0.024].

Although the affirmation had a consistent positive effect for LDC learners, it had a negative effect in MDC learners in the replication experiment. Prior research suggests that affirmation can cause disengagement, particularly for those who are not under psychological threat or who see little possibility to improve (18). These effects have proved inconsistent in field research (10), but insofar as they occur, several reasons are plausible. Absent social identity threat, affirmations may highlight other domains in which people can invest their efforts. This process is especially likely for those whose commitment to the course is only tenuous; indeed, commitment to course completion was lower for learners in MDCs than LDCs (5). Ultimately, as these interventions are brief and involve private writing activities, they can be delivered only to learners expected to benefit.

Beyond material improvements, such as better broadband access, improvements to the psychological climate can improve online educational equity. The interventions took only a few minutes for course designers to implement and for learners to complete. Social identity threat appears to be a barrier to performance in an international learning context, even an online environment with little social interaction. Psychological and learning sciences can help turn an accessible educational experience into an equitable one.

The design of inclusive digital learning environments requires more research on the experiences and cues that trigger and alleviate social identity threat. Even in the absence of threatening cues, learners may wonder if they belong. Course designers may need to take active steps to help learners feel welcome in global learning environments.

References and Notes

  1. Materials and methods are available as supplementary materials.
  2. Acknowledgments: We thank three anonymous reviewers for constructive feedback.
View Abstract

Stay Connected to Science

Navigate This Article