Technical Comments

Response to Comment on “Retrieval Practice Produces More Learning than Elaborative Studying with Concept Mapping”

See allHide authors and affiliations

Science  28 Oct 2011:
Vol. 334, Issue 6055, pp. 453
DOI: 10.1126/science.1204035


Mintzes et al. comment on our study in which we showed that retrieval practice enhances meaningful learning more than elaborative studying with concept mapping. Here, we consider and rebut claims that are based on mischaracterizations of our paper and speculations rather than evidence. We emphasize that randomized, controlled studies in both laboratory and classroom settings are essential to identifying effective strategies that promote meaningful learning.

A crucial goal of educational research is to identify effective strategies for promoting meaningful learning. The Comment by Mintzes et al. (1) gives us the opportunity to address several misunderstandings and mischaracterizations of our research (2). Readers of their Comment might think we claimed that two processes (elaboration and retrieval) or two tasks (concept mapping and free recall) are mutually exclusive and that one should be used “in lieu of” other activities. However, our study drew no such conclusions.

A wealth of research has shown that practicing retrieval enhances learning (3, 4). We tested the mettle of retrieval practice by comparing it to elaborative studying with concept mapping because concept mapping has been widely advocated for years. Practicing retrieval produced more learning than elaborative concept mapping on measures of long-term, meaningful learning that assessed conceptual knowledge, inference making, and knowledge application. Many students were not aware of the benefits of retrieval practice, believing instead that elaborative studying, or even simple repetitive reading, would produce more learning than practicing retrieval.

Mintzes et al. (1) criticize our study because we did not extensively train subjects on concept mapping. They imply that perhaps 10 to 15 weeks of training is necessary. This is puzzling, because many papers, including those by the authors, emphasize that concept mapping requires minimal training (510). In (8), the authors write, “The technique [concept mapping] is easily taught” and can be learned “in a relatively brief period of time” (p. 119). In (10), the authors claim that “virtually all college students can learn to construct a simple concept map in a single 50-minute class period” (p. 12) Another paper (5) claims, “The method can be taught in 10–20 minutes” (p. 302). In (6), the authors write, “The technique is relatively intuitive and requires only a basic understanding of relationships and class inclusion principles” (p. 533). The list of similar quotes goes on. We believe that concept mapping is an effective task for producing elaborative studying with a brief, sufficient amount of instruction, as we provided.

Mintzes et al.’s comment about training, however, raises an important issue: There appears to be no evidence from randomized, controlled experiments that extensive training on concept mapping is necessary. The studies cited by Mintzes et al. (7, 10) are not controlled experiments and did not examine training effects. These studies show that students’ concept maps can improve over time (e.g., over the course of a semester), but that does not mean students must be trained for concept mapping to work. We cannot find any studies that manipulated training to examine whether it enhances the effectiveness of concept mapping. Given the importance of identifying the best ways to implement effective strategies, it is surprising that the role of training in concept mapping has not been rigorously examined.

Mintzes et al. (1) speculate that students are more familiar with retrieval practice than they are with concept mapping. However, it is questionable to assume that students are highly familiar with retrieval practice. Research on students’ metacognitive awareness consistently shows that most students lack awareness of the benefits of retrieval practice and do not use this strategy (11, 12). More important, familiarity with a learning activity need not have anything to do with its effectiveness. For instance, repetitive reading is the most frequently reported strategy among college students (12), yet there are numerous studies in which this familiar strategy produces little or no learning (13). Consider another example: Deleting letters from expository texts improves learning and comprehension because of the generative processing afforded by letter deletion (14). We are confident that students do not commonly read texts with missing letters, yet the activity boosts learning. Mintzes et al. assume a causal link between task familiarity and learning, but that assumption is incorrect.

Mintzes et al. note that we examined long-term retention after 1 week. We agree that it is important to examine longer retention intervals, but without any data to speak to the issue directly, this criticism is speculative. There is no objective criterion for determining the “correct” interval for measuring learning. Does assessment of meaningful learning begin after 2 weeks, a month, a semester or two, or years? Moreover, there is no reason to expect that the pattern of results would reverse and that elaborative concept mapping would be superior to retrieval practice at longer intervals. Retrieval practice enhances learning after long intervals, weeks and months after initial learning (1519), by slowing down the rate of forgetting (19). Therefore, we predict that the advantage of retrieval practice over elaborative concept mapping will only grow larger at longer retention intervals.

Mintzes et al. further imply that we did not measure meaningful learning. We used ideal methods for measuring meaningful, long-term learning widely used in educational research (20). Our second experiment used concept mapping as a criterial measure of meaningful learning, just as the authors have done in their research (610). It is puzzling that the authors regularly use concept mapping to assess meaningful learning yet imply that we did not measure meaningful learning using the same method.

Mintzes et al. say we “imply that science teachers fail to understand the critical role of retrieval.” Certainly, some students and educators understand that retrieval produces learning, but the evidence suggests that this is the exception, not the rule (12). We did not offer recall testing as a solution or prescription for education. Practicing retrieval greatly enhances learning, but we did not claim that recall testing was the only solution or that retrieval practice, however implemented, should be used in lieu of other activities. Indeed, we explicitly said that concept mapping could be used as a retrieval activity.

Mintzes et al. conclude with speculations about the relation between laboratory research and classroom practice. Their comments apply to all research on learning, not just our paper. They suggest that retrieval practice effects observed in the laboratory must be generalized and established in classrooms. We agree—and, fortunately, that has been done. The effectiveness of retrieval practice has been established in randomized, controlled experiments in classrooms (1518).

We recommend that concept mapping and other strategies be held to the same criterion: The effectiveness of educational interventions should be broadly established in controlled experiments in laboratories and classrooms. Numerous papers have used concept mapping in classrooms (610), but that is not the same as conducting randomized, controlled experiments to permit inferences about the effectiveness of concept mapping relative to other plausible educational techniques. There are far fewer randomized, controlled experiments on concept mapping than there are papers reporting demonstrations, opinions, and observations.

We believe that it is imperative to compare and contrast instructional strategies and identify ones that lead to durable, meaningful learning. We compared retrieval practice to concept mapping as one instantiation of this strategy. We hope that future experimental efforts will help lead the way to better instructional strategies using retrieval practice, concept mapping, and other educational techniques.

References and Notes

  1. Acknowledgments:The writing of this paper was supported by grants from the National Science Foundation (0941170) and the Institute of Education Sciences in the U.S. Department of Education (R305A110903). The opinions expressed are those of the authors and do not represent views of the Institute of Education Sciences or the U.S. Department of Education.
View Abstract

Navigate This Article