Report

Prevalence-induced concept change in human judgment

See allHide authors and affiliations

Science  29 Jun 2018:
Vol. 360, Issue 6396, pp. 1465-1467
DOI: 10.1126/science.aap8731

Perceptual and judgment creep

Do we think that a problem persists even when it has become less frequent? Levari et al. show experimentally that when the “signal” a person is searching for becomes rare, the person naturally responds by broadening his or her definition of the signal—and therefore continues to find it even when it is not there. From low-level perception of color to higher-level judgments of ethics, there is a robust tendency for perceptual and judgmental standards to “creep” when they ought not to. For example, when blue dots become rare, participants start calling purple dots blue, and when threatening faces become rare, participants start calling neutral faces threatening. This phenomenon has broad implications that may help explain why people whose job is to find and eliminate problems in the world often cannot tell when their work is done.

Science, this issue p. 1465

Abstract

Why do some social problems seem so intractable? In a series of experiments, we show that people often respond to decreases in the prevalence of a stimulus by expanding their concept of it. When blue dots became rare, participants began to see purple dots as blue; when threatening faces became rare, participants began to see neutral faces as threatening; and when unethical requests became rare, participants began to see innocuous requests as unethical. This “prevalence-induced concept change” occurred even when participants were forewarned about it and even when they were instructed and paid to resist it. Social problems may seem intractable in part because reductions in their prevalence lead people to see more of them.

The deformation of a solid under load is known as “creep.” But in the past few years, that term has crept beyond materials science and has come to describe almost any kind of unintended expansion of a boundary. Software developers worry about feature creep (the unintended expansion of a product’s function over time), project managers worry about scope creep (the unintended expansion of a team’s mandate over time), and military commanders worry about mission creep (the unintended expansion of a campaign’s objectives over time). As it turns out, abstract concepts can creep, too. For example, in 1960, Webster’s dictionary defined “aggression” as “an unprovoked attack or invasion,” but today that concept can include behaviors such as making insufficient eye contact or asking people where they are from (1). Many other concepts, such as abuse, bullying, mental disorder, trauma, addiction, and prejudice, have expanded of late as well (2). Some take these expansions as signs of political correctness and others as signs of social awakening. We take no position on whether these expansions are good or bad. Rather, we seek to understand what makes them happen. Why do concepts creep?

Psychologists have long known that stimuli are judged in the context of the other relevant stimuli that surround them in space or precede them in time (38), and the perceived aggressiveness of a particular behavior will naturally depend on the aggressiveness of the other behaviors the observer is seeing or has seen. When instances of a concept become less prevalent—for example, when unprovoked attacks and invasions decline—the context in which new instances are judged changes as well. If most behaviors are less aggressive than they once were, then some behaviors will seem more aggressive than they once did, which may lead observers to mistakenly conclude that the prevalence of aggression has not declined. When instances of a concept become less prevalent, the concept may expand to include instances that it previously excluded, thereby masking the magnitude of its own decline.

This phenomenon—which we call “prevalence-induced concept change”—can be a problem. When yellow bananas become less prevalent, a shopper’s concept of “ripe” should expand to include speckled ones, but when violent crimes become less prevalent, a police officer’s concept of “assault” should not expand to include jaywalking. What counts as a ripe fruit should depend on the other fruits one can see, but what counts as a felony, a field goal, or a tumor should not, and when these things are absent, police officers, referees, and radiologists should not expand their concepts and find them anyway. Modern life often requires people to use concepts that are meant to be held constant and should not be allowed to expand (916). Alas, research suggests that the brain computes the value of most stimuli by comparing them to other relevant stimuli (1719); thus, holding concepts constant may be an evolutionarily recent requirement that the brain’s standard computational mechanisms are ill equipped to meet (20, 21).

Are people susceptible to prevalence-induced concept change? To answer this question, we showed participants in seven studies a series of stimuli and asked them to determine whether each stimulus was or was not an instance of a concept. The concepts ranged from simple (“Is this dot blue?”) to complex (“Is this research proposal ethical?”). After participants did this for a while, we changed the prevalence of the concept’s instances and then measured whether the concept had expanded—that is, whether it had come to include instances that it had previously excluded.

In Study 1, we showed participants a series of 1000 dots that varied on a continuum from very purple to very blue (see fig. S1) and asked them to decide whether each dot was or was not blue. After 200 trials, we decreased the prevalence of blue dots for participants in the decreasing prevalence condition but not for participants in the stable prevalence condition. Figure 1 shows the percentage of dots at each point along the continuum that participants identified as blue on the initial 200 trials and on the final 200 trials. The two curves in Fig. 1A are nearly perfectly superimposed, indicating that participants in the stable prevalence condition were just as likely to identify a dot as blue when it appeared on an initial trial as when it appeared on a final trial. But the two curves in Fig. 1B are offset, indicating that participants in the decreasing prevalence condition were more likely to identify dots as blue when those dots appeared on a final trial than when those dots appeared on an initial trial. In other words, when the prevalence of blue dots decreased, participants’ concept of blue expanded to include dots that it had previously excluded. Complete methods and results for this and all subsequent studies may be found in the supplementary materials.

Fig. 1 Results for Study 1.

(A) shows the stable prevalence condition, and (B) shows the decreasing prevalence condition. The x axes show the dot’s objective color, and the y axes show the percentage of trials on which participants identified that dot as blue.

In Studies 2 through 5, we examined the robustness of this effect. In Study 2, we replicated the procedure for Study 1, except that instead of telling participants in the decreasing prevalence condition that the prevalence of blue dots “might change” over trials, we told them that the prevalence of blue dots would “definitely decrease” over trials. The effect seen in Study 1 was replicated (see fig. S2). In Study 3, we replicated the procedure for Study 1, except that this time a third of the participants in the decreasing prevalence condition were explicitly instructed to “be consistent” and to not allow their concept of blue to change over the course of the study, and another third were given the same instruction and were also offered a monetary incentive for following it. In all conditions, the effect seen in Study 1 was replicated (see fig. S3). In Study 4, we replicated the procedure for Study 1, except that this time we decreased the prevalence of blue dots gradually for some participants (as we did in the previous studies) and abruptly for others. In all conditions, the effect seen in Study 1 was replicated (see fig. S4). Finally, in Study 5, we replicated the procedure for Study 1, except that this time instead of decreasing the prevalence of blue dots, we increased the prevalence of blue dots. As expected, this change reversed the effect seen in Study 1: When the prevalence of blue dots was increased, participants were less likely to identify a dot as blue when it appeared on a final trial than when it appeared on an initial trial (see fig. S5). In short, the prevalence-induced concept change seen in Study 1 proved remarkably robust and was not eliminated by forewarning (Study 2), by instructions and incentives (Study 3), by sudden decreases in prevalence (Study 4), or by a reversal in the direction of the change in prevalence (Study 5).

Does this finding generalize from simple concepts to complex ones? To find out, in Study 6, we showed participants a series of 800 computer-generated human faces that (according to raters) varied on a continuum from very threatening to not very threatening (see fig. S6). We asked participants to determine whether the person whose face they saw (the “target”) was or was not a threat. After 200 trials, we decreased the prevalence of threatening targets for participants in the decreasing prevalence condition but not for participants in the stable prevalence condition. Figure 2 shows the percentage of targets at each point on the continuum whom participants identified as a threat on the initial 200 trials and on the final 200 trials. Participants in the stable prevalence condition (Fig. 2A) were just as likely to identify a target as a threat when that target appeared on a final trial as when that target appeared on an initial trial, but participants in the decreasing prevalence condition (Fig. 2B) were more likely to identify a target as a threat when the target appeared on a final trial than when the target appeared on an initial trial. In other words, when the prevalence of threatening targets decreased, participants’ concept of threat expanded to include targets that it had previously excluded.

Fig. 2 Results for Study 6.

(A) shows the stable prevalence condition, and (B) shows the decreasing prevalence condition. The x axes show the target’s objective threateningness (as determined by human raters), and the y axes show the percentage of trials on which participants identified that target as a threat.

The foregoing studies suggest that concepts expand when the prevalence of their instances decreases. Does this effect also occur when people are asked to make decisions about purely conceptual rather than visual stimuli? To find out, in Study 7 we asked participants to play the role of a reviewer on an Institutional Review Board. We showed participants a series of 240 proposals for scientific studies that (according to raters) varied on a continuum from very ethical to very unethical, and we asked participants to decide whether researchers should or should not be allowed to conduct the study. After 96 trials, we decreased the prevalence of unethical proposals for participants in the decreasing prevalence condition but not for participants in the stable prevalence condition. Figure 3 shows the percentage of proposals that participants rejected on the initial 48 trials and on the final 48 trials. Participants in the stable prevalence condition (Fig. 3A) were just as likely to reject ethically ambiguous proposals that appeared on a final trial and on an initial trial, but participants in the decreasing prevalence condition (Fig. 3B) were more likely to reject ethically ambiguous proposals that appeared on a final trial than on an initial trial. In other words, when the prevalence of unethical research proposals decreased, participants’ concept of unethical expanded to include proposals that it had previously excluded.

Fig. 3 Results for Study 7.

(A) shows the stable prevalence condition, and (B) shows the decreasing prevalence condition. The x axes show the proposal’s objective ethicality (as determined by raters), and the y axes show the percentage of trials on which participants rejected the proposal.

Across seven studies, prevalence-induced concept change occurred when it should not have. When blue dots became rare, purple dots began to look blue; when threatening faces became rare, neutral faces began to appear threatening; and when unethical research proposals became rare, ambiguous research proposals began to seem unethical. This happened even when the change in the prevalence of instances was abrupt, even when participants were explicitly told that the prevalence of instances would change, and even when participants were instructed and paid to ignore these changes.

These results may have sobering implications. Many organizations and institutions are dedicated to identifying and reducing the prevalence of social problems, from unethical research to unwarranted aggressions. But our studies suggest that even well-meaning agents may sometimes fail to recognize the success of their own efforts, simply because they view each new instance in the decreasingly problematic context that they themselves have brought about. Although modern societies have made extraordinary progress in solving a wide range of social problems, from poverty and illiteracy to violence and infant mortality (22, 23), the majority of people believe that the world is getting worse (24). The fact that concepts grow larger when their instances grow smaller may be one source of that pessimism.

Supplementary Materials

www.sciencemag.org/content/360/6396/1465/suppl/DC1

Materials and Methods

Figs. S1 to S6

References (2631)

References and Notes

Acknowledgments: We thank T. Brady and M. Thornton for technical assistance and M. Banker, S. Carroll, P. Chan, R. Chmielinksi, A. Collinsworth, K. da Silva, I. Droney, C. Fitzgerald, S. Ganley, M. Graether, J. Green, L. Harris, U. Heller, T. Hicks, S. Hoffman, E. Kemp, B. Kollek, R. Lisner, Z. Lu, B. Martinez, T. Murphy, D. Park, D. Peng, M. Powell, M. Sanders, C. Shaw, G. Stern, R. Stramp, A. Strauss, L. Symes, H. Tieh, A. Wang, I. Wang, C. Wu, X. Zeng, and Y. Zheng for research assistance. Statistical support was provided by data science specialists S. Worthington and I. Zahn at the Institute for Quantitative Social Science, Harvard University. Funding: We acknowledge the support of National Science Foundation grant BCS-1423747 to T.D.W. and D.T.G. Author contributions: All authors contributed to the conceptual development of the work, D.E.L. collected and analyzed the data, and D.E.L. and D.T.G. designed the experiments and wrote the manuscript. Competing interests: None. Data and materials availability: The complete materials and data for all studies are available at Zenodo, https://zenodo.org/record/1219833 (25).
View Abstract

Navigate This Article