Measuring and managing bias

See allHide authors and affiliations

Science  01 Sep 2017:
Vol. 357, Issue 6354, pp. 849
DOI: 10.1126/science.aap7679

As someone who grew up with a mother who was a medical researcher, who has been married to a woman very active in scientific research for more than 30 years, and who has had many female colleagues and students, I was surprised when I first took a test to measure implicit gender bias and found that I have a strong automatic association between being male and being involved in science. We all carry a range of biases that are products of our culture and experiences or, in some cases, of outcomes that we desire. Fortunately, many such biases can be measured and, in some cases, effectively managed. A key is to first acknowledge their presence and then to take steps to minimize their influence on important decisions and results.


“Implicit biases…should be acknowledged and managed, rather than denied or ignored.”

Implicit biases—those that we are not consciously aware of—might seem difficult to demonstrate or quantify. However, implicit association tests (IATs; see, for example, https://implicit.harvard.edu/implicit/aboutus.html) can be a useful tool for achieving this. IATs are based on measuring the times needed to classify attributes in a simple computer exercise that takes about 5 minutes to complete. For example, the Gender-Science IAT that I took measures the strength of association between someone being male or female and being involved in science versus liberal arts. On the numerous occasions that I have taken this IAT over a decade, I have found the same strong automatic association between male and science and with female and liberal arts. The good news is that direct awareness of one's own implicit biases can reduce their impact on outcomes, at least in some circumstances. I sometimes catch myself assuming that a scientist is male, and then remind myself of my implicit bias test results, and try to think deliberately to avoid making such assumptions.

Another method for dealing with implicit bias involves removing the possibility that these biases can exert any influence. “Blinding” prevents any individual involved in performing a task from knowing attributes that might be associated with bias. Blinding is used frequently in human clinical trials. In addition, studies can be “randomized” so that the treatment regimen that each subject receives is selected at random to increase the likelihood that the groups subjected to each regimen are as similar as possible.

In contrast to human studies, few animal studies are performed in randomized or blinded fashions because of factors related to training or convenience. However, analyses comparing a large number of animal studies that were or were not randomized or blinded have revealed that apparent effect sizes tend to be smaller when randomization or blinded approaches were used. These observations suggest that biases, likely implicit, are causing some results to appear more robust than they are. Over the past year, several colleagues who have worked in industrial settings have told me that their organizations now strongly encourage randomization and blinding in animal studies and that this can often be accomplished with little additional cost or inconvenience. Their anecdotal accounts support the notion that these approaches can be readily implemented, potentially producing more robust results.

Implicit biases are intrinsic human characteristics that should be acknowledged and managed, rather than denied or ignored. Everyone should consider taking one or more IATs to understand this approach and measure his or her own implicit biases. I have found this to be enlightening and have encouraged Science editors to do the same to increase awareness of their own implicit biases. In a different arena, those involved in research should consider randomizing and blinding experiments, including animal and other studies, whenever feasible. These relatively simple steps may help strengthen the scientific enterprise and increase research reproducibility.


Navigate This Article