EDITORIAL

A Perverted View of “Impact”

Science  14 Jun 2013:
Vol. 340, Issue 6138, pp. 1265
DOI: 10.1126/science.1240456
CREDIT: COURTESY OF HARVARD MEDICAL SCHOOL

Scientists often face vexing professional decisions: whom to hire, what to fund, what to publish, and whom to promote. Because science is about the unknown and its greatest discoveries are often the least expected, scientists often have little to go by except intuition and experience. For this reason, a seductively simple template has recently been introduced: assessment based on “impact and significance.” Thus, the U.S. National Institutes of Health has elevated “significance” to an explicit criterion in funding decisions. It requires that grant reviewers write a paragraph on “impact,” which it defines as the likelihood that the proposed work will have a “sustained and powerful influence.”* Especially in fundamental research, which historically underlies the greatest innovation, the people doing the work often cannot themselves anticipate the ways in which it may bring human benefit. Thus, under the guise of an objective assessment of impact, such requirements invite exaggerated claims of the importance of the predictable outcomes—which are unlikely to be the most important ones. This is both misleading and dangerous.

CREDIT: EVERGIN/ISTOCKPHOTO

One may be able to recognize good science as it happens, but significant science can only be viewed in the rearview mirror. To pretend otherwise distorts science. DNA restriction enzymes, once the province of obscure microbiological investigation, ultimately enabled the entire recombinant DNA revolution. Measurement of the ratios of heavy and light isotopes of oxygen, once a limited area of geochemistry, eventually allowed the interpretation of prior climate change. What is now promoted as high-impact science is usually a narrow extension of existing experimental designs in a program focused on a set of feasible goals. Fuzzy new directions that might fail, but could open up major new questions, are often dismissed as too speculative and considered low-impact. And in biomedical science, there is an increasing tendency to equate significance to any form of medical relevance. This causes biochemical investigations and research on nonmammalian systems to be treated as intrinsically less valuable than studies on human cells. As a result, biomedicine is losing the historically productive cross-fertilization between model systems and human biology.

In science, faster, better, and cheaper are not as important as conceptual, novel, and careful. Focusing resources narrowly on areas that are deemed impactful, while ignoring many others, decreases diversity, making science less productive. Assessments based primarily on impact may also be contributing to an apparent epidemic of irreproducible results in the biomedical literature. Reviewers and editors increasingly insist on major extensions of the submitted work in order to inflate its (narrowly defined) impact, while at the same time making such extensions a condition for acceptance. In today's competitive job and grant market, these demands create a strong inducement for sloppy science.

What ails science today requires an honest diagnosis. Scientists are failing to live up to the trust society has placed in them. The scientific community must create leadership with the courage and independence to take control of the structure of its training, the peer-reviewing of its journals, the organization of grant review panels, and the overall priorities that are set. There are strong political, economic, and institutional interests that are not shy about asserting themselves. Scientists have to be equally assertive and even more persuasive.

I also believe, along with Huda Zoghbi, that scientists must challenge the assumption that translation, rather than fundamental understanding, is the choke point of progress in the application of science to societal problems. They should work hard to encourage risk and exploration, while at the same time rewarding careful, thoughtful investigation. And they should reemphasize humility, banishing the words “impact” and “significance” and seeing them for what they really are: ways of asserting bias without being forced to defend it.

Related Content

Subjects

Navigate This Article