EDITORIAL

A swan in the making

Science  22 Aug 2014:
Vol. 345, Issue 6199, pp. 855
DOI: 10.1126/science.1259740
PHOTO: WENDY GARDNER

Reproducibility is the ugly duckling of science. It provokes distress, denial, and passionate calls for action. With $1.5 trillion spent globally each year on R&D,* the idea that 80% of it is irreproducible can cause downright dread. It threatens the foundations and credibility of the scientific enterprise. But look past the surface, and reproducibility may well be a swan in the making.

In my previous role in industrial microbe and process engineering, I saw quality practices double the productivity of R&D and deliver microbes from half-liter fermentations to 200,000-liter manufacturing with absolute performance reproducibility. That kind of outcome is electrifying.

“Achieving reproducible research…will take a collective movement to change the incentives, culture, and tool sets of science.”

PHOTO: ANDREAS KRONE/ISTOCKPHOTO.COM

Achieving such reproducibility in every lab may offer one of the greatest opportunities in decades to advance the practice of science. The path to transformation already exists; we need only look into the “unsexy” world of manufacturing to learn.

After World War II, the teachings of engineer W. Edwards Deming on quality helped revolutionize a Japanese manufacturing industry then known for cheap, unreliable products. By the 1980s, the quality of Japanese products was shaking the foundations of American electronics and car companies. Today, such uncompromising quality is ubiquitous and expected.

This kind of quality didn't “just happen.” When I was an intern at a General Motors subsidiary in the 1980s, the halls were rife with complaints that it was impossible to meet new and stricter specifications. Discussions today in the halls of science are not so different. Researchers complain about the seemingly impossible task of tracking all the variables needed to ensure reproducibility; they question whether the obsession with reproducibility is overkill; and they fear that the methods of quality improvement will snuff out the flexibility needed to innovate.

Such reactions are actually quite correct. Today's methods of manufacturing quality are too cumbersome and too restrictive for the dynamic and evolutionary nature of scientific thinking.

It's time to fix that.

Behavioral economics has shown that removing even trivial hurdles can vastly improve the adoption of beneficial practices. In one randomized study, people were 29% more likely to attend college when their student aid forms were automatically populated. Similarly, the digital capabilities in your pocket can lower the barriers to quality by automating most of its complexity. For example, my own company is building tools to automate data acquisition via mobile devices, break down lab procedures into modular steps that are evolvable and reusable like lines of software code, and automatically analyze data trends within and across experiments to catch errors and their root causes. Other companies are also contributing to a fast-growing ecosystem of related tools.

Achieving reproducible research—research we can trust and build on efficiently, like high-quality parts in a supply chain—will take a collective movement to change the incentives, culture, and tool sets of science. The good news is that movement has begun. Publishers, funders, foundations, universities, regulators, and companies are raising visibility, developing standards, and creating tools and incentives that make reproducible research more accessible and more rewarding.

It is often said that the first step in solving a problem is recognizing that you have one. Step one…check. The scientific community is aware and taking action. The ugly duckling is becoming a swan.

  • * M. Grueber, T. Studt, R&D Magazine (December 2012).

  • F. Prinz et al., Nat. Rev. Drug Discovery 10, 712 (2011).

  • E. P. Bettinger et al., The Role of Simplification and Information in College Decisions (National Bureau of Economic Research, Cambridge, MA, 2009).

Subjects

Navigate This Article