Science  12 Jul 2013:
Vol. 341, Issue 6142, pp. 109
DOI: 10.1126/science.1242553

How much risk can and should a journal tolerate in publishing papers that describe novel findings—that is, papers that could have a profoundly positive impact within and outside the scientific community if right, but could be broadly harmful by leading investigators in wrong directions if incorrect? I recently engaged a group of the Science editors in a lively discussion on this topic.

We agreed that publishing papers with some such risk is a good thing. Of course, a journal would love for every paper it publishes to turn out to be perfectly correct—but not at the expense of publishing papers that are all perfectly “safe.” Science moves forward by communicating findings that challenge old ideas and force us to test new theories against the evidence. The key is to contain that risk.

The sources of risk and how they are best managed vary within scientific disciplines, notably in my experience between the observational fields versus the experimental fields. Experiments take advantage of controls, use multiple replicates, vary initial conditions and independent variables, and hold constant the factors that might otherwise confound results. Barring outright fabrication, results from the experiments should be reproducible to within known uncertainties. Because the experiments have been designed to test the authors' hypotheses, there is generally a relevant result. Many of these investigations are conducted with great effort and creativity. The paper may still be risky, but the risk is generally quantifiable.


On the other hand, scientific advances that depend on new observations are inherently limited by their availability and quality. We have only one Earth to study, not multiple independent realizations and no “control” planet from which to gain statistical reliability. The initial conditions are lost in time and cannot be determined with any certainty. Astrophysicists and ecologists, for example, face challenges in which the relevant space scales are larger than any laboratory and the time scales exceed human lifetimes. The observations available are not the ideal data set for testing the most pressing questions that need to be answered. The scientists must test hypotheses with the data that they have discovered, not the information that they want to have. And yet the questions that need to be answered are too important to be ignored merely because the definitive experiment cannot be designed.

In fact, the publication of provocative interpretations and the desire to test their validity can prompt the development of new observational tools, which in turn can drive progress in observational fields. The encouragement of risk does not mean that Science wants submissions that are unsubstantiated by data, and clearly we must continue to strive for the highest standards in scientific peer review. But as a brilliant marine geologist once told me, “I refuse to be held responsible for prior interpretations which I have now revised based on newer and more complete information.”

I worry that the judgment of whether or not a paper carries an acceptable amount of risk can be clouded by preconceived bias—on the part of the authors, the reviewers, or even the editors. Biases make it hard to determine whether the risk that the data do not support the conclusions in the paper is real or imagined. Years ago when a decades-long debate raged in the geoscience community about whether the seismic discontinuity at 670 kilometers formed a barrier to convection in Earth's mantle, a former mentor asserted that he was one of the few participants in the conversation who was intellectually honest because he had changed his mind as new observations had come to light. He postulated that some of his colleagues were practicing religion, not science. I urge authors and reviewers to work earnestly with the editors at Science to make good decisions in risky matters by examining the evidence with an open, unbiased mind.


Navigate This Article