Reviewing Peer Review

See allHide authors and affiliations

Science  04 Jul 2008:
Vol. 321, Issue 5885, pp. 15
DOI: 10.1126/science.1162115
CREDIT: GETTY IMAGES

Peer review, in which experts in the field scrutinize and critique scientific results prior to publication, is fundamental to scientific progress, and the achievements of science in the last century are an endorsement of its value. Peer review influences more than just science. The Intergovernmental Panel on Climate Change and other similar advisory groups base their judgments on peer-reviewed literature, and this is part of their success. Many legal decisions and regulations also depend on peer-reviewed science. Thus, thorough, expert review of research results—without compensation—is an obligation that scientists shoulder for both science and the general public.

Despite its successes, peer review attracts its share of criticism. Reviewers can exhibit bias or only support expected, pedestrian results. They can be overtaxed, uninformed, or ask for unnecessary experiments (see Letter by Raff et al., p. 36). Recent studies have explored the value of open review, double-blind review, or whether reviews are useful at all. At Science, we read thousands of reviews and author responses each year. From this vantage point, the system does not appear to be irretrievably broken and continues to serve science well. Reviews improve most papers, some dramatically so. Our authors sometimes thank reviewers for catching an embarrassing conclusion or for revealing a new one. We've seen peer review expose fraud (alas, not always), clarify results, and spur new insights.

But peer review is under increasing stress, in ways that are perhaps not fully appreciated. The growth of scientific publishing is placing a burden on the entire scientific enterprise. Papers today are more interdisciplinary, use more techniques, and have more authors. Many have large volumes of data and supplementary material. To compound the problem, papers are often being reviewed multiple times. Most of those rejected by Science go on to be considered at other journals, where the rejection rates have also increased. Before finding a proper venue, a paper may have received four, six, or even eight reviews. So even if the journal that finally publishes the article responds rapidly, the process is often painful and prolonged.

The responsibility for addressing this growing inefficiency is shared. Scientists can help by selecting the appropriate journal for their work, and seeking critical input from colleagues and all coauthors, before submitting an article for publication. Senior scientists should also mentor their students and postdoctoral fellows in good reviewing practices, enlarging the pool of qualified referees. * The possibility of repurposing reviews among journals, already practiced by some groups of journals with a single publisher, should be considered seriously. We note a recent experiment in which some independent neuroscience journals have agreed to share reviews.

The way scientists and research institutions are evaluated also needs revision. An inappropriately high value is placed on publication in certain journals. Increased competition for the limited slots in these preferred journals exacerbates the natural aggravations of peer review experienced by authors. Efforts like the Faculty of 1000, where experts scan a large set of biology journals and select the best contributions wherever published, can be very helpful. Such efforts can reduce the pressures that many group leaders feel from young scientists, who often place undue emphasis on publishing in a few high-profile journals—where the criteria used for evaluation may not match their research, no matter how valuable the contribution.

Finally, and perhaps most important, authors, reviewers, and journal editors should keep in mind the ultimate goal of scholarly scientific publishing to advance our understanding of the natural world. Competition among labs and personal striving for excellence are forces that can be harnessed to accelerate our progress. But in excess these factors can be impediments. The scientific community must collectively ensure that the peer review process continues to serve the loftier goals of our enterprise, which ultimately benefits us all.

Subjects

Navigate This Article