Chance and consensus in peer review

See allHide authors and affiliations

Science  20 Nov 1981:
Vol. 214, Issue 4523, pp. 881-886
DOI: 10.1126/science.7302566


An experiment in which 150 proposals submitted to the National Science Foundation were evaluated independently by a new set of reviewers indicates that getting a research grant depends to a significant extent on chance. The degree of disagreement within the population of eligible reviewers is such that whether or not a proposal is funded depends in a large proportion of cases upon which reviewers happen to be selected for it. No evidence of systematic bias in the selection of NSF reviewers was found.