Wanted: Better Benchmarks

See allHide authors and affiliations

Science  20 May 2005:
Vol. 308, Issue 5725, pp. 1087
DOI: 10.1126/science.1114801

How much should a nation spend on science? What kind of science? How much from private versus public sectors? Does demand for funding by potential science performers imply a shortage of funding or a surfeit of performers? These and related science policy questions tend to be asked and answered today in a highly visible advocacy context that makes assumptions that are deserving of closer scrutiny. A new “science of science policy” is emerging, and it may offer more compelling guidance for policy decisions and for more credible advocacy.

All developed and many developing nations today have accepted the need to support technical education and research as keys to future economic strength. Studies from the 1990s show that U.S. investment in R&D development led to greater economic productivity, and that information technology, in particular, has been a major factor in sustaining U.S. productivity growth. The question is not whether R&D investments are important, but what investment strategies are most effective in the rapidly changing global environment for science. Here, ideas diverge.

Take the issue of the technical workforce. Sharply differing opinions exist regarding the production of U.S. scientists to meet possible impending shortages.* The differences turn on the interpretation of “benchmark” data regarding the numbers of degree holders produced in the United States and other countries, particularly China and India. In the latter countries, the rates of growth in the numbers of scientists are high, although actual numbers are small relative to those in the United States. Advocates for increased production of U.S. scientists point to our low graduation rates, whereas critics emphasize limited short-term job opportunities for graduates and postdocs. Resolution of this issue requires a broader understanding of socioeconomic factors in a number of nations that would allow us to attach probabilities to different future scenarios. Optimal strategies for large mature economies such as that of the United States will doubtless differ from those for smaller or developing economies. Here, as elsewhere in policy debates, the benchmarks do not speak for themselves.


The data we choose to collect do say something about the framework in which we understand the relations among science, government, and society. Our customary reliance on historical trends in national data, however, creates an inertia that causes data categories to lag far behind changes in the dynamic socioeconomic framework, now evolving internationally. We know that there is a complex linkage between workforce issues and other economic variables. Technical workforces in different countries are increasingly interdependent in a way that makes single-country data unreliable for workforce forecasts.

Globalization and changing modes of science that have blurred disciplinary distinctions have undermined the value of traditional science and engineering data and their conventional interpretations. The old budget categories of basic and applied R&D, still tracked by the U.S. Office of Management and Budget, do not come close to capturing information about the highly interdisciplinary activities thought to fuel innovation. A 1995 U.S. National Research Council (NRC) committee chaired by Frank Press took a step toward data reform when it introduced the combined category of “federal science and technology,” declaring that “the linear sequential view of innovation is simplistic and misleading.” More attention, however, is needed to definitions and models that suit current needs of policy. A recent report from the NRC Committee on National Statistics found that “the structure of … data collection is tied to models of R&D performance that are increasingly unrepresentative of the whole of the R&D enterprise.” Further, “It would be desirable to devise, test and, if possible, implement survey tools that more directly measure the economic output of R&D in terms of short-term and long-term innovation.”

Relating R&D to innovation in any but a general way is a tall order, but not a hopeless one. We need econometric models that encompass enough variables in a sufficient number of countries to produce reasonable simulations of the effect of specific policy choices. This need won't be satisfied by a few grants or workshops, but demands the attention of a specialist scholarly community. As more economists and social scientists turn to these issues, the effectiveness of science policy will grow, and of science advocacy too.

  • *D. Kennedy, J. Austin, K. Urquhart, C. Taylor, Science 303, 1105 (2004).

  • Measuring Research and Development Expenditures in the U.S. Economy, L. D. Brown, T. J. Plewes, M. A. Gerstein, Eds. (National Academies Press, Washington, DC, 2005).

Stay Connected to Science

Navigate This Article