LettersNextgen Voices

Measures of success

+ See all authors and affiliations

Science  01 Apr 2016:
Vol. 352, Issue 6281, pp. 28-30
DOI: 10.1126/science.352.6281.28
ILLUSTRATION: JAMES STEINBERG

In January, we asked young scientists this question: Do publications (number and impact) convey the true value of an early-career scientist? Out of the almost 200 (self-selected) scientists who replied, about 75% did not think that publications conveyed a scientist's true value. About 25% said that publications were indeed the most important metric. Among those who disagreed, several themes emerged:

  • Publication record does not account for the many intangible traits that make a good scientist.

  • A young scientist's publication record reflects his or her country, institution, and adviser more than his or her own skills.

  • Publications overlook good work that led to negative results.

  • Quantity does not equal quality.

Below, we have printed some of the most interesting responses. In some cases, we have printed excerpts of submissions (indicated by ellipses) and lightly copyedited original text for clarity. To read the complete versions, as well as many more, go to http://scim.ag/NG_18R. Follow Science's NextGen VOICES survey on Twitter with the hashtag #NextGenSci

…YOUNG RESEARCHERS HAVE limited control over the number and impact of their publications. But being a good scientist is not just about impact factor; it is asking the right questions, helping your peers find solutions, discussing your work, and sharing negative results….

Claudia S. Barz

Institute for Neuroscience and Medicine, INM-2, Research Centre Jülich, 52428, Jülich, Germany and RWTH Aachen, Medical School, 52074, Aachen, Germany. E-mail: c.barz{at}fz-juelich.de

…UNTIL SCIENCE BECOMES a true meritocracy, and until equality in terms of race, gender, lifestyle, and economic background becomes a reality, the system is inherently flawed. Young scientists should be valued by their commitment to education, their dedication to fighting inequality in science, and their efforts to democratize science.

Melodie Elane Benford

Pearland, TX 77584, USA. E-mail: melodie.benford{at}gmail.com

A SCIENTIST SHOULD be evaluated with respect to his work environment. This can include country, institutions, group, and people with whom he is working. Then his performance should be normalized with these factors.

Manoj Kumar Mishra

Space Physics Laboratory, VSSC, Thiruvananthapuram, 695022, India. E-mail: manojkmishra79{at}gmail.com

YOUNG SCIENTISTS SHOULD be evaluated based on their ability to convey ideas, interpret data, resolve problems, and generate novel scientific ideas. These values can be easily overlooked if one focuses only on publications and impact factors. A mediocre young scientist may have good publications merely by being at the right place, with the right person, and at the right time….

Tiong Sun Chia

Harry Perkins Institute of Medical Research, Perth, 6009, Australia. E-mail: tiongsun{at}gmail.com

…IT IS IMPORTANT to consider achievement relative to opportunity and not to reward opportunity as if it were achievement. For example, success in a rich country or institution is easier than in a poor one; researching in a prestigious institute is an opportunity, not an achievement.

Endymion Cooper

School of Biological and Chemical Sciences Queen Mary University of London, London, E1 4NS, UK. E-mail: endymion.dante.cooper{at}gmail.com

NUMBER AND IMPACT of publications at this stage depend too much on external factors: quality of supervision; public interest in research area; and, more than anything, luck. There is much more to being a good scientist than simply publishing research. Evaluation of young scientists must adopt a holistic approach, taking into account the challenges faced, their level of commitment, plus involvement in nonresearch–focused activities, such as teaching and public engagement.

Kate Newton

Department of Earth Sciences, University of Birmingham, Edgbaston, B15 2TT, UK. E-mail: ken155{at}bham.ac.uk

THE TRUE VALUE of a young scientist's career would be better explained by a “networking index.” This would let us know if rookies could establish new contacts and successfully collaborate with other research groups…rather than just being restricted to their immediate circle….

Nicolás Bonel

División Zoología de Invertebrados I, Departamento de Biología, Bioquímica y Farmacia, Universidad Nacional del Sur-CONICET, Bahía Blanca, Buenos Aires (RA-B8000ICN), Argentina. E-mail: nicobonel{at}gmail.com

…WE NEED A modified h index that also takes into account the number of authors: First three and last three positions get standard h index evaluation. The value for an author in the middle should be the impact factor divided by the number of authors. The h index should be for only the past 5 years. This way, it is not a function of age.

Charles de Bock

Vlaams Instituut voor Biotechnologie (VIB), 3000, Leuven, Belgium. E-mail: charles.debock{at}cme.vib-kuleuven.be

[SCIENTISTS] ARE NEVER remembered for the number of papers we've published, but for how we advance our fields. Suggesting otherwise to young scientists—the future of science—will distract from our true purpose and discourage future Boltzmanns, Faradays, and Bohrs….

Kelsey Farenhem

Vagelos Scholars Program in Molecular Life Sciences, University of Pennsylvania, Philadelphia, PA 19104, USA. E-mail: farek{at}sas.upenn.edu

QUANTIFICATION OF WORK published during the Ph.D. candidacy does not convey the true value due to the large impact of the guidance of the adviser. Therefore, in the years after the Ph.D., …researchers should be funded directly from national agencies. This would allow them…to prove their ability to set their own research goals, manage their work, and lead others in pursuit of new scientific discoveries. After this period, their independently achieved results can be quantified in the same way as those of their more established peers.

Jernej Zupanc

Seyens Ltd., 1000, Ljubljana, Slovenia. E-mail: jernej.zupanc{at}seyens.com

RESEARCH ADDRESSING REAL societal problems and the ability to engage the public convey the true value of a scientist.…We can measure the public interest of the author's work through media circulation of the news or altimetrics….

Esraa Elsanadidy

Department of Chemistry, University of Connecticut, Storrs, CT 06268, USA. E-mail: esraa_elsanadidy{at}yahoo.com

NEITHER NUMBER NOR impact factor of the candidate's publication is sufficient to assess the true value of an early-career scientist. If this were the case, a number of well-known scientists who published path-breaking results after a long lull, as well as Nobel laureates, would have been labeled as underachievers.…

Suchitra D. Gopinath

Translational Health Science and Technology Institute, NCR Biotech Science Cluster, Faridabad, 121001, India. E-mail: sgopinath{at}thsti.res.in

…SOMETIMES THERE IS a huge time lag when measuring publication impact. How long did it take before the first climate change papers really made an impact? Ideas might take a while until they become relevant, known, or established.

Marten Winter

German Centre for Integrative Biodiversity Research (iDiv) Halle-Jena-Leipzig, sDiv—Synthesis Centre, 04103, Leipzig, Germany. E-mail: marten.winter{at}idiv.de

…IT WOULD BE far better to look at an early-career scientist's publication list in relation to a metric that correlates years since publication and citations received. This metric would…reinforce desirable traits such as scientific accuracy (longevity) and applicability (influence).

K. Christian Kemp

POSTECH, Pohang, 37662, South Korea. E-mail: ckemp{at}postech.ac.kr

“…scientists should be valued by papers—by quality, not quantity…”

ILLUSTRATION: JAMES STEINBERG

…NOT HAVING high-impact publications…may indicate discipline-crossing, boundary-breaking work…. Hiring young scholars based on the value of their high-impact publications alone thus risks reinforcing normal science while deferring paradigmatic shifts.

Rense Nieuwenhuis

Swedish Institute for Social Research (SOFI) of Stockholm University, 10691, Stockholm, Sweden. E-mail: rense.nieuwenhuis{at}sofi.su.se

“IMPACT” SHOULD BE JUDGED qualitatively by experts by assessing papers, documentation for public data sets (especially experimental designs and protocols), software and data analysis code, and/or outreach and communication efforts….

J. Steen Hoyer

Computational and Systems Biology Program, Washington University in St. Louis, St. Louis, MO 63132, USA and Donald Danforth Plant Science Center, St. Louis, MO 63132, USA. E-mail: j.s.hoyer{at}wustl.edu

…EARLY-CAREER scientists should also be evaluated by how many revisions and resubmissions they have completed after rejection. Those metrics are an indicator of scientific grit, or the ability to persevere when met with adversity….

Rosa Li

Departments of Psychology and Neuroscience, Center for Cognitive Neuroscience, Duke University, Durham, NC 27708, USA. E-mail: rosa.li{at}duke.edu

YOUNG RESEARCHERS SHOULD be evaluated on their ability to execute solid and honest science, rather than on how flashy their findings are. There needs to be less of a gap in reward between the competent execution of a well-planned experiment that ultimately leads to a negative result and one that leads to a positive one. To that end, evaluations could be partly based on project proposals that demonstrate the ability to plan and think through relevant topics for the field.

Grace Lindsay

Neurobiology and Behavior Program, Columbia University, New York, NY 10032, USA. E-mail: gwl2108{at}columbia.edu

THEORETICALLY ORIENTED early-career scientists should be valued by papers—by quality, not quantity; by citation rates, not impact factors…. Meanwhile,…doctors should be appraised based on their success in curing their patients, and R&D personnel should be valued based on their new product development.

Dayuan Li

Collaborative Innovation Center of Resourceconserving and Environment-Friendly Society and Ecological Civilization, Central South University, Changsha, 410083, China. E-mail: bigolee{at}163.com

…WITH REGARD TO hiring and promotion, factors such as enthusiasm, collegiality, mentoring qualities, and the ethical conduct of research must be evident. It is upon these attributes that a satisfying and influential scientific career will be built….

Anthony O'Mullane

School of Chemistry, Physics, and Mechanical Engineering, Queensland University of Technology, Brisbane, QLD 4001, Australia. E-mail: anthony.omullane{at}qut.edu.au

…DOES TEACHING 33 credits a year prohibit me from being considered a scientist? If publications are the measure, it most certainly does, yet I think I do play an important role in the scientific community: educating future scientists….

Keah C. Schuenemann

Department of Earth and Atmospheric Sciences, Metropolitan State University of Denver, Denver, CO 80217, USA. E-mail: kschuene{at}msudenver.edu

…IN MANY CASES, we are allowed authorship on a publication simply because we are in charge of cleaning the laboratory glassware and helping with very simple experiments. Such a publication is not comparable to a publication on which there are very few authors and the young scientist is the primary contributor.

Daniel Ramirez

Centro de Investigación, Innovación y Desarrollo de Materiales, University of Antioquia, Medellín, Colombia. E-mail: estiben.ramirez{at}udea.edu.co

…A BETTER APPROACH will be to subject all scientists to Yelp-like review criteria. Anonymous…reviewers will be able to grade a scientist based on their past interactions, on a number of points including bench-related criteria (methodology and efficiency), thinking ability (innovativeness of ideas and creativity), and mentorship and leadership abilities. Of course, the scientists themselves can respond directly to reviews posted about them. Furthermore, the quality of the reviewers contributes to the weight of their rating….

Shann S. Yu

École Polytechnique Fédérale de Lausanne, Swiss Institute for Experimental Cancer Research (ISREC), CH-1015, Lausanne, Switzerland. E-mail: shann.yu{at}epfl.ch

PUBLICATIONS ARE THE key parameters for assessing young scientists. During their training phase, they need to learn literature searches, find research questions or problems, design a study…and, of course, learn hands-on techniques for the experiments. Analyzing and interpreting the results is a key part of the process. Building a story based on the data output and publicizing it to the scientific community is the final phase of training….

Matiram Pun

EURAC Institute of Mountain Emergency Medicine, European Academy of Bolzano, Bozen, 39100, Italy. E-mail: matiram.pun{at}eurac.edu

ALTHOUGH NO ONE factor can quantify the “value” of a scientist, publications are a crucial part of the job. If a scientist cannot communicate through writing, even the most groundbreaking research will have no impact.…The number and impact of one's publications can therefore differentiate those who have great ideas from those who are able to influence the scientific community with those great ideas.

Catherine Y. Li

Vagelos Scholars Program in Molecular Life Sciences, University of Pennsylvania, Philadelphia, PA 19104, USA. E-mail: catli{at}sas.upenn.edu

Related Content

Navigate This Article