Research Article

# Quantifying the evolution of individual scientific impact

See allHide authors and affiliations

Science  04 Nov 2016:
Vol. 354, Issue 6312, aaf5239
DOI: 10.1126/science.aaf5239

eLetters is an online forum for ongoing peer review. Submission of eLetters are open to all. eLetters are not edited, proofread, or indexed.  Please read our Terms of Service before submitting your own eLetter.

Compose eLetter

### Plain text

• Plain text
No HTML tags allowed.
• Web page addresses and e-mail addresses turn into links automatically.
• Lines and paragraphs break automatically.
Author Information
Statement of Competing Interests

This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.

## Vertical Tabs

• A small correlation between citation numbers and Nobel Prize winning

The Receiver Operating Characteristic (ROC) in Figure 6(A) of [1] is a misleading way to represent the data on the relation between the Q-factor and Nobel Prize winning. When you look at the plot, it may seem that the classifier works well. At certain rank threshold every Nobel Prize winner in the sample gets in the sieve. The True Positive Rate (TPR) is 100%. The False Positive Rate (FPR) at this rank threshold is merely 25%. But Nobel winners are a tiny fraction of the scientists. So, there should be a lot more of them in the sieve even with the 25% FPR.

To proceed we need to know n_N, the number of Nobel winners, and n_O, the number of other scientists in the sample. I could not find these numbers in Figure captions in [1]. I also could not get them from the authors. So, I had to extract the data from the plots using a plot digitizer. By matching the data from Fig. 6A and Fig. S45 of [1], I could compute the wanted numbers. My estimate is n_N = 25 and n_O ≈ 2900. This means that at the rank threshold R ≈ 760 when TPR = 1 and FPR ≈ 0.25 we get 25 Nobel winners and over 700 other scientists in the sieve. The classifier is poor. Note that if, for example, we had n_O = 28 and the same ROC plot then at TPR = 1 and FPR = 0.25 we would get 25 Nobel winners and 7 other scientists in the sieve. This means that the ROC plot is not the best way to look at the data since it does not separate these drastically different situations.

Fig. S45 in [1] has the plot...