Education ForumASSESSMENT

Global Sex Differences in Test Score Variability

See allHide authors and affiliations

Science  28 Nov 2008:
Vol. 322, Issue 5906, pp. 1331-1332
DOI: 10.1126/science.1162573

Do boys and girls differ in their intellectual and cognitive abilities and, if so, in what way? These questions have raised considerable debate, both in terms of average performance and in terms of variability around the average.

Empirical research on gender differences in achievements produces mixed conclusions, with some evidence that favors boys and some that favors girls (1-4). In many countries, girls show superior performance in school examinations, which is also reflected in higher rates of attendance in tertiary education. In addition, girls have been improving their position relative to boys (5, 6). In countries with a more gender-equal culture, the gender gap that is usually in favor of boys in average mathematics test scores is erased or even reversed in favor of girls (7).

At the same time, some research focuses on the notion that there are more males at the upper end of the distributions of educational and professional success (8). Oft-cited examples include there being more male than female Nobel Prize winners and the inequity of wages in the labor market in favor of males (9, 10). Studies of talented individuals who succeed at the very highest levels, especially in science, highlight substantial male overrepresentation (11).

View this table:

Gaps and ratios. In the table, M refers to male and F to female. Gaps are M-F differences at the mean, 5th, and 95th quantiles in standard deviation units. Variance ratio refers to male variance divided by female variance, and ratios in top and bottom 5% refer to the ratio of number of males and females who scored in the top and bottom 5% of the distribution. Results for the full data are presented here; USA is excluded from OECD results. For full results, see SOM, *P < 0.05; **P < 0.01.

These outcomes can be generated by various kinds of distributions describing the educational and intellectual make-up of boys and girls. Differences in the gender composition of the high-scoring group can be a consequence of gender differences in the mean or variance of the test scores or both. Given that recent research has shown that gender differences in the mean test scores are not a robust phenomenon internationally (7), it is of interest to explore whether the gender differences in the variance of test scores are a more universal outcome.

The idea that males are intellectually and educationally more variable than females dates back a long time and is embedded in cultural history. Early academic entries are represented by late 19th century texts from Ellis (12) and Galton (13). Modern comment [e.g., (14)] can be quite critical of the motives of those who postulate that gender differences in variances exist.

Most of the evidence regarding gender-dependent differences in the mean and variance of intellectual and cognitive abilities is based on U.S. data (15, 16), because most of the data containing test scores are from the United States. But data sources from other countries exist as well. We have therefore investigated whether the phenomenon of “higher variance” is an accurate characterization of boys' educational performance relative to girls from a wider sample of countries (17).

Research Method and Findings

We examined boy-girl differences in test score performance from the 2003 OECD (Organization for Economic Co-operation and Develop-ment) Programme for Inter-national Student Assessment (PISA), a survey of 15-year-olds who are enrolled in full- or part-time education in industrialized countries (18). The PISA survey contains data on individual student performance for 41 countries. Here, we analyze mathematics and reading test scores by country, focusing on differences in the mean and variance of these test scores. The test scores are standardized to have a mean of 500 and a standard deviation of 100 across the OECD countries.

Reading. We calculated the international standardized mean differences in reading test scores as the sex difference in means divided by the standard deviation, and differences in variance as the male-to-female variance ratio. In all 41 countries, the boy-girl mean difference is negative, indicating that girls generally outscore boys (see table, top, and table S1). In 35 out of 41 countries, the boy-girl variance ratio indicates, with statistical significance (P < 0.05), that the population of boys displays greater variance than does the population of girls (see the table above and table S1).

Mathematics. With the same comparisons for mathematics, the mean difference is positive, which indicates that boys generally outscore girls (see table, bottom, and table S2). In 37 of 41 countries, the boy-girl variance ratio indicates with statistical significance (P < 0.05) that the population of boys displays greater variance than does the population of girls (see the figure below and table S2).

Variance ratios in countries that participated in PISA 2003. In the majority of countries, male variability is higher than female variability in both reading and math.

Tip and tail. There are differences for the two tests. On both math and reading tests, boys predominate in two of the four extreme scoring categories (low reading, high math), with girls predominating in the high reading and low math category (see table, top and bottom, and columns 4 and 5 of tables S1 and S2). For mathematics, in 35 of the 41 countries, there are more boys than girls in the top 5%. For reading, 36 of 41 countries have more girls than boys in the top 5% of scores, and 39 of 41 countries have more boys than girls in the bottom 5% of scores.

Gender gaps. Gender gaps at the 5th and 95th quantiles of the reading distribution favor girls, but tend to be smaller at the 95th quantile than at the 5th quantile. In mathematics, on the other hand, the gender difference at the 5th quantile is statistically significant at the 5% level only in 18 countries, and the sign of this difference varies across countries. In mathematics, the gender difference at the 95th quantile favors boys in 36 countries (P < 0.05) (see table S2).

Variance and test performance. For both reading and mathematics tests in all 41 countries, the boy-girl variance ratio shows a positive correlation (r = 0.21 for reading and r = 0.26 for mathematics) with the mean test score performance (fig. S1, A and B). In countries with better test score performance, the boy-girl variance ratio is significantly higher (P < 0.05) than in countries where the children score more poorly (table S3). We find no relation between the Gender Gap Index of the World Economic Forum (19) and the variance ratios for either math or reading (table S3 and fig. S2, A and B), unlike the relation previously identified between the Gender Gap Index and the gender gap in average test scores (7).

Other data sources. To further probe the robustness and wider applicability of these findings, we have also studied gender differences in variance from other international data sources. We analyzed results from (i) the Trends in International Mathematics and Science Study (TIMMS) given in 2003, which tested 9- and 13-year-olds in mathematics and science; and (ii) the Progress in International Reading Literacy Study (PIRLS), given in 2001, which tests reading skills of 10-year-olds (20, 21). Data from both studies indicate higher variance for boys in most countries (table S4).

Implications and Conclusions

Our analysis of international test score data shows a higher variance in boys' than girls' results on mathematics and reading tests in most OECD countries. How this translates into educational achievement is a matter open for discussion. Higher variability among boys is a salient feature of reading and mathematics test performance across the world. In almost all comparisons, the age 15 boy-girl variance difference in test scores is present. This difference in variance is higher in countries that have higher levels of test score performance.

Sex differences in means are easier to characterize: It is evident from the PISA data that boys do better in mathematics, and girls do better in reading. This has a compositional effect on the variance differences as well. The higher boy-girl variance ratio in mathematics comes about because of an increased prevalence of boys in the upper part of the distribution, but the higher variance in reading is due to a greater preponderance of boys in the bottom part of the test score distribution. Because literacy and numeracy skills have been shown to be important determinants of later success in life (for instance, in terms of earning higher wages or getting better jobs), these differing variances have important economic and social implications (22).

We therefore confirm that 15-year-old boys do show more variability than girls in educational performance, with specifics that differ according to whether mathematics or reading are being studied and tested. These results imply that gender differences in the variance of test scores are an international phenomenon and that they emerge in different institutional settings.

References and Notes

Navigate This Article