Technical Comments

Response to Comment on “The extent of forest in dryland biomes”

See allHide authors and affiliations

Science  27 Oct 2017:
Vol. 358, Issue 6362, eaao2070
DOI: 10.1126/science.aao2070


Schepaschenko et al. question our findings, claiming that we did not refer to all existing maps and that we did not account for all sources of uncertainty. In our response, we detail our selection criteria for reference maps, which clarify why the work of Schepaschenko et al. was not used, and we explain why our uncertainty assessment is complete and how it was misunderstood by Schepaschenko et al.

We thank Schepaschenko et al. (1) for their helpful comments on our article (2). We agree that it is crucial to compare state-of-the-art products and to report completely and transparently on uncertainties. However, we believe that (i) we compared our data with comparable data sets (i.e., with quantitative remote-sensing surveys independent of expert-based knowledge and with a spatial resolution comparable to our plot samples), and (ii) our uncertainty calculations, which have been misunderstood by Schepaschenko et al., encompass all the caveats and illustrations discussed by Schepaschenko et al.

Although we appreciate the fact that the “global hybrid forest mask” produced by Schepaschenko et al. in 2015 (3) provides independent support for some of the additional forest area shown in our map, we did not use their map as a basis for comparison for four main reasons:

1) Their forest mask was calibrated with estimates of forest area published in the global Forest Resources Assessments (FRAs) of the Food and Agriculture Organization of the United Nations (FAO), and this may have introduced bias into their map. In the two most recent meetings of the FAO Committee on Forestry, COFO 22 and COFO 23, country representatives stressed the need to improve forest statistics for drylands in particular. These requests were motivated by an awareness by these countries that FRA statistics, which can be traced back to them, are not reliable for this part of the world, with estimates often being based only on expert judgments. Moreover, FRAs are based on country reporting with national territories as the basic unit. The use of these data to characterize a region, which does not reflect a sum of country territories, could be another source of a bias in the Schepaschenko et al. estimate.

2) We decided to exclude references to data sets with spatial resolution of 1 km2, as this resolution is 200 times the size of our sampling unit (0.5 ha). A large difference in spatial resolution consistently reduces the possibility of performing accurate comparisons.

3) The FAO-FRA Global Remote Sensing Survey 2010 (FRA RS 2010) (4), which we used, was independent of FRA statistics but is nonetheless consistent with the FAO definition of “forest.”

4) Schepaschenko et al. did not use their map to report an estimate of global forest area in their paper (3). The most detailed comparisons of our map of tree cover were with two of the “new generation” of wall-to-wall Landsat maps of global forest area, produced by Hansen et al. (5) and Sexton et al. (6), following the pioneering work of Townshend et al. (7). However, all three of these studies focused on mapping “tree cover,” so we needed another map that actually reported global “forest” area in compliance with the FAO definition. This was important to indicate the implications of our findings for the estimate of global forest area of about 4000 Mha that has been accepted for several decades (8) and was actually reported as 3999 Mha in the most recent Forest Resources Assessment, FAO-FRA 2015 (9). In our judgment, FRA RS 2010 was the best available map for this particular purpose.

Schepaschenko et al. also questioned the sources of uncertainty that we considered for the measurement errors. We would like first to point out that what matters is the error on the estimate of forest area that is propagated from the measurement errors. Because measurement errors are symmetric, with underestimations and overestimations canceling out to some extent, the propagation of measurement errors attenuates them so that an increase in the measurement errors will result in a smaller increase of the error on forest area. Moreover, the estimator of forest area is based on a thresholding at 10% of the tree cover, which is a robust approach (just as the median is a robust estimator of central tendency, whereas the arithmetic mean is not). It implies that a measurement error of +5% on the tree cover of a plot with 8% tree cover (thus wrongly accounting it in the forest area) will have the same influence on the forest area as a measurement error of +20%. In other words, the estimation of forest area will only be affected by measurement error if this leads to passing by one side or the other of the threshold.

These principles have been misunderstood by Schepaschenko et al. The 3.5% and 7.4% of errors we report in the main text are related to the “forest area in drylands” and not to plot-by-plot measurement. These forest area errors are calculated by propagating the “measurement errors”—that is, the difference at the plot level between photointerpretation and ground truth—which are 8.2% and 12.7% for tree cover and forest land-use assessment, respectively. Here, Schepaschenko et al. did not compare the correct numbers, as the 14% of “measurement error” they report (1) should have been compared to the 12.7% we report in the supplementary material and not to the 8.2%. Moreover, Schepaschenko et al. calculated their “measurement error” from a double photointerpretation exercise (i.e., with one assessment not being necessarily better than the other), whereas we report the difference between photointerpretation and ground truth. Consequently, these represent two different measures. We therefore suggest that the numbers of Schepaschenko et al. could be used as “measurement uncertainties” rather than a “measurement error.”

We do not deny that our approach has limitations. However, with the exception of biases due to a potential better coverage of very high resolution (VHR) images in Australia than for the rest of the drylands, all the limitations mentioned by Schepaschenko et al. are explicitly recognized and covered in our paper. Verifying the difference in VHR coverage, we found that, contrary to what Schepaschenko et al. are claiming, we had a smaller proportion of VHR images for Australia (74%) than for the entire set of drylands (82%). Our uncertainty analysis is consequently not underestimated by the exceptional quantity of the VHR images available in Australia.

For instance, Schepaschenko et al. (1) mention that our data collection occurred in 2015 but that we report our assessment as for 2015 while VHR images were probably not available for that year. In our supplementary material, we explain that “The Global Dryland Assessment is however reported as for 2015, using the vegetation index computed for the period 2001–2015 (see section on Google Earth and Collect Earth) as a reference to extend the photointerpretation of the latest VHR image to the year 2015.” Our approach is not only based on photointerpretation of VHR images available from Google. It is called “augmented visual interpretation” (10) because it relies on multiple sources of VHR images (Google Earth, Bing Maps) and on medium- to high-resolution data, as we used Landsat 7, Landsat 8, and MODIS data and the corresponding vegetation indices computed for the past 15 years from these to fill the gaps.

Most important, all errors mentioned or illustrated by Schepaschenko et al. are covered by our calculation of uncertainties, as we compared ground-truth data with augmented visual interpretation. Because we report 7.4% errors, this means that about 15,000 of our plots were wrongly assessed. It is consequently easy to find, in the range of our reported errors, examples of plots that were wrongly assessed.

Although we understand the reactions of our colleagues from the International Institute for Applied Systems Analysis and applaud their pioneering work in crowdsourcing, we believe that we compared our study with state-of-the-art comparable data sets and reported completely and transparently on uncertainties.

References and Notes

  1. Acknowledgments: This work was conducted under the Global Forest Survey project of FAO, supported by the International Climate Initiative of the Federal Ministry for the Environment, Nature Conservation, Building and Nuclear Safety of Germany, and under Action Against Desertification, an initiative of the ACP (Africa, Caribbean, and Pacific) group of states, implemented by FAO and funded by the European Union in support of Africa’s Great Green Wall and the United Nations Convention to Combat Desertification action programs and south-south cooperation in ACP countries.
View Abstract

Navigate This Article