Policy ForumRISK ASSESSMENT

Estimating the health benefits of environmental regulations

See allHide authors and affiliations

Science  04 Aug 2017:
Vol. 357, Issue 6350, pp. 457-458
DOI: 10.1126/science.aam8204

Embedded Image

Risk assessment of water pollutants must include noncancer diseases.

PHOTO: SKHOWARD/ISTOCKPHOTO

Assessing health benefits of policies addressing environmental contaminants is important for decision-making and for informing the public about how policy affects their welfare (1). Benefits analysis, one side of benefit-cost analysis (BCA), can be relatively straightforward when sufficient data are available on dose-response relationships, changes in exposure expected from a proposed policy, and other key inputs. But despite progress, benefits analysis for health effects is needlessly constrained by analytic practices that are scientifically outdated and inconsistent with economic theory. These limitations can result in exclusion of important health effects from the estimated benefits of reducing exposure to toxic environmental contaminants, which, in turn, affects net benefits calculations that inform public policy. Fortunately, economic theory and scientific advances in the risk assessment literature provide a way forward.

BCA is widely used in policy-making and has been required in the United States since 1981 for all economically important federal regulations. The goal of the costs side of the analysis is to estimate the value of goods and services that will not be produced as a result of the regulation (i.e., opportunity cost) (2). Cost estimation faces its own challenges (3).

We focus on issues unique to benefits analysis for policies addressing health risks, where the goal is to estimate society's total willingness to pay (WTP) to reduce these risks and thereby improve health. WTP for health improvements encompasses the value of avoided treatment costs, of lost productivity, and of avoided pain, suffering, and discomfort (2). WTP may be estimated from market transactions or through survey techniques. Alternatively, BCAs may use more limited “cost of illness” estimates that reflect only direct medical costs and reduced productivity from missed work, and these values generally underestimate WTP. But, when available, values based on WTP are preferred; they are more comprehensive, represent preferences of affected individuals, and are consistent with economic theory (2). The U.S. Environmental Protection Agency (EPA) regularly produces estimates of health benefits of regulations that reduce fine particles and ozone in air using these values, sometimes amounting to billions of dollars annually (4).

Estimating health benefits from reducing exposure to other toxic environmental contaminants is frequently not done for health effects other than cancer. Thus, benefits of preventing exposure to chemicals linked to adverse health outcomes such as birth defects, neurodevelopmental effects, and cardiovascular disease are typically not quantified. This failure can be due to limited data on health effects from exposure to these contaminants (5) but also due to analytic choices about how to use available data.

We address two problems with the treatment of noncancer health outcomes in many benefits analyses: (i) health effects with less-certain evidence or without a clear summary statement of the strength of the evidence are usually excluded from benefits analysis, even though it is likely that there is some positive value to reduction of those risks; and (ii) analysts frequently do not estimate dose-response relationships that provide changes in the probability of developing a specific health outcome from changes in exposure, and in such cases benefits remain unquantified.

Effects with Less-certain Evidence

Benefits analysis begins with selecting health effects with suitable evidence and dose-response information to estimate risks at varying levels of environmental contaminant exposure and then quantifying population changes in health effects due to exposure reduction. BCAs thus rely on risk assessments that evaluate and synthesize the health effects literature (usually laboratory animal toxicology studies and/or human observational epidemiology studies) for a particular contaminant.

EPA risk assessments for cancer and “criteria” air pollutants (ground-level ozone, lead, particulate matter, carbon monoxide, nitrogen oxides, and sulfur dioxide) use standard terms to summarize the strength of evidence regarding a health effect. A high degree of confidence in the association between exposure and a health outcome, usually based on high-quality epidemiological and/or animal studies, is described as “known,” “causal,” or “likely.” EPA typically includes such outcomes in BCA. Moderate or lesser confidence in the evidence is indicated as “suggestive”—for example, when there is evidence of an association between exposure and health outcome but human studies are few or lacking, and/or there are concerns regarding human or animal study quality (e.g., high risk of bias, or indirect or imprecise evidence), and/or findings are inexplicably inconsistent across studies.

When EPA judges evidence to be “suggestive” or—as is typical for noncancer health effects—there is no summary descriptor from an authoritative review of the evidence, EPA generally excludes the potential health risk from its primary quantitative benefits analysis (4). For example, exclusion of cardiovascular effects (due to uncertainty of the effect at exposure levels relevant in the United States) from an EPA BCA of arsenic in drinking water may have substantially affected the results of the analysis, and benefits related to the excluded cardiovascular effects may have been greater than the included cancer-related benefits (6). This practice implicitly assumes that exposed populations have zero WTP for reduced exposure when there is some evidence of an adverse health effect but that evidence is not unambiguous. This assumption violates economic principles and is contradicted by findings (7).

Moreover, theory and evidence indicate that WTP for reducing risks is higher for more severe health effects (e.g., cancer, cardiovascular disease, and chronic health effects in children). As a result, the benefits of reduced exposure to a chemical with a “suggestive” relationship to serious health end points may be higher than the benefits of reduced exposure to a chemical with a “known” relationship to less serious health end points. It may therefore be misleading to take account of the latter but not of the former.

An important first step to including these less-certain effects is for risk assessments to provide greater clarity on the strength of evidence of each health effect (with, for example, a summary descriptor), even if summarizing health effects evidence with terms like “known” or “probable” can be complex and uncertain (8, 9). Although the foundational theory of BCA supports inclusion and accounting for less-certain health risks, applied research on how to do this is lacking. The best quantitative weight for these less-certain health effects is not zero (10). Ideally, research would reveal how WTP for risk reductions varies systematically with characterizations of uncertainty about those risks. Alternatively, one might posit an expected utility model and reduce the certainty-equivalent values by an estimated probability of causality. These changes require additional research, analysis, and policy choices.

Dose-response Relationship Needs

Epidemiological studies of criteria air pollutants provide dose-response relationships applicable across a range of human exposures. These dose-response functions allow for quantifying and monetizing the benefits of reducing exposures at every level of exposure.

EPA risk assessments for other environmental contaminants typically provide estimated dose-response functions for known or likely carcinogens, but they do not do so for health outcomes other than cancer, such as cardiovascular, respiratory, neurological, or developmental effects. Instead, EPA practice is to estimate a reference dose (RfD) for these effects that assumes a threshold below which exposures have no quantifiable risk. The RfD is a level “likely to be without appreciable risk of deleterious effects” in an exposed population. “Likely” and “appreciable” are not defined, so the RfD is not associated with any quantitative risk target and provides no direct measure of the risk of adverse health outcomes for any level of exposure either above, below, or at the RfD. Without a dose-response relationship, it is impossible to include these risk reductions in a BCA; thus, the related health benefits are implicitly valued at zero. This issue was well recognized in a 2009 report by the U.S. National Academy of Sciences (NAS) (11).

The NAS report noted that the RfD approach, which is based on assumptions developed in the 1950s to 1980s, provides limited information for decision-making and “does not make the best possible use of available scientific evidence” (11). The report observed that the default assumption of a population threshold built into the RfD is questionable for most environmental contaminants. Instead, no population threshold is expected, even for most instances of dose-response relationships that may have thresholds for each individual, due to multiple sources of variability in the population, including differences in both intrinsic factors (e.g., life stage, reproductive status, age, gender, and genetic traits) and acquired factors (e.g., preexisting disease, geography, socioeconomic, cultural, workplace, and exposure to other environmental contaminants) (6, 11). The NAS recommended approaches that do not assume a threshold for population dose-response assessment unless the science supports assuming a population threshold for the contaminant.

Implementing the NAS recommendations for dose-response assessment to assess differences in risk at varying levels of exposure would allow for quantifying and monetizing risk reductions for health effects that are not currently monetized—and in doing so provide quantified estimates of benefits for a range of potential policy choices. Two main approaches to implementing the NAS recommendations are available. First, similar to existing practice for criteria air pollutants, regression models can be used to estimate a dose-response function when adequate data are available. Second, probabilistic models may be used to extrapolate from experimental or epidemiological data, accounting for uncertainties in animal-to-human differences, human population variability, and limitations in the database, as appropriate; the World Health Organization recently issued guidance for this approach (12). In either approach, a single “bright line” akin to the RfD may still be specified by identifying a target level of risk and estimating the dose associated with that risk.

Steps Forward

The rise of benefit-cost analysis as a major input to EPA decision-making has been a gradual process that began in the mid-1980s and continues today. Early development of BCA focused on areas where dose-response relationships were available (i.e., criteria air pollutants and carcinogens). Recently, there has been a growing recognition among decision-makers of the need to better understand the benefits from reducing noncancer health effects from the broad range of environmental contaminants within EPA's jurisdiction. As a result, greater collaboration between risk assessors and economists has become a priority in order to develop methods, tools, and approaches for assessing health effects with less-certain evidence, for developing clear summary statements about the evidence, and for providing dose-response relationships to quantify health risk reductions. Together, these steps would give decision-makers more complete estimates of policy outcomes.

References and Notes

  1. The current practice of ignoring uncertainty associated with “likely” causal health end points may overstate WTP, but in this paper we focus on addressing omission of benefits categories.
  2. Acknowledgments: This article is based on discussions at a 2016 symposium organized by the authors at JPB Foundation in New York, NY. Support for the symposium and this paper were provided to the UCSF Program on Reproductive Health and the Environment by the U.S. Environmental Protection Agency, the JPB Foundation, the Tides Foundation, and the Institute for Policy Integrity at New York University School of Law. The views articulated here are those of the authors and should not be ascribed to any supporting or affiliated organizations.

Navigate This Article