Energy Economics

Uncovering site selection bias

See allHide authors and affiliations

Science  25 Sep 2015:
Vol. 349, Issue 6255, pp. 1501-1502
DOI: 10.1126/science.349.6255.1501-d

Despite internal validity and massive replication, program evaluations may bias out-of-sample predictions and thus misinform policy discussions. Allcott analyzed 111 randomized controlled trials (RCTs) of the Opower energy efficiency intervention, involving 8.6 million U.S. households. Large-scale replication is intended to ensure external validity across RCT sites, populations, contexts, etc., but energy utilities were not all equally willing and able to participate. Utilities having more environmentalist customers were more inclined to partner earlier and to target their most high-consuming customers. Thus, earlier interventions, despite many replications, overestimated impacts relative to effects realized in later trials.

Quart. J. Econ. 10.1093/qje/qjv015 (2015).

Navigate This Article