Policy ForumResearch Ethics: COVID-19

Against pandemic research exceptionalism

See allHide authors and affiliations

Science  01 May 2020:
Vol. 368, Issue 6490, pp. 476-477
DOI: 10.1126/science.abc1731

The global outbreak of coronavirus disease 2019 (COVID-19) has seen a deluge of clinical studies, with hundreds registered on clinicaltrials.gov. But a palpable sense of urgency and a lingering concern that “in critical situations, large randomized controlled trials are not always feasible or ethical” (1) perpetuate the perception that, when it comes to the rigors of science, crisis situations demand exceptions to high standards for quality. Early phase studies have been launched before completion of investigations that would normally be required to warrant further development of the intervention (2), and treatment trials have used research strategies that are easy to implement but unlikely to yield unbiased effect estimates. Numerous trials investigating similar hypotheses risk duplication of effort, and droves of research papers have been rushed to preprint servers, essentially outsourcing peer review to practicing physicians and journalists. Although crises present major logistical and practical challenges, the moral mission of research remains the same: to reduce uncertainty and enable caregivers, health systems, and policy-makers to better address individual and public health. Rather than generating permission to carry out low-quality investigations, the urgency and scarcity of pandemics heighten the responsibility of key actors in the research enterprise to coordinate their activities to uphold the standards necessary to advance this mission.

Rigorous research practices can't eliminate all uncertainty from medicine, but they represent the most efficient way to clarify the causal relationships clinicians hope to exploit in decisions with momentous consequences for patients and health systems. Nevertheless, fastidious research standards may seem a luxury that pandemics can ill accommodate. Commenting on a study using suboptimal design, one group of scientists stated, “Given the urgency of the situation, some limitations…may be acceptable, including the small sample size, use of an unvalidated surrogate end point, and lack of randomization or blinding” (1). The perception that core methodological components of high-quality research are dispensable is underpinned by three problematic assumptions. The first is that some evidence now, even if flawed, seems preferable to expending greater resources on more-demanding studies whose benefits only materialize later. Because the window for learning in pandemics is often short, the need to “balance scientific rigor against speed” seems inevitable (3).

The problem with this view is that challenges that rigorous methods address do not disappear in the face of urgent need. Small studies that build on basic science and preclinical research in early phases of drug development routinely generate signals of promise that are not confirmed in subsequent trials. Even when new drugs are established to be safe and effective, rarely are their benefits so massive that they can be detected in small, open-label, nonrandomized trials. The proliferation of small studies that are not part of an orchestrated trajectory of development is a recipe for generating false leads that threaten to divert already scarce resources toward ineffective practices, slow the uptake of effective interventions because of an inability to reliably detect smaller but clinically meaningful benefits, and engender treatment preferences that make patients and clinicians reluctant to participate in randomized trials. These problems are amplified by published reports of compassionate use, which was designed as an alternative pathway to access interventions outside of research, not to support systematic evaluation.

The second underpinning of research exceptionalism is the view that key features of rigorous research, like randomization or placebo comparators, conflict with clinicians' care obligations. However, when studies begin in, and are designed to disturb a state of, clinical equipoise (meaning that it's uncertain whether a particular treatment is better than the alternatives), they ensure that no study participant receives a standard of care known to be inferior to any available alternative (4). Under this condition, randomized trials with appropriate comparators configure medical practice in a way that allows patients to access investigational interventions under conditions designed to eliminate ineffective strategies and exploit effective alternatives.

The third underpinning of research exceptionalism derives from the expectation that researchers and sponsors are generally free to exercise broad discretion over the organization and design of research. However, that discretion never operates in a vacuum. Even under normal conditions, the goal of research ethics and policy is to use regulations, reporting guidelines, and other social controls to align research conduct with the public interest. Crucially, the information that research produces is a public good on which caregivers, health systems, and policy-makers rely to efficiently discharge important moral responsibilities. As recent international guidelines for ethical research emphasize, the justification for research is its social and scientific value, understood as its ability to produce the information that multiple actors need to make decisions that implicate health, welfare, and the use of scarce resources (5).

To enable stakeholders to fulfill their social responsibilities, research should embody five conditions of informativeness and social value (6). The first is importance. Trials should address key evidence gaps. Interventions selected for testing should capture the most promising therapeutic and prophylactic alternatives as judged from reviews of existing evidence and trials. They should aim to detect effects that are realistic but clinically meaningful. As of this writing, more than 18 clinical trials enrolling more than 75,000 patients have been registered in North America for testing various hydroxychloroquine regimens for COVID-19. This massive commitment concentrates resources on nearly identical clinical hypotheses, creates competition for recruitment, and neglects opportunities to test other clinical hypotheses. Testing different regimens derived from a common clinical hypothesis in uncoordinated protocols increases the probability of false-positive findings due to chance (7). This also frustrates cross-comparisons and squanders opportunities to evaluate regimens side by side (8).

The second component is rigorous design. Trials should be designed to detect clinically meaningful effects so that both positive and negative results serve the informational needs of clinicians and health systems. Studies designed to detect massive effects often eschew randomization or use surrogate end points. Although easily launched, such studies are at high risk for producing inconclusive findings that sow confusion and necessitate further evaluation. The decision to forgo a dummy comparator and use a nonvalidated surrogate end point, absenteeism, in a study testing use of a tuberculosis vaccine to prevent coronavirus infection jeopardizes the study's ability to clarify the merits of this intervention (9).

The third component is analytical integrity. Designs should be prespecified in protocols, prospectively registered, and analyzed in accordance with prespecification. A recent study of hydroxychloroquine reported a beneficial effect on clinical primary outcomes in a preprint, whereas registration documents revealed a different study design and a polymerase chain reaction–based primary end point. The glaring discrepancy, a well-known source of bias in trials, was not flagged in some reporting on the trial (10).

Fourth, trials should be reported completely, promptly, and consistently with prespecified analyses. One reporting challenge present in the best of times, and likely to reemerge during pandemics, is the deposition of positive findings in preprint servers earlier than nonpositive studies. Another challenge is quality control. Qualified peer reviewers are a scarce resource, and the proliferation of low-quality papers saps the ability of scientists to place findings into context before they are publicized (11). Some recent trials garnering press coverage did not adhere to well-established reporting standards.

The fifth component is feasibility: Studies must have a credible prospect of reaching their recruitment target and being completed within a time frame where the evidence is still actionable. This condition is in tension with the others because their resource demands under conditions of scarcity create the prospect that research might never be completed. However, making research feasible by relaxing the other four standards contradicts the social justification for research. The system of incentives normally used to align research actors with the public good is imperfect in noncrisis situations and likely to be ineffective in the context of a pandemic. Therefore, to meet the requirement of feasibility, investigators, sponsors, health systems, and regulators have responsibilities to make exceptional efforts to cooperate and collaborate in a way that concentrates resources on a portfolio of studies that satisfy the above conditions.

Sponsors, research consortia, and health agencies should prioritize research approaches that test multiple interventions, foster modularity, and permit timely adaptation (12). Master protocols enable multiple interventions to be trialed under a common statistical framework, facilitating cross-comparisons and promoting multicenter collaboration (13). Adaptive designs allow flagging interventions to be dropped quickly and promising alternatives to be added with fewer delays than would be incurred from the design and approval of new studies. Seamless trial designs reduce transition time between trial phases and can extend into the provision of care to large numbers of patients.

Individual clinicians should avoid off-label use of unvalidated interventions that might interfere with trial recruitment and resist the urge to carry out uncontrolled, open-label studies. They should instead seek out opportunities to join larger, carefully orchestrated protocols to increase the prospect that high-quality studies will be completed quickly and generate the information needed to advance individual and public health. Academic medical centers can facilitate such coordination by surveying the landscape of ongoing studies and establishing mechanisms for “prioritization review” to triage studies (14). The goal would be to incentivize participation in efforts that uphold the criteria outlined here and to foster robust participation in multicenter studies so that data can be generated from different institutions before their capacity to meet fastidious research requirements is overwhelmed by surging medical demand.


Embedded Image

Greater coordination of research effort, such as this coronavirus disease 2019 (COVID-19) vaccine trial, can help maintain exacting standards during a crisis.

PHOTO: TED S. WARREN/AP PHOTO

Regulatory agencies and public health authorities should play a leading role in identifying studies that meet these standards and fostering collaboration among a sufficient number of centers to ensure adequate recruitment and timely results. They should also avoid making public recommendations or granting emergency use authorization for interventions whose clinical merits remain to be established and, instead, present clinical trials as a mechanism for addressing uncertainty without compromising patient interests. At public briefings, health authorities can point stakeholders to trials being pursued within their catchment and report recruitment milestones to elevate the profile and progress of high-quality studies.

In a report on the ethics and science of research conducted during the 2014–2015 Ebola outbreak (during which ethical and practical concerns about using standard research methodologies, like randomization and placebo comparators, yielded a body of inconclusive findings), a U.S. National Academy of Medicine committee argued that clinical research is an integral part of outbreak response and that “despite [the] sense of urgency, research during an epidemic is still subject to the same core scientific and ethical requirements that govern all research on human subjects” (15). One lesson of the current outbreak is that expeditious research in a crisis situation is feasible. Absent robust leadership from regulators, health authorities, and major funding bodies, however, the responsibility for coordinating research activities falls to the wide range of stakeholders who might normally pursue research on a more independent basis. Although many of these parties face powerful, parochial incentives to conduct research that is feasible with the resources that are locally available, the exigencies of crisis situations like global pandemics require exceptional steps to combine efforts, divide labor, and triage out low-value and duplicative research.

References and Notes

View Abstract

Stay Connected to Science

Navigate This Article