Improving vaccine trials in infectious disease emergencies

See allHide authors and affiliations

Science  14 Jul 2017:
Vol. 357, Issue 6347, pp. 153-156
DOI: 10.1126/science.aam8334


Unprecedented global effort is under way to facilitate the testing of countermeasures in infectious disease emergencies. Better understanding of the various options for trial design is needed in advance of outbreaks, as is preliminary global agreement on the most suitable designs for the various scenarios. What would enhance the speed, validity, and ethics of clinical studies of such countermeasures? Focusing on studies of vaccine efficacy and effectiveness in emergencies, we highlight three needs: for formal randomized trials—even in most emergencies; for individually randomized trials—even in many emergencies; and for six areas of innovation in trial methodology. These needs should inform current updates of protocols and roadmaps.

Preparedness for an epidemic of a new or reemerging disease requires the ability to test whether candidate treatments and vaccines against that disease are safe, efficacious, and effective—that is, beneficial both in principle and in the field. During the 2014–2015 West African Ebola outbreak, one rate-limiting step in this process was that vaccine candidates that had shown promise in animal models had never been tested in humans. The entire sequence of safety and dose-ranging trials (phase I), immunogenicity trials (phase IIa), and efficacy and effectiveness trials (phase IIb and III) had to be performed while the outbreak was under way. To expedite future emergency responses, the new Coalition for Epidemic Preparedness Innovations (CEPI) addresses this challenge by funding safety and immunogenicity trials, in anticipation of outbreaks, for vaccines against infections considered at high risk of emergence (1). Unlike efficacy and effectiveness trials, early-phase trials on healthy volunteers can be completed before the relevant infection is present in populations, and outside of outbreak areas.

The Ebola outbreak also illustrated another obstacle to testing candidate countermeasures: At the height of the epidemic, disputes flared about the scientific validity, feasibility, speed, and ethics of competing efficacy trial designs (2, 3). To achieve an expedient response in a way that preserves public trust, such questions should be settled, to the extent possible, before the need arises; this will require methodological and ethical deliberation in advance. The World Health Organization’s work on a Research and Development Blueprint for Action to Prevent Epidemics includes efforts to create “roadmaps,” protocols, and decision-making tools for trials of vaccines against certain high-priority pathogens (4). A new report by the U.S. National Academies of Science, Engineering, and Medicine recommends expanding such activities to include developing generic research protocols and advance arrangements for legal and administrative details that could save time when an emergency occurs (5). We concur and would endorse an even more ambitious agenda for research and consensus-building about testing countermeasures in anticipation of future outbreaks.

Advance planning is especially urgent for those areas of trial design that remain contentious. Disputes about trial ethics, in particular, proved a bottleneck to vaccine efficacy tests during the 2014–2015 Ebola outbreak (2, 3). During responses to earthquakes, blasts, floods, and some other disasters, rescue workers already triage patients with algorithms that have been settled in advance (6), both to expedite decisions and to avoid corrosive ethical disputes (7). In an infectious disease emergency, especially in research, protecting public trust is key—as is advance agreement on trial design.

Excellent trial design is a subtle and complex art. Advance deliberations would allow input from all disciplines involved, stakeholder involvement, and external expert advice, with ample time for discussion of the intricacies of statistical method, philosophy, and local circumstance and culture of various designs and scenario types. We identify three specific needs of trials in emerging disease outbreaks that such advance agreement ought to include: (i) Formal randomized trials will nearly always be needed before a vaccine can be rolled out, even in an emergency. (ii) Even in outbreaks of highly lethal emerging diseases, randomization of individual participants to different arms is likely to remain scientifically desirable and, notwithstanding recent skepticism, ethically permissible. (iii) Scientific and methodological innovation in six areas would help to keep evaluations of candidate vaccines both rapid and credible. For brevity, we focus on vaccine trials, and discuss diagnostics and therapeutics only briefly.

Needed: Randomized trials

During the 2014–2015 West African Ebola outbreak, some questioned whether randomized efficacy trials should be conducted: Once vaccines, for example, had been shown to be safe and immunogenic in humans, and protective of animals upon challenge with the virus, wouldn’t it be unnecessary and unethical to conduct a randomized trial for vaccine candidates, instead of rolling them out while monitoring the effects? Experimental though they remained, vaccine candidates held out at least the possibility of protection from Ebola (3). This reasoning may hold initial appeal, but it fails for nearly all epidemic situations (8, 9). First, the historical record shows repeated instances where vaccines that seemed highly promising prior to phase III were not proven effective, or were even proven harmful, to everyone or to determinate subgroups. Multiple vaccines against the respiratory syncytial virus have met this fate (10). A vaccine that was efficacious against herpes simplex virus type 2 in serodiscordant couples failed to show efficacy when tested in a more representative population (11). Vaccines against HIV (12) and against Staphylococcus aureus (13) that were protective in animal challenge models and immunogenic in humans were not found to be effective in a wider population and worsened some participants’ outcomes during their respective phase III efficacy trials. A dengue vaccine, although beneficial overall, was found during phase III (14) to be harmful to certain identifiable recipients (15). Remarkably, one-third to one-half of vaccines that enter phase III fail to submit applications for regulatory approval, although not always as a result of trial outcomes (16, 17).

Phase III trials for candidate vaccines are not mere formalities, then. They are performed to quantify the degree of protection offered by the vaccine, as well as to test the real possibility that the vaccine, despite promising preclinical and safety results, may be ineffective or harmful in humans exposed to the infection. Indeed, in several cases where vaccines were judged harmful to some in phase III, the harm was an increase in the probability of infection upon exposure, or the probability of severe disease upon infection—safety flaws undetectable in phase I safety trials, which usually take place in areas where exposure is very rare (10, 1214).

During outbreaks, the waste and potential for toxicity might initially be deemed less crucial than the public health need for a chance at protection. But skipping phase III trials could lead to recalls and even medical harm, undermining the public trust in researchers and responders that is crucial during emergencies. The 2014–2015 Ebola outbreak exemplified another consideration—that several promising vaccine candidates can exist. One point of randomized controlled trials is precisely to establish which of them, if any, is worthiest of the advance investment in large-scale production and rollout.

Despite these considerations, one can envision scenarios so urgent that an experimental vaccine should be rolled out before it is proven effective. In an outbreak of a new disease combining the lethality of severe acute respiratory syndrome (SARS) with the transmission characteristics of pandemic influenza, the months required for vaccine trials might cost millions of lives and risk a breakdown of trust. Such circumstances may warrant replacing advance randomized trials with observational studies during rollout, notwithstanding the notoriously challenging methodology for observational vaccine studies (18). Further development of approaches for combining rollout with randomized evaluations, either through individual randomization (19) or through cluster randomization (2022), could help to expand the options for such extreme scenarios. Even though advance randomized trials would be preferable for most disease outbreaks, the precise conditions warranting immediate rollout and plans for evaluating interventions in such circumstances should also be identified in advance.

Needed: The option for individually randomized trials

The central debate about vaccine trial design during the Ebola outbreak concerned the ethics of individual randomization (2, 3). Is it permissible to deny some study participants any study vaccine, offering them only an unrelated vaccine or placebo? Humanitarian workers, public health leaders, and ethicists who accepted the urgency of randomized trials insisted that it would be unethical to conduct them by enrolling individual participants and then randomizing them to receive either the experimental vaccine or a control substance (either a vaccine against a different infection, or a placebo) (Fig. 1). Many espoused instead a form of cluster-randomized trial in which everyone is offered the experimental vaccine during the trial, yet certain groups receive it earlier than others. Vaccine impact is then assessed by comparing incidence between the early and late groups. Two such designs were considered: (i) a stepped-wedge design in which different communities are offered the vaccine in a sequence determined at random, and the incidence is compared between those who have already received the vaccine and those who have yet to receive it (Fig. 1); and (ii) a ring-vaccination design, implemented for the first time by the 2015 Ebola ça Suffit vaccine trial in Guinea, which randomized “rings” of persons at high risk of infection from a confirmed Ebola case, such that they received either immediate or delayed vaccination; incidence in immediately vaccinated rings was compared to incidence in rings with delayed vaccination. It was argued that these two designs were ethically superior to individual control because no participant was denied access to the vaccine throughout the trial (23).

Fig. 1 Comparing stepped wedge and individual randomization for vaccine protection of trial participants.

Top: Stepped-wedge designs offer vaccine candidates to all eligible trial participants at some point during the trial (dark shading) if they remain uninfected and alive. Bottom: Individually randomized trials, where control participants do not receive the experimental vaccine during data collection, ordinarily take less time or fewer participants or fewer disease cases than other designs to achieve a given degree of statistical power, so data collection can end sooner (vertical dashed line). If control participants in such a trial are offered the vaccine at the end of data collection (light shading), this can permit all trial participants to have access to the vaccine earlier than all stepped-wedge participants (24).

Credit: Adapted by N. Cary

Whatever the other virtues of the stepped-wedge and ring-vaccination designs, they are not ethically superior to individual randomization (24, 25). Just as an individually randomized trial would withhold the experimental vaccine from members of the comparator group, the stepped-wedge and ring-vaccination designs withhold the experimental vaccine from members of the delayed group, albeit temporarily (25). More generally, randomizing participants into arms for any trial design creates a disparity in their prospects whenever (i) the experimental intervention has shown enough promise in earlier trial phases to get to phase III; (ii) no proven alternative intervention exists; and (iii) to get the experimental intervention early is better than to get it late if it works. These conditions will be met in many or in most phase III trials for vaccines against emerging infections, including Ebola in 2014–2015. Participants randomized (individually or as a group) to receive the intervention (or to receive it early) have better prospects than other participants, other things being equal. Fair or unfair, that disparity is inevitable whether control is temporal, spatial, by type of intervention, or based on the difference between intervention and placebo. We have termed it the “near-inevitable disparity within all randomized controlled trials of new interventions”; accepting randomized trials—as one should—means accepting this disparity (24).

Sometimes, doses are initially insufficient regardless of study method, and it is impossible to offer the candidate vaccine immediately to all participants at risk. This may seem to support a stepped-wedge design, which can roll out vaccine as it becomes available. However, an individually randomized design with stepped rollout (19) offers multiple scientific advantages over the stepped-wedge design while delivering vaccine as it becomes available.

Finally, individually randomized designs can offer the experimental vaccine to members of the control group after data are collected—once data analysis suggests efficacy or, if so desired, earlier. Inasmuch as individual randomization reaches adequate sample size earlier than the cluster-randomized alternatives (24), experimental vaccines could therefore become available to controls in individually randomized trials sooner than it could to ones in the cluster-randomized alternative designs (Fig. 1).

In short, individually randomized vaccine trials are typically better than cluster-randomized alternative trials in terms of efficiency (because for any result deemed compelling enough for population rollout, the former require fewer participants and/or fewer disease cases to reach that result); there is also a strong preliminary case that individually randomized trials are otherwise ethically no worse than cluster-randomized ones.

Needed: Design and analysis innovations

As diseases continue to emerge, new design challenges surface, calling for novel approaches to trial design and analysis (22). The Ebola ça Suffit trial is an excellent example. This trial addressed the challenge of dwindling cases with a novel design based on vaccinating individuals at high risk of infection from known cases (23). A similar design might be deployed in future emergencies (26), depending on the circumstances. But many challenges remain in vaccine testing during outbreaks, with concomitant opportunities for methodological innovation:

1) Investigators of many emerging disease outbreaks, including Ebola (2729) and Zika (30), have found that incidence is patchy and (at least initially) unpredictable in space and time. Deploying a trial in an area with uncertain future incidence might expend experimental vaccine doses and trial effort without reaching a conclusive result. Designs that place the trial in the vicinity of known cases [as did Ebola ça Suffit (31)], or in areas predicted by modeling to have high later incidence (22, 29), should be further studied to characterize their suitability to particular situations.

2) It is difficult to create trial designs and analysis methods that can withstand the extreme urgency of testing vaccines in emergencies. Initially, this urgency arises from the approximately exponential growth of a typical infectious disease outbreak (Fig. 2, left), making each week of delay miss more prevention opportunities than the preceding week. After effective public health interventions or exhaustion of the susceptible population, an epidemic begins to wane (Fig. 2, right). Then, a new form of time sensitivity takes over: the need to test countermeasures while there remain enough cases for statistically meaningful results. This transition can be seen in Fig. 2, where early growth of cases gave way, by spring 2015, to a situation where there were too few cases to perform vaccine efficacy trials in Sierra Leone and Liberia; to test a vaccine at the end of the epidemic in Guinea (32), a special design (ring vaccination) had to be invented “on the fly” (23). Zika vaccine trials may face similar challenges, largely because those areas most suitable for transmission (e.g., because of high vector density) will be protected by herd immunity resulting from transmission in prior seasons. As a result, it will be hard to identify populations still at risk for substantial outbreaks with enough cases to test vaccines (33).

Fig. 2 Time sensitivity of testing of countermeasures at different stages of the epidemic, illustrated by the epidemic curve of Ebola cases in Guinea and Sierra Leone.

During the ascending phase of an epidemic, case numbers accumulate at an accelerating rate, meaning that each added week of delay in identifying effective prevention or treatment measures brings a larger number of new infections. Once an epidemic is brought under control, dwindling case numbers threaten trial power, because lower disease incidence ordinarily makes more trial participants necessary for statistically robust results. In sum, a race to identify countermeasures to stop the epidemic in the early stage is replaced in the late stage by a race to test the countermeasures before a research opportunity for developing countermeasures for future outbreaks is missed. [Data from (46)]

Credit: Adapted by N. Cary

3) Trials of vaccines for infections that are sometimes asymptomatic or subclinical, such as Nipah (34) and Zika (35), are challenging to design. In the case of Zika, even asymptomatic infections may contribute to birth defects in the fetuses of infected pregnant women (36) and to transmission (37); hence, it is important to measure the ability of vaccines to prevent asymptomatic infections. To do so, one might repeatedly test healthy trial participants for asymptomatic viral infection or, alternatively, test serum samples at the end of follow-up to detect whether participants were infected. Either strategy risks substantially increasing study cost and complexity, and methodological work on streamlining is needed.

“…the historical record shows repeated instances where vaccines that seemed highly promising prior to phase III were not proven effective, or were even proven harmful…”

4) The latest advances in adaptive designs—in which preset rules govern intra-trial adjustments of the number of participants enrolled, the duration of follow-up, the proportion randomized to different arms, and other parameters—may be applicable in responses to outbreaks (38). These adjustments occur in response to experience in the trial; for example, a vaccine trial may increase enrollment if incidence in the enrolled participants has not yet provided enough cases to achieve statistical power, or a treatment trial may increase the proportion of participants randomized to treatments that show the greatest initial promise. Another form of adaptive design combines more than one of the phases of traditional trials (phase I, II, and III) to achieve greater efficiency and speed (39). The Ebola ça Suffit trial (23) and other proposed vaccine trial designs (19) contained adaptive elements. Further work in this direction could expand the options available for trials in future epidemics.

5) New technologies in other areas can also help to improve the precision of vaccine trials. The increasing speed and declining cost of pathogen genome sequencing may allow detection not only of the infection but also of the likely source of the infection. That would enable greater precision in estimating the effects of prevention measures (40), vaccines included.

6) Host responses to vaccination can now be characterized in exquisite detail, with each individual’s response to vaccination measured at the levels of transcription, protein production, cellular proliferation, and antigenic specificity (41). To date, these measurements have not been deployed on a large scale in clinical trials during an emergency. Their deployment could help to elucidate the mechanisms and correlates of protection (42), useful for many decisions about vaccine use in emergencies [e.g., whether to use fractional doses of a scarce vaccine so as to extend supply (43, 44)].


Consensus is emerging that the time to debate trial designs for vaccine candidates against infections under various scenarios is between epidemics, not during them. Exploring in advance the intricate questions inherent in trial ethics and design is especially important. Toward this end, we have defended the need for randomized efficacy trials prior to rollout of new interventions even in most disease outbreaks, the need for individual randomization, and the need for continued development of innovative designs.

Our focus has been on vaccines. Trials of diagnostics are often simpler and can be performed on blood and other specimens that need not be tested in real time. Treatment trials may enable adaptive designs (45) more readily than vaccine trials, because many adaptive designs can work only if outcomes are known for earlier participants before assigning treatments to later participants, and that is more commonly the case in trials of therapeutics than in trials of vaccines. By contrast, individual control, placebo control, and other design elements are more ethically challenging for therapeutics than they are for vaccines, because only in therapeutic efficacy trials are participants already infected with the relevant serious disease. For all these categories of countermeasures, investing now in innovation and consensus-building is likely to save time and ultimately lives.

References and Notes

Acknowledgments: Supported by National Institute of General Medical Sciences grant U54GM088558 (M.L.). We thank R. Kahn, S. Bellan, and A. Rid for helpful conversations as we were writing this article, R. Kahn for research assistance, and anonymous reviewers for written comments. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of General Medical Sciences or the National Institutes of Health. M.L. served on the Scientific Advisory Group for the Ebola ça Suffit vaccine efficacy trial (unpaid position) and has received consulting fees or honoraria from Merck, Pfizer, Antigen Discovery, and Affinivax. His research has received funding (through his employer) from Pfizer and PATH Vaccine Solutions. Discussions leading to the Oxford University Ethox workshop on Ethical Design of Vaccine Trials in Emerging Infections helped stimulate this paper.

Stay Connected to Science

Navigate This Article