Grading of Recommendations, Assessment, Development, and Evaluation (GRADE): Pfizer-BioNTech COVID-19 Vaccine for Persons Aged 12–15 Years

Overview

A Grading of Recommendations, Assessment, Development and Evaluation (GRADE) review of the evidence for benefits and harms for Pfizer-BioNTech coronavirus disease 2019 (COVID-19) vaccine for persons aged 12–15 years was presented to the Advisory Committee for Immunization Practices (ACIP) on May 12, 2021. GRADE evidence type indicates the certainty in estimates from the available body of evidence. Evidence certainty ranges from type 1 (high certainty) to type 4 (very low certainty)1.

The policy question was, “Should vaccination with Pfizer-BioNTech COVID-19 vaccine be recommended for persons 12–15 years of age during an Emergency Use Authorization?” The potential benefits pre-specified by the ACIP COVID-19 Vaccines Work Group included prevention of symptomatic laboratory-confirmed COVID-19 (critical), hospitalization due to COVID-19 (important), multisystem inflammatory syndrome in children (MIS-C) (important), SARS-CoV-2 seroconversion (important), and asymptomatic SARS-CoV-2 infection (important). The two pre-specified harms were serious adverse events (critical) and reactogenicity grade ≥3 (important).

A systematic review of evidence on the efficacy and safety of a two-dose regimen of Pfizer-BioNTech COVID-19 vaccine among persons aged 12–15 years was conducted. The quality of evidence from one Phase II/III randomized controlled trial was assessed using a modified GRADE approach.

A lower risk of symptomatic COVID-19 was observed with vaccination compared to placebo (relative risk [RR] 0.03, 95% confidence interval [CI]: 0.00, 0.49, evidence type 1). The available data indicated that serious adverse events were more common in vaccine recipients, but certainty in the estimate was very low (RR 2.50; 95% CI: 0.49, 12.84; evidence type 4, serious concern for indirectness, very serious concern for imprecision), and none of these SAEs were assessed by the Food and Drug Administration (FDA) as related to study intervention. Reactogenicity grade ≥3 was associated with vaccination (RR 5.49; 95% CI: 3.51, 8.58; evidence type 1). About 11% of vaccine recipients and 2% of placebo recipients reported any grade ≥3 local or systemic reactions following either dose 1 or dose 2.

Introduction

As of May 12, 2021, three vaccines have been recommended in adults (aged ≥18 years) for prevention of COVID-19, caused by the SARS-CoV-2 virus which emerged in late 2012349. One vaccine has been recommended for adolescents aged 16–17 years; no vaccines are currently recommended for prevention of COVID-19 in adolescents aged 12–15 years.

On May 10, 2021, the FDA updated the Emergency Use Authorization (EUA) for Pfizer-BioNTech COVID-19 (BNT162b2) vaccine for prevention of symptomatic COVID-19 to include persons aged 12–15 years5. As part of the process employed by the ACIP, a systematic review and GRADE evaluation of the evidence for Pfizer-BioNTech COVID-19 vaccine was conducted and presented to ACIP. There were no conflicts of interest reported by CDC and ACIP COVID-19 Vaccines Work Group members involved in the GRADE analysis.

The ACIP adopted a modified GRADE approach in 2010 as the framework for evaluating the scientific evidence that informs recommendations for vaccine use. Evidence of benefits and harms were reviewed based on the GRADE approach.1

The primary policy question was, “Should vaccination with Pfizer-BioNTech COVID-19 vaccine be recommended for persons 12–15 years of age and older during an Emergency Use Authorization?” (Table 1).

Methods

We conducted a systematic review of evidence on the efficacy and safety of a two-dose regimen of Pfizer-BioNTech COVID-19 vaccine. We assessed outcomes and evaluated the quality of evidence using the GRADE approach.

We identified studies in Medline, Embase, and Cochrane Library, written in English, and limited to studies published from 2020 to April 11, 2021. Search terms included coronavirus, COVID-19, SARS-CoV-2, respiratory (symptom, disease, illness, condition), vaccine, immunization, trial, double blind, single blind, placebo, comparative study, phase I, phase II, phase III, immunogenicity, efficacy, effective, adverse, evidence, and variations on these terms (see Appendix 2 for details).

Articles were included if they provided data on vaccination with the Pfizer-BioNTech COVID-19 vaccine and 1) involved human subjects; 2) reported primary data; 3) included adolescents (ages 12–15) at risk for SARS-CoV-2 infection; 4) included data relevant to the efficacy and safety outcomes being measured; and 5) included data for the specific vaccine formulation, dosage, and timing being recommended (BNT162b2, 30 μg, 2 doses IM, 21 days apart). In addition, efforts were made to obtain unpublished and other relevant data by hand-searching reference lists, and consulting with vaccine manufacturers and subject matter experts. Titles and abstracts were screened independently and in duplicate by two separate reviewers. Characteristics of the included studies are shown in Appendix 1.

Patient-important outcomes (including benefits and harms) for assessment were selected by the Work Group during Work Group calls and via online surveys where members were asked to rate and rank the importance of relevant outcomes. The GRADE assessment across the body of evidence for each outcome was presented in an evidence profile.

Outcomes of interest included individual benefits and harms (Table 2). The critical benefit of interest was prevention of symptomatic laboratory-confirmed COVID-19. Other important outcomes included prevention of hospitalization due to COVID-19, prevention of MIS-C, SARS-CoV-2 seroconversion to a non-spike protein, and asymptomatic SARS-CoV-2 infection. The critical harm of interest was serious adverse events, including vaccine-associated enhanced disease; reactogenicity grade ≥3 was deemed an important harm. No events were observed in the study identified in the review of evidence for hospitalization or MIS-C. Hospitalization, MIS-C, SARS-CoV-2 seroconversion and asymptomatic SARS-CoV2 infection were not included in the evidence profile because no data were available.

Relative risks (RR) were calculated from numerators and denominators available in the body of evidence. A standard continuity correction of 0.5 was used when zero events were observed in one or both arms6. Vaccine efficacy (VE) was calculated as 100% x (1 – RR). In addition to data on symptomatic COVID-19 cases, immunobridging data comparing GMTs in 12–15 year-olds to those in 16–25 year-olds in whom clinical efficacy was previously established was provided in support of efficacy. Immunobridging data were considered to supplement the RRs for efficacy; the geometric mean neutralizing antibody titers (GMT) in 12–15-year-olds was compared to the GMT in 16–25 year-olds in whom clinical efficacy was already established, using a geometric mean ratio (GMR).

Results

The results of the GRADE assessment were presented to ACIP on May 12, 2021.

After title and abstract screening of 5,378 records, 38 studies were identified as eligible for full-text review. Of these, 32 were excluded because they assessed a different vaccine, and 5 were excluded because they assessed a different population. This left 1 study for the evidence synthesis and GRADE evidence assessment7. Characteristics of the included study are shown in Appendix 1.

One study was reviewed that provided data on outcomes specified for GRADE (Appendix 1). Data were reviewed from one Phase II/III randomized controlled trial using data provided by the sponsor7. The final GRADE assessment was limited to the Phase II/III randomized control trial data.

The Pfizer-BioNTech COVID-19 vaccine reduced risk of symptomatic laboratory-confirmed COVID-19 when compared to no COVID-19 vaccination (crude VE: 100%; using continuity correction, VE: 97.1%; 95% CI: 51.0%, 99.8%; based on RR: 0.03; 95% CI: 0.00, 0.49) (Table 3a, Table 4). This was observed with a median follow-up of two months, prompting concern for indirectness due to the short duration of follow-up (i.e., observed outcome of vaccine efficacy at two months does not directly inform vaccine efficacy for any duration longer than two months). However, in consideration of the strength of association, it is unlikely that the efficacy estimate for symptomatic COVID-19 would change substantially. We also note that longer-term efficacy from the adult RCT and strong vaccine effectiveness observed during post-authorization use in adults suggest that short-term efficacy w–ill translate to longer-term efficacy. The geometric mean ratio (GMR) for antibodies in 12–15 year-olds compared to 16–25 year-olds was 1.76 (95% CI:1.47, 2.10), and met the noninferiority criteria (i.e., lower bound of the 2-sided 95% confidence interval for GMR >0.67) (Table 3b). These supplemental immunobridging data indicate that the immune response in adolescents is at least as strong as that observed in adults. For evaluation of potential harms, data were reviewed from the Phase II/III randomized controlled trial. Serious adverse events were more common in vaccine recipients, but certainty in the estimate was very low (RR: 2.50; 95% CI: 0.49, 12.84). There was serious concern of indirectness because the body of evidence does not provide certainty that rare serious adverse events were captured due to the short follow-up and sample size. There was also very serious concern for imprecision, due to the width of the confidence interval. No SAEs were judged by FDA to be related to vaccination (Table 3c). There were no cases of vaccine-associated enhanced disease or deaths. Grade ≥3, or severe, local or systemic reactions within 7 days following either vaccination, were reported by 10.7% of vaccine recipients, and occurred more frequently in the vaccine than placebo groups (Table 3d). No serious concerns impacted the certainty of the estimate of reactogenicity.

GRADE Summary

The initial GRADE evidence level was type 1 (high) for each outcome because the body of evidence was from randomized controlled trials (Table 4). In terms of benefits, the available data indicated that the vaccine was efficacious for preventing symptomatic COVID-19, and no serious concerns impacting certainty in the estimate were identified in the context of the time frame of an Emergency Use Authorization for this outcome (type 1, high). The certainty in the estimate of the effect for serious adverse events was downgraded one point due to serious concern of indirectness related to the median two months follow-up and two points for imprecision due to the width of the 95% confidence interval (type 4, very low certainty). No serious concerns impacted the certainty in the estimate of reactogenicity (type 1, high) (Table 4).

Table 1: Policy Question and PICO

Abbreviations: IM = intramuscular.

Table 2: Outcomes and Rankings

^a No events were observed in study identified in the review of evidence.

^b Data on outcome not available in studies identified in the review of evidence.

Table 3a: Summary of Studies Reporting Symptomatic Laboratory-confirmed COVID-19

Abbreviations: RT-PCR = real-time polymerase chain reaction; CI = confidence interval; RR = relative risk.

^a 1131 and 1129 persons were randomized to vaccine and placebo

^b Based on data cutoff March 13, 2021; participants had a median of two months follow-up.

^c Primary outcome, defined as SARS-CoV-2 RT-PCR-positive symptomatic illness, in seronegative adolescents, ≥7 days post second dose.

^d Symptomatic illness defined as least one respiratory or other COVID-19-related symptom (fever, cough, shortness of breath, chills, muscle pain, loss of taste/smell, sore throat, diarrhea, vomiting), confirmed with PCR during or +/-4 days of symptom onset.

^e Vaccine efficacy calculated using the standard continuity correction of 0.5. Vaccine efficacy based on relative risk of 0.03 (95% CI 0.00, 0.49) differs from calculations provided by the sponsor and FDA, which do not include a continuity correction and are based on person-time analyses.

Table 3b: Summary of Studies Reporting Symptomatic Laboratory-confirmed COVID-19 (assessed using immunobridging)

Abbreviations: NT50 = 50% neutralizing titer; GMR= geometric mean ratio; CI = confidence interval; LLOQ = lower limit of quantitation

^a Among participants who had no serological or virological evidence (up to 1 month after receipt of the last dose) of past SARS-CoV-2 infection and had negative NAAT at any unscheduled visit up to one month after dose two.

^b Sampling time point was one month after dose two.

^c Number of subjects with valid and determinate assay results for the specified assay at the given dose and sampling time point.

^d GMRs and 2-sided 95% CIs were calculated by exponentiating the mean difference of the logarithms of the titers (Group 1 [12–15 years] – Group 2 [16–25 years]) and the corresponding CI (based on the Student t distribution).

^e Noninferiority is declared if the lower bound of the 2-sided 95% CI for the GMR is greater than 0.67.

^f Data were only available for a subset of randomly selected participants because of reagent availability, leading to some concern regarding incomplete outcome ascertainment, but this was judged to be not serious.

Table 3c: Summary of Studies Reporting Serious Adverse Events^a

Abbreviations: RR = relative risk; CI = confidence interval; RCT = randomized controlled trial.

^a Death, life-threatening event, hospitalization, incapacity to perform normal life functions, medically important event, or congenital anomaly/birth defect

^b None of these SAEs were assessed by the FDA as related to study intervention.

Table 3d: Summary of Studies Reporting Reactogenicity^a

Abbreviations: RR = relative risk; CI = confidence interval; RCT = randomized controlled trial.

^a Reactogenicity outcome includes local and systemic events, grade ≥3. Grade 3: prevents daily routine activity or requires use of a pain reliever. Grade 4: requires emergency room visit or hospitalization. One participant in the vaccine group reported grade 4 pyrexia (40.4 °C).

^b Based on interim analysis, data cutoff March 13, 2021.

Table 4. Grade Summary of Findings Table

Abbreviations: CI = confidence interval; RR = relative risk; COVID-19 = coronavirus disease 2019; RCT = randomized controlled trial.

a. Risk of bias related to blinding of participants and personnel was present. Although participants and study staff were blinded to intervention assignments, they may have inferred receipt of vaccine or placebo based on reactogenicity. This was deemed unlikely to overestimate efficacy or underestimate risk of serious adverse events, therefore the risk of bias was rated as not serious.

b. The effects noted are from an analysis of the evaluable efficacy population with outcomes assessed at least 7 days post dose 2 among persons who received two doses and had no evidence of prior SARS-CoV-2 infection. In the all-available efficacy population (persons who received at least 1 dose, with or without evidence of prior SARS-CoV-2 infection), there were 3 cases reported among 1,131 persons who received the vaccine, and 35 cases among 1,129 persons who received the placebo, for a relative risk of 0.09 (95% CI: 0.03 to 0.28).

c. The RCT excluded persons with prior COVID-19 diagnosis, pregnant or breastfeeding women, and persons who were immunocompromised. The population included in the RCT may not represent all persons aged 12–15 years.

d. Concern for indirectness was noted due to the short duration of observation in the available body of evidence. The vaccine efficacy observed at a median 2-month follow-up may differ from the efficacy observed with ongoing follow-up. However, in consideration of the strength of association, it is unlikely that the efficacy estimate for symptomatic COVID-19 would change substantially enough to fall below the FDA-defined efficacy threshold for an Emergency Use Authorization for persons aged ≥16 years (i.e. to <50% efficacy).

e. Absolute risk was calculated using the observed risk among placebo recipients in the available body of evidence. Absolute risk estimates should be interpreted in this context.

f. Risk of bias related to blinding of participants was present. Although participants and study staff were blinded to intervention assignments, they may have inferred receipt of vaccine or placebo based on reactogenicity. Some reactogenicity outcomes may also have been reported as serious adverse events, and experiences of reactions immediately after vaccination could have influenced recall or reporting of subsequent serious adverse events. This was rated as not serious.

g. Serious concern of indirectness was noted. The body of evidence does not provide certainty that rare serious adverse events were captured due to the median 2-month follow-up and the sample size.

h. Very serious concern for imprecision was noted based on the 95% confidence interval crossing the line of no effect (1). The width of the confidence interval contains estimates for which different policy decisions might be considered. This outcome may be imprecise due to the small number of events during the observation period.

Appendix 1. Studies Included in the Review of Evidence

Abbreviations: RCT = randomized controlled trial; COVID-19 = coronavirus disease 2019.

Appendix 2. Databases and strategies used for systematic review

a. Most recent search conducted April 11, 2021.