Grading of Recommendations, Assessment, Development, and Evaluation (GRADE): Moderna COVID-19 Vaccine for Persons Aged 12–17 Years

Overview

A Grading of Recommendations, Assessment, Development and Evaluation (GRADE) review of the evidence for benefits and harms for Moderna coronavirus disease 2019 (COVID-19) vaccine for persons aged 12–17 years was presented to the Advisory Committee for Immunization Practices (ACIP) on June 23, 2022. GRADE evidence type indicates the certainty in estimates from the available body of evidence. Evidence certainty ranges from type 1 (high certainty) to type 4 (very low certainty)1.

The policy question was, “Should vaccination with Moderna COVID-19 vaccine (2 doses, 100 µg) be recommended for persons 12–17 years of age during an Emergency Use Authorization?” The potential benefits pre-specified by the ACIP COVID-19 Vaccines Work Group included prevention of symptomatic laboratory-confirmed COVID-19 (critical), hospitalization due to COVID-19 (important), multisystem inflammatory syndrome in children (MIS-C) (important), and asymptomatic SARS-CoV-2 infection (important). The two pre-specified harms were serious adverse events (SAEs) (critical) and reactogenicity grade ≥3 (important).

A systematic review of evidence on the efficacy and safety of a two-dose regimen of Moderna COVID-19 vaccine among persons aged 12–17 years was conducted. The quality of evidence from one Phase II/III randomized controlled trial was assessed using a modified GRADE approach.

A lower risk of symptomatic COVID-19 was observed among the vaccine group compared with the placebo group (Relative risk [RR]: 0.11; 95% Confidence Interval [CI]: 0.02, 0.50; evidence type 2). Immunobridging data were also assessed in support of efficacy. Among adolescents ages 12–17 years, the immune response to vaccine was non-inferior to that observed in adults ages 18–25 years (GMR: 1.1; 95% CI: 0.9, 1.2; evidence type 2). Additionally, a lower risk of asymptomatic SARS-CoV-2 infection was observed among the vaccine group compared with the placebo group, though the confidence interval was wide and crossed the null (RR: 0.61 (0.24, 1.54); evidence type 3). The available data indicated that SAEs were more common in vaccine recipients, but certainty in the estimate was very low (RR 1.50; 95% CI: 0.30, 7.40; evidence type 4), and none of these SAEs were assessed by the Food and Drug Administration (FDA) as related to study intervention. Reactogenicity grade ≥3 was associated with vaccination (RR 5.23; 95% CI: 4.05, 6.76; evidence type 1). About 25% of vaccine recipients and 5% of placebo recipients reported any grade ≥3 local or systemic reactions following either dose 1 or dose 2.

Introduction

On June 23, 2022, the FDA updated the Emergency Use Authorization (EUA) for Moderna (mRNA-1273) vaccine for prevention of symptomatic COVID-19 to include adolescents aged 12–17 years2. As part of the process employed by the ACIP, a systematic review and GRADE evaluation of the evidence for Moderna COVID-19 vaccine was conducted and presented to ACIP. The ACIP adopted a modified GRADE approach in 2010 as the framework for evaluating the scientific evidence that informs recommendations for vaccine use. Evidence of benefits and harms were reviewed based on the GRADE approach1.

The policy question was, “Should vaccination with Moderna COVID-19 vaccine (2 doses, 100 µg) be recommended for persons 12–17 years of age during an Emergency Use Authorization?” (Table 1).

Methods

We conducted a systematic review of evidence on the efficacy and safety of a two-dose regimen (100 µg per dose) of Moderna COVID-19 vaccine. We assessed outcomes and evaluated the quality of evidence using the GRADE approach.

During Work Group calls, members were asked to pre-specify and rate the importance of relevant patient-important outcomes (including benefits and harms) before the GRADE assessment. No conflicts of interest were reported by CDC and ACIP COVID-19 Vaccines Work Group members involved in the GRADE analysis. Outcomes of interest included individual benefits and harms (Table 2). The critical benefit of interest was prevention of symptomatic laboratory-confirmed COVID-19. Other important outcomes included prevention of hospitalization due to COVID-19, prevention of MIS-C, and prevention of asymptomatic SARS-CoV-2 infection. The critical harm of interest was serious adverse events, including death; reactogenicity grade ≥3 was deemed an important harm. Hospitalization and prevention of MIS-C were not included in the evidence profile because no data were available.

We identified clinical trials through clinicaltrials.gov. Records of relevant Phase I, II, or III randomized controlled trials (RTCs) of COVID-19 vaccine were included if they 1) provided data on adolescents aged 12–17 years vaccinated with mRNA-1273; 2) involved human subjects; 3) reported primary data; and 4) included data relevant to the efficacy and safety outcomes being measured. We identified relevant observational studies through an ongoing systematic review conducted by the International Vaccine Access Center (IVAC) and the World Health Organization (WHO)3. Articles were included if they provided data on vaccination with the Moderna COVID-19 vaccine and 1) involved human subjects; 2) reported primary data; 3) included adolescents (ages 12–17) at risk for SARS-CoV-2 infection; 4) included data relevant to the efficacy and safety outcomes being measured; and 5) included data for the specific vaccine formulation, dosage, and timing being recommended (mRNA-1273, 100 μg, 0.5 mL 2 doses IM, 28 days apart). In addition, efforts were made to obtain unpublished and other relevant data by hand-searching reference lists, and consulting with vaccine manufacturers and subject matter experts. Titles and abstracts were screened independently and in duplicate by two separate reviewers. Characteristics of the included studies are shown in Appendix 1.

Patient-important outcomes (including benefits and harms) for assessment were selected by the Work Group during Work Group calls and via online surveys where members were asked to rate and rank the importance of relevant outcomes. The GRADE assessment across the body of evidence for each outcome was presented in an evidence profile.

Relative risks (RR) were calculated from numerators and denominators available in the body of evidence. Vaccine efficacy estimates were defined as 100% x (1-RR). In addition to data on symptomatic COVID-19 cases, immunobridging data comparing geometric mean neutralizing antibody titers (GMTs) in 12–17-year-olds to those in 18–25-year-olds in whom clinical efficacy was previously established was provided using a geometric mean ratio (GMR).

The evidence certainty assessment addressed risk of bias, inconsistency, indirectness, imprecision, and other characteristics. The GRADE assessment across the body of evidence for each outcome was presented in an evidence profile; the evidence certainty of Type 1, 2, 3, or 4 corresponds to high, moderate, low, or very low certainty, respectively.

Results

The results of the GRADE assessment were presented to ACIP on June 23, 2022. One study was reviewed that provided data on outcomes specified for GRADE (Appendix 1). Data were reviewed from one Phase II/III randomized controlled trial using data provided by the sponsor4. Symptomatic laboratory confirmed COVID-19 occurred less frequently in the Moderna COVID-19 vaccination group when compared to the placebo group (vaccine efficacy [VE]: 89.2%; 95% CI: 49.9–97.6%; based on RR: 0.11; 95% CI: 0.02, 0.50) (Table 3a, Table 4). Serious concern for imprecision was noted due to the small number of events and potential for fragility in the estimate. We also note that the RCT excluded persons with a history of COVID-19, pregnant or breastfeeding women, immunocompromised persons and persons who have medical or psychiatric condition that, according to the investigator’s judgment, may pose additional risk as a result of participation, interfere with safety assessments, or interfere with interpretation of results. Due to these exclusions, the clinical trial participants may not represent all persons aged 12–17 years. The immune response to the Moderna COVID-19 vaccine among adolescents aged 12–17 years was non-inferior (GMR: 1.1; 95% CI: 0.9, 1.2) to the immune response among adults aged 18–25 years receiving the Moderna COVID-19 vaccine in whom clinical efficacy had been established (Table 3b). Serious concern of indirectness was noted because immunobridging is a surrogate measure of efficacy. Asymptomatic SARS-CoV-2 infection occurred less frequently in the vaccination group when compared to the placebo group (vaccine efficacy: 36.1%; 95% CI: -22.0–66.5%; based on RR: 0.64; 95% CI: 0.34, 1.22) (Table 3c, Table 4). Very serious concerns of imprecision were noted due to the confidence interval containing estimates for which different policy decisions would be made and failure to meet minimum information requirements.

For evaluation of potential harms, data were reviewed from the Phase II/III randomized controlled trial. Serious adverse events were more common in vaccine recipients, but certainty in the estimate was very low (RR: 1.50; 95% CI: 0.30, 7.40). There was serious concern of indirectness because the body of evidence does not provide certainty that rare serious adverse events were captured due to the short follow-up (median 2-months) and exclusion criteria, listed previously. Very serious concerns of imprecision were noted due to the confidence interval containing estimates for which different policy decisions would be made and failure to meet minimum information requirements. No SAEs were judged by FDA to be related to vaccination (Table 3d). There were no cases of vaccine-associated enhanced disease or deaths. Grade ≥3, or severe, local or systemic reactions within 7 days following either vaccination, were reported by 25.3% of vaccine recipients, and occurred more frequently in the vaccine than placebo group (Table 3e). No serious concerns impacted the certainty of the estimate of reactogenicity.

GRADE Summary

The initial GRADE evidence level was type 1 (high) for each outcome because the body of evidence was from randomized controlled trials (Table 4). In terms of benefits, the available data indicated that the vaccine was efficacious for preventing symptomatic COVID-19, and the certainty was downgraded once for imprecision due to the small number of events (type 2, moderate). Efficacy assessed with immunobridging was downgraded once for indirectness because it was a surrogate outcome (type 2, moderate). The outcome of prevention of asymptomatic SARS-CoV-2 infection was downgraded twice for very serious concerns of imprecision due to the confidence interval containing estimates for which different policy decisions would be made and failure to meet minimum information requirements. The certainty in the estimate of the effect for serious adverse events was downgraded one point due to serious concern of indirectness related to the short duration of follow up (median two months) and twice for very serious concerns of imprecision due to the confidence interval containing estimates for which different policy decisions would be made and failure to meet minimum information requirements (type 4, very low certainty). No serious concerns impacted the certainty in the estimate of reactogenicity (type 1, high) (Table 4).

Table 1: Policy Question and PICO

Abbreviations: IM = intramuscular.

Table 2: Outcomes and Rankings

^a No events were observed in study identified in the review of evidence.

Table 3a: Summary of Studies Reporting Symptomatic Laboratory-confirmed COVID-19

Abbreviations: RT-PCR = real-time polymerase chain reaction; CI = confidence interval; RR = relative risk.

^a Based on data cutoff May 31, 2021; participants had a median of almost 2 months of follow-up.

^b RT-PCR symptomatic illness defined as: a positive post-baseline PCR result and one or more of the following systemic symptoms: fever (temperature ≥38ºC), or chills, cough, shortness of breath or difficulty breathing, fatigue, muscle aches, or body aches, headache, new loss of taste or smell, sore throat, nasal congestion or rhinorrhea, nausea, or vomiting or diarrhea

^c Negative status at baseline was defined as a negative RT-PCR test for SARS-CoV-2 and a negative serology test based on bAb specific to SARS-CoV-2 nucleocapsid on or before Day 1.

Table 3b: Summary of Studies Reporting Symptomatic Laboratory-confirmed COVID-19 (assessed using immunobridging)

Abbreviations: ID50 = 50% inhibitory dose; GMR= geometric mean ratio; CI = confidence interval

^a Immune response was measured at Day 57

^b Among participants immunologic or virologic evidence of prior COVID-19 (i.e., negative NP swab test at Day 1 and/or binding antibodies against SARS-CoV-2 nucleocapsid below limit of detection or lower limit of quantification) at Day 1 before the first dose of IP.

^c GMRs and 2-sided 95% CIs were calculated by exponentiating the mean difference of the logarithms of the titers (Group 1 [12–17 years] – Group 2 [18–25 years]) and the corresponding CI (based on the student t distribution).

^d Noninferiority is declared if the lower bound of the 2-sided 95% CI for the GMR is greater than 0.67.

Table 3c: Summary of Studies Reporting Asymptomatic SARS-CoV-2 Infection

Abbreviations: RT-PCR = real-time polymerase chain reaction; CI = confidence interval; RR = relative risk

^a Absence of symptoms AND bAb levels against SARS-CoV-2 nucleocapsid protein negative at Day 1 then becomes positive starting at day 57 OR positive RT-PCR counted 14 days after the second dose with negative SARS-CoV-2 at baseline.

Table 3d: Summary of Studies Reporting Serious Adverse Events

Abbreviations: RR = relative risk; CI = confidence interval; RCT = randomized controlled trial.

^a Death, life-threatening event, hospitalization, incapacity to perform normal life functions, medically important event, or congenital anomaly/birth defect

^b Number of participants experiencing SAEs (participants may experience more than one SAE)

^c Included all randomized participants who received at least 1 dose of vaccine.

Table 3e: Summary of Studies Reporting Reactogenicity

Abbreviations: RR = relative risk; CI = confidence interval; RCT = randomized controlled trial.

^a Reactogenicity outcome includes local and systemic events, grade ≥3. Grade 3: prevents daily routine activity or requires use of a pain reliever. Grade 4: requires emergency room visit or hospitalization. One participant in the vaccine group reported grade 4 pyrexia (40.4 °C).

^b Based on interim analysis, data cutoff May 8, 2021.

Table 4. Grade Summary of Findings Table

Abbreviations: CI = confidence interval; RR = relative risk; COVID-19 = coronavirus disease 2019; RCT = randomized controlled trial.

^a Risk of bias related to blinding of participants and personnel was present. Although participants and study staff were blinded to intervention assignments, they may have inferred receipt of vaccine or placebo based on reactogenicity. This was deemed unlikely to overestimate efficacy, therefore the risk of bias was rated as not serious.

^b The effects noted are from an analysis of the evaluable efficacy population with outcomes assessed at least 14 days post dose 2 among persons who received two doses and had no evidence of prior SARS-CoV-2 infection.

^c An additional surrogate outcome of efficacy (immunogenicity) was identified. The immune response in 12–17-year-olds was non-inferior to adults 18–25 years, and consistent with efficacy results.

^d The RCT excluded persons with prior COVID-19 diagnosis, pregnant or breastfeeding women, persons who were immunocompromised, and persons who have a medical or psychiatric condition that, according to the investigator’s judgment, may pose additional risk as a result of participation, interfere with safety assessments, or interfere with interpretation of results. The population included in the RCT may not represent all persons aged 12-17 years.

^e Serious concern for imprecision was noted due to the small number of events and potential for fragility in the estimate.

^f Indirectness noted because immunogenicity is an indirect measure of efficacy.

^g The RCT excluded persons with prior COVID-19 diagnosis, pregnant or breastfeeding women, and persons who were immunocompromised. The population included in the RCT may not represent all persons aged 12-17 years.

^h The immune response to vaccine was evaluated using the geometric mean titer ratio of adolescents to young adults. Non-inferiority criteria are met when the lower bound of the 95% confidence interval for the ratio comparing the geometric mean neutralizing antibody titer for the two groups is not less than a pre-set value, which for this study was 0.67. The immune response to vaccine in adolescents aged 12-17 (GMT: 1401.7 [1276.3, 1539.4]) years was non-inferior to that observed in young adults aged 18-25 years (GMT: 1301.3 [1177.0, 1439.0]), with a geometric mean ratio of 1.1 (0.9-1.2), based on SARS-CoV-2 neutralization titers at 1 month after dose 2, in participants without prior evidence of SARS-CoV-2 infection.

ⁱ Absolute effect not applicable for immunobridging outcome.

^j Very serious concern for imprecision was noted based on the confidence interval containing estimates for which different policy decisions would be made and failure to meet minimum information requirements.

^k Risk of bias related to blinding of participants was present. Although participants and study staff were blinded to intervention assignments, they may have inferred receipt of vaccine or placebo based on reactogenicity. Some reactogenicity outcomes may also have been reported as serious adverse events, and experiences of reactions immediately after vaccination could have influenced recall or reporting of subsequent serious adverse events. This was rated as not serious.

^l Serious concern of indirectness was noted. The body of evidence does not provide certainty that rare serious adverse events were captured due to the median 2-month follow-up and the sample size.

Appendix 1. Studies Included in the Review of Evidence

Abbreviations: RCT = randomized controlled trial; COVID-19 = coronavirus disease 2019.

Appendix 2. Databases and strategies used for systematic review

a. Most recent search conducted April 29, 2022.