Grading of Recommendations, Assessment, Development, and Evaluation (GRADE): Moderna COVID-19 Vaccine

FDA Approved Biologics License Application

About

CDC vaccine recommendations are developed using an explicit evidence-based method based on the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach.

Overview

A Grading of Recommendations, Assessment, Development and Evaluation (GRADE) review of the evidence for benefits and harms for Moderna COVID-19 vaccine was presented to the Advisory Committee for Immunization Practices (ACIP) on February 4, 2022. GRADE evidence type indicates the certainty of estimates from the available body of evidence. Evidence certainty ranges from type 1 (high certainty) to type 4 (very low certainty)1.

The policy question was, “Should vaccination with the Moderna COVID-19 vaccine (Spikevax, 2-dose primary series) be recommended for persons 18 years of age and older?” The potential benefits pre-specified by the ACIP COVID-19 Vaccines Work Group included prevention of symptomatic laboratory-confirmed COVID-19 (critical), hospitalization due to COVID-19 (critical), death due to COVID-19 (important), and asymptomatic SARS-CoV-2 infection (important). The two pre-specified harms were serious adverse events (SAEs) (including myocarditis and anaphylaxis) (critical) and reactogenicity (severe, grade ≥3) (important).

A systematic review of evidence on the benefits and harms of a two-dose regimen of Moderna COVID-19 vaccine among persons aged ≥18 years was conducted, based on data available as of December 10, 2021. The evidence from two Phase I open label trials23, one Phase II randomized controlled trial (RCT)4, one Phase III RCT56, 26 vaccine effectiveness studies7891011121314151617181920212223242526272829303132, and two vaccine safety surveillance systems3334 were assessed using a modified GRADE approach1. Pooled efficacy and effectiveness estimates were calculated when multiple sources had data on an outcome.

In terms of benefits, the available data from RCTs demonstrated that, compared with placebo, vaccination was associated with a lower risk of symptomatic laboratory-confirmed COVID-19 (relative risk [RR] 0.07, 95% confidence interval [CI] 0.05–0.09; evidence type 1), hospitalization due to COVID-19 (RR 0.04; 95% CI 0.01–0.31; evidence type 2), death due to COVID-19 (RR 0.14, 95% CI 0.01–2.79; evidence type 3), and asymptomatic SARS-CoV-2 infection (RR 0.43; 95% CI 0.36–0.50; evidence type 1). The certainty of estimates regarding hospitalization and death due to COVID-19 was reduced due to imprecision.

The pooled vaccine effectiveness estimates from observational studies were consistent with these findings. Compared with no vaccination, vaccination with Moderna COVID-19 vaccine was associated with a decreased risk of symptomatic laboratory-confirmed COVID-19 (RR 0.11, 95% CI 0.06–0.18; evidence type 2), hospitalization (RR 0.05, 95% CI 0.04–0.07; evidence type 2), and death due to COVID-19 (RR 0.06, 95% CI 0.05–0.08; evidence type 2). The certainty of each of these estimates was increased for a strong association. Vaccination was also associated with a decreased risk of asymptomatic SARS-CoV-2 infection (RR 0.30, 95% CI 0.23–0.39; evidence type 4); the evidence certainty type was downgraded for inconsistency.

In terms of harms, the available data from RCTs indicated that serious adverse events were balanced between the vaccine and placebo arms (RR 0.92; 95% CI 0.78–1.08, evidence type 2), and 15 serious adverse events reported for 12 persons were judged to be related to vaccination among more than 15,000 persons vaccinated. The certainty of this estimate was reduced due to imprecision. Reactogenicity grade ≥3 was associated with vaccination (RR 5.03; 95% CI 4.65–5.45, evidence type 1). About 21% of vaccine recipients versus 5% of placebo recipients reported grade ≥3 reactions. Two rare but serious adverse events, anaphylaxis and myocarditis, have been associated with vaccination in post-authorization safety surveillance (see results section and Table 3e).

Introduction

On January 31, 2022, the U.S. Food and Drug Administration (FDA) approved the Biologics License Application (BLA) for Moderna COVID-19 Vaccine (Spikevax) for the prevention of COVID-19 in persons aged ≥18 years35. As part of the process employed by the Advisory Committee for Immunization Practices (ACIP), a systematic review and Grading of Recommendations, Assessment, Development and Evaluation (GRADE) assessment of the evidence for Moderna COVID-19 vaccine was conducted and presented to ACIP1. There were no conflicts of interest reported by CDC and ACIP COVID-19 Vaccines Work Group members involved in the GRADE analysis.

ACIP adopted a modified GRADE approach in 2010 as the framework for evaluating the scientific evidence that informs recommendations for vaccine use. ACIP has made modifications to the GRADE approach by presenting assessed evidence as type 1, 2, 3, and 4, which corresponds to high, moderate, low, and very low certainty, whereas standard GRADE has high as level 4 and very low as level 1. Additionally, instead of presenting the overall certainty of evidence across all outcomes, ACIP presents the certainty of evidence for the benefits and harms separately. ACIP includes an option "ACIP recommends the intervention for individuals based on shared clinical decision-making" instead of providing a conditional recommendation for or against an intervention. GRADE was used to evaluate the efficacy and safety of a two-dose regimen of Moderna COVID-19 vaccine among persons aged ≥18 years. Evidence of benefits and harms were reviewed based on the modified GRADE approach. 1

The policy question was, "Should vaccination with Moderna COVID-19 vaccine (Spikevax, 2-dose primary series) be recommended for persons 18 years of age and older?" (Table 1).

Methods

We conducted a systematic review of evidence on the benefits and harms of a two-dose regimen of Moderna COVID-19 vaccine (see Appendix 2 for databases and search strategies). We assessed outcomes and evaluated the quality of evidence using the GRADE approach. Patient-important outcomes (including benefits and harms) for assessment were selected by the Work Group during Work Group calls and via online surveys where members were asked to rate and rank the importance of relevant outcomes.

We identified RCTs through clinicaltrials.gov. Relevant Phase I, II, or III RCTs of Moderna COVID-19 vaccine were included if they: 1) involved human subjects; 2) reported primary data; 3) included adults (aged ≥18 years) at risk for SARS-CoV-2 infection; 4) included data relevant to the efficacy and safety outcomes being measured; 5) included data for the dosage being recommended (100 μg, 2 doses at 0 and 28 days). We identified relevant observational studies through an ongoing systematic review conducted by the International Vaccine Access Center (IVAC) and the World Health Organization (WHO). Relevant observational studies, using case-control, test-negative, or cohort designs, were restricted to the defined population, intervention, comparison, and outcome outlined in the policy question. Outcomes were assessed starting at least 7 days after 2nd dose. We included only 2-dose Moderna vaccine effectiveness estimates, with combined mRNA vaccine effectiveness estimates excluded. We included studies of general populations and special populations. In addition, efforts were made to obtain unpublished and other relevant data by hand-searching reference lists, and consulting with vaccine manufacturers and subject matter experts. We included observational safety data from two vaccine safety surveillance systems based on input from ACIP's COVID-19 Vaccines Safety Technical (VaST) Work Group: Vaccine Safety Datalink (VSD) and Vaccine Adverse Event Reporting Syste (VAERS). Characteristics of all included studies and surveillance systems are shown in Appendix 1. 23456789101112131415161718192021222324252627282930313233

Two reviewers evaluated all studies for study limitations (risk of bias) using the Cochrane Risk of Bias (RoB) tool for RCTs and the Newcastle-Ottawa Scale (NOS) for observational studies. RoB comprises a series of questions structured into domains focusing on different aspects of trial design, conduct, and reporting. Based on question responses, judgement can be "low", "moderate", or "high" risk of bias. NOS is a 9-point scale which assesses study limitations related to participant selection and comparability, and assessment of outcome (cohort studies) or ascertainment of exposure (case-control studies). Studies with NOS scores <7 were considered to have serious study limitations.

From the RCT data, RR were calculated from numerators and denominators available in the body of evidence. Vaccine efficacy estimates were defined as 100% x (1-RR). Vaccine effectiveness estimates and 95% CIs were taken from the published/preprint studies, as defined by the authors using a variety of study designs and analytical approaches; adjusted estimates were used when available. When multiple studies were available, pooled estimates were calculated using random effects (>3 studies) or fixed effects (≤3 studies) meta-analysis (R meta package). When multiple studies provided estimates based on overlapping study populations, the study with the most comprehensive population and follow-up time was selected for inclusion in the pooled estimate. Because there was a relatively large body of evidence from vaccine effectiveness studies, with many available only in the preprint literature, an a priori decision was made to exclude studies judged to have serious study limitations from the main pooled estimate used for GRADE. Sensitivity analyses were performed to assess the influence of study characteristics (e.g., special populations vs. full population, preprint vs. peer-reviewed, standard vs. extended dosing interval, cohort vs. case-control/test-negative study design, study limitations, and circulating variants). The evidence certainty assessment for randomized and observational studies addressed risk of bias, inconsistency, indirectness, imprecision, and other characteristics. The GRADE assessment across the body of evidence for each outcome was presented in an evidence profile.

Results

The results of the GRADE assessment were presented to ACIP on February 4, 2022.

Outcomes of interest included individual benefits and harms. Indirect effects of vaccination (e.g., societal benefits) were not considered as part of GRADE. Benefits of interest deemed critical were prevention of symptomatic laboratory-confirmed COVID-19 and prevention of hospitalization due to COVID-19 (Table 2). Other important benefits included prevention of death due to COVID-19 and prevention of asymptomatic SARS-CoV-2 infection. The critical harm of interest was serious adverse events (SAEs), including myocarditis and anaphylaxis; reactogenicity grade ≥3 was deemed an important harm.

After screening 134 publications, 96 were excluded from full-text review because they were a different intervention (e.g., a different vaccine or a different dose, n=91), or a different outcome (e.g., any infection instead of symptomatic COVID-19 or asymptomatic infection, n=5). Of the 38 publications that were deemed eligible for full-text review, four were excluded because they assessed a different intervention, and one was excluded because it assessed a different outcome. The remaining 33 publications, which reported data on 31 studies or surveillance systems, were included in the evidence synthesis and GRADE evidence assessment (Appendix 1).23456789101112131415161718192021222324252627282930313233 Data were reviewed from five RCT publications, including two publications from the Phase I trial, one publication from a Phase II trial, and two publications from the Phase III trial23456. Data were reviewed from 26 vaccine effectiveness studies. Two vaccine safety surveillance systems, VSD and VAERS, included data for SAEs .3334

In the Phase III RCT, using data on all blinded follow-up (median follow up of 5 months), the Moderna COVID-19 vaccine reduced symptomatic laboratory-confirmed COVID-19 when compared to placebo (vaccine efficacy: 92.7%, 95% CI 90.4–94.4%) (Table 3a). For hospitalization due to COVID-19, 25 events occurred, 24 in the placebo group and one in the vaccine group. Vaccine efficacy against hospitalization due to COVID-19 was 95.9% (95% CI 69.5–99.4%) (Table 3b). Deaths due to COVID-19 were uncommon, zero in the vaccine group and three in the placebo group (vaccine efficacy: 100%) (Table 3c). Numbers of SAEs were comparable between the vaccine group and the placebo group across the two RCTs (Phase III: 268/15,184 (1.8%) vs. 292/15,164 (1.9%); Phase II: 0/200 (0.0%) vs. 0/200 (0.0%)); there were no cases of vaccine-associated enhanced disease or vaccine-related deaths (Table 3e). Grade ≥3 reactions generally were not uncommon and occurred more frequently in the vaccine than placebo groups (Table 3f).

Fourteen vaccine effectiveness studies reported data on symptomatic laboratory-confirmed COVID-19 (Table 3a), 19 reported data on hospitalization due to COVID-19 (Table 3b), five reported data on death due to COVID-19 (Table 3c), and three reported data on asymptomatic SARS-CoV-2 infection (Table 3d). The pooled vaccine effectiveness estimates from the observational studies demonstrated that the Moderna COVID-19 vaccine reduced symptomatic laboratory-confirmed COVID-19 when compared to no vaccination (pooled vaccine effectiveness: 89.2%, 95% CI 82.0–93.6%; based on 11 studies710121316192022242831). The pooled vaccine effectiveness against hospitalization due to COVID-19 was 94.8% (95% CI 93.1–96.1%), based on 14 studies [8-10, 12, 16-20, 25-28, 31]. The pooled vaccine effectiveness for prevention of death due to COVID-19 was 93.8% (95% CI 91.5–95.4%), based on five studies1017182532. The pooled vaccine effectiveness against asymptomatic SARS-CoV-2 infection was 69.8% (95% CI 60.9–76.7%), based on three studies101228.

Observational data on serious adverse events were reviewed. A rapid cycle analysis from VSD evaluated chart-reviewed cases of myocarditis and pericarditis occurring among persons aged 18–39 years following dose 2 of the Moderna COVID-19 vaccine (Table 3e34). The rate of myocarditis and pericarditis was 33.0 cases per million doses in the 0–7-day risk interval. Data from VAERS showed an elevated ratio of observed to expected myocarditis cases in the 7-day interval following vaccination among females aged 18–29 years and among males aged 18–49 years, with higher observed/expected ratios in males than females34. A rapid cycle analysis of data from VSD evaluated chart-reviewed cases of anaphylaxis among all vaccinated persons aged ≥18 years. Based on events occurring in a 0–1 day risk interval after vaccination, the estimated incidence of confirmed anaphylaxis was 5.1 (95% CI 3.3–7.6) per million doses33.

GRADE Summary

The initial GRADE evidence level was type 1 (high) for randomized controlled trials and type 3 (low) for the observational data (Table 4). In terms of benefits, the RCT data indicated that the vaccine reduces the risk of symptomatic laboratory-confirmed COVID-19, and no serious concerns impacting certainty were identified for this outcome (type 1, high). Observational data for symptomatic laboratory-confirmed COVID-19 indicated a similar risk reduction with vaccination, and the certainty was upgraded one point for a strong association (type 2, moderate). The certainty of the evidence from one RCT for hospitalization due to COVID-19 was downgraded one point for serious concern of imprecision (type 2, moderate). Observational data for hospitalization due to COVID-19 indicated a similar risk reduction with vaccination, and the certainty was upgraded one point for a strong association (type 2, moderate). The certainty of the evidence for death due to COVID-19 was downgraded two points for very serious concern of imprecision (type 3, low). Observational data for death due to COVID-19 concurred with a strong risk reduction with vaccination, and the certainty was upgraded one point for a strong association (type 2, moderate). RCT data indicated that the vaccine reduces the risk for asymptomatic SARS-CoV-2 infection, and no serious concerns impacting certainty were identified for this outcome (type 1, high). Observational data for asymptomatic SARS-CoV-2 infection indicated a similar risk reduction with vaccination, and the certainty was downgraded one point for serious concern of inconsistency (type 4, very low). The certainty of evidence for serious adverse events was downgraded one point for serious concern of imprecision (type 2, moderate). Observational data on specific serious adverse events (i.e., myocarditis among persons aged 18–39 years and anaphylaxis among persons aged 12 years and older) demonstrated these events were rare (evidence type 3, low). No serious concerns impacted the certainty of estimates of reactogenicity from RCTs (type 1, high).

The summary of evidence types is shown in Table 5. The final evidence types were type 1 for symptomatic laboratory-confirmed COVID-19, type 2 for hospitalization due to COVID-19 and death due to COVID-19, type 1 for asymptomatic SARS-CoV-2 infection, type 2 for serious adverse events, and type 1 for reactogenicity.

Tables

Table 1: Policy Question and PICO

Abbreviations: IM: intramuscular; PICO: population, intervention, comparison, outcomes.

Policy question:
Should vaccination with Moderna COVID-19 vaccine (Spikevax, 2-dose, primary series) be recommended for persons 18 years of age and older?
Population
Persons aged ≥18 years
Intervention
Moderna COVID-19 vaccine mRNA-1273 (100 μg, 2 doses IM, 28 days apart)
Comparison
No vaccine
Outcomes
Symptomatic laboratory-confirmed COVID-19
Hospitalization due to COVID-19
Death due to COVID-19
Asymptomatic SARS-CoV-2 infection
Serious Adverse Events (SAEs) (including myocarditis and anaphylaxis)
Reactogenicity (proportion with grade 3 or worse reactions)

Table 2: Outcomes and Rankings

Outcome Importancea Included in evidence profile
Symptomatic laboratory-confirmed COVID-19 Critical Yes
Hospitalization due to COVID-19 Critical Yes
Death due to COVID-19 Important Yes
Asymptomatic SARS-CoV-2 infection Important Yes
Serious Adverse Events (SAEs) (including myocarditis and anaphylaxis) Critical Yes
Reactogenicity (proportion with grade 3 or worse reactions) Important Yes

aThree options: 1. Critical; 2. Important but not critical; 3. Not important for decision making

Table 3a: Summary of Studies Reporting Symptomatic Laboratory-confirmed COVID-19

References in this table:567101112131619202122242831

Authors last name, pub year Design, study population No. of patients vaccinated or No. of cases No. of patients unvaccinated or No. of controls Comparator Vaccine Efficacy/Effectiveness (95% CI) Study limitations (Risk of Bias)
Baden 2020, El Sahly 2021a,b [5, 6] Phase III RCT; Age ≥18 years 55 cases/14,287 vaccine recipients 744 cases/14,164 placebo recipients Placebo 92.7 (90.4–94.4) Not serious
Andrews, 2021c [7] Observational (test-negative design); General population ≥18 years; England 24,328 vaccinated/361,470 cases 100,606 vaccinated/570,293 controls No vaccine 94.8 (94.4–95.2)d Not serious
Bruxvoort, 2021 [10] Observational (prospective cohort); General population ≥18 years; United States 254 cases/352,878 vaccinated 1,078 cases/352,878 unvaccinated No vaccine 88.3 (86.5–89.9)d Not serious
Carazo, 2021 [11] Observational (test-negative design); Healthcare workers; Canada 0 vaccinated/2,813 cases 126 vaccinated/18,663 controls No vaccine 100e Not serious
Chemaitelly, 2021 [12] Observational (test-negative design); General population ≥18 years; Qatar 1 vaccinated/32,527 cases 72 vaccinated/32,527 controls No vaccine 98.6 (92.0–100)d Not serious
Chin, 2021 [13] Observational (retrospective cohort); Incarcerated population; United States 4 cases/459 vaccinated 23 cases/329 unvaccinated No vaccine 84.2 (56.4–94.3)d Not serious
Chung, 2021 [14] Observational (test-negative design); General population ≥18 years; Canada 6 vaccinated/51,226 cases 542 vaccinated /252,083 controls No vaccine 94 (86–97)f Not serious
Grannis, 2021 [16] Observational (test negative design); General population ≥18 years; United States 98 vaccinated/3,243 cases 2,558 vaccinated/10,285 controls No vaccine 92 (89–93)d Not serious
Martinez-Baz, 2021 [19] Observational (prospective cohort); General population ≥18 years; Spain 46 cases/1,127 vaccinated 3,278 cases/14,348 unvaccinated No vaccine 85 (80–89)d Not serious
Nasreen, 2021 c [20] Observational (test negative design); General population ≥18 years; Canada 28 vaccinated/1,859 cases 9,893 vaccinated/ 450,284 controls No vaccine 94 (86–97)d Not serious
Nordström, 2021 c [22] Observational (retrospective cohort); General population ≥18 years; Sweden 1.0 case /100,000 person days among vaccinated 2.6 cases/100,000 person days among unvaccinated No vaccine 71 (56–81)d Not serious
Nordström, 2021 [21] Observational (retrospective cohort); General population ≥18 years; Sweden 1.3 cases/100,000 person days among vaccinated 13.5 cases/100,000 person days among unvaccinated No vaccine 87 (84–88)f Not serious
Pilishvili, 2021 [24] Observational (test negative design); Healthcare workers; United States 18 vaccinated/1,472 cases 190 vaccinated/3,420 controls No vaccine 96.3 (91.3–98.4)d Not serious
Tang, 2021 [28] Observational (test-negative design); General population ≥18 years; Qatar 75 vaccinated/1,356 cases 821 vaccinated/5,573 controls No vaccine 73.9 (65.9–79.9)d Not serious
Thompson , 2021 [31] Observational (test negative design); General population ≥50 years; United States 49 vaccinated/2,896 cases 2,427 vaccinated /11,392 controls No vaccine 92 (89–94)f Not serious

aAssessed using a primary outcome of the RCT, defined as SARS-CoV-2 RT-PCR-positive symptomatic illness, in seronegative adults, ≥14 days post second dose. Seronegative status was not a criterion for inclusion of observational studies.

bAdditional data provided by sponsor

cPreprint

dVaccine effectiveness estimate included in main pooled analysis used for GRADE.

e Vaccine effectiveness estimate not included in main pooled analysis used for GRADE because a confidence interval for the effect estimate was not provided

f Vaccine effectiveness estimate not included in main pooled analysis used for GRADE because study population overlapped with another study that was included.

Table 3b: Summary of Studies Reporting Hospitalization due to COVID-19

References in this table: 568910121516171819202325262728293031

Authors last name, pub year Design, study population No. of patients vaccinated or No. of cases No. of patients unvaccinated or No. of controls Comparator Vaccine Efficacy/Effectiveness, % (95% CI) Study limitations (Risk of Bias)
Baden 2020, El Sahly 2021a [5, 6] Phase III RCT, Age ≥18 years 1 hospitalization/14,287 vaccine recipients 24 hospitalizations/14,164 placebo recipients Placebo 95.9 (69.5–99.4) Not serious
Andrews, 2021b [7] Observational (test negative design); General population ≥18 years; United Kingdom 0 vaccinated/Not reported 6,363 vaccinated/Not reported No vaccine 100c Not serious
Bajema, 2021 [8] Observational (test negative design); Hospitalized VA; United States 11 vaccinated/388 cases 136 vaccinated/787 controls No vaccine 91.6 (83.5–95.7)d Not serious
Bruxvoort, 2021 [9] Observational (test negative design); General population ≥18 years; United States 5 vaccinated/141 cases 365 vaccinated/705 controls No vaccine 97.5 (92.7–99.2)d Not serious
Bruxvoort, 2021  [10] Observational (prospective cohort); General population ≥18 years; United States 13 hospitalizations /352,878 vaccinated 182 hospitalizations/352,878 unvaccinated No vaccine 95.8 (92.5–97.6)d Not serious
Chemaitelly, 2021 [12] Observational (test-negative design); General population ≥18 years; Qatar 1 vaccinated/3,394 cases 23 vaccinated/3,394 controls No vaccine 95.7 (73.4–99.9)d Not serious
Embi, 2021 [15] Observational (test negative design); General population ≥18 years; United States 357 vaccinated/10,210 cases 11,984 vaccinated/41,791 controls No vaccine 93 (92–94)e Not serious
Grannis, 2021 [16] Observational (test negative design); General population ≥18 years; United States 70 vaccinated/1,386 cases 2,905 vaccinated/8,549 controls No vaccine 95 (92–97)d Not serious
Irizarry, 2021a [17] Observational (retrospective cohort); General population ≥18 years; United States Not reported Not reported No vaccine 90 (84–94)d Not serious
Lin, 2022 [18] Observational (retrospective cohort); General population ≥18 years; United States Not reported Not reported No vaccine 94.9 (92.4–96.6)d Not serious
Martinez-Baz, 2021 [19] Observational (prospective cohort); General population ≥18 years; Spain 1 hospitalization/1,127 vaccinated 216 hospitalizations/14,348 unvaccinated No vaccine 98 (82–100)d Not serious
Nasreen, 2021a [20] Observational (test negative design); General population ≥18 years; Canada Not reported Not reported No vaccine 98 (93–100)d Not serious
Pawlowski, 2021 [23] Observational (retrospective cohort); ≥18 years with access to Mayo Health system (MN); United States 5 hospitalizations/15,985 vaccinated 52 hospitalizations/15,896 unvaccinated No vaccine 90.6 (76.5–97.1)e Not serious
Puranika , 2021 [25] Observational (retrospective cohort); ≥18 years with access to Mayo Health system (MN); United States 6 hospitalizations /2,215,483 person-days among vaccinated 82 hospitalizations/2,532,948 person-days among unvaccinated No vaccine 91.6 (81–97)d Not serious
Self, 2021 [26] Observational (test negative design); Hospitalized adults ≥18 years; United States 54 vaccinated/1,517 cases 422 vaccinated/1,321 controls No vaccine 93 (91–95)d Not serious
Skowronskia, 2021 [27] Observational (test negative design); General population ≥18 years; Canada (British Columbia and Quebec) British Columbia: 38 vaccinated/1403 cases
Quebec: 24 vaccinated/726 cases
British Columbia: 41,889 vaccinated/118,510 controls
Quebec: 109,861 vaccinated/322,779 controls
No vaccine British Columbia: 97 (96–98)d
Quebec: 97 (95–98)d
Not serious
Tang, 2021 [28] Observational (test-negative design); General population ≥18 years; Qatar 1 vaccinated/104 cases 71 vaccinated/376 controls No vaccine 96.1 (71.6–99.5)d Not serious
Tenforde, November 2021 [30] Observational (test negative design); Hospitalized adults ≥18 years; United States 32 vaccinated/1,701 cases 103 vaccinated/1,247 controls No vaccine 85 (77–91)e Not serious
Tenforde, August 2021 [29] Observational (test negative design); Hospitalized adults ≥18 years; United States 17 vaccinated/473 cases 93 vaccinated/366 controls No vaccine 90.1 (82.3–94.5)e Not serious
Thompson, 2021 [31] Observational (test negative design); General population ≥50 years; United States 95 vaccinated/3,790 cases 6,279 vaccinated/22,990 controls No vaccine 91 (89–93)d Not serious

a Additional data provided by study sponsor. Hospitalization due to COVID-19 was a post-hoc analysis.

b Preprint

c Vaccine effectiveness estimate not included in main pooled analysis used for GRADE because a confidence interval for the effect estimate was not provided.

d Vaccine effectiveness estimate included in main pooled analysis used for GRADE.

e Vaccine effectiveness estimate not included in main pooled analysis used for GRADE because study population overlapped with another study that was included.

Table 3c: Summary of Studies Reporting Death due to COVID-19

References in this table:561017182532

Authors last name, pub year Design, study population No. of patients vaccinated No. of patients unvaccinated Comparator Vaccine Efficacy/Effectiveness, % (95% CI) Study limitations (Risk of Bias)
Baden 2020, El Sahly 2021a [5, 6] Phase III RCT, Age ≥18 years 0 deaths/14,287 vaccine recipients 3 deaths/14,164 placebo recipients Placebo 100 Not serious
Bruxvoort, 2021 [10] Observational (prospective cohort); General population ≥18 years; United States 1 death/352,878 vaccinated 25 deaths/352,878 unvaccinated No vaccine 97.9 (84.5–99.7)b Not serious
Irizarry, 2021c [17] Observational (retrospective cohort); General population ≥18 years; United States Not reported Not reported No vaccine 93 (81–97)b Not serious
Lin, 2021 [18] Observational (retrospective cohort); General population ≥18 years; United States Not reported Not reported No vaccine 93.7 (90.2–95.9)b Not serious
Puranik, 2021a [25] Observational (retrospective cohort); ≥18 years with access to Mayo Health system (MN); United States 0 deaths/2,215,773 person-days among vaccinated 4 deaths/2,537,030 person-days among unvaccinated No vaccine 100 (-70–100)b Not serious
Voko, 2021 [32] Observational (retrospective cohort); General population ≥18 years; Hungary 0.31 deaths/100,000 person-days​ among vaccinated 1.80 deaths/100,000 person-days among unvaccinated No vaccine 93.6 (90.5–95.7)b Not serious

a Additional data provided by study sponsor

b Vaccine effectiveness estimate included in main pooled analysis used for GRADE.

c Preprint

Table 3d: Summary of Studies Reporting Asymptomatic SARS-CoV-2 infection (assessed bAB levels against SARS-CoV-2)

References in this table:56101228

Authors last name, pub year Design, study population No. of patients vaccinated or No. of cases No. of patients unvaccinated or No. of controls Comparator Vaccine Efficacy/Effectiveness, % (95% CI) Study limitations (Risk of Bias)
Baden 2020, El Sahly 2021a [5, 6] Phase III RCT, Age ≥18 years 214 cases/14,287 vaccinated 498 cases/14,164 unvaccinated Placebo 57.4 (50.1–63.6) Not serious
Bruxvoort, 2021 [10] Observational (prospective cohort); General population ≥18 years; United States 35 cases/352,878 vaccinated 66 cases/352,878 unvaccinated No vaccine 72.7 (57.6–82.4)b Not serious
Chemaitelly, 2021 [12] Observational (test-negative design); General population ≥18 years; Qatar 8 vaccinated/37,776 cases 107 vaccinated/37,776 controls No vaccine 92.5 (84.8–96.9)b Not serious
Tang, 2021 [28] Observational (test-negative design); General population ≥18 years; Qatar 53 vaccinated/528 cases 368 vaccinated/2,323 controls No vaccine 53.6 (33.4–67.6)b Not serious

aAdditional data provided by study sponsor

bVaccine effectiveness estimate included in main pooled analysis used for GRADE.

Table 3e: Summary of Studies Reporting Serious Adverse Eventsa

References in this table: 234563334

Authors last name, pub year Age or other characteristics of importance n/N (%) intervention n/N (%) comparison Comparator RR (95% CI) Study limitations (Risk of Bias)
Anderson, 2020a [2] Phase I open label study, persons aged >55 years 1/20 b
Jackson, 2020a [3] Phase I open label study, persons aged 18-55 years 1/15 b
Chu, 2021a [4] Phase II RCT, Adults ≥18 years 0/200 0/200 Placebo 0 Not Serious
Baden 2020, El Sahly 2021a [5, 6] Phase III RCT, Adults ≥18 years 268/15,184c 292/15,164 Placebo 0.92 (0.78, 1.08) Not Serious
VAERS (Myocarditis)d [34] Age ≥18 years Observed cases by age (years) and sex per 1,000,000 second dosesf
Females
Age 18–24: 5.5
Age 25–29: 5.8
Age 30–39: 0.6
Age 40–49: 1.6
Age 50–64: 0.4
Age ≥ 65: 0.5
Males
Age 18–24: 40.0
Age 25–29: 18.3
Age 30–39: 8.4
Age 40–49: 3.5
Age 50–64: 0.9
Age ≥ 65: 0.6
Expected cases per 1,000,000 second doses overallf
0.2 – 2.2
Expected numbers occurring in population Elevated ratio of observed to expected cases among females aged 18–29 years and males 18–49 years. Serious
VSD (Anaphylaxis)e [33] Age ≥18 years 5.1/1,000,000 doses Not serious
VSD (Myocarditis)f [34] 18–39 years 33.0/1,000,000 second doses Not serious

aAdditional data provided by sponsor

bRisk of bias was not formally assessed for these small studies with no comparator; these were not included in the quantitative estimate used for GRADE.

c15 serious adverse events reported in 12 participants were deemed by blinded investigators to be related to vaccination. These included: B-cell small lymphocytic lymphoma, Basedow's disease, autonomic nervous system imbalance, cerebrovascular accident, multiple sclerosis, pericardial effusion, pericarditis, pleural effusion, nausea, vomiting, alopecia areata, angioedema, rheumatoid arthritis, and two reports of facial swelling.

dRisk evaluated in a 7-day interval following vaccination

eRisk evaluated in a 0–1 day risk interval after vaccination

fRisk evaluated in a 7-day interval following dose 2

Table 3f: Summary of Studies Reporting Reactogenicitya

References in this table:123456

Authors last name, pub year Age or other characteristic of importance n/N (%) intervention n/N (%) comparison Comparator RR (95% CI) Study limitations (Risk of Bias)
Anderson, 2020 [2] b Phase I open label study, persons aged >55 years 1/20 (5.0) c
Jackson, 2020 [3] b Phase I open label study, persons aged 18-55 years 1/15 (6.7) c
Chu, 2021 [4] b Phase II RCT, persons aged ≥18 years 32/200 (16.0) 6/200 (3.0) Placebo 5.33 (2.28, 12.47) Not serious
Baden 2020, El Sahly 2021 [5, 6] b Phase III RCT, persons aged ≥18 years 3,243/15,179 (21.4) 679/15,159 (4.5) Placebo 5.03 (4.65, 5.45) Not serious

aGrade 3 or worse. Grade 3 local reactions include pain at injection site or axillary swelling/tenderness that prevents daily activity, redness > 10 cm, and swelling > 10 cm. Grade 3 systemic events include fever >38.9°C to 40.0°C; vomiting that requires intravenous hydration; and headache, fatigue/tiredness, new or worsened muscle pain, or new or worsened joint pain that prevents daily routine activity.

bAdditional data provided by sponsor.

cRisk of bias was not formally assessed for these small studies with no comparator; these were not included in the quantitative estimate used for GRADE.

Table 4. Grade Summary of Findings Table

Certainty assessment № of patients Vaccinated № of patients Unvaccinated Effect Relative
(95% CI)
Effect Absolute
(95% CI)
Certainty Importance
№ of studies Study design Risk of bias Inconsistency Indirectness Imprecision Other considerations
Symptomatic laboratory-confirmed COVID-19
1 Randomized studies not seriousa not serious not seriousb,c not serious none 55/14287 (0.4%) 744/14164 (5.3%) RR 0.07
(0.05–0.09)
4,885 fewer per 100,000
(from 4,990 fewer to 4,780 fewer)d
Type 1 CRITICAL
11e Nonrandomized studiesf not serious not seriousg not serioush not serious strong association f,i f,i RR 0.11
(0.06–0.18)j
4,717 fewer per 100,000
(from 4,982 fewer to 4,346 fewer)d
Type 2 CRITICAL
Hospitalization for COVID-19
1 Randomized studies not seriousa,k not serious not seriousc seriousl none 1/14287 (0.0%) 24/14164 (0.2%) RR 0.04
(0.01 to 0.31)
163 fewer per 100,000
(from 168 fewer to 117 fewer)d
Type 2 CRITICAL
15m Nonrandomized studiesf not serious not seriousn not seriouso not serious strong association f,p f,p RR 0.05
(0.04 to 0.07)j
190 fewer per 100,000
(from 192 fewer to 186 fewer)d
Type 2 CRITICAL
Death due to COVID-19
1 Randomized studies not seriousa not serious not seriousc very seriousq none 0/14287 (0.0%) 3/14164 (0.0%) RR 0.14
(0.01 to 2.79)r
18 fewer per 100,000
(from 21 fewer to 38 more)d
Type 3 IMPORTANT
5s Nonrandomized studiese not serious not serioust not seriousu not serious strong association f,v f,v RR 0.06
(0.05 to 0.08)j
19 fewer per 100,000
(from 19 fewer to 18 fewer)d
Type 2 IMPORTANT
Asymptomatic SARS-CoV-2 infection, assessed with PCR
1 Randomized studies not seriousa not serious not seriousc not serious none 214/14287 (1.5%) 498/14164 (3.5%) RR 0.43
(0.36 to 0.50)
2,004 fewer per 100,000
(from 2,250 fewer to 1,758 fewer)d
Type 1 IMPORTANT
3s Nonrandomized studiesf not serious seriousw not serious not serious none f,x f,x RR 0.30
(0.23 to 0.39)y
2,450 fewer per 100,000
(from 2,695 fewer to 2,135 fewer)d
Type 4 IMPORTANT
Serious adverse events
2z Randomized studies not seriousa not serious not seriousc seriousaa none 268/15384 (1.7%) 292/15364 (1.9%) RR 0.92
(0.78 to 1.08)ab
152 fewer per 100,000
(from 418 fewer to 152 more)d
Type 2 CRITICAL
2w Nonrandomized studies not serious not serious not seriousac not serious none See narrativead,ae,af Type 3 CRITICAL
Reactogenicity, grade >=3
2ah Randomized studies not serious not serious not seriousc not serious none 3275/15379 (21.3%) 685/15359 (4.5%) RR 5.03
(4.65 to 5.45)ab
17,974 more per 100,000
(from 16,279 more to 19,847 more)d
Type 1 IMPORTANT
0 Nonrandomized studies

CI: Confidence interval; RR: Risk ratio

Explanations

a. Risk of bias related to blinding of participants and personnel was present. Although participants and study staff were blinded to intervention assignments, they may have inferred receipt of vaccine or placebo based on reactogenicity. This was deemed unlikely to overestimate efficacy or underestimate risk of serious adverse events, therefore the risk of bias was rated as not serious.
b. The effects noted are from a per protocol analysis with outcomes assessed at least 14 days post dose 2 among persons who received two doses and had no evidence of prior SARS-CoV-2 infection. In an analysis using the full analysis set (persons with or without evidence of prior SARS-CoV-2 infection), there were 58 cases among 15180 persons in the vaccine arm and 754 cases among 15166 persons in the placebo arm (RR = 0.08 (0.06 to 0.10)).
c. The RCT excluded persons with prior COVID-19 diagnosis, pregnant or breastfeeding women, and persons who were in an immunosuppressive or immunodeficient state, had asplenia or recurrent severe infections (HIV-positive participants on stable antiretroviral therapy were not excluded). The population included in the RCT may not represent all persons aged >=18 years.
d. Absolute risk was calculated using the observed risk among placebo recipients in the available body of evidence from randomized controlled trials. Absolute risk estimates should be interpreted in this context.
e. 14 studies were available in the body of evidence. 2 were excluded because the study population was already represented, and 1 was excluded because the confidence interval for the effect estimate was provided.
f. The body of evidence includes preprints.
g. Although I2 value was high (96.6%), no serious concern for inconsistency was judged because all studies showed a high degree of vaccine effectiveness, with point estimates ranging from 79% to 92%.
h. Two included studies measured COVID-19 vaccine effectiveness against laboratory-confirmed COVID-19–associated emergency department and urgent care clinic encounters and hospitalizations. In a sensitivity analysis excluding these studies the pooled RR was 0.11 (95% CI: 0.06, 0.22).
i. Data on numerators and denominators were not consistently reported in the available body of evidence. Seven test negative studies contributed 404,823 cases and 1,083,774 controls. In addition to the test negative study data, four cohort studies contributed data to the pooled estimate.
j. Pooled RR based on a random effects meta-analysis, using adjusted vaccine effectiveness estimates on a log scale.
k. Hospitalization due to COVID-19 was not a protocol defined outcome and was provided in a post-hoc analysis, which may be subject to bias. However, severe COVID-19, defined per FDA guidance, was a protocol defined outcome. There were 2 severe COVID-19 cases among 14164 persons in the vaccine arm and 106 severe COVID-19 cases among 14287 persons in the placebo arm (RR = 0.02 (0.00 to 0.08)). Due to the similarity of these estimates, we deemed the risk of bias as not serious.
l. Serious concerns of imprecision due to fragility in the estimate was present because there were only 25 events observed from a single RCT.
m. 19 studies were available in the body of evidence. Four were excluded because the study population was already represented, and one was excluded because no confidence interval for the effect estimate was provided. One included study provided two VE estimates from distinct study locations.
n. Although I2 value was high (77.6%), no serious concern for inconsistency was judged because all studies showed a high degree of vaccine effectiveness, with point estimates ranging from 93% to 97%.
o. Definitions varied by study. Indirectness was considered given COVID-19 was not necessarily confirmed as the cause of hospitalizations, but this was deemed not serious.
p. Data on numerators and denominators were not consistently reported in the available body of evidence. Ten test negative studies contributed 9,059 cases and 457,807 controls. In addition to the test negative study data, five cohort studies contributed data to the pooled estimate.
q. Very serious concern for imprecision was present due to the small number of events that were observed. In addition to a 95% confidence interval crossing the line of no effect, there was concern for fragility in the estimate due to the small number of events.
r. RR calculated using a standard continuity correction of 0.5.
s. All available studies in the body of evidence were included in the pooled estimate.
t. The relative risk shown is from a pooled analysis of 5 cohort studies conducted in different populations. I2 was 0%.
u. Definitions varied by study. Indirectness was considered given COVID-19 was not necessarily confirmed as the cause of deaths, but this was deemed not serious.
v. Data on numerators and denominators were not consistently reported in the available body of evidence. The n is not included because studies reported rates and did not report the number of cases. The N is not included because studies variously provided person-time.
w. Serious concern for inconsistency was present (I2= 90.3%). The magnitude of the relative risks from the three studies in the body of evidence varied widely, possibly reflecting different prevalence of circulating SARS-CoV-2 variants at the time of data collection or differences in study design.
x. Data on numerators and denominators were not consistently reported in the available body of evidence. Two test negative studies contributed 38,304 cases and 40,099 controls. In addition to the test negative study data, one cohort study contributed data to the pooled estimate.
y. Pooled RR based on a fixed effects meta-analysis, using adjusted vaccine effectiveness estimates on a log scale. Fixed effects model was used for this analysis due to imprecise estimates of the between-studies variance.
z. Two small Phase 1 observational studies with no comparison group were not included in the GRADE analysis. In a total of 35 vaccinated persons, 0 serious adverse events were observed.
aa. Serious concern for imprecision was present. The confidence interval indicates that both reduced and increased risk of serious adverse events are possible.
ab. Pooled RR based on a fixed effects meta-analysis. Fixed effects model was appropriate for this analysis because these RCTs used the same protocol and were conducted in similar populations.
ac. For the outcome of myocarditis evaluated in Vaccine Safety Datalink, data are shown for the age groups 18-39, therefore these were not completely generalizable to the age groups of all persons aged >=18 years as defined in the PICO question. This was deemed not serious.
ad. A rapid cycle analysis from Vaccine Safety Datalink (VSD) evaluated chart-reviewed cases of myocarditis occurring among persons aged 18–39 years following dose 2. Based on events occurring in a 7-day risk interval after vaccination vs. a comparison interval in vaccinated individuals, the adjusted rate ratio was 18.75 (95% CI 6.73–64.94). The excess cases of myocarditis in the risk interval were 31.2 per million doses.
ae. Data from the national Vaccine Adverse Event Reporting System (VAERS) showed an elevated ratio of observed to expected myocarditis cases in the 7-day interval following vaccination among females in age groups 18-29 years, and among males in age groups 16-49 years, with higher observed/expected ratios in males than females. Although VAERS data are subject to the limitations of a passive surveillance system, the elevated risk of myocarditis following Moderna vaccination is consistent with that observed in VSD.
af. A rapid cycle analysis of data from VSD evaluated chart-reviewed cases of anaphalaxis among all vaccinated persons aged 18 years and older. Based on events occurring in a 0–1 day risk interval after vaccination, the estimated incidence of confirmed anaphalaxis was 5.1 (95% CI 3.3–7.6) per million doses.

Table 5: Summary of Evidence for Outcomes of Interest

Outcome Importance Included in evidence profile Certainty
Symptomatic laboratory-confirmed COVID-19 Critical Yes Type 1 (high)
Hospitalization due to COVID-19 Critical Yes Type 2 (moderate)
Death due to COVID-19 Important Yes Type 2 (moderate)
Asymptomatic SARS-CoV-2 infection Important Yes Type 4 (very low)
Serious Adverse Events (SAEs) (including myocarditis and anaphylaxis) Critical Yes Type 2 (moderate)
Reactogenicity (proportion with grade 3 or worse reactions) Important Yes Type 1 (high)

Appendix 1. Studies Included in the Review of Evidence

Randomized Controlled Trial

References in this table:23456

Last name first author, Publication year Study design Country (or more detail, if needed) Age, central tendency or range Total population N vaccinated N unvaccinated Outcomes Funding source
Baden 2020, El Sahly 2021a,b [5, 6] RCT United States ≥18 years 30,415 14,287 14,164
  • Symptomatic COVID-19 (PCR confirmed)c
  • Hospitalization due to COVID-19
  • Death due to COVID-19
  • Asymptomatic infection
  • Serious Adverse Events
  • Reactogenicityd
Industry funding
 Anderson, 2020a [2] RCT United States >55 years 40 20 0
  • Serious Adverse Events
  • Reactogenicityd
Industry funding
Jackson, 2020a [3] RCT United States 18-55 years 45 15 0
  • Serious Adverse Events
  • Reactogenicityd
Industry funding
Chu, 2021a [4] RCT United States ≥18 years 600 200 200
  • Serious Adverse Events
  • Reactogenicityd
Industry funding

Observational Cohort Studies

References in this table:10131718192122232532

Last name first author, Publication year Study design Country (or more detail, if needed) Age, central tendency or range Total population N vaccinated N unvaccinated Outcomes Funding source
Bruxvoort, 2021 [10] Observational (Prospective Cohort) United States ≥18 years 705,756 352,878 352,878
  • Symptomatic COVID-19 (PCR confirmed)c
  • Hospitalization due to COVID-19
  • Death due to COVID-19
  • Asymptomatic infection
Industry funding
Chin, 2021 [13] Observational (Retrospective Cohort) United States, Incarcerated population ≥18 years 3,221 459 329
  • Symptomatic COVID-19 (PCR confirmed)c
University/Academic & Government funding
Irizarry, 2021a [17] Observational (Retrospective Cohort) United States ≥18 years NRe NRe NRe
  • Hospitalization due to COVID-19
  • Death due to COVID-19
Government funding
Lin, 2021 [18] Observational (Retrospective Cohort) United States ≥18 years NRe NRe NRe
  • Hospitalization due to COVID-19
  • Death due to COVID-19
University/Academic & Government funding
Martinez-Baz, 2021 [19] Observational (Prospective Cohort) Spain ≥18 years 30,240 1,127 contacts 14,348 contacts
  • Symptomatic laboratory-confirmed COVID-19c
  • Hospitalization due to COVID-19
Government funding & Other (Horizon 2020 program of the European Commission)
Nordström, 2021 [21] Observational (Retrospective Cohort) Sweden ≥18 years 721,787 312 889
  • Symptomatic laboratory-confirmed COVID-19c
None declared
Nordström, 2021a [22] Observational (Retrospective Cohort) Sweden ≥18 years 1,684,958 NRe NRe
  • Symptomatic laboratory-confirmed COVID-19c
None declared
Pawlowski, 2021 [23] Observational (Retrospective Cohort) United States ≥18 years 136,532 NRe NRe
  • Hospitalization due to COVID-19
Other (nference; data analysis organization)
Puranika,  [25] Observational (Retrospective Cohort) United States ≥18 years 179,546 NRe NRe
  • Hospitalization due to COVID-19
  • Death due to COVID-19
NRe
Voko, 2021 [32] Observational (Retrospective Cohort) Hungary ≥18 years 3,740,066 24 10,956
  • Death due to COVID-19
None declared

Observational Case-Control Studies

References in this table:7811121415162024262728293031

Last name first author, Publication year Study design Country (or more detail, if needed) Age, central tendency or range Total population N vaccinated N unvaccinated Outcomes Funding source
Andrews, 2021a [7] Observational
(Test-Negative Case Control)
England ≥18 years 1,475,391 361,470 570,293
  • Symptomatic laboratory-confirmed COVID-19c
  • Hospitalization due to COVID-19
Government funding
Bajema, 2021 [8] Observational
(Test-Negative Case Control)
United States ≥18 years 1,175 388 787
  • Hospitalization due to COVID-19
Government funding
Bruxvoort, 2021 [9] Observational
(Test-Negative Case Control)
United States ≥18 years 8,153 141 705
  • Hospitalization due to COVID-19
Industry funding
Carazo, 2021a [11] Observational
(Test-Negative Case Control)
Canada 18–74 years 58,476 2,813 18,663
  • Symptomatic laboratory-confirmed COVID-19c
Other (Ministere de la sante’ et des services sociaux du Quebec)
Chemaitelly, 2021 [12] Observational
(Test-Negative Case Control)
Qatar ≥18 years NRe 32,527 32,527
  • Symptomatic laboratory-confirmed COVID-19c
  • Hospitalization due to COVID-19
  • Asymptomatic infection
NRe
Chung, 2021 [14] Observational
(Test-Negative Case Control)
Canada ≥18 years 324,033 51,226 252,083
  • Symptomatic laboratory-confirmed COVID-19c
University/Academic & Government funding
Embi, 2021 [15] Observational
(Test-Negative Case Control)
United States ≥18 years 89,217 10,210 41,791
  • Hospitalization due to COVID-19
Government funding
Grannis, 2021 [16] Observational
(Test-Negative Case Control)
United States ≥18 years 14,636 3,243 10,285
  • Symptomatic laboratory-confirmed COVID-19c
  • Hospitalization due to COVID-19
Government funding
Nasreena, [20] Observational (Test-Negative Case Control) Canada ≥18 years 682,071 1,859 450,284
  • Symptomatic laboratory-confirmed COVID-19c
  • Hospitalization due to COVID-19
Government funding
Pilishvili, 2021 [24] Observational (Test-Negative Case Control) United States ≥18 years 109,865 1,472 3,420
  •  Symptomatic laboratory-confirmed COVID-19c
Government funding
Self, 2021 [26] Observational (Test-Negative Case Control) United States ≥18 years 3,689 1,517 1,321
  • Hospitalization due to COVID-19
NRe
Skowronskia, 2021 [27] Observational (Test-Negative Case Control) British Columbia, Canada ≥18 years 380,532 1,403 118,510
  • Hospitalization due to COVID-19
Government funding
Skowronskia, 2021 [27] Observational (Test-Negative Case Control) Quebec, Canada ≥18 years 854,915 726 322,779
  • Hospitalization due to COVID-19
Government funding
Tang, 2021 [28] Observational (Test-Negative Case Control) Qatar ≥18 years 39,156 1,356 5,573
  • Symptomatic laboratory-confirmed COVID-19c
  • Hospitalization due to COVID-19
  • Asymptomatic infection
NRe
 Tenforde,  August 2021 [29] Observational (Test-Negative Case Control) United States ≥18 years 1,210 473 366
  • Hospitalization due to COVID-19
Government funding
Tenforde, November 2021 [30] Observational (Test-Negative Case Control) United States ≥18 years 4,513 1,701 1,247
  • Hospitalization due to COVID-19
Government funding
Thompson, 2021 [31] Observational (Test-Negative Case Control) United States ≥50 years 41,552 3,790 22,990
  • Symptomatic laboratory-confirmed COVID-19c
  • Hospitalization due to COVID-19
Government funding

Safety Surveillance

References in this table:32333436

Name of system Study design Country (or more detail, if needed) Age, central tendency or range Total population N vaccinated N unvaccinated Outcomes Funding source
Vaccine Adverse Event Reporting System (VAERS) [32,33] Passive surveillance United States ≥18 years (anaphylaxis);
18–49 years (myocarditis)
  • Serious Adverse Events
Government funding
Vaccine Safety Datalink (VSD) [34] Cohort United States ≥12 years (anaphylaxis);
18–39 years (myocarditis)
  • Serious Adverse Events
Government funding

aPre-print

bAdditional data provided by sponsor

cThis was a primary outcome of the RCT, defined as SARS-CoV-2 RT-PCR-positive symptomatic illness, in seronegative persons aged ≥18 years, ≥7 days post second dose. In a secondary analysis among seronegative and seropositive persons, the efficacy was Grade 3 or worse.

dGrade 3 local reactions include pain at injection site that prevents daily activity, redness > 10 cm, and swelling > 10 cm. Grade 3 systemic events include vomiting that requires IV hydration, diarrhea of 6 or more loose stools in 24 hours, or headache, fatigue/tiredness, chills, new or worsened muscle pain, or new or worsened joint pain that prevent daily routine activity.

eNot reported

Appendix 2. Databases and strategies used for systematic review

Database Strategy
clinicaltrials.gov Inclusion: Relevant Phase 1, 2, or 3 randomized controlled trials of Moderna COVID-19 vaccine
  • Involved human subjects
  • Reported primary data
  • Included adults (aged ≥18 years) at risk for SARS-CoV-2 infection
  • Included data relevant to the efficacy and safety outcomes being measured
  • Included data for the dosage and timing being recommended (100 µg, 2 doses at 0 and 28 days)

Additional resources: Unpublished and other relevant data by consulting with vaccine manufacturers and subject matter experts

International Vaccine Access Center (IVAC) Inclusion criteria for IVAC systematic review:
  • Published or preprint study with adequate scientific details
  • Includes groups with and without infection or disease outcome
  • Laboratory confirmed outcome
  • Vaccination status confirmed in ≥90%
  • Studies assess one vaccine or pooled mRNA vaccines
  • Includes participants who did or did not receive a COVID-19 vaccine

Vaccine effectiveness estimate calculated comparing vaccinated to unvaccinated**
Additional criteria for GRADE review:

  • Restricted to PICO-defined population, intervention, comparison, and outcomes
  • Outcomes assess 7 or 14 days after 2nd dose
  • Only Moderna vaccine (not mRNA vaccines as a group)
  • Included studies of general population and special populations (e.g., elderly, pregnant persons, healthcare workers)
Safety Surveillance Systems Evidence Retrieval for Observational Safety Studies:
  • Based on input from ACIP’s COVID-19 Vaccine Safety Technical (VaST) Work Group
  • Data on safety signals identified by vaccine surveillance systems
  • Data have been presented to ACIP

View the complete list of GRADE evidence tables‎

  1. Ahmed, F., et al., Methods for developing evidence-based recommendations by the Advisory Committee on Immunization Practices (ACIP) of the U.S. Centers for Disease Control and Prevention (CDC). Vaccine, 2011. 29(49): p. 9171-6.
  2. Anderson, E.J., et al., Safety and Immunogenicity of SARS-CoV-2 mRNA-1273 Vaccine in Older Adults. N Engl J Med, 2020. 383(25): p. 2427-2438.
  3. Jackson, L.A., et al., An mRNA Vaccine against SARS-CoV-2 – Preliminary Report. N Engl J Med, 2020. 383(20): p. 1920-1931.
  4. Chu, L., et al., A preliminary report of a randomized controlled phase 2 trial of the safety and immunogenicity of mRNA-1273 SARS-CoV-2 vaccine. Vaccine, 2021. 39(20): p. 2791-2799.
  5. El Sahly, H.M., et al., Efficacy of the mRNA-1273 SARS-CoV-2 Vaccine at Completion of Blinded Phase. N Engl J Med, 2021. 385(19): p. 1774-1785.
  6. Baden, L.R., et al., Efficacy and Safety of the mRNA-1273 SARS-CoV-2 Vaccine. N Engl J Med, 2021. 384(5): p. 403-416.
  7. Andrews, N.T., E.; Stowe, J.; Gower, C.; Kirsebom, F.; Simmons, R.; Gallagher, E.; Chand, M.; Brown, K.; Ladhani, S.N.; Ramsay, M.; Lopez Bernal, J., Vaccine effectiveness and duration of protection of Comirnaty, Vaxzevria and Spikevax against mild and severe COVID-19 in the UK. preprint, 2021.
  8. Bajema, K.L., et al., Effectiveness of COVID-19 mRNA Vaccines Against COVID-19-Associated Hospitalization – Five Veterans Affairs Medical Centers, United States, February 1-August 6, 2021. MMWR Morb Mortal Wkly Rep, 2021. 70(37): p. 1294-1299.
  9. Bruxvoort, K.J., et al., Effectiveness of mRNA-1273 against delta, mu, and other emerging variants of SARS-CoV-2: test negative case-control study. BMJ, 2021. 375: p. e068848.
  10. Bruxvoort, K.J., et al., Real-world effectiveness of the mRNA-1273 vaccine against COVID-19: Interim results from a prospective observational cohort study. Lancet Reg Health Am, 2021: p. 100134.
  11. Carazo, S., et al., Single-dose mRNA vaccine effectiveness against SARS-CoV-2 in healthcare workers extending 16 weeks post-vaccination: a test-negative design from Quebec, Canada. Clin Infect Dis, 2021.
  12. Chemaitelly, H., et al., mRNA-1273 COVID-19 vaccine effectiveness against the B.1.1.7 and B.1.351 variants and severe COVID-19 disease in Qatar. Nat Med, 2021. 27(9): p. 1614-1621.
  13. Chin, E.T., et al., Effectiveness of the mRNA-1273 Vaccine during a SARS-CoV-2 Delta Outbreak in a Prison. N Engl J Med, 2021. 385(24): p. 2300-2301.
  14. Chung, H., et al., Effectiveness of BNT162b2 and mRNA-1273 covid-19 vaccines against symptomatic SARS-CoV-2 infection and severe covid-19 outcomes in Ontario, Canada: test negative design study. BMJ, 2021. 374: p. n1943.
  15. Embi, P.J., et al., Effectiveness of 2-Dose Vaccination with mRNA COVID-19 Vaccines Against COVID-19-Associated Hospitalizations Among Immunocompromised Adults – Nine States, January-September 2021. MMWR Morb Mortal Wkly Rep, 2021. 70(44): p. 1553-1559.
  16. Grannis, S.J., et al., Interim Estimates of COVID-19 Vaccine Effectiveness Against COVID-19-Associated Emergency Department or Urgent Care Clinic Encounters and Hospitalizations Among Adults During SARS-CoV-2 B.1.617.2 (Delta) Variant Predominance – Nine States, June-August 2021. MMWR Morb Mortal Wkly Rep, 2021. 70(37): p. 1291-1293.
  17. Irizarry, R.A.R.-F., M. M.; Nieves, E.G.; Cardona-Gerena, I., Time-Varying Effectiveness of Three COVID-19 Vaccines in Puerto Rico. SSRN, 2021.
  18. Lin, D.Y., et al., Effectiveness of Covid-19 Vaccines over a 9-Month Period in North Carolina. N Engl J Med, 2022.
  19. Navarre, Spain, April to August 2021. Euro Surveill, 2021. 26(39).
  20. Nasreen, S.C., H.; He, S.; Brown, K. A.; Gubbay, J.B.; Buchan, S.A.; Fell, D.; Austin, P.C.; Schwartz, K.L.; Sundaram, M.E.; Calzavara, A.; Chen, B.; Tadrous, M.; Wilson, K.; Wilson, S.E.; Kwong, J.C., Effectiveness of mRNA and ChAdOx1 COVID-19 vaccines against symptomatic SARS-CoV-2 infection and severe outcomes with variants of concern in Ontario. medRxiv, 2021.
  21. Nordström, P., M. Ballin, and A. Nordström, Effectiveness of heterologous ChAdOx1 nCoV-19 and mRNA prime-boost vaccination against symptomatic Covid-19 infection in Sweden: A nationwide cohort study. Lancet Reg Health Eur, 2021. 11: p. 100249.
  22. Nordström, P.B., M.; Nordström, A., Effectiveness of Covid-19 Vaccination Against Risk of Symptomatic Infection, Hospitalization, and Death Up to 9 Months: A Swedish Total-Population Cohort Study. SSRN, 2021.
  23. Pawlowski, C., et al., FDA-authorized mRNA COVID-19 vaccines are effective per real-world evidence synthesized across a multi-state health system. Med (N Y), 2021. 2(8): p. 979-992 e8.
  24. Pilishvili, T., et al., Effectiveness of mRNA Covid-19 Vaccine among U.S. Health Care Personnel. N Engl J Med, 2021. 385(25): p. e90.
  25. Puranik, A.L., P.J.; Silvert, E.; Niesen, M.; Corchado-Garcia, J.; O'Horo, J.C.; Virk, A.; Swift, M.D.; Halamka, J.; Badley, A.D; Venkatakrishnan, A.J.; Soundararajan, V., Comparison of two highly-effective mRNA vaccines for COVID-19 during periods of Alpha and Delta variant prevalence. med Rxiv, 2021.
  26. Self, W.H., et al., Comparative Effectiveness of Moderna, Pfizer-BioNTech, and Janssen (Johnson & Johnson) Vaccines in Preventing COVID-19 Hospitalizations Among Adults Without Immunocompromising Conditions – United States, March-August 2021. MMWR Morb Mortal Wkly Rep, 2021. 70(38): p. 1337-1343.
  27. Skowronski, D.M., et al., Two-dose SARS-CoV-2 vaccine effectiveness with mixed schedules and extended dosing intervals: test-negative design studies from British Columbia and Quebec, Canada. medRxiv, 2021: p. 2021.10.26.21265397.
  28. Tang, P., et al., BNT162b2 and mRNA-1273 COVID-19 vaccine effectiveness against the SARS-CoV-2 Delta variant in Qatar. Nat Med, 2021. 27(12): p. 2136-2143.
  29. Tenforde, M.W., et al., Effectiveness of SARS-CoV-2 mRNA Vaccines for Preventing Covid-19 Hospitalizations in the United States. Clin Infect Dis, 2021.
  30. Tenforde, M.W., et al., Association Between mRNA Vaccination and COVID-19 Hospitalization and Disease Severity. JAMA, 2021. 326(20): p. 2043-2054.
  31. Thompson, M.G., et al., Effectiveness of Covid-19 Vaccines in Ambulatory and Inpatient Care Settings. N Engl J Med, 2021. 385(15): p. 1355-1371.
  32. Voko, Z., et al., Nationwide effectiveness of five SARS-CoV-2 vaccines in Hungary-the HUN-VE study. Clin Microbiol Infect, 2021.
  33. Klein, N.P., et al., Surveillance for Adverse Events After COVID-19 mRNA Vaccination. JAMA, 2021. 326(14): p. 1390-1399.
  34. Shimabukuro, T.T., Updates on myocarditis and pericarditis following Moderna COVID-19 vaccination. Presentation to ACIP, 2022.
  35. U.S. Food and Drug Administration, January 31, 2022 Approval Letter – Spikevax. 2022.
  36. International Vaccine Access Center (IVAC). VIEW-hub. 2021; Available from: www.view-hub.org.