Grading of Recommendations, Assessment, Development, and Evaluation (GRADE): Pfizer-BioNTech COVID-19 Vaccine

About

CDC vaccine recommendations are developed using an explicit evidence-based method based on the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach.

Overview

A Grading of Recommendations, Assessment, Development and Evaluation (GRADE) review of the evidence for benefits and harms for Pfizer-BioNTech COVID-19 vaccine was presented to the Advisory Committee for Immunization Practices (ACIP) on August 30, 2021. GRADE evidence type indicates the certainty of estimates from the available body of evidence. Evidence certainty ranges from type 1 (high certainty) to type 4 (very low certainty).1

The policy question was, "Should vaccination with Pfizer-BioNTech COVID-19 vaccine (2-doses, IM) be recommended for persons 16 years of age and older?" The potential benefits pre-specified by the ACIP COVID-19 Vaccines Work Group included prevention of symptomatic laboratory-confirmed COVID-19 (critical), hospitalization due to COVID-19 (critical), death due to COVID-19 (important) and asymptomatic SARS-CoV-2 infection, assessed using PCR (important). The two pre-specified harms were serious adverse events (SAEs) (including myocarditis and anaphylaxis) (critical) and reactogenicity (severe, grade ≥3) (important).

A systematic review of evidence on the benefits and harms of a two-dose regimen of Pfizer-BioNTech COVID-19 vaccine among persons aged ≥16 years was conducted, based on data available as of August 23, 2021. The evidence from one Phase I randomized controlled trial (RCT),2 one Phase II/III RCT,345 26 vaccine effectiveness studies,678910111213141516171819202122232425262728293031 and two vaccine safety surveillance systems323334 were assessed using a modified GRADE approach.1 Pooled efficacy and effectiveness estimates were calculated when multiple sources had data on an outcome.

In terms of benefits, the available data from RCTs demonstrated that, compared with placebo, vaccination was associated with a lower risk of symptomatic laboratory-confirmed COVID-19 (relative risk [RR] 0.09, 95% confidence interval [CI] 0.07–0.11; evidence type 1), hospitalization due to COVID-19 (RR 0.02; 95% CI 0.00–0.12; evidence type 2), and death due to COVID-19 (RR 0.17, 95% CI 0.02–1.39; evidence type 2). The certainty of estimates regarding hospitalization and death due to COVID-19 was reduced due to imprecision.

The pooled vaccine effectiveness estimates from observational studies were consistent with these findings. Compared with no vaccination, vaccination with Pfizer-BioNTech COVID-19 vaccine was associated with a decreased risk of symptomatic laboratory-confirmed COVID-19 (RR 0.07, 95% CI 0.05–0.13; evidence type 2), hospitalization (RR 0.06, 95% CI 0.03–0.12; evidence type 2), and death due to COVID-19 (RR 0.04, 95% CI 0.02–0.09; evidence type 2). The certainty of each of these estimates was increased for a strong association. Vaccination was also associated with a decreased risk of asymptomatic SARS-CoV-2 infection (RR 0.11, 95% CI 0.10–0.12; evidence type 4); the evidence certainty type was downgraded for inconsistency.

In terms of harms, the available data from RCTs indicated that serious adverse events were balanced between the vaccine and placebo arms (RR 1.00; 95% CI 0.85 to 1.18, evidence type 2), and two serious adverse events were judged to be related to vaccination among more than 22,000 persons vaccinated. The certainty of this estimate was reduced due to imprecision. Reactogenicity grade ≥3 was associated with vaccination (RR 4.69; 95% CI 3.83–5.73, evidence type 1). About 11% of vaccine recipients versus 2% of placebo recipients reported grade ≥3 reactions. Two rare but serious adverse events, anaphylaxis and myocarditis, have been associated with vaccination in post-authorization safety surveillance (see results section and Table 3e).

Introduction

On August 23, 2021, the U.S. Food and Drug Administration (FDA) approved the Biologics License Application (BLA) for Pfizer-BioNTech COVID-19 Vaccine (COMIRNATY®) for the prevention of COVID-19 in individuals aged ≥16 years.35 As part of the process employed by the Advisory Committee for Immunization Practices (ACIP), a systematic review and Grading of Recommendations, Assessment, Development and Evaluation (GRADE) assessment of the evidence for Pfizer-BioNTech COVID-19 vaccine was conducted and presented to ACIP.1 There were no conflicts of interest reported by CDC and ACIP COVID-19 Vaccines Work Group members involved in the GRADE analysis.

ACIP adopted a modified GRADE approach in 2010 as the framework for evaluating the scientific evidence that informs recommendations for vaccine use. ACIP has made modifications to the GRADE approach by presenting assessed evidence as type 1, 2, 3, and 4, which corresponds to high, moderate, low, and very low certainty, whereas standard GRADE has high as level 4 and very low as level 1. Additionally, instead of presenting the overall certainty of evidence across all outcomes, ACIP presents the certainty of evidence for the benefits and harms separately. ACIP includes an option "ACIP recommends the intervention for individuals based on shared clinical decision-making" instead of providing a conditional recommendation for or against an intervention. GRADE was used to evaluate the efficacy and safety of a two-dose regimen of Pfizer-BioNTech COVID-19 vaccine among persons aged ≥16 years. Evidence of benefits and harms were reviewed based on the modified GRADE approach.1

The policy question was, "Should vaccination with Pfizer-BioNTech COVID-19 vaccine (2-doses, IM) be recommended for persons 16 years of age and older?" (Table 1).

Methods

We conducted a systematic review of evidence on the benefits and harms of a two-dose regimen of Pfizer-BioNTech COVID-19 vaccine (see Appendix 2 for databases and search strategies). We assessed outcomes and evaluated the quality of evidence using the GRADE approach. Patient-important outcomes (including benefits and harms) for assessment were selected by the Work Group during Work Group calls and via online surveys where members were asked to rate and rank the importance of relevant outcomes.

We identified RCTs through clinicaltrials.gov. Relevant Phase I, II, or III RCTs of COVID-19 vaccine were included if they: 1) involved human subjects; 2) reported primary data; 3) included adults (aged ≥16 years) at risk for SARS-CoV-2 infection; 4) included data relevant to the efficacy and safety outcomes being measured; 5) included data for the dosage being recommended (30 μg, 2 doses at 0 and 21 days). We identified relevant observational studies through an ongoing systematic review conducted by the International Vaccine Access Center (IVAC) and the World Health Organization (WHO).36 Relevant observational studies, using case-control, test-negative, or cohort designs, were restricted to the defined population, intervention, comparison, and outcome outlined in the policy question. Outcomes were assessed starting at least 7 days after 2nd dose. We included only 2-dose Pfizer-BioNTech vaccine effectiveness estimates, with combined mRNA vaccine effectiveness estimates excluded. We included studies of general populations and special populations. In addition, efforts were made to obtain unpublished and other relevant data by hand-searching reference lists, and consulting with vaccine manufacturers and subject matter experts. We included observational safety data from two vaccine safety surveillance systems based on input from ACIP's COVID-19 Vaccines Safety Technical (VaST) Work Group: Vaccine Safety Datalink (VSD) and Vaccine Adverse Event Reporting System (VAERS). Characteristics of all included studies and surveillance systems are shown in Appendix 1.2345678910111213141516171819202122232425262728293031323334

Two reviewers evaluated all studies for study limitations (risk of bias) using the Cochrane Risk of Bias (RoB) tool for RCTs and the Newcastle-Ottawa Scale (NOS) for observational studies. RoB comprises a series of questions structured into domains focusing on different aspects of trial design, conduct, and reporting. Based on question responses, judgement can be "low", "moderate", or "high" risk of bias. NOS is a 9-point scale which assesses study limitations related to participant selection and comparability, and assessment of outcome (cohort studies) or ascertainment of exposure (case-control studies). Studies with NOS scores <7 were considered to have serious study limitations.

From the RCT data, relative risks (RR) were calculated from numerators and denominators available in the body of evidence. Vaccine efficacy estimates were defined as 100% x (1-RR). Vaccine effectiveness estimates and 95% CIs were taken from the published/preprint studies, as defined by the authors using a variety of study designs and analytical approaches; adjusted estimates were used when available. When multiple studies were available, pooled estimates were calculated using random effects (>3 studies) or fixed effects (≤2 studies) meta-analysis (R meta package). When multiple studies provided estimates based on overlapping study populations, the study with the most comprehensive population and follow-up time was selected for inclusion in the pooled estimate. Because there was a relatively large body of evidence from vaccine effectiveness studies, with many available only in the preprint literature, an a priori decision was made to exclude studies judged to have serious study limitations from the main pooled estimate used for GRADE. Sensitivity analyses were performed to assess the influence of study characteristics (e.g., special populations vs. full population, preprint vs. peer-reviewed, standard vs. extended dosing interval, cohort vs. case-control/test-negative study design, study limitations, and circulating variants). The evidence certainty assessment for randomized and observational studies addressed risk of bias, inconsistency, indirectness, imprecision, and other characteristics. The GRADE assessment across the body of evidence for each outcome was presented in an evidence profile.

Results

The results of the GRADE assessment were presented to ACIP on August 30, 2021.

Outcomes of interest included individual benefits and harms. Indirect effects of vaccination (e.g., societal benefits) were not considered as part of GRADE. Benefits of interest deemed critical were prevention of symptomatic laboratory-confirmed COVID-19 and prevention of hospitalization due to COVID-19 (Table 2). Other important benefits included prevention of death due to COVID-19 and prevention of asymptomatic SARS-CoV-2 infection. The critical harm of interest was serious adverse events (SAEs), including myocarditis and anaphylaxis; reactogenicity grade ≥3 was deemed an important harm.

After screening 86 records, 45 were excluded from full-text review because they were a different study design (i.e. screening method, n=2), a different intervention (e.g., a different vaccine or a different dose, n=28), or a different outcome that did not directly align with the PICO outcomes (e.g., any infection instead of symptomatic COVID-19 or asymptomatic infection, n=15). Of the 41 records that were deemed eligible for full-text review, 1 was excluded for not having primary data, 4 were excluded because they assessed a different intervention, and 4 were excluded because they assessed a different outcome. The remaining 33 records, which reported data on 29 studies or surveillance systems, were included in the evidence synthesis and GRADE evidence assessment (Appendix 1).2345678910111213141516171819202122232425262728293031323334 Data were reviewed from four RCT records, including one publication from a Phase I trial, one publication and one preprint from the same Phase II/III trial, and additional data provided by the sponsor.2345 Data were reviewed from 26 vaccine effectiveness studies.678910111213141516171819202122232425262728293031 Two vaccine safety surveillance systems, VSD and VAERS, included data for SAEs.323334

In the Phase II/III RCT, using data on all blinded follow-up (up to 6 months or the unblinding date of March 13, 2021), the Pfizer-BioNTech COVID-19 vaccine reduced symptomatic COVID-19 when compared to placebo (vaccine efficacy: 91.1% (95% CI 88.8–93.1%)) (Table 3a). For hospitalization due to COVID-19, 31 events occurred, all in the placebo group. Vaccine efficacy against hospitalization due to COVID-19 was 100% (95% CI 87.6–100%) (Table 3b). Deaths due to COVID-19 were uncommon, one in the vaccine group and six in the placebo group (83% (-39–98%)) (Table 3c). Numbers of SAEs were comparable between the vaccine group and the placebo group across the two RCTs (Phase II/III: 268/21,926 (1.2%) vs. 268/21,921 (1.2%); Phase I: 1/24 (4.2%) vs. 0/6 (0.0%)); there were no cases of vaccine-associated enhanced disease or vaccine-related deaths (Table 3e). Grade ≥3 reactions generally were not uncommon and occurred more frequently in the vaccine than placebo groups (Table 3f).

Seventeen vaccine effectiveness studies reported data on symptomatic laboratory-confirmed COVID-19 (Table 3a), 13 reported data on hospitalization due to COVID-19 (Table 3b), 6 reported data on death due to COVID-19 (Table 3c), and 5 reported data on asymptomatic SARS-CoV-2 infection (Table 3d). The pooled vaccine effectiveness estimates from the observational studies demonstrated that the Pfizer-BioNTech COVID-19 vaccine reduced symptomatic COVID-19 when compared to no vaccination (pooled vaccine effectiveness: 92.4% (95% CI: 87.5–95.3%), based on 8 studies). 610111417182131The pooled vaccine effectiveness against hospitalization due to COVID-19 was 94.3% (95% CI 87.9–97.3%), based on 8 studies. 1315172122252830The pooled vaccine effectiveness for prevention of death due to COVID-19 was 96.1% (95%CI 91.5–98.2%), based on 4 studies.13151725 The pooled vaccine effectiveness against asymptomatic SARS-CoV-2 infection was 89.3% (95% CI 88.4–90.1%), based on 2 studies.1724

Observational data on serious adverse events were reviewed. A rapid cycle analysis from VSD evaluated chart-reviewed cases of myocarditis occurring among persons aged 18–39 years following dose 2 of the Pfizer-BioNTech COVID-19 vaccine (Table 3e).34 The rates of myocarditis were 368 per million person-years (9/24,432) in the 0–7-day risk interval and 48 per million person-years (3/62,481) in vaccinated comparators (adjusted rate ratio: 9.1 (95%CI 2.1–48.6)). Data from VAERS showed an elevated ratio of observed to expected myocarditis cases in the 7-day interval following vaccination among females in age groups 16–24 years and among males in age groups 16–49 years, with higher observed/expected ratios in males than females.33 A rapid cycle analysis of data from VSD evaluated chart-reviewed cases of anaphylaxis among all vaccinated persons aged ≥12 years. Based on events occurring in a 0–1 day risk interval after vaccination, the estimated incidence of confirmed anaphylaxis was 5.0 (95% CI 3.5–6.9) per million doses.34 The absolute reporting rate to VAERS was 4.7 per million doses administered.32

GRADE Summary

The initial GRADE evidence level was type 1 (high) for randomized controlled trials and type 3 (low) for the observational data (Table 4). In terms of benefits, the RCT data indicate that the vaccine reduces the risk of symptomatic laboratory-confirmed COVID-19, and no serious concerns impacting certainty were identified for this outcome (type 1, high). Observational data for symptomatic laboratory-confirmed COVID-19 indicated a similar risk reduction with vaccination, and the certainty was upgraded one point for a strong association (type 2, moderate). The certainty of the evidence from RCTs for hospitalization due to COVID-19 was downgraded one point for serious concern of imprecision (type 2, moderate). Observational data for hospitalization due to COVID-19 indicated a similar risk reduction with vaccination, and the certainty was upgraded one point for a strong association (type 2, moderate). The certainty of the evidence for death due to COVID-19 was downgraded one point for serious concern of imprecision (type 2, moderate). Observational data for death due to COVID-19 concurred with a strong risk reduction with vaccination, and the certainty was upgraded one point for a strong association (type 2, moderate). The body of evidence for prevention of asymptomatic SARS-CoV-2 infection came from observational studies and was downgraded one point for serious concern for inconsistency (type 4, very low). The certainty of evidence for serious adverse events was downgraded one point for serious concern of imprecision related to sample size (type 2, moderate). Observational data on specific serious adverse events (i.e., myocarditis among persons aged 18–39 years and anaphylaxis among persons aged 12 years and older) demonstrated these events are rare (evidence type 3, low). No serious concerns impacted the certainty of estimates of reactogenicity from RCTs (type 1, high).

The summary of evidence types is shown in Table 5. The final evidence types were type 1 for symptomatic laboratory-confirmed COVID-19, type 2 for hospitalization due to COVID-19 and death due to COVID-19, type 4 for asymptomatic SARS-CoV-2 infection, type 2 for serious adverse events, and type 1 for reactogenicity.

Table 1: Policy Question and PICO

Table 1: Policy Question and PICO
Policy question: Should vaccination with Pfizer-BioNTech COVID-19 vaccine (2-doses, IM) be recommended for persons 16 years of age and older?
Population Persons aged ≥16 years
Intervention Pfizer-BioNTech COVID-19 vaccine BioNTech vaccine BNT162b2

(30 µg, 2 doses IM, 21 days apart)
Comparison No vaccine
Outcomes Symptomatic laboratory-confirmed COVID-19

Hospitalization due to COVID-19

Death due to COVID-19

Asymptomatic SARS-CoV-2 infection

Serious Adverse Events (SAEs) (including myocarditis and anaphylaxis)

Reactogenicity (proportion with grade 3 or worse reactions)

Abbreviations: IM = intramuscular.

Table 2: Outcomes and Rankings

Table 2: Outcomes and Rankings
Outcome Importancea Included in evidence profile
Symptomatic laboratory-confirmed COVID-19 Critical Yes
Hospitalization due to COVID-19 Critical Yes
Death due to COVID-19 Important Yes
Asymptomatic SARS-CoV-2 infection Important Yes
Serious Adverse Events (SAEs) (including myocarditis and anaphylaxis) Critical Yes
Reactogenicity (proportion with grade 3 or worse reactions) Important Yes

a Three options: 1. Critical; 2. Important but not critical; 3. Not important for decision making

Table 3a: Summary of Studies Reporting Symptomatic Laboratory-confirmed COVID-19

References in this table: 4567891011121417181920212224262931

Table 3a: Summary of Studies Reporting Symptomatic Laboratory-confirmed COVID-19
Authors last name, pub year Design, study population No. of patients vaccinated or No. of cases No. of patients unvaccinated or No. of controls Comparator Vaccine Efficacy/Effectiveness (95% CI) Study limitations (Risk of Bias)
Polack 2020, Thomas 2021a [4,5] b, c RCT; Age ≥16 years 77 cases/19,711 vaccine recipients 833 cases/19,741 placebo recipients Placebo 91.1 (88.8–93.1) Not serious
Alalia [6] Observational (retrospective cohort); Healthcare workers; Kuwait 12 cases /90,015 person-days among vaccinated 114 cases /90,367 person-days among unvaccinated No vaccine 94.5 (89.4–97.2)d Not serious
Angel, 2021[7] Observational (retrospective cohort); Healthcare workers; Israel 8 cases/5,372 vaccinated 38 cases/696 unvaccinated No vaccine 97 (94–99)e Not serious
Balicera [8] Observational (prospective cohort); Pregnant women; Israel 67 cases/10,861 vaccinated 144 cases/10,861 unvaccinated No vaccine 97 (91–100)e Not serious
Carazoa [10] Observational (test-negative design); Healthcare workers; Canada 20 vaccinated / 2,813 unvaccinated cases 1,954 vaccinated / 18,663 unvaccinated controls No vaccine 92.2 (87.8–95.1)d Not serious
Chung, 2021 [11] Observational (test-negative design); General population ≥16 years; Canada 51 vaccinated/ 51,271 cases 3,275 vaccinated/ 254,816 controls No vaccine 91 (88–93)d Not serious
Dagan, 2021 [12]
Barda, 2021 [9]		 Observational (retrospective cohort); General population ≥16 years; Israel 2,389 cases/ 596,618 vaccinated 3,607 cases/ 596,618 unvaccinated No vaccine 94 (87–98)e
Updated:
96 (94–97)e		 Not serious
Haas, 2021 [17] Observational (retrospective cohort); General population ≥16 years; Israel 1,692 cases/201,882,183 person-days among vaccinated 39,065 cases /120,076,136 person-days among unvaccinated No vaccine 97.0 (96.7–97.2)d Not serious
Fabiani, 2021 [14] Observational (retrospective cohort); Healthcare workers; Italy 2 cases/216,098 person-days among vaccinated 13 cases /77,073 person-days among unvaccinated No vaccine 93.7 (50.8–99.2)d Not serious
Kissling, 2021 [18] Observational (test-negative design); Symptomatic adults ≥65 years; Europe (England, France, Ireland, the Netherlands, Portugal, Scotland, Spain, and Sweden) 14 vaccinated/519 cases 512 vaccinated/2,857 controls No vaccine 87 (74–93)d Not serious
Lopez Bernal, 2021 [19] Observational (test-negative design); Population >80 years; England 41 vaccinated/8,988 cases 634 vaccinated/15,718 controls No vaccine 85 (79–89)e Not serious
Lopez Bernal, 2021 [20] Observational (test-negative design); General population ≥16 years; England 49 alpha cases / 15,749 vaccinated controls

122 delta cases /15,749 vaccinated controls

 7,313 alpha cases/96,371 unvaccinated controls

4,043 delta cases/96,371 unvaccinated controls

 No vaccine Alpha variant: 93.7 (91.6–95.3)e

Delta variant: 88.0 (85.3–90.1)e

 Not serious
Martínez-Baz, 2021 [21] Observational (prospective cohort); ≥18 years with close contact of lab-confirmed COVID-19 case; Spain 25 cases/491 vaccinated contacts 5,306 cases/19,580 unvaccinated contacts No vaccine 82 (73–88)d Not serious
Nasreena [22] Observational (test-negative design); General population ≥16 years (symptomatic) (Vaccinated/cases)

Non-VOC: 18/28,705

Alpha: 92/36,832

Beta/gamma: 9/ 3,005

Delta: 6/991

 (Vaccinated/controls)

6,914/351,540

 No vaccine Non-variant of concern: 93 (88–96)e
Alpha variant: 89 (86–91)e
Beta/gamma variant: 84 (69–92)e
Delta variant: 87 (64–95)e		 Not serious
Pouwelsa [24] Observational (longitudinal household survey); Aged ≥18 years; United Kingdom Not reported Not reported No vaccine Alpha-dominant period: 97 (96–98)e

Delta-dominant period: 84 (82–86)e

 Not serious
Regev-Yochay, 2021 [26] Observational (prospective cohort); Healthcare workers; Israel 19 cases/ 329,071 person-days 115 cases/
199,126 person-days		 No vaccine 90 (84–94)e Not serious
Tanga [29] Observational (matched case control); Persons with PCR+ SARS-CoV-2 delta variant infections; Qatar 98 vaccinated/571 cases 183 vaccinated/571 controls No vaccine 56.1 (41.4–67.2)f Serious (selection, comparability)
Whitakera [31] Observational (prospective cohort); General population ≥16 years with medically attended COVID-19; England 8 cases/
12,273.3 person-years among vaccinated		 4,228 cases/
1,460,811.4 person-years among unvaccinated		 No vaccine 93.3 (85.8–96.8)d Not serious

a Pre-print

b Assessed using a primary outcome of the RCT, defined as SARS-CoV-2 RT-PCR-positive symptomatic illness, in seronegative adults, ≥7 days post second dose. Seronegative status was not a criterion for inclusion of observational studies.

c Additional data provided by sponsor

d Vaccine effectiveness estimate included in main pooled analysis used for GRADE.

e Vaccine effectiveness estimate not included in main pooled analysis used for GRADE because study population overlapped with another study that was included.

f Vaccine effectiveness estimate not included in main pooled analysis used for GRADE because of study limitations related to selection and comparability.

Table 3b: Summary of Studies Reporting Hospitalization due to COVID-19

References in this table: 34589121315161721222325272830

Table 3b: Summary of Studies Reporting Hospitalization due to COVID-19
Authors last name, pub year Design, study population No. of patients vaccinated or No. of cases No. of patients unvaccinated or No. of controls Comparator Vaccine Efficacy/Effectiveness, % (95% CI) Study limitations (Risk of Bias)
Polack 2020, Thomas 2021a [3,4,5]b Age ≥16 years 0 hospitalizations /19,687 vaccine recipients 31 hospitalizations /19,708 placebo recipients Placebo 100 (87.6–100) Not serious
Balicera [8] Pregnant women; Israel 11 hospitalizations /10,861 vaccinated 25 hospitalizations/
10,861 unvaccinated		 No vaccine 89 (43–100)e Not serious
Dagan, 2021 [12]
Barda, 2021 [9]		 Observational (retrospective cohort); General population ≥16 years; Israel 110 hospitalizations /596,618 vaccinated 259 hospitalizations /596,618 unvaccinated No vaccine 87 (55–100)e
Updated:
92 (85–97)e		 Not serious
Emborga [13] Observational (retrospective cohort); Groups prioritized for vaccination; Denmark 24 hospitalizations /37,429.7 person-years among vaccinated 1,014 hospitalizations /152,171.4 person-years among unvaccinated No vaccine 93 (89–96) d Not serious
Haas, 2021 [17] Observational (retrospective cohort); General population ≥16 years; Israel 596 hospitalizations /201,882,183 person-days among vaccinated 5,526 hospitalizations
/120,076,136 person-days among unvaccinated		 No vaccine 97.2 (96.8– 97.5)d Not serious
Flacco, 2021 [15] Observational (retrospective cohort); General population ≥18 years; Italy Not reported/30,817 vaccinated Not reported/174,023 unvaccinated No vaccine 99 (96–100)d Not serious
Golberga [16] Observational (prospective cohort); General population ≥16 years; Israel 493 hospitalizations /136.8M person-days among vaccinated 10,057 hospitalizations /288.5M person-days among unvaccinated No vaccine 94.2 (93.6–94.7)e Not serious
Martínez-Baz, 2021 [21] Observational (prospective cohort); ≥18 years with close contact of lab-confirmed COVID-19 case; Spain 1 hospitalization /491 vaccinated contacts 548 hospitalizations /19,580 unvaccinated contacts No vaccine 94 (60–99)d Not serious
Nasreena [22] Observational (test-negative design); General population ≥16 years (symptomatic) (Vaccinated/hospitalized cases)c
Non-VOC ≤5/6,327
Alpha 26/6,896
Beta/gamma ≤5/780
Delta ≤5/165		 (Vaccinated/hospitalized SARS-CoV-2 negative controls)c
6,910/351,240		 No vaccine nonVOC: 96 (82–99)f
Alpha 95 (92–97)f
Beta/Gamma 95 (81–99)d
Delta: -		 Not serious
Pawlowski, 2021 [23] Observational (retrospective cohort); ≥18 years with access to Mayo Health system; United States 6 hospitalizations/
1,671,628 person-days among vaccinated		 49 hospitalizations/
1,599,076 person-days among unvaccinated		 No vaccine 88.3 (72.6–95. 9)e Not serious
Puranika [25] Observational (retrospective cohort); ≥18 years with access to Mayo Health system (MN); United States 11 hospitalizations /2,333,145 person-days among vaccinated 82 hospitalizations/2,532,948 person-days among unvaccinated No vaccine 85 (73–93)d Not serious
Saciuka [27] Observational (retrospective cohort; Active members of a large HMO ≥16 years; Israel 105 hospitalizations/
1,353,847 vaccinated		 942 hospitalizations/
1,162,033 unvaccinated		 No vaccine 94.4 (93.2–95.5)d Not serious
Stowea [28] Observational (test negative design); General population ≥16 years; England Not reported Not reported No vaccine Alpha: 95 (78–99)f

Delta: 96 (86–99)d

 Not serious
Tenforde, 2021 [30] Observational (case control); Hospitalized adults ≥18 years; United States 95 vaccinated/1,194 hospitalized cases 571 vaccinated/1,895 hospitalized controls No vaccine 84.4 (74.9–90.4)d Not serious

a Pre-print.

b Additional data provided by study sponsor.

c Outcome defined as hospitalization or death.

d Vaccine effectiveness estimate included in main pooled analysis used for GRADE.

e Vaccine effectiveness estimate not included in main pooled analysis used for GRADE because study population overlapped with another study that was included.

f Vaccine effectiveness estimate not included in main pooled analysis used for GRADE; a different variant-specific estimate from the same study was included.

Table 3c: Summary of Studies Reporting Death due to COVID-19

References in this table: 345131516172527

Table 3c: Summary of Studies Reporting Death due to COVID-19
Authors last name, pub year Design, study population No. of patients vaccinated No. of patients unvaccinated Comparator Vaccine Efficacy/Effectiveness, % (95% CI) Study limitations (Risk of Bias)
Polack 2020, Thomas 2021a [3,4,5]b Age ≥16 years 1 death/19,687 vaccine recipients 6 deaths/19,708 placebo recipients Placebo 83 (-39–98) Not serious
Emborga [13] Observational (retrospective cohort); Groups prioritized for vaccination; Denmark 25 deaths/37631.7 person-years among vaccinated 445 deaths/153,179.6 person-years among unvaccinated No vaccine All priority groups: 94 (90–96)c Not serious
Flacco, 2021 [15] Observational (retrospective cohort); General population ≥18 years; Italy Not reported/30,817 vaccinated Not reported/174,023 unvaccinated No vaccine 98 (87–100)c Not serious
Goldberga [16] Observational (prospective cohort; General population ≥16 years; Israel 136 deaths/136.8M person-days among vaccinated 1749 deaths/288.5 person-days among unvaccinated No vaccine 93.7 (92.5–94.7)d Not serious
Haas, 2021 [17] Observational (retrospective cohort);
General population ≥16 years; Israel		 138 deaths /201,882,183 person-days among vaccinated 715 deaths/120,076,136
person-days among unvaccinated		 No vaccine Adjusted: 96.7 (96.0 – 97.3)c Not serious
Puranika [25] Observational (retrospective cohort); ≥18 years with access to Mayo Health system (MN); United States 0 deaths/2,333,860 person-days among vaccinated 4 deaths/ 2,537,030 person-days among unvaccinated No vaccine 100 (–60–100)c Not serious
Saciuka [27] Observational (retrospective cohort); Active members of a large HMO ≥16 years; Israel 33 deaths/1,354,444 vaccinated 131 deaths/1,166,487 unvaccinated No vaccine 84 (76.6–89.1)d Not serious

a Pre-print.

b Additional data provided by sponsor.

c Vaccine effectiveness estimate included in main pooled analysis used for GRADE.

d Vaccine effectiveness estimate not included in main pooled analysis used for GRADE because study population overlapped with another study that was included.

Table 3d: Summary of Studies Reporting Asymptomatic SARS-CoV-2 infection

References in this table: 717242629

Table 3d: Summary of Studies Reporting Asymptomatic SARS-CoV-2 infection (assessed using PCR)
Authors last name, pub year Design, study population No. of patients vaccinated or No. of cases No. of patients unvaccinated or No. of controls Comparator Vaccine Efficacy/Effectiveness, % (95% CI) Study limitations (Risk of Bias)
Angel, 2021 [7] Observational (retrospective cohort) Healthcare workers; Israel 19 cases/5,372 vaccinated 17 cases/696 unvaccinated No vaccine 86 (69–93)b Not serious
Haas, 2021 [17] Observational (retrospective cohort)
General population ≥16 years; Israel		 3,632 cases/201,882,183 person-days among vaccinated 49,138 cases/120,076,136 person-days among unvaccinated No vaccine 91.5 (90.7–92.2)c Not serious
Pouwelsa [24] Observational (longitudinal household survey) Household survey participants ≥18 years; United Kingdom Not reported Not reported No vaccine Delta period:
74 (69–78%)c		 Not serious
Regev-Yochay, 2021 [26] Observational (prospective cohort) Healthcare workers; Israel 12 cases/1,300 exposure events among vaccinated 48 cases/1,441
exposure events among unvaccinated		 No vaccine 72 (48–86)b,d  Serious (selection, comparability)
Tanga [29] Observational (case-control) Persons with PCR+ SARS-CoV-2 delta variant infections; Qatar 73 vaccinated/757 cases 108 vaccinated/757 controls No vaccine 35.9 (11.1–53.9)d Serious (selection, comparability)

a Pre-print.

b Vaccine effectiveness estimate not included in main pooled analysis used for GRADE because study population overlapped with another study that was included.

c Vaccine effectiveness estimate included in main pooled analysis used for GRADE.

d Vaccine effectiveness estimate not included in main pooled analysis used for GRADE because of study limitations related to selection and comparability.

Table 3e: Summary of Studies Reporting Serious Adverse Eventsa

References in this table: 2345323334

Table 3e: Summary of Studies Reporting Serious Adverse Events
Authors last name, pub year Age or other characteristics of importance n/N (%) intervention n/N (%) comparison Comparator RR (95% CI) Study limitations (Risk or Bias)
Walsh, 2020a,b [2] Age ≥16 years 1/24 (4.2%)c 0/6 (0%) Placebo 0.84 (0.03, 18.44) Not serious
Polack 2020,
Thomas, 2021a,b [3,4,5]		 Age ≥16 years 268/21,926 (1.2%)​d 268/21921 (1.2%) Placebo 1.00 (0.84, 1.18) Not serious
VAERS (Anaphylaxis) [32] All vaccinated ages 4.7/1,000,000 doses None Serious
VAERS (Myocarditis)e [33] Age ≥16 years Observed cases by age (years) and sexf

Females
Age 16–17: 15
Age 18–24: 11
Age 25–29: 4
Age 30–39: 7
Age 40–49: 12
Age 50–64: 9
Age ≥ 65: 4

Males
Age 16–17: 120
Age 18–24: 134
Age 25–29: 30
Age 30–39: 40
Age 40–49: 26
Age 50–64: 5
Age ≥ 65: 4

 Expected cases by age (years) and sexf

Females
Age 16–17: 0–2
Age 18–24: 0–5
Age 25–29: 0–4
Age 30–39: 1–13
Age 40–49: 1–13
Age 50–64: 2–22
Age ≥ 65: 2–22

Males
Age 16–17: 0–3
Age 18–24: 1–7
Age 25–29: 1–5
Age 30–39: 1–11
Age 40–49: 1–11
Age 50–64: 2–19
Age ≥ 65: 2–18

 Expected numbers occurring in population Elevated ratio of observed to expected cases among females aged 16–24 years and males 16–49 years. Serious
VSD (Anaphylaxis)g [34] Age ≥12 years 5.0/1,000,000 doses Not serious
VSD (Myocarditis)h [34] 18–39 years 9/24,232 3/62,481 Comparison interval in vaccinated individuals 9.1 (2.1–48.6) Not serious

a Included all randomized participants who received at least 1 dose of vaccine.

b Additional data provided by sponsor.

c One SAE of neuritis was reported from the phase 1 trial that had not been identified at the time of the Walsh publication. This SAE was deemed unrelated to vaccination.

d Four serious adverse events were deemed by blinded investigators to be related to vaccination. These included: shoulder injury related to vaccine administration, ventricular arrhythmia, lymphadenopathy, and lower back pain and bilateral lower extremity pain with radicular paresthesia. Through further investigation by the FDA, only two were classified as related to vaccination: shoulder injury and lymphadenopathy.

e Risk evaluated in a 7-day interval following vaccination.

f Reported cases and expected number of cases were examined by age group (16–17, 18–24, 25–29, 30–39, 40–49, 50–64, ≥65 years) and sex.

g Risk evaluated in a 0–1 day risk interval after vaccination.

h Risk evaluated in a 7-day interval following dose 2

Table 3f: Summary of Studies Reporting Reactogenicitya

References in this table: 2345

Table 3f: Summary of Studies Reporting Reactogenicity
Authors last name, pub year Age or other characteristic of importance n/N (%) intervention n/N (%) comparison Comparator RR (95% CI) Study limitations (Risk of Bias)
Walsh, 2020b [2] Age ≥16 years 3/24 (8.3%) 0/6 (0%) Placebo 1.96 (0.11, 33.62) Not serious
Polack, 2020
Thomas, 2021b [3,4,5]		 Age ≥16 years 520/4,924 (10.6%) 111/4,915 (2.3%) Placebo 4.68 (3.81, 5.69) Not serious

a Grade 3 or worse. Grade 3 local reactions include pain at injection site that prevents daily activity, redness > 10 cm, and swelling > 10 cm. Grade 3 systemic events include vomiting that requires IV hydration, diarrhea of 6 or more loose stools in 24 hours, or headache, fatigue/tiredness, chills, new or worsened muscle pain, or new or worsened joint pain that prevent daily routine activity.

b Additional data provided by sponsor.

Table 4: Grade Summary of Findings Table

Table 4: Grade Summary of Findings Table
Certainty assessment № of patients Vaccinated № of patients Unvaccinated Effect Relative
(95% CI)		 Effect Absolute
(95% CI)		 Certainty Importance
№ of studies Study design Risk of bias Inconsistency Indirectness Imprecision Other considerations
Symptomatic laboratory-confirmed COVID-19
1 Randomized studies not seriousa not serious not seriousb not serious none 77/19,711 (0.4%) 833/19,741 (4.2%) RR 0.09
(0.07–0.1)		 3,840 fewer per 100,000
(from 3,924 fewer to 3,713 fewer)c		 Type 1 CRITICAL
8d Nonrandomized studiese not serious not seriousf not serious not serious strong association 1,715/g 43,968/g RR 0.08
(0.03–0.13)h		 3,780 fewer per 100,000
(from 3,990 fewer to 3,528 fewer)c		 Type 2 CRITICAL
Hospitalization for COVID-19
1 Randomized studies not seriousa,i not serious not seriousb seriousj none 0/19,687 (0.0%) 31/19,708 (0.2%) RR 0.02
(0.00–0.12)k		 154 fewer per 100,000
(from 116 fewer to --)c		 Type 2 CRITICAL
8l Nonrandomized studiese not serious not seriousm not seriousn not serious strong association 632/o 7,170/ unexposedo RR 0.06
(0.03–0.12)h		 188 fewer per 100,000
(from 194 fewer to 176 fewer)c		 Type 2 CRITICAL
Death due to COVID-19
1 Randomized studies not seriousp not serious not seriousb seriousq none 1/19,687 (0.0%) 6/19,708 (0.0%)r RR 0.17
(0.02–1.39)		 25 fewer per 100,000
(from 30 fewer to 12 more)c		 Type 2 IMPORTANT
4s Nonrandomized studiese not serious not serioust not seriousu not serious strong association 163/- (14.0%)v 0.0% RR 0.04
(0.02–0.09)h		 29 fewer per 100,000
(from 29 fewer to 28 fewer)c		 Type 2 IMPORTANT
Asymptomatic SARS-CoV-2 infection, assessed with PCR
0 Randomized studies
2w Nonrandomized studiese not serious seriousx not serious not serious none 3632/- (7.4%)v 4.2% RR 0.11
(0.10–0.12)y		 3,738 fewer per 100,000
(from 3,780  fewer to 3,696 fewer)z		 Type 4 IMPORTANT
Serious adverse events
2 Randomized studies not seriousaa not serious not seriousb seriousab none 269/21,950 (1.2%) 268/21,927 (1.2%) RR 1.00
(0.85–1.18)ac		 0 fewer per 100,000
(from 183 fewer to 220 more)c		 Type 2 CRITICAL
2w Nonrandomized studies not serious not serious not serious not serious none See narrativead,ae,af Type 3 CRITICALag
Reactogenicity, grade >=3
2 Randomized studies not serious not serious not seriousb not serious none 531/4,948 (10.7%) 111/4,921 (2.3%) RR 4.69
(3.83–5.73)ac		 8,323 more per 100,000
(from 6,383 more to 10,669 more)c		 Type 1 IMPORTANT
0 Nonrandomized studies

CI: Confidence interval; RR: Risk ratio

Explanations

a. Risk of bias related to blinding of participants and personnel was present. Although participants and study staff were blinded to intervention assignments, they may have inferred receipt of vaccine or placebo based on reactogenicity. This was deemed unlikely to overestimate efficacy or underestimate risk of serious adverse events, therefore the risk of bias was rated as not serious.

b. The RCT excluded persons with prior COVID-19 diagnosis, pregnant or breastfeeding women, and persons who were immunocompromised. The population included in the RCT may not represent all persons aged >=16 years.

c. Absolute risk was calculated using the observed risk among placebo recipients in the available body of evidence from randomized controlled trials. Absolute risk estimates should be interpreted in this context.

d. 17 studies were available in the body of evidence. 8 were excluded because the study population was already represented, and 1 was excluded due to serious study limitations.

e. The body of evidence includes preprints.

f. Although I2 value was high (95.0%), no serious concern for inconsistency was judged because all studies showed a high degree of vaccine effectiveness, with point estimates ranging from 87% to 97%. In a sensitivity analysis including results from one study with study limitations identified that had a vaccine effectiveness estimate of 56%, the pooled RR was 0.10 (95% CI 0.05–0.18), and I2 was 98.1%.

g. Data on numerators and denominators were not consistently reported in the available body of evidence. The n shown excludes events from studies that did not report the number of cases. The N is not included because studies variously provided person-time or number of persons. In addition to the numerators from cohort studies shown, the body of evidence included at least 85 cases and 54,603 controls from case-control or test-negative studies.

h. Pooled RR based on a random effects meta-analysis, using adjusted vaccine effectiveness estimates on a log scale.

i. Risk of bias was considered due to concern about misclassification of outcome. Hospitalization due to COVID-19 is not specified in the study protocol, and the data shown include only persons who met the protocol definition of COVID-19 using an approved assay or confirmation in a central laboratory; it was unclear if constructing a non-protocol measure may have resulted in bias. Data on all hospitalizations due to COVID-19 diagnosed by any assay after dose 1 were also obtained and reviewed. Two hospitalizations due to COVID-19 occurred among 21,909 persons in the vaccine arm and 59 occurred among 21,908 persons in the placebo arm (RR 0.03, 95% CI 0.01–0.14); the similar efficacy diminished concerns regarding risk of bias.

j. Serious concerns of imprecision due to fragility in the estimate was present because there were only 31 events observed from a single RCT.

k. RR calculated using a standard continuity correction of 0.5.

l. 13 studies were available in the body of evidence. 5 were excluded because the study population was already represented.

m. Although I2 value was high (91.7%), no serious concern for inconsistency was judged because all studies showed a high degree of vaccine effectiveness, with point estimates ranging from 84% to 99%.

n. Definitions varied by study. Indirectness was considered given COVID-19 was not necessarily confirmed as the cause of hospitalizations, but this was deemed not serious.

o. Data on numerators and denominators were not consistently reported in the available body of evidence. The n shown excludes events from studies that did not report the number of cases. The N is not included because studies variously provided person-time or number of persons. In addition to the numerators from cohort studies shown, the body of evidence included at least 95 cases and 1,359 controls from case-control or test-negative studies.

p. Risk of bias was considered due to possible misclassification of outcomes. One death in a vaccine recipient and 3 deaths among placebo recipients were in persons who had been diagnosed with COVID-19 based on local clinical nucleic acid amplification tests that were not protocol approved; these diagnoses were not confirmed by the central study laboratory and were not counted in the efficacy estimates for symptomatic laboratory-confirmed COVID-19 or hospitalization due to COVID-19. In an analysis using only protocol approved or central laboratory confirmed cases resulting in death, with a standard continuity correction applied, the relative risk was 0.14 (95% CI 0.01–2.77).

q. Serious concern for imprecision was present due to the small number of events that were observed. In addition to a 95% confidence interval crossing the line of no effect, there was concern for fragility in the estimate due to the small number of events.

r. Calculated risk among placebo arm in available body of evidence from RCT was 0.03%, but it appears lower here due to rounding.

s. 6 studies were available in the body of evidence. 2 were excluded because the study population was already represented.

t. The relative risk shown is from a pooled analysis of 4 cohort studies conducted in different populations. I2 was 48.8%.

u. Definitions varied by study. Indirectness was considered given COVID-19 was not necessarily confirmed as the cause of deaths, but this was deemed not serious.

v. Data on numerators and denominators were not consistently reported in the available body of evidence. The n shown excludes events from studies that did not report the number of cases. The N is not included because the type of denominator varied across studies (e.g., person-time or number of persons).

w. 5 studies were available in the body of evidence. 2 were excluded because the study population was already represented, and one study was excluded due to study limitations.

x. Serious concern for inconsistency was present (I2 = 98.1%). The magnitude of the relative risks from the two studies in the body of evidence varied widely, possibly reflecting different prevalence of circulating SARS-CoV-2 variants at the time of data collection or differences in study methods. In a sensitivity analysis including results from one study with study limitations identified that had a vaccine effectiveness estimate of 35.9%, the pooled RR was 0.12% (95% CI 0.11–0.13), and I2 was 99.1%.

y. Pooled RR based on a fixed effects meta-analysis, using adjusted vaccine effectiveness estimates on a log scale. Fixed effects model was used for this analysis due to imprecise estimates of the between-studies variance.

z. Absolute risk was calculated using the observed risk of symptomatic COVID-19 among placebo recipients in the available body of evidence from randomized controlled trials. Absolute risk estimates should be interpreted in this context.

aa. Risk of bias related to blinding of participants was present. Although participants and study staff were blinded to intervention assignments, they may have inferred receipt of vaccine or placebo based on reactogenicity. Some reactogenicity outcomes may also have been reported as serious adverse events, and experiences of reactions immediately after vaccination could have influenced recall or reporting of subsequent serious adverse events. This was rated as not serious.

ab. Serious concern for imprecision was present. The confidence interval indicates that both reduced and increased risk of serious adverse events are possible.

ac. Pooled RR based on a fixed effects meta-analysis. Fixed effects model was appropriate for this analysis because these RCTs used the same protocol and were conducted in similar populations.

ad. A rapid cycle analysis from Vaccine Safety Datalink (VSD) evaluated chart-reviewed cases of myocarditis among persons aged 18–39 years, following dose 2. Based on events occurring in a 7-day risk interval after vaccination vs. a comparison interval in vaccinated individuals, the adjusted rate ratio was 9.1% (95% CI 2.1–48.6). The rates of myocarditis were 368 per 1 million person-years (9/24,432) in the 0–7 day risk interval and 48 per 1 million person-years (3/62,481) in vaccinated comparators.

ae. Data from the national Vaccine Adverse Event Reporting System (VAERS) showed an elevated ratio of observed to expected myocarditis cases in the 7-day interval following vaccination among females in age groups 16–24 years, and among males in age groups 16–49 years, with higher observed/expected ratios in males than females. Although VAERS data are subject to the limitations of a passive surveillance system, the elevated risk of myocarditis following Pfizer vaccination is consistent with that observed in VSD.

af. A rapid cycle analysis of data from VSD evaluated chart-reviewed cases of anaphylaxis among all vaccinated persons aged 12 and older. Based on events occurring in a 0-1 day risk interval after vaccination, the estimated incidence of confirmed anaphylaxis was 5.0 (95% CI 3.5-6.9) per million doses. The absolute reported rate to VAERS was 4.7 per million doses administered.

Table 5: Summary of Evidence for Outcomes of Interest

Summary of Evidence for Outcomes of Interest
Outcome Importance Included in profile Certainty
Symptomatic laboratory-confirmed COVID-19 Critical Yes Type 1 (high)
Hospitalization due to COVID-19 Critical Yes Type 2 (moderate)
Death due to COVID-19 Important Yes Type 2 (moderate)
Asymptomatic SARS-CoV-2 infection Important Yes Type 4 (very low)
Serious Adverse Events (SAEs) (including myocarditis and anaphylaxis) Critical Yes Type 2 (moderate)
Reactogenicity (proportion with grade 3 or worse reactions) Important Yes Type 1 (high)

Appendix 1. Studies Included in the Review of Evidence

Randomized Controlled Trial

References in this table: 2345

Last name first author, Publication year Study design Country (or more detail, if needed) Age, central tendency or range Total population N vaccinated N unvaccinated Outcomes Funding source
 Polack 2020; Thomas, 2021a;  [3,4,5]b RCT United States
Brazil
Argentina
South Africa
Turkey
Germany		 ≥16 years 43,548 19,711 19,741
  • Symptomatic COVID-19 (PCR confirmed)c
  • Hospitalization due to COVID-19
  • Death due to COVID-19
  • Serious Adverse Events
  • Reactogenicityd
 Industry funding
Walsh, 2020 [2] RCT United States 18-55, 65-85 years 195 12 3
  • Serious Adverse Events
  • Reactogenicityd
 Industry funding

Observational Cohort Studies

References in this table: 678912131415161721232425262731

Last name first author, Publication year Study design Country (or more detail, if needed) Age, central tendency or range Total population N vaccinated N unvaccinated Outcomes Funding source
Alalia, [6] Observational (Retrospective Cohort) Kuwait median (IQR) 38 (33 - 44) years 3,246 NRe NRe
  • Symptomatic laboratory-confirmed COVID-19c
 NRe
Angel, 2021 [7] Observational (Retrospective Cohort) Israel mean [SD] 44.3 [12.5] years 6,710  5,372 696
  • Symptomatic laboratory-confirmed COVID-19c
  • Asymptomatic infection (PCR)
 NRe
Balicera, [8] Observational (Prospective Cohort) Israel ≥16 years 21,722 10,861 10,861
  • Symptomatic laboratory-confirmed COVID-19c
  • Hospitalization due to COVID-19
 University/Academic, Industry & Other
Barda, 2021 [9] Observational (Retrospective Cohort) Israel ≥16 years 1,163,534 596,618 596,618
  • Hospitalization due to COVID-19
 NRe
Dagan, 2021 [12] Observational (Retrospective Cohort) Israel median (IQR) 45 (35–62) years 1,163,534 596,618 596,618
  • Symptomatic laboratory-confirmed COVID-19c
  • Hospitalization due to COVID-19
 NRe
Emborga,  [13] Observational (Retrospective Cohort) Denmark ≥18 years 864,096 NRe NRe
  • Hospitalization due to COVID-19
  • Death due to COVID-19
 Government funding
Fabiani, 2021 [14] Observational (Retrospective Cohort) Italy mean (SD) 47.1 (10.8) years 6,423 NRe NRe
  • Symptomatic laboratory-confirmed COVID-19c
 NRe
Flacco, 2021 [15] Observational (Retrospective Cohort) Italy ≥18 years 273,210 30,817  174,023
  • Hospitalization due to COVID-19
  • Death due to COVID-19
 None declared
Goldberga,  [16] Observational (Prospective Cohort) Israel ≥16 years 6,351,903 NRe NRe
  • Hospitalization due to COVID-19
  • Death due to COVID-19
 None declared
Haas, 2021 [17] Observational (Retrospective Cohort) Israel ≥16 years 6,538,911 NRe NRe
  • Symptomatic laboratory-confirmed COVID-19c
  • Hospitalization due to COVID-19
  • Death due to COVID-19
  • Asymptomatic infection (PCR)
 Government funding & Industry funding
Martinez-Baz, 2021 [21] Observational (Prospective Cohort) Spain ≥18 years 20,961 491 contacts 19,580 contacts
  • Symptomatic laboratory-confirmed COVID-19c
  • Hospitalization due to COVID-19
 Government funding & Other (Horizon 2020 program of the European Commission)
Pawlowski, 2021 [23] Observational (Retrospective Cohort) United States ≥18 years 136,532 NRe NRe
  • Hospitalization due to COVID-19
 Other (nference; data analysis organization)
Pouwelsa,  [24] Observational (Longitudinal Household Survey) United Kingdom 18 – 64 years 384,543 NRe NRe
  • Symptomatic laboratory-confirmed COVID-19c
  • Asymptomatic infection (PCR)
 Other: Wellcome Trust [110110/Z/15/Z]
Puranika,  [25] Observational (Matched Retrospective Cohort) United States ≥18 years 179,546 NRe NRe
  • Hospitalization due to COVID-19
  • Death due to COVID-19
 NRe
Regav-Yochay, 2021 [26] Observational (Prospective Cohort) Israel ≥18 years 9,347 NRe NR3
  • Symptomatic laboratory-confirmed COVID-19c
  • Asymptomatic infection (PCR)
 University/Academic
Saciuka,  [27] Observational (Retrospective Cohort) Israel ≥16 years 1,650,885 1,354,444 1,166,487
  • Hospitalization due to COVID-19
  • Death due to COVID-19
 Other: Maccabi HealthCare Services
Whitakera,  [31] Observational (Prospective Cohort) England ≥16 years 5,642,687 NRe NRe
  • Symptomatic laboratory-confirmed COVID-19c
 Government funding

Observational Case-Control Studies

References in this table: 101118192022282930

Last name
first author, Publication year		 Study design Country (or more detail, if needed) Age, central tendency or range Total population N
cases		 N
controls		 Outcomes Funding source
Carazoa, [10] Observational
(Test-Negative Case Control)		 Canada 18–74 years 58,476 5,316 53,160
  • Symptomatic laboratory-confirmed COVID-19c
 Other (Ministere de la sante’ et des services sociaux du Quebec)
Chung, 2021 [11] Observational
(Test-Negative Case Control)		 Canada ≥16 years 324,033 53,270 279,763
  • Symptomatic laboratory-confirmed COVID-19c
 University/Academic & Government funding
Kissling, 2021 [18] Observational (Test-Negative Case Control) France
England
Ireland
Netherlands
Portugal
Scotland
Spain
Sweden		 ≥65 years 4,964 519 2,857
  • Symptomatic laboratory-confirmed COVID-19c
 Other (European Union’s Horizon 2020 research & innovation programme)
Lopez Bernal, 2021 [19] Observational (Test-Negative Case Control) England >80 years 153,441 8,988 15,718
  • Symptomatic laboratory-confirmed COVID-19c
 None declared
Lopez Bernal, 2021 [20] Observational (Test-Negative Case Control) England ≥16 years 19,109 15,749 96,371
  • Symptomatic laboratory-confirmed COVID-19c
 Government funding
Nasreena, [22] Observational (Test-Negative Case Control) Canada ≥16 years 421,073 36,832 351,540
  • Symptomatic laboratory-confirmed COVID-19c
  • Hospitalization due to COVID-19
 Government funding
Stowea,
[28]		 Observational (Test-Negative Case Control) England NRe NRe NRe NRe
  • Hospitalization due to COVID-19
 Government funding
Tanga,
[29]		 Observational (Matched Test-Negative Case Control) Qatar median (IQR)
31 (24-37)		 39,156 757 cases 757 controls
  • Symptomatic laboratory-confirmed COVID-19c
  • Asymptomatic infection (PCR)
 NRe
Tenforde, 2021 [30] Observational (Test-Negative Case Control) United States ≥18 years Pfizer, 2 doses: 482

No Vaccine: 396

 1,194 1,895
  • Hospitalization due to COVID-19
 Government funding

Safety Surveillance

References in this table: 323334

Name of system Study design Country (or more detail, if needed) Age, central tendency or range Total population N vaccinated N unvaccinated Outcomes Funding source
Vaccine Adverse Event Reporting System (VAERS) [32,33] Passive surveillance United States ≥16 years (anaphylaxis);
16–49 years (myocarditis)
  • Serious Adverse Events
 Government funding
Vaccine Safety Datalink (VSD) [34] Cohort United States ≥12 years (anaphylaxis);
18–39 years (myocarditis)
  • Serious Adverse Events
 Government funding

a Pre-print

b Additional data provided by sponsor

c This was a primary outcome of the RCT, defined as SARS-CoV-2 RT-PCR-positive symptomatic illness, in seronegative persons aged ≥18 years, ≥7 days post second dose. In a secondary analysis among seronegative and seropositive persons, the efficacy was Grade 3 or worse.

d Grade 3 local reactions include pain at injection site that prevents daily activity, redness > 10 cm, and swelling > 10 cm. Grade 3 systemic events include vomiting that requires IV hydration, diarrhea of 6 or more loose stools in 24 hours, or headache, fatigue/tiredness, chills, new or worsened muscle pain, or new or worsened joint pain that prevent daily routine activity.

e Not reported

Appendix 2. Databases and strategies used for systematic review

Appendix 2. Databases and strategies used for systematic review
Database Strategy
clinicaltrials.gov Inclusion: Relevant Phase 1, 2, or 3 randomized controlled trials of COVID-19 vaccine
  • Involved human subjects
  • Reported primary data
  • Included adults (age ≥16 years) at risk for SARS-CoV-2 infection
  • Included data relevant to the efficacy and safety outcomes being measured
  • Included data for the dosage and timing being recommended (30 µg, 2 doses at 0 and 21 days)

Additional resources: Unpublished and other relevant data by consulting with vaccine manufacturers and subject matter experts
International Vaccine Access Center (IVAC) Inclusion criteria for IVAC systematic review<:
  • Published or preprint study with adequate scientific details
  • Includes groups with and without infection or disease outcome
  • Laboratory confirmed outcome
  • Vaccination status confirmed in ≥90%
  • Studies assess one vaccine or pooled mRNA vaccines
  • Includes participants who did or did not receive a COVID-19 vaccine

Vaccine effectiveness estimate calculated comparing vaccinated to unvaccinated**
Additional criteria for GRADE review:

  • Restricted to PICO-defined population, intervention, comparison, and outcomes
  • Outcomes assess 7 or 14 days after 2nd dose
  • Only Pfizer-BioNTech vaccine (not mRNA vaccines as a group)
  • Included studies of general population and special populations (e.g., elderly, pregnant persons, healthcare workers)
Safety Surveillance Systems Evidence Retrieval for Observational Safety Studies:
  • Based on input from ACIP’s COVID-19 Vaccine Safety Technical (VaST) Work Group
  • Data on safety signals identified by vaccine surveillance systems
  • Data have been presented to ACIP

View the complete list of GRADE evidence tables‎

List of GRADE evidence tables
Print
Content Source:
National Center for Immunization and Respiratory Diseases (NCIRD)
  1. Ahmed F. U.S. Advisory Committee on Immunization Practices (ACIP) Handbook for Developing Evidence-based Recommendations
  2. Walsh EE, Frenck RW, Falsey AR et al. Safety and Immunogenicity of Two RNA-Based COVID-19 Vaccine Candidates. NEJM. 2020. DOI: 1056/NEJMoa2027906
  3. Pfizer, 2021. personal communication, August 2 – August 21, 2021.
  4. Polack FP, Thomas SJ, Kitchin N, et al. Safety and Efficacy of the BNT162b2 mRNA COVID-19 Vaccine. NEJM. 2020. DOI: 1056/NEJMoa2034577
  5. Thomas, S. J., et al. (2021). Six Month Safety and Efficacy of the BNT162b2 mRNA COVID-19 Vaccine. medRxiv: 2021.2007.2028.21261159.
  6. Alali WQ, Ali LA, Al Seaidan M, Al-Rashidi M. Effectiveness of BNT162b2 and ChAdOx1 Vaccines Against Symptomatic COVID-19 Among Healthcare Workers in Kuwait: A Retrospective Cohort Study. medRxiv 2021: 2021.07.25.21261083. preprint.
  7. Angel Y, Spitzer A, Henig O, et al. Association Between Vaccination With BNT162b2 and Incidence of Symptomatic and Asymptomatic SARS-CoV-2 Infections Among Health Care Workers. JAMA 2021; 325(24): 2457-65.
  8. Balicer R, Dagan N, Barda N, et al. Effectiveness of the BNT162b2 mRNA COVID-19 Vaccine in Pregnancy. https://doi.org/10.21203/rs.3.rs-665725/v1. preprint.
  9. Barda N, Dagan N, Balicer RD. BNT162b2 mRNA COVID-19 Vaccine in a Nationwide Mass Vaccination Setting. Reply. N Engl J Med. 2021 May 20;384(20):1970. doi: 10.1056/NEJMc2104281.
  10. Carazo S, Talbot D, Boulianne N, et al. Single-Dose mRNA Vaccine Effectiveness Against Sars-Cov-2 in Healthcare Workers Extending 16 Weeks Post-Vaccination: A Test-Negative Design from Quebec, Canada. medRxiv 2021: 2021.07.19.21260445. preprint.
  11. Chung H, He S, Nasreen S, et al. Effectiveness of BNT162b2 and mRNA-1273 covid-19 Vaccines Against Symptomatic SARS-CoV-2 Infection and Severe COVID-19 Outcomes in Ontario, Canada: Test Negative Design Study. BMJ 2021; 374: n1943.
  12. Dagan N, Barda N, Kepten E, et al. BNT162b2 mRNA Covid-19 Vaccine in a Nationwide Mass Vaccination Setting. N Engl J Med 2021; 384(15): 1412-23.
  13. Emborg H-D, Valentiner-Branth P, Schelde AB, et al. Vaccine Effectiveness of the BNT162b2 mRNA COVID-19 Vaccine Against RT-PCR Confirmed SARS-CoV-2 Infections, Hospitalizations and Mortality in Prioritized Risk Groups. medRxiv 2021: 2021.05.27.21257583. preprint.
  14. Fabiani M, Ramigni M, Gobbetto V, Mateo-Urdiales A, Pezzotti P, Piovesan Effectiveness of the Comirnaty (BNT162b2, BioNTech/Pfizer) Vaccine in Preventing SARS-CoV-2 Infection Among Healthcare Workers, Treviso Province, Veneto Region, Italy, 27 December 2020 to 24 March 2021. Euro Surveill 2021; 26(17).
  15. Flacco ME, Soldato G, Acuti Martellucci C, et al. Interim Estimates of COVID-19 Vaccine Effectiveness in a Mass Vaccination Setting: Data from an Italian Province. Vaccines 2021; 9(6): 628.
  16. Goldberg Y, Mandel M, Woodbridge Y, et al. Protection of Previous SARS-CoV-2 Infection is Similar to That of BNT162b2 Vaccine Protection: A Three-Month Nationwide Experience from Israel. medRxiv2021: 2021.04.20.21255670. preprint.
  17. Haas EJ, Angulo FJ, McLaughlin JM, et al. Impact and Effectiveness of mRNA BNT162b2 Vaccine Against SARS-CoV-2 Infections and COVID-19 Cases, Hospitalizations, and Deaths Following a Nationwide Vaccination Campaign in Israel: an Observational Study Using National Surveillance Data. Lancet 2021; 397(10287): 1819-29.
  18. Kissling E, Hooiveld M, Sandonis Martín V, et al. Vaccine Effectiveness Against Symptomatic SARS-CoV-2 Infection in Adults Aged 65 Years and Older in Primary Care: I-MOVE-COVID-19 project, Europe, December 2020 to May 2021. Eurosurveillance 2021; 26(29): 2100670.
  19. Lopez Bernal J, Andrews N, Gower C, et al. Effectiveness of the Pfizer-BioNTech and Oxford-AstraZeneca vaccines on COVID-19 Related Symptoms, Hospital Admissions, and Mortality in Older Adults in England: Test Negative Case-Control Study. BMJ 2021; 373: n1088.
  20. Lopez Bernal J, Andrews N, Gower C, et al. Effectiveness of COVID-19 Vaccines Against the B.1.617.2 (Delta) Variant. N EnglJ Med 2021; 385(7): 585-94.
  21. Martinez-Baz I, Miqueleiz A, Casado I, et al. Effectiveness of COVID-19 Vaccines in Preventing SARS-CoV-2 Infection and Hospitalization, Navarre, Spain, January to April 2021. Euro Surveill 2021; 26(21).
  22. Nasreen S, He S, Chung H, et al. Effectiveness of COVID-19 Vaccines Against Variants of Concern, Canada. medRxiv2021: 2021.06.28.21259420. preprint.
  23. Pawlowski C, Lenehan P, Puranik A, et al. FDA-authorized mRNA COVID-19 Vaccines are Effective per Real-World Evidence Synthesized Across a Multi-State Health System. Med (N Y) 2021; 2(8): 979-92 e8.
  24. Pouwels KB, Pritchard E, Matthews PC, et al. Impact of Delta on Viral Burden and Vaccine Effectiveness Against New SARS-CoV-2 Infections in the UK. medRxiv 2021: 2021.08.18.21262237. preprint.
  25. Puranik A, Lenehan PJ, Silvert E, et al. Comparison of two highly-effective mRNA Vaccines for COVID-19 during periods of Alpha and Delta variant prevalence. medRxiv 2021: 2021.08.06.21261707. preprint.
  26. Regev-Yochay G, Amit S, Bergwerk M, et al. Decreased Infectivity Following BNT162b2 Vaccination: A Prospective Cohort Study in Israel. Lancet Reg Health Eur 2021; 7: 100150.
  27. Saciuk Y, Kertes J, Mandel M, Hemo B, Shamir Stein N, Zohar A. Pfizer-BioNTech Vaccine Effectiveness Against SARS-CoV-2 Infection: Findings From a Large Observational Study in Israel. SSRN 2021. http://dx.doi.org/10.2139/ssrn.3868853. preprint.
  28. Stowe J, Andrews N, Gower C, et al. Effectiveness of COVID-19 Vaccines Against Hospital Admission with the Delta (B.1.617.2) Variant. https://fpmag.net/wp-content/uploads/2021/06/Effectiveness-of-COVID-19-vaccines-against-hospital-admission-with-the-Delta-B_1_617_2variant.pdf. preprint.
  29. Tang P, Hasan MR, Chemaitelly H, et al. BNT162b2 and mRNA-1273 COVID-19 Vaccine Effectiveness Against the Delta (B.1.617.2) Variant in Qatar. medRxiv 2021: 2021.08.11.21261885. preprint.
  30. Tenforde MW, Patel MM, Ginde AA, et al. Effectiveness of SARS-CoV-2 mRNA Vaccines for Preventing COVID-19 Hospitalizations in the United States. medRxiv 2021: 2021.07.08.21259776. preprint.
  31. Whitaker H, Tsang R, Byford R, et al. Pfizer-BioNTech and Oxford AstraZeneca COVID-19 Vaccine Effectiveness and Immune Response Among Individuals in Clinical Risk Groups. https://khub.net/documents/135939561/430986542/RCGP+VE+riskgroups+paper.pdf/a6b54cd9-419d-9b63-e2bf-5dc796f5a91f. preprint.
  32. Shimabukuro TT, Cole M, Su JR. Reports of Anaphylaxis After Receipt of mRNA COVID-19 Vaccines in the US—December 14, 2020-January 18, 2021. JAMA. 2021;325(11):1101–1102. doi:10.1001/jama.2021.1967
  33. Shimabukuro TT, Cole M, Su JR. Reports of Anaphylaxis After Receipt of mRNA COVID-19 Vaccines in the US—December 14, 2020-January 18, 2021. JAMA. 2021;325(11):1101–1102. doi:10.1001/jama.2021.1967
  34. Klein, N. Safety update for COVID-19 vaccines: VSD. Presentation to ACIP. August 30, 2021.
  35. U.S. Food and Drug Administration, August 23, 2021 Approval Letter – Comirnaty (fda.gov)
  36. International Vaccine Access Center (IVAC), Johns Hopkins Bloomberg School of Public Health. VIEW-hub. www.view-hub.org. Accessed: 8/20/2021.