Grading of Recommendations, Assessment, Development, and Evaluation (GRADE): Use of JYNNEOS (orthopoxvirus) vaccine primary series for research, clinical laboratory, response team, and healthcare personnel (Policy Questions 1 and 2)

About

  • CDC vaccine recommendations are developed using an explicit evidence-based method based on the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach.

Table 1.1: Policy Question and PICO

Policy question:
Should JYNNEOS® be recommended for persons who are at risk* for occupational exposure to orthopoxviruses?
Population
Persons who are at risk* for occupational exposure to orthopoxviruses
Intervention
Vaccination with JYNNEOS®
Comparison
Vaccination with ACAM2000
Outcomes
a. Prevention of disease
b. Severity of disease
c. Serious adverse events
d. Myo-/ peri- carditis

*Research laboratory personnel, diagnostic laboratory personnel, and orthopoxvirus and healthcare worker response teams designated by appropriate public health and antiterror authorities; clinical laboratory personnel who perform routine chemistry, hematology, and urinalysis procedures are not recommended for vaccination with ACAM2000 or JYNNEOS because the quantity of orthopoxviruses likely to be in clinical specimens of blood and body fluids is low.

Table 1.2: Policy Question 2 and PICO

Policy question:
Policy question: Should JYNNEOS® be recommended, based on shared clinical decision making, for healthcare personnel at risk* for occupational exposure to orthopoxviruses?
Population
Persons who are at risk* for occupational exposure to orthopoxviruses
Intervention
Vaccination with JYNNEOS®
Comparison
Vaccination with ACAM2000
Outcomes
a. Prevention of disease
b. Severity of disease
c. Serious adverse events
d. Myo-/ peri- carditis

*For example, clinical trial physicians or nurses who currently treat or anticipate treating patients with replication-competent orthopoxvirus infections or persons who administer ACAM2000 smallpox vaccine

Table 2: Outcomes and Rankings

Outcome Importance* Included in evidence profile
Prevention of disease Critical Yes
Severity of disease Important Yes
Serious adverse events** Critical Yes
Myo-/ peri- carditis Critical Yes
Minor adverse events Not important No

*Three options: 1. Critical; 2. Important but not critical; 3. Not important for decision making

**Serious adverse events were defined according to the standard FDA definition. In addition, data was collected about any smallpox vaccine-specific adverse event: postvaccinial encephalitis, eczema vaccinatum, progressive vaccinia, and generalized vaccinia.

Appendix 1: Studies Included in the Review of Evidence

Last name first author, Publication year Study design Country (or more detail, if needed) Age (measure central tendency – mean/SD; median/IQR; range) Total population N Intervention N comparison Outcomes Funding source
RCT data
NCT01913353
Pittman
Pittman 2019
Phase 3, open-label,
randomized clinical trial
U.S. military, stationed in Korea Mean 23.5
SD 4.67
433 220 213 Immunogenicity,
Surrogate efficacy (ACAM2000 challenge),
Adverse events
Bavarian Nordic, US Army Medical Research Institute of Infectious Diseases
NCT00082446
Frey1
Frey 2007
Sano 2009
Phase I, randomized, partially blinded,
placebo controlled clinical trial
USA Mean 24.8
SD 3.8
90 30 15 Immunogenicity, Cell-mediated immunity, Surrogate efficacy (Dryvax challenge),
Adverse events
NIAID
VRC 201
Parrino1
Parrino 2007
Phase I/Ib randomized, placebo controlled, double-blinded trial USA Total Mean
SD NR
adults
76 19 21 Immunogenicity, safety, Dryvax challenge, cell mediated/humoral immune responses NIAID
Observational data for the intervention
NCT00437021
Frey 2
Frey et al. 2013
Troy et al. 2015
Phase II, Double-blind, Randomized, Dose-finding Study USA Mean 24.7
SD 4.2
208 67 NA Safety and immunogenicity NIAID
NCT01668537
Greenburg4
2014
Phase II, Randomized, Double-blind, Multicenter USA Mean 27.7
SD 6.28
651 327 NA Safety and immunogenicity Bavarian Nordic
NCT00879762
Frey3
Troy et al. 2015
Frey et al. 2014
Phase II, randomized, double blinded USA Mean 26.5
SD NR
91 45 NA Safety and immunogenicity NIAID
NCT00316602
Greenburg2
Greenberg 2015
Phase II, non-randomized, open-label USA and Mexico Mean 27.7
SD 6.11
632 632 NA Safety and immunogenicity in people with atopic dermatitis NIAID and Bavarian Nordic
NCT00914732
Frey4
Troy et al. 2015
Frey et al. 2015
Phase II, randomized, triple blinded USA Mean 27.2
SD 4.6
523 167 NA Safety and immunogenicity NIAID and Bavarian Nordic
NCT00316524
von Sonnenburg1
Zitzman-Roth et al. 2015
Partially Randomized, Partially Double-blind, Placebo-controlled Phase II Non-inferiority Study Germany Mean 29.8
SD 9.07
745 183 NA Safety and immunogenicity NIAID and Bavarian Nordic
NCT00189904
Greenburg1
Greenberg et al. 2013
Phase I/II, non-randomized, open-label USA Mean 37.9
SD NR
151 60 NA Safety and immunogenicity in HIV positive patients NIAID
NCT01144637
Overton2
Overton et al. 2018
Randomized, Double-Blind, Placebo-Controlled Phase III Trial USA Mean 27.7
SD 6.3
4005 3003 NA immunogenicity, safety, and tolerability Bavarian Nordic and BARDA
NCT00189917
von Sonnenburg2
Darsow et al. 2016
Von Sonnenburg et al. 2014
Open-label, Controlled Phase I Pilot Study Germany Mean NR
SD NR
60 60 NA Safety and immunogenicity NIAID and Bavarian Nordic
NCT00133575
Seaman/Wilck
Seaman et al. 2010
Wilck et al. 2010
Phase I/II, randomized, double blinded, placebo-controlled USA Mean 25.2, SD=3.7 72 10 NA Safety and immunogenicity and surrogate efficacy (Dryvax challenge) NIAID
NCT01827371
Frey5
Anderson et al. 2020
Jackson et al. 2017
Phase II, Randomized, Open-Label USA Mean 27.4
SD 5.3
435 115 NA Safety and immunogenicity NIAID
NCT02038881
Overton3
Overton et al. 2020
Phase II, Randomized, Open-label USA Mean 35
SD 6.7
87 58 NA Safety and immunogenicity in HIV+ patients Bavarian Nordic
NCT00316589
Overton1
Overton et al. 2015
Phase II, Multicenter, Open-label, Controlled USA Mean 37.5
SD 8.0
579 439 NA Safety and immunogenicity HHS and NIAID
NCT00189959
Pokorny
Von Kremplehuber et al. 2010
Phase II, Double-blind, randomized, Dose-finding Study Switzerland Mean 23.3
SD 3.0
165 55 NA Safety and immunogenicity NIAID and Bavarian Nordic
Vollmar
Vollamr et al. 2005
Phase 1, randomized, double-blinded and open-label Germany Mean 32.8 68 16 NA Safety and immunogenicity Bavarian Nordic

Table 3a: Summary of Studies Reporting Outcome A – Prevention of Disease

Authors last name, pub year Age or other characteristic of importance N intervention N comparison Comparator vaccine Absolute difference/effect estimate Study limitations (Risk of Bias)
RCT data
NCT01913353
Pittman
Pittman 2019
Mean age 23.5 y
Healthy adults
220 213 ACAM2000 PRNT GMT: Mean difference 1.93 (95% CI 1.57 to 2.39)
PRNT Seroconversion rate: Risk ratio 1.03 (95% CI 1.00 to 1.05)
Serious concerns due to open-label design and inclusion of only military soldiers (young and very healthy persons)
NCT00082446
Frey 1
Frey 2007
Sano 2009
Mean age 24.8 y
Healthy adults
30 13 Dryvax PRNT GMT: Mean difference 7.54 (95% CI 2.75 to 20.91)
PRNT Seroconversion rate: Risk ratio 1.20 (95% CI 0.93 to 1.53)
Not serious

Table 3b: Summary of Studies Reporting Outcome B – Severity of Disease

Authors last name, pub year Age or other characteristic of importance N intervention N comparison Comparator vaccine Absolute difference/effect estimate Study limitations (Risk of Bias)
RCT data
VRC 201
Parrino1
Parrino 2007
Mean age NR
Healthy adults
15 8 Dryvax RR 1.00 (95% CI 0.83-1.20)
0 fewer developed an attenuated take reaction (from 170 fewer to 200 more)
15/15 participants developed an attenuated1 take reaction in the intervention group
8/8 participants developed an attenuated1 take reaction in the comparison group
Concerns for risk of bias due to attrition. The two groups that contributed data to the intervention and comparison for this outcome lost between 11 and 21% of participants at the time this outcome was assessed.

1 “Attenuated take” was defined as a lesion in category 1 or 2. Categories: 0 (no take, i.e., no skin reaction or skin sensitivity consisting of erythema only within 24-48 hours that resolves by the 5th day without vesicle or pustule formation), 1 (modified take without vesicle, i.e., an attenuated primary vaccine site reaction with the following characteristics: papule with or without surrounding erythema by 3rd day and without vesicle or pustule formation prior to resolution), 2 (modified take with vesicle, i.e., an attenuated primary vaccine site reaction with the following characteristics: papule by 3rd day that becomes vesicular by 5-7th day, and dries shortly thereafter; a relatively small vesicle and areola; and the scar, if present, is usually insignificant), and 3(full take, i.e., primary take is where the inoculation site becomes reddened 3-4 days after vaccination. A vesicle surrounded by a red areola then forms which becomes umbilicated and then pustular by the 7t to 11th day after vaccination. The red areola enlarges by this time. The pustule begins to dry, the redness subsides, and the lesion becomes crusted between the 2nd and 3rd week. By the end of the 3rd week, the scab falls of leaving a permanent scar that at first is pink in color but eventually becomes flesh-colored).

Table 3c: Summary of Studies Reporting Outcome C – Serious Adverse Events (SAE)

Authors last name, pub year Age or other characteristic of importance N intervention N comparison Comparator vaccine Absolute difference/effect estimate Study limitations (Risk of Bias)
RCT
NCT01913353
Pittman
Pittman 2019
Mean age 23.5 y
Healthy vaccinia naïve adults
220 213 ACAM2000 Not estimable,
0/220 participants had vaccine related SAEs in the intervention group.
0/213 participants had vaccine related SAEs in the comparison group
Serious concerns because of open-label design
NCT00082446
Frey 1
Frey 2007
Sano 2009
Mean age 24.8 y
Healthy vaccinia naïve adults
30 15 Dryvax Not estimable,
0/30 participants had vaccine related SAEs in the intervention group.
0/13 participants had vaccine related SAEs in the comparison group
Not serious
VRC 201
Parrino1
Parrino 2007
Mean age NR
Healthy vaccinia naïve adults
18 21 Dryvax Risk ratio: 0.33 (95% CI 0.01 to 7.70)
0/19 participants had vaccine related SAEs in the intervention group.
1/191 participants had vaccine related SAEs in the comparison group
Concerns for risk of bias due to attrition. The two groups that contributed data to the intervention and comparison for this outcome lost between 11 and 21% of participants at the time this outcome was assessed.
Obs
NCT00437021
Frey 2
Frey et al. 2013
Troy et al. 2015
Age: Mean 24.7
SD 4.2
Healthy vaccinia naïve adults
67 NA NA NA2
0 participants had vaccine related SAEs in the intervention group
NCT01668537
Greenburg4
2014
Age: Mean 27.7
SD 6.28
Healthy vaccinia naïve adults
327 NA NA NA2
0 participants had vaccine related SAEs in the intervention group
NCT00879762
Frey3
Troy et al. 2015
Frey et al. 2014
Age: Mean 26.5
SD NR
Healthy vaccinia naïve adults
45 NA NA NA2
0 participants had vaccine related SAEs in the intervention group
NCT00316602
Greenburg2
Greenberg 2015
Age: Mean 27.7
SD 6.11
Healthy vaccinia naïve adults and
Vaccinia naïve adults with atopic dermatitis
623 NA NA NA2
1 participant had a SAE3 that was “probably related” to the vaccine in the intervention group (a healthy vaccinia naïve adult)
NCT00914732
Frey4
Troy et al. 2015
Frey et al. 2015
Age: Mean 27.2
SD 4.6
Healthy vaccinia naïve adults
167 NA NA NA2
0 participants had vaccine related SAEs in the intervention group
NCT00316524
von Sonnenburg1
Zitzman-Roth et al. 2015
Age: Mean 29.8
SD 9.07
Healthy vaccinia naïve adults
183 NA NA NA2
1 participant had a vaccine “related” SAE4 in the intervention group
NCT00189904
Greenburg1
Greenberg et al. 2013
Age: Mean 37.9
SD NR
HIV negative vaccinia naïve adults and HIV positive vaccinia naive adults
60 NA NA NA2
0 participants had vaccine related SAEs in the intervention group
NCT01144637
Overton2
Overton et al. 2018
Age: Mean 27.7
SD 6.3
Healthy vaccinia naïve adults
3003 NA NA NA2
0 participants had vaccine related SAEs in the intervention group
NCT00189917
von Sonnenburg2
Darsow et al. 2016
Von Sonnenburg et al. 2014
Age: Mean NR
SD NR
Healthy vaccinia naïve adults and adults with either history of atopic dermatitis, active atopic dermatitis, or allergic rhinitis
60 NA NA NA2
0 participants had vaccine related SAEs in the intervention group
NCT00133575
Seaman/Wilck
Seaman et al. 2010
Wilck et al. 2010
Age: Mean 25.2, SD=3.7
Healthy vaccinia naïve adults
10 NA NA NA2
0 participants had vaccine related SAEs in the intervention group
NCT01827371
Frey5
Anderson et al. 2020
Jackson et al. 2017
Age: Mean 27.4
SD 5.3
Healthy vaccinia naïve adults
115 NA NA NA2
1 participant had a vaccine “related” SAE5 in the intervention group
NCT02038881
Overton3
Overton et al. 2020
Age: Mean 35
SD 6.7
HIV positive vaccinia naïve adults
58 NA NA NA2
0 participants had vaccine related SAEs in the intervention group
NCT00316589
Overton1
Overton et al. 2015
Age: Mean 37.5
SD 8.0
Healthy vaccinia naïve adults and HIV positive vaccina naïve adults
439 NA NA NA2
1 participant had a SAE6 that was “possibly but unlikely” related to vaccination in the intervention group (an HIV positive vaccina naïve adult)
NCT00189959
Pokorny
Von Kremplehuber et al. 2010
Age: Mean 23.3
SD 3.0
Healthy vaccinia naïve adults
54 NA NA NA2
0 participants had vaccine related SAEs in the intervention group
Vollmar
Vollmar et al. 2005
Age: Mean 32.8
Healthy vaccinia naïve adult males
16 NA NA NA2
0 participants had vaccine related SAEs in the intervention group

1. One participant developed elevated liver enzymes 84 days after receiving Dryvax vaccine on day 0. The SAE was assessed to be “possibly related” to vaccination.

2. Intervention data was drawn from pooled single arm observational studies included in the systematic review. 4/5221 (0.08%) participants from 14 observational studies developed vaccine related serious adverse events. No smallpox vaccine-specific serious adverse event was recorded. Comparison data was drawn from historical data. In a phase III clinical trial for ACAM2000, 3/873 (0.34%) developed vaccine related serious adverse events after ACAM2000 administration. No smallpox vaccine-specific serious adverse event was recorded.

3. Extra ocular muscle paresis event in one person 8 days after second MVA-BN vaccination; deemed “probably related” by investigators.

4. Sarcoidosis event in one person during the 6 months follow up period; deemed “related” because causal relationship with vaccine could not be ruled out.

5. Acute myocardial infarction event in one person 117 days after the first MVA-BN dose. Deemed “related” to vaccination because no other reasonable etiology was found. Number at risk is 116 because one subject was initially randomized to study arm C but was vaccinated out of the window and was analyzed in study arm A.

6. Pneumonia and pleurisy event in one person 1 day after second MVA-BN dose. Deemed “possibly but unlikely” to be associated with vaccination.

Table 3d: Summary of Studies Reporting Outcome D – Myo/pericarditis

Authors last name, pub year Age or other characteristic of importance N intervention N comparison Comparator vaccine Absolute difference/effect estimate Study limitations (Risk of Bias)
RCT
NCT01913353
Pittman
Pittman 2019
Mean age 23.5 y
Healthy vaccinia naïve adults
220 213 ACAM2000 Not estimable,
0/220 participants developed myo-/pericarditis in the intervention group.
0/213 developed myo-/pericarditis in the comparison group
Serious concerns because of open-label trial design
NCT00082446
Frey 1
Frey 2007
Sano 2009
Mean age 24.8 y
Healthy vaccinia naïve adults
30 15 Dryvax Not estimable,
0/30 participants developed myo-/pericarditis in the intervention group.
0/13 developed myo-/pericarditis in the comparison group
Not serious
VRC 201
Parrino1
Parrino 2007
Mean age NR
Healthy vaccinia naïve adults
18 21 Dryvax Not estimable,
0/19 participants developed myo-/pericarditis in the intervention group.
0/19 participants developed myo-/pericarditis in the comparison group
Very serious because unknown how many people were monitored for myopericarditis and when in the trial, this was started
Observational data for the intervention
NCT00437021
Frey 2
Frey et al. 2013
Troy et al. 2015
Age: Mean 24.7
SD 4.2
Healthy vaccinia naïve adults
67 NA NA NA1
0 participants developed myo-/pericarditis in the intervention group
NCT01668537
Greenburg4
2014
Age: Mean 27.7
SD 6.28
Healthy vaccinia naïve adults
327 NA NA NA1
0 participants developed myo-/pericarditis in the intervention group
NCT00879762
Frey3
Troy et al. 2015
Frey et al. 2014
Age: Mean 26.5
SD NR
Healthy vaccinia naïve adults
45 NA NA NA1
0 participants developed myo-/pericarditis in the intervention group
NCT00316602
Greenburg2
Greenberg 2015
Age: Mean 27.7
SD 6.11
Healthy vaccinia naïve adults and
Vaccinia naïve adults with atopic dermatitis
623 NA NA NA1
0 participants developed myo-/pericarditis in the intervention group
NCT00316524
von Sonnenburg1
Zitzman-Roth et al. 2015
Age: Mean 29.8
SD 9.07
Healthy vaccinia naïve adults
183 NA NA NA1
0 participants developed myo-/pericarditis in the intervention group
NCT00189904
Greenburg1
Greenberg et al. 2013
Age: Mean 37.9
SD NR
HIV negative vaccinia naïve adults and HIV positive vaccinia naive adults
60 NA NA NA1
0 participants developed myo-/pericarditis in the intervention group
NCT01144637
Overton2
Overton et al. 2018
Age: Mean 27.7
SD 6.3
Healthy vaccinia naïve adults
3003 NA NA NA1
1 participant developed possible myopericarditis2 in the intervention group
NCT00189917
von Sonnenburg2
Darsow et al. 2016
Von Sonnenburg et al. 2014
Age: Mean NR
SD NR
Healthy vaccinia naïve adults and adults with either history of atopic dermatitis, active atopic dermatitis, or allergic rhinitis
60 NA NA NA1
0 participants developed myo-/pericarditis in the intervention group
NCT00133575
Seaman/Wilck
Seaman et al. 2010
Wilck et al. 2010
Age: Mean 25.2, SD=3.7
Healthy vaccinia naïve adults
10 NA NA NA1
0 participants developed myo-/pericarditis in the intervention group
NCT02038881
Overton3
Overton et al. 2020
Age: Mean 35
SD 6.7
HIV positive vaccinia naïve adults
58 NA NA NA1
0 participants developed myo-/pericarditis in the intervention group
NCT00316589
Overton1
Overton et al. 2015
Age: Mean 37.5
SD 8.0
Healthy vaccinia naïve adults and HIV positive vaccina naïve adults
439 NA NA NA1
0 participants developed myo-/pericarditis in the intervention group
NCT00189959
Pokorny
Von Kremplehuber et al. 2010
Age: Mean 23.3
SD 3.0
Healthy vaccinia naïve adults
54 NA NA NA1
0 participants developed myo-/pericarditis in the intervention group

1. There was no available comparison data from the systematic review. An effect estimate was calculated using intervention data from 12 naively pooled observational studies reporting myopericarditis after MVA vaccine administration (data from systematic review) and comparison data from one study reporting myopericarditis rate after ACAM2000 administration (Source: ACAM2000 package insert, FDA). Risk ratio 0.040 (0.004 to 0.310), 5 fewer events of myo-/pericarditis per 1000 (from 6 fewer to 5 fewer).

2. “One individual in Group 3 experienced symptoms indicating possible acute pericarditis according to protocol criteria (chest pain worsening when lying down) (Table 3). A thorough cardiac examination, including auscultation, ECG, Troponin I testing and echocardiography did not confirm the diagnosis. The echocardiography did not reveal any signs of pericardial effusion, pericardial rub, ECG changes suggestive of pericarditis, Troponin I increased or decreased exercise capacity. A detailed laboratory examination revealed a positive serology for Coxsackie B virus in temporal relation to the reported chest pain, suggesting a possible acute viral infection as the potential cause of the symptoms.” Overton 2008 (#106)

References in this table:12345678910111213141516171819202122232425262728293031323334353637383940

Certainty assessment № of patients Effect Certainty Importance
№ of studies Study design Risk of bias Inconsistency Indirectness Imprecision Other considerations JYNNEOS OPXV vaccine primary series ACAM2000 OPXV vaccine primary series Relative
(95% CI)
Absolute
(95% CI)
A. Prevention of disease (assessed with: geometric mean titer)
21,2,3,4,5,6 randomized trials not serious not serious seriousa,b not serious none 213 199 MD 1.62 titer units higher
(1.32 higher to 1.99 higher)c
Level 2
Moderate
CRITICAL
A. Prevention of disease (assessed with: seroconversion rate)
21,2,3,4,5,6 randomized trials not serious not serious seriousb,d seriouse none 213/213 (100.0%) 192/199 (96.5%) RR 1.02
(0.99 to 1.05)
19 more per 1,000
(from 10 fewer to 48 more)
Level 3
Low
CRITICAL
B. Severity of disease (assessed with: maximum lesion area)
17 randomized trials seriousf not serious not seriousg very seriouse,h none 15/15 (100.0%)i 8/8 (100.0%) RR 1.00
(0.83 to 1.20)
0 fewer per 1,000
(from 170 fewer to 200 more)
Level 4
Very low
IMPORTANT
C. Serious adverse events (SAE) (assessed with: vaccine associated serious adverse event rate)
31,2,3,4,5,6,7 randomized trials not serious not serious not serious very seriousj none 0/269 (0.0%) 1/245 (0.4%) k RR 0.33
(0.01 to 7.70)
3 fewer per 1,000
(from 4 fewer to 27 more)
Level 3
Low
CRITICAL
C. Serious adverse events (SAE) (assessed with: vaccine related serious adverse event rate)
158,9,10,11,12,13,14,15,16,17,
18,19,20,21,22,23,24,25,26,27,
28,29,30,31,32,33,34,35,36,
37,38,39,40
observational studies seriousl not serious seriousm seriousn none 4/5237 (0.1%) o,p 3/873 (0.3%) q,r RR 0.22
(0.05 to 0.99)
3 fewer per 1,000
(from 3 fewer to 0 fewer)
Level 4
Very low
CRITICAL
D. Myo-/pericarditis (assessed with: myo-/pericarditis event rate)
31,2,3,4,5,6,7 randomized trials not serious not serious not serious very seriouss none 0/269 (0.0%) 0/245 (0.0%) not estimable Level 3
Low
IMPORTANT
D. Myo-/pericarditis (assessed with: myo-/pericarditis event rate)
1214,15,16,17,18,19,20,21,22,23,24,
25,26,27,28,29,30,31,32,33,
34,35,36,37,38,39
observational studies seriousl not serious seriousm not serious none 1/4938 (0.0%) t 5/875 (0.6%) u RR 0.040
(0.004 to 0.310)v
5 fewer per 1,000
(from 6 fewer to 4 fewer)
Level 4
Very low
IMPORTANT

Table 5: Summary of Evidence for Outcomes of Interest

Outcome Importance Included in profile Certainty
Prevention of disease Critical Yes Moderate
Severity of disease Important Yes Very low
Serious adverse events Critical Yes Low
Myo-/pericarditis Critical Yes Low
Minor adverse events Not important No N/A

Explanations

a. Geometric mean titer is an indirect measure of efficacy.

b. Frey study used Dryvax in the comparison group. For the immunogenicity outcomes we do not feel there would be a significant difference between the two live vaccines.

c. In order to calculate a mean difference and 95% CI, geometric mean data were transformed to arithmetic mean. The effect estimate was then transformed to geometric mean difference, which you see here.

d. Seroconversion rate is an indirect measure of efficacy.

e. 95% CI includes the potential for both meaningful benefit as well as meaningful harm.

f. Concerns for risk of bias due to attrition. The two groups that contributed data to the intervention and comparison for this outcome lost between 11 and 21% of participants at the time this outcome was assessed.

g. The ideal measure of disease severity is take maximum lesion area. This study, Parrino et al. 2007, reports the proportion of participants with an attenuated take lesion. Clinical difference between categorical (proportion of participants with attenuated take) vs. continuous measurement (take maximum lesion area) is minimal. We feel this won’t affect indirectness. See Parrino et al. 2007 for a description of lesion attenuation criteria.

h. Small sample size.

i. Additionally, for this outcome there are data from single-arm observational studies that provide data for the intervention. These data were not included in the GRADE table because there are no available comparison data in these studies. We naively pooled intervention data from 2 single-arm studies: 42/171 (24.6%) participants who received an MVA [0, 28] primary vaccine series (either MVA-BN or Acambis MVA “ACAM3000”) developed an attenuated take lesion after live vaccinia vaccine challenge (either ACAM2000 or Dryvax). Among the other participants in the two studies, 38/171 (22.2%) developed a full take lesion and 89/171 (52.0%) developed no take lesion. (Pittman et al. 2007, Seaman et al. 2010, Wilck et al. 2010)

j. The sample size is small and does not meet the optimal size to assess this outcome and suggest fragility of the estimate. Also, the 95% CI includes the potential for meaningful harm.

k. One vaccine-related SAE was experienced after Dryvax administration in the comparison group. The SAE was characterized by severe elevated liver enzymes 84 days after the first Dryvax vaccine. This was reported in the Parrino et al. 2007 study. This SAE was deemed “possibly related to vaccination.” No other information is available.

l. There are some concerns with selection bias.

m. Indirect comparison of naively pooled single-arm studies compared to a historical control.

n. Fragility suspected based on few events.

o. Serious adverse events were defined according to the standard FDA definition including: death, life-threatening illness, hospitalization (initial or prolonged), disability or permanent damage, congenital anomaly/birth defect, required intervention to prevent permanent impairment or damage, and other serious medical events. In addition, data was collected about any smallpox vaccine-specific adverse event: postvaccinial encephalitits, eczema vaccinatum, progressive vaccinia, and generalized vaccinia.

p. Vaccine related serious adverse events in the intervention group: 1) Extra ocular muscle paresis event in one person 8 days after second MVA-BN vaccination; deemed probably related by investigators. 2) Sarcoidosis event in one person during the 6 month follow up period; deemed related because causal relationship with vaccine could not be ruled out. 3) Acute myocardial infarction event in one person 117 days after the first MVA-BN dose. Deemed related to vaccination because no other reasonable etiology was found. 4) Pneumonia and pleurisy event in one person 1 day after second MVA-BN dose. Deemed “possibly but unlikely” to be associated with vaccination.

q. Vaccine related serious adverse events from historical data for the comparison. 1) One participant developed severe somatization disorder that was deemed definitely related to vaccination with ACAM2000. 2) One participant developed abnormal ECG changes that was deemed possibly related to vaccination. 3) One participant developed increased cardiac enzymes that was deemed probably related to vaccination. Reference: Rosenthal, S., Merchlinsky, M., & Chowdhury, M. (2007). VRBPAC Background Document: ACAM200 (Live vaccinia Virus Smallpox Vaccine). Trial number H-400-009.

r. Comparison data was drawn from historical data. In a phase III clinical trial for ACAM2000, 3/873 (0.34%) developed vaccine related serious adverse events after ACAM2000 administration.

s. Number of participants is not large enough to capture myopericarditis events.

t. “One individual in Group 3 experienced symptoms indicating possible acute pericarditis according to protocol criteria (chest pain worsening when lying down). A thorough cardiac examination, including auscultation, ECG, Troponin I testing and echocardiography did not confirm the diagnosis. The echocardiography did not reveal any signs of pericardial effusion, pericardial rub, ECG changes suggestive of pericarditis, Troponin I increase or decreased exercise capacity. A detailed laboratory examination revealed a positive serology for Coxsackie B virus in temporal relation to the reported chest pain, suggesting a possible acute viral infection as the potential cause of the symptoms.” Overton ET, Lawrence SJ, Wagner E, et al. Immunogenicity and safety of three consecutive production lots of the non replicating smallpox vaccine MVA: A randomized, double blind, placebo controlled phase III trial. PLoS ONE [Electronic Resource]. 2018;13(4):e0195897.

u. No comparison data was available from the systematic review. Comparison is drawn from historical data, a study reporting myopericarditis rate after ACAM2000 administration. Source: ACAM2000 package insert, FDA.

v. Number of decimal places increased to more accurately present lower limit of confidence interval.

View the complete list of GRADE evidence tables‎

  1. Pittman, P. R., Hahn, M., Lee, H. S., Koca, C., Samy, N., Schmidt, D., Hornung, J., Weidenthaler, H., Heery, C. R., Meyer, T. P. H., Silbernagl, G., Maclennan, J., Chaplin, P.. Phase 3 Efficacy Trial of Modified Vaccinia Ankara as a Vaccine against Smallpox. New England Journal of Medicine; 2019.
  2. Frey, S. E., Newman, F. K., Kennedy, J. S., Sobek, V., Ennis, F. A., Hill, H., Yan, L. K., Chaplin, P., Vollmar, J., Chaitman, B. R., Belshe, R. B.. Clinical and immunologic responses to multiple doses of IMVAMUNE (Modified Vaccinia Ankara) followed by Dryvax challenge. Vaccine; 2007.
  3. Sano, J., Chaitman, B. R., Swindle, J., Frey, S. E.. Electrocardiography screening for cardiotoxicity after modified Vaccinia Ankara vaccination. American Journal of Medicine; 2009.
  4. Bavarian Nordic, , United States Army Medical Research Institute of Infectious Diseases, . A Non-inferiority Trial to Compare MVA-BN® Smallpox Vaccine to ACAM2000®. 2015.
  5. Hughes, C. M., Newman, F. K., Davidson, W. B., Olson, V. A., Smith, S. K., Holman, R. C., Yan, L., Frey, S. E., Belshe, R. B., Karem, K. L., Damon, I. K.. Analysis of variola and vaccinia virus neutralization assays for smallpox vaccines. Clinical & Vaccine Immunology: CVI; 2012.
  6. National Institute of Allergy and Infectious Diseases, . Combination Study With MVA BN and Dryvax. 2004.
  7. Parrino, J., McCurdy, L. H., Larkin, B. D., Gordon, I. J., Rucker, S. E., Enama, M. E., Koup, R. A., Roederer, M., Bailer, R. T., Moodie, Z., Gu, L., Yan, L., Graham, B. S.. Safety, immunogenicity and efficacy of modified vaccinia Ankara (MVA) against Dryvax challenge in vaccinia-naive and vaccinia-immune individuals. Vaccine; 2007.
  8. National Institute of Allergy and Infectious Diseases, . Lyophilized IMVAMUNE® (1×10^8 TCID50) Versus Liquid IMVAMUNE® (1×10^8 TCID50) Administered Subcutaneously and a Lower Dose Liquid IMVAMUNE® (2×10^7 TCID50) Administered Intradermally. 2010.
  9. Frey, S. E., Wald, A., Edupuganti, S., Jackson, L. A., Stapleton, J. T., El Sahly, H., El-Kamary, S. S., Edwards, K., Keyserling, H., Winokur, P., Keitel, W., Hill, H., Goll, J. B., Anderson, E. L., Graham, I. L., Johnston, C., Mulligan, M., Rouphael, N., Atmar, R., Patel, S., Chen, W., Kotloff, K., Creech, C. B., Chaplin, P., Belshe, R. B.. Comparison of lyophilized versus liquid modified vaccinia Ankara (MVA) formulations and subcutaneous versus intradermal routes of administration in healthy vaccinia-naive subjects. Vaccine; 2015.
  10. Lyophilized IMVAMUNE® (1×10^8 TCID50) Versus Liquid IMVAMUNE® (1×10^8 TCID50) Administered Subcutaneously and a Lower Dose Liquid IMVAMUNE® (2×10^7 TCID50) Administered Intradermally. https://clinicaltrials.gov/show/NCT00914732; 2009.
  11. Anderson, E. J., Lai, L., Wrammert, J., Kabbani, S., Xu, Y., Priyamvada, L., Hill, H., Goll, J. B., Jensen, T. L., Kao, C., Yildirim, I., Rouphael, N., Jackson, L., Mulligan, M. J.. Plasmablast, Memory B Cell, CD4+ T Cell, and Circulating Follicular Helper T Cell Responses to a Non-Replicating Modified Vaccinia Ankara Vaccine. 2020.
  12. Jackson, L. A., Frey, S. E., El Sahly, H. M., Mulligan, M. J., Winokur, P. L., Kotloff, K. L., Campbell, J. D., Atmar, R. L., Graham, I., Anderson, E. J., Anderson, E. L., Patel, S. M., Fields, C., Keitel, W., Rouphael, N., Hill, H., Goll, J. B.. Safety and immunogenicity of a modified vaccinia Ankara vaccine using three immunization schedules and two modes of delivery: A randomized clinical non-inferiority trial. Vaccine; 2017.
  13. Phase II Trial to Assess Safety and Immunogenicity of IMVAMUNE®. https://clinicaltrials.gov/show/NCT01827371; 2013.
  14. MVA Post-Event: administration Timing and Boost Study. https://clinicaltrials.gov/show/NCT00437021; 2007.
  15. Troy, J. D., Hill, H. R., Ewell, M. G., Frey, S. E.. Sex difference in immune response to vaccination: A participant-level meta-analysis of randomized trials of IMVAMUNE smallpox vaccine. Vaccine; 2015.
  16. Frey, S. E., Winokur, P. L., Salata, R. A., El-Kamary, S. S., Turley, C. B., Walter, E. B.,Jr., Hay, C. M., Newman, F. K., Hill, H. R., Zhang, Y., Chaplin, P., Tary-Lehmann, M., Belshe, R. B.. Safety and immunogenicity of IMVAMUNE R smallpox vaccine using different strategies for a post event scenario. Vaccine; 2013.
  17. Bavarian Nordic, . A Phase II Trial to Compare a Liquid-frozen and a Freeze-dried Formulation of IMVAMUNE (MVA-BN®) Smallpox Vaccine in Vaccinia-naïve Healthy Subjects. 2013.
  18. A Phase II Trial to Compare a Liquid-frozen and a Freeze-dried Formulation of IMVAMUNE (MVA-BN®) Smallpox Vaccine in Vaccinia-naïve Healthy Subjects. https://clinicaltrials.gov/show/NCT01668537; 2012.
  19. Frey, S. E., Winokur, P. L., Hill, H., Goll, J. B., Chaplin, P., Belshe, R. B.. Phase II randomized, double-blinded comparison of a single high dose (5×10(8) TCID50) of modified vaccinia Ankara compared to a standard dose (1×10(8) TCID50) in healthy vaccinia-naive individuals. Vaccine; 2014.
  20. Greenberg, R. N., Hurley, Y., Dinh, D. V., Mraz, S., Vera, J. G., Von Bredow, D., Von Krempelhuber, A., Roesch, S., Virgin, G., Arndtz-Wiedemann, N., Meyer, T. P., Schmidt, D., Nichols, R., Young, P., Chaplin, P.. A multicenter, open-label, controlled phase II study to evaluate safety and immunogenicity of MVA smallpox vaccine (IMVAMUNE) in 18-40 year old subjects with diagnosed atopic dermatitis. PLoS ONE; 2015.
  21. Bavarian Nordic, , National Institute of Allergy and Infectious Diseases, . A Phase II Study on Immunogenicity and Safety of MVA-BN® (IMVAMUNE™) Smallpox Vaccine in Subjects With Atopic Dermatitis. 2006.
  22. Zitzmann-Roth, E. M., von Sonnenburg, F., de la Motte, S., Arndtz-Wiedemann, N., von Krempelhuber, A., Uebler, N., Vollmar, J., Virgin, G., Chaplin, P.. Cardiac safety of Modified Vaccinia Ankara for vaccination against smallpox in a young, healthy study population. PLoS ONE [Electronic Resource]; 2015.
  23. A partially randomized, partially double-blind, placebo-controlled Phase II non-inferiority study to evaluate immunogenicity and safety of one and two doses of MVA-BN (IMVAMUNE™) smallpox vaccine in 18-55 year old healthy subjects. http://www.who.int/trialsearch/Trial2.aspx?TrialID=EUCTR2005-001781-14-DE; 2006.
  24. A Randomized, Double-blind, Placebo-controlled Study on Immunogenicity and Safety of MVA-BN (IMVAMUNE™) Smallpox Vaccine in Healthy Subjects. https://clinicaltrials.gov/show/NCT00316524; 2006.
  25. Greenberg, R. N., Overton, E. T., Haas, D. W., Frank, I., Goldman, M., von Krempelhuber, A., Virgin, G., Bädeker, N., Vollmar, J., Chaplin, P.. Safety, immunogenicity, and surrogate markers of clinical efficacy for modified vaccinia Ankara as a smallpox vaccine in HIV-infected subjects. J Infect Dis; Mar 1 2013.
  26. Bavarian Nordic, , National Institute of Allergy and Infectious Diseases, . Safety, Tolerability and Immune Response of IMVAMUNE (MVA-BN)Smallpox Vaccine in HIV Infected Patients. 2005.
  27. Overton, E. T., Lawrence, S. J., Wagner, E., Nopora, K., Rosch, S., Young, P., Schmidt, D., Kreusel, C., De Carli, S., Meyer, T. P., Weidenthaler, H., Samy, N., Chaplin, P.. Immunogenicity and safety of three consecutive production lots of the non replicating smallpox vaccine MVA: A randomised, double blind, placebo controlled phase III trial. PLoS ONE [Electronic Resource]; 2018.
  28. Bavarian Nordic, . A Trial to Evaluate Immunogenicity and Safety of Three Consecutive Production Lots of IMVAMUNE® (MVA-BN®) Smallpox Vaccine in Healthy, Vaccinia-naïve Subjects. 2013.
  29. A Trial to Evaluate Immunogenicity and Safety of Three Consecutive Production Lots of IMVAMUNE® (MVA-BN®) Smallpox Vaccine in Healthy, Vaccinia-naïve Subjects. https://clinicaltrials.gov/show/NCT01144637; 2010.
  30. Darsow, U., Sbornik, M., Rombold, S., Katzer, K., von Sonnenburg, F., Behrendt, H., Ring, J.. Long-term safety of replication-defective smallpox vaccine (MVA-BN) in atopic eczema and allergic rhinitis. Journal of the European Academy of Dermatology and Venereology; 2016.
  31. von Sonnenburg, F., Perona, P., Darsow, U., Ring, J., von Krempelhuber, A., Vollmar, J., Roesch, S., Baedeker, N., Kollaritsch, H., Chaplin, P.. Safety and immunogenicity of modified vaccinia Ankara as a smallpox vaccine in people with atopic dermatitis. Vaccine; 2014.
  32. ACAM 3000 MVA at Harvard Medical School. https://clinicaltrials.gov/show/NCT00133575; 2005.
  33. Wilck, M. B., Seaman, M. S., Baden, L. R., Walsh, S. R., Grandpre, L. E., Devoy, C., Giri, A., Kleinjan, J. A., Noble, L. C., Stevenson, K. E., Kim, H. T., Dolin, R.. Safety and immunogenicity of modified vaccinia Ankara (ACAM3000): effect of dose and route of administration. Journal of Infectious Diseases; 2010.
  34. Seaman, M. S., Wilck, M. B., Baden, L. R., Walsh, S. R., Grandpre, L. E., Devoy, C., Giri, A., Noble, L. C., Kleinjan, J. A., Stevenson, K. E., Kim, H. T., Dolin, R.. Effect of vaccination with modified vaccinia Ankara (ACAM3000) on subsequent challenge with Dryvax. Journal of Infectious Diseases; 2010.
  35. Overton, E. T., Lawrence, S. J., Stapleton, J. T., Weidenthaler, H., Schmidt, D., Koenen, B., Silbernagl, G., Nopora, K., Chaplin, P.. A randomized phase II trial to compare safety and immunogenicity of the MVA-BN smallpox vaccine at various doses in adults with a history of AIDS. Vaccine; Mar 4 2020.
  36. Bavarian Nordic, . Randomized, Open-label Phase II Trial to Assess the Safety and Immunogenicity of MVA-BN Smallpox Vaccine in Immunocompromised Subjects With HIV Infection. 2014.
  37. Bavarian Nordic, , National Institutes of Health, . Safety and Immunogenicity of IMVAMUNE® (MVA-BN®) Smallpox Vaccine in HIV Infected Patients. 2006.
  38. Overton, E. T., Stapleton, J., Frank, I., Hassler, S., Goepfert, P. A., Barker, D., Wagner, E., von Krempelhuber, A., Virgin, G., Meyer, T. P., Muller, J., Badeker, N., Grunert, R., Young, P., Rosch, S., Maclennan, J., Arndtz-Wiedemann, N., Chaplin, P.. Safety and Immunogenicity of Modified Vaccinia Ankara-Bavarian Nordic Smallpox Vaccine in Vaccinia-Naive and Experienced Human Immunodeficiency Virus-Infected Individuals: An Open-Label, Controlled Clinical Phase II Trial. Open Forum Infectious Diseases; 2015.
  39. von Krempelhuber, A., Vollmar, J., Pokorny, R., Rapp, P., Wulff, N., Petzold, B., Handley, A., Mateo, L., Siersbol, H., Kollaritsch, H., Chaplin, P.. A randomized, double-blind, dose-finding Phase II study to evaluate immunogenicity and safety of the third generation smallpox vaccine candidate IMVAMUNE. Vaccine; 2010.
  40. Vollmar J, Arndtz N, Eckl KM, et al. Safety and immunogenicity of IMVAMUNE, a promising candidate as a third generation smallpox vaccine. Vaccine. 2006;24(12):2065-70.