Factors That Influence Vaccine Effectiveness

Background

Different outcomes evaluated in vaccine effectiveness studies

Vaccine effectiveness is a measure of how well flu vaccines work among different groups of people, in different settings, and in different real-world conditions (as opposed to randomized controlled trials (RCTs) or "clinical trials). Public health researchers evaluate the benefits of vaccination against illness of varying severity, from mild-to-moderate illness resulting in a doctor visit to more severe illness resulting in hospitalization, ICU admission, and even death. Different studies are used to measure different outcomes, and estimates of vaccine effectiveness may vary based on the outcome measured (as well as other factors, such as the population studied).

Virus factors

Vaccine effectiveness may vary based on how similar or different the vaccine virus is to circulating flu viruses. The more similar the flu vaccine is to circulating viruses, the more likely it is to offer robust protection against flu illness and complications. However, research has shown that even when the viruses are very different, vaccination can still provide as much as 30% reduction in risk123456. (More information is available at How the flu virus can change: "Drift" and "Shift".)

Host factors

Host factors refer to characteristics of the vaccinated person, including, for example, their age, underlying medical conditions, history of prior influenza virus infection, and prior flu vaccinations. All of these factors can affect how well vaccines work.

Study design factors

RCTs provide the most reliable results because they are less susceptible to selection bias and confounding. However, RCTs may be difficult or unethical to conduct when vaccination is universally recommended in a population or for more severe outcomes that are less common. There are several observational study designs; however, many flu vaccine evaluation programs currently use the test-negative design. In the test-negative design, people who seek care for an acute respiratory illness are enrolled at care settings (such as outpatient clinics, urgent care clinics, emergency departments, or in-patient settings), and information is collected about the patients’ flu vaccination status. All participants in a test-negative design study are tested for flu using a highly specific and sensitive test for influenza virus infection, such as reverse transcription polymerase chain reaction (RT-PCR). The ratio of vaccinated to unvaccinated persons (i.e., the odds of flu vaccination) is compared among patients with and without laboratory-confirmed flu. In this way, a test-negative design study estimates VE by comparing vaccination rates among persons with confirmed flu illness (also called “cases”) against vaccination rates among persons with similar illness who do not have flu (also called “controls”) based on laboratory tests. The test-negative design reduces selection bias due to health care seeking behaviors.

Specificity of the outcome

Factors related to measuring specific versus non-specific outcomes

For both RCTs and observational studies, the specificity of the outcome measured in the study is important. Non-specific outcomes, such as pneumonia hospitalizations or influenza-like illness (ILI), can be caused by influenza virus infections or infections with other viruses and bacteria. Vaccine efficacy/effectiveness estimates for non-specific outcomes are generally lower than estimates made for more specific outcomes, depending on what proportion of the outcome measured is attributable to flu. For example, a study among healthy adults found that inactivated flu vaccine (i.e., a flu shot) was 86% effective against laboratory-confirmed flu, but only 10% effective against all respiratory illnesses in the same population and season⁷. When assessing vaccine efficacy/effectiveness, the most specific outcome is laboratory-confirmed influenza virus infection by RT-PCR or viral culture.