Evaluating Vaccination Coverage Trends with the National Immunization Survey-Child (NIS-Child), 2013-2017, United States: Technical Appendix

Executive Summary

• Each year since 1994, the Centers for Disease Control and Prevention (CDC) has collected data to estimate vaccination coverage among U.S. children 19-35 months of age to assess performance of national, state, territorial, and selected local area immunization programs. CDC collects these data through the National Immunization Survey-Child (NIS-Child).
• Starting in 2017, we added a comparison of vaccination coverage by age 19 months for birth cohorts included in both the current and prior year surveys (the “bridging birth cohort”) to our routine accuracy checks. By restricting analysis to children in this bridging birth cohort, and limiting to vaccinations received before 19 months of age, the vaccination coverage estimates from the current and prior survey year should be the same, except for random differences due to sampling. For this reason, we expect to find no difference in vaccination coverage between the estimates from the prior and current survey year, for children in the bridging birth cohort.
• We did not find evidence for change in survey accuracy from 2016 to 2017. For all vaccinations evaluated, there were no statistically significant differences in vaccination coverage by age 19 months from 2016 to 2017 among the bridging birth cohort.
• We assessed trends in vaccination coverage by month and year of birth using NIS-Child data from 2013–2017. Among children born during January 2012 through January 2016, we found that vaccination coverage was stable for most vaccinations. Coverage by 24 months of age decreased by 0.5 percentage points per 12 birth months for ≥ 3 doses of hepatitis B vaccine, and increased by 1.1 percentage points per 12 birth months for ≥ 2 doses of hepatitis A vaccine.
• CDC plans to report vaccination coverage from NIS-Child by annual birth cohorts in future reports.

Introduction

Details regarding the statistical methodology of NIS-Child are available in the NIS-Child Data User’s Guide 2016. Starting in 2017, we added an analysis of the current and prior survey year data to assess changes in survey accuracy, by examining vaccination coverage among children who were eligible for inclusion in both survey years, i.e. the “bridging birth cohort” to our routine accuracy checks. This approach for assessing changes in accuracy of NIS-Child estimates was developed in 2015 (1) and was added to routine data quality checks in 2017 (2). In 2017, we also evaluated how to interpret differences in vaccination coverage estimates by survey year and assessed national trends in vaccination coverage by month and year of birth (2).

We describe below updated findings of these evaluations based on analysis of NIS-Child data from 2013 through 2017.

Findings

Bridging Birth Cohort Analysis to Assess Change in Survey Accuracy

Specifically for the 2016 and 2017 NIS-Child data, we compared estimates of vaccination coverage by age 19 months for children born during January 2014 through May 2015, the bridging birth cohort included in both the most recent survey year and the prior survey year. Differences in vaccination coverage by age 19 months among this bridging birth cohort could signal a change in survey bias, a systematic difference that could affect validity of vaccination coverage estimates. We conducted national bridging birth cohort analysis for survey years 2016-2017, for fifteen different vaccination measures. We estimated vaccination coverage by 19 months (before the day the child reached 19 months of age) within each survey year by month and year of birth. We then computed the average difference of these vaccination coverage estimates within each of the 17 monthly birth cohorts (January 2014- May 2015) included in the bridging cohort across the two survey years.

For the 15 vaccinations assessed, differences in average vaccination coverage among the bridging cohort ranged from -1.9 to 2.2 percentage points, and none was statistically significant (Table 1 [1 page, 508]).

Interpreting Differences in Annual NIS-Child Estimates

As previously described (2), we divided the overall difference in annual NIS-Child estimates for 2017 compared to 2016 into four parts, with contributions from the non-overlapping and overlapping (bridging) birth cohorts further split by age at vaccination (vaccination by age 19 months and vaccination at or after age 19 months). This categorization shows how each component contributes to the overall difference in vaccination coverage estimates. The contribution from the bridging birth cohort for vaccination by age 19 months does not reflect change in vaccination over time because it compares two independent estimates of the same birth cohorts of children by the same age. The contribution from the non-overlapping birth cohorts for vaccination by age 19 months would reflect changes in vaccination coverage over time.

For each of 16 vaccination measures (including the percent of children with no vaccinations), we showed how the overall estimated difference in vaccination coverage between survey years 2017 and 2016 resulted from differences in estimates among the bridging birth cohort and non-overlapping birth cohorts (Table 2 [2 pages, 508]). Two of the vaccination measures had overall statistically significant increases from 2016 to 2017; coverage with the hepatitis B birth dose increased by 2.5 percentage points, and the percent of children with no vaccinations increased by 0.3 percentage points. For each of these vaccinations, the contribution to the overall difference from the non-overlapping cohort and bridging cohort were approximately the same.

Assessing Trends in Vaccination Coverage by Month and Year of Birth

To more directly assess changes in vaccination coverage (2), we estimated linear trends in national vaccination coverage by month and year of birth, using NIS-Child data from 2013–2017 (Table 3 [2 pages, 508], Figures 1-16 [4 MB, 16 pages]).

For each of 13 vaccination measures, we estimated national coverage by age 24 months by month and year of birth. Hepatitis B birth dose was assessed by 3 days of birth, and rotavirus vaccination was assessed by age 8 months. We also estimated coverage with ≥2 doses of hepatitis A vaccine by 35 months. We used the Kaplan-Meier method to account for censoring of vaccination status after age 19 months for children assessed for vaccination status before reaching age 24 or 35 months. We assessed linear trends over births from January 2012 through January 2016 for most vaccinations, through May 2016 for the hepatitis B birth dose and rotavirus vaccination, and through February 2015 for 2 or more doses of hepatitis A vaccination by age 35 months. We also assessed more recent linear trends including children born from January 2014 and later. We did not asses the trend by month and year of birth for the percent of children unvaccinated by 24 months because of imprecision resulting from low prevalence.

Our analysis included estimates of vaccination coverage from two to three survey years for the same monthly birth cohort. We treated the estimates from each survey year as independent samples.

Among children born during January 2012 and later, estimated linear trends by month and year of birth indicated stable coverage for most vaccinations. Coverage declined by 0.5 percentage points per 12 monthly birth cohorts for the primary series of Haemophilus influenzae type b conjugate vaccine and ≥3 doses of hepatitis B vaccine. Coverage increased by 1.1 percentage points per 12 monthly birth cohorts for ≥2 doses of hepatitis A vaccine by age 24 months.

Among children born January 2014 and later, coverage was stable except for an increase of 3.0 percentage points per 12 monthly birth cohorts for the birth dose of hepatitis B vaccine. The statistically significant increase in hepatitis B birth dose coverage by survey year, from 2016 to 2017, may reflect this observed increase among the more recent birth cohorts.

Conclusions

Routine annual data quality checks did not identify any signals of change in accuracy of the NIS-Child national vaccination coverage estimates from 2017 compared to 2016. When we assessed trends in coverage by month and year of birth, we found stable coverage over the births from January 2012 and later. There were annual changes of one percentage point or less for coverage by age 24 months with ≥3 doses of hepatitis B vaccine (a decline), and with ≥2 doses of hepatitis A vaccine (an increase).  Limitations of our approach to analyzing these data have been discussed previously (2),

Both the traditional comparison of annual estimates and the birth cohort analysis generally indicate that childhood vaccination coverage is high and stable. The observed increases or decreases found from either approach were small differences of less than one to three percentage points.

CDC plans to report NIS-Child estimates by year of birth in future reports.

Authors:

James A. Singleton, PhD, David Yankey, PhD, Holly A. Hill, MD, PhD, Laurie Elam-Evans, PhD, Immunization Services Division, National Center for Immunization and Respiratory Diseases, Centers for Disease Control and Prevention

Benjamin Fredua, MS, Leidos

References

1. Yankey D, Hill HA, Elam-Evans LD, Khare M, Singleton JA, Pineau V, Wolter K. Estimating change in telephone survey bias in an era of declining response rates and transition to wireless telephones – evidence from the National Immunization Survey (NIS), 1995-2013. Presented at the American Association of Public Opinion Research (AAPOR) 70^th Annual Conference, May 15, 2015, Hollywood, FL.
2. Evaluating vaccination coverage trends with the National Immunization Survey—Child (NIS-Child), 2012–2016, United States. Atlanta, GA: US Department of Health and Human Services, CDC; 2018.