Quality Improvement Coaching for Human Papillomavirus Vaccination Coverage: A Process Evaluation in 3 States, 2018–2019

Jennifer Leeman, MDiv, MPH, DrPH^1,2; Victoria Petermann, RN, BSN¹; Jennifer Heisler-MacKinnon, MPH³; Adam Bjork, PhD^4,5; Noel T. Brewer, PhD^2,3; Brigid K. Grabert, JD, PhD^2,3; Melissa B. Gilkey, PhD^2,3 (View author affiliations)

Suggested citation for this article: Leeman J, Petermann V, Heisler-MacKinnon J, Bjork A, Brewer NT, Grabert BK, et al. Quality Improvement Coaching for Human Papillomavirus Vaccination Coverage: A Process Evaluation in 3 States, 2018–2019. Prev Chronic Dis 2020;17:190410. DOI: http://dx.doi.org/10.5888/pcd17.190410.

PEER REVIEWED

Abstract

Purpose and Objectives

Quality improvement (QI) coaching improves human papillomavirus (HPV) vaccination coverage, but effects of coaching have been small, and little is known about how and when QI coaching works. To assess implementation outcomes and explore factors that might explain variation in outcomes, we conducted a process evaluation of a QI coaching intervention for HPV vaccination.

Intervention Approach

QI coaches received tools and training to support 4 core coaching competencies: 1) expertise in using clinic-level adolescent vaccination data to drive change, 2) knowledge of the evidence base to support change in HPV vaccination practice, 3) familiarity with improvement strategies and action planning, and 4) skill in building relationships.

Evaluation Methods

Our mixed methods evaluation involved collecting quantitative data through effort-tracking logs and gathering qualitative data through in-depth interviews with QI coaches (N = 11) who worked with 89 clinics in 3 US states. Data were collected on implementation outcomes and on contextual factors that might explain variations in those outcomes. Implementation outcomes included adoption by clinics, reach to providers and staff (ie, participation in the coaching visit), and implementation fidelity.

Results

States achieved either high adoption or high reach, but not both. For example, state A had high adoption with 94% of clinics accepting a coaching visit, but low reach with a median of 1 participant per clinic. In contrast, state C had lower adoption (29%, P < .01) than state A but higher reach (median of 4 participants per clinic, P < .01). Generally, states had high coaching protocol fidelity with the exception of advising on strategies and action planning. QI coaches described factors that might explain these variations, including strength of relationships with clinic staff and whether they recruited clinics directly or through large clinic networks.

Implications for Public Health

Our findings have implications for the design of future QI coaching initiatives, including how coaches recruit clinics to ensure full clinic engagement, refinements to coaching visits, and how QI coaches can effectively engage with clinic networks. Findings could inform future QI coaching interventions to strengthen their impact on public health.

Top

Introduction

Persistent human papillomavirus (HPV) infection leads to over 34,000 new cancer diagnoses per year in the United States (1). HPV vaccination is highly effective at preventing HPV cancers, yet only 51% of US adolescents aged 13–17 have received the recommended number of doses (2). The Centers for Disease Control and Prevention (CDC) provides funding for immunization quality improvement (QI) coaching to 61 state, local, and territorial immunization programs, with the goal of increasing immunization rates for routinely recommended vaccines, including HPV vaccine (3). CDC provided this coaching through the AFIX (Assessment, Feedback, Incentives, and eXchange) program, which was replaced by IQIP (Immunization Quality Improvement for Providers) in July 2019. The AFIX program engaged staff in state and regional health departments to deliver QI coaching to improve clinics’ immunization practices (2). QI coaching, also referred to as “practice facilitation,” is an evidence-based implementation strategy defined as a process of interactive problem solving and support that occurs in the context of a recognized need for improvement and a supportive interpersonal relationship (4).

Researchers at the University of North Carolina (UNC) developed an intervention that provides tools and training to support the delivery of QI coaching to improve clinics’ HPV vaccination rates (5). In a previous study, we found that the intervention demonstrated a small but significant improvement in HPV vaccination rates in clinics that received HPV vaccination QI coaching, as compared with those that did not (6,7). In the evaluation presented here, we explore factors that influenced the implementation of HPV vaccination QI coaching, with the goal of further refining the intervention.

Top

Purpose and Objectives

The UNC research team sought to compare the effectiveness of HPV vaccination QI coaching to clinical medical education by conducting a randomized controlled trial in primary care clinics in 3 states (clincaltrials.gov NCT 5108275). The purpose of this process evaluation was to assess the implementation outcomes for the QI coaching aspect of the trial and explore contextual factors that might explain variations in implementation outcomes across the states.

Top

Intervention Approach

The coaching intervention provided tools and training to support QI coaches who worked in CDC-funded AFIX programs in state or regional departments of public health. AFIX-based QI coaches typically were trained in public health or nursing and made an annual 1–2 hour, in-person coaching visit to a subset of primary care clinics in their geographic regions, following the in-person visit with additional coaching by email or telephone. CDC originally designed the AFIX program to improve vaccination coverage for infants and young children and only later expanded it to improve adolescent vaccination coverage. Given persistently low rates of HPV vaccination, the goal of the HPV QI coaching intervention was to provide coaches with additional training and tools to improve HPV vaccination rates for adolescents.

QI coaching is a widely tested implementation strategy. In a systematic review of 22 studies involving 1,429 clinics, reviewers found that clinics that received QI coaching were more likely to adopt evidence-based guidelines than those that did not (8). The activities involved in QI coaching vary across studies but generally require the 4 core competencies of 1) expertise in using data to drive change, 2) knowledge of the evidence base that drives the change in practice, 3) familiarity with available strategies to implement change, and 4) skills in relationship building (9). UNC’s QI coaching tools for HPV are freely available on the project website (https://www.hpviq.org). The tools are designed to support the 4 core coaching competencies and include an immunization report card template to increase QI coach competence in using data to drive change, a PowerPoint presentation to support clinic education on the evidence base driving the change in practice, a menu of recommended improvement strategies to implement change, and protocols for guiding clinics in action planning to initiate change.

Immunization report card template. The report card template provided QI coaches with a tool that supports their efforts to use data to drive change (Figure). Coaches used the template to translate data from the state’s immunization registry into a report card that provides feedback on a clinic’s current coverage or percentage of adolescents vaccinated for HPV, as compared with 2 other adolescent vaccines — meningococcal conjugate vaccine and the tetanus toxoid, reduced diphtheria toxoid, and acellular pertussis (Tdap) booster. Assessing and providing feedback on performance is an evidence-based implementation strategy. A Cochrane review found that assessment and feedback generally led to modest (4%–7%) clinically important improvements in pediatric health outcomes (10). Coaches also used the report card to prompt clinics to set specific goals for improvement.

Figure.
Template for coaches’ immunization report card. Source: https://www.hpviq.org. [A text version of this figure is also available.]

Presentation. Coaches received an HPV PowerPoint slide set to present to clinic providers and staff. The 28-slide presentation began with an overview of the evidence base in support of HPV vaccination as an effective means of preventing cancer in both males and females. The presentation then guided coaches through each step of the QI coaching session, including reviewing the report card, setting a goal for improvement, selecting strategies to improve HPV vaccination, and creating an action plan.

Menu of recommended improvement strategies. CDC’s standard AFIX protocols provided QI coaches with a list of 19 improvement strategies that varied in feasibility and potential to increase HPV vaccination rates in clinics. The research team reviewed the list and identified strategies with the greatest potential for feasibility and improvements, based on input from academic and practice-based stakeholders with expertise in HPV vaccination and the AFIX program. As a result, the coaching protocols specified 1 primary strategy (advising providers to recommend same-day HPV vaccination for all patients aged ≥11 years) and 3 secondary strategies (reviewing CDC guidelines with all immunization staff, training front desk staff on scheduling, and establishing standing orders).

Protocols for guiding clinics in action planning. The PowerPoint presentation included slides that prompted participants to identify who is responsible for specific vaccination roles in their clinic, such as scheduling appointments, reviewing and flagging charts, and prescribing vaccines. It also prompted participants to start planning how they would work with people identified to improve clinic vaccination coverage.

AFIX encouraged QI coaches to invite additional clinic staff to participate in the 1-time coaching visit. To increase participation and motivate providers to attend, the intervention offered clinical medical education (CME) credits.

To support QI coaches in using the tools and protocols, the research team engaged 8 coaches and 5 of their supervisors in a 2-day, in-person training. The research team then engaged coaches in weekly conference calls to review coaching and data collection protocols and to solve problems that challenged implementation as they were encountered.

Top

Evaluation Methods

This mixed methods evaluation involved the collection of quantitative data through effort-tracking logs, and qualitative data by using in-depth interviews with QI coaches who worked in AFIX programs in 3 states, from 2018 to 2019. The University of North Carolina Institutional Review Board reviewed the trial and classified it as exempt.

We selected 3 states to participate in the trial on the basis of their geographic diversity, robust state immunization registries, active AFIX programs, and interest of AFIX program staff. These states were in the southwest (state A), the northeast (state B), and the midwest (state C). Baseline HPV vaccine initiation coverage (for patients aged 13–17 y) was 67.2% for state A, 69.8% for state B, and 64.1% for state C (2).

In each state, clinics were eligible for inclusion if they were pediatric or family medicine practices, Vaccines for Children providers, had between 200 and 7,000 patients (aged 11–17 y), and had baseline HPV vaccine initiation coverage below 85%. The federally funded Vaccines for Children program provides free vaccines to high-priority populations, including children who are under- or uninsured. Clinics were ineligible if they belonged to a network with over 30 clinics or were pharmacies or school health clinics. States A and C conducted work statewide, while state B worked in 3 large counties. AFIX visits were open to providers, other members of the primary care team including nurses and medical assistants, and administrative staff.

We collected data on the implementation outcomes of clinic adoption, providers and staff reach, and QI coaching fidelity (11), and on contextual factors that might explain variations in those outcomes. AFIX staff in each state maintained a tracking log of the number and proportion of clinics that agreed to participate (ie, adoption) and the number of staff and providers who participated in each visit (ie, reach). Providers were physicians, nurse practitioners, and physician assistants. All other participants were classified as staff. A researcher conducted in-depth phone interviews with QI coaches in each state to assess how they used the HPV coaching tools (fidelity) and to explore contextual factors that might have contributed to variations in implementation outcomes. Interviews followed a semi-structured interview guide that queried coaches about their experiences recruiting clinics and providers for participation in an HPV QI coaching visit and how they delivered QI coaching. The focus of the guide was the coaches’ use of recommended tools and fidelity to protocols.

For quantitative data, we compared adoption and reach among the 3 states. We determined if adoption rates varied across states by using logistic regressions followed by Wald tests. We determined if reach to providers and staff (number per clinic who attended AFIX visit) varied across states by comparing rank means using the Kruskal–Wallis and Dunn tests because of skewing of data. Because of the skewed nature of those data, we report medians rather than means for reach. For qualitative data, we recorded interviews, with consent of participants, and transcribed them. We used content analysis to analyze data (12). We developed a set of codes for implementation outcomes and contextual factors that influenced outcomes. For implementation outcomes, codes specified performance of core components of HPV QI coaching, including distributing the clinic report card, setting a goal for improvement, sharing the presentation, selecting improvement strategies, and creating an action plan. During the coding process, codes were developed as needed to fully capture all relevant contextual factors. Coders used ATLAS.ti (Scientific Software Development GmbH), a qualitative software management program, to code the interviews. Coders met to compare and reconcile coding. Once coding was complete, data were put into a matrix that organized findings by QI coach and state, and the research team identified themes related to factors influencing implementation outcomes and how those factors varied across states.

Top

Results

The total sample for our process evaluation (N = 11) consisted of 9 AFIX QI coaches and 2 supervisors who provided HPV QI coaching to 89 clinics in 3 states. State A had 2 QI coaches with a mean of 11.0 years of experience (range, 4–18 years), state B had 5 QI coaches with a mean of 2.3 years of experience (range, 1–6 years), and state C had 2 QI coaches with a mean of 2.3 years of experience (range, 1.5–3 years).

Implementation Outcomes

• Clinic adoption of HPV QI coaching. Overall, 63% of invited clinics agreed to participate in QI coaching and completed a coaching visit. Adoption was higher in states A and B, as compared with state C (both P < 0.01), with 30 of 32 (94%) clinics adopting in state A, 40 of 44 clinics adopting in state B (91%), and 19 of 65 (29%) adopting in state C.
• Staff and provider reach. A median of 2 providers and other staff from clinics participated in the QI coaching visit. The total number of participants varied across all 3 states. AFIX visits in state A had a median of 1 participant per visit, and all were staff. State B had a median of 2 participants (1 staff and 2 providers), and state C had a median of 4 participants (4 staff and 1 provider).
• QI coach fidelity to coaching protocols. QI coaches consistently reported that they used the report card to provide feedback on clinic vaccination data and worked with clinics to set a goal for improving vaccination rates over the next 6 months. Coaches in states B and C reviewed the presentation with participants in the coaching visit. In state A, coaches converted the information into a 1-page handout that they reviewed with participants. Across all 3 states, coaches reported that they worked with clinics to select specific strategies for improving their HPV vaccination coverage. However, coaches did not consistently promote the short list of recommended strategies from the coaching protocol. When interviewed, one-third of QI coaches could not name any of the 4 recommended strategies, and most coaches continued to recommend strategies from the longer list of 19 from the standard AFIX protocol. Only the QI coaches in state C reported conducting action planning with clinics and reported that it was fairly limited. QI coaches in states A and B reported conducting no action planning (Table 1).

Factors that might explain variations in implementation outcomes

Analysis of the qualitative data suggested several factors that might explain variability in the 3 implementation outcomes (Table 2) of clinic adoption, staff and providers reached, and QI coach fidelity to intervention tools and protocols, as follows.

• Clinic adoption. QI coaches perceived strong relationships with clinic staff as key to gaining entry to the clinics. In state A, where coaches reported strong relationships with clinic staff, clinic-level adoption was 94%. Low turnover rates of both AFIX coaches and clinic staff in state A may have contributed to the strength of the relationships. As one of the coaches in state A reported, “When I call them, they see my name. I’ve been here for 18 years. A lot of the people know me already, so they know when I call, I’m like, ‘Guess what? It’s time for me to come out again,’ and they’ll be like, ‘All right.’”
• In states A and B, coaches reported that they promoted clinic adoption by presenting the coaching visit as a required meeting, with some coaches adding that the visit is required for participation in the Vaccines for Children program. Coaches in state C did not tell clinics they were required to participate in an HPV vaccination coaching visit, which may have contributed to low adoption rates in that state. The prevalence of large clinic networks in state C also might have contributed to a low adoption rate. More than 85% of the clinics in state C were part of a network, and more than 10% of those networks involved 11 or more clinics. In contrast, in state A, none of the clinics were part of a network, and in state B, 52% of clinics were part of networks, most of which (>94%) were small with 4 or fewer clinics. QI coaches in state C reported that they had difficulty directly recruiting clinics that were part of a network. Instead, they had to navigate the network’s multilevel hierarchy to identify an individual with the authority to approve a visit. As one QI coach from state C noted, “And then there’s also a lot of big systems where the person you call at the clinic is not authorized to say, ‘Yes, you can come in and talk to our providers.’ And so, [we] may have to go up the chain, but often gets lost…” In state C, administrative personnel often contacted clinics to schedule the HPV vaccination coaching visit, as compared with the other 2 states where most scheduling was done by the QI coaches.
• Reach to providers and staff. Although state C had the lowest clinic adoption rates, it had the highest average number of staff and providers participating in coaching visits. In contrast, state A had the highest clinic adoption rates and the lowest rate of staff and provider participation in coaching visits. One factor that might account for this is the staff with whom the coach scheduled the visits. In state C, coaches often scheduled visits with a higher-level representative of the healthcare system, such as a quality improvement leader. In state A, QI coaches scheduled their visits with medical assistants in individual clinics who might have lacked influence in the clinic and been unable to persuade providers to attend the visit.
• Across all states, time of year appeared to influence provider participation. QI coaches reported that providers were least available to participate in visits during the summer when vacations reduced the number of providers and staff in the clinic; however, workload increased because of sports- and school-related physical examinations. Only a few of the QI coaches viewed CME as an effective incentive to get providers to participate in the visit. Several QI coaches were either not aware that CMEs were available as an incentive or did not know how to request the CME. Several QI coaches reported that, although physicians typically did not participate, they often came in for at least part of the visit.
• QI coach fidelity to HPV coaching protocols. Factors that appeared to influence QI coaches’ overall fidelity to the HPV vaccination coaching protocols were perceived tension for change, knowledge of and beliefs about the HPV vaccination coaching protocols, perceptions of the needs and capacity of the person participating in the QI coaching visit, and perceptions of the HPV vaccination coaching tools. Coaches who provided high fidelity coaching perceived a need to change their current approach to QI coaching (ie, tension for change). QI coaches in states B and C acknowledged that their QI coaching could be improved, whereas those in state A were content with their current approach and saw little need to change. QI coaches in state A had the most years of experience providing QI coaching and the greatest success persuading clinics to adopt an AFIX visit, both of which might have contributed to a low perceived tension for change.
• Knowledge of HPV vaccination coaching protocols varied among coaches, which appeared to limit their ability to deliver the protocols with fidelity. Knowledge of the protocols did not appear to vary by state, but did vary by whether coaches were among the 8 who attended the 2-day training on HPV coaching. In many interviews, when asked about their process of advising clinics on the selection of improvement strategies, QI coaches who did not attend the training reported that they suggested strategies other than those on the list of strategies from the HPV QI coaching protocol.

QI coaches also described how they tailored delivery to match their perceptions of the needs and capacity of clinic staff and providers. This was particularly salient in state A, where medical assistants were the only participants in AFIX visits. QI coaches reported that their perceptions of participant needs and capacity particularly influenced use of the presentation slides and action planning. For example, QI coaches skipped sections of the slides if they felt that the content was already well known. Similarly, they skipped action planning if they felt the person they were meeting with lacked the capacity to make changes to clinic processes. Low fidelity to protocols for coaches working with clinics to develop an action plan was largely due to the perception that action planning fell outside the role of the person participating in the coaching visit. In the words of a QI coach, “[Clinic staff] weren’t able to see kind of beyond their role and to talk about other people’s roles.” In a few cases, reviewers stated that clinics were already performing well and did not perceive a need to participate in efforts to improve their HPV vaccination rates.

QI coaches’ positive perceptions of the report card, goal setting protocols, and the presentation slides appeared to encourage their relatively high levels of fidelity in the use of those tools. Across all 3 states, QI coaches reported appreciation for the report card’s clarity and conciseness. They all reported using the report card during the coaching visit; however, their perception of the utility of the report card was dampened by clinic staff and provider skepticism about the accuracy of the data reported. Data came from the state immunization registry rather than the clinic’s electronic health records. In some cases, clinic electronic health records lacked a direct interface with the state registry, and delays could occur in clinics uploading their data.

The high level of coaches’ fidelity to goal setting might be explained by their positive perceptions of the way the vaccination report card stated goals as both a number and percentage of patients. As one QI coach noted, “Putting [the goal] into people, rather than a percentage, really helped them understand it. And they’re like, ‘Oh…that’s only like X amount of kids a month.’”

Coaches in all 3 states valued the way the slide presentation outlined the evidence in support of HPV vaccination. In states B and C, QI coaches also valued the PowerPoint format as a way to structure the visit. A QI coach reported, when people “start going off topic, or they start talking about other stuff, it’s really useful to have that PowerPoint to keep the meeting on pace and kind of helping you bring people back.”

Top

Implications for Public Health

The HPV vaccination coaching intervention provided training and tools to support delivery of evidence-based, data-driven coaching to participating clinics in 3 states. Research has demonstrated that QI coaching is effective in improving the implementation of evidence-based interventions (8); however, the effects are often small and little is known about how, why, and when QI coaching works (13). Implementation outcomes in our study varied across the 3 states, providing an opportunity to explore variations in how QI coaching was delivered and factors that might begin to explain when and why those variations occurred.

Among the 3 states, rates of clinic adoption and provider participation varied such that QI coaches either achieved high adoption or high reach, but not both. In states A and B, where QI coaches gained easy access to clinics, coaches were less successful at getting additional staff to participate in the coaching visit. In contrast, in state C, QI coaches had difficulty gaining entry to clinics but were successful in getting multiple providers and staff to participate. This finding suggests that successful implementation of HPV coaching depends not only on clinic adoption of the approach but also on clinic readiness to improve their HPV vaccination practices. In implementation science, readiness is conceptualized as organizational commitment to implementing an innovation, including commitment and involvement of managers and leaders (14). In states A and B, where adoption rates were relatively high, coaches presented the visit as a mandate, and in state A, they additionally scheduled visits with medical assistants who had limited authority. Although clinics in these states adopted the HPV vaccination coaching visit, their leadership was not involved in the decision making, and the clinics might not have been ready to implement change. In state C, where the adoption rate was low, QI coaches had to work through multiple levels of network hierarchy to gain access to clinics. The extra effort required to get network leadership to approve HPV vaccination QI coaching might explain the success those coaches had engaging multiple staff to participate in the coaching visit. Coaches in state C chose not to present coaching as a mandate but instead focused on the benefits of the visit to patient care, which possibly contributed to higher levels of commitment to and participation in the coaching visit. Our findings have implications for the design of future QI coaching initiatives. Careful thought needs to be given to how coaches recruit clinics to ensure that clinics are committed to engaging providers and staff in efforts to improve their HPV vaccination processes. The format of the coaching visit might need refinement to take advantage of the finding that providers often join the visit only briefly. For example, the QI coach may include a brief visit with providers as part of a longer visit with staff. Lastly, as clinic networks continue to grow, QI coaches will need to learn how to effectively engage with these networks.

QI coaches in all 3 states demonstrated high fidelity to some HPV vaccination coaching tools and protocols and mixed or low fidelity to others. Coaches’ knowledge of and attitudes toward the tools were central factors influencing fidelity. In all 3 states, QI coaches reported consistent use of the report card and goal setting, which they perceived to be clear and concise. Coaches also valued the content of the presentation slides, particularly the concise presentation of evidence supporting HPV vaccination. Fidelity using the presentation slides was mixed, however, with coaches in 1 state converting slides into a 1-page hand-out. In the other 2 states, coaches reported skipping information or sections of the presentation that were not applicable to staff participating in the coaching visit.

Coaches reported limited fidelity to HPV vaccination coaching protocols for selecting QI strategies and action planning. Although coaches engaged clinic staff in strategy selection, they often promoted the strategies recommended by the traditional AFIX program, rather than the shorter list of strategies included in the HPV vaccination coaching protocols. This highlights the importance of providing booster trainings to ensure that all QI coaches are knowledgeable about the tools and protocols. QI coaches reported that they did little action planning, largely because the staff person participating in the coaching visit lacked the authority to do so. The lack of fidelity to action planning provides further support for the importance of recruiting clinics that are ready to improve HPV vaccination coverage, and, therefore, have the capacity and motivation to develop an action plan. Lastly, study findings on fidelity raise the question of when adaptation might be appropriate to improve the suitability of a tool or protocol to the needs of a particular state or clinic, and when adaptation is not appropriate because it alters 1 of the intervention’s 4 core components. Further research is needed to answer questions about which aspects of HPV vaccination QI coaching are essential to its effectiveness and which can be adapted.

Our study had several limitations. The HPV vaccination QI coaching intervention was evaluated in only 3 states; therefore, the ability to generalize the findings to other states remains to be established. Factors identified to explain variations in implementation outcomes are based on QI coaches’ perceptions; therefore, they are exploratory. Further study is needed to establish which factors influence QI implementation outcomes as well as the impact of those outcomes on vaccine coverage rates.

One of our intervention’s strengths was its alignment of tools and protocols with widely recognized QI coaching competencies, including skill in using data to drive change, knowledge of the evidence base informing change, familiarity with strategies to implement change, and skills in communication and relationship building. These 4 competencies provide a foundation for exploring core components of QI coaching. We found that QI coaches maintained fidelity to tools and protocols related to using data to drive change (report card, goal setting). They also maintained fidelity to the presentation of the evidence supporting the change. Even when they switched content to a 1-page format, they retained focus on informing clinics of the evidence base supporting HPV vaccination. Fidelity results were mixed for tools and protocols related to strategies to implement change and begin action planning. This is a concern, because evidence suggests that using data to drive change is most effective when it is coupled with guidance on QI and other change strategies (10). Findings also point to the importance of designing tools and protocols to support communication and relationship building. Careful consideration needs to be given to ensuring that QI coaches engage someone in the clinic (or delivery system hierarchy) who will verify the clinic’s readiness to improve HPV vaccination rates and engage providers and staff in the HPV vaccination QI coaching visit.

Top

Acknowledgments

Funding for the project, Impact of AFIX and Physician-to-Physician Engagement on HPV Vaccination in Primary Care: an RCT, was provided to the University of North Carolina at Chapel Hill, Department of Health Behavior, Gillings School of Global Public Health by the Centers for Disease Control and Prevention through Cooperative Agreement no. U01IP001073-02. Authors Victoria Petermann and Brigid Grabert were funded by the Cancer Control Education Program at University of North Carolina at Chapel Hill, Lineberger Comprehensive Cancer Center, grant no. T32CA057726-28. No copyrighted instruments or tools were used in this evaluation.

Top

Author Information

Corresponding Author: Jennifer Leeman, DrPH, School of Nursing, University of North Carolina, Chapel Hill, Carrington Hall, CB #7460, Chapel Hill, NC 27599-7460. Telephone: 919-600-0518 Email: jleeman@email.unc.edu.

Author Affiliations: ¹School of Nursing, University of North Carolina, Chapel Hill, North Carolina. ²Lineberger Comprehensive Cancer Center, University of North Carolina, Chapel Hill, North Carolina. ³Department of Health Behavior, Gillings School of Global Public Health, University of North Carolina, Chapel Hill, North Carolina. ⁴Immunization Services Division, Centers for Disease Control and Prevention, Atlanta, Georgia. ⁵United States Public Health Service, Commissioned Corps, Rockville, Maryland.

Top

References

1. Saraiya M, Unger ER, Thompson TD, Lynch CF, Hernandez BY, Lyu CW, et al. ; HPV Typing of Cancers Workgroup. US assessment of HPV types in cancers: implications for current and 9-valent HPV vaccines. J Natl Cancer Inst 2015;107(6):djv086. CrossRef PubMed
2. Walker TY, Elam-Evans LD, Yankey D, Markowitz LE, Williams CL, Fredua B, et al. National, regional, state, and selected local area vaccination coverage among adolescents aged 13–7 years — United States, 2018. MMWR Morb Mortal Wkly Rep 2019;68(33):718–23. CrossRef PubMed
3. Centers for Disease Control and Prevention. AFIX (Immunization Quality Improvement for Providers). 2017; https://www.cdc.gov/vaccines/programs/afix/index.html. Accessed August 7, 2020.
4. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci 2015;10(1):21–35. CrossRef PubMed
5. Calo WA, Gilkey MB, Leeman J, Heisler-MacKinnon J, Averette C, Sanchez S, et al. Coaching primary care clinics for HPV vaccination quality improvement: Comparing in-person and webinar implementation. Transl Behav Med 2019;9(1):23–31. CrossRef PubMed
6. Gilkey MB, Dayton AM, Moss JL, Sparks AC, Grimshaw AH, Bowling JM, et al. Increasing provision of adolescent vaccines in primary care: a randomized controlled trial. Pediatrics 2014;134(2):e346–53. CrossRef PubMed
7. Gilkey MB, Moss JL, Roberts AJ, Dayton AM, Grimshaw AH, Brewer NT. Comparing in-person and webinar delivery of an immunization quality improvement program: a process evaluation of the adolescent AFIX trial. Implement Sci 2014;9(1):21–9. CrossRef PubMed
8. Baskerville NB, Liddy C, Hogg W. Systematic review and meta-analysis of practice facilitation within primary care settings. Ann Fam Med 2012;10(1):63–74. CrossRef PubMed
9. Knox L, Brach C. Practice facilitation handbook: training modules for new facilitators and their trainers. Rockville, MD: Agency for Healthcare Research and Quality; 2013. Accessed May 30, 2019.
10. Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Systematic Review. 2012;(6):CD000259.
11. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health 2011;38(2):65–76. CrossRef PubMed
12. Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res 2005;15(9):1277–88. CrossRef PubMed
13. Berta W, Cranley L, Dearing JW, Dogherty EJ, Squires JE, Estabrooks CA. Why (we think) facilitation works: insights from organizational learning theory. Implement Sci 2015;10(1):141–54. CrossRef PubMed
14. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009;4(1):50–65. CrossRef PubMed

Top

Tables

Abbreviation: QI, quality improvement.

Top