Example of DTBE Program Evaluation Plan & Annual Report

Print

Updated December 11, 2020

Related Pages

Return to Main Menu

PDF Version

Disclaimer: The activities in this document are intended for example purposes only. The actual activities implemented as part of TB program evaluation should be identified by state or local TB program officials in collaboration with other stakeholders. The example provided here is not intended to be applied directly to any specific TB program.

Increase Use of Nucleic Acid Amplification Tests (NAAT) at Local Health Jurisdictions to Improve Treatment Initiation Performance

Planned Start Date: October 1, 2020 – Planned Completion Date: August 1, 2021

Background, Rationale, & Expected Outcome

In August 2020, our state TB program completed an evaluation that identified challenges to meeting national performance targets for TB case management activities. The program evaluator led a discussion about the evaluation findings between state TB program staff and collaborating state and local partners. A main concern by partners for not meeting the national TB program objective for treatment initiation performance (initiate treatment within 7 days of specimen collection) was that local health jurisdictions (LHJs) were not utilizing nucleic acid amplification tests (NAAT). NAAT can be a rapid and efficient option to guide clinicians’ decisions to initiate patient therapy and is recommended in the guidelines for diagnosing TB in adults and children. [1] Thus, TB program staff and state and local partners collaborated to develop a strategic plan to increase the use of NAAT by LHJs, with a long-term goal of improving treatment initiation performance. This long-term goal is aligned with the 2025 national TB program objective and performance target for treatment initiation.

Current Goal: To increase the use of NAAT results within local health jurisdictions (LHJ).

On This Page
Objectives
By August 01, 2021 (Time bound), increase the use of NAAT by LHJs for three TB case groups:
  • Increase the proportion of sputum smear-positive TB patients with a NAAT result reported prior to a culture result from 52% to 60% (Specific, Measurable, Achievable, and Relevant)
  • Increase the proportion of sputum smear-negative TB patients with a NAAT result reported prior to a culture result from 47% to 55% (Specific, Measurable, Achievable, and Relevant
  • Increase the proportion of HIV-positive TB patients with a NAAT result reported prior to a culture result from 68% to 75% (Specific, Measurable, Achievable, and Relevant)

State TB Program Objective and Performance Target for 2025: For TB patients with positive acid-fast bacillus (AFB) sputum-smear results (Specific and Relevant), increase the proportion who initiated treatment within 7 days of specimen collection from 68% to 80% (Measurable and Achievable) by August 1, 2021 (Time-bound).

Relevant National TB Program Objective and Performance Target for 2025: For TB patients with positive acid-fast bacillus (AFB) sputum-smear results, increase the proportion who initiated treatment within 7 days of specimen collection to 96% by 2025.
Corresponding National TB Indicators Project Indicator

Program evaluation efforts should reflect upon the national indicators. Please indicate the core NTIP indicator(s) addressed by this evaluation. If the evaluation is not linked to an NTIP indicator, please provide an explanation of how the evaluation relates to your scope of work in the CDC Cooperative Agreement.

Relevant National TB Program Indicator and Performance Target for Case Management and Treatment

Indicator: TB Disease Treatment Initiation

For TB patients with positive acid-fast bacillus (AFB) sputum-smear results, increase the proportion who initiated treatment within 7 days of specimen collection.

2025 National Performance Target: 96%

Evaluation Question Indicator/Deliverable Data Source Data Calculation/ Method Benchmark/Target Timeline
To what extent do program activities in the strategic plan with LHJs increase their use of NAAT results?

 

 Increase proportion of pulmonary TB patients with NAAT result reported prior to a culture result for smear-positive, smear-negative and HIV-positive patients Report of Verified Case of Tuberculosis (RVCT) For each case group: Number of patients with a NAAT result reported prior to a culture result / Number of pulmonary TB patients X 100 Target for sputum smear-positive:
60% by 08/2021

Target for sputum smear-negative:
55% by 08/2021

HIV-positive:
75% by 08/2021

 Work with LHJs to begin collecting data starting in October 2020 and ending in July 2021

Analyze and report findings by August 2021
Did increased NAAT use by LHJs contribute to improved treatment initiation performance? NTIP Treatment Initiation

Number of days between specimen collection and treatment initiation by LHJs

 

 

 NTIP NTIP Treatment Initiation

Compare the median number of days from specimen collection to treatment initiation for TB patients with and without NAAT result

 Target Treatment Initiation:
80% by 08/2021

Fewer median days to treatment initiation for TB patients with NAAT result

 Work with LHJs to begin collecting data starting in October 2020 and ending in July 2021

Analyze and report findings by August 2021

 

Part B: Evaluation Findings and Conclusions
(This section should be updated with submission of the annual report, if new findings are available, when the evaluation is complete.)

Is this a final report of a completed evaluation or an interim report of an ongoing evaluation? Final

Evaluation Question or Indicator Outcomes
Evaluation Question or Indicator Outcomes
(Listed results are by evaluation question and per NTIP indicator, if appropriate.)		 Benchmark
Met or Not Met
To what extent does the implemented program activities in the strategic plan with LHJs increase their use of NAAT results?
  • The state TB program education and training team identified 15 LHJs to implement targeted outreach activities; 13 LHJs agreed to participate in the study and integrate greater use of NAAT as a diagnostic tool for TB
  • Across all participating LHJs, the proportion of pulmonary TB patients with NAAT results reported prior to a culture result increased:
    • from a baseline of 52% to 60% at follow-up for sputum smear-positive
    • from a baseline of 47% to 57% at follow-up for sputum smear-negative
    • from a baseline of 68% to 76% for HIV-positive patients
 Met the 60% target for sputum smear-positive

Met the 55% for sputum smear-negative

Met the 75% for sputum HIV-positive
Did increased NAAT use by LHJs contribute to improved treatment initiation performance?
  • The state performance on TB Treatment Initiation did not reach the target of 80% by August 01, 2021. The current state performance for TB Treatment Initiation is 71.5%. However, this is an improvement from 2020 when the performance was 68%.
  • Overall, TB patients with a NAAT result initiated treatment in a shorter median amount of time than those without a NAAT result.
    • Sputum smear-positive TB patients who had a NAAT result were more likely to start treatment in a shorter amount of time (median of 5 days) compared to sputum smear-positive patients without a NAAT result (median of 16 days).
    • Sputum smear-negative TB patients who had a NAAT results were more likely to start treatment in a shorter amount of time (median of 7 days) compared to sputum smear-negative patients without a NAAT result (median of 24 days).
    • TB patients who are HIV-positive with a NAAT results were more likely to start treatment in a shorter amount of time (median of 11 days) compared to TB patients who are HIV-positive without a NAAT result (median of 18 days).
 Not Met with 71.5% of treatment initiation

Met with a lower median number of days from specimen collection to treatment initiation for TB patients with a NAAT result compared to those without a NAAT result

Having a timeline and stakeholders involved in the process of developing strategies, prioritizing objectives, and establishing deadlines and activities were helpful in creating a feasible strategy and gaining buy-in from LHJs.

Using existing data and protocols already in place for data reporting to CDC was beneficial in developing a manageable timeline to complete evaluation activities and in obtaining useful findings that are relevant to national TB program objectives.

Major Challenges in the Evaluation for Not Reaching Targets

While targeted LHJs increased their use of NAAT and improved their performance of TB treatment initiation, this contribution was not enough for the state to reach the national performance target. A challenge identified during the outreach process were reports by many LHJs that their NAAT ordering protocols varied compared to their partnering laboratory’s protocols, potentially causing delays. For example, there were several discrepancies in their testing and reporting schedules for when to offer a test and conduct the test, when to send the test to the lab, and when to report results back to LHJs to inform provision of treatment.

How findings were used: We held a meeting with TB program staff and partnering stakeholders in August 2021 to discuss new evaluation findings and challenges. The group has developed a new strategy to facilitate working partnerships between LHJs and laboratories to improve TB testing procedures for the long-term goal of further contribution to improved program treatment initiation performance.

  1. Gather more information on current TB diagnostic protocols from LHJs.
  2. Gather more information on current TB testing protocols from laboratories.
  3. Finalize a strategy and begin promoting the strategy to improve TB program treatment initiation performance.
  4. Identify solutions to improve the data quality, completeness, and timeliness of NAAT result reporting to local health jurisdictions.
  5. Evaluate collaboration efforts and effectiveness of targeted strategies to achieve long-term goals to achieve national TB Program objectives for laboratory reporting and case management and treatment.
  1. Brief report to share strategies and findings with LHJs, laboratories and other partners.
  2. Comprehensive report to share with state partners, CDC colleagues and other TB programs who wish to adopt the same study or strategies.
  3. Revised outreach materials to share with LHJs.
  4. New plan to promote partnerships between LHJs and laboratories for the next targeted evaluation.
  5. Success story to share with state partners, CDC colleagues and LHJs to promote use of NAAT for earlier TB diagnosis.
References

[1] Lewinsohn DM, Leonard MK, LoBue PA, et al. Official American Thoracic Society/Infectious Diseases Society of America/Centers for Disease Control and Prevention Clinical Practice Guidelines: Diagnosis of Tuberculosis in Adults and Children. Clin Infect Dis. 2017;64(2):111-115. doi:10.1093/cid/ciw778