OBNE Metrics

Purpose

OutbreakNet Enhanced sites incorporate performance metrics and model practices into their capacity building plans. OutbreakNet Enhanced performance metrics are a subset of FoodCORE and Council to Improve Foodborne Outbreak Response (CIFOR) measures used to document the burden, timeliness, and completeness of enteric disease surveillance and response. Performance metrics can also be used to identify areas for process improvements.

microscopic image of Salmonella

Isolate/specimen-based metrics

Rationale: The intent of these metrics is to evaluate the timeliness and completeness/availability of laboratory surveillance and subtyping data for Salmonella, Shiga toxin-producing Escherichia coli (STEC), Listeria, Shigella, and Campylobacter. These metrics can be used to determine if there are gaps in the laboratory isolate handling and testing processes. If gaps are identified, knowing the detailed circumstances around the gap will help develop targeted actions to address them specifically.

1a. (OPTIONAL) Total number of SSL(SC) isolates and isolate-yielding specimens submitted to or recovered at the Public Health Lab (PHL)

  • Intent: To allow evaluation of the burden of isolate submissions and testing at the PHL.
  • Note: This is the total number, not just primary isolates.
  • Note: This measure is optional as it can also be collected through the ELC Performance Measures

1b. Number of primary isolates and primary isolate-yielding specimens submitted to the PHL

  • Intent: To allow evaluation of laboratory testing associated with the first or representative isolate or sample for each clinical case non-human testing unit versus duplicate isolates from repeat testing or sampling protocols.
  • Note: If multiple samples are tested from the same source, the primary isolates from each of the samples for which you completed testing can be counted.

2a. Total number of preliminary positive clinical specimens or samples (i.e. positive result from CIDT or other preliminary testing) received at PHL as a specimen or sample (regardless of if isolate-yielding or not)

  • Intent: To allow evaluation of the submissions of preliminary positive clinical specimens (i.e. CIDTs) to the PHL.

2b. Total number of preliminary positive clinical specimens or samples received at the PHL that yielded isolates

  • Intent: To allow evaluation of the outcome of clinical specimen testing and to calculate the percent of isolate-yielding clinical specimens or samples received at PHL.
  • Note: Could be used to identify gaps in submission protocols if a high proportion of specimens or samples are not viable; this also indicates the utility of testing multiple clinical specimens to try to identify cases that might otherwise be missed if only using isolate submissions.
  • Note: This is a sub-measure of 2a and calculated with 2a as the denominator.
  • Example: 100 CIDT positive specimens or samples are received by your lab (not as isolates) and PHL is able to culture an isolate from 75; 2a = 100 2b =75 (75%)

3a. Percent of SSL(SC) primary isolates with WGS testing at PHL

  • Intent: To allow evaluation of the completeness of WGS subtyping of SSL(SC) isolates.

4a. Time from SSL(SC) isolate receipt (or recovery) or WGS request at PHL to sequence upload to PulseNet

  • Intent: To allow evaluation of the timeliness of WGS subtyping (e.g., sequencing) at the PHL.
  • Note: Time is measured in median days, measurements will exclude weekend days. For laboratory time measurements, only isolates tested at the PHL should be included.
  • Note: To reduce duplication, this measure is collected through the ELC Performance Measures. This measure will be included in annual summaries.

Case-based metrics

Rationale: The intent of these metrics is to evaluate the timeliness and completeness/availability of epidemiologic data for reported cases. These metrics can be used to determine if there are gaps in the epidemiologic interviewing process. If gaps are identified, knowing the detailed circumstances around the gap will help develop targeted actions to address them specifically.

5a. Total number of SSL(SC) cases reported to epidemiology staff

5a.i (OPTIONAL) Percent of SSL(SC) cases with isolates or specimens submitted to the PHL

5b. Total number of confirmed SSL(SC) cases reported to epidemiology staff

5c. Total number of probably SSL(SC) cases reported to epidemiology staff

5d. Total number of suspect SSL(SC) cases reported to epidemiology staff

  • Intent: To allow evaluation of the burden of cases reported to epidemiology staff.
  • Note: The number of confirmed cases reported to epidemiology staff may not be equivalent to laboratory isolate counts because of duplicate isolates submitted to the PHL, and/or because cases may be reported to epidemiology staff from outside the PHL in their health department (i.e., laboratory or clinical report of cases not submitted to the PHL).
  • Note: Definitions for confirmed, probable, and suspect cases will follow CSTE/NNDSS case definitions for each pathogen (https://ndc.services.cdc.gov/).

6a. Percent of SSL(SC) cases with attempted interview

6a.i Percent of confirmed cases with attempted interview

6a.ii Percent of probable cases with attempted interview

6a.iii Percent of suspect cases with attempted interview

  • Intent: To allow evaluation of interviewing capacity to try to reach cases.
  • Note: To be based off #5 for calculation.
  • Note: Definitions for confirmed, probable, and suspect cases will follow CSTE/NNDSS case definitions for each pathogen (https://ndc.services.cdc.gov/).

6b. Time from SSL(SC) case report to initial interview attempt

  • Intent: To allow evaluation of turnaround time for attempted interviews. Attempted interviews were used here because factors that impact completion of an interview may be outside the control of epidemiology staff (e.g., case refuses), whereas making an attempt to interview is a turnaround time that can be controlled through data exchange and capacity. Time is measured in median days; measurements will exclude weekend days. This metric should include all confirmed, probable, and suspect cases with attempted interviews.

6c. Percent of SSL(SC) cases with exposure history obtained

  • Intent: To evaluate proportion of cases with assessment of exposures prior to onset of illness.
  • Note: To be based off #6a for calculations. The intended evaluation could be made using either #5 or #6a, but using #6a has the added benefit of allowing assessment of how many attempted interviews are successful in obtaining exposure history.

6c.i (OPTIONAL) Among SSL(SC) cases with an exposure history, percent with full shotgun or case exposure obtained

  • Intent: To evaluate the completeness of conducted interviews—e., how many interviews may have been a shorter interview and how many had collection of a full shotgun or exposure assessment.
  • Note: This is a sub-measure of #6c.

6c.ii (OPTIONAL) Among cases with an exposure history, time from case report to when an exposure history is obtained

  • Intent: To allow evaluation of turnaround time for obtaining exposure histories.
  • Note: This is a sub-metric for cases where an exposure history was obtained. Cases lost to follow-up and cases where no exposure history was obtained should not be included. Time is measured in median days; measurements will exclude weekend days.

Investigation-based metrics

Rationale: The intent of these metrics is to evaluate epidemiologic activity related to cluster and outbreak monitoring, evaluation, and investigation. These metrics can be used to determine if there are gaps in cluster and outbreak investigation. If gaps are identified, knowing the detailed circumstances around the gap will help develop targeted actions to address them specifically.

7. Number of SSL(SC) cluster and outbreak investigations

  • Intent: To allow evaluation of the burden of clusters, outbreaks, and investigational needs.

8a. (OPTIONAL) Total number and percent of SSL(SC) cluster and outbreak investigations with routine interview of cases

  • Intent: To allow evaluation of completeness of cluster and outbreak investigation response activities.
  • Note: This metric would indicate that initial interviews were conducted with a case(s) in your jurisdiction.

8b. Total number and percent of SSL(SC) cluster and outbreak investigations where an analytic epidemiologic study was conducted

  • Intent: To allow the evaluation of conducting or participating in analytic epidemiologic investigations.
  • Note: This metric would indicate that your jurisdiction was responsible for (i.e., led) or participated in analytic hypothesis testing. There may be clusters that do not warrant analytic epidemiologic investigation based on the hypothesis generating data.

9a. Total number and percent of SSL(SC) cluster and outbreak investigations with confirmed vehicle/source identified

9b. (OPTIONAL) Total number and percent of SSL(SC) cluster and outbreak investigations with suspect vehicle/source identified

  • Intent: To allow the evaluation of how often cluster and outbreak investigations result in identifying confirmed and suspect vehicles/sources. These investigations can still contribute to the body of knowledge of risky foods, practices, or other gaps in the food safety system in order to inform prevention efforts.
  • Note: There is not always a relationship between the completeness and/or timeliness of an investigation and identification of a confirmed vehicle/source.

10. Total number and percent of SSL(SC) cluster and outbreak investigations where food or environmental sample collected for testing

  • Intent: To allow the evaluation of how often food or environmental samples are collected for testing within your jurisdiction as part of a cluster or outbreak investigation.
  • Note: Not all investigations will yield evidence that support taking this kind of action.

11. Total number and percent of SSL(SC) cluster and outbreak investigations where environmental health, agriculture, regulatory, or food safety program staff were contacted

  • Intent: To allow the evaluation of how often environmental health, agriculture, regulatory, or food safety program staff within your jurisdiction were engaged in cluster and outbreak investigation activities.
  • Note: Not all investigations will yield evidence that supports taking this kind of action. Additionally, contacting partners during an investigation does not necessarily imply that a regulatory action would be indicated or taken.

Outbreak-based metrics

Rationale: The intent of these metrics is to evaluate outbreak reporting activity. These metrics can be used to determine if there are gaps in outbreak reporting. If gaps are identified, knowing the detailed circumstances around the gap will help develop targeted actions to address them specifically.

12a. Total number of SSL(SC) outbreaks, excluding multi-state outbreaks

12b. (OPTIONAL) Total number and percent of SSL(SC) outbreaks (excluding multi-state outbreaks) reported/submitted to National Outbreak Reporting System (NORS)

  • Intent: To determine the burden and completeness of outbreak reporting through NORS.
  • Note: It is understood that this value may not be 100% during specific reporting periods if an outbreak investigation is ongoing and therefore not ready to be submitted to NORS.

Definitions

Listed below are terms and definitions associated with the OutbreakNet Enhanced metrics. Terms are listed in the order that they appear in the metrics. These definitions were adopted from the FoodCORE core definitions.

Measurements will exclude weekend days. For laboratory time measurements, only isolates subtyped at the PHL should be included.

This will include all isolates (human, food, environmental, etc.) submitted to PHL and isolates recovered from specimen submitted to PHL. This can be further broken down to report total number of each category of isolates (human, food, environmental, etc.).

To be limited to the first or representative isolate or sample for each case or testing unit for any non-human isolates (i.e., isolates that are intended to be part of a subtyping workflow, such as whole genome sequencing). Any samples that were collected but not processed or tested can be de-duplicated and not counted as a primary isolate. If multiple samples are from the same source, the primary isolates from each of the samples for which you completed testing can be counted.

Refer to NNDSS case definitions for each pathogen

To include State, County, Birth Month, Birth Year, Sex, Race

To include an interview (of any format) that assesses exposures prior to onset of illness, via an open-ended exposure history, or via a list of potential exposures. The key factor to be considered an exposure history is an interview that goes beyond assessment of high-risk settings and prevention education to ascertain food consumption/preference, or other exposure data.

Two or more cases of infection where the number of cases meets one or more of the following criteria:

  1. The number of isolates/cases represent an increase over the expected baseline (as determined by PulseNet definition or other subtyping criteria)
  2. Demographic or other epidemiologic characteristics among cases indicate some deviation from expected values for the region (and is consistent with subtyping information)
  3. There is a non-human isolate that would indicate a potential source of human infections
  4. In the absence of meeting the above criteria in a catchment area, a case-patient should be considered cluster-associated if the above criteria are met when including isolates from other jurisdictions or catchment areas.
  5. In the absence of meeting any of the above criteria, ill persons should be considered cluster-associated if there are demographic or epidemiologic indications of a common source of infection even without laboratory subtyping data to link cases.

This definition also includes clusters that may be defined as outbreaks in your jurisdiction.

Any active epidemiologic follow-up resulting from the identification of a cluster. This could be initiating contact with a case (or the public health authority under whose jurisdiction a case falls) to ascertain direct case-based epidemiologic data, or active review of previously collected case-based data for cases later identified as cluster-associated.

Report of a case or cluster (depending on the metric) to epidemiology staff, i.e., when epidemiology staff first were made aware of a specific case or an identified cluster. This could be via routine communication such as a laboratory report or accessing a database, or via direct complaints, reports from another health authority (local, other state, federal, etc.), media report, or other means of communication.

A systematic, statistical analysis against a comparison group or within a cohort to test a hypothesis

SUSPECT vehicle/source clusters

Clusters of infection where investigational and/or laboratory data indicate a likely source/vehicle of infection without confirmation: vehicle is a known risk factor, established errors in food preparation, or reported consumption by a high proportion of cluster-associated cases.

CONFIRMED vehicle/source clusters

Clusters of infection where the etiologic agent has either been cultured from the vehicle or the vehicle has been statistically implicated in an analytic study.

To include interventions such as exclusion of an ill person(s) from high risk setting, remediation or closure of an establishment linked to illness, educational campaigns during daycare outbreaks, etc. To be considered a control measure, activities should extend beyond the routine educational component of an interview or exposure assessment.

To include media, public messaging (web updates, press release, etc.), or regulatory action (recall, hold, etc.). To be considered a public health action, activities should extend beyond the routine investigation activities and reach at-risk individuals beyond identified cases. A public health action should be included in the metrics if the FoodCORE Center was directly involved in the action, or is aware that a public health action was taken during a multijurisdictional investigation. For example, if CDC produces public messaging during a multistate outbreak investigation that a FoodCORE Center is involved in, that investigation would be associated with a public health action for the purposes of the metrics.