BRFSS Annual Survey Data

	Please note: This page is best printed in landscape view.

2001 Summary Data Quality Report

This report provides selected statistical indicators of data quality in the Behavioral Risk Factor Surveillance System (BRFSS). The report presents data on four types of measures by state¹:

The measures in this report are designed to document the quality of BRFSS data.² "Data quality" in this report refers to the accuracy of BRFSS data.

Outcome Measures

The factors affecting the distribution of disposition codes by state may be grouped into differences in telephone systems, sample designs, surveyed populations, and data collection processes. Different outcome measures are variously affected by differences in these factors.

Table 1 presents the frequency distributions of individual disposition codes by state. The percent completes by state is the Efficiency Rate.

Table 2 presents brief definitions of each disposition code.

Table 3.

The categories used in the calculations of the outcome rates in Table 6 are defined below.

The BRFSS Cooperation Rate is an outcome rate with the number of completes in the numerator and the number of eligible respondents who are capable of completing the survey in the denominator. Records with disposition codes indicating a respondent who does not speak English—or in some states, English or Spanish—or a respondent with a physical or mental impairment are excluded from the denominator.³ The denominator of the Cooperation Rate consists of records with disposition codes of 01, 02, 07, 09, 13, 14. Thus, the formula for the BRFSS Cooperation Rate is

Table 3.

A response rate is an outcome rate with the number of completes in the numerator and an estimate of the number of eligible units in the sample in the denominator. The BRFSS CASRO Response Rate calculation treats telephone numbers used by someone aged 18 years or older who speaks English—or in some states, English or Spanish—well enough to complete the interview and with no physical or mental impairment that prevents completion of the interview as eligible units. Eligible records are records with disposition codes of 01, 02, 07, 09, 12, 13, or 14. Ineligible records are records in which a respondent is unable to complete an interview (those with disposition codes of 11 or 08) or those in which no adult resident uses the number (disposition codes 03, 05, and 06). Added to this total is an estimate of the number of households among the records whose eligibility is undetermined. These are records with disposition codes of 04 or 10. The BRFSS CASRO Rate calculation assumes that the unresolved numbers contain the same percentage of eligible households as the records whose eligibility or ineligibility are determined. Thus, the formula for the BRFSS CASRO Rate is:

A response rate is an indicator of the potential for bias in the results of a survey. It does not indicate the actual amount of bias. The actual amount of bias can be conceptualized as a function of two factors—the amount of non-response, which is measured by a response rate, and the differences between the responders and the non-responders. A response rate does not address the latter factor.

Table 4 presents frequency distributions of individual household disposition codes by state. The percent completes by state may be termed the Household Completion Rate. It is the percent of completes among identified households.

Table 5 presents frequency distributions of individual disposition codes by state for listed household telephone numbers.⁴ These distributions may be less subject to the influences of characteristics of the telephone systems and of sample designs than the distributions of all sample records. By definition, all of these numbers were listed household numbers at one time. While states will differ in the extent to which these numbers will have changed status since being listed, the resulting differences should be smaller than the differences among numbers that began with different statuses. Thus, the percent of completes among listed household numbers may be considered an alternative measure of data quality.

Selection Biases

Tables 6 through 13 present data on the differences between BRFSS and population data with respect to gender, age, and race/ethnicity by state. In these tables, BRFSS data are weighted for the characteristics of the sample design—disproportionate sampling by geographic and density strata (where they exist), number of phones, and number of adults in the household.⁵ Because these factors are built into the sample design, they should be adjusted for before comparing survey distributions to population distributions. No definitive standards exist with respect to what constitutes a substantial difference between survey and population percentages. One approach would be to examine the distribution of discrepant values with the purpose of identifying extreme values, or outliers, which may indicate biased data.
 

Income Missing Values

Table 14 presents the percent missing (Don’t know/Not sure or Refused or either) income by state. Income is the variable in the survey data with the largest percentage of missing values. A larger percentage of missing values for income implies lower quality data for income and, by extension, for other variables.