Featured CCQDER Projects

Non-binary gender survey question

Project: Development and evaluation of a single, non-binary gender question for population-based federal health surveys

Purpose: This study examined the performance of a single non-binary gender item for federal population-based health surveys. The study is part of a larger research agenda to develop a non-binary gender measure for a range of federal data collection systems. The research agenda includes both quantitative and qualitative methodologies.

Findings: The study found that a single question could be used to collect non-binary gender information in population-based health surveys. The study supported earlier findings that context affected the ways that respondents made sense of questions and came up with answers. This context included why the information was being collected and the respondent's understanding of the survey's privacy and confidentiality protections.

Conclusions: A single-item gender question is appropriate for the needs of most surveys. That question should offer a "transgender, non-binary-or another gender" response option and allow selection of more than one response option. In addition, researchers should consider how respondents understand why the question is being asked and should explain to respondents how their answers will be protected.

Citation: Miller, K, Willson, S. Development and evaluation of a single, non-binary gender question for population-based federal health surveys. 2023.

Study year: 2022

U.S. passport 'X' gender marker

Project: Cognitive interview evaluation of 'X' gender marker definitions for the U.S. passport application form

Partner: U.S. Department of State

Background: When this project began, several countries offered an X gender option on passports. Twenty-one states and the District of Columbia also offered the category on driver's licenses and birth certificates. Those government agencies used varying definitions of the X marker. It was not known how the general population interpreted and used the marker.

Purpose: The U.S. Department of State partnered with CCQDER as it prepared to add an X gender option to U.S. passports. This study assessed how potential passport applicants would interpret different versions of the definition of an X gender option. It also evaluated how various options would inform response choices.

Recommendation: Based on findings from two rounds of interviews, the optimal definition of an X gender marker for U.S. passports is "Unspecified or another gender identity."

Citation: Willson, S, Miller, K. Cognitive interview evaluation of X gender marker definitions for the U.S. passport application form. 2022.

Study year: 2022

Telemedicine availability during COVID-19

Project: Mixed method evaluation of the RANDS during COVID-19 telemedicine availability question: Results from the first two rounds of RANDS during COVID-19

Background: To provide the public with timely data about health-related pandemic effects, the National Center for Health Statistics (NCHS) expanded the mission of its Research and Development Survey (RANDS) program. RANDS—the center's methodological research survey—provided experimental estimates of a limited number of health-related outcomes during the pandemic.

NCHS's Division of Research and Methodology consulted both internal and external stakeholders when developing the RANDS during COVID-19 questionnaire. Through this process, RANDS during COVID-19 worked to provide timely information about telemedicine availability and use before and during the pandemic.

Purpose: While researchers tested the initial draft question about current telemedicine availability, the text of the final question was not tested before use. The question's language had been changed during the Office of Management and Budget clearance process. To get information about how respondents interpreted the final question language, CCQDER added an open-ended probe to the first round of the RANDS during COVID-19 questionnaire.

Findings: Analysis indicated that older respondents and respondents with lower levels of educational attainment appeared to have higher levels of potential measurement error. In addition, there may have been cultural differences in how the telemedicine availability question was understood that needed to be addressed to produce more consistent and valid survey data.

Citation: Scanlon, P. Mixed method evaluation of the RANDS during COVID-19 telemedicine availability question: Results from the first two rounds of RANDS during COVID-19. 2022.

Study year: 2020

COVID-19 survey questions

Project: Cognitive testing evaluation of survey questions on COVID-19

Purpose: This cognitive interview evaluation supported federal surveys that had added (or intended to add) questions related to COVID-19. The study looked at the validity of the COVID-19 questions. This helped survey data analysts understand what constructs the questions captured based on observed patterns of interpretation by respondents. The study also explored the question-response process to identify possible sources of response error.

Findings: The COVID-19 pandemic created a contextual backdrop that influenced respondents' interpretations of the questions, sometimes in unexpected ways. The evolving pandemic timeline created challenges for answering some of the questions because respondents had to decide which phase of their experience in the pandemic to consider. In addition, while some new pandemic-related terms were becoming more common, shared understanding of their meaning was not widespread.

Citation: Willson S. Cognitive testing evaluation of survey questions on COVID-19. 2021.

Study year: 2020

Opioid-related survey questions

Project: 2019 evaluation of opioid-related questions for federal household surveys

Purpose: This project examined the performance of opioid use, impairment, misuse, and disorder questions intended for population-based federal surveys. The project included a large, nationwide cognitive interviewing study (in English and Spanish) and a companion pile sorting activity. The interviews focused on the ways in which respondents interpreted the various questions.

Findings: Interviewers were able to determine false-negative and false-positive responses as well as reasons for that error. Respondents interpreted questions and formed responses to the various opioid survey questions in varied, inconsistent ways. Three different aspects of respondents' understanding of opioids informed how respondents formulated answers—

1. Their general awareness of opioids
2. The connotations they associated with opioids
3. Their personal understanding of their own relationship with opioids

Citation: Miller, K, Willson, S, Scanlon, P, Vickers, B, Massey, M, Creamer, L. 2019 Evaluation of opioid-related questions for federal household surveys. 2022.

Study year: 2018