|
|
Volume
3:
No. 1, January 2006
COMMUNITY CASE STUDY
The Annual African American Conference on Diabetes: Evolving Program Evaluation With Evolving Program Implementation
Jacquelyn M. Houston, MPH, APRN, BC, Maurice Martin, PhD, MEd, CHES, Joel E. Williams, MPH, PhD, Rhonda L. Hill, PhD, CHES
Suggested citation for this article: Houston JM, Martin M, Williams JE, Hill RL. The
Annual African American Conference on Diabetes: evolving program evaluation with
evolving program implementation. Prev Chronic Dis [serial online] 2006 Jan [date cited]. Available from: URL: http://www.cdc.gov/pcd/issues/2006/
jan/05_0119.htm.
PEER REVIEWED
Abstract
Background
According to 2003 Behavioral Risk Factor Surveillance System data, South Carolina has the fourth highest rate of overall diabetes among the 50 states (9.3%) but the second highest rate among African Americans (15.5%). Nationwide, African Americans are disproportionately affected by diabetes.
In addition, 40% of the African American population in South Carolina lives in a rural area, and
approximately 26% live at or below the poverty level. Lack of access to health care and diabetes education are additional barriers for people with diabetes and their families.
Context
Since 1997, the South Carolina Diabetes Prevention and Control Program and the Diabetes Today Advisory Council have sponsored the African American Conference on Diabetes, which targets African Americans with diabetes, their families, and
their caregivers. This article describes the evolution of the conference and its evaluation.
Methods
In 2002, we conducted focus groups with 20 African American conference
attendees with diabetes to 1) assess the program’s effects, 2) determine how to reach more individuals, and 3) improve programming. In 2004, we incorporated the preconference and postconference
Diabetes Understanding Scale survey to assess the cognitive impact of the conference on participants.
Consequences
Focus group results revealed that participants wanted to attend the conference because of the
opportunity to increase their knowledge and change their behaviors
through 1) education, 2) social support, 3) resources, and 4) logistics. Self-rated understanding increased significantly after the conference for each cognitive understanding item on the Diabetes Understanding
Scale.
Interpretation
Focus group results suggested that participants who continue to attend the conference year after year may improve diabetes self-management skills. A quantitative evaluation showed that this 1-day diabetes education conference significantly increased short-term, self-rated cognitive understanding of diabetes behaviors.
Back to top
Background
Diabetes is a significant public health problem that affects approximately 18 million people (1). The prevalence among adults is expected to double by 2025 (2). Diabetes is more prevalent among older people, and it disproportionately affects people in minority populations (3). Diabetes prevalence also differs geographically. According to 2003 Behavioral Risk Factor Surveillance System data,
South Carolina has the fourth highest overall rate of diabetes among the 50 states (9.3%) (4) but the second highest rate among African Americans (15.5%). Nationwide, African Americans are disproportionately affected by diabetes (4).
Self-management is the cornerstone of diabetes care and treatment, yet most people with diabetes do not receive any formal self-management education (5). Diabetes education integrated into comprehensive diabetes care has effectively improved self-management and diabetes clinical outcomes (6). Regardless of
race or ethnicity, diabetes and its complications can be controlled through early
diagnosis and proper self-management (1). Studies show that intensive glucose control can prevent retinopathy, nephropathy, neuropathy, and microvascular complications among people with diabetes (7,8).
Because African Americans in South Carolina are disproportionately affected by diabetes, the South Carolina Diabetes Prevention and Control Program (SC DPCP) and the Diabetes Today Advisory Council (DTAC) have sponsored the African American Conference on Diabetes (AACD). Since 1997, the AACD has
been convened to help educate African Americans with diabetes, their families, and
their caregivers. The
rigor of evaluation techniques to assess the effects of AACD has increased since
2002. This article describes the AACD's evolution and the simultaneous improvement in
its evaluation.
Back to top
Context
Diabetes in South Carolina
The goal of the Centers for Disease Control and Prevention’s (CDC’s) National Diabetes Prevention and Control Program
is to help people with diabetes have long, healthy, satisfying lives (9). One of the CDC's
national objectives is to reduce diabetes-related disparities among high-risk populations (9). The shared mission of the SC DPCP and DTAC is to prevent diabetes and its
complications among African Americans through diabetes education and management and to make individuals aware of community resources.
South Carolina is a rural, medically underserved state with a significant diabetes problem among African Americans. In South Carolina, 40% of African Americans live in rural areas, with 26% of them living
below the poverty level (10).
Specialized care is primarily available in areas with larger populations but is often inaccessible for poor individuals who live in rural areas (10).
Barriers to diabetes self-management include the lack of funds or insurance to cover the cost of ongoing care, medicines, supplies, and diabetes self-management education. Among the 12 counties with a diabetes prevalence that
is higher than the state average, two of the counties do not have even one certified diabetes educator (10).
Furthermore, six South Carolina counties have a ratio of less than one certified diabetes educator per 10,000 people (10).
Before 1997, SC DPCP's efforts to educate rural African American communities
about diabetes through health fairs and presentations in churches had limited exposure. In 1997, the SC DPCP and DTAC hosted the first annual AACD and focused on diabetes self-management. Organizers of the AACD hoped it would be a forum to provide diabetes education and resources to African Americans with
diabetes and their families and caregivers across South Carolina. Organizers selected Columbia, the state capital, as the conference site because of its central location and accessibility. Since its inception, conference attendance has continued to increase almost every year; more than 1000 people attended in 2004.
Evolution of the AACD
The AACD is held each November during National Diabetes Awareness Month in Columbia, SC. In 1997, the AACD’s first year,
the conference was sponsored by two community-based agencies. By 2004, sponsorship funding had increased through educational grants from pharmaceutical companies and start-up money from private businesses. The funding increase allowed
attendance to grow steadily from 195 participants in 1997 to 1044 in 2004, with a slight decrease in 2002 (Figure). The 2002 decrease in attendance coincided with the origination of a $5 registration fee; even this nominal fee excluded some people from participating.
Since 2003, community partners have assisted people who needed help with
the registration cost.
Figure. African American Diabetes Conference attendance, 1997−2004. The 2002 decrease in attendance was attributed to the origination of a $5 registration fee.
[A tabular version of this chart is also available.]
The AACD features educational encounter sessions (EESs), which are brief, didactic, skill-building sessions led by experts. AACD sessions educate participants about diabetes care practices (e.g., visiting primary care physicians, having a hemoglobin A1c test, monitoring blood glucose levels regularly) and encourage people with diabetes to adopt diabetes self-management skills and behaviors. At
these sessions, participants share and learn not only from experts but also from each other. The SC DPCP and DTAC expanded their partnerships with health professionals and community volunteers to increase the number of people who could receive
the AACD's resources.
In 2002, a session titled “Ask the Doctor” was incorporated into the conference. During the session, participants are allowed to question a panel of physician specialists: an internist, a dentist, an optometrist, a podiatrist, and a pharmacist.
In a foot care session, a clinician examined participants’ feet and then taught them how to examine their own feet.
Participants were shown how to use glucometers and products that make blood glucose testing easier and less painful.
In addition, they were taught the importance of taking prescribed medication, monitoring their blood glucose levels, and keeping records to share with their health care providers.
Sessions on physical activity focused on having fun while moving to music and burning calories. A nutrition session was designed to teach participants how to prepare healthy soul food that was low in sodium, fat, and sugar.
Health and community organizations and exhibitors presented their resources and products.
New formats and topics for future AACD sessions evolve on the basis of participant feedback and interest. For example, the initial AACD format
included morning plenary sessions. Because participants were reluctant to voice their concerns and ask questions in the plenary format, the sessions were replaced with the more intimate concurrent group EESs. Attendee feedback indicated a desire
for more diverse diabetes-related topics and more EESs. For example, in 2002, a session about depression and diabetes was incorporated. Participants practiced relaxation techniques and were encouraged to talk with health care providers about their mental and emotional health. Also as a result of participant feedback, AACD offered screening stations for blood pressure, cholesterol levels, and
kidney disease. The AACD is the only source of diabetes-related education and other resources for many of its participants.
Back to top
Methods
The AACD's program evaluation became more rigorous as the conference became more complex. Before 2002, only participant registration records and feedback on individual sessions were evaluated.
In 2003, a participant questionnaire was added to gauge satisfaction with the AACD programming. Each ensuing year, participant feedback was used formatively to plan the AACD
programming.
The evaluation was conducted in two phases: phase 1 in 2002 and phase 2 in
2004. In 2002, program planners realized that the AACD conference had matured and that impact evaluation methods should be used to 1) assess the program’s effect, 2) increase the number of people affected by the conference, and 3) improve programming. We selected the focus group method to gain a more comprehensive
understanding of participants’ thoughts and feelings about diabetes and the AACD.
Data from focus groups can provide insight into the cultural norms that shape diabetes self-management perspectives. A focus group can also create an interactive environment that allows participants to freely discuss issues (11). Qualitative and quantitative methods used in tandem work well for evaluating and planning educational interventions (12). In 2004, the evaluation included a
quantitative assessment to determine whether attending the AACD improved participants’ basic understanding of diabetes and awareness of important self-management skills.
Phase 1: focus groups
The CDC gave the SC DPCP a nonresearch determination for program evaluation in public health practice, so no Institutional Review Board approval was required for data collection.
Recruitment
Focus group participants were selected from the AACD registration forms, which included a question about participants’ previous
AACD attendance. Results showed that 70 registrants had attended at least two previous conferences. These registrants were contacted by telephone to confirm their prior attendance and to determine whether
they 1) wanted to participate in a focus group and 2) had been diagnosed with diabetes. The majority of registrants were willing to participate; most who were excluded did not meet the disease status requirement (i.e., did not have diabetes). Of the 70 contacted, 28 met the inclusion criteria and were invited to participate. Of the 28 potential participants, two declined to participate because they were not
planning to attend the 2002 conference. Later, six additional people dropped out because they were unable to attend the AACD. The remaining
20 participants were assigned to group 1 or group 2 based on the numerical order in which their names were listed on the original list of 28 eligible registrants. Odd numbers were assigned to group 1 (n = 12) and even numbers to group 2 (n = 8).
Facilitation
Two independent, 1-hour focus groups were conducted at the 2002 AACD. Before each focus group, participants wrote
on paper their demographic data and diabetes history. Each participant received $20 at the end of the session.
The facilitator told the participants that the focus group was being used as a program evaluation tool to improve the AACD. Participants were assured that their responses would be confidential, told that participation was voluntary, and told that their continued participation would be considered permission to report the aggregate information to stakeholders.
The facilitator was an African American certified health education specialist trained in focus group facilitation and experienced in diabetes prevention and control. The same structured discussion guide (designed to be flexible to allow probing for clarification) was used for each session to ensure
that the presentations were consistent. During each session, a staff member
wrote the themes of the conversation on a flip chart so that participants could review and validate their responses. Two staff members took notes on the responses and another person audiotaped the sessions.
Analysis
Audiotapes were transcribed verbatim, and the content was analyzed to find recurring themes. The transcription’s accuracy was confirmed by comparing it with the field notes. Two individuals coded the transcribed records for themes, one of whom had no previous involvement with the evaluation or program. Both coders were
experts in diabetes prevention and control and experienced in qualitative analysis. The coders discussed the themes and came to a consensus before issuing the report. Because no significant demographic differences existed between the two focus groups, we reported aggregated results.
Phase 2: quantitative measurement of diabetes understanding
In 2004, conference planners expanded the evaluation by gathering data on the short-term effect of the AACD educational sessions on participants’ understanding of diabetes and its treatment.
Data collection instrument
A modified 13-item Diabetes Understanding Scale was developed from section IV of the Diabetes Care Profile (DCP), an instrument for assessing understanding of diabetes and its treatment (13). The
scale addresses topics presented during the AACD. The scale was modified by combining the diet, exercise, and medication items into one
item and eliminating a diabetes and pregnancy item because no sessions were held on this topic. Two items on prevention and treatment of high “blood sugar” and low “blood sugar” were presented as
the following four items: 1) prevention of high blood sugar, 2) prevention of low blood sugar, 3) treatment of high blood sugar, 4) and treatment of low blood sugar.
The item alterations allowed evaluators to consider separately changes in participant understanding of prevention and treatment for high and low blood sugar.
Administration
All participants at the 2004 AACD were invited to participate in the program evaluation by completing the preconference and postconference surveys in their registration packets. Unique identifiers on the forms allowed us to match the participants’ preconference forms with their postconference forms. During the presession breakfast, an
introduction to the evaluation process was presented. Confidentiality was assured, and participants’ completed surveys were considered consent to use their data in an aggregate form.
Participants completed and submitted the preconference diabetes understanding survey and a short demographic questionnaire before the morning sessions began. After the closing session, participants completed the postconference
diabetes understanding survey. To encourage participation, participants received free raffle tickets, which they could only turn in with their completed evaluation forms. Of the 1044 attendees, 628 attendees
completed at least one of the surveys and provided useable data. Fewer postconference evaluation forms were completed than preconference forms because many participants left the conference before filling out the postconference
survey. Using a 5-point Likert scale (with 1 = poor, and 5 = excellent),
respondents ranked how well they understood specific diabetes issues. The data were entered into an EpiData
version 3.02 database (EpiData Association, Odense, Denmark) and exported as a SAS file (SAS Institute Inc, Cary, NC).
Analysis
Survey data were analyzed using SAS version 9.1. Descriptive statistics were used to describe the respondent sample by sex, race, age, and diabetes status. The mean
survey scores, standard error, and number of respondents for each subscale item preconference and postconference were calculated. Cronbach
α for internal consistency was computed for the preconference subscale items. Proc Ttest was used to examine significant differences in participants’ mean
survey scores, which had been matched by repeated measures at preconference and postconference (by unique
identification numbers).
Back to top
Consequences
Phase 1: focus groups
Twenty adults, predominantly African American women, participated in two focus groups
(Table 1). Participants freely engaged in discussions, sharing personal and family stories about their diabetes experiences. The devastation caused
by diabetes in their families and concern for preventing diabetes and its complications among loved ones dominated the conversations.
Table 2 includes direct
quotes for each theme.
Motivation for participation
When participants were asked, “What inspired you to participate in this conference over the years?” education and learning was the most frequently reported theme among the answers. Participants also frequently cited family and communication issues — a social support theme. They reported that the AACD provided a unique
opportunity to meet and talk to others with similar problems.
Logistics
Participants were also asked about which components of the conference were helpful
and should be continued in future conferences. Again, the themes of education, social support, and information about resources emerged in their answers. When asked about items that needed improvement, their
answers focused on logistics (e.g., the setting, programming issues, exhibitors, conference amenities). Participants were most concerned about
the conference center’s limited space and getting the registration information early. Participants also expressed
a desire to
be involved in the conference planning.
Knowledge and behavior change
When asked what they would have done differently as a result of the conference to handle their diabetes, participants said that they were more confident in their ability to manage their diabetes because of what they learned at the conference. They were better able to use the social support systems in
their communities and families, making them more effective at performing self-care tasks and more willing to seek professional health care when needed. Access to resources (human and material) was a third theme that emerged. Participants found the information about resources useful.
Phase 2: diabetes understanding
In 2004, 36% of AACD attendees responded to all
of the items on the preconference and postconference surveys.
Table 3 includes a description of the survey participants.
Table 4 presents preconference-to-postconference
score changes for each item, the mean number of respondents, and the significance levels for change by item.
The mean item scores of participants who completed the preconference and postconference surveys indicate a significant increase in self-rated understanding for each item on the scale. Internal consistency analyses of the preconference data revealed that the scale was highly reliable (Cronbach
α = 0.96).
Back to top
Interpretation
In 1997, the AACD was a small conference with fewer than 200 attendees and limited sponsorship, resources, and marketing ability. It has evolved into a program with more than 1000 attendees and greater sponsorship, resources, and marketing. As the AACD has evolved, so too has the
quality of its programming, stakeholder expectations, and evaluation rigor. Qualitative
and quantitative methods were used to effectively evaluate and plan this educational intervention.
Qualitative findings from focus groups suggest that participants at the AACD were motivated to attend the EESs because they received quality diabetes education, social support, and resources in an inviting, interactive environment. They felt empowered to help others manage diabetes. They also reported that previous AACD attendance improved their ability to adopt effective diabetes self-care
practices.
The participants’ perspectives were used in planning the logistics of each succeeding AACD. In 2004, the program site was relocated to a larger convention center to provide more space and address the logistical issues identified in 2002. To let more people know about the conference and allow more people to register early, the organizers marketed the conference through multiple media channels. The AACD
continued to offer an array of topics on diabetes self-management and expanded the variety of concurrent EESs to provide participants more opportunities to learn about different topics.
The AACD has been providing diabetes education and resources to the community since 1997; however, 2004 was the first year that the cognitive impact of the AACD on participants was evaluated. Quantitative findings from 2004 suggest that the AACD conference format with EESs improved participants’ self-reported understanding of diabetes self-management. The
survey reliability measures were
high and similar to those found in other studies (14,15). Examination of preconference to postconference changes in self-rated understanding suggests that the AACD had a significant effect on diabetes-related understanding, at least in the short term.
Increased conference attendance and increased understanding about diabetes self-management does not necessarily lead to behavior change, which is the primary purpose of the AACD. Although focus groups suggest that the behavior change occurred among multiyear attendees with diabetes, focus groups cannot be used to determine the actual proportion of participants who changed
their behavior. Understanding how the AACD benefits stakeholders and participants is important for planners of future programs, as is determining how to transform program evaluation into behavioral research. As the rigor of evaluation techniques increases, new research questions will emerge.
The evolutionary process described in this article is an example of the way that program evaluation not only improves programming but also plays a role in behavioral research. From the AACD evaluation, important research questions have already emerged:
- How much of the target audience can we reach using the AACD format?
- How can we recruit more attendees?
- What is the long-term effect of the AACD on participants’ diabetes-related knowledge, attitudes, and behaviors? How can we measure the effects?
As the next step in the evolution of the AACD’s program evaluation plan, we are considering using the Behavioral Risk Factor Surveillance System diabetes module to track the behavior of a sample of individuals who attend the conference each year.
Back to top
Acknowledgments
This project was supported by cooperative agreement U32/CCU410626 from the
CDC, which was awarded to the SC DPCP Department of Health and Environmental Control, Columbia, SC. The authors thank Barbara Wright-Mallory,
lay health facilitator with the SC DPCP, for
her leadership in planning the conference, and the DTAC for
its commitment over the years as a cosponsor.
Back to top
Author Information
Corresponding Author: Jacquelyn M. Houston, MPH, APRN, BC, Centers for Disease Control and Prevention, 2600 Bull St, Columbia, SC 29201. Telephone: 803-545-4472. E-mail: houstojm@dhec.sc.gov.
Author Affiliations: Maurice Martin, PhD, MEd, CHES, Centers for Disease Control and Prevention, Atlanta, Ga; Joel E. Williams, MPH, PhD, and Rhonda L. Hill, PhD, CHES, South Carolina Diabetes Prevention and Control Program, South Carolina Department of Health and Environmental Control,
Columbia, SC. Dr Williams is currently affiliated with the Department
of Psychology, University of South Carolina, Columbia, SC.
Back to top
References
- Centers for Disease Control and Prevention. National diabetes fact sheet: general information and national estimates on diabetes in the United States, 2003. Atlanta (GA): Centers for Disease Control and Prevention; 2004.
- Lamendola C.
Early and more vigorous detection of diabetes. J Cardiovasc Nurs 2003; 18(2):103-7.
- Gavin JR 3rd.
Slowing cardiovascular disease progression in African-American patients: diabetes management. J Clin Hypertens 2004;6(4 Suppl 1):26-33.
- Centers for Disease Control and Prevention. Behavioral Risk Factor Surveillance System
survey data. Atlanta (GA): Centers for Disease Control and Prevention; 2003.
- Mensing C, Boucher J, Cypress M, Weinger K, Mulcahy K, Barta P, et al.
National standards for diabetes self-management education. Diabetes Care 2005;28 Suppl 1:S72-9.
- American Association of Clinical Endocrinologists.
The American Association of Clinical Endocrinologists Medical Guidelines for Management of Diabetes Mellitus: The AACE
system of intensive diabetes self-management--2002 update. Endocr Pract
2002;8(1):40-82.
- The Diabetes Control and Complications Trial Research Group.
The effect of intensive treatment of diabetes on the development and progression of long-term complications in insulin-dependent diabetes mellitus.
N Engl J Med 1993;329:977-86.
- UK Prospective Diabetes Study (UKPDS) Group.
Intensive blood-glucose control with sulphonylureas or insulin compared with conventional treatment and risk of complications in patients with type 2 diabetes (UKPDS 33). Lancet 1998;352:837-53.
- Vinicor F, Jack L Jr.
25 years and counting: Centers for Disease Control and Prevention identifies opportunities and challenges for diabetes prevention and control. Ann Intern Med 2004;140(11):943-4.
- Diabetes Initiative of South Carolina, South Carolina Department of Health and Environmental Control. Burden of diabetes in
South Carolina 2003. Columbia (SC): South Carolina Department of Health and
Environmental Control; 2005.
- Blanchard MA, Rose LE, Taylor J, McEntee MA, Latchaw LL.
Using a focus group to design a diabetes education program for an African American population. Diabetes Educ 1999;25(6):917-24.
- Larme AC, Meyer JS, Pugh JA.
Use of qualitative methods to evaluate diabetes education programs. Diabetes Educ 1998;24(4):499-500, 504, 507-8.
- Fitzgerald JT, Davis WK, Connell CM, Hess GE, Funnell MM, Hiss RG.
Development and validation of the
Diabetes Care Profile. Eval Health Prof 1996;19:208-30.
- Fitzgerald JT, Anderson RM, Gruppen LD, Davis WK, Aman LC, Jacober SJ, et al.
The reliability of the
Diabetes Care Profile for African Americans. Eval Health Prof 1998;21:52-65.
- Fitzgerald JT, Gruppen LD, Anderson RM, Funnell MM, Jacober SJ, Grunberger G, et al.
The influence of treatment modality and ethnicity on attitudes in type 2 diabetes. Diabetes Care 2000;23:313-8.
Back to top
|
|
