|
||||||||
|
Volume 8: No. 6, November 2011
TOOLS AND TECHNIQUES |
Box 1. Community Eligibility Criteria for Children in Balance Replication Trial | |
---|---|
Criterion |
Description |
Urban |
Must be an incorporated, urban city (US Census definition of urban). |
Diversity |
Racially, ethnically, and economically diverse. (City can make case for diversity. Benchmark used was 60% of school children were eligible for free or reduced-priced lunch.) |
Size |
Population of 50,000 to 125,000. |
Leadership |
Independent government structure including an elected mayor. |
Coalition |
Community-based coalition working on issues of or related to childhood obesity to demonstrate capacity to mobilize around the issue. |
Readiness |
Must demonstrate an appropriate level of readiness to act, while having not yet engaged in any major prior or current school-wide or community-wide childhood obesity intervention. |
Independent food service |
Must have a school district with a self-operating food service department. Food service cannot be outsourced. |
Professional development |
Demonstrated willingness to set aside 1 professional development day per year for teachers, food service staff, and nurses. |
Curriculum implementation |
Be willing to implement a nutrition and physical activity curriculum at least once per week during the school day (for grades 1 through 3). |
Leadership support |
Letter of support from the school district superintendent. |
Sustainability |
Applicant must identify how it would contribute $100,000 in cash or in-kind during 2 years of the project and identify programs that the community or coalition has been successful in piloting and sustaining. |
Box 2. 9-Point Readiness Scale for Community Readiness Modela |
||
---|---|---|
Score |
Stage |
Description |
1 |
No awareness |
Issue is not generally recognized by the community or leaders as a problem (or it may truly not be an issue). |
2 |
Denial/resistance |
At least some community members recognize that it is a concern, but there is little recognition that it might be occurring locally. |
3 |
Vague awareness |
Most feel that there is a local concern, but there is no immediate motivation to do anything about it. |
4 |
Preplanning |
There is clear recognition that something must be done, and there may even be a group addressing it. However, efforts are not focused or detailed. |
5 |
Preparation |
Active leaders begin planning in earnest. Community offers modest support of efforts. |
6 |
Initiation |
Enough information is available to justify efforts. Activities are under way. |
7 |
Stabilization |
Activities are supported by administrators or community decision makers. Staff are trained and experienced. |
8 |
Confirmation/expansion |
Efforts are in place. Community members feel comfortable using services, and they support expansions. Local data are regularly obtained. |
9 |
High level of community ownership |
Detailed and sophisticated knowledge exists about prevalence, causes, and consequences. Effective evaluation guides new directions. Model is applied to other issues. |
a Source: Plested et al (29). |
The request for applications was publicized on 33 professional listserves and distribution lists, posted on the Children in Balance website, and disseminated through a direct-mail campaign to mayors and school superintendents in 426 urban communities whose populations met the criteria listed in Box 1. The study team held an optional webinar and conference call for interested applicants 2 weeks before the letter of intent deadline. Attendees received information about the study background, design, theory, eligibility criteria, and application time line and asked questions. Potential applicants also e-mailed study staff with questions.
Sixty-seven participants from 54 communities in 26 states participated in the webinar. The study team reviewed 30 letters of intent and invited all 26 eligible communities to submit full applications. Twenty-two communities from 17 states submitted full applications. A team of 12 reviewers, 6 internal (staff and faculty) and 6 external (from foundations, academia, and government), independently reviewed and scored the 22 applications, using criteria and a scorecard developed by staff. The team ranked communities by average total score to identify the 10 highest-scoring communities.
Through the competitive grant-application process, the review team sought to identify communities that did not have comprehensive obesity prevention programming but demonstrated existing efforts, leadership, and collaboration as evidence of their potential to implement intervention components.
For the next step, the study team used the CRM protocol (29) to select 6 finalist communities from the top 10. Two team members attended a CRM training session at the Tri-Ethnic Center in the fall of 2007 and then trained 4 team members to conduct and score the semistructured interviews by using the Tri-Ethnic Center’s protocol (www.triethniccenter.colostate.edu/CRhandbookcopy.htm) (29). This protocol uses a 9-point readiness scale (Box 2) to evaluate 6 dimensions: community efforts to address the issue, community knowledge about the efforts, leadership, community climate (prevailing community attitude), community knowledge about the issue, and resources. The interview protocol includes 36 semistructured questions; 21 are “anchored” questions that directly address at least 1 of the 6 dimensions and are required for assessment (29). The 15 nonanchored questions are optional and can be modified according to researchers’ needs. At least 4 to 6 people in each community should be interviewed to assess a community’s readiness (29).
The study team modified the CRM interview script to focus on childhood obesity. The study team substituted 1 anchored question with a nonanchored question. The final script (Appendix) had 23 questions, including 20 anchored questions. The 3 additional questions addressed existing policies, practices, or laws related to obesity and the identification of community leaders on the issue. These questions were relevant because of the project’s objectives to create policy change and sustain leadership support.
Interviewers conducted the interviews during 2 weeks in January 2008. In each community, interviewers contacted the mayor or city manager, school district superintendent, school food service director, and a representative from the community coalition that submitted the application. Four people were interviewed in each community, for a total of 40 interviews. Interviewers used voice-over Internet protocol telephones and digitally recorded the calls with permission from interviewees. Interviews lasted approximately 30 minutes. A transcription agency transcribed the audio files.
Paired scorers reviewed the transcripts and assigned scores (Box 2) for each of the 6 dimensions independently. Scores are assigned on a scale of 1 to 9, in 0.25 increments (ie, 1.00, 1.25, 1.50, etc.). Each pair met to discuss the dimension scores and achieve consensus. According to CRM protocol, reviewers averaged the consensus scores for each dimension and rounded them down to the nearest integer to determine the overall CRM score. No study member scored transcripts from communities in which they had conducted interviews.
The mean overall CRM score for the 10 communities assessed was 4.28 (SD, 0.68), corresponding with the preplanning stage of readiness. Overall scores ranged from 2.97 to 5.36. Among the scores for the 6 dimensions (averaged for the 10 communities), the lowest were for community climate (mean, 3.11; SD, 0.64), knowledge about the issue (mean, 3.61; SD, 0.80), and knowledge of efforts on the issue (mean, 3.80; SD, 0.55) — all corresponding with the vague awareness stage. The leadership score (mean, 4.79; SD, 1.13) varied the most, corresponding with preplanning. The score for resources related to the issue (mean, 4.76; SD, 0.68) also corresponded with preplanning. The score for existing community efforts (mean 5.64; SD, 0.96) was the highest and corresponded with the preparation stage, indicating that these communities were likely launching and planning efforts with modest support from the community.
The review committee reconvened to discuss the CRM scores, CRM interview results, and other application components, including community demographics and coalition formation. The committee was particularly interested in the scores for leadership and community knowledge of the efforts because they reflect the support of key community decision makers and the level of awareness among citizens and leaders. The committee then ranked the communities to identify 6 finalists and 2 alternates.
The study team visited the 6 finalist communities to confirm eligibility and application information. Each visit included an observation of a school lunch, a community tour to understand local infrastructure, and semistructured interviews with community leaders in child health, which included some CRM interview participants. During these site visits, the study team interviewed an average of 11 people (range, 6-20) per community; some sites organized 15 to 20 coalition members for a group interview. The site-visit interviews addressed resources, local leadership, current initiatives, and community motivation. Questions for the superintendent and food service director focused on willingness and ability to promote physical activity, healthy eating, and school wellness practices. Study team members asked community leaders who were not part of the CRM interviews 8 questions from the CRM script. These interviews were not scored. As a result of the site visits, 1 community was deemed ineligible because the school district outsourced its food service. The study team visited and confirmed the eligibility of an alternate community.
The overall CRM scores demonstrate that the applicant communities ranged from vague awareness to preparation stages of change (scores of 3-5); these scores matched the review committee’s interest in identifying communities that were eager to address childhood obesity but did not already have comprehensive efforts in place. The letter of intent asked applicants to describe previous efforts that had been sustained. In the CRM stages, evidence of collaboration on the issue — a desired attribute — would correspond more closely with an overall score of preplanning (score of 4) or preparation (score of 5).
The low score (vague awareness) for community climate may indicate limited community empowerment (29). Similarly, the vague awareness of knowledge about obesity suggests limited recognition of childhood obesity as a local problem. In contrast, the mean score for existing community efforts was the highest of the dimensions. The difference between these scores indicates that efforts existed and communities could have been informed of them; the limited awareness suggests a communication gap between the community and the people involved in obesity prevention efforts.
The variability among the leadership scores indicated differences in community leaders’ prioritization of childhood obesity prevention among competing issues. Leadership support is necessary for ensuring the sustainability of a community-wide project; for this reason, low leadership scores (≤3.9) were a red flag to review committee members.
Communities with a high overall score (eg, a score of 6, corresponding with the initiation stage of readiness) may already have sufficient motivation and momentum to initiate and sustain intervention components on their own (19). Communities with a low overall score (eg, a score of 2, corresponding with denial/resistance) would need to dedicate significant efforts to raising awareness and building relationships in advance of implementing any intervention components. For this reason, the review committee removed from consideration a community with an overall score of 2.97.
The qualitative information gained from the interviews enhanced the study team’s understanding of the applicant communities and in 1 case, led to the elimination of an applicant. This finalist demonstrated a robust coalition, innovative programming, and a multicultural population in its written application. During the CRM interviews, however, it emerged that the community was deeply divided along ethnic lines, which would likely impede implementation of a community-wide, collaborative program.
The CRM transcripts provided descriptive information about existing programs, policies, challenges, and resources that can be leveraged for the intervention (32). Identifying local experts and seeking their opinion through interviews had additional value as an entry point to securing support from community leaders.
The 4 community members selected for the interviews were in leadership positions whose support and collaboration seemed integral to achieving the study objectives. The community, however, may not have seen these people in the same light. By preselecting interview participants on the basis of predetermined criteria, the study team may have overlooked other important community perspectives.
Because the interviews took place within a competitive application process, they may reflect social desirability bias. Respondents may have overstated community activities and commitment to childhood obesity prevention in hope of securing funding. Additionally, the transcript scoring process demands interpretive discretion, which the consensus process aims to attenuate.
The study team did not conduct reliability tests for the modified protocol. Because the only change was the substitution of a question addressing community climate, the change is not expected to have compromised accuracy of either the dimension or overall scores; of the 6 dimensions, community climate is seen as the least concrete and is often inferred from answers to anchored questions addressing other dimensions. The study team trained the scoring pairs to identify statements corresponding to each dimension throughout the interview transcripts.
Community readiness is a subjective construct. The CRM scoring system assigns numerical values to ease comparison; however, the data are fundamentally qualitative (26). The CRM captures a snapshot of a community during the interview period; a community, however, is constantly changing and readiness can be in flux. A crisis, or a change in leadership, may redirect community motivation and resources. Establishing the validity of a community readiness measure is challenging in the absence of a true readiness value that could be captured through an objective protocol.
A previous application of the CRM to obesity prevention used the overall readiness score to design an intervention that would be appropriate for a specific community and its level of readiness (32). That approach was similar to previous uses of the CRM for strategic planning for public health issues. Researchers have also applied the CRM as a pre–post measurement for a randomized community intervention (19,36); the CRM score was used to identify communities that would be able to implement an existing intervention model. To our knowledge, this is the first application of the CRM to compare readiness within a competitive request for applications addressing the issue of childhood obesity prevention.
The CRM protocol enabled the study team to gather information remotely about community capacity for an issue of interest. This ability reduced the number of site visits needed, thereby lowering travel costs and overall costs in the review process.
The stage model of the CRM is sufficiently concrete to be useful to researchers, consultants, and evaluators who wish to provide feedback to communities (25,28). Finding comparison sites for community-level interventions is difficult (37). The CRM scores provided an additional level of comparison among 10 communities of similar size, diversity, socioeconomic status, and perceived ability to implement a 2-year community-based obesity prevention program. The ability to evaluate readiness is central to determining whether a community can successfully execute a given intervention and to identifying a starting place for researchers and practitioners who are designing programs or interventions (29,38). Without this information, programs risk over- or underestimating what communities are capable of implementing (39), making for an inefficient use of resources.
Grant funding from the PepsiCo Foundation and core funding from the New Balance Foundation supported these research efforts.
Corresponding Author: Sarah Sliwa, MS, New Balance Doctoral Fellow in Childhood Nutrition, Friedman School of Nutrition Science and Policy, Tufts University, 150 Harrison Ave, Boston, MA 02111. Telephone: 617-636-3547. E-mail: sarah.sliwa@tufts.edu.
Author Affiliations: Jeanne P. Goldberg, Valerie Clark, Bridgid Junot, Elizabeth Nahar, Miriam E. Nelson, Alison Tovar, Christina D. Economos, John Hancock Research Center on Physical Activity, Nutrition, and Obesity Prevention, Friedman School of Nutrition Science and Policy, Tufts University, Boston, Massachusetts; Jessica Collins, Partners for a Healthier Community, Springfield, Massachusetts; Ruth Edwards, FlashStone Research and Consulting LLC, Stilwell, Kansas; Raymond R. Hyatt, Public Health and Family Medicine, Tufts University School of Medicine, Boston, Massachusetts.
|
|
|
|
The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.
Privacy Policy | Accessibility This page last reviewed March 30, 2012
|
|