Health Worker Performance After Training in Integrated Management of Childhood Illness -- Western Province, Kenya, 1996-1997

Each year, approximately 12 million children die in developing countries before age 5 years; 70% of these deaths are caused by respiratory infections, diarrhea, malaria, measles, and malnutrition, alone or in combination (1). In 1994, the World Health Organization (WHO) and the United Nations Children's Fund (UNICEF) developed the Integrated Management of Childhood Illness (IMCI) guidelines, which call for nonphysician health workers (HWs) to evaluate every sick child presenting to a first-level health facility (HF) for each of these conditions, regardless of the child's presenting complaint(s). Even though IMCI is being incorporated into the national health-care programs of many developing countries, little is known about HW performance after IMCI training. To measure the level of performance achieved and maintained by IMCI-trained HWs, during 1996-1997 CDC, the Kenya-Finland Primary Health Care Program, and the Ministry of Health of Kenya prospectively evaluated the level of performance achieved by IMCI-trained HWs at the end of training (EOT) and the level of performance maintained during the first 3 months post-training (1-3MPT) with monthly or bimonthly clinical supervision. This report summarizes the results of this evaluation, which indicate that HWs achieved reasonably high performance levels managing ill children with mild and moderate disease classifications but performed at a much lower level when managing severely ill children at EOT.

The IMCI algorithm follows four main steps. First, the HW assesses for signs that indicate the child is severely ill and needs referral and asks whether the child has a cough or difficult breathing, diarrhea, fever, or an ear problem. A more detailed assessment is performed if any of these symptoms are present. Children are assessed for signs of malnutrition and anemia, and their vaccination status is checked. Second, the child is classified according to the assessment findings. IMCI classifications are organized into three categories of illness severity: severe, moderate, and mild. Third, the child is treated and, fourth, his caretaker is counseled.

During the evaluation and monitoring of HW performance, supervisors used observation checklists to record the care provided to sick children aged 2-59 months, then reassessed the children to evaluate the accuracy of HWs' classifications. After the reassessment of each child, the supervisors provided immediate, individual feedback to the HWs on their performance.

Overall performance scores were developed that gave equal weight to each assessment task, classification, treatment, or counseling message. Assessment and counseling performance scores were calculated by dividing the total number of tasks or messages required and completed for each child by the total number required. Classification and treatment scores were calculated by dividing the number of correct classifications made or treatments given by the number that should have been made or given. Correct classifications were based on signs and symptoms recorded by supervisors during their reassessment of the child. The principal performance scores were measures of sensitivity rather than specificity for two reasons: a child is likely to suffer more from the omission of treatments that should have been given than from the provision of treatments not required by the IMCI guidelines, and treatments not required by the guidelines may have been for diseases not covered by the guidelines.

A total of 478 children were observed during the EOT evaluation, and 307 children were observed during supervisory visits 1-3MPT. Because feedback was given to the HW after each child seen, only the first child seen during each supervisory visit (n=117) was included in the analysis of HW performance. Because fewer children were seen with severe classifications than with moderate or mild classifications, all children with severe classifications observed during supervisory visits were included in the analysis to ensure an adequate sample size of severe disease classifications.

In general, performance levels reached at EOT were maintained 1-3MPT. Overall scores for the completion of assessment tasks were 81% (8781 of 10,896) at EOT and 75% (1988 of 2662) at 1-3MPT. Overall classification scores were 79% (1535 of 1939) at EOT and 78% (394 of 505) at 1-3MPT. Overall treatment scores were 72% (680 of 951) at EOT and 67% (172 of 258) at 1-3MPT, and overall scores for counseling during these periods were 69% (3480 of 5069) and 67% (829 of 1237). Overall classification and treatment scores primarily reflect performance classifying and treating the more common moderate and mild disease classifications. Performance scores for the classification and treatment of severe disease were much lower (Table_1): only 31% of children's illnesses were correctly classified and 32% correctly treated at EOT and 24% correctly classified and 26% correctly treated at 1-3MPT. HW performance classifying and treating two potentially life-threatening moderate diseases (i.e., pneumonia and anemia) show declining trends.

Reported by: A Odhacha, Kenya-Finland Primary Health Care Program; H Orone, M Pambala, R Odongo, J Lungapher, A Kituye, J Mwanza, C Nedda, S Ulenya, Ministry of Health, Kenya. International Child Survival and Emerging Infections Program Support Activity, Div of Parasitic Diseases, National Center for Infectious Diseases; and an EIS Officer, CDC.

Editorial Note

Editorial Note: In this evaluation, HWs performed well overall at EOT in completing assessment tasks and in classifying and treating moderate disease, but performed poorly classifying and treating severe disease. With clinical supervision, performance levels during 1-3MPT were generally maintained at the level achieved by EOT. Further investigation is necessary to determine why HWs perform poorly in classifying and treating severe disease and to modify training and clinical supervision to improve HWs' performance. Further investigation also is needed to evaluate how HWs perform using IMCI when they are not being observed. Because HWs are observed managing only three children during each supervisory visit, HWs may demonstrate more accurate adherence to the IMCI guidelines than when working under greater time pressure. Consequently, performance scores may differ when measured by alternative methods, such as surveys in which HWs are evaluated throughout an entire day, or reviews of medical records that capture HWs assessment findings and management plans.

Since the early 1980s, symptom-specific algorithms and training programs developed by WHO have been incorporated into the national health programs of many developing countries. Training courses in symptom-specific programs have become one of the principal means for improving HW performance after basic training. However, a review of HF surveys conducted in 28 countries with national training programs in the control of diarrheal diseases indicated that a median of only 16% of cases were correctly assessed, and a median of only 20% of children were correctly rehydrated (2). A 1994 survey in Bungoma and Vihiga Districts of Kenya indicated that HWs trained in the control of diarrheal diseases performed at the same level as HWs not trained (CDC, unpublished data, 1994). Because most training programs do not evaluate the level of performance achieved by HWs at EOT and then measure performance after training, it is unknown whether HWs perform poorly after training because they do not reach a satisfactory level of performance by the EOT, or if they attain a satisfactory level by the EOT but are unable to maintain it after returning to their HFs.

The introduction of IMCI guidelines is expected to improve HWs' performance and, as a result, substantially reduce childhood mortality (WHO, unpublished data, 1997). Like symptom-specific programs, IMCI provides guidelines to HWs with little previous clinical training in classifying and treating children. A major advantage of IMCI over the symptom-specific programs is that IMCI requires the HW to assess the child for all main symptoms regardless of the child's presenting complaint.

The approach to training and supervision used in western Kenya allowed supervisors to monitor HW performance levels, identify and provide immediate feedback on the performance of individual HWs, and identify problems associated with inadequate skill levels at EOT or failure to maintain or apply skills after training. This approach should be considered in other countries where IMCI is being implemented.

References

1. Pelletier DL, Frongillo EA, Schroder DG, Habicht JP. The effects of malnutrition on child mortality in developing countries. Bull World Health Organ 1995;73:443-8.

2. World Health Organization. CDD Ninth Programme Report, 1992-1993. Geneva, Switzerland: World Health Organization, Programme for the Control of Diarrheal Diseases, 1993.

TABLE 1. Number of cases of illnesses and percentage of illnesses correctly classified
and treated by health workers trained in the Integrated Management of Childhood
Illness (IMCI) guidelines at end of training (EOT) and 1-3 months post-training
(1-3MPT) -- Bungoma and Vihiga Districts, Kenya, 1996-1997*
======================================================================================================
                                    Correctly classified             Correctly treated+
                                -----------------------------    ---------------------------
                                     EOT             1-3MPT          EOT            1-3MPT
                                -------------      ----------    -----------      ----------
Classification                   No.      (%)      No.    (%)    No.     (%)      No.    (%)
--------------------------------------------------------------------------------------------
Severe
  All severe classifications&    173     (31)       72   (24)    173    (32)       74   (26)
  Severe pneumonia or             71     (45)       25   (32)     71    (38)       25   (48)
    very severe disease
  Very severe febrile dis-        48     (23)       24   (13)     48    (31)       24   ( 8)
    ease
  Severe malnutrition             36     (19)       17   (18)     36    (11)       17   ( 6)

Moderate
  All moderate classifica-       677    (85)@      156   (83)    656    (84)      152   (85)
    tions&
  Pneumonia                      115     (90)       27   (78)    115    (88)       27   (67)
  Malaria                        384     (96)       96   (96)    384    (95)       96   (99)
  Acute ear infection             32     (28)        7   (43)     32    (63)        7   (86)
  Anemia                          80     (73)       16   (56)     80    (54)       16   (38)

Mild
  All mild classifications&     1089     (83)      277   (90)
  No pneumonia, cough            151     (69)       39   (77)                       @
   or cold
  No dehydration                 122     (80)       32   (88)    122    (61)       32   (75)
  No anemia                      391     (91)       98   (95)                       @
  Not very low weight            425     (82)      108   (90)                       @
--------------------------------------------------------------------------------------------
* Percentages and numbers refer to classifications. Each child may have multiple classifications.
+ Correct medication prescribed (not including dosage) and child referred if indicated.
& Categories of severe and moderate disease, which include <7 cases seen are not listed indi-
  vidually but are included in the analysis of all classifications correctly classified and treated. 
  Severe disease classifications from all children seen during each supervisory visit are included. 
  Moderate and mild disease classifications from only the first child seen each supervisory visit 
  are included. 
@ Children with the moderate classification of very low weight or the mild classifications other 
  than measles or no dehydration received only symptomatic treatment and counseling.
====================================================================================================== 

Disclaimer   All MMWR HTML versions of articles are electronic conversions from ASCII text into HTML. This conversion may have resulted in character translation or format errors in the HTML version. Users should not rely on this HTML document, but are referred to the electronic PDF version and/or the original MMWR paper copy for the official text, figures, and tables. An original paper copy of this issue can be obtained from the Superintendent of Documents, U.S. Government Printing Office (GPO), Washington, DC 20402-9371; telephone: (202) 512-1800. Contact GPO for current prices.

Page converted: 11/25/98

HOME  |  ABOUT MMWR  |  MMWR SEARCH  |  DOWNLOADS  |  RSS |  CONTACT POLICY  |  DISCLAIMER  |  ACCESSIBILITY

Morbidity and Mortality Weekly Report Centers for Disease Control and Prevention 1600 Clifton Rd, MailStop E-90, Atlanta, GA 30333, U.S.A Department of Health and Human Services

This page last reviewed 5/2/01