ENDLINE SURVEY FINAL REPORT. World Vision, Niger. Prepared by ICF and World Vision for WHO Rapid Access Expansion (RAcE) Program March 2017

Transcription

1 ENDLINE SURVEY FINAL REPORT World Vision, Niger Prepared by ICF and World Vision for WHO Rapid Access Expansion (RAcE) Program March 2017 AUTHORS: Grace Nganga, Yodit Fitigu, Kirsten Zalisk

2 ACKNOWLEDGEMENTS ICF and World Vision would like to thank le Ministère de la Sante Publique and l Institute National de la Statistique for their contributions to this work. We would also like to thank the Relais Communautaires (Niger s Community Health Workers), who work hard to provide services to caregivers and children in communities, and the caregivers who give so much to ensure and improve the health of their children. This work was made possible by the World Health Organization through funding by the Canadian Government. RAcE Niger Endline Survey Final Report ii

3 TABLE OF CONTENTS ABBREVIATIONS... iv EXECUTIVE SUMMARY... v 1 BACKGROUND RAcE Program Goals and Objectives World Vision Project Background World Vision Endline Survey Objectives SURVEY METHODS Survey Implementation and Partnership Survey Design Survey Questionnaire Selection and Training of Survey Staff Data Collection Data Entry and Management Data Analysis Survey Indicators Survey Limitations FINDINGS Characteristics of Sick Children and Caregivers Decision-making Caregiver Knowledge and Perception of iccm RComs Care-Seeking Assessment Treatment Coverage First Dose of Treatment and Counseling from RCom Referral Adherence Sick Child Follow-Up Illness Management and Diagnostics by Sex DISCUSSION Annex A. List of Persons Involved in the Survey Annex B. Endline Sample Annex C. Detailed Sampling Design Annex D. Survey Questionnaire Annex E. Survey Training Schedule Annex F. Fieldwork Schedule Annex G. Details of Data Cleaning and Analysis Annex H. Indicator Definitions RAcE Niger Endline Survey Final Report iii

4 ABBREVIATIONS ACT CCM CSI iccm INS MSP ODK ORS PPS RAcE RCom WHO artemisinin-based combination therapy community case management Case de Santé Intégré (Health Hut) integrated community case management Institute National de la Statistique (National Institute of Statistics) Ministère de la Santé Publique (Ministry of Health) Open Data Kit oral rehydration solution probability proportional to size Rapid Access Expansion Relais Communautaire (community health worker) World Health Organization RAcE Niger Endline Survey Final Report iv

5 EXECUTIVE SUMMARY World Vision implemented the Rapid Access Expansion (RAcE) program in four health districts in Niger Boboye, Dogondoutchi, Dosso, and Keita since July In October 2016, World Vision conducted the RAcE endline survey, with technical assistance from ICF, and in collaboration with the Division de la Statistique (Division of Statistics) of the Ministère de la Santé Public (Ministry of Health) and the Institute National de la Statistique (National Institute of Statistics). This report presents endline data and compares baseline and endline data to assess changes in care-seeking, assessment, and treatment of sick children. Baseline and endline data are also used to assess caregivers knowledge of childhood illnesses and their perceptions of services provided by Relais Communautaire (RComs, or community health workers). This information is used to present project accomplishments. Results for key indicators are presented in Table 1. Caregivers knowledge and perceptions of RCom increased significantly over the course of the project, as expected, given that RCom were trained and deployed to provide iccm services after the baseline survey was implemented. The percentage of caregivers who know the RCom who works in their community significantly increased, from 1 percent at baseline to 99.8 percent at endline (p<0.001). At endline, caregiver s trust in RCom was nearly universal: 99 percent of caregivers viewed RCom as trusted health providers, and 98 percent believe RCom provide quality services. There was no significant change observed in caregiver knowledge of childhood illnesses over the course of the project. Care-seeking from an appropriate provider was high at baseline (68 percent) and remained high at endline (85 percent). As expected following the extension of health services to the communities via RCom, care-seeking from RComs was high at endline; of cases of illness among children 2-59 months who sought care from an appropriate provider, 88 percent of those sought care from an RCom The percentage of cases of illness among children aged 2-59 months taken to an RCom as the first source of care increased significantly (p<0.001), from 0.1 percent at baseline to 75 percent at endline. At baseline iccm services had not yet been rolled out to communities and no RCom administered RDTs to assess cases of fever. At endline, 75 percent of the cases of fever among children 2-59 months that were assessed by an RCom in the two weeks prior the survey were administered an RDT. The overall percentage of illnesses receiving appropriate treatment increased significantly over the course of the project, from 37 percent at baseline to 59 percent at endline (p<0.001). The largest increase observed was for the appropriate treatment for diarrhea, which increased from 23 percent at baseline to 64 percent at endline (p<0.001). Among those who sought care from an RCom, 58 percent received appropriate treatment. Despite the overall increase in appropriate treatment from any provider, the percentage of cases receiving appropriate treatment is low (overall, and from RCom). Little more than half of the cases treated by the RCom received the first dose of the treatment in the presence of the RCom and that 98 percent of caregivers were counseled by the RCom on treatment administration. At endline, 32 percent of cases managed by an RCom were referred. Overall, reported adherence to referrals made by RCom was high at 91 percent at endline. Overall, more than 68 percent of cases were followed up by an RCom, 93 percent of which were followed-up by the RCom within three days. The findings from the endline survey suggest the value of iccm in hard to reach communities in Niger, but greater improvements in access to quality care are needed. There is need to further investigate RAcE Niger Endline Survey Final Report v

6 reasons why appropriate treatment for iccm illnesses remains low despite high care-seeking from and assessment by RCom, and to put solutions in place to address this challenge. Table 1. Key indicator summary table Indicator Caregiver knowledge Percentage of caregivers of children age 2-59 months who have been sick in the two 1 weeks preceding the survey who are aware of the presence of the CCM-trained RCom in their community Percentage of caregivers of children age 2-59 months who have been sick in the two 2 weeks preceding the survey who know the role of the CCM-trained RCom in their community Percentage of caregivers of children age 2-59 months who have been sick in the two weeks preceding the survey who know two 3 or more signs of childhood illness that require immediate assessment by an appropriately trained provider Caregiver perceptions of iccm services Percentage of caregivers of children age 2-59 months who have been sick in the two 4 weeks preceding the survey who view CCMtrained RComs as trusted health care providers Percentage of caregivers of children age 2-59 months who have been sick in the two 5 weeks preceding the survey who believe CCM-trained RComs provide quality services Percentage of caregivers of children age 2-59 months who have been sick in the two 6 weeks preceding the survey who found the CCM-trained RCom at first visit Percentage of caregivers of children age 2-59 months who have been sick in the two 7 weeks preceding the survey who cite the CCM-trained RCom as a convenient source of treatment Sick child care-seeking Percentage of children age 2-59 months who have been sick in the two weeks preceding the survey for whom advice or treatment was sought from an appropriate provider 8 9 Overall Fever Diarrhea Fast breathing Percentage of children age 2-59 months who were sick in two weeks preceding the survey taken to a CCM-trained RCom as first source of care Baseline Endline % Point % (CI %) % (CI %) change 1.0 ( ) 40.0 ** ( ) 75.8 ( ) 20.0 ** ( ) 20.0 ** ( ) 0* 60.0 ** ( ) 68.8 ( ) 72.4 ( ) 65.8 ( ) 68.3 ( ) 99.8 ( ) 77.4 ( ) 81.1 ( ) 98.5 ( ) 97.6 ( ) 73.8 ( ) 87.9 ( ) 84.7 ( ) 88.9 ( ) 84.9 ( ) 80.1 ( ) p-value N/A RAcE Niger Endline Survey Final Report vi

7 Indicator Baseline Endline % Point % (CI %) % (CI %) change p-value Overall ( ) ( ) Fever ( ) ( ) Diarrhea ( ) Fast breathing ( ) Sick child assessment 10 Percentage of children age 2-59 months with fever in the two weeks preceding the survey ( ) ( ) who had finger or heel stick Percentage of children age 2-59 months for whom their caregiver received the results of the malaria diagnostic test of the children ( ) ( ) who had had finger or heel stick in the two weeks preceding the survey Percentage of children age 2-59 months with cough with difficult or fast breathing (suspected pneumonia) in the two weeks 12 preceding the survey who had their respiratory rate counted to assess fast breathing Sick child assessment by RCom Percentage of children age 2-59 months with fever in the two weeks preceding the survey 13 who had a finger or heel stick by an RCom among those who sought care from an RCom Percentage of children age 2-59 months for whom their caregiver received the results of the malaria diagnostic test of the children 14 who had a finger or heel stick by an RCom in the two weeks preceding the survey among those who sought care from an RCom Percentage of children age 2-59 months with cough with difficult or fast breathing in the two weeks preceding the survey who had 15 their respiratory rate counted to assess fast breathing by an RCom among those who sought care from an RCom Sick child treatment Percentage of children age 2-59 months who have been sick in two weeks preceding the survey who received appropriate treatment Overall Confirmed Malaria (ACT within 24 hours) Diarrhea (ORS and zinc) Cough with difficult or fast breathing (amoxicillin) Percentage of children age 2-59 months who have been sick in the two weeks preceding the survey who received appropriate treatment from a CCM-trained RCom 53.5 ( ) 0** 0* 0* 37.0 ( ) 79.6 ( ) 23.3 ( ) 44.6 ( ) 52.4 ( ) 75.4 ( ) 87.7 ( ) 63.8 ( ) 59.4 ( ) 73.4 ( ) 64.4 ( ) 46.2 ( ) n/a 63.8 n/a RAcE Niger Endline Survey Final Report vii

8 Indicator Baseline Endline % Point % (CI %) % (CI %) change p-value Overall ( ) Confirmed Malaria (ACT within 24 hours ) ( ) Diarrhea (ORS and zinc) ( ) Cough with difficult or fast breathing (amoxicillin) ( ) Percentage of children age 2-59 months who have been sick in the two weeks preceding the survey who received the first dose of treatment in the presence of an RCom among those who received prescription medicines for a CCM condition in the two weeks preceding the survey Overall 0* ( ) 55.0 n/a Fever (ACT) 0* 57.3 ( ) 57.3 n/a Diarrhea (ORS and zinc) 0* 50.3 ( ) 50.3 n/a Cough with difficult or fast breathing * (amoxicillin) ( ) 60.3 n/a Percentage of sick children age 2-59 months for whom their caregivers received counseling on how to provide the treatment(s) among those who received prescription medicines for a CCM condition in the two weeks preceding the survey 19 Overall 0* 98.1 ( ) 98.1 n/a Fever (ACT) 0* 98.9 ( ) 98.9 n/a Diarrhea (ORS and zinc) 0* 96.5 ( ) 96.5 n/a Cough with difficult or fast breathing (amoxicillin) 0* n/a Sick child referral and follow-up Percentage of sick children age 2-59 who 20 were referred in the two weeks preceding the * survey whose caregiver adhered to referral ( ) 91.4 n/a advice Percentage of sick children age 2-59 months receiving treatment from an RCom in the two 21 weeks preceding the survey who received a follow-up visit from an RCom according to country protocol *No cases **Fewer than 10 cases. Fever with positive blood test 0* 68.5 ( ) 68.5 n/a RAcE Niger Endline Survey Final Report viii

9 1 BACKGROUND 1.1 RAcE Program Goals and Objectives In 2013, the World Health Organization (WHO) launched the Rapid Access Expansion (RAcE) program in five sub-saharan African countries Democratic Republic of Congo, Malawi, Mozambique, Niger, and Nigeria. The goal of the program was to increase coverage of diagnostic, treatment, and referral services for malaria, pneumonia, and diarrhea to decrease overall mortality and the number of severe cases among children aged 2 59 months. The program would accomplish this goal through the following objectives: Catalyze the scale-up of integrated community case management (iccm) as an integral part of government-provided health services in sub-saharan Africa. Stimulate policy review and regulatory update in each country on disease case management. Accelerate adaptation of supply management and surveillance systems to include services at the community level. This effort came at a time when there was great momentum for iccm at the country level and a high degree of interest among the global health community to understand how to best measure success and how to build country ownership and capacity to sustain iccm interventions. 1.2 World Vision Project Background World Vision, in collaboration with the Ministère de la Santé Publique (MSP, or Ministry of Health), implemented the RAcE project in four health districts in Niger Boboye, Dogondoutchi, Dosso, and Keita from July 2013 to September 2017, with a target population of children aged 2-59 months. The objective of the project is to use the iccm approach by extending health care and treatment to households from health facilities to Case de Santé Intégré (CSI, or Health Hut) through trained community health volunteers, referred to as Relais Communautaire (RCom), based within the project area. The project will also strengthen the health system, supply chain management, and health information management. This project was implemented in accordance with the National iccm Strategy developed in 2012, part of the larger National Child Survival Strategy that focuses on harmonizing interventions that promote healthy family practices in health, nutrition, hygiene, and sanitation at the community level. One of the objectives of the MSP and RAcE strategy is to ensure that the implementation of primary health care involves community participation as a means of empowering communities in support of their health problems. The areas of the four districts in which RAcE was implemented have a total population of 1,872,929, including an estimated 414,079 children aged 2-59 months. MSP, together with WHO and World Vision, selected Boboye, Dogondoutchi, Dosso, and Keita as RAcE project areas due to high incidence rates of malaria, diarrhea, and pneumonia. Targeted areas are those that are iccm eligible, meaning that they are villages located at least five km from a health facility, have limited number of staff at the health facilities, have insufficient access and treatment options due to lack of trained health workers, and have limited RAcE Niger Endline Survey Final Report 1

10 equipment to provide adequate care. A total of 1,227 RComs were actively providing iccm services in all four health districts at the time of the endline survey. A baseline survey was conducted from August 26 to September 18, 2013 by World Vision and ADESEN-NAFA, a local nongovernmental organization, and with technical support from ICF. The baseline survey was administered in three steps, including two preparatory phases: training of the coordinating team took place from August 26 to 29, followed by training of the supervisors and enumerators from September 1 to 6. The field data collection was conducted over a 10-day period from September 9 to September 18 in sampled households in the project area. 1.3 World Vision Endline Survey Objectives The objective of the RAcE endline household survey was to assess care-seeking behavior for sick children, iccm coverage, and caregiver knowledge, attitudes, and practices related to pneumonia, diarrhea, and malaria in the RAcE Niger intervention areas. We compared baseline and endline data to assess changes in sick child care-seeking, assessment, and treatment coverage as well as caregivers knowledge of childhood illnesses and perceptions of RCom services, and used the information to make inferences about project accomplishments. RAcE Niger Endline Survey Final Report 2

11 2 SURVEY METHODS 2.1 Survey Implementation and Partnership World Vision, in collaboration with the MSP Division de la Statistique (Statistical Division), conducted the RAcE endline survey, with technical assistance from ICF, and l Institute National de la Statistique (INS, or National Institute of Statistics). World Vision and Division de la Statistique worked with ICF to finalize the questionnaire, led the training of enumerators and supervisors, and provided oversight of the survey implementation. INS, together with World Vision, conducted the training on mobile data collection and prepared data for analysis by ICF. The survey protocol received ethical approval from ICF s Institutional Review Board and administrative approval from MSP. Annex A contains a complete list of the people involved in the survey and their roles. 2.2 Survey Design This was a cross-sectional cluster-based household survey, targeting primary caregivers of children aged 2-59 months who had recently been sick with diarrhea, fever, or cough with fast breathing. All primary caregivers of children aged 2-59 months reported to have experienced diarrhea, fever, or cough with fast breathing in the two weeks prior to the interview were considered eligible for inclusion in the survey. ICF developed standardized sampling guidance for all RAcE projects, which was adapted for World Vision Niger. To be able to detect a 20 percent difference at 90 percent power with a two-tailed test and 95 percent confidence using cluster sampling, 263 cases were needed for each disease. ICF rounded up to 300 cases to ensure a consistent number of interviews per cluster and a slight increase in the precision of the coverage estimates. The Niger household survey used a 30x30 multi-stage cluster sampling methodology. At baseline, the RAcE project area, iccm-eligible areas more than five km from a health facility comprised the target population in Boboye, Dogondoutchi, Dosso, and Keita. At baseline, 30 clusters were selected using probability proportional to size (PPS). The baseline survey was conducted prior to training and equipping RCom to provide iccm services. The same clusters sampled at baseline were planned to be sampled at endline. However, nearly the full baseline sample had to be re-selected because they were not active RAcE project areas. RComs had been recruited to provide services in all of the baseline clusters, but in 21 of the baseline clusters RAcE interventions were never implemented 1. The sampling frame was updated accordingly and 21 replacement clusters were selected using PPS. Annex B contains the endline sample with the complete list of clusters and communities. 1 World Vision reported that RCom recruited in these areas either did not show up for training or did not pass the test qualifying them to provide iccm services. RAcE Niger Endline Survey Final Report 3

12 Within each cluster, 10 interviews were conducted for each of the three illness modules diarrhea, fever, and fast breathing for a total of 30 interviews per cluster, or 300 interviews per each illness across the project area. Within each cluster, the survey team randomly selected the first household for interview and proceeded to the household with its front door nearest to the front door of the current household until the team conducted 10 interviews for each illness. Because two communities visited did not have 10 cases of each illness, interviewers went to the nearest community of the nearest cluster to complete the questionnaires of 10 cases per illness. See Annex C for the detailed sampling design. At each household, the interviewer first determined if an eligible child lived there. An eligible child was aged 2-59 months and had been sick with diarrhea, fever, cough with rapid breathing, or any combination of the three illnesses in the two weeks preceding the survey. If there was an eligible child in the household, the interviewer administered the questionnaire, including all applicable illness modules, to the caregiver of the eligible child. If more than one child was eligible, and they were sick with different illnesses, their caregiver was asked about each instance of illness. If there was more than one eligible child in the household for an illness, the interviewer randomly selected one of the eligible children and interviewed his or her caregiver. 2.3 Survey Questionnaire ICF developed a standard questionnaire for all RAcE grantees to use for their surveys. World Vision worked with ICF to adapt the questionnaire to fit the Niger iccm program and country context. This included including appropriate terminology for community health workers in Niger, care-seeking locations, and treatment options. The survey questionnaire contains seven modules: caregiver and household background information; caregivers knowledge of iccm activities in their community; caregivers knowledge of childhood illness danger signs; household decision-making; and a module for each major childhood illness: fever, diarrhea, and fast breathing. In addition to collecting information about caregiver knowledge, care-seeking, and treatment coverage, the questionnaire collects standard Demographic and Health Survey data on household ownership of selected assets, materials used for housing construction, and types of water access and sanitation facilities, which ICF will analyze and use for the final evaluation. The endline survey questionnaire is approximately 30 pages in length. ICF had the questionnaire professionally translated into French, after which it was reviewed by WHO Niger and World Vision Niger prior to being fielded. However, prior to data collection, World Vision decided to use a mobile data collection program (Open Data Kit [ODK]) instead of the paper questionnaire because mobile data collection was used at baseline. With the help of Division de la Statistique, a programmer from INS programmed the mobile devices. This late decision left a short timeframe for developing the mobile program and did not provide time for ICF to review the program prior to fielding. Pretesting of the mobile questionnaire took place in Dosso in RAcE villages that were not part of the survey sample. The pretest was conducted on October 22, directly following the enumerator and supervisor training. The questionnaire was fielded in the local languages, using verbal translation by enumerators. A debriefing session was held with all coordinators, supervisors, and interviewers to discuss their pretesting experiences and identify and address problems with preparedness, field procedures, and instruments. Adjustments to the mobile questionnaire to resolve some of the missing RAcE Niger Endline Survey Final Report 4

13 items in a handful of the devices were made by the INS programmer immediately after pretesting, which took a considerable amount of time. Thus, the start of data collection was postponed for a few days to update the program. Questions and the structure of the questionnaire were not changed from the paper version. The survey questionnaire is provided in Annex D. 2.4 Selection and Training of Survey Staff Selection and recruitment of enumerators and supervisors was done by the MSP Division de la Statistique; participants with the strongest technical and leadership skills were selected to be supervisors. The enumerators were selected by their skill set and their previous experience in data collection. Many of the enumerators and supervisors who were recruited for the endline survey had also taken part in the baseline survey. World Vision, together with ICF, the MSP Division de la Statistique, and INS facilitated the training of enumerators and supervisors that took place in Niamey from October 17 to 21. The five-day training covered the following: Overview of the RAcE project goals and objectives Objectives of the endline survey Review of methodology, sampling, and respondent selection Roles and responsibilities of interviewers, supervisors, and all others in the study; rules; behaviors and ethics Detailed review of the use of mobile data collection tablets Question by question review of the household questionnaire Group practices, mock interviews, and role playing The survey training schedule is provided in Annex E. 2.5 Data Collection Endline survey data collection took place from October 27 to November 6 in Boboye, Dosso, and Keita, lasting approximately 10 days. The data collection team members included four coordinators and eight teams consisting of one supervisor and three enumerators. Data collection was done using the mobile program on tablets. The fieldwork schedule is provided in Annex F. Verbal informed consent was obtained from each caregiver prior to the start of the interview. Respondents were not compensated for their time away from income-earning activities or daily duties for participating in the data collection. The average length of the interview was approximately one hour per respondent. Quality control procedures during fieldwork included quality control checks by supervisors throughout the data collection process. This was done by observing interviews and going back to the households to re-interview at least 10 of the interviews to ensure quality and validity. The supervisor also oversaw correction of all errors that were detected while in the field and discussed the issues with the RAcE Niger Endline Survey Final Report 5

14 enumerators before the team left the community. The supervisor edited any errors in the questionnaire to the best of his or her knowledge. Administering the questionnaire using tablets removed the data entry step, which greatly helped in ensuring the availability of data in real time. 2.6 Data Entry and Management The endline data were collected using tablets with the ODK mobile program and downloaded to a cloud data storage at INS. INS was responsible for housing the data, and together with the MSP Division de la Statistique, was responsible for cleaning and preparing the data for ICF to conduct analysis. During the baseline survey, the data were stored in World Vision Canada s data cloud and were cleaned in Niger for analysis. Names of participants were collected only for purposes of listing and were not used during any stage of data analysis. Data entered cannot be traced back to the individuals. Access to data was restricted to authorized personnel only. After data for all clusters were validated, INS stripped the final dataset of any identifying information and shared it with ICF for analysis. However, ICF did not receive a clean dataset. After repeated requests for cleaned data from INS, ICF continued to find inconsistencies and inaccuracies with the final survey dataset sent by INS. 2.7 Data Analysis After a lengthy process of trying to obtain a clean dataset from INS, ICF did intensive data cleaning. ICF analyzed the survey data using Stata v14 and Microsoft Excel. The ICF analyst imported the Excel and Stata files sent by INS into a single, merged Stata file. The ICF analyst checked the endline data file for missing values. A list of missing data was sent to INS for corrected files, including the child roster file and missing location information in the caregiver file. There were also duplicate entries in the fever, diarrhea, and caregiver modules. A detailed explanation of the data cleaning and analysis process, including how missing values were handled, is provided in Annex G. Before analyzing the endline data, the ICF analyst had to redo the analysis of the baseline data; the baseline analysis that World Vision conducted did not account for cluster effects and did not include confidence intervals around the point estimates. Redoing the baseline analysis also allowed ICF to append the baseline and endline data files and calculate changes between the baseline and endline surveys. However, the results generated in World Vision s initial analysis are different from those that ICF generated. World Vision was unable to share a file that detailed the data cleaning that took place. Thus, ICF excluded records that did not include cluster information and records that contained fewer than 6 of 10 completed modules. The baseline data files required a substantial amount of cleaning before they could be merged and appended to the endline dataset. Household number fields in the various modules had to be cleaned, dropping duplicates (for example, in several cases, the household number was missing or differed for the same parent record). There was no cluster variable in any of the data files. Therefore, the ICF analyst performed the following steps: RAcE Niger Endline Survey Final Report 6

15 Used text fields (community and other_placename) that interviewers manually filled to generate a cluster variable; ensured that each community name was spelled the same way in all records. Inferred six missing community names from dates, times, interviewer name, and information in other entries. Dropped 17 records; 3 records had community names that could not be matched to any of the clusters, and 14 records were missing community information. Encoded community names to produce a cluster number for each of the 30 communities. Added the cluster variable to all records as the modules were merged into one baseline data file. Dropped 13 records from the dataset that did not include at least one completed sick child module. Dropped 144 records that had fewer than 6 completed modules out of 10. The ICF analyst did not verify or clean the child roster file; however, the analyst was able to pull age, sex, and two-week illness history for most of the sick child records. Roster information was missing for between 45 and 60 children, depending on the indicator. All variables were then recoded so that their names and values aligned with the names and values of the variables in the endline dataset. The cleaned baseline dataset was then appended to the cleaned endline dataset for analysis. The ICF analyst calculated survey indicator point estimates and 95 percent confidence intervals accounting for cluster effects, and used Pearson s chi-squared test to determine statistical significance for binary and categorical variables and regression for continuous variables. We considered indicators with p-values less than 0.05 to show a statistically significant change between baseline and endline. Endline data are displayed disaggregated by child s sex and illness. For the comparison of indicators between baseline and endline, we only disaggregated data by sex if we found the differences between males and females to be statistically significant. 2.8 Survey Indicators The survey collected data on 18 key indicators related to caregiver knowledge of RComs and child illnesses; caregiver perceptions of RComs; and sick child care-seeking, assessment, treatment, referral adherence, and follow-up. The survey also collected information on household and caregiver characteristics and household decision-making. Annex H contains a complete list of indicators and their definitions. 2.9 Survey Limitations The data collection team reported limitations in the ODK application. Since the mobile phones were verified twice before going to the field, it is feared that, once in the field, some data collectors may have deleted modules by mistake. These modules were, however, re-uploaded in the field. The other major issue was that the modules were not systematically linked, making it easy to skip an entire module without realizing it if a data collector was not paying adequate attention. This may explain some of the missing data, an issue identified during data analysis. The survey provides estimates for the RAcE project area as a whole. The survey was not powered to provide district-level estimates. Additionally, there are known potential biases and limitations with the RAcE Niger Endline Survey Final Report 7

16 indicators that assess caregiver recall of malaria diagnostic testing and coverage of appropriate treatment for children with fever and cough with difficult or fast breathing. The potential biases and limitations of these indicators are further detailed in the findings section. 3 FINDINGS 3.1 Characteristics of Sick Children and Caregivers The endline survey collected data on a total of 889 cases of illness among children aged 2-59 months. Table 2 illustrates characteristics of the children experiencing cases of illness in the two weeks prior to the survey. Of the cases of illness, 54.7 percent were among male children, and 45 percent were among female children. The largest age group represented in the survey were children aged months (24 percent), followed by children aged 2-11 months and aged months (both at about 21 percent). In the baseline survey, 52 percent of sick child cases were among males, and 48 percent were among females. The largest age group during baseline was children aged months (23 percent), followed by children aged months (21 percent). Of the cases of illness among children aged 2-59 months who were sick in the two weeks preceding the endline survey, 82 percent had fever, 78 percent had diarrhea, and 44.5 percent had cough with difficult or fast breathing. The characteristics of caregivers are shown in Table 3. A total of 489 caregivers were surveyed at endline. Table 2. Characteristics of sick children included in survey Characteristic* Baseline % (CI %) Sex of sick children included in survey Male, % 51.6 ( ) Female, % 48.4 ( ) Age (months) of sick children included in survey 2-11, % 19.2 ( ) 12-23, % 23.1 ( ) 24-35, % 20.5 ( ) 36-47, % 19.2 ( ) 48-59, % 18.0 ( ) Two week history of illness of children included in survey Had fever, % 75.5 ( ) Had diarrhea, % 61.0 ( ) Had cough with difficult or 56.3 fast breathing, % ( ) Endline % (CI %) 54.7 ( ) 45.3 ( ) 20.7 ( ) 23.8 ( ) 17.2 ( ) 17.8 ( ) 20.5 ( ) 82.4 ( ) 78.3 ( ) 44.5 ( ) RAcE Niger Endline Survey Final Report 8

17 Average number of illnesses, N Total number of sick children included in survey Cases of illness included in survey Fever, N Diarrhea, N Cough with difficult or fast breathing, N Total number of sick child cases included in survey * Missing sex for 45 children at baseline and 1 child at endline; missing age for 43 children at baseline and 1 child at endline; missing fever 2-week history for 46 children at baseline and 1 child at endline; missing diarrhea 2-week history for 54 children at baseline and 1 child at endline; missing cough with difficult or fast breathing 2-week history for 56 children at baseline and 1 child at endline. Table 3. Caregiver characteristics Characteristic Baseline % (CI %) Endline % (CI %) Age (years)* ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Mean age (years) Education* None ( ) ( ) Primary, year ( ) ( ) Primary, year ( ) ( ) Secondary or higher ( ) ( ) Marital status* Currently married or living with partner ( ) Not married but living with a 1.6 partner ( ) Not in union 5.1 ( ) Partner living with caregiver (among those in union)** Yes 79.8 ( ) Total number of caregivers *Missing age and education for 3 caregivers at baseline and 7 caregivers at endline; missing marital status for 2 caregivers at baseline and 28 caregivers at endline **486 caregivers in a union at baseline, and 450 caregivers in a union at endline ( ) 0.2 ( ) 2.2 ( ) 92.2 ( ) Table 4 shows the results for reported distance and mode of transportation to the nearest health facility. It is important to note that even though baseline data were collected for these indicators, there were a number of missing data. Similarly, there were a lot of missing responses at endline. Therefore, RAcE Niger Endline Survey Final Report 9

18 these data should be interpreted with caution. Please see below the table for notes on specific missing information. Table 4. Reported distance and mode of transport to nearest health facility Baseline Endline % (CI%) % (CI%) Distance to nearest facility < 5 km N/A 24.2 ( ) 5-9 km N/A 49.8 ( ) 20.3 N/A km ( ) 20 km N/A 5.7 ( ) Mean distance to nearest facility N/A 7.8 km Number of caregivers N/A 458 Mode of transport Walk ( ) ( ) Motorbike/taxi/bus ( ) ( ) Other ( ) ( ) Number of caregivers Time to nearest facility (among those who go to the facility) < 30 minutes ( ) ( ) minutes ( ) ( ) 1 < 2 hours ( ) ( ) 2 < 3 hours ( ) ( ) 3 hours or more ( ) ( ) Mean time to nearest facility 40 minutes 55 minutes Total number of caregivers *Missing distance to nearest facility for 31 caregivers at endline; the baseline data did not make sense so this indicator was not calculated. Missing mode of transport for 1 caregiver at baseline and 31 caregivers at endline; additionally, 1 caregiver in each survey stated that he or she does not go to the health facility. Missing time to nearest facility for 1 caregiver at baseline and 2 caregivers at endline. **The indicator for baseline was not calculated because the data do not make sense. 3.2 Decision-making As shown in Table 5, at endline, among caregivers who sought care for their child aged 2-59 months who had been sick in the two weeks before the survey, 62 percent made the decision to seek care jointly with their spouse or partner. This joint careseeking increased significantly from baseline (p<0.05). Table 5. Joint decision-making to seek care for sick child by illness Illness Overall Decided to seek care jointly with partner Baseline Endline % (CI %) % (CI %) ( ) ( ) p-value Baseline N Endline N RAcE Niger Endline Survey Final Report 10

19 Fever ( ) ( ) Diarrhea ( ) ( ) Cough with difficult or fast breathing ( ) ( ) * Missing information for caregivers of children with diarrhea with for 6 caregivers at baseline and 7 caregivers at endline; missing information for caregivers of children with fever with for 2 caregivers at baseline. 3.3 Caregiver Knowledge and Perception of iccm RComs There was no significant change observed in caregiver knowledge of childhood illnesses over the course of the project. Caregivers knowledge and perceptions of RCom increased significantly over the course of the project, as expected, given that RCom were trained and deployed to provide iccm services after the baseline survey was implemented. The percentage of caregivers who know the RCom who works in their community significantly increased, from 1 percent at baseline to 99.8 percent at endline (p<0.001). Of caregivers who reported knowing of an RCom in their community, the percentage who knew at least two curative services provided by an RCom was 77 percent at endline. Of the activities that RComs perform in their communities, caregivers most noted that RCom provide treatment for malaria (73 percent), malaria testing (64 percent), and oral rehydration solution (ORS) treatment for diarrhea (60 percent). At endline, caregiver s trust in RCom was nearly universal. At endline 99 percent of caregivers viewed RCom as trusted health providers, and 98 percent believe RCom provide quality services. Table 6. Caregiver knowledge of childhood illnesses Caregiver knowledge Baseline Endline % (CI %) % (CI %) Knows 2+ child illness signs ( ) ( ) Knows cause of malaria ( ) ( ) Knows fever is a sign of malaria ( ) ( ) Knows malaria treatment ( ) ( ) Total number of caregivers p-value Table 7. Caregiver knowledge of RCom Baseline Endline Caregiver knowledge p-value % (CI %) % (CI %) Knows CCM-trained RCom works in community ( ) ( ) Total number of caregivers Knows 2+ RCom curative services* ( ) ( ) Total number of caregivers *Only asked of caregivers who stated that there was a CCM-trained RCom in their community RAcE Niger Endline Survey Final Report 11

20 Table 8. Caregiver perceptions of iccm RComs Caregiver perceptions Baseline Endline % (CI %) % (CI %) p-value View CCM-trained RComs as trusted health care providers ( ) ( ) Believe CCM-trained RComs provide quality services ( ) ( ) Found the CCM-trained RCom at first visit (for all instances of 73.9 N/A care-seeking included in survey)* ( ) N/A Cite the CCM-trained RCom as a convenient source of treatment ( ) ( ) Total number of caregivers N/A=not available * Denominator is 406 caregivers at endline only those who sought care from an RCom for at least one sick child are included. Response missing for one caregiver who reportedly sought care from an RCom at baseline. 3.4 Care-Seeking Careseeking from an appropriate provider was high at baseline (68 percent) and remained high at endline (85 percent). As expected following the extension of health services to the communities via RCom, care-seeking from RComs was high at endline; of cases of illness among children 2-59 months who sought care from an appropriate provider, 88 percent of those sought care from an RCom The percentage of cases of illness among children aged 2-59 months taken to an RCom as the first source of care increased significantly (p<0.001), from 0.1 percent at baseline to 75 percent at endline. Looking at the sources of care sought, findings show a significant shift from public facility to RComs over the course of the project. Caregivers sought care at public facilities more at baseline (93 percent) than at endline (15 percent). However, caregivers have shifted to RComs as the source where most sought care, from 0.1 percent at baseline to 91 percent at endline. At endline, of cases of illness among children 2-59 months who sought care from an appropriate provider in the two weeks prior the survey, only 9 percent sought care from a source other than the RCom. Tables 9-12 illustrate the results of care-seeking behavior throughout the project timeframe. Table 9. Source of care by illness Illness Overall Fever Diarrhea Cough with difficult or fast breathing Sought care from RCom was first source of appropriate provider* care p-value Baseline Endline Baseline Endline % (CI %) % (CI %) % (CI %) % (CI %) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) 68.3 ( ) 80.1 ( ) ( ) p-value Baseline N Endline N * Appropriate providers include national, regional, or district hospitals; integrated health center; RCom; health hut and mobile or private clinics. RAcE Niger Endline Survey Final Report 12

21 Table 10. Care-seeking from RCom RCom was first source of care among Illness those who sought any care Baseline Endline % (CI %) % (CI %) Overall ( ) ( ) Fever ( ) ( ) Diarrhea ( ) Cough with difficult or fast breathing ( ) p-value Baseline N Endline N Table 11. Cases of illness for which no care was sought Illness Overall Fever Diarrhea Cough with difficult or fast breathing Total number of sick child cases Did not seek care Baseline Endline % (CI %) % (CI %) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Sought care but not p-value from RCom Baseline Endline % (CI %) % (CI %) ( ) ( ) ( ) ( ) ( ) ( ) p-value Table 12. Sources of care and first source of care Source of Care First Source Location Baseline Endline Baseline Endline % (CI %) % (CI %) % (CI %) % (CI %) Public facility ( ) ( ) ( ) ( ) Private clinic ( ) ( ) RCom ( ) ( ) ( ) ( ) Store, pharmacy, or market ( ) ( ) ( ) ( ) Traditional practitioner ( ) ( ) ( ) ( ) Other ( ) ( ) ( ) ( ) Total number of sick child cases Assessment Caregiver recall of malaria diagnostic testing is poor, which could affect the malaria diagnosis and appropriate treatment indicators calculated. According to the Indicator Guide: Monitoring and Evaluating Integrated Community Case Management, Studies have found poor sensitivity and specificity of maternal recall for malaria diagnostic tests (finger/heel stick). Consequently, the current RAcE Niger Endline Survey Final Report 13

22 recommendation is that household surveys track treatment coverage of fever and, where possible, supplement with data from service delivery assessment to better understand the proportion of suspected malaria cases that receive appropriate diagnosis and treatment. 2 Results show a significant increase in the assessment of cases of fever among children aged 2-59 months by any provider from 21 percent at baseline to 68 percent at endline (p<0.001). At baseline iccm services had not yet been rolled out to communities and no RCom administered RDTs to assess cases of fever. At endline, 75 percent of the cases of fever among children 2-59 months that were assessed by an RCom in the two weeks prior the survey were administered an RDT. The shift in care-seeking is similarly reflected in shifts in provider assessments. At endline, 90 percent of the cases of fever among children 2-59 months that were assessed by any provider were assessed by a RCom; 9 percent by nurses; and 0 percent by doctors or medical assistants. These results are likely due to the accessibility and presence of the RComs in the villages. Table 13. Malaria assessment among children with fever Cases managed by RCom All cases Fever assessment Baseline Endline p-value Baseline Endline % (CI %) % (CI %) % (CI %) % (CI %) Child had blood drawn ( ) ( ) ( ) Caregiver received result N/A of blood test ( ) ( ) ( ) Blood test positive for N/A malaria ( ) ( ) ( ) Received ACT* after positive blood test, among N/A those who had a positive ( ) ( ) ( ) blood test Total number of fever cases ACT=artemisinin-based combination therapy N/A=not available * Malaria treatment (ACT Coartem and Artesun Amodiaquine) p-value Table 14. Fast breathing assessment Respiratory rate assessment Respiratory rate assessed Total number of cough with difficult or fast breathing cases N/A=not available Cases managed by RCom Baseline Endline %(CI %) %(CI %) N/A 63.8 ( ) All cases Baseline Endline %(CI %) %(CI %) ( ) ( ) p-value The Maternal and Child Health Integrated Program (MCHIP). Indicator Guide: Monitoring and Evaluating Integrated Community Case Management, July RAcE Niger Endline Survey Final Report 14

23 3.6 Treatment Coverage The overall percentage of illnesses receiving appropriate treatment increased significantly over the course of the project, from 37 percent at baseline to 59 percent at endline (p<0.001). The largest increase observed was for the appropriate treatment for diarrhea, which increased from 23 percent at baseline to 64 percent at endline (p<0.001). At endline, the percentage of cases of illness among children aged 2-59 months who received appropriate treatment from an RCom was 48 percent. Among those who sought care from an RCom, 58 percent received appropriate treatment. Despite the overall increase in appropriate treatment from any provider, the percentage of cases receiving appropriate treatment is low (overall, and from RCom). Appropriate treatment provided by RCom, among cases who sought care from RCom, varied by illness. At endline, 73 percent of confirmed malaria cases were provided with artemisinin-based combination therapy (ACT) within the same or next day, 71 percent of cases with diarrhea received appropriate treatment of ORS and zinc, and 34 percent of cases of cough with difficult or fast breathing received treatment with amoxicillin. Treatment of cough with difficult or fast breathing must be interpreted carefully. Pneumonia treatment, for which this indicator is a proxy, is globally recognized to have validity issues 3 because diagnosis of presumptive pneumonia is often inaccurate in comparison with clinical diagnosis of pneumonia at health facilities. Therefore, the number of cases of cough with difficult or fast breathing is likely an overestimate of actual clinical pneumonia cases, and the percentage of these treated with amoxicillin can, and should, reasonably not be 100 percent. It is important to note that at baseline zero illness cases received treatment from an RCom; this is expected because iccm had not yet been rolled out to communities. For one fever case at baseline, a caregiver sought care from an RCom but received treatment from a CSI. Table 15. Treatment coverage Received appropriate Received appropriate Condition treatment from RCom treatment p-value (treatment) Baseline Endline Baseline Endline % (CI %) % (CI %) % (CI %) % (CI %) Overall* ( ) ( ) ( ) Confirmed malaria (ACT)** ( ) ( ) ( ) Confirmed malaria (ACT within same or ( ) ( ) ( ) next day)** Diarrhea (ORS and zinc) Cough with difficult or fast breathing (amoxicillin) ( ) 25.3 ( ) ( ) 44.6 ( ) 64.4 ( ) 46.2 ( ) p-value Baseline N Endline N *Calculated for confirmed malaria (ACT within 24 hours), diarrhea (ORS and zinc), and cough with fast breathing (amoxicillin) **Calculated among fever cases with a positive blood test result; malaria treatment (ACT Coartem and Asucam) 3 Campbell H, el Arifeen S, Hazir T, O'Kelly J, Bryce J, Rudan I, et al. (2013) Measuring Coverage in MNCH: Challenges in Monitoring the Proportion of Young Children with Pneumonia Who Receive Antibiotic Treatment. PLoS Med 10(5): e doi: /journal.pmed RAcE Niger Endline Survey Final Report 15