A systematic review of strategies to increase demand, uptake and quality of community-based diagnosis and case management of malaria

Transcription

1 A systematic review of strategies to increase demand, uptake and quality of community-based diagnosis and case management of malaria December 2012 Maternal, Newborn and Child Health Working Paper UNICEF Health Section, Programme Division unite for children

2 A systematic review of strategies to increase demand, uptake and quality of community-based diagnosis and case management of malaria United Nations Children s Fund (UNICEF), New York, 2012 Knowledge Management and Implementation Research Unit, Health Section, Programme Division UNICEF 3 UN Plaza, New York, NY December 2012 This is a working document. It has been prepared to facilitate the exchange of knowledge and to stimulate discussion. The findings, interpretations and conclusions expressed in this paper are those of the authors and do not necessarily reflect the policies or views of UNICEF or of the United Nations. The text has not been edited to official publication standards, and UNICEF accepts no responsibility for errors. The designations in this publication do not imply an opinion on legal status of any country or territory, or of its authorities, or the delimitation of frontiers. The editors of the series are Theresa Diaz, Julia Kim and Alyssa Sharkey of UNICEF Programme Division. For more information on the series, or to submit a working paper, please contact tdiaz@unicef.org, jukim@unicef.org or asharkey@unicef.org. PHOTOGRAPHS: Children, parents and community health workers in Madarounfa, Maradi and Kollo, Tillabéri, Niger by Juliet Bedford Anthrologica Ltd ii

3 MATERNAL, NEWBORN AND CHILD HEALTH WORKING PAPER December 2012 A systematic review of strategies to increase demand, uptake and quality of community-based diagnosis and case management of malaria Lucy Smith Paintain, Barbara Willey, Alyssa Sharkey, Julia Kim, Valentina Buj, David Schellenberg & Ngashi Ngongo Keywords: systematic review, community case management, care-seeking, diagnosis, treatment, malaria, child health, Africa Comments may be addressed by to Lucy Smith Paintain (Lucy.Paintain@lshtm.ac.uk) cc: asharkey@unicef.org, vbuj@unicef.org, nngongo@unicef.org and tdiaz@unicef.org iii

4 iv

5 Table of Contents Abbreviations... vii Executive Summary... viii 1. Introduction Aim Objectives Methods Systematic literature review Results Overview of included studies Presumptive community case management of malaria RDT-confirmed case management of malaria Community case management of malaria integrated with other health services Community-level demand generation for care seeking for malaria Quality of included studies Discussion Interventions to improve quality of CHW performance Interventions to strengthen referrals from community to facility-based providers Interventions to strengthen health system capacity to support malaria CCM Integrated CCM of malaria and other interventions Community-level promotion of prompt care seeking Issues of study design Missing/ unreported information Conclusions and recommendations References Appendix 1: MEDLINE search strategy Sub-search 1: to identify interventions to improve community case management of malaria including quality, health system support, referrals and integration with other health interventions (objectives 1-4) Sub-search 2: to identify community level interventions to improve prompt and effective care seeking for treatment of malaria from any source (objective 5) Appendix 2: Sub-Saharan African countries eligible for inclusion v

6 Tables 1: Overview of included studies 9 2: Overview of included studies according to type of intervention, scale of implementation and evaluation design. 18 3: Community-based management of malaria based on presumptive treatment: Intervention details and evaluation outcomes. 23 4: Community-based management of malaria based on RDT-confirmed diagnosis: Intervention details and evaluation outcomes : Community-based management of malaria integrated with other health services: Intervention details and evaluation outcomes : Community-based promotion of care seeking for malaria, with or without integration with other health services: Intervention details and evaluation outcomes : Summary of study quality for randomised controlled trials and controlled before & after studies included in the review..47 Figures 1: Flow chart of literature search and screening. 7 2: Number of studies addressing each objective, by classification of intervention : Scatterplots of quality of malaria CCM programmes against length of intervention period, in terms of (a) proportion of children under five with fever/ malaria given the correct dose of anti-malarial; and (b) the proportion of children with fever receiving correct anti-malarial within 24 hours : Number of studies addressing each review objective, by study design. 61 Summary boxes 1: Key Findings and recommendations - Interventions to improve quality of diagnosis and CCM of malaria by CHWs : Key Findings and recommendations - Interventions to strengthen effective referral from the community level : Key findings and recommendations - Interventions to strengthen health system capacity to support CCM : Key findings and recommendations - Interventions to improve integration of malaria CCM & other interventions : Key findings and recommendations - Interventions to increase treatment seeking for malaria..60 6: Key findings -- Programme evaluation design.. 62 vi

7 Abbreviations ACT AL AQ ARI AS CBA CCM CDD CDI CHW CMD CORP CQ DOT HBMF HF HMIS HMM ICCM IMCI IQR ITN MCA MOH NGO OR ORS PCR PW RCT RBM RDT SP TDR Tx U5 VHC Artemisinin combination therapy Artemether-lumefantrine Amodiaquine Acute respiratory infection Artesunate Controlled before and after [study design] Community case management Community drug distributor Community-directed intervention Community health worker Community medicine distributor Community-owned resource person Chloroquine Directly observed therapy [TB treatment] Home-based management of fever Health facility Health management information system Home management of malaria Integrated community case management Integrated management of childhood illness interquartile range Insecticide-treated net Malaria control agent Ministry of Health Non-governmental organisation Odds ratio Oral rehydration salts Polymerase chain reaction Pregnant women Randomised controlled trial Roll Back Malaria Partnership Rapid diagnostic test Sulfadoxine-pyrimethamine Special Programme for Research and Training in Tropical Diseases Treatment Child under five Village health committee vii

8 Executive Summary A systematic literature review has been conducted to assess the published and unpublished evidence on the effectiveness, cost-effectiveness, equity and sustainability of strategies to increase demand and uptake and to improve the quality of community-based diagnosis and case management of malaria, towards achieving universal coverage. Specific objectives of the review were to investigate interventions to (i) improve the quality of services provided by community health workers (CHWs) responsible for malaria case management; (ii) strengthen referrals by CHWs to facility-based providers; (iii) build the capacity of health systems to support community case management (CCM); (iv) integrate malaria diagnosis and case management with other health services at the community level; and (v) increase care seeking behaviour for fever. In total, 42 studies from 16 countries in sub-saharan Africa met the inclusion criteria for the review, addressing one or more of the stated objectives. Following is a summary of key findings by category. Interventions to improve quality of CHW performance High adherence by CHWs to correct doses of anti-malarials was seen across the vast majority of studies, irrespective of diagnosis or anti-malarial policy, or strength of study design; in large part this is due to the benefit of pre-packaged anti-malarials and sufficient practical, interactive training techniques. Prompt and correct treatment of malaria is less consistent and tends to be lower than indicators of treatment with the correct anti-malarial and dose alone. Community mobilisation towards prompt treatment seeking should be emphasised. Larger scale studies with less external support had more modest results for prompt and effective treatment of malaria than more rigorously controlled research studies. CHWs demonstrated high ability to safely use rapid diagnostic tests (RDTs) and adhere to results, prescribing Artemisinin-based combination therapies (ACTs) for majority of RDT-positive patients (and minimum ACT prescription for RDT-negatives); challenges remain with respect to the action to be taken for RDT-negative patients. Cost effectiveness of use of RDTs depends on level of parasite prevalence. Evidence on CHW ability to diagnose and treat pneumonia is mixed. However, findings suggest that strong practical training, clear guidelines and regular supportive supervision with opportunities for problem solving are critical for maintaining CHW quality, especially for pneumonia treatment. Few studies evaluated integration of malaria CCM with other interventions. However, there is no indication that integration reduces the quality of CHW malaria treatment. Interventions to strengthen referrals from the community level Less than one-third of studies reported on referrals between the community and health facility levels, despite this being a crucial function of any CHW programme. CHWs may make judgement decisions as to whom to refer, for example, depending on the likelihood that the patient will be taken to a health facility or on the cost for the patient s family. Children with severe disease, of very young age, and those given clear instructions by the CHW, were more likely to comply with referral advice. viii

9 Interventions to build health system capacity to support malaria CCM Elements of health system capacity found to be critical for effective CHW programmes include: (i) ability to treat referred cases; (ii) regular supervision of CHWs; and (iii) a reliable and consistent supply chain for essential medicines and equipment to reach the community level. Information on critical health system capacity elements from included studies was generally qualitative. Development of complementary quantitative indicators to monitor health system support is desirable. Once areas of weakness have been identified, innovative and sustainable interventions are required. Design and evaluation of appropriate intervention studies are needed. Integration of malaria CCM with other health services Malaria CCM has been integrated with various other health interventions, including treatment of other common childhood diseases and promotion of preventive interventions (vaccination, vitamin A, insecticide-treated mosquito nets (ITNs), nutrition). Additional tasks do not seem to reduce the quality of malaria CCM, provided sufficient training, supervision and support is maintained. With the exception of pneumonia treatment, reporting on the quality of delivery of other interventions is limited. Interventions to increase prompt care seeking for fever Community mobilization activities to encourage prompt treatment seeking for fever were more successful when conducted alongside an intervention to improve malaria treatment provision. Malaria CCM interventions with insufficient mobilization support resulted in low demand for CHW services. No conclusive evidence was found on the impact of user fees for consultations or treatment on CHW utilization or socioeconomic equity of access. Only one study disaggregated results according to wealth quintile; none assessed achievement of improved geographic access to treatment through malaria CCM. Remaining questions Evidence for this review is drawn from studies with a spectrum of designs, from randomised controlled trials (RCTs) and controlled before-and-after (CBA) studies to post-only studies without control. Questions remain about the relative effectiveness of interventions relating to particular aspects of CHW quality, for example the relative importance of supervision or different models of training. More rigorous study designs may be required to investigate these specific questions. Less rigorous study designs can nevertheless still offer valuable programmatic insights on interventions to improve the quality of malaria CCM, provided it is possible to attribute the outcome of interest directly to the CHW, namely data from CHW registers or household surveys where information is collected on treatment by source. ix

10 1. Introduction Malaria is one of the leading causes of mortality worldwide. Over the past few years there has been a substantial increase in the amount of attention and funding focused on attempts to reduce the burden of the disease. This focus has been accompanied by the development of various short- and mediumterm targets for intervention coverage and disease reduction. Prompt and effective treatment of episodes of malaria illness has been a cornerstone of malaria control for the past fifteen years. In 2008, the UN Secretary General, set a target for achieving universal coverage by 2010 with all malaria interventions. This call was taken up by the wider Roll Back Malaria (RBM) Partnership and for prompt and effective management of malaria has been defined as 80% of malaria patients are diagnosed and treated with effective anti-malarial medicines, e.g. artemisinin-based combination therapies (ACT), within one day of the onset of illness [1]. Alongside a shift in national treatment policies to include and advance the use of more effective ACTs, the World Health Organization (WHO) policy on malaria case management was changed in 2010 to include parasitological diagnosis rather than presumptive treatment of all fevers as malaria [2]. This policy is mostly being implemented in the public sector [3, 4] (ie. health facilities); however, increasingly discussions are also moving towards introducing use of rapid diagnostic tests (RDTs) by CHWs and possibly into the private sector. According to the 2011 World Malaria Report, RDT and ACT use at the community level is approved national policy in 20 and 24 of 43 endemic African countries, respectively [5]. A result of these recommendations is that quality malaria treatment now implies the timely parasitological diagnosis and treatment with ACTs of confirmed malaria episodes. Available data on progress towards the goal of universal treatment shows that the mean proportion of children under five years of age with a fever that were treated with an anti-malarial drug in sub-saharan Africa between 2007 and 2008 was 32% (varying across countries from 6% to 57%), although less than 15% of children were given an ACT (range: 3% to 25%) [6]. These data do not include a measure of the promptness of treatment and assume presumptive diagnosis. Considering delays in treatment seeking [7], it is likely that the proportion of children receiving an appropriate anti-malarial drug within one day of the onset of illness will be lower still. It is now also important to consider parasitologically-confirmed cases of malaria, not just all febrile illnesses, when assessing the proportion of malaria cases receiving prompt, effective and appropriate treatment. A crucial factor contributing to the low coverage of prompt and effective malaria treatment (as well as other key preventive and curative services) is the weakness of national health systems in most low income countries and limitations in their ability to deliver interventions. Various studies have demonstrated inequitable access to malaria treatment according to socioeconomic status [8, 9] or distance from a formal health facility [10, 11]; the equity dimension must be considered in future analyses. The practice of unsupervised treatment of fevers in the home with ineffective remedies or incomplete doses of anti-malarials is widespread [12, 13]. One potential solution to improve access to appropriate treatment for malaria is to provide services closer to communities by training local CHWs in the provision of diagnostic and effective treatment services [14]. As the latest figures show, there is still a long way to go before universal coverage of prompt and effective treatment of malaria is achieved. Patients with fever may seek care from a range of sources, including formal public or private facilities, informal private drug vendors or community sources such as CHWs [15]. To achieve universal coverage of prompt and effective malaria treatment, quality of 1

11 diagnosis and treatment services need to be optimised at each of these sources, and improvements made in prompt care seeking by patients or their caretakers. Existing literature reviews provide information on the demand-side factors that influence treatmentseeking behaviour for childhood fever, including self-treatment [7, 15, 16] and adherence to treatment regimens [13]. Other systematic reviews have investigated provider characteristics that influence demand [17], interventions to improve malaria treatment behaviour of different types of provider [18-20], or broader interventions to improve demand for and access to health care such as removal of user fees [21] or conditional cash transfers [22]. A recent review in The Lancet by Haines et al. highlights the resurgence of interest in the role of CHWs in the context of challenges to achieving the child and maternal survival Millennium Development Goals (MDGs) [23]. A Cochrane Review of the potential role of lay health workers has also recently been updated [24], although the majority of evidence that met the inclusion criteria for this review is from developed countries. A systematic review by Hopkins et al. published in 2007 on the health impact of home-based management of malaria (HMM) provides a comprehensive analysis of studies conducted and published in the pre-act era [20], reflecting the earlier WHO strategy of HMM that recommends presumptive treatment of all fevers with an effective anti-malarial [14]. More recently, several large-scale studies and programmes have been funded that represent a shift in the international focus from HMM or community case management (CCM) of individual diseases to integrated community case management (ICCM). There is a clear need to expand the review by Hopkins et al. in the current context of WHOrecommended confirmed malaria diagnosis, and the use of ACTs and RDTs at the community level, which have increased the level of complexity of the tasks for which CHWs are responsible. Likewise it is important to understand the impact of interventions on outcomes other than health impact [25] to inform best practice for existing CHW programmes and identify areas where further research may be needed. Specifically, questions remain concerning strategies to increase: (i) quality of community-based services, including the use and interpretation of RDTs; (ii) linkages with the formal health system, including supervision, monitoring data, sustainable supply of commodities, and referral of patients with severe illness; (iii) integration of malaria diagnosis and treatment with management of other childhood illnesses such as pneumonia and diarrhoea. In addition, it is of interest to assess the evidence base for the effectiveness of community mobilisation to generate demand for prompt and effective malaria treatment. 2. Aim To assess the published and unpublished evidence on effectiveness, cost-effectiveness, equity and sustainability of strategies to increase demand and uptake and improve quality of community-based diagnosis and case management of malaria, towards achieving universal coverage. 3. Objectives 1. To assess and report the effectiveness of strategies to improve the quality of services provided by community health workers responsible for malaria case management. 2

12 2. To assess and report the effectiveness of strategies to strengthen referrals from CHWs to facility-based providers, with a focus on the management of malaria. 3. To assess and report the effectiveness of strategies to strengthen the capacity of health systems to support case management, including universal diagnosis, at the community level. 4. To assess and report the effectiveness of strategies to integrate malaria diagnosis and case management with other health services at the community level. 5. To assess and report the effectiveness of strategies at the community level aiming to increase care seeking behaviour for fever. 4. Methods 4.1 Systematic literature review Search strategy The search strategy was divided in to two sub-searches, the first to identify interventions to improve CCM of malaria including quality, health system support, referrals and integration with other health interventions (objectives 1-4); and the second to identify community-level interventions to improve prompt and effective care-seeking for treatment of malaria from any source (objective 5). For each sub-search, the online Ovid databases of CAB Abstracts, EMBASE, Global Health, MEDLINE and PsychINFO were searched using combinations of relevant free text and Ovid subheadings. The searches were limited to publications from 2000 onwards. However, no limits were placed on study location or language at this stage. The search terms included in the first sub-search included (i) malaria/fever; and (ii) community, home management/treatment. The search terms for the second sub-search included (i) malaria/fever; (ii) community; (iii) treatment-seeking, care-seeking, prompt, access; and (iv) intervention, mobilisation, sensitisation, education, promotion (Appendix 1). Although we anticipated a large degree of overlap in the two sub-searches, the second sub-search enabled a more focused investigation of the demand creation objective, which refers to demand creation activities at the community level, with the emphasis on prompt treatment seeking from any source, i.e., not necessarily a community source. The last sub-searches were conducted in August The results of the two sub-searches were combined and exported into EndNote, where duplicates were removed. Titles and abstracts were screened by two reviewers (LSP, BW) according to the study inclusion/exclusion criteria. Disagreements were resolved by discussion. A second stage of screening was conducted after reading the full text of the remaining papers, against the same inclusion/exclusion criteria to produce the final papers for analysis. Reference lists in the final papers were checked for additional studies of relevance. To identify any additional relevant reports in the grey literature, publications on the following websites were searched using broad terms such as malaria, community, community health worker, home management of malaria, integrated community case management: ELDIS [26], 3ie [27], WHOLIS & WHO publications [28], TDR [29], World Bank [30], DfID [31], USAID [32], MSH [33], Human Resources for Health [34], CHW Central [35]. 3

13 Inclusion criteria Study conducted in sub-saharan Africa (Appendix 2). Study involves: (i) EITHER, evaluation of an intervention to improve community-based management of malaria, where: o o Treatment is with first-line anti-malarial at time of study, i.e., not limited to studies with ACT; Diagnosis may be presumptive or confirmed using an RDT; (ii) OR, evaluation of an intervention delivered at the community level to improve prompt treatment seeking for malaria from any source. Community-based management of malaria was defined as provision of malaria diagnosis and treatment by lay members of the community without previous formal medical training but trained in basic management of malaria (with or without other basic health problems) in the context of the intervention. Private medicine vendors and drug shopkeepers were excluded from the analysis due to a recent systematic review of interventions to improve their role in home-based management of malaria [19]. Studies with the following designs were included for analysis: randomised controlled trial (RCT); clustered RCT; before-and-after studies, with or without control; interrupted time series (ITS) with at least three time points pre- and post-intervention. Post-only studies were also included where the outcome of interest could be attributed to the community-based intervention under study, for example direct evaluation of CHW performance via register review or household surveys with questions on treatment by source. If linked to an included publication describing a community-based treatment or demand creation intervention, qualitative studies, case studies, process evaluations, and cost-effectiveness studies were included to identify barriers to and facilitators of effective intervention implementation. Papers were only included where objective and standardised impact or outcome measures were reported. These include: clinical outcome; proportion of patients seen by CHW that are diagnosed and treated correctly; correct use of RDT by CHW and treatment according to result; correct referral to formal health facilities; referral completion; proportion of febrile patients seeking care promptly (within 24 hours). Papers where the outcome measures were limited to CHW caseload were not included (as they don't reveal anything about the quality of services provided). Similarly, user-side studies which looked at knowledge of malaria transmission, etc., without knowledge on the need for prompt treatment seeking, were excluded, as were studies which demonstrated increased treatment seeking from a CHW post-intervention without reporting timeliness of such treatment seeking. Studies evaluating the role of CHWs in providing pre-referral treatment for severe malaria in the absence of management of uncomplicated malaria were not included in the analysis. Papers that cover surveys of provider or user behaviour without any intervention were not included. 4

14 Papers were also excluded if they related to chemoprophylaxis, mass drug administration, intermittent preventive treatment of malaria in children or infants (IPTc or IPTi), imported malaria, or traveller health. Data extraction Data was extracted on the following themes using a standardised Microsoft Access database: Characteristics of the intervention: country; scale and period of implementation; implementation partners; target population; CHW recruitment process; length, style and content of training; supervision; record keeping; supply chain & financing; system of motivation (financial/non-financial); ratio of CHWs to population; details of integration with other health interventions. Relevant malaria contextual information: malaria transmission intensity and seasonality; antimalarial drug policy at time of study. Reported outcomes, including stratification by socioeconomic status and other measures of equity where available. Appropriate outcomes include: clinical outcomes (mortality, anaemia, clinical malaria, parasitaemia etc.); proportion of patients seen by CHW that are diagnosed and treated correctly; correct use of RDT by CHW and treatment according to result; correct referral to formal health facilities where indicated; referral compliance; proportion of febrile patients seeking care within 24 hours. Facilitating factors and barriers to increasing demand and uptake and improving quality of community-based malaria diagnosis and case management, including (but not restricted to): CHW recruitment and training; CHW capacity and availability; financial and non-financial motivation of the CHWs; supervision and commodity supply; and community acceptance. Study quality and strength of evidence, including study design and potential sources of bias. Assessment of study quality Studies with an RCT or CBA design were assessed by two authors (LSP, BW) for risk of bias, based on guidelines from the Cochrane Effective Practice and Organisation of Care (EPOC) group [36]; each study was given a classification of DONE, NOT CLEAR, or NOT DONE in the following categories: Appropriate sample size calculations reported; Completeness of follow-up of participants, e.g., community health worker attrition rate, patient loss to follow-up from trial; Whether those measuring the outcome(s) were blinded to the intervention group; Similarity of intervention and control groups at baseline in terms of the primary outcome; Similarity of intervention and control groups at baseline in terms of socio-demographic characteristics; In the case of RCTs, quality of the randomisation procedure to minimise selection bias. 5

15 6

16 5. Results 5.1 Overview of included studies In total, 2373 papers were found by searching the online publications databases. After their titles and abstracts were screened for relevance by two reviewers, 115 were retained for full paper screening. After double-screening the full papers against the same eligibility criteria, 78 more were excluded, leaving 37 papers from the published literature for final analysis. An additional 3 papers were included from the grey literature (Figure 1). The final 40 papers cover a total of 42 studies, where an individual study is defined as evaluation of an intervention conducted in one site (country). Some papers report on the same intervention conducted at multiple sites, [37-39] and each of these sites is regarded as a separate study for the purposes of this review; thus one paper may report on more than one study. Conversely some studies may have relevant data reported in multiple papers. Figure 1: Flow chart of literature search and screening Community-based malaria treatment CAB Abstracts: 1274 EMBASE: 1802 Global Health: 1311 Medline: 1309 PsychINFO: 101 TOTAL papers: 5797 Community-based demand creation for prompt malaria treatment seeking CAB Abstracts: 197 EMBASE: 646 Global Health 201: Medline: 400 PsychINFO: 32 TOTAL papers: 1476 Removal of duplicates Total papers remaining: 2373 Screening of titles & abstracts Papers from grey literature: 3 Total papers remaining: 115 Screening of full papers Total papers remaining: 40 (42 studies) 7

17 Geographical distribution of studies The 42 final studies covered sixteen countries in sub-saharan Africa. Of these, 24 were conducted in East or Southern Africa (7 in Uganda; 5 in Tanzania; 3 each in Ethiopia & Zambia; 2 in Rwanda; 1 each in Kenya, Malawi, Mozambique & Sudan), and 18 in West or Central Africa (6 in Ghana; 4 in Nigeria; 3 in Burkina Faso; 2 in Cameroon; 1 each in DRC, Mali and Sierra Leone). The majority of studies were conducted in rural settings (78.6%, 33/42), 16.7% (7) in urban settings, and 4.8% (2) in mixed urban/rural settings. Twenty-five of the studies were conducted at sub-district scale, 15 at district level, one at regional level and one at national level (Table 1). Malaria diagnosis & treatment policies Of the 36 studies where malaria treatment was provided by a CHW, 27 were conducted when national policy was presumptive diagnosis of malaria based on fever with or without other clinical symptoms. Four of these assessed the feasibility of RDT use by CHWs in advance of a potential policy change, meaning that CHWs were assessed on their ability to perform an RDT but did not treat according to the RDT result (malaria treatment continued to be provided to all febrile patients). Seven were conducted after a change in policy to require confirmed malaria diagnosis (using RDTs) before treatment in the community; a further two studies spanned the policy change from presumptive to confirmed diagnosis [40, 41]. Diagnosis policy was not specified for the eight studies involving demand generation for prompt malaria treatment seeking, although since all of these were implemented between 2000 and 2006, it is highly likely that presumptive malaria diagnosis was still the recommended practice [2]. Just over half of the studies (25) were conducted with an ACT as the national drug policy for community level management of malaria, eight involved chloroquine (CQ) or sulfadoxine-pyrimethamine (SP) monotherapy (6 and 2 studies, respectively), and four CQ-SP (all in Uganda). Two studies spanned a change in national policy from SP to ACT [40, 42], and three did not specify the first-line anti-malarial policy at the time of the study (all of which related to demand generation activities only). Classification of interventions The studies can be classified according to presumptive or confirmed diagnosis of malaria; whether the intervention involved management of malaria only or integration with other health services; and those that involved community-level mobilisation activities to promote prompt care seeking for malaria without direct delivery of treatment at the community level (Table 1): Sixteen studies involved CCM of malaria alone based on presumptive diagnosis of fever. Eleven involved CCM of malaria alone with confirmed diagnosis based on an RDT; all other nonmalaria fevers as well as any children with danger signs were referred to the nearest formal health facility. Nine studies involved integration of malaria CCM with other health services, such as diagnosis and treatment of acute respiratory infections or pneumonia, diarrhoea; delivery of vitamin supplementation or insecticide-treated nets; promotion of vaccinations. Six studies evaluated community-level interventions to improve care seeking for fever without treatment provision; five of these were specific to malaria and one involved wider health promotion of practices to improve child survival. 8

18 Table 1: Overview of included studies Author, year of publication [Ref] Country, setting & scale Malaria transmission intensity & seasonality Intervention Overview Community mobilisation activities Objectives addressed* Malaria policy at time of study Treatment Diagnosis Community-based management of malaria only; presumptive diagnosis & treatment Ajayi et al., 2008 [37, 43, 44] Akweongo et al., 2011 [38] Chinbuah et al., 2006 [45] Eriksen et al., 2010 [46] Fapohunda et al., 2004 [47] Franco et al., 2008 [48] Ghana (2 sites), Nigeria & Uganda; Rural; District Burkina Faso & Ghana; Urban; Sub-district Ghana; Rural; Sub-district Tanzania; Rural; Sub-district Uganda; Rural; District Rwanda; Rural; District High, perennial High, seasonal High, perennial High, perennial High, perennial Not described Community medicine distributors; presumptive Tx of uncomplicated malaria with ACT; inform caretakers of how to take full dose, danger signs, adverse events; referral of children with danger signs. Community medicine distributors; presumptive diagnosis & Tx with ACT, counselling caretakers on dosage Community-based agents; presumptive diagnosis & Tx of malaria with AL; identify danger signs for referral. Community meetings; 'handouts' of Tx instructions given to caretakers. Not described AS-AQ (Burkina Faso & Ghana); AL (Burkina Faso) Initial community meetings to explain rationale for study, discuss number of CHWs needed, nomination and acceptance of CHW. IEC on AL given to community groups for first 2 months AS-AQ Presumptive (Ghana); AL (Nigeria, Uganda) Presumptive AL Presumptive Women leaders; presumptive Tx of malaria with SP given as DOT; follow-up Community sensitisation meetings after selection & training of women SP Presumptive on days 2&3. Referral of severe malaria or leaders on aims of project & need to non-malaria illness. All children also received paracetamol for 2d. seek prompt Tx for fever from women leaders. Community drug distributors; Sensitisation of communities & CQ-SP Presumptive management of febrile under-fives, meetings to select CDDs. CDDs presumptive Tx using pre-packaged CQ-SP counselled communities on need for (Homapak). prompt care seeking for malaria. Community health workers; presumptive Village sensitisation meetings AL Presumptive Tx of fever with pre-packaged AL (PRIMO); RDTs & ICCM pilot just starting at time of study. 9

19 Author, year of publication [Ref] Country, setting & scale Malaria transmission intensity & seasonality Intervention Overview Community mobilisation activities Objectives addressed* Malaria policy at time of study Treatment Diagnosis Kallander et al., 2006 [49] Kidane & Morrow, 2000 [50] Kouyate et al., 2008 [51] Nsabagasani et al., 2007; Nsungwa- Sabiiti et al., 2007 [52, 53] Sirima et al., 2003 [54] Staedke et al., 2009 [55] Uganda; Rural; Sub-district Ethiopia; Rural; Sub-district Burkina Faso; Rural; Sub-district Uganda; Rural; National Burkina Faso; Rural; Regional Uganda; Urban; Sub-district High, perennial High, seasonal High, seasonal High, perennial High, seasonal Medium, perennial Community drug distributors; management of febrile under-fives, presumptive Tx using CQ-SP (Homapak); danger signs for immediate verbal referral to nearest HF; follow-up after 3d. Referral slips or counter-referral feedback from HFs not practiced. Mother co-ordinators to teach all mothers in neighbour group to recognise possible malaria and correctly treat with CQ (supplied to all HHs by mother coordinator); pictorial treatment charts of dosage used as job aids; referrals; record births & deaths. Women leaders provided educational meetings with other mothers in their subvillage; held pre-packaged CQ & paracetamol stocks for presumptive Tx of febrile children; supervision of first dose, home visits on days 2&3; referral of severe illness. Community drug distributors (CDD); free distribution of pre-packaged CQ-SP (Homapak) to all under fives with fever; counselling sessions during consultations & village meetings using pictorial charts on Tx seeking. Community health workers; promote recognition of symptoms of uncomplicated malaria, provide advice on Tx & referral as necessary; sell caretakers pre-packaged CQ & aspirin; emphasis for malaria Tx on caretakers. Mothers in intervention arm given 3 packs pre-packaged AL at randomisation, stocks replenished to this level as necessary 10 Not described CQ-SP Presumptive Initial community meetings; mother coordinators spread health promotion messages to their neighbour group mothers on recognition of malaria symptoms, correct Tx with CQ. Sensitisation campaign conducted in intervention villages after selection & training of women leaders to mobilise mothers. Women leaders spent half day with ~15 mothers to encourage prompt Tx seeking for CQ treatment. CDDs mobilised communities to seek prompt care for fever through village meetings and consultations. Sensitisation tours of communities, involving treatment charts, posters, radio jingles. CQ Presumptive CQ Presumptive CQ-SP Presumptive CQ Presumptive Not described AL; CQ-SP for HBMF Presumptive

20 Author, year of publication [Ref] Country, setting & scale Malaria transmission intensity & seasonality Intervention Overview Community mobilisation activities Objectives addressed* Malaria policy at time of study Treatment Diagnosis during monthly visits by study team. Community-based management of malaria only; confirmed diagnosis with RDT & treatment Akweongo et al., 2011 [38] Chanda et al., 2011 [56, 57] Elmardi et al., 2009 [58] Harvey et al., 2008 [59] Hawkes et al., 2009 [60] Ishengoma et al., 2011 [40] Lemma et al., 2010 [41] Ethiopia & Ghana; Urban; Sub-district Zambia; Rural; Sub-district Sudan; Rural; Sub-district Zambia; Rural; Sub-district DR Congo; Rural; Sub-district Tanzania; Rural; Sub-district Ethiopia; Rural & urban; District Low (Ethiopia) High (Ghana), seasonal Medium, perennial Medium, seasonal Not described High, perennial Medium, perennial High, seasonal Community health volunteers; Tx of RDT+ve children with ACT, counselling caretakers on dosage. Community health workers; Tx of RDT+ve children with AL; referral of RDT -ve with suspected non-malaria illness to nearest HF. Malaria control assistants; Use of RDTs & AS-SP; pre-referral rectal AS for severe malaria; village health committees formed to support, facilitate & supervise MCAs. Existing CHWs divided in to 3 arms to assess effectiveness of RDT use: manufacturer's instructions only, job aidonly, job aid plus training. Performance assessed in local HC with real patients, though all treated presumptively as per current policy. Existing CHWs trained on incorporation of RDTs in to malaria management. Performance assessed in village with real patients, though all then treated presumptively as per current policy. Community-owned resources persons; Presumptive Tx with SP changed to AL, confirmed diagnosis for >5s (Jan 07); referral of severe cases. Year 1: CHWs in intervention district treated fever presumptively as falciparum malaria with AL; Year 2: Half of intervention CHWs given RDTs, RDT+ve 11 Not described AL (Ethiopia); AS-AQ (Ghana) Confirmed (RDT) Not described AL Confirmed (RDT) Community leaders sensitised communities early in intervention; MCA graduation ceremony. MCAs promote appropriate malaria Tx in community. AS-SP Confirmed (RDT) Not described ACT (nonspecified) Presumptive Not described AS-AQ Presumptive Villages informed to consult CORPs for febrile illness at any time of day or night. Not described SP then AL Presumptive then confirmed AL for treatment of falciparum Presumptive then confirmed

21 Author, year of publication [Ref] Country, setting & scale Malaria transmission intensity & seasonality Intervention Overview Community mobilisation activities Objectives addressed* Malaria policy at time of study Treatment Diagnosis Mubi et al., 2011 [61] Ngasala et al., 2011 [62] Thomson et al., 2011 [63] Tanzania; Rural; Sub-district Tanzania; Rural; Sub-district Sierra Leone; Rural; Sub-district High, seasonal High, seasonal High, perennial Community-level malaria management integrated with other health services Akweongo et al., 2011 [38] CDI Study Group, 2008; Ndyomugyenyi et al., 2010 [39, 64] Malawi; Urban; Sub-district Cameroon, Nigeria & Uganda; Rural; District Medium Not described treat with AL, RDT-ve assume vivax and treat with CQ. CHWs in control district malaria prevention only. Community health workers; comparison of clinical versus RDT-confirmed diagnosis of malaria; Tx with pre-packaged AL (first dose supervised), paracetamol for all febrile patients; recognition of danger signs for referral; follow-up on days 3&7. Community health workers; Tx of RDT+ves with AL; referral of non-malaria fever, severe cases. Community malaria volunteer; Tx of RDT+ves with AS-AQ & paracetamol; recognition of danger signs & referral of RDT-ves to nearest HF. Health surveillance agents; presumptive diagnosis & Tx with ACT, counselling caretakers on dosage. Plus: Vaccinations, growth monitoring, nutrition, water & sanitation, family planning. Community directed intervention (CDI); presumptive Tx of malaria with antimalarial. Plus: Different CDI tasks added in intervention districts each year (Vitamin A, TB-DOTS, ITNs for U5s & PW); impact on existing CDI for delivery of ivermectin (oncho) assessed. Sensitisation meetings to inform communities about the purpose and details of the study malaria; CQ for vivax AL Presumptive Not described AL Confirmed (RDT) Not described AS-AQ Confirmed (RDT) Not described AL Presumptive Community members collectively decide health interventions, identify resources; plan how, when, where, by whom intervention will be implemented; community-directed implementer. Health staff facilitate process: training, supervision, supplies AL (Cameroon ); ACT (Nigeria); CQ-SP (Uganda) Presumptive 12

22 Author, year of publication [Ref] Country, setting & scale Malaria transmission intensity & seasonality Intervention Overview Community mobilisation activities Objectives addressed* Malaria policy at time of study Treatment Diagnosis Kelly et al., 2001; Rowe et al [65, 66] Mukanga et al., 2011 [67] Rwanda MOH, 2009 [68] Winch et al., 2003 [69] Yeboah-Antwi et al., 2010 [70] Kenya; Rural; District Uganda; Rural; Sub-district Rwanda; Rural; District Mali; Rural; Sub-district Zambia; Rural; Sub-district High, perennial High, perennial Not described Not described High, seasonal Community health workers; use of modified IMCI algorithm to diagnose malaria; referral of severely ill children to HFs; counsel caregivers on correct Tx and preventive behaviours. Plus: Integrated management of ARIs & diarrhoea (also clinical diagnosis only). Not described SP Presumptive Community health workers; use of RDTs Not described AL Presumptive to diagnose malaria and treat with AL. Plus: use of respiratory rate timers to diagnose pneumonia & treat with an antibiotic. Community health workers; expanded Not described AL Presumptive community-imci tasks added to current CHWs providing presumptive Tx for malaria. Plus: Integrated management of pneumonia & diarrhoea (also clinical diagnosis only). Village drug kit managers; Additional Village meetings in intervention zone to CQ Presumptive training on CQ doses to recommend & explain improvements to drug kit sell; use of pictorial visual aids to assist system & encourage Tx seeking from counselling tasks, recognition of danger drug kit managers signs for referral to primary HF; training at HF level on referral system from community level. Plus: drug kits have paracetamol, eye ointment, ORS, bandages. Tx of RDT+ve with AL & anti-pyretic in intervention group, presumptively in control group; identification of danger signs for referral Plus: Diagnosis of uncomplicated pneumonia using timer to measure respiratory rate; intervention group treated with amoxicillin, control group 13 Not described AL; SP for CCM Confirmed (RDT)

23 Author, year of publication [Ref] Country, setting & scale Malaria transmission intensity & seasonality Intervention Overview Community mobilisation activities Objectives addressed* Malaria policy at time of study Treatment Diagnosis referred to nearest HF. Community-level promotion of prompt treatment seeking for fever (no direct supply of drugs) Ajayi, 2008 [71] Alba et al., 2010; Hetzel et al., 2007 [42, 72] Chirdan et al, 2008 [73] De La Cruz et al., 2009 [74] Nkuo Akenji, 2005 [75] Nigeria; Rural; District Tanzania; Rural & urban; District Nigeria; Urban; Sub-district Ghana; Rural; Sub-district Cameroon; Rural; Sub-district High, perennial High, perennial Not described Not described High, perennial Mother trainers to provide education & pictorial guidelines but not drugs; facilitate home Tx by caretakers with CQ from usual source. Medicine vendors trained on correct prescription of CQ, danger signs for referral. Social marketing to generate demand for prompt and appropriate Tx of fever from HFs & accredited drug distribution outlets (ADDOs) with trained providers. Education of mothers to improve prompt & effective Tx of fever from usual source; posters & CQ charts used as job aids. Malaria education delivered via microfinance groups to improve prompt & effective Tx of fever from usual source. Community health education to promote prompt Tx seeking for fever from rehabilitated health post, run by research team. Initial community sensitisation meetings. Mother trainers provide education to neighbouring mothers on malaria, diagnosis & Tx. Pictorial Tx guideline (made using participatory approach) distributed by mother trainers. Community sensitisation meetings (av. 40 leaders) & road shows (100s to 1000s attended)in every village with dancing competition, comedies, lecture, short films, Q&A, promotion materials; billboards, posters. Random selection of 150 mothers; divided in to 8 groups and given three 45minute training sessions on cause of malaria, recognition of symptoms, danger signs, Tx & prevention. Intervention microfinance groups, compared to comparison credit associations which received diarrhoea education only, and non-clients. Twice weekly community meetings on causes of malaria, signs & symptoms, preventive measures, early Tx seeking and appropriate home Tx, referral; carried out by doctor, nurse & field technicians (from rehabilitated health post) using charts & pictures CQ Presumptive Policy change from SP to AL during study period Presumptive Presumptive Presumptive Presumptive 14

24 Author, year of publication [Ref] Country, setting & scale Malaria transmission intensity & seasonality Intervention Overview Community mobilisation activities Objectives addressed* Malaria policy at time of study Treatment Diagnosis Edward et al., 2007 [76] Mozambique Rural; District Not described Community health volunteers; promotion of prompt Tx seeking for malaria, recognition of danger signs, use of ITNs. Plus: Promotion of integrated package of child survival interventions: breast feeding, ORS for diarrhoea, nutrition, supervised delivery, immunisations, Tx of pneumonia at HF; Integration of community surveillance & HMIS. 173 'care groups' each with female community health volunteers. Each volunteer assigned 10 neighbouring HHs, visited monthly to provide health education & register vital events. Support to strengthen VHCs, pastors & traditional healer involvement. Not specified * Themes addressed by the objectives: (1) Quality of malaria CCM; (2) Referrals from community to health facility level; (3) Health system capacity to support malaria CCM; (4) Integration of malaria CCM with other health services; (5) Community level strategies to increase treatment seeking for fever. Abbreviations: ACT=artemisinin combination therapy ; AL=artemether-lumefantrine; AQ=amodiaquine; ARI=acute respiratory infection; AS=artesunate; CDD=community drug distributor; CDI=community-directed intervention; CHW=community health worker; CORP=community-owned resource person; CQ=chloroquine; HBMF=home-based management of fever; HF=health facility; HMIS=health management information system; ICCM=integrated community case management; IMCI=integrated management of childhood illness; ITN=insecticide-treated net; MCA=malaria control agent; ORS=oral rehydration salts; PW=pregnant women; RDT=rapid diagnostic test; SP=sulfadoxine-pyrimethamine; Tx=treatment; U5=child under five; VHC=village health committee Presumptive 15

25 Characteristics of CHWs Various titles were given to trained community members in the studies, with the most common being community drug/medicine distributors (CDD/CMD) or community health workers (CHW); other titles included community health volunteers, health surveillance agents, community implementers, community-owned resource persons, women leaders and mother coordinators. In the current document, the generic term community health worker is used unless referring to a specific programme (Table 1). In the vast majority of studies, the CHWs were selected by their own communities during or following sensitisation meetings. One exception was a RCT in Tanzania where the community nominated suitable candidates and the study team made the final selection [46]; another was the Malawi site within a multicountry study of malaria CCM in urban settings where the community implementers were health surveillance agents employed by the ministry of health (MOH) who did not necessarily work in their own communities [38]. Common requirements or characteristics of CHWs were that they should be permanent residents in the community (with the exception of Malawi, described above), available, trusted and respected, demonstrating a willingness to serve/volunteer, and having basic literacy skills for simple record keeping. Where described, most CHWs were farmers, mothers or unemployed persons in the rural studies; occupations in the urban studies were more varied and included shop keepers, petty traders, labourers, teachers or students, as well as those that were unemployed. Twenty-five of the studies reported the gender balance of CHWs, with a median of 39% male (interquartile [IQR] range: 6%-75%). Twenty-one studies reported sufficient data to estimate the ratio of CHW to total population in their catchment area with a median of 553 people per CHW (IQR range: ) (Tables 2a, 2b, 2c & 3). Training and supervision Twenty-eight of the studies reported the length of training received by the CHWs, which seemed to grow with increasing complexity of the interventions they were expected to deliver. For example, CHWs presumptively treating only malaria cases received a median of 3 days training (data for 12/16 studies; IQR range: 2-4 days). Interestingly, those using RDTs to confirm malaria cases also received a median of 3 days training (data for 9/11 studies; IQR range 3-7 days); although in seven of these 11 studies the CHWs were already trained to implement presumptive malaria treatment and so the training on RDT was additional with some general refresher training [41, 57-63]. For those CHWs delivering multiple interventions along with malaria treatment, the median length of training (where reported) was 30 days (data for 5/9 studies; IQR range: days); village drug kit managers in Mali had a 7-week literacy course before they received specific drug kit management training [69]. The main topics covered included: appropriate anti-malarials to give to a child diagnosed with malaria (presumptively or with an RDT), including dose and duration; importance of counselling caretakers on taking the full treatment and prompt future care seeking; recognition of danger signs and referral to nearest health facility. Seventeen of the studies explicitly described the training methods as interactive and had opportunity for practice; in the remainder, insufficient detail was given to determine the training style. Only six studies explicitly described any refresher training [38, 40, 41, 65, 70], although as previously discussed some of the studies were building upon an existing network of CHWs, so any new training delivered is likely to have included some level of refresher training on key principles of malaria treatment. 16

26 Frequency of supervision varied between studies, with the most commonly reported frequency being monthly (although it was not always clear whether this was the intended or actual frequency of supervision); in some of the studies, the CHWs were monitored more frequently, e.g., every other day in a Zambian RCT [70], weekly in one study in Ghana and Malawi [38] and fortnightly in a cluster RCT in Tanzania [62], often by members of the study team. Supervision by local health workers or district staff was also common. In some cases, supervision was via monthly meetings at the nearest health facility of all CHWs within the catchment area with chance for problem solving. Supervisory visits or meetings were often reportedly used to collect report forms and to collect/receive new drug and equipment stocks. User charges and CHW motivation Of the 36 studies in which anti-malarials were dispensed at the community level (Tables 2a, 2b, 2c), 11 charged a small amount per dose of anti-malarial (ranging from US$0.05 for chloroquine to US$0.45 for an ACT); all of which were in West Africa (Burkina Faso, Ghana, Mali, Nigeria), with the exception of the home management of malaria programme in Rwanda [48]. In 17 studies, all of which were in East or Southern Africa, anti-malarials were distributed for free; eight studies did not provide information on any charges to the user. Community health workers in 13 of the 36 treatment studies received some financial incentive, ranging from a small commission of a few US cents per dose of anti-malarial sold, to a monthly allowance of US$15 or US$25 in a couple of cases (both in Tanzania [61, 62]). In 7 studies, non-financial incentives were provided in the form of bicycles or other equipment or payment in kind from neighbours; in three, intrinsic motivation such as pride and satisfaction in serving their community was specified. Only two studies explicitly stated that no motivation was given at all, although 14 studies did not provide any information on CHW motivation. Study design Seven studies used a cluster RCT to evaluate the intervention, 5 used a controlled before and after (CBA) design, 5 before and after without control, and 4 studies were post-only with a control. The most popular study design was post-only without control, of which there were 21 studies (Table 2); all of these involved a direct evaluation of CHW performance, for example via review of CHW registers, or household surveys with questions on treatment by source, hence the quality of performance could be related to the intervention and any contextual factors described. 17

27 Table 2: Overview of included studies according to type of intervention, scale of implementation and evaluation design RCT cluster or non-cluster Before & after with control Before & after without control Post only with control Post only without control Malaria CCM only with presumptive diagnosis Malaria CCM only with confirmed diagnosis Malaria CCM integrated with other health services Demand generation for malaria treatment Sub-district scale: Eriksen et al., Tanzania [46] Kidane & Morrow, Ethiopia [50] Kouyate et al., Burkina Faso [51] Staedke et al., Uganda [55] Sub-district scale: Mubi et al., Tanzania [61] Sub-district scale: Winch et al., Mali [69] Yeboah-Antwi et al., Zambia [70] National scale: Nsungwa-Sabiiti et al., Uganda [52] District scale: Fapohunda et al., Uganda [47] District scale: Lemma et al., Ethiopia [41] District scale: Ajayi et al., Nigeria [71] Sub-district scale: De La Cruz et al., Ghana [74] Sub-district scale: Chinbuah et al., Ghana [45] District scale: Alba et al., Tanzania [42] Edward et al., Mozambique [76] Sub-district scale: Chirdan et al., Nigeria [73] Nkuo Akenji et al., Cameroon [75] Sub-district scale: Harvey et al., Zambia [59] District scale: CDI Group, Cameroon, Nigeria, Uganda [39] Regional scale: Sirima et al., Burkina Faso [54] District scale: Ajayi et al., Ghana (x2), Nigeria, Uganda [43] Franco et al., Rwanda [48] Sub-district scale: Akweongo et al., Ghana, Burkina Faso [38] Kallender et al., Uganda [49] Sub-district scale: Akweongo et al., Ghana, Ethiopia [38] Chanda et al., Zambia [57] Elmardi et al., Sudan [58] Hawkes et al., DRC [60] Ishengoma et al., Tanzania [40] Ngasala et al., Tanzania [62] Thomson et al., Sierra Leone [63] District scale: Akweongo et al., Malawi [38] Kelly et al., Kenya [65] Rwanda MOH [68] Sub-district scale: Mukanga et al., [67] 18

28 The outcomes and findings of each of the four classifications of study are presented here in more detail, as follows: (i) Presumptive community case management of malaria; (ii) RDT-confirmed community case management of malaria; (iii) Community case management of malaria integrated with other health services; and (iv) Community-level demand generation for care seeking for malaria. 5.2 Presumptive community case management of malaria Sixteen studies across seven countries investigated presumptive treatment of malaria by CHWs (Table 3): five studies in Uganda; four in Ghana; three in Burkina Faso; and one each in Ethiopia, Nigeria, Rwanda and Tanzania. The majority of studies evaluated interventions conducted at the sub-district or district level (8 and 6 studies, respectively); one study in Burkina Faso evaluated implementation at the regional level [54] and one in Uganda evaluated the national home management of malaria programme [52, 53]. Four studies employed a cluster RCT design, two were controlled before and after studies, one before and after without a control, and the remaining nine were post only without control. Eight studies reported impact of the intervention in terms of a clinical or health outcome; fourteen reported at least one indicator of the quality of CHW performance; four reported on referral rates by the CHW and three of these reported the completion rates of referrals at a formal health facility. Impact on clinical outcomes In total, eight of the sixteen studies reported some indicator of clinical impact of the CHW intervention. A cluster RCT by Kidane & Morrow in Tigray Region of Ethiopia found a 40.6% reduction in all-cause under-five mortality (95% CI; 29.2%, 50.6%) in the intervention clusters compared to control clusters (29.8 versus 50.2 deaths per 1000 child-years, respectively) [50]; mother coordinators in intervention villages held CQ stocks and educated other mothers to seek treatment promptly from them for fever. Evaluation of an intervention in Burkina Faso found that 5.1% (93/1806) of children promptly receiving pre-packaged CQ progressed to severe malaria, compared to 11.0% (153/1396) for those that didn't (adjusted OR: 0.47; 95%CI: 0.35, 0.64; p<0.001) [54]. Three more recent cluster RCTs have found mixed clinical impact. For example, in the intervention arm of a study in Kampala, Uganda, mothers were given AL supplies and responsibility for treating their children's fever in the home; mothers in the control arm followed usual treatment practice. Although 51.5% (444/862) febrile illnesses in the home management arm were treated promptly and effectively, compared to 5.2% (30/570) in the standard care arm, there was no significant difference in prevalence of moderate anaemia, haemoglobin level or splenomegaly. However, parasite prevalence was significantly lower post-intervention in the home management arm (2%, 4/200) compared to standard care arm (10%, 17/170; p=0.006) [55]. Studies conducted in Burkina Faso and Tanzania used similar models of training women leaders in intervention villages to educate neighbouring mothers and provide malaria treatment [46, 51]. Both found that although moderate anaemia decreased in the intervention villages, it also decreased in control villages over the intervention period, suggesting broader health improvements may have been responsible rather than the CHW intervention itself: in Burkina Faso prevalence of moderate anaemia decreased from 28.0% to 16.7% in intervention villages, and 29.9% to 14.5% in control villages (p=0.32) [51]; in Tanzania the reduction was from 43.9% to 0.8% in intervention villages, and from 30.8% to 0.17% in control villages (p=0.04) [46]. Three of the four sites that participated in the TDR multi-country study of ACT use for home management of malaria investigated polymerase chain reaction (PCR)-adjusted cure rates of a subsample of patients treated with ACT by a CHW. Twenty-eight days after treatment, 90.9%, 91.4% and 19

29 97.2% of patients had cleared their original infections in the Nigeria, Ghana and Uganda sites, respectively [37], supporting that ACT can be effectively administered by CHWs and adhered to by the users of these services. Impact on CHW quality Eleven of the sixteen studies reported the proportion of febrile children given the correct dose of antimalarial. Nine of these were post-only studies where the data was collected from the CHW register; one was a cluster RCT and one a CBA study for both of which the data were collected using household surveys. The proportion of febrile children given the correct dose of anti-malarial as extracted from CHW registers was high across all studies, ranging from 92.0% in Rwanda [48] to 99.0% in Ghana [43]. Where household survey data was also available on this indicator for children visiting a CHW, two studies in Ghana found similarly high proportions of febrile children receiving the correct dose (93.1% and 95.3% [38, 45]); however one study in urban Burkina Faso found that only 25.9% caretakers reported receiving the correct dose of anti-malarial from a CHW, compared to 94.8% from the CHW registers [38]. The reason for this discrepancy is not clear from the paper, although it may be due to reporting bias by the CHW and issues of recall for the caretakers. The CBA study reporting correct dose of anti-malarial was an evaluation of the HMM programme in Uganda, comparing baseline treatment practices with those twelve months after implementation of the HMM strategy in districts with or without the intervention [47]. The proportion of febrile children receiving the correct dose of anti-malarial increased from 14% (134/959) to 37% (671/1815) in the intervention districts, and from 4% (17/431) to 7% (70/995) in the control group, giving an overall 20% increase in the intervention districts (p<0.001). The cluster RCT in Burkina Faso trained women leaders in intervention villages to provide education and pre-packaged CQ to other mothers; the control villages sought treatment for fever from usual sources [51]. Treatment of fever with CQ increased from 36.9% to 86.3% (p=0.002) in intervention villages, and from 65.9% to 67.7% in control villages (p=0.71), giving an overall increase of 47.6% in intervention villages. It is important to note that the indicator was not reported by source of treatment for either study and so may include children treated at sources other than the CHW; however, 88% of febrile children in the intervention villages in Burkina Faso were treated by women leaders [51]. Four studies reported the proportion of febrile children treated promptly by a CHW which ranged from 85.8% in Uganda [43], to 96.2% in Nigeria [43]. Eight post-only studies reported the combined indicator of proportion of febrile children receiving prompt and correct malaria treatment from a CHW, which demonstrated a considerably wider range, spanning from 18.4% in urban Burkina Faso [38] to 87.2% in rural Ghana [38], with a median of 69.8%. Data on both of these indicators was derived from household surveys. In Uganda, a CBA study evaluating the HMM programme (conducted one year after the one previously reported by Fapohunda et al.) found that prompt and effective treatment increased from 7.4% (12/163) to 13.5% (21/156) in intervention districts, and declined from 7.3% (7/96) to 0% in comparison districts; giving an overall 13.5% increase in intervention districts (p=0.01) [52]. Again, this figure is for all febrile children regardless of where they sought treatment; however, 32.3% of those treated with the HMM pre-packaged drug, Homapak, received the correct dose promptly compared to 1.3% treated with other anti-malarials (OR: 37.0; 95%CI: 4.8,286; p=0.001). This is the only study reviewed that reported results according to wealth quintile: 23% of febrile children in the poorest wealth quintile received Homapak, compared to 50% in the least poor quintile (numbers not reported), giving an equity ratio of 0.46 [52]. 20

30 Impact on referrals Only four of the sixteen studies reported on any indicator of referral. One study in Ghana reported that 29.4% (5/17) of children with severe signs were referred to the nearest health facility; of these 60% (3/5) completed the referral [45]. Interviews with the CHWs revealed that this low level of referral may be due to their having followed-up severe cases in the patients homes to monitor progress, rather than having immediately referred them to a health facility [45]. A study in Tanzania found that 2.2% (59/2675) of all children seen by the CHWs were referred to a health facility; the proportion that completed the referral was not reported [46]. A review of the records of 72 CHWs in Rwanda found that 92 children had been referred over a 3-month period (total number of children seen was not reported); of these, 70% (64/92) had completed counter-referral forms from the health facility, indicating that they had acted on the referral advice [48]. A case series follow-up of the referrals of 40 community drug distributors (CDD) in Uganda found an overall referral rate of 8% (117/1454). Sixteen of these were lost to follow-up and 24 were excluded due to non-febrile illness; 87% (67/77) of those children followed up at home had completed their referral at a hospital or health centre [49]. Promptness of referral was more likely for children under one and for those classified as urgent by the CDD; the main barriers reported were lack of money, no instructions from the CDD or improvement of the child s condition [49]. Impact on health system capacity to support CCM Three main areas can be described to understand health system capacity to support CCM: (i) ability to treat referred cases; (ii) regular supervision of CHWs; and (iii) reliable and sufficient medical supplies at the health facility and with the CHW. Referral completion was reasonably high for the studies in Ghana, Rwanda and Uganda [45, 48, 49] where it was reported, suggesting that children referred from the community in these settings were able to access the health facility for care and that the health workers recognised and treated children referred from a CHW. An additional study in Uganda measured the capacity of health facilities to support referrals by assessing the proportion of health workers at the facility with training on integrated management of childhood illness (IMCI) and knowledge of the management of severe malaria; it was found that fewer than 50% of health workers in the study districts had sufficient knowledge of either [47]. The remaining twelve studies investigating presumptive treatment of malaria by CHWs did not provide any information on referrals, including the capacity of the formal health system to support them. Ten of the sixteen studies reported that supervision visits were conducted by local health workers or members of the district health team; for the other six studies, supervision was provided by the study investigators (4) or not specified (2). Four of the ten studies explicitly described training health facility staff on skills and knowledge required for supporting the intervention [46, 48, 51, 54], and one study described holding monthly meetings with the study team, district staff and CHWs to resolve problems [46]. Reported frequency of supervision varied from weekly or twice weekly by study teams [38, 45, 46] to monthly or longer by local health workers. Although it is hard to quantify the impact of training or other interventions (such as providing money for transport [43]) on the capacity of health workers to conduct regular effective supervision of CHWs, a number of studies reported qualitative information collected from stakeholders on supervision. For example, CHWs negative experiences with formal health workers in Nigeria led them to bypass health facilities in favour of supervision by community leaders and provision of drug supplies directly from the district stores [44]. Early evaluations of the HMM programme in Uganda found that skill levels of health workers were insufficient to provide 21

31 effective supervisory support or treatment of referred cases [47]; CDDs and communities were concerned about lack of supervisory support from health workers which undermined motivation [53]. Evaluation of the HMM programme in Rwanda also found infrequent supervision of CHWs due to other priorities, lack of time and shortages of fuel [48]. In twelve of the sixteen studies, drugs were supplied to CHWs through the district system, usually during supervision visits; however in some cases it was not clear whether the research team paid for the drugs made available to the CHWs. The RCT in Kampala explicitly stated that the monthly anti-malarials supplied to mothers in the intervention arm were funded by the study [55]; three studies did not describe the drug supply mechanism. In two studies, pharmacy staff responsible for monitoring CHW drug supplies received training as part of the intervention [48, 54]. Availability of drugs at the CHW or district level was only reported for two of the studies: the early evaluation of the HMM programme in Uganda found serious problems with drug shortages [47]; conversely the evaluation of the Rwanda HMM programme found good availability of AL from CHWs and health facilities [48]. Sustainability The median length of implementation of the interventions under study was 12 months (IQR: months), meaning that the results represent the quality of CHW performance after a reasonable length of time. One potential barrier to sustainability is the issue of financial motivation for the CHWs, which was raised in just under half (7/16) of the studies [43-45, 48, 53]. There was no apparent pattern to this issue, for example financial motivation was requested by CHWs at sites where small incentives were already provided [43, 48, 71] as well as by those without any financial reward [45, 53]. User charges for drugs or consultations could present another potential barrier. However, five of the eight studies which involved user charges (ranging from US$0.05 per dose in Burkina Faso [54] to US$0.64 in Nigeria [43]) reported that caretakers appreciated the lower costs associated with seeking treatment from a CHW compared to their usual source of care [38, 43, 44, 48, 54, 71]. For the remaining three, the topic was not discussed, although utilisation of the CHWs for treatment of fever was high [43, 51]. 22

32 Country; Scale; Int length [Ref] Tanzania; Sub-district; 11mo ( ) [46] Ethiopia; Sub-district; 12mo ( ) [50] Table 3: Community-based management of malaria based on presumptive treatment: Intervention details and evaluation outcomes Burkina Faso; Sub-district; 2 years ( ) [51] Uganda; Sub-district; 18mo ( ) [55] CHW characteristics & recruitment; % male; CHW ratio: popn Existing member of women s group, primary school education, permanent resident; nominated by village & selected by study group. 0% male. Ratio: 450 Neighbour groups of 20 HHs with U5s formed by consensus of community leaders, women's associations, mothers. Each group selected mother coordinator. 0% male. Women Leaders selected by community based on: permanent residence, age 30-50y, honesty, respect. 0% male. Mothers in intervention arm given AL supplies and responsibility for treating children's fever in the home; mothers in control arm followed usual Tx practice Length & style of training 4d lectures, 3d practice (hospital setting). HWs given refresher training on malaria case management & oriented to study objectives. Length of training not detailed. No refresher training described. 2d (discussions, practical sessions, pictorial job aids) One refresher session over study period Not described Supervision Supply User charges & CHW motivation 2-weekly, collect records; monthly meetings at district hospital with study team. Monthly by study supervisors, collection of reports Monthly supervision by HWs, collection of report forms, drug provision. 1-3 monthly by study team. Monthly visits by study team to replenish drug stocks, collect diaries Drug supplies restocked every 2w during supervision Supplies of CQ from HF, responsibility of mother coordinator First 6mo drug stock free, then revolving fund to buy new stock; drugs delivered by supervisors. Replaced monthly by study team User charges: Free CHW Incentives: US$20 per month for period of study User charges: Not described CHW Incentives: Not described User charges: Cost per dose US$ (CQ & paracetamol) CHW Incentives: US$ per dose sold. User charges: Free CHW Incentives: None 23 Evaluation design & Key outcomes Cluster RCT 97.8% (2616/2675) febrile U5s seen by women leaders received SP & paracetamol. 2.2% (59/2675) referred to HF; completion rate N/A. Prevalence of moderate anaemia (<8g/dL) pre- & post-intervention: 43.9% to 0.8% in intervention & 30.8% to 0.17% in control villages (p=0.04). Cluster RCT All-cause U5 mortality of 29.8 versus 50.2 per 1000 child-years in intervention versus control clusters; mortality rate reduction of 40.6% (95% CI; 29.2%,50.6%); no difference for age & sex. 19% (13/70) potential malaria deaths in intervention versus 57% (68/120) in control clusters. Cluster RCT Prevalence of moderate anaemia (<24% haematocrit) before & after: 28.0% to 16.7% in intervention & 29.9% to 14.5% in control villages (p=0.32). Tx of fever with CQ before & after: 36.9% to 86.3% (p=0.002) in intervention & 65.9% to 67.7% in control villages (p=0.71); overall +47.6% diff (p<0.001) Facilitators & Barriers Facilitators: In addition to intervention, increased ITN levels & food security may explain reductions in intervention & control villages. Barriers: Potential contamination between intervention & control areas; anaemia may not be most appropriate outcome as nonspecific for malaria intervention Facilitators: Post-conflict setting means disciplined population used to self support, strong community solidarity, no alternative income opportunities for mother coordinators Facilitators: Strong experience of community support, most villages have women's agricultural groups; intervention increased coverage of appropriate malaria Tx (but not health impact) Barriers: Potential contamination between intervention & control villages; higher malaria Tx n control villages at baseline; increasing CQ resistance. Cluster RCT Facilitators: Having AMs in home improved 51.5% (444/862) febrile illnesses in HMM prompt Tx of fever, but likely to have arm treated effectively <24h, 5.2% resulted in considerable over treatment in (30/570) in standard care arm. this urban setting Parasite prevalence in HMM arm 2% Barriers: Higher cost and few benefits when (4/200) post-intervention, 10% in compared to standard care, authors standard care arm (17/170; p=0.006); no conclude this approach may not be other significant changes in clinical appropriate for an urban setting such as outcomes. Kampala Median provider & user costs of US$33.83 & US$6.00 for HMM, US$23.16 & US$8.74 for standard care per child per year;

33 Country; Scale; Int length [Ref] Uganda; District; 12mo ( ) [47] CHW characteristics & recruitment; % male; CHW ratio: popn CDDs selected by community (no further details) Length & style of training Supervision Supply User charges & CHW motivation 2 days Not described Not described User charges: Free CHW Incentives: Not described Evaluation design & Key outcomes Before/after with control Tx seeking <24h: 50% (241/483) & 58% (1053/1816) before & after in intervention districts, 31% (41/134) & 52% (517/995) in control group; overall - 11% diff (p=0.002) Tx with correct AM: 14% (134/959) & 37% (671/1815) before & after in intervention districts, 4% (17/431) & 7% (70/995) in control group; overall +20% diff (p<0.001) Facilitators & Barriers Facilitators: Improvements in prompt care seeking and Tx with correct AM. Barriers: Considerable problems with stockouts at HFs supporting CDDs. Only 12 months after start of HMM programme, not all districts achieved full implementation. Insufficient skills at HF to supervise & treat referrals. Need to improve use of CDD reports. Uganda; National; 2 years ( ) [52, 53] CDDs selected by community Ratio: 250 3d. No refresher training described. Monthly supervision by local HW, quarterly by district Drug stocks replenished from nearest HF User charges: Free CHW Incentives: None Before/after with control Tx seeking <24h: 52.2% (85/163) & 76.8% (120/156) before & after in int group, 26.0% (25/96) & 64.5% (49/76) in comp group; overall -13.8% diff (p=0.12) Prompt & effective Tx: 7.4% (12/163) & 13.5% (21/156) before & after in int group, 7.3% (7/96) & 0% in comp group; overall +13.5% diff (p=0.01) 32.3% (13/39) those treated with Homapak received full dose <24h; 1.3% (1/72) treated with other AM (OR: 37.0; 95%CI: 4.8, 286; p=0.001). 23% Homapak use by febrile children in poorest wealth quintile, 50% in least poor (n=?), equity ratio 0.46 Facilitators: High appreciation of Homapak as accessible, free & effective in reducing severe malaria; Pre-packaging helped improved adherence to dose & duration Barriers: Rapid national scale-up meant Insufficient community sensitisation or ownership of HBMF; demand for integrated management of diseases; CDD & communities concerned about lack of financial and supervisory support from HWs for motivation; reasons for SES inequities unclear. Ghana; Sub-district; 10mo ( ) [45] CBAs honest, basic literacy, hardworking, patient, available, accessible. Community selection. 27% male. Ratio: 374 2w classroom with practice; training material developed from WHO HMM, IMCI & Novartis manuals.1d for HW to support referrals. No refresher training described. 2x in first 2w by research team. Monthly meetings to collect records, problem solving. Not described User charges: Free CHW Incentives: No financial motivation. Bicycle Before/after no control 20% (n=?) sought any care <24h preintervention, compared to 89.1% (299/334) post-intervention. HH survey: 95.3% (307/322) febrile children 6-59mo received correct dose of AL from CBA; CBA register: 98.3% (354/360). 29.4% (5/17) children with severe signs referred to HF; 60% (3/5) of these completed. Facilitators: CBA confidence increased with observed patient recovery; CBA commitment & interest appreciated by community; longer training; close supervision by research team; initiative in patient follow up (though at expense of referral) Barriers: Financial motivation required for sustainability (e.g. US$20-50/month), community prefer external provision or small consultation charge (US$ ). CHWs would rather have money than payment in kind. 24

34 Country; Scale; Int length [Ref] Nigeria; District; 18mo ( ) [37, 43, 44] CHW characteristics & recruitment; % male; CHW ratio: popn Permanent residents, trusted & respected, basic literacy, willingness to serve (drug sellers, PHC workers, mothers). Selected by community leaders after public meetings. 7% male. Ratio: 717 Length & style of training 2d with pictorial treatment guidelines. Plus 1d on prep. of blood slides & filter paper for drug effectiveness. No refresher training. Supervision Supply User charges & CHW motivation Not described Drugs free from MOH; new stocks collected by CMDs when needed. Sometimes delivered by supervisors. User charges: US$ per dose & US$0.24 commission. Free after 6mo. CHW Incentives: US$0.24 per pack sold. T-shirts, travel, certificate. Evaluation design & Key outcomes Post-only no control 98.0% children received correct dose (1019/1040). HH survey: 96.2% treated same or next day (278/289); 79.9% treated promptly AND correctly (231/289) 90.9% PCR adjusted cure rate at day 28 (140/154) Facilitators & Barriers Facilitators: Overall acceptance of AL & intervention by caretakers, CMDs & health staff. AL effective and few adverse events. No need for transport costs, lower drug costs. Easy access. Pre-packed drugs facilitated adherence to dose. Barriers: Sustainability seen as challenge (free drugs, CMD motivation, training). Negative experiences with health staff, led to bypassing of HFs (preferred community leader supervision, drugs directly from LGA stores). More incentives requested by CMDs. Ghana; District; 12mo (no dates) [37, 43, 44] Permanent residents, trusted & respected, basic literacy, willingness to serve (farmers, teachers, drug sellers) Selected by communities. 94% male. Ratio: 704 5d (no further details) Plus 1d on prep. of blood slides & filter paper for drug effectiveness. No refresher training described. Every 2mo by research team; monthly by district HWs. Transport funded by project. Drugs free from MOH; Collected by CMDs when needed. Sometimes supervisors delivered. User charges: US$ per dose. CHW Incentives: US$3.50 per month. Bicycles, boots. Post-only no control 99.0% children received correct dose (4473/4518). HH survey: 88.6% treated same or next day (259/289); 87.2% treated promptly AND correctly (252/289) 91.4% PCR adjusted cure rate at day 28 (160/197) Facilitators: Accessibility & availability of CMDs - Majority of caretakers found CMD first time Barriers: Financial incentives repeatedly requested by CMDs Ghana; District; 12mo (no dates) [43, 44] Permanent residents, trusted & respected, basic literacy, willingness to serve (farmers, teachers, drug sellers) Selected by communities. 63% male. Ratio: 553 2d (no further details). No refresher training described. Monthly supervision by local HWs & quarterly meetings Drugs free from MOH; Collected by CMDs when needed. Sometimes supervisors delivered. User charges: US$ per dose. CHW Incentives: US$8 quarterly. T-shirts, watches, raincoats, torches Post-only no control 98.0% children received correct dose (3900/3979). HH survey: 95.7% treated same or next day (89/93); 74.2% treated promptly AND correctly (69/93) Facilitators: Accessibility & availability of CMDs - Majority of caretakers found CMD first time Barriers: Financial incentives repeatedly requested by CMDs Uganda; District; 12mo (no dates) [37, 43, 44] Pre-existing cadre of CMDs, previously selected by communities 53% male. Ratio: d (no further details) Plus 1d on prep. of blood slides & filter paper for drug effectiveness. No refresher training. Monthly supervision by local HWs Drugs free from MOH; new stocks collected by CMDs when needed. Some supervisors delivered. User charges: None CHW Incentives: US$1.16 per meeting. T-shirts, baseball caps, certificates Post-only no control 97.0% children received correct dose (10671/11000). HH survey: 85.8% treated same or next day (531/619); 70.8% treated promptly AND correctly (438/619) 97.2% PCR adjusted cure rate at day 28 (210/216) Facilitators: Accessibility & availability of CMDs - Majority of caretakers found CMD first time. Well established system of HBMF in this setting. Barriers: Financial incentives repeatedly requested by CMDs 25

35 Country; Scale; Int length [Ref] Burkina Faso; Sub-district; 12mo ( ) [38] CHW characteristics & recruitment; % male; CHW ratio: popn Resident, available, acceptable, read & write French, willingness to volunteer (traders, labourers, students, 50% unemployed) Community selection 17% male. Ratio: 290 Length & style of training 2 days (no further details) No refresher training described. Supervision Supply User charges & CHW motivation Regular supervision by local HWs ACTs supplied by WHO via MOH; stocks available for sale and return to CMDs when needed User charges: Cost per dose US$ CHW Incentives: 10% incentive per dose sold Evaluation design & Key outcomes Post only no control* HH survey: 25.9% (73/282) children that attended CMD treated with correct ACT dose; CMD register: 94.8% (1818/1918). HH survey: 18.4% (52/282) febrile children that attended CMD treated with correct dose ACT <24h. CMD register: not available. Facilitators & Barriers Facilitators: Pre-packaged drugs effective & affordable. CMDs convenient, nearby, provided quick Tx, occasional home visits. Intensive supervision (study team) encouraged good CMD performance. Barriers: Rural-style IEC ineffective & wider range of established alternative Tx sources. Low community confidence in CMD training & skills. CMD unavailable. Inability to pay. Ghana; Sub-district; 12mo ( ) [38] Uganda; Sub-district; 5mo (2004) [49] Rwanda; District; 12mo ( ) [48] Resident, basic literacy, trusted & willingness to volunteer (traders, teachers, artisans, local leaders, 25% unemployed). Community selection 51% male. Ratio: 16,200 CDDs selected by community (no further details) CHWs selected by community during public meeting; one male, one female per village; integrity, literacy, cleanliness 5d classroom with practice. Monthly updates in first 3mo, followed by 2-monthly updates 3d. No refresher training described. Length of training not detailed. No refresher training described. 2-weekly supervision by study team Supervised by local HWs (frequency not specified) Supervised by local HWs less than monthly. Monthly meetings, deliver reports. HW & pharmacy training on stock mgmt AS-AQ supplied by MoH every 2 weeks; medicines not used in 2 weeks were withdrawn Not described CHW travel to nearest HF to replenish stocks (at no charge) User charges: Cost per dose US$ CHW Incentives: Not detailed User charges: Free CHW Incentives: Not described User charges: US$0.20 per dose CHW Incentives: US$0.20 per dose 26 Post only no control* HH survey: 93.1% (167/179) febrile children that attended CMD treated with correct ACT dose; CMD register: 93.2% (890/955). HH survey: 68.7% (123/179) children that attended CMD treated with correct ACT dose <24h; CMD register: 60.4% (577/955). Post-only no control Case series follow-up of referrals of 40 CDDs. Overall referral rate of 8% (117/1454); 16 lost to follow-up, 24 excluded due to non-febrile illness. 87% (67/77) completed referral at hospital or health centre. Post-only no control 92% (370/402) febrile children given correct AL dose. 3mo review of 72 CHWs: 92 children referred (N=?); 70% (64/92) counterreferrals completed Facilitators: Pre-packaged drugs effective & affordable. CMDs convenient, nearby, provided quick Tx, occasional home visits. Intensive supervision (study team) encouraged good CMD performance. Barriers: Rural-style IEC ineffective & wider range of established alternative Tx sources. Low community confidence in CMD training & skills. CMD unavailable. Inability to pay. Facilitators: High completion of referral compared to HF-to-hospital level, may be due to low costs for primary care seeking. Promptness greater for U1s or "urgent" referrals Barriers: Main barriers to successful referral include lack of money & mother not being told to go immediately, child improvement. Lack of regular follow-up by CDD could also contribute. Recommend use of referral & counter-referral forms. Facilitators: Good availability of AL in HFs and with CHWs. CHWs & AL accessible & acceptable to caretakers, recognise capacity & limitations. Affordable & known costs of AL (CHW & HF). CHW have pride in role & serving community. Barriers: Infrequent supervision due to other priorities, lack of time & fuel. Demand for capacity to treat more diseases. Improvements needed in counselling & checking for danger signs. Some requests for bonuses & equipment.

36 Country; Scale; Int length [Ref] Burkina Faso; Regional; 14mo ( ) [54] CHW characteristics & recruitment; % male; CHW ratio: popn CHWs (no further details) Length & style of training Length & style of CHW training not described; pharmacies & HFs also trained to support intervention Supervision Supply User charges & CHW motivation Supervision by HWs (frequency not specified); project team supervised nurses and occasionally CHWs directly Initial free stock of pre-packaged CQ & aspirin; used to initiate a revolving fund User charges: Cost per dose US$ CHW Incentives: 10% margin per dose sold Evaluation design & Key outcomes Post-only no control 5.1% (93/1806) children promptly receiving pre-packaged AM progressed to severe malaria, 11.0% for those (153/1396) that didn't; OR: 0.47 (95%CI: 0.35,0.64; p<0.001), age, sex, mother's literacy, village adjusted. 56.4% (1806/3202) children with fever treated promptly with pre-packaged AM from CHW; Ranged 30%-84% by village), correlated with CHW:popn ratio Facilitators & Barriers Facilitators: Higher ratio CHW: population increased proportion children promptly accessing pre-packaged AM from CHW. Low cost of the pre-packaged AMs compared to commonly used drug shops Barriers: Still some problems with incorrect dose suggest further improvements in prepackaging needed e.g. blister-packs * Baseline household surveys conducted, but no data collected on CHW performance or prompt treatment seeking Abbreviations: ACT=artemisinin combination therapy ; AL=artemether-lumefantrine; AQ=amodiaquine; ARI=acute respiratory infection; AS=artesunate; CDD=community drug distributor; CDI=community-directed intervention; CHV=community health volunteer; CHW=community health worker; CI=community implementer; CMD=community medicine distributor; CORP=community-owned resource person; CQ=chloroquine; HBMF=home-based management of fever; HH=household; HMM=home management of malaria; HF=health facility; HMIS=health management information system; HSA=health surveillance agent; HW=health worker; IMCI=integrated management of childhood illness; int=intervention; ITN=insecticide-treated net; MCA=malaria control agent; ORS=oral rehydration salts; PW=pregnant women; RCT=randomised controlled trial; RDT=rapid diagnostic test; SP=sulfadoxine-pyrimethamine; Tx=treatment; U5=child under five; VHC=village health committee. 27

37 5.3 RDT-confirmed case management of malaria Eleven studies investigated the use of RDTs by CHWs for the case management of malaria (Table 4). These were conducted in seven countries: three in Tanzania; two each in Ethiopia and Zambia; and one each in DRC, Ghana, Sierra Leone and Sudan. The majority (10/11) of studies were relatively small scale sub-district level pilot investigations of the potential for use of RDTs in the community; one study in Ethiopia evaluated implementation at the district scale [41]. One study was an RCT, one was a CBA design, and the remaining nine were post only without control. Two studies reported impact of the intervention in terms of a clinical or health outcome; eight reported at least one indicator of the quality of CHW performance; and three reported on referral from the community level to a formal health facility. Impact on clinical outcomes Two of the eleven studies involving confirmed diagnosis of malaria by CHWs measured impact on a clinical outcome. The study by Lemma et al. in Ethiopia used a before and after design with a comparison group and demonstrated reduced risk of malaria-specific mortality in the intervention versus comparison clusters (adjusted IRR: 0.60; 95%CI: 0.40, 0.90; p=0.01), although there was no impact on all-cause mortality rate (adjusted IRR: 1.03; 95%CI: 0.87, 1.21; p=0.75) [41]. Only 50% of CHWs in the intervention clusters received training on RDTs; however no CHW performance data was collected to determine whether RDTs improved diagnosis and treatment of malaria patients. Nevertheless, as the authors report, similar high levels of vector control in intervention and control districts and the higher altitude of the control district add plausibility that the impact on malaria-specific mortality may have been due to the treatment of malaria with ACT in the community by CHWs [41]. The second paper with a clinical outcome was a follow-up to the study by Mubi et al. in Tanzania [61], measuring PCR-adjusted cure rates for patients with an RDT-confirmed malaria diagnosis that received ACT from a CHW; high day 28 and day 42 cure rates of 95.1% (229/241) and 93.0% (224/241), respectively support the effectiveness of ACT for confirmed malaria at the community level in Tanzania [62]. Interestingly, 75.4% (300/398) of febrile children considered for the study were RDT positive, however, only 83.3% (250/300) of these also had a positive blood slide upon further investigation and so were enrolled in the study [62]. It is not clear if this discrepancy was due to how the CHWs used the RDTs or read the results, or to imperfect sensitivity of the RDTs themselves. Impact on CHW quality Six of the eleven confirmed diagnosis studies reported on the proportion of febrile, RDT-positive patients seen by CHWs; this figure varied considerably from 23.3% (2298/9847) in Zambia [57] to 93.0% (332/357) in DRC [60], with a median of 58.5% (IQR: 47.1%, 73.2%). Two of the studies measured CHW quality according to the number of steps correctly conducted in preparing and conducting an RDT and to the proportion that read the result correctly, since use of RDTs at the community level was not policy at that time in Zambia [59] and DRC [60]; all febrile patients were therefore treated with ACT regardless of the RDT result. The Zambian study investigated the influence of different types of training on correct RDT use for existing CHWs involved in malaria treatment. A mean of 57% (N=32) of 16 steps for RDT use were completed correctly in the arm that used the manufacturer's guidelines only, compared to 80% (N=21) in the arm that received a more detailed job aid (p<0.05), and 90% (N=26) in the arm that received both the job aid and practical training (p<0.05); the mean proportion of RDT results read correctly was 54% (N=32), 82% (N=21, p<0.05) and 93% (N=26, p<0.05), 28

38 respectively [59]. In the DRC study, a median of 100% of 16 steps for RDT use were completed correctly (N=12) and 100% of CHWs read RDT results correctly after one day of practical training on RDT use for existing CHWs involved in malaria treatment [60]. Although interactive training with opportunity for practice and resolution of common problems led to good performance of RDTs in both settings immediately after training, extremely high RDT positivity in DRC (93%) led the authors to conclude that use of RDTs for malaria treatment would not be cost effective; basic decision modelling found that at 80% parasite prevalence, the cost per unnecessary course of ACT averted would be $8.79 which is approximately eight times the cost of a course of ACT, or 60% of the annual per capita health budget [60]. Appropriate treatment according to RDT result in terms of proportion of RDT-positive and RDT-negative patients treated with ACT was reported by six and four studies, respectively. Amongst these, appropriate treatment by the CHWs was high with a median of 98.5% RDT-positive patients receiving ACT (IQR: 91.0%, 99.3%) and a median of 8.4% of RDT-negative patients receiving ACT (IQR: 4.6%, 15.5%). One study of 20 CHWs in Sudan was an exception: according to their records, 30% (6/20) of CHWs did not treat according to the RDT result, and interviews revealed that they preferred to rely on clinical judgement to treat RDT-negative patients with ACT; interestingly, this was not related to prior experience with malaria treatment or education level [58]. The CHWs recruited in seven of the eleven RDT studies were already treating malaria presumptively in their communities, so their training on use of RDTs built upon prior malaria knowledge; this was also the case for approximately half of the CHWs involved in the Sudan study [58]. CHWs in two of the studies were recruited for the RDT intervention with no indication of prior malaria experience [38]. Finally, the longitudinal study by Ishengoma et al. took place over a four-year period and spanned the change in policy from presumptive treatment of all fevers with SP to the confirmation of malaria in over-fives with an RDT and treatment of all ages with ACT [40]. This study found that before RDTs were introduced in January 2007, 98.6% (5478/5556) of febrile patients seen by CHWs were treated with SP. After the introduction of RDTs, 96.6% (4122/4267) of febrile under-fives were treated with AL; 88.8% (3942/4440) of over-5s treated with AL were RDT-positive, and 10.7% (475/4440) were RDT-negative [40], which suggests that the addition of RDTs did not affect CHW adherence to treatment policy. All other studies only reported on CHW performance after RDT training, thereby preventing comparison with their performance when diagnosis was presumptive. However, since appropriate treatment with an antimalarial according to RDT result was high, there does not appear to be cause for concern in this respect. Impact on referrals Three of the eleven studies reported on the proportion of cases referred to a formal health facility [58, 61, 63], and three reported on completion of referrals [57, 63]. The study of 20 CHWs in Sudan reported that 0.9% (35/3889) of patients were referred over a 6-month period for non-malaria febrile illness [58]. CHWs in this study could only treat patients with malaria and should have referred all other non-malaria fevers to a health facility. RDT positivity was not reported but a pre-rainy season household survey found a parasite prevalence of just 0.1%, so it seems unlikely that all patients who should have received a referral did so. An RCT in Tanzania which compared RDT-confirmed diagnosis with clinical diagnosis found a 17.2% (250/1457) referral rate in the RDT group compared to 5.0% (73/1473) in the clinical diagnosis group over a 5-month period; the main reasons for referral (80%) were non-malaria fever, diarrhoea, vomiting, cough or difficult breathing [61]. CHWs were trained to recognise danger signs and indications for referral, suggesting that a clinical judgement was needed rather than simple referral of all non-malaria fevers. Although referral was higher in the RDT group, 49.7% (724/1457) of these 29

39 patients were RDT negative; it is not clear what proportion of these were eligible for referral to assess the quality of the referral process. A study by Thomson et al. evaluated an intervention designed to improve the referral system for community-based treatment of malaria in Sierra Leone; the main focus of the intervention was the introduction of referral registers at health facilities and training of health staff in the completion of these registers for patients that had been referred by a CHW. CHWs received training to refer all cases with severe symptoms and those febrile patients with a negative RDT. The study found an overall referral rate of 17.2% (2093/12312) for children under five and of 8.5% (419/4929) for pregnant women over a 3-month period. Of these, 99.3% (2442/2459) were due to a negative RDT result with 0.9% (21) being completed according to the health facility registers; 0.7% (17/2459) of referrals were RDT-positive with severe signs with an 88.2% (15) completion rate [63]. The authors conclude that there is a need to improve the completion rate of referrals for RDT negative patients, for example by increased training or supervision. Alternatively, the criteria for referral of RDT negative patients could be narrowed to those with danger signs, or the CHWs could be provided with the capacity to treat common alternative childhood illnesses such as pneumonia and diarrhoea. The study by Chanda et al. in Zambia found a referral rate of 8.7% (815/9847), despite training on the need for referral of non-malaria fever to a health facility and 76.7% (7549/9847) of febrile patients being RDT negative. All 20 cases with signs of severe malaria were referred; however reasons for referral or lack of referral of the remaining RDT-negative patients were not detailed. The completion rate of those referred was 40.2% (328/815), as determined by written reports from the health facility [57]. Although health workers in the facilities supporting the CHWs were trained on supervision and feedback mechanisms for referrals, the CHWs reported that written feedback is rarely received, leading the authors to suggest that two-way communication needs to be improved. Impact on health system capacity to support CCM Only two of the eleven studies reported on completion of CHW referrals at a health facility. Feedback of referral completion to CHW was only 40.2% in Zambia, and communication with health workers on referrals was raised as an issue by CHWs during focus group discussions; the CHW complained that lack of feedback increased their workload as they had to follow-up the patients at home to check on their condition and on whether they had gone to a health facility as recommended[57]. Referral completion in the Sierra Leone study seemed to be strongly influenced by severity of illness with 88.2% of referred severe malaria cases seeking care at a health facility compared to just 0.9% of referred RDT-negative patients [63]. It is not clear where the reason for this low level of completion lies; it may be that health workers were less likely to complete referral records for these patients than severe malaria cases, but it is more likely that patients with a negative RDT result did not seek further treatment. The lack of information on the level of health system support for referrals from 80% (9/11) of studies investigating confirmed diagnosis of malaria is worrisome. In terms of supervision of CHWs implementing RDTs, only three studies described supervisory visits being conducted by district health workers [38, 57, 58], one of which was conducted jointly with the study team [38]. Supervision in the Sierra Leone study was conducted by NGO staff responsible for health care in the districts [63]; three further studies described supervision being conducted by the study team [38, 40, 61]. Four did not provide details of supervision activities, although two of these were cross-sectional evaluations of CHW ability to use RDTs immediately after training rather than evaluation of performance under operational conditions [59, 60]. Frequency of supervision was not always described; however in general those CHWs supervised by the study team were visited more 30

40 often than those visited by district health workers. For the four studies where the study team was involved in supervision, regular and frequent supervisory visits by experienced personnel was reported as a facilitating factor for good CHW performance [38, 40, 57]. Two studies reported challenges with supervision; two members of staff were allocated to visit CHWs in the Sierra Leone study, but it was reported that the number was insufficient to adequately cover all 130 villages [63]; three of the CHWs in the Zambia study reported that they were supervised at the health facility, which they disliked due to transport costs [57]. Where information was provided (6/9 studies), supplies of RDTs and ACT tended to be supplied during supervisory visits; for those studies where supervision was provided by health workers, supplies were also delivered through the district system [38, 57, 58]. Although only two studies explicitly stated that supplies were replenished for free, no study reported that CHWs had to pay for their RDTs or ACT stocks. Only one study gave any information on stock levels during the period of the evaluation: in Zambia, RDT stocks were reported as being consistent with just one stock out of two weeks in one of the two intervention districts over the 18 months of the intervention [57]. Two of the eleven studies described training for district health workers to help them support the CHWs, both of which involved orientation on the new referral system [57, 63]; lack of a comparison in either study means it is not possible to say whether this training was responsible for any improvements in health system capacity. No study described provision of training on supervision or supply management. Sustainability The median length of implementation of the interventions under study was 8 months (IQR: 6-21 months). The two evaluations of CHWs use of RDTs conducted immediately after training showed high performance [59, 60], although translation of this result to operational conditions and over time is not clear. Excluding these studies, the median length of intervention under evaluation was 12 months (IQR: 6-24 months) suggesting that CHW ability to correctly treat malaria according to RDT result remained high a considerable time after training. However, it is important to note that only a minority of studies were evaluating interventions conducted under operational conditions, a fact which raises concerns for sustainability. For example, the longest running study by Ishengoma et al. in Tanzania relied on weekly supervision by the study team which the authors acknowledged would not be feasible if implemented by the district [40]. Three of the eleven studies provided financial incentives to CHWs, ranging from US$0.5 per consultation in Sudan [58] to a monthly allowance of US$15 [61] or US$25 [62] in two of the Tanzania studies. One study in Zambia reported that the CHWs each received a bicycle [57]; financial or non-financial incentives were not described for the remaining seven studies. Only one of the studies reported CHW opinion on their incentives: 13 of the 20 CHWs in the Sudan study reported that the US$0.5 consultation fee was insufficient compensation for the time they spent away from their fields [58]. It is therefore difficult to comment on the acceptability or sustainability of CHW motivation in the included studies. Users in Ghana were charged US$0.15 per course of ACT [38] and those in Sudan were charged US$0.5 per consultation [58]; no details were given of any charges for RDTs, or whether the CHW had to pay for their supplies. In both cases, users commented that the CHW services were affordable. Five studies reported that users did not incur any charges for services provided by a CHW, and four did not describe whether user fees were charged or not. 31

41 Country; Scale; Int length [Ref] Tanzania; Sub-district; 5mo (2006) [61] Ethiopia; District; 2 years (no dates) [41] Ethiopia; Sub-district; 2 years ( ) [38] Ghana; Sub-district; 12mo ( ) [38] Table 4: Community-based management of malaria based on RDT-confirmed diagnosis: Intervention details and evaluation outcomes CHW characteristics & recruitment; % male; CHW ratio: popn Existing CHWs selected by community, although health promotion role only (at least primary education; age 20-50y; majority farmers). 55% male. Ratio: 400 Existing CHWs (no further details). Acceptable to community, willing to volunteer, availability, member of local council for at least one year (traders, labourers, students, 70% unemployed). Community selection. 35% male. Ratio: 1730 Existing CDDs. Basic education, responsible & reliable (petty traders, drug shop owners, urban farmers). Community selection. 36% male. Ratio: 2600 Length & style of training 7d on RDT use, Tx with AL, IMCI algorithm to identify danger signs & referrals. CHWs used RDTs or clinical diagnosis on alternate weeks. No refresher training described. 3 days refresher training. CHWs using RDTs in Y2 received additional specific training on RDT use. 3 days No refresher training described. 2 weeks, including 2d RDT practice Supervision Supply User charges & CHW motivation Field supervision by study team (frequency not specified) Supplies during field supervisors User charges: Free CHW Incentives: US$15 per month during study Not described Not described User charges: Not described CHW Incentives: Not described Monthly monitoring & supervision meetings with study team & district staff Weekly supervision by study team, collect reports, replenish supplies; monthly meetings of CDDs ACTs supplied by MOH; frequency & mechanism for replenishing RDT & drug stocks not detailed AS-AQ supplied by UNICEF; supplies replenished weekly during supervision visits User charges: Free CHW Incentives: Not described User charges: Cost per dose US$0.15; costs of RDTs not specified CHW Incentives: Not described Evaluation design & Key outcomes RCT 50.3% (733/1457) patients RDT +ve; 99.3% (731/733) RDT +ve treated with AL; 6.1% (44/724) RDT -ve given AL. Overall 96.8% (1411/1457) compliance with RDT. 96.5% (1422/1473) febrile patients in clinical diagnosis weeks given ACT. 17.2% (250/1457) referral rate in RDT group compared to 5.0% (73/1473) in CD group Before/after with control Poisson multivariate modelling adjusting for main confounders & clustering at village level, significant difference in malaria-specific mortality between intervention & control clusters (IRR: 0.60; 95%CI: 0.40,0.90; p=0.01), no significant difference in all-cause mortality (IRR: 1.03; 95%CI: 0.87,1.21; p=0.75) Post only no control* CHV register: 99.6% (1206/1211) children with confirmed malaria treated with correct ACT dose. HH survey: 100% (32/32) CHV register: 64.6% (784/1211) children with confirmed malaria treated with correct ACT dose <24h. HH survey: 56.3% (18/32). 46.0% (1211/2634) febrile children RDT+ve, assuming correct interpretation of RDT. Post only no control* CDD register: 97.7% (888/909) children with confirmed malaria treated with correct ACT dose. HH survey: 50.0% (53/106) CDD register: 30.6% (278/909) children with confirmed malaria treated with correct ACT dose <24h. HH survey: 34.9% (37/106) 66.7% (909/1363) febrile children RDT+ve, assuming correct interpretation of RDT Facilitators & Barriers Facilitators: High CHW compliance with guidelines compared to formal HWs; encourage compliance amongst patients, along with prepackaged doses and monitoring completion Facilitators: Gametocidal effect of AL may have caused reduction in malaria transmission in intervention district (3-fold lower burden during epidemic); similar high levels of vector control in both districts. Barriers: No process data on CHW Tx practice to determine if higher % malaria cases treated appropriately; malaria deaths identified using VA model; sampling for mortality history unclear Facilitators: Pre-packaged drugs effective & affordable. CMDs convenient, nearby, provided quick Tx, occasional home visits. Intensive supervision (study team) encouraged good CMD performance. Barriers: Rural-style IEC ineffective & wider range of established alternative Tx sources. Low community confidence in CMD training & skills. CMD unavailable. Inability to pay. Facilitators: Pre-packaged drugs effective & affordable. CMDs convenient, nearby, provided quick Tx, occasional home visits. Intensive supervision (study team) encouraged good CMD performance. Barriers: Rural-style IEC ineffective & wider range of established alternative Tx sources. Low community confidence in CMD training & skills. CMD unavailable. Inability to pay. 32

42 Country; Scale; Int length [Ref] Zambia; Sub-district; 18mo ( ) [56, 57] CHW characteristics & recruitment; % male; CHW ratio: popn Existing CHWs, nominated by community to receive training at HF. All had secondary level education, >35y. 87% male. Ratio: 2147 Length & style of training Training on RDT use, including practice & (length of training not detailed). CHW & HWs trained on new referral system Supervision Supply User charges & CHW motivation 1-3 monthly supervision by HWs. RDTs & drugs issued by HWs to the CHWs every 2 weeks upon request. Only one stock out (2weeks, national) reported. User charges: Free CHW Incentives: Bicycle Evaluation design & Key outcomes Post-only no control 23.3% (2298/9847) patients with RDT +ve uncomplicated malaria. 99.3% (2282/2298) RDT+ve treated with AL; 0.2% (13/7549) RDT -ve given AM. 40.2% (328/815) of CHW referrals completed. Cost per case (2009 US$) appropriately diagnosed & treated: 4.22 for HMM, 6.61 for health facility; ICER 4.18 [( )/( )]. Provider perspective only, no opportunity costs for CHW time included. Facilitators & Barriers Facilitators: Continuous availability of RDT; regular supervision; useful job aids. Increased accessibility & availability of CHWs versus HFs Barriers: Incomplete written referral feedback by HWs for CHWs; need improved communication. 3 CHWs supervised at HF, dislike due to cost of transport. Some patient demands for AM with RDT-ve; demand to treat non-malaria illness (ICCM) Sudan; Sub-district; 8mo ( ) [58] Zambia; Sub-district; 0mo** (2006) [59] Available, acceptable, literate, willingness to volunteer (mean age 38y, varied education level, 45% malaria Tx experience, farmers, merchants) 100% male. Ratio: 1750 Existing CHWs (mean age 42y; mean 5.6y as a CHW; 19% completed secondary education; 90% malaria Tx experience, 7.6% RDT experience) 75% male 7d classroom & practice on RDTs, referral of danger signs, severe malaria after admin of rectal AS In the arm receiving job-aid plus training, CHWs received 3 hours training with demo of RDT use & lecture on safe blood handling, followed by practice on each other. Monthly supervision in first 3 months, reduced over time. N/A (evaluation immediately followed training) Supplies & equipment provided free from nearest HF. N/A (evaluation immediately followed training) User charges: Post-only no control US$0.5 14/20 (70%) MCAs treated appropriately according consultation fee to RDT result, remaining 6 gave AS-SP to RDT-ve, CHW using 'clinical judgement'. Incentives: 0.9% (35/3889) cases referred to hospital with US$0.5 per non-malaria febrile illness. consultation User charges: Not described CHW Incentives: Not described Post-only with control Mean of 57% (N=32) 16 steps for RDT use completed correctly in manufacturer's guidelines arm, 80% (N=21) in job aid-only arm (p<0.05), 90% (N=26) job aid & training arm (p<0.05). Mean % RDT results read correctly: 54% (N=32), 82% (N=21, p<0.05) & 93% (N=26, p<0.05), respectively. Cost US$175 per CHW trained (including materials, per diems, trainer salaries, transport) Facilitators: Acceptance of MCAs & caring attitude; easy, close, affordable service; Barriers: Financial motivation insufficient for 13/20 MCAs due to time away from usual work; most felt VHC supportive. Request for expansion of services MCAs provide Facilitators: Interactive training style with demo & practice; sufficient ratio of trainers to CHWs to observe and counsel on performance. Practice improved performance in all 3 arms, although not as much as job aid or training. Barriers: Common errors found: obtaining correct quantity of blood for RDT, waiting 15mins for result, interpreting results (especially faint positives or invalid tests). Training group had fewer errors. DRC; Sub-district; 0mo ** (2009) [60] Existing CHWs (median age 32y; median of 4.5y malaria Tx experience, 16.7% RDT experience) 75% male. 1d lectures & practical demo of RDT use. Not described CHWs supplied with thermometers, RDTs & AS-AQ after training User charges: Not described CHW Incentives: Not described Post-only no control Median of 100% of 16 steps for RDT use completed correctly (N=12); 100% CHWs read RDT results correctly 93% (332/357) children in village RDT+ve, 1.1% RDT-ve, 5.0% indeterminate. At 80% prevalence, decision model found cost of $8.79 per unnecessary course of ACT averted; ~8x cost of ACT dose, or 60% annual per capita health budget Facilitators: High acceptance of RDTs by CHWs (utility & ease of use); experienced CHWs therefore quickly picked up new RDT skills. Barriers: At very high levels of parasite prevalence, RDTs not cost effective; associated costs likely to be major barrier to integration of RDT use for CHW 33

43 Country; Scale; Int length [Ref] Tanzania; Sub-district; 4.5 years ( ) [40] Tanzania; Sub-district; 6mo (2009) [62] Sierra Leone; Sub-district; 6mo (2009) [63] CHW characteristics & recruitment; % male; CHW ratio: popn Length & style of training CORPs (no further details). Initial training on clinical diagnosis & Tx with SP. Refresher training after policy change, on RDT use for over-5s & AL. Existing CHWs selected based on gender balance, residential area, ability to keep records. 100% male. Existing CMVs able to read & write, speak English, respected & liked. Community selection Ratio: days No refresher training described 3 days, including practical sessions on RDT use & training on new referral procedures Supervision Supply User charges & CHW motivation Weekly supervision by study team Not described Random visits and monthly meetings where RDTs & drugs distributed and reporting forms collected RDTs provided weekly by supervisors to avoid storage at high temps; AM supply unclear Free prepackaged AL, RDTs, job aids paracetamol RDTs & drugs provided for free monthly by supervisors User charges: Not described CHW Incentives: Not described User charges: Free CHW Incentives: US$25 per month User charges: Free (MSF funds) CHW Incentives: Not described Evaluation design & Key outcomes Facilitators & Barriers Post-only no control Facilitators: Intensive support supervision Pre-RDTs, 98.6% (5478/5556) patients treated with (weekly) by experienced team; level of training. SP. Post-RDTs, 96.6% (4122/4267) U5s treated Barriers: For implementation by district, such with AL; 88.8% (3942/4440) over-5s treated were intense supervision impractical; will need to RDT+ve, 10.7% (475/4440) RDT-ve design sufficient programme to maintain CORPs confidence in RDT results & guideline adherence. Post-only no control Not described 95.1% PCR adjusted cure rate at day 28 (229/241); 93.0% at day 42 (224/241) Post-only no control Overall referral rate of 17.2% (2093/12204) of U5s & 8.5% (419/4901) of PW over 3mo. 99.3% (2442/2459) referrals due to RDT-ve result; of these 0.9% (21) referrals completed. 0.7% (17/2459) referrals RDT+ve with severe signs; of these 88.2% (15) referrals completed Facilitators: CMV behaviour correct, need to improve referral completion, or give CMVs capacity to treat uncomplicated RDT-ve patients. Large cluster variation, perhaps due to distance, CMV performance, follow-up by HF personnel. Barriers: Supervision a challenge with 2 supervisors for 130 villages * Baseline household surveys conducted, but no data collected on CHW performance or prompt treatment seeking; ** Evaluation conducted immediately after intervention training Abbreviations: ACT=artemisinin combination therapy ; AL=artemether-lumefantrine; AQ=amodiaquine; ARI=acute respiratory infection; AS=artesunate; CDD=community drug distributor; CDI=community-directed intervention; CHV=community health volunteer; CHW=community health worker; CI=community implementer; CMD=community medicine distributor; CORP=community-owned resource person; CQ=chloroquine; HBMF=home-based management of fever; HH=household; HMM=home management of malaria; HF=health facility; HMIS=health management information system; HSA=health surveillance agent; HW=health worker; IMCI=integrated management of childhood illness; int=intervention; ITN=insecticide-treated net; MCA=malaria control agent; ORS=oral rehydration salts; PW=pregnant women; RCT=randomised controlled trial; RDT=rapid diagnostic test; SP=sulfadoxine-pyrimethamine; Tx=treatment; U5=child under five; VHC=village health committee 34

44 5.4 Community case management of malaria integrated with other health services Nine studies across eight countries investigated CHW treatment of malaria integrated with other child health services (Table 5): two studies in Uganda; and one each in Cameroon, Kenya, Mali, Malawi, Nigeria, Rwanda and Zambia. The majority of studies evaluated interventions conducted at the subdistrict or district scale (4 and 5 studies, respectively). Two studies employed a cluster RCT design, one before and after without a control, three were post only with a comparison group and the remaining three were post only without control. None of the studies reported impact of the intervention in terms of a clinical or health outcome; all nine reported at least one indicator of the quality of CHW performance; three reported on referral rates by the CHW and one of these reported the completion rate of referrals at a formal health facility. CHWs in four of the studies were trained to diagnose and treat non-severe pneumonia and reported pneumonia treatment indicators. Impact on CHW quality All nine studies reported some indicator of CHW quality, although these were diverse and reflected the varied responsibilities of CHWs in different programmes and the different approaches to the evaluation of integrated service provision. In terms of CCM of malaria, CHWs presumptively diagnosed malaria in seven of the studies: a study of CHW management of multiple childhood illnesses in Kenya evaluated CHW performance at three time points over a four-year intervention period; the proportion of patients clinically diagnosed with malaria that were correctly treated with SP increased from 41.0% (39/95) one year after initial training to 92.7% (115/124) twelve months later following refresher training; appropriate treatment remained high at 90.5% (237/262) in the third evaluation two years later [65, 66]. Three studies measured CHW performance in terms of the proportion of febrile patients treated with the correct anti-malarial at the correct dosage: a cluster RCT in Mali found that 58.9% (89/151) of drug kit managers sold CQ at the correct dose in the intervention zones (which received refresher training on CQ treatment and an improved referral system), compared to 48.1% (63/131) in the control zones (which relied on their original basic training) (p=0.04) [69]. Two other studies (both post only no control) reported the same indicator: 99.6% (798/801) and 84.8% (774/913) of febrile children that attended a CHW were treated with the correct ACT dose in Malawi and Rwanda, respectively [38, 68]. Prompt and appropriate treatment was received by 73.7% (590/801) of febrile children seen by a CHW in Malawi [38]. Evaluation of community-directed interventions (CDI) in Cameroon, Nigeria and Uganda that included HMM found 17.3%, 62.5% and 58.5% of febrile children respectively received prompt and appropriate treatment (from any source) in the intervention districts where CHWs had supplied anti-malarials for one year, compared to 10.5% (p=0.3), 30.3% (p=0.04) and 29.4% (p<0.002) in the respective comparison districts without HMM [39]. In those districts where HMM had been implemented for two years, 27.9% (p=0.01), 77.2% (p<0.002) and 76.7% (p=0.002) of febrile children in Cameroon, Nigeria and Uganda respectively received prompt and appropriate treatment (from any source) [39]. One of the main reasons for the lower figures in Cameroon is a change in policy to confirmed diagnosis partway through the intervention period that could not be incorporated into HMM as part of CDI [39]. In two of the more recent studies in Zambia and Uganda, malaria diagnosis was confirmed by RDT [67, 70]. The study in Zambia was a cluster RCT to investigate the potential for CHWs to diagnose and treat malaria and pneumonia, compared to presumptive diagnosis and treatment of malaria and referral of suspected pneumonia: 27.8% (271/975) of children with fever in the intervention arm were RDT positive and of these 97.8% received AL (only 0.4% of RDT-negative patients were treated with AL), compared to 35

45 99.1% (2066/2085) of febrile children in the control arm that received AL presumptively [70]. CHW performance was high in both arms of the study, but the results highlight the considerable overtreatment with ACT if diagnosis is presumptive in this setting of relatively low RDT positivity. By contrast, 78.0% (142/182) of febrile patients were RDT positive in the Ugandan study which investigated CHW diagnosis and treatment of malaria and pneumonia immediately after a training programme; 97.1% of these children were correctly treated with an ACT (4.7% of RDT-negative patients were treated with an ACT) [67]. Along with the treatment of malaria, CHWs in four of the studies were able to diagnose uncomplicated pneumonia and treat with antibiotics. In the Kenyan longitudinal study by Kelly et al., the clinical algorithm adapted from IMCI included steps to diagnose uncomplicated pneumonia; amongst children classified with pneumonia by the study clinician, 58.3% (28/48), 65.1% (43/66) and 50.0% (46/92) received treatment with cotrimoxazole at the 12, 24 and 48 month evaluations, respectively [65, 66]. This is in large part due to errors in performing the assessment tasks and in using the assessments to classify the child s diagnosis: CHWs used a normal watch to measure respiratory rate and were in agreement with the study clinician in classifying pneumonia only 33.3%, 54.5% and 31.5% of the time during the first, second and third evaluations respectively. Reasons proposed by the authors for the poor adherence to treatment guidelines were the complexity of the algorithm and the fact that it changed a number of times over the study period. A multivariate analysis of the third evaluation data found that none of the intervention-related factors studied (refresher training, supervision, adequate medicine supplies) were significant predictors of guideline adherence [66]. Conversely, in an evaluation of CHW performance conducted immediately following training on the use of RDT for malaria diagnosis and respiratory timers for pneumonia diagnosis in Uganda, 84.6% of respiratory timer readings were in agreement with the study clinician and 82.5% of clinician-confirmed pneumonia cases were treated with an antibiotic [67]. In the evaluation of community-imci in Rwanda, 79.0% (158/200) of children diagnosed with pneumonia by a CHW received the correct dose of antibiotic; however, only 72.8% (67/92) of CHWs observed correctly measured respiratory rate [68], so it may be that appropriate treatment would decrease if a clinician s diagnosis were used as the denominator (as reported for the Kenya and Uganda studies). In the Zambian cluster RCT previously described, the proportion of patients classified with non-severe pneumonia that received prompt and appropriate treatment was 68.2% (247/362) in the arm where CHWs could treat with antibiotics, compared to 13.3% (27/203) in the control arm where pneumonia cases had to be referred to the nearest health facility for treatment (RR: 5.32; 95%CI: 2.19, 8.94); no validation of CHW diagnosis of pneumonia was reported in this paper, although the CHWs were regularly supervised [70]. The five other studies did not provide CHWs with antibiotics: in addition to anti-malarials, drug kit managers in Mali also had oral rehydration solution (ORS) for the treatment of diarrhoea as well as basic first aid supplies and eye ointment [69]. Health surveillance agents in Malawi were also responsible for growth monitoring, nutrition, vaccinations, family planning and water and sanitation activities [38]; however, no indicators on these additional non-malaria services were reported for either intervention. The three CDI studies in Cameroon, Nigeria and Uganda involved community-level treatment of TB, Vitamin A supplementation, and ITN delivery in addition to presumptive treatment of malaria. Overall, the most impressive improvements between community and conventional delivery channels were seen for prompt and effective treatment of malaria and ITN delivery; coverage of TB-DOT and vitamin A supplementation were high across intervention and comparison districts [39]. 36

46 Impact on referrals Three of the nine studies reported the proportion of cases referred to a formal health facility [65, 68, 69] and one on the proportion of referrals completed [69]. A large component of the intervention in Mali was to improve CHW referral of patients that could not be treated and those with recognised danger signs, along with better record keeping in a designated referral register. Data from a household survey found that 42.1% (64/152) of children seen by a CHW were referred in the intervention zones with 64.5% (40/62) of caretakers reporting that they complied with the referral, compared to 11.2% (15/134) referral and 85.7% (12/134) compliance in control zones; however, from the CHW registers in the intervention zones, the overall referral rate was 16.1% (46/286), of which 87.0% (40/46) were completed [69]. The authors report that following discussions with drug kit managers, this discrepancy was likely to be due to drug kit managers verbally referring more patients, but only recording those where they believe there was a high chance of compliance [69]. In Kenya, the referral rate for severe classifications was 70.0% (63/90), one year after initial training, 42.1% (32/76) after two years and 52.7% (64/122) after four years; CHWs reported that they were concerned about criticism from caretakers and health staff if they referred a child to the health facility if they did not really need to go [65]. The referral rate reported in the evaluation of the Rwandan C-IMCI programme was 18.8% (249/1326); a low return of counter-referral forms from the health facility to the CHW was described, ranging from 20%-67% (no numbers or further details provided) [68]. Impact on health system capacity to support CCM The study by Winch et al. in Mali was the only one to report on the completion of referrals at a formal health facility and this depended on the source of data: 87.0% of referrals were completed according to CHW records and 64.5% according to caretaker interviews, believed to be due to under-reporting of referrals made by CHWs when they were not confident that the caretakers would comply [69]. However, there was good consistency between the CHW and health facility record books and management of referred children was reported to be generally appropriate [69], suggesting that capacity of the district health workers to support the CHWs in terms of referral was a strength of the programme. The situation regarding supervision of CHWs was mixed: two of the studies did not describe any supervisory activities [39], and for one it was not appropriate as the evaluation was conducted immediately following training [67]. The two cluster RCTs had regular follow-up of the CHWs by the study team: every other day in the Zambia trial [70] and fortnightly in the Mali trial [69]. Supervision was also intensive in Malawi at three times weekly, although it was conducted by district staff; in addition, monthly meetings were held with the community, CHWs and the district and study teams [38]. In the remaining three studies, supervision was limited by understaffing and fuel shortages [39, 65], and in the case of Rwanda did not take place at all three months post-training [68]. In Kenya, the supervisors performance in terms of case management was no better than the CHWs [65]. The link between supervisory support and CHW performance was not clear: two of the three studies of intensive supervision found strong CHW performance [38, 70]; the reasons for lower appropriate treatment of fever with CQ in Mali were unclear, as they were for the reasonably high treatment performance of CHWs in Rwanda and Uganda in the absence of regular supervision (although this may have been due to a long established CHW system in these countries). Seven of the studies reported that CHW supplies were provided through the district system, generally on a monthly basis with no details provided for the remaining two studies. Stock-outs were reported in four of these studies: all three of the CDI countries, which experienced frequent problems, particularly in the first year [39], and in Rwanda where the evaluation team found delays in supplies after the initial 37

47 training and stock-outs of basic medicines due to poor motivation of district pharmacies and inconsistent training standards among health facilities [68]. Sustainability The median length of implementation of the interventions under study was 12 months (IQR: 3-24 months), again providing indication of CHW performance after a reasonable length of implementation. Four studies did not describe any motivation system for the CHWs; the Zambian RCT provided nonfinancial incentives including a bicycle [70]; the three CDI studies stated that there were no formal financial incentives, although some CHWs received payment in kind from their communities and in some cases small financial gifts [39]; and, the full-time health surveillance agents in Malawi received a monthly allowance from the MOH, although the amount of this was not specified. Overall, however, CHWs from five of the nine integrated CCM studies reported that they had strong intrinsic motivation (derived from serving their communities and the respect they gained) or non-financial benefits (such as gifts or help with chores), and no requests were made for financial incentives. Services were provided free to users in five of the studies, three did not describe any user charges. Only the Mali study reported that drugs were sold to caretakers, but the costs were not described [69]; the authors of this study hypothesised that cost was still a potential barrier since the malaria season coincides with the time of lowest income, so it may be necessary to develop interventions to reduce the financial costs of careseeking further [69]. 38

48 Country; Scale; Int length [Ref] Mali; Sub-district; 3mo (2002) [69] Zambia; Sub-district; 12mo ( ) [70] Cameroon; District; 3 years ( ) [39] Table 5: Community-based management of malaria integrated with other health services: Intervention details and evaluation outcomes CHW characteristics & recruitment; % male; CHW ratio: popn Existing drug kit managers selected by community (no further details) Existing CHWs selected by community, based in fixed community health posts (mean age 40y, ~70% with secondary education). 89% male Selected for diligence, honesty, skill, experience. In some villages leaders appointed the CIs. Some villages divided tasks, others all done by one CI. Length & style of training Original training 35d literacy plus 5d drug kit management (classroom & practice). Additional 5d for Intervention villages on visual aids, dangers signs, new referral system, CQ doses, counselling Existing 6wk training. All study CHWs had 4d interactive & practical training on simplified algorithms for diagnosis & Tx of malaria &/or pneumonia; Half-day on RDTs for int CHW. 2d refresher course after 6mo. Group training at nearest HF on CDI philosophy, principles of an integrated approach, and specific knowledge of each intervention. Refresher training not described. Supervision Supply User charges & CHW motivation 2-weekly follow-up by study team. Skills assessment after 1mo, then 3-monthly; CHWs visited every other day by study team; monthly visits to HF with registers. Not described Not described AL & anti-pyretic supplied monthly by district team, Intervention CHWs also given RDTs & amoxicillin. From formal health system. Challenges with consistent supply. User charges: Drugs sold to caretakers, although prices not reported CHW Incentives: Not described User charges: Free CHW Incentives: Bicycle, umbrella, stationery User charges: Not described CHW Incentives: No compulsory payments; payment in kind (labour, food, prayers), or small financial gifts. 39 Evaluation design & Key outcomes Cluster RCT 58.9% (89/151) drug kit managers in intervention zones sold CQ at correct dose, 48.1% (63/131) in control zones (p=0.04). HH survey: referral rate 42.1% (64/152) in intervention zones, 64.5% (40/62) completion; 11.2% (15/134) & 85.7% (12/134) in control. From registers: overall referral rate 16.1% (46/286), completion 87.0% (40/46). Cluster RCT 27.5% (268/975) children with fever in int arm received AL (all of RDT+ve & 3 RDT-ve), 99.1% (2066/2085)in control arm Of children classified with non-severe pneumonia, 78.8% (285/362) sought Tx from CHW <48h in int group, 75.4% (153/203) in control group (RR: 1.06; 95%CI: 0.91,1.23). Of children classified with nonsevere pneumonia, 68.2% (247/362) received prompt & appropriate Tx in int group, 13.3% (27/203) in control group (RR: 5.32; 95%CI: 2.19,8.94). Post-only with control Y3 results: % febrile children prompt & effective Tx was 10.5% (2/19) in comp districts, 17.3% (13/75) where HMM in Y3 only (p=0.3), 27.9% (19/68) where HMM in Y2&3 (p=0.01). Mean annual cost (2005 US$) to implement 5 interventions of 34,755 per comparison district, 25,211 per CDI district. Salaries major of cost in all sites, as well as maintenance, training & social mobilisation Facilitators & Barriers Facilitators: Good completion rate of referrals, although discrepancy btw drug kit managers' records & HH survey may be due to only recording referrals when know high chance of completion (2 verbal for each written referral). Barriers: Compliance with correct dose lower for syrup than tablets; cost still a potential barrier as malaria season coincides with time of lowest income Facilitators: High adherence to Tx algorithms in both intervention & control arms; prompt & appropriate Tx outcomes improved with use of RDT & amoxicillin at community level; refresher training, frequent supervision & high education levels, simple guidelines; Most CHWs reported service to their community main motivation. Barriers: Caretakers that didn t seek care from a CHW was due to unavailability, severity of illness, proximity to HF; caretakers took children with suspected pneumonia to int CHWs as knew amoxicillin available Facilitators: Stakeholder consultation & mobilisation; little extra advocacy needed to gain support for CDI HMM. Maturation over time. Community selection of implementers. Strong nonfinancial/intrinsic motivation. Community and female empowerment. HW more engaged in outreach activities Barriers: Ensuring consistent commodity supplies (big problem in year 1); Resistance by HWs to use of CDI for TB- DOTS (too complex) and VitA (HWs lose per diems from vaccination days)

49 Country; Scale; Int length [Ref] Nigeria; District; 2 years ( ) [39] Uganda; District; 2 years ( ) [39, 64] Malawi; Sub-district; 12mo ( ) [38] CHW characteristics & recruitment; % male; CHW ratio: popn Selected for diligence, honesty, skill, experience. In some villages leaders appointed the CIs. Some villages divided tasks, others all done by one CI. Selected for diligence, honesty, skill, experience. In some villages leaders appointed the CIs. Some villages divided tasks, others all done by one CI. Existing full-time employees of MOH. Secondary level education; not resident in community. 25% male. Ratio: 3000 Kenya; Unpaid literate District; 4 volunteers, permanent years (1997- residents. ~50% CHWs 2001) [65, 66] reported that women had strong influence in Length & style of training Group training at nearest HF on CDI philosophy, principles of an integrated approach, and specific knowledge of each intervention. Refresher training not described. Group training at nearest HF on CDI philosophy, principles of an integrated approach, and specific knowledge of each intervention. Refresher training not described. 6wk, including 2wk on Tx with AL (classroom & practical) Initial 3wk training; 2x one-week refresher training session in Y2&3. Simplified version of IMCI algorithms to guide Supervision Supply User charges & CHW motivation Not described Limited supervision by HWs due to understaffing & fuel shortages 3x per wk by district officer; monthly meetings with community, district & study team Supervision conducted at HF& community, although From formal health system. Challenges with consistent supply. From formal health system. Challenges with consistent supply. ACT supplied by MOH; drug stocks replenished monthly by study team CHW medicine kits restocked at community pharmacies, based on cost User charges: Free CHW Incentives: No compulsory payments; some payment in kind (labour, food, prayers), or small financial gifts User charges: Free CHW Incentives: No compulsory payments; some payment in kind (labour, food, prayers), or small financial gifts User charges: Free CHW Incentives: Monthly salary from MOH (not specified) User charges: Not described CHW Incentives: Not described 40 Evaluation design & Key outcomes Post-only with control Y3 results: % febrile children with prompt & effective Tx 30.3% (59/195) in comp districts, 62.5% (218/349) where HMM in Y3 only (p<0.002), 77.2% (260/337) where HMM in Y2&3 (p<0.002). Mean annual cost (2005 US$) to implement 5 interventions of 28,806 per comparison district, 15,062 per CDI district. Salaries major of cost in all sites, as well as maintenance, training & social mobilisation. Post-only with control Y3 results: % febrile children prompt & effective Tx 29.4% (5/17) in comp districts, 58.5% (31/53) where HMM in Y3 only (p=0.04), 76.7% (23/30) where HMM in Y2&3 (p=0.002). Mean annual cost (2005 US$) to implement 5 interventions of 48,139 per comparison district, 29,331 per CDI district. Salaries major of cost in all sites, as well as maintenance, training & social mobilisation. Post-only no control HH survey: 99.6% (798/801) children that attended HSA treated with correct ACT dose. HSA register: 100% (2131/2131). HH survey: 73.7% (590/801) febrile children that attended HSA treated with correct ACT dose <24h. HSA register: not available. Post-only no control* Tx of uncomplicated malaria with SP: 41.0% (39/95), 92.7% (115/124) & 90.5% (237/262) at 1st, 2nd & 3rd evaluations. Tx of uncomplicated pneumonia with Facilitators & Barriers Facilitators: Stakeholder consultation & mobilisation; little extra advocacy needed to gain support for CDI HMM. Maturation over time. Community selection of implementers. Strong nonfinancial/intrinsic motivation. Community and female empowerment. HW more engaged in outreach activities Barriers: Ensuring consistent commodity supplies (big problem in year 1); Resistance by HWs to use of CDI for TB- DOTS (too complex) and VitA (HWs lose per diems from vaccination days) Facilitators: Strong community support as malaria important perceived problem, improved access compared to poor quality of HF. CIs reported strong intrinsic motivation and would continue as don't want to let down community. Integrated tasks considered easy Barriers: Health system constraints limit supervision (fuel, staff shortages); some shortages of ITNs which caused conflict; only task found to be challenging was TB- DOT due to stigma, may be more suitable to HW delivery Facilitators: Pre-packaged drugs effective & affordable. CMDs convenient, nearby, provided quick Tx, occasional home visits. Intensive supervision (study team) encouraged good CMD performance. Barriers: Rural-style IEC ineffective; wider range of established alternative Tx sources. Low community confidence in CMD skills. CMD unavailable. Inability to pay. Facilitators: Multivariate regression predictors of good performance: older children; no danger signs; use of job aid; CHWs thought they received benefits from role (e.g. Money, respect, ability to

50 Country; Scale; Int length [Ref] CHW characteristics & recruitment; % male; CHW ratio: popn their selection (VHC in other cases) 4% male. Ratio: 417 Uganda; Subdistrict; age 42y, at least Existing CHWs; median 0mo** (2009) primary education [67] (farmers, teachers) 43% male. Ratio: Rwanda; District; 3mo (2009) [68] Existing CHWs malaria tasks expanded to include diarrhoea & pneumonia (no further details) Length & style of training diagnosis & Tx for ARI, diarrhoea & malaria. Mix of lectures, role-play, clinical observation, job aids. 8d classroom and practice on use of RDTs, respiratory timers, danger signs Interactive training curriculum, including practice; length not specified Supervision Supply User charges & CHW motivation infrequent; monthly CHW meetings N/A (evaluation immediately followed training) None conducted by 3mo posttraining recovery system Supplies & job aids provided after training Not described User charges: Free CHW Incentives: Not described User charges: Not described CHW Incentives: Not described Evaluation design & Key outcomes cotrimoxazole: 58.3% (28/48), 65.1% (43/66) & 50.0% (46/92) at 1st, 2nd & 3rd evaluations. Referral rate for severe classifications: 70.0% (63/90), 42.1% (32/76) & 52.7% (64/122) at 1st, 2nd & 3rd evaluations. Post-only no control Mean 96.3% score for RDT preparation; 96.0% for use of respiratory timer (14 CHW). 95.5% (85/89) children with malaria prescribed AL, 40% (4/10) with pneumonia prescribed antibiotic; 90.6% (48/53) with both presc both. Post-only no control 84.8% (774/913) children with fever received correct drug & dose, 72.1% (155/215) with diarrhoea, & 79.0% (158/200) with pneumonia. Referral rate 18.8% (249/1326); low return of counter-referrals (20-67%, numbers not detailed) Facilitators & Barriers help community, gifts or help with chores); higher CHW education. Barriers: Long & complex guidelines, algorithm changed twice during study period with remaining inconsistencies, confusing to CHWs; supervision rare; performance of supervisors no better than CHWs; worry about unnecessary referral; lack of confidence Facilitators: Lots of practice with RDTs during training led to high levels of correct interpretation. Barriers: Some CHW had difficulty linking diagnostic results to classification, especially pneumonia; main reason for incorrect Tx. Evaluation in HF setting. Facilitators: Caretakers satisfied with CHWs as available, welcoming, information & Tx received. Integration not at expense of existing malaria Tx. Barriers: Delay in supplies after initial training. Stock-outs of basic medicines due to poor motivation of district pharmacies. Inconsistent training standards btw HFs. Lack of supervision. Recognition of danger signs & referral system need improvement. * Baseline household surveys conducted, but no data collected on CHW performance or prompt treatment seeking; ** Evaluation conducted immediately after intervention training. Abbreviations: ACT=artemisinin combination therapy ; AL=artemether-lumefantrine; AQ=amodiaquine; ARI=acute respiratory infection; AS=artesunate; CDD=community drug distributor; CDI=community-directed intervention; CHV=community health volunteer; CHW=community health worker; CI=community implementer; CMD=community medicine distributor; CORP=community-owned resource person; CQ=chloroquine; HBMF=home-based management of fever; HH=household; HMM=home management of malaria; HF=health facility; HMIS=health management information system; HSA=health surveillance agent; HW=health worker; IMCI=integrated management of childhood illness; int=intervention; ITN=insecticide-treated net; MCA=malaria control agent; ORS=oral rehydration salts; PW=pregnant women; RCT=randomised controlled trial; RDT=rapid diagnostic test; SP=sulfadoxine-pyrimethamine; Tx=treatment; U5=child under five; VHC=village health committee 41

51 5.5 Community-level demand generation for care seeking for malaria Amongst the 36 treatment studies, 16 described initial community sensitisation meetings to raise awareness of the intervention, fourteen described ongoing community mobilisation activities by the CHWs to promote prompt treatment seeking for fever (with some overlap in studies conducting both these activities), and seventeen studies did not describe any sensitisation or mobilisation activities. Only two of these, plus a further six studies that did not include treatment of malaria at the community level were assessed in more detail based on sufficient study design. Thus, a total of eight studies involved community level interventions to improve demand for prompt and effective treatment of fever (from any source) that were evaluated using at least a before and after study design with or without control (Table 6). The studies were conducted across six countries: two each in Nigeria, and Uganda; one each in Cameroon, Ghana, Mozambique and Tanzania. Two of the eight studies evaluated the home management of malaria programme in Uganda where antimalarials were provided at the community level for treatment of fever by CHWs [47, 52] (which has been previously discussed), one study in Cameroon was part of a wider intervention to improve quality of care at the health facility serving the community [75] and the ACCESS programme in Tanzania also involved interventions to improve the supply of anti-malarials through the informal private sector [42, 72]; the remaining four studies did not involve any provider side interventions and focused on user demand generation only. Four of the eight studies evaluated the intervention using a controlled before and after study design; three used a before and after without control, although provided sufficient contextual information to assess the likelihood that any changes in treatment seeking were due to the promotion intervention under study. One study evaluated the national HMM programme in Uganda [52], four evaluated interventions implemented at district scale and three at sub-district scale. Impact on prompt and effective treatment of malaria In four of the studies, promotion of prompt treatment seeking for fever/malaria was carried out by volunteering members of the target communities. In Nigeria, mother trainers selected by their communities provided education to approximately fifty neighbouring mothers on malaria, diagnosis and treatment and distributed a locally-developed pictorial treatment guideline. Prompt treatment seeking was 100% at baseline in the intervention and comparison groups and remained at 99% in both groups after the intervention. Impressive improvements were seen however in the correct use of CQ, which increased from 2.6% (3/116) to 52.3% (69/132) in the intervention group pre- and post-intervention (p<0.001), compared to 4.1% (3/72) and 15.8% (9/57) in control group (p=0.05). Use of the pictorial guideline was significantly associated with correct use of CQ (OR: 44.6; 95%CI: 7.6, 433.4; p<0.001) [71]. In the Ugandan HMM programme, community sensitisation meetings were held to select the community drug distributors (CDDs) who then had the responsibility of promoting the importance of prompt treatment seeking and providing treatment to those children under five brought to them with fever. Both of the evaluations of the programme, conducted 12 and 24 months after the first districts had started implementation, found that the proportion of febrile children for whom care was sought within 24 hours increased in the intervention districts after the introduction of HMM; however, similar increases were also seen in the districts that had not yet received HMM [47, 52]. Both studies found that the proportion of febrile children that actually received treatment promptly with an appropriate antimalarial was significantly greater in the intervention districts than the comparison districts after the HMM intervention was introduced, leading to a 20% overall increase 12 months after implementation [47] and a 13% overall increase a further 12 months later [52] (Table 3). In Mozambique, female 42

52 community health volunteers were trained to promote an integrated package of child survival interventions to their peers, including breast feeding, ORS for diarrhoea, nutritional advice, supervised delivery, immunisations, and prompt care seeking for common childhood illnesses; over the course of the two-year intervention, the proportion of febrile children treated at a health facility within 24 hours increased from 28% (7/25) to 90% (18/20) (p<0.001) and under five mortality rate was reduced by 41.6%, from 180 to 105 deaths per 1000 children (p<0.001) [76]. In two of the remaining four studies, community mobilisation activities were implemented by noncommunity members without a provider-side component to the intervention. For example, microfinance groups in Ghana organised health education sessions on malaria transmission, prevention and treatment and compared the impact on the proportion of children treated with an anti-malarial within 24 hours in these groups with those that either received diarrhoea messages or no education. All of the groups had approximately the same level of prompt treatment of a fever with an anti-malarial post-intervention (around 48%), and there was no statistically significant improvement in prompt treatment seeking from baseline levels in any of the groups [74] (Table 3). The authors suggest that having only one session on appropriate fever treatment may partly explain the lack of effect; in addition, treatment seeking at baseline differed among the three groups, and the two surveys were conducted at different times of the year a serious flaw for comparison of malaria treatment behaviour. A health education programme for mothers in Nigeria consisted of three 45-minute sessions on the cause of malaria, recognition of symptoms, danger signs, treatment and prevention, and incorporated a job aid for the use of CQ. Similar to the other Nigerian demand generation study (which also included a pictorial job aide) [71], prompt treatment seeking was already high before the intervention, so no significant improvement was found; however, the proportion of children treated appropriately with CQ did significantly increase from 68.0% to 92.8% (p=0.003) [73]. The final two studies evaluated interventions that had a provider component to improve availability of anti-malarials in addition to demand creation. The ACCESS project in Tanzania implemented a social marketing campaign with a range of multi-media promotional events including community meetings, road shows and billboards to promote prompt treatment seeking and demand for appropriate antimalarials along with strengthening of informal private medicine retailers. Prompt use of the correct antimalarial increased from 73% (59/81) of febrile children under five at baseline to 88% (44/50) after four years of implementation (OR: 1.30, p=0.03); prompt use of the correct dose of anti-malarial also increased significantly from 32% (49/154) of all febrile respondents to 63% (80/127) in four years (p<0.001). These figures still include SP as the implementation of the policy change to AL only took place a few months before the final survey. Problems with AL stock when the policy first changed meant that the proportion of febrile patients receiving appropriate dose of anti-malarial was only 22% (28/127) if SP was excluded. Although there were no comparison groups in the study, detailed contextual information on the lack of other interventions in the area suggests that the combination of social marketing and improved anti-malarial accessibility through drug shops was responsible for most or all of the improvements in timely treatment seeking [42, 72]. In Cameroon, health staff from a rehabilitated health post conducted community health education sessions in the surrounding villages to promote prompt treatment seeking for fever from the facility. In the study area, there was a significant increase in prompt treatment seeking from 73.3% (74/101) of febrile patients to 89.6% (43/48) (p=0.03); the improvement in prompt and appropriate treatment seeking was also significant and of a greater magnitude, rising from 49.5% (50/101) to 81.3% (39/48) after the intervention (p=0.01). 43

53 Country; Scale; Int length [Ref] Nigeria; District; 2.5 years ( ) [71] Ghana; Sub-district; 5mo (2005) [74] Uganda; District; 12mo ( ) [47] Uganda; National; 2 years ( ) [52, 53] Table 6: Community-based promotion of care seeking for malaria, with or without integration with other health services: Intervention details and evaluation outcomes CHW characteristics & recruitment 2-10 Mother Trainers per village (1 per 50 HHs) selected by community N/A (CHWs didn t deliver intervention) Community drug distributors (CDDs) selected by community (no further details) CDDs selected by community Community mobilisation activities, including training of CHWs, user charges & CHW motivation (where applicable) Initial community sensitisation meetings. Mother trainers provide education to neighbouring mothers on malaria, diagnosis & Tx. Pictorial Tx guideline (made using participatory approach) distributed by mother trainers. No malaria CCM. Supervision activities not described. User charges: not described CHW Incentives: not described Malaria education delivered using interactive techniques via microfinance groups to improve prompt & effective Tx of fever from usual source. 10x 1h sessions on malaria transmission, prevention & Tx. In comparison groups, 6 sessions on cause, prevention & Tx of diarrhoea only. No malaria CCM. User charges: not described CHW Incentives: N/A Sensitisation of communities & meetings to select CDDs. CDDs counselled communities on need for prompt care seeking for malaria. CDDs also responsible for malaria CMM. Supervision not described. User charges: not described CHW Incentives: N/A CDDs mobilised communities to seek prompt care for fever through village meetings and consultations. CDDs also responsible for malaria CMM. Monthly supervision by local HW, quarterly by district User charges: Free CHW Incentives: None Evaluation design; Key treatment seeking outcomes Before/after with control 100% Tx seeking <24h in int & control groups at baseline; 98.9% (176/178) and 99.3% (138/139) post-intervention. Correct use of CQ: 2.6% (3/116) before & 52.3% (69/132) after in int group (p<0.001); 4.1% (3/72) & 15.8% (9/57) in control group (p=0.05). Use of guideline significantly associated with correct use of CQ (OR: 44.6; 95%CI: ; p<0.001) Before/after with control Post-intervention: 48.3% (29/60) children treated with AM <24h in malaria education group, 44.4% (32/72) in diarrhoea group (p=0.7), 48.3% (29/60) in no intervention group (p=1.0). Absolute % change in children treated with AM <24h from baseline: -2.9% in malaria education group (p=0.7), +8.3% in diarrhoea group (p=0.2), +4.4% no intervention group (0.53) Before/after with control Tx seeking <24h: 50% (241/483) & 58% (1053/1816) before & after in intervention districts, 31% (41/134) & 52% (517/995) in control group; overall -11% diff (p=0.002) Tx with correct AM: 14% (134/959) & 37% (671/1815) before & after in intervention districts, 4% (17/431) & 7% (70/995) in control group; overall +20% diff (p<0.001) Before/after with control Tx seeking <24h: 52.2% (85/163) & 76.8% (120/156) before & after in intervention group, 26.0% (25/96) & 64.5% (49/76) in control group; overall -13.8% diff (p=0.12) Prompt & effective Tx: 7.4% (12/163) & 13.5% (21/156) before & after in intervention group, 7.3% (7/96) & 0% in control group; overall +13.5% diff (p=0.01) 44 Facilitators & Barriers Facilitators: Pictorial guidelines & training medicine vendors in addition to education efforts likely to have increased correct CQ Tx. Barriers: None described Facilitators: None described Barriers: Single session on Tx & referral insufficient to motivate behaviour change, compared to 3 sessions on ITNs; presence of other malaria interventions; differences btw 3 groups at baseline; surveys done in different seasons. Facilitators: Improvements in prompt care seeking and Tx with correct AM. Barriers: Considerable problems with stockouts at HFs supporting CDDs. Only 12 months after start of HMM programme, not all districts achieved full implementation. Insufficient skills at HF to supervise & treat referrals. Need to improve use of CDD reports. Facilitators: High appreciation of community CCM as accessible, free & effective; Prepackaging helped improved adherence to dose & duration Barriers: Rapid national scale-up meant insufficient community sensitisation or ownership of HBMF; demand for integrated management of diseases; CDD & communities concerned about lack of financial and supervisory support from HWs for motivation

54 Country; Scale; Int length [Ref] Tanzania; District; 4 years ( ) [42, 72] Nigeria; Sub-district; 0mo* (mo dates) [73] Cameroon; Sub-district; (no dates) [75] Mozambiqu e District; 2.5 years ( ) [76] CHW characteristics & recruitment N/A (CHWs didn t deliver intervention) N/A (CHWs didn t deliver intervention) N/A (CHWs didn t deliver intervention) Community health volunteer s (no further details) Community mobilisation activities, including training of CHWs, user charges & CHW motivation (where applicable) Community sensitisation meetings (av. 40 leaders) & road shows (100s to 1000s attended) in every village with dancing competition, comedies, lecture, short films, Q&A, promotion materials; billboards, posters. No malaria CCM. User charges: not described CHW Incentives: N/A Random selection of 150 mothers; divided in to 8 groups and given three 45minute training sessions on cause of malaria, recognition of symptoms, danger signs, Tx & prevention. No malaria CCM. User charges: not described. CHW Incentives: N/A Community health education to promote prompt Tx seeking for fever from rehabilitated health post, run by research team. No malaria CCM. User charges: not described. CHW Incentives: N/A Health educators trained female CHWs to promote integrated package of child survival interventions: breast feeding, ORS for diarrhoea, nutrition, supervised delivery, immunisations, prompt Tx seeking at HF. Locally appropriate techniques, including role play & drama. Close supervision from programme staff (no further details). No malaria CCM. User charges: not described. CHW Incentives: not described Evaluation design; Key treatment seeking outcomes Before/after no control 73% (59/81) febrile U5s with correct AM <24h in 2004, 88% (44/50) in 2008 (OR: 1.30, p=0.03). 32% (49/154) all febrile respondents receiving correct dose AM <24h: in 2004, 63% (80/127) in 2008 (p<0.001) Before/after no control 84.0% (42/50) treatment seeking <24h pre-intervention, 85.7% (48/56) post-intervention (p=0.8) 68.0% (34/50) febrile children treated with CQ preintervention, 92.8% (52/56) post-intervention (p=0.003) Before/after no control 73.3% (74/101) and 89.6% (43/48) prompt Tx seeking prepost intervention (p=0.03); 49.5% (50/101) and 81.3% (39/48) prompt & appropriate Tx seeking pre-post intervention (p=0.01) 27.1% (50/185) and 13.3% (16/120) fever (p=0.001), 21.6% (40/185) and 13.3% (16/120) moderate anaemia (PCV 15-30%) (p=0.001) pre-post intervention Before/after no control 28% (7/25) febrile children treated at HF <24h at baseline, 90% (18/20) post-intervention (p<0.001) Reduction in U5 mortality from Mar00 to Feb03: 62.2% from community data (119 to 45 per 1000); 41.6% pregnancy history survey (180 to 105 per 1000) 45 Facilitators & Barriers Facilitators: SM considered effective in increasing knowledge and HF attendance as no other local interventions at same time (although some national radio spots); timely Tx due to SM and increased accessibility of AMs at ADDOs. Barriers: Change in drug policy & associated stock-outs (SP in HFs & AL in ADDOs) during the transition reduced % patients receiving appropriate Tx. Facilitators: None described Barriers: None described Facilitators: None described Barriers: None described Facilitators: VHC supported community initiatives & spread health messages; programme started in post-emergency context (floods); integrated routine data system & supportive supervision for timely decision-making; high volunteer: popn ratio, support network Barriers: None described * Evaluation conducted immediately after intervention training. Abbreviations: ACT=artemisinin combination therapy ; ADDO=accredited drug distribution outlet; AL=artemether-lumefantrine; AM=anti-malarial; CDI=community-directed intervention; CHV=community health volunteer; CHW=community health worker; CI=community implementer; CQ=chloroquine; HH=household; HMM=home management of malaria; HF=health facility; int=intervention; ITN=insecticide-treated net; N/A=not applicable; NGO=non-governmental organisation; ORS=oral rehydration salts; PCV=packed cell volume; SM=social marketing; SP=sulfadoxine-pyrimethamine; Tx=treatment; U5=child under five; VHC=village health committee

55 5.6 Quality of included studies The seven RCT and five CBA studies included in the review were assessed for risk of bias, based on guidelines from the Cochrane Effective Practice and Organisation of Care (EPOC) group [36]; each study was given a classification of DONE, NOT CLEAR or NOT DONE on the following categories: sample size calculations; randomization and blinding procedures (where appropriate); similarity of intervention and control areas at baseline; and completion of participant follow-up (Table 7). Overall, the quality of the seven RCTs was reasonably high, particularly in appropriateness and clarity of sample size calculations and minimization of selection bias through randomization. Similarity of the intervention and control arms at baseline was generally good in terms of the primary outcome of interest and general socio-demographic indicators. However, three studies did not provide data on outcome measures at baseline, meaning it was not possible to assess quality. Three studies presented outcome and socio-demographic data at baseline but were classified as NOT DONE due to significant differences in either the primary outcome (haemoglobin levels and parasite positivity [46]) or sociodemographic characteristics (immunization status [70], or ethnicity [51]), however none of the other measured characteristics were significantly different. The quality of studies in terms of measuring and reporting completion of follow-up for study participants was mixed: three studies included loss to follow-up (that was minimal in all cases) in a trial profile, three did not include a trial profile hence loss to follow-up could not be assessed, and one included a trial profile in the paper but did not specify loss to follow-up in the study (Table 7). Blinding of the outcome assessment was either not clear or not done across all RCTs. However, it could be argued that this would not always have implications for introducing bias, for example where the outcome could be objectively measured (haemoglobin levels or parasite positivity by RDT). The study which measured intervention impact on malaria mortality using verbal autopsy could however be subject to bias in interpretation depending on the assessor knowing whether the patient belonged to the intervention or control arm [50]. With the exception of sample size calculations (which were clearly described for four studies), the quality of the five studies with a CBA design was considerably lower than the RCTs included in this review: Four of the five studies found significant differences at baseline in primary outcome and sociodemographic characteristics between the intervention and control groups (Table 7). Similarly, blinding of measurement of the outcomes and loss to follow-up were either not done or not reported for all studies. The lack of similarity of intervention and control groups at baseline makes interpretation of results difficult as it is not clear whether any changes in outcome are due to the intervention or other contextual factors. Blinding and loss to follow-up are not conventionally reported for CBA designs, however, may have influence on the validity of results, for example if there is a risk of measurement or exclusion bias having occurred. 46

56 Table 7: Summary of study quality for randomised controlled trials and controlled before & after studies included in the review Study Completeness Similarity of int & Similarity of int & Concealment of primary Blinding of Power of follow-up control groups at control groups at calculation of study baseline - primary baseline allocation outcome participants outcome demographics Randomised controlled trials Staedke et al [55] DONE DONE DONE NOT CLEAR DONE DONE Mubi et al [61] Winch et al [69] Yeboah- Antwi et al. [70] Kidane & Morrow 2000 [50] Eriksen et al [46] Kouyate et al. [51] DONE DONE DONE NOT CLEAR (not specified) DONE DONE NOT CLEAR (no trial profile) DONE DONE (villages >15km from HF excl) DONE DONE NOT DONE (loss to followup not in trial profile) DONE DONE NOT CLEAR (no trial profile) DONE DONE NOT CLEAR (no trial profile) Controlled before & after studies Ajayi et al [71] DONE N/A NOT CLEAR (not reported) Nsungwa- Sabiiti et al [52] Fapohunda et al [47] De La Cruz et al [74] NOT CLEAR DONE NOT CLEAR (not reported) DONE NOT CLEAR NOT CLEAR (not reported) NOT CLEAR (not specified) DONE DONE DONE NOT DONE (sig difference in immunisation status) NOT CLEAR (not reported) NOT DONE NOT DONE (sig differences in Hb & parasite positivity) DONE NOT DONE DONE NOT DONE (sig difference in ethnicity) NOT DONE DONE NOT CLEAR (descriptive text about setting, no clear table) DONE N/A NOT DONE NOT DONE NOT CLEAR (similar on overall Tx indicator, different Tx <24h) DONE N/A NOT CLEAR (not reported) NOT CLEAR (no detail of sample size calculations) N/A NOT CLEAR (not reported) NOT CLEAR NOT DONE NOT DONE (sig differences in prompt Tx seeking & anaemia) NOT DONE (sig differences in some malaria knowledge indicators) NOT CLEAR (descriptive text but no clear table) NOT DONE (not reported) NOT DONE (sig difference in age, water source) 47

57 Study Lemma et al [41] Power calculation Concealment of allocation Completeness of follow-up of study participants Blinding of primary outcome DONE N/A NOT DONE NOT DONE (not specified if verbal autopsy blinded) Similarity of int & control groups at baseline - primary outcome NOT DONE (difference in baseline parasite positivity at high transmission) Similarity of int & control groups at baseline demographics NOT DONE (sig difference in altitude, distance to health facility) 6. Discussion The 42 studies included in this review represented a range of 16 countries across sub-saharan Africa. The profiles of the CHW programmes varied, including those in which users paid for consultations or treatment and those where services were delivered free of charge, and those where financial or nonfinancial incentives were given to the CHW or where the intervention relied on intrinsic motivation. Other programme variables included the size of population covered by each CHW (median of 553 people per CHW; IQR: 374, 1750) and the proportion of CHWs that were male (median 46.5% male; IQR: 9.5%, 72.0%). However, there were certain characteristics that were consistent across the studies. Namely, CHWs were members of the community that they served and in the vast majority of cases were elected during public village meetings. Likewise, the minimum requirements for a CHW to be elected included basic literacy, availability and accessibility, and a willingness to volunteer and serve. The majority of the studies were conducted at a time when national policy called for presumptive diagnosis of malaria by CHWs. However, there is a growing evidence base for the ability of CHWs to use RDTs and treat appropriately according to the result, with eleven studies reporting data on RDT use. Likewise, around sixty percent of studies involved the use of ACT by CHWs. This review therefore presents a useful update of current policy context to the 2007 review by Hopkins et al. which could only draw upon published literature on presumptive treatment of malaria by CHWs using CQ, SP or CQ-SP [20]. Overall, 32 studies reported on indicators of CHW quality addressing objective one of this review; eleven provided information on referrals from the community to health facility level (objective two); eleven reported on some element of health system capacity to support malaria CCM, such as supervision, supply chain management or treatment of referred cases (objective 3); ten investigated integration of malaria CCM with other health services (objective 4); seven involved demand generation activities at the community level (objective 5); and a total of eleven studies reported clinical outcomes. Although most studies were conducted at a time when presumptive treatment of malaria remained policy at the community level, they still provide a reasonable evidence base for each objective from the more recent approaches of malaria CCM with confirmed treatment, and treatment integrated with other services (Figure 2). 48

58 No. studies Figure 2: Number of studies addressing each objective, by classification of intervention Malaria CCM only; presumptive Tx Malaria CCM only; confirmed Tx Malaria CCM & other services Demand generation only Objective 1: Quality Objective 2: Referral Objective 3: HS Capacity Objective 4: Integration Objective(s) addressed Objective 5: Demand Clinical outcomes 6.1 Interventions to improve quality of CHW performance Results from the sixteen studies that involved presumptive treatment of malaria found that CHW performance was very high in terms of delivering the correct anti-malarial and at the correct dose; amongst the nine studies that reported this outcome indicator, the median proportion of febrile children that received the correct dose of anti-malarial was 97.8% (IQR: 94.8%, 98.0%). All of these studies involved pre-packaged anti-malarials which have been previously demonstrated to improve provider and user adherence to correct dose [77, 78]. Prompt treatment seeking from a CHW was also high at around 90%, although this finding was only reported in four of the presumptive studies [43]. Four other studies reported on a combined indicator of prompt treatment with the correct dose which was lower, with a median of 69.8% (IQR: 55.2%, 75.6%) of febrile children under five receiving the correct dose of anti-malarial the same day or the next day across the eight studies. One of the lowest results for this indicator was the study by Akweongo et al. in Burkina Faso where there was a large discrepancy between household survey data and CHW records in the proportion of children that received the correct dose of anti-malarial from a CHW (25.9% and 94.8%, respectively) [38]; the composite indicator of prompt and effective treatment came from household survey data and was low. The authors did not suggest any reasons for this difference, although it may perhaps have related to caretaker recall or misunderstanding of the questions on dose. One of the two studies that had a CBA design also found a high level of prompt and effective treatment of 86.3% in the post-intervention areas [51]; however, in the study of the national HMM programme in Uganda, only 13.5% of all febrile children received prompt and effective treatment post-intervention (although it should be noted that this was from any source) [53]. Taking into consideration changes in the comparison areas, both studies found an overall increase in prompt and effective treatment that could be reasonably attributed to the CHW intervention of 47.6% in Tanzania and 13.5% in Uganda [51, 53]. The evaluations of the national HMM programme in Uganda and a regional programme in Burkina Faso are the largest scale studies in the review. Both were carried out with limited external support, for 49

59 example for supervision or anti-malarial supply, in comparison to some of the other studies. It is perhaps revealing that these studies showed more modest achievements in terms of CHW performance: 32.3% of children treated with pre-packaged CQ-SP (Homapak) in Uganda received the correct dose within 24 hours [53]; in Burkina Faso, 56.4% of febrile children were treated promptly with a pre-packaged antimalarial [54]. Reasons for this result offered by the authors include the need for more community mobilisation and stronger supervision in Uganda, and for improvements to the pre-packaging in Burkina Faso which was conducted before blister packs were commonly available [53, 54]. The eleven studies that investigated RDT-confirmed diagnosis of malaria showed encouraging results in the ability of CHWs to use RDTs and treat according to the result when they received practical training with an opportunity for observation of practice and problem solving; as one study in Zambia showed, manufacturing guidelines alone are insufficient to achieve good RDT practice [59]. In many of the RDT studies, the CHWs already had experience of treating malaria presumptively but this did not seem to influence their willingness to treat only RDT-positive cases. The one exception was a small study in Sudan where 30% (7/20) of CHWs admitted preferring to use clinical criteria for treating malaria; the reasons were unclear [58]. These findings were in contrast to a number of studies of formal health worker use of RDTs which found that significant proportions of RDT-negative patients were still prescribed ACT [79, 80]. Although ACT was correctly given to a high proportion of RDT-positive patients and few RDT-negative patients, there remains a dilemma about what is the best course of action for RDT-negative patients at the community level. A number of the RDT studies found that CHWs would like to be able to treat the alternative causes of fever in these patients [57, 58], a finding for which there is argument in light of the limited evidence for effective referral. However, the four studies that also investigated the ability of CHWs to treat uncomplicated pneumonia found that correct classification of pneumonia by CHWs based on breathing rate can be challenging [40, 65-67, 70]. Nevertheless, with simple guidelines, appropriate practical training and close supervision, accurate classification can be achieved; and, as in the instance of the Zambian RCT, significant improvements can be made in the prompt and appropriate treatment of pneumonia when compared to referral of non-malaria fevers to a health facility [40, 70]. Another important point to note is that, in addition to the ability of CHWs to use RDTs, parasite positivity and the relative cost effectiveness of confirmed versus presumptive diagnosis should be considered in deciding whether to include RDTs in malaria CCM [81]. For example, in DRC 93.0% of febrile children under five tested by a CHW were RDT-positive. The authors conducted a simple cost effectiveness threshold analysis and found that at 80% prevalence, the cost per unnecessary course of ACT averted was US$8.79, approximately eight times the cost of an ACT dose, or 60% of the annual per capita health budget [60]. Alternatively, in Zambia, where only 23.3% of children tested with an RDT were positive, the relative cost of CCM was US$4.22 per malaria case treated compared to US$6.61 in a formal health facility [56]. Eight of the sixteen presumptive studies and two of the eleven RDT studies reported clinical outcomes, with varying outcome measures and mixed findings. For example, the significant reductions in all-cause mortality and severe malaria morbidity respectively shown by Kidane & Morrow in Ethiopia and Sirima et al. in Burkina Faso in the early 2000s provided strong support for the early home-based treatment of fever programmes [50, 54]. One more recent study in Ethiopia by Lemma et al. also found a significant reduction in malaria-specific mortality in intervention districts where CHW had RDTs and ACT, compared to controls districts without malaria CCM [41]. However, RCTs of home management of malaria in 50

60 Uganda, Burkina Faso and Tanzania did not find a significant impact on morbidity outcomes such as moderate anaemia, haemoglobin level or splenomegaly [46, 51, 55]. One encouraging finding in terms of clinical outcomes is that amongst four studies that measured day 28 PCR-adjusted cure rates, over 90% of patients had completely cleared their infection after treatment with ACT from a CHW, supporting the ability of CHWs to prescribe appropriate dose and for users to adhere to this dose [37, 62]. This limited and mixed evidence on clinical impact of malaria CCM is supported by another recent systematic review that found very few studies that have measured the health impact of CHW programmes for childhood diseases [25]. The few studies that did measure clinical impact did not tend to measure process or outcome indicators of prompt and effective treatment; hence it is difficult to ascertain the relationship between CHW performance and ultimate health impact. An interesting pattern noted was the apparent maturation of effect of CHW performance over time (Figure 3a). Although measurement of CHW knowledge or practice immediately following training revealed strong results [40, 59, 60], most of the studies were evaluations of programmes that had been running for at least 12 months. CHW performance was generally good; however, those studies that followed the CHWs for longer periods and included refresher training tended to find improvements. For example, the Siaya study in Kenya and two of the CDI study sites (Uganda and Nigeria) found that prompt and effective treatment was greater after two years of intervention compared to one year postintervention [39, 65]. This finding perhaps suggests that initial teething problems for operational programmes (such as insufficient community mobilisation or supervision [47, 48, 53]) could offset knowledge that is fresh in CHWs minds after training. Figure 3a suggests that there may be a reduction in quality over the longer term; however, only two studies followed CHWs for longer than two years, and in both of them the proportion of children receiving the correct dose of anti-malarial was still around 90% [40, 65]. As found by other reviews and commentaries on CHW programmes, ongoing supervision was crucial to sustainable success [23]. Interestingly, there was no clear overall pattern associated with increasing length of intervention when the correct treatment indicator includes promptness of treatment (Figure 3b). On the whole, prompt and effective treatment is reasonably high (median 68.7%, IQR: 51.5%, 76.7%), although there are clear outliers when the proportion of children receiving prompt and correct treatment is below the median, as previously discussed. Differences in study design and sources of data mean it is important to be cautious in interpreting findings; however, the need for sustained community mobilisation activities to encourage prompt treatment seeking is clear. 51

61 Figure 3: Scatterplots of quality of malaria CCM programmes against length of intervention period, in terms of (a) proportion of children under five with fever/ malaria given the correct dose of antimalarial; and (b) the proportion of children with fever receiving correct anti-malarial within 24 hours (a) (b) 52

62 Summary box 1: Key findings and recommendations - Interventions to improve quality of diagnosis and CCM of malaria by CHWs Key findings High adherence by CHWs to correct dose of anti-malarial was seen across the vast majority of studies, irrespective of diagnosis or anti-malarial policy, or strength of study design; in large part this is due to the benefit of pre-packaged anti-malarials and sufficient practical, interactive training techniques. Prompt and correct treatment of malaria is less consistent and tends to be lower. Community mobilisation towards prompt treatment seeking should be emphasised. Larger scale studies with less external support had more modest results for prompt & effective treatment of malaria than more rigorously controlled research studies. CHWs also demonstrated high ability to safely use RDTs and adhere to results, prescribing ACTs for majority of RDT-positive patients (and minimum ACT prescription for RDTnegatives); challenges remain with action to take for RDT-negative patients. Cost effectiveness of use of RDTs will depend upon level of parasite prevalence. Evidence on CHW ability to diagnose and treat pneumonia is mixed. Findings suggest strong practical training, clear guidelines and regular supportive supervision with opportunity for problem solving are critical for maintaining CHW quality, especially for pneumonia treatment. Few studies evaluated integration of malaria CCM with other interventions. However, there is no indication that integration reduces quality of CHW malaria treatment. Recommendations for programme managers Interactive training of at least three days is a minimum requirement to achieve CHW adherence to correct anti-malarial treatment. At least one additional day of training on safe use of RDTs is needed, including practice and opportunity for problem-solving to achieve accurate malaria diagnosis and treatment. Accurate classification of simple pneumonia by CHW is challenging and demands close training, supervision and monitoring. Opportunity for regular problem-solving with supervisors improves CHW motivation. Recommendations for future research More evidence is needed on appropriate treatment of RDT-negative patients at the community level, including impact of different strategies on health outcomes. Investigation of the minimum frequency and format of supervision required to maintain CHW performance. Limited evidence is currently available on integrated CCM. 6.2 Interventions to strengthen referrals from community to facility-based providers Overall less than a third (11) of the thirty-six studies involving treatment of malaria by CHWs reported on any indicator of referral, either the proportion of patients referred to a health facility and/or the proportion that completed their referral. This finding reflects a low level of reporting, considering the 53

63 effective referral of cases that the CHW does not have the capacity to treat is a crucial element to safe implementation of CCM. However, it is recognised that these indicators can be difficult to monitor, especially referral completion, which requires two-way communication with the nearest health facility, for example to establish a system of written counter-referrals from the health facility back to the CHW. Only two of the studies found by this review specifically evaluated interventions to improve the referral system from the community to formal health facility level. The study by Winch et al. in Mali found increases in the proportion of children referred and completing referral post-intervention, although there was a discrepancy between household survey results and the CHW referral record. After further investigation, the authors found that CHWs were more likely to give non-formal verbal referral advice to those patients whom they did not expect to complete the referral, but to make a formal written reference for those that were likely to comply [69]. The study by Kelly et al. in Kenya also found evidence of CHWs making judgements on which patients to refer, reporting that they were cautious about referring patients unnecessarily and causing them to incur costs that they could not afford, often choosing instead to monitor patients at home rather than refer immediately [65]. The study by Thomson et al. in Sierra Leone found a very high completion rate for patients referred to a health centre with severe malaria (almost 90%), compared to RDT-negative patients (only 1%) following the introduction of new referral registers with training at the community and facility levels [63]. Another study in Uganda reported that referrals for urgent cases, children less than one year old, and those to whom clear instructions were given were more likely to complete the referral [49]. This finding highlights the importance of reporting a disaggregated indicator for referral completion according to the reason for referral, e.g., RDT negative and/or severe disease. When CHWs were trained to treat all cases of fever as malaria, criteria for referral were relatively straightforward and generally involved training on recognition of danger signs and referral of such patients to a health facility after treatment with an anti-malarial. However, with the use of RDTs by CHWs arises a challenge of what action should be taken for patients with a negative RDT. One option is that all negative cases be referred to a health facility for further diagnosis. However, as the study by Thomson et al. shows, the chance of completion for such patients may be low [63]. Reasons for this action are likely to involve judgements by the caretaker, balancing the resources involved in attending a health facility (including time, transport and user charges) and the quality of care expected when they reach the facility with the possibility that the child s fever may resolve on its own. The alternative is that CHWs be given the capacity to treat RDT-negative patients. However, other causes of childhood fever are not always clear; some may be classified as pneumonia based on increased breathing rate and other symptoms, shown to be possible for CHWs with appropriate training [67, 70], while other fevers may be due to bacterial infections which could also be treated with broadspectrum antibiotics. Nevertheless, a relatively high proportion of fevers are likely to be due to nonspecific viral infections which would not respond to antibiotics and may only be treatable with an antipyretic. Therefore, the risk remains that children with a negative RDT may develop severe disease if not treated promptly, either at the community or facility level. Although only one study investigated referral completion by children with a negative RDT, the findings are dramatically low and may support the argument for CHWs to have capacity to diagnose and treat simple pneumonia. In addition, clear criteria for which RDT-negative patients must be referred (at a minimum those with any danger signs) and appropriate counselling of caretakers of these children is crucial. Development of further point-of-use diagnostic tests to complement RDTs and help differential diagnosis is desirable. In the meantime, clear process monitoring of CHW referral systems and 54

64 qualitative studies of provider and caretaker behaviour around referrals is essential to ensure patient safety. Summary box 2: Key findings and recommendations - Interventions to strengthen effective referral from the community level Key findings Less than one third of studies reported on referrals between the community and health facility levels, despite this being a crucial function of any CHW programme. CHW may make judgement decisions on who to refer, for example depending on the likelihood that the patient will be taken to a health facility or the user costs involved. Children with severe disease, of very young age, and those given clear instructions by the CHW were more likely to comply with referral advice. Recommendations for programme managers Develop clear guidelines for referral procedures and ensure sufficient training of CHWs and health facility staff on these guidelines. Monitor and report proportion of cases referred and proportion of referrals completed; disaggregate these measures by reason for referral (RDT positive/ negative, severe/ non-severe illness). Recommendations for future research Basic research into point-of-use diagnostics to guide treatment for RDT-negative patients. Further in-depth studies of provider and user behaviour around referrals at the community level (including CHWs and facility staff). 6.3 Interventions to strengthen health system capacity to support malaria CCM Three main areas have been highlighted in this review in order to understand and build health system capacity to support CCM: (i) ability to treat referred cases; (ii) regular supervision of CHWs; and (iii) reliable and sufficient medical supplies at the health facility and community levels. Although data on these indicators were collected and reported by Fapohunda et al. in their early evaluation of the Ugandan HMM programme [47], it was not possible to extract such information from any of the other studies. Indeed the only other objective measure of health system capacity was the proportion of referrals that were completed and this was only reported by six of the 42 studies [45, 48, 49, 57, 63, 67]. Also, although referral completion may give some indication of the functionality of the referral system and an indirect indication of the accessibility and perceived quality of health facility care, it does not give any information on the actual quality of care likely to be received at the reference facility; for example, health worker capacity to manage severe malaria and other common childhood illnesses is also important to measure and improve if necessary. Studies that specified a separate health worker or health system strengthening component to the malaria CCM intervention were few. In addition to the two referral system interventions previously discussed [63, 69], only four studies reported specifically on training health workers to support CHWs [46, 48, 51, 54]. For example, two studies reported that pharmacists were trained to strengthen stock management at the health facility and maintain the supply chain to the community level; for both of these it was reported that there was good availability of anti-malarial stock during the period of 55

65 evaluation [48, 54]. Few other studies, however, reported any indication of continuity of anti-malarial stock during the intervention, a crucial element to understand CHW and health system performance. In one of the other two studies, health workers were retrained on malaria case management [46], and in the other, staff at the health facility were trained to supervise the CHWs, collect reporting forms and deliver drug supplies [48]; the impact of this training was not clear from the papers. Although supportive supervision could be expected to improve quality, and the frequency of supervision was reported for a number of studies, attributing any impact on CHW performance to supervision amongst other elements of the intervention is challenging. In the study by Franco et al., infrequent supervision was reported to be due to other health worker priorities and a lack of time and fuel [48]. This finding highlights that training alone is likely to be insufficient to improve health system capacity and that more in-depth assessment of the needs at the supervising health facility level will be needed to design appropriate interventions to support CCM. For malaria CCM programmes to be sustainable, they need a minimum level of external support. Ideally this support will be provided by the local health system. However, community-based interventions are often introduced to improve access to effective basic preventive and curative services by underserved populations where existing health systems are dysfunctional. Hence additional support from donors, bilateral agencies or NGOs may be required. CHWs can provide quality services for hard-to-reach populations. However, there remains a fine balance between sustained motivation of CHWs, especially as new tasks are added to their responsibilities, and the financial resources required to effectively support them in terms of supervision, supplies and potentially financial incentives. Finding and funding this balance is fundamental to the sustainability of CHW programmes. 56

66 Summary box 3: Key findings and recommendations - Interventions to strengthen health system capacity to support CCM Key findings Elements of health system capacity found to be critical for effective CHW programmes include: (i) ability to treat referred cases; (ii) regular supervision of CHWs; and (iii) a reliable and consistent supply chain for essential medicines and equipment to the community level. Information on these elements from included studies was generally qualitative. Development of complementary quantitative indicators to monitor health system support is desirable. Once areas of weakness have been identified, innovative and sustainable interventions are required. Design & evaluation of appropriate intervention studies are needed. Recommendations for programme managers Systems to ensure regular supervision, collection and use of monitoring data, and supply of medicines are crucial for any CHW programme. District & health facility staff must be included from the start of the intervention and any training needs identified to enable them to effectively support CHWs in their catchment e.g. refresher training on IMCI, supply chain management, supportive supervision. Respect & support from formal health workers is a source of CHW motivation. Recommendations for future research Evaluation of interventions to improve effective supervision and professional support to CHWs and supervisors within resource-limited settings. Evaluation of interventions to improve consistent supply of essential medicines, especially ACTs and RDTs throughout the health system, extending to the community level. 6.4 Integrated CCM of malaria and other interventions The nine studies in which CHWs delivered other child health services in addition to case management of malaria varied in content. Two studies in Uganda and Zambia that introduced RDTs also trained on the ability to diagnose and treat uncomplicated pneumonia with antibiotics [40, 70] with positive results in terms of CHW performance. The other seven studies included broader health interventions: the three CDI studies in Cameroon, Nigeria and Uganda that trained CHWs to deliver ITNs, DOTS for tuberculosis and vitamin A supplementation [39]; in Kenya and Rwanda CHWs could treat diarrhoea and uncomplicated pneumonia [65-67]; in Mali village drug kit managers had ORS for diarrhoea, and could treat eye infections and basic first aid [69]; and in Malawi a cadre of health surveillance agents received six weeks of training to deliver vaccinations and broader primary health care such as family planning, nutrition and promotion of improved water and sanitation [38]. In general, the findings were consistent with the malaria-only programmes in terms of achieving high standards of malaria CCM, both presumptive and confirmed. As previously discussed, ability of CHWs to treat pneumonia effectively was strong when given appropriate training [40, 67, 70]. Quality of CHW performance in the Kenyan study was lower, although over-complicated guidelines and infrequent supervision were identified as the reasons [65, 66]. In the evaluation of the Rwandan ICCM programme, the proportion of patients with diarrhoea or pneumonia that received correct treatment was over 70% [67]. The studies of integrated service delivery by CHWs in Mali and Malawi did not report on nonmalaria indicators to assess performance on their wider tasks [38, 69]. 57

67 Overall it seems that integration of additional activities in malaria CCM programmes does not have a detrimental effect on the quality of CHW performance, within the previously discussed conditions that sufficient training and supervision are provided. Interestingly, there is no indication that increased responsibility and number of tasks led to an increased demand for financial motivation from CHWs involved. However, it is hard to determine if this issue was objectively assessed; it may be that CHW motivation was not formally evaluated and, even if a study does report that CHWs made requests for remuneration, it is hard to know if such remuneration is vital to their motivation or a response to being asked if there is anything they would like to improve. Considering the increasing number of countries that have an ICCM programme, or are at least piloting the approach, there is currently limited published evidence on the effectiveness or quality of implementation. It will be important to synthesise the findings from current and planned ICCM programmes to inform the future direction of malaria CCM and the lessons learnt for effective implementation. Summary box 4: Key findings and recommendations - Interventions to improve integration of malaria CCM & other interventions Key findings Malaria CCM has been integrated with various other health interventions, including treatment of other common childhood diseases and promotion of preventive interventions (vaccination, vitamin A, ITNs, nutrition). Additional tasks do not seem to reduce the quality of malaria CCM, provided sufficient training, supervision and support is maintained. With the exception of pneumonia treatment, reporting on the quality of delivery of these other interventions is limited. Recommendations for programme managers Keep integrated treatment guidelines and algorithms clear and as simple as possible to avoid confusion and errors of misclassification. Regular supervision of practice and refresher training is crucial to maintain quality over time. Recommendations for future research What is the threshold in terms of time commitment and responsibility for expecting CHWs to carry out their tasks on a voluntary basis? What lessons can be learned on coordination of community activities that fall under different programmes? 6.5 Community-level promotion of prompt care seeking Amongst the 36 treatment studies, sixteen described initial community sensitisation meetings to raise awareness of the intervention and fourteen described community mobilisation activities by the CHWs to promote prompt treatment seeking for fever (with some overlap in the interventions carrying out these activities); seventeen studies did not describe any sensitisation or mobilisation activities. Only two of these 36 studies, plus six further non-treatment studies, were analysed in more detail based on sufficient study design to assess the effectiveness of community-level demand creation activities. 58

68 The eight studies showed limited evidence of success, especially in terms of increasing prompt treatment seeking for malaria which was generally already high in the study areas. There was a more impressive improvement in the correct use of anti-malarials, for example in two studies in Nigeria that found an increase in the proportion of febrile children receiving the correct dose of chloroquine; However, both of these interventions also gave the CHWs pictorial treatment guidelines to promote appropriate treatment, suggesting that these were the more successful component of the intervention [71, 73]. In general, the interventions that also involved a component to improve provider behaviour and availability of quality anti-malarials achieved better results; key examples are the establishment of a network of accredited drug dispensing outlets in Tanzania [42] and provision of improved services at a public health post in Cameroon [75] that both achieved increases of around 30 percentage points in children receiving prompt and appropriate treatment from baseline levels. There were exceptions however: the evaluation of the national HMM programme which had user and provider components achieved limited success due to reasons of intermittent stock, lack of supervision, and inconsistent mobilisation activities as previously discussed [53]; reasons suggested for a successful demand-only intervention in Mozambique were a high CHW-to-population ratio and the post-emergency setting with many NGOs providing health care [76]. It is important to note that the indicators for these demand generation interventions were slightly different to the CHW treatment studies, as the measures of prompt treatment seeking or prompt and effective treatment were for all febrile children not disaggregated by source of treatment; in most of the CHW studies discussed, the indicators were for children visiting a CHW. The fact that the results tend to be less impressive for those studies that evaluated success based on overall treatment seeking from any source is not surprising given that care may be sought from numerous sources and that these are likely to be of varying quality. This finding supports the notion that multi-sectoral interventions are needed at all levels where treatment quality is low if one is to expect greater improvements in access to prompt and effective treatment, and ultimately a positive impact on health outcomes. Considering potential barriers to access to health care, user fees present a notable one, particularly for the poorest groups. Recent literature reviews have found that removal of user fees in the formal health system tended to increase utilisation of curative services [21, 82]. However, it is important to note that the topic is complex and there are numerous challenges to the policy process of user fee removal that must be managed to avoid unintended consequences, not least reductions in quality of care [5, 81]. A point of interest from the studies included in this review is that there seems to be a trend for removal of user fees for consultations or treatment over time, although cost was not necessarily perceived as a barrier by caretakers during focus group discussions in those studies with a user charge [38, 43, 48, 54]. However, since only one paper disaggregated results on prompt and effective treatment by socioeconomic status [53] it is not possible to judge in a more objective manner whether or not user charges present a barrier to equitable access and more data is needed on this. Similarly it is not possible to comment on the achievement of one of the fundamental objectives and stated strengths of CHWs in improving socio-economic (and geographic) access to care [14]. Often CHW programmes are designed to target hard to reach communities, for example those further than 5km from a health facility or in the lowest socioeconomic quintiles; hence, it may be that equity was incorporated into the design of the intervention rather than being directly measured. Nevertheless, it is still important to evaluate whether a CHW programme has actually achieved improved equity of access to included interventions. However, in order to do this, a household survey is required that measures treatment seeking by source before and after the intervention with results disaggregated by socio- 59

69 economic status and distance from the treatment source; this requirement may be overly resource intensive for many programmes. Another recent literature review suggests that in order to fully understand treatment seeking behaviour, characteristics of providers should be considered in addition to those of users [17]. The review by Kizito et al. found that traditional barriers such as cost and proximity are important; however, they also report that health services may increase their appeal if they responded to user preferences for friendliness, effectiveness, compassion and shorter waiting times; these positive attributes were often drawn from CHW studies included in their review [17]. Summary Box 5: Key findings and recommendations - Interventions to increase treatment seeking for malaria Key findings Community mobilization activities to encourage prompt treatment seeking for fever were more successful when conducted alongside an intervention to improve provision of malaria treatment. Malaria CCM interventions with insufficient mobilization efforts had low demand for CHW services. There is no conclusive evidence on the impact of user fees for consultations or treatment on CHW utilization or socioeconomic equity of access. Only one study disaggregated results according to wealth quintile; none assessed achievement of improved geographic access to treatment through malaria CCM. Recommendations for programme managers Community sensitisation and mobilisation is essential to ensure demand for CCM services; demand and respect from their own community acts as strong motivation for CHW. Promotion of prompt treatment seeking for fever without parallel improvements in service provision has little effect. Monitor accessibility of CCM services according to measures of equity (e.g. socioeconomic, geographic) Recommendations for future research What are the implications of user fees on equitable access to CCM versus sustainability of the programme? What are the reasons for delayed treatment seeking by caretakers of children with fever, and how can interventions be designed to address this? 6.6 Issues of study design Study design is important for the reliability and validity of the measured effect of an intervention. Randomised controlled trials are considered the gold standard for evaluating an intervention, although the Cochrane EPOC group also accepts CBA studies or interrupted time series with a sufficient number of data points [36]. Using criteria recommended by the Cochrane EPOC group, risk of bias was assessed for the seven RCT and five CBA studies included in this review. On the whole, the RCTs were of good quality with low risk of bias; however the CBA studies suffered from considerable risk of bias. One particular finding of note was the significant differences between intervention and control groups at baseline for the CBAs, highlighting the challenge of matching districts or communities as units of allocation to intervention or control, and reflecting the reality of evaluating implementation under programme conditions compared to RCTs. 60

70 No. Studies Thus, although RCTs are considered the gold standard in epidemiological terms, there are a number of limitations to using this design to evaluate an intervention under operational conditions or when delivered at a large scale, and alternative designs may be appropriate, depending on the audience and role of the evaluation [83]. This is the justification for the inclusion in this review of before and after studies without control, and post only studies with or without control; despite methodological limitations, these were considered to avoid exclusion of useful programme data on CHW performance. It is important to note that post only studies were only included where it was possible to attribute the outcome of interest directly to the CHW, namely data from CHW registers or household surveys where information was collected on treatment by source [84]. Similarly, before and after studies without a comparison group were only included if the study provided sufficient contextual information to support attribution of any changes to the intervention being evaluated [17]. Figure 4: Number of studies addressing each review objective, by study design Objective 1: Quality Objective 2: Referral 6 Objective 3: HS Capacity Objective(s) RCT (cluster or non-cluster) Before & after with control Before & after no control Post only with control Post only no control Objective 4: Integration 4 4 Objective 5: Demand Clinical outcomes As Figure 4 shows, the greatest proportion of the evidence for each objective of this review was drawn from post only studies without control groups, relying in large part upon CHW register review. Interestingly, for three of the five objectives RCTs provided the next most common evidence base. Both clinical and non-clinical performance-related outcomes were measured by both types of study and results were not noticeably different in magnitude or direction according to the quality of the study design; in particular the positive findings on potential quality of CHWs in managing malaria at the community level were consistent, including in the era of RDTs and ACTs. However, studies were not appropriately designed to answer questions about the relative role of different components of a successful malaria CCM programme in improving CHW performance, for example whether training (and what aspect of training), supervision (what frequency and format), or job aides were most important. All of these elements are important, but to distinguish between them specific study designs would be needed. 61

71 It is important to note the potential limitations in terms of the reliability of data extracted from the included studies. Many of the studies reported data on anti-malarial use and dosage from reviews of the CHW registers, which may suffer from incomplete or inaccurate reporting. Others report the proportion of febrile children receiving an effective anti-malarial at the correct dose from household surveys which can have problems with recall and understanding. Direct observation of CHW performance is considered the gold standard, yet it also has implications for influencing behaviour and is both time and resource intensive, so it is not suitable for routine monitoring. However, despite these limitations, the small number of studies that collected data from both registers and household surveys generally found encouragingly similar results [38, 43]. Where possible, validation of data from different sources is desirable. Summary Box 6: Key findings -- Programme evaluation design Evidence for this review is drawn from studies with a spectrum of designs, from RCTs and controlled before and after studies to post only studies without control. Questions remain about the relative effectiveness of interventions relating to particular aspects of CHW quality, for example the relative importance of supervision or different models of training. More rigorous study designs may be required to investigate these specific questions. Less rigorous study designs can nevertheless still offer valuable programmatic insights on interventions to improve quality of malaria CCM, provided it is possible to attribute the outcome of interest directly to the CHW, namely data from CHW registers or household surveys where information collected on treatment by source. 6.7 Missing/ unreported information There is a need for objective indicators of health system capacity to support CCM. As suggested by Fapohunda et al. in their evaluation of home-based management of malaria in Uganda [47], these should include objective measures of (i) the capacity for health workers to treat referred cases effectively, meaning some measure of knowledge (or ideally practice) of the management of severe malaria and IMCI; (ii) consistent and sufficient supply of essential drugs and equipment, including RDTs, at the health facility level; (iii) frequency of supportive supervision and opportunity for problem solving; and (iv) use of CHW records in monitoring practice and informing commodity supply. As previously discussed, only one study in Uganda collected and reported such indicators [47]. Likewise, although supportive supervision could be expected to improve quality, and the frequency of supervision was reported for a number of studies, attributing any impact on CHW performance to this amongst other elements of the intervention is challenging and would require a carefully designed operational study. Only one of the 42 studies included in this review reported an outcome measure stratified by wealth quintile as an indication of socioeconomic equity of the CHW intervention [52]. None of the studies reported quantitatively on whether CHW programmes improved geographic access by bringing malaria treatment closer to people s homes as intended, although some of the qualitative findings suggest that this was the case. In part, this low level of reporting may be because socioeconomic and geographic equity are intrinsic to the design of malaria CCM programmes. Nonetheless, it would be useful to include quantitative indicators in future evaluations to assess whether this basic objective of improving equity of access to malaria diagnosis and treatment is being achieved through community-based delivery. 62

72 Another area of limited evidence is the cost effectiveness of malaria CCM. Seven of the included studies reported a costing element [39, 55, 56, 59, 60], with variations in purpose and methodology of data collection. These elements range from cost per CHW of training in the use of RDTs in Zambia [59] to district level costs of implementation of five interventions through a community directed approach compared to delivery through usual channels [39]. Studies in Uganda and Zambia also compared costs of community-based treatment of malaria to the costs of standard care at a health facility, although comparison of results across these studies is difficult due to different outcome indicators (cost per child per year versus cost per malaria episode treated, respectively) and costing perspective (societal versus provider, respectively) [55, 56]. One study in DRC explored the reduced cost efficiency of using RDTs for confirmed malaria diagnosis at high parasite prevalence levels [60]. Although malaria CCM appears to be a less costly approach compared to standard care for malaria treatment at a health facility, the variety of costing methods means it is not possible to present a clear picture of the cost effectiveness of the approach. In particular, data is needed on indirect opportunity costs for CHWs volunteering their time. More economic evaluations of malaria CCM are needed, and standardisation of methodology to allow cross-study comparison will be extremely valuable. There is still relatively little evidence that CHWs can positively impact mortality or morbidity, although there is reasonable evidence they can perform well with adequate supervision and health system support. Health impact studies (e.g. [41, 50]) rarely reported process or outcome indicators that would help understand the health outcomes and relate these to the strength of implementation; similarly, most of the programme evaluations focused on process and outcome indicators but did not report health outcomes (and were not designed to do so). This suggests an area for further investigation, namely for studies that measure impact of malaria CCM on morbidity or mortality to also include process evaluation in their design so that inference can be drawn on the level of implementation of malaria CCM needed to achieve health impact. Such evidence is particularly needed from countries implementing ICCM as there is currently a shortage of data on such programmes, despite their relatively widespread and large-scale implementation. According to the 2011 World Malaria Report, RDT and ACT use at the community level is approved national policy in 20 and 24 of 43 endemic African countries, respectively [5]. Despite a large number of countries adopting a national policy of malaria CCM, there remains a paucity of data on studies conducted at large scale; the majority of studies included in the review evaluated interventions implemented at the sub-district or district level, and only one study evaluated a regional level programme and one a national programme. Evidence from larger scale implementation is needed to add to the evidence base on effectiveness of malaria CCM. 7. Conclusions and recommendations The findings from this review support the potential for CHWs to implement quality care at the community level, even with increasing complexity of their roles and responsibilities. Important facilitating factors for success include interactive and practical training, and regular supportive supervision. However, there is an urgent need for more evidence on the most appropriate course of action for RDT-negative patients and the effective referral by CHWs of cases that they are unable to treat; that few studies report on this is a cause for concern, given the crucial role that referral must play in ensuring patient safety when responsibility for case management is transferred to low-level community providers. 63

73 Evidence from this review also highlights that for any CCM programme to succeed and be sustained there must be strong support from the formal health system, in terms of ability to manage referred cases effectively, to provide regular supportive supervision, and to ensure consistent and adequate supply of commodities. Often this capacity is lacking, and more evidence is needed on effective interventions to improve health system capacity in these areas in the context of community case management. Further evidence is required on the integration of malaria CCM with other health services, particularly from programmes following the ICCM approach. Results of evaluations should reflect the equity aims of CCM by reporting key outcomes stratified by wealth quintile and distance from source of treatment wherever possible. Similarly, more evidence is needed on the cost effectiveness of malaria CCM compared to standard care. The limited evidence from the few studies that met the inclusion criteria for investigating communitylevel demand generation provides further support for multi-faceted interventions; namely that for userside demand generation interventions to be successful, they must be complemented by interventions that also strengthen service provision. 64

74 References 1. RBM, Global strategic plan: Roll back malaria , Roll Back Malaria Partnership: Geneva. 2. WHO, Guidleines for the treatment of malaria. 2010, World Health Organisation: Geneva. 3. Drakeley, C. and H. Reyburn, Out with the old, in with the new: the utility of rapid diagnostic tests for malaria diagnosis in Africa. Trans R Soc Trop Med Hyg, (4): p Perkins, M.D. and D.R. Bell, Working without a blindfold: the critical role of diagnostics in malaria control. Malar J, Suppl 1: p. S5. 5. WHO, World Malaria Report , World Health Organisation: Geneva. 6. WHO, World Malaria Report , World Health Organisation: Geneva. 7. McCombie, S.C., Treatment seeking for malaria: a review of recent research. Soc Sci Med, (6): p Schellenberg, J.A., et al., Inequities among the very poor: health care for children in rural southern Tanzania. Lancet, (9357): p Barat, L.M., et al., Do malaria control interventions reach the poor? A view through the equity lens. Am J Trop Med Hyg, (2 Suppl): p Onwujekwe, O., Inequities in healthcare seeking in the treatment of communicable endemic diseases in Southeast Nigeria. Soc Sci Med, (2): p Muller, I., et al., The effect of distance from home on attendance at a small rural health centre in Papua New Guinea. Int J Epidemiol, (5): p Foster, S., Treatment of malaria outside the formal health services. J Trop Med Hyg, (1): p Yeung, S. and N.J. White, How do patients use antimalarial drugs? A review of the evidence. Trop Med Int Health, (2): p WHO, The Roll Back malaria strategy for improving access to treatment through home management for malaria. 2005, World Health Organisation: Geneva. 15. Williams, H.A. and C.O. Jones, A critical review of behavioral issues related to malaria control in sub-saharan Africa: what contributions have social scientists made? Soc Sci Med, (3): p McCombie, S.C., Self-treatment for malaria: the evidence and methodological issues. Health Policy Plan, (4): p Kizito, J., et al., Improving access to health care for malaria in Africa: a review of literature on what attracts patients. Malar J, (1): p Smith, L.A., et al., Review: Provider practice and user behavior interventions to improve prompt and effective treatment of malaria: do we know what works? Am J Trop Med Hyg, (3): p Wafula, F.N. and C.A. Goodman, Are interventions for improving the quality of services provided by specialized drug shops effective in sub-saharan Africa? A systematic review of the literature. Int J Qual Health Care, (4): p Hopkins, H., et al., Impact of home-based management of malaria on health outcomes in Africa: a systematic review of the evidence. Malar J, : p Lagarde, M. and N. Palmer, The impact of user fees on access to health services in low- and middle-income countries. Cochrane Database Syst Rev, 2011(4): p. CD

75 22. Lagarde, M., A. Haines, and N. Palmer, The impact of conditional cash transfers on health outcomes and use of health services in low and middle income countries. Cochrane Database Syst Rev, 2009(4): p. CD Haines, A., et al., Achieving child survival goals: potential contribution of community health workers. Lancet, (9579): p Lewin, S., et al., Lay health workers in primary and community health care for maternal and child health and the management of infectious diseases. Cochrane Database Syst Rev, 2010(3): p. CD Christopher, J.B., et al., Thirty years after Alma-Ata: a systematic review of the impact of community health workers delivering curative interventions against malaria, pneumonia and diarrhoea on child mortality and morbidity in sub-saharan Africa. Hum Resour Health, (1): p ELDIS. Eldis: Sharing the best in development policy, practice & research. Available from: ie. 3ie: International Initiative for Impact Evaluation. Available from: WHOLIS, World Health Organisation Library and Information Networks for Knowledge. 29. TDR. Special Programme for Research and Training in Tropical Diseases (TDR). Available from: World Bank, The World Bank: Publications & Documents. 31. DfID. Department for International Development. Available from: USAID. United States Agency for International Development (USAID): Knowledge Services Center. Available from: MSH. Managemente Sciences for Health: Resource Center. Available from: HRH. Human Resources for Health: Global Resource Center. Available from: CHW Central. USAID: Health Care Improvement Project. Available from: Cochrane EPOC. Cochrane Effective Practice and Organisation of Care (EPOC) Group: Resources for review authors. 2010; Available from: Ajayi, I.O., et al., Effectiveness of artemisinin-based combination therapy used in the context of home management of malaria: a report from three study sites in sub-saharan Africa. Malaria Journal, (190): p. (27 September 2008). 38. Akweongo, P., et al., Feasibility and acceptability of ACT for the community case management of malaria in urban settings in five African sites. Malar J, (1): p CDI Study Group, Community-directed interventions for major health problems in Africa: A multicountry study: Final report. 2008, TDR: Geneva. 40. Ishengoma, D.S., et al., Accuracy of malaria rapid diagnostic tests in community studies and their impact on treatment of malaria in an area with declining malaria burden in north-eastern Tanzania. Malar J, : p Lemma, H., et al., Deploying artemether-lumefantrine with rapid testing in Ethiopian communities: impact on malaria morbidity, mortality and healthcare resources. Tropical Medicine & International Health, (2): p

76 42. Alba, S., et al., Improvements in access to malaria treatment in Tanzania following community, retail sector and health facility interventions -- a user perspective. Malaria Journal, (pp 163). 43. Ajayi, I.O., et al., Feasibility and acceptability of artemisinin-based combination therapy for the home management of malaria in four African sites. Malaria Journal, (6): p. (8 January 2008). 44. Ajayi, I.O., et al., A qualitative study of the feasibility and community perception on the effectiveness of artemether-lumefantrine use in the context of home management of malaria in south-west Nigeria. BMC Health Services Research, (119): p. (01 June 2008). 45. Chinbuah, A.M., et al., Feasibility and acceptability of the use of artemether-lumefantrine in the home management of uncomplicated malaria in children 6-59 months old in Ghana. Tropical Medicine and International Health, (7): p Eriksen, J., et al., Effectiveness of a community intervention on malaria in rural Tanzania - a randomised controlled trial. African Health Sciences, (4): p Fapohunda, B., et al., Home-Based Management of Fever Strategy in Uganda: A Report of the 2003 Survey. 2004, MOH, WHO and BASICS II: Arlington, Virginia, USA. 48. Franco, C., et al., Evaluation of the Home Based Management of Malaria Strategy in Rwanda: , BASICS, SPS, USAID: Arlington, Virginia, USA. 49. Kallander, K., et al., Community referral in home management of malaria in western Uganda: a case series study. BMC International Health and Human Rights, (2): p. (16 March 2006). 50. Kidane, G. and R.H. Morrow, Teaching mothers to provide home treatment of malaria in Tigray, Ethiopia: a randomised trial. Lancet (British edition), (9229): p Kouyate, B., et al., Process and effects of a community intervention on malaria in rural Burkina Faso: randomized controlled trial. Malaria Journal, (50): p. (25 march 2008). 52. Nsungwa-Sabiiti, J., et al., Home-based management of fever and malaria treatment practices in Uganda. Transactions of the Royal Society of Tropical Medicine and Hygiene, (12): p Nsabagasani, X., et al., Home-based management of fever in rural Uganda: community perceptions and provider opinions. Malaria Journal, (11): p. (26 January 2007). 54. Sirima, S.B., et al., Early treatment of childhood fevers with pre-packaged antimalarial drugs in the home reduces severe malaria morbidity in Burkina Faso. Tropical Medicine and International Health, (2): p Staedke, S.G., et al., Home management of malaria with artemether-lumefantrine compared with standard care in urban Ugandan children: a randomised controlled trial. Lancet, (9675): p Chanda, P., et al., Relative costs and effectiveness of treating uncomplicated malaria in two rural districts in Zambia: implications for nationwide scale-up of home-based management. Malar J, : p Chanda, P., et al., Community case management of malaria using ACT and RDT in two districts in Zambia: achieving high adherence to test results using community health workers. Malar J, : p Elmardi, K.A., et al., Feasibility and acceptability of home-based management of malaria strategy adapted to Sudan's conditions using artemisinin-based combination therapy and rapid diagnostic test. Malaria Journal, (39): p. (9 March 2009). 59. Harvey, S.A., et al., Improving community health worker use of malaria rapid diagnostic tests in Zambia: package instructions, job aid and job aid-plus-training. Malaria Journal, (160): p. (22 August 2008). 67

77 60. Hawkes, M., J.P. Katsuva, and C.K. Masumbuko, Use and limitations of malaria rapid diagnostic testing by community health workers in war-torn Democratic Republic of Congo. Malaria Journal, (308): p. (23 December 2009). 61. Mubi, M., et al., Malaria rapid testing by community health workers is effective and safe for targeting malaria treatment: randomised cross-over trial in Tanzania. PLoS One, (7): p. e Ngasala, B.E., et al., Effectiveness of artemether-lumefantrine provided by community health workers in under-five children with uncomplicated malaria in rural Tanzania: an open label prospective study. Malaria Journal, (64): p. (16 March 2011). 63. Thomson, A., et al., Low referral completion of rapid diagnostic test-negative patients in community-based treatment of malaria in Sierra Leone. Malaria Journal, (94): p. (17 April 2011). 64. Ndyomugyenyi, R. and A.T. Kabali, Community-directed interventions for integrated delivery of a health package against major health problems in rural Uganda: perceptions on the strategy and its effectiveness. International Health (RSTMH), (3): p Kelly, J.M., et al., Community health worker performance in the management of multiple childhood illnesses: Siaya District, Kenya, American Journal of Public Health, (10): p Rowe, S.Y., et al., Effect of multiple interventions on community health workers' adherence to clinical guidelines in Siaya district, Kenya. Transactions of the Royal Society of Tropical Medicine and Hygiene, (2): p Mukanga, D., et al., Can lay community health workers be trained to use diagnostics to distinguish and treat malaria and pneumonia in children? Lessons from rural Uganda. Trop Med Int Health, Rwanda MOH, Community case management: Evaluation report of community health workers' performance. 2009, Ministry of Health: Kigali, Rwanda. 69. Winch, P.J., et al., Increases in correct administration of chloroquine in the home and referral of sick children to health facilities through a community-based intervention in Bougouni District, Mali. Transactions of the Royal Society of Tropical Medicine and Hygiene, (5): p Yeboah-Antwi, K., et al., Community case management of fever due to malaria and pneumonia in children under five in Zambia: a cluster randomized controlled trial. PLoS Medicine, (9): p. e Ajayi, I.O., et al., Assessment of a treatment guideline to improve home management of malaria in children in rural south-west Nigeria. Malaria Journal, (24). 72. Hetzel, M.W., et al., Understanding and improving access to prompt and effective malaria treatment and care in rural Tanzania: the ACCESS Programme. Malaria Journal, (83): p. (29 June 2007). 73. Chirdan, O.O., A.I. Zoakah, and C.L. Ejembi, Impact of health education on home treatment and prevention of malaria in Jengre, North Central Nigeria. Annals of African Medicine, (3): p De La Cruz, N., et al., Microfinance against malaria: impact of Freedom from Hunger's malaria education when delivered by rural banks in Ghana. Transactions of the Royal Society of Tropical Medicine & Hygiene, (12): p Nkuo Akenji, T.K., et al., Evaluating a malaria intervention strategy using knowledge, practices and coverage surveys in rural Bolifamba, southwest Cameroon. Transactions of the Royal Society of Tropical Medicine and Hygiene, (5): p

78 76. Edward, A., et al., Examining the evidence of under-five mortality reduction in a communitybased programme in Gaza, Mozambique. Transactions of the Royal Society of Tropical Medicine and Hygiene, (8): p Ansah, E.K., et al., Improving adherence to malaria treatment for children: the use of pre-packed chloroquine tablets vs. chloroquine syrup. Trop Med Int Health, (7): p Orton, L. and G. Barnish, Unit-dose packaged drugs for treating malaria. Cochrane Database Syst Rev, 2005(2): p. CD Chandler, C.I., et al., Guidelines and mindlines: why do clinical staff over-diagnose malaria in Tanzania? A qualitative study. Malar J, : p Hamer, D.H., et al., Improved diagnostic testing and malaria treatment practices in Zambia. JAMA, (20): p WHO, Everybody's business: Strengthening health systems to improve health outcomes: WHO s framework for action. 2007, World Health Organisation: Geneva. 82. Smith Paintain, L., et al., A systematic review of strategies to increase demand, uptake and quality of commuity-based diagnosis and case management of malaria. 2012, UNICEF Working Paper. 83. Habicht, J.P., C.G. Victora, and J.P. Vaughan, Evaluation designs for adequacy, plausibility and probability of public health programme performance and impact. Int J Epidemiol, (1): p Webster, J., D. Chandramohan, and K. Hanson, Methods for evaluating delivery systems for scaling-up malaria control intervention. BMC Health Serv Res, Suppl 1: p. S8. 69

79 Appendix 1: MEDLINE search strategy Sub-search 1: to identify interventions to improve community case management of malaria including quality, health system support, referrals and integration with other health interventions (objectives 1-4). 1. malaria.mp. or exp Malaria/ 2. fever.mp. or exp Fever/ 3. 1 or 2 4. community.mp. 5. case and management).mp or treat*.mp 6. (home and management and malaria).mp. 7. (home and management and fever).mp. 8. (integrat* and case and management).mp. 9. referral*.mp or 6 or 7 or 8 or and 4 and limit 11 to (humans and yr="2000 -Current") Sub-search 2: to identify community level interventions to improve prompt and effective care seeking for treatment of malaria from any source (objective 5). 1. malaria.mp. or exp Malaria/ 2. fever.mp. or exp Fever/ 3. 1 or 2 4. (community or household or home).mp. 5. prompt.mp. 6. treatment-seeking.mp. 7. care-seeking.mp. 8. access.mp or 6 or 7 or exp epidemiologic study characteristics as topic/ or exp intervention studies/ or exp epidemiologic research design/ 11. intervention*.mp. 12. (mobili* or sensiti* or educat* or promot*).mp or 11 or and 4 and 9 and limit 14 to (humans and yr="2000 -Current") 70

80 Appendix 2: Sub-Saharan African countries eligible for inclusion 1. Angola 2. Benin 3. Botswana 4. Burkina Faso 5. Burundi 6. Cameroon 7. Cape Verde 8. Central African Republic 9. Chad 10. Comoros 11. Congo 12. Cote d Ivoire 13. Democratic Republic of Congo 14. Djibouti 15. Equatorial Guinea 16. Eritrea 17. Ethiopia 18. Gabon 19. The Gambia 20. Ghana 21. Guinea 22. Guinea Bissau 23. Kenya 24. Liberia 25. Madagascar 26. Malawi 27. Mali 28. Mauritania 29. Mozambique 30. Namibia 31. Niger 32. Nigeria 33. Rwanda 34. Sao Tome & Principe 35. Senegal 36. Sierra Leone 37. Somalia 38. South Africa 39. North Sudan 40. South Sudan 41. Swaziland 42. The United Republic of Tanzania 43. Togo 44. Uganda 45. Zambia 46. Zanzibar 47. Zimbabwe 71

81 72