The promise of RBF to reach the Health MDGs and the Evidence Gap: How Impact Evaluation Can Inform Policy Dialogue

Transcription

1 The promise of RBF to reach the Health MDGs and the Evidence Gap: How Impact Evaluation Can Inform Policy Dialogue African Development Bank Agnes Soucat, MD, Ph.D Director Department of Human Development 1

2 Progress towards the health MDGs is inadequate 25 Number of children dying before the age of Africa Asia Other with achievement of with current 2 Trend

3 Percent Treated fever among children under 5 by wealth quintile Lowest Second Middle Fourth Highest Wealth Quintile Source: Demographic and Health Surveys 3

4 Most mortality causes still avoidable with low cost interventions Cause-specific proportional mortality in the Africa region Neonatal HIV/AIDS 2% 6% 21% Diarrhoeal diseases 21% Measles 7% Malaria Respiratory infections 17% 17% Injuries 4% Others 4

5 Billions of US$ The Health MDGs are within reach. Progress towards MDGs, Marginal Cost, and Fiscal Space 100.0% 90.0% 80.0% 70.0% 60.0% 50.0% $151.0 $89.0 $160.0 $140.0 $120.0 $100.0 $ % 30.0% 20.0% 10.0% $48.0 $60.0 $40.0 $ % $- Comprehensive Medium Focussed MDG 1 MDG 4 MDG 5 MDG 6 MDG7 Source : high Level Task Force on Innovative Financing,

6 Percent of total health expenditure And money is here Africa is growing at 5% a year and external aid is an important source of health spending in Sub-Saharan Africa External aid as % of total health spending (2002) East Asia & Pacific Eastern Europe & Central Asia Latin America & the Caribbean Middle East & North Africa South Asia Sub-Saharan Africa Region 6

7 But more money does not always give better results But increasing public spending is also not enough * Percent deviation from rate predicted by GDP per capita Source: Spending and GDP from World Development Indicators database. Under-5 mortality from Unicef 72002

8 Countries use well-designed policies to achieve growth and human development outcomes Health, Education, Poverty But * 8

9 But, what looks good on paper seems to break down in practice Leakage of Funds Bad policy Poor budget handling Primary education Low quality services Provider incentives unclear, absenteeism Hard to monitor, users helpless Sub-optimal spending (Big salary bills but insufficient commodities) Financing problems Information & monitoring Local govt. incentives skewed Local capacity issues Lack of demand Externalities Community norms Budget constraints Intra-household behavior 9

10 Public Money Benefits Richer Groups Expenditure incidence Health Source: Filmer 2003b 10

11 Public Money Does Not Reach the Frontline Nonwage funds not reaching health services: Evidence from PETS (%) Health Country Mean Chad Senegal Cameroon Rwanda Source: World Bank 11

12 And the Results Chain to achieving the MDGs has missing links Health Systems Outcomes Upstream diagnostic Country Status Report on Health, Poverty, Health Systems and the MDGs Policy dialogue One country, one costed plan for country supported by all partners (IHP+) Financing Programs Focused funds:,malaria HIV/AIDS, Tuberculosis etc. Problem specific and health system focused operations (eg: SWAPs, PBS, PRSC) Joint Bank-Partners Programs (eg: Resultsbased financing) Health Systems Health Financing Human Resources For Health Supply Chain and Pharmaceuticals Governance and Service Delivery Infrastructure and ICT Immunization, ORT ARI treatment, Exclusive Breastfeeding, Vitamin A supplementation, ITNs, Antimalarial treatment, Attended Deliveries, Antenatal Care, Contraception, etc Multisectoral Outcomes Female Education Access to water Income School Health Health Outcomes Nutrition (MDG1) Child health (MDG4) Maternal Health and mortality (MDG5) Malaria,AIDS, TB, prevalence and mortality (MDG6) Financial protection Control of Non Communicable Diseases Control of Neglected Tropical Diseases 12

13 the missing middle problem. Financing Health systems Health systems Outcomes MDGs Focused funds:,malaria HIV/AIDS, Tuberculosis etc. Problem specific and health system focused operations (eg: SWAPs, PBS, PRSC) Joint Bank- Partners Programs (eg: Results-based financing) Human Resources For Health Supply chain Governance and Service Delivery Public finance and health insurance Infrastructure and ICT Other outcomes Education Water Social Protection Immunization ORT ARI treatment Exclusive breastfeeding Vitamin A supplementation ITNs Antimalarial treatment Attended Deliveries Antenatal Care Contraception etc Other sectors Outcomes Access to safe water Female education Income Nutrition (MDG1) Child health (MDG4) Maternal Health and mortality (MDG5) Malaria,AIDS, TB, prevalence and mortality (MDG6) Financial protection Non Communicable Diseases 13

14 What do we know and don t know We know that high impact interventions lead to improved health outcomes.. We do not know how to best produce demand and effective coverage with these interventions in specific contexts.what economists call the production function We know the inputs.. But not the incentives, institutional and management arrangements The missing middle or black box.. 14

15 When we do not know there are consequences Government s policies are shooting in the dark Public money is not used efficiently.. Aid effectiveness is low.. Ideology prevails.. 15

16 An example of an ideology driven debate user fees Proponents: user fees provide incentives to providers, reduce absenteeism, improve quality, improve accountability to users/clients Opponents: user fees reduce access and utilization, and are a burden on the poor s households budgets Evidence? 16

17 An example of an ideology driven debate user fees Hundreds of publications on the topic.. Yet less than 5 rigorous studies based on population surveys of service use and randomized control group Conclusions of these studies are mixed.. 17

18 Uganda: Utilization of Curative Services Trending Upwards Utilization rates in Uganda, User fees eliminated Overall Private Government NGO Year Source: UBOS (various years) The World Bank Public Expenditure Review 2008 The World 18 Bank

19 % Uganda: little effect of user fees abolition on MDGs related High Impact Interventions % delivered in a health facility TOTAL DPT3 (%) Currently Using any modern FP method (%) % U5 who slept under an ITN the past night Source: DHS 2000 and 19

20 The promise of Results Based Financing.. Setting Evidence Based Policies : The Rwanda example 20

21 U5MR per 1,000 Rwanda is back on track to reach the MDGs Under five mortality trends with MDG target for level 200 MDG target for Observed Trends since Trends required to reach the 2015 target 21

22 Increase in utilization of high impact services Trends in assistance at delivery : Years 2000, 2005, 2007 Percentage (%) of women delivered by a health professional 22

23 Average number per month Rwanda : Intake of Family planning tripled in three years.. Family Planning, Modern Methods, Users at the End of the Month Average Per Health Center per Month R 2 =

24 Performance-based Financing (PBF) Developed after extensive piloting from Objectives Focus on maternal and child health as well as communicable diseases (MDGs 4 & 5) Increase quantity and quality of health services provided Increase health worker motivation Financial incentives to providers to see more patients and provide higher quality of care Operates through contracts between Government Public and Private Health facilities providing services 24 24

25 Table 1: Output Indicators (U s) and Unit Payments for PBF Formula OUTPUT INDICATORS Amount paid per unit (US$) Visit Indicators: Number of 1 curative care visits first prenatal care visits women who completed 4 prenatal care visits first time family planning visits (new contraceptive users) contraceptive resupply visits deliveries in the facility child (0-59 months) preventive care visits 0.18 Content of care indicators: Number of 8 women who received tetanus vaccine during prenatal care women who received malaria prophylaxis during prenatal care 10 at risk pregnancies referred to hospital for delivery emergency transfers to hospital for obstetric care children who completed vaccinations (child preventive care) malnourished children referred for treatment other emergency referrals

26 Evaluation Questions: Did PBF Increase the quantity of contracted health services delivered? Improve the quality of contracted health services provided? 26

27 Researcher & Policy Maker Collaboration Research Team Paulin Basinga, National University of Rwanda Paul Gertler, UC Berkeley Jennifer Sturdy, World Bank and UC Berkeley Christel Vermeersch, World Bank Policy Counterpart Team A collaboration between the Rwanda Ministry of Health, CNLS, and SPH, the INSP in Mexico, UC Berkeley and the World Bank Agnes Binagwaho, Rwanda MOH and CNLS Louis Rusa, Rwanda Rwanda MOH Claude Sekabaraga, Rwanda MOH Agnes Soucat, World Bank 27

28 Acknowledgments Funding by: World Bank Government of Rwanda (PHRD grant) Bank-Netherlands Partnership Program (BNPP) ESRC/DFID GDN 28

29 Evaluation Design During decentralization, phased rollout at district level Identified districts without PBF in 2005 Group districts into similar pairs based on population density & livelihoods Randomly assign one to treatment and other to control MOH reallocated some districts to treatment With decentralization, some new districts had PBF in an area of the new district must be treatment 29 29

30 Isolating the incentive effect PBF Performance incentives Additional resources Compensate control facilities with equal resources Average of what treatments receive Not linked to performance Money allocated by the health center management 30

31 Sample: Panel 165 Facilities households in catchment areas Random sample of 14 per clinic 31

32 Log Expenditures Year Treatment Control Difference P-Value (1.042) (1.007) (0.71) (1.08) (0.14) Randomization balanced baseline Follow-up balanced, so difference in followup outcomes due to incentives not resources 32

33 A dramatic increase of utilization of services in Rwanda Rates of Assisted deliveries increased in both treatment and control groups P4P Begins treatment facilities 33

34 0.7 Impact on 4+ Prenatal Visits and Facility Delivery No PBF 2008 PBF prenatal vists Facility Delivery 34

35 Proportion of of institutional deliveries Delivery at the health facility increased overall in Rwanda, but 7% more in PBF facilities % increase due to PBF Baseline (2006) Follow up (2008) Control facilities Treatment (PBF facilities) 35 35

36 0.5 Impact of PBF on Probability of Child Preventive Care Visit in Last 4 Weeks NO PBF 2008 PBF Months Months 36

37 37

38 Quality Conceptual Framework What They Do: (Quality) Production Possibility Frontier What They Know (Ability/Technology) 38

39 Goal: Use Pay for Performance to Close Productivity Gap Production Possibility Frontier What They Do Productivity Gap Conditional on Ability Actual Performance Ability/Technology 39

40 Prenatal Competency & Quality Provider knowledge/competency Standardized vignette presented to provider Compare answers to Rwandan CPG Measure of ability/knowledge Process quality Patient exit interview of clinical services provided Clinical content of care Provider effort 40

41 Standardized Prenatal effort score In the last years, PBF has increased prenatal care quality significantly Baseline (2006) Follow up (2008) 15 % Standard deviation increase due to PBF Control facilities Treatment (PBF facilities) 41 41

42 Kernel Non parametric regression practice-competency at baseline Control facilities Treatment facilities Competence 42

43 Kernel Non parametric regression practice-competency at follow up Control facilities Treatment facilities Competence 43

44 Results Summary Balanced at baseline Expenditures same, so isolate incentives Impact on utilization Delivery & Child prevention, but not prenatal Impact on prenatal quality Bigger for better doctors Reduced child morbidity & Taller children Effect sizes bigger than most other interventions 44

45 Discussion Rwanda is back on track towards the health MDGs because of many different factors including Strong political leadership Micro-Insurance (Mutuelles) Autonomy and Fiscal Decentralization (Imihigo) HIV services and earmarked funding PBF effect seen despite many other national level intervention: possible bigger effect in other countries 45 45

46 Average number per month Beware the pre-post trap : the example of Family Planning in Rwanda Family Planning, Modern Methods, Users at the End of the Month Average Per Health Center per Month R 2 =

47 Conclusions Reaching the MDGs or at least making a major dent towards reaching them- is possible even within the next five years.. Results Based financing can be a powerful way to address the problems of the missing middle and inject incentives into the implementation black box Only because of the rigorous impact evaluation conducted in Rwanda can we conclude that RBF played a role in the increased utilization of services Impact E valuation should systematically be nested into major policy interventions 47