Monitoring and Evaluation in the Global HIV/AIDS Program. Joseph Valadez GHAP/GAMET The World Bank February 24, 2005

Transcription

1 Monitoring and Evaluation in the Global HIV/AIDS Program Joseph Valadez GHAP/GAMET The World Bank February 24, 2005

2 Approved and Active HIV/AIDS Projects and Disbursements by Region as of November 7, 2004 Total (Net) Comm. USD Estimated AIDS Comm. USD Total Disbursed USD Estimated AIDS Disb'd USD AFR 1, , EAP ECA LAC MNA SA TOTAL 3, , ,

3 Approved and Active Projects and Disbursements in Africa as of November 7, 2004 Country Project Title FY Approved Type Total (Net) Comm. USD Estimated AIDS Comm. USD Total Disbursed USD % Disbursed Estimated AIDS Disb'd USD Cote d'ivoire Integrated Health Services FY96 IDA Guinea-Bissau National Health Devl. FY98 IDA Eritrea National Health Devl. FY98 IDA Malawi Secondary Education FY98 IDA The Gambia Particip. Health, Pop., and Nut. FY98 IDA Ethiopia Health Sector Devlp FY99 IDA Chad Health System Support FY00 IDA Nigeria Second Primary Education FY00 IDA Rwanda Human Resources Develp. FY00 IDA Lesotho Health Sector Reform FY00 IDA Kenya Decentralized RH & AIDS (DARE) FY01 IDA Ethiopia Multisectoral HIV/AIDS FY01 IDA Kenya HIV/AIDS Project FY01 IDA Eritrea HAMSET (HIV/Malaria/TB/STI) FY01 IDA Ghana AIDS Response Project FY01 IDA Gambia HIV/AIDS Rapid Response FY01 IDA Uganda HIV/AIDS Control FY01 IDA Cameroon Multisectoral HIV/AIDS FY01 IDA Burkina Faso HIV/AIDS Disaster Relief FY02 IDA Nigeria HIV/AIDS Response FY02 IDA Madagascar Multisectoral STI/HIV/AIDS FY02 IDA CAR HIV/AIDS FY02 IDA Benin HIV/AIDS Multisectoral FY02 IDA Senegal HIV/AIDS Prevention FY02 IDA Sierra Leone HIV/AIDS Response FY02 IDA Cape Verde HIV/AIDS FY02 IDA Burundi HIV/AIDS and Orphans FY02 IDA Nigeria Second Health Sys. Devlp FY02 IDA Chad Second Pop. & AIDS FY02 IDA Guinea Multisectoral AIDS FY03 IDA Grant Zambia HIV/AIDS (ZANARA) FY03 IDA Grant

4 Approved and Active Projects and Disbursements in Africa as of November 7, 2004 (cont d.) Country Project Title FY Approved Type Total (Net) Comm. USD Estimated AIDS Comm. USD Total Disbursed USD % Disbursed Estimated AIDS Disb'd USD Mozambique HIV/AIDS Response FY03 IDA Grant Rwanda Multisectoral HIV/AIDS FY03 IDA Grant Niger Support for Multisector HIV/AIDS Program FY03 IDA Grant Mauritania Multisector HIV/AIDS & Endemic Disease FY04 IDA Grant Tanzania Multisectoral AIDS FY04 IDA Grant Malawi Multisector AIDS FY04 IDA Grant Western Africa Abidjan-Lagos Transport Cooridor FY04 IDA Grant Congo (DRC) Multisectoral HIV/AIDS FY04 IDA Grant Congo, Rep.of HIV/AIDS and Health FY04 IDA Grant not effective N/A 0.00 Guinea Bissau AFR Sub- Regional HIV/AIDS Global Mitigation Support Project Regional HIV/AIDS Treatment Acceleration Project FY04 IDA Grant not effective N/A 0.00 FY04 IDA Grant not effective N/A 0.00 Mali Multisectoral HIV/AIDS Project FY04 IDA Grant not effective N/A 0.00 Lesotho HIV/AIDS Capacity Bldg & TA FY05 IDA Grant TOTAL

5 Approved and Active Projects and Disbursements in East Asia and the Pacific as of November 7, 2004 Country Project Title FY Approved Type Total (Net) Comm. USD Estimated AIDS Comm. USD Total Disbursed USD % Disbursed Estimated AIDS Disb'd USD China Health IX FY99 IBRD Vietnam Reg. Blood Transfusion Centers FY02 IDA Cambodia Health Sector Support FY03 IDA Credit & Grant TOTAL

6 Approved and Active Projects and Disbursements in Europe and Central Asia as of November 7, 2004 Country Project Title FY Approved Type Total (Net) Comm. USD Estimated AIDS Comm. USD Total Disbursed USD % Disbursed Estimated AIDS Disb'd USD Bulgaria Health Sector Reform FY00 WB Ukraine TB & HIV/AIDS Control FY03 IDA/IBRD Russian Fed. TB & HIV/AIDS Control FY03 IDA/IBRD Moldova AIDS Control Project FY03 IDA Grant TOTAL

7 Approved and Active Projects and Disbursements in Latin America and the Caribbean as of November 7, 2004 Country Project Title FY Approved Type Total (Net) Comm. USD Estimated AIDS Comm. USD Total Disbursed USD % Disbursed Estimated AIDS Disb'd USD Mexico Mexico Basic Health FY01 IBRD Venezuela Caracas Metro. Health Services FY01 IBRD Barbados Multi-Country HIV/AIDS Prevention & Control FY01 IBRD Dominican Multi-Country HIV/AIDS Prevention Republic & Control FY01 IBRD Jamaica HIV/AIDS Prevention & Control FY02 IBRD Honduras Health System Reform FY02 IDA Grenada HIV/AIDS Prevention & Control FY03 IDA/IBRD St. Kitts & Nevis HIV/AIDS Prevention & Control FY03 IBRD Trinidad & Tobago HIV/AIDS Prevention & Control FY03 IBRD Brazil AIDS/STD Control (3) FY03 IBRD Pan-Caribbean Pan Caribbean AIDS FY04 IDA Grant not effective N/A 0.00 Guyana HIV/AIDS Prevention & Control FY04 IDA Grant not effective N/A 0.00 St. Vincent & IDA Grant, HIV/AIDS Prevention & Control FY05 Genadines IDA/IBRD not effective N/A 0.00 St. Lucia HIV/AIDS Prevention & Control FY05 IDA Grant, IDA/IBRD not effective N/A 0.00 TOTAL

8 Approved and Active Projects and Disbursements in the Middle East and North Africa as of November 7, 2004 Country Project Title FY Approved Type Total (Net) Comm. USD Estimated AIDS Comm. USD Total Disbursed USD % Disbursed Estimated AIDS Disb'd USD Djibouti HIV/AIDS, Malaria & TB Control FY03 IDA Grant TOTAL

9 Approved and Active Projects and Disbursements in South Asia as of November 7, 2004 Country Project Title FY Approved Type Total (Net) Comm. USD Estimated AIDS Comm. USD Total Disbursed USD % Disbursed Estimated AIDS Disb'd USD Bangladesh Health and Population FY98 IDA India AIDS Prevention II FY99 IDA India Maharashtra Health Sys. Devp't FY99 IDA India UP Health Sys. Devlp FY00 IDA Bangladesh HIV/AIDS Prevention FY01 IDA Sri Lanka National AIDS Program FY03 IDA Grant Pakistan HIV/AIDS Prevention FY03 IDA Credit and Grant Bhutan HIV/AIDS & STI Prevention and Control Project FY04 IDA Grant TOTAL

10 Proposed HIV/AIDS Projects FY05-FY08 Country Project Title FY Approved Product Line Total IDA Funding US$ Total AIDS Funding US$ Africa Great Lakes Initiative on HIV/AIDS FY05 IBRD/IDA Central Asia Central Asia AIDS Control Project FY05 IBRD/IDA Uzbekistan Health II FY05 IBRD/IDA Vietnam HIV/AIDS Prevention FY05 IBRD/IDA PNG Highlands Highway Rehabilitation Project FY05 IBRD/IDA Bangladesh Health SWAP FY05 IBRD/IDA Burkina Faso Second HIV/AIDS Disaster Response Project FY05 IBRD/IDA Africa African Regional Capacity Building Network for HIV/AIDS Prevention, Treatment, and Care FY05 IBRD/IDA Ghana MAPII FY05 IBRD/IDA Madagascar Second Multisectoral HIV/AIDS Project FY05 IBRD/IDA Central America Central America AIDS Project FY05 IBRD/IDA Nicaragua Health SWAP FY05 IBRD/IDA Cote d'ivoire HIV/AIDS Multi-Sector Response FY05 IBRD/IDA Angola HIV/AIDS, Malaria and TB Control Project FY05 IBRD/IDA Africa Sub-S Africa HIV/AIDS Communications & Education Capacity FY05 IBRD/IDA India AIDS III FY06 IBRD/IDA Dominica HIV/AIDS Prevention and Control FY06 IBRD/IDA Benin Health FY06 IBRD/IDA Kenya MAP II FY06 IBRD/IDA Africa Horn of Africa HIV/AIDS Sub-Regional Project FY06 IBRD/IDA Suriname Suriname HIV/AIDS Prevention and Control FY06 IBRD/IDA Togo HIV/AIDS Emergency Response FY06 IBRD/IDA Cape Verde HIV/AIDS Prevention FY07 IBRD/IDA Nigeria MAP II FY07 IBRD/IDA Uganda MAP II FY07 IBRD/IDA Africa West Africa Sub-Regional HIV/AIDS Project FY07 IBRD/IDA Senegal HIV/AIDS Prevention FY08 IBRD/IDA Haiti AIDS and Communicable Diseases FY08 IBRD/IDA Haiti CARIBBEAN HIV/AIDS II - HAITI FY08 IBRD/IDA TOTAL

11 Global HIV/AIDS Monitoring and Evaluation Support Team: GAMET! A team which aids nations to develop M&E systems so that they can learn from the process of implementing their HIV/AIDS projects! GAMET works in close association with UNAIDS, bilateral donors, GFATM and other technical agencies

12 Learning By Doing! Use of M&E Systems to aid managers to use evidence to make management decisions to steer and guide their programs! Make tactical and strategic changes to increase project impacts

13 Promotes Harmonization! Actively supports the Three Ones! Supports National AIDS Coordinating Authority to establish a national M&E framework " Planning " Implementation " Using data for decision making! Coordinates with other agencies working on M&E issues

14 GAMET s Product Lines! Works with nations to establish national M&E frameworks.! Community Learning Established decentralize M&E systems to track outcome indicators to aid local managers to learn from the programs and make strategic and tactical changes to enhance impacts " Districts, NGOs! Strategic thinking methods that aid managers to replan programs using M&E data! Monitoring the quality of ARV services

15 GAMET Team! Country Support Team: About 20 long term consultants! Working in the three product lines in more than 30 nations! Heavy emphasis in Africa! Now expanded beyond to Africa to Asia, Caribbean, Eastern Europe, Central Asia! Support Bank and non-bank Projects

16 M&E and the Management Cycle INPUTS OUTPUTS OUTCOMES RESULTS IMPACTS COSTS Process Policy & Planning Monitoring Evaluation

17 Ideal M&E Information Density Inputs Outputs Outcomes Most Studies & Attention ALL MOST SOME Impacts FEW Resource Staff Funds Supplies Condoms Distributed Trained Staff Quality Services Clients Served Workshops Held Short to Long Term Behavior Change Knowledge Change Attitude Change Policy Change HIV Prevalence AIDS Related Mortality STI Trends Economic Impacts

18 Ideal M&E Information Density Inputs Outputs Outcomes Information Needed by Program Managers ALL MOST SOME Impacts FEW Resource Staff Funds Supplies Condoms Distributed Trained Staff Quality Services Clients Served Workshops Held Short to Long Term Behavior Change Knowledge Change Attitude Change Policy Change HIV Prevalence AIDS Related Mortality STI Trends Economic Impacts

19 Macro vs Local M&E Data District Program NATIONAL LEVEL

20 Stylized Structure of HIV/AIDS Assistance National HIV/AIDS Strategy Development Partners National HIV/AIDS Committee Public Sector Private and NGO Sector Administrative Layers Local Administrative Layers Community Led/Based Initiatives Partner Organizations

21 Stylized Structure of HIV/AIDS Assistance National HIV/AIDS Strategy I. Development Partners National HIV/AIDS Commission Public Sector Private and NGO Sector II. III. Administrative Layers Local Administrative Layers Community Led/Based Initiatives Partner Organizations

22 Community Learning Has Three Components Screening Diagnostics Treatment Project Area + - Why Is There A Problem? What Program Changes Should We Make? LQAS

23 LQAS!Lot Quality Assurance Sampling or!local Quality Assurance and Supervision

24 An analysis method that can: What is LQAS?! be used locally in supervision areas # to identify priority areas: those not reaching an established performance benchmark for an indicator # data for local management decision making and for sharing information across supervision areas! measure coverage at an aggregate level (e.g., district or province or nation) Catchment Area # Suitable for Reporting Purposes! LQAS uses small samples # Most frequently used size = <20 per Supervision Area # Larger sizes are seldom needed

25 A B Good C D E F G Below Average or Established Benchmark

26 LQAS Table: Decision Rules for Sample Sizes of and Coverage Benchmarks or Average Coverage of 10% to 95% Coverage Benchmarks or Average Coverage Sample Sizes 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 12 N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A For all coverage benchmarks (except where noted) LQAS is at least 92% sensitive and specific N/A = Not Applicable -- Indicates that LQAS should not be used since coverage is too low for LQAS to detect. Alpha and Beta Errors are > 10% Alpha and Beta Errors are > 15%

27 Delivery Attended by a Clinically Trained Provider Benchmark = 75%, Decision Rule=12; Mean = 65%, Decision Rule=10 * = Below Benchmark; # = Below Mean District Area Yes No Total 1 11 * * # * # Total

28 Nile River Project Districts Arua Bushenyi Iganga Kabale Kam pala Kam uli Kam wenge Kayunga Kyenjojo Lira NMasaka Masindi W E Mayuge S Mbale Mbarara Mukono Rakai Sironko Wakiso Lakes Non-project districts Kilometers

29 S.A. = 19 District S.A. = 19 S.A. = 19 S.A. = 19 Nation or Region District S.A. = 19 S.A. = 19 S.A. = 19 S.A. = 19 District SA= Supervision Area S.A. = 19 S.A. = 19 S.A. = 19 S.A. = 19

30 Antenatal Care & Institutional Birth in Uganda by District W N Masindi Arua Kamwenge Lira Kampala 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% C Kayunga Wakiso Mukono Kamuli E Bushenyi Iganga Mbarara Sironko Kabale Mbale Rakai Mayuge S Masaka ANC ANCard InstBirth

31 Kabarole Hoima Youth Condom Use Pattern: Ever, 1st Contact, & Last Contact Kitigum Moyo Arua Nakapiripiriti Lira Apac Kampala 100% 80% 60% 40% 20% 0% Wakiso Sironko Mbale Soroti Tororo Kaberamaido Kayunga Mukono Masindi Kamwenge Kalangala Bushenyi Mbarara Kabale CondomEverYth CondLastSexYth Kamuli Iganga Mayuge Jinja Masaka Rakai Cond1stSex

32 Condom Use in the Last 12-Months among Sexually Active People Living with HIV/AIDS 100% 80% 60% 51% 48% 40% 20% 26% 30% 24% 22% 0% ALWAYS SOMETIMES NEVER Male Female

33 Capacity Building Results 2003!337 Participants Trained!203 Organizations in 19 Districts!2109 Villages Visited!In each Village an Interview of $ Orphan 6-18 years $ Mother of an Infant $ Youth $ Woman Years $ Man $ PLWHAs $ TOTAL OF 2109 x 6 = 12,654 INTERVIEWS

34 Next Steps Carried Out % Understanding why key indicators are low or have district variation % Mobilize strategic thinking workshop % Mobilize re-planning of map activities % Include decentralized M&E in the annual work plans of all participating Districts