Appendix F. Results of the Electronic Survey of World Bank Task Team Leaders

Appendix F. Results of the Electronic Survey of World Bank Task Team Leaders 1. An electronic survey was administered in April-May 2013 to 112 World Bank Task Team Leaders of Bank-supported health projects during the period 2006 2012, in which either child health or health systems performance have been listed as themes. Only 24 TTLs responded to the survey for a response rate of 21.4 percent. The results lack statistical validity and should be interpreted cautiously. The results are reported here only for documentation purposes. Background Questions to World Bank Task Team Leaders 2. Question 1. Please indicate all the countries and the respective time periods during which you were the TTL of record for Bank-supported health projects between 2006 and 2012, inclusive. Region Number of Countries Indicated Africa 16 East Asia & Pacific 5 Latin America & Caribbean 6 South Asia 5 Europe & Central Asia 13 Middle East & North Africa 2 Total 47 3. Question 2. If you identified more than one country, please answer this survey from the point of view of the country in which you worked the longest during 2006 2012 and in which GAVI was active in the country. Please identify this country. World Bank TTL by Region Region Number of Share of Africa 5 a + 2 b (7) 32% East Asia & Pacific 3 a + 3 b (6) 27% Latin America & Caribbean 2 a 9% South Asia 2 a + 2 b (4) 18% Europe & Central Asia 2 a 9% Middle East & North Africa 1 b 5% Total 22 a. identified more than one country and selected a country in this region to answer the survey. b. identified only one country. 79

World Bank s Involvement in Immunization 4. Question 3. During the years that you were working in this country, in which of the following immunization-related activities were you, other Bank staff, or consultants reporting to you involved? (Please indicate all that apply.) Response Count Share of Policy dialogue 13 57% Investment projects 16 70% Technical assistance 5 22% Economic and sector work 7 30% Other 0 0% None of us were involved in any immunization-related activities supported by the Bank during 2006-2012 2 9% 5. Question 4. Did you ever supervise or implement any World Bank activities funded by the ImGAVI Trust Fund? (The ImGAVI Trust Fund financed immunization-related health systems strengthening activities during the 2007 2010 period.) Response Count Share of Yes 7 30% No 16 70% Total 23 100% 6. Question 5. What results did the Immunization and GAVI Trust Fund (ImGAVI)-financed activities achieve? ( answered this question only if they have supervised or implemented any World Bank activities funded by the ImGAVI Trust Fund.) 80

Comments 1 1. Strengthening the technical and management capacity of NIP staff at the different levels of health care, to ensure a systematic, effective, and quality delivery of immunization services, within the framework of comprehensive health care; 2. Strengthening supervision and monitoring to ensure compliance with vaccine standards, quality and safety, and tracking results at the different levels; 3. Effectively covering municipalities with at-risk areas due to difficult access; 4. Strengthening the Epidemiological Surveillance System, detecting and investigating suspected cases of vaccinepreventable diseases as well as launching immediate responses; 5. Reviewing the EPI budget and sources of funds. 2 Financed two studies related to quality of child health services including immunization and child health statistics in Tajikistan. 3 The GAVI funding has made it possible to address the core problems that afflict immunization services in Ukraine: (a) low knowledge on the major part of key medical workers about vaccine safety and efficacy; (b) gaps and inconsistencies in adverse events following immunization surveillance and response systems and communications related to it. The activities involved regular technical consultation and joint training implementation by UNICEF, the World Bank Office in Ukraine, the WHO Country Office, WHO Euro and the Ministry of Health have been ensured throughout the project. WHO Euro and the WHO Country Office provided technical support, recommended qualified consultants, and provided guidance for the development of the country assessment tool. The Ministry of Health ensured participation of medical workers via its Ministerial Orders. Ministry of Health specialists participated in the training sessions and delivered presentations. 1 4 Helped identify key issues being faced by immunization program and fiscal and operation issues which need to be addressed to introduce new vaccine in Pakistan. 5 They helped to complete the CSR health sector report and to define an HSS strategy, ultimately supported by a lending project. At this time, GAVI was providing the only available trust fund for HSS. Note: These are answers by survey respondents and do not necessarily reflect IEG s findings. 7. Question 6. In your opinion, how did these ImGAVI-financed activities contribute to the country s immunization-related activities? Please explain. ( answered this question only if they had supervised or implemented any World Bank activities funded by the ImGAVI Trust Fund.) 1 2 3 Comments GAVI contributions were: The project contributed to achieve key elements within the NIP, which strengthened its operations and deliver high quality, safe immunization services in a sustainable and effective manner, within the framework of comprehensive care and wide inter- and intra-sectoral and community participation. Still, the sustainability of program strategies and financial support is essential and arrangements should be made to fill gaps and ensure that activities continue. GAVI Funds such as this fund used in 2011 in Nicaragua serve as a key agglutinators in support of the goals stated in institutional plans. Strengthened the immunization program, provided training, and supported immunization program supervision. The studies revealed some critical gaps in services in Tajikistan, and have been used by the Bank and other development partners in formulating current and future priorities for HSS in Tajikistan. 81

4 5 6 Comments The activities funded under the Trust Fund aimed to: A. Understand the perception of households on health service quality in general and immunization services in particular. B. Conduct a formal assessment of the Adverse Effects from Immunization (AEFI) surveillance and response system (detection, reporting, and causality assessment) in order to identify existing gaps and ways to improve it; C. Develop and implement trainings on AEFI and crisis communication in relation to AEFI for health professionals in order to equip them with better skills and knowledge on AEFI as well as improve skills on communication with the media following adverse events. These activities and training have led to adjustments in the Ukrainian AEF) surveillance and response system which will contribute to formation of a more efficient system for management of AEFI cases. Trainings on AEFI and crisis communication in relation to AEFI for medical workers funded under the GAVI Trust Fund is expected to boost their capacity to respond efficiently to AEFI and communicate more accurately with media on AEFI. As a result of the initiative, it is expected that medical workers will gain clarity on what AEFI is, how causality assessment is done, and how to collect and analyze data from the field. Once a better understanding on these issues is obtained, inaccurate communication and argumentation by speakers on AEFI issues to the media is expected to decrease. As a result, concerns of medical workers, journalists, and the public on safety of vaccines will decrease. In a longer-term perspective, doctors confidence on immunization issues is expected to increase and consequently vaccination refusal levels decrease. Although it raised the understanding of the issues faced by the program - its impact on making policy and related changes are just appearing - to date the impact has been minimal. As for the ESW part, the impact was very indirect, given that we were looking at the overall health system. As for the lending product (prepared thanks to the ESWs), the impact is more clear, as the product is a Results-based Financing project, which includes an indicator on immunization. World Bank s Partnership with GAVI at the Country Level 8. Question 7. To what extent was GAVI active in the country during the years you were working on the country? Rating Response Count Share of Negligibly 1 5% Modestly 1 5% Substantially 13 59% Highly 3 14% No opinion 4 18% Total 22 100% 9. Question 8. What were the principal frameworks within which World Bank- GAVI engagement occurred? (Please indicate all that apply.) (Sorted in descending order.) Response Count Share of Health Systems Funding Platform 8 36% ImGAVI Trust Fund 7 32% International Health Partnerships+ 7 32% Request from the national government 6 27% Joint Assessment of National Strategies 4 18% Sector-Wide Approach (SWAp) 4 18% Request from other GAVI implementing partners (UNICEF and/or WHO) 4 18% Initiative of GAVI (such as GAVI s Country Responsible Officers) 4 18% 82

Response Count Share of None 3 14% Personal initiative of the World Bank TTL 2 9% Other 1 5% Intervention of World Bank s sector director/ manager 0 0% 10. Question 9. During the years that you were working in the country, in which of the following GAVI related country-level committee(s) did the World Bank participate? (Please indicate all that apply.) Response Count Share of Interagency Coordination Committee for Immunization (ICC) 5 23% Health Sector Coordination Committee (HSCC) 7 32% None of the above 11 50% 11. Question 10. During the years that you were working in the country, in which ways was the World Bank involved in country-level activities that directly or indirectly contributed to the work of GAVI? (Please indicate all that apply.) (Sorted in descending order.) Response Count Share of Financing investments in relation to immunization-related health systems strengthening 11 50% Helping to prepare country strategies (such as the five-year plan proposal or the immunization strategy) 10 45% Supporting analytical work in relation to the financial sustainability of immunization activities 7 32% Supporting analytical work in relation to immunization-related health systems strengthening 7 32% Helping with oversight/supervision of GAVI-supported activities (such as participating in joint supervision missions) 6 27% Helping to prepare grant applications to GAVI 5 23% Reviewing immunization strategies (such as introducing new vaccines) 4 18% Helping with implementation of GAVI-supported activities (including financial management and procurement) 3 14% None 3 14% Other 1 5% 12. Question 11. In your experience, which of the following factor(s) have made it easier or more difficult for the World Bank to engage with GAVI-supported activities at the country level? (Sorted in descending order from much easier to much more difficult. ) 83

Response by Sub-question The World Bank and GAVI participate in the Health Systems Funding Platform The focus of GAVI on low-income countries The fact that GAVI provides financial assistance in the form of grants The World Bank and GAVI are partners in the International Health Partnership The World Bank participates in the Interagency Coordinating Committee Much easier Somewhat easier Neither easier nor more difficult Somewhat more difficult Much more difficult Total 4 7 9 0 0 20 4 5 11 0 0 20 4 3 12 1 0 20 1 8 10 0 1 20 1 5 13 0 0 19 The absence of written Bank-wide guidelines or directives for engaging with GAVI beyond the general language 1 2 9 7 1 20 contained in the 2007 HNP Strategy The absence of a Memorandum of Understanding between the GAVI and the World Bank for collaborating at the 1 1 12 4 2 20 country level The World Bank and GAVI participate in the Joint Assessment of National 0 8 12 0 0 20 Strategies The limited country presence of GAVI 0 1 8 7 4 20 The different project cycle of GAVI compared to the World Bank 0 0 15 5 0 20 13. Question 12. Indicate other factors, if any, which have made it EASIER for the World Bank and GAVI to engage at the country level. Regular communication from GAVI Secretariat about the developments, especially application process for HSS platform. More commitment of the recipient. Common goals of Bank project under implementation Multi-agency oversight group looking at GAVI and Global Fund health system strengthening work, including World Bank. The presence of an established donor coordination mechanism through SWAp. Personalities that respect each other s corporate background and mandate. Whenever there was a visit from Geneva, we met, beyond that there was very little engagement or coordination. Willing to enter into partnership. The Bank engagement is wider so GAVI was used to facilitate their financing support. But unfortunately that has been one sided - they listen to the Bank and if it is supportive they appreciate it, but if issues and concerns are raised they usually sidetrack them. 84

IHP+ has helped, as well as the HSS platform (we created one in Benin, with the World Bank, GAVI, and the Global Fund). 14. Question 13. Indicate other factors, if any, which have made it MORE DIFFICULT for the World Bank and GAVI to engage at the country level. Better presence of GAVI at the country level. The absence of the above bodies at the country level. GAVI s policy of withdrawal. Limited country capacity. Political context. GAVI not based in the country, only intermittent missions. The fact that there were not clear guidelines or mandate for collaboration and no clear outcomes from collaboration made it very vague and as a result there was little actual effort. GAVI sidetracking issues being raised by the Bank and just pushing their financing. 15. Question 14. In your experience, how would you best characterize the relationship between the World Bank and GAVI during the years that you were working in the country? (Choose only one.) Response Count Share of Collaborative: The two organizations staff, consultants, and agents worked together 5 25% on common activities in the pursuit of commonly agreed objectives. Complementary: The two organizations staff, consultant, and agents worked 3 15% alongside each other in the pursuit of common objectives. Consultative: The two organizations staff, consultants, and agents consulted each 2 10% other regularly in the course of their own activities. Sharing information only: The two organizations staff, consultants, and agents only 7 35% shared information about each other s activities. Unrelated and independent: The two organizations worked independently of each 1 5% other supporting different health initiatives in the country. Competitive: The two organizations competed for business among the same 0 0% potential clients. Other 2 10% 85

16. Question 15. In your opinion, should the World Bank have been more or less engaged with GAVI during the years you were working in the country? Response Count Share of More engaged 6 30% Neither more or less engaged. The degree of engagement was appropriate. 12 60% Less engaged 2 10% World Bank s Partnership with GAVI at the Global Level 17. Question 16. Overall, to what extent do you consider the World Bank currently to be a partner of GAVI at the following levels? Negligible Modest Substantial High Total COUNTRY level 4 10 5 1 20 GLOBAL level 0 7 10 3 20 18. Question 17. GAVI is now the largest provider of financial resources for childhood immunization in developing countries. In your opinion, to what extent has GAVI s presence had the following impacts on the World Bank since its establishment in 2000? Response by Sub-question World Bank lending to the overall health sector is LOWER OR HIGHER than it otherwise would have been? World Bank lending for childhood immunization is LOWER OR HIGHER than it otherwise would have been? World Bank lending for health systems strengthening is LOWER OR HIGHER than it otherwise would have been? Much lower Lower No change Higher Much higher Total 0 0 18 2 0 20 1 6 11 1 1 20 0 1 15 3 1 20 86

Question 18. Which of the following do you consider the most important COMPARATIVE ADVANTAGES OF GAVI among international development agencies in terms of achieving results for child immunization at the country level? Please rank in order of importance with 1 being the most important and 7 being the least important. (Sorted in ascending order from 1 being the most important and 7 being the least important.) Priority Rating Average Mobilizing donor resources for childhood immunization in the short term 2.85 Building institutional and human resource capacity for immunization 3.45 Promoting country-owned strategies for immunization 3.55 Developing specialized expertise in childhood immunization 3.7 Lowering the transaction costs of development assistance from the point of view of beneficiaries 4.45 Sustaining financial resources for childhood immunization over the long term 4.5 Promoting a results focus to development assistance 5.5 87