What Works in HIV Prevention: Lessons from Randomized Trials Rachel Glennerster Abdul Latif Jameel Poverty Action Lab at MIT povertyactionlab.org 1
Overview What is J-PAL? Why is measuring impact is so hard? Benefits of randomized trials Examples relevant to PEPFAR HIV/AIDS education Know your status programs Lessons 2
What is J-PAL? Established by 3 Professors of Economics at MIT, now a network of researchers throughout the country Goal is to fight poverty by ensuring that policy decisions are based on scientific evidence We do this by: Running randomized impact evaluations of poverty programs Building capacity of others to do randomized evaluations Disseminating the results to decision makers Currently, 50 ongoing programs in Ghana, India, Indonesia, Kenya, Madagascar, Malawi, Mexico, Morocco, Peru, Philippines, Sierra Leone, US, and Zambia 3
Why is Measuring Impact so Hard? To know the impact of a program must be able to answer counterfactual: How would individual have fared without the program But can t observed same individual with and without the program Need an adequate comparison group Common approaches: Before and after Cross section But programs are usually started in particular places at particular times for a reason 4
Randomized Evaluation Determine treatment and control randomly By construction program beneficiaries are not more motivated, richer, more educated etc than nonbeneficiaries Gives nice clean results hard to manipulate or dispute Randomization can be incorporated in different ways lottery beta testing randomized phase in encouragement design Must be done ex ante Not just relevant for medical issues test what messages people respond to 5
HIV/AIDS Education 6 A B D U L L A T I F J A M E E L P O V E R T Y A C T I O N L A B
HIV/AIDS Education HIV/AIDS curricula developed by Government of Kenyan, with input from local groups and UNICEF Not being implemented because few trained teachers Chose (randomly) some schools to receive training in the curricula Also compared to: reducing the cost of education telling children about relative infection rates by age Debate about whether condoms should be discussed in school and essays on how to protect yourself 7
8
9
HIV/AIDS Education Results (I) Training in the curricula $2 per student led to more teaching of the curricula no change in child bearing If had a child, 6 percent increase in number married Reducing the cost of schooling by giving free school uniforms $11-12 per student Reduced drop out rates reduced teenage childbearing by 10 percent Cost per pregnancy averted, US$750 10
HIV/AIDS Education Results (II) Debates and essays US1 per student Increased students knowledge Increased self reported use of condoms No change in self reported sexual activity No pregnancy results collected yet Sugar Daddy talk US1 per student Reduced child bearing with older men by 65 percent No increase in child bearing with younger men Cost per pregnancy averted, $91 11
HIV Testing: Know Your Status 12 A B D U L L A T I F J A M E E L P O V E R T Y A C T I O N L A B
Knowing Your Status as Prevention 50 percent of spending on HIV/AIDS prevention in Africa is on know your status campaigns Many of those tested do not pick up their result Small incentive can get people to pick up their result study conducted in Malawi (Thornton, 2006) less than 40 percent collected their results random sample given incentive to get results 10-20 cents enough to increase take up by 50 percent Does testing change behavior? Offered very subsidized condoms (and money to buy them) Only HIV+ve in long-term partnerships bought more (than those who did not learn status), and then only a few Now looking at impact on HIV status (with Kohler) 13
Evaluation and PEPFAR Impact evaluations are a public good and we under-invest in them Currently organizations often try and do a confused mix of process/output and impact evaluations in too many places Instead, do good process evaluations everywhere and a moderate number of high quality impact evaluations that focus on a few key questions run several different approaches against each other like what are the most effective ways of changing behavior Do not rely on self reported behavior, likely to be biased Use learnings to guide what PEPFAR funds Learn from evaluations done by others in deciding what to fund Teach about relative HIV infection rates of different ages Subsidize school for girls (improves education and health) Are knowing your status campaigns prevention? 14