Evaluating the Matlab Interventions Using Non-standardOctober Methods10, and2012 Outcomes 1 / 31

Transcription

1 Evaluating the Matlab Interventions Using Non-standard Methods and Outcomes Julia Driessen University of Pittsburgh October 10, 2012 Evaluating the Matlab Interventions Using Non-standardOctober Methods10, and2012 Outcomes 1 / 31

2 Overview Overarching goal: Use non-standard methods and outcomes to generate more nuanced understanding of the effects of health interventions Premise: Overall effect may mask significant variation within the population, and interventions effect change beyond immediate avoidance of morbidity/mortality These interventions offer a nice test of economic theories (labor, family, investment, education, etc.) Evaluating the Matlab Interventions Using Non-standardOctober Methods10, and2012 Outcomes 2 / 31

3 D88 Block Boundaries Embankment ICDDR,B Hospital ICDDR,B Subcenter Main River Village Boundaries Block A-D (ICDDR,B area) Block E-G (Government area) VB4 D97 D89 VB8 D96 D90 D95 Block - G VB6 C00 VB5 V50 VB7V96 D94 D93 V51 Matlab D98 VB0 V53 V95 D99 V09 A00 B00 Block - F V05 V36 V08 V04 V06 V02 V01 V03 V61 V60 V32 VB3 V97 V07 V62 D35 V99 VB1 D34 G00 V81 F00 D00 J00 V10 V14 Block - A V31 V98 V84 D33 V38 D30 V35 V89 V55 V54 V52 W00 V11 D32 VB2 D29 V65 V15 V67 U00 V12 V59 V66 Block - D V17 V18 V16 V63 D28 D31 V13 R00 V34 V72 Block - E V56 V49 V19 V20 V47 V57 V23 0 V82 V48 S00 T00 H00 V21 V64 P00 V33 Q00 V22 V75 K00 Block - B V83 VBB VBC V M00 V29 V73 V24 V68 V43 O00 V30 L00 V V74 V85 V40 V39 V44 V41 Block - C V28 V90 N00 V V42 V27 V46 V45 DX1 V86 V80 DX0 V71 V79 V78 V88 VBA V76 Matlab HDSS Study Area Legend? > Meghna River? > > > > E GIS/HDSU, ICDDR,B Meters M. Zahirul Haq Figure: Matlab, Bangladesh Surveillance site established in Pop: 200,000. Divided into treatment and control areas, and a variety of health interventions have been introduced since the late 1970s. Evaluating the Matlab Interventions Using Non-standardOctober Methods10, and2012 Outcomes 3 / 31

4 Family Planning-Fertility Heterogeneity Family Planning-Fertility Heterogeneity Evaluating the Matlab Interventions Using Non-standardOctober Methods10, and2012 Outcomes 4 / 31

5 Family Planning-Fertility Heterogeneity Background Overview Questions: 1. What additional insight is gained by sequentially estimating the fertility response to a family planning program? 2. How does this effect vary based on family characteristics and over time? Hypothesis: Variation in the relative utility of an additional child over time and based on factors such as SES variation in the fertility effect of family planning access Evaluating the Matlab Interventions Using Non-standardOctober Methods10, and2012 Outcomes 5 / 31

6 Family Planning-Fertility Heterogeneity Background Contribution: Shows relative benefit of sequential fertility approach- significant variation in sign and magnitude of fertility response Offers more nuanced evaluation of how program operates while controlling for endogeneity Evaluating the Matlab Interventions Using Non-standardOctober Methods10, and2012 Outcomes 6 / 31

7 Family Planning-Fertility Heterogeneity Background Total fertility Looks at total outcomes over an individual s lifetime (e.g., total births, total losses) Theoretically: in period 2 Choose total children desired in period 1, survival realized Empirically: Regress total births on total child deaths No time-varying components in information set Evaluating the Matlab Interventions Using Non-standardOctober Methods10, and2012 Outcomes 7 / 31

8 Family Planning-Fertility Heterogeneity Background Sequential fertility Looks at the individual outcomes (e.g., is there a birth in year t), allows for updating of information set over time. Allows for consideration of time-varying factors: Contraception trends (stopping vs. spacing) Fertility trends (replacement effect, recent birth effect, etc.) Societal trends/shocks (ex: shigella outbreak in 1984) Introduction of other interventions Evaluating the Matlab Interventions Using Non-standardOctober Methods10, and2012 Outcomes 8 / 31

9 Family Planning-Fertility Heterogeneity Background Matlab Family Planning Program Family planning program was initiated in 1977 in the treatment area Entailed fortnightly visits from female community health workers, who provided contraceptive information and products Women in the comparison area only had standard government family planning services available, and had to visit local health clinics to obtain contraceptives The outreach component of the intervention was crucial given women s limited physical mobility (Sinha, 2003) In the first 18 months of the program contraception use increased from 5 to 33% (Koenig, Rob, Khan, Chakraborty & Fauveau, 1992) Evaluating the Matlab Interventions Using Non-standardOctober Methods10, and2012 Outcomes 9 / 31

10 Family Planning-Fertility Heterogeneity Model Model builds on Heckman & Willis (1976): T U = β t [u(n t, Z t ) f (e t )] t=0 w = p N N t + p Z Z t + p e e t Choice of contraception usage and intensity each period (e t (0, 1)) affects likelihood of an additional birth: p t = p(1 e t ) The choice of e t is non-trivial when there is value in preventing an additional birth: V t+1 ( b N+1 ) = V t+1 ( b N+1 ) V t+1 (b N+1 ) > 0 Evaluating the Matlab Interventions Using Non-standardOctober Methods 10, and 2012 Outcomes 10 / 31

11 Family Planning-Fertility Heterogeneity Model e t is chosen such that: p V t+1 ( b N+1 ) + [p V }{{} t+1(b e N+1 ) + (1 p )Vt+1( e b N+1 )] }{{} (1) (2) = f (e t ) }{{} (3) In this framework, the introduction of a family planning program has 3 effects: e t Fertility (1) Decline in disutility of unplanned birth (2) Decline in long-run consumption effect (3) Decline in psychic/utility cost Thus, it is possible for fertility to remain unchanged or even increase (for some mothers in some years) in the presence of the Matlab FP program Evaluating the Matlab Interventions Using Non-standardOctober Methods 10, and 2012 Outcomes 11 / 31

12 Family Planning-Fertility Heterogeneity Model Method & Data Panel data set of Matlab families from (fertility timeline ) Data source: 1996 Matlab Health & Socioeconomic Survey Prior total analyses find significant fertility declines across mother s age (Joshi & Schultz 2007, Sinha 2003) Undertake a sequential analysis using a fixed effects logit approach: Outcome: whether there is a birth to family i in year t Measure of interest: family planning program availability (not usage) Inclusion of time-sensitive covariates (such as exposure to other interventions) Fixed effects control for nonrandom placement in unobservables (fertility preferences) Evaluating the Matlab Interventions Using Non-standardOctober Methods 10, and 2012 Outcomes 12 / 31

13 Family Planning-Fertility Heterogeneity Fertility effect Overall effect... Table: Fixed Effect Logit Analysis of Fertility Effect Coefficient (Std. err.) % Family planning available (FP) (0.077) Mother age (0.067) Mother age (0.109) Mother age (0.156) Mother age (0.205) Mother age (0.261) Mother age (0.351) Other intervention indicators YES Year indicators YES Observations (1776) Hausman test for FE 80.86*** Covariates include indicators for mother s age category, other interventions, and year; outcome is probability of having a birth in year i Evaluating the Matlab Interventions Using Non-standardOctober Methods 10, and 2012 Outcomes 13 / 31

14 Family Planning-Fertility Heterogeneity Fertility effect...masks significant variation in magnitude... Table: Fixed Effect Logit Analysis of Fertility Effect (1) (2) FP x Mother age (0.148) FP x Mother age (0.116) FP x Mother age (0.101) FP x Mother age (0.104) FP x Mother age (0.123) FP x Mother age (0.182) FP x Mother age (0.377) FP x Muslim x Schooling FP x Muslim x No schooling FP x non-muslim x Schooling FP x non-muslim x No schooling (0.101) (0.090) (0.256) (0.169) Covariates include indicators for mother s age category, other interventions, and year; outcome Julia Driessen is probability (Pitt) of having a birth in year i Evaluating the Matlab Interventions Using Non-standardOctober Methods 10, and 2012 Outcomes 14 / 31

15 Family Planning-Fertility Heterogeneity Fertility effect...and sign Table: Heterogeneity of Fertility Response to Family Planning Access Year 1 of FP program availability (0.127) Year 2 of FP program availability (0.127) Year 3 of FP program availability (0.227) Year 4 of FP program availability (0.213) Year 5 of FP program availability (0.200) Year 6 of FP program availability (0.126) Year 7 of FP program availability (0.140) Year 8 of FP program availability (0.145) Year 9 of FP program availability (0.164) Year 10 of FP program availability (0.281) Year 11 of FP program availability (0.357) Year 12 of FP program availability (0.379) (1) (2) Years 1-2 of FP program availablity (0.096) Years 3-5 of FP program availability (0.181) Years 6+ of FP program availability (0.094) Covariates include indicators for mother s age category, other interventions, and year; outcome is probability of having a child mortality in year i Evaluating the Matlab Interventions Using Non-standardOctober Methods 10, and 2012 Outcomes 15 / 31

16 Family Planning-Fertility Heterogeneity Fertility effect Findings Significant variation in fertility response Overall effect driven by families with older mothers FP program also affects fertility gaps due to religion, education Interesting time dynamics- non-uniform decline Evaluating the Matlab Interventions Using Non-standardOctober Methods 10, and 2012 Outcomes 16 / 31

17 Family Planning-Fertility Heterogeneity Fertility effect Conclusion Overall, more detailed understanding of how the intervention operates in the population. Underscores the importance of considering timing and unobservable preferences, looking beyond overall effect Sequential fertility accounted for these factors and captured the trajectory of the intervention s role in fertility decisions Total fertility only identified the overall, long-term negative effect, but not the dynamics behind it Evaluating the Matlab Interventions Using Non-standardOctober Methods 10, and 2012 Outcomes 17 / 31

18 Measles Vaccination-School Enrollment Measles Vaccination-School Enrollment Evaluating the Matlab Interventions Using Non-standardOctober Methods 10, and 2012 Outcomes 18 / 31

19 Measles Vaccination-School Enrollment Background Questions 1. Does childhood measles vaccination improve school enrollment? 2. How does the schooling effect vary by gender? Builds on earlier work on vaccination-fertility link to identify whether fertility changes lead to investment changes Evaluating the Matlab Interventions Using Non-standardOctober Methods 10, and 2012 Outcomes 19 / 31

20 Measles Vaccination-School Enrollment Background Pathway Childhood vaccination reduces illness and saves lives Early childhood morbidity hampers physical and cognitive development (Cebu) Delays persist to affect adult health and worker productivity Should be able to identify achievement gap given data on long-term outcomes Evaluating the Matlab Interventions Using Non-standardOctober Methods 10, and 2012 Outcomes 20 / 31

21 Measles Vaccination-School Enrollment Background Measles Vaccination in Matlab Measles vaccine introduced into half of treatment area (blocks A and C) in February 1982 Extended to other half of treatment area (blocks B and D) in December 1985 Target age group: 9-59 months Recommended age group: 9-12 months Uptake was a choice not randomized Evaluating the Matlab Interventions Using Non-standardOctober Methods 10, and 2012 Outcomes 21 / 31

22 Measles Vaccination-School Enrollment Background Vaccination Uptake: Recommended % of Recommended Age Measles Vaccination, by Birth Year and Block Year A B C D E Evaluating the Matlab Interventions Using Non-standardOctober Methods 10, and 2012 Outcomes 22 / 31

23 Measles Vaccination-School Enrollment Background Vaccination Uptake: Non-recommended % of Measles Vaccination Months, by Birth Year and Block Year A B C D E Evaluating the Matlab Interventions Using Non-standardOctober Methods 10, and 2012 Outcomes 23 / 31

24 Measles Vaccination-School Enrollment Data and Methods Data All births from in Matlab Data source: 1982 & 1996 Matlab Health & Socioeconomic Surveys, Record-Keeping System (RKS) N=32,319 Variable Mean Std. Dev. Min Max Educational enrollment Boys Girls Fraction of years 1-5 with vaccination Boys Girls Entered treatment area by 12 months of age Entered treatment area after 12 months and before 5 years of age Evaluating the Matlab Interventions Using Non-standardOctober Methods 10, and 2012 Outcomes 24 / 31

25 Measles Vaccination-School Enrollment Data and Methods Methods Used instrumental variables estimation to deal with endogeneity of choice to vaccinate Outcome: whether child i is enrolled in school by 1996 Measure of interest: % of time from years 1-5 with coverage against measles Instruments: indicators for in treatment area by 12 months, and after 12 months but before 5 years of age Captures total effect = direct effect + spillover effect Covariates: parents education levels, family size, child gender, mother s age, block of residence, year of birth, exposure to other health interventions Evaluating the Matlab Interventions Using Non-standardOctober Methods 10, and 2012 Outcomes 25 / 31

26 Measles Vaccination-School Enrollment Results Overall Impact of measles vaccination on school enrollment: Probit model Dependent Variable Probit School Enrollment IV Probit First Stage Measles Vaccination IV Probit School Enrollment Fraction of years 1-5 with vaccination 0.024* (0.013) (0.039) In a treatment area by 12 months of age 0.844*** (0.072) In a treatment year after 12 months and before 5 years of age 0.443*** (0.084) N Pseudo R-squared Log pseudo-likelihood Wald test of exogeneity of measles (chi-sq) p-value of Wald Test Evaluating the Matlab Interventions Using Non-standardOctober Methods 10, and 2012 Outcomes 26 / 31

27 Measles Vaccination-School Enrollment Results By gender Impact of measles vaccination on school enrollment: Probit model Dependent Variable Fraction of years 1-5 with vaccination B In a treatment area by 12 months of age In a treatment year after 12 months and before 5 years of age Fraction of years 1-5 with vaccination G In a treatment area by 12 months of age In a treatment year after 12 months and before 5 years of age Probit IV Probit IV Probit School Measles School Enrollment Vaccination Enrollment ** (0.015) (0.044) 0.835*** 0.029* (0.017) (0.073) 0.440*** (0.083) 0.853*** (0.070) 0.446*** (0.085) (0.045) Evaluating the Matlab Interventions Using Non-standardOctober Methods 10, and 2012 Outcomes 27 / 31

28 Measles Vaccination-School Enrollment Results Findings Measles vaccination increases enrollment of boys, but not girls ( 10%) Why boys? Different health effects Social norms Higher observed effect with IV suggests large group effects Evaluating the Matlab Interventions Using Non-standardOctober Methods 10, and 2012 Outcomes 28 / 31

29 Measles Vaccination-School Enrollment Results Conclusion Evidence of longer-term, socioeconomic effects of childhood health interventions Strengthened argument for investment in vaccines Again, heterogeneity is important in delivering fuller picture Evaluating the Matlab Interventions Using Non-standardOctober Methods 10, and 2012 Outcomes 29 / 31

30 Final Thoughts This work identified effects of very common health programs that were captured by standard evaluations. Sequential fertility analysis showed that the fertility response to the family planning intervention was surprisingly dynamic Evidence of longer-term, socioeconomic effects of childhood measles vaccination intervention Argument for two-fold awareness/expansion of program evaluation: Longer-term outcomes Focus on heterogeneity to highlight significant deviations from overall effect Evaluating the Matlab Interventions Using Non-standardOctober Methods 10, and 2012 Outcomes 30 / 31

31 Next Steps Look at other interventions Look at other outcomes (especially longer-term) Evaluating the Matlab Interventions Using Non-standardOctober Methods 10, and 2012 Outcomes 31 / 31