Applied Quantitative Methods II

Transcription

1 Applied Quantitative Methods II Lecture 7: Endogeneity and IVs Klára Kaĺıšková Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 1 / 36

2 Outline 1 OLS and the treatment effect 2 OLS and endogeneity 3 Dealing with endogeneity 4 Instrumental variables Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 2 / 36

3 OLS assumptions: short reminder The regression model is linear in the coefficients, is correctly specified, and has an additive error term The error term has a zero population mean Observations of the error term are uncorrelated with each other The error term has a constant variance All explanatory variables are uncorrelated with the error term No explanatory variable is a perfect linear function of any other explanatory variable(s) (The error term is normally distributed) Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 3 / 36

4 Does the OLS estimate treatment effect? If the OLS assumptions hold, OLS is BLUE and estimates the treatment effect correctly Assumption of no correlation b/w explanatory variables and error term crucial! Variables correlated with error term are endogenous variables their coefficients are inconsistent and biased OLS does not estimate the treatment effect If x and the error term are correlated, OLS estimate attributes to x some of the variation in y that actually comes form the error term Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 4 / 36

5 Outline 1 OLS and the treatment effect 2 OLS and endogeneity 3 Dealing with endogeneity 4 Instrumental variables Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 5 / 36

6 Typical cases of endogeneity 1. Omitted variables 2. Selection explanatory variable is not in the equation makes part of the error term unobservable characteristics influence both participation in treatment (dependent variable) and explanatory variables most important and widespread problem in treatment effect estimation 3. Simultaneity - reversed causation relationship between dependent variable and explanatory variable goes in both directions 4. Measurement error Some of independent variables measured with (a non-random) error Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 6 / 36

7 Example 1: Class size and test scores Classroom size and test scores Goal: Estimate the effect of class size on educational attainment of children test scores = α + βclass size + ɛ Problems: Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 7 / 36

8 Example 1: Class size and test scores Classroom size and test scores Goal: Estimate the effect of class size on educational attainment of children test scores = α + βclass size + ɛ Problems: 1 Omitted variables Unobserved factors influencing test scores that are correlated with class size e.g. funding, teachers ability, students motivations... 2 Reverse causality Students with low test scores might be put into a smaller classes 3 Measurement error test-scores done with pen-and-paper, which brings errors on Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 7 / 36

9 1. Regression with Omitted variables Classroom size and test scores Suppose the true model is test scores = α + βclass size + δfunding + ω But we don t include funding in the estimation test scores = α + βclass size + ɛ now residuals include funding: ɛ = δfunding + ω and funding is correlated with class-size: E[ɛ class size] 0 richer schools may pay more teachers smaller classes negative corr. BIAS: We expect β < 0, and OLS estimate will overstate the true value: β OLS = 10: part of this negative correlation is due to negative corr. of funding we overstate the true effect (negative bias): β TRUE = 5 Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 8 / 36

10 1. Omitted variables example Example IQ in Africa Do people in Africa have genetically lower IQ scores? IQ score = α + βafrica + ɛ Forgetting education! Education attainment is much lower in Africa and education affects the IQ scores Q: What bias will we have, if we assume β TRUE = 0? Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 9 / 36

11 1. Omitted variables example Example IQ in Africa Do people in Africa have genetically lower IQ scores? IQ score = α + βafrica + ɛ Forgetting education! Education attainment is much lower in Africa and education affects the IQ scores Q: What bias will we have, if we assume β TRUE = 0? A: Negative bias (e.g.β OLS = 10) which can lead to wrong policy conclusions! Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 9 / 36

12 2. Regression with reversed causality Eli Berman - Can Hearts and Minds be bought? Goal: Effect of development spending on violence in Afghanistan violence = α + βspending + ɛ We expect β < 0 but we get β > 0: Q: Why? A: regions with more violence get more money two-way relationship the reversed effect dominates result - positive bias of OLS estimate Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 10 / 36

13 3. Regression with measurement error ME in independent variable! Goal: Does spending on advertising increase sales? The true model: sales t = α + βa t + ɛ Assume e.g. β TRUE = 10 What if advertising measured with error? measured advertising:m t = A t + v t correlation with sales will be weaker: β OLS = 7 if really high measurement error, we may not find any sig. effect at all Results in downward bias (also called attenuation bias) sales t = α + β(m t v t ) + ɛ sales t = α + β 1 M t + (β 2 v + ɛ) Measurement error in dependent variable leads to inefficiency, not bias or inconsistency Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 11 / 36

14 4. Regression with self-selection Angrist (1991): Vietnam war Goal: Estimate effect of being in army (MP - 0/1) on lifetime earnings (LI) LI = α + βmp ɛ Idea: Military participation should decrease lifetime earnings: β TRUE < 0 MP depends on many unobserved factors in error term that are crucial for LI (self-selection): Who goes voluntarily into army? Does he like office work? Value money? Academic attitudes? People who choose to go to army are likely to be the ones who would have lower earnings anyway - positive or negative bias? Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 12 / 36

15 4. Regression with self-selection Angrist (1991): Vietnam war Goal: Estimate effect of being in army (MP - 0/1) on lifetime earnings (LI) LI = α + βmp ɛ Idea: Military participation should decrease lifetime earnings: β TRUE < 0 MP depends on many unobserved factors in error term that are crucial for LI (self-selection): Who goes voluntarily into army? Does he like office work? Value money? Academic attitudes? People who choose to go to army are likely to be the ones who would have lower earnings anyway - positive or negative bias? OLS estimates upward biased: it will we bigger in absolute value (more negative), because it includes the negative correlation caused by self-selection Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 12 / 36

16 Outline 1 OLS and the treatment effect 2 OLS and endogeneity 3 Dealing with endogeneity 4 Instrumental variables Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 13 / 36

17 Ways to deal with endogeneity 1 Controlled (social) experiment RCT: Direct randomization of treated and untreated 2 Natural experiment Finding naturally occurring treated and untreated group that are as similar as possible - usually leads to diff-in-diffs method 3 Propensity score matching If treatment assignment only depends on observable characteristics 4 Instrumental variable (IV) Finding variable that assigns into treatment but does not affect outcome 5 Discontinuity design Probability of treatment is changing discontinuously with a characteristic (i.e. age) 6 Panel data can solve endogeneity if unobserved characteristics are time-invariant Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 14 / 36

18 Dealing with endogeneity: Example Angrist (1991): Vietnam war Goal: Estimate effect of being in army on lifetime earnings LI = α + βmp ɛ Story: Some soldiers volunteered, but it was not enough lottery was used as a mechanism to choose who goes to war they assigned a random number to each day of year and if the number is above some threshold, all men of sufficient age born that day are selected to go (eligibility:z i = 1) lottery - random and exogenous selection of eligibility some were eligible but not participated (health reasons...) some were not eligible but volunteered still, the lottery had a substantial impact on who went to war Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 15 / 36

19 Dealing with endogeneity: Example Angrist (1991): Vietnam war Individual level: Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 16 / 36

20 Dealing with endogeneity: Example Angrist (1991): Vietnam war Intuition: eligibility (lottery result) highly correlated with participation eligibility not connected with any unobserved factors that affect lifetime earnings Use eligibility as an instrument for participation in military comparing lifetime earnings of those who went to war becasue they were eligible (chosen) with those who were not difference in earnings then solely due to military participation, not self-selection! Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 17 / 36

21 Outline 1 OLS and the treatment effect 2 OLS and endogeneity 3 Dealing with endogeneity 4 Instrumental variables Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 18 / 36

22 Instrumental variable An instrumental variable (IV) for X is one solution to the endogenity problem Requirements for Z to be a valid instrument for X : Relevant = Correlated with X Exogenous = Not correlated with Y in other way except through its correlation with X Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 19 / 36

23 Terminology First stage: X = α 0 + α 1 Z + ω Structural model:y = β 0 + β 1 X + ɛ Reduced form: Y = δ 0 + δ 1 Z + ξ an interesting equality follows: δ 1 = α 1 β 1 so β 1 = δ 1 /α 1 Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 20 / 36

24 Are IV requirements satisfied? Is IV relevant? How can we detect weak (not relevant) IVs? Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 21 / 36

25 Are IV requirements satisfied? Is IV relevant? How can we detect weak (not relevant) IVs? Use intuition - is it likely that X and Z are correlated? Formal test: look at the first stage regression - F-statistics should be >10 Weak IV estimates are not consistent! Do not use weak IVs! Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 21 / 36

26 Are IV requirements satisfied? Is IV relevant? How can we detect weak (not relevant) IVs? Use intuition - is it likely that X and Z are correlated? Formal test: look at the first stage regression - F-statistics should be >10 Weak IV estimates are not consistent! Do not use weak IVs! Is IV exogenous? Can we test for exogeneity? Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 21 / 36

27 Are IV requirements satisfied? Is IV relevant? How can we detect weak (not relevant) IVs? Use intuition - is it likely that X and Z are correlated? Formal test: look at the first stage regression - F-statistics should be >10 Weak IV estimates are not consistent! Do not use weak IVs! Is IV exogenous? Can we test for exogeneity? Use intuition! Is there any way Z can be correlated with unobserved determinants of Y? There is not formal test for exogeneity! (see next slide) Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 21 / 36

28 Can we formally test for exogeneity? Is it true that a good instrument should not be correlated with the dependent variable? Can we look at correlation of Y and Z to prove if Z is indeed exogenous? Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 22 / 36

29 Can we formally test for exogeneity? Is it true that a good instrument should not be correlated with the dependent variable? Can we look at correlation of Y and Z to prove if Z is indeed exogenous? No!Z has to be correlated with Y, otherwise it is useless as an instrument But it can only be correlated with Y through X A good instrument must not be correlated with the unobserved determinants of Y otherwise inconsistent! Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 22 / 36

30 Bad instruments 1 Wage and education We estimate wage i = α + βeduc i + u i with an instrument parent educ i Requirments for good IV: 1 There should be some correlation between parental education and own education. Is it likely that more educated parents have more educated children? Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 23 / 36

31 Bad instruments 1 Wage and education We estimate wage i = α + βeduc i + u i with an instrument parent educ i Requirments for good IV: 1 There should be some correlation between parental education and own education. Is it likely that more educated parents have more educated children? Educated parents may provide motivation, role models, better start,... OK Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 23 / 36

32 Bad instruments 1 Wage and education We estimate wage i = α + βeduc i + u i with an instrument parent educ i Requirments for good IV: 1 There should be some correlation between parental education and own education. Is it likely that more educated parents have more educated children? Educated parents may provide motivation, role models, better start,... OK 2 Parental education should not be correlated with unobserved factors determining wages. Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 23 / 36

33 Bad instruments 1 Wage and education We estimate wage i = α + βeduc i + u i with an instrument parent educ i Requirments for good IV: 1 There should be some correlation between parental education and own education. Is it likely that more educated parents have more educated children? Educated parents may provide motivation, role models, better start,... OK 2 Parental education should not be correlated with unobserved factors determining wages. What about parental income? Parental income is likely to be positively related to children s wages and parental education - not OK! Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 23 / 36

34 Bad instruments 2 Classroom attendance and test scores We estimate test scores = α + βclass attendance + u i unobserved: ability, interest more interested/able in class more likely to attend Instrument: being pregnant Conditions: 1 There should be correlation b/w pregnancy and class attendance. Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 24 / 36

35 Bad instruments 2 Classroom attendance and test scores We estimate test scores = α + βclass attendance + u i unobserved: ability, interest more interested/able in class more likely to attend Instrument: being pregnant Conditions: 1 There should be correlation b/w pregnancy and class attendance. pregnant will miss classes often - negative correlation - OK Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 24 / 36

36 Bad instruments 2 Classroom attendance and test scores We estimate test scores = α + βclass attendance + u i unobserved: ability, interest more interested/able in class more likely to attend Instrument: being pregnant Conditions: 1 There should be correlation b/w pregnancy and class attendance. pregnant will miss classes often - negative correlation - OK 2 Pregnancy should not be correlated with unobserved factors affecting test scores. Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 24 / 36

37 Bad instruments 2 Classroom attendance and test scores We estimate test scores = α + βclass attendance + u i unobserved: ability, interest more interested/able in class more likely to attend Instrument: being pregnant Conditions: 1 There should be correlation b/w pregnancy and class attendance. pregnant will miss classes often - negative correlation - OK 2 Pregnancy should not be correlated with unobserved factors affecting test scores. teenage pregnancies prob. associated with lower interest/ability those less interested more likely to get pregnant? - not OK Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 24 / 36

38 Estimation:Two-stage least squares (2SLS or TSLS) Step 1: X = a 0 + a 1 Z 1 + a 2 Z a k Z k + a Q Q + u Step 2: Y = b 0 + b 1 X + b 3 Q+e Substitute the fitted X in place of original X Note: if done manually in two stages, standard errors are based on wrong residuals e = Y b 0 b 1 X when it should be e = Y b 0 b 1 X Let the software do it for you ;-) Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 25 / 36

39 Control variables in 2SLS model Control variables should be in both stages Stage 1: X = a 0 + a 1 Z + a 2 Q + u Stage 2: Y = b 0 + b 1 X + b 2 Q + e Control vars considered instruments, they are just not excluded instruments they are not excluded from the second stage they are their own instrument Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 26 / 36

40 Example Classroom attendance and test scores We estimate test scores = α + βclass attendance + γsat + u i unobserved: ability, interest more interested/able in class more likely to attend Instrument: distance, public transport quality Procedure: 1 Run class attendance = δ 0 + δ 1 SAT + δ 2 dist + δ 3 pubtrans + v i 2 Obtain fitted values: class attendance ˆ = ˆδ 0 + ˆδ 1 SAT + ˆδ 2 dist + ˆδ 3 pubtrans 3 Run test scores = α + βclass attendance ˆ + γsat + u i Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 27 / 36

41 Finding an instrument Geography as an instrument distance, rivers, small area variation Legal/political institutions as an instrument laws, election dynamics Administrative rules as an instrument wage/staffing rules, reimbursement rules, eligibility rules Naturally occurring randomization draft, birth date, lottery, roommate assignment, weather Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 28 / 36

42 How IV estimation works Not all of the available variation in X is used Only that portion of variation in X,which is explained by Z, is used to explain Y Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 29 / 36

43 How IV estimation works Reality: Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 30 / 36

44 What IV estimation measures? The average treatment effect for individuals who can be induced to change [treatment] status by a change in the instrument Imbens and Angrist (1994) Local Average Treatment Effect (LATE) LATE is instrument-dependent different instruments give different estimates! LATE is the average causal effect of X on Y for compliers people affected by instrument (as opposed to always takers or never takers ) Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 31 / 36

45 Example: Proximity to college study and LATE Who are the compliers? Proximity to college means lower costs of college education Low-income individuals might base their decision to go to college on college proximity Who are never-takers? Individual that never go to college, no matter how close they live Who are always-takers? Individuals that always go to college, no matter how close they live (smart and/or rich) Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 32 / 36

46 Important point about IV models The IV estimator is biased E(b IV ) β but with endogeneity: E(b LS ) β, plim(b IV ) β anyway The good thing is, IV estimator is consistent E(b IV ) β as N IV studies thus need large samples Asymptotic behavior or IV plim(b IV ) = β + Cov(Z, e)/cov(z, X ) if Z truly exogenous, then Cov(Z, e) = 0 Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 33 / 36

47 Examples of IV Studies Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 34 / 36

48 Conclusion When can we use IVs? What is the underlying idea? What are the basic conditions? Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 35 / 36

49 Example: Police Hiring Goal: Estimate effect of police force on crime rate at regional level Problems: 1 Measurement error crime = α + βpolice + ɛ Mobilization of police officers (M.E. in X ) as well as differential crime reporting (M.E. in Y ) 2 Simultaneity More police might be hired during a crime wave 3 Omitted variables Unobserved characteristics of regions that affect crime rate e.g. severity of punishment, activity of police Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 36 / 36