Lecture II: Difference in Difference and Regression Discontinuity
|
|
- Dorcas Goodwin
- 5 years ago
- Views:
Transcription
1 Review Lecture II: Difference in Difference and Regression Discontinuity it From Lecture I Causality is difficult to Show from cross sectional observational studies What caused what? X caused Y, Y caused X Omitted Variable Bias/Confounding In some cases you can say whether the estimate is an upper-bound or lower bound estimate Other times impossible ibl to sign bias since omitted variables bias the coefficient of interest positively and negatively. Net effect impossible to determine a-priori. 2 Review (cont.) Discussed Randomized Control Trials as a simple (but not necessarily practical) way to solve the causality problem Randomization works because we can be sure about temporal precedence Randomization works because treatment and control groups are balanced on observables bl and un-observables Review (cont.) Also quickly presented some other commonly used research designs X 01 - Observe only data from post treatment (X) 01 X 02 Observe data from pre and post treatment periods X 03 Observe data from pre and post treatment; observe a longer pre period Common Feature of all these designs is that there is NO CONTROL GROUP 3 4
2 Difference in Difference I 01 X is the pre-period treatment group data 02 is the post intervention ti treatment t t group data 03 is the pre-period control group data 04 is the post intervention ti control group data 5 Difference in Difference I (cont.) Let s Compare to 01 X 02 design How is this different from difference-in-difference design?` No control group, leads to the strong assumption that over time, without an intervention, dependent variable of interest would not have changed 01 X Diff. in Diff. treatment design accounts for the fact that dependent variable might change even if there were no interventioni Similar to the RCT framework the control group provides the counterfactual 6 Difference in Difference I (cont.) Difference in Difference I (cont.) Simplest representation is 2 X 2 Table Diff. in Diff. Estimate= [E(Y T1 ) E(Y T0 )] [E(Y C1 ) E(Y C0 )] Same result even if you calculate = [E(Y T1 ) E(Y C1 )] [E(Y T0 ) E(C T0 )] E(Y T0 ) E(Y T1 ) E(Y C0 ) E(Y C1 ) You are subtracting out the change in the control group from the change in the treatment group If treatment had no effect what does this imply about the magnitudes of the two terms? The two differences are equal If the treatment t t had an effect then either the first term is bigger than the second term (positive effect) or the second term is bigger than the first term (negative effect) 7 8
3 Difference in Difference II Why is Diff. and Diff. powerful? MAIN REASON: We have a control group Another problem with cross-sectional studies is that we worry about unobserved and hard to measure differences between the treatment t t and control group In the difference in difference estimate, Unobserved differences across treatment and control that stay constant over time are differenced out Another way of saying this is that these unobserved unchanging characteristics effect the level but not the changes 9 Difference in Difference III Problems with Difference in Difference Estimation Lets remember What made RCT powerful We knew the assignment mechanism: RANDOMIZATION Note that there is no randomization in Diff. in Diff Unit of observation (for ex. state) still chooses whether or not to get treatment Choice leads to the potential problem that treatment and control groups are different Consequently we are still concerned with some of the usual problems from cross-sectional sectional studies 10 Difference in Difference III (cont.) Main Concern is History How can we be sure that other interventions are also not simultaneously occurring with treatment? For ex. Some states in an effort to reduce smoking might enact anti-smoking laws in public spaces Very possible that the states that enact anti-smoking laws simultaneously enact other anti-smoking measures as well (increase advertising, i increase taxes etc.) For these changes not to bias the difference-in difference estimate we would have to argue that the control group also enacted these other changes at the same time.» Or we would have to adjust for them explicitly in the the regression 11 Difference in Difference III (cont.) Specification Checks Plot pre intervention trends over time for dependent variables separately by treatment and control groups. IF trends are parallel in treatment and control groups and you see sudden change after intervention then you are potentially safe If trends are not parallel then possible bias from other sources Create False Treatments t and Redo estimation For ex. If intervention happened in 1990, assign intervention in treatment group to 1989 and see if you still find an effect If you find an effect then likely that something else is driving your findings 12
4 Difference in Difference III (cont.) Use an outcome that shouldn t be affected by the intervention and redo estimation Difference in Difference IV Still Other Concerns Policy intervention is tied to outcome Difference in Difference will overstate true effect Mean reversion is again a potential problem My sense is that this is only a problem for some outcomes (wages is a good ex.) Long term effect ect might be difficult to estimate Estimate is most reliable right after intervention Long term effects likely confounded by other variables Functional Form Means or Logs Card & Krueger - An Example What is the Effect of a Minimum Wage increase on employment? Theory says rise in wages should lead to less employment Firms are profit-maximizing already, taxing one input (labor) should lead to a decrease in it s use Card & Krueger (cont.) NJ enacted a state law that increased the minimum wage from 4.25 to 5.05 Effective April 1, 1992 Card and Krueger(1994) use a Diff. in Diff. research design to examine whether this change led to lower employment Control group is Pennsylvania where the minimum wage did not change over this time period 15 16
5 Card & Krueger (cont.) Card & Krueger Look at the effects in Fast Food Industry, Why? Leading employer of low-wage workers Easier to measure prices, employment and wages in this industry Survey Burger King, KFC, Wendy s and Roy Roger s chains Exclude McDonalds because McDonalds had a poor response rates to surveys in previous work Initial survey conducted in late February and early March 1992, A month before the NJ minimum wage increase Secondary Survey conducted in November and December 1992 Card & Krueger (cont.) Around 80% response rate in pre-period 90% of these 80% responded in post-period One Key question: Is the wage increase in N.J. meaningful? Yes, average starting wage in New Jersey restaurants increased by 10% (4.61 to 5.05) In wave 1: 31% had a starting wage of 4.25 In Pennsylvania, In wave 1, average starting wage in Pennsylvania was 4.63 and In wage two there was no change NJ PA Card & Krueger: Results Avg. Full Time Employees Before Avg. Full Time Employees After Diff in Diff Estimate: [ ] [ ] [.59]-[-2.16]=2.75 Standard Error on estimate is 1.36 Conclusion: Estimate is positive but not statistically significant at the 5% level C&K Results (cont.) Lets compare to the 01X02 design Question: Given the C&K data what would you have concluded about the effect of the increase in minimum wage if you used this design? This simpler design would have said that the effect of the minimum wage hike is positive and the magnitude=.59 The Diff. in Diff. estimate also says the effect ect of the minimum wage hike is positive but the magnitude is now 2.7 Including a control group increases the 01X02 estimate by a factor of close to
6 C&K Results (cont.) Regression Framework Each observation in the data is a store Dependent variable is Change in employment Independent variables include region, chain dummies (burger king etc.) State Dummy for whether or not in New Jersey Regression coefficient on State Dummy: 2.33 On average the law leads to an increase of 2.33 employees But standard d error on the estimate t is 1.33 so not statistically ti ti different from zero C&K-Other Specifications (cont.) Some stores not affected if they are already above the minimum wage Create a GAP variable 0 for stores in Pennsylvania 0 for stores in NJ whose wage is already above 5.05 (5.05-initial wage)/initial wage for other NJ states % increase in wages Again find a positive effect but not statistically significant C&K-Other Specifications (cont.) % change in employment in the dependent d variable Exclude management employees Include part time workers in employment Exclude stores along the coast of NJ These stores might have a different seasonal pattern Finally surveyors called some stores in NJ more often to get data. Exclude these stores from sample NONE OF THESE CHANGES AFFECT THE BASIC RESULTS C&K - Other Specifications (cont.) Non Wage-Offsets Offset raise in minimum wage by reducing non-wage compensation (fringe benefits) Main fringe benefit is free and reduced price meals Do not find any changes in this measure Future wage offsets Reduce the rate at which salaries increase Examined the average time to first pay raise 23 24
7 Regression Discontinuity (RD) Arbitrary Threshold determines whether or not a unit gets assigned to treatment or Control group Anti-Discrimination law only applies to firms with at least 15 employees Rabbinic Scholar Maimonides says Class size cannot exceed 40, if so must group student into smaller classes For ex. 42 students means average class size is students means two classes of size 40 but 81 students means average class size of 27 Regression Discontinuity (cont.) In this research design being above or below some threshold implies you are in the treatment group Look for a Change in magnitude of the outcome variable right around this pre- specified threshold Regression Discontinuity RD can provide unbiased estimates of the relationship between X and Y If the assignment criteria is explicitly followed there are no concerns of omitted variables bias The key is that we know precisely the assignment mechanism to the treatment and control groups Regression Discontinuity (cont.) Important to note there is NO assignment of individuals to treatment and control at the same value of X This is unlike what happens in most matching estimators where we find very comparable groups based on set of covariates except that some in the group get a treatment and some don t So we have to use the fact that we observe units with values very close to the threshold to get an estimate of the effect of X on Y 27 28
8 Regression Discontinuity (cont.) This research design might make you think of 01 X 02 But it s not? Why is that? There is no time component We are not sure X is the only change happening RD design in Practice I Two types of regression discontinuity Sharp Regression Discontinuity W i = 1 if X i >= C All units with X >= C are assigned the treatment All units with X< C are assigned to control RD Estimate is: E[Y X>=C] E[Y X<C] Probability of Assignment Potential RD Outcome probability of treatment Outco ome Treatment Criteria Treatment criteria 31 32
9 RD in practice II Fuzzy Regression Discontinuity Design Probability of receiving does not have to be 1 at the threshold For ex. Individuals above some threshold could be offered a treatment t t The offer does not lead all individuals to take up treatment As an example think of a voucher scheme that allows people to move neighborhoods. For some individuals voucher amount offered is not enough to get them to comply 33 RD in Practice II (cont.) The key is that offer is only made to individuals who are above threshold Note that it would be wrong to estimate treatment effects by comparing individuals who were offered treatment at threshold C but did not take it, with individuals who were offered treatment at threshold C but did take it. 34 RD in Practice III Estimates from RD design are useful only for providing treatment effects for sub- populations i.e. the subpopulation around the threshold RD design does not provide an overall average treatment t t effect like what we get from Randomized control trials Implies limited external validity 35 RD in practice III (cont.) Threshold h can be determined d by complicated procedures Colleges can create a numerical rating for financial aid that is based on a function of several variables (SAT, family Income, etc.) Would be nice to verify whether decision i rule is strictly being followed or being potentially manipulated Administrators of programs have leeway and can potentially confound the design Researcher no longer knows the assignment mechanism You can check for this by looking at the numbers of individuals near the threshold 36
10 RD in Practice III (cont.) Use Graphs If you don t see a change in the Mean of the Outcome variable around the threshold then likely no effect First important specification check: Look for other discontinuities in the Dependent variable that are comparable in magnitude to the one found near the specified thresholdh If you find others that you can t explain then you question this design RD in Practice III (cont.) Second Important Specification Check Mean values of other variables near pre- specified threshold should be similar Inclusion of these covariates in a regression specification should NOT change your treatment effect Cites Bruce Meyer, Natural and Quasi-Experiments i in Economics, Journal of Business and Economic Statistics, Vol. 13 (2), pp Card and Krueger, Minimum Wages and Employment: A case study of the fast-food industry in NJ and Pennsylvania, American Economic Review, Vol. 84(4), pp Imbens, Lemieux, Regression Discontinuity Designs: A Guide to Practice, NBER Working Paper,
Lecture II: Difference in Difference. Causality is difficult to Show from cross
Review Lecture II: Regression Discontinuity and Difference in Difference From Lecture I Causality is difficult to Show from cross sectional observational studies What caused what? X caused Y, Y caused
More informationCarrying out an Empirical Project
Carrying out an Empirical Project Empirical Analysis & Style Hint Special program: Pre-training 1 Carrying out an Empirical Project 1. Posing a Question 2. Literature Review 3. Data Collection 4. Econometric
More informationInstrumental Variables I (cont.)
Review Instrumental Variables Observational Studies Cross Sectional Regressions Omitted Variables, Reverse causation Randomized Control Trials Difference in Difference Time invariant omitted variables
More informationECON Microeconomics III
ECON 7130 - Microeconomics III Spring 2016 Notes for Lecture #5 Today: Difference-in-Differences (DD) Estimators Difference-in-Difference-in-Differences (DDD) Estimators (Triple Difference) Difference-in-Difference
More informationGeographic Data Science - Lecture IX
Geographic Data Science - Lecture IX Causal Inference Dani Arribas-Bel Today Correlation Vs Causation Causal inference Why/when causality matters Hurdles to causal inference & strategies to overcome them
More informationInstrumental Variables Estimation: An Introduction
Instrumental Variables Estimation: An Introduction Susan L. Ettner, Ph.D. Professor Division of General Internal Medicine and Health Services Research, UCLA The Problem The Problem Suppose you wish to
More informationTRACER STUDIES ASSESSMENTS AND EVALUATIONS
TRACER STUDIES ASSESSMENTS AND EVALUATIONS 1 INTRODUCTION This note introduces the reader to tracer studies. For the Let s Work initiative, tracer studies are proposed to track and record or evaluate the
More informationIntroduction to Program Evaluation
Introduction to Program Evaluation Nirav Mehta Assistant Professor Economics Department University of Western Ontario January 22, 2014 Mehta (UWO) Program Evaluation January 22, 2014 1 / 28 What is Program
More informationMeasuring Impact. Program and Policy Evaluation with Observational Data. Daniel L. Millimet. Southern Methodist University.
Measuring mpact Program and Policy Evaluation with Observational Data Daniel L. Millimet Southern Methodist University 23 May 2013 DL Millimet (SMU) Observational Data May 2013 1 / 23 ntroduction Measuring
More informationCorrelation Ex.: Ex.: Causation: Ex.: Ex.: Ex.: Ex.: Randomized trials Treatment group Control group
Ch. 3 1 Public economists use empirical tools to test theory and estimate policy effects. o Does the demand for illicit drugs respond to price changes (what is the elasticity)? o Do reduced welfare benefits
More informationPropensity Score Methods for Estimating Causality in the Absence of Random Assignment: Applications for Child Care Policy Research
2012 CCPRC Meeting Methodology Presession Workshop October 23, 2012, 2:00-5:00 p.m. Propensity Score Methods for Estimating Causality in the Absence of Random Assignment: Applications for Child Care Policy
More informationThe Limits of Inference Without Theory
The Limits of Inference Without Theory Kenneth I. Wolpin University of Pennsylvania Koopmans Memorial Lecture (2) Cowles Foundation Yale University November 3, 2010 Introduction Fuller utilization of the
More informationApplied Quantitative Methods II
Applied Quantitative Methods II Lecture 7: Endogeneity and IVs Klára Kaĺıšková Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 1 / 36 Outline 1 OLS and the treatment effect 2 OLS and endogeneity 3 Dealing
More informationEvaluating Social Programs Course: Evaluation Glossary (Sources: 3ie and The World Bank)
Evaluating Social Programs Course: Evaluation Glossary (Sources: 3ie and The World Bank) Attribution The extent to which the observed change in outcome is the result of the intervention, having allowed
More informationIntroduction to Applied Research in Economics Kamiljon T. Akramov, Ph.D. IFPRI, Washington, DC, USA
Introduction to Applied Research in Economics Kamiljon T. Akramov, Ph.D. IFPRI, Washington, DC, USA Training Course on Applied Econometric Analysis June 1, 2015, WIUT, Tashkent, Uzbekistan Why do we need
More informationConfidence Intervals and Sampling Design. Lecture Notes VI
Confidence Intervals and Sampling Design Lecture Notes VI Statistics 112, Fall 2002 Announcements For homework question 3(b), assume that the true is expected to be about in calculating the sample size
More informationProblems to go with Mastering Metrics Steve Pischke
Problems to go with Mastering Metrics Steve Pischke Chapter 1 1. Consider the following three causal questions: Many firms, particularly in southern European countries, are small, and owned and run by
More informationWhy randomize? Rohini Pande Harvard University and J-PAL.
Why randomize? Rohini Pande Harvard University and J-PAL www.povertyactionlab.org Agenda I. Background on Program Evaluation II. What is a randomized experiment? III. Advantages and Limitations of Experiments
More informationCHAPTER 2: TWO-VARIABLE REGRESSION ANALYSIS: SOME BASIC IDEAS
CHAPTER 2: TWO-VARIABLE REGRESSION ANALYSIS: SOME BASIC IDEAS 2.1 It tells how the mean or average response of the sub-populations of Y varies with the fixed values of the explanatory variable (s). 2.2
More informationPublic Policy & Evidence:
Public Policy & Evidence: How to discriminate, interpret and communicate scientific research to better inform society. Rachel Glennerster Executive Director J-PAL Global Press coverage of microcredit:
More informationEc331: Research in Applied Economics Spring term, Panel Data: brief outlines
Ec331: Research in Applied Economics Spring term, 2014 Panel Data: brief outlines Remaining structure Final Presentations (5%) Fridays, 9-10 in H3.45. 15 mins, 8 slides maximum Wk.6 Labour Supply - Wilfred
More informationPLS 506 Mark T. Imperial, Ph.D. Lecture Notes: Reliability & Validity
PLS 506 Mark T. Imperial, Ph.D. Lecture Notes: Reliability & Validity Measurement & Variables - Initial step is to conceptualize and clarify the concepts embedded in a hypothesis or research question with
More informationWhat is: regression discontinuity design?
What is: regression discontinuity design? Mike Brewer University of Essex and Institute for Fiscal Studies Part of Programme Evaluation for Policy Analysis (PEPA), a Node of the NCRM Regression discontinuity
More informationThe Effectiveness of Captopril
Lab 7 The Effectiveness of Captopril In the United States, pharmaceutical manufacturers go through a very rigorous process in order to get their drugs approved for sale. This process is designed to determine
More informationAn Introduction to Regression Discontinuity Design
An Introduction to Regression Discontinuity Design Laura Wherry Assistant Professor Division of GIM & HSR RCMAR/CHIME Methodological Seminar November 20, 2017 Introduction to Regression Discontinuity Design
More informationWhat Colorado Employers Need To Know About Marijuana and Workers Compensation
What Colorado Employers Need To Know About Marijuana and Workers Compensation In partnership with Pinnacol Assurance Page 1 ALTHOUGH IT S BEEN A FEW YEARS since the recreational use of marijuana was legalized
More informationVersion No. 7 Date: July Please send comments or suggestions on this glossary to
Impact Evaluation Glossary Version No. 7 Date: July 2012 Please send comments or suggestions on this glossary to 3ie@3ieimpact.org. Recommended citation: 3ie (2012) 3ie impact evaluation glossary. International
More informationRegression Discontinuity Design
Regression Discontinuity Design Regression Discontinuity Design Units are assigned to conditions based on a cutoff score on a measured covariate, For example, employees who exceed a cutoff for absenteeism
More informationCitation for published version (APA): Ebbes, P. (2004). Latent instrumental variables: a new approach to solve for endogeneity s.n.
University of Groningen Latent instrumental variables Ebbes, P. IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite from it. Please check the document
More informationMultiple Linear Regression (Dummy Variable Treatment) CIVL 7012/8012
Multiple Linear Regression (Dummy Variable Treatment) CIVL 7012/8012 2 In Today s Class Recap Single dummy variable Multiple dummy variables Ordinal dummy variables Dummy-dummy interaction Dummy-continuous/discrete
More informationDylan Small Department of Statistics, Wharton School, University of Pennsylvania. Based on joint work with Paul Rosenbaum
Instrumental variables and their sensitivity to unobserved biases Dylan Small Department of Statistics, Wharton School, University of Pennsylvania Based on joint work with Paul Rosenbaum Overview Instrumental
More informationICPSR Causal Inference in the Social Sciences. Course Syllabus
ICPSR 2012 Causal Inference in the Social Sciences Course Syllabus Instructors: Dominik Hangartner London School of Economics Marco Steenbergen University of Zurich Teaching Fellow: Ben Wilson London School
More informationProblem Set 5 ECN 140 Econometrics Professor Oscar Jorda. DUE: June 6, Name
Problem Set 5 ECN 140 Econometrics Professor Oscar Jorda DUE: June 6, 2006 Name 1) Earnings functions, whereby the log of earnings is regressed on years of education, years of on-the-job training, and
More informationMethods for Addressing Selection Bias in Observational Studies
Methods for Addressing Selection Bias in Observational Studies Susan L. Ettner, Ph.D. Professor Division of General Internal Medicine and Health Services Research, UCLA What is Selection Bias? In the regression
More informationImpact Evaluation Toolbox
Impact Evaluation Toolbox Gautam Rao University of California, Berkeley * ** Presentation credit: Temina Madon Impact Evaluation 1) The final outcomes we care about - Identify and measure them Measuring
More informationEXPERIMENTAL RESEARCH DESIGNS
ARTHUR PSYC 204 (EXPERIMENTAL PSYCHOLOGY) 14A LECTURE NOTES [02/28/14] EXPERIMENTAL RESEARCH DESIGNS PAGE 1 Topic #5 EXPERIMENTAL RESEARCH DESIGNS As a strict technical definition, an experiment is a study
More informationFolland et al Chapter 4
Folland et al Chapter 4 Chris Auld Economics 317 January 11, 2011 Chapter 2. We won t discuss, but you should already know: PPF. Supply and demand. Theory of the consumer (indifference curves etc) Theory
More informationIssues in African Economic Development. Economics 172. University of California, Berkeley. Department of Economics. Professor Ted Miguel
Economics 172 Issues in African Economic Development Professor Ted Miguel Department of Economics University of California, Berkeley Economics 172 Issues in African Economic Development Lecture 10 February
More informationChapter 23. Inference About Means. Copyright 2010 Pearson Education, Inc.
Chapter 23 Inference About Means Copyright 2010 Pearson Education, Inc. Getting Started Now that we know how to create confidence intervals and test hypotheses about proportions, it d be nice to be able
More informationACCTG 533, Section 1: Module 2: Causality and Measurement: Lecture 1: Performance Measurement and Causality
ACCTG 533, Section 1: Module 2: Causality and Measurement: Lecture 1: Performance Measurement and Causality Performance Measurement and Causality Performance Measurement and Causality: We ended before
More informationPropensity Score Analysis Shenyang Guo, Ph.D.
Propensity Score Analysis Shenyang Guo, Ph.D. Upcoming Seminar: April 7-8, 2017, Philadelphia, Pennsylvania Propensity Score Analysis 1. Overview 1.1 Observational studies and challenges 1.2 Why and when
More informationStatistical Power Sampling Design and sample Size Determination
Statistical Power Sampling Design and sample Size Determination Deo-Gracias HOUNDOLO Impact Evaluation Specialist dhoundolo@3ieimpact.org Outline 1. Sampling basics 2. What do evaluators do? 3. Statistical
More informationRegression Discontinuity Designs: An Approach to Causal Inference Using Observational Data
Regression Discontinuity Designs: An Approach to Causal Inference Using Observational Data Aidan O Keeffe Department of Statistical Science University College London 18th September 2014 Aidan O Keeffe
More informationDoctors Fees in Ireland Following the Change in Reimbursement: Did They Jump?
The Economic and Social Review, Vol. 38, No. 2, Summer/Autumn, 2007, pp. 259 274 Doctors Fees in Ireland Following the Change in Reimbursement: Did They Jump? DAVID MADDEN University College Dublin Abstract:
More informationWrite your identification number on each paper and cover sheet (the number stated in the upper right hand corner on your exam cover).
STOCKHOLM UNIVERSITY Department of Economics Course name: Empirical methods 2 Course code: EC2402 Examiner: Per Pettersson-Lidbom Number of credits: 7,5 credits Date of exam: Sunday 21 February 2010 Examination
More informationCross-Lagged Panel Analysis
Cross-Lagged Panel Analysis Michael W. Kearney Cross-lagged panel analysis is an analytical strategy used to describe reciprocal relationships, or directional influences, between variables over time. Cross-lagged
More informationTechnical Track Session IV Instrumental Variables
Impact Evaluation Technical Track Session IV Instrumental Variables Christel Vermeersch Beijing, China, 2009 Human Development Human Network Development Network Middle East and North Africa Region World
More informationIntroduction to Observational Studies. Jane Pinelis
Introduction to Observational Studies Jane Pinelis 22 March 2018 Outline Motivating example Observational studies vs. randomized experiments Observational studies: basics Some adjustment strategies Matching
More informationQuasi-experimental analysis Notes for "Structural modelling".
Quasi-experimental analysis Notes for "Structural modelling". Martin Browning Department of Economics, University of Oxford Revised, February 3 2012 1 Quasi-experimental analysis. 1.1 Modelling using quasi-experiments.
More informationSchool Autonomy and Regression Discontinuity Imbalance
School Autonomy and Regression Discontinuity Imbalance Todd Kawakita 1 and Colin Sullivan 2 Abstract In this research note, we replicate and assess Damon Clark s (2009) analysis of school autonomy reform
More informationEconometric analysis and counterfactual studies in the context of IA practices
Econometric analysis and counterfactual studies in the context of IA practices Giulia Santangelo http://crie.jrc.ec.europa.eu Centre for Research on Impact Evaluation DG EMPL - DG JRC CRIE Centre for Research
More informationCASE STUDY 2: VOCATIONAL TRAINING FOR DISADVANTAGED YOUTH
CASE STUDY 2: VOCATIONAL TRAINING FOR DISADVANTAGED YOUTH Why Randomize? This case study is based on Training Disadvantaged Youth in Latin America: Evidence from a Randomized Trial by Orazio Attanasio,
More informationNBER WORKING PAPER SERIES ALCOHOL CONSUMPTION AND TAX DIFFERENTIALS BETWEEN BEER, WINE AND SPIRITS. Working Paper No. 3200
NBER WORKING PAPER SERIES ALCOHOL CONSUMPTION AND TAX DIFFERENTIALS BETWEEN BEER, WINE AND SPIRITS Henry Saffer Working Paper No. 3200 NATIONAL BUREAU OF ECONOMIC RESEARCH 1050 Massachusetts Avenue Cambridge,
More informationIntroduction to Econometrics
Global edition Introduction to Econometrics Updated Third edition James H. Stock Mark W. Watson MyEconLab of Practice Provides the Power Optimize your study time with MyEconLab, the online assessment and
More informationChapter 13. Experiments and Observational Studies. Copyright 2012, 2008, 2005 Pearson Education, Inc.
Chapter 13 Experiments and Observational Studies Copyright 2012, 2008, 2005 Pearson Education, Inc. Observational Studies In an observational study, researchers don t assign choices; they simply observe
More informationVocabulary. Bias. Blinding. Block. Cluster sample
Bias Blinding Block Census Cluster sample Confounding Control group Convenience sample Designs Experiment Experimental units Factor Level Any systematic failure of a sampling method to represent its population
More informationIntroduction to Quantitative Research and Program Evaluation Methods
Introduction to Quantitative Research and Program Evaluation Methods Dennis A. Kramer II, PhD Assistant Professor of Education Policy Director, Education Policy Research Center Agenda for the Day Brief
More informationThreats and Analysis. Shawn Cole. Harvard Business School
Threats and Analysis Shawn Cole Harvard Business School Course Overview 1. What is Evaluation? 2. Outcomes, Impact, and Indicators 3. Why Randomize? 4. How to Randomize? 5. Sampling and Sample Size 6.
More informationWork, Employment, and Industrial Relations Theory Spring 2008
MIT OpenCourseWare http://ocw.mit.edu 15.676 Work, Employment, and Industrial Relations Theory Spring 2008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
More informationChapter 13 Summary Experiments and Observational Studies
Chapter 13 Summary Experiments and Observational Studies What have we learned? We can recognize sample surveys, observational studies, and randomized comparative experiments. o These methods collect data
More informationThe General Equilibrium Impacts of Unemployment Insurance: Evidence from a Large Online Job Board 1
The General Equilibrium Impacts of Unemployment Insurance: Evidence from a Large Online Job Board 1 Ioana Marinescu, University of Chicago Abstract During the Great Recession, U.S. unemployment benefits
More informationThe Economics of Obesity
The Economics of Obesity John Cawley Cornell University Usefulness of Economics in Studying Obesity Offers widely-accepted theoretical framework for human behavior (constrained maximization) We ask different
More informationThe Effect of the Smoke- Free Illinois Act on Casino Admissions and Revenue
The Effect of the Smoke- Free Illinois Act on Casino Admissions and Revenue Frank J. Chaloupka, University of Illinois at Chicago Society for Research on Nicotine & Tobacco Paper Session 10, Florence,
More informationMethods of Randomization Lupe Bedoya. Development Impact Evaluation Field Coordinator Training Washington, DC April 22-25, 2013
Methods of Randomization Lupe Bedoya Development Impact Evaluation Field Coordinator Training Washington, DC April 22-25, 2013 Content 1. Important Concepts 2. What vs. Why 3. Some Practical Issues 4.
More informationDiurnal Pattern of Reaction Time: Statistical analysis
Diurnal Pattern of Reaction Time: Statistical analysis Prepared by: Alison L. Gibbs, PhD, PStat Prepared for: Dr. Principal Investigator of Reaction Time Project January 11, 2015 Summary: This report gives
More informationExperiments. ESP178 Research Methods Dillon Fitch 1/26/16. Adapted from lecture by Professor Susan Handy
Experiments ESP178 Research Methods Dillon Fitch 1/26/16 Adapted from lecture by Professor Susan Handy Recap Causal Validity Criterion Association Non-spurious Time order Causal Mechanism Context Explanation
More information7 Statistical Issues that Researchers Shouldn t Worry (So Much) About
7 Statistical Issues that Researchers Shouldn t Worry (So Much) About By Karen Grace-Martin Founder & President About the Author Karen Grace-Martin is the founder and president of The Analysis Factor.
More informationQUASI-EXPERIMENTAL HEALTH SERVICE EVALUATION COMPASS 1 APRIL 2016
QUASI-EXPERIMENTAL HEALTH SERVICE EVALUATION COMPASS 1 APRIL 2016 AIM & CONTENTS Aim to explore what a quasi-experimental study is and some issues around how they are done Context and Framework Review
More informationClass 1: Introduction, Causality, Self-selection Bias, Regression
Class 1: Introduction, Causality, Self-selection Bias, Regression Ricardo A Pasquini April 2011 Ricardo A Pasquini () April 2011 1 / 23 Introduction I Angrist s what should be the FAQs of a researcher:
More informationStrategies for handling missing data in randomised trials
Strategies for handling missing data in randomised trials NIHR statistical meeting London, 13th February 2012 Ian White MRC Biostatistics Unit, Cambridge, UK Plan 1. Why do missing data matter? 2. Popular
More informationIn this chapter we discuss validity issues for quantitative research and for qualitative research.
Chapter 8 Validity of Research Results (Reminder: Don t forget to utilize the concept maps and study questions as you study this and the other chapters.) In this chapter we discuss validity issues for
More informationChapter 9 Experimental Research (Reminder: Don t forget to utilize the concept maps and study questions as you study this and the other chapters.
Chapter 9 Experimental Research (Reminder: Don t forget to utilize the concept maps and study questions as you study this and the other chapters.) In this chapter we talk about what experiments are, we
More informationRegression Discontinuity Analysis
Regression Discontinuity Analysis A researcher wants to determine whether tutoring underachieving middle school students improves their math grades. Another wonders whether providing financial aid to low-income
More informationWe re going to talk about a class of designs which generally are known as quasiexperiments. They re very important in evaluating educational programs
We re going to talk about a class of designs which generally are known as quasiexperiments. They re very important in evaluating educational programs and policies because often we might not have the right
More informationRegression Discontinuity Design (RDD)
Regression Discontinuity Design (RDD) Caroline Flammer Ivey Business School 2015 SMS Denver Conference October 4, 2015 The Identification Challenge Does X cause Y? Tempting to regress Y on X Y = a + b
More informationThe Logic of Data Analysis Using Statistical Techniques M. E. Swisher, 2016
The Logic of Data Analysis Using Statistical Techniques M. E. Swisher, 2016 This course does not cover how to perform statistical tests on SPSS or any other computer program. There are several courses
More informationBugbears or Legitimate Threats? (Social) Scientists Criticisms of Machine Learning. Sendhil Mullainathan Harvard University
Bugbears or Legitimate Threats? (Social) Scientists Criticisms of Machine Learning Sendhil Mullainathan Harvard University This is a Poorly Titled Talk Arbitrage Outline of Talk Some past papers of mine
More informationREVIEW FOR THE PREVIOUS LECTURE
Slide 2-1 Calculator: The same calculator policies as for the ACT hold for STT 315: http://www.actstudent.org/faq/answers/calculator.html. It is highly recommended that you have a TI-84, as this is the
More informationAP Stats Review for Midterm
AP Stats Review for Midterm NAME: Format: 10% of final grade. There will be 20 multiple-choice questions and 3 free response questions. The multiple-choice questions will be worth 2 points each and the
More informationMULTIPLE REGRESSION OF CPS DATA
MULTIPLE REGRESSION OF CPS DATA A further inspection of the relationship between hourly wages and education level can show whether other factors, such as gender and work experience, influence wages. Linear
More informationIdentification with Models and Exogenous Data Variation
Identification with Models and Exogenous Data Variation R. Jay Kahn Toni M. Whited University of Michigan University of Michigan and NBER June 11, 2016 Abstract We distinguish between identification and
More informationFirming Up Inequality
Firming Up Inequality Jae Song Social Security Administration Fatih Guvenen Minnesota, FRB Mpls, NBER David Price Stanford Nicholas Bloom Stanford and NBER Till von Wachter UCLA and NBER February 22, 2016
More informationSection The Question of Causation
Section 2.5 - The Question of Causation Statistics 104 Autumn 2004 Copyright c 2004 by Mark E. Irwin Causation Does smoking cause cancer? Did chemical weapons exposure cause health problems in Gulf War
More informationEcon 270: Theoretical Modeling 1
Econ 270: Theoretical Modeling 1 Economics is certainly not the only social science to use mathematical theoretical models to examine a particular question. But economics, since the 1960s, has evolved
More informationBusiness Statistics Probability
Business Statistics The following was provided by Dr. Suzanne Delaney, and is a comprehensive review of Business Statistics. The workshop instructor will provide relevant examples during the Skills Assessment
More informationDoing Quantitative Research 26E02900, 6 ECTS Lecture 6: Structural Equations Modeling. Olli-Pekka Kauppila Daria Kautto
Doing Quantitative Research 26E02900, 6 ECTS Lecture 6: Structural Equations Modeling Olli-Pekka Kauppila Daria Kautto Session VI, September 20 2017 Learning objectives 1. Get familiar with the basic idea
More informationThose Who Tan and Those Who Don t: A Natural Experiment of Employment Discrimination
Those Who Tan and Those Who Don t: A Natural Experiment of Employment Discrimination Ronen Avraham, Tamar Kricheli Katz, Shay Lavie, Haggai Porat, Tali Regev Abstract: Are Black workers discriminated against
More informationThreats and Analysis. Bruno Crépon J-PAL
Threats and Analysis Bruno Crépon J-PAL Course Overview 1. What is Evaluation? 2. Outcomes, Impact, and Indicators 3. Why Randomize and Common Critiques 4. How to Randomize 5. Sampling and Sample Size
More informationGlossary From Running Randomized Evaluations: A Practical Guide, by Rachel Glennerster and Kudzai Takavarasha
Glossary From Running Randomized Evaluations: A Practical Guide, by Rachel Glennerster and Kudzai Takavarasha attrition: When data are missing because we are unable to measure the outcomes of some of the
More informationChapter 7. Marketing Experimental Research. Business Research Methods Verónica Rosendo Ríos Enrique Pérez del Campo Marketing Research
Chapter 7 Marketing Experimental Research Business Research Methods Verónica Rosendo Ríos Enrique Pérez del Campo CHAPTER 7. MARKETING EXPERIMENTAL RESEARCH No great marketing decisions have ever been
More informationIssues in African Economic Development. Economics 172. University of California, Berkeley. Department of Economics. Professor Ted Miguel
Economics 172 Issues in African Economic Development Professor Ted Miguel Department of Economics University of California, Berkeley Economics 172 Issues in African Economic Development Lecture 11 February
More informationRandomized Evaluations
Randomized Evaluations Introduction, Methodology, & Basic Econometrics using Mexico s Progresa program as a case study (with thanks to Clair Null, author of 2008 Notes) Sept. 15, 2009 Not All Correlations
More informationSTA Module 9 Confidence Intervals for One Population Mean
STA 2023 Module 9 Confidence Intervals for One Population Mean Learning Objectives Upon completing this module, you should be able to: 1. Obtain a point estimate for a population mean. 2. Find and interpret
More informationAudio: In this lecture we are going to address psychology as a science. Slide #2
Psychology 312: Lecture 2 Psychology as a Science Slide #1 Psychology As A Science In this lecture we are going to address psychology as a science. Slide #2 Outline Psychology is an empirical science.
More informationLennart Hoogerheide 1,2,4 Joern H. Block 1,3,5 Roy Thurik 1,2,3,6,7
TI 2010-075/3 Tinbergen Institute Discussion Paper Family Background Variables as Instruments for Education in Income Regressions: A Bayesian Analysis Lennart Hoogerheide 1,2,4 Joern H. Block 1,3,5 Roy
More informationSection 9.2b Tests about a Population Proportion
Two-Tailed Tests The basketball player problem and the potato problem were both examples of a single sided or one tailed test of significance. The next problem is a situation that involves a two-sided
More informationHUMAN-COMPUTER INTERACTION EXPERIMENTAL DESIGN
HUMAN-COMPUTER INTERACTION EXPERIMENTAL DESIGN Professor Bilge Mutlu Computer Sciences, Psychology, & Industrial and Systems Engineering University of Wisconsin Madison CS/Psych-770 Human-Computer Interaction
More informationObjectives. Quantifying the quality of hypothesis tests. Type I and II errors. Power of a test. Cautions about significance tests
Objectives Quantifying the quality of hypothesis tests Type I and II errors Power of a test Cautions about significance tests Designing Experiments based on power Evaluating a testing procedure The testing
More information