Inference with Difference-in-Differences Revisited
|
|
- Julia Wiggins
- 5 years ago
- Views:
Transcription
1 Inference with Difference-in-Differences Revisited M. Brewer, T- F. Crossley and R. Joyce Journal of Econometric Methods, 2018 presented by Federico Curci February 22nd, 2018 Brewer, Crossley and Joyce Inference D-D Applied Reading Group 1 / 28
2 This paper What we know Misleading inference in D-D because of serial correlation in the errors Brewer, Crossley and Joyce Inference D-D Applied Reading Group 2 / 28
3 This paper What we know Misleading inference in D-D because of serial correlation in the errors What has been proposed Cluster-robust standard errors can deal with error correlation Not possible with small number of groups Brewer, Crossley and Joyce Inference D-D Applied Reading Group 2 / 28
4 This paper What we know Misleading inference in D-D because of serial correlation in the errors What has been proposed Cluster-robust standard errors can deal with error correlation Not possible with small number of groups What they propose Combination of feasible GLS with cluster-robust inference solve this problem even with small number of groups Brewer, Crossley and Joyce Inference D-D Applied Reading Group 2 / 28
5 Presentation Inference problem in D-D: Bertrand et al. (2004) Cluster-based solutions: Cameron and Miller (2015) Brewer et al. (2018) solution Brewer, Crossley and Joyce Inference D-D Applied Reading Group 3 / 28
6 Previously on Brewer, Crossley and Joyce Inference D-D Applied Reading Group 4 / 28
7 Bertrand et al. (2004) Standard D-D model y i g t = α g + δ t + βt g t + γw i g t + v i g t (1) Brewer, Crossley and Joyce Inference D-D Applied Reading Group 5 / 28
8 Bertrand et al. (2004) Standard D-D model y i g t = α g + δ t + βt g t + γw i g t + v i g t (1) Standard errors might be inconsistent (even if estimator unbiased) v i g t may not be iid within group: v i g t = ɛ g t + u i g t Brewer, Crossley and Joyce Inference D-D Applied Reading Group 5 / 28
9 Bertrand et al. (2004) Standard D-D model y i g t = α g + δ t + βt g t + γw i g t + v i g t (1) Standard errors might be inconsistent (even if estimator unbiased) v i g t may not be iid within group: v i g t = ɛ g t + u i g t...because of serial correlation DD estimation usually relies on fairly long time-series Most commonly used dependent variables in DD are serially correlated Treatment variable changes itself very little within a state over time Brewer, Crossley and Joyce Inference D-D Applied Reading Group 5 / 28
10 Bertrand et al. (2004) Standard D-D model y i g t = α g + δ t + βt g t + γw i g t + v i g t (1) Standard errors might be inconsistent (even if estimator unbiased) v i g t may not be iid within group: v i g t = ɛ g t + u i g t...because of serial correlation DD estimation usually relies on fairly long time-series Most commonly used dependent variables in DD are serially correlated Treatment variable changes itself very little within a state over time...and cross sectional correlation Use micro-data but estimate effect treatment which varies only at group level Brewer, Crossley and Joyce Inference D-D Applied Reading Group 5 / 28
11 Bertrand et al. (2004) Wrong inference in terms of both size and power Size: type I error Probability to reject null hypothesis when the null hypothesis is true Probability to assert something that is absent Power: type II error Probability to not reject null hypothesis when the null hypothesis is false Ability of a test to detect an effect, if the effect actually exist Brewer, Crossley and Joyce Inference D-D Applied Reading Group 6 / 28
12 To solve cross-sectional variation First-stage aggregation Regress using micro data y i g t on w i g t and take the mean residual within each group-time cell (Ŷ g t ) Regress Ŷ i g t on fixed effects and treatment Brewer, Crossley and Joyce Inference D-D Applied Reading Group 7 / 28
13 To solve cross-sectional variation First-stage aggregation Regress using micro data y i g t on w i g t and take the mean residual within each group-time cell (Ŷ g t ) Regress Ŷ i g t on fixed effects and treatment We still have a problem of serial correlation Brewer, Crossley and Joyce Inference D-D Applied Reading Group 7 / 28
14 Bertrand et al. (2004) Assess the extent of serial correlation problem in DD Examine how DD performs on placebo laws: treated states and year of passage are chosen at random Since law are fictitious a significant effect at the 5 percent level should be found roughly 5 percent of the time Use Monte Carlo simulations Brewer, Crossley and Joyce Inference D-D Applied Reading Group 8 / 28
15 Assess magnitude of serial correlation problem Use women s wages from Current Population Survey Years Women between 25 and 50 with positive earnings 50*21 state-years couples (1050) Brewer, Crossley and Joyce Inference D-D Applied Reading Group 9 / 28
16 Assess magnitude of serial correlation problem Use women s wages from Current Population Survey Years Women between 25 and 50 with positive earnings 50*21 state-years couples (1050) Randomly generate laws that affect some states and not other Draw a year at random from a uniform distribution between Select half the states at random Create treatment dummy Brewer, Crossley and Joyce Inference D-D Applied Reading Group 9 / 28
17 Assess magnitude of serial correlation problem Use women s wages from Current Population Survey Years Women between 25 and 50 with positive earnings 50*21 state-years couples (1050) Randomly generate laws that affect some states and not other Draw a year at random from a uniform distribution between Select half the states at random Create treatment dummy Estimate w i g t = α g + δ t + βt g t + γw i g t + v i g t Generate estimate ˆβ and standard error Repeat this exercise large number of times each time drawing new laws at random Brewer, Crossley and Joyce Inference D-D Applied Reading Group 9 / 28
18 Bertrand et al. 200 independent draws of placebo laws Brewer, Crossley and Joyce Inference D-D Applied Reading Group 10 / 28
19 Bertrand et al. Reject null of no effect 67.5 percent of the time Brewer, Crossley and Joyce Inference D-D Applied Reading Group 10 / 28
20 Bertrand et al. Power: reject null of no effect against alternative of 2 percent 66 times Brewer, Crossley and Joyce Inference D-D Applied Reading Group 10 / 28
21 Presentation Inference problem in D-D: Bertrand et al. (2004) Cluster-based solutions: Cameron and Miller (2015) Brewer et al. (2018) solution Brewer, Crossley and Joyce Inference D-D Applied Reading Group 11 / 28
22 Cameron and Miller (2015) Cluster-robust standard errors can deal with error correlation Cluster-Robust Standard Error (CRSE): [ ( V clu ˆβ] = X X ) ( 1 ) Gg (X =1 X g v g v g X g X ) 1 Consistent and Wald statistics based on it asymptotical normal, as G Brewer, Crossley and Joyce Inference D-D Applied Reading Group 12 / 28
23 Cameron and Miller (2015) Cluster-robust standard errors can deal with error correlation Cluster-Robust Standard Error (CRSE): [ ( V clu ˆβ] = X X ) ( 1 ) Gg (X =1 X g v g v g X g X ) 1 Consistent and Wald statistics based on it asymptotical normal, as G Wald t-statistics w = ˆβ β 0 s ˆβ If G, then w N(0,1) under H 0 : β = β 0 Finite G: unknown distribution of w. Common to use w T (G 1) Brewer, Crossley and Joyce Inference D-D Applied Reading Group 12 / 28
24 Cameron and Miller (2015) Problem with Few Clusters Despite reasonable precision in estimating β, ˆV clu ( ˆβ) can be downwards-biased "Overfitting": estimated residuals smaller than true errors Small-G bias larger when distribution of regressors is skewed (Mackinnon and Webb, 2017) Imbalance between number of treated and control Brewer, Crossley and Joyce Inference D-D Applied Reading Group 13 / 28
25 Cameron and Miller (2015) Solution 1: Bias-Corrected CRSE ṽ g = G(N 1) (G 1)(N K ) ˆv g Reduce but not eliminate over-rejection when there are few clusters Brewer, Crossley and Joyce Inference D-D Applied Reading Group 14 / 28
26 Cameron and Miller (2015) Solution 1: Bias-Corrected CRSE ṽ g = G(N 1) (G 1)(N K ) ˆv g Reduce but not eliminate over-rejection when there are few clusters Solution 2: Wild Cluster Bootstrap Idea Estimate main model imposing null hypothesis that you wish to test to give estimate of β H0 Example: test statistical significance of one OLS regressor. Regress y i g on all components of x i g except regressor with coefficient 0 Obtain residuals ṽ i g = y i g x i g β H0 Less trivial to implement and computationally more intensive Montecarlo Brewer, Crossley and Joyce Inference D-D Applied Reading Group 14 / 28
27 Presentation Inference problem in D-D: Bertrand et al. (2004) Cluster-based solutions: Cameron and Miller (2015) Brewer et al. (2018) solution Brewer, Crossley and Joyce Inference D-D Applied Reading Group 15 / 28
28 Brewer et al. (2018) Solution: Combination of feasible GLS with cluster-robust inference Brewer, Crossley and Joyce Inference D-D Applied Reading Group 16 / 28
29 Brewer et al. (2018) Solution: Combination of feasible GLS with cluster-robust inference Tests of correct size can be obtained with standard statistical software vce(cluster clustervar) Even with few groups Brewer, Crossley and Joyce Inference D-D Applied Reading Group 16 / 28
30 Brewer et al. (2018) Solution: Combination of feasible GLS with cluster-robust inference Tests of correct size can be obtained with standard statistical software vce(cluster clustervar) Even with few groups Real problem is power Gains in power can be achieved using feasible GLS even with small groups Brewer, Crossley and Joyce Inference D-D Applied Reading Group 16 / 28
31 Brewer et al. (2018) Solution: Combination of feasible GLS with cluster-robust inference Tests of correct size can be obtained with standard statistical software vce(cluster clustervar) Even with few groups Real problem is power Gains in power can be achieved using feasible GLS even with small groups Combination feasible GLS and cluster-robust inference can also control test size Brewer, Crossley and Joyce Inference D-D Applied Reading Group 16 / 28
32 Structure of the paper Replicate Monte Carlo simulations of Bertrand et al. (2004) CRSE and wild-bootstrap have low size distortion CRSE and wild-bootstrap have low power to detect the real effects Increasing Power with Feasible GLS Brewer, Crossley and Joyce Inference D-D Applied Reading Group 17 / 28
33 Rejection rates when the Null is True Brewer, Crossley and Joyce Inference D-D Applied Reading Group 18 / 28
34 Rejection rates when the Null is True Assuming iid errors, reject more than 40 % Brewer, Crossley and Joyce Inference D-D Applied Reading Group 18 / 28
35 Rejection rates when the Null is True Cluster-robust SE solve serial correlation with big G Brewer, Crossley and Joyce Inference D-D Applied Reading Group 18 / 28
36 Rejection rates when the Null is True Problem cluster-robust SE with small G Brewer, Crossley and Joyce Inference D-D Applied Reading Group 18 / 28
37 Rejection rates when the Null is True Bias-correction of cluster-robust SE have low size distortion, even with small number of groups Brewer, Crossley and Joyce Inference D-D Applied Reading Group 18 / 28
38 Rejection rates when the Null is True Similar for wild cluster bootstrap Brewer, Crossley and Joyce Inference D-D Applied Reading Group 18 / 28
39 Rejection rates when the Null is True Results robust Binary dependent variable Wide range of error process Brewer, Crossley and Joyce Inference D-D Applied Reading Group 19 / 28
40 Rejection rates when the Null is True Results robust Binary dependent variable Wide range of error process Bias-corrected CRSE not robust Large imbalance between treatment and control groups Different from bootstrap Brewer, Crossley and Joyce Inference D-D Applied Reading Group 19 / 28
41 Structure of the paper Replicate Monte Carlo simulations of Bertrand et al. (2004) CRSE and wild-bootstrap have low size distortion CRSE and wild-bootstrap have low power to detect the real effects Increasing Power with Feasible GLS Brewer, Crossley and Joyce Inference D-D Applied Reading Group 20 / 28
42 Power to detect the real effects MDE Brewer, Crossley and Joyce Inference D-D Applied Reading Group 21 / 28
43 Power to detect the real effects Low power for both methods: 2 % effect detected only 40 % times MDE Brewer, Crossley and Joyce Inference D-D Applied Reading Group 21 / 28
44 Power to detect the real effects Higher power with higher effect MDE Brewer, Crossley and Joyce Inference D-D Applied Reading Group 21 / 28
45 Power to detect the real effects Problem of power higher with lower groups MDE Brewer, Crossley and Joyce Inference D-D Applied Reading Group 21 / 28
46 Power to detect the real effects Problem of power higher with lower groups even for large true effects MDE Brewer, Crossley and Joyce Inference D-D Applied Reading Group 21 / 28
47 Structure of the paper Replicate Monte Carlo simulations of Bertrand et al. (2004) CRSE and wild-bootstrap have low size distortion CRSE and wild-bootstrap have low power to detect the real effects Increasing Power with Feasible GLS Brewer, Crossley and Joyce Inference D-D Applied Reading Group 22 / 28
48 Feasible GLS Way to increase efficiency by exploiting knowledge of serial correlation y = X β + v, Cov [v X ] = Ω, ˆβGLS = ( X ΩX ) 1 X Ωy Assume AR(k) process for group-time shocks Estimate model Estimate k parameters using OLS residuals Apply GLS transformation Estimate model on transformed variables Brewer, Crossley and Joyce Inference D-D Applied Reading Group 23 / 28
49 Feasible GLS Problems Estimate k lag parameters inconsistent: Hansen s bias correction may not work for small G Misspecification of the error process Brewer, Crossley and Joyce Inference D-D Applied Reading Group 24 / 28
50 Feasible GLS Problems Estimate k lag parameters inconsistent: Hansen s bias correction may not work for small G Misspecification of the error process Not affect consistency and (likely) more efficient than OLS estimator, but affect test size Possible to use cluster-robust inference to improve test size Plug FGLS residuals into the CRSE formula Brewer, Crossley and Joyce Inference D-D Applied Reading Group 24 / 28
51 Increasing power with Feasible GLS MDE Brewer, Crossley and Joyce Inference D-D Applied Reading Group 25 / 28
52 Increasing power with Feasible GLS FGLS improves power wrt OLS, even with small G MDE Brewer, Crossley and Joyce Inference D-D Applied Reading Group 25 / 28
53 Increasing power with Feasible GLS FGLS combined with bias-corrected CRSE improves size, even with small G MDE Brewer, Crossley and Joyce Inference D-D Applied Reading Group 25 / 28
54 Increasing power with Feasible GLS Results not robust Parametric assumption about serial correlation (power advantage FGLS if process is not MA) Number of time periods (power advantage FGLS with high T) Brewer, Crossley and Joyce Inference D-D Applied Reading Group 26 / 28
55 What we learn from this paper Over-rejection null of no effect not a problem Both with simple bias-correction CRSE and bootstrap Even with small groups Bias-correction CRSE: vce(cluster clustervar) We know it already from Cameron and Miller (2015) Brewer, Crossley and Joyce Inference D-D Applied Reading Group 27 / 28
56 What we learn from this paper Over-rejection null of no effect not a problem Both with simple bias-correction CRSE and bootstrap Even with small groups Bias-correction CRSE: vce(cluster clustervar) We know it already from Cameron and Miller (2015) Bias-correction CRSE not robust in case of imbalances treatment-control and reduce (but not eliminate) over-rejection Bootstrap Brewer, Crossley and Joyce Inference D-D Applied Reading Group 27 / 28
57 What we learn from this paper Problem to detect real effect Combining FGLS and CRSE possible to solve power with correct test size Even with small groups Brewer, Crossley and Joyce Inference D-D Applied Reading Group 28 / 28
58 What we learn from this paper Problem to detect real effect Combining FGLS and CRSE possible to solve power with correct test size Even with small groups FGLS+CRSE not robust in case to small T periods and parametric assumption of serial correlation + improve but not solve power problem Bootstrap Brewer, Crossley and Joyce Inference D-D Applied Reading Group 28 / 28
59 Solution 2: Wild Cluster Bootstrap, cont d Implementation 1 Obtain b th resample 1 Obtain ũ i g = y i g x β i g H0 { 1 w.p Randomly assign cluster g the weight d g = 1 w.p Generate new pseudo-residuals u i g = d g ũ i g and new outcome variables y i g = x i g β H0 + u i g 2 Compute OLS estimate ˆβ b 3 Calculate the Wald test statistics w b = ˆβ b ˆβ s ˆβ b Essentially Replacing y g in each resample with y g = X β H0 + ũ g or y g = X β H0 ũ g Obtain 2 G unique values of w 1,..., w B Back to Solutions Brewer, Crossley and Joyce Inference D-D Applied Reading Group 29 / 28
60 Cameron and Miller (2015): Montecarlo Back to Solutions Back to Conclusion Brewer, Crossley and Joyce Inference D-D Applied Reading Group 30 / 28
61 Size robustness to error specification Simulate state-time shocks, changing degree of serial correlation and non-normality. Assume AR(1) process ( ) (d 2) ɛ g t = ρɛ g t 1 + w g t (2) d 0.004d ɛ g 1 = d 1 w g 1 (3) Back to Robustness Brewer, Crossley and Joyce Inference D-D Applied Reading Group 31 / 28
62 Size robustness to error specification Back to Robustness Brewer, Crossley and Joyce Inference D-D Applied Reading Group 32 / 28
63 Size robustness to imbalance groups Back to Robustness Brewer, Crossley and Joyce Inference D-D Applied Reading Group 33 / 28
64 Power MDE Minimum Detectable Effects (MDE) Smallest effect that would lead to a rejection of the null hypothesis of no effect with given probabilities Back to Power ( MDE }{{} x = se ˆ clu ˆβ) Power c u }{{} Upper critical value t G 1 p1 x t }{{} (1-x)th percentile of t G 1 under null of no effect (4) Brewer, Crossley and Joyce Inference D-D Applied Reading Group 34 / 28
65 Power MDE Back to Power Brewer, Crossley and Joyce Inference D-D Applied Reading Group 35 / 28
66 Power MDE with FGLS Back to Power FGLS Brewer, Crossley and Joyce Inference D-D Applied Reading Group 36 / 28
67 Power robustness to error specification Back to Robustness Brewer, Crossley and Joyce Inference D-D Applied Reading Group 37 / 28
68 Power robustness to number of time periods Back to Robustness Brewer, Crossley and Joyce Inference D-D Applied Reading Group 38 / 28
Example 7.2. Autocorrelation. Pilar González and Susan Orbe. Dpt. Applied Economics III (Econometrics and Statistics)
Example 7.2 Autocorrelation Pilar González and Susan Orbe Dpt. Applied Economics III (Econometrics and Statistics) Pilar González and Susan Orbe OCW 2014 Example 7.2. Autocorrelation 1 / 17 Questions.
More informationEC352 Econometric Methods: Week 07
EC352 Econometric Methods: Week 07 Gordon Kemp Department of Economics, University of Essex 1 / 25 Outline Panel Data (continued) Random Eects Estimation and Clustering Dynamic Models Validity & Threats
More informationMEA DISCUSSION PAPERS
Inference Problems under a Special Form of Heteroskedasticity Helmut Farbmacher, Heinrich Kögel 03-2015 MEA DISCUSSION PAPERS mea Amalienstr. 33_D-80799 Munich_Phone+49 89 38602-355_Fax +49 89 38602-390_www.mea.mpisoc.mpg.de
More informationCitation for published version (APA): Ebbes, P. (2004). Latent instrumental variables: a new approach to solve for endogeneity s.n.
University of Groningen Latent instrumental variables Ebbes, P. IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite from it. Please check the document
More informationCarrying out an Empirical Project
Carrying out an Empirical Project Empirical Analysis & Style Hint Special program: Pre-training 1 Carrying out an Empirical Project 1. Posing a Question 2. Literature Review 3. Data Collection 4. Econometric
More informationStatistical Tests Using Experimental Data
Statistical Tests Using Experimental Data Alec Brandon July 15, 2015 Alternative title So you ve worked your tail off and have some experimental data. Now what? Why are we even talking about statistics?
More informationSession 1: Dealing with Endogeneity
Niehaus Center, Princeton University GEM, Sciences Po ARTNeT Capacity Building Workshop for Trade Research: Behind the Border Gravity Modeling Thursday, December 18, 2008 Outline Introduction 1 Introduction
More informationScore Tests of Normality in Bivariate Probit Models
Score Tests of Normality in Bivariate Probit Models Anthony Murphy Nuffield College, Oxford OX1 1NF, UK Abstract: A relatively simple and convenient score test of normality in the bivariate probit model
More informationSession 3: Dealing with Reverse Causality
Principal, Developing Trade Consultants Ltd. ARTNeT Capacity Building Workshop for Trade Research: Gravity Modeling Thursday, August 26, 2010 Outline Introduction 1 Introduction Overview Endogeneity and
More informationNPTEL Project. Econometric Modelling. Module 14: Heteroscedasticity Problem. Module 16: Heteroscedasticity Problem. Vinod Gupta School of Management
1 P age NPTEL Project Econometric Modelling Vinod Gupta School of Management Module 14: Heteroscedasticity Problem Module 16: Heteroscedasticity Problem Rudra P. Pradhan Vinod Gupta School of Management
More informationThe Late Pretest Problem in Randomized Control Trials of Education Interventions
The Late Pretest Problem in Randomized Control Trials of Education Interventions Peter Z. Schochet ACF Methods Conference, September 2012 In Journal of Educational and Behavioral Statistics, August 2010,
More informationLecture 14: Adjusting for between- and within-cluster covariates in the analysis of clustered data May 14, 2009
Measurement, Design, and Analytic Techniques in Mental Health and Behavioral Sciences p. 1/3 Measurement, Design, and Analytic Techniques in Mental Health and Behavioral Sciences Lecture 14: Adjusting
More informationLec 02: Estimation & Hypothesis Testing in Animal Ecology
Lec 02: Estimation & Hypothesis Testing in Animal Ecology Parameter Estimation from Samples Samples We typically observe systems incompletely, i.e., we sample according to a designed protocol. We then
More informationWrite your identification number on each paper and cover sheet (the number stated in the upper right hand corner on your exam cover).
STOCKHOLM UNIVERSITY Department of Economics Course name: Empirical methods 2 Course code: EC2402 Examiner: Per Pettersson-Lidbom Number of credits: 7,5 credits Date of exam: Sunday 21 February 2010 Examination
More informationThe Effects of Autocorrelated Noise and Biased HRF in fmri Analysis Error Rates
The Effects of Autocorrelated Noise and Biased HRF in fmri Analysis Error Rates Ariana Anderson University of California, Los Angeles Departments of Psychiatry and Behavioral Sciences David Geffen School
More informationECON Microeconomics III
ECON 7130 - Microeconomics III Spring 2016 Notes for Lecture #5 Today: Difference-in-Differences (DD) Estimators Difference-in-Difference-in-Differences (DDD) Estimators (Triple Difference) Difference-in-Difference
More informationStatistical Power Sampling Design and sample Size Determination
Statistical Power Sampling Design and sample Size Determination Deo-Gracias HOUNDOLO Impact Evaluation Specialist dhoundolo@3ieimpact.org Outline 1. Sampling basics 2. What do evaluators do? 3. Statistical
More informationConsequences of effect size heterogeneity for meta-analysis: a Monte Carlo study
Stat Methods Appl (2010) 19:217 236 DOI 107/s10260-009-0125-0 ORIGINAL ARTICLE Consequences of effect size heterogeneity for meta-analysis: a Monte Carlo study Mark J. Koetse Raymond J. G. M. Florax Henri
More information1 Simple and Multiple Linear Regression Assumptions
1 Simple and Multiple Linear Regression Assumptions The assumptions for simple are in fact special cases of the assumptions for multiple: Check: 1. What is external validity? Which assumption is critical
More informationSample size and power calculations in Mendelian randomization with a single instrumental variable and a binary outcome
Sample size and power calculations in Mendelian randomization with a single instrumental variable and a binary outcome Stephen Burgess July 10, 2013 Abstract Background: Sample size calculations are an
More informationEc331: Research in Applied Economics Spring term, Panel Data: brief outlines
Ec331: Research in Applied Economics Spring term, 2014 Panel Data: brief outlines Remaining structure Final Presentations (5%) Fridays, 9-10 in H3.45. 15 mins, 8 slides maximum Wk.6 Labour Supply - Wilfred
More informationInstrumental Variables Estimation: An Introduction
Instrumental Variables Estimation: An Introduction Susan L. Ettner, Ph.D. Professor Division of General Internal Medicine and Health Services Research, UCLA The Problem The Problem Suppose you wish to
More informationUnit 1 Exploring and Understanding Data
Unit 1 Exploring and Understanding Data Area Principle Bar Chart Boxplot Conditional Distribution Dotplot Empirical Rule Five Number Summary Frequency Distribution Frequency Polygon Histogram Interquartile
More informationCHAPTER 4 Designing Studies
CHAPTER 4 Designing Studies 4.2 Experiments The Practice of Statistics, 5th Edition Starnes, Tabor, Yates, Moore Bedford Freeman Worth Publishers Experiments Learning Objectives After this section, you
More informationComparison of Some Almost Unbiased Ratio Estimators
Comparison of Some Almost Unbiased Ratio Estimators Priyaranjan Dash Department of Statistics, Tripura University, India. ---------------------------------------------------------------------***---------------------------------------------------------------------
More informationCheap Donuts and Expensive Broccoli: The Effect of Relative Prices on Obesity
Cheap Donuts and Expensive Broccoli: The Effect of Relative Prices on Obesity Jonah B. Gelbach University of Maryland, Department of Economics Florida State University, College of Law Jonathan Klick Florida
More informationChapter 11: Advanced Remedial Measures. Weighted Least Squares (WLS)
Chapter : Advanced Remedial Measures Weighted Least Squares (WLS) When the error variance appears nonconstant, a transformation (of Y and/or X) is a quick remedy. But it may not solve the problem, or it
More informationMarno Verbeek Erasmus University, the Netherlands. Cons. Pros
Marno Verbeek Erasmus University, the Netherlands Using linear regression to establish empirical relationships Linear regression is a powerful tool for estimating the relationship between one variable
More informationApplied Quantitative Methods II
Applied Quantitative Methods II Lecture 7: Endogeneity and IVs Klára Kaĺıšková Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 1 / 36 Outline 1 OLS and the treatment effect 2 OLS and endogeneity 3 Dealing
More informationOLS Regression with Clustered Data
OLS Regression with Clustered Data Analyzing Clustered Data with OLS Regression: The Effect of a Hierarchical Data Structure Daniel M. McNeish University of Maryland, College Park A previous study by Mundfrom
More informationProblem set 2: understanding ordinary least squares regressions
Problem set 2: understanding ordinary least squares regressions September 12, 2013 1 Introduction This problem set is meant to accompany the undergraduate econometrics video series on youtube; covering
More informationMeasuring the Impacts of Teachers: Reply to Rothstein
Measuring the Impacts of Teachers: Reply to Rothstein Raj Chetty, Harvard University John Friedman, Brown University Jonah Rockoff, Columbia University February 2017 Abstract Using data from North Carolina,
More informationAssessing Studies Based on Multiple Regression. Chapter 7. Michael Ash CPPA
Assessing Studies Based on Multiple Regression Chapter 7 Michael Ash CPPA Assessing Regression Studies p.1/20 Course notes Last time External Validity Internal Validity Omitted Variable Bias Misspecified
More informationAssignment #6. Chapter 10: 14, 15 Chapter 11: 14, 18. Due tomorrow Nov. 6 th by 2pm in your TA s homework box
Assignment #6 Chapter 10: 14, 15 Chapter 11: 14, 18 Due tomorrow Nov. 6 th by 2pm in your TA s homework box Assignment #7 Chapter 12: 18, 24 Chapter 13: 28 Due next Friday Nov. 13 th by 2pm in your TA
More informationWhere does "analysis" enter the experimental process?
Lecture Topic : ntroduction to the Principles of Experimental Design Experiment: An exercise designed to determine the effects of one or more variables (treatments) on one or more characteristics (response
More informationFinal Exam - section 2. Thursday, December hours, 30 minutes
Econometrics, ECON312 San Francisco State University Michael Bar Fall 2011 Final Exam - section 2 Thursday, December 15 2 hours, 30 minutes Name: Instructions 1. This is closed book, closed notes exam.
More informationGlossary From Running Randomized Evaluations: A Practical Guide, by Rachel Glennerster and Kudzai Takavarasha
Glossary From Running Randomized Evaluations: A Practical Guide, by Rachel Glennerster and Kudzai Takavarasha attrition: When data are missing because we are unable to measure the outcomes of some of the
More informationReflection Questions for Math 58B
Reflection Questions for Math 58B Johanna Hardin Spring 2017 Chapter 1, Section 1 binomial probabilities 1. What is a p-value? 2. What is the difference between a one- and two-sided hypothesis? 3. What
More informationMeta-Analysis and Publication Bias: How Well Does the FAT-PET-PEESE Procedure Work?
Meta-Analysis and Publication Bias: How Well Does the FAT-PET-PEESE Procedure Work? Nazila Alinaghi W. Robert Reed Department of Economics and Finance, University of Canterbury Abstract: This study uses
More informationEmpirical Tools of Public Finance. 131 Undergraduate Public Economics Emmanuel Saez UC Berkeley
Empirical Tools of Public Finance 131 Undergraduate Public Economics Emmanuel Saez UC Berkeley 1 DEFINITIONS Empirical public finance: The use of data and statistical methods to measure the impact of government
More informationDescribe what is meant by a placebo Contrast the double-blind procedure with the single-blind procedure Review the structure for organizing a memo
Business Statistics The following was provided by Dr. Suzanne Delaney, and is a comprehensive review of Business Statistics. The workshop instructor will provide relevant examples during the Skills Assessment
More informationExamining Relationships Least-squares regression. Sections 2.3
Examining Relationships Least-squares regression Sections 2.3 The regression line A regression line describes a one-way linear relationship between variables. An explanatory variable, x, explains variability
More informationStill important ideas
Readings: OpenStax - Chapters 1 13 & Appendix D & E (online) Plous Chapters 17 & 18 - Chapter 17: Social Influences - Chapter 18: Group Judgments and Decisions Still important ideas Contrast the measurement
More informationBayesian and Frequentist Approaches
Bayesian and Frequentist Approaches G. Jogesh Babu Penn State University http://sites.stat.psu.edu/ babu http://astrostatistics.psu.edu All models are wrong But some are useful George E. P. Box (son-in-law
More informationReadings: Textbook readings: OpenStax - Chapters 1 13 (emphasis on Chapter 12) Online readings: Appendix D, E & F
Readings: Textbook readings: OpenStax - Chapters 1 13 (emphasis on Chapter 12) Online readings: Appendix D, E & F Plous Chapters 17 & 18 Chapter 17: Social Influences Chapter 18: Group Judgments and Decisions
More informationDISCUSSION PAPER SERIES. No MEASURING THE IMPACTS OF TEACHERS: RESPONSE TO ROTHSTEIN (2014) Raj Chetty, John Friedman and Jonah Rockoff
DISCUSSION PAPER SERIES No. 10768 MEASURING THE IMPACTS OF TEACHERS: RESPONSE TO ROTHSTEIN (2014) Raj Chetty, John Friedman and Jonah Rockoff LABOUR ECONOMICS and PUBLIC ECONOMICS ABCD ISSN 0265-8003 MEASURING
More informationMS&E 226: Small Data
MS&E 226: Small Data Lecture 10: Introduction to inference (v2) Ramesh Johari ramesh.johari@stanford.edu 1 / 17 What is inference? 2 / 17 Where did our data come from? Recall our sample is: Y, the vector
More informationDo Your Online Friends Make You Pay? A Randomized Field Experiment on Peer Influence in Online Social Networks Online Appendix
Forthcoming in Management Science 2014 Do Your Online Friends Make You Pay? A Randomized Field Experiment on Peer Influence in Online Social Networks Online Appendix Ravi Bapna University of Minnesota,
More informationBusiness Statistics Probability
Business Statistics The following was provided by Dr. Suzanne Delaney, and is a comprehensive review of Business Statistics. The workshop instructor will provide relevant examples during the Skills Assessment
More informationIntroduction to Observational Studies. Jane Pinelis
Introduction to Observational Studies Jane Pinelis 22 March 2018 Outline Motivating example Observational studies vs. randomized experiments Observational studies: basics Some adjustment strategies Matching
More informationPsychology Research Process
Psychology Research Process Logical Processes Induction Observation/Association/Using Correlation Trying to assess, through observation of a large group/sample, what is associated with what? Examples:
More informationn Outline final paper, add to outline as research progresses n Update literature review periodically (check citeseer)
Project Dilemmas How do I know when I m done? How do I know what I ve accomplished? clearly define focus/goal from beginning design a search method that handles plateaus improve some ML method s robustness
More informationPropensity scores: what, why and why not?
Propensity scores: what, why and why not? Rhian Daniel, Cardiff University @statnav Joint workshop S3RI & Wessex Institute University of Southampton, 22nd March 2018 Rhian Daniel @statnav/propensity scores:
More informationChapter 1: Exploring Data
Chapter 1: Exploring Data Key Vocabulary:! individual! variable! frequency table! relative frequency table! distribution! pie chart! bar graph! two-way table! marginal distributions! conditional distributions!
More informationPerformance of Median and Least Squares Regression for Slightly Skewed Data
World Academy of Science, Engineering and Technology 9 Performance of Median and Least Squares Regression for Slightly Skewed Data Carolina Bancayrin - Baguio Abstract This paper presents the concept of
More informationEPI 200C Final, June 4 th, 2009 This exam includes 24 questions.
Greenland/Arah, Epi 200C Sp 2000 1 of 6 EPI 200C Final, June 4 th, 2009 This exam includes 24 questions. INSTRUCTIONS: Write all answers on the answer sheets supplied; PRINT YOUR NAME and STUDENT ID NUMBER
More informationFor general queries, contact
Much of the work in Bayesian econometrics has focused on showing the value of Bayesian methods for parametric models (see, for example, Geweke (2005), Koop (2003), Li and Tobias (2011), and Rossi, Allenby,
More informationMantel-Haenszel Procedures for Detecting Differential Item Functioning
A Comparison of Logistic Regression and Mantel-Haenszel Procedures for Detecting Differential Item Functioning H. Jane Rogers, Teachers College, Columbia University Hariharan Swaminathan, University of
More informationEstablishing Causality Convincingly: Some Neat Tricks
Establishing Causality Convincingly: Some Neat Tricks Establishing Causality In the last set of notes, I discussed how causality can be difficult to establish in a straightforward OLS context If assumptions
More informationRegression Discontinuity Design (RDD)
Regression Discontinuity Design (RDD) Caroline Flammer Ivey Business School 2015 SMS Denver Conference October 4, 2015 The Identification Challenge Does X cause Y? Tempting to regress Y on X Y = a + b
More informationDichotomizing partial compliance and increased participant burden in factorial designs: the performance of four noncompliance methods
Merrill and McClure Trials (2015) 16:523 DOI 1186/s13063-015-1044-z TRIALS RESEARCH Open Access Dichotomizing partial compliance and increased participant burden in factorial designs: the performance of
More informationWhat is: regression discontinuity design?
What is: regression discontinuity design? Mike Brewer University of Essex and Institute for Fiscal Studies Part of Programme Evaluation for Policy Analysis (PEPA), a Node of the NCRM Regression discontinuity
More informationTreatment effect estimates adjusted for small-study effects via a limit meta-analysis
Treatment effect estimates adjusted for small-study effects via a limit meta-analysis Gerta Rücker 1, James Carpenter 12, Guido Schwarzer 1 1 Institute of Medical Biometry and Medical Informatics, University
More informationEconometric Game 2012: infants birthweight?
Econometric Game 2012: How does maternal smoking during pregnancy affect infants birthweight? Case A April 18, 2012 1 Introduction Low birthweight is associated with adverse health related and economic
More informationStudy Guide for the Final Exam
Study Guide for the Final Exam When studying, remember that the computational portion of the exam will only involve new material (covered after the second midterm), that material from Exam 1 will make
More informationRUNNING HEAD: TESTING INDIRECT EFFECTS 1 **** TO APPEAR IN JOURNAL OF PERSONALITY AND SOCIAL PSYCHOLOGY ****
RUNNING HEAD: TESTING INDIRECT EFFECTS 1 **** TO APPEAR IN JOURNAL OF PERSONALITY AND SOCIAL PSYCHOLOGY **** New recommendations for testing indirect effects in mediational models: The need to report and
More informationAn Introduction to Modern Econometrics Using Stata
An Introduction to Modern Econometrics Using Stata CHRISTOPHER F. BAUM Department of Economics Boston College A Stata Press Publication StataCorp LP College Station, Texas Contents Illustrations Preface
More informationA COMPARISON OF IMPUTATION METHODS FOR MISSING DATA IN A MULTI-CENTER RANDOMIZED CLINICAL TRIAL: THE IMPACT STUDY
A COMPARISON OF IMPUTATION METHODS FOR MISSING DATA IN A MULTI-CENTER RANDOMIZED CLINICAL TRIAL: THE IMPACT STUDY Lingqi Tang 1, Thomas R. Belin 2, and Juwon Song 2 1 Center for Health Services Research,
More informationStatistical Science Issues in HIV Vaccine Trials: Part I
Statistical Science Issues in HIV Vaccine Trials: Part I 1 2 Outline 1. Study population 2. Criteria for selecting a vaccine for efficacy testing 3. Measuring effects of vaccination - biological markers
More informationThe Limits of Inference Without Theory
The Limits of Inference Without Theory Kenneth I. Wolpin University of Pennsylvania Koopmans Memorial Lecture (2) Cowles Foundation Yale University November 3, 2010 Introduction Fuller utilization of the
More informationMeta-Analysis. Zifei Liu. Biological and Agricultural Engineering
Meta-Analysis Zifei Liu What is a meta-analysis; why perform a metaanalysis? How a meta-analysis work some basic concepts and principles Steps of Meta-analysis Cautions on meta-analysis 2 What is Meta-analysis
More informationComparing Pre-Post Change Across Groups: Guidelines for Choosing between Difference Scores, ANCOVA, and Residual Change Scores
ANALYZING PRE-POST CHANGE 1 Comparing Pre-Post Change Across Groups: Guidelines for Choosing between Difference Scores, ANCOVA, and Residual Change Scores Megan A. Jennings & Robert A. Cribbie Quantitative
More informationRussian Journal of Agricultural and Socio-Economic Sciences, 3(15)
ON THE COMPARISON OF BAYESIAN INFORMATION CRITERION AND DRAPER S INFORMATION CRITERION IN SELECTION OF AN ASYMMETRIC PRICE RELATIONSHIP: BOOTSTRAP SIMULATION RESULTS Henry de-graft Acquah, Senior Lecturer
More informationSESUG Paper SD
SESUG Paper SD-106-2017 Missing Data and Complex Sample Surveys Using SAS : The Impact of Listwise Deletion vs. Multiple Imputation Methods on Point and Interval Estimates when Data are MCAR, MAR, and
More informationEMPIRICAL STRATEGIES IN LABOUR ECONOMICS
EMPIRICAL STRATEGIES IN LABOUR ECONOMICS University of Minho J. Angrist NIPE Summer School June 2009 This course covers core econometric ideas and widely used empirical modeling strategies. The main theoretical
More informationThe Dynamic Effects of Obesity on the Wages of Young Workers
The Dynamic Effects of Obesity on the Wages of Young Workers Joshua C. Pinkston University of Louisville June, 2015 Contributions 1. Focus on more recent cohort, NLSY97. Obesity
More informationINTRODUCTION TO ECONOMETRICS (EC212)
INTRODUCTION TO ECONOMETRICS (EC212) Course duration: 54 hours lecture and class time (Over three weeks) LSE Teaching Department: Department of Economics Lead Faculty (session two): Dr Taisuke Otsu and
More informationInstrumental Variables I (cont.)
Review Instrumental Variables Observational Studies Cross Sectional Regressions Omitted Variables, Reverse causation Randomized Control Trials Difference in Difference Time invariant omitted variables
More informationWDHS Curriculum Map Probability and Statistics. What is Statistics and how does it relate to you?
WDHS Curriculum Map Probability and Statistics Time Interval/ Unit 1: Introduction to Statistics 1.1-1.3 2 weeks S-IC-1: Understand statistics as a process for making inferences about population parameters
More informationInsight Assessment Measuring Thinking Worldwide
California Critical Thinking Skills Test (CCTST). The CCTST measures the reasoning skills human beings use in the process of reflectively deciding what to believe or what to do. Skill/Attribute Name SE
More informationMark J. Koetse 1 Raymond J.G.M. Florax 2 Henri L.F. de Groot 1,3
TI 7-5/3 Tinbergen Institute Discussion Paper The Impact of Effect Size Heterogeneity on Meta-Analysis: A Monte Carlo Experiment Mark J. Koetse 1 Raymond J.G.M. Florax Henri L.F. de Groot 1,3 1 Department
More informationMostly Harmless Simulations? On the Internal Validity of Empirical Monte Carlo Studies
Mostly Harmless Simulations? On the Internal Validity of Empirical Monte Carlo Studies Arun Advani and Tymon Sªoczy«ski 13 November 2013 Background When interested in small-sample properties of estimators,
More informationSUPPLEMENTARY INFORMATION
doi:10.1038/nature16467 Supplementary Discussion Relative influence of larger and smaller EWD impacts. To examine the relative influence of varying disaster impact severity on the overall disaster signal
More informationTesting the Predictability of Consumption Growth: Evidence from China
Auburn University Department of Economics Working Paper Series Testing the Predictability of Consumption Growth: Evidence from China Liping Gao and Hyeongwoo Kim Georgia Southern University and Auburn
More informationSection on Survey Research Methods JSM 2009
Missing Data and Complex Samples: The Impact of Listwise Deletion vs. Subpopulation Analysis on Statistical Bias and Hypothesis Test Results when Data are MCAR and MAR Bethany A. Bell, Jeffrey D. Kromrey
More informationChecking the counterarguments confirms that publication bias contaminated studies relating social class and unethical behavior
1 Checking the counterarguments confirms that publication bias contaminated studies relating social class and unethical behavior Gregory Francis Department of Psychological Sciences Purdue University gfrancis@purdue.edu
More informationMethods of Reducing Bias in Time Series Designs: A Within Study Comparison
Methods of Reducing Bias in Time Series Designs: A Within Study Comparison Kylie Anglin, University of Virginia Kate Miller-Bains, University of Virginia Vivian Wong, University of Virginia Coady Wing,
More informationIndividual preference heterogeneity, targeting and welfare effects of soda taxes
Individual preference heterogeneity, targeting and welfare effects of soda taxes Pierre Dubois, Rachel Griffith and Martin O Connell Institute for Fiscal Studies Lisbon, October 2017 1 / 44 Motivation:
More informationQuasi-experimental analysis Notes for "Structural modelling".
Quasi-experimental analysis Notes for "Structural modelling". Martin Browning Department of Economics, University of Oxford Revised, February 3 2012 1 Quasi-experimental analysis. 1.1 Modelling using quasi-experiments.
More information1.4 - Linear Regression and MS Excel
1.4 - Linear Regression and MS Excel Regression is an analytic technique for determining the relationship between a dependent variable and an independent variable. When the two variables have a linear
More informationGetting started with Eviews 9 (Volume IV)
Getting started with Eviews 9 (Volume IV) Afees A. Salisu 1 5.0 RESIDUAL DIAGNOSTICS Before drawing conclusions/policy inference from any of the above estimated regressions, it is important to perform
More informationChapter 02. Basic Research Methodology
Chapter 02 Basic Research Methodology Definition RESEARCH Research is a quest for knowledge through diligent search or investigation or experimentation aimed at the discovery and interpretation of new
More informationAP Statistics. Semester One Review Part 1 Chapters 1-5
AP Statistics Semester One Review Part 1 Chapters 1-5 AP Statistics Topics Describing Data Producing Data Probability Statistical Inference Describing Data Ch 1: Describing Data: Graphically and Numerically
More informationDescribe what is meant by a placebo Contrast the double-blind procedure with the single-blind procedure Review the structure for organizing a memo
Please note the page numbers listed for the Lind book may vary by a page or two depending on which version of the textbook you have. Readings: Lind 1 11 (with emphasis on chapters 10, 11) Please note chapter
More informationShould a Normal Imputation Model Be Modified to Impute Skewed Variables?
Sociological Methods and Research, 2013, 42(1), 105-138 Should a Normal Imputation Model Be Modified to Impute Skewed Variables? Paul T. von Hippel Abstract (169 words) Researchers often impute continuous
More informationSampling Weights, Model Misspecification and Informative Sampling: A Simulation Study
Sampling Weights, Model Misspecification and Informative Sampling: A Simulation Study Marianne (Marnie) Bertolet Department of Statistics Carnegie Mellon University Abstract Linear mixed-effects (LME)
More informationRandomization as a Tool for Development Economists. Esther Duflo Sendhil Mullainathan BREAD-BIRS Summer school
Randomization as a Tool for Development Economists Esther Duflo Sendhil Mullainathan BREAD-BIRS Summer school Randomization as one solution Suppose you could do a Randomized evaluation of the microcredit
More informationeconstor Make Your Publications Visible.
econstor Make Your Publications Visible. A Service of Wirtschaft Centre zbwleibniz-informationszentrum Economics Bruns, Stephan B. Working Paper The fragility of meta-regression models in observational
More informationF1: Introduction to Econometrics
F1: Introduction to Econometrics Feng Li Department of Statistics, Stockholm University General information Homepage of this course: http://gauss.stat.su.se/gu/ekonometri.shtml Lecturer F1 F7: Feng Li,
More information