Program Evaluations and Randomization. Lecture 5 HSE, Dagmara Celik Katreniak
|
|
- Phebe Owen
- 5 years ago
- Views:
Transcription
1 Program Evaluations and Randomization Lecture 5 HSE, Dagmara Celik Katreniak
2 Overview Treatment Effect under perfect conditions Average Treatment Effect Treatment effect under imperfect conditions Intention To Treat (ITT) Treatment On the Treated (TOT) Local Average Treatment Effect (LATE) Other Externality Effect
3 Average Treatment Effect (ATE) SUTVA assumption Non-interference Being in the treatment or control group does not influence the outcome of anyone No variation in treatment All subjects are subject to the same treatment Unconfoundedness/Ignorability Assignment to treatment not connected with outcome
4 Average Treatment Effect (ATE) Randomly assign subjects to T or C Calculate ATE ATE = measures the effect of treatment on a randomly selected person Randomization not conditional on observables Perfect compliance No attrition SUTVA and unconfoundedness
5 Example Blimpo (AEJ: Applied Microeconomics, 2014) All takeovers, no attrition Randomized at school level T1= Individual target group T2= Team target group T3= Team Tournament group C = Control group sssss i,s = β k=1 β k T i,s k + ε i,s
6 Randomization balance
7
8
9 The conditional probability of selection Stratify your sample By stratas, randomize into T/C The probability to be picked to Treatment depends on observables (stratification) The allocation of T unequal across stratas Possible for example if the number of people in T and C not equal but fixed number of awards in T
10 Stratified sample Checking randomization balance Stratum level But then no overall balance check Problem of too many tests at once Possibly low power cause of small strata size Overall/general level But may not be balanced by stratas F-test by Firpo, Foguel and Borges Jales (conference paper) What if imbalances exist?
11 Example Voucher program in Colombia Students picked to schools by lottery Given number of winners in each city The ratio of winners differs by town Lottery random within each city conditional on whether households have an access to a phone Causal effect of the program on voucher s applicants with access to a telephone in surveyed cities (Angrist et al., 2001)
12 The conditional probability of selection Weighted average over all strata Weight = proportion of treated subjects within strata Regression Control for all strata variables Include all dummies plus their interactions Be careful about the degrees of freedom
13 The conditional probability of selection Conditional Randomization implies: E Y i C X, T E Y i C X, C = 0 E x E Y T i X, T E Y C i X, T = E Y T i x, T E Y C i x, C P X = x T dd If discrete comparison of means using proportion of treatment within cells as weights
14 (Im)Perfect Compliance Subjects from Treatment/Control group without/with treatment Spillover effects Low take up rate Switching of subjects during the experiment What is the question of interest? The effect of the intervention itself? The effect of a diet rich in iron The effect of the instrument? Common for policy relevant research Do we need all participants to participate or we want to see the effect on those who choose to participate?
15 (Im)Perfect Compliance Perfect compliance of interest Prepare design accordingly Example: Thomas, Frankenberg, Friedman, Habicht, & Al (2003) on iron supplementation Partial compliance of interest Esp. encouragement designs Duflo and Saez (2003) Partial compliance as the only option Deworming program (Miguel & Glewwe, 2004) Mostly because tracking not possible
16 TOT=? ITT=?
17 Intention to Treat Estimate (ITT) The effect of offered intervention Differ from the effect of the treatment in case of imperfect compliance Based on the initial division into treatment and control group Perfect compliance ITT=ATE
18 Average Treatment Effect on Treated Estimate (TOT) Y(T) = A PPPPPPPPPPP TTTTTTT T) + B (1 PPPPPPPPPPP TTTTTTT T)) Y C = A PPPPPPPPPPP TTTTTTT C) + B (1 PPPPPPPPPPP TTTTTTT C)) TTT = Y T Y C PPPP TTTTTTT T) PPPP TTTTTTT C)
19 Scenario 1: Example Researcher chooses 100 people into treatment group (he gives them $100) and 100 people into control group (gives them 0) Collects information on rewards All people in T got money No person from C got money ITT=? TOT=? ITT=TOT=$100
20 Example Scenario 2: Researcher tells 100 students if they come to his office, he gives them $100 (= treatment), he says nothing to the control group Cca 20% of students do not trust him/lazy to come Collects information on rewards On average treatment group received $80, control $0 ITT=?, TOT=? ITT=$80 TOT=$100
21 Example Scenario 3: Researcher tells subjects to come at midnight to his office to pick up $100 (treatment), he says nothing to the control group Only 50% of treatment group show up Some 10% of control group students show up Collects information on rewards ITT=?, TOT=? ITT=$40 TOT=$100
22 Example Scenario 4: Researcher gives $25, $50, $75, $100 to four groups of students (each 1/4 in size), $0 to control 30% of treated pick up money, no control Collects information on rewards ITT=?, TOT=? ITT=$21 If everyone came, TOT=$62.5 When 30% came, TOT=$21/0.3=$70
23 ITT versus ATE Two identification assumptions (Imbens and Angrist, 1994): Independence Outcomes are not directly affected by the instrument (Duflo, Glennerster & Kremer, 2007, p.52) Y i C, Y i T, T i 1, T i 0 Z Monotonicity The monotonicity assumption requires that the instrument makes every person either weakly more or less likely to actually participate in the treatment (Duflo, Glennerster & Kremer, 2007, p.52) T i 1 T i 0 fff aaa i or T i 1 T i 0 fff aaa i
24 ITT versus ATE Wald estimate local average treatment effect (LATE) on compliers, i.e. on those whose treatment status was affected by the instrument IV Estimate of β (using Z as an instrument) can be interpreted as the ATE for a well-defined group of individuals, namely those who are induced by the instrument Z to take advantage of the treatment (Duflo, Glennerster & Kremer, 2007, p.53)
25 ITT versus ATE β W = E Y i Z i =1 E Y i Z i =0 E T i Z i =1 E T i Z i =0 β W = E Y i T Y i C T i 1 T i 0 = 1
26 IV estimate (LATE) No person from control group got treatment LATE = TOT All people from treatment group got treatment Oversubscription design Interpretation! Causal effect for compliers Not necessarily the effect for the entire population
27 Example Voucher program in Colombia PACES = governmental program, scholarships for private secondary schools assigned by lottery Lottery winners and losers picked from PACES program applicants Stratification by localities, conditional to access to a phone What effects do they measure? The effect of winning the lottery The effect of scholarship
28 Angrist et al. (2002)
29 Estimates of the effects The effect of winning the lottery on scholarship use, school choice and schooling 6-7% more likely to begin the 6 th grade 15-16% more likely to be in private school Decision to be in private school seems to be sensitive to price, decision to attend the school not Lower repetition rate for winners Higher likelihood to be in 8 th grade for winners Angrist et al. (2002)
30 Estimates of the effects The effect of winning the lottery on test scores Subsample of children invited by phone to be tested + by letter More than one test opportunity if failed Refreshments, after each test other rewards picked by lottery (e.g.bicycle) Travel costs covered NO SIGNIFICANT relationship between being tested and voucher status Angrist et al. (2002)
31 Angrist et al. (2002)
32
33 Estimates of the effects The effect of receiving scholarship Using voucher win/loss status as IV Initial randomization = IV versus Actual treatment 6% (24%) of lottery users used voucher 90% of winners used scholarship Two stage least squares estimation The 2SLS estimates based on this difference are necessarily larger than the reduced form effects of winning the lottery since winning the lottery is only imperfectly correlated with receiving a scholarship. Angrist et al. (2002)
34
35 Spillovers How would you measure them? Vary the exposure to a treatment Duflo and Saez (2003) Use variation in exposure across groups that arises from randomization Duflo, Kremer and Robinson (2006) Random assignment to peer groups
36 Attrition May or may not be a problem Problem if there is correlation between attrition and treatment Biases results Random attrition Lowers statistical power Non-random attrition No independence of outcomes Try to limit ex-ante
37 Attrition Parametric techniques Heckman correction Survival model and attrition Non-parametric techniques Manski-Lee bounds Matching matching lottery winners and losers Angrist et al (2006) Imputation methods
38 Case Study Please, work on the case study
39 Summary Randomization Experimental Designs Sample size and power calculations Budget, CBA and CEA Data Analysis under perfect and imperfect randomization *** Further use of randomization Topics Presentations
Threats and Analysis. Shawn Cole. Harvard Business School
Threats and Analysis Shawn Cole Harvard Business School Course Overview 1. What is Evaluation? 2. Outcomes, Impact, and Indicators 3. Why Randomize? 4. How to Randomize? 5. Sampling and Sample Size 6.
More informationThreats and Analysis. Bruno Crépon J-PAL
Threats and Analysis Bruno Crépon J-PAL Course Overview 1. What is Evaluation? 2. Outcomes, Impact, and Indicators 3. Why Randomize and Common Critiques 4. How to Randomize 5. Sampling and Sample Size
More informationRandomization as a Tool for Development Economists. Esther Duflo Sendhil Mullainathan BREAD-BIRS Summer school
Randomization as a Tool for Development Economists Esther Duflo Sendhil Mullainathan BREAD-BIRS Summer school Randomization as one solution Suppose you could do a Randomized evaluation of the microcredit
More informationWhat can go wrong.and how to fix it!
What can go wrong.and how to fix it! Megha Pradhan Policy and Training Manager, J-PAL South Asia Kathmandu, Nepal 29 March 2017 Introduction Conception phase is important and allows to design an evaluation
More informationGroup Work Instructions
Evaluating Social Programs Executive Education at the Abdul Latif Jameel Poverty Action Lab June 18, 2007 June 22, 2007 Group Work Instructions You will be assigned to groups of 5-6 people. We will do
More informationGlossary From Running Randomized Evaluations: A Practical Guide, by Rachel Glennerster and Kudzai Takavarasha
Glossary From Running Randomized Evaluations: A Practical Guide, by Rachel Glennerster and Kudzai Takavarasha attrition: When data are missing because we are unable to measure the outcomes of some of the
More informationHow to Randomise? (Randomisation Design)
J-PAL Africa Executive Education Course How to Randomise? (Randomisation Design) Lecture 4, January 24, 2012 Martin Abel University of Cape Town Roadmap to Randomised Evaluations 1 Environment / Context
More informationNBER TECHNICAL WORKING PAPER SERIES USING RAMDOMIZATION IN DEVELOPMENT ECONOMICS RESEARCH: A TOOLKIT. Esther Duflo Rachel Glennerster Michael Kremer
NBER TECHNICAL WORKING PAPER SERIES USING RAMDOMIZATION IN DEVELOPMENT ECONOMICS RESEARCH: A TOOLKIT Esther Duflo Rachel Glennerster Michael Kremer Technical Working Paper 333 http://www.nber.org/papers/t0333
More informationComplier Average Causal Effect (CACE)
Complier Average Causal Effect (CACE) Booil Jo Stanford University Methodological Advancement Meeting Innovative Directions in Estimating Impact Office of Planning, Research & Evaluation Administration
More informationDISCUSSION PAPER SERIES. No USING RANDOMIZATION IN DEVELOPMENT ECONOMICS RESEARCH: A TOOLKIT
DISCUSSION PAPER SERIES No. 6059 USING RANDOMIZATION IN DEVELOPMENT ECONOMICS RESEARCH: A TOOLKIT Esther Duflo, Rachel Glennerster and Michael Kremer DEVELOPMENT ECONOMICS ABCD www.cepr.org Available online
More informationEvaluating Social Programs Course: Evaluation Glossary (Sources: 3ie and The World Bank)
Evaluating Social Programs Course: Evaluation Glossary (Sources: 3ie and The World Bank) Attribution The extent to which the observed change in outcome is the result of the intervention, having allowed
More informationMIT LIBRARIES. MiltmilWv I! ''.'''' ;i ill:i!.(;!!:i i'.ir-il:>!\! ItllillPlli. ' iilpbilisil HB31 .M415. no AHHH
' MIT LIBRARIES 3 9080 02618 0981 MiltmilWv I! ''.'''' ;i ill:i!.(;!!:i i'.ir-il:>!\! ItllillPlli v ' iilpbilisil HB31.M415 no.06-36 2006 11 AHHH Digitized by the Internet Archive in 2011 with funding
More informationEMPIRICAL STRATEGIES IN LABOUR ECONOMICS
EMPIRICAL STRATEGIES IN LABOUR ECONOMICS University of Minho J. Angrist NIPE Summer School June 2009 This course covers core econometric ideas and widely used empirical modeling strategies. The main theoretical
More informationInstrumental Variables I (cont.)
Review Instrumental Variables Observational Studies Cross Sectional Regressions Omitted Variables, Reverse causation Randomized Control Trials Difference in Difference Time invariant omitted variables
More informationWhat is: regression discontinuity design?
What is: regression discontinuity design? Mike Brewer University of Essex and Institute for Fiscal Studies Part of Programme Evaluation for Policy Analysis (PEPA), a Node of the NCRM Regression discontinuity
More informationTHREATS TO VALIDITY. Presentation by: Raul Sanchez de la Sierra
THREATS TO VALIDITY Presentation by: Raul Sanchez de la Sierra What I want you to avoid Captain, sorry, I forgot the spillovers! Lecture Overview External Validity Attrition Spillovers Partial Compliance
More informationG , G , G MHRN
Estimation of treatment-effects from randomized controlled trials in the presence non-compliance with randomized treatment allocation Graham Dunn University of Manchester Research funded by: MRC Methodology
More informationApplied Econometrics for Development: Experiments II
TSE 16th January 2019 Applied Econometrics for Development: Experiments II Ana GAZMURI Paul SEABRIGHT The Cohen-Dupas bednets study The question: does subsidizing insecticide-treated anti-malarial bednets
More informationBrief introduction to instrumental variables. IV Workshop, Bristol, Miguel A. Hernán Department of Epidemiology Harvard School of Public Health
Brief introduction to instrumental variables IV Workshop, Bristol, 2008 Miguel A. Hernán Department of Epidemiology Harvard School of Public Health Goal: To consistently estimate the average causal effect
More informationInstrumental Variables Estimation: An Introduction
Instrumental Variables Estimation: An Introduction Susan L. Ettner, Ph.D. Professor Division of General Internal Medicine and Health Services Research, UCLA The Problem The Problem Suppose you wish to
More informationPlanning Sample Size for Randomized Evaluations.
Planning Sample Size for Randomized Evaluations www.povertyactionlab.org Planning Sample Size for Randomized Evaluations General question: How large does the sample need to be to credibly detect a given
More informationVersion No. 7 Date: July Please send comments or suggestions on this glossary to
Impact Evaluation Glossary Version No. 7 Date: July 2012 Please send comments or suggestions on this glossary to 3ie@3ieimpact.org. Recommended citation: 3ie (2012) 3ie impact evaluation glossary. International
More informationDoes AIDS Treatment Stimulate Negative Behavioral Response? A Field Experiment in South Africa
Does AIDS Treatment Stimulate Negative Behavioral Response? A Field Experiment in South Africa Plamen Nikolov Harvard University February 19, 2011 Plamen Nikolov (Harvard University) Behavioral Response
More informationDemystifying causal inference in randomised trials
Demystifying causal inference in randomised trials Ian White 1, Sabine Landau 2, Graham Dunn 3,4 and Richard Emsley 3,4 1 MRC Biostatistics Unit Cambridge 2 Department of Biostatistics, Institute of Psychiatry,
More informationEvaluating the Impact of Health Programmes
Evaluating the Impact of Health Programmes Justine Burns, Malcolm Keswell, and Rebecca Thornton April 13, 2009 Abstract This paper has two broad objectives. The first objective is broadly methodological
More informationStatistical Power Sampling Design and sample Size Determination
Statistical Power Sampling Design and sample Size Determination Deo-Gracias HOUNDOLO Impact Evaluation Specialist dhoundolo@3ieimpact.org Outline 1. Sampling basics 2. What do evaluators do? 3. Statistical
More informationSupplementary Appendix
Supplementary Appendix This appendix has been provided by the authors to give readers additional information about their work. Supplement to: Howard R, McShane R, Lindesay J, et al. Donepezil and memantine
More informationMethods for Addressing Selection Bias in Observational Studies
Methods for Addressing Selection Bias in Observational Studies Susan L. Ettner, Ph.D. Professor Division of General Internal Medicine and Health Services Research, UCLA What is Selection Bias? In the regression
More informationTitle:Bounding the Per-Protocol Effect in Randomized Trials: An Application to Colorectal Cancer Screening
Author's response to reviews Title:Bounding the Per-Protocol Effect in Randomized Trials: An Application to Colorectal Cancer Screening Authors: Sonja A Swanson (sswanson@hsph.harvard.edu) Oyvind Holme
More informationThe Stable Unit Treatment Value Assumption (SUTVA) and Its Implications for Social Science RCTs
The Stable Unit Treatment Value Assumption (SUTVA) and Its Implications for Social Science RCTs Alan S. Gerber & Donald P. Green Yale University From Chapter 8 of Field Experimentation: Design, Analysis,
More informationA NEW TRIAL DESIGN FULLY INTEGRATING BIOMARKER INFORMATION FOR THE EVALUATION OF TREATMENT-EFFECT MECHANISMS IN PERSONALISED MEDICINE
A NEW TRIAL DESIGN FULLY INTEGRATING BIOMARKER INFORMATION FOR THE EVALUATION OF TREATMENT-EFFECT MECHANISMS IN PERSONALISED MEDICINE Dr Richard Emsley Centre for Biostatistics, Institute of Population
More informationChapter 3. Producing Data
Chapter 3. Producing Data Introduction Mostly data are collected for a specific purpose of answering certain questions. For example, Is smoking related to lung cancer? Is use of hand-held cell phones associated
More informationCHL 5225 H Advanced Statistical Methods for Clinical Trials. CHL 5225 H The Language of Clinical Trials
CHL 5225 H Advanced Statistical Methods for Clinical Trials Two sources for course material 1. Electronic blackboard required readings 2. www.andywillan.com/chl5225h code of conduct course outline schedule
More informationSection 1: The Science of Clinical Investigation
INTRODUCTION TO CLINICAL RESEARCH Scientific Concepts for Clinical Research Scott L. Zeger, Ph.D. July 14, 2014 Section 1: The Science of Clinical Investigation 1. Platonic model: science as the search
More informationRandomized Experiments with Noncompliance. David Madigan
Randomized Experiments with Noncompliance David Madigan Introduction Noncompliance is an important problem in randomized experiments involving humans Includes e.g. switching subjects to standard therapy
More information1. Introduction Consider a government contemplating the implementation of a training (or other social assistance) program. The decision to implement t
1. Introduction Consider a government contemplating the implementation of a training (or other social assistance) program. The decision to implement the program depends on the assessment of its likely
More informationGreg Pond Ph.D., P.Stat.
Statistics for Clinical Trials: Basics of a Phase III Trial Design Greg Pond Ph.D., P.Stat. Ontario Clinical Oncology Group Escarpment Cancer Research Institute Department of Oncology, McMaster University
More informationSAMPLING AND SAMPLE SIZE
SAMPLING AND SAMPLE SIZE Andrew Zeitlin Georgetown University and IGC Rwanda With slides from Ben Olken and the World Bank s Development Impact Evaluation Initiative 2 Review We want to learn how a program
More informationEconomics 270c. Development Economics. Lecture 9 March 14, 2007
Economics 270c Development Economics Lecture 9 March 14, 2007 Lecture 1: Global patterns of economic growth and development (1/16) The political economy of development Lecture 2: Inequality and growth
More informationIntroduction to Program Evaluation
Introduction to Program Evaluation Nirav Mehta Assistant Professor Economics Department University of Western Ontario January 22, 2014 Mehta (UWO) Program Evaluation January 22, 2014 1 / 28 What is Program
More information14.11 Lecture 2: Public Health Messages How to Convince Others to Adopt Healthy Behaviors?
14.11 Lecture 2: Public Health Messages How to Convince Others to Adopt Healthy Behaviors? Prof. Esther Du o February 14, 2006 In this lecture, we are going to examine various ways which have been proposed
More informationWorkshop on Experiments in Political Economy May Columbia Center for the Study of Development Strategies & the Harriman Institute
Workshop on Experiments in Political Economy 18-20 May 2011 Columbia Center for the Study of Development Strategies & the Harriman Institute Contents Preface v 1 Causal Inference I: The Fundamental Problem
More informationCausal Validity Considerations for Including High Quality Non-Experimental Evidence in Systematic Reviews
Non-Experimental Evidence in Systematic Reviews OPRE REPORT #2018-63 DEKE, MATHEMATICA POLICY RESEARCH JUNE 2018 OVERVIEW Federally funded systematic reviews of research evidence play a central role in
More informationTrial designs fully integrating biomarker information for the evaluation of treatment-effect mechanisms in stratified medicine
Trial designs fully integrating biomarker information for the evaluation of treatment-effect mechanisms in stratified medicine Dr Richard Emsley Centre for Biostatistics, Institute of Population Health,
More informationLecture II: Difference in Difference. Causality is difficult to Show from cross
Review Lecture II: Regression Discontinuity and Difference in Difference From Lecture I Causality is difficult to Show from cross sectional observational studies What caused what? X caused Y, Y caused
More informationIntroduction to Observational Studies. Jane Pinelis
Introduction to Observational Studies Jane Pinelis 22 March 2018 Outline Motivating example Observational studies vs. randomized experiments Observational studies: basics Some adjustment strategies Matching
More informationAnalysis of TB prevalence surveys
Workshop and training course on TB prevalence surveys with a focus on field operations Analysis of TB prevalence surveys Day 8 Thursday, 4 August 2011 Phnom Penh Babis Sismanidis with acknowledgements
More informationTRACER STUDIES ASSESSMENTS AND EVALUATIONS
TRACER STUDIES ASSESSMENTS AND EVALUATIONS 1 INTRODUCTION This note introduces the reader to tracer studies. For the Let s Work initiative, tracer studies are proposed to track and record or evaluate the
More informationMethods of Randomization Lupe Bedoya. Development Impact Evaluation Field Coordinator Training Washington, DC April 22-25, 2013
Methods of Randomization Lupe Bedoya Development Impact Evaluation Field Coordinator Training Washington, DC April 22-25, 2013 Content 1. Important Concepts 2. What vs. Why 3. Some Practical Issues 4.
More informationAP Statistics Exam Review: Strand 2: Sampling and Experimentation Date:
AP Statistics NAME: Exam Review: Strand 2: Sampling and Experimentation Date: Block: II. Sampling and Experimentation: Planning and conducting a study (10%-15%) Data must be collected according to a well-developed
More informationGUIDE 4: COUNSELING THE UNEMPLOYED
GUIDE 4: COUNSELING THE UNEMPLOYED Addressing threats to experimental integrity This case study is based on Sample Attrition Bias in Randomized Experiments: A Tale of Two Surveys By Luc Behaghel, Bruno
More informationCausal Interaction and External Validity: Obstacles to the Policy Relevance of Randomized Evaluations
Causal Interaction and External Validity: Obstacles to the Policy Relevance of Randomized Evaluations Seán M. Muller The ability to generalize effects estimated from randomized experiments is critical
More informationEmpirical Strategies 2012: IV and RD Go to School
Empirical Strategies 2012: IV and RD Go to School Joshua Angrist Rome June 2012 These lectures cover program and policy evaluation strategies discussed in Mostly Harmless Econometrics (MHE). We focus on
More informationBeyond the intention-to treat effect: Per-protocol effects in randomized trials
Beyond the intention-to treat effect: Per-protocol effects in randomized trials Miguel Hernán DEPARTMENTS OF EPIDEMIOLOGY AND BIOSTATISTICS Intention-to-treat analysis (estimator) estimates intention-to-treat
More informationTRANSLATING RESEARCH INTO ACTION. Why randomize? Dan Levy. Harvard Kennedy School
TRANSLATING RESEARCH INTO ACTION Why randomize? Dan Levy Harvard Kennedy School Your background Course Overview 1. What is evaluation? 2. Measuring impacts (outcomes, indicators) 3. Why randomize? 4. How
More informationHigher Psychology RESEARCH REVISION
Higher Psychology RESEARCH REVISION 1 The biggest change from the old Higher course (up to 2014) is the possibility of an analysis and evaluation question (8-10) marks asking you to comment on aspects
More informationDichotomizing partial compliance and increased participant burden in factorial designs: the performance of four noncompliance methods
Merrill and McClure Trials (2015) 16:523 DOI 1186/s13063-015-1044-z TRIALS RESEARCH Open Access Dichotomizing partial compliance and increased participant burden in factorial designs: the performance of
More informationReflection Questions for Math 58B
Reflection Questions for Math 58B Johanna Hardin Spring 2017 Chapter 1, Section 1 binomial probabilities 1. What is a p-value? 2. What is the difference between a one- and two-sided hypothesis? 3. What
More informationIncentives versus sorting in tournaments: Evidence from a field experiment forthcoming Journal of Labor Economics
Incentives versus sorting in tournaments: Evidence from a field experiment forthcoming Journal of Labor Economics Edwin Leuven, CREST (ENSAE) Hessel Oosterbeek, University of Amsterdam and Tinbergen Institute
More informationEstimating average treatment effects from observational data using teffects
Estimating average treatment effects from observational data using teffects David M. Drukker Director of Econometrics Stata 2013 Nordic and Baltic Stata Users Group meeting Karolinska Institutet September
More informationTechnical Track Session IV Instrumental Variables
Impact Evaluation Technical Track Session IV Instrumental Variables Christel Vermeersch Beijing, China, 2009 Human Development Human Network Development Network Middle East and North Africa Region World
More informationSampling. (James Madison University) January 9, / 13
Sampling The population is the entire group of individuals about which we want information. A sample is a part of the population from which we actually collect information. A sampling design describes
More informationEmpirical Strategies
Empirical Strategies Joshua Angrist University of Lausanne August 2013 These lectures cover program and policy evaluation strategies discussed in Mostly Harmless Econometrics (MHE). The main theoretical
More informationLecture Slides. Elementary Statistics Eleventh Edition. by Mario F. Triola. and the Triola Statistics Series 1.1-1
Lecture Slides Elementary Statistics Eleventh Edition and the Triola Statistics Series by Mario F. Triola 1.1-1 Chapter 1 Introduction to Statistics 1-1 Review and Preview 1-2 Statistical Thinking 1-3
More informationImpact Evaluation Methods: Why Randomize? Meghan Mahoney Policy Manager, J-PAL Global
Impact Evaluation Methods: Why Randomize? Meghan Mahoney Policy Manager, J-PAL Global Course Overview 1. What is Evaluation? 2. Outcomes, Impact, and Indicators 3. Why Randomize? 4. How to Randomize? 5.
More informationIssues of Non-Compliance and Their Effect on Validity in Field Experiments - A case study of the field experiment Taxis and Contracts
Issues of Non-Compliance and Their Effect on Validity in Field Experiments - A case study of the field experiment Taxis and Contracts Department of Economics Uppsala University Bachelor Thesis Author:
More information14.74 Foundations of Development Policy Spring 2009
MIT OpenCourseWare http://ocw.mit.edu 14.74 Foundations of Development Policy Spring 2009 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms. 14.74 Lecture
More informationPractitioner s Guide To Stratified Random Sampling: Part 1
Practitioner s Guide To Stratified Random Sampling: Part 1 By Brian Kriegler November 30, 2018, 3:53 PM EST This is the first of two articles on stratified random sampling. In the first article, I discuss
More informationPrediction, Causation, and Interpretation in Social Science. Duncan Watts Microsoft Research
Prediction, Causation, and Interpretation in Social Science Duncan Watts Microsoft Research Explanation in Social Science: Causation or Interpretation? When social scientists talk about explanation they
More informationVocabulary. Bias. Blinding. Block. Cluster sample
Bias Blinding Block Census Cluster sample Confounding Control group Convenience sample Designs Experiment Experimental units Factor Level Any systematic failure of a sampling method to represent its population
More informationUniversity student sexual assault and sexual harassment survey. Notes on reading institutional-level data
University student sexual assault and sexual harassment survey Notes on reading institutional-level data This is a note on reading the institutional-level data from the Australian Human Rights Commission
More informationQuasi-experimental analysis Notes for "Structural modelling".
Quasi-experimental analysis Notes for "Structural modelling". Martin Browning Department of Economics, University of Oxford Revised, February 3 2012 1 Quasi-experimental analysis. 1.1 Modelling using quasi-experiments.
More informationRandomized experiments are increasingly used in development economics.
American Economic Journal: Applied Economics 9, 1:4, 3 http://www.aeaweb.org/articles.php?doi=1.157/app.1.4. In Pursuit of Balance: Randomization in Practice in Development Field Experiments By Miriam
More informationRisk of bias assessment for. statistical methods
Risk of bias assessment for experimental and quasiexperimental designs based on statistical methods Hugh Waddington & Jorge Garcia Hombrados The haphazard way we individually and collectively study the
More informationEmpirical Strategies
Empirical Strategies Joshua Angrist BGPE March 2012 These lectures cover many of the empirical modeling strategies discussed in Mostly Harmless Econometrics (MHE). The main theoretical ideas are illustrated
More informationEstimating drug effects in the presence of placebo response: Causal inference using growth mixture modeling
STATISTICS IN MEDICINE Statist. Med. 2009; 28:3363 3385 Published online 3 September 2009 in Wiley InterScience (www.interscience.wiley.com).3721 Estimating drug effects in the presence of placebo response:
More informationRegression Discontinuity Designs: An Approach to Causal Inference Using Observational Data
Regression Discontinuity Designs: An Approach to Causal Inference Using Observational Data Aidan O Keeffe Department of Statistical Science University College London 18th September 2014 Aidan O Keeffe
More informationRandomized Controlled Trial
Randomized Controlled Trial Training Course in Sexual and Reproductive Health Research Geneva 2016 Dr Khalifa Elmusharaf MBBS, PgDip, FRSPH, PHD Senior Lecturer in Public Health Graduate Entry Medical
More informationChapter 13 Summary Experiments and Observational Studies
Chapter 13 Summary Experiments and Observational Studies What have we learned? We can recognize sample surveys, observational studies, and randomized comparative experiments. o These methods collect data
More informationTitle: Intention-to-treat and transparency of related practices in randomized, controlled trials of anti-infectives
Author s response to reviews Title: Intention-to-treat and transparency of related practices in randomized, controlled trials of anti-infectives Authors: Robert Beckett (rdbeckett@manchester.edu) Kathryn
More informationImpact Evaluation Toolbox
Impact Evaluation Toolbox Gautam Rao University of California, Berkeley * ** Presentation credit: Temina Madon Impact Evaluation 1) The final outcomes we care about - Identify and measure them Measuring
More informationInferential Statistics
Inferential Statistics and t - tests ScWk 242 Session 9 Slides Inferential Statistics Ø Inferential statistics are used to test hypotheses about the relationship between the independent and the dependent
More informationSchool Autonomy and Regression Discontinuity Imbalance
School Autonomy and Regression Discontinuity Imbalance Todd Kawakita 1 and Colin Sullivan 2 Abstract In this research note, we replicate and assess Damon Clark s (2009) analysis of school autonomy reform
More informationIntroduction to NHANES and NAMCS
Introduction to NHANES and NAMCS CMU Summer Institute 2002 NHANES: National Health and Nutrition Examination Survey Home Page: http://www.cdc.gov/nchs/nhanes.htm Goal: To provide national estimates of
More informationChapter 13. Experiments and Observational Studies. Copyright 2012, 2008, 2005 Pearson Education, Inc.
Chapter 13 Experiments and Observational Studies Copyright 2012, 2008, 2005 Pearson Education, Inc. Observational Studies In an observational study, researchers don t assign choices; they simply observe
More informationThe Fallacy of Taking Random Supplements
The Fallacy of Taking Random Supplements Healthview interview with Dr. Paul Eck Healthview: We can see from our conversations that you are totally against people taking random supplements even if people
More informationWORKING PAPER SERIES
WORKING PAPER SERIES Evaluating School Vouchers: Evidence from a Within-Study Kaitlin P. Anderson Patrick J. Wolf, Ph.D. April 3, 2017 EDRE Working Paper 2017-10 The University of Arkansas, Department
More informationTRANSLATING RESEARCH INTO ACTION
TRANSLATING RESEARCH INTO ACTION This case study is based on a current study by Esther Duflo and Tavneet Suri. J-PAL thanks the authors for allowing us to use their project 17 Key Vocabulary 1. Equivalence:
More informationApplied Quantitative Methods II
Applied Quantitative Methods II Lecture 7: Endogeneity and IVs Klára Kaĺıšková Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 1 / 36 Outline 1 OLS and the treatment effect 2 OLS and endogeneity 3 Dealing
More informationWelcome to this series focused on sources of bias in epidemiologic studies. In this first module, I will provide a general overview of bias.
Welcome to this series focused on sources of bias in epidemiologic studies. In this first module, I will provide a general overview of bias. In the second module, we will focus on selection bias and in
More informationWhy do Psychologists Perform Research?
PSY 102 1 PSY 102 Understanding and Thinking Critically About Psychological Research Thinking critically about research means knowing the right questions to ask to assess the validity or accuracy of a
More informationEconomics 270c Graduate Development Economics. Professor Ted Miguel Department of Economics University of California, Berkeley
Economics 270c Graduate Development Economics Professor Ted Miguel Department of Economics University of California, Berkeley Economics 270c Graduate Development Economics Lecture 12 April 14, 2009 Macroeconomic
More informationEPI 200C Final, June 4 th, 2009 This exam includes 24 questions.
Greenland/Arah, Epi 200C Sp 2000 1 of 6 EPI 200C Final, June 4 th, 2009 This exam includes 24 questions. INSTRUCTIONS: Write all answers on the answer sheets supplied; PRINT YOUR NAME and STUDENT ID NUMBER
More informationThis article appeared in a journal published by Elsevier. The attached copy is furnished to the author for internal non-commercial research and
This article appeared in a journal published by Elsevier. The attached copy is furnished to the author for internal non-commercial research and education use, including for instruction at the authors institution
More informationMATH-134. Experimental Design
Experimental Design Controlled Experiment: Researchers assign treatment and control groups and examine any resulting changes in the response variable. (cause-and-effect conclusion) Observational Study:
More informationPublic Policy & Evidence:
Public Policy & Evidence: How to discriminate, interpret and communicate scientific research to better inform society. Rachel Glennerster Executive Director J-PAL Global Press coverage of microcredit:
More informationCausal Effect Heterogeneity
Causal Effect Heterogeneity Jennie E. Brand Juli Simon-Thomas PWP-CCPR-2011-012 Latest Revised: March 30, 2012 California Center for Population Research On-Line Working Paper Series CAUSAL EFFECT HETEROGENEITY
More informationConfounding by indication developments in matching, and instrumental variable methods. Richard Grieve London School of Hygiene and Tropical Medicine
Confounding by indication developments in matching, and instrumental variable methods Richard Grieve London School of Hygiene and Tropical Medicine 1 Outline 1. Causal inference and confounding 2. Genetic
More information