Introduction to Program Evaluation
|
|
- Clifford Goodwin
- 6 years ago
- Views:
Transcription
1 Introduction to Program Evaluation Nirav Mehta Assistant Professor Economics Department University of Western Ontario January 22, 2014 Mehta (UWO) Program Evaluation January 22, / 28
2 What is Program Evaluation? Using statistics to determine the effect of a treatment on an outcome (or outcomes) of interest. What is a treatment? It can be: a policy: Introducing school choice into a public school district an individual decision: Attending university for one year Finishing university Eating a burrito Two ways of recovering the effect of a treatment Experimental: Randomization of treatment Use observational data and a combination of statistical and behavioral assumptions Mehta (UWO) Program Evaluation January 22, / 28
3 My perspective I am currently working on projects in: the Economics of Education How school choice affects student achievement The effect of ability tracking on student achievement Health Economics The design of optimal physician incentive schemes Mehta (UWO) Program Evaluation January 22, / 28
4 What can we use program evaluation for? Three types of analyses: Retrospective: How did introducing a school choice program affect student achievement? Prospective: How would introducing a school choice program that has already been implemented on Group A affect student achievement for students in Group B? Prospective: How would a school choice program, which has never been implemented, affect student achievement for students in Group A or Group B or anyone else? Using retrospective analyses to prospectively evaluate programs requires extrapolation (i.e. additional assumptions). Mehta (UWO) Program Evaluation January 22, / 28
5 Leading example There is a public school district with one public school. A new public school enters the district. How does attending the new school affect student achievement? Mehta (UWO) Program Evaluation January 22, / 28
6 A little notation to fix ideas Individual i, time t Observed characteristics X it (household income, parental education,...) Unobserved characteristics ɛ it (motivation, ability, waking up on the right/wrong side of the bed,...) Treatment status D Dit = 0 means i didn t have the treatment at time t Dit = 1 means i had the treatment at time t Outcome Y is a function of individual characteristics and treatment status (score on standardized test or probability of graduating high school) Y (X, ɛ, D) Treatment effect it Y (X it, ɛ it, 1) Y (X it, ɛ it, 0) combination of behavioral responses and input changes Mehta (UWO) Program Evaluation January 22, / 28
7 Treatment effects There is in general a distribution of treatment effects. Put another way, there s no reason to expect that i = for all people. The effect of being in a new school not only reflects the potentially different characteristics of those students. Also incorporates behavioral responses that can affect a student s learning. For example, parents might help their child more or less when their child is in a particular school. These responses could also depend on student characteristics: the amount and efficacy of parent help on home may depend on parental education. Generalizing our findings to other students or another school requires us to make assumptions about these behavioral responses. Mehta (UWO) Program Evaluation January 22, / 28
8 Leading example First question: What is the treatment? We will focus on the students attending the new school for this talk. Note: We could also see what the effect of attending the old school when the new school enters (spillover effect of competition) is! Second question: Which summary of the treatment effect? Focus on average for today. Third question: Which students? (average for whom?) students attending the new school TT: Treatment on the treated students attending the old school TU: Treatment on the untreated all students who attended the old school last year? ATE: Average treatment effect Mehta (UWO) Program Evaluation January 22, / 28
9 Interpreting averages Most researchers focus on the average effect of a program on some subgroup of the population. Although convenient, this almost never innocuous! A small, positive, average treatment effect could be consistent with a small improvement for most people. a very large, positive change for some people. e.g. the worst students learn how to read e.g. the best students get into their super top choice university a very large, positive change for some and a large, negative change for some other people! Mehta (UWO) Program Evaluation January 22, / 28
10 Why is program evaluation hard? Look at student i, who attended the old public school in t 1 but then switched to the new public school in year t. D Outcome D Outcome t 1 0 Y (X i,t 1, ɛ i,t 1, 0) t 0 Y (X it, ɛ it, 0) 1 Y (X it, ɛ it, 1) Missing data problem: The object of interest is it Y (X it, ɛ it, 1) - Y (X it, ɛ it, 0) Y (X it, ɛ it, 0) is a counterfactual outcome We can t observe outcomes under both treatment conditions Therefore, we need to find a valid comparison group. Mehta (UWO) Program Evaluation January 22, / 28
11 Counterfactual outcomes Our definition of the treatment effect, and the summary of the treatment effect we re interested in (i.e. the average for some group of students) provide criteria for a comparison group. We observe Y (X it, ɛ it, 1). We need to come up with Y (X it, ɛ it, 0). Mehta (UWO) Program Evaluation January 22, / 28
12 Counterfactual outcomes In the language of our model, someone with the same observable characteristics (X ) and the same unobservable characteristics (ɛ) who did not participate in the treatment (D = 0) would suffice, given no further assumptions on Y. What if treatment status was related to unobservable characteristics, e.g. more motivated students are more likely to enroll in a new, demanding school. Many methods try to make people comparable across these unobservable characteristics. More on this later. Mehta (UWO) Program Evaluation January 22, / 28
13 Strategies for program evaluation Trade-off: The more (or stronger) assumptions you make, the more you can extrapolate. Economists, and other social scientists, have used the following: 1. Randomized control trial 2. Cross sectional comparisons 3. Fixed effects 4. Fixed effects and common time trend (Difference in differences) 5. Multivariate regression 6. Matching (e.g. on propensity scores) 7. Regression discontinuity design 8. Instrumental variables 9. Structural estimation Mehta (UWO) Program Evaluation January 22, / 28
14 1: Randomized control trial Say we randomly assigned attendance at the new school amongst all students at the old school. This is like having two subgroups of students who had the same distribution of (X, ɛ), but different treatment statuses D. We can recover the average for students on those subgroups by taking the average difference in outcomes between the two groups! Mehta (UWO) Program Evaluation January 22, / 28
15 Interpreting results from randomized control trials People like to say that RCTs are the gold standard for evaluating programs. The monetary gold standard is an obsolete relic that people like talking about all the time. So, I agree. If we are unwilling to assume that all students would be affected in exactly the same manner we have to make more assumptions to make use of findings from RCTs. Mehta (UWO) Program Evaluation January 22, / 28
16 Problems with RCTs They are retrospective. They are expensive. I don t have enough grant money or time to conduct RCTs every time I want to study something new. It s hard to generalize findings from one RCT to another if treatment effects are heterogeneous. We need further structure to understand the results. What is it about the new school that resulted in those amazing outcomes, and is replicable? Similar to generalizability. Will the next new school be exactly the same? This could be a HUGE deal. Education interventions, found effective through RCTs, often don t scale up. If we put further assumptions on Y, we start down the path of using observational (think: survey or administrative) data. Mehta (UWO) Program Evaluation January 22, / 28
17 Methods using observational data Mehta (UWO) Program Evaluation January 22, / 28
18 2: Cross-section Let s invoke the commonly used additively separable framework: Y it = X it β + it D it + ɛ it and assume that the effect of treatment is constant Y it = X it β + D it + ɛ it The inferential problem is that the random variable D it may be correlated with ɛ it. If highly motivated students (large, positive ɛ) were also the ones who switched to the new school, we might overstate the effect of attending the new school. Therefore, comparing people who received the treatment with those who did not may bias our estimate of. Mehta (UWO) Program Evaluation January 22, / 28
19 3: Fixed effects Assumption: What if motivation were constant over time? Y it = X it β + D it + α i + η t + µ it }{{} ɛ it We could then difference outcome equation within each student, over time. Run regression on differenced data. The year where there s a switch will identify. If a student switched because they were even more motivated in year t, we d have a problem! Mehta (UWO) Program Evaluation January 22, / 28
20 4: Difference in differences What is the new school was also introduced when there was a common, unobserved, shock? Say the new school entered because the district is in turmoil, which lowers achievement. Take two students, in the same district, one of whom had the treatment and the other who did not have the treatment. Take the difference between their differences! This gets rid of both α and η. Run regression on differenced data ( Diff in diff ) Mehta (UWO) Program Evaluation January 22, / 28
21 Condition on observables: 5. Regression and 6. Matching Include as many variables as you can in the linear regression. Hope you capture offending terms in ɛ. This is the same in principle as matching, basically find people with X as similar as possible. Just making an assumption about how the X enter the outcome equation. Both of these can look like data mining. beneath social scientists statistical issues, as well Mehta (UWO) Program Evaluation January 22, / 28
22 Local methods: Thinking inside the box The inferential problem was that we didn t know whether the distribution of ɛ was the same for students who attend the new school and attend the old school after the new school has entered. Sometimes, policymakers design programs that are assigned to people on only one side of a cutoff. If we can see the variable used in calculating group membership, we can form a local comparison group. Mehta (UWO) Program Evaluation January 22, / 28
23 7: Regression discontinuity design Outcome Delta(x) Index x Mehta (UWO) Program Evaluation January 22, / 28
24 7: Regression discontinuity design Outcome Treatment effect Index Index Mehta (UWO) Program Evaluation January 22, / 28
25 8: Instrumental variables Similar idea underlying instrumental variables: Find something that toggles treatment status without otherwise affecting outcome. Similarly, instrumental variables tell us about treatment effects for only a subgroup of the population! Those whose treatment status is affected by the instrument More generally, we can model the selection process and use a control function to solve the inferential problem. While we re modeling selection, why not just go all the way? Mehta (UWO) Program Evaluation January 22, / 28
26 9: Structural estimation Specify outcomes as the result of optimization problems. In the leading example, write down a student s utility from attending the old school and the new one, in terms of outcome of interest, which may depend on other choices like effort other factors, like distance between the two schools We then use data to estimate parameters of developed economic model that we can assume to be policy-invariant. For this we typically use different assumptions than other methods. Assumptions commonly grounded in theory (mine grounded in economic theory). We can use the estimated model parameters to then extrapolate to situations that haven t yet happened: Ex ante policy evaluation. Mehta (UWO) Program Evaluation January 22, / 28
27 Conclusion We talked about some commonly used methods to evaluate the effects of programs. Takeaways: 1. There almost always exists a set of assumptions under which a statistical model returns an estimate of the treatment effect. 2. How plausible are those assumptions? We need to go beyond statistics. 3. All methods for program evaluation involve assumptions! 4. Interpretation of : it s a combination of agent input choices and equilibrium responses. It may not be policy invariant! 5. It s imperative to understand the implications of the mathematical models we use before we run them. Mehta (UWO) Program Evaluation January 22, / 28
28 Suggested readings See Petra Todd s lecture notes for a more formal treatment: World Bank book on impact evaluation: worldbank.org/external/default/wdscontentserver/wdsp/ IB/2009/12/10/ _ /Rendered/PDF/ PUB0EPI1101Official0Use0Only1.pdf Mehta (UWO) Program Evaluation January 22, / 28
Version No. 7 Date: July Please send comments or suggestions on this glossary to
Impact Evaluation Glossary Version No. 7 Date: July 2012 Please send comments or suggestions on this glossary to 3ie@3ieimpact.org. Recommended citation: 3ie (2012) 3ie impact evaluation glossary. International
More informationGlossary From Running Randomized Evaluations: A Practical Guide, by Rachel Glennerster and Kudzai Takavarasha
Glossary From Running Randomized Evaluations: A Practical Guide, by Rachel Glennerster and Kudzai Takavarasha attrition: When data are missing because we are unable to measure the outcomes of some of the
More informationLecture II: Difference in Difference. Causality is difficult to Show from cross
Review Lecture II: Regression Discontinuity and Difference in Difference From Lecture I Causality is difficult to Show from cross sectional observational studies What caused what? X caused Y, Y caused
More informationLecture II: Difference in Difference and Regression Discontinuity
Review Lecture II: Difference in Difference and Regression Discontinuity it From Lecture I Causality is difficult to Show from cross sectional observational studies What caused what? X caused Y, Y caused
More informationImpact Evaluation Toolbox
Impact Evaluation Toolbox Gautam Rao University of California, Berkeley * ** Presentation credit: Temina Madon Impact Evaluation 1) The final outcomes we care about - Identify and measure them Measuring
More informationEvaluating Social Programs Course: Evaluation Glossary (Sources: 3ie and The World Bank)
Evaluating Social Programs Course: Evaluation Glossary (Sources: 3ie and The World Bank) Attribution The extent to which the observed change in outcome is the result of the intervention, having allowed
More informationAnalysis Plans in Economics
MIT December 2012 Where did this come from... for me. Where did this come from... for me. High school internship in a medical lab doing research on anti-stroke drugs Where did this come from... for me.
More informationInstrumental Variables Estimation: An Introduction
Instrumental Variables Estimation: An Introduction Susan L. Ettner, Ph.D. Professor Division of General Internal Medicine and Health Services Research, UCLA The Problem The Problem Suppose you wish to
More informationPropensity Score Methods for Estimating Causality in the Absence of Random Assignment: Applications for Child Care Policy Research
2012 CCPRC Meeting Methodology Presession Workshop October 23, 2012, 2:00-5:00 p.m. Propensity Score Methods for Estimating Causality in the Absence of Random Assignment: Applications for Child Care Policy
More informationCASE STUDY 2: VOCATIONAL TRAINING FOR DISADVANTAGED YOUTH
CASE STUDY 2: VOCATIONAL TRAINING FOR DISADVANTAGED YOUTH Why Randomize? This case study is based on Training Disadvantaged Youth in Latin America: Evidence from a Randomized Trial by Orazio Attanasio,
More informationImpact Evaluation Methods: Why Randomize? Meghan Mahoney Policy Manager, J-PAL Global
Impact Evaluation Methods: Why Randomize? Meghan Mahoney Policy Manager, J-PAL Global Course Overview 1. What is Evaluation? 2. Outcomes, Impact, and Indicators 3. Why Randomize? 4. How to Randomize? 5.
More informationThe Logic of Data Analysis Using Statistical Techniques M. E. Swisher, 2016
The Logic of Data Analysis Using Statistical Techniques M. E. Swisher, 2016 This course does not cover how to perform statistical tests on SPSS or any other computer program. There are several courses
More informationTRANSLATING RESEARCH INTO ACTION. Why randomize? Dan Levy. Harvard Kennedy School
TRANSLATING RESEARCH INTO ACTION Why randomize? Dan Levy Harvard Kennedy School Your background Course Overview 1. What is evaluation? 2. Measuring impacts (outcomes, indicators) 3. Why randomize? 4. How
More informationApplied Quantitative Methods II
Applied Quantitative Methods II Lecture 7: Endogeneity and IVs Klára Kaĺıšková Klára Kaĺıšková AQM II - Lecture 7 VŠE, SS 2016/17 1 / 36 Outline 1 OLS and the treatment effect 2 OLS and endogeneity 3 Dealing
More informationExperimental Methods. Policy Track
Experimental Methods Policy Track East Asia Regional Impact Evaluation Workshop Seoul, South Korea Nazmul Chaudhury, World Bank Reference Spanish Version & French Version also available, Portuguese soon.
More informationProblem Situation Form for Parents
Problem Situation Form for Parents Please complete a form for each situation you notice causes your child social anxiety. 1. WHAT WAS THE SITUATION? Please describe what happened. Provide enough information
More informationThe Limits of Inference Without Theory
The Limits of Inference Without Theory Kenneth I. Wolpin University of Pennsylvania Koopmans Memorial Lecture (2) Cowles Foundation Yale University November 3, 2010 Introduction Fuller utilization of the
More informationEc331: Research in Applied Economics Spring term, Panel Data: brief outlines
Ec331: Research in Applied Economics Spring term, 2014 Panel Data: brief outlines Remaining structure Final Presentations (5%) Fridays, 9-10 in H3.45. 15 mins, 8 slides maximum Wk.6 Labour Supply - Wilfred
More informationHow Early Health Affects Children s Life Chances
How Early Health Affects Children s Life Chances David Figlio* Director, Institute for Policy Research Northwestern University Sulzberger Lecture, Duke University, January 13, 2015 *Collaborative research
More informationMeasuring Impact. Program and Policy Evaluation with Observational Data. Daniel L. Millimet. Southern Methodist University.
Measuring mpact Program and Policy Evaluation with Observational Data Daniel L. Millimet Southern Methodist University 23 May 2013 DL Millimet (SMU) Observational Data May 2013 1 / 23 ntroduction Measuring
More informationLecture Slides. Elementary Statistics Eleventh Edition. by Mario F. Triola. and the Triola Statistics Series 1.1-1
Lecture Slides Elementary Statistics Eleventh Edition and the Triola Statistics Series by Mario F. Triola 1.1-1 Chapter 1 Introduction to Statistics 1-1 Review and Preview 1-2 Statistical Thinking 1-3
More informationPublic Policy & Evidence:
Public Policy & Evidence: How to discriminate, interpret and communicate scientific research to better inform society. Rachel Glennerster Executive Director J-PAL Global Press coverage of microcredit:
More informationInstrumental Variables I (cont.)
Review Instrumental Variables Observational Studies Cross Sectional Regressions Omitted Variables, Reverse causation Randomized Control Trials Difference in Difference Time invariant omitted variables
More informationPros. University of Chicago and NORC at the University of Chicago, USA, and IZA, Germany
Dan A. Black University of Chicago and NORC at the University of Chicago, USA, and IZA, Germany Matching as a regression estimator Matching avoids making assumptions about the functional form of the regression
More informationChapter 1 Review Questions
Chapter 1 Review Questions 1.1 Why is the standard economic model a good thing, and why is it a bad thing, in trying to understand economic behavior? A good economic model is simple and yet gives useful
More informationTRACER STUDIES ASSESSMENTS AND EVALUATIONS
TRACER STUDIES ASSESSMENTS AND EVALUATIONS 1 INTRODUCTION This note introduces the reader to tracer studies. For the Let s Work initiative, tracer studies are proposed to track and record or evaluate the
More informationMeasuring Impact. Conceptual Issues in Program and Policy Evaluation. Daniel L. Millimet. Southern Methodist University.
Measuring mpact Conceptual ssues in Program and Policy Evaluation Daniel L. Millimet Southern Methodist University 23 May 2013 DL Millimet (SMU) Measuring mpact May 2013 1 / 25 ntroduction Primary concern
More informationIntroduction to Research Methods
Introduction to Research Methods 8-10% of the AP Exam Psychology is an empirical discipline. Psychologists develop knowledge by doing research. Research provides guidance for psychologists who develop
More informationAP Psychology -- Chapter 02 Review Research Methods in Psychology
AP Psychology -- Chapter 02 Review Research Methods in Psychology 1. In the opening vignette, to what was Alicia's condition linked? The death of her parents and only brother 2. What did Pennebaker s study
More informationECON Microeconomics III
ECON 7130 - Microeconomics III Spring 2016 Notes for Lecture #5 Today: Difference-in-Differences (DD) Estimators Difference-in-Difference-in-Differences (DDD) Estimators (Triple Difference) Difference-in-Difference
More informationMethods of Randomization Lupe Bedoya. Development Impact Evaluation Field Coordinator Training Washington, DC April 22-25, 2013
Methods of Randomization Lupe Bedoya Development Impact Evaluation Field Coordinator Training Washington, DC April 22-25, 2013 Content 1. Important Concepts 2. What vs. Why 3. Some Practical Issues 4.
More informationVocabulary. Bias. Blinding. Block. Cluster sample
Bias Blinding Block Census Cluster sample Confounding Control group Convenience sample Designs Experiment Experimental units Factor Level Any systematic failure of a sampling method to represent its population
More informationThreats and Analysis. Shawn Cole. Harvard Business School
Threats and Analysis Shawn Cole Harvard Business School Course Overview 1. What is Evaluation? 2. Outcomes, Impact, and Indicators 3. Why Randomize? 4. How to Randomize? 5. Sampling and Sample Size 6.
More informationThe 5 Things You Can Do Right Now to Get Ready to Quit Smoking
The 5 Things You Can Do Right Now to Get Ready to Quit Smoking By Charles Westover Founder of Advanced Laser Solutions Copyright 2012 What you do before you quit smoking is equally as important as what
More informationIntroduction: Statistics, Data and Statistical Thinking Part II
Introduction: Statistics, Data and Statistical Thinking Part II FREC/STAT 608 Dr. Tom Ilvento Department of Food and Resource Economics Let s Continue with our introduction We need terms and definitions
More informationIssues in African Economic Development. Economics 172. University of California, Berkeley. Department of Economics. Professor Ted Miguel
Economics 172 Issues in African Economic Development Professor Ted Miguel Department of Economics University of California, Berkeley Economics 172 Issues in African Economic Development Lecture 10 February
More informationScript for Contacting People that you Know
Script for Contacting People that you Know Hi this is, I ve been thinking a lot about you and wanted to share something with you that I feel passionate about that has changed my life. Is this a good time
More informationRandomization as a Tool for Development Economists. Esther Duflo Sendhil Mullainathan BREAD-BIRS Summer school
Randomization as a Tool for Development Economists Esther Duflo Sendhil Mullainathan BREAD-BIRS Summer school Randomization as one solution Suppose you could do a Randomized evaluation of the microcredit
More informationEconomics 2010a. Fall Lecture 11. Edward L. Glaeser
Economics 2010a Fall 2003 Lecture 11 Edward L. Glaeser Final notes how to write a theory paper: (1) A highbrow theory paper go talk to Jerry or Drew don t listen to me. (2) A lowbrow or applied theory
More informationEconometric analysis and counterfactual studies in the context of IA practices
Econometric analysis and counterfactual studies in the context of IA practices Giulia Santangelo http://crie.jrc.ec.europa.eu Centre for Research on Impact Evaluation DG EMPL - DG JRC CRIE Centre for Research
More informationRegression Discontinuity Design
Regression Discontinuity Design Regression Discontinuity Design Units are assigned to conditions based on a cutoff score on a measured covariate, For example, employees who exceed a cutoff for absenteeism
More informationStatistics Mathematics 243
Statistics Mathematics 243 Michael Stob February 2, 2005 These notes are supplementary material for Mathematics 243 and are not intended to stand alone. They should be used in conjunction with the textbook
More informationTeresa Anderson-Harper
Teresa Anderson-Harper Teresa was nominated as a Reunification Month Hero by a parent attorney who has seen her grow from a parent in a series of dependency cases to the first-ever Family Recovery Support
More informationWRITTEN PRELIMINARY Ph.D. EXAMINATION. Department of Applied Economics. January 17, Consumer Behavior and Household Economics.
WRITTEN PRELIMINARY Ph.D. EXAMINATION Department of Applied Economics January 17, 2012 Consumer Behavior and Household Economics Instructions Identify yourself by your code letter, not your name, on each
More informationInstructions for Printing Outbreak Scenario Cards:
Instructions for Printing Outbreak Scenario Cards: 1. Print pages 2-14 of this document This document contains 4 scenarios for each outbreak. To determine the number of copies needed, divide the number
More informationAn Economic Approach to Generalize Findings from Regression-Discontinuity Designs
An Economic Approach to Generalize Findings from Regression-Discontinuity Designs Nirav Mehta November 6, 2015 Abstract Regression-discontinuity (RD) designs estimate treatment effects at a cutoff. This
More informationClass #5: THOUGHTS AND MY MOOD
: THOUGHTS AND MY MOOD CLASS OUTLINE I. Announcements & Agenda II. III. IV. General Review Personal Project Review Relaxation Exercise V. New Material VI. Personal Project I. Any Announcements? II. GENERAL
More informationCausality and Statistical Learning
Department of Statistics and Department of Political Science, Columbia University 29 Sept 2012 1. Different questions, different approaches Forward causal inference: What might happen if we do X? Effects
More informationChapter 1 Introduction to Educational Research
Chapter 1 Introduction to Educational Research The purpose of Chapter One is to provide an overview of educational research and introduce you to some important terms and concepts. My discussion in this
More informationCSE 258 Lecture 1.5. Web Mining and Recommender Systems. Supervised learning Regression
CSE 258 Lecture 1.5 Web Mining and Recommender Systems Supervised learning Regression What is supervised learning? Supervised learning is the process of trying to infer from labeled data the underlying
More informationClass 1: Introduction, Causality, Self-selection Bias, Regression
Class 1: Introduction, Causality, Self-selection Bias, Regression Ricardo A Pasquini April 2011 Ricardo A Pasquini () April 2011 1 / 23 Introduction I Angrist s what should be the FAQs of a researcher:
More informationIntroduction: Statistics and Engineering
Introduction: Statistics and Engineering STAT:2020 Probability and Statistics for Engineering and Physical Sciences Week 1 - Lecture 1 Book Sections 1.1-1.2.4, 1.3: Introduction 1 / 13 Where do engineering
More informationQuasi-experimental analysis Notes for "Structural modelling".
Quasi-experimental analysis Notes for "Structural modelling". Martin Browning Department of Economics, University of Oxford Revised, February 3 2012 1 Quasi-experimental analysis. 1.1 Modelling using quasi-experiments.
More informationThe Diabetes Breakthrough: Dr. Osama Hamdy on his 12-week Plan
Transcript Details This is a transcript of an educational program accessible on the ReachMD network. Details about the program and additional media formats for the program are accessible by visiting: https://reachmd.com/programs/book-club/the-diabetes-breakthrough-dr-osama-hamdy-on-his-12-weekplan/7059/
More informationBusiness Statistics Probability
Business Statistics The following was provided by Dr. Suzanne Delaney, and is a comprehensive review of Business Statistics. The workshop instructor will provide relevant examples during the Skills Assessment
More informationPlanning for a time when you cannot make decisions for yourself
Planning for a time when you cannot make decisions for yourself An information leaflet for members of the public Version: October 2013 Introduction The Mental Capacity Act 2005 allows you to plan ahead
More informationSTAT 201 Chapter 3. Association and Regression
STAT 201 Chapter 3 Association and Regression 1 Association of Variables Two Categorical Variables Response Variable (dependent variable): the outcome variable whose variation is being studied Explanatory
More informationwith Deborah Gruenfeld Professor, Stanford Graduate School of Business People decide how competent you are in a fraction of a second.
with Deborah Gruenfeld Professor, Stanford Graduate School of Business KEY POINTS People decide how competent you are in a fraction of a second. Factors used to determine your competence: 7% words, 38%
More informationRegression Discontinuity Designs: An Approach to Causal Inference Using Observational Data
Regression Discontinuity Designs: An Approach to Causal Inference Using Observational Data Aidan O Keeffe Department of Statistical Science University College London 18th September 2014 Aidan O Keeffe
More informationResearch Questions, Variables, and Hypotheses: Part 2. Review. Hypotheses RCS /7/04. What are research questions? What are variables?
Research Questions, Variables, and Hypotheses: Part 2 RCS 6740 6/7/04 1 Review What are research questions? What are variables? Definition Function Measurement Scale 2 Hypotheses OK, now that we know how
More informationQUESTIONS ANSWERED BY
Module 16 QUESTIONS ANSWERED BY BERNIE SIEGEL, MD 2 Q How do our thoughts and beliefs affect the health of our bodies? A You can t separate thoughts and beliefs from your body. What you think and what
More informationI. Introduction and Data Collection B. Sampling. 1. Bias. In this section Bias Random Sampling Sampling Error
I. Introduction and Data Collection B. Sampling In this section Bias Random Sampling Sampling Error 1. Bias Bias a prejudice in one direction (this occurs when the sample is selected in such a way that
More informationAnthony Robbins' book on success
Anthony Robbins' book on success This is a motivational book that provides you with the inspiration and techniques with which you can achieve your goals. In this book you will be taught to not give up
More informationPolitical Science 15, Winter 2014 Final Review
Political Science 15, Winter 2014 Final Review The major topics covered in class are listed below. You should also take a look at the readings listed on the class website. Studying Politics Scientifically
More informationBehaviorism: An essential survival tool for practitioners in autism
Behaviorism: An essential survival tool for practitioners in autism What we re going to do today 1. Review the role of radical behaviorism (RB) James M. Johnston, Ph.D., BCBA-D National Autism Conference
More informationWhen Your Partner s Actions Seem Selfish, Inconsiderate, Immature, Inappropriate, or Bad in Some Other Way
When Your Partner s Actions Seem Selfish, Inconsiderate, Immature, Inappropriate, or Bad in Some Other Way Brent J. Atkinson, Ph.D. In the article, Habits of People Who Know How to Get their Partners to
More informationTRANSLATING RESEARCH INTO ACTION
TRANSLATING RESEARCH INTO ACTION This case study is based on a current study by Esther Duflo and Tavneet Suri. J-PAL thanks the authors for allowing us to use their project 17 Key Vocabulary 1. Equivalence:
More informationUnderstanding Science Conceptual Framework
1 Understanding Science Conceptual Framework This list of conceptual understandings regarding the nature and process of science are aligned across grade levels to help instructors identify age-appropriate
More informationQUASI-EXPERIMENTAL APPROACHES
QUASI-EXPERIMENTAL APPROACHES Experimental approaches work by comparing changes in a group that receives a development intervention with a group that does not. The difference is then attributed to the
More informationEcon 270: Theoretical Modeling 1
Econ 270: Theoretical Modeling 1 Economics is certainly not the only social science to use mathematical theoretical models to examine a particular question. But economics, since the 1960s, has evolved
More informationThreats and Analysis. Bruno Crépon J-PAL
Threats and Analysis Bruno Crépon J-PAL Course Overview 1. What is Evaluation? 2. Outcomes, Impact, and Indicators 3. Why Randomize and Common Critiques 4. How to Randomize 5. Sampling and Sample Size
More informationWhat can go wrong.and how to fix it!
What can go wrong.and how to fix it! Megha Pradhan Policy and Training Manager, J-PAL South Asia Kathmandu, Nepal 29 March 2017 Introduction Conception phase is important and allows to design an evaluation
More informationAn Experimental Investigation of Self-Serving Biases in an Auditing Trust Game: The Effect of Group Affiliation: Discussion
1 An Experimental Investigation of Self-Serving Biases in an Auditing Trust Game: The Effect of Group Affiliation: Discussion Shyam Sunder, Yale School of Management P rofessor King has written an interesting
More informationIntroduction to Applied Research in Economics Kamiljon T. Akramov, Ph.D. IFPRI, Washington, DC, USA
Introduction to Applied Research in Economics Kamiljon T. Akramov, Ph.D. IFPRI, Washington, DC, USA Training Course on Applied Econometric Analysis June 1, 2015, WIUT, Tashkent, Uzbekistan Why do we need
More informationHow To Get Mentally Fit & Motivated
How To Get Mentally Fit & Motivated Disclaimer: Note as always when dealing with health-related issues, seek the advice of your healthcare provider or family doctor for your own individual needs. This
More informationRecent advances in non-experimental comparison group designs
Recent advances in non-experimental comparison group designs Elizabeth Stuart Johns Hopkins Bloomberg School of Public Health Department of Mental Health Department of Biostatistics Department of Health
More informationThe Practice of Statistics 1 Week 2: Relationships and Data Collection
The Practice of Statistics 1 Week 2: Relationships and Data Collection Video 12: Data Collection - Experiments Experiments are the gold standard since they allow us to make causal conclusions. example,
More informationQUASI-EXPERIMENTAL HEALTH SERVICE EVALUATION COMPASS 1 APRIL 2016
QUASI-EXPERIMENTAL HEALTH SERVICE EVALUATION COMPASS 1 APRIL 2016 AIM & CONTENTS Aim to explore what a quasi-experimental study is and some issues around how they are done Context and Framework Review
More informationAnalysis A step in the research process that involves describing and then making inferences based on a set of data.
1 Appendix 1:. Definitions of important terms. Additionality The difference between the value of an outcome after the implementation of a policy, and its value in a counterfactual scenario in which the
More informationWhat is: regression discontinuity design?
What is: regression discontinuity design? Mike Brewer University of Essex and Institute for Fiscal Studies Part of Programme Evaluation for Policy Analysis (PEPA), a Node of the NCRM Regression discontinuity
More informationKey questions when starting an econometric project (Angrist & Pischke, 2009):
Econometric & other impact assessment approaches to policy analysis Part 1 1 The problem of causality in policy analysis Internal vs. external validity Key questions when starting an econometric project
More informationFREQUENTLY ASKED QUESTIONS MINIMAL DATA SET (MDS)
FREQUENTLY ASKED QUESTIONS MINIMAL DATA SET (MDS) Date in parentheses is the date the question was added to the list or updated. Last update 6/25/05 DEFINITIONS 1. What counts as the first call? (6/24/05)
More information3 CONCEPTUAL FOUNDATIONS OF STATISTICS
3 CONCEPTUAL FOUNDATIONS OF STATISTICS In this chapter, we examine the conceptual foundations of statistics. The goal is to give you an appreciation and conceptual understanding of some basic statistical
More information12 hours. Your body has eliminates all excess carbon monoxide and your blood oxygen levels become normal.
Balance March 2018 What happens after the last cigarette? You know that smoking is one of the leading causes of preventable deaths but the process of quitting seems too daunting. After all, you ve tried
More informationReassessing Quasi-Experiments: Policy Evaluation, Induction, and SUTVA. Tom Boesche
Reassessing Quasi-Experiments: Policy Evaluation, Induction, and SUTVA Tom Boesche Abstract This paper defends the use of quasi-experiments for causal estimation in economics against the widespread objection
More informationChapter Eight: Multivariate Analysis
Chapter Eight: Multivariate Analysis Up until now, we have covered univariate ( one variable ) analysis and bivariate ( two variables ) analysis. We can also measure the simultaneous effects of two or
More informationFinding True Program Impacts Through Randomization
David Evans (World Bank) Finding True Program Impacts Through Randomization Impact Opportunities: Evidence and Action to Save Lives in Nigeria Uyo, Nigeria, May 7-10, 2013 Objective Evaluate the causal
More informationA NON-TECHNICAL INTRODUCTION TO REGRESSIONS. David Romer. University of California, Berkeley. January Copyright 2018 by David Romer
A NON-TECHNICAL INTRODUCTION TO REGRESSIONS David Romer University of California, Berkeley January 2018 Copyright 2018 by David Romer CONTENTS Preface ii I Introduction 1 II Ordinary Least Squares Regression
More informationI have dementia... First steps after diagnosis
I have dementia... First steps after diagnosis Contents Each section of the booklet has its own colour to make it easy to locate the section you want to read. Message from the Chair of the Working Group
More informationA Practical Guide to Getting Started with Propensity Scores
Paper 689-2017 A Practical Guide to Getting Started with Propensity Scores Thomas Gant, Keith Crowland Data & Information Management Enhancement (DIME) Kaiser Permanente ABSTRACT This paper gives tools
More informationWorkbook Relapse Prevention Name of the patient
Workbook Relapse Prevention Name of the patient The Workbook Relapse Prevention According to research 30-57% of the patients relapse into their eating disorder after having received successful clinical
More informationMeasuring impact. William Parienté UC Louvain J PAL Europe. povertyactionlab.org
Measuring impact William Parienté UC Louvain J PAL Europe povertyactionlab.org Course overview 1. What is evaluation? 2. Measuring impact 3. Why randomize? 4. How to randomize 5. Sampling and Sample Size
More informationGUIDE 4: COUNSELING THE UNEMPLOYED
GUIDE 4: COUNSELING THE UNEMPLOYED Addressing threats to experimental integrity This case study is based on Sample Attrition Bias in Randomized Experiments: A Tale of Two Surveys By Luc Behaghel, Bruno
More informationLiving Well with Diabetes. Meeting 12. Welcome!
12-1 Welcome! Welcome back and congratulations! Today is a time to celebrate all of your accomplishments. For the past few months we have learned a great deal about managing diabetes. Today, we will talk
More information2 INSTRUCTOR GUIDELINES
STAGE: Not Ready to Quit (Ready to cut back) You have been approached by Mr. Faulk, a healthy young male, aged 28, who has applied to become a fireman and has a good chance of being offered the job. His
More informationWorries and Anxiety F O R K I D S. C o u n s e l l i n g D i r e c t o r y H a p p i f u l K i d s
Worries and Anxiety F O R K I D S C o u n s e l l i n g D i r e c t o r y H a p p i f u l K i d s Contents What is anxiety? Types of anxiety What does anxiety look like? Top tips for tackling worries Asking
More informationComputer Science 101 Project 2: Predator Prey Model
Computer Science 101 Project 2: Predator Prey Model Real-life situations usually are complicated and difficult to model exactly because of the large number of variables present in real systems. Computer
More informationWhat makes us special? Ages 3-5
What makes us special? Ages 3-5 Pudsey s learning aims I can think about how we are different and how we are the same. I can think about how we are special and what makes us unique. Key Words Special Unique
More informationMixed Methods Study Design
1 Mixed Methods Study Design Kurt C. Stange, MD, PhD Professor of Family Medicine, Epidemiology & Biostatistics, Oncology and Sociology Case Western Reserve University 1. Approaches 1, 2 a. Qualitative
More information