What can go wrong.and how to fix it!
|
|
- Adele Smith
- 6 years ago
- Views:
Transcription
1 What can go wrong.and how to fix it! Megha Pradhan Policy and Training Manager, J-PAL South Asia Kathmandu, Nepal 29 March 2017
2 Introduction Conception phase is important and allows to design an evaluation enabling to answer the research questions But the implementation phase of the evaluation is also extremely important: many things can go wrong J-PAL POST-DESIGN CHALLENGES 2
3 Objectives To be able to identify the main threats to validity during the implementation phase of the evaluation To define strategies to prevent each of these threats To know some of the methods that can be used during analysis phase J-PAL POST-DESIGN CHALLENGES 3
4 Lecture Overview Attrition Unexpected Spillovers Partial Compliance Intention to Treat (ITT) & Treatment on Treated (ToT) J-PAL POST-DESIGN CHALLENGES 4
5 Lecture Overview Attrition Spillovers Partial Compliance Intention to Treat (ITT) & Treatment on Treated (ToT) J-PAL POST-DESIGN CHALLENGES 5
6 Attrition Attrition occurs when not all of the individuals we intend to survey for the evaluation are available to be surveyed. Why is there attrition? People migrate and can t be tracked People refuse to get surveyed Businesses shut down Children don t come to school Why is attrition a problem?
7 Attrition Problem Attrition is a threat when it significantly reduces the sample size and power to detect significant impacts. Treatment Control No longer detect smallest effect size = motivated, entrepreneurial type
8 Attrition Problem Attrition is a threat when the treatment and control groups are no longer comparable. Treatment Control Started / expanded businesses; too busy to be interviewed Understating impact? = motivated, entrepreneurial type
9 Attrition Problem: An Example Problem: Some children are malnourished and sick because they don t get enough food Intervention: You start a school feeding program where free lunch is provided to children Evaluation: You want to measure children s weight as the outcome. You assign schools to randomized treatment and a control groups, and you weigh everyone who is in school on a given day Impacts: 1. Weight of children: Children who receive school meals gain more weight 2. Increased enrollment: Weak, stunted children start going to school more if they are enrolled in a treatment school
10 Before Treatment After Treament T C T C Ave. Difference Difference J-PAL THREATS AND ANALYSIS 10
11 Before Treatment After Treament T C T C Ave Difference 0 Difference 2 J-PAL THREATS AND ANALYSIS 11
12 What What if if only children children < 21 > 21 Kg Kg are come less to likely school? to come to school? J-PAL THREATS AND ANALYSIS 12
13 What if children < 21 Kg are less likely to come to school? Before Treatment A. Will you underestimate the impact? B. Will you overestimate the impact? C. Neither After Treament T C T C D. Ambiguous E. Don t know 0% 0% 0% 0% 0% A. B. C. D. E. J-PAL THREATS AND ANALYSIS 13
14 What if children < 21 Kg are less likely to What if only children > 21 Kg come to school? come to school? Before Treatment After Treament T C T C [absent] [absent] 22 [absent] Ave Difference 0 Difference -0.5 J-PAL THREATS AND ANALYSIS 14
15 Limiting Attrition: Partial Solutions Devote resources to tracking participants after they leave the program If there is still attrition, do some checks: Does it seem as if the missing people were different to begin with than those who remained in the study? Try to bound the extent of the bias Suppose everyone who dropped out from the treatment got the lowest score that anyone got Suppose everyone who dropped out of control got the highest score that anyone got.
16 Lecture Overview Attrition Spillovers Partial Compliance Intention to Treat (ITT) & Treatment on Treated (ToT) J-PAL POST-DESIGN CHALLENGES 16
17 Spillover Problem Spillovers are a problem when the outcomes of the control group reflect indirect program effects and not what would have happened in the absence of the program (i.e., the counterfactual). Evaluation Sample Random Assignment Treatment Group Control Group If positive If negative understate impact overstate impact
18 Spillovers: Solutions Two strategies seen during design phase: avoid them or measure them 1. Design the level of randomization so that it encompasses the spillovers If we expect spillovers that are all within school, randomization at the level of the school allows for estimation of the overall effect 2. It can be good to measure spillovers as they can have important implications for policy design Compare spillover group to pure control to measure spillover effect
19 Lecture Overview Attrition Unexpected Spillovers Partial Compliance Intention to Treat (ITT) & Treatment on Treated (ToT) J-PAL POST-DESIGN CHALLENGES 19
20 Partial Compliance Individuals assigned to comparison group could move into treatment group Alternatively, individuals allocated to treatment group may not receive treatment Can be due to project implementers or to participants themselves J-PAL POST-DESIGN CHALLENGES 20
21 Non compliers Target Population Not in evaluation What can you do? Can you switch them? Evaluation Sample Random Assignment Treatment group Control group Participants No-Shows Non- Participants Cross-overs J-PAL POST-DESIGN CHALLENGES 21
22 Non compliers Target Population Not in evaluation What can you do? Can you drop them? Evaluation Sample Random Assignment Treatment group Control group Participants No-Shows Non- Participants Cross-overs J-PAL POST-DESIGN CHALLENGES 22
23 Sample Selection Bias Sample selection bias could arise if factors other than random assignment influence program allocation Even if intended allocation of program was random, the actual allocation may not be J-PAL POST-DESIGN CHALLENGES 23
24 Non compliers Not in evaluation Target Population You can compare the original groups Evaluation Sample Random Assignment Treatment group Control group Participants No-Shows Non- Participants Cross-overs J-PAL POST-DESIGN CHALLENGES 24
25 What can be done? Ideally: prevent it during design or implementation phase => cannot always be done Monitor it during implementation phase => important to be aware that it happens Interpret it during analysis phase J-PAL POST-DESIGN CHALLENGES 25
26 Lecture Overview Attrition Unexpected Spillovers Partial Compliance Intention to Treat (ITT) & Treatment on Treated (ToT) J-PAL POST-DESIGN CHALLENGES 26
27 A school feeding program Let s take the example of a school feeding program Some schools receive the program, some don t (random allocation) But allocation is imperfectly respected J-PAL POST-DESIGN CHALLENGES 27
28 Compliance is imperfect School 1 Intention to treat? Treated? Pupil 1 Yes Yes Pupil 2 Yes Yes Pupil 3 Yes Yes Pupil 4 Yes No Pupil 5 Yes Yes Pupil 6 Yes No Pupil 7 Yes No Pupil 8 Yes Yes Pupil 9 Yes Yes Pupil 10 Yes No School 2 Intention to Treat? Treated? Pupil 1 No No Pupil 2 No No Pupil 3 No Yes Pupil 4 No No Pupil 5 No No Pupil 6 No Yes Pupil 7 No No Pupil 8 No No Pupil 9 No No Pupil 10 No No J-PAL POST-DESIGN CHALLENGES 28
29 Compliance is imperfect School 1 Intention to treat? Treated? Observed Change in weight Pupil 1 Yes Yes 4 Pupil 2 Yes Yes 4 Pupil 3 Yes Yes 4 Pupil 4 Yes No 0 Pupil 5 Yes Yes 4 Pupil 6 Yes No 2 Pupil 7 Yes No 0 Pupil 8 Yes Yes 6 Pupil 9 Yes Yes 6 Pupil 10 Yes No 0 Avg. Change among Treated A = 4.67 School 1: Average of treated in School 1 School 2: Average of treated in School 2 (A) 4.67 (B) 0.5 A-B 4.17 School 2 Pupil 1 No No 2 Pupil 2 No No 1 Pupil 3 No Yes 3 Pupil 4 No No 0 Pupil 5 No No 0 Pupil 6 No Yes 3 Pupil 7 No No 0 Pupil 8 No No 0 Pupil 9 No No 1 Pupil 10 No No 0 Avg. Change among Not-Treated B = 0.5
30 Intention to Treat (ITT) Easiest way to deal with partial compliance: Calculate the Intent to Treat (ITT): The difference between the average outcome of the group that was randomly assigned to treatment and the group that was randomly assigned to control, regardless of whether they actually received the treatment. What does intention to treat measure? What happened to the average child who is in a treated school in this population?
31 Intention To Treat School 1 Intention to treat? Treated? Observed Change in weight Pupil 1 Yes Yes 4 Pupil 2 Yes Yes 4 Pupil 3 Yes Yes 4 Pupil 4 Yes No 0 Pupil 5 Yes Yes 4 Pupil 6 Yes No 2 Pupil 7 Yes No 0 Pupil 8 Yes Yes 6 Pupil 9 Yes Yes 6 Pupil 10 Yes No 0 Avg. Change among Treated A = School 1: Avg. Change among Treated School 2: Avg. Change among Not-Treated A-B (A) (B) School 2 Pupil 1 No No 2 Pupil 2 No No 1 Pupil 3 No Yes 3 Pupil 4 No No 0 Pupil 5 No No 0 Pupil 6 No Yes 3 Pupil 7 No No 0 Pupil 8 No No 0 Pupil 9 No No 1 Pupil 10 No No 0 Avg. Change among Not-Treated B =
32 Intention To Treat School 1 Intention to treat? Treated? Observed Change in weight Pupil 1 Yes Yes 4 Pupil 2 Yes Yes 4 Pupil 3 Yes Yes 4 Pupil 4 Yes No 0 Pupil 5 Yes Yes 4 Pupil 6 Yes No 2 Pupil 7 Yes No 0 Pupil 8 Yes Yes 6 Pupil 9 Yes Yes 6 Pupil 10 Yes No 0 Avg. Change among Treated A = 3 School 1: Avg. Change among Treated School 2: Avg. Change among Not-Treated (A) 3 (B) 1 A-B 2 School 2 Pupil 1 No No 2 Pupil 2 No No 1 Pupil 3 No Yes 3 Pupil 4 No No 0 Pupil 5 No No 0 Pupil 6 No Yes 3 Pupil 7 No No 0 Pupil 8 No No 0 Pupil 9 No No 1 Pupil 10 No No 0 Avg. Change among Not-Treated B = 1
33 ITT / ToT Intention To Treat Treatment on Treated What happened to the average child who is in a treated school in this population? What happened to a child that actually received the treatment? Measuring the impact of launching the program Measuring the impact of the program itself - ITT and ToT are two different ways to analyze the data - ITT may relate more to actual programs, especially if imperfect compliance is likely to happen => Let s now see how we do it J-PAL POST-DESIGN CHALLENGES 33
34 From ITT to ToT We conceptually divide our treatment and control groups into three categories: 1/ The always takers, who will get the meals no matter if they are in the treatment or the control group 2/ The never takers, who won t get the meals no matter if they are in the treatment or the control group 3/ The compliers, who will behave according to the group they have been assigned to J-PAL POST-DESIGN CHALLENGES 34
35 A situation of imperfect compliance Treatment Group Control Group
36 Division into the three categories Treatment Group Control Group Always-takers Compliers Never-takers As the assignation was done randomly, the proportion of each category should be similar in Treatment and Control
37 Comparing the compliers Treatment Group Control Group Always-takers Compliers Never-takers To measure the impact of receiving the treatment, we compare compliers from Treatment and Control This measure of the impact is local : it is only valid for compliers. It can have a different impact for always-takers or never-takers.
38 ToT Estimator What values do we need? Y(T) Y(C) Prob[treated T] Prob[treated C] Y T Y C Prob treated T Prob[treated C] J-PAL POST-DESIGN CHALLENGES 38
39 ToT estimator School 1 Intention to treat? Treated? Observed Change in weight A = Gain if Treated B = Gain if not Treated Pupil 1 Yes Yes 4 Pupil 2 Yes Yes 4 Pupil 3 Yes Yes 4 Pupil 4 Yes No 0 Pupil 5 Yes Yes 4 Pupil 6 Yes No 2 Pupil 7 Yes No 0 Pupil 8 Yes Yes 6 Pupil 9 Yes Yes 6 Pupil 10 Yes No 0 Avg. Change Y(T) = School 2 Pupil 1 No No 2 Pupil 2 No No 1 Pupil 3 No Yes 3 Pupil 4 No No 0 Pupil 5 No No 0 Pupil 6 No Yes 3 Pupil 7 No No 0 Pupil 8 No No 0 Pupil 9 No No 0 Pupil 10 No No 0 Avg. Change Y(C) = ToT Estimator: A-B A-B = Y(T)-Y(C) Prob(Treated T)-Prob(Treated C) Y(T) Y(C) Prob(Treated T) Prob(Treated C) Y(T)-Y(C) Prob(Treated T)-Prob(Treated C) A-B
40 ToT estimator School 1 Intention to treat? Treated? Observed Change in weight Pupil 1 Yes Yes 4 Pupil 2 Yes Yes 4 Pupil 3 Yes Yes 4 Pupil 4 Yes No 0 Pupil 5 Yes Yes 4 Pupil 6 Yes No 2 Pupil 7 Yes No 0 Pupil 8 Yes Yes 6 Pupil 9 Yes Yes 6 Pupil 10 Yes No 0 School 2 Avg. Change Y(T) = 3 Pupil 1 No No 2 Pupil 2 No No 1 Pupil 3 No Yes 3 Pupil 4 No No 0 Pupil 5 No No 0 Pupil 6 No Yes 3 Pupil 7 No No 0 Pupil 8 No No 0 Pupil 9 No No 1 Pupil 10 No No 0 Avg. Change Y(C) = 1 A = Gain if Treated B = Gain if not Treated ToT Estimator: A-B A-B = Y(T)-Y(C) Prob(Treated T)-Prob(Treated C) Y(T) 3 Y(C) 1 Prob(Treated T) 60% Prob(Treated C) 20% Y(T)-Y(C) 2. Prob(Treated T)-Prob(Treated C) 40% A-B 540
41 ToT estimator The intuitive idea: Let s say the ITT effect of afterschool classes is a 2 point test score difference between treatment and control schools. But only 40% of the children in the treatment schools actually went to the classes (for simplicity let s assume no children in control schools got the classes). If the effect of 40% take-up is to increase scores by 2 points, then we can say that if everyone were to take the classes, the effect would be points J-PAL POST-DESIGN CHALLENGES 41
42 The ITT estimate will always be smaller (e.g., closer to zero) than the ToT estimate A. True B. False C. Don t Know 0% 0% 0% A. B. C. J-PAL THREATS AND ANALYSIS 42
43 ITT / ToT: Conclusions Both ITT and ToT can provide valuable information to decision-makers ToT gives the effect of the intervention on the ones that take-up the programme ITT gives the overall effect of the intervention, admitting that partial compliance can happen (which is inherent to any policy) J-PAL POST-DESIGN CHALLENGES 43
44 Conclusions Internal validity is the great strenghth of Randomized Evaluations so everything undermining it must be carefully considered Design phase and power calculation are important but so is the ability to face challenges during implementation phase Distinguish well between attrition, spillovers and partial compliance J-PAL POST-DESIGN CHALLENGES 44
45 Thank you Follow us on Explore our website
Threats and Analysis. Shawn Cole. Harvard Business School
Threats and Analysis Shawn Cole Harvard Business School Course Overview 1. What is Evaluation? 2. Outcomes, Impact, and Indicators 3. Why Randomize? 4. How to Randomize? 5. Sampling and Sample Size 6.
More informationThreats and Analysis. Bruno Crépon J-PAL
Threats and Analysis Bruno Crépon J-PAL Course Overview 1. What is Evaluation? 2. Outcomes, Impact, and Indicators 3. Why Randomize and Common Critiques 4. How to Randomize 5. Sampling and Sample Size
More informationTHREATS TO VALIDITY. Presentation by: Raul Sanchez de la Sierra
THREATS TO VALIDITY Presentation by: Raul Sanchez de la Sierra What I want you to avoid Captain, sorry, I forgot the spillovers! Lecture Overview External Validity Attrition Spillovers Partial Compliance
More informationTRANSLATING RESEARCH INTO ACTION
TRANSLATING RESEARCH INTO ACTION This case study is based on a current study by Esther Duflo and Tavneet Suri. J-PAL thanks the authors for allowing us to use their project 17 Key Vocabulary 1. Equivalence:
More informationWhy randomize? Rohini Pande Harvard University and J-PAL.
Why randomize? Rohini Pande Harvard University and J-PAL www.povertyactionlab.org Agenda I. Background on Program Evaluation II. What is a randomized experiment? III. Advantages and Limitations of Experiments
More informationGUIDE 4: COUNSELING THE UNEMPLOYED
GUIDE 4: COUNSELING THE UNEMPLOYED Addressing threats to experimental integrity This case study is based on Sample Attrition Bias in Randomized Experiments: A Tale of Two Surveys By Luc Behaghel, Bruno
More informationProgram Evaluations and Randomization. Lecture 5 HSE, Dagmara Celik Katreniak
Program Evaluations and Randomization Lecture 5 HSE, 10.11.2014 Dagmara Celik Katreniak Overview Treatment Effect under perfect conditions Average Treatment Effect Treatment effect under imperfect conditions
More informationCASE STUDY 4: DEWORMING IN KENYA
CASE STUDY 4: DEWORMING IN KENYA Addressing Threats to Experimental Integrity This case study is based on Edward Miguel and Michael Kremer, Worms: Identifying Impacts on Education and Health in the Presence
More informationRandomization as a Tool for Development Economists. Esther Duflo Sendhil Mullainathan BREAD-BIRS Summer school
Randomization as a Tool for Development Economists Esther Duflo Sendhil Mullainathan BREAD-BIRS Summer school Randomization as one solution Suppose you could do a Randomized evaluation of the microcredit
More informationEXERCISE: HOW TO DO POWER CALCULATIONS IN OPTIMAL DESIGN SOFTWARE
...... EXERCISE: HOW TO DO POWER CALCULATIONS IN OPTIMAL DESIGN SOFTWARE TABLE OF CONTENTS 73TKey Vocabulary37T... 1 73TIntroduction37T... 73TUsing the Optimal Design Software37T... 73TEstimating Sample
More informationHere s a list of the Behavioral Economics Principles included in this card deck
Here s a list of the Behavioral Economics Principles included in this card deck Anchoring Action Goals Availability Bias Decision Paralysis Default Bias Disposition Effect Ego Depletion Endowment Effect
More informationEvaluating Social Programs Course: Evaluation Glossary (Sources: 3ie and The World Bank)
Evaluating Social Programs Course: Evaluation Glossary (Sources: 3ie and The World Bank) Attribution The extent to which the observed change in outcome is the result of the intervention, having allowed
More informationRegression Discontinuity Design
Regression Discontinuity Design Regression Discontinuity Design Units are assigned to conditions based on a cutoff score on a measured covariate, For example, employees who exceed a cutoff for absenteeism
More informationMeasuring impact. William Parienté UC Louvain J PAL Europe. povertyactionlab.org
Measuring impact William Parienté UC Louvain J PAL Europe povertyactionlab.org Course overview 1. What is evaluation? 2. Measuring impact 3. Why randomize? 4. How to randomize 5. Sampling and Sample Size
More informationChapter 13 Summary Experiments and Observational Studies
Chapter 13 Summary Experiments and Observational Studies What have we learned? We can recognize sample surveys, observational studies, and randomized comparative experiments. o These methods collect data
More informationGlossary From Running Randomized Evaluations: A Practical Guide, by Rachel Glennerster and Kudzai Takavarasha
Glossary From Running Randomized Evaluations: A Practical Guide, by Rachel Glennerster and Kudzai Takavarasha attrition: When data are missing because we are unable to measure the outcomes of some of the
More informationChapter 13. Experiments and Observational Studies. Copyright 2012, 2008, 2005 Pearson Education, Inc.
Chapter 13 Experiments and Observational Studies Copyright 2012, 2008, 2005 Pearson Education, Inc. Observational Studies In an observational study, researchers don t assign choices; they simply observe
More informationWhy we get hungry: Module 1, Part 1: Full report
Why we get hungry: Module 1, Part 1: Full report Print PDF Does Anyone Understand Hunger? Hunger is not simply a signal that your stomach is out of food. It s not simply a time when your body can switch
More informationInstrumental Variables Estimation: An Introduction
Instrumental Variables Estimation: An Introduction Susan L. Ettner, Ph.D. Professor Division of General Internal Medicine and Health Services Research, UCLA The Problem The Problem Suppose you wish to
More informationPATIENT AND FAMILY SATISFACTION. Dr. James Stallcup, M.D.
PATIENT AND FAMILY SATISFACTION Dr. James Stallcup, M.D. PREMISE In medicine we tend to focus on the clinical care delivered, not the patient s perception of the care delivered. In a trauma this is acceptable
More informationInstrumental Variables I (cont.)
Review Instrumental Variables Observational Studies Cross Sectional Regressions Omitted Variables, Reverse causation Randomized Control Trials Difference in Difference Time invariant omitted variables
More informationCritical Appraisal Istanbul 2011
Critical Appraisal Istanbul 2011 The Conviction with which many Nephrologists hold an opinion varies inversely with the Evidence Ed Lewis The Good Old Times. The Google Generation. ASN Kidney Daily LEADING
More informationData Collection: Validity & Ethics
Data Collection: Validity & Ethics Data Integrity Expectancy effects & their control Experimenter expectancy effects Participant Expectancy Effects Single- and Double-blind designs Researcher and Participant
More informationIntroduction to Program Evaluation
Introduction to Program Evaluation Nirav Mehta Assistant Professor Economics Department University of Western Ontario January 22, 2014 Mehta (UWO) Program Evaluation January 22, 2014 1 / 28 What is Program
More informationThis is an Easy Read booklet. This booklet is about changes we want made to social care support for disabled people.
This is an Easy Read booklet. This booklet is about changes we want made to social care support for disabled people. What is in this booklet? 1. What is this booklet about? Who is it for? 2. Who wrote
More informationWelcome to this series focused on sources of bias in epidemiologic studies. In this first module, I will provide a general overview of bias.
Welcome to this series focused on sources of bias in epidemiologic studies. In this first module, I will provide a general overview of bias. In the second module, we will focus on selection bias and in
More informationReview: Conditional Probability. Using tests to improve decisions: Cutting scores & base rates
Review: Conditional Probability Using tests to improve decisions: & base rates Conditional probabilities arise when the probability of one thing [A] depends on the probability of something else [B] In
More informationChapter 2 Quiz Self-Concept and Self-Esteem in Human Relations
Chapter 2 Quiz Self-Concept and Self-Esteem in Human Relations Name True or False: 1. Self-esteem is the way you conceive of or see yourself. 2. Self-image is the self that you assume others see when they
More informationThe Fallacy of Taking Random Supplements
The Fallacy of Taking Random Supplements Healthview interview with Dr. Paul Eck Healthview: We can see from our conversations that you are totally against people taking random supplements even if people
More informationRecent developments for combining evidence within evidence streams: bias-adjusted meta-analysis
EFSA/EBTC Colloquium, 25 October 2017 Recent developments for combining evidence within evidence streams: bias-adjusted meta-analysis Julian Higgins University of Bristol 1 Introduction to concepts Standard
More informationMore on Experiments: Confounding and Obscuring Variables (Part 1) Dr. Stefanie Drew
1 More on Experiments: Confounding and Obscuring Variables (Part 1) Dr. Stefanie Drew stefanie.drew@csun.edu 2 Previously, On Research Methods Basic structure of an experiment Now in more detail Internal
More informationImpact Evaluation Methods: Why Randomize? Meghan Mahoney Policy Manager, J-PAL Global
Impact Evaluation Methods: Why Randomize? Meghan Mahoney Policy Manager, J-PAL Global Course Overview 1. What is Evaluation? 2. Outcomes, Impact, and Indicators 3. Why Randomize? 4. How to Randomize? 5.
More informationTeresa Anderson-Harper
Teresa Anderson-Harper Teresa was nominated as a Reunification Month Hero by a parent attorney who has seen her grow from a parent in a series of dependency cases to the first-ever Family Recovery Support
More informationELEPHANT IN THE OFFICE!
Ethics ELEPHANT IN THE OFFICE! Ethical Choices Learn to distinguish between right & wrong Professional groups or an employer s code of ethics can help Restrain yourself from choosing the wrong path Don
More informationContinuous or Intermittent Calorie Deficits: Which is Better for Fat Loss?
Continuous or Intermittent Calorie Deficits: Which is Better for Fat Loss? Byrne et al., Int J Obes, September, 2017 [Epub ahead of print] James Krieger, M.S. Background As you lose weight, your body tries
More informationSAMPLING AND SAMPLE SIZE
SAMPLING AND SAMPLE SIZE Andrew Zeitlin Georgetown University and IGC Rwanda With slides from Ben Olken and the World Bank s Development Impact Evaluation Initiative 2 Review We want to learn how a program
More informationWhen Your Partner s Actions Seem Selfish, Inconsiderate, Immature, Inappropriate, or Bad in Some Other Way
When Your Partner s Actions Seem Selfish, Inconsiderate, Immature, Inappropriate, or Bad in Some Other Way Brent J. Atkinson, Ph.D. In the article, Habits of People Who Know How to Get their Partners to
More informationEXPERIMENTAL RESEARCH DESIGNS
ARTHUR PSYC 204 (EXPERIMENTAL PSYCHOLOGY) 14A LECTURE NOTES [02/28/14] EXPERIMENTAL RESEARCH DESIGNS PAGE 1 Topic #5 EXPERIMENTAL RESEARCH DESIGNS As a strict technical definition, an experiment is a study
More informationYes, you do have a choice. Understanding and using Choice Theory with clients Presented by: Dana Hampson, MBA, BCC, LPC-S The Balanced Life, LLC
Yes, you do have a choice. Understanding and using Choice Theory with clients Presented by: Dana Hampson, MBA, BCC, LPC-S The Balanced Life, LLC What is Choice Theory? Theory developed by William Glasser,
More informationWhy do Psychologists Perform Research?
PSY 102 1 PSY 102 Understanding and Thinking Critically About Psychological Research Thinking critically about research means knowing the right questions to ask to assess the validity or accuracy of a
More informationStatistical Power Sampling Design and sample Size Determination
Statistical Power Sampling Design and sample Size Determination Deo-Gracias HOUNDOLO Impact Evaluation Specialist dhoundolo@3ieimpact.org Outline 1. Sampling basics 2. What do evaluators do? 3. Statistical
More informationADDITIONAL CASEWORK STRATEGIES
ADDITIONAL CASEWORK STRATEGIES A. STRATEGIES TO EXPLORE MOTIVATION THE MIRACLE QUESTION The Miracle Question can be used to elicit clients goals and needs for his/her family. Asking this question begins
More informationTRANSLATING RESEARCH INTO ACTION. Why randomize? Dan Levy. Harvard Kennedy School
TRANSLATING RESEARCH INTO ACTION Why randomize? Dan Levy Harvard Kennedy School Your background Course Overview 1. What is evaluation? 2. Measuring impacts (outcomes, indicators) 3. Why randomize? 4. How
More informationPsychology and personality ASSESSMENT
Reading Practice A Psychology and personality ASSESSMENT Our daily lives are largely made up of contacts with other people, during which we are constantly making judgments of their personalities and accommodating
More informationResearch Process. the Research Loop. Now might be a good time to review the decisions made when conducting any research project.
Research Process Choices & Combinations of research attributes Research Loop and its Applications Research Process and what portions of the process give us what aspects of validity Data Integrity Experimenter
More informationEveryone is Responsible for Safety But who is accountable and for what?
Everyone is Responsible for Safety But who is accountable and for what? Patrick Hudson 24-3-2011 Delft University of Technology Challenge the future Introduction Responsibility for Accidents Cognitive
More informationCollecting Data Example: Does aspirin prevent heart attacks?
Collecting Data In an experiment, the researcher controls or manipulates the environment of the individuals. The intent of most experiments is to study the effect of changes in the explanatory variable
More informationAddress : S. 27th Street, Suite 201 Franklin, Wisconsin, WebSite : https://www.patienttrak.net/review-site-monitoring
Address : - 7441 S. 27th Street, Suite 201 Franklin, Wisconsin,53132 Email ID : info@patienttra.net Phone No : 888-766-2862 WebSite : https://www.patienttrak.net/review-site-monitoring There are so many
More informationExperimental Methods. Policy Track
Experimental Methods Policy Track East Asia Regional Impact Evaluation Workshop Seoul, South Korea Nazmul Chaudhury, World Bank Reference Spanish Version & French Version also available, Portuguese soon.
More informationHow to Randomise? (Randomisation Design)
J-PAL Africa Executive Education Course How to Randomise? (Randomisation Design) Lecture 4, January 24, 2012 Martin Abel University of Cape Town Roadmap to Randomised Evaluations 1 Environment / Context
More informationInterviewer: Tell us about the workshops you taught on Self-Determination.
INTERVIEW WITH JAMIE POPE This is an edited translation of an interview by Jelica Nuccio on August 26, 2011. Jelica began by explaining the project (a curriculum for SSPs, and for Deaf-Blind people regarding
More informationJohn Smith 20 October 2009
John Smith 20 October 2009 2009 MySkillsProfile.com. All rights reserved. Introduction The Emotional Competencies Questionnaire (ECQ) assesses your current emotional competencies and style by asking you
More information125M 7/07 (Sells) Copyright The A.A. Grapevine, Inc., February 1958 Reprinted with permission by A.A. World Services, Inc.
ALCOHOLICS ANONYMOUS is a fellowship of men and women who share their experience, strength and hope with each other that they may solve their common problem and help others to recover from alcoholism.
More informationChapter 11: Experiments and Observational Studies p 318
Chapter 11: Experiments and Observational Studies p 318 Observation vs Experiment An observational study observes individuals and measures variables of interest but does not attempt to influence the response.
More informationPublic Policy & Evidence:
Public Policy & Evidence: How to discriminate, interpret and communicate scientific research to better inform society. Rachel Glennerster Executive Director J-PAL Global Press coverage of microcredit:
More informationResearch Design. Miles Corak. Department of Economics The Graduate Center, City University of New York
Research Design The Advantages and Challenges of Randomized Controlled Trials Miles Corak Department of Economics The Graduate Center, City University of New York MilesCorak.com @MilesCorak Lecture 1 Labor
More informationBeyond the intention-to treat effect: Per-protocol effects in randomized trials
Beyond the intention-to treat effect: Per-protocol effects in randomized trials Miguel Hernán DEPARTMENTS OF EPIDEMIOLOGY AND BIOSTATISTICS Intention-to-treat analysis (estimator) estimates intention-to-treat
More informationValidity and reliability of measurements
Validity and reliability of measurements 2 3 Request: Intention to treat Intention to treat and per protocol dealing with cross-overs (ref Hulley 2013) For example: Patients who did not take/get the medication
More informationTip sheet. A quick guide to the dos and don ts of mental health care and inclusion. 1. Ask questions. Practical tips
A quick guide to the dos and don ts of mental health care and inclusion Much of the rejection felt by those in church with mental health problems comes from accidental actions and words, delivered with
More informationLetter to the teachers
Letter to the teachers Hello my name is Sasha Jacombs I m 12 years old and I have had Type 1 Diabetes since I was four years old. Some of the people reading this may not know what that is, so I had better
More informationSocial Biases and Pressures. Critical Thinking
Social Biases and Pressures Critical Thinking Humans are Social Animals We are very sensitive to what others do and think. Most of our deep-rooted beliefs and values come from family and society we grow
More informationSAMMY BEAR S MUMMY IS IN HOSPITAL
SAMMY BEAR S MUMMY IS IN HOSPITAL A story for children aged 3 6 to be read with an adult 2 Notes for adults We hope this rhyming story will be useful for family, carers, social workers and teachers who
More informationMILIEU MANAGEMENT THROUGH TRAUMA INFORMED CARE AND MOTIVATIONAL INTERVIEWING. Chauna Brocht, LCSW-C Health Care for the Homeless Maryland
MILIEU MANAGEMENT THROUGH TRAUMA INFORMED CARE AND MOTIVATIONAL INTERVIEWING Chauna Brocht, LCSW-C Health Care for the Homeless Maryland What are some challenging client behaviors that you commonly face?
More informationORGANISATIONAL BEHAVIOUR
ORGANISATIONAL BEHAVIOUR LECTURE 3, CHAPTER 6 A process through which Individuals organize and interpret their sensory impressions in order to give meaning to their environment. PERCEPTION Why is Perception
More informationExperiments in the Real World
Experiments in the Real World Goal of a randomized comparative experiment: Subjects should be treated the same in all ways except for the treatments we are trying to compare. Example: Rats in cages given
More informationChapter 6. Experimental Studies
Chapter 6 Experimental Studies 1 Chapter 6 Outline 6.1 Introduction 6.2 Historical perspective 6.3 General concepts 6.4 Data analysis 2 Epi Experiments ( Trials ) Trials - from the French trier (to try)
More informationLogic Model. When do you create a Logic Model? Create your LM at the beginning of an evaluation and review and update it periodically.
RRC Evaluation Tool Basket: Logic Model 1 Logic Model What is a Logic Model? A logic model is a series of if-then statements that outlines what you expect to happen in your program. For example, If these
More informationSupporting children with anxiety
Supporting children with anxiety Healthy risk takers Nourishing Environment Effective Coping Strategies Effective Problem Solving Skills Healthy Thinking Habits RESILIENCE The capacity to cope and stay
More informationPerfectionism and mindset
Perfectionism and mindset Perfectionism Being perfect sounds like a good thing, but perfectionism gets seriously in the way of learning. Rates of perfectionism are higher at Nossal than in other schools.
More informationThe Single Digit Body Fat Manual
The Single Digit Body Fat Manual Published by FitnessUnderOath.com Copyright 2016 1 Legal Notices No part of this publication may be reproduced or transmitted in any form or by any means, mechanical or
More informationLecture II: Difference in Difference and Regression Discontinuity
Review Lecture II: Difference in Difference and Regression Discontinuity it From Lecture I Causality is difficult to Show from cross sectional observational studies What caused what? X caused Y, Y caused
More informationPLS 506 Mark T. Imperial, Ph.D. Lecture Notes: Reliability & Validity
PLS 506 Mark T. Imperial, Ph.D. Lecture Notes: Reliability & Validity Measurement & Variables - Initial step is to conceptualize and clarify the concepts embedded in a hypothesis or research question with
More informationSection 4 - Dealing with Anxious Thinking
Section 4 - Dealing with Anxious Thinking How do we challenge our unhelpful thoughts? Anxiety may decrease if we closely examine how realistic and true our unhelpful/negative thoughts are. We may find
More informationControlled Trials. Spyros Kitsiou, PhD
Assessing Risk of Bias in Randomized Controlled Trials Spyros Kitsiou, PhD Assistant Professor Department of Biomedical and Health Information Sciences College of Applied Health Sciences University of
More informationLets Talk about Down Syndrome & Family Mental Health. Yona Lunsky, PhD CPsych October 28, 2017
Lets Talk about Down Syndrome & Family Mental Health Yona Lunsky, PhD CPsych October 28, 2017 What is mental health? Whose mental health are we talking about? Whose mental health are we talking about?
More informationWorking Papers Project on the Public and Biological Security Harvard School of Public Health 17.
Working Papers Project on the Public and Biological Security Harvard School of Public Health 17. FLU VACCINE SURVEY Robert J. Blendon, Harvard School of Public Health, Project Director John M. Benson,
More informationLifestyle Education in Diabetes Care Bonnie L. Hinds
Lifestyle Education in Diabetes Care Bonnie L. Hinds RHAT Annual Conference November 2018 2 Diabetes Facts -- Diabetes rates have tripled over the last 20 years. -- Diabetes is the seventh leading cause
More informationexposure/intervention
Kathleen A. Kennedy, MD, MPH University of Texas Medical School at Houston Medical School Most clinical research evaluates an association between exposure/intervention outcome. 1 Does the investigator
More informationMethods for Addressing Selection Bias in Observational Studies
Methods for Addressing Selection Bias in Observational Studies Susan L. Ettner, Ph.D. Professor Division of General Internal Medicine and Health Services Research, UCLA What is Selection Bias? In the regression
More informationVersion No. 7 Date: July Please send comments or suggestions on this glossary to
Impact Evaluation Glossary Version No. 7 Date: July 2012 Please send comments or suggestions on this glossary to 3ie@3ieimpact.org. Recommended citation: 3ie (2012) 3ie impact evaluation glossary. International
More informationFamily Weekender. What to expect when you volunteer
Family Weekender What to expect when you volunteer Type 1 Events are a unique volunteering opportunity and we want to make your experience as rewarding as possible. Volunteering at one of our events isn
More informationNonviolent Communication
Nonviolent Communication Nonviolent Communication (NVC) is sometimes referred to as compassionate communication. Its purpose is to: 1. create human connections that empower compassionate giving and receiving
More informationCognitive Restructuring
Cognitive Restructuring Cognitive Restructuring Cognitive Restructuring is an evidence based intervention for the treatment of low mood or anxiety, recommended by the National Institute for Health and
More informationANA and Antibody Series Changes in ANA and Antibody Levels in Scleroderma
ANA and Antibody Series Changes in ANA and Antibody Levels in Scleroderma Background This article was prompted by an excellent question that was recently sent to the Scleroderma Education Project: You
More informationTechnical Track Session IV Instrumental Variables
Impact Evaluation Technical Track Session IV Instrumental Variables Christel Vermeersch Beijing, China, 2009 Human Development Human Network Development Network Middle East and North Africa Region World
More informationChapter 4 Review. Name: Class: Date: Multiple Choice Identify the choice that best completes the statement or answers the question.
Name: Class: Date: Chapter 4 Review Multiple Choice Identify the choice that best completes the statement or answers the question. 1. Use Scenario 4-1. The newspaper asks you to comment on their survey
More informationRisk Aversion in Games of Chance
Risk Aversion in Games of Chance Imagine the following scenario: Someone asks you to play a game and you are given $5,000 to begin. A ball is drawn from a bin containing 39 balls each numbered 1-39 and
More information20. Experiments. November 7,
20. Experiments November 7, 2015 1 Experiments are motivated by our desire to know causation combined with the fact that we typically only have correlations. The cause of a correlation may be the two variables
More informationThe t-test: Answers the question: is the difference between the two conditions in my experiment "real" or due to chance?
The t-test: Answers the question: is the difference between the two conditions in my experiment "real" or due to chance? Two versions: (a) Dependent-means t-test: ( Matched-pairs" or "one-sample" t-test).
More informationCASE STUDY 2: VOCATIONAL TRAINING FOR DISADVANTAGED YOUTH
CASE STUDY 2: VOCATIONAL TRAINING FOR DISADVANTAGED YOUTH Why Randomize? This case study is based on Training Disadvantaged Youth in Latin America: Evidence from a Randomized Trial by Orazio Attanasio,
More informationConfounding and Bias
28 th International Conference on Pharmacoepidemiology and Therapeutic Risk Management Barcelona, Spain August 22, 2012 Confounding and Bias Tobias Gerhard, PhD Assistant Professor, Ernest Mario School
More information716 West Ave Austin, TX USA
Practical Ethics for Fraud Examiners GLOBAL HEADQUARTERS the gregor building 716 West Ave Austin, TX 78701-2727 USA Introduction I. INTRODUCTION Ethics as a branch of philosophy has been developing since
More informationHere are a few ideas to help you cope and get through this learning period:
Coping with Diabetes When you have diabetes you may feel unwell and have to deal with the fact that you have a life long disease. You also have to learn about taking care of yourself. You play an active
More informationFeedback. What is feedback? Why is feedback important to humans?
Presentation Feedback Feedback What is feedback? Why is feedback important to humans? Helpful Feedback Helpful feedback is intentional or unintentional responses that come from others Naturally uplifting
More informationChapter 9 Experimental Research (Reminder: Don t forget to utilize the concept maps and study questions as you study this and the other chapters.
Chapter 9 Experimental Research (Reminder: Don t forget to utilize the concept maps and study questions as you study this and the other chapters.) In this chapter we talk about what experiments are, we
More informationOVERVIEW OF 15-MONTH FOLLOW-UP
MOST Operations Manual page 1 TABLE OF CONTENTS OVERVIEW OF 15-MONTH FOLLOW-UP 1. Overview of measurements...2 1.1 Telephone interview...2 1.2 Clinic visit...2 1.2.1 Clinic visit with x-ray (potential
More informationWhat is anxiety? What does it look like? Why is my child anxious? What can I do to help my child?
What is anxiety? What does it look like? Why is my child anxious? What can I do to help my child? Group Activity: How do you know when your child is anxious? What is anxiety? Anxiety is a feeling of unease,
More information