Threats to validity in intervention studies. Potential problems Issues to consider in planning

Size: px
Start display at page:

Download "Threats to validity in intervention studies. Potential problems Issues to consider in planning"

Transcription

1 Threats to validity in intervention studies Potential problems Issues to consider in planning

2 An important distinction Credited to Campbell & Stanley (1963) Threats to Internal validity Threats to External validity Problems we might encounter in this particular study. Problems that might limit the generalizability of the results Was it really the new program (the intervention) that caused a difference, or was it something else? Okay, the program seemed to work in this study, but would it work elsewhere? Next time? With new students, teachers, etc.?

3 Threats to Internal validity Outside events (History) Attrition Maturation Selection Practice (testing) Instrumentation Regression (statistical regression) Selection-Maturation Experimenter effects Subject expectations Diffusion

4 Outside events (History) Problems with issues that happen out in the real world Events might influence the outcome we are studying, but they are not really part of our new treatment News events The Sesame Street example There are almost always lots of factors that are influencing things, so other factors can have an influence on our participants Control groups can help deal with this

5 Attrition Drop outs A big issue in longitudinal studies College studies are a classic example Differential attrition Can make new programs look good e.g., if weaker students drop out Can make new programs look bad e.g., if weaker students are retained Or if weaker students drop out of control group Not always a problem in short-term

6 Maturation Normal development may occur while the new treatment is being studied An important issue with young children Comparison groups can help

7 Selection effects In comparative studies How are people selected for the treatment and nontreatment conditions? Volunteers? Intact groups? What biases might be present? This is the classic problem we encounter when we do not have a randomized experiment and have to use an observational study or a quasi-experimental design.

8 Practice effects Can occur with tests Also a potential problem with interviews Observations Other situations? Can be tricky to control

9 Instrumentation Changes in the instruments Different tests Different questions (in interviews or questionnaires) Different interviewers Different observers Different scoring systems Different raters

10 Regression to the mean Regression artifacts High scores tend to be not quite as high Low scores tend to be not quite so low Only an issue in special circumstances Can have important effects The Head Start example Here is a link to a nice discussion of regression to the mean: ward_mean.html

11 Selection-Maturation interaction Even if two groups appear to be similar at the beginning of the study (at pretest), they may be growing at different rates. (Or their future growth may be different.) This can be a challenge for quasi-experimental designs.

12 Expectancy effects Experimenter effects Even unintentional influences can play a role One group may be influenced more than another Subject expectations (participant expectations) Hawthorne effects (just being in a study can be an influence) Knowing about new treatment may be an influence Compensatory rivalry in control group Demoralized control (knowing about control group may be an influence) Placebo effects in medical trials How to control these?

13 Diffusion of treatment Some of the new treatment ideas might diffuse (spread) to others Teachers may share ideas with other teachers So control group is not quite what you wanted People may find alternative ways to gain access to some new program or idea (This might be desirable in some ways, but it can certainly be a limitation for research studies.)

14 Treatment implementation Was the new treatment really implemented? (What was the Fidelity of the treatment implementation? ) Sometimes the control group implements some of the new ideas (treatment diffusion) Other times the treatment classrooms may not fully implement the new ideas A tricky issue that we run into in lots of studies especially studies that take place over long periods of time. Things can get better over time Or things might get worse when the novelty wears off

15 Threats to External validity Limited samples Special arrangements Expectancy effects Sensitization Multiple treatments

Study Design. Svetlana Yampolskaya, Ph.D. Summer 2013

Study Design. Svetlana Yampolskaya, Ph.D. Summer 2013 Study Design Svetlana Yampolskaya, Ph.D. Summer 2013 Study Design Single point in time Cross-Sectional Research Multiple points in time Study only exists in this point in time Longitudinal Research Study

More information

Experimental and Quasi-Experimental designs

Experimental and Quasi-Experimental designs External Validity Internal Validity NSG 687 Experimental and Quasi-Experimental designs True experimental designs are characterized by three "criteria for causality." These are: 1) The cause (independent

More information

26:010:557 / 26:620:557 Social Science Research Methods

26:010:557 / 26:620:557 Social Science Research Methods 26:010:557 / 26:620:557 Social Science Research Methods Dr. Peter R. Gillett Associate Professor Department of Accounting & Information Systems Rutgers Business School Newark & New Brunswick 1 Overview

More information

Rival Plausible Explanations

Rival Plausible Explanations Ci Criminology i 860 Quasi-Experimentation and Evaluation Research Remember John Stuart Mill s 3 Criteria for inferring the existence of a causal relation: 1. Temporal precedence 2. Existence of a relationship

More information

Experimental Design Part II

Experimental Design Part II Experimental Design Part II Keith Smolkowski April 30, 2008 Where Are We Now? esearch eview esearch Design: The Plan Internal Validity Statements of Causality External Validity Statements of Generalizability

More information

Levels of Evaluation. Question. Example of Method Does the program. Focus groups with make sense? target population. implemented as intended?

Levels of Evaluation. Question. Example of Method Does the program. Focus groups with make sense? target population. implemented as intended? Program Evaluation Jonathan Howland, PhD, MPH 1 Formative Process Level Outcome Levels of Evaluation Question Example of Method Does the program Focus groups with make sense? target population. Was the

More information

Threats to Validity in Experiments. The John Henry Effect

Threats to Validity in Experiments. The John Henry Effect Threats to Validity in Experiments Office Hours: M 11:30-12:30, W 10:30-12:30 SSB 447 The John Henry Effect Housekeeping Homework 3, due Tuesday A great paper, but far from an experiment Midterm review

More information

Sociology 201: Social Research Design

Sociology 201: Social Research Design Sociology 201: Social Research Design 29. Review of the Course 1 Experiments Review deductive model--"traditional" model Theory Hypothesis Operationalization Hypothesis-testing Vitamin C preventing colds

More information

8 people got 100 (I rounded down to 100 for those who went over) Two had perfect exams

8 people got 100 (I rounded down to 100 for those who went over) Two had perfect exams Mid-Term Exams Understanding your grade The number printed on your answer sheet is the number correct (out of 27). Quadruple the number correct to learn your grade (out of 100 points) The mean score was

More information

OBSERVATION METHODS: EXPERIMENTS

OBSERVATION METHODS: EXPERIMENTS OBSERVATION METHODS: EXPERIMENTS Sociological Research Methods Experiments Independent variable is manipulated, and the dependent variable respond to the manipulation. e.g. Eating a chocolate bar prior

More information

Experimental Research. Types of Group Comparison Research. Types of Group Comparison Research. Stephen E. Brock, Ph.D.

Experimental Research. Types of Group Comparison Research. Types of Group Comparison Research. Stephen E. Brock, Ph.D. Experimental Research Stephen E. Brock, Ph.D., NCSP California State University, Sacramento 1 Types of Group Comparison Research Review Causal-comparative AKA Ex Post Facto (Latin for after the fact).

More information

Classic experimental design. Definitions. Decisions. Example: Manhattan Bail Project

Classic experimental design. Definitions. Decisions. Example: Manhattan Bail Project Classic experimental design Definitions Classic experimental design Experimental group O 1 O 2 O 2 -O 1 =d e = random assignment O = observation = experimental stimulus (= independent var) andomization

More information

In this chapter we discuss validity issues for quantitative research and for qualitative research.

In this chapter we discuss validity issues for quantitative research and for qualitative research. Chapter 8 Validity of Research Results (Reminder: Don t forget to utilize the concept maps and study questions as you study this and the other chapters.) In this chapter we discuss validity issues for

More information

Reliability and Validity checks S-005

Reliability and Validity checks S-005 Reliability and Validity checks S-005 Checking on reliability of the data we collect Compare over time (test-retest) Item analysis Internal consistency Inter-rater agreement Compare over time Test-Retest

More information

HPS301 Exam Notes- Contents

HPS301 Exam Notes- Contents HPS301 Exam Notes- Contents Week 1 Research Design: What characterises different approaches 1 Experimental Design 1 Key Features 1 Criteria for establishing causality 2 Validity Internal Validity 2 Threats

More information

CHAPTER LEARNING OUTCOMES

CHAPTER LEARNING OUTCOMES EXPERIIMENTAL METHODOLOGY CHAPTER LEARNING OUTCOMES When you have completed reading this article you will be able to: Define what is an experiment Explain the role of theory in educational research Justify

More information

VALIDITY OF QUANTITATIVE RESEARCH

VALIDITY OF QUANTITATIVE RESEARCH Validity 1 VALIDITY OF QUANTITATIVE RESEARCH Recall the basic aim of science is to explain natural phenomena. Such explanations are called theories (Kerlinger, 1986, p. 8). Theories have varying degrees

More information

UNIT III: Research Design. In designing a needs assessment the first thing to consider is level for the assessment

UNIT III: Research Design. In designing a needs assessment the first thing to consider is level for the assessment UNIT III: Research Design SWK 330 Kimberly Baker-Abrams What is a needs assessment? Why is it helpful? an assessment to determine the existence of services, client population, client access to services,

More information

PLS 506 Mark T. Imperial, Ph.D. Lecture Notes: Reliability & Validity

PLS 506 Mark T. Imperial, Ph.D. Lecture Notes: Reliability & Validity PLS 506 Mark T. Imperial, Ph.D. Lecture Notes: Reliability & Validity Measurement & Variables - Initial step is to conceptualize and clarify the concepts embedded in a hypothesis or research question with

More information

The Logic of Data Analysis Using Statistical Techniques M. E. Swisher, 2016

The Logic of Data Analysis Using Statistical Techniques M. E. Swisher, 2016 The Logic of Data Analysis Using Statistical Techniques M. E. Swisher, 2016 This course does not cover how to perform statistical tests on SPSS or any other computer program. There are several courses

More information

Research Approach & Design. Awatif Alam MBBS, Msc (Toronto),ABCM Professor Community Medicine Vice Provost Girls Section

Research Approach & Design. Awatif Alam MBBS, Msc (Toronto),ABCM Professor Community Medicine Vice Provost Girls Section Research Approach & Design Awatif Alam MBBS, Msc (Toronto),ABCM Professor Community Medicine Vice Provost Girls Section Content: Introduction Definition of research design Process of designing & conducting

More information

Georgina Salas. Topics EDCI Intro to Research Dr. A.J. Herrera

Georgina Salas. Topics EDCI Intro to Research Dr. A.J. Herrera Homework assignment topics 37-42 Georgina Salas Topics 37-42 EDCI Intro to Research 6300.62 Dr. A.J. Herrera Topic 37 1. What is the purpose of an experiment? The purpose of an experiment is to explore

More information

Lecture 3. Previous lecture. Learning outcomes of lecture 3. Today. Trustworthiness in Fixed Design Research. Class question - Causality

Lecture 3. Previous lecture. Learning outcomes of lecture 3. Today. Trustworthiness in Fixed Design Research. Class question - Causality Lecture 3 Empirical Research Methods IN4304 design (fixed design) Previous lecture A research plan Consider: purpose, theory, research question, mode inquiry, sampling method Research question Descriptive/

More information

Designing and Analyzing RCTs. David L. Streiner, Ph.D.

Designing and Analyzing RCTs. David L. Streiner, Ph.D. Designing and Analyzing RCTs David L. Streiner, Ph.D. Emeritus Professor, Department of Psychiatry & Behavioural Neurosciences, McMaster University Emeritus Professor, Department of Clinical Epidemiology

More information

The validity of inferences about the correlation (covariation) between treatment and outcome.

The validity of inferences about the correlation (covariation) between treatment and outcome. Threats Summary (MGS 9940, Fall, 2004-Courtesy of Amit Mehta) Statistical Conclusion Validity The validity of inferences about the correlation (covariation) between treatment and outcome. Threats to Statistical

More information

STATISTICAL CONCLUSION VALIDITY

STATISTICAL CONCLUSION VALIDITY Validity 1 The attached checklist can help when one is evaluating the threats to validity of a study. VALIDITY CHECKLIST Recall that these types are only illustrative. There are many more. INTERNAL VALIDITY

More information

One slide on research question Literature review: structured; holes you will fill in Your research design

One slide on research question Literature review: structured; holes you will fill in Your research design Topics Ahead Week 10-11: Experimental design; Running experiment Week 12: Survey Design; ANOVA Week 13: Correlation and Regression; Non Parametric Statistics Week 14: Computational Methods; Simulation;

More information

CHAPTER NINE DATA ANALYSIS / EVALUATING QUALITY (VALIDITY) OF BETWEEN GROUP EXPERIMENTS

CHAPTER NINE DATA ANALYSIS / EVALUATING QUALITY (VALIDITY) OF BETWEEN GROUP EXPERIMENTS CHAPTER NINE DATA ANALYSIS / EVALUATING QUALITY (VALIDITY) OF BETWEEN GROUP EXPERIMENTS Chapter Objectives: Understand Null Hypothesis Significance Testing (NHST) Understand statistical significance and

More information

Random Assignment/Selection DESIGNS AND THEIR LIMITATIONS. Limitations

Random Assignment/Selection DESIGNS AND THEIR LIMITATIONS. Limitations Designs and Their Limitations 153 Education level could be a moderator because it would explain the relationship between desire to quit and company citizenship. When you add education to the analysis,

More information

Previous Example. New. Tradition

Previous Example. New. Tradition Experimental Design Previous Example New Tradition Goal? New Tradition =? Challenges Internal validity How to guarantee what you have observed is true? External validity How to guarantee what you have

More information

Educational Research. S.Shafiee. Expert PDF Trial

Educational Research. S.Shafiee. Expert PDF Trial Educational Research: Experimental Studies Educational Research S.Shafiee Research... The systematic application of a family of methods employed to provide trustworthy information about problems an ongoing

More information

The following are questions that students had difficulty with on the first three exams.

The following are questions that students had difficulty with on the first three exams. The following are questions that students had difficulty with on the first three exams. Exam 1 1. A measure has construct validity if it: a) really measures what it is supposed to measure b) appears, on

More information

INVENTORY OF POSITIVE PSYCHOLOGICAL ATTITUDES (IPPA-32R) Self-Test Version

INVENTORY OF POSITIVE PSYCHOLOGICAL ATTITUDES (IPPA-32R) Self-Test Version ==RESOURCES FOR RESILIENCE== BUILDING A RESILIENT WORLDVIEW INVENTORY OF POSITIVE PSYCHOLOGICAL ATTITUDES (IPPA-32R) Self-Test Version Jared D. Kass, Ph.D., LMHC, Director Lynn Kass, M.A., M.A.T., LMHC,

More information

What We Will Cover in This Section

What We Will Cover in This Section Pre and Quasi-Experimental Design 9/2/2006 P767Intro.ppt 1 What We Will Cover in This Section What are they? Threats to internal validity. Pre-Experimental Designs Quasi- Experimental Designs. Summary

More information

GUIDE 4: COUNSELING THE UNEMPLOYED

GUIDE 4: COUNSELING THE UNEMPLOYED GUIDE 4: COUNSELING THE UNEMPLOYED Addressing threats to experimental integrity This case study is based on Sample Attrition Bias in Randomized Experiments: A Tale of Two Surveys By Luc Behaghel, Bruno

More information

What We Will Cover in This Section

What We Will Cover in This Section Pre and Quasi-Experimental 1 What We Will Cover in This Section Pre-Experimental Quasi- Experimental. Summary 2 Internal Validity Revisited 3 Common Threats to Internal Validity 1. History. 2. Maturation.

More information

Measuring and Assessing Study Quality

Measuring and Assessing Study Quality Measuring and Assessing Study Quality Jeff Valentine, PhD Co-Chair, Campbell Collaboration Training Group & Associate Professor, College of Education and Human Development, University of Louisville Why

More information

Regressions usually happen at the following ages 4 months, 8 months, 12 months, 18 months, 2 years.

Regressions usually happen at the following ages 4 months, 8 months, 12 months, 18 months, 2 years. A sleep regression is when your child s sleep becomes more challenging. This is usually due to your child reaching a developmental stage and learning something new. It s easy to tell when your child is

More information

Research Design. Source: John W. Creswell RESEARCH DESIGN. Qualitative, Quantitative, and Mixed Methods Approaches Third Edition

Research Design. Source: John W. Creswell RESEARCH DESIGN. Qualitative, Quantitative, and Mixed Methods Approaches Third Edition Research Design Source: John W. Creswell RESEARCH DESIGN Qualitative, Quantitative, and Mixed Methods Approaches Third Edition The Three Types of Designs Three types Qualitative research Quantitative research

More information

PYSC 224 Introduction to Experimental Psychology

PYSC 224 Introduction to Experimental Psychology PYSC 224 Introduction to Experimental Psychology Session 7 Extraneous Variables Lecturer: Dr. Margaret Amankwah-Poku, Dept. of Psychology Contact Information: mamankwah-poku@ug.edu.gh College of Education

More information

RESEARCH METHODS. Winfred, research methods, ; rv ; rv

RESEARCH METHODS. Winfred, research methods, ; rv ; rv RESEARCH METHODS 1 Research Methods means of discovering truth 2 Research Methods means of discovering truth what is truth? 3 Research Methods means of discovering truth what is truth? Riveda Sandhyavandanam

More information

Patrick Breheny. January 28

Patrick Breheny. January 28 Confidence intervals Patrick Breheny January 28 Patrick Breheny Introduction to Biostatistics (171:161) 1/19 Recap Introduction In our last lecture, we discussed at some length the Public Health Service

More information

RESEARCH METHODS. Winfred, research methods,

RESEARCH METHODS. Winfred, research methods, RESEARCH METHODS Winfred, research methods, 04-23-10 1 Research Methods means of discovering truth Winfred, research methods, 04-23-10 2 Research Methods means of discovering truth what is truth? Winfred,

More information

M6728. Goals. Depression Scores. Research Designs

M6728. Goals. Depression Scores. Research Designs M6728 Research Designs Goals Match appropriate research designs to the study purpose and questions Differentiate between experimental and non-experimental studies Discuss epidemiologic designs Evaluate

More information

Overview of the Logic and Language of Psychology Research

Overview of the Logic and Language of Psychology Research CHAPTER W1 Overview of the Logic and Language of Psychology Research Chapter Outline The Traditionally Ideal Research Approach Equivalence of Participants in Experimental and Control Groups Equivalence

More information

Chapter 9 Experimental Research (Reminder: Don t forget to utilize the concept maps and study questions as you study this and the other chapters.

Chapter 9 Experimental Research (Reminder: Don t forget to utilize the concept maps and study questions as you study this and the other chapters. Chapter 9 Experimental Research (Reminder: Don t forget to utilize the concept maps and study questions as you study this and the other chapters.) In this chapter we talk about what experiments are, we

More information

Evaluation and Assessment of Health Policy. Gerard F. Anderson, PhD Johns Hopkins University

Evaluation and Assessment of Health Policy. Gerard F. Anderson, PhD Johns Hopkins University This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike License. Your use of this material constitutes acceptance of that license and the conditions of use of materials on this

More information

PST-PC Appendix. Introducing PST-PC to the Patient in Session 1. Checklist

PST-PC Appendix. Introducing PST-PC to the Patient in Session 1. Checklist PST-PC Appendix Introducing PST-PC to the Patient in Session 1 Checklist 1. Structure of PST-PC Treatment 6 Visits Today Visit: 1-hour; Visits 2-8: 30-minutes Weekly and Bi-weekly Visits Teach problem

More information

Formative and Impact Evaluation. Formative Evaluation. Impact Evaluation

Formative and Impact Evaluation. Formative Evaluation. Impact Evaluation Formative and Impact Evaluation Formative Evaluation 2 An evaluation designed to produce qualitative and quantitative data and insight during the early developmental phase of an intervention, including

More information

Evidence Informed Practice Online Learning Module Glossary

Evidence Informed Practice Online Learning Module Glossary Term Abstract Associations Attrition Bias Background and Significance Baseline Basic Science Bias Blinding Definition An abstract is a summary of a research article. It usually includes the purpose, methods,

More information

Validity and Quantitative Research. What is Validity? What is Validity Cont. RCS /16/04

Validity and Quantitative Research. What is Validity? What is Validity Cont. RCS /16/04 Validity and Quantitative Research RCS 6740 6/16/04 What is Validity? Valid Definition (Dictionary.com): Well grounded; just: a valid objection. Producing the desired results; efficacious: valid methods.

More information

Roots of Empathy. Evaluation of. in Scotland Executive Summary. for Action for Children December 2015

Roots of Empathy. Evaluation of. in Scotland Executive Summary. for Action for Children December 2015 Evaluation of Roots of Empathy in Scotland 2014-15 Executive Summary for Action for Children December 2015 Qa Research Mill House, North Street, York, YO1 6JD 01904 632039 Dephna House, 24-26 Arcadia Ave,

More information

More on Experiments: Confounding and Obscuring Variables (Part 1) Dr. Stefanie Drew

More on Experiments: Confounding and Obscuring Variables (Part 1) Dr. Stefanie Drew 1 More on Experiments: Confounding and Obscuring Variables (Part 1) Dr. Stefanie Drew stefanie.drew@csun.edu 2 Previously, On Research Methods Basic structure of an experiment Now in more detail Internal

More information

TECH 646 Analysis of Research in Industry and Technology. Experiments

TECH 646 Analysis of Research in Industry and Technology. Experiments TECH 646 Analysis of Research in Industry and Technology Experiments Lecture note based on the Ch. 9 of the text book and supplemental materials: Cooper, D.R., & Schindler, P.S., Business Research Methods

More information

The Singapore Copyright Act applies to the use of this document.

The Singapore Copyright Act applies to the use of this document. Title Authors Source The use of multi comparison groups in quasi-experimental designs Lee Ngan Hoe, Agnes Chang Shook Cheong and Lee Peng Yee Asia-Pacific Education Research Association Conference 2008,

More information

Experimental Validity

Experimental Validity Experimental Validity 2/16/2003 P365 Experimental Validity 1 What We Will Cover in This Section Overview Internal Validity. External Validity. Key Learning Points. 2/16/2003 P365 Experimental Validity

More information

1. Temporal precedence 2. Existence of a relationship 3. Elimination of Rival Plausible Explanations

1. Temporal precedence 2. Existence of a relationship 3. Elimination of Rival Plausible Explanations Ci Criminology i 860 Experimentation: Deconstructing Donnerstein & Berkowitz Remember John Stuart Mill s 3 Criteria for inferring the existence of a causal relation 1. Temporal precedence 2. Existence

More information

Research Plan And Design

Research Plan And Design Research Plan And Design By Dr. Achmad Nizar Hidayanto Information Management Lab Faculty of Computer Science Universitas Indonesia Salemba, 1 Agustus 2017 Research - Defined The process of gathering information

More information

Topic #2. A key criterion in evaluating any test, measure, or piece of research is validity.

Topic #2. A key criterion in evaluating any test, measure, or piece of research is validity. ARTHUR SYC 302 (EXERIMENTAL SYCHOLOGY) 18C LECTURE NOTES [08/23/18] RESEARCH VALIDITY AGE 1 Topic #2 RESEARCH VALIDITY A key criterion in evaluating any test, measure, or piece of research is validity.

More information

Abhinav: So, Ephraim, tell us a little bit about your journey until this point and how you came to be an infectious disease doctor.

Abhinav: So, Ephraim, tell us a little bit about your journey until this point and how you came to be an infectious disease doctor. Announcer: Welcome to the Science is the Best Medicine podcast with your host Dr. Abhinav Sharma, exploring the pressing scientific and healthcare issues of our time. Dr. Abhinav Sharma: Superbugs we hear

More information

Sooner really isn t better TEENS AND DRINKING:

Sooner really isn t better TEENS AND DRINKING: Sooner really isn t better TEENS AND DRINKING: Enjoy your teenage years People often say that teens, i.e. you, are awkward. Perhaps. There s no doubt that transitioning from childhood to adulthood is a

More information

The Helping Orientations Inventory Open Materials 1

The Helping Orientations Inventory Open Materials 1 The Helping Orientations Inventory Open Materials 1 From Maki, A., Vitriol, J. A., Dwyer, P. C., Kim, J. S., & Snyder, M. (in press). The Helping Orientations Inventory: Measuring propensities to provide

More information

TECH 646 Analysis of Research in Industry and Technology. Experiments

TECH 646 Analysis of Research in Industry and Technology. Experiments TECH 646 Analysis of Research in Industry and Technology Experiments Lecture note based on the Ch. 9 of the text book and supplemental materials: Cooper, D.R., & Schindler, P.S., Business Research Methods

More information

Empirical Knowledge: based on observations. Answer questions why, whom, how, and when.

Empirical Knowledge: based on observations. Answer questions why, whom, how, and when. INTRO TO RESEARCH METHODS: Empirical Knowledge: based on observations. Answer questions why, whom, how, and when. Experimental research: treatments are given for the purpose of research. Experimental group

More information

Common Measurement Framework: Possible Front Runner Measures

Common Measurement Framework: Possible Front Runner Measures Common Measurement Framework: Possible Front Runner Measures WORKING TOWARDS A COMMON OUTCOMES FRAMEWORK 1 WORKING TOWARDS A COMMON OUTCOMES FRAMEWORK 1) Socially significant improvement of the mental

More information

Lecture 4: Research Approaches

Lecture 4: Research Approaches Lecture 4: Research Approaches Lecture Objectives Theories in research Research design approaches ú Experimental vs. non-experimental ú Cross-sectional and longitudinal ú Descriptive approaches How to

More information

A Patient s Guide to Chronic Pain Management

A Patient s Guide to Chronic Pain Management A Patient s Guide to Chronic Pain Management 264 Pleasant Street Concord, NH 03301 Phone: 6032243368 Fax: 6032287268 marketing.copa@concordortho.com DISCLAIMER: The information in this booklet is compiled

More information

Communication Research Practice Questions

Communication Research Practice Questions Communication Research Practice Questions For each of the following questions, select the best answer from the given alternative choices. Additional instructions are given as necessary. Read each question

More information

The Basics of Experimental Design [A Quick and Non-Technical Guide]

The Basics of Experimental Design [A Quick and Non-Technical Guide] The Basics of Experimental Design [A Quick and Non-Technical Guide] Sid Sytsma Website Administrator's Note: I have always considered Sid Sytsma's short article on experimental design one of the best short

More information

Survey Research Centre. An Introduction to Survey Research

Survey Research Centre. An Introduction to Survey Research Survey Research Centre An Introduction to Survey Research Outline Introduction to the SRC Beginning with data Two design choices: cross-sectional survey and experiments Coffee break Best practices Your

More information

Assessing Studies Based on Multiple Regression. Chapter 7. Michael Ash CPPA

Assessing Studies Based on Multiple Regression. Chapter 7. Michael Ash CPPA Assessing Studies Based on Multiple Regression Chapter 7 Michael Ash CPPA Assessing Regression Studies p.1/20 Course notes Last time External Validity Internal Validity Omitted Variable Bias Misspecified

More information

Lecture 15. There is a strong scientific consensus that the Earth is getting warmer over time.

Lecture 15. There is a strong scientific consensus that the Earth is getting warmer over time. EC3320 2016-2017 Michael Spagat Lecture 15 There is a strong scientific consensus that the Earth is getting warmer over time. It is reasonable to imagine that a side effect of global warming could be an

More information

DEPENDENT VARIABLE. TEST UNITS - subjects or entities whose response to the experimental treatment are measured or observed. Dependent Variable

DEPENDENT VARIABLE. TEST UNITS - subjects or entities whose response to the experimental treatment are measured or observed. Dependent Variable EXPERIMENT Experimental Research A RESEARCH INVESTIGATION IN WHICH CONDITIONS ARE CONTROLLED ONE INDEPENDENT VARIABLE IS MANIPULATED (SOMETIMES MORE THAN ONE) ITS EFFECT ON A DEPENDENT VARIABLE IS MEASURED

More information

PM12 Validity P R O F. D R. P A S Q U A L E R U G G I E R O D E P A R T M E N T O F B U S I N E S S A N D L A W

PM12 Validity P R O F. D R. P A S Q U A L E R U G G I E R O D E P A R T M E N T O F B U S I N E S S A N D L A W PM12 Validity P R O F. D R. P A S Q U A L E R U G G I E R O D E P A R T M E N T O F B U S I N E S S A N D L A W Internal and External Validity The concept of validity is very important in PE. To make PE

More information

UNIT 7 EXPERIMENTAL RESEARCH-I1

UNIT 7 EXPERIMENTAL RESEARCH-I1 UNIT 7 EXPERIMENTAL RESEARCH-I1 Structure 7.1 Introduction 7.2 Objectives 7.3 Types of Experimental Design 7.4 Pre-experimental Designs 7.4.1 One Shot Case Study Design 7.4.2 One Group Pre-test Post-test

More information

GCSE PSYCHOLOGY UNIT 2 FURTHER RESEARCH METHODS

GCSE PSYCHOLOGY UNIT 2 FURTHER RESEARCH METHODS GCSE PSYCHOLOGY UNIT 2 FURTHER RESEARCH METHODS GCSE PSYCHOLOGY UNIT 2 SURVEYS SURVEYS SURVEY = is a method used for collecting information from a large number of people by asking them questions, either

More information

Statistical Sampling: An Overview for Criminal Justice Researchers April 28, 2016

Statistical Sampling: An Overview for Criminal Justice Researchers April 28, 2016 Good afternoon everyone. My name is Stan Orchowsky, I'm the research director for the Justice Research and Statistics Association. It's my pleasure to welcome you this afternoon to the next in our Training

More information

Higher Psychology RESEARCH REVISION

Higher Psychology RESEARCH REVISION Higher Psychology RESEARCH REVISION 1 The biggest change from the old Higher course (up to 2014) is the possibility of an analysis and evaluation question (8-10) marks asking you to comment on aspects

More information

Research Methodologies

Research Methodologies Research Methodologies Quantitative, Qualitative, and Mixed Methods By Wylie J. D. Tidwell, III, Ph.D. www.linkedin.com/in/wylietidwell3 Consider... The research design is the blueprint that enables the

More information

Internal Validity and Experimental Design

Internal Validity and Experimental Design Internal Validity and Experimental Design February 18 1 / 24 Outline 1. Share examples from TESS 2. Talk about how to write experimental designs 3. Internal validity Why randomization works Threats to

More information

Public Policy & Evidence:

Public Policy & Evidence: Public Policy & Evidence: How to discriminate, interpret and communicate scientific research to better inform society. Rachel Glennerster Executive Director J-PAL Global Press coverage of microcredit:

More information

WHOLE HEALTH: INFORMATION FOR VETERANS. The Healing Power of Hope and Optimism

WHOLE HEALTH: INFORMATION FOR VETERANS. The Healing Power of Hope and Optimism WHOLE HEALTH: INFORMATION FOR VETERANS The Healing Power of Hope and Optimism Whole Health is an approach to health care that empowers and enables YOU to take charge of your health and well-being and live

More information

Neural circuits PSY 310 Greg Francis. Lecture 05. Rods and cones

Neural circuits PSY 310 Greg Francis. Lecture 05. Rods and cones Neural circuits PSY 310 Greg Francis Lecture 05 Why do you need bright light to read? Rods and cones Photoreceptors are not evenly distributed across the retina 1 Rods and cones Cones are most dense in

More information

COMMENTS ON THE PAPER OF PROF. PARTHA DASGUPTA

COMMENTS ON THE PAPER OF PROF. PARTHA DASGUPTA 21_Stiglitz_A_G.qxd:Layout 1 16-10-2008 12:07 Pagina 563 COMMENTS ON THE PAPER OF PROF. PARTHA DASGUPTA I thought Prof. Dasgupta s paper was extremely interesting and I think it raised a number of points.

More information

CRITICALLY APPRAISED PAPER

CRITICALLY APPRAISED PAPER CRITICALLY APPRAISED PAPER FOCUSED QUESTION Does wearing a weighted vest improve in-seat behavior, task completion speed, and attention-to-task in learners diagnosed with attention deficit hyperactivity

More information

Letter to the teachers

Letter to the teachers Letter to the teachers Hello my name is Sasha Jacombs I m 12 years old and I have had Type 1 Diabetes since I was four years old. Some of the people reading this may not know what that is, so I had better

More information

Chapter Three Research Methodology

Chapter Three Research Methodology Chapter Three Research Methodology Research Methods is a systematic and principled way of obtaining evidence (data, information) for solving health care problems. 1 Dr. Mohammed ALnaif METHODS AND KNOWLEDGE

More information

Introductory: Coding

Introductory: Coding Introductory: Coding Sandra Jo Wilson Editor, Education Coordinating Group Associate Director, Peabody Research Institute Research Assistant Professor, Dept. of Special Education Vanderbilt University,

More information

Preliminary Research Considerations. Lecture Overview. Stephen E. Brock, Ph.D., NCSP

Preliminary Research Considerations. Lecture Overview. Stephen E. Brock, Ph.D., NCSP Preliminary Research Considerations Stephen E. Brock, Ph.D., NCSP California State University, Sacramento 1 Lecture Overview Research Hypotheses Research Resources Exemptions from Human Subjects Committee

More information

Overview of Perspectives on Causal Inference: Campbell and Rubin. Stephen G. West Arizona State University Freie Universität Berlin, Germany

Overview of Perspectives on Causal Inference: Campbell and Rubin. Stephen G. West Arizona State University Freie Universität Berlin, Germany Overview of Perspectives on Causal Inference: Campbell and Rubin Stephen G. West Arizona State University Freie Universität Berlin, Germany 1 Randomized Experiment (RE) Sir Ronald Fisher E(X Treatment

More information

The Parent's Perspectives on Autism Spectrum Disorder

The Parent's Perspectives on Autism Spectrum Disorder Transcript Details This is a transcript of an educational program accessible on the ReachMD network. Details about the program and additional media formats for the program are accessible by visiting: https://reachmd.com/programs/autism-spectrum/the-parents-perspectives-on-autism-spectrumdisorder/6809/

More information

Experimental Methods in Health Research A. Niroshan Siriwardena

Experimental Methods in Health Research A. Niroshan Siriwardena Experimental Methods in Health Research A. Niroshan Siriwardena Community and Health Research Group: Improvement science and methods 2012 Introduction What is an experiment? Components of an experiment

More information

CogSysIII Lecture 4/5: Empirical Evaluation of Software-Systems

CogSysIII Lecture 4/5: Empirical Evaluation of Software-Systems CogSysIII Lecture 4/5: Empirical Evaluation of Software-Systems Human Computer Interaction Ute Schmid Applied Computer Science, Bamberg University last change October 30, 2007 CogSysIII Lecture 4/5: Empirical

More information

Research Landscape. Qualitative = Constructivist approach. Quantitative = Positivist/post-positivist approach Mixed methods = Pragmatist approach

Research Landscape. Qualitative = Constructivist approach. Quantitative = Positivist/post-positivist approach Mixed methods = Pragmatist approach Empirical Methods Research Landscape Qualitative = Constructivist approach Build theory from data Quantitative = Positivist/post-positivist approach Mixed methods = Pragmatist approach Experimental Analysis

More information

Realist Interviewing and Realist Qualitative Analysis

Realist Interviewing and Realist Qualitative Analysis Realist Interviewing and Realist Qualitative Analysis IIQM Webinar June 2017 Dr Gill Westhorp Professorial Research Fellow, Charles Darwin University Director, Community Matters Pty Ltd Associate, RMIT

More information

Use of the Quantitative-Methods Approach in Scientific Inquiry. Du Feng, Ph.D. Professor School of Nursing University of Nevada, Las Vegas

Use of the Quantitative-Methods Approach in Scientific Inquiry. Du Feng, Ph.D. Professor School of Nursing University of Nevada, Las Vegas Use of the Quantitative-Methods Approach in Scientific Inquiry Du Feng, Ph.D. Professor School of Nursing University of Nevada, Las Vegas The Scientific Approach to Knowledge Two Criteria of the Scientific

More information

We re going to talk about a class of designs which generally are known as quasiexperiments. They re very important in evaluating educational programs

We re going to talk about a class of designs which generally are known as quasiexperiments. They re very important in evaluating educational programs We re going to talk about a class of designs which generally are known as quasiexperiments. They re very important in evaluating educational programs and policies because often we might not have the right

More information

Introduction: Research design

Introduction: Research design Introduction: Research design 17.871 1 Statistics 1. a. In early use, that branch of political science dealing with the collection, classification, and discussion of facts (especially of a numerical kind)

More information

EXPERIMENTAL AND EX POST FACTO DESIGNS M O N A R A H I M I

EXPERIMENTAL AND EX POST FACTO DESIGNS M O N A R A H I M I EXPERIMENTAL AND EX POST FACTO DESIGNS M O N A R A H I M I EXPERIMENTAL AND EX POST FACTO DESIGN To strongly identify cause-and-effect relationships Experimental Design EXPERIMENTAL AND EX POST FACTO DESIGN

More information

What is Psychology? chapter 1

What is Psychology? chapter 1 What is Psychology? chapter 1 Overview! The science of psychology! What psychologists do! Critical and scientific thinking! Correlational studies! The experiment! Evaluating findings What is psychology?

More information