Experimental Design Part II
|
|
- Maximillian George
- 5 years ago
- Views:
Transcription
1 Experimental Design Part II Keith Smolkowski April 30, 2008 Where Are We Now? esearch eview esearch Design: The Plan Internal Validity Statements of Causality External Validity Statements of Generalizability Designs Experimental Designs Quasi-experimental Designs Poor Designs Non-Experimental Designs 1 2 External Validity Statements of Generalizability Extent to which findings can be applied to other individuals or groups and other settings. Two aspects of external validity Population validity: extent to which you can generalize results to a specified larger group. Ecological validity: extent to which you can generalize results to other environmental conditions. A variety of threats to external validity identified by Glass and Bract (1968). Threats to External Validity Population Validity Population validity: generalization from a sample to a specific larger group Threats Inability to generalize from experimental sample to a defined population, the target population The extent to which personological variables interact with treatment effects 3 4 Generalize from experimental sample to a defined population Public schools and students by type of locale: Sample may differ from target population on location or environment Can only generalize to the population from which the sample was drawn Compare populations on relevant characteristics and hope results will generalize isky 5 regon Mississippi California 6
2 Interaction between personoligical variables and treatment effects Sample may differ from target population on personological variables Personological variables include locus of control, gender, SES, education, alcohol consumption, anxiety level, various academic or social skills, confidence level, and so on Public school student membership by racial & ethnic category regon Hawaii Mississippi 7 8 Threats to External Validity, Ecological Validity Explicit Description of IV Ecological Validity: Generalization from study conditions to a different environment 1) Explicit description of experimental treatment 2) Hawthorne effect 3) Multiple treatment interference 4) Novelty and disruption effects 5) Experimenter effect 6) Pretest sensitization 7) Posttest sensitization 8) Interaction of history and treatment 9) Measurement of dependent variable 10) Interaction of time of measurement and treatment effects 9 Give a complete and detailed description of method such that another researcher could replicate the procedure Write out steps Draw pictures Build a timeline 4. Infect with a cold virus 5. Lock treatment group in cold room 6. Have participants count symptoms 10 Multiple-Treatment Interference Hawthorne Effect Use multiple interventions in an uncontrolled manner Which treatment caused the change? Cannot generalize findings to situations with only one treatment Design studies that systematically separate the participants into different intervention groups I know you re watching me phenomenon When knowledge of the study influences performance by participants Aware of the aims or hypothesis eceive special attention (Not really the cause of the problem in the original Hawthorne study; see Gilbert, 1996) 11 12
3 Novelty and Disruption Effects Experimenter Effect Novel interventions may cause changes in DV simply because they are new or different Disruption effects occur when a disruption in routine inhibits performance Effectiveness or ineffectiveness of intervention due to the individual administering the intervention Importance of replication Pretest Sensitization Posttest Sensitization Pretest may interact with intervention and produce different results than if the participants had not taken the pretest Pretest clues in the participants Applies to both groups Most common with self-report of attitudes or personality Posttest is itself a learning experience Participants performance affected by the the test Test extends the intervention Helps participants put the pieces together Practice in intervention was probably insufficient Interaction of History and Treatment Effects Measurement of DV It may be difficult to generalize the finding outside of the time period in which the research was done Example: School Safety Study March to June 98: Intervene to improve safety in non-classroom settings May 98: Thurston Shooting Difficult to avoid! 17 esults limited to the particular mode of assessment used in the study Example: Project DAE Sample: 6 th graders Intervention: Police officer DAE s kids to say no to drugs Assessment officer plays a drug dealer: Wanna buy some drugs? esults: DAE 6 th graders say no DAE unrelated to later drug use 18
4 Interaction of Time of Measurement and Treatment Effects Threats to External Validity Population Validity eview esults may depend on the amount of time between intervention and assessment Last day of intervention A month later A year later Is assessment immediately after intervention better? Population validity: generalization from a sample to a specific larger group Threats Inability to generalize from experimental sample to a defined population, the target population The extent to which personological variables interact with treatment effects Threats to External Validity, Ecological Validity eview Threats to Study Validity in Examples on Slides 18, 19, & 20 from Part I Ecological Validity: Generalization from your study to some other environment Generalization from research project to real world Lab to School Clinic to Home Threats Explicit treatment description Hawthorne effect Multiple treatment interference Novelty and disruption Experimenter effects Pretest sensitization Posttest sensitization Interaction of history & treatment Measurement of DV Interaction of time of measurement & treatment 21 a) Experimental treatment diffusion (i9), also called contamination. b) History (i1). c) Controls for differential selection (i6). d) A true placebo may control for experimental treatment diffusion (i9), compensatory equalization of treatments (i11) or resentful demoralization of the control group (i12). e) Multi-treatment interference (e2) threat to external validity. i = internal, p = population, e = ecological f) Not a typical threat to internal validity, but probably falls under maturation (i2). g) Differential selection (i6). h) Compensatory rivalry by the control group (i10). i) Generalization from sample to undefined population (i6; see also slides 5-8 in Part II). j) Statistical regression towards the mean (i5). k) Testing they become test wise (i3). l) Hawthorne effect (e3). 22 Group Designs Experimental Designs Experimental Show Causality Two or More Groups Comparisons Between Groups andom Assignment Equivalent Groups No Adjustment Unnecessary Manipulate IV Provide Intervention ne or More IVs Separate Groups for IVs Study Designs Quasi-Experimental Imply Causality Two or More Groups Comparisons Between Groups Not andom Assignment Nonequivalent Groups Must Adjust for Differences Manipulate IV Provide Intervention ne or More IVs Separate Groups for IVs Nonexperimental Show elationships Descriptive Correlational Causal Comparative Case Studies Single Group Pre-Post 23 Shows Causality equirements andomly assigned groups, two or more Manipulation of one IV (or more) Examples Post-only control group design Pre-post control-group design Solomon four group design Factorial designs 24
5 Post-nly Control Group Design Pre-Post Control Group Design Collect data () only after manipulation of the IV Interv n: X or X 1 Control: <blank> or X 2 Internal (causation): differential mortality External (generalization): potentially many X Alternate Specification X 1 X 2 Data collection () before & after intervention (X) Internal: attrition, treatment diffusion External: pretest sensitization, novelty X Solomon 4-Group Design Factorial Designs Combination of post-only and pre-post designs Ideal but difficult to use equires larger sample size Cumbersome analysis Internal (causality): attrition External (generalizability): experimenter, disruption, but not pretest sensitization X X Similar to pre-post, but with two IVs X = intervention (drug) Y = setting (cold room) Internal: mortality External: interaction of testing & treatment X 1 X 1 X 2 X 2 Y 1 Y 2 Y 1 Y Unit of Analysis: Individuals Unit of Analysis: Groups Standard procedure Sample individuals andomize individuals to groups Unit of analysis = individuals Analyze individuals Inference based on individuals But what if you cannot recruit and assign individuals, only intact groups? 29 Challenges to standard procedure Can only recruit intact groups, say, classrooms Intervention applies to groups, such as schools or communities Alternative procedure andomize groups (e.g., classrooms, cities) Unit of analysis = groups Intervention applies to groups Analyze group means (other options available) Inference based on groups equires more groups & more people 30
6 Quasi-Experimental Designs Static Group Comparison Quasi means resembling equirements Two or more groups, Not randomly assigned Manipulation of one or more IVs Examples Static-group comparison design Nonequivalent-group design Interrupted time series design Exactly like a post-only design, but no random assignment Threats to validity (examples) Internal: differential selection, mortality External: interaction of selection and treatment Not andom Design X Nonequivalent Group Designs Interrupted Time Series Design Similar to experimental pre-post design, but not randomized Internal (causality): differential selection External: Interaction of testing and treatment Experimenter effect Not andom X 33 X Many assessments with IV in the middle Threats to Validity (Examples) Internal (causality): history, maturation External (generalizability): interaction of testing and treatment Contrast with true single-case research, which can achieve good internal validity (single-subject research sequence highly recommended) 34 egression Discontinuity Nonexperimental Designs Two-group design Assign to condition based on pretest Cut score must be used for assignment Accounts for selection bias Ex: Assign all students reading below 20 on Beck Depression Inventory to intervention Discontinuity in regression egress pretest on posttest Test for discontinuity at assignment cut score Show nly elationships equirements: Few Examples Descriptive Correlational Causal comparative design Case studies Single-group pre-post 35 36
7 Correlational Designs Causal Comparative Designs Also: ex post facto studies Correlation does not imply causation Determining the relationship between two variables Example: Teacher training and student performance in 1 st grade Variable 1: Hours spent in practicum Variable 2: Students reading scores 37 Studies temporal relationships Suspected cause: past event or experience Likely effect: present event or state Experimental control is not possible Shows relationships over time Cannot to establish causality Example: Gang membership School dropout may lead to gang membership Alternatives: poor school environment leads to both dropout and gang membership 38 ne group pretest-posttest Group Design eview Assess, intervene, assess a bad idea Threats to Validity (Examples) Internal Design History Maturation Testing X Interaction of selection and other factors External Interaction of testing and treatment Interaction of selection and treatment How do design types differ? Experimental? Quasi-experimental? Nonexperimental? Valid experiments What is internal validity? What is external validity? Single-Case esearch (a digression) Experimental Validity eview Powerful, flexible, and parsimonious Very different from group designs 1 to 5 participants Each participant serves as his or her own control May achieve excellent internal and external validity with appropriate design Many experimental designs Multiple baseline ABAB Alternating treatments Beyond the scope of the current presentation Key sources Kennedy, Single Case Designs for Ed. esearch, 2005 Zhan & ttenbacher, Single subject research designs... in Disability and ehab., 2001, 23(1) Carr, A classroom demonstration of singlesubject research designs in Teaching of Psychology, 1997, 24(3) Fisher, Kelley, & Lomas, Visual aids and structured criteria for... single-case designs in JABA, 2003, 36(3) 41 Internal Validity Valid Statements about causality Can we draw conclusions about cause? External Validity Valid statements about generalization Can we expect the same results at other places or times, with other people, & with the intervention we reported? 42
8 Design eview the Plan Design: esearch Question Hypothesis Statement, esearch Question Design overview: timeline or design figure Participants: sampling & recruitment Intervention (IV): theory, implementation, fidelity, strength Data collection (DV): Measures with reliability & validity Carefully identify procedures & timing Intended analysis & power Critique: strengths & weaknesses 43 elationship between IV(s) and DVs Identifies the effect of the IV on the DV for the study sample Must represent falsifiable hypotheses Suggests empirical tests Implies measurable, well defined DVs State clearly and unambiguously (Kerlinger & Lee, 2000) 44 Design: verview & Sample Design: Independent Variable verview Draw design picture or timeline Define relationships among variables Sample To whom do I want to generalize? How will I sample participants? How do I assign participants? How large of a sample do I need? perational definition of IV Expected strength of IV Fidelity of Implementation the extent to which the treatment conditions, as implemented, conform to the researcher s specifications (G, B, & G, 1996, p. 481) Also called Treatment Fidelity Design: Dependent Variable Design: Analysis & Critique Choose measures carefully Borrow measures in research literature Create new, which usually requires considerable pilot work eport reliability & validity Carefully identify procedures & assessment schedule 47 Choose an analysis method Factors to Consider esearch question Type of data collected Number of groups: treatment condition as well as other groups Type of design (e.g., pre-post, correlational) eport Power coming next Critique: strengths & weaknesses 48
9 Design: Power Analysis Design: Power Analysis (cont d) How large must my sample be? A big question Assumes true differences: how likely will we see them? eality & our glimpse of it eal world: not known ur sample: what we know We try to infer reality by what we know from our sample 49 A step back: two realities eality I: no difference in real world Assumption for statistical tests (not power) Type I error: we accidentally concluding we have a difference when it does not really exist The chance of Type I error: p-value or alpha (α) eality II: differences exist in real world Assumption for power Type II error: accidentally concluding we have no difference when one exists in reality The chance of Type II error: called beta (β) 50 Error Types Design: Computing Power Probability of Type I error: α Probability of Type II error: β What We Know from ur Sample (Test esults) No Difference Accept Null Groups Differ eject Null eal World Unknown to Us No Difference Correct Type I Error (α) Difference Type II Error (β) Correct 51 Important considerations Analysis: t-test, ANVA, correlation, etc. Alpha level: level of statistical significance chosen to reject null, often.05 or.01 Direction of hypothesis: one- or two-tailed Expected magnitude of effect, effect size Desired power: 1 β, often 80%, so β =.20 Attrition rate Consult Cohen (1988) or similar source for power tables or get G*Power (free) 52 Questions? esearch designs? Internal validity? Power? External validity? 53
Experimental Research. Types of Group Comparison Research. Types of Group Comparison Research. Stephen E. Brock, Ph.D.
Experimental Research Stephen E. Brock, Ph.D., NCSP California State University, Sacramento 1 Types of Group Comparison Research Review Causal-comparative AKA Ex Post Facto (Latin for after the fact).
More informationStudy Design. Svetlana Yampolskaya, Ph.D. Summer 2013
Study Design Svetlana Yampolskaya, Ph.D. Summer 2013 Study Design Single point in time Cross-Sectional Research Multiple points in time Study only exists in this point in time Longitudinal Research Study
More informationThreats to validity in intervention studies. Potential problems Issues to consider in planning
Threats to validity in intervention studies Potential problems Issues to consider in planning An important distinction Credited to Campbell & Stanley (1963) Threats to Internal validity Threats to External
More informationCHAPTER NINE DATA ANALYSIS / EVALUATING QUALITY (VALIDITY) OF BETWEEN GROUP EXPERIMENTS
CHAPTER NINE DATA ANALYSIS / EVALUATING QUALITY (VALIDITY) OF BETWEEN GROUP EXPERIMENTS Chapter Objectives: Understand Null Hypothesis Significance Testing (NHST) Understand statistical significance and
More information26:010:557 / 26:620:557 Social Science Research Methods
26:010:557 / 26:620:557 Social Science Research Methods Dr. Peter R. Gillett Associate Professor Department of Accounting & Information Systems Rutgers Business School Newark & New Brunswick 1 Overview
More informationEXPERIMENTAL RESEARCH DESIGNS
ARTHUR PSYC 204 (EXPERIMENTAL PSYCHOLOGY) 14A LECTURE NOTES [02/28/14] EXPERIMENTAL RESEARCH DESIGNS PAGE 1 Topic #5 EXPERIMENTAL RESEARCH DESIGNS As a strict technical definition, an experiment is a study
More informationLecture 4: Research Approaches
Lecture 4: Research Approaches Lecture Objectives Theories in research Research design approaches ú Experimental vs. non-experimental ú Cross-sectional and longitudinal ú Descriptive approaches How to
More informationRESEARCH METHODS. Winfred, research methods, ; rv ; rv
RESEARCH METHODS 1 Research Methods means of discovering truth 2 Research Methods means of discovering truth what is truth? 3 Research Methods means of discovering truth what is truth? Riveda Sandhyavandanam
More informationExperimental and Quasi-Experimental designs
External Validity Internal Validity NSG 687 Experimental and Quasi-Experimental designs True experimental designs are characterized by three "criteria for causality." These are: 1) The cause (independent
More informationRESEARCH METHODS. Winfred, research methods,
RESEARCH METHODS Winfred, research methods, 04-23-10 1 Research Methods means of discovering truth Winfred, research methods, 04-23-10 2 Research Methods means of discovering truth what is truth? Winfred,
More informationPSY 250. Designs 8/11/2015. Nonexperimental vs. Quasi- Experimental Strategies. Nonequivalent Between Group Designs
Nonexperimental vs. Quasi- Experimental Strategies PSY 250 Quasi and Non-experimental designs Chapter 10 Resemble experiments but contain some threat such as a confounding variable that can not be completely
More informationIn this chapter we discuss validity issues for quantitative research and for qualitative research.
Chapter 8 Validity of Research Results (Reminder: Don t forget to utilize the concept maps and study questions as you study this and the other chapters.) In this chapter we discuss validity issues for
More informationExperimental Research I. Quiz/Review 7/6/2011
Experimental Research I Day 3 Quiz/Review Quiz Review Normal Curve z scores & T scores More on the normal curve and variability... Theoretical perfect curve. Never happens in actual research Mean, median,
More information04/12/2014. Research Methods in Psychology. Chapter 6: Independent Groups Designs. What is your ideas? Testing
Research Methods in Psychology Chapter 6: Independent Groups Designs 1 Why Psychologists Conduct Experiments? What is your ideas? 2 Why Psychologists Conduct Experiments? Testing Hypotheses derived from
More informationSTATISTICAL CONCLUSION VALIDITY
Validity 1 The attached checklist can help when one is evaluating the threats to validity of a study. VALIDITY CHECKLIST Recall that these types are only illustrative. There are many more. INTERNAL VALIDITY
More informationTECH 646 Analysis of Research in Industry and Technology. Experiments
TECH 646 Analysis of Research in Industry and Technology Experiments Lecture note based on the Ch. 9 of the text book and supplemental materials: Cooper, D.R., & Schindler, P.S., Business Research Methods
More informationPLS 506 Mark T. Imperial, Ph.D. Lecture Notes: Reliability & Validity
PLS 506 Mark T. Imperial, Ph.D. Lecture Notes: Reliability & Validity Measurement & Variables - Initial step is to conceptualize and clarify the concepts embedded in a hypothesis or research question with
More information2013/4/28. Experimental Research
2013/4/28 Experimental Research Definitions According to Stone (Research methods in organizational behavior, 1978, pp 118), a laboratory experiment is a research method characterized by the following:
More informationCHAPTER LEARNING OUTCOMES
EXPERIIMENTAL METHODOLOGY CHAPTER LEARNING OUTCOMES When you have completed reading this article you will be able to: Define what is an experiment Explain the role of theory in educational research Justify
More informationVALIDITY OF QUANTITATIVE RESEARCH
Validity 1 VALIDITY OF QUANTITATIVE RESEARCH Recall the basic aim of science is to explain natural phenomena. Such explanations are called theories (Kerlinger, 1986, p. 8). Theories have varying degrees
More informationResearch Design. Source: John W. Creswell RESEARCH DESIGN. Qualitative, Quantitative, and Mixed Methods Approaches Third Edition
Research Design Source: John W. Creswell RESEARCH DESIGN Qualitative, Quantitative, and Mixed Methods Approaches Third Edition The Three Types of Designs Three types Qualitative research Quantitative research
More informationTopic #2. A key criterion in evaluating any test, measure, or piece of research is validity.
ARTHUR SYC 302 (EXERIMENTAL SYCHOLOGY) 18C LECTURE NOTES [08/23/18] RESEARCH VALIDITY AGE 1 Topic #2 RESEARCH VALIDITY A key criterion in evaluating any test, measure, or piece of research is validity.
More informationCommunication Research Practice Questions
Communication Research Practice Questions For each of the following questions, select the best answer from the given alternative choices. Additional instructions are given as necessary. Read each question
More informationEducational Research. S.Shafiee. Expert PDF Trial
Educational Research: Experimental Studies Educational Research S.Shafiee Research... The systematic application of a family of methods employed to provide trustworthy information about problems an ongoing
More informationOne slide on research question Literature review: structured; holes you will fill in Your research design
Topics Ahead Week 10-11: Experimental design; Running experiment Week 12: Survey Design; ANOVA Week 13: Correlation and Regression; Non Parametric Statistics Week 14: Computational Methods; Simulation;
More informationTECH 646 Analysis of Research in Industry and Technology. Experiments
TECH 646 Analysis of Research in Industry and Technology Experiments Lecture note based on the Ch. 9 of the text book and supplemental materials: Cooper, D.R., & Schindler, P.S., Business Research Methods
More informationHPS301 Exam Notes- Contents
HPS301 Exam Notes- Contents Week 1 Research Design: What characterises different approaches 1 Experimental Design 1 Key Features 1 Criteria for establishing causality 2 Validity Internal Validity 2 Threats
More informationResearch Questions, Variables, and Hypotheses: Part 2. Review. Hypotheses RCS /7/04. What are research questions? What are variables?
Research Questions, Variables, and Hypotheses: Part 2 RCS 6740 6/7/04 1 Review What are research questions? What are variables? Definition Function Measurement Scale 2 Hypotheses OK, now that we know how
More information9 research designs likely for PSYC 2100
9 research designs likely for PSYC 2100 1) 1 factor, 2 levels, 1 group (one group gets both treatment levels) related samples t-test (compare means of 2 levels only) 2) 1 factor, 2 levels, 2 groups (one
More informationDesign of Experiments & Introduction to Research
Design of Experiments & Introduction to Research 1 Design of Experiments Introduction to Research Definition and Purpose Scientific Method Research Project Paradigm Structure of a Research Project Types
More informationCHAPTER 8 EXPERIMENTAL DESIGN
CHAPTER 8 1 EXPERIMENTAL DESIGN LEARNING OBJECTIVES 2 Define confounding variable, and describe how confounding variables are related to internal validity Describe the posttest-only design and the pretestposttest
More informationChapter 10 Quasi-Experimental and Single-Case Designs
Chapter 10 Quasi-Experimental and Single-Case Designs (Reminder: Don t forget to utilize the concept maps and study questions as you study this and the other chapters.) The experimental research designs
More informationFormative and Impact Evaluation. Formative Evaluation. Impact Evaluation
Formative and Impact Evaluation Formative Evaluation 2 An evaluation designed to produce qualitative and quantitative data and insight during the early developmental phase of an intervention, including
More informationRESEARCH METHODS. A Process of Inquiry. tm HarperCollinsPublishers ANTHONY M. GRAZIANO MICHAEL L RAULIN
RESEARCH METHODS A Process of Inquiry ANTHONY M. GRAZIANO MICHAEL L RAULIN STA TE UNIVERSITY OF NEW YORK A T BUFFALO tm HarperCollinsPublishers CONTENTS Instructor's Preface xv Student's Preface xix 1
More informationEvaluating Social Programs Course: Evaluation Glossary (Sources: 3ie and The World Bank)
Evaluating Social Programs Course: Evaluation Glossary (Sources: 3ie and The World Bank) Attribution The extent to which the observed change in outcome is the result of the intervention, having allowed
More informationinvestigate. educate. inform.
investigate. educate. inform. Research Design What drives your research design? The battle between Qualitative and Quantitative is over Think before you leap What SHOULD drive your research design. Advanced
More informationVariables Research involves trying to determine the relationship between two or more variables.
1 2 Research Methodology Week 4 Characteristics of Observations 1. Most important know what is being observed. 2. Assign behaviors to categories. 3. Know how to Measure. 4. Degree of Observer inference.
More informationEmpirical Knowledge: based on observations. Answer questions why, whom, how, and when.
INTRO TO RESEARCH METHODS: Empirical Knowledge: based on observations. Answer questions why, whom, how, and when. Experimental research: treatments are given for the purpose of research. Experimental group
More informationLecture 3. Previous lecture. Learning outcomes of lecture 3. Today. Trustworthiness in Fixed Design Research. Class question - Causality
Lecture 3 Empirical Research Methods IN4304 design (fixed design) Previous lecture A research plan Consider: purpose, theory, research question, mode inquiry, sampling method Research question Descriptive/
More informationThe following are questions that students had difficulty with on the first three exams.
The following are questions that students had difficulty with on the first three exams. Exam 1 1. A measure has construct validity if it: a) really measures what it is supposed to measure b) appears, on
More informationLevels of Evaluation. Question. Example of Method Does the program. Focus groups with make sense? target population. implemented as intended?
Program Evaluation Jonathan Howland, PhD, MPH 1 Formative Process Level Outcome Levels of Evaluation Question Example of Method Does the program Focus groups with make sense? target population. Was the
More informationResearch Methodology. Characteristics of Observations. Variables 10/18/2016. Week Most important know what is being observed.
Research Methodology 1 Characteristics of Observations 1. Most important know what is being observed. 2. Assign behaviors to categories. 3. Know how to Measure. 4. Degree of Observer inference. 2 Variables
More informationThe essential focus of an experiment is to show that variance can be produced in a DV by manipulation of an IV.
EXPERIMENTAL DESIGNS I: Between-Groups Designs There are many experimental designs. We begin this week with the most basic, where there is a single IV and where participants are divided into two or more
More informationOBSERVATION METHODS: EXPERIMENTS
OBSERVATION METHODS: EXPERIMENTS Sociological Research Methods Experiments Independent variable is manipulated, and the dependent variable respond to the manipulation. e.g. Eating a chocolate bar prior
More informationChapter 9 Experimental Research (Reminder: Don t forget to utilize the concept maps and study questions as you study this and the other chapters.
Chapter 9 Experimental Research (Reminder: Don t forget to utilize the concept maps and study questions as you study this and the other chapters.) In this chapter we talk about what experiments are, we
More informationPrevious Example. New. Tradition
Experimental Design Previous Example New Tradition Goal? New Tradition =? Challenges Internal validity How to guarantee what you have observed is true? External validity How to guarantee what you have
More informationUNIT 7 EXPERIMENTAL RESEARCH-I1
UNIT 7 EXPERIMENTAL RESEARCH-I1 Structure 7.1 Introduction 7.2 Objectives 7.3 Types of Experimental Design 7.4 Pre-experimental Designs 7.4.1 One Shot Case Study Design 7.4.2 One Group Pre-test Post-test
More information11/2/2017. Often, we cannot manipulate a variable of interest
Often, we cannot manipulate a variable of interest Quasi-independent variables: Participant variable = individual characteristic used to select participants to groups Natural treatment = exposure in the
More informationTopic #6. Quasi-experimental designs are research studies in which participants are selected for different conditions from pre-existing groups.
ARTHUR PSYC 204 (EXPERIMENTAL PSYCHOLOGY) 17A LECTURE NOTES [03/08/17] QUASI-EXPERIMENTAL DESIGNS PAGE 1 Topic #6 QUASI-EXPERIMENTAL DESIGNS Again, central issue is one of research validity. Quasi-experimental
More informationDefine the population Determine appropriate sample size Choose a sampling design Choose an appropriate research design
Numbers! Observation Study: observing individuals and measuring variables of interest without attempting to influence the responses Correlational Research: examining the relationship between two variables
More information10/23/2017. Often, we cannot manipulate a variable of interest
Often, we cannot manipulate a variable of interest Quasi-independent variables: Participant variable = individual characteristic used to select participants to groups Natural treatment = exposure in the
More informationIndependent Variables Variables (factors) that are manipulated to measure their effect Typically select specific levels of each variable to test
Controlled Experiments experimental investigation of a testable hypothesis, in which conditions are set up to isolate the variables of interest ("independent variables") and test how they affect certain
More informationResearch Design & Protocol Development
Research Design & Protocol Development Paul J Kim, DPM, MS, FACFAS Associate Professor Director of Research Definitions Randomization Sampling Allocation Variables Independent Variable Treatment, intervention,
More informationThe validity of inferences about the correlation (covariation) between treatment and outcome.
Threats Summary (MGS 9940, Fall, 2004-Courtesy of Amit Mehta) Statistical Conclusion Validity The validity of inferences about the correlation (covariation) between treatment and outcome. Threats to Statistical
More informationResearch Approach & Design. Awatif Alam MBBS, Msc (Toronto),ABCM Professor Community Medicine Vice Provost Girls Section
Research Approach & Design Awatif Alam MBBS, Msc (Toronto),ABCM Professor Community Medicine Vice Provost Girls Section Content: Introduction Definition of research design Process of designing & conducting
More informationExperimental Validity
Experimental Validity 2/16/2003 P365 Experimental Validity 1 What We Will Cover in This Section Overview Internal Validity. External Validity. Key Learning Points. 2/16/2003 P365 Experimental Validity
More informationQuasi-Experimental and Single Case Experimental Designs. Experimental Designs vs. Quasi-Experimental Designs
1 Quasi-Experimental and Single Case Experimental Designs RCS 6740 6/30/04 Experimental Designs vs. Quasi-Experimental Designs Experimental Designs include: Random Assignment of Participants to groups
More informationThreats to Validity in Experiments. The John Henry Effect
Threats to Validity in Experiments Office Hours: M 11:30-12:30, W 10:30-12:30 SSB 447 The John Henry Effect Housekeeping Homework 3, due Tuesday A great paper, but far from an experiment Midterm review
More informationEXPERIMENTAL AND EX POST FACTO DESIGNS M O N A R A H I M I
EXPERIMENTAL AND EX POST FACTO DESIGNS M O N A R A H I M I EXPERIMENTAL AND EX POST FACTO DESIGN To strongly identify cause-and-effect relationships Experimental Design EXPERIMENTAL AND EX POST FACTO DESIGN
More informationClassic experimental design. Definitions. Decisions. Example: Manhattan Bail Project
Classic experimental design Definitions Classic experimental design Experimental group O 1 O 2 O 2 -O 1 =d e = random assignment O = observation = experimental stimulus (= independent var) andomization
More informationM6728. Goals. Depression Scores. Research Designs
M6728 Research Designs Goals Match appropriate research designs to the study purpose and questions Differentiate between experimental and non-experimental studies Discuss epidemiologic designs Evaluate
More informationWWC STUDY REVIEW STANDARDS
WWC STUDY REVIEW STANDARDS INTRODUCTION The What Works Clearinghouse (WWC) reviews studies in three stages. First, the WWC screens studies to determine whether they meet criteria for inclusion within the
More informationGeorgina Salas. Topics EDCI Intro to Research Dr. A.J. Herrera
Homework assignment topics 37-42 Georgina Salas Topics 37-42 EDCI Intro to Research 6300.62 Dr. A.J. Herrera Topic 37 1. What is the purpose of an experiment? The purpose of an experiment is to explore
More informationUNIT III: Research Design. In designing a needs assessment the first thing to consider is level for the assessment
UNIT III: Research Design SWK 330 Kimberly Baker-Abrams What is a needs assessment? Why is it helpful? an assessment to determine the existence of services, client population, client access to services,
More informationCHAPTER TEN SINGLE-SUBJECTS (QUASI-EXPERIMENTAL) RESEARCH
CHAPTER TEN SINGLE-SUBJECTS (QUASI-EXPERIMENTAL) RESEARCH Chapter Objectives: Understand the purpose of single subjects research and why it is quasiexperimental Understand sampling, data collection, and
More informationChapter Three Research Methodology
Chapter Three Research Methodology Research Methods is a systematic and principled way of obtaining evidence (data, information) for solving health care problems. 1 Dr. Mohammed ALnaif METHODS AND KNOWLEDGE
More informationchapter thirteen QUASI-EXPERIMENTAL AND SINGLE-CASE EXPERIMENTAL DESIGNS
chapter thirteen QUASI-EXPERIMENTAL AND SINGLE-CASE EXPERIMENTAL DESIGNS In the educational world, the environment or situation you find yourself in can be dynamic. You need look no further than within
More informationHypothesis Testing. Richard S. Balkin, Ph.D., LPC-S, NCC
Hypothesis Testing Richard S. Balkin, Ph.D., LPC-S, NCC Overview When we have questions about the effect of a treatment or intervention or wish to compare groups, we use hypothesis testing Parametric statistics
More informationInferences: What inferences about the hypotheses and questions can be made based on the results?
QALMRI INSTRUCTIONS QALMRI is an acronym that stands for: Question: (a) What was the broad question being asked by this research project? (b) What was the specific question being asked by this research
More informationOther Designs. What Aids Our Causal Arguments? Time Designs Cohort. Time Designs Cross-Sectional
Statistics in & Things to Consider Your Proposal Bring green folder of readings and copies of your proposal rough draft (one to turn in) Other Designs November 26, 2007 1 2 What Aids Our Causal Arguments?
More informationQUASI EXPERIMENTAL DESIGN
UNIT 3 QUASI EXPERIMENTAL DESIGN Factorial Design Structure 3. Introduction 3.1 Objectives 3.2 Meaning of Quasi Experimental Design 3.3 Difference Between Quasi Experimental Design and True Experimental
More informationPsych 1Chapter 2 Overview
Psych 1Chapter 2 Overview After studying this chapter, you should be able to answer the following questions: 1) What are five characteristics of an ideal scientist? 2) What are the defining elements of
More informationSample Exam Questions Psychology 3201 Exam 1
Scientific Method Scientific Researcher Scientific Practitioner Authority External Explanations (Metaphysical Systems) Unreliable Senses Determinism Lawfulness Discoverability Empiricism Control Objectivity
More informationValidity and Quantitative Research. What is Validity? What is Validity Cont. RCS /16/04
Validity and Quantitative Research RCS 6740 6/16/04 What is Validity? Valid Definition (Dictionary.com): Well grounded; just: a valid objection. Producing the desired results; efficacious: valid methods.
More informationExperimental Design. Dewayne E Perry ENS C Empirical Studies in Software Engineering Lecture 8
Experimental Design Dewayne E Perry ENS 623 Perry@ece.utexas.edu 1 Problems in Experimental Design 2 True Experimental Design Goal: uncover causal mechanisms Primary characteristic: random assignment to
More informationChapter 13. Experiments and Observational Studies. Copyright 2012, 2008, 2005 Pearson Education, Inc.
Chapter 13 Experiments and Observational Studies Copyright 2012, 2008, 2005 Pearson Education, Inc. Observational Studies In an observational study, researchers don t assign choices; they simply observe
More informationResearch Methods. Donald H. McBurney. State University of New York Upstate Medical University Le Moyne College
Research Methods S I X T H E D I T I O N Donald H. McBurney University of Pittsburgh Theresa L. White State University of New York Upstate Medical University Le Moyne College THOMSON WADSWORTH Australia
More informationPYSC 224 Introduction to Experimental Psychology
PYSC 224 Introduction to Experimental Psychology Session 6 Quasi Experiments and Faulty Experimental Designs Part 1 & 2 Lecturer: Dr. Margaret Amankwah-Poku, Dept. of Psychology Contact Information: mamankwah-poku@ug.edu.gh
More informationExperimental Research in HCI. Alma Leora Culén University of Oslo, Department of Informatics, Design
Experimental Research in HCI Alma Leora Culén University of Oslo, Department of Informatics, Design almira@ifi.uio.no INF2260/4060 1 Oslo, 15/09/16 Review Method Methodology Research methods are simply
More informationRival Plausible Explanations
Ci Criminology i 860 Quasi-Experimentation and Evaluation Research Remember John Stuart Mill s 3 Criteria for inferring the existence of a causal relation: 1. Temporal precedence 2. Existence of a relationship
More informationUse of the Quantitative-Methods Approach in Scientific Inquiry. Du Feng, Ph.D. Professor School of Nursing University of Nevada, Las Vegas
Use of the Quantitative-Methods Approach in Scientific Inquiry Du Feng, Ph.D. Professor School of Nursing University of Nevada, Las Vegas The Scientific Approach to Knowledge Two Criteria of the Scientific
More informationPolitical Science 15, Winter 2014 Final Review
Political Science 15, Winter 2014 Final Review The major topics covered in class are listed below. You should also take a look at the readings listed on the class website. Studying Politics Scientifically
More informationOverview of Experimentation
The Basics of Experimentation Overview of Experiments. IVs & DVs. Operational Definitions. Reliability. Validity. Internal vs. External Validity. Classic Threats to Internal Validity. Lab: FP Overview;
More informationThe Singapore Copyright Act applies to the use of this document.
Title Authors Source The use of multi comparison groups in quasi-experimental designs Lee Ngan Hoe, Agnes Chang Shook Cheong and Lee Peng Yee Asia-Pacific Education Research Association Conference 2008,
More informationStatistical Literacy in the Introductory Psychology Course
Statistical Literacy Taskforce 2012, Undergraduate Learning Goals 1 Statistical Literacy in the Introductory Psychology Course Society for the Teaching of Psychology Statistical Literacy Taskforce 2012
More informationDesigns. February 17, 2010 Pedro Wolf
Designs February 17, 2010 Pedro Wolf Today Sampling Correlational Design Experimental Designs Quasi-experimental Design Mixed Designs Multifactioral Design Sampling Overview Sample- A subset of a population
More informationResearch Landscape. Qualitative = Constructivist approach. Quantitative = Positivist/post-positivist approach Mixed methods = Pragmatist approach
Empirical Methods Research Landscape Qualitative = Constructivist approach Build theory from data Quantitative = Positivist/post-positivist approach Mixed methods = Pragmatist approach Experimental Analysis
More informationLecture 9 Internal Validity
Lecture 9 Internal Validity Objectives Internal Validity Threats to Internal Validity Causality Bayesian Networks Internal validity The extent to which the hypothesized relationship between 2 or more variables
More informationVillarreal Rm. 170 Handout (4.3)/(4.4) - 1 Designing Experiments I
Statistics and Probability B Ch. 4 Sample Surveys and Experiments Villarreal Rm. 170 Handout (4.3)/(4.4) - 1 Designing Experiments I Suppose we wanted to investigate if caffeine truly affects ones pulse
More informationOverview of Perspectives on Causal Inference: Campbell and Rubin. Stephen G. West Arizona State University Freie Universität Berlin, Germany
Overview of Perspectives on Causal Inference: Campbell and Rubin Stephen G. West Arizona State University Freie Universität Berlin, Germany 1 Randomized Experiment (RE) Sir Ronald Fisher E(X Treatment
More informationRESEARCH METHODS: PSYCHOLOGY AS A SCIENCE
RESEARCH METHODS: PSYCHOLOGY AS A SCIENCE TODAY S OBJECTIVE Explain basic methods of psychological research. IMPORTANT KEY TERMS Variable: anything that can change Theory: broad idea or set of closely
More informationSINGLE-CASE RESEARCH. Relevant History. Relevant History 1/9/2018
SINGLE-CASE RESEARCH And Small N Designs Relevant History In last half of nineteenth century, researchers more often looked at individual behavior (idiographic approach) Founders of psychological research
More informationWhat We Will Cover in This Section
Pre and Quasi-Experimental Design 9/2/2006 P767Intro.ppt 1 What We Will Cover in This Section What are they? Threats to internal validity. Pre-Experimental Designs Quasi- Experimental Designs. Summary
More informationExperimental Design and the struggle to control threats to validity
EXPERIMENTAL DESIGN Experimental Design and the struggle to control threats to validity INCREASINGLY CONSTRAINED LOW NATURALISTIC CASE-STUDY CORRELATIONAL DIFFERENTIAL EXPERIMENTAL HIGH Experimental design
More informationControlled Variables
Controlled Variables A controlled variable is not changed Also called constants Allow for a fair test Answers the question "What do I keep the same?" Students of different ages were given the same jigsaw
More informationResearch Designs. Internal Validity
Reviewing a few things Research Designs Review of a few things Demonstrations vs. Comparisons Experimental & Non-Experimental Designs IVs and DVs Between Group vs. Within-Group Designs Kinds of bivariate
More informationPSY 250. Experimental Design: The Basic Building Blocks. Simple between subjects design. The Two-Group Design 7/25/2015. Experimental design
Experimental Design: The Basic Building Blocks PSY 250 Experimental design The general plan for selecting participants, assigning participants to experimental conditions, controlling extraneous variables,
More informationChapter 13 Summary Experiments and Observational Studies
Chapter 13 Summary Experiments and Observational Studies What have we learned? We can recognize sample surveys, observational studies, and randomized comparative experiments. o These methods collect data
More informationCHAPTER EIGHT EXPERIMENTAL RESEARCH: THE BASICS of BETWEEN GROUP DESIGNS
CHAPTER EIGHT EXPERIMENTAL RESEARCH: THE BASICS of BETWEEN GROUP DESIGNS Chapter Objectives: Understand that the purpose of experiments and group quasi-experiments is to investigate differences between
More informationThe Basics of Experimental Design [A Quick and Non-Technical Guide]
The Basics of Experimental Design [A Quick and Non-Technical Guide] Sid Sytsma Website Administrator's Note: I have always considered Sid Sytsma's short article on experimental design one of the best short
More information