RESEARCH METHODS A Process of Inquiry ANTHONY M. GRAZIANO MICHAEL L RAULIN STA TE UNIVERSITY OF NEW YORK A T BUFFALO tm HarperCollinsPublishers
CONTENTS Instructor's Preface xv Student's Preface xix 1 CURIOSITY, CREATIVITY, AJVD COMMITMENT 1 Science Is a Way of Thinking 2 Asking Questions 2 Science and Art 5 Common Methods of Acquiring Knowledge 6 Tenacity 6 Intuition 6 Authority 6 Rationalism 7 Empiricism 8 Science 9 Emerging Modern Science 9 Early Civilization 10 Greek Science 11 Medieval Science 14 The Scientific Revolution 15 The Science of Psychology 18 Summary 18 Review Exercises 20 2 RESEARCH IS A PROCESS OF INQUIRY 22 Basic Assumptions of Science 23 Observation and Inference: Facts and Constructs 23 Conceptual Models in Science 26 Inductive and Deductive Thinking 29 Models and Theories in Science 31 v
A Model of the Research Process 32 Phases of Research 32 Idea-generating Phase 32 / Problem-Definition Phase 34 / Procedures Design Phase 34 / Observation Phase 35 / Data-Analysis Phase 35 / Interpretation Phase 35 / Communication Phase 36 Levels of Constraint 37 Naturalistic Observation 40 / Case-Study Method of Observation 41 / Correlational Research 41 / Differential Research 41 / Experimental Research 41 Summary 42 Review Exercises 43 THE STARTING POINT: ASKING QUESTIONS 45 Asking Questions 46 Sources of Questions 46 Researcher's Personal Interests and Observations 46 / Other Investigators' Theories and Research 46 / Seeking Solutions to Practical Problems 47 Refining Questions for Research 48 Types of Variables in Research 51 Behavioral Variables 51 Stimulus Variables 51 Organismic or Subject Variables 52 Independent and Dependent Variables 52 Validity and the Control of Extraneous Variables 54 Research Ethics 55 Ethical Guides for Human Research 56 Ethical Checks 60 / Institutional Review Boards (IRBs) 61 Ethical Principles in Research with Animals 61 Summary 63 Review Exercises 64 DATA AND THE NATURE OF MEASUREMENT 67 Research Variables 68 Measurement 68 Scales of Measurement 71 Nominal Scales 71 Ordinal Scales 72 Interval Scales 72 Ratio Scales 73 Types of Data 73 Nominal Data 74 Ordered Data 74 Score Data 74
CONTENTS VII Measuring and Controlling Variables 75 Measurement Error 75 Operational Definitions 75 Reliability, Effective Range, and Validity 78 Scale Attenuation Effects 80 The Need for Objective Measurement 80 Statistical Analyses 81 Summary 81 Review Exercises 82 STATISTICAL ANALYSIS OF DATA 85 Individual Differences and Statistical Procedures 86 Descriptive Statistics 87 Frequency Counts and Distributions 89 Nominal and Ordinal Data 89 / Score Data 89 Graphical Representation of Data 90 Summary Statistics 93 Measures of Central Tendency: Mode, Median, and Mean 93 / Measures of Variability: Range, Variance, and Standard Deviation 95 / Measures of Relationship (Correlations) 99 Summary of Descriptive Statistics 102 Inferential Statistics 102 Populations and Samples 102 The Null Hypothesis 103 Statistical Decisions and Alpha Levels 104 Type I and Type II Errors 104 Testing for Mean Differences 104 Simple Mest 105 / Correlated Mest 105 / Analysis of Variance (ANOVA) 105 Summary 106 Review Exercises 106 NATÜRALISTIC AND CASE-STÜDY RESEARCH 109 Examples of Naturalistic Research 111 Examples of Case-Study Research 114 The Value of Low-Constraint Methods 116 Conditions for Using Low-Constraint Research 116 Information Gained from Low-Constraint Research 117 Problem Statements and Hypotheses in Naturalistic and Case-Study Research 120 Using Naturalistic and Case-Study Methods 121 Making Observations 121 Ethical Issues 123 Sampling of Subjects 124
VÜi CONTENTS Sampling of Situations 125 Sampling of Behaviors 125 Evaluating and Interpreting Data 125 Limitations of Naturalistic and Case-Study Methods 126 Poor Representativeness 126 Poor Replicability 127 Causal Inferences and Low-Constraint Research: The Ex Post Facto Fallacy 127 Limitations of the Observer 129 Summary 130 Review Exercises 130 7 CORRELATIONAL AND DIFFERENTIAL METHODS OF RESEARCH 133 Correlational Research Methods 134 Differential Research Methods 135 Artifacts and Confounding Variables 136 What Makes Differential Research Higher Constraint than Correlational Research? 138 When to Use Correlational and Differential Research 139 Conducting Correlational and Differential Research 140 Conducting Correlational Research 140 Measurement 140 / Sampling 141 / Analysis of Data 141 / Interpreting the Correlation 142 Conducting Differential Research 143 Measurement 143 / Selecting Appropriate Control Groups 143 / Sampling 146 / Analysis of Data 148 / Interpretation of Data 148 Limitations of Correlational and Differential Research 148 Problems in Determining Causation 148 Confounding Variables 150 Summary 150 Review Exercises 151 8 HYPOTHESIS TESTING, VALIDITY, AND THREATS TOVALIDITY 153 Hypothesis Testing 154 Statement of the Problem 155 Operational Definitions 157 Research Hypothesis 158 Testing the Research Hypothesis 159 Null Hypothesis 160 / Confounding Variable Hypothesis 160 / Causal Hypothesis 161 Summary 163
Validity and Threats to Validity 163 CONTENTS Types of Validity 164 Statistical Validity 164 / Construct Validity 164 / External Validity 165 / Internal Validity and Confounding Variables 166 Some Major Confounding Factors 168 Maturation 168 History 168 Testing 169 Instrumentation 169 Regression to the Mean 170 Selection 170 Attrition 170 Diffusion of Treatment 171 Sequencing Effects 171 Summary 172 Threats to Validity from Subject and Experimenter Effects 172 Subject Effects 172 Experimenter Effects 173 Statistical Versus Practical Significance 174 Validity, Control, and Constraint 175 Summary 175 Review Exercises 176 CONTROLS TO REDUCE THREATS TO VALIDITY IN EXPERIMENTAL RESEARCH 178 Threats to Validity 179 General Control Procedures 180 Preparation of Setting 180 Response Measurement 181 Replication 181 Control Over Subject and Experimenter Effects 182 Single- and Double-Blind Procedures 182 Automation 184 Using Objective Measures 185 Multiple Observers 185 Using Deception 185 Control Through Subject Selection and Assignment 187 Subject Selection 187 Random Sampling 188 / Stratified Random Sampling 189 / Ad hoc Samples 189 Subject Assignment 190 Free Random Assignment 191 / Matched Random Assignment 191 / Other Matching Procedures 193 Control Achieved Through Specific Experimental Design 194 The True Experiment 195
Summary 197 Review Exercises 198 CONTROL OF VARIANCE THROUGH EXPERIMENTAL DESIGN: SINGLE-VARIABLE, INDEPENDENT-GROUPS DESIGN 200 Experimental Design 201 Variance 201 Forms of Variance 202 Systematic Between-Groups Variance 202 / Nonsystematic Within-Groups Variance 203 Controlling Variance in Research 205 Maximizing Experimental Variance 205 / Controlling Extraneous Variance 205 / Minimizing Error Variance 206 Nonexperimental Research Designs 207 Ex Post Facto Design 207 Single-Group, Posttest-Only Design 208 Single-Group, Pretest-Posttest Design 209 Pretest-Posttest, Natural Control-Group Design 210 Experimental Research Designs 211 Designs for Testing One Independent Variable 211 Randomized, Posttest-Only, Control-Group Design 211 / Randomized, Pretest-Posttest, Control-Group Design 212 / Multilevel, Completely Randomized, Between-Subjects Design 213 / Solomon Four-Group Design 213 Summary 215 Statistical Analyses of Completely Randomized Designs 215 Mest 215 / Analysis of Variance (ANOVA) 215 / Specific Means Comparisons in ANOVA 219 Other Experimental Designs 220 Independent- Versus Correlated-Groups Designs 220 / Single-Variable Versus Multivariable or Factorial Designs 220 Summary 221 Review Exercises 221 CONTROL OF VARIANCE THROUGH EXPERIMENTAL DESIGN: SINGLE-VARIABLE, CORRELATED-GROUPS DESIGN 224 Correlated-Groups Design 225 Within-Subjects Design 226 Using Within-Subjects Design 226 / Analyzing Within-Subjects Design 228 / Strengths and Weaknesses of Within-Subjects Design 229 / Summary 233
CONTENTS Xi Matched-Subjects Design 233 Using Matched-Subjects Design 233 / Analyzing Matched-Subjects Design 236 / Strengths and Weaknesses of Matched-Subjects Design 237 / Summary 237 Single-Subject Design 238 Summary 238 Review Exercises 238 CONTROL OF VARIANCE THROUGH EXPERIMENTAE DESIGN: FACTORIAL DESIGN 240 Factorial Design 241 Main Effects and Interactions 242 Possible Outcomes of Factorial Design 246 An Example: Children's Dark-Fears Study 250 Analysis of Variance in Factorial Design 254 ANOVA Summary Table for Children's Dark-Fears Study 255 Variations of Basic Factorial Design 256 Within-Subjects (Repeated Measures) Factorial 256 Mixed Designs 258 Between-Subjects and Within-Subjects Variables 258 / Manipulated and Nonmanipulated Variables 260 / Summary 261 ANOVA: A Postscript 261 Summary 264 Review Exercises 264 FIELD RESEARCH: A SECOND LOOK AT RESEARCH IN NATURAL SETTINGS 267 Conducting Field Research 268 Reasons for Doing Field Research 268 Difficulties in Field Research 270 Quasi-experimental Design 271 Nonequivalent Control-Group Design 272 Interrupted Time-Series Design 276 Single-Subject Design 281 Reversal Design (ABA Design) 285 Multiple Baseline Design 287 Single-Subject, Randomized, Time-Series Design 288 Program Evaluation 291 Practical Problems in Program Evaluation Research 291 Issues of Control 292 Selecting Appropriate Dependent Measures 293 / Minimizing Bias in Dependent Measures 293 / Control Through Research Design in Program Evaluation 293
XÜ CONTENTS Typical Program Evaluation Designs 293 Randomized, Control-Group Design 293 / Single-Group, Time-Series Design 294 / Pretest-Posttest Design 294 Summary of Program Evaluation 294 Survey Research 294 Types of Surveys 294 Status Survey 294 / Survey Research 295 Steps in Survey Research 295 Developing the Instrument 295 / Sampling Considerations 295 / Sampling Procedures 296 / Sample Size and Confidence Intervals 297 Research Design of Surveys 297 Cross-Sectional Design 297 / Longitudinal Design 298 Summary of Survey Research 298 Longitudinal and Cross-Sectional Research 298 Summary 298 Review Exercises 299 14 SELECTING APPROPRIATE STATISTICAL PROCEDURES 302 An Initial Example 303 A Decision-Tree Model 306 Decision-Tree Flowcharts 306 Identifying Research Variables 307 Identifying Major Characteristics of the Study 312 Selecting Appropriate Descriptive and Inferential Statistics 313 Statistical Procedures for the First Hypothesis 313 / Statistical Procedures for Other Hypotheses 314 Secondary Analyses 314 Post hoc Analyses 315 / Secondary Analyses to Help Explain Results 315 / Data Snooping 315 Caveats and Disclaimers 315 Summary 317 Review Exercises 317 15 RESEARCH METHODOLOGY: AN EVOLVEVG DISCIPLINE 320 New Directions in Research Methodology 322 New Questions and New Methods 322 The Evolution of Research Questions and Methods 322 / The Impact of Other Disciplines 323 / Multidisciplinary Research 324 / Moving Research Out of the Laboratory 324 / The Impact of Computers 324 Science: An Interaction Between Empiricism and Rationalism 327 Review Exercise 328
CONTENTS XÜi APPENDIX A Writing a Research Report: APA Publication Style 329 Structure of a Research Article 330 Writing the Research Report 330 Using Levels of Headings to Organize 330 Sections of a Research Report 331 Title Page 331 / Abstract 331 / Introduction 332 / Method 332 / Results 333 / Discussion 335 / References 335 Writing Style 336 Summary 341 APPENDIX B Statistical Tables 342 B.1 Area Under the Standard Normal Curve 343 B.2 Critical Values of Student's t 345 B.3 Critical Values of the F Distribution 346 B.4 Critical Values of the Pearson Product-Moment Correlation 350 B.5 Critical Values of the Spearman Rank-Order Correlation 351 B.6 Probability Values of the Chi-Square Distribution 351 B.7 Random Numbers 352 APPENDIX C Statistical Computation Procedures 357 Descriptive Statistics 358 Measures of Central Tendency 358 Mean 358 / Median 358 / Mode 358 Measures of Variability 358 Variance 358 / Standard Deviation 359 Measures of Relationship 359 Pearson Product-Moment Correlation 359 / Spearman Rank-Order Correlation 360 Inferential Statistics 360 Parametric Statistics 360 Independent Samples /-Test 361 / Correlated /-Test 362 / Analysis of Variance (ANOVA) 363 Nonparametric Statistics 371 Chi-Square Goodness-of-Fit Test 371 / Chi-Square Test for Independence 371 / Cautions Regarding Chi-Square 372 / Mann-Whitney U-Jesi 373 / Other Nonparametric Statistics 374 Summary 374
XiV CONTENTS APPENDIX D Selected Answers to Section III Exercises 375 Glossary 383 References 399 Name Index 405 Subject Index 407