Research Design Source: John W. Creswell RESEARCH DESIGN Qualitative, Quantitative, and Mixed Methods Approaches Third Edition
The Three Types of Designs Three types Qualitative research Quantitative research Mixed methods research Research design Plan or proposal to conduct research Intersection of: Philosophical worldviews Strategies of inquiry Research methods
Strategies of Inquiry Quantitative Qualitative Mixed Methods Experimental designs Nonexperimental designs, such as surveys Narrative research Phenomenology Ethnographies Grounded theory studies Case study Sequential Concurrent Transformative
Research Methods Quantitative Methods Pre-determined Instrument-based questions Performance, attitude, observational, and census data Statistical analyses Statistical interpretation Mixed Methods Both pre-determined and emerging methods Both open- and closed-ended questions Multiple forms of data drawing on all possibilities Statistical and text analyses Across databases interpretation Qualitative Methods Emerging methods Open-ended questions Interview, observation, document, and audiovisual data Text and image analyses Themes, patterns interpretation
Criteria for Selecting a Research Design The Research Problem An issue or concern that needs to be addressed If the problem calls for Explanation or theory testing: Quantitative Exploration or understanding: Qualitative One approach alone is inadequate: Mixed methods Personal Experiences Training, preferences, time, resources Audience Advisors, journal editors, graduate committees, etc.
Significance and Meaning of a Purpose Statement A Qualitative Purpose Statement A Quantitative Purpose Statement A Mixed Methods Purpose Statement
Purpose Statement The purpose statement Is the central controlling idea in a study Is written in a sentence or several sentences Sets the objectives, the intent, or the major idea of a proposal or a study The purpose statement is NOT The problem leading to a need for the study The questions to be answered by collecting data
Quantitative Purpose Statements A good quantitative purpose statement contains: The variables (independent, dependent, and mediating, moderating or control) The participants The research site Include the following: Use words such as purpose, intent, or objective Identify the theory, model, or conceptual framework Use words to connect the independent and dependent variables such as related to or comparison of Position the independent before the dependent variable; other variables may go between the two or after the dependent variable Mention the specific strategy of inquiry Provide general definitions of key variables
A Script for Writing Quantitative Purpose Statements The purpose of this (experiment? survey?) study is (was? will be?) to test the theory of (theory name) that (compares? relates?) the (independent variable) to (dependent variable), controlling for (control variables) for (participants) at (the research site). The independent variable(s) will be defined as (provide a definition). The dependent variable(s) will be defined as (provide a definition), and the control and intervening variable(s) will be defined as (provide a definition).
Quantitative Research Questions and Hypotheses Quantitative researchers pose research questions or hypotheses to focus the study's purpose Quantitative research questions: Questions about the relationships among variables that the investigator seeks to know Quantitative hypotheses: Predictions that the researcher makes about the expected relationships among variables Predictions about the population values that the researcher will estimate based on data from a sample Quantitative objectives: Indicate a study's goals Used frequently in proposals for funding
Writing Quantitative Research Questions and Hypotheses Write questions or hypotheses, not both Consider 3 approaches to the variables for a question or hypothesis: Compare groups Relate variables Describe responses Specify questions and hypotheses based on theory if possible Measure the independent and dependent variables separately Generally use demographic information as intervening variables Use consistent words and ordering for independent and dependent variables
Scripts for Writing Quantitative Research Questions and Hypotheses Does (name the theory) explain the relationship between (independent variable) and (dependent variable), controlling for the effects of (control variable)? There is no significant difference between (the control and experimental groups on the independent variable) on (dependent variable).
Forms for Writing Quantitative Research Questions and Hypotheses If writing hypotheses, use a consistent form: Null hypotheses (predict no difference or no relationship) Directional hypotheses (predict direction of difference or relationship) Nondirectional hypotheses (predict a difference or relationship, but not its direction) If writing research questions: First, specify descriptive questions for each important variable Next, state inferential questions that relate variables or compare groups Finally, add questions in which variables are controlled
Defining Surveys Survey Design To provide a quantitative description of trends, attitudes, or opinions of a population Components of a Survey Method Plan The Survey Design The Population and Sample Instrumentation Variables in the Study Data Analysis and Interpretation
A Survey Method Plan Instrumentation Provide detailed information about the survey instrument How developed, Pilot testing Sample items, Types of scores Describe the validity and reliability scores of past and/or current uses of the instrument Validity: whether one can draw meaning and useful inferences from scores on the instruments Reliability: whether scores resulting from past use are internally consistent, have high test-retest correlations, and result from consistent scoring Describe steps for administering survey and ensuring a high response rate
A Survey Method Plan Variables in the Study Relate the variables to research questions and items on the instrument Data Analysis and Interpretation Present the steps for analyzing the data Step 1. Report response rate Step 2. Determine response bias: the effect of nonresponses on survey estimates Step 3. Conduct descriptive analyses Step 4. Check instrument's scales Step 5. Conduct inferential statistical analyses (see Table 8.3) Step 6. Present and interpret results
Defining Experiments Experimental Design To test the impact of a treatment on an outcome, controlling for other factors that might influence that outcome Components of an Experimental Method Plan Participants Variables Instrumentation and Materials Experimental Procedures Threats to Validity
An Experimental Method Plan Participants Describe the selection of participants Random or convenience Describe the assignment of participants to groups Random or not; Consider matching participants Describe the procedures for determining the number of participants per group Variables Clarify the groups Identify the independent variable(s), including the treatment variable Identify the dependent variable(s), the outcomes
An Experimental Method Plan Instrumentation and Materials Discuss instruments development, items, and scales reliability and validity reports of past uses Thoroughly discuss materials used for the treatment Experimental Procedures Identify the type of experiment Pre-experimental, true experiment, quasi-experiment, and singlesubject designs Identify the type of comparisons: within-group or between-subject Provide a visual model X = treatment O = observation
An Experimental Method Plan Consider Threats to Validity Threats to internal validity: procedures, treatments, or experiences of the participants that threaten the researcher's ability to draw conclusions about cause and effect Threats to external validity: characteristics of the sample, setting, or timing that threaten the researcher's ability to generalize the conclusions to a population Threats to statistical conclusion validity: inadequate statistical power or violation of statistical assumptions that threaten the researcher's ability to draw statistical inferences Threats to construct validity: inadequate definitions and measures of variables that threaten the researcher's ability to measure relevant constructs
Threats to Validity Threats to Internal Validity History Maturation Regression Selection Mortality Diffusion of treatment Compensatory/resentful demoralization Compensatory rivalry Testing Instrumentation Threats to External Validity Interaction of selection and treatment Interaction of setting and treatment Interaction of history and treatment
An Experimental Method Plan The Procedure Describe in detail the procedure for conducting the experiment Procedures for pre-test post-test control group design Measure dependent variable as a pre-test Assign participants to matched pairs based on scores Randomly assign one member of each pair to the control and experimental group Expose experimental group to the treatment Measure dependent variable as a post-test from both groups Compare groups statistically
An Experimental Method Plan Data Analysis Report descriptive statistics (e.g., means, standard deviations, ranges) Conduct inferential statistical tests (e.g., t test, ANOVA, ANCOVA, or MANOVA) Use line graphs for single subject designs Report confidence intervals and effect sizes in addition to statistical tests Interpreting Results Discuss results, limitations, and implications