Chapter 9: Factorial Designs

Transcription

1 Chapter 9: Factorial Designs A. LEARNING OUTCOMES. After studying this chapter students should be able to: Describe different types of factorial designs. Diagram a factorial design. Explain the advantages and limitations of factorial designs. Discuss how factorial designs can be used to identify nonlinear effects. Describe why and how subject variables are often included in factorial designs, and what are the cautions needed when interpreting findings that involve subject variables. Discuss how factorial designs can be used to examine changes in behavior over time. Describe different types of outcomes that can occur in a factorial design that has two independent variables. Examine relatively simple sets of findings from factorial designs and identify whether main effects and interactions are likely to present. Describe the main effects that are possible in an experiment with three independent variables. Identify the total number of interactions possible in an experiment with three independent variables, and define the concept of a three-way interaction. B. KEYWORDS Between-subjects factorial design Factorial design Interaction Main effect Mixed-factorial design Person x situation factorial design Simple main effect Three-way interaction Two-way interaction Within-subjects factorial design C. BRIEF CHAPTER OUTLINE I. Basic Characteristics of Factorial Designs A. Describing a Factorial Design B. Advantages of Factorial Designs C. Limitations of Factorial Designs II. Designing a Factorial Experiment A. Examining Nonlinear Effects B. Incorporating Subject Variables C. Examining Changes in a Dependent Variable over Time 101

2 102 CHAPTER 9: Factorial Designs III. Understanding Main Effects and Interactions A. Possible Outcomes in a 2 x 2 Design B. Interactions and External Validity C. Analyzing the Results: General Concepts IV. Experiments with Three Independent Variables D. EXTENDED CHAPTER OUTLINE *Much of this summary is taken verbatim from the text. Introduction This chapter discusses factorial designs, which researchers use to study two or more independent variables within a single experiment. In a factorial design each level of an independent variable is combined with the each level of the other independent variables. Part I: Basic Characteristics of Factorial Designs The most basic factorial design is one in which two independent variables, each of which has two levels, are simultaneously manipulated to create four treatment conditions. A. Describing a factorial design. Factorial designs in which each participant engages in only one condition is a between-subjects factorial design. In contrast, a factorial design in which each participant engages in all treatment conditions is a within-subjects factorial design. A mixedfactorial design is one that has at least one between-subjects variable and one within-subjects variable. Factorial designs are typically described numerically. The most basic factorial design, for example, is a 2 x 2 design. The number of numbers indicates how many variables there are, and the value of each number indicates how many levels there are in the variable. The individual cells within a factorial design are typically described by name (i.e., child son condition, or hot humid condition). Alternatively, the components of a factorial design can be identified alphanumerically. For example, the levels of the first variable, A, are A1, A2, and so forth. The subsequent variables are identified the same way using the letters B, C, and so on. B. One advantage of a factorial design is that since most behaviors are caused by more than one factor, the design becomes a way to better approximate what happens in the real world. Other advantages of examining multiple variables in a single study include efficiency and the ability to examine situational factors on behavior. In addition, using multiple factors enables one to examine whether each has a main effect behavior, as well as whether factors combine to create interactions on behavior. a. A main effect is a single factor s overall effect on the dependent variable. In a factorial design the number of main effects is equal to the number of independent variables.

3 CHAPTER 9: Factorial Designs 103 b. An interaction occurs when the way in which an independent variable influences behavior differs depending on the level of another independent variable in which it s combined. The ability to examine whether unique combinations of independent variables affect a dependent variable is one of the greatest advantages of factorial designs. Some theories predict that behaviors are dependent upon interactions among variables, so factorial designs can be used to assess their validity. In addition, factorial designs can be used to test hypotheses about interactions as well as explore whether interactions exist. c. Moderator variables (discussed previously in Chapter 4) are factors that alter the strength or direction of the relation between an independent and dependent variable. Since an interaction is when an independent variable affects behavior but only at a specific level of another independent variable, the second variable is considered a moderator variable. For example, cellphone use while driving may impair driving performance but only when traffic density is high. In this example, traffic density is a moderator for whether cell phone use alters driving behavior. C. The key limitation of a factorial design is that as the number of independent variables increase, and as the number of levels of an independent variable increase, the total number of treatment conditions grows exponentially. In addition, factorial designs with many factors, or many levels of a factor, increase the complexity of a study, thereby making interpretation of its results difficult. Part II: Designing a Factorial Experiment A. Examining nonlinear effects. Factorial designs can be used to determine whether the nonlinear effects of one variable occur under different levels of another variable. Recall from Chapter 8 that nonlinear effects may be observed when an independent variable has three or more levels. B. Incorporating subject variables. Subject variables (see Chapter 8) are those factors that vary due to the characteristics of participants. a. Factorial designs allow a researcher to examine subject variables along with other variables. For example, in a person x situation factorial design, one subject variable (e.g., gender) with two or more levels is manipulated simultaneously along two or more levels of a situational variable (e.g., cognitive demand). b. As described in Chapter 8 one must think critically about subject variables when interpreting their results on behavior. This is true for simple, one-factor experiments as well as for factorial designs. Because subject variables provide naturally occurring treatment conditions, experimental control is sacrificed. Less experimental control means that one must be careful about making causal statements. C. Some factorial designs examine how an independent affects a dependent variable over time. The factor, time, is treated as an independent variable and must include at least two levels. In addition to calling this factor time, it may also be referred to as trial or testing session.

4 104 CHAPTER 9: Factorial Designs Part III: Understanding Main Effects and Interactions A. Possible outcomes in a 2 x 2 Design. Factorial designs are diagramed such that they create a table. A 2 x 2 design produces a table with two rows and two columns. The combination of rows and columns create cell means. In a 2 x 2 design there are four cell means. In addition, there are four marginal means, the average of a particular row or column. When interpreting the outcome of a factorial design one must ask the following questions: Is there a main effect for factor A? This is when the mean score of level A1 differs significantly from the mean score of level A2. This is determined by comparing the difference between the column marginal means, which are calculated based on all scores across the levels of B within the specific level of A associated with the column. Likewise, is there a main effect for factor B? This is when the mean score of the marginal mean for level B1 differs significantly from the mean score of the marginal mean for level B2. Finally, is there an interaction between A and B? This occurs when there is a difference between the levels of one factor, but only at a certain level of the other factor in which it s combined. A x B interactions are based on differences among cell means. The answer to these questions can produce a number of outcomes. A few examples include: a. Main effect without an interaction. This occurs when the means of the levels of one factor differ from one another in the same way at each level of the other factor. In a line graph this relationship is illustrated by two horizontal parallel lines. b. Two main effects without an interaction. This occurs when the means of the levels of one factor differ from one another at both levels of the other variable, but in a smaller (or greater) magnitude at one level of the one factor compared to the other. This relationship is illustrated on a line graph by two parallel lines that have an equal slope. c. Interaction with one main effect. In this outcome the means of the levels differ from one another at each level of the other factor. However, the direction of the difference between the two means is different at each level of the other factor. This relationship is illustrated by two intersecting lines. d. Interaction with two main effects. Here the means of the levels of an independent variable also differ at each level of the other variable. The direction of the difference between the two levels is the same at each level of the other variable, but the difference between the means at one level of the other factor is much greater than the difference of the means at the other level of the other factor. This is illustrated on a line graph by two lines, each going in the same direction, but with different slopes. B. Interactions and external validity. The external validity of an experiment is increased in factorial designs. Whereas in a single-factor study you could argue that the effect of an independent variable is limited to that particular situation, multifactor designs can demonstrate that the independent variable affects behavior under a variety of different situations.

5 CHAPTER 9: Factorial Designs 105 C. Analyzing the results: General concepts. There are several ways to analyze factorial designs but the most common is an analysis of variance (ANOVA). A two-factor ANOVA enables the researcher to make a statistical decision about the main effects of factor A, B, C, and so on, on behavior, as well as whether the combination of the factors leads to a significant change in the dependent variable. When there is a main effect for a factor with more than two levels, a post hoc analysis must be used to determine which levels of the factor are different from the other. Likewise, when there is a statistically significant interaction, the researcher follows it up with tests that examine simple main effects. These tests perform simple contrasts to determine, specifically, at which level of one factor the other factor varies (e.g., A1 is different from A2 but only at B1). Part IV: Experiments with Three Independent Variables Up to this point the examples used to illustrate factorial designs have been those in which only two factors are manipulated (each with only two levels). However, some studies manipulate three or more factors. The simplest three-factor study is a 2 x 2 x 2 design. Three-factor designs examine three main effects; one for A, B, and C. The third variable also produces the potential for a two-way interaction (A x B, A x C, or A x C) as well as a three-way interaction (A x B x C). E. LECTURE AND CLASSROOM ENHANCEMENTS PART I: Basic Characteristics of Factorial Designs A. Lecture/Discussion Topics The order in which factors are described in factorial designs. Sometimes factorial designs are called row by column designs. In a basic two-factor experiment, the first variable, A, is divided so that each level is a row of data. The second variable, B, is divided so that each level of it is a column of data. Likewise, factor A is usually presented on the horizontal axis of a line graph and B is plotted as the third dimension (the dependent variable is still plotted along the vertical axis). B. Classroom Exercise Factorial design worksheet. This link provides an exercise for students to help them understand factorial research design and analysis: ConceptualExercise.pdf C. Web Resources An introduction to factorial designs. The Methodology Center at Penn State describes factorial designs specific to randomized control trials.

6 106 CHAPTER 9: Factorial Designs D. Additional References Factorial designs: Interpretation and considerations. Dziak, J. J., Nahum-Shani, I., & Collins, L. M. (2012). Multilevel factorial experiments for developing behavioral interventions: power, sample size, and resource considerations. Psychological Methods, 17, McAllister, F. A., Strauss, S. E., Sackett, D. L., & Altman, D. G. (2003). Analysis and reporting of factorial trials: A systematic review. JAMA, 289, Montgomery, A. A., Peters, T. J., & Little, P. (2003). Design, analysis and presentation of factorial randomised controlled trials. BMC Medical Resource Methodology, 3, 26. PART II: DESIGNING a Factorial Experiment A. Lecture/Discussion Topics The use of subject variables is an easy way to create a factorial design. The most basic experiment includes the manipulation of a single factor. Adding another factor adds a level of complexity to the study that has a variety of advantages. Subject variables can easily transform any basic, single-factor experiment into a more sophisticated factorial design. Rather than simply examining whether a drug affects behavior, examine the effects of the drug in both men and women, or in young and older adults. Because subject variables exist naturally they take less work (or virtually no work!) to create, compared to true independent variables, thus providing a relatively quick and easy way to explore the effects of two or more factors on behavior. Fractional factorial designs. Factorial designs become exponentially larger with each added variable. One way to manage experiments with multiple independent variables is to create a fractional factorial design. In a traditional factorial design (full factorial design) every possible combination of each variable is examined. In contrast, a fractional factorial design examines a fraction of factorial combinations. For example, a 4 x 2 design creates 8 different treatment conditions, and a 3 x 6 design creates 18. It may not be feasible to conduct a full-factorial design. B. Classroom Exercise The pros and cons of adding time as a factor. Time is often a nonmanipulated factor that is incorporated into experiments to create factorial designs. Ask students to work in groups to think of single-factor experiments that would benefit from having time as a second factor. Have them provide rationales for why time would be a valuable addition, given that it could potentially increase participant mortality and the time it takes to complete the study. C. Web Resource Factorial design flashcards. This website contains, for student download, flashcards of terms associated with factorial designs. IV x SV factorial designs. This website summarizes a lecture specific to how factorial designs may include true independent variables as well as subject variables.

7 CHAPTER 9: Factorial Designs 107 D. Additional References On fractional factorial designs. Gunst, R. F., & Mason, R. L. (2009). Fractional factorial design. Wiley Interdisciplinary Reviews: Computational Statistics, 1(2), PART III: Understanding Main Effects and Interactions A. Lecture/Discussion Topics But only I like to introduce interactions as being a but only type of outcome. For example, I can get a lot of writing done when my kids are around, but only if I m at home. If I m at work with my kids I get virtually nothing done (which begs the question, why do I even bother to go into work with my kids, but I digress ). If I were to simply measure the amount of work I can get done only when my kids around, the data would provide overwhelming support for me to stay at home, which would not go over well with my Dean. However, if I measured my productivity at work and at home, as well as when my kids are around and when I m alone, the data would suggest something totally different. It would likely show that in general I m most productive at work, but only when my kids aren t there. My kids have a negative effect on my ability to write when I m at work because it s not a kid-friendly environment and I m constantly have to give them new things to entertain themselves with. At home I can get a good deal done because they have a playroom, television, and other electronic devices to keep them occupied. This example demonstrates the power that factorial designs have in detecting interactions between variables. What other but only situations can your students think of? The relative importance of significant interactions and significant main effects. Interpreting main effects in factorial designs is no different than interpreting the effect of the variable in a one-factor design. Interactions, by nature, reveal more complex relations among variables, and interpreting them can be tricky, especially for novice researchers. Students need to know that even though main effects are the easiest effects to describe and interpret, when an interaction is present it must be the effect that is showcased. This is because interactions can produce artifacts. An artifact is when there appears, statistically, to be a main effect but it only exists because of its combination with another variable. As I tell my students, when there is a significant interaction it usually trumps any main effect that also exists.

8 108 CHAPTER 9: Factorial Designs B. Classroom Exercises Understanding the complexity of multifactor designs. Perhaps the best way to teach students how to understand main effects and interactions is to present them with figures and then have them visualize (1) a main effect for factor A, (2) a main effect for factor B, and (3) an interaction between factors A and B. To that end, have students work in groups to create figures that would illustrate results from a study in which cognitive demand (factor A; high, low) and room temperature (factor B; comfortable, 102 o ) were manipulated and then the ability to detect grammatical errors was measured. The students estimate of the mean number of grammatical errors that will produce the following effects should be illustrated using a line graph: o Main effect for A o Main effect for B o Main effect for A and main effect for B o Interaction between A and B o Main effect for A and an interaction between A and B o Main effect for B and an interaction between A and B o Main effect for A, main effect for B, and an interaction between A and B C. Web Resources The Everyday Research Methods blog provides examples of factorial designs that are highly applicable to real life. Following each example there are questions about the study that students can answer to better understand factorial designs and the types of variables that are used to understand behavior. 2 x 2 factorial design calculator. At this website students can enter data for two factors, each of which may have up to four levels, to calculate descriptive and inferential results. Interpreting line graphs. This document provides a variety of line graphs describing results from factorial designs. Following each figure there is an explanation of the effect(s) it illustrates. D. Additional References Interactions in factorial research designs. Rosnow, R. L., & Rosenthal, R. (1989). Definition and interpretation of interaction effects. Psychological Bulletin, 105(1), Rosnow, R. L., & Rosenthal, R. (1995). Some things you learn aren't so : Cohen's paradox, Asch's paradigm, and the interpretation of interaction. Psychological Science, 6(1), 3 9.

9 CHAPTER 9: Factorial Designs 109 PART IV: Experiments with Three Independent Variables A. Lecture/Discussion Topics Complex designs make for complex analyses. Students are taught that the addition of more independent variables to an experiment increase the study s external validity, since most behaviors are not dependent on a single factor, but are the combination of many factors. However, when three or more factors are included in a research design, the interpretation of the experiment s results become exponentially more difficult. In addition to more than two main effects, there are multiple potential interactions. B. Classroom Exercise Diagraming a three-factor experiment. To illustrate further the complexity of three-plus factor experiments ask the students to diagram a 3 x 2 x 2 (drug x exercise x therapy) and 3 x 2 x 2 x 2 (drug x exercise x therapy x gender) study. The dependent measure is one s score on the Beck Depression Inventory. The purpose of the study is to examine the contribution of antidepressant use, aerobic activity, and behavioral therapy on depression. C. Web Resource Three-factor experimental designs. o UCLA s Institute for Digital Research and Education s Guide to Understanding Three-Factor Experiments: o This YouTube video created by students gives a very clear description and explanation of threefactor research designs: D. Film Suggestion The Antarctica Challenge is a documentary about the effects of the changing climate on our environment. You can use this film to demonstrate the various factors scientists have identified as contributing to environmental change and how complex (three or more factor) experiments best examine the effect these factors have alone, and in combination with one another, on the environment. Stavrides, S. C., Terry, H. F., Aarons, J., Terry, M., Dayne, R., & Kelley, C. (Producers). Terry, M. (Director). (2009). The Antarctica Challenge. Toronto, ON: Polar Cap Productions. E. Additional References Studies involving three factors. Eysenk, S. B., & Eysenk, H. J. (1970). Crime and personality: An empirical study of the three-factor theory. British Journal of Criminology, 10, 225. Granholm, E., Link, P., Fish, S., Kraemer, H., & Jeste, D. (2010). Age-related practice effects across longitudinal neuropsychological assessments in older people with schizophrenia. Neuropsychology, 24(5), 616.

10