Interactions between predictors and two-factor ANOVA

Transcription

1 Interactions between predictors and two-factor ANOVA March 6, 2017 psych10.stanford.edu

2 Announcements / Action Items Quiz 4 is on Wednesday 3/7 Option to earn extra credit (added to Quiz 3) New topics Χ 2 statistic and goodness-of-fit Χ 2 statistic and independence F-statistic and one-factor analysis of variance (ANOVA) Probability: many comparisons, conditional probability and Bayes rule, and independence Statistical significance, effect size, and making decisions Survey 5 due today at 2 PM

3 Last time In a one-factor ANOVA we partition the total sum of squares (SStotal) into SSbetween and SSwithin, and use these quantities to calculate an F-statistic If we infer that not all population means are equal, we can use follow-up confidence intervals to test whether each pair of conditions is significantly different and to make decisions based on the plausible magnitude of these differences Statistical significance tells us if we should infer if there is any effect at all; effect size and confidence intervals give us information about how meaningful that effect is

4 This time How can we describe the effects of multiple predictor variables? How can we use F-statistics to make inferences about main effects and interactions in the population?

5 This time How can we describe the effects of multiple predictor variables? How can we use F-statistics to make inferences about main effects and interactions in the population?

6 Multiple predictor variables Previously: the effect of a single predictor (grouping variable) on a quantitative response variable t-test: grouping variable has 2 categories ( levels ) ANOVA: grouping variable has 2+ categories ( levels ) Today: the effects of multiple predictor (grouping) variables on a quantitative response variable grouping variables have 2+ categories ( levels ) we ll only consider independent samples Examining multiple predictor variables will allow us to: explain additional variance in our observations formally examine people or situations to which our findings generalize vs. do not generalize

7 Some terminology in ANOVA Predictor variables (categorical) are called factors When using notation, we ll often call them Factor A and Factor B Conditions within predictor variables are called levels An example: Factors: Form of chocolate, with levels chocolate ice-cream and hot chocolate Season, with levels summer and winter Response variable: amount a person is willing to pay for it

8 Chocolate and season Amount willing to pay is determined by the combination of type of chocolate and season Your guess about the pattern of population means Chocolate ice-cream Hot chocolate Collapsed Summer μsummer,icecream μsummer,hotchoc μsummer Winter μwinter,icecream μwinter,hotchoc μwinter Collapsed μicecream μhotchoc

9 Chocolate and season Dollars Summer Winter Chocolate ice-cream Hot chocolate Collapsed Summer μsummer,icecream μsummer,hotchoc μsummer Winter μwinter,icecream μwinter,hotchoc μwinter Collapsed μicecream μhotchoc

10 Chocolate and season Dollars Summer Winter Simple effects (in this case, four) (keep level of one variable constant) - effect of season, only looking at ice-cream - effect of season only looking at hot chocolate - effect of chocolate type, only looking at summer - effect of chocolate type, only looking at winter Chocolate ice-cream Hot chocolate Collapsed Summer μsummer,icecream μsummer,hotchoc μsummer Winter μwinter,icecream μwinter,hotchoc μwinter Collapsed μicecream μhotchoc

11 Chocolate and season Dollars Summer Winter Simple effects (in this case, four) (keep level of one variable constant) - effect of season, only looking at ice-cream - effect of season only looking at hot chocolate - effect of chocolate type, only looking at summer - effect of chocolate type, only looking at winter Chocolate ice-cream Hot chocolate Collapsed Summer μsummer,icecream μsummer,hotchoc μsummer Winter μwinter,icecream μwinter,hotchoc μwinter Collapsed μicecream μhotchoc

12 Chocolate and season Dollars Summer Winter Simple effects (in this case, four) (keep level of one variable constant) - effect of season, only looking at ice-cream - effect of season only looking at hot chocolate -the effect effect of chocolate of season type, depends only looking on what summer - level effect of of chocolate type, we are only considering looking at winter Chocolate ice-cream Hot chocolate Collapsed Summer μsummer,icecream μsummer,hotchoc μsummer Winter μwinter,icecream μwinter,hotchoc μwinter Collapsed μicecream μhotchoc

13 Chocolate and season Dollars Summer Winter Simple effects (in this case, four) (keep level of one variable constant) - effect of season, only looking at ice-cream - effect of season only looking at hot chocolate - effect of chocolate type, only looking at summer - effect of chocolate type, only looking at winter Chocolate ice-cream Hot chocolate Collapsed Summer μsummer,icecream μsummer,hotchoc μsummer Winter μwinter,icecream μwinter,hotchoc μwinter Collapsed μicecream μhotchoc

14 Chocolate and season Dollars Summer Winter Simple effects (in this case, four) (keep level of one variable constant) - effect of season, only looking at ice-cream - effect of season only looking at hot chocolate - effect of chocolate type, only looking at summer - effect of chocolate type, only looking at winter the effect of chocolate depends on what level of season we are considering Chocolate ice-cream Hot chocolate Collapsed Summer μsummer,icecream μsummer,hotchoc μsummer Winter μwinter,icecream μwinter,hotchoc μwinter Collapsed μicecream μhotchoc

15 Chocolate and season Dollars Summer Winter A confounded comparison (differ on multiple factors) - for example, ice-cream in the summer vs. hotchocolate in the winter - we don t know if any differences are due to season, type of chocolate, or both - we won t make these comparisons Chocolate ice-cream Hot chocolate Collapsed Summer μsummer,icecream μsummer,hotchoc μsummer Winter μwinter,icecream μwinter,hotchoc μwinter Collapsed μicecream μhotchoc

16 Chocolate and season Dollars Summer Winter Interaction is present if the effect of one factor depends on the level of the other factor the effect of season depends on what level of chocolate we are considering Chocolate ice-cream Hot chocolate Collapsed Summer μsummer,icecream μsummer,hotchoc μsummer Winter μwinter,icecream μwinter,hotchoc μwinter Collapsed μicecream μhotchoc

17 Chocolate and season Dollars Summer Winter Interaction is present if the effect of one factor depends on the level of the other factor the effect of season depends on what level of chocolate we are considering the effect of chocolate depends on what level of season we are considering we have an interaction between chocolate and season Chocolate ice-cream Hot chocolate Collapsed Summer μsummer,icecream μsummer,hotchoc μsummer Winter μwinter,icecream μwinter,hotchoc μwinter Collapsed μicecream μhotchoc

18 Chocolate and season Dollars Summer Winter Main effects consider the effect of one factor, collapsed across the levels of the other factors (ignoring the other factor) Chocolate ice-cream Hot chocolate Collapsed Summer μsummer,icecream μsummer,hotchoc μsummer Winter μwinter,icecream μwinter,hotchoc μwinter Collapsed μicecream μhotchoc

19 Chocolate and season Dollars Summer Winter Main effects consider the effect of one factor, collapsed across the levels of the other factors (ignoring the other factor) - effect of season, ignoring chocolate - effect of chocolate, ignoring season Chocolate ice-cream Hot chocolate Collapsed Summer μsummer,icecream μsummer,hotchoc μsummer Winter μwinter,icecream μwinter,hotchoc μwinter Collapsed μicecream μhotchoc

20 Chocolate and season Dollars Summer Winter Main effects consider the effect of one factor, collapsed across the levels of the other factors (ignoring the other factor) - effect of season, ignoring chocolate - effect of chocolate, ignoring season Chocolate ice-cream Hot chocolate Collapsed Summer μsummer,icecream μsummer,hotchoc μsummer Winter μwinter,icecream μwinter,hotchoc μwinter Collapsed μicecream μhotchoc

21 Chocolate and season Dollars Summer Winter Summary - four simple effects (# levels A x # levels B) - two main effects (# of factors) - one interaction effect (# sets of 2+ factors) - approach: start by examining the interaction and main effects, examine simple effects if necessary Chocolate ice-cream Hot chocolate Collapsed Summer μsummer,icecream μsummer,hotchoc μsummer Winter μwinter,icecream μwinter,hotchoc μwinter Collapsed μicecream μhotchoc

22 Medication and therapy Consider the effects of therapy and medication on depression outcomes in the population (Navarro text) why would we want to consider these variables together? Two factors (predictor, grouping variables) Therapy, with levels no therapy or therapy (CBT) Medication, with levels placebo or drug (joyzepam) Response variable Gain in mood after treatment

23 Potential outcomes drug placebo No interaction (effects of one factor do not depend on level of the other factor) mood gain neither therapy nor drug improves mood mood gain therapy improves mood, drug does not no_therapy therapy drug improves mood, therapy does not no_therapy therapy mood gain mood gain both therapy and drug improve mood no_therapy therapy no_therapy therapy

24 Potential outcomes drug placebo With interaction (effects of one factor do depend on level of the other factor) mood gain direction of effect of one depends on level of other factor ( crossover ) mood gain magnitude of effect of one depends on level of the other no_therapy therapy no_therapy therapy mood gain existence of effect of one depends on level of the other There are many more patterns that we could see that indicate an interaction, such that: (μdrug,therapy - μdrug,no_therapy) (μplacebo,therapy - μplacebo,no_therapy) no_therapy therapy

25 Potential outcomes drug placebo With interaction (effects of one factor do depend on level of the other factor) mood gain direction of effect of one depends on level of other factor ( crossover ) mood gain magnitude of effect of one depends on level of the other no_therapy therapy no_therapy therapy mood gain existence of effect of one depends on level of the other no_therapy therapy There are many more patterns that we could see that indicate an interaction, such that: (μdrug,therapy - μdrug,no_therapy) (μplacebo,therapy - μplacebo,no_therapy) non-parallel lines

26 Potential outcomes drug placebo No interaction (effects of one factor do not depend on level of the other factor) mood gain neither therapy nor drug improves mood mood gain therapy improves mood, drug does not no_therapy therapy drug improves mood, therapy does not no_therapy therapy mood gain mood gain both therapy and drug improve mood no_therapy therapy no_therapy therapy

27 Main effects vs. simple effects drug placebo With interaction (effects of one factor do depend on level of the other factor) mood gain direction of effect of one depends on level of other factor ( crossover ) mood gain magnitude of effect of one depends on level of the other no_therapy therapy no_therapy therapy mood gain existence of effect of one depends on level of the other no_therapy therapy No main effect of therapy No main effect of drug Is it accurate to say that drug has no effect on mood gain and therapy has no effect on mood gain? misleading to describe main effects instead describe simple effects

28 Main effects vs. simple effects drug placebo With interaction (effects of one factor do depend on level of the other factor) mood gain direction of effect of one depends on level of other factor ( crossover ) mood gain magnitude of effect of one depends on level of the other no_therapy therapy no_therapy therapy mood gain existence of effect of one depends on level of the other no_therapy therapy Main effect of therapy Main effect of drug Is it accurate to say that drug has an effect on mood gain and therapy has an effect on mood gain? yes, this is true regardless of other factor also describe simple effects

29 Main effects vs. simple effects drug placebo With interaction (effects of one factor do depend on level of the other factor) mood gain direction of effect of one depends on level of other factor ( crossover ) mood gain magnitude of effect of one depends on level of the other no_therapy therapy no_therapy therapy mood gain existence of effect of one depends on level of the other no_therapy therapy Main effect of therapy Main effect of drug Is it accurate to say that drug has an effect on mood gain and therapy has an effect on mood gain? not really, it depends on other factor describe simple effects

30 Mini-recap An interaction means that the effect of one variable depends on the level of another variable Main effects describe the effect of one variable ignoring ( collapsing across ) another variable Simple effects describe the effect of one variable when only consider observations corresponding to one level of another variable If we have an interaction, we need to think carefully about how to interpret the main effects In ANOVA we call these predictor / grouping variables factors

31 This time How can we describe the effects of multiple predictor variables? How can we use F-statistics to make inferences about main effects and interactions in the population?

32 Two factors and sample data We don t have access to the population means We can observe sample means Best prediction of the population means? Can we infer that there is a main effect of Factor A in the population? A main effect of Factor B in the population? An interaction in the population? we ll test all of these with one procedure We will consider only situations with an equal number of participants in each cell (combination of levels of each factor), called a balanced design (this matters there are different procedures for handling unbalanced designs)

33 Partitioning sums of squares / df SS A is based on deviations from a pooled mean if we only consider Factor A, ignoring Factor B SStotal SS B is based on deviations from a pooled mean if we only consider Factor B, ignoring Factor A SSbetween SSwithin SS AxB is based on deviations from a pooled mean that aren t explained by simply considering deviations predicted by the combination of Factor A and Factor B SSA SSB SSA x B dftotal SS within is (still) based on deviations of individual values from their group (cell) mean dfbetween dfwithin MS = SS / df dfa dfb dfa x B F = MS / MS within

34 ANOVA table Where k A = # of levels of A, k B = # of levels of B, k A x B = # of cells, N = total number of participants; all sums are sums over all individual observations Source SS sum of squares df degrees of freedom A Ʃ(x A - x pooled) 2 ka - 1 B Ʃ(x B - x pooled) 2 kb - 1 A x B Ʃ(x AxB - x A - x B + x pooled) 2 (ka - 1)(kB - 1) Within Ʃ(xi - x group) 2 N - kaxb Total Ʃ(xi - x pooled) 2 (= SSA + SSB + SSAxB + SSw) N - 1 (= dfa + dfb + dfaxb + dfw) MS mean square (variance) MSA = SSA / dfa MSB = SSB / dfb MSAxB = SSAxB / dfaxb MSw = SSw / dfw F FA = MSA / MSw FB = MSB / MSw FAxB = MSAxB / MSw

35 Sample data Sample with 3 participants per cell (group; combination of each factor) We want to know about the main effects and interactions in the population We have three null hypotheses: H 0,drug : no main effect of drug (μ placebo = μ joyzepam ) H 0,therapy : no main effect of therapy (μ none = μ CBT ) H 0,interaction : no interaction between therapy and drug ((μ drug,therapy - μ drug,no_therapy ) = (μ placebo,therapy - μ placebo,no_therapy )) 2.0 mood.gain four sample means (x ) *not real data 0.0 no.therapy therapy CBT drug placebo joyzepam error bars are one standard deviation above / below the mean

36 ANOVA table Where k A = # of levels of A, k B = # of levels of B, k A x B = # of cells, N = total number of participants; all sums are sums over all individual observations Source SS sum of squares df degrees of freedom Therapy ka - 1 Drug kb - 1 Interaction (ka - 1)(kB - 1) Within N - kaxb Total (sum of all sources) N - 1 (= dfa + dfb + dfaxb + dfw) MS mean square (variance) MSA = SSA / dfa MSB = SSB / dfb MSAxB = SSAxB / dfaxb MSw = SSw / dfw F FA = MSA / MSw FB = MSB / MSw FAxB = MSAxB / MSw

37 ANOVA table Where k A = # of levels of A, k B = # of levels of B, k A x B = # of cells, N = total number of participants; all sums are sums over all individual observations Source SS sum of squares df degrees of freedom Therapy = 1 Drug = 1 Interaction (2-1)*(2-1) = 1 Within = 8 Total (sum of all sources) 12-1 = 11 (sum of all sources) MS mean square (variance) MSA = SSA / dfa MSB = SSB / dfb MSAxB = SSAxB / dfaxb MSw = SSw / dfw F FA = MSA / MSw FB = MSB / MSw FAxB = MSAxB / MSw

38 ANOVA table Where k A = # of levels of A, k B = # of levels of B, k A x B = # of cells, N = total number of participants; all sums are sums over all individual observations Source SS sum of squares df degrees of freedom Therapy = 1 Drug = 1 Interaction (2-1)*(2-1) = 1 Within = 8 Total (sum of all sources) 12-1 = 11 (sum of all sources) MS mean square (variance) / 1 = / 1 = / 1 = / 8 = F FA = MSA / MSw FB = MSB / MSw FAxB = MSAxB / MSw

39 ANOVA table F critical,df=(1,8) = 5.32 * careful, df is not always the same for each F Where k A = # of levels of A, k B = # of levels of B, k A x B = # of cells, N = total number of participants; all sums are sums over all individual observations Source SS sum of squares df degrees of freedom Therapy = 1 Drug = 1 Interaction (2-1)*(2-1) = 1 Within = 8 Total (sum of all sources) 12-1 = 11 (sum of all sources) MS mean square (variance) / 1 = / 1 = / 1 = / 8 = F / = /.061 = MS / MS = 0.87

40 Sample data Sample with 3 participants per cell (group; combination of each factor) We want to know about the main effects and interactions in the population We have three null hypotheses: H 0,drug : no main effect of drug (μ placebo = μ joyzepam ) H 0,therapy : no main effect of therapy (μ none = μ CBT ) H 0,interaction : no interaction between therapy and drug ((μ placebo - μ joyzepam ) = (μ none - μ CBT )) mood.gain no.therapy therapy CBT drug placebo joyzepam Infer that the (population) mean mood gain is greater for joyzepam than for placebo Not enough evidence to conclude that the the (population) mean mood gain is different for no therapy vs. placebo Not enough evidence to conclude that there is an interaction between drug and therapy error bars are one standard deviation above / below the mean

41 Effect size often called η 2 pronounced ( eta ) Source SS sum of squares Therapy = 1 Drug = 1 Interaction (2-1)*(2-1) = 1 Within = 8 Total (sum of all sources) r 2 = SS / SS total therapy r 2 = df / =.02 MS we degrees can explain of 2% mean of the square variance in our Fsample mood freedom gains by knowing (variance) whether a participant had no therapy or CBT 12-1 = 11 (sum of all sources) / 1 = / 1 = / 1 = / 8 = / = /.061 = MS / MS = 0.87 drug r 2 = / =.84 we can explain 84% of the variance in our sample mood gains by knowing whether a participant took a placebo or joyzepam interaction r 2 = / =.01 we can explain 1% of the variance in our sample mood gains by considering the specific combination of levels of each factor, beyond the main effects

42 Effect size often called partial η 2 pronounced ( eta ) denominator is changing cannot make direct comparisons can sum to > 1 partial r 2 = SS / (SS + SS within ) therapy SS df MS Source sum of r 2 degrees = of / (0.083 mean ) square =.15 F squares we freedom can explain 15% (variance) of the variance in our sample mood gains that is not explained Therapy / 1 = / by 2 other - 1 = factors 1 in the experiment by = 1.36 knowing whether a participant had no Drug / 1 = /.061 therapy 2-1 = or 1 CBT = Interaction (2 drug - 1)*(2-1) / 1 = MS / MS r 2 = = 1 / ( ) =.87 = 0.87 Within we can explain 87% of / the 8 = variance in our sample 12-4 = mood 8 gains that is not explained Total by 12 other - 1 = factors 11 in the experiment by (sum of all knowing (sum of all whether a participant took a sources) placebo sources) or joyzepam

43 Sample data Sample with 3 participants per cell (group; combination of each factor) We want to know about the main effects and interactions in the population We have three null hypotheses: H 0,drug : no main effect of drug (μ placebo = μ joyzepam ) H 0,therapy : no main effect of therapy (μ none = μ CBT ) H 0,interaction : no interaction between therapy and drug ((μ placebo - μ joyzepam ) = (μ none - μ CBT )) mood.gain no.therapy therapy CBT drug placebo joyzepam Can perform follow-up one-factor ANOVAs or confidence intervals as appropriate based on the results here, we already have our answer error bars are one standard deviation above / below the mean

44 ANOVA as a general procedure Notice that mood gain is really a measure of a difference in post-treatment mood and pre-treatment mood We could instead set this up as a three-factor ANOVA with factors of time, drug, and therapy, where time is a repeated-measures (paired) factor Instead here we have used strategy of converting paired observations to difference scores Reminder that ANOVA procedures can be extended to an arbitrary number of factors, and using independent or dependent samples

45 Why we need balanced designs placebo joyzepam collaped no therapy n = 3 n = 3 3 people w/ placebo, 3 people with joyzepam CBT n = 3 n = 3 3 people w/ placebo, 3 people with joyzepam balanced design: the effect of drug (joyzepam vs. placebo) will balance out between no therapy and CBT placebo joyzepam collaped no therapy n = 3 n = people w/ placebo, 100 people with joyzepam CBT n = 3 n = 3 3 people w/ placebo, 3 people with joyzepam unbalanced design: the effect of drug (joyzepam vs. placebo) will not balance out between no therapy and CBT, comparison reflects both effect of therapy and effect of drug (therapy and drugs are confounded)

46 Some disambiguation Two-way ANOVA: 2 grouping variables Each combination of levels has multiple observations that all provide a quantitative measure on a response variable Χ 2 test for independence 2 grouping variables Each combination of levels has a count of observations (Can also think of as 1 grouping variable, with counts of a response variable measured separately for each level of a grouping variable)

47 Recap When considering multiple predictor variables, we can think of main effects (ignore other variable), simple effects (restrict to one level of another variable), or interaction effects (does effect of one variable depend on level of another variable) We can do hypothesis tests about main effects and interactions in a single test using F-statistics with a twofactor ANOVA We can perform follow-up tests (such as simple effects) to better understand the pattern of findings, when appropriate

48 Questions