Comparing 3 Means- ANOVA

Transcription

1 Comparing 3 Means- ANOVA Evaluation Methods & Statistics- Lecture 7 Dr Benjamin Cowan

2 Research Example- Theory of Planned Behaviour Ajzen & Fishbein (1981) One of the most prominent models of behaviour Used a lot in behaviour change research Has 3 core components: Beliefs (right) Intentions (middle) Behavior (left)

3 Research Example- Theory of Planned Behaviour Beliefs: Attitude- Persons attitude towards an action Subjective norms- what people perceive others to believe about the action Perceived behavioural control- peoples perceptions on how able they are to do a certain behaviour

4 Research Example- Theory of Planned Behaviour Intentions: A person s internal declaration to act Behavior: The action

5 TPB and Pro Environmental Behaviour Perceived Behavioural Control Self Efficacy concept We want to see whether 2 interventions we design impact pro environmental behaviour self efficacy compared to a control group with no intervention. We therefore have 3 conditions in our experiment

6 How would we design this experiment?

7 How would we design this experiment? IV- Intervention Level 1- Control Group (No Intervention) Level 2- Intervention 1 (Generic Information Condition) Level 3- Intervention 2 (Tailored Information condition) DV- Self Efficacy/Perceived Behavioural Control Measured by questionnaire

8 One Way ANOVA- The Idea Compare more than 2 means to identify whether they are significantly different i.e. whether they come from different populations We could do 3 t-tests Compare control to group 1 Compare control to group 2 Compare group 2 to group 3

9 Familywise error rate If we have 3 tests in a family of tests and assume each is independent If we use Fishers level of 0.05 as our level of significance The probability of no Type I error in all of these tests 0.95 x 0.95 X 0.95 = This is because we would expect to get a chance significant results 5% of the time. => Probability of Type 1 error is =0.143 That is far greater than the Type I error for each test separately (0.05) We therefore use ANOVA rather than lots of t-tests

10 ANOVA- The Idea Compare 3 (or more) means to identify whether they are significantly different i.e. whether they come from different populations Or more accurately..we are testing the null hypothesis that the samples come from the same population. It is what we call an omnibus test It tells us there is a significant difference, not where it is.

11 ANOVA- The Idea

12 What this means in our example We are testing whether: The scores on perceived behavioural control in each condition come from the same population of scores (i.e. our interventions had no effect - H0) If they don t (i.e. they come from different populations) then we can reject the null hypothesis

13 The Key: ANOVA & F Ratio We found a significant effect of condition on perceived behavioural control [F(2, 56)= 11.78, p<0.001] F ratio is the ratio of explained (that accounted for by the model we are proposing) to unexplained variation This is calculated using the Mean Squares

14 The mindset of models Remember the mean is a statistical model, just sometimes not a very good one. We want to see whether the statistical model we have proposed explains the variation in our data better

15 Step 1- Total Sum of Squares The total amount of variation in our data This should look familiar (see Lecture 3) SS T = ( x x ) 2 i grand

16 Step 1- Graphically What the equation is doing x grand x i

17 Step 2- Model Sum of Squares We now need to know how much variation our model can explain How much the total variation can be explained due to data points coming from different groups in the perfect model n k is the amount of people in that condition SS M = n (x k k x grand ) 2

18 Step 2- Graphically What the equation is doing x grand x k

19 Step 3- Residual Sum of Squares How much of the variation cannot be explained by the model i.e. what error is there in the model prediction? Easy way to calculate: SS R = SS T SS M But here is the real formula SS R = ( x x ) 2 ik k

20 Step 3- Graphically What the equation is doing x k x ik

21 Great.. So what next? These are summed values Therefore impacted by the number of scores in the sum (remember variance in Lecture 3) We can get around this by dividing by the respective degrees of freedom for each SS

22 Degrees of Freedom for each SS Degrees of Freedom for SS T (dft) : N-1 Degrees of Freedom for SS M (dfm): Amount of Conditions (k) -1 Degrees of Freedom for SS R (dfr): N-k

23 F Ratio The F Ratio is calculated using the: Mean Squares model (MS M ): SS M /df M Mean Squares residual (error) (MS R ): SS R /df R

24 F Ratio Variation explained by our model Variation unexplained by our model

25 F Ratio Mean Square Model (MS M ) Mean Square Residual (MS R )

26 F Distribution F Distribution for specific pair of degrees of freedom Table of Critical Values

27 One Way Independent ANOVA- Assumptions Normally distributed data (Shapiro-Wilkes test) Equality of Variance (Levene s test) Interval or ratio data Independent data

28 Reporting ANOVA F ratio Degrees of Freedom (dof M, dof R ) P value Back to the example we saw earlier: F(2, 56)= 11.78, p<0.001 We can therefore state that there is a significant effect of our independent variable on perceived behavioural control

29 One Way Repeated Measures ANOVA Experiment measuring self efficacy at: 1. Time 1 (before Intervention) 2. Time 2 (directly after intervention) 3. Time 3 (1 month after the intervention) This is a Repeated Measures design (Within Subjects- see last week) Reduces unsystematic variability due to individual differences

30 Repeated Measures ANOVA- Sphericity The assumption of independence of data doesn t hold as the data is from the same participants Instead we look for sphericity The assumption that the variance of the differences between scores in each treatment are equal Calculate the difference between pairs of scores in all possible combination of treatment levels,then calculate the variance of these differences

31 Mauchly s test of sphericity It tests the hypothesis that the variances of the differences are equal (H0) If I got p<0.05 for this test would it be good or bad?

32 Mauchly s test of sphericity It tests the hypothesis that the variances of the differences are equal (H0) If I got p<0.05 for this test would it be good or bad? It would be bad as it states there is a significant difference between variance of differences Corrections exist if this is the case, usually Greenhouse-Geisser correction is used

33 One Way Repeated Measures ANOVA-Theory Within-Participant Variance This includes Experiment effect (as all people have taken part in all conditions) Error variance- that not explained by the model

34 Step 1- Total Sum of Squares The total amount of variation in our data dft = N-1 SS T = ( x x ) i grand 2

35 Step 2- Within Participant Sum of Squares How much of the variation is within participant variance x ik is a person (i) s score on a condition (k) and x i _ across_ conditions is the person s mean across those conditions dfw= Number of participants X (Number of conditions-1) SS w = ( x x ) 2 ik i _ across_ conditions

36 So far we know.. The total amount of variation in our data (SSt) How much this is caused by individual s performance under the different experiment conditions (SSw)

37 Step 3- Model Sum of Squares We now need to know how much variation our model can explain In other words how much variation is attributed to our experiment and how much isn t This calculation is the same as in Independent ANOVA SS M = n (x k k x grand ) 2

38 Step 4- Residual Sum of Squares We now need to know how much of this variation is noise or error variation Simplest way to calculate this is: SSR= SSW-SSM Degrees of freedom are also calculated using: dfr = dfw-dfm

39 One Way Repeated Measures ANOVA- Assumptions Normally distributed data for each condition (Shapiro-Wilkes test) Sphericity Interval or ratio data

40 Omnibus test & Post Hoc If we had a significant main effect of intervention on self efficacy [F(2, 56)= 11.78, p<0.001] This tells us there is a significant effect of our experiment conditions on self efficacy i.e. that the means of the conditions are not equal But how does this break down? Information >Control condition? Tailored Information > Information & Control conditions? We then need post hoc tests

41 Post Hoc Tests Used when no specific a priori predictions about the data we have They are used for exploratory data analysis Pairwise comparisons Like performing t-tests on all the pairs of mean in our data But they control the Type I error rate by correcting the significance level across all tests Using the Bonferroni correction (0.05/Ntests) There are many to choose from..

42 A selection of common post hoc tests LSD (Least Significant Difference) Analogous to multiple t-tests Bonferroni Uses Bonferroni correction to control for Type I With multiple comparisons this may be too conservative (increase chance of Type II error) Tukey s test Control Type I and better when testing large amounts of means

43 Which one to choose? Trade off between: Type I error rate likelihood Statistical power (ability to find an effect if there is one) Whether assumptions of ANOVA have been violated, although most are robust to minor variations

44 Lecture Readings and Further concepts to consider Core:- Field (2009) Chapters 8 & 11 (pages ) Other concepts to consider: Statistical Power: Cohen (1992). A power primer. Psychological Bulletin Planned Contrasts: Field (2009), Chapter 8, p