Analysis of Variance (ANOVA)

Transcription

1 Research Methods and Ethics in Psychology Week 4 Analysis of Variance (ANOVA) One Way Independent Groups ANOVA

2 Brief revision of some important concepts To introduce the concept of familywise error rate. To explain the logic of ANOVA To demonstrate the calculations for one-way independent groups ANOVA To introduce partitioning of variance and degrees of freedom To demonstrate the construction of the ANOVA summary table and enable students to interpret it.

3 the P value is the most frequently used approach to evaluating statistical significance of a given test P represents the probability that a finding was reached by chance If p <.05, it is the same thing as saying that there is a less than 5% likelihood that the finding has occurred by chance Sir Ronald A. Fisher

4 When should we reject the null hypothesis? When should we fail to reject the null hypothesis? What would you conclude from the p values below? p =.003 p > 0.5 p <.05 p =.87

5 No. of people An assumption of ANOVA is Homogeneity of variance Population mean Population mean The variance (spread) within each of the populations is equal

6 Variance: the natural individual differences between people in a sample. Individual scores vary around the mean value x x s Formula for variance is: N 1 So it is the mean of the squared differences AKA mean of squares Independent Variable: the one you manipulate, the grouping variable AKA the factor AKA the treatment (think clinical trials)

7 Type 1 error: reporting something as significant when you shouldn t. The probability of incorrectly rejecting H 0 We use p <.05 to indicate significance 95% sure that we have found something significant (as we can never be 100% sure) This means that there s always a 5% chance of being wrong (5% chance that our sample has indeed come from H 0 distribution)

8 Type error: not finding a significant result when you should. The probability of incorrectly failing to reject H 0 Various reasons why our study isn t good enough: measuring problems, sampling problems, etc Type error is the POWER of the study we talk about this later

9 Error rate is the amount of Type 1 error in your study Normally 5% of course We call this the alpha level or Familywise error rate: Simply put, this is the probability of making one or more false discoveries, (or type I errors) among all our hypotheses when performing multiple pairwise tests

10 Modern Mechanix, December 1931

11 Research question 1: do cows milk better to music? measures: music versus no music Analysis = one repeated measures t-test Error rate = 0.05 (α level as normal) Research question : which music makes cows milk better? 7 groups: no music; rock music; dance music; hiphop; heavy metal; folk music; classical music Analysis = 1 t-tests to compare each group with the others Familywise error rate = a very high chance of making a mistake (i.e., reporting an effect that we shouldn t)

12

13 The probability of Type1 error is.05 (The probability of not having any Type 1 errors is therefore.95) We multiply.95 by itself for each additional time a test is conducted If there were 5 tests, it would be:.95x.95x.95x.95x.95 = (or 77% chance of not having any Type 1 errors) Having at least 1 Type 1 error = 1-.77=.3 or 3%

14 We could keep using t-tests to look at all the pairs of scores until we find which ones differ (and which one is best) but this lead to an increase in error rates. Note change below p familywise 0.95 to the power of 1 (i.e., 1 tests) = This means there will be a high probability of making a T1 error (saying significant when it isn t) - in this case 0.66, 66% chance ANOVA tests if there is any significant differences between the many means with just one test. So the error rate stays at 0.05

15 H H1 etc There is at least one significant difference Although we are testing for differences between means, ANOVA looks for an overall test for differences against the H 0 that says there are no differences at all ANOVA does this by analysing variance ** ANOVA can also be linked to regression (we do this later), therefore it is AKA The General Linear Model (see Field book)

16 Breaking VARIANCE Down (Independent groups ANOVA) Differences between individual scores come from various sources 1.Within groups variation a) Individual differences/error: Within each sample/group there will be variation. (Remember Std Dev?). Between Groups Variation a) Sampling error: natural differences between the two samples; no two groups are exactly the same (remember that the sampling distribution has a std dev) b) Treatment effect: the actual variation between groups due to your IV; the one you re interested in!

17 Research question: does culture affect response to academic failure? Cross cultural study = between groups design 5 groups of students: American; Japanese; Nigerian; Chilean; Turkish H 0 = no differences in (group) mean responses to failure H 1 = differences between mean responses Sources of variance come from: individual differences on responses to failure due to personality etc (i.e., nothing to do with culture) differences between samples due to understanding of the task, experimenter, etc... (again nothing to do with culture) actual systematic differences due to culture: the treatment effect

18 Think of the t-test formula: mean difference std deviations This is essentially: observed/treatment differences uncontrollable/individual difference In ANOVA we are looking for differences in variance not mean but the logic is still the same: Systematic Variation (treatment effect) VS. Unsystematic variation (error, two types) All tests are basically asking if the systematic differences in the scores, the ones we are targeting, are BIGGER than the natural ones

19 So we calculate a ratio (which is basically a division sum) Systematic variance / unsystematic variance In ANOVA variance terms this becomes: Between groups variance (effect + error) Within groups variance (error only) So all the maths involved is to find out these two numbers so we can divide them and get our ANOVA statistic (F) F = differences including treatment effects differences with no treatment effects So if H 0 is true (no differences) then this ratio would be (0 + error) / error = 1 We are looking for an F bigger than 1, to illustrate a difference

20 Compare the two estimates. F MS MS between within Variance including treatment effects Variance with no treatment effects If the null hypothesis is true then the two estimates will be the same and F = 1. If the between groups variation is larger than the within groups variation then F > 1. Look F up in tables and find the probability of it occurring by chance. Compare this value to alpha and if p then the null hypothesis can be rejected.

21 1) ANOVA makes two assumptions about the data in order to work. These have to be true for the test to reliably work on the data a) The samples are from Normal distribution populations b) Homogeneity of variance : the samples have come from populations with the same variance (so sampling error is minimised, we can use within groups error as between group error) e ANOVA is all about estimating this number, the error variance. The bottom of the ratio is key

22 So we need to work out two forms of variance in our data (to make it less repetitive we are now going to call variance Mean Squares) e s s s 1) The error variance (denominator): a measure of the within groups variation, also called MS within (mean square within) e e e i.e., the sample variances are estimates of the population variance (natural individual differences) To get a better estimate, they can be averaged. The pooled (averaged) sample variance give us the error variance s pooled Note that this estimate is independent of the truth or falsity of 0 e H

23 ) The between groups variance (numerator): a measure of the treatment effect MS between (mean square between) s x e x s x e The variance between sample means (we want this to be large to show the groups differ and there is a treatment effect) Note that this is only a good estimate if the null hypothesis is true. If the null hypothesis is false then this estimate will be too big.

24 (SPSS will do them for you, but you need to know how) Its all about variance so we go back to the formula for variance: Top is called the Sum of Squares (SS) s x x Aka MS (mean squares) N 1 Bottom is called Degrees of freedom (df) Remember that the only data we actually have is mean and N number of our samples. Calculations are conducted on Sums of Squares for ease of calculation as they are more easily manipulated

25 So we are aiming to solve the overall equation (division): F Ratio h) g) i) (variance) F (Variance) MSbetweengroups MSwithingroups MSbetweengroups MS withingroups and both the MS parts can be broken down into individual variance equations SSbetweengroups df betweengroups SSwithingroups df withingroups Problem is, we don t have the values of any of these 4 things we have to find a way to get these values from our sample means only

26 a) first, we are able to get an overall SS SS total When comparing two or more samples the mean of all the scores together is known as the grand mean Any score can be expressed as a deviation from the grand mean ie so x x SS total How much each score deviates from the grand mean can be broken into two parts x x x x x x x x x 1) how much it deviates from the mean of its own sample ) how much the mean of its sample deviates from the grand mean

27 SStotal SSbetweengroups SSwithingroups SS total SS betweengroups SS withingroups

28 Using the SS total to get at SS within and between b) SS within SS within x x c) SS between SS between ni x i x n i Where is the number of scores in that particular group.

29 Partitioning Degrees of Freedom d) e) f) df 1 total df k 1 between df k n 1 within df total Where N is the total number of scores K is the number of groups n is the number of observations within a group df betweengroups df withingroups

30 Now we have all the individual values, we just put them into the formula: SSbetweengroups MSbetweengroups df betweengroups F MSbetweengroups MSwithingroups SSwithingroups df withingroups MS withingroups

31 Martin, C., Tweed, A., & Metcalfe, M. (004). A psychometric evaluation of the Hospital Anxiety and Depression Scale in patients diagnosed with end-stage renal disease. British Journal of Clinical Psychology, 43, To determine the psychometric properties of the Hospital Anxiety and Depression Scale (HADS) in patients with end-stage renal disease (ESRD) and determine the suitability of the instrument for use with this clinical group

32 (1) Are there differences in HADS-assessed anxiety and depression in patients diagnosed with ESRD as a function of either dialysis modality, or kidney transplant origin? () Is the HADS a robust and reliable measure able to assess separate domains of anxiety and depression in patients diagnosed with ESRD?

33

34

35

36 x x x x N You may find some of your text books use this computational formula for Sums of Squares.

37 Post hoc testing for the oneway independent groups ANOVA type I and type II errors Bonferroni t-test Tukey s HSD Student Newman Keuls Fisher s LSD