AMSc Research Methods Research approach IV: Experimental [2] Marie-Luce Bourguet mlb@dcs.qmul.ac.uk Statistical Analysis 1
Statistical Analysis Descriptive Statistics : A set of statistical procedures used to organize, summarize and present the data collected in an experiment. Inferential Statistics : A series of procedures based on samples or bits of information used to make statements about some broader set of circumstances. Statistical Analysis 2
Descriptive Statistics [1] Graphs Some mathematical calculations : e.g. means and measures of dispersion Group A (reading scheme) 4 1 12 7 5 6 5 8 11 5 9 7 9 4 10 8 6 6 7 5 7 Group B (no reading scheme) 9 2 12 8 4 3 11 1 10 9 3 4 5 4 7 3 9 5 8 10 2 9 3 Statistical Analysis 3
Descriptive Statistics [2] Frequency distributions (histograms): Group A x x x x x x x x x x x x x x x x x x x x x x x x 1 2 3 4 5 6 7 8 9 10 11 12 Group B x x x x x x x x x x x x x x x x x x x x x x x x 1 2 3 4 5 6 7 8 9 10 11 12 SX Means: X = N Mean (A) = 6.375 Mean (B) = 6.125 Variances: S 2 = S(X-X) 2 S 2 (A) = 6.901 N S 2 (B) = 10.193 Standard Deviations: S S (A) = 2.623 S (B) = 3.193 Statistical Analysis 4
Inferences and Statistical Tests In experimental research we make inductions or inferences based on our observations. But data and observations obtained from people are often extremely varied. Many of the things which influence their behaviour may have nothing to do with the experiment. We have to sort out whether experimental results are really significant. This is what statistical tests enable you to do. Statistical Analysis 5
Variability of Scores - Illustration Number of ideas correctly recalled (out of 10) Condition 1 Condition 2 (simple texts) 10 2 5 1 6 3 3 4 9 4 8 4 7 2 5 5 6 7 5 4 Tot 64 36 (complex texts) 4 3 2 1 4 3 2 1 Condition 1 0 1 2 3 4 5 6 7 8 9 10 Condition 2 Mean = 6.4 S = 2.01 Mean = 3.6 S = 1.62 0 1 2 3 4 5 6 7 8 9 10 Statistical Analysis 6
Statistical Tests: Basic Aim test whether any differences predicted by the alternative hypothesis are significant; or whether a researcher should instead accept the null hypothesis that such differences are only due to chance fluctuations in people s performance. Statistical Analysis 7
How to Choose a Statistical Test? The whole art of using statistics is to match up the experimental designs with the statistical tests. It depends on: the number of experimental conditions (number of independent variables / number of conditions per variable) what level of measurement is used for measuring a dependent variable (nominal / ordinal / numerical) Statistical Analysis 8
Parametric Versus Non-parametric Tests Non-parametric tests can be used when you can measure your experimental data only at the ordinal level ((Wilcoxon, Friedman, etc.). There are some non-parametric tests that can be used even when your data is only nominal (Chi-square). There are 3 requirements for parametric tests: - scores are measured on an interval scale (numerical) - scores are normally distributed - homogeneity of variance (the variability of scores for each experimental condition should be roughly the same) Statistical Analysis 9
Standard Normal Distribution z = X - X s Statistical Analysis 10
Decision Chart Differences Correlations or differences between conditions? Correlations Para Pearson Non P Spearman One variable One variable or two or more variables? Two or more variables How many experimental conditions? Within (or matched) Within (or matched) or between subjects in each condition? Between 2 way ANOVA (related) 2 way ANOVA (mixed) 2 way ANOVA (unrelated) Statistical Analysis 11
Decision Chart - Continuation Two How many experimental conditions? Three or more Within (or matched) Para t test (related) Within (or matched) or between subjects in each condition? Non P Wilcoxon Para t test (unrelated) Between Non P Mann- Whitney Chi-square Para 1way ANOVA (related) Within (or matched) or between subjects in each condition? Within (or matched) Non P Friedman Pages L Trend Para 1 way ANOVA (unrelated) Between Non P KrustalWallis Jonkheere Trend Chi-Square Statistical Analysis 12
Looking up Probabilities in Statistical Tables Whichever type of statistical test you use, you will end up by having to look up percentage probabilities in the statistical table appropriate for that particular test. Things to know about your experimental design: degrees of freedom (N-1, C-1) one-tailed and two-tailed hypotheses Statistical Analysis 13
Looking up Probabilities in Statistical Tables - Illustration (t-test) Level of significance for one-tailed test.05.025.01.0005 Level of significance for two-tailed test.10.05.02.001 Statistical Analysis 14
Selecting a Level of Significance Choosing a significance level is a matter of deciding what odds you are prepared to accept that your results are due to chance. In psychology, it is a convention to accept odds of either 1 in 100 (i.e. 1%) or 5 in 100 (i.e. 5%). It means that the probability of a result being due to chance is less than 1% (p <.01) or less than 5% (p <.05) With a two-tailed hypothesis there is a higher probability that it is a chance result (it s double). Statistical Analysis 15
Selecting a Level of Significance - Illustration (t-test) Level of significance for one-tailed test.05.025.01.0005 Level of significance for two-tailed test.10.05.02.001 Statistical Analysis 16
Non-parametric Tests 2 conditions (ordinal data) 3 or more conditions (ordinal data) Trends (ordinal data) 2 or more categories (nominal data) Related designs (within or matched subjects) Wilcoxon Friedman Page s L Trend Unrelated designs (between subjects) Mann-Whitney Kruskal-Wallis Jonckheere Trend Chi-square Statistical Analysis 17
Non-parametric Test: Wilcoxon Statistical Analysis 18
Wilcoxon Statistical Analysis 19
Parametric Tests for One Independent Variable Related designs (within or matched subjects) Unrelated designs (between subjects) 2 conditions Related t test Unrelated t test 3 or more conditions One-way Related ANOVA One-way Unrelated ANOVA Statistical Analysis 20
Parametric Test: Related t test Statistical Analysis 21
t test Level of significance for one-tailed test.05.025.01.0005 Level of significance for two-tailed test.10.05.02.001 Statistical Analysis 22
References Learning to use Statistical Tests in Psychology A Student Guide. Judith Green and Manuela D Oliveira. The Open University Press. Statistics without tears A primer for Non-mathematicians. Derek Rowntree. Penguin Books. Fundamentals of Behavioural Statistics. R. Runyon, A. Haber, D. Pittenger & K. Coleman. McGraw-Hill. Statistical Analysis 23