The t-test: Answers the question: is the difference between the two conditions in my experiment "real" or due to chance?

Transcription

1 The t-test: Answers the question: is the difference between the two conditions in my experiment "real" or due to chance? Two versions: (a) Dependent-means t-test: ( Matched-pairs" or "one-sample" t-test). Same subjects do both experimental conditions e.g., two conditions A and B: half subjects do A then B; rest do B then A. (Randomly allocated to one order or the other). Both types of t-test have one independent variable, with two levels (the two different conditions of our experiment). (b) Independent-means t-test: ( Two-sample" t-test). Different subjects do each experimental condition. e.g., two conditions A and B: half subjects do A; rest do B. (Randomly allocated to A or B). There is one dependent variable (the thing we actually measure). Effects of alcohol on reaction-time performance. I.V. is "alcohol consumption". Two levels - drunk and sober. D.V. is RT. Use a repeated-measures t-test: measure each subject's RT twice, once while drunk and once while sober. Effects of personality type on a memory test. I.V. is "personality type". Two levels - introversion and extraversion. D.V. is memory test score. Use an independent-measures t-test: measure each subject's memory score once, then compare introverts and extraverts. 1

2 Rationale behind the t-test: Experiment on the effects of alcohol on RT. Measure RT for subjects when drunk, and when sober. Null hypothesis: alcohol has no effect on RT: variation between the drunk sample mean and the sober sample mean is due to sampling variation. The drunk and sober scores are samples from the same population (sober RTs). If the difference between the sober and drunk means is large, we might prefer to believe that alcohol has affected RT: the difference is not due to sampling variation, but arose because the drunk and sober scores are samples from two different populations - the population of sober RTs and the population of drunk RTs. How large is large? The t-test enables us to decide. Both types of t-test are similar in principle to the z-score. t-distribution becomes progressively more like the normal distribution as sample size (n) increases: Observed difference between sample means Predicted difference between sample means (that there will be no difference at all) measure of the extent to which pairs of sample means might differ 2

3 1. We have two sample means, which differ. 2. Null hypothesis is that the two samples come from the same population; if so, ideally the sample means should be identical. RT for drunk sample: 800 ms RT for sober sample: 300 ms Difference: 500 ms Sober sample RT really reflects sober population RT: say, 600 ms Drunk sample RT also really reflects sober population RT: 600 ms Difference: 0 The difference between 500 ms and 0 ms is due to chance (sampling variation). 3. Alternative hypothesis is that our experimental manipulation has affected our subjects. The two samples (drunk and sober) are samples from different populations with different means. If so, the samples might well have different means. (e.g. the sober sample mean of 300 ms might reflect a sober population mean of 300 ms; the drunk sample mean of 800 ms might reflect a drunk population mean of 800 ms). A big difference between our two sample means therefore suggests that either: (a) the two sample means are poor reflections of the mean of the single population that they are supposed to represent (i.e., our samples are atypical ones). OR (b) The two sample means are actually from two different parent populations, and our initial assumption that the samples both come from the same population is wrong. The bigger the difference between our two sample means, the less plausible (a) becomes, and the more likely that (b) is true. Repeated Measures t-test, step-by-step: Does Prozac affect driving ability? Ten subjects have their driving performance tested twice on a sheep farm: test A after they have taken Prozac ( experimental condition); test B while they are drug-free ( control condition). Each subject thus provides two scores (one for each condition). Five do A then B, five do B then A. 3

4 Number of sheep hit during a 30-minute driving test: Subject: Test A Score Test B Score Difference, D ΣD 6 t the average difference between scores in our two samples (should be close to zero if there is no difference between the two conditions) D µ D( hypothesised ) S D the predicted average difference between scores in our two samples (usually zero, since we assume the two samples don t differ ) estimated standard error of the mean difference (a measure of how much the mean difference might vary from one occasion to the next). 1. Add up the differences: ΣD 6 2. Find the mean difference: D D N Estimate of the population standard deviation (the standard deviation of the differences): S D ( D D ) n Estimate of the population standard error (the standard error of the differences between two sample means): S D S D n 4

5 5. Hypothesised difference between the sample means. Our null hypothesis is usually that there is no difference between the two sample means. (In statistical terms, that they have come from two identical populations): µ D (hypothesised) 0 6. Work out t: t "Degrees of freedom" (d.f.) are the number of subjects minus one: d.f. n Find the critical value of t from a table (at the back of many statistics books; also on my website). (a) Two-tailed test : if we are predicting a difference between tests A and B find the critical value of t for a "two-tailed" test. With 9 d.f., critical value (b) One-tailed test : if we are predicting that A is bigger than B, or A is smaller than B, find the critical value of t for a "one-tailed" test. For 9 d.f., critical value critical values of t (two-tailed): If obtained t is bigger or equal to the critical t-value, "reject the null hypothesis" - the difference between our sample means is probably too large to have arisen by chance. Here, obtained t 65. This is less than There was no significant difference between performance on the two tests; the observed difference is so small, it probably arose by chance. Conclusion: Prozac does not significantly affect driving ability. Results of analysis using Excel: (Tools> Data Analysis > t-test: Paired Two Sample for Means)! " # # $ #%% & '() * # %,-./ #+ #++%# $ %,-./ $ #+ # +%

6 Results of analysis using SPSS: (Analyze >Compare Means > Paired samples t-test) Paired Samples Statistics Pair 1 Test A Test B Std. Std. Error Mean N Deviation Mean Paired Samples Correlations Pair 1 Test A & Test B N Correlation Sig Paired Samples Test Pair 1 Test A - Test B Paired Differences 95% Confidence Interval of the Std. Std. Error Difference Sig. Mean Deviation Mean Lower Upper t df (2-tailed)