MAT Mathematics in Today's World

Transcription

1 MAT 1000 Mathematics in Today's World

2 Last Time 1. What does a sample tell us about the population? 2. Practical problems in sample surveys.

3 Last Time Parameter: Number that describes a population Statistic: number that describes a sample Error: discrepancies between statistics and the parameters being estimated Bias: in repeated samples, the sample statistic consistently misses the population parameter in the same direction Variability: different samples from the same population may yield different values of the sample statistic

4 Last Time Bias and variability are possible sources of sampling error. To reduce bias, use random sampling. To reduce variability, use larger samples. There are also sources of nonsampling error such as processing errors, response errors, and poorly worded questions.

5 Today 1. Collecting data with experiments 2. Practical issues with experiments Confounding variables Randomized and comparative Statistical significance

6 Experiments An experiment, like an observational study, is a way to collect data. In an experiment, the people collecting the data attempt to modify, influence, or affect the individuals they are studying. The individuals in an experiment are called subjects. Subjects may be people, or animals, or even objects.

7 Experiments The actions the experimenters take in order to affect the subjects are called treatments. The word treatment means more than just medical treatments. What do the experimenters test? Does imposing a treatment on the subjects produces a certain response or reaction. The data produced by the experiment is evidence for or against this (cause and effect) relationship.

8 Experiments Examples of questions that could be studied with an experiment: 1. In K-12 education, will decreased class sizes lead to improved scores on a standardized test? 2. Will taking vitamin A lead to an increased lifespan? 3. Are nicotine patches an effective way to quit smoking?

9 Experiments Notice in each example we are interested in two variables. Example 1 Subjects: K-12 students Variable 1: Class size Variable 2 Standardized test scores Experiment 2 Subjects: People Variable 1: Vitamin A intake Variable 2: Lifespan

10 Experiments The question is whether there is a cause and effect relationship between these two variables: 1. Will decreasing class size cause an increase in test scores? 2. Will increasing vitamin A intake increase lifespan? The (hypothetical) cause is called the explanatory variable The effect is called the response variable

11 Experiments Using this terminology: experiments test the hypothesis that changes in the explanatory variable will lead to changes in the response variable. We can picture this as: Explanatory variable Response variable

12 Experiments Will decreased class sizes lead to improved scores on a standardized test? The explanatory variable is class size, and the response variable is standardized test scores. Class size Test scores

13 Experiments Will using a nicotine patch help people to quit smoking? Explanatory variable: using a nicotine patch Response variable: quitting smoking (yes/no) Nicotine patch Quitting smoking

14 Confounding How should an experiment be run? We could collect a group of individuals, impose a treatment and see what happens. For example, reduce all class sizes in a school district and see if test scores go up. Problem: if test scores increase, how do we know it was due only to the smaller class size? If we cannot distinguish the effects of different possible causes, we say those causes are confounded.

15 Confounding Here is an illustration of confounding: Explanatory variable Response variable Confounding variable

16 Confounding Example Researchers wanted to compare online college classes with traditional classes. They compared scores on standardized tests and found that students in online classes scores just as high as students in traditional classes. Explanatory variable: two values: traditional class or online class. Response variable: test scores

17 Confounding But, the students who enrolled in the online class tended to have more prior knowledge of the course material: On a test given before the semester started, the average score for students in the online course was For students in the traditional class it was How much of the performance of the students in the online course was due to their better preparation? We have no idea. The variables of taking an online course and being more prepared are confounded.

18 Confounding Taking an online course Test score Being more prepared

19 The placebo effect A very important example of a confounding variable is the placebo effect. A placebo is any medical treatment which ought to have no effect on a patient s condition. A placebo could be a pill with no active medication in it, say a sugar pill. But a placebo could also be a wide variety of other things: creams, inhalants, injections, even faked surgeries. Logically, if a patient is given a placebo, we expect to see nothing happen.

20 The placebo effect It may be surprising to find out that patients who are given a placebo sometimes do show signs of improvement this is called the placebo effect. A sick person given nothing but a placebo will probably experience fewer or less intense symptoms. It s important to realize that this isn t all in the patient s imagination. For example, it has been shown that painkillers that are administered without the patient s knowledge are less effective knowing you are getting a painkiller is part of what makes it work.

21 The placebo effect For an experimenter testing a medical treatment, the placebo effect is a very important practical problem. We can t just give a bunch of smokers the nicotine patch and see what happens. Just by giving the smokers any kind of patches (active or placebo), and telling them that the patches will help them to quit smoking, some of the smokers will probably quit.

22 The placebo effect Nicotine patches Quitting smoking Placebo effect

23 Preventing confounding What can we do to prevent confounding (from the placebo effect or any other extraneous variable)? Our experiments should be comparative, which means they should compare one or more treatment. Instead of giving all of the subjects nicotine patches, we give half of them a placebo (an inactive patch that looks the same).

24 Preventing confounding As a result of the placebo effect, we expect that all of the subjects will show some improvement, whether or not they get an active medication or a placebo. But now that we have two groups, we can look at whether the group getting the active treatment shows more improvement. Having a group that gets a placebo allows us to control for the placebo effect. That s why the group getting the placebo is sometimes called the control group. You will also see the term placebo controlled.

25 Preventing confounding But our other example of an experiment, comparing online and traditional classes, was comparative. Yet it still had confounding. Our other issue is to make sure the treatment groups are not different in any significant way. This is a similar problem to one we dealt with last class: how to pick a sample that represents a population. The solution is the same: randomize.

26 Preventing confounding In that earlier example, if we had randomly assigned students to an online or traditional class, then we can be confident that the best prepared students will not all be in one or the other class. So the best design for an experiment is one that is comparative and randomized. We choose two or more treatments, then randomly assign our subjects to one of the treatment groups. Simple random sampling is one way of making the assignments.

27 Preventing confounding Medical experiments are often double blind. This means that neither the subjects getting a treatment, nor the doctor or nurse administering that treatment knows whether it is a placebo. It makes sense not to tell the subject: we want to control for the placebo effect. If a patient knows they aren t getting an active medication, there will be no placebo effect. Why not tell the doctor? We want all of our subjects to be treated the same. A doctor could (even unconsciously) treat subjects differently if she knows they are getting a placebo.

28 Statistical significance When reading the results of an experiment, you may encounter the term statistical significance. The results of an experiment are statistically significant if they were unlikely to have occurred by chance alone. Think about it this way: if we take a group of students, randomly divide them into two groups, and give each group the same test, what do we expect? The average scores for each group should be similar. But, we don t expect them to be identical.

29 Statistical significance These chance variations are always present in the outcome of an experiment. We want to see that our treatment has made a bigger difference than what we d see by chance. If we randomly assign half of a group of students to take a traditional class, and the other half to take an online class, we want to see a difference in the results that is greater than what we d see just by chance alone. If we do, then our results are statistically significant.

30 Statistical significance A word of caution about the phrase statistically significant. The results of an experiment could be statistically significant, but practically meaningless. If we have a large number of subjects in an experiment, even very small differences can be statistically significant. For example, with enough participants, we could see a very small difference which is statistically significant.

31 Statistical significance So it may be that online class has a statistically significant difference in the average standardized test scores, but that the difference could be only 1 or 2 percentage points. In that case putting all students into online classes may not be worth the expense. So remember that statistically significant is something to look for in the results of an experiment, but by itself it does not mean important.