MATH& 146 Lesson 6. Section 1.5 Experiments

Transcription

1 MATH& 146 Lesson 6 Section 1.5 Experiments 1

2 Experiments Studies where the researchers assign treatments to cases are called experiments. When this assignment includes randomization (such as coin flips) to decide which treatment a patient receives, it is called a randomized experiment. Randomized experiments are fundamentally important when trying to show a causal connection between two variables. 2

3 Experiment For example, consider a medical study designed to test whether large doses of Vitamin C can help prevent colds. To conduct this study, researchers must ask some people in the sample to take large doses of Vitamin C. This is an experiment because some participants receive a treatment that they would not otherwise receive. 3

4 Experiment It is difficult to determine whether an experimental treatment works unless you compare groups that receive the treatment to groups that don't. In the Vitamin C study, for example, researchers might create two groups of people: a treatment group that takes large doses of Vitamin C and a control group that does not take Vitamin C. Having a control group is usually crucial to interpreting the results of experiments. 4

5 Treatment and Control Groups The treatment group in an experiment is the group of sample members who receive the treatment being tested. The control group in an experiment is the group of sample members who do not receive the treatment being tested. It is important for the treatment and control groups to be selected randomly and to be alike in all respects except for the treatment. 5

6 Principles of Experimental Design Randomized experiments are generally built on four principles. Controlling so that subjects receive identical treatments Randomizing subjects between groups Replicating the number of subjects between groups Blocking for other variables that might influence the results 6

7 Controlling Researchers assign treatments to cases, and they do their best to control any other differences in the groups. For example, when patients take a drug in pill form, some patients take the pill with only a sip of water while others may have it with an entire glass of water. To control for water consumption, a doctor may ask all patients to drink a 12 ounce glass of water with the pill. 7

8 Randomization Researchers randomize patients into treatment groups to account for variables that cannot be controlled. For example, some patients may be more susceptible to a disease than others due to their dietary habits. Randomizing patients into the treatment or control group helps even out such differences, and it also prevents accidental bias from entering the study. 8

9 Replication The more cases researchers observe, the more accurately they can estimate the effect of the explanatory variable on the response. In a single study, we replicate by collecting a sufficiently large sample. Additionally, a group of scientists may replicate an entire study to verify an earlier finding. 9

10 Blocking Researchers sometimes know or suspect that variables, other than the treatment, might influence the response. Under these circumstances, they may first group individuals based on this variable into blocks and then randomize cases within each block to the treatment groups. This strategy is often referred to as blocking. 10

11 Blocking For instance, if we are looking at the effect of a drug on heart attacks, we might first split patients in the study into low-risk and high-risk blocks, then randomly assign half the patients from each block to the control group and the other half to the treatment group. 11

12 Principles of Experimental Design It is important to incorporate the first three experimental design principles (controlling, randomization, replication) into any experiment, and the material we will learn in this course will describe methods for analyzing data from such experiments. Blocking is a slightly more advanced technique, but the statistical methods we will learn may be extended to analyze data collected using blocking. 12

13 Reducing Bias in Human Experiments Randomized experiments are the gold standard for data collection, but they do not ensure an unbiased perspective into the cause and effect relationships in all cases. Human studies are perfect examples where bias can unintentionally arise. The problem is that people can be affected by their beliefs as well as by real treatments. 13

14 Reducing Bias in Human Experiments For example, stress and other psychological factors have been shown to affect resistance to colds. If people taking Vitamin C get fewer colds than people who don't, we can't necessarily conclude that the Vitamin C was responsible. It might be that people stayed healthier because they believed that Vitamin C works. 14

15 Reducing Bias in Human Experiments Put yourself in the place of a person in the study. If you are in the treatment group, you anticipate that the Vitamin C will help you. On the other hand, a person in the other group sits idly, hoping their participation doesn't increase her risk of a cold. 15

16 Reducing Bias in Human Experiments These perspectives suggest there are actually two effects: the one of interest is the effectiveness of the drug, and the second is an emotional effect that is difficult to quantify. Researchers aren't usually interested in the emotional effect, which might bias the study. To circumvent this problem, researchers do not want patients to know which group they are in. 16

17 Reducing Bias in Human Experiments As long as the participants don't know whether they are in the treatment or control group (that is, whether they got the real pills or the placebo), any effect arising from psychological factors known as a placebo effect should affect both groups equally. Then, if people in the Vitamin C group get fewer colds than people in the control group, we have evidence that Vitamin C really works. 17

18 Blinding In statistical terminology, the practice of keeping people in the dark about who is in the treatment group and who is in the control group is called blinding. A single-blind experiment is one in which the participants don't know which group they belong to, but the experimenters (the people administering the treatment) do know. Using a placebo is one way to create a single-blind experiment. 18

19 Blinding However, sometimes a single-blind experiment can still be unreliable if the experimenters can influence outcomes. For example, in an experiment that involves interviews, the experimenters might speak differently to people who received the real treatment than to those who received the placebo. 19

20 Blinding This type of problem can be avoided by making the experiment double-blind, which means neither the participants nor the experimenters know who belong to each group. (Of course, someone must keep track of the two groups in order to evaluate the results at the end. In typical double-blind experiments, researchers hire experimenters to make any necessary contact with the participants.) 20

21 Blinding An experiment is single-blind if the participants do not know whether they are members of the treatment group or the control group, but the experimenters do know. An experiment is double-blind if neither the participants nor the experimenters (people administering the treatment) know who belongs to the treatment group and who belongs to the control group. 21

22 Example 1 For the experiment below, identify any problems and explain how the problems could have been avoided. A chiropractor wants to know if his adjustments relieve back pain. He performs adjustments on 25 patients with back pain. After interviewing all 25 patients, 18 say they feel better. He concludes that the adjustments are an effective treatment. 22

23 Example 2 For the experiment below, identify any problems and explain how the problems could have been avoided. A new drug for attention deficit disorder (ADD) is supposed to make the affected children more polite. Randomly selected children suffering from ADD are divided into treatment and control groups. Those in the control group receive a placebo that looks like the real drug. The experiment is single-blind. Experimenters interview the children one-on-one to decide whether they became more polite. 23

24 Inference Statistical inference means drawing conclusions about a study population based on data from a sample. There are two main types of inferences: Population Inference: The results seen in the data can be extended to a wider population. Causal Inference: The differences seen in the data were due to the group assignment. Cause and effect relationships can be drawn. 24

25 Scope of Inference Scope of inference refers to the type of inferences (conclusions) that can be drawn from a study. The types of inferences we can make (inferences about the population and inferences about causeand-effect) are determined by two factors in the design of a study: how the subjects were selected from the population and how the subjects were assigned to groups. 25

26 Scope of Inference If the subjects of a study are randomly selected from a population, we can assume our results can be extended to the overall population (assuming the sample is large enough). Once selected, if the subjects are then randomly assigned to the treatment and control groups, then we can claim causal inference. Differences between the treatment and control group will even out, except for the treatment itself. 26

27 Example 3 A dentist convinces some his dental patients to be subjects in an experiment. Participants are split into two groups, those who are taking at least 500 mg of Vitamin C each day and those who don't. He then asks them how often they have canker sores in their mouth. He finds that the proportion of those who take Vitamin C daily and complain of canker sores is significantly smaller than the proportion of those who don't. Can this study be extended to the overall population? Can this study be used to show Vitamin C reduces canker sores? 27

28 Example 4 A dentist convinces some his dental patients to be subjects in an experiment. He randomly assigns half of them to take 500 mg of Vitamin C each day and the other half to take a placebo for three months. He finds that the proportion of those who take Vitamin C daily and complain of canker sores is significantly smaller than the proportion of those who don't. Can this study be extended to the overall population? Can this study be used to show Vitamin C reduces canker sores? 28

29 Example 5 A dentist selects a random sample of dental patients in town and convinces them to be subjects in an experiment. Participants are split into two groups, those who are taking at least 500 mg of Vitamin C each day and those who don't. He finds that the proportion of those who take Vitamin C daily and complain of canker sores is significantly smaller than the proportion of those who don't. Can this study be extended to the overall population? Can this study be used to show Vitamin C reduces canker sores? 29

30 Example 6 A dentist selects a random sample of dental patients in town and convinces them to be subjects in an experiment. He randomly assigns half of them to take 500 mg of Vitamin C each day and the other half to take a placebo for three months. He finds that the proportion of those who take Vitamin C daily and complain of canker sores is significantly smaller than the proportion of those who don't. Can this study be extended to the overall population? Can this study be used to show Vitamin C reduces canker sores? 30