CHAPTER 6. Experiments in the Real World

Transcription

1 CHAPTER 6 Experiments in the Real World

2 EQUAL TREATMENT FOR ALL SUBJECTS The underlying assumption of randomized comparative experiments is that all subjects are handled equally in every respect except with regard to the treatments being compared. If the subjects are not handled equally in other respects, there might be bias. However, handling subjects exactly the same way is difficult, if not impossible. 2

3 DOUBLE-BLIND EXPERIMENTS Patients who know they are receiving a placebo may feel worse just because they know that they are not being treated. A doctor who knows a patient is receiving a placebo may treat that patient differently. A double-blind experiment is one in which neither the subject nor the people who work with them know which treatment each subject is receiving. The standard for experiments is a randomized, double-blind, placebo-controlled trial. 3

4 EXAMPLE 6.1 A researcher told 13 people who were very sensitive to poison ivy that the stuff being rubbed on one arm was poison ivy. It was a placebo, but all 13 broke out in a rash. The stuff rubbed on the other arm really was poison ivy, but the subjects were told it was harmless and only 2 of the 13 developed a rash. 4

5 EXAMPLE 6.2 An experiment that claimed to show that meditation reduces anxiety proceeded as follows. The experimenter interviewed the subjects and rated their anxiety level. Then the subjects were randomly assigned to 2 groups. The experimenter taught one group how to meditate and they meditated daily for a month. The other group was simply told to relax more. At the end of the month, the experimenter interviewed all the subjects and rated their anxiety level. The meditation group now has less anxiety. Psychologists said the results were suspect because the ratings were not blind. Explain what this means and how lack of blindness could bias the reported results. 5

6 PROBLEMS DURING EXPERIMENTS Experiments suffer from similar problems as sample surveys that may cause bias in the results. Refusal to participate is a serious problem for medical experiments. Often the groups that refuse to participate are systematically different from those who cooperate. Nonadherers are subjects who don't follow their assigned treatment. They may not take the treatment as prescribed, or take additional treatments on their own. Dropouts are subjects who begin the experiment but do not complete it. This is a grave problem if subjects drop out because of their reaction to one of the treatments. 6

7 UNWARRANTED GENERALIZATION Technically, the results of an experiment only tell us what happened to the subjects in the experiment. But we want to draw conclusions about the entire population (e.g., all adults, all students, all patients with high blood pressure,...). Can we generalize the experimental results to a wider population? 7

8 UNWARRANTED GENERALIZATION In order to generalize: We need to make sure the results are statistically significant. The environment of the experiment should be realistic to allow generalization. If the application of the treatment in the real world is very different from the experimental setup, the effects might be very different. We can only generalize for the population that was considered in the study. 8

9 COMPLETELY RANDOMIZED DESIGN The basic kind of experimental design is a completely randomized design (CRD) In a completely randomized experimental design, all the experimental subjects are allocated at random among all the treatments 9

10 EXAMPLE 6.3 A pharmaceutical company has developed an experimental drug to relieve the symptoms associated with the common cold. The company found 300 adults in New York aged years old and randomly divided them into three groups. Group 1 is given the new drug, group 2 is given the old drug, and group 3 is given a placebo. After one week of treatment, the percentage of each group that still has cold symptoms is compared. 10

11 EXAMPLE 6.3 Let s draw a diagram of this experiment (to show that it is a completely randomized design). What should we do to avoid the placebo effect? There are still a few weaknesses with this experiment. Can you spot them? 11

12 QUESTION TO CONSIDER What if men and women respond differently to treatments? A completely randomized design considers all subjects, both men and women, as a single pool If we subject gender might affect the response variables, a better design considers women and men separately 12

13 BLOCK DESIGNS A block is a group of experimental subjects that are known, prior to the experiment, to be similar in some way that is expected to affect the response in treatments. Examples of blocks? Gender Age Race A block design involves the random assignment of subjects to treatments being carried out separately within blocks. 13

14 EXAMPLE 6.4 Acme Medicine is conducting an experiment to test a new vaccine against a placebo. Acme is worried that gender may play a role in the vaccine s effectiveness, so they use gender as a blocking variable. To test the vaccine, Acme has 500 subjects 300 men and 200 women. Draw a diagram to illustrate a good way to design this experiment. 14

15 MATCHED-PAIRS DESIGNS Matched-pairs designs are designed to compare two treatments. There are two common types: Design 1: For each subject, try to find another subject who is as closely matched as possible, i.e., similar in the variables that might influence the outcome. Then randomly decide which subject receives which treatment. Design 2: Use each subjects as his or her own control by assigning both treatments to each subject and randomize the order of these treatments. 15

16 EXAMPLE 6.5 Example 1: In order to test if using a new instructional software in the classrooms improves test scores in middle schools, we find 2 similar schools (Note that here schools are our subjects, not students). We randomly assign one school the new software and the other school continues their current routine. Example 2: In an experiment to compare if Coke drinkers preferred the taste of Pepsi over Coke, all subjects tasted both colas. To avoid that the responses depend on which cola is tasted first, the order of tasting is chosen at random for each subject. 16

17 REMINDERS Chapter 6 problems are posted online. They do not need to be turned in. They are just suggested problems to work on. Chapter 5 homework is due today Exam 1 is tomorrow 17