Experimental Design Be sure you understand that: Experiments are studies in which the researcher imposes a treatment on experimental units. Sometimes different treatments are simply compared with one another. If no treatment is assigned or imposed, the study is called an observational study. Some experiments have a control group (a group of experimental units that receive no treatment or receive only a placebo), but this is not necessary for a well-designed experiment. Experimental units are the smallest independent objects to which treatments are assigned and on which a response is measured. o Consider an experiment that is designed to determine which of several types of fish food will result in the greatest weight gain for fish. If tanks contain several fish, and food is added to the water in the tank, then the tank is the experimental unit (not the individual fish), since the fish in a tank are not independent of one another, but tanks are independent of one another. Replication refers to having multiple experimental units in each treatment group (repeating the treatment), not to repeating the entire experiment. In an experiment, randomization refers to randomly assigning experimental units to the treatments. Often the experimental units are not a random sample of the population of interest. While this is not a problem with the experimental design, it may limit the scope of inference for the experimental results. (Note that random samples are necessary in surveys.) The purpose of random assignment (of experimental units to treatment groups or of treatments to experimental units) is to even out extraneous variables and make treatment groups that are approximately similar in all respects except for the treatment. In a double blind experiment, someone must know which treatment the experimental unit received! The subjects (assuming they are people) are blind to which treatments they are receiving, and anyone who interacts with the subjects should also be blinded to treatments. o If the response variable is in any way a subjective evaluation, then the person performing that evaluation must be blind to what treatments were applied. But obviously, some person or people on the research team must have a record of what treatments have been applied to what subjects!
A confounding variable is one that affects the response variable and also is related to group membership. A variable that affects the response variable and is not related to group membership (that is, the variable would be expected to even out across the groups) is not a confounding variable. You may refer to this type of variable as an extraneous variable. It is best to avoid using the term lurking variable. It will almost always get you in trouble! o For example: It has been observed that people who take long vacations have, on average, significantly longer lifespans than people who don t. Can we conclude that vacationing is a way to extend your lifespan? Not necessarily: a person s income could be a confounding variable people with higher incomes are more likely to be able to take long vacations, and they re also more likely to afford health care that could lead to longer lifespans. Note that something like exercise would probably be an extraneous variable and not a confounding variable. Exercise may indeed be associated with longer lifespans, but is there an association between getting exercise and taking long vacations? Blocks are groups of experimental units that are homogeneous with respect to some inherent characteristic that is expected to affect the response to treatments. o Blocks are considered a form of control blocks help control known sources of variability among the experimental units so that the experimenter is better able to detect differences in the response variable that are due to the treatments. o Blocking is used to control the factors you can see; randomization helps balance the ones you cannot see. Richard L. Scheaffer, AP Statistics Chief Faculty Consultant, 1997 1999
Multiple choice questions: 1. Which of the following is a key distinction between well designed experiments and observational studies? A. More subjects are available for experiments than for observational studies. B. Ethical constraints prevent large-scale observational studies. C. Experiments are less costly to conduct than observational studies. D. An experiment can show a direct cause-and-effect relationship, whereas an observational study cannot. E. Tests of significance cannot be used on data collected from an observational study. 2. A new medication has been developed to treat sleep-onset insomnia (difficulty in falling asleep). Researchers want to compare this drug to a drug that has been used in the past by comparing the length of time it takes subjects to fall asleep. Of the following, which is the best method for obtaining this information? A. Have subjects choose which drug they are willing to use, then compare the results. B. Assign the two drugs to the subjects on the basis of their past sleep history without randomization, then compare the results. C. Give the new drug to all subjects on the first night. Give the old drug to all subjects on the second night. Compare the results. D. Randomly assign the subjects to two groups, giving the new drug to one group and no drug to the other group, then compare the results. E. Randomly assign the subjects to two groups, giving the new drug to one group and the old drug to the other group, then compare the results.
3. A researcher wishes to test a new drug developed to treat hypertension (high blood pressure). A group of 40 hypertensive men and 60 hypertensive women is to be used. The experimenter randomly assigns 20 of the men and 30 of the women to the placebo and assigns the rest to the treatment. The major reason for separate assignment for men and women is that A. it is a large study with 100 subjects B. the new drug may affect men and women differently C. the new drug may affect hypertensive and nonhypertensive people differently D. this design uses matched pairs to detect the new-drug effect E. there must be an equal number of subjects in both the placebo group and the treatment group 4. Automobile brake pads are either metallic or nonmetallic. An experiment is to be conducted to determine whether the stopping distance is the same for both types of brake pads. In previous studies, it was determined that car size (small, medium, large) is associated with stopping distance, but car type (sedan, wagon, coupe) is not associated with stopping distance. This experiment would best be done A. by blocking on car size B. by blocking on car type C. by blocking on stopping distance D. by blocking on brake pad type E. without blocking
5. A group of students has 60 houseflies in a large container and needs to assign 20 to each of three groups labeled A, B, and C for an experiment. They can capture the flies one at a time when the flies enter a side chamber in the container that is baited with food. Which of the following methods will be most likely to result in three comparable groups of 20 houseflies each? A. Label the first 20 flies caught as group A, the second 20 caught as group B, and the third 20 caught as group C. B. Write the letters A, B, and C on separate slips of paper. Randomly pick one of the slips of paper and assign the first 20 flies caught to that group. Pick another slip and assign the next 20 flies to that group. Assign the remaining flies to the remaining group. C. When each fly is caught, roll a die. If the die shows an even number, the fly is labeled A. If the die shows an odd number, the fly is labeled B. When 20 flies have been labeled A and 20 have been labeled B, the remaining flies are then labeled C. D. Place each fly in its own numbered container (numbered from 1 to 60) in the order that it was caught. Write the number from 1 to 60 on slips of paper, put the slips in a jar, and mix them well. Pick 20 numbers out of the jar. Assign the flies in the containers with those numbers to group A. Pick 20 more numbers and assign the flies in the containers with those numbers to group B. Assign the remaining 20 flies to group C. E. When each fly is caught, roll a die. If the die shows a 1 or 2, the fly is labeled A. If the die shows a 3 or 4, the fly is labeled B. If the die shows a 5 or 6, the fly is labeled C. Repeat this process for all 60 flies.
Free Response Questions: 6. When a tractor pulls a plow through an agricultural field, the energy needed to pull that plow is called the draft. The draft is affected by environmental conditions such as soil type, terrain, and moisture. A study was conducted to determine whether a newly developed hitch would be able to reduce draft compared to the standard hitch. (A hitch is used to connect the plow to the tractor.) Two large plots of land were used in this study. It was randomly determined which plot was to be plowed using the standard hitch. As the tractor plowed that plot, a measurement device on the tractor automatically recorded the draft at 25 randomly selected points in the plot. After the plot was plowed, the hitch was changed from the standard one to the new one, a process that takes a substantial amount of time. Then the second plot was plowed using the new hitch. Twenty-five measurements of draft were also recorded at randomly selected points in this plot. (a) What was the response variable in this study? Identify the treatments. What were the experimental units? (b) Given that the goal of the study is to determine whether a newly developed hitch reduces draft compared to the standard hitch, was randomization used properly in this study? Justify your answer.
(c) Given that the goal of the study is to determine whether a newly developed hitch reduces draft compared to the standard hitch, was replication used properly in this study? Justify your answer. (d) Plot of land is a confounding variable in this experiment. Explain why.
7. Because of concerns about employee stress, a large company is conducting a study to compare two programs (tai chi or yoga) that may help employees reduce their stress levels. Tai chi is a 1,200-yearold practice, originating in China, that consists of slow, fluid movements. Yoga is a practice, originating in India, that consists of breathing exercises and movements designed to stretch and relax muscles. The company has assembled a group of volunteer employees to participate in the study during the first half of their lunch hour each day for a 10-week period. Each volunteer will be assigned at random to one of the two programs. Volunteers will have their stress levels measured just before beginning the program and 10 weeks later at the completion of it. (a) A group of volunteers who work together ask to be assigned to the same program so that they can participate in that program together. Give an example of a problem that might arise if this is permitted. Explain to this volunteer group why random assignment to the two programs will address this problem. (b) Someone proposes that a control group be included in the design as well. The stress level would be measured for each volunteer assigned to the control group at the start of the study and again 10 weeks later. What additional information, if any, would this provide about the effectiveness of the two programs? (c) Is it reasonable to generalize the findings of this study to all employees of this company? Explain.