Chapter 13: Experiments

Transcription

1 Chapter 13: Experiments The objective of sampling is to describe a population. In the process of collecting the sample, sample units are not to be modified or affected by the researcher. In contrast, experimental studies focus on how population units responds to an imposed treatment or manipulation of units by the researcher. In studying the relationship between two variables, it may be possible to identify response variable and explanatory variables. In the context of experiments, a response is elicited by the explanatory variables. Example: The human papillomavirus (HPV) vaccine is thought to prevent infection by some species of human papillomavirus. An experiment aimed at assessing the effectiveness of the vaccine would assign some subjects to a group receiving the vaccine and the remaining subjects to a group receiving a placebo 1. A diagram outlining experiment shows the assignment of subjects to groups: Observational units (Subjects) Group 1 (Placebo) Group 2 (HPV vaccine) Compare (Contract HPV: y/n) Explanatory Response Variable Variable The explanatory variable is a binary variable with two levels (vaccine or placebo) and the response variable is the presence or absence of HPV (after some time has lapsed) is also binary. It s important to distinguish between observational and experimental studies because the scope of inference differs between the two type of studies. Observational studies: The researcher collects data without manipulating the units. Examples: 1. Smoking and lung cancer studies involving humans are observational because the usual protocol is to observe whether a person smokes or not (or how much they smoke) and whether they develop lung cancer. 2. Recruit a set of volunteers (children) and administer the Sabin polio vaccine. The proportion of subjects that contract the disease provides some information on the effectiveness of the vaccine. However, it is not possible to be confident that the results 1 A treatment with no value. 93

2 are attributable to the vaccine since there may be other factors are responsible. 2 The 1960 Sabin polio vaccine trial carried out in the Soviet Union used 10 million children in this manner. Observational studies are often classified as one of two types: 1. Prospective studies: The explanatory variable (but not the response variable) is observed at the beginning of the study. The subjects are observed a later time to record the value of the response variable. Example: Researchers hope to find out how well genetic markers can predict the development of Type 1 diabetes (or juvenile diabetes). A large number of children are sampled, some having this marker and others not having this marker. Subjects are followed for a few years. Researchers then observe the presence or absence of Type I diabetes in each child. 2. Retrospective studies: The current condition of the subjects are known (for example diseased or not) and data are collected on the past history of the subjects. Example: A 1995 study carried out in Wales examined the association between oral contraception use and heart attacks. Two samples of women were collected. The first sample consisted of women that had been treated for a heart attack, the second consisted of women that had not experienced a heart attack. The rate of contraceptive use was compared between groups. Differences in the rate of oral conceptive use implies that there is an association between contraceptive use and the incidence of heart attacks. 3 Controlled Experiments: The researcher controls (manipulates) the value of the explanatory variable on each unit. Each experimental unit is randomly assigned to a group. Units in each group receive the same treatment, and no two groups receive the same treatment. Before receiving the treatments, there are little or no systematic differences between groups because of the random assignment of units to groups. If there are substantive differences between groups with respect to the response variable after the treatments are administered, then it can be concluded that differences in the explanatory variable were the cause of the differences. It s possible to draw a statistically valid conclusion stating that a cause-and-effect relationship exists between the response and explanatory variables. 2 Polio is a seasonal disease, and the severity of the polio season depends on weather and other factors. 3 In a long-term follow-up study, the relative risk of death in oral contraceptive users from circulatory diseases as a group was reported to be 4.2 greater than in the group that did not use oral contraceptives. 94

3 HPV vaccine experiments have found significant evidence that the vaccine confers some level of immunity to recipients of the vaccine. The effectiveness has been estimated to be 83 percent. In the 1954 Salk vaccine trials, volunteer children were randomly assigned to either group A or group B. Group A volunteers received the Salk vaccine; group B received a placebo injection. The response variable was the incidence (y/n) of paralytic polio. The effectiveness of the Salk vaccine was estimated to be 71.3 percent. Strength of inferences - observational studies While observational studies can establish association between two variables, they cannot establish cause-and-effect because there may be other uncontrolled variables that affect the response variable. When other variables affect the response variable, then the effect of the explanatory variable is confounded with the other variables. 4 For example, long-term smokers tend to contract more respiratory infections (bronchitis and pneumonia) than nonsmokers. Consequently, smoking and number and severity of respiratory infections may explain why smokers are 10 to 20 times more likely to get lung cancer or die from lung cancer than people who do not smoke. 5 It is not possible to exclude the possibility that repeated respiratory infections are responsible (or partly responsible) for the development of lung cancer in smokers. Strength of inferences - controlled experiments It is possible to establish cause-and-effect with a high degree of confidence by conducting a properly designed controlled experiment. The key is to randomly assign subjects to different treatment groups. Each treatment group receives a different level of the explanatory variable. The only systematic differences between groups are the levels (values) of the explanatory variable. The potential effect of confounding variables is minimized and if large differences in the response variable are observed, then it is valid to conclude that differences are attributable to the explanatory variable. Random assignment of experimental units to treatment groups cannot completely eliminate the possibility of systematic differences between groups before the treatments are applied. The risk of residual systematic differences can be minimized by using large numbers of experimental units. 4 Confounding variables were discussed in Chapters 3 and 7. 5 CDC website 95

4 Experimental design terminology 1. The experimental units are the units used in the experiment. If the units are people, they are referred to as subjects or participants. The experimental units ought to be representative of the population. 2. The explanatory variable is referred to as a factor. A factor may be quantitative or categorical. 3. A factor has several levels (or values) in an experiment. Levels are chosen by the researcher. In the 1954 Salk polio vaccine trials, the factor was the presence/absence of the vaccine in the injection given to a subject. Levels were present and absent. 4. A treatment is a particular factor level applied to some units in the experiment. In the 1954 Salk polio vaccine trials, the treatments are present and absent. If there is more than one factor in the experiment, then a particular combination of levels applied to some units is a treatment. 5. An experiment is a study wherein the researcher controls the levels of the factors to create the treatments. Units are randomly assigned to treatment groups. Example: To assess the effect of diet on mice lifetimes, two factors are identified: 1. Caloric intake. Levels: 100 calories/day (normal), 80 calories/day (reduced) 2. Protein intake. Levels: 5 grams/day (high), 2 grams/day (low) The four treatments are (normal,high), (normal,low), (reduced,high), (reduced,low). The number of mice randomly assigned to the following four treatments are shown in Table 1: Table 1: Factors, levels and number of experimental units assigned to each treatment. Calorie level Protein level normal reduced high low Essential elements of a controlled experiment: 1. Comparison: The principal objective is to compare two or more treatments. Often, there are just two treatments: the control treatment (no action or manipulation) and the experimental treatment (the new action, drug or therapy). 2. Randomization: Subjects are assigned randomly to groups. Randomization helps ensure that the effects of confounding variables are minimized since the confounding effects are roughly equalized across treatment groups. 96

5 3. Replication: The effects of the treatments are replicated by assigning more than one (perhaps many) experimental units to each treatment group. Replication reduces differences between groups that might be present despite randomization. In experiments with human subjects, further control may be accomplished by employing one or more of the following strategies: 1. Blinding: The subjects in an experiment are blind if they do not know which treatment group they are in. The researchers are blind if, when measuring the response variable for a subject, they don t know which treatment group the subject belongs to. An experiment is double-blind if both the subjects and the researchers are blind. Blindness for the subjects is important because knowing which treatment they received may influence their perception of well-being. Blindness for the researchers is important when they are measuring the response because knowing which treatment the subject received may (unintentionally) bias their evaluation. It is not always possible to make an experiment double-blind. 2. Placebo: The control group receives a placebo to eliminate the possible confounding caused by one group receiving something (a pill, for example) while the other doesn t. In clinical studies of drug therapies, there is a often a perception of improvement from receiving a pill even when no active agent is present in the pill. This is known as the placebo effect. The use of a placebo equalizes the placebo effect because all subjects receive the psychological benefit of knowing that they may be receiving something beneficial. It isn t always possible to use a placebo; if for example, the experiment compares two medical treatments, one of which is surgery and the other of which is nonsurgical, then it s considered unethical to give the control group a placebo surgery. Example: The National Institutes of Health funded the first double-blind fetal cell transplant study for the purpose to treating Parkinson s disease. 6 This study, conducted at the University of Colorado, involved 40 patients with Parkinson s and spanned over five years ( ). The (new) treatment for Parkinson s disease surgically injected fetal tissue into the brain of the patient to replace damaged brain cells and compensate for the loss of essential nerve cell groups. Researchers randomly assigned 20 subjects to receive fetal tissue and 20 subjects to receive a placebo surgery. After some time had passed, the subjects were assessed by a doctor and the progression of the disease was observed. 7 6 Parkinson s disease is a disorder of the brain leading to tremors and difficulty with walking, movement, and coordination. 7 Surgery helped a small number of Parkinson s patients but not all who underwent the experimental therapy. 97

6 What are the experimental units? What is the factor? What are the levels of the factor? Diagram of this study: 40 subjects (Parkinson s patients) 20 - fetal tissue 20 - placebo surgery Disease progression Explanatory Variable Response Variable Suppose those receiving the fetal cell tissue experienced less progression of the disease than those receiving the placebo surgery. If the difference in disease progression between the two treatment groups is significantly large, then it would be concluded that there is statistical evidence of a beneficial effect due to fetal tissue injection. Example: Anorexia therapies To assess the efficacy two behavioral therapies for the treatment of anorexia 8, 72 volunteer anorexia patients participated in an experiment in which each was randomly assigned to either the placebo group, family therapy, cognitive behavioral therapy. Each subject underwent 6 weeks of treatment. Pre- and post-experiment weights were recorded. What are the experimental units? What is the factor? How many levels does the factor have? What is the response variable? Pre-experiment weight is presumably related to post-experiment weight. How might the potential confounding effects of pre-experiment weight be reduced or eliminated? 8 Anorexia is an eating disorder characterized by refusal to maintain a healthy body weight and an obsessive fear of gaining weight. 98

7 Blocking Random assignment of subjects to treatments does not guarantee that the groups will be perfectly balanced in every way. It only ensures that there is no bias in assigning units to treatments. If it is known that the responses may differ among treatments because of a confounding variable (such as pre-experiment weight), then blocking may be used to help isolate the treatment effects. Blocking promotes balance among treatment groups with respect to the confounding variable. This situation is analogous to stratification in sampling. Back to the example: Pre-experiment weight is presumed to be associated with post-experiment weight. It s expected that pre-experiment weight partially determines post-experiment weight even if a treatment promotes weight gain. To introduce blocking to the experiment, three blocks of subjects are defined based on pre-experiment weight: less than 80 lb, between 80 and 85 lb, and greater than 85 lb. Approximately 24 subjects would be in each block. Within each block, randomly assign the subjects to a treatment group (control, family therapy, cognitive behavioral therapy). Blocks and treatments and number of subjects are shown below: Treatments Blocks (pre-experiment weight) (therapy) < 80 lb lb 85 lb Control Family Cognitive behavioral Example: Researchers in the US and Europe analyzed the results of 7 clinical trials on the use of tamiflu in the prevention of secondary infections following influenza. Tamiflu is used to shorten the duration of flu and reduce its symptoms. 3,815 adolescents and adults were randomly assigned to four groups. Two groups received tamiflu, in either inhalable form or in a combination of inhaled powder and intranasal spray. The other two groups received placebo treatments administered in similar fashions. What are the factor(s) and levels? 99

8 How many treatments were present? Adolescents and adults Tamiflu Placebo Inhalable form Combination form Inhalable form Combination form 7 Compare secondary infection rates 17% of those who received the placebo treatments developed secondary infections that required antibiotic use compared to 11% of those receiving the drug. 9 Conclusion? In a completely randomized design, all experimental units have an equal chance of receiving any treatment. In a randomized block design, the randomization occurs only within blocks. Experiments with more than one factor are called multifactor or factorial designs. It s sometimes of interest to investigate interactions between factors. For example, to suppress HIV, drugs A and B may be used alone or in combination. The set-up for an experiment to investigate the drugs may use subjects randomly assigned to treatments as tabled below. Table 2: Number of subjects per treatment group Drug A Drug B 0 mg/day 100 mg/day 300 mg/day 0 mg/day mg/day mg/day Archives of Internal Medicine, Nov