Handout 10: Association vs. Causation and Observational Studies vs. Designed Experiments STAT 100 Spring 2016

Transcription

1 Example: Adoption and Suicide In September of 2013, researchers from the University of Minnesota published a study in the journal Pediatrics. This study was described in a Fox News article titled Adopted teens may be at higher risk of suicide. This article described the results as follows: [The researchers] examined data from an existing University of Minnesota study of 692 adopted children and 540 non-adopted siblings in Minnesota All of the adopted kids, who were between 11 and 21 years old during the study period, had been taken in by their families before age two. Almost three quarters of the adopted children were born abroad, most of the foreign-born children were from South Korea and 60 percent of those were girls. At the beginning of the study, and again about three years later, the researchers asked participating parents families if the children had made a suicide attempt. Over the three years of the study, 56 children attempted suicide at least once, according to the family members' reports. Of those kids, 47 were adopted and nine were not adopted. 1. Create a table of counts based on the data obtained in this study: Suicide Attempt No Suicide Attempt Totals Adopted Not adopted Totals 2. Find the risk of suicide attempt for both those teens that were adopted and those that were not adopted. 1

2 3. Use your results from Question 2 to complete the 100% stacked column chart. 4. Calculate the risk ratio and use it to express the increase in risk of suicide attempt associated with adoption. 5. Suppose a news magazine plans to write an article about this study. They are considering using the title, Adoption Increases the Risk of Suicide Attempt or Adoption Quadruples the Risk of Suicide Attempt. Do you think these titles are appropriate? Why or why not? 2

3 Association vs. Causation For an association to exist between two variables, say A and B, they simply must be related to one another. This is a causal relationship only if one variable directly influences the other. Note that associations between variables A and B may be due to any of the following: Causation: Changes in A cause changes in B (or changes in B cause changes in A) Common Response: Changes in both A and B are caused by a third variable, C Confounding: Changes in B are caused by both changes in A and C, and the variables A and C are related It is important to note that ASSOCIATION DOES NOT IMPLY CAUSATION! Source: Statistics, Concepts and Controversies by David S. Moore (2 nd Edition) 3

4 6. Reconsider the study linking adoption to increased risk of suicide attempt. Can you think of any other variable that might be affecting both adoption and suicide attempts? Note that the association between adoption and suicide attempts could be due to a common response to this third variable; thus, it would not be fair to say that adoption has a causal effect on suicide. 7. Can you think of another variable related to adoption that might also affect suicide attempts? Note that the association between adoption and suicide attempts could be due to this confounding variable; thus, it would once again not be fair to say that adoption has a causal effect on suicide. 8. In this study, it is quite possible that other variables not accounted for could be explaining the association between adoption and increased risk of suicide attempt; thus, one should not conclude that adoption increases the risk of attempting suicide (i.e., there is association, but not evidence of causation). Does this change your answer to Question 5? 4

5 9. Evaluate the following actual headlines used to publicize the results of this study. Do you think these headlines are fair, or are they misleading? Explain your reasoning. Adopted teens may be at higher risk of suicide (foxnews.com) Adopted teens face high suicide risk (medpagetoday.com) Adoptees four times more likely to attempt suicide (medscape.com) 10. Read through the Fox News article. Overall, do you think the author provided an appropriate overview of this study? Why or why not? 5

6 Example: Breakfast Cereal and Healthy Weights A 2012 study was conducted to investigate the relationship between eating breakfast cereal and weights of children. One summary of this study was headlined, Breakfast Cereals Prevent Overweight in Children. The article claims that based on this study, we can conclude that regularly eating cereal for breakfast every day (even super-sugary cereals!) is tied to healthy weights for children. Source: worldhealthme.blogspot.com. 1. Can you think of another variable that might be affecting both whether a child eats breakfast cereal and whether they are at a healthy weight? Note that the association between eating breakfast cereal and having a healthy weight could be due to a common response to this third variable; thus, it would not be fair to say that eating breakfast has a causal effect on whether a healthy weight is maintained. 2. Note that this headline uses the word prevent. Does this imply that there is an association or a causal relationship between eating breakfast cereal and having a healthy weight? Is this fair? Explain. 3. Write a more appropriate headline for this article. 6

7 4. Read through this article. Overall, do you think the author provided an appropriate overview of this study? Why or why not? 5. Here is another media outlet s account of this study: foxnews.com. Is their headline more appropriate? Do they provide an appropriate overview of the study, and do you think their conclusions are fair? Why or why not? 6. One last point of discussion related to this study. Consider the following quote from the study s lead author: "[Cereal] is an excellent breakfast choice, it's simple, and gets those essential nutrients that children need, especially low income minority children," said Frantzen, who is employed by Dairy MAX, a regional dairy council in Grand Prairie, Texas. The original published manuscript provides the following information: Does this information impact your overall impressions from this study? Explain. 7

8 Confounding and Lurking Variables Confounding occurs when two independent variables are both associated with a dependent variable but are also associated with each other. If a variable has the potential for confounding but is not measured and accounted for in the study, it is called a lurking variable. For example, in the breakfast cereal study, physical activity level could be viewed as a lurking variable because it is reasonable to assume this is associated with both eating breakfast and maintaining a healthy weight. Similarly general quality of nutrition in the home could be viewed as a lurking variable. Neither one of these variables were accounted for in the study. The potential for lurking variables to affect associations makes it very difficult for us to establish causal relationships from study results. However, it s not impossible for us to do so! The remainder of this handout will discuss statistical methods for establishing causation. To understand this, we must first discuss different methods for collecting data: observational studies vs. designed experiments. Types of Studies: Experimental and Observational A designed experiment involves randomly assigning subjects to certain experimental conditions and then observing outcomes on the dependent variable. These experimental conditions correspond to the values of the independent variable. On the other hand, an observational study involves simply observing the values of the independent and dependent values for subjects in the study. In this case, subjects are not randomly assigned to experimental conditions. 1. Reconsider the Adoption/Suicide Attempt example. Is this a designed experiment or an observational study? Explain. 2. Reconsider the Breakfast Cereal/Healthy Weight example. Is this a designed experiment or an observational study? Explain. 8

9 When Can We Establish Causation? When an observational study indicates that there is an association between the two variables under study, it is quite possible that lurking variables may be affecting the association. For example, consider the cereal study. Recall that those who ate cereal for breakfast tended to be at healthier weights. Does the cereal cause this effect? Not necessarily! It also seems reasonable to conclude that students with high physical activity levels will be hungry in the morning and will eat more cereal; moreover, students with a high physical activity level will be at a healthier weight. This set of relationships could be the real reason for the study s results! Now, suppose that the study had been conducted differently. Instead of simply observing the children s habits at breakfast, suppose that the researchers randomly assigned half of the children to eat breakfast cereal every day and the other half to not eat breakfast cereal. Now, what would we conclude if the study results were similar? That is, what if we still saw that those who tended to eat cereal for breakfast tended to be at healthier weights? Now, with a designed experiment, we wouldn t be able to argue that the results were attributable to physical activity level. The random assignment of subjects to experimental conditions would have taken care of this theoretically, half of the students with high physical activity levels would have been eating breakfast cereal, and the other half would not. Because of the random assignment, the groups of subjects receiving the different experimental conditions are theoretically balanced on all other variables. The only way the two groups differ is in whether or not they re eating cereal every day. So, even though lurking variables are still present, designed experiments account for lurking variables by balancing out their effects between the different groups under study. If a designed experiment indicated that kids who ate breakfast cereal every day tended to have healthier weights, we would have been much more certain that we had uncovered a causal relationship. The Big Idea: Observational studies allow us to conclude only that an association exists. The best method for determining causal relationships is to conduct a designed experiment! 9

10 Adjusting for Confounding Variables Consider the Adoption/Suicide example one last time. The following quote was taken from the Fox News article (foxnews.com). When previous self-harm behavior was taken into account, researchers calculated that adopted teens were 3.7 times more likely to attempt suicide than the other teens When the researchers adjusted for other factors often linked with suicidal thinking or behavior, including drug use, depression, academic struggles and personality traits like alienation and impulsivity, the increased risk for adopted kids remained Other mediating factors, not considered in our study, may include: heritable risk, prenatal factors, factors unique to relinquishment by a biological parent, early trauma, weak attachment to adoptive families and loss of cultural identity and ethnic discrimination," Keyes told Reuters Health by If the researchers really wanted to establish a causal connection between being adopted and attempting suicide, one might argue that they should design an experiment. Is this even possible for a study such as this? Why or why not? 2. What confounding variables were identified by the researchers? 3. What lurking variables were identified? Are there any others you can think of that aren t identified? 4. What do the researchers mean when they say they adjusted for other factors? 5. If they did adjust for confounding factors in their analysis and still saw an association between adoption and suicide, why isn t it fair to conclude that it is the adoption that is causing increased suicide risk? 10

11 Other Examples Consider the following studies (read the articles, if links are provided) and answer the related questions. Walking Cuts Breast Cancer Risk 1. Is this an observational study or a designed experiment? Explain. 2. Obesity has also been linked to breast cancer, and those who tend to walk less will likely be more overweight. Thus, this could explain the association between walking and the incidence of breast cancer. Though it isn t mentioned in this article, the researchers did actually control for this one potential confounding factor (obesity). Is this enough to make you believe that it really is the walking that causes a decrease in the risk of breast cancer? Why or why not? Happiness Wards off Heart Disease 1. Is this an observational study or a designed experiment? Explain. 2. In your opinion, is this headline appropriate? Why or why not? 3. What is your overall impression of the merit of this article? Explain. 11

12 Swapping Sugary Drinks out of Teens Homes Curbs Weight Gain 1. Is this an observational study or a designed experiment? Explain. 2. In your opinion, is this headline appropriate? Why or why not? 3. What is your overall impression of the merit of this article? Explain. Does Smoking Cause Lung Cancer? 1. Are the studies that investigate this question observational studies or designed experiments? Explain. 2. Given your answer to the previous question, one could argue that we can t definitively establish causation. Why is it, then, that we as a society overwhelmingly believe that smoking does in fact cause lung cancer? 12