Causation and experiments. Phil 12: Logic and Decision Making Spring 2011 UC San Diego 5/24/2011

Transcription

1 Causation and experiments Phil 12: Logic and Decision Making Spring 2011 UC San Diego 5/24/2011

2 Review Diagramming causal relations Variables as nodes (boxes) Causal relations as arrows Not tracing the flow of activity, but causal relations If there are conditions under which changing one variable will result in change of another variable, include a arrow between the variables Sometimes there are important intermediate causes such that a more ultimate cause only produces its effect through a more proximate cause Match struck [yes, no] Tip temperature [>350, <350 ] Match lit [yes, no]

3 Review 2 Common cause A positive correlation between two variables may be the result of a common cause for both Pine needles [on tree, dropped] Fish [alive, dead] Toxic waste [no, yes] If the causation is direct, there should be no way to screen off E from C

4 Review 3 Causal Fallacies treating mere coincidences as causal post hoc, ergo propter hoc confusing cause and effect ignoring a common cause assuming a common cause

5 Confusing cause and effect Is a child difficult because the parents are shorttempered? Or are the parents short of temper because the child is difficult? Parents shorttempered [no, yes] Child difficult [no, yes]

6 Assuming/Ignoring common causes Assuming common causes: Based on a correlation between two variables, assume there must be a common cause of them both Ignoring common causes: Based on a correlation between two variables, assume a direct causal relationship, when there really is a common cause for both Test for common cause: must identify the cause and determine that varying it alone alters both effects If varying the purported cause fails to alter both effects, there is no common cause, but rather two correlated causes

7 Clicker question What causal fallacy is illustrated in this example: Immigration to Alberta from Ontario increased. Soon after, the welfare rolls increased. Therefore, the increased immigration caused the increased welfare rolls. A. Ignoring a common cause B. Assuming a common cause C. Post hoc, ergo propter hoc D. Confusing cause and effect

8 Clicker question What causal fallacy is illustrated in this example: Two students in a large class turn in identical term papers. The teacher accuses them of copying from one another, even though the students can prove that they did not know one another and had no access to one another s work. The teacher maintains nonetheless that the identical papers were caused by copying each from one another. A. Ignoring a common cause B. Assuming a common cause C. Post hoc, ergo propter hoc D. Confusing cause and effect

9 Clicker question What causal fallacy is illustrated in this example: Bob is visiting his cousin Fred on his farm. While standing in Fred s hay field, Bob gets watery eyes and starts sneezing. Bob claims that his watery eyes caused him to sneeze. A. Ignoring a common cause B. Assuming a common cause C. Post hoc, ergo propter hoc D. Confusing cause and effect

10 Need for Experiments or WellControlled Observations The best evidence as to whether something is a cause of some effect is whether manipulating it changes the value of the effect When that isn t possible, one must rely on controlled observations that rule out other possible causes (confounds) we ll cover these Thursday

11 The basic idea of an experiment If the independent variable is the cause of the dependent variable, then a manipulation of the independent variable should produce a change in the value of the dependent variable And if it were not the cause, we would not expect such a result from manipulation Manipulation Independent variable [values]? Dependent variable [values]

12 Experiments on regular deterministic systems When there is no variance in the population being studied, statistical analysis is not necessary The main danger is affirming the consequent The key is to test a causal hypothesis in which you would not expect the effect to occur unless you were right about the cause. Manipulation (vary the object dropped) Object dropped [no, yes]? Object falls [no, yes]

13 Experiments on regular deterministic systems When there is no variance in the population being studied, statistical analysis is not necessary The main danger is affirming the consequent The key is to test a causal hypothesis in which you would not expect the effect to occur unless you were right about the cause. Manipulation (vary the object dropped) Object dropped [mass]? Object falls [m/s 2 ]

14 Variability in nondeterministic systems Most biological, cognitive, social systems are nondeterministic Different systems of the same kind will vary in their responses depending on their particular composition (genetics, etc.) their particular history, etc. Your reaction time will differ depending on how much sleep you have had what you have had to drink, etc. The same system may respond differently on different occasions Challenge: how to detect causal relations in the face of background variability (noise)?

15 Experiments on nondeterministic systems Since complex systems (e.g., biological, cognitive, social) are not perfectly regular in their behavior: Researchers cannot simply do an experiment on one instance and draw a conclusion about the whole population Must rather work with samples and draw conclusions based on statistical analysis Are the differences in the values of the dependent variable greater than expected by chance?

16 Confounding Variables Different individuals behave differently because some extraneous variables may be related to the effect of interest Confounding variable: an extraneous variable that correlates with the IV, and so may be responsible for the effect produced in a study Subject variable confounds: Differences between subjects in the study Procedural variable confounds: Differences in the way different groups are treated An experiment is confounded if it has any confounding variables

17 Strategies for controlling confounding variables 1. Randomization 2. Locking 3. Matching 4. Making confounding variables into studied variables (additional factors)

18 Procedural variable confounds When you conduct a manipulation, generally more than one thing will be changed These variables will then be correlated with the independent variable So it rather than the IV may be the cause

19 Confounding Procedural Variables The president of the AGL corporation wanted to get his workers to be more productive She found that when each employee was presented with a jar of jellybeans, productivity increased Was it the jellybeans that caused the increased productivity? Or was it: Novelty of the situation Attention from the president Desire to reciprocate

20 Controlling confounding procedural variables Manipulation Correlation or causation X Procedural variable X Independent variable [values]? Dependent variable [values] Strategy: break the correlation thereby breaking the effect of the confounding variable

21 Clicker question What method(s) could you use to eliminate the effects of the confounding procedural variables in the jellybean case (e.g., novelty of situation, attention from president, desire to reciprocate)? A. Random assignment B. Lock the value of each variable C. Match subjects on the values of each variable D. Make the variable a factor in the experiment

22 Controlling confounding procedural variables Manipulation Correlation or causation X Procedural variable X Independent variable [values]? Dependent variable [values] Strategy: break the correlation thereby breaking the effect of the confounding variable commonly achieved via locking

23 Subject variable confounds Subjects in an experiment may be differently affected by different values of other variables People of different ages sleep different amounts Women might be affected differently than men If these aren t the DV or IV, these variables are extraneous If there is a correlation between these variables and the independent variable: they, rather than the variable you are focusing on, may be what produce the change in the dependent variable Such variables are confounds

24 Controlling confounding subject variables Manipulation Independent variable [values] X Correlation or causation Dependent variable [values] Strategy: break the correlation thereby breaking the effect of the confounding variable Random assignment of subjects is a main strategy for breaking the correlation? X Subject variable

25 Controlling subject confounds: Between subjects randomization

26 Manipulate independent variable Task Key Behavior IV Task Key Behavior Statistically Significant DV Difference?

27 Controlling subject confounds: Within subjects design

28 Subjects serving as own control Task Task Key Behavior Key Behavior Statistically Significant DV Difference?

29 Betweensubject design Good news: Participants are not contaminated in one condition as a result of having participated in the other Bad news: Requires a larger number of participants Runs the risk of nonequivalence of subject groups

30 Withinsubjects designs Good news: Requires a smaller number of participants Rules out any differences between subjects Bad news: Potential contamination of participants behavior from previous trial: carryover effect Subjects might learn from one condition and that could alter their behavior in the second condition Practice effect Fatigue effect

31 Counterbalancing 1. Withinsubject counterbalancing Reversing order: ABBA 2. Acrosssubject counterbalancing Complete: every possible sequence requires n! runs Partial Random Latin Square: each condition appears once and only once in a given ordinal position no two conditions are juxtaposed in the same order more than once Order 1: A B D C Order 2: B C A D Order 3: C D B A Order 4: D A C B

32 PretestPosttest Design There is always a danger in an experiment that the members of the two (or more) groups being studied already differ on the dependent variable Best control is to focus on change, not raw value of the DV Pretest: measure the DV before the intervention Posttest: measure the DV after the intervention Change = Posttest Pretest Manipulation Independent variable [values]? Dependent variable [change] [values]

33 Limitations of pretestposttest design Just measuring change in one group using a pretest and a posttest allows for confounds Time has elapsed and subjects have gotten older (maturation) Events occurring between the pretest and posttest could affect the DV (history) Experience with previous test may change performance Pretest and posttest may vary in difficulty Use of pretestposttest does not obviate the need for a control group