Reconciling Compensatory and Noncompensatory Strategies of Cue Weighting: A Causal Model Approach

Transcription

1 Journal of Behavioral Decision Making, J. Behav. Dec. Making (2016) Published online in Wiley Online Library (wileyonlinelibrary.com).1978 Reconciling Compensatory and Noncompensatory Strategies of Cue Weighting: A Causal Model Approach ABIGAIL B. SUSSMAN 1 *, DANIEL M. OPPENHEIMER 2 AND MATTHEW M. LAMONACA 3 1 University of Chicago, Chicago, IL USA 2 UCLA, Los Angeles, CA USA 3 Princeton University, Princeton, NJ USA ABSTRACT When forming a judgment about any unknown item, people must draw inferences from information that is already known. This paper examines causal relationships between cues as a relevant factor influencing how people determine the amount of weight to place on each piece of available evidence. We propose that people draw from their beliefs about specific causal relationships between cues when determining how much weight to place on those cues, and that understanding this process can help reconcile differences between predictions of compensatory and lexicographic heuristic strategies. As causal relationships change, different cues become more or less important. Across three experiments, we find support for the use of causal models in determining cue weights, but leave open the possibility that they work in concert with other strategies as well. We conclude by discussing relative strengths and weaknesses of the causal model approach relative to existing models, and suggest areas for future research. Copyright 2016 John Wiley & Sons, Ltd. Additional supporting information may be found in the online version of this article at the publisher s web-site. key words judgment; compensatory heuristics; noncompensatory heuristics; causal reasoning When making judgments about the world, we typically have access to various pieces of information, or cues, which might facilitate those judgments. Doctors might consider the age of the patient and the severity of symptoms to determine how much medication to prescribe. Jurors might consider the number of people injured by a product and the extent of those injuries when determining damages. Bankers might look at a person s income and credit history when deciding the size and interest rate of a housing loan. One question that has been central in the study of judgment is how these cues are used and combined. The manner in which people combine cues impacts judgments, attitudes, and choices, and has important ramifications for human welfare. Within the field of judgment and decision making, it is often assumed that people do not fully integrate all information provided to them in complex environments and instead rely on heuristics when making evaluations. In this context, many heuristic strategies have been proposed to understand how people combine available information to form judgments. Two of the most prominent classes of such strategies are compensatory and noncompensatory (see Shah & Oppenheimer, 2008 for a review). Some researchers have posited that people rely on compensatory heuristics, in which they consider multiple cues and make evaluations by integrating information and making tradeoffs across these cues. Evaluations derived from compensatory heuristics approximate those made by linear models in many situations. However, most compensatory heuristics posit that people streamline the evaluation process *Correspondence to: Abigail Sussman, University of Chicago Booth School of Business, 5807 S. Woodlawn Ave, Chicago, IL 60637, USA. abigail.sussman@chicagobooth.edu by simplifying their approach to cue integration. For example, people could place equal weight on each cue (e.g. the equal weighting or tallying heuristic; Dawes, 1979; Payne, Bettman, & Johnson, 1993) rather than weighting each cue in direct proportion to its relevance. Other researchers have argued that people use noncompensatory (i.e. lexicographic) decision rules in which they rely only on the single most informative cue that can differentiate between options (e.g. Fishburn, 1974; Gigerenzer, Todd, & the ABC research group, 1999). For example, many of the fast and frugal heuristics proposed by the ABC research group (e.g. Take the best, Take the last, The minimalist, etc.; Gigerenzer et al., 1999) posit a stopping rule in which people stop seeking information (from memory or the environment) as soon as the first cue that discriminates among options is found. Reconciling compensatory and noncompensatory strategies Each type of heuristic can accurately predict human judgment under certain circumstances (Bergert & Nosofsky, 2007; Brehmer & Brehmer, 1988; Hammond, 1955; Hoffman, 1960; Payne, Bettman, & Johnson, 1988), but both approaches also have systematic gaps in their ability to make predictions under various (and predictable) environmental situations. Because compensatory heuristics excel at explaining judgment in some environments, and noncompensatory accounts are more effective in other environments, many researchers have argued that people switch between the different strategies depending on the particulars of the judgment task (e.g. Gigerenzer et al., 1999; Hogarth & Karelaia, 2005, 2007; Newell & Shanks, Copyright 2016 John Wiley & Sons, Ltd.

2 Journal of Behavioral Decision Making 2003; Payne et al., 1988, 1993; Swait & Adamowicz, 2001). In fact, strategy switching has largely emerged as the dominant approach to understanding cue weighting. This approach posits that people have a number of cognitive strategies in their repertoire and switch between them depending on the nature of the task environment. This widely accepted method has led researchers to devote a great deal of attention to determining which environments lead people to use one strategy over the other. General patterns have emerged from this analysis showing that people s judgments align more with the predictions of noncompensatory rules when they are under time pressure, the task is complex, the cost of information is high, or they have low cognitive ability, among other factors (Bröder, 2000, 2003; Bröder & Schiffer, 2006; Johnson & Payne, 1985; Payne, 1976; Payne et al., 1988). The relative validity of available cues and which approach is more likely to yield a correct answer influence cognitive strategies as well. For example, noncompensatory rules better predict judgments in environments where a single cue is much more predictive than any other cue or cues are mostly redundant, whereas compensatory rules better predict judgments in environments that have multiple cues predicting unique variance (Rieskamp & Otto, 2006). Although strategy switching accounts have become dominant in the field, this paper explores whether an approach based upon causal reasoning may be able to subsume both types of heuristic rules to account for judgments observed across various environments, in a single, unified framework. Even after accounting for the possibility of strategy switching, the compensatory versus lexicographic debate takes causal independence of cues as an underlying assumption. However, this assumption is too strong. We propose that we can capture patterns of strategy switching accounts without evoking heuristics by relaxing this assumption and incorporating this causal element. This is not a paper about causal reasoning, but rather looks to literature in causal reasoning to gain a better understanding of the cue weighting process. A causal model approach presents a single strategy that mimics multi-cue combinations under some conditions, and mimics single-cue use under others (and predicts unique patterns of cue usage in still others). While such a model could be more parsimonious than strategy switching, and thus could be desirable from the perspective of Occam s razor, few such models have been proposed in the literature. Exceptions include memory-based models (e.g. Dougherty, Gettys, & Ogden, 1999), exemplar models (e.g. Juslin & Persson, 2002), neural network models (e.g. Gluck & Bower, 1988), and Bayesian and sequential sampling models (e.g. Busemeyer & Townsend, 1993; Lee & Cummins, 2004). However, none consider causal relationships between cues. For example, decision field theory (Busemeyer & Townsend, 1993) posits that cues are sampled independently and stochastically, but does not specify the mechanism by which sampling occurs, and leaves no space for a causal framework. Thus, even for other unified accounts, causal reasoning has not played a central role, but could be important for setting parameters. It is worth noting that either of these rules (lexicographic or compensatory) can lead to normative cue weights under certain conditions, depending on the causal relationships between those cues. For example, consistent with a lexicographic strategy, a person could normatively place zero weight on a cue that is causally independent of another cue when information about a direct causal relationship is known. The current research proposes that a model based upon causal reasoning may be able to subsume both compensatory and non-compensatory rules to account for judgments across various environments and provide a novel way of conceptualizing cue weighting in judgment. A causal approach to modeling cue weighting Causal models are applicable across a wide array of judgment domains and, together with research on causal learning, can specify which cues are relevant and when (for reviews see Glymour, 2001; Gopnik & Schulz, 2007; Shanks, Holyoak, & Medin, 1996; Sloman, 2005). People are known to engage in spontaneous causal reasoning, particularly when confronted with negative or surprising outcomes (Weiner, 1985). Researchers have also shown that causal models play a role in how people make decisions (Hagmayer & Sloman, 2009; Sloman & Hagmayer, 2006). Further, causal narratives have been shown to influence jury decision making (Pennington & Hastie, 1993), and causal assumptions have been shown to affect risk assessment (Morgan, Fischhoff, Bostrom, & Atman, 2002). There is even some evidence that causal reasoning can influence how people consider cues when making judgments. For example, research examining which cues people focus on in judgments demonstrates that people prefer to rely on cues that are causally related (Garcia-Retamero & Hoffrage, 2006; Garcia-Retamero, Wallin, & Dieckmann, 2007). Further, Oppenheimer (2004) and Oppenheimer and Monin (2009) examined the classic availability heuristic (Tversky & Kahneman, 1973) in which people substitute the accessible cue of how easy it is to bring exemplars to mind for more difficult frequency judgments. Oppenheimer showed that people spontaneously considered the reason that the information was easy to bring to mind. When information was personally relevant, or was easy to retrieve because of fame rather than frequency, the availability heuristic ceased to operate. While there is ample evidence that people use causal information, there has been little work to investigate whether causal models might be able to explain cue weighting, particularly in relation to the use of compensatory versus non-compensatory rules. Furthermore, prior research in causal reasoning has generally been limited to predicting binary outcomes, while the current approach aims to explain continuous judgments. For example, while Garcia-Retamero et al. (2007) have shown the importance of the existence of causal associations in binary judgment, this research does not specify how the nature of the causal relationship might influence how the cues are integrated. That is, the previous work looks at whether or not cues are causally related, but does not investigate whether different types of causal structures lead to different patterns of judgment. However,

3 A. B. Sussman et al. it is possible that apparent switches in strategy arise because of differences in the causal relationships of the cues. This could account for the findings supporting both compensatory and non-compensatory rules without resorting to the notion of strategy switching (and also make valid novel predictions). Causal model-based cue weighting The present approach posits that when exposed to a set of cues, people spontaneously attempt to understand the causal relationships that exist between those cues and the criterion of interest (c.f. Oppenheimer, 2004). Importantly, different causal relationships will lead different cues to become more or less relevant. Existing literature in causal reasoning has closely examined how people use causal models when making inferences across a variety of tasks, including those related to decision making and categorization (e.g. Hagmayer & Sloman, 2009; Morais, Olsson, & Schooler, 2011; Rehder & Burnett, 2005; Rottman & Hastie, 2014; Sloman & Hagmayer, 2006; Sloman & Lagnado, 2005; Sloman, Love, & Ahn, 1998; Waldmann & Hagmayer, 2005). These investigations examine discrete outcomes, but provide a foundation for helping us understand how similar inferences could be used in determining continuous cue weights. The causal model approach posits that people considering how to weight cues first generate a single causal model that links the cues and target of judgment. They then use these causal models to guide the cue weighting process. Thus, the most basic prediction of a causal model approach is that different patterns of causal relationships between cues should yield different patterns of cue weighting. Neither compensatory rules, nor lexicographic rules, nor sequential sampling models (e.g. Busemeyer & Townsend, 1993; Lee & Cummins, 2004) predict differences in cue weighting based on how cues are causally related. Therefore, observing systematic variation in cue weights because of variations in causal relations would be an important, novel insight into the judgment literature. Observing such systematic variation based on causal relationships is an essential starting point; failure to observe this fundamental prediction would falsify a causal framework to cue weighting. Hence, our central hypothesis is that judgments will vary as a function of the causal relationship across cues. The primary contribution of this research is to provide evidence for this basic prediction. Notably, this research investigates how people form continuous judgments, rather than limiting findings to binary outcomes, as is the case in most prior literature on causal reasoning. The reason that we expect causal relations will play a role in cue weighting is that it is normative to do so. Of course, given limited cognitive capacity, research in bounded rationality has identified a number of deviations from normative judgment (see Kahneman, 2003 for a review), and so we would not expect perfect alignment of human behavior to the normative models. Nonetheless, a normative framework can serve as a starting point for identifying the types of causal relations that will likely yield cue weighting Causal Model Approach to Cue Weighting differences. As such, we outline a set of prescriptive guidelines of how subjects would optimally use causal information for cue weighting, as derived by normative Bayesian models, as described in the context of causal Bayes nets (see Charniak, 1991 for a detailed discussion of Bayesian normativity). Later, we will compare the behavior of participants to these normative models, to determine not just the extent to which they use causal relationships to determine cue weights, but also how close they fall to the normative standards. Consider the following causal structures, where A B represents a situation in which A causes B (i.e. increasing the value of A will directly lead to an increase in the value of B), and where C represents the target of judgment A B C 2. C B A 3. A B C In these cases, the normative framework for causal models would emulate those of a lexicographic heuristic. In all cases A is likely correlated with C, but not through a direct causal relationship. For the causal relationships presented in 1 3, B screens off A from C, so that once a person knows the value of B, there is no additional predictive information conveyed about C through A (Pearl, 2000; Sloman, 2005). In other words, for these three cases, A and C are independent, conditional on the value of B. Thus, if all assumptions were met, the normative response for causal structures such as those presented in 1 3 is that people who have access to B will use it to the exclusion of A when making judgments about C, just like lexicographic heuristics. Contrast those causal structures with the following three cases: 4. A C B 5. A C B 6. A C B A normative benchmark for the use of causal models in these relationships emulates the predictions of a compensatory rule. Both A and B are directly and independently causally related to C. As such, both A and B would be useful predictors of C in some combination. Just as compensatory rules would incorporate multiple cues, causal models would weight cues based on the strength and the structure of the perceived causal relationship (c.f. Griffiths & Tenenbaum, 2005). As such, a normative response relying on causal structures such as those presented in 4 6, involves a combination of A and B, just like in compensatory rules. In the absence of additional information, we make the simplifying assumption that people weight each cue approximately 1 This relationship is deliberately underspecified because of the additional assumptions required for more complete specification. For our purposes, we assume that for any relationship in which A causes B, increasing (decreasing) A increases (decreases) B unless B is at ceiling (floor). While there may be self-sustaining systems such that a decrease in A does not lead to a decrease in B, this is beyond the scope of the current paper.

4 Journal of Behavioral Decision Making equally when two cues are jointly informative about a third cue. 23 Importantly, while a causal model approach can, in a single unifying framework, explain both compensatory and non-compensatory heuristic patterns of judgment, it can also make unique predictions for certain causal structures. Consider: 7. A B C In feature set 7, normative reliance on a causal model would lead to a unique pattern of cue use. Just as in 1 3, there is no direct causal relationship between A and C. However unlike in 1 3, the utility of cue B in judging C is conditional on the value of A. That is, in 7, a high value of B could be explained by either a high value of A or C (or both). Thus, while A does not directly impact judgments of C, it changes the amount of weight placed on B. In the causal reasoning literature, this is known as explaining away or discounting (for a review, see Khemlani & Oppenheimer, 2011). And in fact, this pattern of results has been found in several studies of judgment (e.g. Oppenheimer, 2004, 2005; Oppenheimer & Frank, 2007; Oppenheimer & Monin, 2009). Importantly, neither compensatory nor noncompensatory heuristic rules directly predict this pattern of results. Although these strategies cover a wide range of possibilities that could possibly account for many of these observations, neither makes a straightforward prediction of this kind. Noncompensatory rules most commonly predict that reasoners would rely on a single cue (B) for their judgment because it is the most highly correlated with C. Compensatory rules would factor in both A and B, but because A is uncorrelated with C, the weight would likely be near zero and the predictions would align with those of heuristic rules in most cases. For ease of discussion, the seven causal models discussed above will be grouped by the type of heuristic that would normatively make the same predictions as the causal model approach for the stated causal relationship. Thus, the following naming conventions will be used: models 1 3 will be referred to as Lexicographic 1 (A B C), Lexicographic 2 (C B A), and Lexicographic 3 (A B C); 2 This framework is built around continuous values, (rather than binary estimates) for which an assumption of averaging seems plausible. In the case of structure 4 above (as well as structure 7), this prediction relies on an assumption of an averaging function where C is a combination of A and B cues rather than a function where either A or B is sufficient to cause a particular level for C. For example, in structures above, if someone were trying to determine whether C were present or not, knowledge that either A or B were present could be sufficient, even if the other were absent. In that case, people might factor in only one cue deemed to be the most informative. However, in making a continuous estimate, normative models would factor in both relevant cues and, absent additional discriminating information, use a strategy of equal weighting. See Waldmann, 2007, for a discussion of contextdependent approaches to cue integration. 3 While causal structures 4 and 5 are clearly symmetric, the prediction of equal weighting for structure 6 requires an assumption that predictive and diagnostic reasoning are equivalent an assumption that, while normative, has been shown empirically not to describe human behavior (e.g. Fernbach, Darlow, & Sloman, 2011; Tversky & Kahneman, 1982; Waldmann & Holyoak, 1992). However, in the absence of a principled way to set values for predictive vs. diagnostic cues, any alternative assumptions would be arbitrary. models 4 6 will be referred to as Compensatory 1 (A C B), Compensatory 2 (A C B), and Compensatory 3 (A C B); and model 7 will be referred to as Discounting (A B C). We recognize that there may be differences within each of these larger groups. For example, descriptive research has shown that people often make stronger inferences in predictive versus diagnostic reasoning (i.e. reasoning from cause to effect rather than effect to cause; e.g. Fernbach et al., 2011; Tversky & Kahneman, 1982; Waldmann & Holyoak, 1992). This might lead to differences in the strength of weight placed on cue B across models 1 and 2, or on cue A across models 5 and 6. Moreover, people sometimes violate the principle of screening off (most relevant to causal models 1, 2, and 3 above; e.g. Rehder & Burnett, 2005). And, in cases where people s judgments are directionally consistent with those of compensatory rules, they are often insufficiently sensitive to the specific parameters of the causal environment (see Rottman & Hastie, 2014 for a review). The remainder of this paper will examine whether people s judgments vary as a function of the causal relationships between cues. As causal relationships vary, we will compare the accuracy of predictions of causal models to those made using representative compensatory or lexicographic rules. To foreshadow results, we find support for our primary hypothesis that causal models affect judgments. However, we find significant deviations from the normative predictions of the Bayesian approach. In Experiment 1, we show that causal models can predict participant judgments when we explicitly manipulate causal relationships among sets of novel cues in a carefully controlled experimental setting. In Experiment 2, we use process tracing methods to demonstrate that participant search patterns match those that would be predicted by a causal model approach. Finally, in Experiment 3, we provide evidence that participants spontaneously rely on prior knowledge of causal relationships to determine cue weights even in more naturalistic decision environments with meaningful content. Overall, data supports the use of causal models in determining cue weights. However, it suggests that their use does not conform to Bayesian ideals, and leaves open the possibility that they work in concert with other methods. Primary contributions This is a paper about cue weighting. Our goal is to highlight causal reasoning as an important determinant of cue weighting that has been neglected in prior literature and to set an agenda for future examination of this factor. This approach examines how specific causal relationships alter judgments, rather than determining that the presence of causal relationships alone can influence patterns. In other words, we examine whether people will respond differently to cues that are causally related to an outcome, but connected in different ways. Our predictions incorporate extended elements of a causal system. Consequently, if two parts are causally relevant to each other, knowledge of one can alter a judgment about the other, even if there is not a direct causal relationship.

5 A. B. Sussman et al. Given the number of strong assumptions required to make predictions based on causal models (e.g. how might weighting differ when reasoning up a causal chain vs. down a causal chain), the current research does not endorse the particulars of each causal model but instead presents an illustrative placeholder (based on a normative Bayesian framework) for how a causal model approach could influence cue weighting. Additionally, while prior research on incorporating causal reasoning into judgment has focused on binary outcomes, the current approach examines a more complex situation that includes continuous predictors and outcome variables. Because existing methods incorporating causal reasoning rely on binary cues and outcomes, we make another valuable contribution to this literature by introducing a method for analyzing continuous data. EXPERIMENT 1 The primary prediction of the causal model approach is that qualitatively different cue weighting patterns will arise depending on causal relationships between the cues. In particular, cue weighting will mimic the predictions of lexicographic approaches for certain causal structures, will mimic the predictions of compensatory rules for other causal structures, and will show patterns that differ from either model for still other causal structures. The first experiment aimed to determine whether this pattern could be observed using carefully controlled experimental stimuli. We examined how participants weighted cues when provided with explicit information about the causal relationships between cues. Importantly, these causal relationships were the only elements that changed across conditions. Pretests We pretested three similar experimental manipulations, using online samples from Amazon.com s Mechanical Turk platform, to determine how to convey the intended causal relationships most effectively. These variations were primarily run to improve the rate at which the participants passed the manipulation check (indicated understanding of the intended causal model). In these pretests, the participant agreement with the intended causal model was unacceptably low (37%, 44%, and 46%, respectively), and so we do not present them in detail here. Nonetheless, as the data does speak to the primary hypothesis and provide converging evidence for the causal model theory, we present the details of these studies and results in the Supporting Information. For the purposes of the main text, the important take-home message is that in all cases the qualitative results are very similar to the primary study that we are reporting. Method Participants Three hundred ninety participants were recruited through an online platform hosted by Amazon.com and completed the Causal Model Approach to Cue Weighting experiment for monetary compensation. 4 The population was 44% female, with a mean age of 34. Design and procedure Participants were randomly assigned to one of seven conditions, in a between-subjects design, corresponding to each of the seven causal models described in the introduction. The experiment used a Star Trek theme to keep participants engaged in the task. Participants were told that their crew was on a mission to rescue human hostages from the Romulans on a faraway planet. Their ship had been facing a string of mechanical problems and participants needed to incorporate known information about mechanical parts to fix it. Pages with encouragement from Star Trek characters were interspersed throughout the experiment to keep participants engaged. Participants were instructed: In the questions that follow, you will be presented with background information about a series of gears on the Enterprise-K and told how they relate to each other. You will then learn the speed of some gears and be asked to predict the value of others. This speed of gears is measured on a 1 10 scale where 1 is the absolute slowest possible, 5 is average speed, and 10 is the absolute fastest possible. Your responses should also be on this 1 10 scale. Although the values of the gears will change from question to question, the relationships between the gears will stay the same. Drawings will be provided on the following pages to help you understand the relationship. Participants were then introduced to a novel (blank predicate) domain involving a set of gears and told how those gears relate to each other; the causal relationship between the parts varied by condition. For example, in the conditions involving a causal chain relationship (Lexicographic 1, Lexicographic 2, and Compensatory 3), participants were told: When the BLUE GEAR spins faster, it causes the ORANGE GEAR to spin faster. And, when the ORANGE GEAR spins faster, it causes the BLACK GEAR to spin faster. In the conditions involving a common cause relationship (Lexicographic 3 and Compensatory 2), the relationship described was: When the ORANGE GEAR spins faster, it causes the BLACK GEAR to spin faster. And, when the ORANGE GEAR spins faster, it causes the BLUE GEAR to spin faster. And, in conditions involving a common effect relationship (Compensatory 1 and Discounting), the relationship described was instead: 4 This data pooled 74 participants from a pretest designed to ensure that participants grasped the concepts presented in the task with 316 from the central study, using an identical procedure. Details about this and related pretests can be found in the Supporting Information.

6 Journal of Behavioral Decision Making When the BLACK GEAR spins faster, it causes the ORANGE GEAR to spin faster. And, when the BLUE GEAR spins faster, it causes the ORANGE GEAR to spin faster. Given our knowledge from pretests that participants often failed to report the causal model intended by the experimental design, the current experiment used a strong manipulation to induce the desired causal relationships. One method that has been particularly effective in facilitating causal learning is the use of interventions (e.g. Bramley, Lagnado, & Speekenbrink, 2015; Coenen, Rehder, & Gureckis, 2015; Hagmayer, Sloman, Lagnado, & Waldmann, 2007; Steyvers, Tenenbaum, Wagenmakers, & Blum, 2003). This methodology also helped us to convey more complex but qualitative information about the relationship between parts, such as the strength of the causal relationship. As such, after the introduction, participants saw a gear machine with separate vertical bars representing the speed of each of the three gears, to help explain the causal relationships among the devices, see Figure 1. Participants could set the speed of one gear at a time, and observe resulting movement in the other gears. All interventions were chosen by the participant. No questions were asked, no specific values for any cues were predetermined, and no feedback was provided during the training phase. A change in one gear s speed led to a change in the others as determined by the underlying causal structure, although the specific movements included a stochastic element. On initialization, the bars fluctuated randomly between 1 and 9 pixels high on a 150 pixel scale. Clicking Set let participants specify an exact level for the bar as indicated by vertical height. However, no numeric markers were present to quantify levels changing. A bar that was directly influenced by another bar deterministically moved to match the causal bar value within 8 pixels, after which it continuously fluctuated randomly in the specified range until the next intervention was made or the participant progressed to the next screen. This fluctuation implied that the causal relationships were sustaining (rather than discrete). A bar that was not influenced by the other bars would hover around zero. If a bar was influenced by two of the other bars, it moved to match the average of those two bars within 8 pixels, consistent with the averaging assumptions used to determine causal model predictions in the introduction. Because participants set only one gear at a time, this averaging manifested itself such that when two gears jointly influenced a third gear but were not causally connected (e.g. causal structure 4, A C B) and one cause was set to a high level, the other cause would remain unchanged hovering around zero while the effect would move to a value near the midpoint between zero and the level of the set bar. 5 Participants could spend as long as desired intervening on the system before proceeding to the following page, where 5 In the case of Discounting, note that C (the target of judgment) is not the average of the other bars. Instead, B was the average of A and C. So, if a participant moved A, it had no effect on C, but changed B to be the average of A and C. experimental trials began. In each trial, participants were given values for two of the gears on a 10 point scale where 1 was described as the absolute slowest possible, 5 was average, and 10 was the absolute fastest possible. Their task was to estimate the value for the third gear using the same scale. Participants completed 10 trials during which the cue values of the different gears were systematically varied (although the causal relationships between the gears stayed the same). The item values changed each trial, but remained constant across conditions. No feedback was provided. The values were selected so that each model would make a distinct prediction in most cases (see Table 1 for exact numbers used). After the 10 trials were complete, a manipulation check used a series of multiple choice questions to determine whether participants comprehended the stated causal model. For example, participants were asked: When the ORANGE GEAR spins faster, what does it DI- RECTLY cause? The BLUE GEAR to spin faster only The BLACK GEAR to spin faster only Both the BLUE GEAR and the BLACK GEAR to spin faster Neither the BLUE GEAR nor the BLACK GEAR to spin faster Parallel questions were asked for each of the three gears. Results and discussion Participants correctly responded to 82% of questions across the manipulation checks, and 85% of participants correctly responded to at least two of the three multiple choice questions. However, despite efforts at ensuring that participants understood the causal models intended, only 63% of participants (247 of 390) correctly answered all three multiple choice questions in the manipulation check. 67 Held to this standard, participants understood and were able to report that each of the intended causal links was present and that there were no additional direct links. To err on the side of conservatism, two analyses were done: one of all participant data and one only including participants who correctly responded to all parts of the manipulation check. We discuss the latter analysis (only subjects who passed the manipulation check) here. Although excluded participants do represent a source of noise, this should not take away from the primary analysis on the filtered participants. Results from all participants, as well as from participants whose causal models did not match those intended by the experiment are broken out separately in the Supporting Information. 6 Participants who failed the manipulation check can be thought of as adding noise to the data. Thus, including them does not affect the general trends of the data, although it does decrease the power. 7 Approximately 10% of participants incorrectly labeled indirect causes as direct causes (e.g. in the compensatory rule A B C, including A C), and approximately 10% (inclusive) incorrectly stated bidirectional relationships (e.g. where A B indicating A B and B A), with the remainder of errors stemming from assorted other responses.

7 A. B. Sussman et al. Causal Model Approach to Cue Weighting Figure 1. Still screenshot of training phase from Experiments 1 and 2. In the actual experiments, participants interacted with this interface to set the value of one gear and watch the value of the other gears move, relying on visual cues without explicit values. In this case, the participant had set the height of the orange gear and observed movement in the black gear, but no movement in the blue gear. The height of each bar hovered around a set point rather than remaining still Table 1. Stated values for cues A and B, used across all experiments, and predictions of the Causal Model hypothesis, used to generate cue weights Question Stated Cue A Stated Cue B Predicted Cue C (lexicographic conditions) Predicted Cue C (compensatory conditions) Predicted Cue C (discounting condition) Analytic approach We conducted three distinct sets of analyses to examine the predictive validity of the causal model approach. First, we used regression analysis to calculate the difference in weight that participants placed on cue A and cue B in determining cue C, and we compared this difference in weights to predictions of the causal model. Second, we used histograms to examine individual differences in the distribution of relative weights placed on cue A and cue B. Third, we examined a measure of the absolute deviation across participants stated C value and predictions of causal models, lexicographic rules, and compensatory rules. Each of these is described in detail below for each experiment, generally beginning with analysis at the level of grouped condition (individual conditions grouped by the predictions of the causal model approach) and then followed by analysis of individual conditions (each causal model examined independently). We primarily rely on pattern level analysis (e.g. vs. determining statistical differences) for comparisons across individual condition as a result of the large number of comparisons and consequently small sample size.

8 Journal of Behavioral Decision Making To examine the difference in weight that participants placed on cue A and cue B, standardized beta weights were first calculated for each participant using linear regression to determine the weight given to the values of cue A and cue B in participant estimates of cue C. This method of estimation of participants weighting policy is common in both the Brunswikian tradition of linear modeling (e.g. Cooksey, 1996) and in heuristic approaches to attribute substitution (e.g. Monin, 2003). However, to our knowledge, this methodology has not previously been used in the context of causal reasoning. We use a regression as a statistical tool to determine weights placed on each cue (see Cooksey, 1996 for a review of this methodology in the context of lens model analysis with inter-correlated cues). We do not intend to imply that this approach parallels or describes the underlying cognitive process. While lexicographic rules primarily predict that participants should weight one cue substantially more heavily than the other, and compensatory rules primarily predict that weights should be split more or less evenly between the cues, neither predicts that weights would change based on differences in causal structure. Because the judgment environment was held constant across conditions (e.g. no time pressure, identical cue values, etc.) there is no a priori reason to expect strategy switching. The causal model, however, predicts qualitative differences in the weights placed on cues depending on the causal structure that the participants were presented with. While the normative instantiation of the causal model approach makes specific predictions about weights for each causal model, our central prediction is that weights will vary across causal models, while allowing for human deviance from normativity. Table 2a displays the predictions of the causal model approach and the actual weights generated by participants, along with the average model fit. In each case, predicted weights were generated by simulating responses from a single participant for cue C on each of the 10 trials that were consistent with the causal model approach as outlined below. These values were then entered into a linear regression, relying on the same methodology used to generate weights for each participant. Predicted weights for the Lexicographic conditions (β A =0; β B = 1) were based on the logic of screening off. As A and C are independent conditional on the value of B, the judgment of C should be driven solely by the value of B. Predicted weights for the Compensatory conditions (β A =.63; β B =.55) are a result of both A and B having an equal and independent causal impact on the value of C. Thus, the prediction for C is the average of A and B values. 8 In the case of the Discounting condition, predictions may vary depending on the amount of causal discounting that takes place (as described in the theoretical background above). However, the most straightforward prediction (β A = 0.70; β B = 1.07), is based on the assumption that B takes on the average value 8 Predictions for standardized beta weights were generated using the cue values actually shown to participants, which were not perfectly balanced around the midpoint of the scale. Although unstandardized beta weights are β A =.5; β B =.5, stated predictions are for standardized weights which gives the appearance of deviating from equal cue weighting. of A and C. Table 2b compares the observed predictions of cue C for each model against the value of cue C stated by participants. Note that although we make specific, quantitative predictions for the purpose of testing an overall pattern of results, many of the assumptions used to generate those predictions are somewhat arbitrary, and it is the qualitative pattern of results that is most informative. For example, the quantitative predictions were generated under the assumption that the causal strength of each cue is consistent. This is a simplifying assumption that is likely inaccurate, but allows for more straightforward testing during the initial development of the causal model approach. We make similar simplifying assumptions for compensatory and lexicographic rules as well, as described in the introduction. Analysis by grouped condition To provide an overall evaluation of the causal model approach to cue weighting, we first grouped conditions based on the predictions of the approach i.e. the three Lexicographic conditions and the three Compensatory conditions were grouped together. Differences in patterns of cue weighting across groups were largely consistent with the causal model approach. To examine whether these differences were statistically reliable, we ran a one-way ANOVA on the difference between the beta weights that each participant placed on cue B and on cue A when making their judgments (β B β A, which we will refer to as β DIFF ). A one-way analysis of variance at the grouped level revealed a significant main effect of condition on β DIFF (F (2, 244) = 40.93, p <.001, η 2 =.25), see Table 2a. Consistent with the causal model approach, post-hoc contrasts indicated that the mean β DIFF for Lexicographic conditions (M=.55, SD =.90) was significantly greater than for Compensatory conditions (M =.05, SD =.31, t(244) = 5.27, p <.001, d =.74). Additionally, as predicted by the causal model approach, β DIFF for the Lexicographic conditions was significantly lower than for the Discounting condition (M = 1.13, SD =.89, t(244) = 8.66, p <.001, d =.65). Next, we turn to the comparison of observed predictions of cue C against the values predicted by each model or rule. Table 3a displays the average absolute deviation between participant responses and the C value predicted by each model on a trial-by-trial basis, which we will refer to as C DEV for simplicity. Consistent with predictions of the causal model approach, C DEV is lowest for the causal model predictions (1.39) relative to values predicted by the lexicographic (1.65; within-subject t(246) = 6.29, p <.001, d = 0.35) or compensatory (1.53; within-subject t(246) = 2.42, p =.016, d = 0.16) rules when examined in aggregate across participants. This level of analysis is particularly meaningful because it shows overall predictive power, and the ability of the causal model to make more accurate predictions irrespective of the individual condition. However, we continue examination at the level of grouped condition to gain a better understanding of how and when the causal model predictions are most accurate. Recall that the causal model makes the same predictions as

9 A. B. Sussman et al. Causal Model Approach to Cue Weighting Table 2. Actual and predicted cue weights in estimating missing cue values by predictions of the causal model approach in Experiment 1. Conditions are displayed grouped by predictions of the causal model approach (a) as well as listed individually (b) Causal model prediction Participant responses Model fit Condition (2a) β A β B β A β B R 2 Lexicographic conditions Compensatory conditions Discounting condition (2b) Lexicographic 1 (A B C) Lexicographic 2 (C B A) Lexicographic 3 (A B C) Compensatory 1 (A C B) Compensatory 2 (A C B) Compensatory 3 (A C B) Discounting (A B C) Note: As measured by a one-way, between subjects ANOVA testing differences in weights placed on the A cue and the B cue, condition 1 is significantly different from conditions 2, 3, 4, 5, and 6 (ps.001); Condition 2 is significantly different from conditions 3, 4, 5, and 6 (ps.001); Condition 3 is significantly different from conditions 6 and 7 (ps.05); Conditions 4, 5, and 6 are significantly different from condition 7 (ps <.001). Table 3. The absolute deviation between model predictions and participant responses for cue C (C DEV ), averaged across questions and across participants in Experiment 1. Average deviation is calculated at the level of the participant. Conditions are displayed grouped by predictions of the causal model approach (a) as well as listed individually (b) C DEV Condition (3a) Causal model Lexicographic Compensatory Lexicographic Compensatory Discounting Aggregate (3b) Lexicographic Lexicographic Lexicographic Compensatory Compensatory Compensatory Discounting Aggregate the lexicographic and compensatory rules in certain conditions (this is how they were initially grouped), and thus this deviation will be equal across equivalent models in those cases. Thus, we would anticipate the C DEV would be equal to the causal model for the rule making matching predictions (either compensatory or lexicographic) and lower for the causal model and the matched rule than for the other rule. As predicted, C DEV is the lowest for the causal model and the compensatory rules for the Compensatory conditions (0.97 vs for the lexicographic rules; within-subject t(120) = 9.89, p <.001, d = 0.97). While C DEV is directionally lower for the causal model and the lexicographic predictions for the Lexicographic conditions (1.63 vs for compensatory rules; within-subject t(87) = 1.18, ns), this difference is not meaningful in magnitude. In the case of the Discounting condition, where the causal model prediction differs from both the lexicographic and compensatory rules, C DEV is directionally (but not significantly) lower for the causal model prediction (2.14) than for the Lexicographic conditions (2.27; within-subject t(38) < 1), and significantly lower than for the Compensatory conditions (2.79; withinsubject t(38) = 0.03, p =.034, d = 0.36). Analysis by individual condition The causal model approach predicts that the Lexicographic conditions should yield distinct judgment patterns from the Compensatory conditions and the Discounting condition, and this is supported by the above analysis. We can also look at the predictions of the normative model to determine the extent to which people are not just influenced by causal relationships, but how closely they conform to Bayesian standards. To that end, we can measure whether Lexicographic conditions yield similar judgment patterns to one another (and similarly whether the Compensatory conditions yield similar patterns to one another). To test this possibility, we did a condition-by-condition analysis of the data, see Table 2b. Participants responses to the different Compensatory conditions were nearly identical to one another. However, the results for the Lexicographic conditions did not fit the normative patterns. Participants in the Lexicographic 1 condition (A B C) and the Lexicographic 2 condition (C B A) placed negative (rather than zero) weight on cue A. Moreover, the expected pattern disappears entirely in the Lexicographic 3 condition (A B C), where participants placed approximately equal weight on cues A and B. In other words, the Lexicographic 1 and 2 conditions look more like the Discounting condition while the Lexicographic 3 condition looks like the Compensatory conditions, which is counter to normative behavior.

10 Journal of Behavioral Decision Making In addition to looking at the average β DIFF across subjects, looking at individual differences in β DIFF can provide further insight into the nature of the pattern of results. For example, one alternative to the causal model approach would suggest that participants in each of the Compensatory conditions could have been placing the full weight of their judgment on cue A one-half of the time and on cue B one-half of the time. When averaged, these results could have appeared as equal cue weights. To ensure that this was not the case, we looked at the β DIFF distribution (see Figure 2). This analysis revealed that β DIFF was generally close to zero, rather than at one or negative one, and ruled out this alternative hypothesis. Similarly, an examination of individual responses in the Discounting condition reveals that the majority of participants were placing much higher weights on cue B than cue A. The data is not as uniform in examination of individual responses for each of the Lexicographic conditions, consistent with the β DIFF analysis above. Individual responses in the Lexicographic 1 and 2 conditions approximately align with predictions, although the negative value placed on cue A described above becomes apparent through the large number of participants (52% and 20% in Lexicographic 1 and 2, respectively) with β DIFF greater than However, we see near opposite patterns for the Lexicographic 3 condition (A B C). Rather than merely revealing equal weighting on cues A and B, as suggested by the β DIFF analysis, the histogram shows that more than half of participants placed greater weight on cue A than cue B, the exact opposite of our initial prediction. We propose that this pattern may stem from continued reliance on causal relationships, but a difference in interpretation. While we generated our assumptions from Bayesian principles of screening off, participants may have been noting the symmetric relationship between cues A and C in relation to B and generalizing from there. If B A and B C and participants only know the values of A and B, they may conclude that they can learn more about cue C from cue A than from cue B, because they would expect a parallel consequence of the identical causal relationship. In other words, participants may be concluding that whatever B is doing to A it must also be doing to C. Therefore, rather than looking back to B and reasoning through the consequences of the causality, which would require additional assumptions about the meaning of the causal relationship, participants may be noting the parallel causal relationship and generalizing from there. Examining the comparison of observed predictions of cue C against the values predicted by each model (C DEV ) at the level of the individual condition (Table 3b), we see similar patterns to those revealed by the histograms. In all cases except for Lexicographic 3, the causal model prediction minimizes the deviation from participant responses relative to the other models. Overall, results provided evidence for the predictions of the causal model approach, showing that causal relationships among cues altered the weight that people placed on each cue when forming judgments. However, all of the evidence was at the outcome level, through examination of judgments and corresponding cue weights. In Experiment 2, we aim to gain additional insight into whether the use of causal models underlies the observed patterns by investigating whether the process used to arrive at these judgments is consistent with the predictions of the causal model approach. EXPERIMENT 2 To this point we have examined the judgment outcome rather than the process that underlies it. However, advances in Figure 2. Distribution of differences in weights placed on each cue (β B β A ) for each condition in Experiment 1. Each bar represents the percent of total responses falling into the β DIFF category described on the X-axis