RUNNING HEAD: TWO MECHANISMS OF CONTINGENCY LEARNING

Transcription

1 TWO MECHANISMS OF CONTINGENCY LEARNING 1" RUNNING HEAD: TWO MECHANISMS OF CONTINGENCY LEARNING Two Mechanisms of Human Contingency Learning Daniel A. Sternberg and James L. McClelland Department of Psychology Stanford University Correspondence: Daniel A. Sternberg Jordan Hall, Building 420 Stanford University 450 Serra Mall Stanford, CA

2 TWO MECHANISMS OF CONTINGENCY LEARNING 2" Abstract How do humans learn contingencies between events? Several types of process models have been proposed, including pathway strengthening and inference-based models. We propose that each of these processes is used in different task conditions. Human participants viewed displays containing single or paired objects and learned which displays were usually followed by a dot. Some participants predicted whether the dot would appear and then saw the outcome, while others were required to respond quickly if the dot appeared shortly after the objects. For predict participants, instructions guiding participants to infer which objects had the power to cause the outcome determined whether contingencies associated with one object affected predictions about its pair mate. For respond participants, contingencies associated with one object affected responses to the mate, whether or not independent these instructions were provided. The results challenge single-mechanism accounts and support the proposal that the mechanisms underlying performance in the two tasks are distinct.

3 TWO MECHANISMS OF CONTINGENCY LEARNING 3" Two Mechanisms of Human Contingency Learning Understanding how people learn contingencies between events has been a focus of research for many years (Krechevsky, 1932; Pavlov, 1927; Tolman, 1948, 1949). In standard contingency learning tasks, participants view situations in which cues are followed by outcomes, and are later asked to predict outcomes for test cases. Two kinds of accounts have been offered to describe the process underlying performance in such tasks. One type of account is based on strengthening of pathways linking representations of cues to representations of outcomes or responses (Rescorla & Wagner, 1972; Pearce & Hall, 1980). The other is based on an explicit reasoning process that leads to inferences about the causal relations between the cues and outcomes in light of evidence (De Houwer, 2009; Mitchell, De Houwer & Lovibond, 2009). Pathway strengthening has been proposed as a mechanism for gradually learning contingent response tendencies. Stronger pathways promote fast, automatic responding (Cohen, Dunbar & McClelland, 1990), and pathway strengthening models can account for the gradual speeding of contingency-sensitive responding in fast-paced sequence learning tasks (Cleeremans & McClelland, 1991). On the other hand, considerable evidence now supports accounts of contingency learning that rely on a resource-intensive process of making explicit inferences in many situations, leading some to propose that a complete account of contingency learning is possible based only on explicit inference-based process (De Houwer, 2009, Mitchell, De Houwer & Lovibond, 2009). We suggest that both processes may be at work, depending on the task situation. In support of our view, we rely on a difference in the predictions the two accounts make about how instructions should influence what we call indirect effects in contingency learning. Indirect effects can arise when potential predictive cues can occur either alone or in

4 TWO MECHANISMS OF CONTINGENCY LEARNING 4" combination. In some training events, two cues (X 1 and X 2, perhaps a light and a tone) are presented together, and shortly afterward, an outcome (O, perhaps a shock) occurs. In other training events, X 1 occurs without X 2. If X 1 alone is followed by O, the learner will be less likely to anticipate O to the X 2 alone (this effect is called blocking, Kamin, 1968, 1969). However, if X 1 alone is not followed by O, the learner will anticipate O when X 2 occurs on its own (screening or reduced overshadowing, Carr, 1974). The learner s experience with X 2 is the same in both cases; the response to it is indirectly affected by what happened when the X 1 was presented without X 2. Pathway strengthening can explain indirect effects if changes in strengths of pathways linking cues to outcomes or responses are proportional to the error, or difference between the prediction from existing connection strengths and the observed outcome: If cue X 1 alone is followed by O, it will develop a strong connection to O. There will then be little error when X 1 and another cue X 2, are followed by O, and little strengthening between X 2 and O (blocking); conversely, if X 1 alone is not followed by O, the connection from X 1 to O will be kept weak, leading to error when O occurs after X 1 and X 2 together, and therefore a strong association between X 2 and O (screening). On this account, indirect effects emerge directly from the same process as learning the training contingencies, and their occurrence does not require any extra processing steps. The earliest models incorporating indirect effects, and in particular blocking, relied on this kind of mechanism (Rescorla & Wagner, 1972). An inference-based process can also produce indirect effects if the learner adopts a causal scenario in which X 1 and X 2 are treated as potential independent causes of the outcome. For example, with foods and allergies, individual foods may be treated as potential independent causes of an allergic reaction. If so, and no reaction occurs when X 1 alone is eaten, but does

5 TWO MECHANISMS OF CONTINGENCY LEARNING 5" occur when both X 1 and X 2 are eaten, one can infer that X 2 must be the cause of the allergy O. However, if X 1 alone does produce an allergy, a reaction when X 1 and X 2 occur together provides little evidence about X 2. Thus, participants who are relying on an inference process and who adopt the independent cause assumption should predict the allergy more strongly in the screening situation than in the blocking situation. These alternative accounts of indirect effects make contrasting predictions. Within the inference-based approach, the assumption that causes act independently is considered necessary to license the inference that gives rise to the indirect effect (Lovibond, et al., 2003). If independence is not assumed, both the blocking and the screening cases are ambiguous. In the screening case, for example, one could instead suppose that X 1 and X 2 must occur together for O to occur, leaving no clear implication that O will occur if X 2 occurs on its own. Evidence from contingency learning tasks supports the idea that this assumption is important: giving participants a cover story that is consistent with the independent cause assumption has been shown to increase indirect effects (Lovibond et al., 2003; Williams, Sagness & McPhee, 1994; Waldmann & Holyoak, 1992, 1997). On an inference-based account, the effect also depends on the learner having the necessary time or resources to make an inference from the evidence provided by the training events to the novel X 2 test event, a claim that is supported by the finding that introducing a secondary task reduces indirect effects (De Houwer & Beckers, 2003). Thus, an inferencebased account predicts that instructions encouraging a search for independent causes will promote indirect effects compared with neutral instructions. Limitations on available resources (including time to retrieve relevant information from memory about other inference-relevant training events) might further restrict the circumstances in which indirect effects are observed. On the other hand, pathway strengthening approaches based on error-correcting learning

6 TWO MECHANISMS OF CONTINGENCY LEARNING 6" do not predict that instructions promoting the independent cause assumption will be necessary to produce indirect effects, because pathway strengthening is thought to operate in neural circuits that are not penetrated by verbal instructions. Under the assumption that this kind of process underlies indirect effects, we would thus expect them to occur whether or not participants are given instructions that promote attribution of causal powers to individual cues. We suggest that both inference and pathway strengthening may underlie contingency learning, but under different circumstances. Simply put, when participants are given time and a task context that encourages them to make the relevant inferences, they will do so, so that performance should conform to the predictions of inference-based accounts, showing sensitivity to instructions promoting a search for independent causes. But when participants are placed in a situation where they must respond quickly, and there is little time for reflection and inference, sensitivity to contingencies will occur via pathway strengthening, which should give rise to indirect effects whether or not instructions promoting independent causes are provided. In order to address these issues, we designed two tasks that exposed participants to the same cue-outcome contingencies, differing only in terms of the time-structure of the trials and the timing within the trial of responses. Participants in our untimed prediction task were given unlimited time to make predictions about whether cues would be followed by a particular outcome, then (during training trials) they were given outcome information confirming or disconfirming their prediction. Half of these participants were given causal framing instructions that promoted the idea that they should endeavor to learn which objects had causal powers, while other participants were given more neutral instructions (called object-framing instructions) that did not guide them to make such inferences. In our fast-paced reaction time (RT) task, given to another two groups of participants,

7 TWO MECHANISMS OF CONTINGENCY LEARNING 7" outcomes were programmed to occur according to the same cue-outcome contingencies as in the untimed prediction task. However, the outcomes automatically occurred shortly after the objects appeared, and participants were instructed to respond very quickly if the outcome occurred, and to refrain from responding when it did not. The predictions of the inference-based and pathway-strengthening accounts for the four conditions of the experiment are spelled out in the first two panels of Table 1. Our two mechanism hypothesis predicts a third pattern (shown in the third panel), such that the predictions of the inference-based account will be upheld in the untimed prediction task, while the predictions of the pathway-strengthening account will be upheld in the fast-paced reaction time task. Method Participants 48 members of the Stanford psychology paid participants pool participated in the prediction task for payment, and 96 members of the same pool participated in the fast-paced RT task. Participants were paid in part based on their performance in the task, earning $0.02 for each correct response and losing $0.01 for each incorrect response (see the supplementary information available on-line for further details regarding payment, materials and methods). One participant in the RT task was removed for failing to respond on any trial to one of the tested items, leaving 95 participants: 47 in the causal framing condition and 48 in the object framing condition. Materials The same stimuli and events were used in both tasks. The cues consisted of clip-art objects taken from the Open Clipart Database ( which were randomly shuffled for each participant. Eleven distinct object displays were used during

8 TWO MECHANISMS OF CONTINGENCY LEARNING 8" training; each display consisted of one object or a pair of objects. For the outcomes, a dot either appeared after the objects on one side of the screen (randomly determined for each trial), or it did not appear. The materials included two critical sets of displays, the blocking set and the screening set (Table 2 describes all displays used). For both the blocking and screening sets, there was a compound event (B 1 B 2 and S 1 S 2 ) that was followed by the dot on 90% of trials. One of the two members of each set also occurred on its own. The blocking training singleton (B 1 ) was followed by the dot on approximately 90% of trials, while the screening training singleton (S 1 ) was only followed by the dot on approximately 10% of trials. If the participants learned the training contingencies, they should more strongly expect the dot to appear following B 1 than S 1 at test. If they show an indirect effect, they should more strongly expect the dot to appear following S 2 than B 2. Each participant was given one of two sets of framing instructions (shown in Table 3) in addition to a set of task-specific instructions (prediction or RT). The causal framing instructions were designed to promote the assumption that individual objects had the power to make the dot appear and to encourage participants to infer which objects had that power. The object framing instructions were meant to ensure that participants knew that the computer followed rules based on the object displays, and to encourage attention to the objects, without further describing the nature of the rules and without encouraging participants to make inferences. Procedure Figure 1 describes the layout and time-course of trials in both tasks. Objects appeared inside a bounding rectangle. On each trial, either two objects appeared (pair trial), or a single object appeared (singleton trial). On pair trials, one object was randomly chosen to appear in the center of the upper half of the box, while the other appeared in the center of the lower half. On

9 TWO MECHANISMS OF CONTINGENCY LEARNING 9" singleton trials, a single object appeared randomly in one of these two locations. In the prediction task, participants had as much time as they wished to respond following the presentation of the object display, and responded by pressing a yes or no button corresponding to their prediction of whether the dot would appear. Once a participant made a response, the outcome was presented and was followed by visual and auditory feedback. During the training phase, each event occurred 24 times, and the most likely outcome for each event (dot or no dot) occurred on 22 of these presentations. At test, each item occurred 5 times, and outcomes were not shown. Instead, after participants responded, the word Recorded appeared in the center of the box. In the fast-paced RT task, after the object(s) appeared, the outcome was automatically shown and was followed by the same feedback information as in the prediction task. Participants were required to respond to the dot outcome within a variable response window that began at the onset of dot; responses were treated as too late if they occurred after the end of the response window. The outcome and response period was followed by the same visual and auditory feedback used in the prediction task. Each training event occurred 72 times during the training block, and the most likely outcome for the event occurred on 65 of these presentations. During testing, the same set of test events was used as in the prediction task. All test events occurred 16 times during the test block. In order to obtain sufficient response time data for the relevant items, all single item events were followed by the dot on 8 of the 16 test trials. Pair events retained their probabilities from training, receiving the dot on either 14 or 2 of the 16 trials. Results Responses during the test phase of the experiment indicate that participants in all four conditions were sensitive to the contingencies on which they had been trained (Table 2).

10 TWO MECHANISMS OF CONTINGENCY LEARNING 10 Participants in both the causal and object framing conditions of the prediction task were more likely to predict the dot for the B 1 item than the S 1 item at test (Wilcoxon signed rank tests causal: z=4.61, p<.0001, object: z=4.34, p<.0001; Figure 2, upper left), and participants in both conditions of the RT task responded faster on dot trials for the B 1 item than the S 1 item at test (causal: t(46)=10.9, p<.0001, object: t(47)=10.14, p<.0001; Figure 2, lower left). Learning curves for both tasks are available in the supporting information available online. In the untimed prediction task, the critical indirect effect appeared in the causal framing condition, but not in the object framing condition. If these participants showed indirect effects in the expected direction, they should have been more likely to predict the dot for the S 2 item than the B 2 item. Prediction participants given the causal framing showed this indirect effect, z=2.47, p<.05, while those given the object framing did not z=-0.32, p=.78 (Figure 2, upper right). This difference between the two groups was significant (Wilcoxon-Mann-Whitney test comparing the difference scores, z=2.10, p<.05). In contrast, in the RT task, both groups showed the expected indirect effect: participants responded faster to the S 2 item than the B 2 item (causal: t(46)=2.60, p<.05, object: t(47)=3.13, p<.01), and the size of the effect did not differ by group, t(93)=0.54, p=.59 (Figure 2, lower right). Mixed effects regression analyses in the supporting information available on-line provide further confirmation of these findings. The lack of an indirect effect in the prediction task under the object framing instructions led us to look further at the pattern of responding to all test items in this condition. In order to determine whether predictions for these items were sensitive to the contingencies for the pair events in which they had occurred during training, we compared the average of each participant s predictions for the C 1, C 2, B 2, and S 2, items, which had occurred in dot-likely pairs

11 TWO MECHANISMS OF CONTINGENCY LEARNING 11 during training, to their predictions for the N 2 item, which had occurred in a dot-unlikely pair. Indeed, object framing participants predicted the dot more often for the dot-likely items than for the N 2 item according to a Friedman test, χ 2 (1)=9.78, p <.01. However, there were no differences among the dot-likely items, χ 2 (2)=0.222,,p=.89, consistent with the view that participants responses were not affected by the contingencies of their pair-mates. These findings indicate that participant in the prediction task who were given object framing instructions learned and relied on the direct co-occurrence relations between the test objects and the dot outcome, based on the training events in which these objects occurred in pairs, but did not take the further step of making an inference based on outcomes associated with training events in which their pair-mates occurred alone. Discussion We observed a striking dissociation in our experiment: On the one hand, when making untimed predictions, causal but not object framing instructions led to an indirect effect. This is consistent with previous results showing that indirect effects in explicit tasks are framing dependent. When a causal framing was lacking, predictions about items that occurred only in pairs during training appeared to be based only on the outcome of the relevant pair trials, and not on the outcome associated with separate appearances of the item with which it had been paired. On the other hand, when participants make responses under high time pressure, indirect effects occurred in both framing conditions, as we would expect if such effects were based on pathway strengthening via error-correcting learning. This dissociation is consistent with the hypothesis that indirect effects are driven by an inference-based process in our untimed prediction task, but rely on pathway strengthening in our fast-paced responding task. While supporting our predictions, the dissociation we observed in the effect of

12 TWO MECHANISMS OF CONTINGENCY LEARNING 12 instructions in the two tasks challenges unitary approaches to contingency learning based solely on an inference-based process or solely on pathway strengthening. An inference-based process nicely accounts for the effect of instructions in our prediction task, but would predict either that the indirect effect would emerge in the causal framing condition of the RT task as well, or that no indirect effect would occur in either condition of this task due to the lack of time to make the extra inference about the critical test items. It is unclear to us how an inference-based account would predict an indirect effect in the RT task whether or not causal framing was used, given the assumption of such models that the indirect effect requires an inference that is more likely under a causal framing, and more likely when time and cognitive resources are less constrained. The pattern of results is also difficult to explain based only on a pathway strengthening mechanism. It is true that some proponents of associative accounts of contingency learning have addressed framing effects by positing that causal framing promotes a shift from configural to elemental representations of compound events (Melchers, Shanks & Lachnit, 2008; Williams, Sagness & McPhee, 1994): if a compound event is represented completely distinctly from the elements that make it up, an error-correcting learning process would not predict indirect effects. However, applying this account to our prediction task would predict, contra the data, that objectframed participants would respond equivalently to all test events seen during training only in pairs. We found instead that object framing participants predictions for the singleton test events were sensitive to the outcomes associated with the pair events in which they occurred during training. Furthermore, a pathway strengthening account would need to invoke further assumptions to explain why the instruction manipulation affected performance only in the prediction task, but not in the RT task, if pathway strengthening is thought to be at work in both cases.

13 TWO MECHANISMS OF CONTINGENCY LEARNING 13 There is a third approach to understanding contingency learning that we have not yet explored in this paper. This theory posits that learners attempt to determine the causal relations between events by performing Bayesian inference over possible instantiations of causal graphical models representing these relations (Gopnik et al., 2004, Griffiths & Tenenbaum, 2005, Sobel, Tenenbaum & Gopnik, 2004; Tenenbaum & Griffiths, 2003). While this account is similar to an explicit inference account in that it invokes the concept of inference, it is framed at a computational level that is agnostic to process and mechanism, and refers only to the space of alternative hypotheses and the evidence provided by experience as a basis for Bayesian inference. Proponents of this approach might predict that framing instructions would make a difference, in that the object framing instructions leave open a large space of possible hypotheses about how objects relate to outcomes, while the causal framing instructions are far more restrictive. However, several difficulties remain. First, considering only the explicit prediction task, it is unclear what consistent alternative hypotheses object framing participants might consider that are both (a) consistent with the training data and (b) fit their predictions at test. For example, training experience with the screening pair (S 1 S 2 +) and screening training singleton (S 1 - ) rules out a causal graphical model in which the outcome occurs unless one or more objects in the display prevents it, and participants responses to test singletons rule out any hypothesis that treats pair and singleton events completely independently. While it is difficult to rule out all versions of this general type of account, since it leaves open what the full range of hypotheses might be, the data in the explicit object framing condition are easily explained by assuming that participants are not considering a large hypothesis space at all, but are simply responding to test singletons based on cue-outcome co-occurrence statistics experienced when these items occurred

14 TWO MECHANISMS OF CONTINGENCY LEARNING 14 in pairs during training. Indeed, we have implemented a very simple model to explain the predictions made by participants who received the object framing instructions in the prediction task. The model is a standard exemplar model (e.g., Medin & Schaffer, 1978) and assumes that participants in this condition respond to test displays by comparing the test display to memory representations of the displays encountered during training, then computing a predicted probability for the occurrence of the dot based on a similarity-weighted average of the dot probabilities associated with each of these displays. Because of the similarity-based weighting, exact matches to training displays have greater weight than partial matches. For example, the singleton from the blocking pair that was presented during training exactly matches a representation of itself, and partially matches the representation of the pair, while the singleton from this pair that was not presented as such during training does not exactly match any item presented during training, but does partially match the representation of the pair. The model provides a very good fit to the data (see supplementary Figure 2 available online), and in particular, captures the fact that participants predict the dot about equally often for the blocking test singleton, the screening test singleton, and the two control test singletons, while predicting the dot far lass frequently for the negative test singleton. These probabilities are less extreme (less close to 1 for the positive cases and less close to 0 for the negative case) than those associated with displays participants actually saw during training, due to the fact that the test item only partially matches the representation of the pair stored in memory. A second challenge for the computational level account is to explain why causal framing instructions are necessary for indirect effects in the explicit prediction task, but not under fastpaced responding conditions. One could appeal to the idea that the independent cause

15 TWO MECHANISMS OF CONTINGENCY LEARNING 15 assumption is a default assumption relied on when resources are limited in the fast-paced responding task, but one could then ask why, if indeed this is a default, it is not considered in the explicit object framing condition, where it is perfectly consistent with the data. One reason the hypothesis space might differ between the two tasks is that the tasks actually engage different mechanisms with different constraints, one capable of considering a wide range of alternatives and another that is more restricted. Such stipulations would acknowledge our key point, that different processes underlie indirect effects in untimed prediction and fast-paced responding situations. The specific processes we propose go one step further, making clear predictions about what patterns of results should be observed in our two task situations. Our findings accord with dual process models that include separate implicit and explicit learning mechanisms (e.g., Ashby, Alfonso-Reese, Turken & Waldron, 1998; Sun, Slusarz & Terry, 2005). Such a distinction already has considerable support in the domain of category learning (Ashby & Maddox, 2005; Maddox, Love, Glass & Filoteo, 2008; Spiering & Ashby, 2008, Zeithamova & Maddox, 2006). Previous attempts to separate implicit and explicit knowledge in contingency learning tasks have often relied on participants reports of their awareness of contingencies. Such reports are notoriously sensitive to depth of probing by the experimenter (Maia & McClelland, 2004), and explicit awareness might arise in parallel with connection-based learning (Cleeremans, 1993; Cleeremans & McClelland, 1991), making it difficult to use awareness measures as definitive evidence for the existence of two separate mechanisms. Another type of evidence that has been used to support a two-mechanism account is based on individual differences in learning a simple abstract rule underlying a complex set of contingent relationships (Shanks & Darby, 1998). The individual differences observed in this study, however, have been addressed under an inference-based account by assuming that

16 TWO MECHANISMS OF CONTINGENCY LEARNING 16 different participants are simply relying on different hypotheses, rather than different mechanisms (De Houwer, 2009). Our study s finding that causal framing instructions affect performance in an untimed explicit prediction task but not in a fast-paced responding task provides a different kind of evidence that may provide a stronger challenge to hypothesis-based approaches. It remains possible to argue that some form of associative or connection strengthening process is at work in all forms of learning, while at the same time allowing for the possibility that these processes operate in different learning systems, one subserving the rapid formation of explicit memories for items, hypotheses, and explicitly-formulated inferences, and another subserving the gradual strengthening of connections in processing pathways (McClelland, McNaughton & O Reilly, 1995). Ultimately, learning in all contingency learning tasks may thus be in some sense associative (Shanks, 2010), or, in our terms, pathway strengthening-based. However, this underlying similarity should not obscure the functional differences between these different types of learning. Further research will clearly be needed to more fully delineate the characteristics of these quite different kinds of learning systems. Author Notes This work was supported by the Air Force Research Laboratory, agreement number FA References Ashby, F.G., Alfonso-Reese, L.A., Turken, A.U., & Waldron, E.M. (1998). A neuropsychological theory of multiple systems in category learning. Psychological Review, 105(3), Ashby, F.G., & Maddox, W.T. (2005). Human category learning. Annual Review of Psychology, 56,

17 TWO MECHANISMS OF CONTINGENCY LEARNING Carr, A.F. (1974). Latent inhibition and overshadowing in conditioned emotional response conditioning with rats. Journal of Comparative and Physiological Psychology, 86(4), Cohen, J.D., Dunbar, K., & McClelland, J.L. (1990). On the control of automatic processes: A parallel distributed processing account of the Stroop task. Psychological Review, 97, Cleeremans, A. (1993). Attention and awareness in sequence learning. In Proceedings of the Fifteenth Annual Conference of the Cognitive Science Society, pp Hillsdale, NJ: Erlbaum. Cleeremans, A. & McClelland, J.L. (1991). Learning the structure of event sequences. Journal of Experimental Psychology: General, 120, De Houwer, J. (2009). The propositional approach to associative learning as an alternative for association formation models. Learning & Behavior, 37(1), De Houwer, J. & Beckers, T. (2003). Secondary task difficulty modulates forward blocking in human contingency learning. Quarterly Journal of Experimental Psychology: B, 56, Gopnik, A., Glymour, C., Sobel, D., Schulz, L., Kushnir, T., & Danks, D. (2004). A theory of causal learning in children: causal maps and Bayes nets. Psychological Review, 111, Griffiths, T.L., & Tenenbaum, J.B. (2005). Structure and strength in causal induction. Cognitive Psychology, 51(4), Kamin, L.J. (1968). Attention-like processes in classical conditioning. In M. R. Jones (Ed.), Miami symposium on the prediction of behavior: Aversive stimulation, pp Coral Gables, FL: University of Miami Press. Kamin, L.J. (1969). Predictability, surprise, attention, and conditioning. In B. A. Campbell & R. M. Church (Eds.), Punishment and Aversive Behavior. New York: Appleton-Century-Crofts.

18 TWO MECHANISMS OF CONTINGENCY LEARNING 18 Krechevsky, I. (1932). Hypotheses in rats. Psychological Review, 39(6), Lovibond, P.F., Been, S., Mitchell, C.J., Bouton, M.E., & Frohardt, R. (2003). Forward and backward blocking of causal judgment is enhanced by additivity of effect magnitude. Memory & Cognition, 31, Maddox, W.T., Love, B.C., Glass, B.D., & Filoteo, J.V. (2008). When more is less: Feedback effects in perceptual category learning. Cognition, 108, Maia, T.V., & McClelland, J.L. (2004). A reexamination of the evidence for the somatic marker hypothesis: What participants really know in the Iowa gambling task. Proceedings of the National Academy of Sciences, 101(45), McClelland, J.L., McNaughton, B.L., & O Reilly, R.C. (1995). Why there are complementary learning systems in the hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory. Psychological Review, 102(3), Medin, D.L., & Schaffer, M.M. (1978). Context theory of classification learning. Psychological Review, 85(3) Melchers, K.G., Shanks, D.R., & Lachnit, H. (2008). Stimulus coding in human associative learning: Flexible representations of parts and wholes. Behavioural Processes, 77(3), Mitchell, C.J., De Houwer, J., & Lovibond, P.F. (2009). The propositional nature of human associative learning. Behavioral and Brain Sciences, 32, Pavlov, I.P. (1927). Conditioned reflexes. London: Oxford University Press. Pearce, J.M., & Hall, G. (1980). A model of Pavlovian learning. Psychological Review, 87, Rescorla, R.A., & Wagner, A.R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In Black, A.H., & Prokasy, W.F.

19 TWO MECHANISMS OF CONTINGENCY LEARNING 19 (Eds.), Classical conditioning II: Current theory and research. New York: Appleton- Century-Crofts. Shanks, D.R. (2010). Learning: From association to cognition. Annual Review of Psychology, 61(1), Shanks, D.R., & Darby, R.J. (1998). Feature- and rule-based generalization in human associative learning. Journal of Experimental Psychology. Animal Behavior Processes, 24(4), Sobel, D.M., Tenenbaum, J.B., & Gopnik, A. (2004). Children s causal inferences from indirect evidence: Backwards blocking and Bayesian inference in preschoolers. Cognitive Science, 28, Spiering, B.J., & Ashby, F.G. (2008). Initial training with difficult items facilitates information integration, but not rule-based category learning. Psychological Science, 19(11), Sun, R., Slusarz, P., & Terry, C. (2005). The interaction of the explicit and the implicit in skill learning: A dual-process approach. Psychological Review, 112(1), Tenenbaum, J.B., & Griffiths, T.L. (2003). Theory-based causal inference. In S. Becker, S. Thrun, & K. Obermeyer (Eds.), Advances in Neural Information Processing Systems 15, Cambridge: MIT Press. Tolman, E.C. (1948). Cognitive maps in rats and men. Psychological Review, 55(4), Tolman, E.C. (1949). There is more than one kind of learning. Psychological Review, 56(3), Waldmann, M.R., & Holyoak, K.J. (1992). Predictive and diagnostic learning within causal models: Asymmetries in cue competition. Journal of Experimental Psychology: General, 121(2), Waldmann, M.R., & Holyoak, K.J. (1997). Determining whether causal order affects cue selection in human contingency learning: Comments on Shanks and Lopez (1996). Memory & Cognition, 25(1),

20 TWO MECHANISMS OF CONTINGENCY LEARNING 20 Williams, D.A., Sagness, K.E., & McPhee, J.E. (1994). Configural and elemental strategies in predictive learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 20(3), Zeithamova, D., & Maddox, W. T. (2006). Dual-task interference in perceptual category learning. Memory and Cognition, 34(2), "

21 TWO MECHANISMS OF CONTINGENCY LEARNING 21 Table 1. Predictions about indirect effects for inference-based accounts, pathway strengthening accounts and our combined account. Inference-based accounts predict that these effects should depend on causal framing, and may not occur at all in our fast-paced task. Pathway strengthening using errorcorrecting learning predicts indirect effects in all conditions. On our account, participants make explicit inferences when afforded time to do so, but rely on pathway strengthening when under time pressure. Thus we predict that indirect effects should depend on a causal framing in the untimed prediction task, but should occur under both framing conditions in the fast-paced task. Inference-based Causal framing Object framing Untimed Prediction X Fast-paced RT task / X X Pathway strengthening Causal framing Object framing Untimed Prediction Fast-paced RT task Our account Causal framing Object framing Untimed Prediction X Fast-paced RT task

22 TWO MECHANISMS OF CONTINGENCY LEARNING 22 Table 2. Event contingencies during training and test, prediction task responses at test, and RT task response times at test. Events labeled + were followed by the dot on 90% of presentations, while those labeled - were followed by the dot on 10% of presentations. During the training phase of both tasks, filler events were included in order to reduce the base rate of the dot outcome below 0.5. Prediction task data reports proportion of test trials on which participants predicted that the dot would appear for each item. RT task data reports the mean (and standard deviation) of participants median response time for each item in milliseconds. Object Trained Pair Trained Singleton Test Singleton Item contingencies Untimed Prediction Task Fast-Paced RT Task Experimental items Fillers (only appeared during training block) causal framing object framing causal framing object framing Blocking B 1 B 2 + B 1 + B 2 Screening S 1 S 2 + S 1 - S 2 Negative N 1 N 2 - N 1 - N 2 Control C 1 C C 1, C 2 NP 1 NP PS1+ NS 1 - NS 2 - Blocking Screening Negative Control Blocking Screening Negative Control Blocking (64) (62.9) (48.5) Screening (54.8) (49.8) (62.6) Negative (59.4) (52.6) (49.1) Control (59.9) (44.9) Blocking (49.9) (58.4) (51.1) Screening (52.1) (45.2) (59.9) Negative (71.3) (41.4) (41.7) Control (61.5) (49.5) Table 3. Framing instructions for the causal framing and object framing conditions. Each

23 TWO MECHANISMS OF CONTINGENCY LEARNING 23 sentence was presented on a separate instruction screen, to ensure participants read them carefully. Condition Causal framing Object framing Instructions Some of the objects that you will see during the experiment have the power to make the dot appear, while others do not. The computer has randomly decided which objects have this power at the beginning of the experiment. You cannot tell just by looking at the object whether it has the power to make the dot appear. However, you can learn which objects have this power based on whether or not the dot appears when the object is inside the box. If two objects appear inside the box and at least one of them has this power, the dot will usually appear. Sometimes the box may malfunction, and the dot may occasionally fail to appear when it should, and may occasionally appear when it shouldn't. If you can determine which objects have the power to make the dot appear, this will help you to make predictions during the experiment. The computer follows rules to determine whether the dot will appear, based on the object(s) currently displayed on the screen. Sometimes the dot may fail to appear when it should, and may sometimes appear when it shouldn t. If you pay attention to the object(s) displayed on the screen, this will help to improve your performance during the experiment.

24 TWO MECHANISMS OF CONTINGENCY LEARNING 24 Figure Captions Figure 1. The sequence of displays and timing of trials in the two tasks. In each task, the appearance of one or two objects appear on the screen was followed by an outcome (either dot or no dot) and feedback. In the prediction task, after the object(s) appeared, participants had unlimited time to make a prediction. After making their prediction, they were given visual feedback (10 points or -5 points) and auditory feedback (pleasant sound or buzzer) for 500 ms. In the RT task, the dot automatically did or did not appear 350 ms after the object(s) appeared, and on dot trials, the participant had to respond within a response window. The duration of the response window was initially 400 ms, decreasing by 12.5 ms every 10 trials for the first 200 training trials, and then by 2.5 ms every 10 trials for the next 100 trials. After this, the duration remained at 250 ms through the rest of training and test. Figure 2. Responses and RTs for the key training and test items in the prediction and RT tasks, respectively. Error bars indicate standard errors of the mean.

25 Feedback Outcome Object(s) 10 Prediction Untimed RT 850 ms Object(s) 500 ms Object(s) 500 ms 500 ms Outcome Feedback response deadline (finalized at 250 ms) 500 ms Outcome Feedback

26 Direct effect Indirect effect B1 Prediction task Proportion of responses predicting dot S1 Causal framing Object framing Proportion of responses predicting dot B2 S2 Causal framing Object framing RT task rt (ms) B1 S1 Causal framing Object framing rt (ms) B2 S 2 Causal framing Object framing

27 Additional Methods and Analyses Methods Payment to participants All participants in the prediction task were paid at least $5.00 (mean: $7.30, s.d. $0.73). All participants in the RT task were paid at least $10.00 (mean: $12.66, s.d. $1.57). Timing details There was a delay of random length between trials (prediction task: ms, RT task: ms). Task instructions Each line of the instructions appeared as a separate screen, followed by Press the spacebar to continue. Framing instructions, shown in the main text, were identical for both tasks and followed immediately after the main task instructions. Untimed prediction task In this experiment we will be looking at how you learn the relationships between events. Throughout the experiment, you will see a box in the center of the screen. On each trial, one or two objects will appear in the box. On some trials, a dot will appear on either the left or right side of the screen shortly afterward Your job is to predict whether a dot will appear on each trial. After the objects appear in the box, you need to make your prediction by pressing Z (the "Yes" key) if you think the dot will appear on that trial, or / (the "No" key) if you think the dot will not appear on that trial. You will then see the outcome (dot or no dot), and receive feedback You will receive points based on your ability to predict the outcome on each trial If you correctly predict the outcome on a trial, you will hear a pleasant "ding" and receive 10 points ("10" will appear in the center of the screen in green). If you incorrectly predict the outcome, you will hear a buzzer and lose 5 points ("-5" will appear in the center of the screen in red).

28 Every 5 points you receive is equal to 1 cent. Fast-paced RT task In this experiment we will be looking at how you learn the relationships between events. Throughout the experiment, you will see a box in the center of the screen. On each trial, one or two objects will appear in the box. On some trials, a dot will appear on either the left or right side of the screen shortly afterward. These are called ''go trials''. On other trials, no dot will appear. These are called ''no-go trials''. Your job is to respond as quickly as possible when the dot appears by pressing the spacebar, regardless of the side where the dot appears. You will receive points based on your ability to perform correctly on each trial. If you press the spacebar fast enough on go trials, or correctly keep from pressing on nogo trials, you will earn 10 points. You will hear a pleasant ''ding'', and ''10'' will appear in green. However, if you do not press quickly enough on a go trial, or press the spacebar by mistake on a no-go trial, you will lose 5 points. You will hear a buzzer and ''-5'' will appear in red. If you press before the outcome occurs, you will also lose 5 points. You will hear a beep, and ''-5'' will appear in red after the outcome is shown. Every 5 points you receive is equal to 1 cent. Mixed regression analyses In order to more closely examine the effects of instructions in both tasks, we also designed a pair of mixed effects regression models. We submitted predictions for the indirect items in the prediction task to a mixed effects logistic regression including Item (B 2, S 2 ) and Condition (causal framing, object framing) as fixed factors, and Participant as a random factor. This analysis revealed a significant main effect of item, z = 4.171, p <.0001, and a significant item X condition interaction, z = 4.866, p <.0001, confirming our finding that indirect effects only emerged in the causal framing condition of the prediction task. In the RT task, we submitted response times for the indirect items at test to a mixed-effects linear regression, again including Item and Condition as fixed factors and Participant as a random factor. This analysis revealed a main effect of item F(1,93) = 16.47, p <.001, while neither the main effect of condition or the

29 Item X Condition interaction were significant, all F(1,93) < 1, p >.5. This more sensitive analysis provided no support for a framing effect in this task. Exemplar-based context model The model assumes that each display encountered during training is stored as an exemplar representation. Thus, the trained blocking singleton, and the blocking pair, the trained screening singleton, the screening pair, the control pair, the trained negative singleton, and the negative pair, are all represented as exemplars in memory. At test, the presented display is compared to all exemplars. Each stored item is represented as a vector where the elements stand for the different cue objects and the values stand for the presence or absence of the cue (+/-). For example, if the first two elements of the vector are the cues in the blocking pair, its vector would be: The 13 elements of the vector correspond to the 13 cues used in the experiment. The test display is likewise represented as such a vector, so the trained blocking singleton would be: and the blocking test singleton would be: The context model is then used to determine the probability that a dot will occur. Specifically, all exemplar representations positively associated with the dot vote that the probability of a dot is 11/12 (the empirical probability of a dot's occurring for these items), and all exemplar representations that are not associated with the dot vote that the probability of a dot is 1/12 (the empirical dot probability for these items). The strength of each exemplar s vote is calculated in proportion to the following equation:!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!(1) where #matches is the number of features that match between the probe and stored exemplar. The votes are summed and normalized using the equation:!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!(2)!

30 to give the predicted probability that participants will predict the dot. A single gain parameter g was fitted to the actual participant prediction proportions using maximum likelihood (g = 2.51, negative log likelihood = 37.6). A comparison of the model predictions and the experimental data from the test phase of the untimed prediction task for participants who received object framing instructions is provided in Figure S2. As expected, when a test display matches a display presented during training (e.g. blocking pair, B 1 singleton), the model, like participants, relies on the exact matching display, so the p(dot) responses given are close to the experienced values. Interestingly, in the case of the screening pair, which is positively associated with the dot, the p(dot) is a bit reduced in both model and data compared to the blocking pair and the control pair due to the influence from the screening singleton, which is negatively associated with the dot. The effect is small, because the voting is dominated by the exact match, as expected in the context model. Similarly, the screening singleton is endorsed a bit more than the negative singleton, due to the weak positive influence of the screening pair on the screening singleton versus the weak additional negative influence of the negative pair on the negative singleton. Crucially, the test singletons derived from pairs presented during training produce a partial match to the memory representation of the pair. This results in a tendency to predict the dot for all test items from positive training pairs, and a tendency not to predict the dot for the single test item from the negative training pair (N 2 ), but these tendencies are weaker than for items that exactly match a trained display because of the weak contribution to the outcome from the partial match. Very weak background activations spread over the other training items pull the overall endorsement rates toward a middling neutral value. Figure S1 Training curves for both conditions of both tasks. The prediction task graph plots the proportion of trials on which participants predicted the most likely outcome for events as a function of the number of presentations. The RT task graph plots mean response time on trials where the dot appeared as a function of the number of presentations on which they had seen the event followed by the dot. The lower axis corresponds to the dot-likely items (which were followed by the dot on 65 of 72 presentations), while the upper axis corresponds to the dot-unlikely items (which were followed by the dot on 7 of 72 presentations).! Figure'S2' ' Comparison!of!actual!participant!prediction!proportions!for!test!events!(top)!to!exemplar! model!prediction!probabilities!(bottom).!the designation 'pair' refers to the blocking, screening, control, or negative pair seen by participants during training. For the members of the blocking, screening, and negative pairs, the designation '1' refers to the member of the pair that occurred as a singleton during training, and the designation '2' refers to the member of the pair that only occurred during the test phase. In the case of the control singletons, neither occurred at