ALM: An R Package for Simulating Associative Learning Models

Similar documents
The Mechanics of Associative Change

Representations of single and compound stimuli in negative and positive patterning

A configural theory of attention and associative learning

The Rescorla Wagner Learning Model (and one of its descendants) Computational Models of Neural Systems Lecture 5.1

Simulation of associative learning with the replaced elements model

The Influence of the Initial Associative Strength on the Rescorla-Wagner Predictions: Relative Validity

An Attention Modulated Associative Network

Contemporary associative learning theory predicts failures to obtain blocking. Comment on Maes et al. (2016)

THEORIES OF ASSOCIATIVE LEARNING

Comparing Patterning and Biconditional Discriminations in Humans

Changes in attention to an irrelevant cue that accompanies a negative attending discrimination

An attention-modulated associative network

Context and Pavlovian conditioning

Representation and Generalisation in Associative Systems

Metadata of the chapter that will be visualized online

COMPUTATIONAL MODELS OF CLASSICAL CONDITIONING: A COMPARATIVE STUDY

Changes to Associative Learning Processes in Later Life

Amount of training effects in representationmediated food aversion learning: No evidence of a role for associability changes

Classical Conditioning Classical Conditioning - a type of learning in which one learns to link two stimuli and anticipate events.

Discrimination blocking: Acquisition versus performance deficits in human contingency learning

Modulatory mechanisms in Pavlovian conditioning

Classical Conditioning V:

Classical Conditioning. Learning. Classical conditioning terms. Classical Conditioning Procedure. Procedure, cont. Important concepts

Associative learning

The Fate of Redundant Cues in Human Predictive Learning

Learned changes in the sensitivity of stimulus representations: Associative and nonassociative mechanisms

Topics in Animal Cognition. Oliver W. Layton

After Rescorla-Wagner

Discrimination of Structure: I. Implications for Connectionist Theories of Discrimination Learning

AP PSYCH Unit 6.1 Learning & Classical Conditioning. Before ever opening this book, what did you think learning meant?

Chapter 6. Learning: The Behavioral Perspective

Evaluative and Non-Evaluative Conditioning: Theory and Implications of Conditioning of Valence and Attributes

Associative Learning

Transfer after Serial Feature Positive Discrimination Training

A response rule for positive and negative stimulus interaction in associative learning and performance

The Feature-Label-Order Effect In Symbolic Learning

Myers PSYCHOLOGY. (7th Ed) Chapter 8. Learning. James A. McCubbin, PhD Clemson University. Worth Publishers

Unit 6 Learning.

The organism s problem: Avoid predation and injury so you can reproduce

TEMPORALLY SPECIFIC BLOCKING: TEST OF A COMPUTATIONAL MODEL. A Senior Honors Thesis Presented. Vanessa E. Castagna. June 1999

Functional Relationships Between Arbitrary Stimuli and Fixed Responses: Reflex Conditioning

A Simulation of Sutton and Barto s Temporal Difference Conditioning Model

An elemental model of associative learning: I. Latent inhibition and perceptual learning

Occasion Setting without Feature-Positive Discrimination Training

The interaction between response effects during the acquisition of response priming

Parameter Invariability in the TD Model. with Complete Serial Components. Jordan Marks. Middlesex House. 24 May 1999

Positive and negative patterning in human classical skin conductance response conditioning

Learning. AP PSYCHOLOGY Unit 5

Symmetrical generalization decrements: Configural stimulus processing in human contingency learning

Summation and transfer of negative occasion setting

Transfer of Control in Ambiguous Discriminations

Association. Operant Conditioning. Classical or Pavlovian Conditioning. Learning to associate two events. We learn to. associate two stimuli

WHAT S ELEMENTARY ABOUT ASSOCIATIVE LEARNING?

DISCRIMINATION OF SERIAL AUDITORY PATTERNS IN RATS - roles of configural and elemental associations

Predictive Accuracy and the Effects of Partial Reinforcement on Serial Autoshaping

Blocking Effects on Dimensions: How attentional focus on values can spill over to the dimension level

Within-compound associations in retrospective revaluation and in direct learning: A challenge for comparator theory

Lateral Inhibition Explains Savings in Conditioning and Extinction

How do we Learn? Chapter 6 Learning. Class Objectives: What is learning? What is Classical Conditioning? How do you know you ve learned something?

Foundations of Addictions

Cue competition as a retrieval deficit

Dikran J. Martin. Psychology 110. Name: Date: Principal Features. "First, the term learning does not apply to (168)

Extinction and retraining of simultaneous and successive flavor conditioning

Psychological Hodgepodge. Mr. Mattingly Psychology

Human latent inhibition and the density of predictive relationships in the context in which the target stimulus occurs

Chapter 5 Study Guide

Name: Period: Chapter 7: Learning. 5. What is the difference between classical and operant conditioning?

Acquired Relational Equivalence: Implications for the Nature of Associative Structures

PSY 402. Theories of Learning Chapter 4 Nuts and Bolts of Conditioning (Mechanisms of Classical Conditioning)

What is Learned? Lecture 9

Chapter 7 - Learning

Chapter 5: How Do We Learn?

Behavioural Processes

NEED RECURRENT GATED DIPOLE TO EXPLAIN SECONDARY INHIBITORY CONDITIONING 1. Pair Onset of CS 1 with shock READ (I) CIRCUIT

Challenge for Comparator Theory. Klaus Lober* and Harald Lachnit. Philipps-Universität Marburg. David R. Shanks. University College London

Learning. Learning is a relatively permanent change in behavior acquired through experience or practice.

PROBABILITY OF SHOCK IN THE PRESENCE AND ABSENCE OF CS IN FEAR CONDITIONING 1

Magazine approach during a signal for food depends on Pavlovian, not instrumental, conditioning.

Learning. Learning. Learning

Contemporary Study of Pavlovian Conditioning

The effects of Pavlovian CSs on two food-reinforced baselineswith and without noncontingent shock

University of Southampton Research Repository eprints Soton

Learning. Learning. Stimulus Learning. Modification of behavior or understanding Is it nature or nurture?

Value transfer in a simultaneous discrimination by pigeons: The value of the S + is not specific to the simultaneous discrimination context

City Research Online. Permanent City Research Online URL:

Contrasting AAC and ABC renewal: the role of context associations

Chapter Six. Learning. Classical Conditioning Operant Conditioning Observational Learning

Rescorla-Wagner (1972) Theory of Classical Conditioning

Two Kinds of Attention in Pavlovian Conditioning: Evidence for a Hybrid Model of Learning

Conditioning The Behavior of the Listener. Conditioning the Behavior of the Listener

Configural Olfactory Learning in Honeybees: Negative and Positive Patterning Discrimination

Teaching Classical and Operant Conditioning in a Laboratory-Based Course: Eight Effective Experiments

Transverse patterning in pigeons

Classical Conditioning II. Generalization and Discrimination. Generalization Example. Generalization/Discrimination; Time and the Order of Events

Com bin ing CSs Associated w ith the Sam e or Differen t USs

PSYC 337 LEARNING. Session 4 Classical Conditioning II. Lecturer: Dr. Inusah Abdul-Nasiru Contact Information:

Combining Configural and TD Learning on a Robot

Classical Conditioning. AKA: Pavlovian conditioning

Learning. Learning is a relatively permanent change in behavior acquired through experience.

Overshadowing by fixed and variable duration stimuli. Centre for Computational and Animal Learning Research

Transcription:

ALM: An R Package for Simulating Associative Learning Models Ching-Fan Sheu & Teng-Chang Cheng National Cheng Kung University, Taiwan 9 July 2009 Sheu & Cheng (NCKU) ALM 9 July 2009 1 / 24

Outline 1 Introduction & Motivation 2 Pavlovian Conditioning 3 The Rescorla-Wagner Model 4 Configural or Elementary Association 5 Human Associative Learning The Configural Model of Pearce The Elemental Model of Harris 6 Two Discrimination Problems 7 Conclusions Sheu & Cheng (NCKU) ALM 9 July 2009 2 / 24

Introduction & motivation Psychologists have been using a variety of experimental paradigms to study associative learning. A computer software is needed to implement models of associative learning for teaching and research. Macho (2002) implemented the configural model of Pearce with Microsoft Excel. Schultheis, Thorwart, & Lachnit (2008) implemented the elemental model of Harris with MATLAB. Excel and MATLAB are commercial software. Programming with spreadsheets is inefficient. Sheu & Cheng (NCKU) ALM 9 July 2009 3 / 24

Pavlovian conditioning (Pavlov, 1927) Theories of associative learning are concerned with the factors that govern the association formation when two stimuli are presented together (Pearce & Bouton, 2001). Conditioning Before During After Unconditioned Stimulus (US) Unconditioned Response (UCR) Food Salivation Bell - Conditioned Stimulus (CS) + US UCR Bell + Food Salivation CS Conditioned Response (CR) Bell Salivation Sheu & Cheng (NCKU) ALM 9 July 2009 4 / 24

Blocking (Kamin, 1969) Condition Stage 1 Stage 2 Test Treatment Light Light+Tone Tone Shock Shock Control - Light+Tone Tone - Shock Trial 16 8 Rats in the treatment group showed no fear of the tone, while rats in the control group were afraid of it. Sheu & Cheng (NCKU) ALM 9 July 2009 5 / 24

The Rescorla-Wagner model (1972) At each trial, the association strength between a given CS and the US changes in proportion to the discrepancy between the maximum strength supported by the US and the total strength of all conditioned stimuli present at the current trial: n V cs = α cs β us (λ us V i ) i=1 Sheu & Cheng (NCKU) ALM 9 July 2009 6 / 24

Blocking Effect Explained Conditioning End of stage 1 CS Present Weight CR Light 1 1 1 Tone 0 0 0 During stage 2 Light 1 1 1 Tone 1 0 0 Test Light 0 1 0 Tone 1 0 0 Sheu & Cheng (NCKU) ALM 9 July 2009 7 / 24

Configural or elemental associations HARRIS Two Problems for Elemental Theories he core assumption of any elemental theory, including stimuampling theory and the Rescorla Wagner model, is that stimelements become independently associated with the US. How-, the soundness of this assumption is seriously questioned by onstrations that animals can learn certain conditional discrimions between stimulus compounds that cannot be solved by a le elemental process (Spence, 1952). Two examples are nege patterning and biconditional discrimination. In negative pating, two CSs (A and B) are presented on separate trials, and is followed by the US ( ). Intermixed among these A and Figure 1. Elemental and configural representations in the Rescorla trials are trials in which the two stimuli are presented simulously but not followed by the US (AB ). A simple elemental circles) that become associated with the unconditioned stimulus (US). A Wagner model. Individual stimuli (A and B) activate representations (black predicts that the associative strengths of A and B will compound stimulus (AB) activates these same representations but additionally activates a configural representation (black square) that specifi- ease on A and B trials and will decrease on AB trials. ever, responding should always be greater the AB trials cally codes for the conjunction of A and B and forms an independent Conditioning with a use the combined associative strengths of A and B will proe more responding than that elicited by either stimulus alone. association compound with the US. stimuli results in a unitary iew of this prediction, representation it is important that, in a variety of ofthe inhibitory compound associations from the configural entering elements of the into nonreinforced compounds (AC and BD in the previous example) and association with the itioning paradigms, animals have been shown to master nege patterning discriminations, albeit with considerable diffiy: They learn to respond reinforcer. excitatory associations from the configural elements of the reinforced compounds (AB and CD). more on A and B trials than on trials (e.g., Pavlov, 1927; Rescorla, 1972, 1973; Whitlow & The addition of a configural element to the representation of a ner, 1972). iconditional discriminations When present two an evenor more more stimulus complex stimuli compoundare enabledpresented the Rescorla Wagner model for to explain how animals can learn to withhold responses to a nonrein- conditioning, each. Four distinctive stimuli (A, B, C, and D) are presented in four forced compound whose component CSs are excitatory. However, wise combinations, element two of which are reinforced may (AB enter and in into the absence association of such explicit training, with the model predicts the thereinforcer. ) and two of which are not reinforced (AC and BD ). summation of responding when two or more CSs are presented in s, each of the four CSs is reinforced when presented in one compound. Although there are many demonstrations that animals pound and not reinforced in another compound. Therefore, all do respond to the compound of two CSs more than to each CS uli have equivalent reinforcement history and so provide no individually (e.g., Kehoe, 1982, 1986; Rescorla, 1997), there are rential information to cue the animal to respond or not red. In other words, like negative patterning, biconditional dis- also many reported failures to observe summation in Pavlovian conditioning paradigms. Many of these failures have arisen in inations are not solvable by a simple elemental mechanism. autoshaping experiments with pigeons (e.g., Aydin & Pearce, etheless, animals can solve such discriminations, learning to 1995, 1997; Rescorla & Coldwell, 1995), but both successes and ond more on AB and CD trials than on AC and BD Sheu & Cheng (NCKU) failures to observe summationalm have been reported in other para- 9 July 2009 8 / 24

Human associative learning (Shanks, et al. 1998) Stage 1 Stage 2 Test A+ B+ A+ AB- GH+ AB- AC+ IJ- AC- D+ - D+ DE- - DE- DF+ - DF- Trial 15 10 10 Participants were asked to predict whether an allergy would occur (+) for the food (A) presented and received feedback trial by trial. Learning at stage 2 should lead to more positive responses for AB than for DE if patterns are learned elementwise. Sheu & Cheng (NCKU) ALM 9 July 2009 9 / 24

Food Stimuli A Cheese Fromage B Chocolate Chocolat C Milk Lait D Cucumber Concombre E Fish Poisson F Banana Banane G Olive oil Huile d olive H Vinegar Vinaigre I Onion Oignon Sheu & Cheng (NCKU) ALM 9 July 2009 10 / 24

except as described below. The design is given in Table 1. The foods were the same as in Experiment 2 with the addition of vinegar and onions. In Stage 1, participants received A Oh Human AB * associative no 0, and AC * Oi trials together learning with functionally (Shanks, et al. 1998) equivalent D -» O2, DE -» no O, and DF O2 trials. Each trial type was presented on 15 occasions, the order of which was randomized. The combination of stimuli was determined by Latin squares such that each of the critical foods (cheese, chocolate, milk) was presented equally often as cue A, B, and C and likewise for the foods assigned to D-F (cucumber, fish, banana). In Stage 2, AC no O, D O2, DE -»no O, and DF -»no O test trials in Stage 3. First, it is clear that the results of Experiment 2 have been replicated in that AB elicited far fewer allergy responses than AC. Second, and equally unsurprisingly, DE elicited fewer allergy responses than DF. The crucial result, however, is that no more allergy responses were made to AB than to DE, despite the fact that one constituent of the AB compound had been revalued in Stage 2. In fact, on the first test trial, slightly fewer allergy responses were made to AB 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 Stage 1 Stage 2 Test Figure 3. Mean percentage of allergy predictions in the three stages of Experiment 3. A-F are foods. Responses to AB and DE at the test stage were virtually the same. Sheu & Cheng (NCKU) ALM 9 July 2009 11 / 24

The configural model of Pearce (1987) Sheu & Cheng (NCKU) ALM 9 July 2009 12 / 24

The configural model of Pearce a c j = (a i t w c j) σ J a o = w o j a c j w o j = α j β k (λ k a o ) j a c j is the activation of configural unit j. a i is the input vector. w c j is the configural vector of configural unit j. σ is the specificity parameter. a o is the activation of output unit. w o j is the weight change between configural unit j to output. Sheu & Cheng (NCKU) ALM 9 July 2009 13 / 24

The elemental model of Harris (2006) m w y = w y w j V j y { V xy = j=i w x β y w y w x β y w y n R(A) = wa i VA i i=1 if w y t if w y < t A stimulus activates a population of elements. Activated elements compete for entry to an attention buffer with capacity t. Each element has a fixed probability of being connected to any other elements. V xy is the change in association strength between element x and element y. w y is the difference between self-generated weight, w y, and the activated weight of y by association. R(A) is the response strength of A. Sheu & Cheng (NCKU) ALM 9 July 2009 14 / 24

Input file Cue A B C D E F US Phase Feedback Iseval A 1 0 0 0 0 0 1 1 1 1 AB 1 1 0 0 0 0 0 1 1 1 AC 1 0 1 0 0 0 1 1 1 1 D 0 0 0 1 0 0 1 1 1 1 DE 0 0 0 1 1 0 0 1 1 1 DF 0 0 0 1 0 1 1 1 1 1 B 0 1 0 0 0 0 1 2 1 1 AB 1 1 0 0 0 0 0 2 0 0 DE 0 0 0 1 1 0 0 2 0 0 Sheu & Cheng (NCKU) ALM 9 July 2009 15 / 24

Configural model of Pearce in R CMP=function(dat, itemlabel, items, phase, US, feedback, nb1=15, nb2=10, sigma=2, alpha=1, beta=0.15, lambda=c(0,100)) dat: input dataframe itemlabel: column index for item label items: column indices for items phase: column index for phase US: column index for unconditioned stimulus feedback: column index for feedback nb1: number of learning trials in phase 1 nb2: number of learning trials in phase 2 sigma: specificity parameter alpha: salience parameter beta: learning rate parameter lambda: asymptotic value of the unconditioned stimulus Sheu & Cheng (NCKU) ALM 9 July 2009 16 / 24

Elemental model of Harris in R EMH=function(dat, itemlabel, items, phase, feedback, nelements=20, nruns=20, ntrials=c(20,20), beta=2, gain=1, fraction=0, cdensity=.5) dat: input dataframe itemlabel: column index for item label items: column indices for items phase: column index for phase feedback: column index for feedback nelements: number of elements nruns: number of simulation runs ntrials: number of trials in each of two phases beta: learning rate parameter gain: gain parameter fraction: fraction parameter cdensity: connectivity density parameter Sheu & Cheng (NCKU) ALM 9 July 2009 17 / 24

An example script > input=read.table( xp3data.asc, h=t) > cmpout=cmp(input[1:9,1:10],1,c(2:7),9,8,10,sigma=2); > CMP.plot(cmpOut$sumdata,cmpOut$itemlabel, sigma=2 ) # > emhout=emh(input[1:9,1:10],1,c(2:8),9,10,ntrial=c(30,20)) > EMH.plot(emhOut$sumdata,emhOut$evallabs, EMH Plot ) Sheu & Cheng (NCKU) ALM 9 July 2009 18 / 24

Human associative learning - CMP CMP Plot, specificity=2 CMP Plot, specificity=10 Response strength 20 0 20 40 60 80 100 120 A AB AC D DE DF B Response strength 20 0 20 40 60 80 100 120 A AB AC D DE DF B 5 10 15 20 25 5 10 15 20 25 Trials Trials Figure: σ=2 vs. σ=10 Sheu & Cheng (NCKU) ALM 9 July 2009 19 / 24

Human associative learning CMP Plot, specificity=10 EMH Plot Response strength 20 0 20 40 60 80 100 120 A AB AC D DE DF B Response strength 20 0 20 40 60 A AB AC D DE DF B 5 10 15 20 25 0 10 20 30 40 50 Trials Trials Figure: CMP vs. EMH Sheu & Cheng (NCKU) ALM 9 July 2009 20 / 24

Two discrimination problems In positive patterning, two stimuli are not reinforced when each is presented alone (A-, B-), but a US follows when the two are presented together (AB+). In negative patterning, a US is presented after each of two stimuli when they are presented alone (A+, B+), but not when they are presented together (AB-). Sheu & Cheng (NCKU) ALM 9 July 2009 21 / 24

Positive Patterning: A-,B-, AB+ Positive Patterning CMP Postive Patterning EMH Response strength 20 0 20 40 60 80 100 120 A B AB Response strength 20 10 0 10 20 30 40 A B AB 5 10 15 20 25 0 20 40 60 80 100 Trials Trials Figure: CMP vs. EMH Sheu & Cheng (NCKU) ALM 9 July 2009 22 / 24

Negative Patterning: A+, B+, AB- Negative Patterning CMP Negative Patterning EMH Response strength 20 0 20 40 60 80 100 120 A B AB Response strength 20 10 0 10 20 30 40 A B AB 5 10 15 20 25 0 50 100 150 200 Trials Trials Figure: CMP vs. EMH Sheu & Cheng (NCKU) ALM 9 July 2009 23 / 24

Summary We implemented in R two models for associative learning. The unique cue theory and the replaced elements model are yet to be implemented. The ALM R package enables users to easily reproduce, modify, and extend these models for teaching and research. Sheu & Cheng (NCKU) ALM 9 July 2009 24 / 24

2024 © healthdocbox.com Privacy Policy | Terms of Service | Feedback