1. Learning theory. Cambridge MRCPsych course Tue 23 Oct 2018

Transcription

1 1. Learning theory Rudolf Cardinal University Lecturer in Clinical Informatics, Department of Psychiatry, University of Cambridge Honorary consultant psychiatrist, Liaison Psychiatry Service, CPFT & CUH Cambridge MRCPsych course Tue 23 Oct 2018

2 MRCPsych syllabus (2013, 2018):

3 Routes to action Unlearned behaviour: stuff you re born with simple, e.g. reflexes complex, e.g. fixed action patterns Pavlovian conditioning (= classical conditioning): predicting the world and responding accordingly Instrumental conditioning (= operant conditioning): controlling the world Also some more complex situations in which these interact. see Cardinal et al. 2002; Cardinal 2001; Doty & Bosma 1956; Lorenz 1939; Tinbergen 1948

4

5 Classical (Pavlovian) conditioning Pavlov (1927): dogs, salivation. The experimenter arranges a relationship between stimuli. (The subject s responses are measured.) One is initially neutral: the (to-be-)conditioned stimulus (CS). One is biologically relevant: the unconditioned stimulus (US), which normally evokes an unconditioned response (UR). When the CS and US are paired by the experimenter, the CS comes to elicit the same response, termed a conditioned response (CR). Other basic phenomena: extinction: after training, present CS without US blocking: A US; then A+B US; what s learned about B? Pavlov (1927). Kamin (1969), blocking.

6 Classical conditioning: what s being learned? (1) Simple model: CS1 CS2 US response initially weak; plastic synapse strong / fixed Rescorla Wagner model: ΔVCS = αβ(γ Vtotal) V = current associative strength γ = degree of associative strength that the US will support thus: learn to the degree that an unexpected outcome occurs [α = CS learning rate parameter; β = US learning rate parameter] Describes learning, extinction, blocking, etc. Rescorla & Wagner (1972).

7 Classical conditioning: Rescorla Wagner illustration ΔVCS = αβ(γ Vtotal)

8 Classical conditioning: what s being learned? (2) Reality is more complex, including associations formed from the CS to the US and to affect, not just the response.

9 Generalization Condition to something. e.g. 2 khz tone (CS) shock (US) leads to freezing to tone (CR) Generalize: respond to something similar. e.g. 2.1 khz tone freeze (CR) Clearly sensible to a degree. Over-generalization in e.g. anxiety disorders? For example: (1) panic attack (US?) on train (initially neutral stimulus; becomes CS?) (2) generalize to enclosed spaces, public transport, situations with mechanical noises, interoceptive stimuli vaguely like panic symptoms? e.g. Dymond et al. (2015)

10 Stimulus preparedness A predisposition to learn about some specific stimuli more than others. Easier to induce fears in the lab to e.g. snakes, spiders, heights. Predisposition attributable to evolutionary history? Example: Subjects: non-fearful lab-reared monkeys Observe wild-reared monkey displaying fear to live/toy snakes, even on video display fear reactions themselves to toy snakes but not if flowers or toy rabbits replaced the snakes in the videos even though flowers etc. work fine as CSs in appetitive experiments suggests something special and innate about snakes; predisposed to activate the fear module? e.g. Seligman 1971, Öhman 1979; Öhman & Mineka 2001

11 Incubation [ALLOW MIND TO SWITCH OFF] Traditional definition of fear incubation : not a real thing? (1) CS fearful US (2) wait... response to CS increases Eysenck s: (1) CS US (2) more CS presentations... response to CS increases if true, particularly concerning for CS exposure therapies like desensitization. there is experimental evidence against this (e.g. Malloy & Lewis) so: a term to be aware of (as per the syllabus), but probably not a thing. Incubation also used re creativity (answers come in the shower / during sleep, etc.). This probably is a thing. McAllister & McAllister (1967); Eysenck (1968 onwards); Malloy & Lewis (1990)

12 Instrumental (operant) conditioning [MIND BACK ON] For controlling the world. The experimenter arranges a contingency between the subject s behaviour and some (motivationally significant) outcome. Action ( operant ) is instrumental in producing the outcome. Thorndike s Law of Effect (cats, puzzle boxes): an effective outcome strengthens the connection between stimuli and the action. A stimulus response (S R) habit theory. But animals and people can also be goaldirected. Thorndike (1905), Grindley (1932), Tolman (1932), Skinner (1938), Dickinson (1980)...

13 Behaviourism: positive and negative reinforcement Skinner (e.g. 1932, 1938, 1953): behaviourism and reinforcement. Positive reinforcement A food, leads to more A Negative reinforcement (inc. escape, avoidance) A less shock, leads to more A Punishment A shock, leads to less A Negative punishment A less food, leads to less A Extinction A nothing leads to less A Positive reinforcer / negative reinforcer = food / shock. Defined by consequences on behaviour. Other useful terms: appetitive, aversive [but not in strict behaviourist tradition as imply emotional states!].

14 Basic schedules of reinforcement Ratio schedules: outcome is delivered by the nth response fixed ratio, e.g. n = 10 FR-1 also called continuous reinforcement like piecework pause (e.g. to eat), then resume variable ratio, e.g. n = varying between 5 15 like door-to-door sales, gambling high, quite steady rates of responding Interval schedules: outcome is delivered by the first response after t seconds have elapsed since the last outcome fixed interval, e.g. t = 60 seconds pause after reinforcer, then towards expected reinf. time variable interval, e.g. t = seconds moderate, steady responding models consumption of a replenishable resource, like grazing?

15 Some clinical aspects of negative reinforcement Successful avoidance implies lack of exposure to the US. If avoidance action predicts no US, and no US comes, no reason to alter your beliefs (or synaptic weights). So you ll continue to avoid. Avoidance behaviours are very resistant to extinction. Particularly resistant if the initial relationship involved some uncertainty about when the US would come ( lack of contrast when US is absent). Think phobias and anxiety.

16 An aside: another schedule of practical use How to get rid of behaviour without punishment, if extinction alone is difficult? Differential reinforcement of other behaviour (DRO schedule) = differential reinforcement of alternative behaviour (DRA schedule). Subject exhibits undesirable behaviour. Reinforce him/her after e.g. 1 minute of not exhibiting that behaviour. If undesirable behaviour occurs, reset the clock. With success, gradually increase the time. e.g. Petscher et al. (2009)

17 Instrumental versus Pavlovian conditioning

18 Animals work for reinforcement for several reasons, including... after Dickinson (1980)

19 Goal-directed action Train rats to press a lever for food A. Give them food B for free. Poison either food A (group P for poisoned) or food B (group U for unpoisoned). Test responding in extinction (no food). If their lever-pressing is goaldirected and they represent the value of the goal, then group P should press less than group U. Adams & Dickinson (1981) They do.

20 Stimulus response habits develop after extended training Adams (1982)

21 Is alcohol-seeking more habitual than goal-directed? Dickinson et al. (2002) experiment 2, figure 6 redrawn

22 Companies may have learned from rat experiments! Sucrose fading procedure: from e.g. 20% sucrose 20% sucrose, 5% ethanol 5% sucrose, 10% ethanol 40% ethanol Samson (1986), rats; 1995 saw introduction of alcopops to UK

23 Cues paired with reinforcement can also motivate

24 Secondary (conditioned) reinforcement Associate neutral stimulus (CS) with appetitive stimulus (US), e.g. by classical conditioning. Then you may work for the CS now termed a conditioned reinforcer (or sometimes a secondary reinforcer). Easily demonstrable in animals: tone food, then work for tone. People often think of money in this context, but money is probably a lot more complicated psychologically (e.g. fungible, not just associated with a specific outcome). There s an equivalent concept of conditioned punishers.

25 Conditioning and addiction Environmental stimuli (cues and contexts) may become associated with the effects of drugs such as cocaine through Pavlovian conditioning. They become conditioned stimuli (CSs). They may motivate an addict to seek out drugs cueinduced (conditioned) craving.

26 Cues paired with reinforcement can also motivate

27 Pavlovian instrumental transfer depends on motivational state Dickinson (1986); Dickinson & Dawson (1987a, 1987b)

28 Pavlovian instrumental transfer? Supermarkets Static advertising, of course, and advertising to children (works: e.g. Galst & White 1976 Child Dev 47:1089), but also auditory/visual stimuli: Tesco TV is being established to provid[e] offers and value propositions from Tesco, its partners and advertisers where it can be of most value, in-store where many purchase decisions are made 7 zones were identified in-store where programming could be targeted to make the best use of dwell time to create a positive effect for the customer and advertisers [Grocery, Beers/Wines/Spirits, etc.] The trial began in 3 stores and its impact was comprehensively researched with Tesco customers proposed roll-out to 300 stores Feb 2004

29 Shopping and motivational state Mela et al. (1996)

30 Summary: routes to action Unlearned behaviour: stuff you re born with simple, e.g. reflexes complex, e.g. fixed action patterns Pavlovian conditioning (= classical conditioning): predicting the world and responding accordingly Instrumental conditioning (= operant conditioning): controlling the world goal-directed action instrumental contingency incentive value of the outcome complex; learned; involves hedonic value stimulus response habits Also some more complex situations in which these interact. conditioned reinforcement Pavlovian-to-instrumental transfer

31 Reciprocal inhibition, habituation, chaining, shaping, cueing Google: this combination leads you almost exclusively to MRCPsych links. This tells you something. So, definitions: reciprocal inhibition = you can t be both anxious and relaxed, so evoke (in behaviour therapy) the opposing response while presenting the anxiety-provoking stimulus habituation = ceasing to respond after repeated presentations (e.g. startle to a loud noise get used to it) shaping = reward animal for doing something close to what you want... ever closer... behaviour is shaped like clay chaining = if you want a behavioural sequence (chain) ABCDE, reinforce A, then reinforce AB, etc. cueing = use of a teaching signal to shape behaviour (e.g. smile, frown) (educational psychology)

32 Observational learning A kind of social learning. Perceive (observe) behaviour, then perform (imitate) that behaviour. Transmission of aggression through imitation of aggressive models : Bobo doll experiment. Acquisition simply through observation, including in one-trial learning. Delayed performance, mediated by cognitive processes (cognitive mediation). Vicarious, rather than direct, reinforcement. What are the consequences for someone else when they perform a behaviour? Bandura argues it involves: attentional processes (e.g. salience, functional value); retention processes (e.g. cognitive construction, rehearsal); production processes (e.g. monitoring of own enactment); motivational processes (e.g. perceived benefits/costs). Bandura (1961, 1971, 1972, 2004); Deguchi (1984).

33 Cognitivism / cognitive learning theories Broad group of theories. Developed from Gestalt psychology: the way the mind acquires meaningful perceptions in a complex environment. Emphasize internal mental processes (and contrasted with radical behaviourism that de-emphasized these), such as attention working memory computation You ll come across specific examples, like the Baddeley & Hitch model of working memory [see memory lectures].

34