NSCI 324 Systems Neuroscience

Transcription

1 NSCI 324 Systems Neuroscience Dopamine and Learning Michael Dorris Associate Professor of Physiology & Neuroscience Studies

2 NSCI 324 Systems Neuroscience Dopamine and Learning Suggested/source readings: Schultz, W. (2006) Behavioural Theories and the Neurophysiology of Reward. (review) Annual Reviews of Psychology, 57: Hollerman and Schultz (1998) Dopamine Neurons Report and Error in the Temporal Prediction of Reward during Learning. Nature Neuroscience, 1: Articles uploaded online.

3 4. Recording of DA neurons a. transient burst for unpredicted rewards Ø Tonically active Ø Reward results in Phasic Burst Ø Latencies of 100ms, lasting 300ms Ø Do not discriminate between rewards Ø Respond also to salient stimuli

4 The role of DA neurons in classical (Pavlovian) conditioning:

5 Do DA neurons conform with Learning Theory? Learning theory suggests that three factors govern conditioning: 1. Contiguity 2. Contingency 3. Prediction Error All three requirements need to be fulfilled for learning to occur

6 Contiguity Definition: sequential occurrence of stimulus and response In learning: Temporal proximity of a conditioned stimulus (CS) or action and the reward A reward needs to follow a CS by an optimal interval of less than a few seconds

7 Contiguity Acquisition CS predicts reward Extinction Previously reward predicting CS suddenly no longer predicts reward Horizontal lines indicate duration of predictive licking

8 Neurophysiological Basis of Contiguity chronology CS

9 Neurophysiological Basis of Contiguity chronology CS CS CS CS CS Reward

10 Contingency The conditional probability of reward occurring in the presence of a CS as opposed to its absence Higher the probability of pairing CS the higher the likelihood/faster learning occurs 100% Vs 1%

11 Contingency CS fully predicts reward CS fully predicts absence of reward CS has no predictive value with respect to reward. Time of Stimulus Presentation

12 Reward Prediction Error To learn, we require a discrepancy between the predicted reinforcer and the actual reinforcer received. This is known as prediction error.

13 Reward Prediction Error No CS presented and reward given randomly (+ve prediction error) Reward predicted and reward given (no prediction error) Reward predicted but no reward given (-ve prediction error)

14 Reward Prediction Error Note: Unlike contingency which looked at the response following the CS, here we are looking at the response following the reward and comparing it to that predicted by the CS No CS presented and reward given randomly (+ve prediction error) Reward predicted and reward given (no prediction error) Reward predicted but no reward given (-ve prediction error)

15 Neurophysiological Basis of Reward Error Prediction Reward Greater then Expected = response in DA neuron Reward Expected = no change in DA neuron response Reward Less then Expected = in DA neuron response

16 So, YES, it appears as though DA neurons satisfy the three criteria of learning theory (i.e., contiguity, contingency, reward prediction error) during Pavlovian (classical) conditioning. What is the role of DA in operant (trial and error) conditioning?

17 DA & Operant Conditioning Discrimination Learning Task Learning curves for familiar and new stimuli Two stimuli were presented for ~60 trials Randomized position between left and right side of screen One stimulus was associated with reward the other not Hollerman and Schultz, 1998

18 DA & Operant Condition Fast learning session Slow learning session Hollerman and Schultz, 1998

19 DA & Operant Conditioning DA responses to the temporal prediction of rewards Expected time b/w lever and reward Hollerman and Schultz, 1998