NEED RECURRENT GATED DIPOLE TO EXPLAIN SECONDARY INHIBITORY CONDITIONING 1. Pair Onset of CS 1 with shock READ (I) CIRCUIT

Transcription

1 NEED RECURRENT GATED DIPOLE TO EXPLAIN SECONDARY INHIBITORY CONDITIONING 1. Pair Onset of with shock Fear 2. Pair Onset of with offset of Relief ON OFF + GATE + READ (I) CIRCUIT Grossberg & Schmajuk (1987) LTM GATES REcurrent Associative Diople 1

2 CONDITIONING PARADIGMS Primary Excitatory Primary Inhibitory Secondary Excitatory Secondary Inhibitory Partial PRIMARY EXCITATORY CONDITIONING Aquisition No passive forgetting US Extinction NO US US Read-out No passive forgetting ON OFF LTM (ON) 2

3 PRIMARY INHIBITORY CONDITIONING becomes a "conditioned inhibitor" Aquisition US Extinction NO US US REBOUND Read-out ON OFF LTM (ON) SECONDARY EXCITATORY CONDITIONING -US - LTM (ON-Primary) LTM (ON-Secondary) 3

4 SECONDARY INHIBITORY CONDITIONING -US - OFF LTM (ON-Primary) LTM (OFF-Secondary) READ I EQUATIONS 4

5 ASSOCIATIVE LEARNING FORGETTING e.g., Remember childhood experiences Forgetting is NOT PASSIVE Forgetting is SELECTIVE Learn Remember Learn Forget SELECTIVE: Larger memory capacity PROBLEM: Why doesn't memory SATURATE? Learn SELECTIVE FORGETTING Not "just" gated memory decay GATED STEEPEST DESCENT x i z ij x i STM LTM STM d dt z ij= f(x i ) -z ij +g(x j ) >0 0 d dt z ij= g(x j ) -z ij +f(x i ) NECESSARY, BUT NOT SUFFICIENT 5

6 EXPERIMENTAL SUPPORT Levy et al., Singer et al., Hebbian Anti-Hebbian An early NN prediction DISCONFIRMED EXPECTATIONS LEAD TO SELECTIVE FORGETTING 6

7 PAVLOVIAN CONDITIONING Forgetting = Extinction = Conditioned Stimulus (Bell, Light, ) US = Unconditioned Stimulus (Shock, Food, ) MAIN FACT ABOUT FORGETTING A conditioned excitor extinguishes A conditioned inhibitor does not extinguish DeVito, 1980 Owren and Kaplan, 1981 Witcher, 1978 Zimmer-hart and Rescorla, 1974 NOT YET EXPLAINED BY OTHER MODELS CONDITIONED EXCITOR EXTINGUISHES LEARNING PHASE FORGETTING PHASE (BELL) US (BELL) FORGETTING FEAR(-) LET US = SHOCK 7

8 CONDITIONED INHIBITOR DOES NOT EXTINGUISH LEARNING PHASE FORGETTING PHASE 1) (LIGHT) SHOCK FEAR (-) 2) + (BELL) NO SHOCK RELIEF(+) (BELL) NO FORGETTING is relevant; it predicts US change SAME could be used! SAME "teacher" in forgetting phase! Something else must be going on, or else CAUSALITY would be violated! CONDITIONED EXCITOR EXTINGUISHES LEARNING PHASE FORGETTING PHASE (BELL) US (BELL) FORGETTING FEAR(-) SHOCK is conditioned to an expectation of shock THE EXPECTATION OF SHOCK IS DISCONFIRMED 8

9 CONDITIONED INHIBITOR DOES NOT EXTINGUISH LEARNING PHASE FORGETTING PHASE 1) (LIGHT) SHOCK FEAR (-) 2) + (BELL) NO SHOCK RELIEF(+) NO SHOCK (BELL) NO FORGETTING THE EXPECTATION THAT NO SHOCK FOLLOWS IS NOT DISCONFIRMED! CONCLUSIONS 1. No passive forgetting (inhibitor) NO SHOCK NO SHOCK RELIEF 2. Nonoccurrence of EXPECTED shock causes active forgetting (excitor) SHOCK NO SHOCK FEAR DISCONFIRMED EXPECTATION FORGETTING 9

10 DISCONFIRMED EXPECTATION = EXPLICIT TEACHER At the moment of unexpected disconfirmation the correct answer is unknown to the system NOVELTY Learn sensory expectations (1) PARALLEL LEARNING PROCESSES NO ONE "ENGRAM Sensory representation of US Learn conditioned reinforcer (2) Emotional representation of US Disconfirmation of (1) forgetting of (2) (1) recognition learning (2) reinforcement learning 10

11 WHAT ARE THE MECHANISMS? I. ANTAGONISTIC REBOUND DISCONFIRMED EXPECTATION Nonoccurrence of expected punishment (-) relief (relaxation) (+) Nonoccurrence of expected reward (+) frustration (-) SUDDEN REDUCTION OF REINFORCING EVENT Sudden reduction of punishment (-) relief (+) Sudden reduction of reward (+) frustration (-) II. OPPONENT EXTINCTION Grossberg (1975) Grossberg and Schmajuk (1987) Selective forgetting = opponent extinction No passive forgetting No associative saturation Disconfirmed expectation active forgetting 11

12 A CAN BE CONDITIONED TO BOTH THE ON AND OFF CHANNELS OF A GATED DIPOLE Gated dipole Competition LTM ON "fear" OFF "relief" LEARN ON-RESPONSE ON Disconfirmed expectation antagonistic rebound OFF-channel is conditioned Zero net output ON OFF What about ASSOCIATIVE SATURATION? 12

13 LEARN NET DIPOLE OUTPUT PATTERN Opponent "decision" controls learning Cf.,competitive learning Learning signal ON OFF Opponent extinction DISSOCIATION OF LTM READ-OUT AND READ-IN LTM Dendritic Spines Rall (1960's) Opponent Competition Teaching Signal retrograde action potential Old Predictions Ca ++ currents in learning (1968) Role of dendritic spines in learning (1975) Cf., recent work of Hasselmo 13

14 READ (I) CIRCUIT Grossberg & Schmajuk (1987) LTM GATES REcurrent Associative Diople LEARNING AND MISMATCH OF SENSORY EXPECTATION Adaptive Resonance Theory Forgetting controlled by 2 interacting ART systems (1) Recognition learning (1976) (2) Recognition-reinforcement learning (1975) Sensory mismatch in (1) Nonspecific arousal burst Antagonistic rebound in (2) Opponent extinction (forgetting) 14

15 SAME EXPLANATION FOR UNBLOCKING! Conditioned Inhibitor: + No shock Unblocking: + US 2 ( US 1 ) Relief if US 2 < US 1 Fear if US 2 > US 1 Also need: Unexpected event ( ) is stored more actively in STM due to STM reset by expectancy mismatch FOCUS ON HOW ADAPTIVE TIMING INTERACTS WITH RECOGNITION, REINFORCEMENT, & ACTION Role of Hippocampal formation (dendrate granule cells, CA3 pyramid cells, NMDA receptors) Outline Behavioral problems Computational principles Local neural circuits Global neural architecture 15

16 CONNECTION WITH RECOGNITION LEARNING ADAPTIVE RESONANCE THEORY (Carpenter & Grossberg, 1987) 16