ccopyright by June-Seek Choi 1999 All Rights Reserved

Transcription

1 CLASSICAL EYEBLINK CONDITIONING WITH MIXED INTERSTIMULUS INTERVALS: TEMPORAL INTEGRATION OF RESPONSE TOPOGRAPHY AND NEURONAL CORRELATES A Dissertation Presented by JUNE-SEEK CHOI University of Massachusetts Amherst in partial fulllment of the requirements for the degree of DOCTOR OF PHILOSOPHY September 1999 Neuroscience and Behavior Program

2 ccopyright by June-Seek Choi 1999 All Rights Reserved

3 CLASSICAL EYEBLINK CONDITIONING WITH MIXED INTERSTIMULUS INTERVALS: RESPONSE TOPOGRAPHY AND NEURONAL CORRELATES A Dissertation Presented by JUNE-SEEK CHOI Approved as to style and content by: John W. Moore, Chair Andrew G. Barto, Member Neil E. Berthier, Member Gordon A. Wyse, Member Katherine V. Fite, Program Head Neuroscience and Behavior Program

4 ACKNOWLEDGMENT I thank my mentor, Professor John Moore for years of caring and teaching. He invited me to this continent and helped me through the journey with patience and discipline. I am indebted to him for my growth as a scientist. I thank Professors Andrew G. Barto, Neil E. Berthier and Gordon A. Wyse for their guidance and advice in this project. I am grateful to Marcy Roseneld for her love and voluntary care of the animals day and night. I also like to thank Kevin for his friendship and advice. I thank my parents and my wife Hee-Ryoung for their love and trust in me. This is probably too small and too incomplete to be called an achievement, but I owe them most of it if there is any. 1 1 The research described in this dissertation was partially supported by grant MH57893, J. W. Moore PI. iv

5 ABSTRACT CLASSICAL EYEBLINK CONDITIONING WITH MIXED INTERSTIMULUS INTERVALS: TEMPORAL INTEGRATION OF RESPONSE TOPOGRAPHY AND NEURONAL CORRELATES JUNE-SEEK CHOI, B.S., SOGANG UNIVERSITY M.A., KOREA UNIVERSITY Ph.D., UNIVERSITY OF MASSACHUSETTS AMHERST Directed by: Professor John W. Moore The purpose of this thesis was to investigate the ring pattern of single neurons in cerebellar nucleus interpositus (NI) related to a conditioned response (CR) instilled by conditioning with mixed interstimulus intervals (ISIs). The conditioning with mixed ISIs is a protocol of classical conditioning that involves two dierent interstimulus intervals (ISIs) alternating randomly. The conditioned stimulus (CS) is a tone and the unconditioned stimulus (US) is an electrical shock to periorbital region that causes eyeblink. This protocol resulted in bimodal responses with peaks corresponding to the timing of the US. A related purpose of this investigation was to explore mechanisms of CR timing and temporal integration using a computational method. Sutton and Barto's (1990) temporal dierence (TD) model with a complete serial compound (CSC) representation of CS was used to implement the idea of a neuron-like processing element that receives time-tagged inputs and adjusts their associative strength to generate an appropriate output adaptive to the given conditioning environment. The TD model v

6 with CSC representation of time can be aligned with the cerebellum. Within this context, there are several scenarios as to how time is segmented in the cerebellum and how this information is integrated to produce the CR. The current investigation presents evidence that single neurons of NI express a ring pattern closely related to the bimodal response. All forty-two CR-related units recorded in NI showed neuronal activity closely related to the time-course of eyeblink CRs, i.e. a neuronal activity pattern with two distinctive increases in ring rate. Most of the units preceded the behavioral response but the degree by which the neuronal activity preceded the behavioral response varied. Among the 42 CR-related units, 9 units responded to the tone CS with short latency(<100 ms), CS-locked activity. Among twenty four units tested on a US-only trial, 22 units increased ring rate or remained at the same level, and 2 units decreased ring rate after US presentation. The CR topography on short-isi reinforced trials was unimodal implying that the US has become a conditioned inhibitor. The corresponding neuronal activity ofsingle neurons were also unimodal. The TD (CSC) model and its cerebellar implementation could account for the suppression of the second peak by employing a US-initiated timing cascade. The suppression of the neuronal activity on short-isi trials suggests that the excitation and the inhibition could be expressed at the level of single neuron. vi

7 TABLE OF CONTENTS LIST OF TABLES : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : x LIST OF FIGURES : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : xi Chapter 1. INTRODUCTION : : : : : : : : : : : : : : : : : : : : : : : : : : : Classical eyeblink conditioning : : : : : : : : : : : : : : : : : : Eyeblink conditioning in rabbits : : : : : : : : : : : : : : Cerebellar plasticity : : : : : : : : : : : : : : : : : : : : : : : : Eyeblink conditioning and CR timing and topography : : : : : Conditioning with mixed-isi : : : : : : : : : : : : : : : : Two approaches to the problem of temporal integration : The TD model : : : : : : : : : : : : : : : : : : : : : : : The TD (CSC) model : : : : : : : : : : : : : : : : : : : : : : : The TD (CSC) model: Formal representation : : : : : : The TD (CSC) model: A computational example : : : : Cerebellar implementation of the TD (CSC) model : : : : : : : Temporal integration in mixed-isi conditioning : : : : : METHODS : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Computer simulation of classical conditioning with the TD (CSC) model : : : : : : : : : : : : : : : : : : : : : : : : : : : Recording from the cerebellum : : : : : : : : : : : : : : : : : : Subject : : : : : : : : : : : : : : : : : : : : : : : : : : : Behavioral training : : : : : : : : : : : : : : : : : : : : : Surgery : : : : : : : : : : : : : : : : : : : : : : : : : : : 30 vii

8 2.2.4 Recording : : : : : : : : : : : : : : : : : : : : : : : : : : Histology : : : : : : : : : : : : : : : : : : : : : : : : : : Data analysis : : : : : : : : : : : : : : : : : : : : : : : : : : : : Digitization : : : : : : : : : : : : : : : : : : : : : : : : : Cross-correlational analyses : : : : : : : : : : : : : : : : Average response topography and peri-stimulus time histogram (PSTH) : : : : : : : : : : : : : Peri-response time histogram (PRTH) : : : : : : Lead time estimation using inection points : : : : : : : Template matching analysis : : : : : : : : : : : : : : : : RESULTS : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Overview : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : TD (CSC) model simulations and predictions : : : : : : : : : : Overview : : : : : : : : : : : : : : : : : : : : : : : : : : Conditioning under single-isi with xed-duration CS (delay conditioning) : : : : : : : : : : : : : : : : : : : : : Conditioning under single-isi with xed-duration CS (trace conditioning) : : : : : : : : : : : : : : : : : : : : Simulations of mixed-isi conditioning procedure : : : : : Conditioning with mixed ISIs: US as a conditioned inhibitor : : : : : : : : : : : : : : : : : : Mixed-ISI xed-cs conditioning : : : : : : : : : Mixed-ISI variable-cs conditioning : : : : : : : Comparison of behavioral data with simulation data : : : : : : Bimodal response : : : : : : : : : : : : : : : : : : : : : : US as conditioned inhibitor : : : : : : : : : : : : : : : : Decrease in response latency : : : : : : : : : : : : : : : : Summary of simulation and behavioral data : : : : : : : Recording : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 65 viii

9 3.4.1 Selection of the units : : : : : : : : : : : : : : : : : : : : Histology : : : : : : : : : : : : : : : : : : : : : : : : : : Bimodal CRs and neuronal activity : : : : : : : : : : : : Temporal relationship between CRs and neuronal activity Analysis of CS-locked, short latency activity : : : : : : : Analysis of US-related activity : : : : : : : : : : : : : : Eect of the US on the CR : : : : : : : : : : : : : : : : Activity of adjacent cells : : : : : : : : : : : : : : : : : : Eect of marking lesion : : : : : : : : : : : : : : : : : : DISCUSSION : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Summary : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Implications for cerebellar theories of CR learning : : : : : : : Neural plasticity inni : : : : : : : : : : : : : : : : : : : : : : : The scenarios revisited : : : : : : : : : : : : : : : : : : : : : : : US as a conditioned inhibitor : : : : : : : : : : : : : : : : : : : Temporal integration in the aerents to the cerebellum : : : : : Implications for delay line structure : : : : : : : : : : : : : : : 114 Appendix A. REAL-TIME MODELS AND REPRESENTATION OF TIM- ING TRACE : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 115 B. NEURAL SUBSTRATE UNDERLYING EYEBLINK CONDI- TIONING : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 118 C. THE TD MODEL : : : : : : : : : : : : : : : : : : : : : : : : : : : : 122 C.1 The RW model : : : : : : : : : : : : : : : : : : : : : : : : : : : 122 C.2 _ Y models : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 124 C.2.1 The TD model : : : : : : : : : : : : : : : : : : : : : : : 127 D. AN EXAMPLE PROGRAM CODE USED IN SIMULATIONS OF THE TD (CSC) MODEL : : : : : : : : : : : : : : : : : : : : 129 E. SIMULATION PARAMETERS : : : : : : : : : : : : : : : : : : : 148 F. EYEBLINK CRS AND PSTHS OF ALL RECORDED UNITS 150 ix

10 LIST OF TABLES 3.1 Summary of analysis of unit activity recorded from the cerebellum : Summary of analysis of unit activity recorded from the cerebellum : : 78 E.1 Parameters used in simulations : : : : : : : : : : : : : : : : : : : : : 149 x

11 LIST OF FIGURES 1.1 Representation of synaptic encoding of CSs. A. A neuron-like processing element receiving multiple CSs. Dierent synapses encode dierent CSs. B. The CSC representative of a single CS. Dierent synapses encode time-tagged serial components of the CS. : : : : : : : : : : : : : : : : : : Dynamics of the TD (CSC) model. Trial 1. CS is presented for the rst time. t = 1. The rst serial component CS is activated. t = 2. The second serial component is activated. The rst component is no longer active, but it is eligible for synaptic modication.t =3.The third serial component is activated, and the US occurs. Because it is eligible, the second component increases its synaptic weight. t = 4. The fourth component isactivated. Though it is eligible, the third component does not undergo modication. The lower panels show the predicted output before and after Trial 1. : : : : : : : : : : : : : : : : : : : : : : : : : : : Dynamics of the TD (CSC) model. Trial 2. CS is presented a second time. t = 1. The rst serial component CS is activated. t = 2. The second serial component isactivated, producing a CR, and the rst component undergoes synaptic enhancement because it is eligible. t = 3. The third serial component is activated, and the US occurs. The second component undergoes synaptic enhancement because it is eligible. t = 4. The fourth component is activated. The lower panels show the predicted output before and after Trial 2. : : : : : : : : : : : : : : : : : Cerebellar and brain stem circuits underlying eyeblink conditioning (after Roseneld and Moore, 1995). See text for details. : : : : : : : : : : : : The complete TD (CSC) implementation showing three sequentially activated tapped-delay components of CS. : : : : : : : : : : : : : : : : : : : Three scenarios for temporal integration of CRs in the cerebellum. A. The cerebellum expresses temporal integration. B. Cerebellar cortex expresses temporal dierentiation. Temporal integration is expressed at the level of cerebellar deep nucleus (NI). C. Temporal integration is expressed at the level of single cerebellar PCs. : : : : : : : : : : : : : : : Illustration of the template analysis method. A template based on nomodulation hypothesis is compared against the other based on modulation hypothesis. : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 37 xi

12 3.1 Simulated response topographies, Y(t), with various values of and. Dotted-line box represents the US. : : : : : : : : : : : : : : : : : : : : : Simulation of complex response topography by altering the stimulus synchrony after trace conditioning. Top Stimuli conguration. Middle CR topography tothecs. Bottom CR topography toalongcs. : : : : : : Simulated CRs in mixed-isi xed-cs conditioning procedure. Top panel shows stimuli conguration, and panels A, B, and C show CRs on probe trial (A) and two reinforced trials (B, C). Right panels of A, B and C are underlying CS onset (white bars) and the US processes (black bars). Arrows in B and C mark the US. : : : : : : : : : : : : : : : : : : : : : : Simulation of xed-cs conditioning with mixed ISI. : : : : : : : : : : : : Simulation of mixed-isi procedure used in Millenson et al. (1977). : : : Averaged eyeblink responses. Top panel, Averaged response to 300-ms CS. Bottom panel, Averaged response topography to 500-ms CS. Black horizontal bars represent CS duration. : : : : : : : : : : : : : : : : : : : Distribution of latencies of response peaks. : : : : : : : : : : : : : : : : : Response topographies on short-isi reinforced trials at dierent stages of training. Each panel is composed of average response topographies from individual rabbits (dotted line) and the average response topography of all the rabbits included in the gure (solid line). Data from 12 rabbits were plotted. In each panel, the black bar represents the CS and the arrow represents the US. : : : : : : : : : : : : : : : : : : : : : : : : : : : Simulated bimodal responses over 500 trials. : : : : : : : : : : : : : : : : Conditioned bimodal responses on long-isi trials over 14 sessions. Each session is composed of 80 trials. Eyelid position after the US onset have not been plotted since it is contaminated by the UR. Responses were averaged over 12 rabbits. : : : : : : : : : : : : : : : : : : : : : : : : : : : Reconstruction of cerebellar sections with anatomical landmarks. Abbreviations: BC: brachium conjunctivum, NIa: anterior nucleus interpositus, VN: vestibular nucleus, LC: nucleus coeruleus, IO: inferior olive, ml: medial lemniscus, Tz: nucleus of trapezoidal body, SO: superior olive, 7: facial nerve, NIp: posterior nucleus interpositus, ND: dentate nucleus, NF: fastigial nucleus, icp: inferior cerebellar peduncle. : : : : : : : : : : Reconstruction of recording sites. A double circle represents the site where eyeblinks were observed when making a marking lesion. : : : : : : : : : : Reconstruction of recording sites-continued. : : : : : : : : : : : : : : : : : Reconstruction of recording sites-continued. : : : : : : : : : : : : : : : : : 70 xii

13 3.15 Raster plots and PSTH of a recorded unit (Unit 24). A A CR and raw spike activity on a single trial. Black bar represents the CS. The unit shows a ring pattern highly related to the CR. Inside the box are two representative spike waveforms. B Raster plots and PSTH. Individual response and raster plots are arranged according to the order of CR onset. Top trace is averaged eyelid position. Scale bar = 100 ms. Bin size = 10 ms. : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Raster plots and PSTH of a recorded unit (Unit 47). A A CR and raw spike activity on a single trial. Black bar represents the CS. The unit shows a ring pattern not related to the CR. Inside the box are two representative spike waveforms. B Raster plots and PSTH. Individual response and raster plots are arranged according to the order of CR onset. Top trace is averaged eyelid position. Scale bar = 100 ms. Bin size = 10 ms. : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : A CR-locked unit (Unit 19). Top trace Eyelid position synchronized on the onset. Top trace is an averaged eyelid position of the synchronized response topography. Middle trace A PRTH with raster plots. Bottom trace A CUSUM. : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Average cross-correlograms using unsynchronized average response topography and PSTH (black circle) and synchronized average response topography and PRTH (black square). Two vertical dotted-lines indicate the mode of each correlogram. Error bars represent standard errors. : : : correlation between tau's computed by dierent method : : : : : : : : : : Leadtime determined by the rst and the second peak : : : : : : : : : : : Frequency histogram of lead time distributions : : : : : : : : : : : : : : : Raster plots and PSTH of a recorded unit (Unit 5). A Eyelid position and neuronal response on a single trial. Black bar represents the CS. The unit shows a ring pattern time-locked to the CS onset. Inside the box are two representative spike waveforms. An asterisk (*) marks the onset of CS-locked activity. B Raster plots and PSTH. Individual response and raster plots are arranged according to the order of CR onset. Top trace is an averaged eyelid position. Scale bar = 100 ms. Bin size = 10 ms. : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Responses to the US. A-B Units showing an evoked response to the US. The left trace is eyelid position and neuronal response on a single trial. The right trace is a blown-up version of the left trace. The arrow marks the US and the square marks the rst burst of spiking after the US. : : : Frequency histogram of the ratio of pre- and post-us neuronal activity. : 91 xiii

14 3.25Average response topography and PSTH on probe and reinforced trials with 312-ms ISI (Unit 7). Top Average response topography. Bottom PSTH from probe trials (white bar) and from 312-ms ISI reinforced trials (black bar). The arrow marks the US on the reinforced trial. : : : : : : : Average response topography and PSTH on probe and reinforced trials with 312-ms ISI (Unit 25). Top Average response topography. Bottom PSTH from probe trials (white bar) and from 312-ms ISI reinforced trials (black bar). The arrow marks the US on the reinforced trial. : : : : : : : Raster plots and PSTH of a recorded unit (Unit 20). A ACRandraw spike activity on a single trial. Black bar represents the CS. The unit pauses the ring in response to the tone. Inside the box are two representative spike waveforms. B Raster plots and PSTH. Individual response and raster plots are arranged according to the order of CR onset. Top trace is an averaged eyelid position. Scale bar = 100 ms. Bin size =10ms. : : Raster plots and PSTH of a recorded unit (Unit 27). A ACRandraw spike activity on a single trial. The unit shows a ring pattern related to the CR as well as a CS-locked activity. Inside the box are two representative spike waveforms. B Raster plots and PSTH. Individual response and raster plots are arranged according to the order of CR onset. Top trace is an averaged eyelid position. Black bar represents the CS. Scale bar = 100 ms. Binsize=10 ms. : : : : : : : : : : : : : : : : : : : : : : : : : : CR rate and response topography after a small lesion. For a exact location of the lesion, see Fig The asterisk (*) marks the session when the lesion was made. : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Raster plots and PSTH of a recorded unit (Unit 29). The CRs and neuronal activity of this unit was recorded four sessions after a marking lesion was made on Unit 27. A A CR and raw spike activity on a single trial. The unit displays predominantly unimodal response pattern. Inside the box are two representative spike waveforms. B Raster plots and PSTH. Individual response and raster plots are arranged according to the order of CR onset. Toptraceisanaveraged eyelid position. Black bar represents the CS. Scale bar = 100 ms. Bin size =10ms. : : : : : : : : : : : : : : US responsiveness as a function of CR-relatedness. : : : : : : : : : : : : : Three scenarios for temporal integration of CRs in the cerebellum. A. The cerebellum expresses temporal integration. B. Cerebellar cortex expresses temporal dierentiation. Temporal integration is expressed at the level of cerebellar deep nucleus (NI). C. Temporal integration is expressed at the level of single cerebellar PCs. : : : : : : : : : : : : : : : 108 xiv

15 4.3 A recording from a unit in cerebellar cortex showing CR-related decrease in ring A. Spike prole and eyeblink CR on a single probe trial. B. Eyeblink CRs, raster plot and PSTH on probe trials. C. Eyeblink CRs, raster plot and PSTH on the reinforced trials with 300-ms ISI. D. Eyeblink CRs, raster plot and PSTH on the reinforced trials with 700-ms ISI. Top trace in each panel is an averaged eyeblink response. Two vertical lines represent the CS. The triangle in C and D represents the US. : : : A recording from a unit in cerebellar cortex showing CR-related increase in ring A. Spike prole and eyeblink CR on a single probe trial. B. Eyeblink CRs, raster plot and PSTH on probe trials. C. Eyeblink CRs, raster plot and PSTH on the reinforced trials with 300-ms ISI. D. Eyeblink CRs, raster plot and PSTH on the reinforced trials with 700-ms ISI. Top trace in each panel is an averaged eyeblink response. Two vertical lines represent the CS. The triangle in C and D represents the US. : : : Relations between latency of rst and second amplitude peaks. : : : : : : 114 B.1 Schematic diagram showing cerebellar and brainstem structures identied as essential to eyeblink conditioning in rabbits. Abbreviations are: PC, Purkinje cell NI, nucleus interpositus CTX, cerebellar cortex PF, parallel ber MF, mossy ber CF, climbing ber PC, Purkinje cell St, stellate cell Ba, basket cell Go, Golgi cell gr, granule cell MN, motor nuclei RN, red nucleus PN, pontine nucleus Sp5, spinal trigeminal nucleus Coch, cochlear nucleus IO, inferior olive (+), excitatory synapse (;), inhibitory synapse. Modied from Roseneld and Moore (1995) Kim and Thompson (1997). : : : : : : : : : : : : : : : : : : : : : : : : : 120 F.1 Peristimulus histograms and rasterplots : : : : : : : : : : : : : : : : : : : 151 F.2 Peristimulus histograms and rasterplots{continued : : : : : : : : : : : : : 152 F.3 Peristimulus histograms and rasterplots{continued : : : : : : : : : : : : : 153 F.4 Peristimulus histograms and rasterplots{continued : : : : : : : : : : : : : 154 F.5 Peristimulus histograms and rasterplots{continued : : : : : : : : : : : : : 155 F.6 Peristimulus histograms and rasterplots{continued : : : : : : : : : : : : : 156 F.7 Peristimulus histograms and rasterplots{continued : : : : : : : : : : : : : 157 F.8 Peristimulus histograms and rasterplots{continued : : : : : : : : : : : : : 158 F.9 Peristimulus histograms and rasterplots{continued : : : : : : : : : : : : : 159 F.10Peristimulus histograms and rasterplots{continued : : : : : : : : : : : : : 160 F.11Peristimulus histograms and rasterplots{continued : : : : : : : : : : : : : 161 F.12Peristimulus histograms and rasterplots{continued : : : : : : : : : : : : : 162 xv

16 C H A P T E R 1 INTRODUCTION The primary purpose of the current study was to investigate the ring pattern of single neurons of the cerebellum related to conditioned eyeblink responses in rabbits acquired under a mixed-interstimulus interval (mixed-isi) procedure. Mixed-ISI conditioning involves training with two randomly alternating ISIs. When the two ISIs are suciently dierent (e.g., 300 ms vs. 700 ms), the conditioned response (CR) becomes bimodal with distinctive amplitude peaks timed at the temporal loci of the two times of unconditioned stimulus (US) occurrence. Using mixed-isi conditioning, the current study addressed the question of how the brain integrates experience with two ISIs to generate a CR with a bimodal waveforms. Here I present evidence suggesting that the activity of single neurons in an area known to be critical for acquisition and execution of eyeblink CRs, the nucleus interpositus (NI), is closely related to the full complexity of a bimodal response topography learned with mixed-isi conditioning. The data are consistent with several contemporary models of classical conditioning and the cerebellum (e.g., Bullock, Fiala, and Grossberg, 1994 Desmond and Moore, 1988 Gluck, Reifsnider, and Thompson, 1990 Mauk and Donegan, 1997 Moore and Choi, 1997a). Among them, the 1990 Sutton and Barto's temporal dif- 1

17 ference model with complete serial compound representation (TD (CSC)) of conditioned stimulus (CS) and its cerebellar implementation (Moore and Choi, 1997a,b, 1998 Moore, Choi, and Brunzell, 1998), provide a framework for understanding how complex CR topographies are represented in the cerebellum. Conditioned response waveform obtained with mixed-isi conditioning were contrasted with simulations of the model. 1.1 Classical eyeblink conditioning Classical Pavlovian conditioning is one of the most widely used forms of associative learning and found in almost all animal species ranging from planaria to human (For an example, see Gormezano and Moore, 1969). Some researchers believe thatby studying the neural mechanism underlying classical conditioning, it is possible to nd a basic mechanism of learning shared across phylogeny. In a typical demonstration of classical conditioning, the CS is repeatedly paired with a US. The US is usually a stimulus with biological signicance suchasanoxious stimulus or food. TheUSelicits a reexive response, an unconditioned response (UR). A CR emerges after multiple pairings of the CS and the US. Moore, Desmond, and Berthier (1989) described the development of an eyeblink CR as follows. The latency of the CR with respect to onset of a CS changes during training. Initially the CR appears as an enhanced UR. The CR appears just prior to the onset of the US at this stage. With further 2

18 training, the CR emerges as a lengthening shadow of the UR cast forward in time toward CS onset. The latency of the CR depends on the ISI. If this interval is long, the CR is delayed. If the temporal locus of the US changes, the peak amplitude of the CR shifts to the new US locus. In trace conditioning protocols, in which CS oset precedes the US, CR initiation and peak amplitude tend to occur within the trace portion of the ISI at the same temporal locus as in forward-delay conditioning protocols. Eyeblink conditioning has been a popular paradigm of classical conditioning for several decades (Hilgard, 1936). In eyeblink conditioning, the CS (usually a simple sensory cue such as a tone or a light) is repeatedly paired with the US (usually an airpu to the cornea or mild electric shock to the periorbital region) which causes defensive blink. A CR emerges after multiple pairings of the CS and the US. The CR is predictive because it is initiated by the CS and precedes the US in time. Because the UR can interfere with the CR, in experiments, CS-only probe trials are intermittently presented with reinforced trials. Eyeblink conditioning has been used in testing and developing theories of learning (e.g., Moore and Gormezano, 1977), as a model system to nd a neural basis of learning and memory (e.g., Thompson, 1986) and in diagnosing human brain disease such as Alzheimer's disease (Woodru-Pak, Papka, Romano, and Li, 1996). 3

19 1.1.1 Eyeblink conditioning in rabbits A defensive eyeblink in rabbits is a group of responses that include globe retraction, nictitating membrane (NM) sweep, and eyelid closure. Several discrete muscles are involved in the eyeblink reex including obicularis oculi (OO), which causes eyelid closure, retractor bulbi, recti and oblique muscles, which cause a retraction of a globe, accompanied by apassive sweep of NM (Berthier and Moore, 1980). The nictitating membrane is sometimes called the third membrane which exists in most mammals except for primates and humans. The nictitating membrane response has been a popular measure of eyeblink CR for many species including rabbits. On the other hand, eyelid movement has been used in more recent eyeblink conditioning literature because it is easier to measure and allows direct comparisons with humans. 1 The cerebellum, which has been known to be critical for rabbit eyeblink conditioning, has been implicated in eyeblink conditioning in humans (Logan and Grafton, 1995 Molchan, Sunderland, McIntosh, Herscovitch, and Schreurs, 1994). 1.2 Cerebellar plasticity Accumulating evidence indicates that the cerebellum is a brain structure where the essential plasticity for eyeblink conditioning is formed and expressed. A brief 1 It has been shown that the obicularis oculi muscle controlling the eyelid has tight coupling with the retractor bulbi muscle controlling the NM and the time shift between two movements is minimal (Berthier, 1992). The two responses are used interchangeably in conditioning literature. 4

20 summary on neural circuit critical for eyeblink CRs is attached in Appendix A and Fig. B.1. An alternative view is that the cerebellum plays a regulatory role in the formation of the CR, and lesions of the cerebellum aect the performance of the CR but do not abolish the learning itself. In one study in which aweak airpu was used as the US, Welsh and Harvey (1989) showed that lesions of anterior interpositus nucleus (NIa), which has been known to be critical for eyeblink CRs, decreased the amplitude of the UR as well as that of the CR. This result implies that the cerebellar damage caused a general motor decit rather than specic memory loss. However, experiments from several laboratories using similar manipulations of NI showed results that contradict Welsh and Harvey (1989) (Steinmetz and Sengelaub, 1992 Yeo and Hardiman, 1992). Kelly, Zuo, and Bloedel (1990) showed that CR could be acquired in decerebrate rabbits without the cerebellum (But see Yeo, 1991) (For a recent review on this subject, see Yeo and Hesslow, 1998). Given that the learning occurs in the cerebellum, there has been some controversy about the relative importance of cerebellar cortex. Lesions of the cerebellar cortex, especially HVI in rabbits, completely abolished the CRs and prevented further learning in one study (e.g., Yeo, Hardiman, and Glickstein, 1985a) but not in another (e.g., Lavond and Steinmetz, 1989). One explanation for this discrepancy is that the completeness of the lesion diers slightly between researchers as there are several dierent subdivisions of cerebellar cortex known to be related to the CR, such as lobule HV or HVII which border HVI. Studies using genetically engineered 5

21 mutant mice with specic deciencies in Purkinje cells (PCs), showed impaired CR acquisition (e.g., Chen, Bao, Lockard, Kim, and Thompson, 1996 Shibuki, Gomi, Chen, Bao, Kim, Wakatsuki, Fujisaki, Ikeda, Chen, Thompson, and Itohara, 1996). These studies support the argument that cerebellar cortex is essential for learning eyeblink CR. Another possible explanation about the dierent results of cerebellar cortical lesions has been recently proposed: Conditioning may establish a learning trace in the contralateral cerebellum to a small degree which could survive ipsilateral damage. Gruart and Yeo (1995) showed that when contralateral cerebellum as well as the ipsilateral cerebellum were damaged, the CR was completely abolished and prevented further relearning as opposed to ipsilateral damage which also abolished the CR but could not prevent relearning. There is a much larger degree of agreement about the eect of NI manipulations. The nucleus interpositus is the brain structure that receives major converging inputs from HVI. Lesions of NI disrupt the CR (e.g., Clark et al., 1992 Yeo et al., 1985b). Inactivation of NI abolished learning related activity in the trigeminal complex, but not vice versa (Clark and Lavond, 1996). A similar result was obtained from pontine nucleus region (Clark, Gohl, and Lavond, 1997). CR-related activity in pontine nucleus (PN) region is dependent on NI. Together, these studies suggest that NI is critically involved in the CR-related activity foundinvarious brain structures. Recent experimental results suggest that the cerebellum is involved in CR timing. Lesions in cerebellar cortex disrupt the timing of eyeblink CRs (Perrett, Ruiz, and Mauk, 1993). Several models have been proposed to describe the timing of the CR 6

22 in the cerebellum (e.g., Mauk and Donegan, 1997 Moore, Desmond, and Berthier, 1989). It is essential, therefore, that the role of cerebellar circuit in conditioned timing proposed in these models be tested using a task where the response timing is a distinct feature of the response. 1.3 Eyeblink conditioning and CR timing and topography A prime feature of the conditioned eyeblink response is its dependency on ISI. Interstimulus interval determines the rate of learning. The so-called ISI function has an inverted-u shape. Learning does not occur at an ISI shorter than ms (Schneiderman and Gormezano, 1964). The rate of learning reaches the maximum at an ISI of approximately 250 ms then slowly declines as the ISI increases (Gormezano and Moore, 1969 Schneiderman, 1966 Schneiderman and Gormezano, 1964). 2 As noted in a previous section, ISI determines CR topography. In eyeblink conditioning, response topography is formed such that the latency of CR amplitude peaks coincides with the temporal loci of the US (Schneiderman, 1966 Schneiderman and Gormezano, 1964). If switched to a dierent ISI, the peak latency shifts to the new location of the US (Coleman and Gormezano, 1971). In Coleman and Gormezano (1971), rabbits were conditioned to a CS with 200-ms ISI and then switched to 700-ms 2 It has been known that the ISI function is not absolute. Instead, it is dependent on the distribution of trials within a training session. The intertrial interval (ITI) which is an index of how densely the trials are distributed in the session determines the kurtosis of the ISI curve (e.g., Levinthal, 1973 Levinthal, Tartell, Margolin, and Fishman, 1985). 7

23 ISI. Immediately after being switched to the new ISI, the rabbits showed a bimodal response with two peaks, one on the original ISI and the other on the new ISI. As the training went on, however, the number of CRs timed at the original, short ISI decreased but the number of CRs at the new, longer ISI increased. Eventually, all CRs peaked at the long ISI. Switching from the long ISI to the short ISI resulted in a similar latency shift. Attempts to relate neuronal activity to the timing of behavioral response come from literature on voluntary movement of hand and arm (e.g., van Kan, Houk, and Gibson, 1993). However, most of the studies on voluntary movement are specically concerned with spatial parameters such as position and velocity. Multijoint nature of such movement makes it dicult to analyze temporal parameters. Rabbit eyeblink conditioning is advantageous in investigating timing using various neurobiological methods, because an eyeblink is a one-dimensional movement and therefore the movement is almost exclusively constrained by temporal factors. Therefore, the eect of neurobiological manipulation on the temporal aspect of the response could be more easily assessed Conditioning with mixed-isi My research involves training rabbits with a mixture of two ISIs in order to produce complex CR waveform, waveforms with two amplitude peaks. Hoehler and Leonard (1976) and Millenson, Kehoe, and Gormezano (1977) were the rst who employed this procedure. Millenson et al.'s procedure was one in which there were two ISIs, one at 200 ms and the other at 700 ms. Both trial types in Millenson et 8

24 al.'s procedure were delay conditioning. That is, the CS terminated when the US occurred. They also included CS-only test trials, in order to evaluate CR waveforms uncontaminated by URs. Their ndings are summarized as follows: 1) In mixed-isi training, the CR waveform was bimodal with each peak timed at the corresponding ISI. 2) The second peak of the bimodal response was suppressed on the short-cs duration test trials, presumably because CS oset became a conditioned inhibitor. The conditioning procedure used by Millenson et al. will be called variable- CS mixed-isi conditioning because the CS duration covaries with the ISI. The idea that CS oset becomes a conditioned inhibitor was incorporated into Desmond and Moore's model of response timing (Desmond and Moore, 1988), which can predict the salient features of Millenson et al.'s experiment. In the present experiment, xed-cs mixed-isi conditioning, the CS duration was held constant at 300 ms (Choi, Hirl, and Moore, 1993). In xed-cs mixed-isi procedure, CS oset is not a reliable cue for the occurrence of the US. As a consequence, on CS-only test trials there is no suppression of the second peak. Later on, however, I present data suggesting that the second amplitude peak is suppressed on short-isi reinforced trials. The TD (CSC) model can predict this result by allowing the US to have the signaling properties of a CS Two approaches to the problem of temporal integration How does the brain integrate experience with two ISIs, in mixed-isi conditioning, and express it as a bimodal response topography? One approach to this problem is to use a computational model. Sutton and Barto's (1990) temporal dierence (TD) model with a complete serial compound (CSC) representation of CSs was chosen 9

25 to approach the question of how response topography is learned under mixed-isi conditioning procedure. The TD model was chosen because: 1) it is a general model of predictive timing and action. 2) it has been proven that the model is capable of simulating wide variety of real-time conditioning phenomena including blocking and ISI eects (Sutton and Barto, 1990). 3) it can be aligned with the cerebellar structures implicated in eyeblink conditioning in a relatively straightforward manner (Moore and Choi, 1997a). Another approach to the problem of temporal integration is to examine the neural representation of bimodal CRs in the cerebellum. Recording studies of the cerebellum (Berthier and Moore, 1986, 1990 McCormick and Thompson, 1984) indicate that a learning-related activity is formed and expressed in the cerebellum but did not approach the problem of temporal integration. Using mixed-isi conditioning procedure, the current study investigated not only the existence of learning-related activity in the cerebellum, but also the extent to which the neuronal activity represents the temporal complexity ofbimodal CRs. The two approaches are complementary because the TD (CSC) model can be aligned with cerebellar circuit elements known to be involved in eyeblink conditioning. The cerebellar implementation of the TD (CSC) model proposed by Moore and Choi (1997a), provides a scheme by which the timing of CR is learned and represented in the cerebellum. This implementation scheme provides a direct comparison with the recording data which will be discussed later on. Before describing this scheme, the next session describe the TD model. 10

26 1.3.3 The TD model Barto, Sutton, and Anderson (1983) described a neuron-like element, called the Adaptive Critic Element (ACE), which is a part of two-element connectionist network that can learn to balance a pole on a cart moving on a one-dimensional track. The ACE receives various inputs regarding the state of the system and provides an appropriate evaluation signal to guide the action generated by another element of the system, the Adaptive Search Element (ASE). The ACE receives four inputs to represent the state of the system position of the cart and angular position of the pole and their rst derivatives. The ACE's job is to learn to predict future reinforcement on a given state, using an occasional reinforcement signal: a fall of the pole. The diculty of this task lies in the fact that the actual (negative) reinforcement signal is temporally scarce, and therefore a state being evaluated can be remote from the reinforcement. This diculty can be summarized by a category of problems called credit assignment problem (Sutton, 1984). The credit assignment problem is similar to that proposed by a classical conditioning situation. Figure 1.1A shows one example. Multiple CSs, labeled as CS1 CS2 CS n, converge on a single neuron-like processor. The neuron-like processor needs to learn to assign appropriate connection weights to each CSs. For the processor to function eciently, it must evaluate CSs even on time steps when the US is not present. Therefore, the problem in classical conditioning with multiple CSs is similar to that faced by the ACE. 11

27 Figure 1.1B is a slightly dierent situation where a single CS is divided into multiple components, each one capable of acquiring own connection weights. All CSs are dierent serial components originating from the same CS. Each CS component is dierentiated by time index, t = 1 2 n. This type of spatial representation of a temporal stimuli have been an integral part of Desmond and Moore's model of classical conditioning with multiple delay-lines (Desmond and Moore, 1988). Other models expressed similar ideas of spatial representation of temporal stimulus. Bullock, Fiala, and Grossberg (1994) and Gluck, Reifsnider, and Thompson (1990) proposed that a CS is represented in dierent frequency components of a spectrum of oscillating activity. In Mauk and Donegan (1997), a CS is represented in a series of unique time-varying activity patterns in the cerebellar parallel bers. A brief review of these and other models is attached in Appendix B. The delay-line representation of a CS will be described in more detail with the TD (CSC) model later on. 1.4 The TD (CSC) model One way to represent response topography is to assume that the CS is composed of serial components. Sutton and Barto (1990) showed that the TD model with a CSC representation of CS can generate realistic response topography. In CSC, a stimulusresponse mapping is achieved by assuming that a nominal CS triggers cascades of activation. Elements of these cascades, in turn, gain associative values specic to the timing of the CR. As pointed out by Moore and Choi (1997a), this representation 12

28 A Spatial encoding of CSs on neuron-like processor US CS 1 CS 2 CS 3 V CS1 V CS 2 V CS3 CR V C S n CS n B Temporal encoding of CS on neuron-like processor US CS ( t = 1 ) CS ( t = 2 ) CS ( t = 3 ) V 2 V 3 V 1 CR V n CS ( t = n ) Figure 1.1: Representation of synaptic encoding of CSs. A. A neuron-like processing element receiving multiple CSs. Dierent synapses encode dierentcss. B. The CSC representative of a single CS. Dierent synapses encode time-tagged serial components of the CS. 13

29 of CS is, for all intents and purposes, the same as delay-line inputs of Desmond and Moore's 1988 model. The TD (CSC) model has evolved from a series of contemporary connectionist models such as the Rescorla-Wagner (RW) model (Rescorla and Wagner, 1972), the Sutton-Barto (SB) model (Sutton and Barto, 1981), and the TD model (Sutton and Barto, 1990). A summary of formal descriptions of these models are attached in Appendix C. The TD (CSC) model has been adopted to various applications including simulation of behavior of dopaminergic neurons in operant conditioning (Schultz, Dayan, and Montague, 1997) The TD (CSC) model: Formal representation The TD (CSC) model computations are similar to the TD model computations presented by Eq. C.8 through Eq. C.10 in Appendix C. The dierence is, however, that the representation of a CS is more complicated with added dimensions of onset and oset and the corresponding timing cascades. Therefore, three subscripts are needed to represent a single CSC component. Following equations describe computations of the TD (CSC) model. A connection weight of a single CSC component, V ijk (t), is dened as, V ijk (t +1) = V ijk (t)+[(t) ; (Y (t ; 1) ; Y (t))] i X ijk (t) (1.1) where Y (t) = b X i XX j k V ijk (t)x ijk (t)c: (1.2) 14

30 V ijk (t) is identied by type of CS (i), onset (j =1)oroset(j = 2), and the position within a cascade initiated by the CS onset or oset (k). For example, a V123(t) would mean the connection weight of the third CSC component of the oset of the rst CS. Y (t) is the sum of the output produced by all CSC components that exist on the given time step, t. i is saliency of acs,i. is a learning constant. P is a discount factor. i,, and take values between 0 and 1. The notation b VXc species this constraint: If this quantity is less than 0, it is set equal to 0 (See Sutton & Barto, 1990, p 533). X ijk (t) refers to eligibility for modication. X ijk (t +1) = X ijk (t)+[x ijk (t) ; X ijk (t)]: (1.3) is an eligibility constant which determines the rate of decay of CS trace. Due to, a CS component can undergo synaptic modication on time steps where it no longer exists. Equation 1.1 highlights a key feature in the TD (CSC) model. Changes in connection weights depend on two types of reinforcement: One donated by the US ((t)) and the other by what Sutton and Barto (1990) called Y _ component of reinforcement, Y (t) ; Y (t ; 1). The Y _ component represents the dierence between a prediction of future reinforcement (Y (t)) and the actual reinforcement (Y (t ; 1)). The prediction is discounted by the discount factor,, because Y (t) must be estimated using the connection weights computed on the previous time step, V (t ; 1). 15

31 1.4.2 The TD (CSC) model: A computational example To explain workings of the TD (CSC) model, this section describes a step-by-step computation of the model in a simple delay conditioning task. Figure 1.2 and Fig. 1.3 show operations of the TD (CSC) model in acquiring associative strength of each CSC component. The two gures show the rst and the second trial of the acquisition. The top trace in the gures is a stimulus conguration. The CS is 4 time steps long, and the US is presented on the fourth time step. The middle trace is composed of 4 diagrams showing change in connection weights on each time step. In the diagrams, two dierent representation of CS described above, CSC and tapped-delay line are contrasted. By the CSC representation, the onset of the nominal CS initiates a series of short CSs, CS1 CS2 CS3 andcs4, which have a duration of only one time step. The bottom trace shows the output of the system before and after the given trial. On time-step 1 of the rst trial, CS1 is active. On time-step 2, CS2 is active and CS1 is eligible for synaptic modication. No synaptic change occurs because neither nor Y _ term has any value. On time-step 3, the US is presented and CS 2 which is eligible at the time, gains an increase in synaptic strength by contributed by the US. Notice that CS3 synapse does not change because it is not eligible for the synaptic change. In other words, a CS is not eligible until the next time step after it 16

32 becomes active. 3 By this mechanism, in the TD (CSC) model, the CS immediately preceding the US acquires more synaptic strength that the one that's on the same time step as the US. On time-step 4, CS4 is active and CS3 is eligible. No synaptic change occurs. On the next trial, the acquisition process is similar until it reaches the time-step 2. No synaptic change occurred on time-step 2 of the rst trial. On the second trial, however, CS2 gains some synaptic strength because the output response of the processing unit is not zero, due to the synaptic strength of CS2 gained from the previous trial. Therefore, Y _ is positive and this is converted into an increase in synaptic ecacy of the CS that is eligible at that time, CS1. Second-order conditioning of one serial component of the CS by the next is responsible for the sysnaptic change on this time step. Notice that the synaptic strength increases on CS1 is not as big as CS2 although it could eventually approach the same level of synaptic strength if the discount factor is big enough. On time-step 3, CS2 gains an increase in synaptic strength from again. On time-step 4, no synaptic change occurs. 1.5 Cerebellar implementation of the TD (CSC) model Moore and Choi (1997a) presented a neural implementation of the TD (CSC) model based on cerebellar architecture and neurobiological data. Detailed description 3 Although a CS can remain eligible for an arbitrary amount oftimewhich is determined by in the TD model equation, in the given example it is only eligible for one time step which is the case when equals 1. 17

33 Trial 1 t = 1 CS US t t = 2 inactive synapse active synapse eligible synapse CSC CSC CS on Processing unit CS on Processing unit t = 3 t = 4 CSC CSC CS on Processing unit US CS on Processing unit Before CS After CR t Figure 1.2: Dynamics of the TD (CSC) model. Trial 1. CS is presented for the rst time. t = 1. The rst serial component CS is activated. t = 2. The second serial component is activated. The rst component is no longer active, but it is eligible for synaptic modication.t = 3. The third serial component is activated, and the US occurs. Because it is eligible, the second component increases its synaptic weight. t = 4. The fourth component is activated. Though it is eligible, the third component does not undergo modication. The lower panels show the predicted output before and after Trial 1. 18

34 Trial 2 t = 1 CS US t t = 2 inactive synapse active synapse eligible synapse CSC CSC CS on Processing unit CS on Processing unit t = 3 t = 4 CSC CSC CS on Processing unit US CS on Processing unit Before CS After CR t Figure 1.3: Dynamics of the TD (CSC) model. Trial 2. CS is presented a second time. t = 1. The rst serial component CS is activated. t = 2. The second serial component is activated, producing a CR, and the rst component undergoes synaptic enhancement because it is eligible. t = 3. The third serial component is activated, and the US occurs. The second component undergoes synaptic enhancement because it is eligible. t = 4. The fourth component is activated. The lower panels show the predicted output before and after Trial 2. 19