Climbing fibers encode a temporal-difference prediction error during cerebellar learning in mice

Transcription

1 Climing fiers enoe temporl-ifferene preition error uring ereellr lerning in mie Shogo Ohme & Jvier F Mein npg 25 Nture Ameri, In. All rights reserve. Climing fier inputs to Purkinje ells re thought to e involve in generting the instrutive signls tht rive ereellr lerning. To investigte how these instrutive signls re enoe, we reore the tivity of iniviul liming fiers uring ereellum-epenent eyelink onitioning in mie. We foun tht liming fiers signle oth the unexpete elivery n the unexpete omission of the perioulr irpuff tht serve s the instrutive signl for eyelink onitioning. In ition, we oserve tht liming fiers tivte y perioulr irpuffs lso respone to stimuli from other sensory molities if those stimuli were novel or if they preite tht the perioulr irpuff ws out to e presente. This pttern of liming fier tivity is mrkely similr to the responses of opmine neurons uring reinforement lerning, whih hve een shown to enoe prtiulr type of instrutive signl known s temporl ifferene preition error. Climing fiers originting in the inferior olive projet to the ereellr ortex, where they re thought to provie the instrutive signls neessry for ereellr lerning 2 4. Some of the strongest support for this hypothesis omes from stuies of Pvlovin eyelink onitioning 5, ereellr tsk in whih nimls lern to lose the eyeli in response to onitione stimulus (CS) suh s n light if it is repetely pire with link-eliiting unonitione stimulus (US) suh s perioulr irpuff. Consistent with their presume role s tehers, liming fiers rry signls out the instrutive US in this ssoitive lerning tsk 9. Furthermore, iret eletril stimultion of liming fiers n serve s the US uring onitioning, proviing tehing signl tht is s effetive s perioulr stimultion 2. It hs een suggeste tht the tehing signls trnsmitte y liming fiers re enoe s preition errors in ereellr lerning tsks 4,3 5. During eyelink onitioning, for exmple, liming fiers fire if the US is presente unexpetely 9 (positive preition error) n they re inhiite if n expete US is omitte (negtive preition error). This type of error oing n e use to generte n effetive tehing signl 6,7 y lerting the rin tht urrent expettions out the likelihoo of the instrutive US re inorret n nee to e upte. Inee, liming fier signls out positive n negtive US preition errors feture prominently in mny omputtionl moels of ereellrepenent onitioning 5,,9. Preition error signls out the US re well suite for riving simple forms of ssoitive lerning, suh s first-orer quisition n extintion of the onitione eyeli response 5 9. However, for higher orer lerning in whih nimls must lern from nonprimry reinforers suh s the CS, tehing signls relte to the US re not enough 2. Theories se on the influentil temporl ifferene (TD) moel 2,2 hve propose tht higher orer instrutive signls must lso lert the rin out the CS events tht relily preit the ourrene of the US. Suh CS-triggere signls hve een foun in mirin opmine neurons uring reinforement lerning tsks 4,22,23. We sought to exmine whether liming fiers my enoe the sme type of preitive TD signls uring ereellr-epenent ssoitive lerning. Tking vntge of new system for eyelink onitioning in he-fixe mie 24, we exmine the neurl oing of preition errors in liming fiers. On the sis of the preitions of the TD moel, we hypothesize tht, in ition to their well-known tivtion y n unexpete US, liming fiers shoul lso fire in response to presenttions of the CS t the en of onitioning, fter the primry ssoition etween the CS n the US hs een estlishe. RESULTS Monitoring liming fiers uring eyelink onitioning We sought to exmine the signls tht liming fiers sen to Purkinje ells uring ereellr lerning n to evlute whether these signls onform to the preitions of the TD moel 2,2. We use hefixe pprtus to trin mie in simple ely eyelink onitioning tsk tht is ritilly epenent on the ereellum (Fig.,) Dily onitioning sessions omprise 2 trils in whih CS suh s tone or n light ws followe fter 22 ms y n versive irpuff irete t the eye, whih serve s the instrutive linkeliiting US. All mie (n = 7) lerne to mke well-time onitione responses (CR) over the ourse of 5 onitioning sessions; tht is, they lerne to link in response to the CS, losing their eyelis in ntiiption of the versive perioulr irpuff. To mesure liming fier signls on ny given onitioning session, we lowere n eletroe into n ientifie eyelink region of ereellr ortex 27,2 n reore the extrellulr tivity of iniviul Purkinje ells (Fig. ). Eh tivtion of the powerful liming fier input resulte in mssive epolriztion of the postsynpti Purkinje ell 2,4, whih oul e etete in the rw extrellulr reor s hrteristi omplex spike 29 (; Fig. ). The wveform Deprtment of Neurosiene, Bylor College of Meiine, Houston, Texs, USA. Corresponene shoul e resse to J.F.M. (jvier.mein@m.eu). Reeive 5 August; epte 4 Otoer; pulishe online 9 Novemer 25; oi:.3/nn VOLUME NUMBER 2 DECEMBER 25 nture NEUROSCIENCE

2 npg 25 Nture Ameri, In. All rights reserve. Figure Experimentl esign n pproh. (,) Exmples of wveforms (top, ) n rw extrellulr signl (ottom, ) for SSs n s fire y representtive Purkinje ell uring eyelink onitioning in tremill pprtus for he-fixe mie (). ( e) Eyeli movements (men (line) ± s.. (she region)) n simultneously reore s of the Purkinje ell in in trils with unexpete perioulr irpuff (), pire n perioulr irpuff (), n without perioulr irpuff (e). FEC, frtion eyeli losure. Rw extrellulr tres in e show s (ots) in n exmple tril (thin lk eyeli tre). (f) Peristimulus time histogrms (in size = ms) for the s fire in the trils orresponing to the three rster plots of e. of the liming fier riven oul e istinguishe from norml Purkinje ell tion potentils known s simple spikes 29 (SS; Fig. ). Beuse eh Purkinje ell is known to reeive input from single liming fier,4 (Fig. ), the s fire y well-isolte Purkinje ell provie strightforwr wy to mesure the tivity of n iniviul liming fier. To evlute the oing of preition error signls, we first trine the mie n then exmine the liming fier riven s of iniviul Purkinje ells in ily sessions with three types of trils: unexpete presenttions of the perioulr irpuff US (unexpete; Fig.,f), pire trils in whih the CS (tone or ) ws presente 22 ms efore the US, s ws the se for norml onitioning trils uring trining (pire; Fig.,f), n trils in whih the CS ws presente y itself, without the US (omitte; Fig. e,f). Note tht, even in well-trine mie, there ws onsierle tril-y-tril vrition in performne (Fig.,e) n tht the size of the eyelink CR efore the US ws presente oul rnge from very smll in some iniviul trils (Fig. ) to very lrge in others (Fig. e). Climing fiers enoe US-relte preition errors Our first nlysis ws esigne to exmine liming fier riven s in the 2-ms time winow fter the perioulr irpuff (US perio; Fig. 2). We will refer to s in this US perio s US. Beuse liming fiers o not moulte their firing rte muh n typilly fire just one in response to eh stimulus presenttion 2, we ompute the tivity of eh liming fier y onstruting peristimulus time histogrm tht verges the numer of s fire y the sme Purkinje ell ross multiple trils (Fig. f n Online Methos). Consistent with previous reports,,2, we foun tht the liming fier input of mny Purkinje ells () in the eyelink region provies iiretionl signl tht enoes oth positive n negtive preition errors out the US 4 : the proility of US ws higher (Hz) SS (Hz) Unexpete Pire Omitte CS perio Big CR No CR US perio 22 Time from puff onset (ms) US (Hz), unexpete 6 (Hz), seline Unexpete Pire, no CR Pire, ig CR Omitte, no CR Omitte, ig CR US (Hz), omitte *** ** ns US (Hz), pire *** *** 4 2 US moultion (Hz) SS SS Purkinje ell He plte Airpuff Cylinril tremill ms Eletroe Climing fier ms e f mv mv rster mv rster (Hz) 6 ms Unexpete puff thn seline when the perioulr irpuff ws presente unexpetely (unexpete, Wiloxon signe rnk test, P <. with Bonferroni orretion for five omprisons; Figs.,f n 2,,) n in pire onitioning trils in whih the mouse file to mke CR (pire no CR, Wiloxon signe rnk test, P <. with Bonferroni orretion for five omprisons; Figs. n 2). In ontrst, the proility of US ws lower thn seline in trils in whih the mouse me CR n the perioulr irpuff ws omitte (omitte; Figs. e,f n 2,). Note tht the reution of liming fier tivity in the omitte trils ws very relile (omitte ig CR, Wiloxon signe rnk test, P <. with Bonferroni orretion for five omprisons; Fig. 2), ut the size of the moultion ws smll euse liming fiers fire t very low rtes roun Hz uring seline 2. We exmine the SS tivity of the sme group of Purkinje ells (Fig. 2) to gin some insight out the mehnisms unerlying the oing of preition errors in liming fiers. Purkinje ells reue their SS firing rte prior to the US perio, ut only in trils with CR (ig CR; Fig. 2), n not when the mouse file to mke Pire puff Omitte puff Unexpete Pire Omitte 22 Time from puff onset (ms) Figure 2 Climing fier responses in the US perio. () Popultion verges of eyeli movements n simultneously reore s n SSs in trils with unexpete perioulr irpuff (lk), pire n perioulr irpuff (re), n without perioulr irpuff (lue). For SS, t in trils without irpuff re plotte seprtely for trils with onitione eyeli response (ig CR) n trils without (no CR). Time winows for CS n US nlysis re inite. ( ) The proility of uring seline n in the US perio of ifferent types of trils is plotte for ll iniviul Purkinje ells (,) n summrize for the popultion (). In, moultion of tivity in the US perio is reltive to seline. Mein (re line), interqurtile rnge (ox), t rnge (whisker), n outliers (ots). An outlier ws omitte from the unexpete puff onition (first row). **P <., ***P <., fter using Bonferroni test to orret for five omprisons; ns inites not signifint. nture NEUROSCIENCE VOLUME NUMBER 2 DECEMBER

3 Figure 3 Climing fier responses in the CS perio. (,) Proility of tivity in the CS perio reltive to the proility fter unexpete elivery of perioulr irpuff, plotte for ll iniviul Purkinje ells in well-trine mie (), n in nive mie (). All of the responses hve een normlize to the tivity of eh Purkinje ell uring seline. () The responsiveness of liming fiers to the CS in well-trine n nive mie is plotte s rtio of response proilities to the CS n to the unexpete presenttion of the perioulr irpuff stimulus. Dt for iniviul Purkinje ells (ots) n mein (re line) re shown. Numers in prentheses inite the numer of mie use (,) n the numer of ells reore (). CS (normlize to seline) 5 5 Trine mie (6) n = US, unexpete (normlize to seline) 5 Nive mie (2) n = 29 CS / US 2 (29) (34) Nive Trine npg 25 Nture Ameri, In. All rights reserve. CR (no CR; Fig. 2). It hs een suggeste tht suh reution in SS firing rte oul e use to inhiit liming fier tivity t the time of the US,3 vi oule inhiitory pthwy from Purkinje ells to neurons in the eep ereellr nulei n then to the inferior olive where liming fiers originte 3. In this moel, liming fier tivity uring the US perio is moulte y two inputs onverging in the inferior olive: exittion from US-riven trigeminl neurons 32,33 n inhiition from CR-relte neurons in the eep ereellr nuleus,3,34. This ntomil orgniztion oul help to explin why liming fier tivity ws highest in positive preition error trils when there ws no CR-relte inhiition of the inferior olive t the time of the US (unexpete n pire no CR; Fig. 2), reue when US-relte exittion ws ounterlne y CR-relte inhiition (pire ig CR; Fig. 2), ner seline in the sene of US-relte n CR-relte inputs (omitte no CR; Fig. 2), n suppresse elow seline in negtive preition trils in whih CR-relte inhiition of the inferior olive ourre without the US (omitte ig CR; Fig. 2). Climing fiers responses to the CS in well-trine mie Previous eletrophysiology stuies 9,3, inluing the nlysis presente ove (Fig. 2 ), hve fouse exlusively on liming fier signls roun the time of the instrutive US. However, to omply with the requirements of the TD moel 2,2, liming fiers must fire in response to stimuli suh s the CS tht relily preit the ourrene of the US. To ssess whether liming fiers meet this riterion, we exmine s in -ms time winow strting 5 ms fter the or tone stimulus use s the CS (CS perio; Fig. 2). We will refer to s in this CS perio s CS. (Hz) CS, ig CR CS Big CR.2 No CR 5 n = US 22 Time (ms) n = 3 Normlize 5 5 CS, no CR CS US US ISI 22 ISI 37 5 n = 3 n = e CS lteny (ms) Time (ms) 3 (3) () 5 ISI 22 ISI 37 f CS lteny (ms) erly CR lte CR FEC 6 CS US Erly onset.2 Lte onset 5 n = 3 22 Time (ms) n = 3 4 CR lteny (ms) erly CR lte CR Presenttion of the CS relily eliite s in well-trine mie (Fig. 2). The proility of CS ws higher in Purkinje ells tht lso h relile response to unexpete perioulr irpuffs (R =.72; Fig. 3). In ontrst, the proility of CS ws ner seline in the mjority of Purkinje ells reore in nive mie, even those tht fire s relily in response to unexpete presenttion of the perioulr irpuff (Fig. 3). Thus, Purkinje ells in this region of ereellr ortex fire s muh more roustly to the unexpete US thn to the CS in nive mie (nive; Fig. 3), n eqully well to oth stimuli fter trining (trine; Fig. 3). These results re onsistent with hypothesis in whih the mjority of liming fiers re initilly unresponsive to the CS, n grully quire response uring onitioning, s the CS eomes preitive of the instrutive US. However, it is ler tht the CS n tivte some liming fiers even in nive mie (Fig. 3,). We will return to this oservtion elow. In ition to proviing the tehing signls neessry for ereellr lerning, it hs een suggeste tht liming fiers lso ontriute to the ongoing ontrol of movement timing 35. We performe three nlyses to ssess whether there is reltionship etween CS tivity n the CR-relte movement of the eyeli (Fig. 4). First, we foun tht, in most Purkinje ells, the proility of CS ws only mrginlly higher in trils with ig CR thn in trils without CR (Wiloxon signe rnk test, P <.; Fig. 4,). Seon, we reore n itionl eight Purkinje ells in mouse trine with 37-ms CS-US interstimulus intervl (ISI; Fig. 4,e) n foun tht the lteny of the CS fell in the rnge oserve for Purkinje ells in mie trine with the 22-ms intervl (Wiloxon rnk sum test, P =.22; Fig. 4e), even though the temporl profile of the CR ws ifferent 24,36 (Fig. 4). Thir, we onfirme tht the lteny of the CS i not epen on the lteny of the CR n ws essentilly the sme for trils in whih the eyeli strte losing erly or lte (Fig. 4); tht is, the ifferene in CS lteny etween trils with erly CRs n lte CRs ws ner zero for most of the Purkinje ells (Wiloxon signe rnk test, P =.4; Fig. 4f). Thus, s expete for TD preition error signl 2,2, liming fiers fire t fixe lteny fter the CS in well-trine mie regrless of the expete time of Figure 4 Complex spikes in CS perio re not riven y eyeli movement. (,) Comprison of popultion-verge tivity in trils with (ig CR) n without (no CR) onitione eyeli response (), n the orresponing response in the CS perio for ll iniviul Purkinje ells (). (,e) Comprison of popultion-verge tivity in mie trine with 22-ms ISI (ISI 22) n mouse trine with 37-ms ISI (ISI 37) (), n the orresponing mein lteny in the CS perio for ll iniviul Purkinje ells (ots) n popultion mein (re line) (e). Eyeli tres in hve een normlize to provie iret omprison of movement time ourse. () Comprison of popultion-verge tivity in trils with erly-onset n lte-onset CR movements. Numers in prentheses inite the numer of ells reore. (f) The ifferene etween the lteny of the in trils with erly-onset n lte-onset CRs is plotte for ll iniviul Purkinje ells. VOLUME NUMBER 2 DECEMBER 25 nture neuroscience

4 npg 25 Nture Ameri, In. All rights reserve. Figure 5 Climing fier responses to novel stimuli. (,) Experimentl esign for mie in hitution () n ifferentil onitioning () groups. (,) Proility of response to the unexpete irpuff (lk) n to the CS+ (lue) n CS (re) over the ourse of multiple sessions of hitution () or ifferentil onitioning (). Eh session shows t for ifferent Purkinje ell. All responses hve een normlize to the tivity of eh Purkinje ell uring seline. (e,f) Proility of response to the CS in the first three (e) n lst three (f) reoring sessions. The t for tone n stimuli n for hitution n ifferentil onitioning sessions were omine. (g) The responsiveness of liming fiers to the CS in the first three n the lst three sessions is plotte s rtio of the response proilities to the CS n to the unexpete presenttion of the perioulr irpuff stimulus. Dt for iniviul Purkinje ells (ots) n mein (re line) re shown. Numers in prentheses inite the numer of ells reore. the US or the timing of the CR. Furthermore, we n rule out the possiility tht CS s re riven y CR-relte movement of the eyeli, s they were lerly present oth in trils with n without CR (Fig. 4,). Climing fier responses to novel stimuli In reent omputtionl stuies se on the TD moel, instrutive signls inorporte omponent relte to the novelty of the CS 37,3. We performe two itionl experiments to ssess whether there is reltionship etween liming fier tivity n stimulus novelty (n = 4 mie): hitution trining, in whih nive mie reeive ily sessions omprising 2 5 repete presenttions of tone or stimulus (CS ) without the perioulr puff (Fig. 5), n ifferentil onitioning, in whih mie were first trine y piring one stimulus (CS+) with the perioulr puff n, fter lerning, reeive ily sessions in whih the sme CS+ ontinue to e pire with the irpuff in 75% of ll trils, n ifferent CS ws presente without the irpuff in 25% of ll trils (Fig. 5). None of the mie me eyeli movements in response to the CS in ny of our experiments (Supplementry Fig. ). Figure 5, summrizes the responses of Purkinje ells reore in two representtive mie uring hitution n ifferentil onitioning (one Purkinje ell per session). Purkinje ells reore in the first few sessions of hitution (Fig. 5) n ifferentil onitioning (Fig. 5) fire s relily in response to the CS. In ontrst, Purkinje ells reore in susequent sessions were muh less likely to fire s to the CS thn to the CS+ or to unexpete presenttions of the perioulr irpuff (Fig. 5, n Supplementry Fig.,f,i). Pooling the t for ll the mie revele reltionship etween liming fier tivity n the novelty of the CS (Fig. 5e g): the CS triggere with high proility when it ws reltively novel, tht is, in the first three sessions of hitution or ifferentil onitioning (Wiloxon signe rnk test, P <.; Fig. 5e,g), ut not in the lst three sessions when it h een presente mny times n ws more fmilir (Wiloxon signe rnk test, P =.9; Fig. 5f,g). The sme result ws otine y ompring the responses to the CS in the first two n the lst two sessions, or in the first five n the lst five sessions (Supplementry Fig. 2). This fining explins our previous oservtion tht the CS n trigger liming fier riven s in some Purkinje ells even in nive mie (Fig. 3,). Inee, the eight responsive Purkinje ells shown in Figure 3, were ll reore in the first two ys of experiments, uring initil exposure to the CS. DISCUSSSION We exmine the tivity of liming fiers uring eyelink onitioning in mie y reoring the s of Purkinje ells in n eyeli e Hitution CS (without US) n Session numer US, unexpete (normlize) to CS (normlize) to CS (normlize) Session numer Bseline First 3 sessions n = (4 mie) CS f Pre trining Differentil onitioning CS + US CS (without US) n Session numer region of ereellr ortex. Our results onfirm those of previous reports 9, suggesting tht mny liming fiers in this region signl oth the unexpete elivery n the unexpete omission of the perioulr irpuff tht serves s the instrutive US. In ition, we foun tht, uner ertin onitions, the sme liming fiers n lso fire in response to CS from ifferent sensory molity, suh s tone or n light. Here we isuss the implitions of these results for unerstning the neurl representtion of instrutive signls uring ereellr lerning. We foun tht liming fiers enoe preition error signl tht stisfies three si priniples of TD moels 2,2 : liming fiers fire in response to unexpete presenttion of the US (positive preition error), liming fiers re inhiite if n expete US is omitte (negtive preition error), n liming fiers fire in response to the preitive CS in well-trine mie, fter the reltionship etween the CS n the US hs een lerne. Aoring to the TD moel 2,2, the response to the CS shoul evelop grully uring onitioning, progressively inresing in strength s lerning proees n the ssoition etween the CS n US is estlishe. We were unle to test this hypothesis iretly euse ereellr-epenent eyelink onitioning in mie is slow lerning proess 24, n it is not urrently fesile to mintin goo isoltion n trk the tivity of the sme liming fier over long perios of time. Until the right tools eome ville, we n t lest note tht, in our experiments, liming fiers with similr responses to the unexpete presenttion of the US fire muh more relily to the CS in well-trine mie thn in nive mie. This fining is onsistent with the hypothesis tht liming fier responses to the CS strt out reltively wek n get progressively stronger uring lerning, s preite y TD moels 2,2. Our experiments lso revele tht liming fiers often fire in response to the initil presenttions of the CS in nive mie. Beuse CS CS+ 6 Lst 3 sessions n = (4 mie) 6 6 US, unexpete (normlize to seline) Session numer Bseline 2 g CS / US 2 () () First 3 Lst 3 nture NEUROSCIENCE VOLUME NUMBER 2 DECEMBER 25

5 npg 25 Nture Ameri, In. All rights reserve. this response hitutes if the CS ontinues to e repetely presente without the US, we interpret it s signl relte to stimulus novelty. Novelty signls were not prt of the originl TD moel 2,2, ut they hve een reently inorporte into the generl TD frmework, where they re importnt for triggering explortory n orienting tions tht help to etermine the mening of stimuli with high potentil importne 37,3. Thus, in our working hypothesis, liming fiers hve the pity to multiplex: they generte novelty signls out stimuli whose mening is yet to e etermine, s well s preition error signls out stimuli whose mening is known or hs een lerne. An lterntive hypothesis tht n ount for mny of our oservtions is tht liming fiers enoe signl out stimulus sliene, whih is thought to e importnt for ssoitive lerning 39. However, our results inite tht liming fiers o more thn provie slieny informtion. For exmple, we hve shown tht liming fier tivity is suppresse elow seline when n expete US is omitte, whih is hllmrk of negtive preition error signl, ut it s iffiult to explin with pure slieny oe. Climing fier responses to visul n uitory CSs were ovious n highly prevlent in our experiments, ut they re onspiuously sent in previous stuies of eyelink onitioning 9. In ft, we oul only fin one preliminry report (Egley, S.A., Mostofi, A. & Holtzmn, T. So. Neurosi. Astr. 76.4, 2) n three pulitions tht riefly mention in pssing wht my hve een osionl CS-relte responses,4,4. There re two importnt fetures of our experimentl pproh tht oul help to explin this pprent isrepny. First, we only exmine the tivity of liming fiers tht projet to n ientifie eyelink region of ereellr ortex 27,2 n tht respone relily to unexpete presenttions of the perioulr irpuff stimulus. This seletion is likely to e importnt euse the response properties of liming fiers n vry wiely epening on their ext lotion in the ntomil mirozonl orgniztion of the ereellr ortex 2. Seon, we performe ll of our experiments in he-fixe mie tht were free to loomote on top of tremill 24, wheres others efore us hve often immoilize their sujets 9 or use eererte preprtion to inrese reoring stility. Whether ny of these methoologil onsiertions n ount for the lk of CS-relte liming fier responses in previous stuies remins n open question. However, we note tht liming fier responses to peripherl stimultion re strongly moulte y ehviorl stte 42 n iffer gretly etween resting n wlking onitions 43,44. Our results provie some lues s to the origin of the multimol responses of liming fiers to somtosensory, visul n uitory stimuli. All our reorings trgete Purkinje ells in smll eyelink region of ereellr ortex tht reeives liming fier inputs from neurons lote in three suivisions of the IO 2 : the orsl essory olive, the meil essory olive n the meil orsl olive. Neurons in these three suivisions of the IO pper to e well positione for multisensory proessing, s they reeive onverging inputs from proprioeptive n utneous reeptors vi sening spino-olivry pthwys n from multiple sensorimotor res of the ortex vi esening erero-olivry pthwys 3,45,46. In ition, some ells in the meil essory olive respon to flshes of light n tpping souns 47, possily vi tivtion of inputs from the superior olliulus 4. Notly, we foun tht liming fiers of well-trine mie fire in response to uitory or visul CSs even in trils in whih the mouse file to mke onitione eyeli movement. This oservtion rules out the possiility tht wht my hve looke like sensory response to the CS ws relly proprioeptive response or refferene response riven y the neurons or musles tht ontrol the eyeli. However, non-eyeli motor soure relte to orienting or strtle movements to the CS is still possile 4. Our results re isruptive n ll for mjor revision of existing theories out the funtion of liming fiers uring ereellr lerning tsks suh s eyelink onitioning. In the previling view, liming fiers rry preition error signl out the instrutive US 5,5,,9. The ntomy supports suh view euse it is wellknown tht the neurons of the inferior olive (IO) tht sen their liming fier xons to eyelink regions of the ereellr ortex reeive hrwire exittory input from US-responsive res in the trigeminl nuleus 32,33. In ition, the sme IO neurons lso reeive n inhiitory input from ells of the eep ereellr nuleus tht fire uring the CR,3,34, therey estlishing n ntomil sis for omputing positive- n negtive-preition error signls out the US; tht is, IO ells will e exite y the US when it is not preite, n they will e inhiite y CR-relte tivity if preite US is omitte. This ntomil rrngement of exittory n inhiitory synpti inputs in the IO is the fountion for urrent moels of ereellr lerning se on the omprtor hypothesis 5,,9. However, none of the existing theories tke into ount our isovery tht the sme liming fiers tht rry error-relte informtion out the instrutive US lso respon to visul n uitory CSs if these stimuli re novel or if they hve een previously onitione. The pttern of liming fier responses tht we reore uring eyelink onitioning ers mrke resemlne to the responses of mny opmine neurons uring reinforement lerning tsks 22,49,5. We lrey knew tht, in oth ses, these responses enoe positive n negtive preition error signls out the instrutive US, tht is, out the perioulr irpuff use for eyelink onitioning in the se of liming fiers, n out the rewr use for reinforement lerning in the se of opmine neurons. Here we foun tht the similrities go eyon oing of US-relte instrutive signls. As with the liming fiers reore in our experiments, opmine neurons in reinforement lerning tsks fire in response to uitory or visul stimuli if these stimuli re novel 22,49 or if they preit tht the US is out to e presente 22,49,5. In theories of reinforement lerning se on the TD moel, these opmine responses re importnt for riving higher orer quisition of pproh ehvior to potentil rewr 23,37,3,49,5. Future experiments will help to etermine whether the novelty n CS-relte signls of liming fiers my hve similr tehing role uring ereellr lerning. Methos Methos n ny ssoite referenes re ville in the online version of the pper. Note: Any Supplementry Informtion n Soure Dt files re ville in the online version of the pper. Aknowlegments We thnk K. Ohme for tehnil support n S. Heiney for help with nlysis n neurophysiologil pproh. This work ws supporte y grnt to J.F.M. from the US Ntionl Institutes of Helth (R MH93727), n grnt to S.O. from Jpn Soiety for the Promotion of Siene (Grnt-in-Ai for JSPS Fellows) n from the Uehr Memoril Fountion. AUTHOR CONTRIBUTIONS S.O. n J.F.M. esigne the reserh pln. S.O. performe ll of the experiments n nlyze t. J.F.M. n S.O. prepre the figures n wrote the mnusript. COMPETING FINANCIAL INTERESTS The uthors elre no ompeting finnil interests. Reprints n permissions informtion is ville online t reprints/inex.html. 2 VOLUME NUMBER 2 DECEMBER 25 nture neuroscience

6 npg 25 Nture Ameri, In. All rights reserve.. Eles, J.C., Llinás, R. & Sski, K. The exittory synpti tion of liming fires on the Purkinje ells of the ereellum. J. Physiol. (Lon.) 2, (966). 2. Njfi, F. & Mein, J.F. Beyon ll-or-nothing liming fiers: gre representtion of tehing signls in Purkinje ells. Front. Neurl Ciruits 7, 5 (23). 3. De Zeeuw, C.I. et l. Miroiruitry n funtion of the inferior olive. Trens Neurosi. 2, 39 4 (99). 4. Simpson, J.I., Wylie, D.R. & De Zeeuw, C.I. On liming fier signls n their onsequene(s). Behv. Brin Si. 9, (996). 5. Mein, J.F., Nores, W.L., Ohym, T. & Muk, M.D. Mehnisms of ereellr lerning suggeste y eyeli onitioning. Curr. Opin. Neuroiol., (2). 6. Christin, K.M. & Thompson, R.F. Neurl sustrtes of eyelink onitioning: quisition n retention. Lern. Mem., (23). 7. Freemn, J.H. & Steinmetz, A.B. Neurl iruitry n plstiity mehnisms unerlying ely eyelink onitioning. Lern. Mem., (2).. Thompson, R.F., Thompson, J.K., Kim, J.J., Krup, D.J. & Shinkmn, P.G. The nture of reinforement in ereellr lerning. Neuroiol. Lern. Mem. 7, 5 76 (99). 9. Sers, L.L. & Steinmetz, J.E. Dorsl essory inferior olive tivity iminishes uring quisition of the rit lssilly onitione eyeli response. Brin Res. 545, 4 22 (99).. Niholson, D.A. & Freemn, J.H. Jr. Aition of inhiition in the olivoereellr system n the ontogeny of motor memory. Nt. Neurosi. 6, (23).. Rsmussen, A., Jirenhe, D.A. & Hesslow, G. Simple n omplex spike firing ptterns in Purkinje ells uring lssil onitioning. Cereellum 7, (2). 2. Muk, M.D., Steinmetz, J.E. & Thompson, R.F. Clssil onitioning using stimultion of the inferior olive s the unonitione stimulus. Pro. Ntl. A. Si. USA 3, (96). 3. Ito, M. Error etetion n representtion in the olivo-ereellr system. Front. Neurl Ciruits 7, (23). 4. Shultz, W. & Dikinson, A. Neuronl oing of preition errors. Annu. Rev. Neurosi. 23, (2). 5. Den, P. & Porrill, J. Deorreltion lerning in the ereellum: omputtionl nlysis n experimentl questions. Prog. Brin Res. 2, (24). 6. Resorl, R.A. & Wgner, A.R. A theory of Pvlovin onitioning: vritions in the effetiveness of reinforement n nonreinforement. in Clssil Conitioning II: Current Reserh n Theory (es. Blk, A.H. & Proksy, W.F.) (Appleton- Century-Crofts, New York, 972). 7. Wirow, B. & Hoff, M.E. Aptive swithing iruits. in IRE WESCON Convention Reore 96 4 (Institute of Rio Engineers, New York, 96).. Muk, M.D. & Donegn, N.H. A moel of Pvlovin eyeli onitioning se on the synpti orgniztion of the ereellum. Lern. Mem. 4, 3 5 (997). 9. Mein, J.F. & Muk, M.D. Computer simultion of ereellr informtion proessing. Nt. Neurosi. 3 (suppl. ): 25 2 (2). 2. Sutton, R.S. & Brto, A.G. Time-erivtive moels of Pvlovin reinforement. in Lerning n Computtionl Neurosiene: Fountions of Aptive Networks (es. Griel, M. & Moore, J.W.) (MIT Press, Cmrige, Msshusetts, 99). 2. Sutton, R.S. Lerning to preit y the methos of temporl ifferenes. Mh. Lern. 3, 9 44 (9). 22. Shultz, W. Preitive rewr signl of opmine neurons. J. Neurophysiol., 27 (99). 23. Shultz, W., Dyn, P. & Montgue, P.R. A neurl sustrte of preition n rewr. Siene 275, (997). 24. Heiney, S.A., Wohl, M.P., Chettih, S.N., Ruffolo, L.I. & Mein, J.F. Cereellrepenent expression of motor lerning uring eyelink onitioning in he-fixe mie. J. Neurosi. 34, (24). 25. Yng, Y., Lei, C., Feng, H. & Sui, J.F. The neurl iruitry n moleulr mehnisms unerlying ely n tre eyelink onitioning in mie. Behv. Brin Res. 27, (25). 26. Boele, H.J., Koekkoek, S.K. & De Zeeuw, C.I. Cereellr n extrereellr involvement in mouse eyelink onitioning: the ACDC moel. Front. Cell. Neurosi. 3, 9 (2). 27. Heiney, S.A., Kim, J., Augustine, G.J. & Mein, J.F. Preise ontrol of movement kinemtis y optogeneti inhiition of Purkinje ell tivity. J. Neurosi. 34, (24). 2. Mostofi, A., Holtzmn, T., Grout, A.S., Yeo, C.H. & Egley, S.A. Eletrophysiologil loliztion of eyelink-relte mirozones in rit ereellr ortex. J. Neurosi. 3, (2). 29. Thh, W.T. Jr. Somtosensory reeptive fiels of single units in t ereellr ortex. J. Neurophysiol. 3, (967). 3. Hesslow, G. & Ivrsson, M. Inhiition of the inferior olive uring onitione responses in the eererte ferret. Exp. Brin Res., (996). 3. Anersson, G., Grwiz, M. & Hesslow, G. Eviene for GABA-meite ereellr inhiition of the inferior olive in the t. Exp. Brin Res. 72, (9). 32. Vn Hm, J.J. & Yeo, C.H. Somtosensory trigeminl projetions to the inferior olive, ereellum n other preereellr nulei in rits. Eur. J. Neurosi. 4, (992). 33. Swenson, R.S. & Cstro, A.J. The fferent onnetions of the inferior olivry omplex in rts. An nterogre stuy using utoriogrphi n xonl egenertion tehniques. Neurosiene, (93). 34. Svensson, P., Bengtsson, F. & Hesslow, G. Cereellr inhiition of inferior olivry trnsmission in the eererte ferret. Exp. Brin Res. 6, (26). 35. Llinás, R.R. Cereellr motor lerning versus ereellr motor timing: the liming fire story. J. Physiol. (Lon.) 59, (2). 36. Chettih, S.N., MDougle, S.D., Ruffolo, L.I. & Mein, J.F. Aptive timing of motor output in the mouse: the role of movement osilltions in eyeli onitioning. Front. Integr. Neurosi. 5, 72 (2). 37. Kke, S. & Dyn, P. Dopmine: generliztion n onuses. Neurl Netw. 5, (22). 3. Lurent, P.A. The emergene of slieny n novelty responses from Reinforement Lerning priniples. Neurl Netw. 2, (2). 39. Pere, J.M. & Hll, G. A moel for Pvlovin lerning: vritions in the effetiveness of onitione ut not of unonitione stimuli. Psyhol. Rev. 7, (9). 4. Berthier, N.E. & Moore, J.W. Cereellr Purkinje ell tivity relte to the lssilly onitione nititting memrne response. Exp. Brin Res. 63, (96). 4. Niholson, D.A. & Freemn, J.H. Jr. Developmentl hnges in eye-link onitioning n neuronl tivity in the inferior olive. J. Neurosi. 2, (2). 42. Bengtsson, F. & Jorntell, H. Ketmine n xylzine epress sensory-evoke prllel fier n liming fier responses. J. Neurophysiol. 9, (27). 43. Ozen, I., Domek, D.A., Hoogln, T.M., Tnk, D.W. & Wng, S.S.-H. Wiespre stte-epenent shifts in ereellr tivity in loomoting mie. PLoS ONE 7, e4265 (22). 44. Apps, R. Gting of liming fire input to ereellr ortil zones. Prog. Brin Res. 24, 2 2 (2). 45. Jörntell, H., Grwiz, M. & Ekerot, C.F. Reltion etween utneous reeptive fiels n musle fferent input to liming fires projeting to the ereellr C3 zone in the t. Eur. J. Neurosi., (996). 46. Proe, J., Egley, S.A., Drew, T. & Apps, R. Chnges in exitility of sening n esening inputs to ereellr liming fiers uring loomotion. J. Neurosi. 24, (24). 47. Gellmn, R., Houk, J.C. & Gison, A.R. Somtosensory properties of the inferior olive of the t. J. Comp. Neurol. 25, (93). 4. Sint-Cyr, J.A. & Courville, J. Desening projetions to the inferior olive from the mesenephlon n superior olliulus in the t. An utoriogrphi stuy. Exp. Brin Res. 45, (92). 49. Bromerg-Mrtin, E.S., Mtsumoto, M. & Hikosk, O. Dopmine in motivtionl ontrol: rewring, versive, n lerting. Neuron 6, 5 34 (2). 5. Dw, N.D. & Doy, K. The omputtionl neuroiology of lerning n rewr. Curr. Opin. Neuroiol. 6, (26). nture NEUROSCIENCE VOLUME NUMBER 2 DECEMBER 25 3

7 npg 25 Nture Ameri, In. All rights reserve. ONLINE METHODS Animl preprtion. All proeures were performe in C57BL/6 mle mie pproximtely 4 weeks of ge (Jkson Lortories, n = ), following the guielines of the US Ntionl Institutes of Helth n protool pprove y the University of Pennsylvni Animl Cre n Use Committee. Before strting our experiments, we implnte he plte n reoring hmer ove the right ereellr ortex following surgil proeures esrie previously 24. Stimulus ontrol n ehviorl proeures. In ll experiments, mie were he-fixe on top of ylinril tremill n llowe to wlk freely 24. Fil n eyeli movements ipsilterl to the reoring site were monitore with high-spee monohrome mer (GE6, Allie Vision) uner infrre illumintion. Vieo frmes (2 fps) were triggere y Sys3 proessor (RZ5, TDT) n store with the MATLAB Vieo Aquisition Toolox. A Sys3 proessor (RZ5, TDT) ws use to ontrol the timing of stimuli uring onitioning. The US ws n irpuff of nitrogen (4 psi, 2 ms) elivere vi 23-guge flt-ene neele ple ~4 mm in front of the right orne of the mouse. The CS were either 5-ms lue light positione m in front of the mouse s fe, or 5-ms tone of white noise elivere vi speker (4-Ω mgneti speker, FF, TDT) positione 2 m in front of the mouse. The volume of the white noise ws set efore the first onitioning session to just elow the threshol for eliiting short-lteny strtle movement of the eyeli. After 3 of limtiztion to the ylinril tremill, mie were sumitte to one of three ehviorl proeures: (i) hitution: ily sessions omprising 2 5 unpire presenttions of the CS ( or tone without the US), (ii) eyelink onitioning: ily sessions omprising 2-79 pire trils in whih the CS+ ( or tone) ws presente for 5 ms n the US ws elivere 22 ms fter the CS+ onset (22-ms ISI). For one mouse, the ISI ws 37 ms, In ~25% of the trils the US ws omitte, ut this frtion ws inrese in few reoring sessions to ollet enough t for nlysis, (iii) ifferentil onitioning: mie first reeive 5 3 sessions of eyelink onitioning omprising pire presenttions of the CS+ ( or tone) n the US (22-ms ISI; see ove). After the mie lerne to mke onitione eyeli responses to the CS+, they egn reeiving ily sessions omprising mixture of trils in whih the sme CS+ ontinue to e pire with the US (pproximtely 75% of ll trils), n CS trils in whih ifferent stimulus (for exmple, when the CS+ ws tone, the CS ws n, n vie vers) ws presente without the US (pproximtely 25% of ll trils). In ll reoring sessions, the US ws elivere unexpetely to the ipsi- n ontrlterl ornes in osionl trils rnomly interleve uring eh session. The minimum intervl etween trils (intertril intervl; ITI) ws 7 6 s, ut trils oul only strt if the eyeli position ws stle for t lest.6 s. Behviorl nlysis. Movement of the eyeli ws lulte frme y frme y ounting the numer of white pixels in threshole inry imge of the eye n surrouning fur, oring to proeure esrie previously 24. Eyeli losure ws mesure in units of frtion eyeli losure (FEC), rnging from (fully open) to (fully lose). Trils in whih FEC i not reh t lest. in the ISI perio were efine s no-cr trils (Figs. 2 n 4,). For Figure 4, the eyeli tres were normlize for eh session so tht the verge eyeli position ws t the onset of the CS n t the onset of the US. For Figure 4, the trils of eh reoring session were ivie into three equl-size groups oring to the onset lteny of the CR: erly-onset, mile-onset n lte-onset CRs. CR onset ws efine for eh tril in the session s the time when the low-pss filtere eyeli veloity tre rehe threshol of.2 FEC ms. No-CR trils were exlue from this nlysis. For Figure 4f, the mein CR ltenies for the erly-onset n lte-onset groups were sutrte from eh other for eh reoring session n plotte on the x xis. Single-unit reoring. Extrellulr reoring of simple spikes n s in Purkinje ells ws performe with 5 MΩ tungsten miroeletroes (75 µm of shft imeter, FHC) or glss pillry eletroes (BF5-6-, Sutter instrument) with 2 7-µm tip n 3 6-MΩ impene (P-, Sutter instrument). The eletroes were ontrolle with hyruli mirorive (MMO-22A, Nrishige) mounte on three-xis mnul miromnipultor (M325, WPI). The voltge signl ws quire t 24,44-Hz smpling rte, n n-pss filtere etween. khz using igitl proessor (RZ5, TDT). To trget the eyelink mirozone lote ner the floor of the primry fissure 27,2 (2, 2,4 µm elow the surfe of ur mtter), eletroes were irete long 5-eg ngle xis from posterior orsl to nterior ventrl, reltive to the vertil plne. The eyelink mirozone oul e ientifie y the presene of lrge negtive LFP signl (4 µv) in the moleulr lyer of the ereellr ortex in response to ipsilterl perioulr irpuffs ( 2 ms), n y the orresponing reore in iniviul Purkinje ells. As reporte previously for the eyelink mirozone in rits 2, ontrlterl perioulr irpuffs i not evoke s s relily. Eletrophysiology nlysis. We reore totl of 5 Purkinje ells (29 from mie in hitution group, 5 from mie trine with the 22-ms ISI, ells from mouse trine with the 37-ms ISI, n 63 from mie in the ifferentil onitioning group). s oul e isolte in ll 5 Purkinje ells, n simple spikes in 22 of them. s n simple spikes were sorte off-line using the threshol-rossing n templte-mthing lgorithm of Spike2 softwre (Cmrige Eletroni Design). For s we lso exmine the voltge wveform tres tril-y-tril n performe itionl mnul sorting, lin to tsk onition n ehvior. Finlly, to onfirm tht the s n the simple spikes originte from the sme Purkinje ell, we heke tht there ws 4-ms puse in simple spike tivity fter eh 4. Although liming fiers typilly fire just one in response to the CS+ n to the unexpete US, we oserve osionl oulets of s in quik suession (rw reor in Fig. n rster in Fig. ). Similr oulets hve een oserve efore 5. Also note tht fter firing to the CS+, the refrtory perio of liming fiers (pproximtely ms) 4 is too rief to ffet how the liming fier will fire in response to the US (whih is elivere ms fter the CS, fter the refrtory perio is over). A peristimulus time histogrm (PSTH) ws onstrute for eh liming fier n for eh tril type (see exmple PSTH s for unexpete, pire n omitte trils types of representtive liming fier in Fig. f). Climing fier tivity in the PSTH ws expresse s frequeny in Hz, y ing ll the s fire in eh time in n iviing y the numer of trils multiplie y the in size (. s). We took three mesurements from eh PSTH: (i) the spontneous (seline) frequeny of the liming fier in the 5-ms time winow efore the trils strte, (ii) the frequeny of the liming fier in the 2-ms winow fter elivery (or omission) of the US, n (iii) the frequeny of the liming fier in the -ms time winow fter elivery of the tone CS, or 5 ms fter elivery of the CS to tke into ount longer elys relte to visul proessing. The response to the CS or the unexpete US ws onsiere to e sttistilly signifint if it ws t lest two s.. higher thn the seline frequeny. Of the 5 ells in mie trine with 22-ms ISI, 34 h sttistilly signifint response to the unexpete presenttion of the US (nlyze in Figs. 2 n 3), n 3 h sttistilly signifint response to the CS (nlyze in Fig. 4). In ition, sttistilly signifint responses to the unexpete US were foun in 29 ells from mie in the nive/hitution group (nlyze in Figs. 3 n 5e g; note tht, in Fig. 5e g, only frtion of the 29 ells were reore in the first three or the lst three sessions), eight ells from mouse trine with the 37-ms ISI (nlyze in Fig. 4,e), n 45 ells from mie in the ifferentil onitioning group (nlyze in Fig. 5e g, note tht only frtion of these 45 ells were reore in the first three or the lst three sessions). The frequeny of eh liming fier ws normlize y its seline frequeny for Figures 3,, 4 n 5e,f. Sttistil nlysis. Men n within-session vriility (s..) re isplye for eyeli tres in Figure,e. Averge of ll sessions is isplye for eyeli tres n histogrms in Figures 2 n 4. All sttistil nlyses were performe using the Sttistis toolox in MATLAB. We pplie the Bonferroni orretion for multiple omprisons In Figure 2, to test the ifferene of moultion (firing frequeny ove seline) from zero. We use nonprmetri sttistil tests without ssuming normlity, exept for t in Figure 3, where we use Person s orreltion oeffiient. All tests were two-sie. No rnomiztion ws use, ut mie were ssigne to speifi experimentl group without is n no nimls were exlue. The experimenter ws lin to tsk onition n ehvior uring spike sorting n further nlyses. A Supplementry Methos Cheklist is ville. 5. Eles, J.C., Sh, N.H., Shmit, R.F. & Torikov, H. Cutneous mehnoreeptors influening impulse ishrges in ereellr ortex. 3. In Purkyne ells y liming fier input. Exp. Brin Res. 5, (972). nture NEUROSCIENCE oi:.3/nn.467