Macmillan Publishers Limited. All rights reserved

Transcription

1 Vol June 29 doi:1.138/nture828 LETTERS Two types of dopmine neuron distinctly convey positive nd negtive motivtionl signls Msyuki Mtsumoto 1 & Okihide Hikosk 1 Midbrin dopmine neurons re ctivted by rewrd or sensory stimuli predicting rewrd 1 4. These excittory responses increse s the rewrd vlue increses. This response property hs led to hypothesis tht dopmine neurons encode vlue-relted signls nd re inhibited by versive events. Here we show tht this is true only for subset of dopmine neurons. We recorded the ctivity of dopmine neurons in monkeys (Mcc multt) during Pvlovin procedure with ppetitive nd versive outcomes (liquid rewrds nd irpuffs directed t the fce, respectively). We found tht some dopmine neurons were excited by rewrdpredicting stimuli nd inhibited by irpuff-predicting stimuli, s the vlue hypothesis predicts. However, greter number of dopmine neurons were excited by both of these stimuli, inconsistent with the hypothesis. Some dopmine neurons were lso excited by both rewrds nd irpuffs themselves, especilly when they were unpredictble. Neurons excited by the irpuff-predicting stimuli were locted more dorsolterlly in the substnti nigr prs compct, wheres neurons inhibited by the stimuli were locted more ventromedilly, some in the ventrl tegmentl re. A similr ntomicl difference ws observed for their responses to ctul irpuffs. These findings suggest tht different groups of dopmine neurons convey motivtionl signls in distinct mnners. If midbrin dopmine neurons ctully encode vlue-relted signls, their ctivity should be inhibited by versive stimuli becuse versive stimuli hve negtive motivtionl vlues. However, the results re inconsistent, some studies showing inhibitions 6 nd others showing both inhibitions nd excittions 7 11 by versive stimuli. Few of these studies exmined the effects of rewrds on the sme dopmine neurons 12,13, prtly becuse the nimls were nesthetized. To test whether dopmine neurons encode motivtionl vlues, we conditioned two monkeys using Pvlovin procedure with two distinct contexts (Fig. 1): one in which liquid rewrd ws expected (ppetitive block; Fig. 1) nd one in which n versive irpuff ws nticipted (versiveblock;fig.1b).inechblock,threeconditionedstimuliwere ssocited with the unconditioned stimulus (rewrd or irpuff) with 1%, % nd % probbility, respectively. These three conditioned stimuli were considered to convey three different levels of motivtionl vlue. In the ppetitive block, nticiptory licking incresed s the probbility of rewrd incresed (Fig. 1c). In the versive block, nticiptory blinking incresed s the probbility of irpuff incresed (Fig. 1d). While the monkeys were conditioned using the Pvlovin procedure, we recorded single-unit ctivity from 13 puttive dopmine neurons (68 in monkey N nd 3 in monkey D) in nd round the substnti nigr prs compct (SNc) nd ventrl tegmentl re (VTA). Their electrophysiologicl properties were distinctly different from other neurons in the SNc nd VTA (Supplementry Fig. 1), nd we henceforth cll them dopmine neurons. Most previous studies on midbrin dopmine neurons hve chrcterized dopmine neurons s functionlly homogeneous popultion 1. We found tht this is not true. In Fig. 2, e, we show the ctivity of two dopmine neurons, seprtely for different conditioned stimuli. Their b c Normlized mgnitude of licking ITI ITI Free rewrd Timing cue (1 s) Free irpuff Timing cue (1 s) % % 1% Rewrd probbility CS (1. s) CS (1. s) d Number of blinks 4 2 Rewrd Rewrd No rewrd Airpuff Airpuff No irpuff No rewrd or US or US No irpuff % % 1% Airpuff probbility Figure 1 Pvlovin procedure., Appetitive block. Three conditioned stimuli were ssocited with pple juice with 1%, % nd % probbility, respectively. b, Aversive block. Three conditioned stimuli were ssocited with n versive irpuff with 1%, % nd % probbility, respectively. In both blocks, ech tril strted fter the presenttion of timing cue (centrl smll spot) on the screen. After 1 s, the timing cue disppered nd one of the three conditioned stimuli ws presented. After 1. s, the conditioned stimulus disppered nd the unconditioned stimulus (rewrd or irpuff) ws delivered. In ddition to the cued trils, uncued trils were included in which rewrd lone (free rewrd) ws delivered during the ppetitive block nd n irpuff lone (free irpuff) ws delivered during the versive block. c, Averge normlized mgnitude of nticiptory licking during the presenttion of the rewrd-predicting conditioned stimuli for monkey D (solid line) nd monkey N (dshed line). d, Averge number of nticiptory blinks during the presenttion of the irpuff-predicting conditioned stimuli for monkey D (solid line) nd monkey N (dshed line). Double sterisks indicte significnt difference between two dt points (P,.1, Wilcoxon rnk-sum test). Error brs, s.d. ITI, inter-tril intervl; CS, conditioned stimulus; US, unconditioned stimulus. 1 Lbortory of Sensorimotor Reserch, Ntionl Eye Institute, Ntionl Institutes of Helth, Bethesd, Mrylnd , USA. 29 Mcmilln Publishers Limited. All rights reserved 837

2 LETTERS NATURE Vol June 29 1% rewrd CS % rewrd CS % rewrd CS e 1% rewrd CS % rewrd CS % rewrd CS 1% irpuff CS % irpuff CS % irpuff CS 1, 1, b Rewrd CS c Airpuff CS d f Rewrd CS g Airpuff CS h % % 1% Probbility Figure 2 Responses of dopmine neurons to conditioned stimuli., e, Activity of two exmple neurons in the ppetitive block (top row) nd versive block (bottom row), which were clssified s ACS-inhibited type () nd ACS-excited type (e). Histogrms (2-ms bins) nd rsters re ligned t the strt of the conditioned stimulus nd re shown for 1% rewrd CS, % rewrd CS, % rewrd CS, 1% irpuff CS, % irpuff CS nd % irpuff CS. b, c, Averged ctivity of 24 ACS-inhibited neurons. f, g, Averged ctivity of 38 ACS-excited neurons. Spike density functions re shown for 1% rewrd CS (red), % rewrd CS (pink) nd % rewrd CS (grey) in the ppetitive block (b, f), nd for 1% irpuff CS (drk blue), % ctivities were similr in the ppetitive block (top row). Both of them were excited by 1% rewrd conditioned stimulus (the conditioned stimulus ssocited with rewrd with 1% probbility). This excittion decresed in response to % rewrd conditioned stimulus, nd chnged to n inhibition in response to % rewrd conditioned stimulus. However, the dopmine neurons showed completely different responses in the versive block (bottom row). In response to 1% irpuff conditioned stimulus, the neuron shown in Fig. 2 ws inhibited wheres the neuron shown in Fig. 2e ws excited. Furthermore, s the probbility of irpuff decresed, their response mgnitudes were grded in opposite directions. To chrcterize the responses to conditioned stimuli, we clssified the 13 neurons into three groups bsed on the response to 1% irpuff conditioned stimulus (Supplementry Tble 1). Neurons showing significnt inhibition nd excittion were clssified s irpuff conditioned stimulus (ACS)-inhibited type (n 24) nd ACSexcited type (n 38), respectively (P,., Wilcoxon signed-rnk test). Neurons showing no significnt response were clssified s ACS-non-responsive type (n 41, P.., Wilcoxon signed-rnk test). The responses of individul neurons to conditioned stimuli re shown in Supplementry Fig. 2 nd Supplementry Tble 2. In the following, we will focus on the ACS-inhibited nd ACS-excited neurons (see Supplementry Fig. 3 for ACS-non-responsive neurons; see lso Supplementry Note A nd Supplementry Tble 3 for the electrophysiologicl properties of ech type). The verged ctivity of the ACS-inhibited neurons ws modulted by the rewrd probbility (Fig. 2b) nd irpuff probbility (Fig. 2c) in opposite directions. The excittory response to the rewrd-predicting conditioned stimuli decresed nd becme n inhibition s the rewrd probbility decresed (Fig. 2b, red line in Fig. 2d). By contrst, the inhibitory response to the irpuff-predicting conditioned stimuli decresed s the irpuff probbility decresed 838 1, Mcmilln Publishers Limited. All rights reserved 1% irpuff CS % irpuff CS % irpuff CS 1, 1, (Fig. 2c, blue line in Fig. 2d). The sme trend ws found in individul ACS-inhibited neurons (Supplementry Note B nd Supplementry Fig. 4). These results suggest tht the ACS-inhibited neurons encode motivtionl vlue on single scle, nd re most strongly excited in response to the most positive stimulus (1% rewrd conditioned stimulus) nd most strongly inhibited in response to the most negtive stimulus (1% irpuff conditioned stimulus). The verged ctivity of the ACS-excited neurons ws lso modulted by the rewrd probbility (Fig. 2f) nd irpuff probbility (Fig. 2g), but in the sme direction. The excittory response decresed s the outcome probbility decresed for both rewrdpredicting nd irpuff-predicting conditioned stimuli (Fig. 2h; see lso Supplementry Note B nd Supplementry Fig. 4b for individul neurons). These results suggest tht the ACS-excited neurons do not encode motivtionl vlue. Previous studies hve repetedly shown tht dopmine neurons re excited by rewrd when it is unexpected 1. However, it is still debtble whether they re excited or inhibited by versive stimuli nd, if so, in wht context. Figure 3 shows the responses to rewrd nd irpuff of the sme neuron shown in Fig. 2. This neuron ws strongly excited when rewrd ws presented without preceding conditioned stimulus (free rewrd) nd inhibited when irpuff ws presented without preceding conditioned stimulus (free irpuff), consistent with vlue coding. By contrst, the neuron shown in Fig. 3e ws excited by both free rewrd nd free irpuff. We then reclssified the 13 neurons into three groups on the bsis of the response to free irpuff (Supplementry Tble 1). Neurons showing significnt inhibition nd excittion were clssified s irpuff unconditioned stimulus (AUS)-inhibited type (n 47) nd AUS-excited type (n 11), respectively (P,., Wilcoxon signed-rnk test). Neurons showing no significnt response were clssified s AUS-nonresponsive type (n 4, P.., Wilcoxon signed-rnk test). The * 1, % % 1% Probbility irpuff CS (light blue) nd % irpuff CS (grey) in the versive block (c, g). Grey res indicte the period tht ws used to nlyse responses to conditioned stimuli. d, h, The mgnitudes of the responses of the ACSinhibited neurons (d) nd ACS-excited neurons (h) to the rewrd-predicting conditioned stimuli (red) nd irpuff-predicting conditioned stimuli (blue). Filled symbols indicte significnt devition from zero (P,., Wilcoxon signed-rnk test). Red nd blue sterisks indicte significnt difference between two responses for the rewrd-predicting nd irpuffpredicting conditioned stimuli, respectively (double sterisk, P,.1; single sterisk, P,.; Wilcoxon signed-rnk test). Error brs, s.d.

3 NATURE Vol June 29 LETTERS 1% rewrd % rewrd Free rewrd e 1% rewrd % rewrd Free rewrd % irpuff % irpuff Free irpuff 1, 1, 1, 2 1% irpuff % irpuff Free irpuff 2 1, 1, 1, b Rewrd c Airpuff d f Rewrd g Airpuff h Free % 1% Probbility Figure 3 Responses of dopmine neurons to unconditioned stimuli., e, Activity of two exmple neurons in the ppetitive block (top row) nd versive block (bottom row), which were clssified s AUS-inhibited type () nd AUS-excited type (e). Histogrms nd rsters re ligned t the strt of the unconditioned stimulus nd re shown for 1% rewrd, % rewrd, free rewrd, 1% irpuff, % irpuff nd free irpuff. b, c, Averged ctivity of 47 AUS-inhibited neurons. f, g, Averged ctivity of 11 AUSexcited neurons. Spike density functions re shown for 1% rewrd (red), responses of individul neurons to unconditioned stimuli re shown in Supplementry Fig. nd Supplementry Tble 2. We note tht this clssifiction differs from tht bsed on the response to 1% irpuff conditioned stimulus. In the following, we will focus on the AUSinhibited nd AUS-excited neurons (see Supplementry Figs 6 nd 7 for AUS-non-responsive neurons, see lso Supplementry Note C nd Supplementry Tble 4 for the electrophysiologicl properties of ech type). The verged responses to the rewrd nd irpuff re shown for the AUS-inhibited neurons in Fig. 3b, c nd for the AUS-excited neurons in Fig. 3f, g. In both types, the excittory response to rewrd disppered when the rewrd ws completely predictble by following 1% rewrd conditioned stimulus, nd decresed when the rewrd ws prtly predictble by following % rewrd conditioned stimulus (Fig. 3b, f). This is consistent with the rewrd-prediction-error hypothesis tht the ctivity of dopmine neurons represents the difference between the expected nd ctul vlues of rewrd 14,1. The AUS-inhibited neurons ppered to encode prediction error even for versive outcomes, lbeit prtly, becuse these neurons were inhibited by n unexpected versive irpuff (free irpuff; Fig. 3c) nd this inhibitory response decresed monotoniclly s the irpuff becme more predictble (Fig. 3d; see Supplementry Note D nd Supplementry Fig. 8 for individul neurons). We note tht the excittory response of the AUS-excited neurons to the irpuff lso decresed s the irpuff becme more predictble (Fig. 3g, h; see Supplementry Note D nd Supplementry Fig. 8b for individul neurons). The prediction-error hypothesis predicts tht when n outcome is unexpectedly omitted, neurons should respond in the direction opposite to tht in which they respond when the sme outcome is unexpectedly delivered 14,1. We found tht AUS-inhibited neurons tended to show this kind of response to both rewrd omission nd * Free % 1% Probbility % rewrd (pink) nd free rewrd (grey) in the ppetitive block (b, f), nd for 1% irpuff (drk blue), % irpuff (light blue) nd free irpuff (grey) in the versive block (c, g). Grey res indicte the period tht ws used to nlyse responses to unconditioned stimuli. d, h, The mgnitudes of the responses of the AUS-inhibited neurons (d) nd AUS-excited neurons (h) to rewrd (red) nd irpuff (blue). Significnce mesures nd error brs re the sme s Fig. 2d, h. 3 mm Response to 1% irpuff CS (spikes per second) Recording depth (mm) 29 Mcmilln Publishers Limited. All rights reserved b c Recording depth (mm) Figure 4 Loctions of dopmine neurons in reltion to their responses to irpuff-predicting conditioned stimulus., Recording sites of 68 dopmine neurons in monkey N re plotted on five coronl sections shown rostrocudlly from left to right (intervl, 1 mm). Red circles indicte neurons showing significnt excittions to 1% irpuff CS (tht is, ACSexcited neurons). Blue circles indicte neurons showing significnt inhibitions to 1% irpuff CS (tht is, ACS-inhibited neurons). White circles, no significnce (tht is, ACS-non-responsive neurons). Blck lines indicte electrode penetrtion trcks, which were tilted lterlly by 3u. b, c, Reltion between recording depth nd the response to 1% irpuff CS for monkey N (b) nd monkey D (c). Red, blue nd white circles indicte ACS-excited, ACS-inhibited nd ACS-non-responsive neurons, respectively. The recording depth ws mesured from reference depth set by mnipultor to dvnce the recording electrode. * 839

4 LETTERS NATURE Vol June 29 irpuff omission, wheres AUS-excited neurons showed no response to the omission of the outcome (Supplementry Note E nd Supplementry Fig. 9). The current consensus, tht dopmine neurons crry rewrdrelted informtion, is thought to hold for ll dopmine neurons locted in the midbrin, including both the SNc nd the VTA 1. Becuse we hve found types of dopmine neuron tht differ with regrd to their responses to versive events, we now sk whether they were locted in different regions in the midbrin. Figure 4 shows the recording sites of the 68 dopmine neurons in monkey N in reltion to the response to 1% irpuff conditioned stimulus. Neurons showing significnt excittion (tht is, ACS-excited neurons; red circles) tended to be locted in the more dorsolterl prt, nd neurons showing significnt inhibition (tht is, ACS-inhibited neurons; blue circles) tended to be locted in the more ventromedil prt. To test this trend sttisticlly, we exmined the reltion between the recording depth nd the response to 1% irpuff conditioned stimulus for monkey N (Fig. 4b) nd monkey D (Fig. 4c). As shown by the sctter plots, significnt negtive correltion ws found for both monkeys (monkey N: correltion coefficient r 2., P,.1; monkey D: r 2.7, P,.1). This negtive correltion confirmed the dorsolterl ventromedil differentition of the excittory nd inhibitory responses evoked in dopmine neurons by the irpuffpredicting conditioned stimulus. Similr loction differences were found in reltion to response to irpuff itself (Supplementry Note F nd Supplementry Fig. 1). It hs generlly been ssumed tht midbrin dopmine neurons form unified functionl group, ll representing rewrd-relted signls in similr mnner 1. Our results re roughly consistent with this ide s fr s the rewrd-relted signls re concerned. However, cler heterogeneity ws reveled when we exmined their responses to versive events. We found two types of dopmine neuron, one inhibited nd the other excited by irpuff or its predictor. This suggests tht the unified concept of dopmine neurons needs to be chnged (see Supplementry Note G for the reltionship between our findings nd previous studies). We propose tht there re t lest two functionl groups of dopmine neurons. Dopmine neurons in the first group (irpuff-inhibited type, tht is, ACS- nd AUS-inhibited types) would represent motivtionl vlue. Their responses co-vried with prediction errors ssocited with both rewrd nd irpuff, nd therefore would be useful in lerning to pproch rewrds nd void versive stimuli. The function of the second group (irpuff-excited type, tht is, ACS- nd AUSexcited types) is not immeditely cler, but we found tht their response to the conditioned stimulus ws correlted with the ltency of the monkey s orienting response (gze shift) to the conditioned stimulus nd tht this correltion ppered only fter the conditioned stimulus ws pired with rewrd or irpuff (Supplementry Note H nd Supplementry Fig. 11). These results rise the possibility tht the responses of the irpuff-excited dopmine neurons to conditioned stimulus reflect the motivtionl slience of the conditioned stimulus. However, this interprettion my not be vlid for the responses of these neurons to unconditioned stimulus or its omission. We note tht the two types of dopmine neuron were distributed differently, the irpuff-excited type in the dorsolterl region in the SNc nd the irpuff-inhibited type in the ventromedil region in the SNc s well s the VTA (see Supplementry Note I for detils). In monkeys 16 nd rts 17, dopmine neurons in the dorsolterl SNc project minly to the dorsl stritum, wheres those in the ventromedil SNc nd VTA project minly to the ventrl stritum. The irpuff-inhibited dopmine neurons in the ventromedil region in the SNc nd VTA my thus trnsmit vlue-relted informtion to the ventrl stritum, which is thought to process rewrd vlues 18 2.On the other hnd, the irpuff-excited dopmine neurons in the dorsolterl region in the SNc respond to motivtionlly slient stimuli, whether they re ppetitive or versive, nd send the signl to the dorsl stritum, which is relted to orienting behviour This my 84 be prt of the mechnism by which orienting behviour such s sccdic eye movement is induced by motivtionlly slient stimuli 24. The two types of dopmine neuron my receive inputs from different sources. The irpuff-excited dopmine neurons my receive inputs from res such s the bsl forebrin, in which neurons lso show excittory responses to both ppetitive nd versive events 2,26 (see Supplementry Note J for further discussion). The irpuff-inhibited dopmine neurons my receive inputs, t lest prtly, from the lterl hbenul. Using the sme Pvlovin procedure, we hve shown tht lterl hbenul neurons re excited by the irpuff-predicting conditioned stimulus nd inhibited by the rewrd-predicting conditioned stimulus, indicting tht they encode motivtionl vlue similrly to the irpuff-inhibited dopmine neurons, but in the opposite mnner 27.The vlue signls in the lterl hbenul would then be trnsmitted to the dopmine neurons by inhibiting them 28, nd this effect ws stronger on dopmine neurons locted in the ventromedil SNc or the VTA, where the irpuff-inhibited type domintes (Supplementry Note K nd Supplementry Fig. 12). So fr, we hve clssified dopmine neurons into two types. However, the rel picture is more complex. First, the difference between the two types ws not distinct; there ws nother group of dopmine neurons tht did not belong to either type (tht is, the type nonresponsive to irpuff or its predictor). Second, the clssifiction ws different for conditioned nd unconditioned stimuli (Supplementry Note L, Supplementry Tble 1 nd Supplementry Fig. 13b). More neurons were excited by the irpuff-predicting conditioned stimulus, wheres more neurons were inhibited by the irpuff itself. This might indicte flexible opertion of the dopmine system. If slient stimulus (tht is, conditioned stimulus) is presented, it would be beneficil to orient ttention to the stimulus nd judge whether it predicts rewrding event or n versive event. This is the time when mjority of dopmine neurons re excited, thus promoting the orienting behviour. If n versive event occurs (tht is, unconditioned stimulus), it would be crucil to lern to void the ction tht led to the versive event. This is the time when mjority of dopmine neurons re inhibited, thus promoting voidnce lerning. METHODS SUMMARY Two dult rhesus monkeys (Mcc multt) were used for the experiments. All procedures for niml cre nd experimenttion were pproved by the Animl Cre nd Use Committee of the Ntionl Eye Institute nd complied with the Public Helth Service Policy on the humne cre nd use of lbortory nimls. A plstic hed holder nd plstic recording chmber were fixed to the skull under generl nesthesi nd sterile surgicl conditions. The recording chmber ws plced over the frontoprietl cortex, tilted lterlly by 3u, nd imed t the SNc nd VTA. Two serch coils were surgiclly plced under the conjunctiv of the eyes. The hed holder, the recording chmber nd the eye coil connectors were ll embedded in dentl crylic tht covered the top of the skull, nd were connected to the skull using crylic screws. We conditioned two monkeys using Pvlovin procedure with n ppetitive unconditioned stimulus (liquid rewrd) nd n versive unconditioned stimulus (irpuff). During the Pvlovin procedure, we recorded the ctivity of dopmine neurons in nd round the SNc nd VTA. We estimted the position of the SNc nd VTA by mgnetic resonnce imging nd identified dopmine neurons by their electrophysiologicl properties. After the end of recording sessions in one monkey, we confirmed the recording sites histologiclly. We nlysed nticiptory licking, nticiptory blinking nd neuronl responses during the Pvlovin procedure. We focused on three kinds of neuronl responses: (1) responses elicited by conditioned-stimulus presenttion, (2) responses elicited by unconditionedstimulus delivery nd (3) responses elicited by unconditioned-stimulus omission. Detils of the Pvlovin procedure, identifiction of dopmine neurons, nlysis methods, nd histologicl procedure cn be found in the full Methods. Full Methods nd ny ssocited references re vilble in the online version of the pper t Received 26 Jnury; ccepted 27 Mrch 29. Published online 17 My Mcmilln Publishers Limited. All rights reserved 1. Schultz, W. Predictive rewrd signl of dopmine neurons. J. Neurophysiol. 8, 1 27 (1998).

5 NATURE Vol June 29 LETTERS 2. Stoh, T., Nki, S., Sto, T. & Kimur, M. Correlted coding of motivtion nd outcome of decision by dopmine neurons. J. Neurosci. 23, (23). 3. Tkikw, Y., Kwgoe, R. & Hikosk, O. A possible role of midbrin dopmine neurons in short- nd long-term dpttion of sccdes to position-rewrd mpping. J. Neurophysiol. 92, (24). 4. Morris, G., Arkdir, D., Nevet, A., Vdi, E. & Bergmn, H. Coincident but distinct messges of midbrin dopmine nd stritl toniclly ctive neurons. Neuron 43, (24).. Tobler, P. N., Fiorillo, C. D. & Schultz, W. Adptive coding of rewrd vlue by dopmine neurons. Science 37, (2). 6. Ungless, M. A., Mgill, P. J. & Bolm, J. P. Uniform inhibition of dopmine neurons in the ventrl tegmentl re by versive stimuli. Science 33, (24). 7. Chiodo, L. A., Antelmn, S. M., Cggiul, A. R. & Lineberry, C. G. Sensory stimuli lter the dischrge rte of dopmine (DA) neurons: evidence for two functionl types of DA cells in the substnti nigr. Brin Res. 189, (198). 8. Coizet, V., Dommett, E. J., Redgrve, P. & Overton, P. G. Nociceptive responses of midbrin dopminergic neurones re modulted by the superior colliculus in the rt. Neuroscience 139, (26). 9. Schultz, W. & Romo, R. Responses of nigrostritl dopmine neurons to highintensity somtosensory stimultion in the nesthetized monkey. J. Neurophysiol. 7, (1987). 1. Mntz, J., Thierry, A. M. & Glowinski, J. Effect of noxious til pinch on the dischrge rte of mesocorticl nd mesolimbic dopmine neurons: selective ctivtion of the mesocorticl system. Brin Res. 476, (1989). 11. Gurrci, F. A. & Kpp, B. S. An electrophysiologicl chrcteriztion of ventrl tegmentl re dopminergic neurons during differentil Pvlovin fer conditioning in the wke rbbit. Behv. Brin Res. 99, (1999). 12. Mirenowicz, J. & Schultz, W. Preferentil ctivtion of midbrin dopmine neurons by ppetitive rther thn versive stimuli. Nture 379, (1996). 13. Joshu, M., Adler, A., Mitelmn, R., Vdi, E. & Bergmn, H. Midbrin dopminergic neurons nd stritl cholinergic interneurons encode the difference between rewrd nd versive events t different epochs of probbilistic clssicl conditioning trils. J. Neurosci. 28, (28). 14. Schultz, W., Dyn, P. & Montgue, P. R. A neurl substrte of prediction nd rewrd. Science 27, (1997). 1. Montgue, P. R., Dyn, P. & Sejnowski, T. J. A frmework for mesencephlic dopmine systems bsed on predictive Hebbin lerning. J. Neurosci. 16, (1996). 16. Lynd-Blt, E. & Hber, S. N. The orgniztion of midbrin projections to the stritum in the primte: sensorimotor-relted stritum versus ventrl stritum. Neuroscience 9, (1994). 17. Ikemoto, S. Dopmine rewrd circuitry: two projection systems from the ventrl midbrin to the nucleus ccumbens olfctory tubercle complex. Brin Res. Rev. 6, (27). 18. Knutson, B., Adms, C. M., Fong, G. W. & Hommer, D. Anticiption of incresing monetry rewrd selectively recruits nucleus ccumbens. J. Neurosci. 21, RC19 (21). 19. Cromwell, H. C. & Schultz, W. Effects of expecttions for different rewrd mgnitudes on neuronl ctivity in primte stritum. J. Neurophysiol. 89, (23). 2. Schultz, W., Apicell, P., Scrnti, E. & Ljungberg, T. Neuronl ctivity in monkey ventrl stritum relted to the expecttion of rewrd. J. Neurosci. 12, (1992). 21. Kitm, T., Ohno, T., Tnk, M., Tsubokw, H. & Yoshid, K. Stimultion of the cudte nucleus induces contrversive sccdic eye movements s well s hed turning in the ct. Neurosci. Res. 12, (1991). 22. Hikosk, O., Tkikw, Y. & Kwgoe, R. Role of the bsl gngli in the control of purposive sccdic eye movements. Physiol. Rev. 8, (2). 23. Crli, M., Evenden, J. L. & Robbins, T. W. Depletion of unilterl stritl dopmine impirs initition of contrlterl ctions nd not sensory ttention. Nture 313, (198). 24. Hollnd, P. C. & Gllgher, M. Amygdl circuitry in ttentionl nd representtionl processes. Trends Cogn. Sci. 3, 6 73 (1999). 2. Lin, S. C. & Nicolelis, M. A. Neuronl ensemble bursting in the bsl forebrin encodes slience irrespective of vlence. Neuron 9, (28). 26. Richrdson, R. T. & DeLong, M. R. Electrophysiologicl studies of the functions of the nucleus bslis in primtes. Adv. Exp. Med. Biol. 29, (1991). 27. Mtsumoto, M. & Hikosk, O. Representtion of negtive motivtionl vlue in the primte lterl hbenul. Nture Neurosci. 12, (29). 28. Mtsumoto, M. & Hikosk, O. Lterl hbenul s source of negtive rewrd signls in dopmine neurons. Nture 447, (27). Supplementry Informtion is linked to the online version of the pper t Acknowledgements We thnk S. Hong, E. Bromberg-Mrtin, M. Ysud, S. Ymmoto nd Y. Tchibn for discussion, M. K. Smith for his histologicl expertise, J. W. McClurkin, A. M. Nichols, T. W. Ruffner, A. V. Hys nd L. P. Jensen for technicl ssistnce, nd G. Tnsey, D. Prker nd B. Ngy for niml cre. This reserch ws supported by the Intrmurl Reserch Progrm t the Ntionl Institutes of Helth, Ntionl Eye Institute. Author Contributions M.M. designed the Pvlovin procedure, performed the experiments nd nlysed the dt. O.H. supported ll of these processes. M.M. nd O.H. discussed the results nd wrote the mnuscript. Author Informtion Reprints nd permissions informtion is vilble t Correspondence nd requests for mterils should be ddressed to M.M. (mtsumotom@nei.nih.gov). 29 Mcmilln Publishers Limited. All rights reserved 841

6 doi:1.138/nture828 METHODS Pvlovin procedure. Our Pvlovin procedure consisted of two blocks of trils, n ppetitive block (Fig. 1) nd n versive block (Fig. 1b). In the ppetitive block, three conditioned stimuli (red circle, green cross nd blue squre for monkey N; yellow ring, cyn tringle nd blue squre for monkey D) were ssocited with liquid rewrd (pple juice) s n unconditioned stimulus with 1%, % nd % probbility, respectively. In the versive block, three conditioned stimuli (yellow ring, cyn tringle nd blue squre for monkey N; red circle, green cross nd blue squre for monkey D) were ssocited with n irpuff directed t the monkey s fce s n unconditioned stimulus with 1%, % nd % probbility, respectively. The liquid rewrd ws delivered through spout tht ws positioned in front of the monkey s mouth. The irpuff (2 3 p.s.i.) ws delivered through nrrow tube plced 6 7 cm from the fce. Ech tril strted fter the presenttion of timing cue for both blocks. The monkeys were not required to look t the timing cue. After 1 s, the timing cue disppered nd one of the three conditioned stimuli ws presented pseudorndomly. After 1. s, the conditioned stimulus disppered nd the unconditioned stimulus ws delivered. In ddition to the cued trils, uncued trils were included in which rewrd lone (free rewrd) ws delivered during the ppetitive block nd n irpuff lone (free irpuff) ws delivered during the versive block. All trils were presented with rndom inter-tril intervl tht verged s (3 7 s) for monkey N nd 4. s (3 6 s) for monkey D. One block consisted of 42 trils with fixed proportions of tril types (1%, 12 trils; %, 12 trils; %, 12 trils; uncued, 6 trils). For % trils, the conditioned stimulus ws followed by the unconditioned stimulus in six trils nd ws not followed by the unconditioned stimulus in the other six trils. The block chnged without ny externl cue. For ech neuron, we collected dt by repeting the ppetitive nd versive blocks twice or more. We monitored licking nd blinking of the monkeys. To monitor licking, we ttched strin guge to the rewrd spout nd mesured strins on the spout resulting from licking. To monitor blinking, mgnetic-serch-coil technique ws used. A smll Teflon-coted stinless-steel wire (, mm in dimeter, five or six turns) ws tped to n eyelid. Eye closure ws identified by the verticl component of the eyelid-coil signl. Identifiction of dopmine neurons. We serched for dopmine neurons in nd round the SNc nd VTA. Dopmine neurons were identified by their irregulr firing, tonic bseline ctivity round five spikes per second, brod spike potentil nd phsic excittion to free rewrd. Dt nlysis. We nlysed nticiptory licking, nticiptory blinking nd neuronl ctivity during the Pvlovin procedure. To evlute the frequency nd strength of nticiptory licking, the strin-guge signl ws used. We first clculted the velocity of the signl chnge under licking. Then we integrted the bsolute velocity during conditioned-stimulus presenttion for ech tril. This integrted velocity becomes lrger if the monkeys more frequently nd strongly lick the spout. We defined this vlue s the mgnitude of nticiptory licking in the tril. The mgnitude ws normlized ccording to the following formul: normlized mgnitude equls (X 2 Min)/(Mx 2 Min). Here X is the mgnitude of nticiptory licking in the tril, Mx is the mximum mgnitude in the recording session nd Min is the minimum mgnitude in the recording session. To count the number of nticiptory blinks during conditioned-stimulus presenttion, the verticl component of the eyelid signl ws used. We first clculted the downwrd velocity of eyelid movement. We set threshold nd counted how mny times the velocity crossed the threshold during conditioned-stimulus presenttion for ech tril. This count ws defined s the number of nticiptory blinks in the tril. In nlyses of neuronl ctivity, responses to ech conditioned stimulus were defined s the dischrge rte during the intervl 1 to 32 ms fter conditioned stimulus onset minus the bckground dischrge rte during the 2 ms before conditioned stimulus onset. Response to rewrd ws defined s the dischrge rte during the intervl 2 to 4 ms fter rewrd onset minus the bckground dischrge rte during the 2 ms before rewrd onset. Response to irpuff ws defined s the dischrge rte during the intervl to 2 ms fter irpuff onset minus the bckground dischrge rte during the 2 ms before irpuff onset. Response to rewrd omission ws defined s the dischrge rte during the intervl 2 to ms fter the conditioned stimulus ended minus the bckground dischrge rte during the 2 ms before the conditioned stimulus ended. Response to irpuff omission ws defined s the dischrge rte during the intervl 1 to 3 ms fter the conditioned stimulus ended minus the bckground dischrge rte during the 2 ms before the conditioned stimulus ended. These time windows were determined on the bsis of the verged ctivity of dopmine neurons. Specificlly, we set the time windows such tht they include mjor prts of the excittory nd inhibitory responses. Becuse the % rewrd conditioned stimulus nd % irpuff conditioned stimulus were physiclly identicl, they could only be distinguished by the block context (ppetitive block or versive block). Therefore, to nlyse responses to % rewrd conditioned stimulus nd % irpuff conditioned stimulus, we excluded ll trils with the % rewrd conditioned stimulus or the % irpuff conditioned stimulus tht were presented before the block context could be known, tht is, before the block s first presenttion of 1% conditioned stimulus, % conditioned stimulus or free outcome. We chrcterized the electrophysiologicl properties of recorded neurons by (1) bseline firing rte, (2) irregulrity of firing pttern nd (3) spike wveform. Bseline firing rte is the men firing rte during the 2 ms before the onset of the timing cue. To quntify irregulrity of firing pttern, we used n irregulrity metric introduced in ref. 29 nd clled IR. First, interspike intervl (ISI) ws computed s follows: if spike i 2 1, spike i nd spike i 1 1 occurred in this order, the intervl between spike i 2 1 nd spike i corresponds to ISI i nd the intervl between spike i nd spike i 1 1 corresponds to ISI i11. Second, the difference between djcent ISIs ws computed s jlog(isi i /ISI i11 )j. This vlue ws then ssigned to the time spike i occurred. Thus, smll IR vlues indicte regulr firing nd lrge IR vlues indicte irregulr firing. We then computed medin of ll IR vlues during the inter-tril intervl (during the 1, ms before timing-cue onset). To quntify spike wveform, we mesured the spike durtion of 67 dopmine neurons (whose spike wveforms were successfully recorded). The typicl spike consisted of the following wves: first, shrp negtive; second, shrp positive; third, slow negtive; fourth, slow positive. We mesured the spike durtion from the pek of the first wve (shrp negtive) to the pek of the third wve (slow negtive). Histology. After the end of the recording session in monkey N, we selected representtive loctions for electrode penetrtion. When typicl dopmine ctivity ws recorded, we mde electrolytic microlesions t the recording sites (12 ma nd 3 s). Then monkey N ws deeply nesthetized using pentobrbitl sodium, nd perfused with 1% formldehyde. The brin ws blocked nd equilibrted with 1% sucrose. Frozen sections were cut every mm in the coronl plne. The sections were stined with cresyl violet. 29. Dvies, R. M., Gerstein, G. L. & Bker, S. N. Mesurement of time-dependent chnges in the irregulrity of neurl spiking. J. Neurophysiol. 96, (26). 29 Mcmilln Publishers Limited. All rights reserved