Abstract reward and punishment representations in the human orbitofrontal cortex

Transcription

1 rticles Astrct rewrd nd punishment representtions in the humn oritofrontl cortex J. O Doherty 1,2, M. L. Kringelch 1,2, E. T. olls 1, J. Hornk 1 nd C. Andrews 2 1 Deprtment of Experimentl Psychology, University of Oxford, South Prks od, Oxford OX1 3UD, UK 2 Oxford Centre for Functionl Mgnetic esonnce Imging (FMIB), John dcliffe Hospitl, Hedley Wy, Oxford OX3 9DU, UK Correspondence should e ddressed to E.T.. (Edmund.olls@psy.ox.c.uk) The oritofrontl cortex (OFC) is implicted in emotion nd emotion-relted lerning. Using eventrelted functionl mgnetic resonnce imging (fmi), we mesured rin ctivtion in humn sujects doing n emotion-relted visul reversl-lerning tsk in which choice of the correct stimulus led to proilisticlly determined monetry rewrd nd choice of the incorrect stimulus led to monetry loss. Distinct res of the OFC were ctivted y monetry rewrds nd punishments. Moreover, in these res, we found correltion etween the mgnitude of the rin ctivtion nd the mgnitude of the rewrds nd punishments received. These findings indicte tht one emotionl involvement of the humn oritofrontl cortex is its representtion of the mgnitudes of strct rewrds nd punishments, such s receiving or losing money. Dmge to the OFC (long with other sectors of the ventrl nd medil prefrontl cortex) in humns is ssocited with impirments in emotionl nd socil ehvior chrcterized y disinhiition, socil inppropriteness nd irresponsiility 1 4. This dmge in humns is lso ssocited with impirments in gmling tsks. In these tsks, ptients choose from set of stimuli nd, on the sis of the monetry rewrd nd loss otined following ech selection, lern to dptively choose those stimuli tht mximize the overll monetry rewrd, nd minimize the monetry loss otined in the tsk 5,6. A fundmentl deficit shown y non-humn primtes nd humns with OFC lesions is difficulty in reversing ehviorl responses to ojects ssocited with rewrds nd punishments, following reversl of the reinforcement contingencies 4,7 10. Such impirments my e due to representtion in the OFC of rewrds nd punishments tht re received, nd involvement of the OFC in lerning nd updting ssocitions etween visul stimuli nd rewrding nd punishing outcomes 3. Consistent with this hypothesis re findings from single-cell neurophysiologicl investigtions in non-humn primtes tht the rewrd vlue of tste, olfctory nd visul stimuli is represented in the oritofrontl cortex, nd tht some neurons respond only when reinforcement contingencies chnge 3,11. However, less is known out the nture of the representtions in the humn rin for rewrd nd punishment, especilly for strct types of rewrd nd punishment (such s prise or losing money). Previous PET (positron emission tomogrphy) imging studies hve found tht the humn OFC cn e ctivted with monetry rewrd 12, nd with written positive nd negtive feedck during performnce of guessing tsk 13. However, very little is known out whether strct punishment, such s losing money, ctivtes the humn OFC. If it does, then the question is rised of whether the representtions for monetry rewrd nd punishment in the OFC re distinct or overlpping. Further, it is not known how the mgnitudes of monetry rewrds nd punishments received fter choice re represented in the rin; the mount won or lost on ny tril clerly is importnt informtion for the computtion of the overll rewrd vlue ssocited with prticulr stimulus. To ddress these issues, we used event-relted fmi to investigte the involvement of the OFC while sujects did vrint of stndrd reversl lerning tsk (Fig. 1 nd ; Methods), in which symolic monetry gins nd losses were used s the rewrds nd punishments. Sujects were informed in dvnce tht they would not receive ny ctul remunertion sed on the totl money ccumulted throughout the experiment, ut were encourged to try to do well in the tsk. (The verge totl money gined y the sujects t the end of the tsk ws 5061.) The event-relted fmi design used here offered considerle dvntges over the previous PET studies descried ove, s fmi offers etter sptil nd temporl resolution, nd enles trily-tril mesurement of the individul hemodynmic responses relted to the receipt of ech rewrd or punishment 14. Our fmi design therefore ws cple of showing whether rin ctivtion in prticulr regions followed ech rewrd or punishment received. This is in contrst to PET studies, which, ecuse of the long imging period nd lock design, cnnot show whether ny effect is due to some ongoing generl effect of eing in the rewrd condition, to nticiption of rewrd, or to the rewrd ctully received. In ddition, our event-relted design enled us to correlte the BOLD signl with the individul rewrds nd punishments received, to determine where in the rin the mgnitudes of the rewrds nd punishments re represented. To chieve this, the monetry gins nd losses were distriuted ccording to differing proilistic reinforcement schedules nd rnges of mgnitude for the rewrding stimulus (S+) nd punishing stimulus (S ). An importnt feture of this tsk ws tht money could e won or lost on oth the S+ nd the S, ut choosing the S+ gve lrger rewrds nd smller punishments, wheres the converse nture neuroscience volume 4 no 1 jnury

2 rticles S+ S ewrd:punishment rtio 70:30 40:60 ewrd rnges Punishment rnges Fig. 1. Experimentl design. () Two unfmilir nd esily discernle frctl ptterns were displyed on the screen, with the totl score displyed numericlly nd grphiclly s r chrt. After suject hd selected stimulus, flshing text messge ws superimposed ove the selected stimulus, indicting how much money the suject hd won or lost. The two frctls were rndomly ssigned to the top or the ottom of the screen on ech tril. At the eginning of the tsk, one of the stimuli ws designted s the S+. Whether tril ws rewrded or punished ws decided on the sis of pseudo-rndom sequence ccording to the rtios shown in (). The mgnitude of the rewrds nd punishments were lso vried using uniform distriution etween the rnges shown in the sme figure. The specific rtios nd rnges used were chosen on the sis of pilot experiments to give the tsk n pproprite level of difficulty. Consistent selection of the current S+ resulted in n overll monetry gin, wheres consistent selection of the S resulted in n overll monetry loss. The criterion for successful cquisition of the S+ in the tsk ws four of the previous five consecutive responses to the S+. After etween one nd three trils following criterion (rndomly determined), the rewrd contingencies were reversed so tht the old S+ ecme the new S nd the old S ecme the new S+. The suject s tsk ws, y tril nd error, to determine which stimulus ws the more profitle to choose, nd to keep trck of this when reversl occurred. The different event types used in the sttisticl comprisons re shown in (c). ws true of the S. The sujects tsk ws, y tril nd error, to determine which stimulus ws the more profitle to choose, to keep trck of the profitility, nd to reverse their choice when we reversed the rewrd nd punishment contingencies. The specific rtios nd rnges of rewrds nd punishments were chosen on the sis of pilot experiments; the tsk ws given n pproprite level of difficulty, sensitive enough to detect impirments in ptients with OFC dmge (J.O., J.H., E.T.., D. Wde nd J. McGrth, unpulished dt). The tsk ws designed with proilistic reinforcement contingencies to mke it less likely tht sujects would use lterntive cognitive strtegies, such s use of explicit verl coding ( stimulus x gives rewrd ), to solve the tsk. During the sme scnning session, sujects lso prticipted in control tsk in which they mde similr responses without ssurnce of monetry rewrd or punishment. c ewrd Punishment Acquisition phse S+ ewrd cquisition Punishment cquisition [ewacq] [PunAcq] eversl phse Old S+ ewrd reversl Punishment reversl [new S ] [ewev] [Punev] ESULTS The principl events of interest (Fig. 1c) were s follows: first, lrge rewrd otined during the cquisition phse, when the sujects were consistently touching the S+ (ewacq); second, smll punishment otined during the cquisition phse (PunAcq); third, lrge punishment otined during the reversl phse when the contingencies hve reversed (Punev); fourth, smll rewrd otined during the reversl phse (ewev). Sttisticl comprisons of the BOLD ctivtions relted to these different events were sed on group rndom effects with sttistic threshold t, which followed t-tests done for ech suject, for ech comprison (Methods). Comprisons considered first re those tht show the rin res more ctivted following rewrding outcome thn punishing outcome. When the rewrd cquisition events were compred to the punish reversl events (ewacq Punev), voxels were found to survive the sttisticl threshold in the medil OFC ilterlly (Fig. 2; this ws the cse in ll nine of the individul sujects). We lso found significntly ctivted voxels in the medil OFC when compring the rewrd cquisition events to the punishment cquisition events (ewacq PunAcq), demonstrting tht the ctivtion ws genuine response to the receipt of the rewrd, rther thn response to the differences etween the cquisition nd reversl phses of the tsk. The verged percent chnge time course of significntly ctivted voxels in the medil OFC (Fig. 3) showed cler increse in the BOLD signl following rewrd, with lso some evidence of decrese in the BOLD signl following punishment. To determine whether the ctivtion in the medil OFC ws relted to the mgnitude of the rewrd otined, we mesured the correltion etween the mgnitude of the rewrds nd punishments on ech tril nd the corresponding BOLD signl on ech tril in ech voxel (see Methods). The group nlysis (Fig. 4) showed tht left medil OFC ctivtion ws correlted with the mgnitude of the rewrd otined. (Significntly ctivted clusters of voxels showed this effect in the individul suject nlyses for six of the nine sujects, t threshold of, nd n extent threshold of p < 0.05.) This finding indictes tht n increse in neurl ctivity (s mesured y the BOLD signl) in the medil OFC is prticulrly relted to the receipt of rewrd fter selecting stimulus, nd tht the mgnitude of this ctivity increses with the mgnitude of the rewrd. To exmine the nture of this correltion further, we show in Fig. 5 the men percent chnge of the BOLD signl for different mgnitudes of rewrd or punishment in the most significnt voxels in the sutrction nlysis shown in Fig. 2. It is evident from Fig. 5 (for the pek ewacq Punev voxels cross sujects) tht there ws grded increse in these voxels ccording to the mgnitude of the rewrd, nd grded decrese ccording to the mgnitude of the punishment. Indeed, the pek voxels from the sutrction nlysis lso showed significnt correltion etween the pek BOLD signl on ech tril nd the mgnitude of the rewrd otined (significnt t in ll 6 sujects tht showed significnt effects in the correltion nlysis). Thus, the single-event correltion nlysis showed tht voxels in the medil OFC hd BOLD chnges relted to the mgnitude of the rewrd otined (Fig. 4). 96 nture neuroscience volume 4 no 1 jnury 2001

3 rticles ewacq Punev p < 2x10 7 c ewacq PunAcq Punev ewacq PunAcq ewacq p < 7x10-5 p < 7x10-5 p < 1x ewev ewacq p < 7x Fig. 2. The results from the group rndom effects nlysis cross the nine sujects with threshold t (uncorrected) superimposed on the structurl MI scn of single suject. The equivlent y-vlues in Tlirch spce for ech coronl section shown is provided on the ottom left of ech frme. () Ares of ilterl medil OFC significntly ctivted y rewrd, s shown in the ewacq Punev comprison. Tlirch coordintes of ctivted clusters in the OFC, [x, y, z], [6, 42, 24], [ 10, 40, 22], [ 4, 28, 18] nd the ewacq PunAcq comprison [ 4, 48, 28]. () An re of right lterl OFC ctivted y punishment is shown in the Punev ewacq comprison [36, 58, 12]. (c) ight inferior prefrontl sulcus (BA 44/45) significntly ctivted in the PunAcq ewacq comprison [48, 33, 23] nd ewev ewacq comprison [45, 23, 23]. The ctivtion extends dorslly to include prts of res 9/46. We next descrie comprisons tht show the rin res tht re more ctivted following punishing outcome thn rewrding outcome. We compred the punish reversl events to the rewrd cquisition events (Punev ewacq), nd sw ctivtion in lterl re of nterior OFC (BA 10/11) s well s in region of nery ventrl prefrontl cortex. The re ctivted ws in the right hemisphere in the group nlysis (Fig. 2), nd on the right in eight of nine sujects in the individul suject nlyses (t threshold of, extent threshold of p < 0.05). In five of those eight sujects, ctivtion ws lso nture neuroscience volume 4 no 1 jnury

4 rticles found on the left, nd in the ninth suject, the ctivtion ws on the left only. The time course of significntly ctivted voxels in this region is shown in Fig. 3. In ddition, correltion nlysis, in which voxels were selected tht were positively correlted with the mgnitude of the punishments received, reveled n djcent re of the lterl OFC (nd ventrl prefrontl cortex) where the BOLD signl ws correlted with the mgnitude of the punishment otined (Fig. 4, group nlysis; the sme correltion existed in eight of nine sujects in the individul suject nlyses t, with p < 0.05 extent threshold). These results indicte tht n increse in the neurl ctivity of the lterl OFC is prticulrly relted to the suject s receipt of punishment fter selecting stimulus, nd tht the mgnitude of ctivtion in this region is relted to the mgnitude of the punishments otined. Confirmtion of this finding comes from the dt indicting tht, in the pek voxels cross sujects from the Punev ewacq sutrction (Fig. 2), there ws grded increse in these voxels ccording to the mgnitudes of the punishment, nd grded decrese ccording to the mgnitude of the rewrd (Fig. 5). Indeed, the pek voxels from the sutrction nlysis lso showed significnt correltion etween the pek BOLD signl in the lterl OFC re on ech tril nd the mgnitude of the punishment otined (significnt t p < 0.01 in 5 sujects nd t p < 0.05 in 3 other sujects). Thus, the single-event correltion nlysis showed tht voxels in the lterl OFC hd BOLD chnges tht were relted to the mgnitude of the punishment otined (Fig. 4). We conducted control condition in which the sujects hd to choose one of two frctl imges presented in the sme formt s the reversl tsk, ut without receiving ny monetry rewrd or punishment s feedck following the stimulus selection (see Methods). The ims of this tsk were to provide n ffectively neutrl seline condition, with the sme response selection nd motor components s in the reversl tsk, ut without the monetry feedck. In this nlysis, we compred the ctivtions during ll of the trils in which rewrds were received (ewacq nd ewev trils) nd the ctivtions on the control trils, to revel res ctivted y rewrd reltive to neutrl seline (ewrd Control). Seven of the nine sujects prticipted in this condition. (Two sujects were unle to prticipte ecuse of their time constrints.) Consistent with the previous results, n re of medil OFC ws ctivted in this contrst (Fig. 6). Similrly, when ll of the trils in which the sujects received monetry loss (PunAcq nd Punev trils) were sutrcted from the neutrl seline (Punish Control), n re of lterl oritofrontl cortex ws ctivted ilterlly (Fig. 6). The results of the comprisons with the control tsk support the findings descried ove. Other rin res Activtion of some other rin res ws lso found to e relted to performnce of the tsk. One such re included prts of the medil prefrontl cortex (BA 10/32), which were significntly ctivted in contrsts reveling rin ctivtion to rewrd (Fig. 2). A second region, the cortex in the right posterior inferior prefrontl sulcus (BA 44/45), showed most ctivtion fter trils in which the suject either received smll punishment on the currently correct stimulus (PunAcq), or smll rewrd for touching the currently incorrect stimulus (ewev; Fig. 2c). A third ctivted region ws the dorsl nterior cingulte cortex (BA 24/32), where effects were found in contrsts tht reveled ctivtion to punishment s well s in the ewev ewacq contrst. Fig. 3. Typicl verged time course reltive to the time of the ehviorl response mde to select one of the stimuli from single suject of significntly ctivted voxels from () the ewacq Punev contrst in the medil OFC nd () the Punev ewacq contrst in the lterl OFC. DISCUSSION The results from this study indicte tht the lterl nd medil oritofrontl corticl res re together responsive to the rewrding nd punishing outcomes produced y selecting stimulus. The lterl re of the OFC is ctivted following punishing outcome, nd the medil OFC is ctivted following rewrding outcome. The results dvnce erlier understnding of the oritofrontl cortex otined, for exmple, y neurophysiology in the mcque, or y PET neuroimging in humns. The results show the following. Astrct rewrds nd punishments cn ctivte the oritofrontl cortex. Different res re ctivted y these rewrds nd punishers. The ctivtions re produced y the rewrds nd punishments given (in tht they follow the delivery of ech rewrd nd punishment, finding mde possile y the fmi design). The mgnitude of the ctivtions reflects the mgnitude of the rewrd or punishment delivered ( finding mde possile y the event-relted spect of the fmi design, which enled correltions to e mesured). The medil region tht showed incresed ctivtion to rewrd lso showed decresed BOLD signl when punishment ws delivered, nd vice vers for the lterl oritofrontl cortex region. The dissocition found in this study etween the lterl nd medil oritofrontl cortex my e relted to ntomicl nd neuropsychologicl differences etween these regions in mcques. The lterl oritofrontl cortex receives most of the sensory fferents 15 17, wheres the medil oritofrontl cortex hs the strongest connections with the rostrl cingulte cortex. The intrinsic connectivity of the mcque lterl oritofrontl cortex is such tht res within this region project strongly to ech other, ut very wekly to res in the medil oritofrontl cortex, wheres res in the medil oritofrontl cortex connect strongly with ech other nd wekly to lterl oritl res 18. On these grounds, those uthors drw firm distinction etween the medil nd lterl 98 nture neuroscience volume 4 no 1 jnury 2001

5 rticles p < 2 x p < 2 x Fig. 4. esults from the correltion nlysis. () Voxels in the OFC nd 1.5 other regions whose ctivity increses reltive to the incresing mgnitude 1 of rewrd or punishment otined. Voxels in n re of left medil OFC [0,80] [ 6, 34, 28] correlted positively with rewrd (ove), nd voxels in n 0.5 [80,159] re of right lterl OFC [28, 60, 6] correlted positively with punishment (elow). The verge pek cross-correltion coefficient for the re 0 [160,250] [-600;-425] [-250,-1] correlted positively with rewrd ws 0.18, wheres the verge pek correltion coefficient for the re correlted positively with punishment ws 0.5 [-425,-250] () The medin percent chnge in BOLD signl from seline cross 1 sujects (with the vlue for ech suject with significnt effect shown t 1.5 in the single event correltion nlysis) for six different ctegory rnges of rewrd nd punishment. The signl ws verged cross ctegory rnge within ech suject nd then the verge signl chnge from ech ctegory ws verged cross sujects. This is plotted for the pek voxels in the medil OFC (ove), which significntly correlted with rewrd, nd in the lterl OFC (elow), which significntly correlted with punishment. The rnges of monetry rewrd nd punishment in ech ctegory re shown on the chrt, nd were determined y their reltive frequencies, which follow from the experimentl design. For the time series cross-correltion dt shown here, the verge of the Person correltion coefficient from ech suject etween the inned mgnitudes of rewrd nd the corresponding BOLD signls ws 0.66, nd the verge of the Person correltion coefficient from ech suject etween the inned mgnitudes of punishment nd the corresponding BOLD signls ws Percent chnge in BOLD signl Percent chnge in BOLD signl [-600,-425] ewrd correltion [-425,-250] [-250,-1] Punishment correltion [0,80] [80,159] [160,250] oritofrontl cortex nd suggest tht the two res re seprte networks. In neuropsychologicl investigtion in mcques, differences were reported etween the effects of lesions in different prts of the oritofrontl cortex during emotion-relted visul discrimintion reversl lerning 7. Following reversl, mcques with lesions in the lterl oritofrontl cortex nd inferior convexity continue responding to the previously rewrded ut now unrewrded stimulus. In contrst, monkeys with medil OFC lesions mke more errors during cquisition efore criterion, nd do not show the sme persevertive responding following reversl. This dissocition in lesion effects could e explined y the results from the current study; if the lterl oritofrontl cortex is more involved in representing the punishing consequences of continuing to select the previously rewrded stimulus following reversl, then this could ccount for difficulty in reversing responses to the old S+ following reversl. If, lterntively, the medil oritofrontl cortex is involved in representing the rewrding consequences of choosing the currently rewrded S+ during cquisition, then dmge to this region could result in filure to consistently choose the currently rewrded stimulus evident, for exmple, during cquisition. The finding of correltion etween the size of the rewrd or punishment otined on n individul tril nd the mgnitude of the BOLD response in the medil nd lterl oritofrontl corticl res, respectively, is evidence tht the mgnitude of the rewrds nd punishments received re represented in those regions. This finding my ccount in prt for the deficits shown y ptients with OFC lesions during gmling tsks, in which the ptients hve to decide to choose dvntgeously from numer of choices, to mximize monetry gin 5. An inility to represent the mgnitude of the rewrds nd punishments received my then result in difficulty in judging the degree to which prticulr stimulus choice is dvntgeous on the sis of cumultive monetry gin. The finding of inferior prefrontl sulcus ctivtion during ewev nd PunAcq trils suggests tht this re my e more ctivted during trils tht re more miguous in their potentil for signling chnge in the reinforcement contingencies. The ctivted re ws very similr to the ctivted re seen in numer of imging studies feturing tsks ssocited with prefrontl function, such s in the cognitive set-shifting component of the Wisconsin crd sorting tsk 19, nd in the no-go trils of go/no-go tsk 20. Tken together, the findings suggest tht the cortex in the inferior prefrontl sulcus my e involved in inhiiting inpproprite ehviorl strtegies, such s to switch ehvior fter smll punishment or to sty fter smll rewrd. nture neuroscience volume 4 no 1 jnury

6 rticles Fig. 5. The men percent chnge in BOLD signl from seline cross sujects for six different ctegory rnges of rewrd nd punishment, shown for the pek voxels from the sutrction nlysis. () Percent chnge in BOLD signl for the ewacq Punev contrst. () Percent chnge in BOLD signl for the Punev ewacq contrst. The signl ws verged cross ctegory rnge within ech suject, nd the verge signl chnge from ech ctegory ws then verged cross sujects. The rnges of monetry rewrd nd punishment in ech ctegory re shown on the chrt, nd were determined y their reltive frequencies, which follow from the experimentl design. The mens nd stndrd errors of the mens re shown. For the sutrction nlysis shown here, the verge of the Person correltion coefficient from ech suject, etween the inned mgnitudes of rewrd nd the corresponding BOLD signls, ws 0.61, nd the verge of the Person correltion coefficient from ech suject, etween the inned mgnitudes of punishment nd the corresponding BOLD signls, ws Elliot nd collegues 21 reviewed results of functionl imging experiments conducted in their lortory in which they lso report dissocition etween the medil nd lterl oritofrontl cortex. They suggest, on the sis of findings in diverse tsks including sentence completion, story comprehension, guessing nd delyed mtch to smple, tht the medil oritofrontl cortex is involved in monitoring nd holding in mind rewrd vlues. Our direct evidence on this issue indictes tht the medil oritofrontl cortex ws ctivted y monetry rewrd, nd showed less BOLD signl reltive to seline following the punishment of monetry loss. On the sis of findings in tsks including delyed non-mtch to smple, sentence completion, guessing nd hypothesis testing, these uthors suggest tht the lterl oritofrontl cortex is likely to e ctivted when response previously ssocited with rewrd hs to e suppressed. Our evidence showed tht the lterl oritofrontl cortex ws ctivted y punishment, nd tht it indeed reflected the mgnitude of the punishment, with some dditionl evidence for decrese in ctivtion following rewrd. A region sensitive to the mgnitude of punishment could provide useful signl for response suppression. However, in mcques, neurophysiologicl evidence ewrd Control +50 Punish Control -5 p < 2 x10 Percent chnge in BOLD signl Percent chnge in BOLD signl [-600,-425] [-424,-250] [-600,-425] [-424,-250] [-249,-1] [-249,-1] [0,79] [0,79] [80,159] [160, 250] [80,159] [160,250] suggests tht sensory stimuli tht produce rewrds nd punishments, rther thn the motor responses eing mde, re represented in the oritofrontl cortex, nd we therefore suggest tht the response suppression itself is likely to e medited outside the oritofrontl cortex. The time to pek of the ctivtion in the oritofrontl cortex (pproximtely 10 seconds) ws lter thn the time to pek often found y fmi investigtions in other rin res such s the primry visul or somtosensory cortex, in which period etween four to six seconds is commonly reported 23. The reltively long ltency of the BOLD response in the oritofrontl cortex oserved here hs een noted previously. In prticulr, using somtosensory stimultion protocol, we oserved tht the ctivtion in the oritofrontl cortex peked much lter thn the ctivtion in the primry somtosensory cortex 24. A similr effect hs lso een oserved in the nterior inferior prefrontl cortex 25. Possile resons for the slow hemodynmic response in the OFC include 24 the reltively low pek firing rtes of neurons in the OFC (frequently in the rnge of spikes/s 26,27, which contrst with frequent pek responses of spikes/s in the temporl loe corticl visul res 28 ), nd the sprseness of the representtions found in the OFC 26,29. A full understnding of this effect my only e possile when the reltionship etween the underlying neurl firing rte nd the chrcteristics of the BOLD response re more completely understood p < 2 x10-5 Fig. 6. Comprison of rewrds nd punishments with control condition. () A region of ilterl medil OFC [0, 44, 26] nd medil prefrontl cortex significntly ctivted reltive to neutrl seline in the ewrd Control contrst () A region of ilterl lterl OFC significntly ctivted reltive to neutrl seline in the Punish Control contrst. ight OFC, [34, 52, 12]; left OFC, [ 28, 64, 8]. 100 nture neuroscience volume 4 no 1 jnury 2001

7 Imge nlysis. The imge nlysis ws crried out in MEDx (Sensor Systems, Sterling, Virgini). The dtsets were motion corrected using three-dimensionl utomted imge registrtion lgorithm, sptilly smoothed using Gussin filter (full width t hlf mximum, 5 mm), intensity normlized, nd temporlly filtered. (None of the nine sujects were excluded from the nlysis ecuse of excessive motion.) The motion cross ll nine sujects rnged from mm in the x-direction, mm in the y-direction nd mm in the z-direction, nd we verified tht the three-dimensionl utomted imge registrtion lgorithm corrected for this smll degree of motion, which ws, in ny cse, smll in reltion to the voxel size. Further, the pttern of results could not e due to stimulus-correlted motion rtifct, s the time course of the OFC ctivtions found shows pek t round 8 14 s, which is consistent with norml hemodynmic response. If the results were relted to ny stimulus-induced motion rtifct, it would e nticipted tht the pek of the response would occur t the tril onset, nd not 8 14 s fterwrd 35 ). The trils were designted s one of four event types (Fig. 1c). Events from the cquisition phse were selected from the second tril, fter sujects hd chnged their choice of stimulus to the new S+, provided tht the sujects consistently chose tht S+ until the next reversl (denoting stle cquisition of the S+). Events from the reversl phse were selected fter reversl of the reinforcement contingencies, when the suject touched the old S+ nd efore the sujects hd successfully switched their choice of stimulus to the new S+. Three volumes were selected from ech tril corresponding to 8 14 seconds fter the tril onset, s this represented the pek of the hemodynmic response in the OFC. The selected volumes from ech tril were then used in the t-tests etween ech of the events of interest for ech individul suject. A sttisticl threshold of with n extent threshold of p < 0.05 ws pplied to the individul z-mps resulting from the sttisticl contrsts using the MEDx cluster detection function. For the nlysis involving the control condition, the volumes from the control condition were registered to the first volume in the reversl tsk, nd intensity normlized to the sme glorticles A much discussed issue is the extent to which there is hemispheric speciliztion in the processing of rewrds nd punishments 31. In the group nlysis, the voxels tht correlted with rewrd were predominntly left-sided, wheres the voxels correlted with punishment were predominntly on the right. However, ilterl ctivtion in the rewrd nd punishment correltion nlyses ws commonly seen in the individul sujects (five sujects in rewrd nlysis nd four sujects in punishment nlysis). Furthermore, when the rewrd nd punishment trils were sutrcted from neutrl seline, ctivtion in the group nlysis ws found ilterlly in the OFC for oth rewrd nd punishment. Consequently, the most consistent seprtion etween the rin res responding to rewrd nd punishment in this study were etween lterl nd medil res of the OFC, not the left nd right OFC. Other res ctivted in the tsk include prts of the medil prefrontl cortex (re 10) nd the dorsl nterior cingulte. These results indicte tht severl prefrontl regions re ctivted during performnce of the tsk. However, the evidence from lesion studies suggests tht the OFC, not other regions of prefrontl cortex, is essentil for performnce on n emotion-relted reversl lerning tsk 9,32. Furthermore, single-cell neurophysiologicl recordings in non-humn primtes emphsize the importnce of OFC neurons in the responses to rewrds nd punishments 11,29,33. These results re consistent with the hypothesis tht the deficits in emotion-relted lerning tsks following OFC lesions re due to n impirment in using informtion from rewrding nd punishing outcomes to guide ehviorl choice. The finding tht the mgnitude of the ctivtions depended on the size of the rewrd or punishment received is consistent with neurophysiologicl evidence from primtes tht rewrding nd punishing stimuli re represented in the OFC, ut this finding does not exclude possile representtion in other rin res of rewrds nd punishments 3,29. Our evidence tht there is representtion for rewrds nd punishments in these regions is lso consistent with the hypothesis tht the chnged emotionl nd socil ehvior demonstrted y humns with this dmge is due to fundmentl impirment in the processing of rewrds nd punishments 3. METHODS Sujects nd tsk. Nine helthy, right-hnded sujects prticipted in this study (6 femle nd 3 mle; yers old; verge ge, 25.4). Before scnning, the sujects were trined on version of the tsk designed for use outside the scnner, s prt of neuropsychologicl study of ptients with OFC lesions (J.O., J.H., E.T.., D. Wde nd J. McGrth, unpulished dt). This tsk consisted of two prts: simple cquisition tsk, where the suject hd to find which of two stimuli ws the S+ nd touch tht stimulus consistently, nd reversl tsk similr to the one used in the scnner. Previously unseen frctl imges were used in the scnner, so tht lerning ws required during the imging. The sujects were trined on 50 trils of the reversl tsk outside the scnner (or until they hd ttined t lest two reversls). In the scnner, stimuli were generted on PC, nd ppered on ck-projection screen mounted outside the scnner ore, which sujects viewed using mirror glsses. Sujects used their right hnd to press one of two uttons corresponding to the top or ottom of the screen, to select one of the two frctls. The sujects did the tsk (descried fully in Fig. 1) for 30 min, during which they completed n verge of 136 trils nd 15 reversls. After selection of one of the stimuli, the mount won or lost on stimulus ws displyed for 3 s; then the screen ws clered, nd fixtion cross ws presented for n dditionl 10 s, efore the presenttion of the next tril. Seven sujects lso prticipted in control condition in the sme scnning session, in which two novel frctls were presented in n identicl fshion to the reversl tsk, nd the sujects were requested to consistently select one of the two frctls. (They were told in dvnce which stimulus to select.) After selection of one of the stimuli, the sujects were not provided with ny monetry rewrd or punishment ut insted were given neutrl feedck, which consisted of the words pttern selected. The screen ws then clered, nd fixtion cross ws shown efore the presenttion of the next tril. The sujects did the control tsk for 5 min, during which they completed n verge of 18 trils. Scnning procedures. Imges were cquired with 3.0-T VAIAN/SIEMENS (Plo Alto, Cliforni) whole-ody scnner t Oxford Centre for functionl mgnetic resonnce imging (FMIB, Oxford, UK). Fourteen T2* weighted EPI slices were cquired every 2 s (T = 2). The following prmeters were crefully selected in order to minimize susceptiility nd distortion rtifct in the oritofrontl cortex. First, the dt were cquired in coronl rther thn xil slicing direction, s this ligned the slices to e perpendiculr to the predominnt direction of the intrinsic susceptiility-induced field grdients, nd helped to minimize through-plne dephsing. Second, the voxel resolution ws minimized y using 3-mm in-plne resolution nd 5-mm slice thickness, which results in less phse cncelltion thn would e produced y lower voxel resolutions. Third, reltively low TE of 25 ms ws selected to decrese the signl dropout, s less phse dispersion is creted cross the voxels. Fourth, ech suject ws individully shimmed using oth liner nd second-order shimming to minimize sttic field inhomogeneities in the oritofrontl cortex. Finlly, geometric distortion ws minimized y using specilist hed insert grdient coil (Mgnex SGAD III, Aingdon, UK) with reltively high grdient switching frequency of 960 Hz. The mtrix size ws 64 64, nd the field of view ws mm. Continuous coverge ws otined from +60 (A/P) to 10 (A/P). Acquisition ws done during tsk performnce, which lsted totl of 30 min, yielding 900 volumes in totl. A whole-rin T2* weighted EPI volume of the ove dimensions nd n ntomicl T1 volume with slice thickness of 5 mm nd in-plne resolution of mm were lso cquired. The cquisition protocol used in this study is similr to tht used in previous studies from our lortory, in which we used fmi to imge the oritofrontl cortex 24,34. nture neuroscience volume 4 no 1 jnury

8 rticles l men s the volumes from the reversl tsk. Further, we pplied sptil nd temporl filtering using the sme prmeters s those descried for the reversl tsk ove. For the sttisticl comprisons, three volumes were selected from ech tril in the control condition corresponding to 8 14 s fter the tril onset, nd t-tests were done etween the selected volumes from the rewrd nd punishment events nd the control events. The rewrd events used in the ewrd Control contrst consisted of ll the events during the reversl tsk in which rewrd ws received, irrespective of the tril type (tht is, the trils were pooled cross oth ewacq nd ewev events). Similrly, the punishment events used in the Punish Control contrst consisted of ll trils during the reversl tsk in which punishment ws received (pooled cross oth Punev nd PunAcq events). The individul z-mps for ech suject were registered to tht suject s corresponding ntomicl volume. The men difference imges for ech suject from every comprison were registered to the ntomicl volume from single suject nd rndom effects sttistics were clculted for ech of the contrsts (using loclly developed FMIB tool). A threshold for sttisticl significnce of (uncorrected) ws pplied to the rndom effects z-mps. The sttisticl criteri incorported in the rndom effects methods nd used in this investigtion re intrinsiclly roust, nd within this context, the use of uncorrected sttistics is resonle, ut is in ny cse further justified on the sis tht the min re of interest is the oritofrontl cortex, region in which ctivtion ws predicted on the sis of our priori hypothesis. Tlirch coordintes for ech of the ctivted clusters were otined following registrtion of the single suject ntomicl to stndrd rin in the coordinte spce of Tlirch nd Tournoux 36. For the correltion nlysis, time-series wveform ws constructed, which represented the mgnitude of the rewrds nd punishments received during the experiment. The wveform ws constructed strting with simple ON-OFF squre lock wveform in which the mgnitude in 6-s ON period fter the tril onset (llowing 8 s for hemodynmic lg) ws set to the mount of monetry gin or loss otined on ech individul tril (positive for rewrd, negtive for punishment), nd the OFF period mgnitude ws set to zero. The resulting squre wveform ws convolved using Poisson distriution (with λ = 1.0) to give closer pproximtion to rel hemodynmic response. This wveform ws then correlted on voxel-y-voxel sis with the ctul hemodynmic responses cross the experiment for ech of the sujects using the MEDx correltion nlysis function. A group rndom effects sttistic ws clculted from comintion of the individul correltion mps, nd sttisticl threshold of (uncorrected) ws pplied. Note: supplementry informtion is ville on the Nture Neuroscience we site ( ACKNOWLEDGEMENTS This reserch is funded y the MC (to FMIB), the MC (to E.T..), the MC IC for Cognitive Neuroscience, the MC, Oxford McDonnell- Pew Centre for Cognitive Neuroscience nd Firmenich (to J.O.D.) nd the Dnish eserch Agency, Arhus, Denmrk (to M.L.K.). We thnk P. Mtthews, S. Smith, P. Jezzrd, S. Clre nd I. Trcey for their help. ECEIVED 5 JUNE; ACCEPTED 20 OCTOBE Dmsio, A.. Descrtes Error (Putnm, New York, 1994). 2. Anderson, S. W., Bechr, A., Dmsio, H., Trnel, D. & Dmsio, A.. Impirment of socil nd morl ehvior relted to erly dmge in humn prefrontl cortex. Nt. Neurosci. 2, (1999). 3. olls, E. T. The Brin nd Emotion (Oxford Univ. Press, Oxford, 1999). 4. olls, E. T., Hornk, J., Wde, D. & McGrth, J. Emotion-relted lerning in ptients with socil nd emotionl chnges ssocited with frontl loe dmge. J. Neurol. Neurosurg. Psychitry 57, (1994). 5. Bechr, A., Dmsio, A.., Dmsio, H. & Anderson, S. W. Insensitivity to future consequences following dmge to humn prefrontl cortex. Cognition 50, 7 15 (1994). 6. Bechr, A., Dmsio, H., Trnel, D. & Dmsio, A.. Deciding dvntgeously efore knowing the dvntgeous strtegy. Science 275, (1997). 7. Iversen, S. D. & Mishkin, M. Persevertive interference in monkeys following selective lesions of the inferior prefrontl convexity. Exp. Brin es. 11, (1970). 8. Jones, B. & Mishkin, M. Limic lesions nd the prolem of stimulus reinforcement ssocitions. Exp. Neurol. 36, (1972). 9. Meunier, M., Bchevlier, J. & Mishkin, M. Effects of oritl frontl nd nterior cingulte lesions on oject nd sptil memory in rhesus monkeys. Neuropsychologi 35, (1997). 10. Freedmn, M., Blck, S., Eert, P. & Binns, M. Oritofrontl function, oject lterntion nd persevertion. Cere. Cortex 8, (1998). 11. olls, E. T., Critchley, H. D., Mson,. & Wkemn, E. A. Oritofrontl cortex neurons: role in olfctory nd visul ssocition lerning. J. Neurophysiol. 75, (1996). 12. Thut, G. et l. Activtion of the humn rin y monetry rewrd. Neuroreport 8, (1997). 13. Elliott,., Frith, C. D. & Doln,. J. Differentil neurl response to positive nd negtive feedck in plnning nd guessing tsks. Neuropsychologi 35, (1997). 14. Buckner,. L. Event-relted fmi nd the hemodynmic response. Hum. Brin Mpp. 6, (1998). 15. Cvd, C., Compny, T., Tejedor, J., Cruz izzolo,. J. & einoso Surez, F. The ntomicl connections of the mcque monkey oritofrontl cortex. A review. Cere. Cortex 10, (2000). 16. Crmichel, S. T. & Price, J. L. Limic connections of the oritl nd medil prefrontl cortex in mcque monkeys. J. Comp. Neurol. 363, (1995). 17. Morecrft,. J., Geul, C. & Mesulm, M. -M. Cytorchitecture nd neurl fferents of oritofrontl cortex in the rin of the monkey. J. Comp. Neurol. 232, (1992). 18. Crmichel, S. T. & Price, J. L. Connectionl networks within the oritl nd medil prefrontl cortex of mcque monkeys. J. Comp. Neurol. 371, (1996). 19. Konishi, S. et l. Trnsient ctivtion of inferior prefrontl cortex during cognitive set shifting. Nt. Neurosci. 1, (1998). 20. Konishi, S., Nkjim, K., Uchid, I., Sekihr, K. & Miyshit, Y. No-go dominnt rin ctivity in humn inferior prefrontl cortex reveled y functionl mgnetic resonnce imging. Eur. J. Neurosci. 10, (1998). 21. Elliott,., Doln,. J. & Frith, C. D. Dissocile functions in the medil nd lterl oritofrontl cortex: evidence from humn neuroimging studies. Cere. Cortex 10, (2000). 22. olls, E. T. in The Frontl Loes (eds. Stuss, D. T. & Knight,. T.) (Oxford Univ. Press, Oxford, in press). 23. Miezin, F. M., Mccott, L., Ollinger, J. M., Petersen, S. E. & Buckner,. L. Chrcterizing the hemodynmic response: effects of presenttion rte, smpling procedure, nd the possiility of ordering rin ctivity sed on reltive timing. Neuroimge 11, (2000). 24. Frncis, S. et l. The representtion of the plesntness of touch in the humn rin, nd its reltion to tste nd olfctory res. Neuroreport 10, (1999). 25. Buckner,. L. et l. Functionl-ntomic study of episodic retrievl: II. Selective verging of event-relted fmi trils to test the retrievl success hypothesis. Neuroimge 7, (1998). 26. olls, E. T. & Bylis, L. L. Gusttory, olfctory, nd visul convergence within the primte oritofrontl cortex. J. Neurosci. 14, (1994). 27. olls, E. T., Yxley, S. & Sienkiewicz, Z. J. Gusttory responses of single neurons in the cudolterl oritofrontl cortex of the mcque monkey. J. Neurophysiol. 64, (1990). 28. olls, E. T. & Tovee, M. J. Sprseness of the neuronl representtion of stimuli in the primte temporl visul cortex. J. Neurophysiol. 73, (1995). 29. olls, E. T. The oritofrontl cortex nd rewrd. Cere. Cortex 10, (2000). 30. ees, G., Friston, K. & Koch, C. A direct quntittive reltionship etween the functionl properties of humn nd mcque V5. Nt. Neurosci. 3, (2000). 31. Dvidson,. J. Anterior cererl symmetry nd the nture of emotion. Brin Cogn. 6, (1992). 32. Dis,., oins, T. & oerts, A. Dissocition in prefrontl cortex of ffective nd ttentionl shifts. Nture 380, (1996). 33. Tremly, L. & Schultz, W. eltive rewrd preference in primte oritofrontl cortex. Nture 398, (1999). 34. O Doherty, J. et l. Sensory-specific stiety-relted olfctory ctivtion of the humn oritofrontl cortex. Neuroreport 11, (2000). 35. Birn,. M., Bndettini, P. A., Cox,. W. & Shker,. Event-relted fmi of tsks involving rief motion. Hum. Brin Mpp. 7, (1999). 36. Tlirch, J. & Tournoux, P. Co-Plnr Stereotxic Atls of the Humn Brin (Thieme, Stuttgrt, 1988). 102 nture neuroscience volume 4 no 1 jnury 2001