Reward Changes Salience in Human Vision via the Anterior Cingulate

Transcription

1 11096 The Journl of Neuroscience, August 18, (33): Behviorl/Systems/Cognitive Rewrd Chnges Slience in Humn Vision vi the Anterior Cingulte Clyton Hickey, 1 Leonrdo Chelzzi, 2,3 nd Jn Theeuwes 1 1 Deprtment of Cognitive Psychology, VU University Amsterdm, 1081 BT Amsterdm, The Netherlnds, 2 Deprtment of Neurologicl nd Visul Sciences, University of Veron, Veron, Itly, nd 3 Ntionl Institute of Neuroscience, Veron, Itly Rewrd-relted mesolimic dopmine steers niml ehvior, creting utomtic pproch towrd rewrd-ssocited ojects nd voidnce of ojects unlikely to e eneficil. Theories of dopmine suggest tht this reflects underlying ises in perception nd ttention, with rewrd enhncing the representtion of rewrd-ssocited stimuli such tht ttention is more likely to e deployed to the loction of these ojects. Using mesures of ehvior nd rin electricity in mle nd femle humns, we demonstrte this to e the cse. Sensory nd perceptul processing of rewrd-ssocited visul fetures is fcilitted such tht ttention is deployed to ojects chrcterized y these fetures in susequent experimentl trils. This is the cse even when prticipnts know tht strtegic decision to ttend to rewrd-ssocited fetures will e counterproductive nd result in suoptiml performnce. Other results show tht the mgnitude of visul is creted y rewrd is predicted y the response to rewrd feedck in nterior cingulte cortex, n re with strong connections to dopminergic structures in the midrin. These results demonstrte tht rewrd hs n impct on vision tht is independent of its role in the strtegic estlishment of endogenous ttention. We suggest tht rewrd cts to chnge visul slience nd thus plys n importnt nd undervlued role in ttentionl control. Introduction Rewrd plys fundmentl role in humn cognition, ut surprisingly few studies hve exmined the impct of rewrd on erly cognitive processes, such s visul perception nd ttention (ut see Dell Lier nd Chelzzi, 2009). This is reflected in theory: models of visul ttention propose tht selection is guided y utomtic exogenous fctors, which is ttention towrd slient stimuli, nd volitionl endogenous fctors, which direct ttention towrd tsk-relevnt ojects nd loctions (Treismn nd Gelde, 1980; Wolfe et l., 1989). Rewrd plys no explicit role in this frmework. In shrp contrst, theories of the function of dopmine in reinforcement lerning nd niml pproch ehvior plce rewrd firmly t the center of ttentionl control (Berridge nd Roinson, 1998; Ikemoto nd Pnksepp, 1999; Redgrve et l., 1999; Wise, 2004; Alcro et l., 2007). For exmple, the incentive slience hypothesis of Berridge nd Roinson (1998) proposes tht rewrd-relted mesencephlic dopmine is specificlly responsile for chnging the perceptul representtion of rewrd-conditioned stimuli such tht they ecome slient nd ttention-drwing. Other theories propose more generl role for dopmine in reinforcement lerning ut lso suggest tht rewrd hs direct impct on vision (Schultz et l., 1997; Schultz, 2002) (for review nd integrtion of dopmine models, see McClure et l., 2003). Received Fe. 26, 2010; revised June 9, 2010; ccepted July 2, L.C. is supported y Fondzione Criveron. Correspondence should e ddressed to Clyton Hickey, vn der Boechorststrt 1, 1055 AB Amsterdm, The Netherlnds. E-mil: c.hickey@psy.vu.nl. DOI: /JNEUROSCI Copyright 2010 the uthors /10/ $15.00/0 The direct nture of this influence deserves emphsis; the ide is tht rewrd utomticlly chnges the visul slience of rewrd-ssocited perceptul fetures, nd this is theoreticlly distinct from the known role of rewrd in the strtegic estlishment of ttentionl set (Munsell, 2004). Unfortuntely, much of the extnt literture is sed on n experimentl prdigm tht fils to differentite these mnners of influence. In this type of experiment, humn or niml oservers re presented with stimuli tht predict rewrd outcome for the current experimentl tril. Results show tht visul processing of ojects tht predict good outcome is etter thn processing of other ojects (Pltt nd Glimcher, 1999; Iked nd Hikosk, 2003; Roesch nd Olson, 2003; Kiss et l., 2008; Peck et l., 2009). However, stimuli tht predict rewrd hve n inherent spect of motivtionl significnce; liquid-deprived monkey trets cue indicting forthcoming liquid s type of rewrd in itself, even when this informtion hs no ering on tsk performnce (Bromerg- Mrtin nd Hikosk, 2009). Anticiption of rewrd cues might therefore trigger endogenous ises in visul ttention. These ises would fcilitte detection nd discrimintion of cue stimuli y enhncing visul processing, possily from very erly in visul cortex (Hopf et l., 2004). This mkes it uncler whether the fcilitted visul processing of rewrd-predictive stimuli reflects the direct impct of rewrd or the strtegic impct of endogenous ttention. The current study ws designed to determine whether rewrd hs direct impct on vision tht is distinct from its impct vi endogenous ttention. Our pproch to this prolem ws to ssocite rewrd to visul fetures tht chrcterized ojects prticipnts were ctively trying to ignore. To gin insight into the mechnisms involved in trnslting rewrd outcome into visul

2 Hickey et l. Rewrd Chnges Visul Slience J. Neurosci., August 18, (33): Figure 1. Generl prdigm nd results from experiment 1., The trget nd slient distrctor re denoted. In experiment 1, these ojects could e presented t ny of six loctions, s illustrted. In experiment 2, there were 10 possile stimulus loctions. Correct trget discrimintion ws rewrded with 1 or 10 points (worth 0.2 or 2, respectively)., Rection times from experiment 1. Error rs here nd in ll susequent figures reflect within-prticipnt 95% confidence intervls (Cousineu, 2005). Accurcy results were lso nlyzed: prticipnts responded with men SD ccurcy of 95 4%, nd no relile ptterns were oserved (ll P vlues 0.4). slience, we recorded oth ehvior nd electricl rin ctivity while prticipnts completed our experimentl tsk. Mterils nd Methods In our generl prdigm, which ws used in ll experiments, prticipnts serched for uniquely-shped trget presented mong numer of homogenous distrctors (Fig. 1) (Theeuwes, 1991). Response ws sed on the orienttion of smll line contined within the trget. In some trils, ll the stimuli were of the sme color, ut more often one of the distrctors hd color tht ws different from ll other ojects. Color singletons such s this re known to drw ttention to their loction during serch for unique shpe (Hickey et l., 2006). Two criticl prmeters of the tsk vried from tril to tril. First, the colors of the trget nd distrctor chnged such tht the distrctor could e red, with ll other stimuli including the trget green, or vice vers. The colors could therefore swp etween trils, with the color of the trget ecoming tht of the distrctor, or could remin the sme. Second, prticipnts received either high-mgnitude or low-mgnitude monetry rewrd fter correct responses. Importntly, prticipnts were instructed to mximize rewrd, ut rewrd mgnitude ws ctully rndomized on tril-y-tril sis. Prticipnts. Forty helthy dults were rndomly ssigned to tke prt in experiment 1 (men SD ge, 20 2 yers; three left hnded; 16 men), eight to the control experiment (men SD ge, 21 2 yers; ll right hnded; three women), nd 14 to experiment 2 (men SD ge, 21 3 yers; ll right hnded; six women). All gve informed consent efore tking prt in the experiment nd reported norml or corrected-to-norml vision, nd none hd previous experience with the experimentl tsk. Stimuli nd procedure. Experimentl stimuli were very similr to those used y Hickey et l. (2006) with the ddition of monetry feedck fter prticipnt response. Briefly, in experiment 1, visul serch rrys contined six oject outlines (line thickness of 0.3 visul ngle), ech presented equidistnt (9.1 ) from centrl fixtion point nd from ech other. Ojects could e dimonds ( ) or circles (3.4 dimeter), with ech disply contining only one uniquely shped item. This unique item could e dimond, with ll other stimuli circles, or circle, with ll other stimuli dimonds. In 80% of trils, one of the homogenously shped nontrget items ws of unique color, either red with ll other ojects green or vice vers. Trget nd distrctor colors were rndomly determined for ech tril. The onset of the visul serch rry ws preceded y fixtion dot for rndom durtion of ms. Ech oject contined gry line ( ) rndomly oriented verticlly or horizontlly. Responses were mde on stndrd computer keyord; the z utton pressed with the index finger of the left hnd denoted verticl trget line, nd the m utton pressed with the index finger of the right hnd denoted horizontl trget line. Correct responses to the serch trget were immeditely followed y the replcement of the centrl fixtion dot with n indiction of rewrd feedck in lue text (65 point font; 5 height), either 10, denoting the receipt of 10 points, or 1, denoting the receipt of 1 point. The visul serch disply remined onscreen during the presenttion of feedck, nd the serch disply nd feedck were presented together for 1000 ms. Incorrect responses resulted in the removl of 10 points, denoted y 10. Ech point hd vlue of pproximtely 0.2 euro cent. Prticipnts were pid sed on the numer of points received, ut, ecuse rewrd mgnitude fter correct response ws rndom nd most prticipnts performed very well, there ws little vriility in compenstion; cross ll experiments, prticipnts never erned less thn 8.50/h nd never more thn 9.25/h. All prmeters in the control experiment nd in experiment 2 were s in experiment 1 with the following exceptions. In the control experiment, serch rrys contined four ojects (9.1 from fixtion; 12.9 from ech other), nd, in experiment 2, rrys contined 10 ojects (9.1 from fixtion; 5.6 from ech other). In experiment 2, the uniquely shped item ws lwys circle (with ll other stimuli dimonds), responses were mde with stndrd computer mouse (the left mouse utton pressed with the index finger of the right hnd denoted verticl trget line, nd the right mouse utton pressed with the middle finger of the right hnd denoted horizontl trget line), nd ech rewrd point ws worth pproximtely 0.4 euro cent (to compenste prticipnts for the time required to set up the electroencephlogrphic recording). Experiment 1 nd the control experiment took plce in soundttenuted room, nd experiment 2 took plce in sound-ttenuted Frdy cge. All stimuli were presented on cthode ry tue monitor locted 60 cm wy from the eyes. Prticipnts in experiment 1 nd the control experiment completed 15 locks of 30 trils for totl of 450 trils, which took 0.5 h, nd prticipnts in experiment 2 completed 45 locks of 30 trils for totl of 1350 trils, which took 1.5 h. Feedck regrding ccurcy nd response speed followed the completion of ech lock. All prticipnts were given detiled instructions regrding the experimentl tsk. Importntly, in the predictive condition of experiment 1 nd in the control experiment, prticipnts were explicitly informed of the informtion encoded in rewrd mgnitude. These prticipnts were sked to descrie how rewrd mgnitude predicted fetures of the next tril nd only egn the experiment when they expressed cler understnding of this spect of the experimentl design. Trils in which response occurred sooner thn 100 ms fter stimulus or lter thn 1500 ms fter stimulus were rejected from ll nlysis, s were ll trils resulting in error. Approximtely 99% of correct responses occurred within this time window. All rection times reflect men verges of rw, untrnsformed dt. Electroencephlogrm recording nd nlysis. In experiment 2, electroencephlogrm ws recorded from 134 Ag/AgCl electrodes using the Biosemi Active2 system. This included 128 encephlic sites, two electrodes plced 1 cm lterl to the externl cnthi of ech eye, two electrodes locted 1 cm ove nd elow the right eye socket, nd two electrodes plced on the mstoids (500 Hz smple rte; encephlic electrode loctions re illustrted in the figures). Independent component nlysis (Bell nd Sejnowski, 1995; Delorme nd Mkeig, 2004) ws pplied to the dt nd the results used for the rejection of trils tinted y eye movements (8 4% of trils per prticipnt, men SD) nd correction of rtifcts stemming from links nd muscle ctivity. Eventrelted potentils (ERPs) were clculted from the resulting dt using stndrd signl verging procedures (Luck, 2005), referenced to the verge of mstoids, nd digitlly filtered ( Hz; finite impulse

3 11098 J. Neurosci., August 18, (33): Hickey et l. Rewrd Chnges Visul Slience lest-squre kernels with 6 d trnsition of 0.01 Hz for low-pss filtering nd 6 d trnsition of 2 Hz for high-pss filtering). Sttistics were clculted efore filtering. All ERPs were selined on period eginning 100 ms efore stimulus onset nd ending 50 ms fter. The seline ws clculted efore filtering. Visul ctivity in sensory rely nuclei does not crete potentil t sclp surfce, nd the first ERP evidence of visul ctivity in unimodl visul experiments occurs etween 50 nd 60 ms fter stimulus in the C1 component, mking the first 50 ms of the visul ERP vlid spect of the seline (Rugg nd Coles, 1995). During the editoril process, n nonymous reviewer sked us to explicitly ddress the possiility tht the 50 ms poststimulus intervl contined evoked ctivity. A pired t test compring ctivity in the 50 ms preceding stimulus onset to the 50 ms fter onset reveled no trend towrd relile difference (t (14) 0.755, p 0.464; collpsed cross experimentl conditions nd sed on unfiltered dt recorded t the sme lterl posterior electrodes used to generte the visul ERPs presented in the figures). Note tht, in ERPs elicited t nterior electrode sites, the ppliction of filters resulted in n pprent rmping up of the N1 tht egins t 45 ms fter stimulus; this erly spect of the N1 is filter rtifct nd does not reflect ctul evoked ctivity. Becuse seline ws clculted efore filtering, this hd no impct on the clcultion of ERP mplitude. Source nlysis. For the purposes of oth topogrphic mpping nd source nlysis, dt were referenced to the verge of ll 128 encephlic signls. Topogrphic mps re sed on sphericl spline interpoltion (Perrin et l., 1989). Source nlysis ws conducted using BESA version 5.1 (Megis Softwre). The BESA lgorithm opertes y recursively seeding electricl sources in three-shell ellipticl model of humn hed, clculting n estimte of the electricl potentil t sclp surfce generted y the model nd compring this with the recorded ctivity. The dipoles re djusted in terms of position nd orienttion until the model optimlly fits the recorded dt, nd good fit is reflected in low mesure of residul vrince. Figure 2. Conditionl results from experiment 1., Results when rewrd mgnitude ws independent of the likelihood of color swp., Results when rewrd mgnitude predicted the likelihood of color swp. Results Rewrd guides serch utomticlly Our interest ly in how rewrd in one tril ffected visul processing in the next, nd we mde two predictions sed on the ide tht rewrd directly impcts slience. First, high-mgnitude rewrd should fcilitte processing of the fetures tht chrcterize trget such tht visul ttention is ised towrd these fetures in the next tril (Berridge nd Roinson, 1998; Ikemoto nd Pnksepp, 1999; Dell Lier nd Chelzzi, 2009). Prticipnts should therefore respond quickly when the sme color chrcterizes the trget s did so in the preceding tril ut should respond slowly when the colors swp. Second, low-mgnitude rewrd should result in reltive devlution of fetures tht chrcterize trget such tht ttention is less likely to e deployed to these fetures (Berridge nd Roinson, 1998; Frnk et l., 2004; Dell Lier nd Chelzzi, 2009). Prticipnts should therefore respond slowly when the sme color chrcterizes the trget s did so in the preceding tril, ut quickly when the colors swp. As illustrted in Figure 1, these predictions were orne out in men rection time (rewrd color swp, F (1,38) 11.86, p 0.001; ll other effects involving rewrd nd color swp fctors, F vlues 1). The pttern of results reveled in experiment 1 suggests tht humns preferentilly ttend to ojects with visul fetures ssocited with rewrd (Fig. 1). It is importnt to note tht the impct of rewrd is tied to color ut tht the trget in this prdigm ws defined y unique shpe. Color ws therefore tsk irrelevnt, mking it unlikely tht the influence of rewrd reflects endogenous ttentionl set; there ws no strtegic motivtion to py color heed. However, the sic prdigm does not llow us to rule out the possiility tht prticipnts dopted strtegy of ttending to stimuli chrcterized y rewrd-ssocited perceptul fetures, possily motivted y the enefits this strtegy provides in life outside the lortory. To determine whether such strtegic ccount for the results ws vile, we included etween-prticipnt fctor in the design of experiment 1. For hlf of prticipnts, rewrd mgnitude ws unrelted to the likelihood tht the trget nd distrctor colors would swp etween trils. For the others, high-mgnitude rewrd ws 80% predictive of swp in colors, nd low-mgnitude rewrd ws 80% predictive of no swp. These ltter prticipnts were informed of the reltionship etween rewrd nd color swp nd told to use this informtion to optimize performnce. Our resoning ws tht prticipnts in the predictive condition should discrd the useless strtegy of mintining ttentionl set fter high-mgnitude rewrd nd dopt the eneficil strtegy of prepring for the colors to swp (nd, correspondingly, discrd the useless strtegy of prepring for color swp fter low-mgnitude rewrd nd dopt the eneficil strtegy of prepring for the trget nd distrctor colors to remin the sme). Importntly, were they to do so, the experimentl results in this condition would e opposite those illustrted in Figure 1: they would e fster when color swp followed high-mgnitude rewrd (rther thn slower) nd slower when no swp followed low-mgnitude rewrd (rther thn fster). In fct, results from the two conditions were sttisticlly indistinguishle (Fig. 2) (min effect of predictiveness, F (1,38) 1.20, p 0.28; ll other effects involving predictiveness, nd criticlly the three-wy interction, F vlues 1). The interction etween rewrd nd color swp ws relile when oth the nonpredictive condition (Fig. 2)(F (1,19) 7.24, p 0.01; ll other F vlues 1) nd the predictive condition (Fig. 2)(F (1,19) 4.69, p 0.04; ll other F vlues 1) were exmined in isoltion. These results re inconsistent with the ide tht the impct of rewrd on serch reflects endogenous strtegy; prticipnts in the predictive condition were explicitly wre tht high-mgnitude rewrd ment tht the colors would likely swp in the next tril. Under these circumstnces, they hd no strtegic motivtion to continue to ttend to ojects chrcterized y the sme color. This strtegy would in fct e counterproductive, guiding ttention to n oject tht ws unlikely to e the trget. Despite this, the results show continuing propensity to select the stimulus chrcterized y the color recently ssocited with rewrd. There is smll possiility tht results from the predictive condition of experiment 1 reflect n inility or unwillingness to

4 Hickey et l. Rewrd Chnges Visul Slience J. Neurosci., August 18, (33): high-mgnitude rewrd predicted color swp. Why does their ehvior show no evidence of this knowledge? We elieve tht, in the predictive condition of experiment 1, the receipt of high-mgnitude rewrd cued prticipnts to strtegiclly prepre for color swp, ut receipt of high-mgnitude rewrd lso initited the utomtic priming of perceptul fetures tht chrcterized the trget. This perceptul effect ws strong enough to overwhelm nd negte the impct of endogenous strtegy. Figure 3. Prdigm nd results from the control experiment., As in experiment 1, prticipnts serched for the uniquely shped oject. Rewrd in ny one tril predicted the loction of the trget on the next tril., Rection times. Prticipnts were fster to respond to trgets presented t cued loctions. Figure 4. Behviorl results from experiment 2, replicting findings from experiment 1. extrct informtion encoded in rewrd. Prticipnts in this condition were mde explicitly wre tht rewrd predicted the likelihood of color swp, nd ll prticipnts could clerly descrie this spect of the experiment efore eginning the tsk. However, if for ny reson they did not tke the informtion encoded in rewrd into ccount during experimentl prticiption, this might ccount for the sence of ny difference etween the predictive nd nonpredictive conditions. To ensure tht this ws not the cse, we conducted control experiment. Here, rewrd predicted trget loction rther thn the likelihood of color swp: high-mgnitude rewrd indicted tht the trget in the next tril ws more likely to e on the horizontl meridin, nd lowmgnitude rewrd indicted tht it would more likely e on the verticl meridin (Fig. 3). Sptil cues of this nture re known to hve strong impct on visul serch (Posner nd Cohen, 1984), nd prticipnts in this experiment were indeed fster to respond to trgets t cued loctions (Fig. 3) (trget loction, F (1,13) 1.54, p 0.255; rewrd, F 1; trget loction rewrd, F (1,13) 13.88, p 0.007). This demonstrtes tht prticipnts re le to extrct strtegic informtion encoded in rewrd nd tht they mke n effort to use this informtion to improve performnce. Prticipnts in experiment 1 were presumly wre of the fct tht Electrophysiologicl indices of the impct of rewrd on perception nd ttention Becuse ehvior reflects the outcome of processing in multiple cognitive stges nd chnge in response cn reflect modultion t ny point in this sequence, mesures such s response ltency nd ccurcy provide reltively corse insight into the cognitive stges ffected y experimentl mnipultions. In contrst, noninvsive mesures of rin electricl ctivity, ERPs (Luck, 2005), cn sometimes e used to demonstrte chnge in discrete processing stges. With this in mind, in second experiment, we hd 14 new prticipnts complete the nonpredictive version of our visul serch tsk while we recorded ERPs from the sclp surfce. As illustrted in Figure 4, ehviorl results from this experiment replicted those of experiment 1 (rewrd color swp, F (1,13) 6.30, p 0.026). Prticipnts in experiment 2 dditionlly responded more rpidly when colors did not swp etween trils, lthough not relily so (color swp, F (1,13) 2.75, p 0.121; ll other F vlues 1). To index chnges in perceptul nd ttentionl processing, we nlyzed lterlized ERPs elicited over visul cortex y the onset of serch displys. We concentrted on ERPs elicited when the trget nd distrctor were presented to opposite visul hemifields ecuse lterlized effects oserved under these circumstnces cn e ssocited with reltive increse in the perceptul nd ttentionl processing of one or the other of the two slient ojects. This is the cse ecuse of the contrlterl nd retinotopic nture of the visul system; effects ssocited with trget processing will e primrily evident t electrode loctions contrlterl to the trget, wheres effects ssocited with distrctor processing will e evident t electrode loctions contrlterl to the distrctor (Luck nd Hillyrd, 1994,; Hillyrd et l., 1998; Woodmn nd Luck, 1999; Luck, 2005; Hickey et l., 2006). It is importnt to note tht this design does not mke is possile to link signl to one discrete stimulus; ERPs hve inherently d sptil resolution, nd, ecuse of this, minority of the signl oserved contrlterl to the trget my reflect distrctor processing. However, ecuse our experimentl design llows us to compre the lterlized potentils elicited y identicl stimulus rrys under vrying conditions of rewrd, it provides perspective on the reltive degree to which perceptul nd ttentionl resources re eing pplied to either the trget or distrctor. As illustrted in Figure 5, nd, multiple effects were pprent in the ERPs elicited fter high-mgnitude rewrd, eginning with n increse in the mplitude of the lterl P1 component t 100 ms fter stimulus. This increse in P1 reflects n mplifiction of erly visul processing stges in extrstrite visul cortex (for review, see Hillyrd et l., 1998). Criticlly, in no-swp trils fter high-mgnitude rewrd, the P1 increse ws opposite the

5 11100 J. Neurosci., August 18, (33): Hickey et l. Rewrd Chnges Visul Slience Rewrd processing in nterior cingulte cortex predicts the mgnitude of the impct of rewrd on the deployment of ttention A crucil gol of experiment 2 ws to seek direct signture of rewrd-relted processing in the rin such tht this ctivity could e loclized nd relted to ehvior. We pproched nlysis with the hypothesis tht the mesolimic dopmine system would ply role in creting the impct of rewrd on vision, nd we exmined the ERP elicited y rewrd feedck to determine whether this ws the cse. Midrin dopminergic structures re too deep in the rin to e detected in electricl recordings t sclp surfce, ut reinforcement-monitoring spects of the dop c d Figure 5. ERPs elicited y serch rrys over lterl occipitl cortex. These ERPs were recorded t the posterior electrodes identified in white in Figure 6. The contrlterl wveforms re n vergeofsignlsrecordedtthethreelefthemisphereelectrodesiteswhenthetrgetwsintherightfieldndvicevers, ndtheipsilterlwveformsrenvergeofthethreelefthemisphere signls when the trget ws in the left field together with the three right hemisphere signls when the trget ws in the right field., Wveforms recorded when the colors defining trget nd distrctor did not swp fter high-mgnitude rewrd nd the trget nd distrctor were presented to opposite hemifields. The loction of ttention identified from the N2pc is denoted in the visul serch rry y roken circle., Wveforms elicited fter high-mgnitude rewrd when the colors swpped. The N2pc is elicited y the distrctor in this condition. Note the presence of lte contrlterl component, possily reflecting recovery nd redeployment of ttention to the trget. c, Wveforms recorded fter low-mgnitude rewrd when the colors did not swp. d, Wveforms recorded fter low-mgnitude rewrd when the colors swpped. trget (Fig. 5), demonstrting n increse in sensory nd perceptul processing of the trget, ut in swp trils fter highmgnitude rewrd, the P1 increse ws opposite the distrctor (Fig. 5), demonstrting n increse in sensory nd perceptul processing of the distrctor (electrode lterlity swp condition, F (1,13) 4.98, p 0.044; ll other F vlues 1; nlysis sed on pek mplitude difference). Notice tht, in oth cses, it is the stimulus chrcterized y the color reinforced with high rewrd on the immeditely preceding tril tht elicited the enhnced P1 effect. No corresponding pttern ws oserved in the ERPs elicited fter low-mgnitude rewrd (Fig. 5c,d), consistent with the sence of ehviorl effect in this condition (Fig. 4). There re two importnt points to e mde in the context of this P1 effect. First, the displys tht elicited ll four of the ERPs presented in Figure 5 were identicl, nd therefore the chnge in lterlized P1 mplitude oserved in the high-mgnitude rewrd condition cnnot e product of chnges in visul input. Second, the lterl P1 is not normlly sensitive to endogenous ttentionl set for visul fetures in serch; the lterl P1 elicited y unique item tht is the trget is not ny lrger thn the lterl P1 elicited y unique item tht is distrctor (Luck nd Hillyrd, 1994). The increse in lterl P1 mplitude in response to n oject chrcterized y rewrd-ssocited color thus suggests tht rewrd cn hve n impct on perceptul processing tht cnnot e ccounted for s product of endogenous ttentionl set. The P1 results support the ide tht high-mgnitude rewrd fcilittes susequent processing of stimuli chrcterized y rewrd-ssocited visul fetures from very erly in the visul processing sequence. A similr pttern ws evident in the N2pc component. The N2pc is n increse in negtive ERP mplitude from 200 to 300 ms fter stimulus t posterior electrode sites contrlterl to n ttended oject (Luck nd Hillyrd, 1994,), nd it constitutes relile index of visuosptil ttention (Luck nd Hillyrd, 1994,; Woodmn nd Luck, 1999; Hickey et l., 2009). In the high-rewrd no-swp condition, the N2pc ws elicited contrlterl to the trget (Fig. 5), reflecting rpid trget selection, ut in the high-rewrd swp condition, the N2pc ws elicited contrlterl to the distrctor, demonstrting the deployment of ttention to the distrctor loction (Fig. 5) (electrode lterlity swp condition, F (1,13) 7.29, p 0.018; ll other F vlues 1; simple effect electrode lterlity in no-swp condition, t (13) 2.42, p 0.015; simple effect electrode lterlity in swp condition, t (13) 2.14, p 0.026; ll N2pc sttistics sed on men mplitude ms fter stimulus). No corresponding effects were oserved in the ERPs elicited fter low-mgnitude rewrd, lthough the trget-elicited N2pc in this condition ws numericlly lrger in the swp condition, possily reflecting etter trget selection when the trget hs not een ssocited with suoptiml outcome (Fig. 5c,d) (electrode lterlity swp condition, F (1,13) 2.04, p 0.177). Together, the P1 nd N2pc results reinforce the conclusions we took from ehvior, nmely tht rewrd primes erly visul processing such tht ojects chrcterized y rewrdssocited fetures re more likely to e ttended.

6 Hickey et l. Rewrd Chnges Visul Slience J. Neurosci., August 18, (33): Figure6. Electrophysiologiclresponsetorewrdfeedck., ERPselicitedyhigh- ndlow-mgnituderewrdfeedckover mediofrontlcortex. TheseERPsreflectnvergeofsignlsrecordedtthefournteriormidlineelectrodesitesidentifiedinwhite in., Topogrphy of the difference in MFN elicited y low-mgnitude versus high-mgnitude rewrd. c Figure 7. Results from inverse dipole model of the difference in MFN., Visul inspection of the topogrphic mp presented in Figure 6 suggested the presence of three ctivity sources, two locted ilterlly over occipitotemporl cortex nd third locted more frontlly on the midline, nd source nlysis ccordingly egn with n unconstrined fit of three-dipole model. Reltively high residul vrince in this model motivted the inclusion of fourth dipole, nd n unconstrined fit of four dipoles generted theillustrtedmodel.inclusionofdditionldipolesdidnotcretenypprecileimprovementtothismodel( 0.25%increse in ccounted vrince). Dipole coordintes in Tlrich nd Tournoux(1988) spce were clculted y coregistering the BESA hed model to the verge of 24 individul structurl mgnetic resonnce imges: occipitl 38, 71, 11 nd 30, 76, 13; cingulte 3, 11, 46; eye/oritofrontl 22, 72, 4., Cingulte source wveform, representing isolted ctivity of the cingulte source over time. c, Per-prticipnt reltionship etween isolted cingulte ctivity ( ms) nd the impct of rewrd on ehvior. Note tht correltion coefficients here nd in d reflect Spermn s, which is less sensitive to outliers thn other mesures of correltion. Sttisticl vlues reflect outcome of permuttion tests. d, Per-prticipnt reltionship etween recorded difference in MFN mplitude nd the impct of rewrd on ehvior. d mine system extend to the nterior cingulte cortex (Holroyd nd Coles, 2002), consistent with the known connectivity etween these res (Willims nd Goldmn-Rkic, 1993). Rewrd processing in nterior cingulte cortex cn e indexed in midline nterior ERP component known s the medil frontl negtivity (MFN). The MFN is pprent from 200 to 300 ms fter rewrd feedck, nd it ppers to reflect ctivity involved in the ssessment of motivtionl impct (Gehring nd Willoughy, 2002). The MFN is generlly lrger for low-mgnitude rewrd, s ws the cse in the present study (Fig. 6)(t (13) 4.20, p 0.001; ll MFN nlyses re sed on men mplitude ms fter stimulus). Figure 6 presents the sclp topogrphy of the difference in ERP ctivity elicited y high-mgnitude versus low-mgnitude rewrd oserved cross the pek intervl of the MFN ( ms). This topogrphy suggests tht differences in MFN cooccurred with chnge in posterior corticl processing. Exmintion of ERPs elicited over visul cortex suggested tht the posterior effect my reflect n occipitl selection negtivity (Hrter nd Aine, 1984). This is consistent with the ide tht the ttentive response to rewrd feedck vried s function of the mgnitude of rewrd denoted. We creted reverse dipole model (Scherg, 1992) of the difference in ctivity oserved cross the pek of the MFN ( ms) to isolte nterior rewrd-relted processing from concurrent ctivity in visul cortex. The estfitting unconstrined model suggested four discrete sources (residul vrince, 3.69%) (Fig. 7). Two sources were locted in occipitl cortex. Another source ws locted in extrcorticl spce in the vicinity of the right eye nd likely reflects comintion of residul eye movement ctivity in the ERP nd rewrd processing in oritofrontl cortex (Rolls, 2000). A finl source ws locted on the order of the nterior cingulte nd medil frontl gyrus. By pplying this model to the complete ERP intervl, we clculted n ctivity wveform for this cingulte source (Fig. 7) tht ws mrkedly similr in terms of onset nd pek ltency to the rw difference in MFN (Fig. 6). To determine whether nterior cingulte cortex ws involved in creting the influence of rewrd on perceptul nd ttentionl processing, we exmined the reltionship etween the modeled cingulte ctivity nd the impct of high-mgnitude rewrd on ehvior. We pplied the dipole model to individul prticipnt dt, creting per-prticipnt versions of the cingulte source wveform, nd mesured ctivity of the cingulte source for ech prticipnt cross the pek intervl identified in the grnd-verge MFN wveform ( ms) (identified y roken ox in Figs. 6,7). We lso quntified the ehviorl impct of high-mgnitude rewrd for ech prticipnt y clculting the men ltency difference etween swp nd no-swp responses. These mesures correlted strongly (Fig. 7c) (Spermn s 0.618, p 0.021). A similr reltionship ws identified when the MFN difference oserved in the rw ERPs ws used rther thn the dipole model output (Fig. 7d) (Spermn s 0.662, p 0.020). These correltions were primrily driven y vriility in the cingulte response to high-mgnitude rewrd. They show tht s the difference etween cingulte ctivity elicited y low- nd highmgnitude rewrd decreses driven y the incresing mplitude of the signl elicited y high-mgnitude rewrd the impct of high-mgnitude rewrd on ehvior ecomes stronger. We interpret this s evidence of individul differences in sensitivity to the motivtionl impct of positive feedck. Prticipnts who re sensitive to positive feedck s reflected in lrger cingulte response

7 11102 J. Neurosci., August 18, (33): Hickey et l. Rewrd Chnges Visul Slience to high-mgnitude rewrd show correspondingly lrger effect of rewrd on vision. Discussion The results demonstrte tht rewrd hs direct, non-volitionl impct on humn perception nd ttention tht is independent of its impct on strtegy nd endogenous ttentionl control. Behviorl mesures show tht prticipnts re fst to respond to trget chrcterized y color recently ssocited with highmgnitude rewrd ut slow to respond to trget when distrctor chrcterized y this color competes for ttentionl resources. In contrst, prticipnts re slow to respond to trget ssocited with low-mgnitude rewrd ut fst to respond when it is the distrctor color tht hs een ssocited with suoptiml outcome. This ehviorl pttern is evident even when prticipnts were wre tht strtegy to select ojects chrcterized y rewrd-ssocited fetures would e counterproductive nd much etter strtegy ws mde ville to them. Electrophysiologicl mesures confirm tht this reflects chnges in perceptul nd ttentionl processing: the P1 ERP component is enhnced contrlterl to n oject chrcterized y rewrd-ssocited color, reflecting fcilitted perceptul ctivity, nd this stimulus elicits n N2pc, indexing the deployment of ttention to its loction. Criticlly, these effects re oserved regrdless of whether the stimulus is the serch trget: when the slient distrctor is chrcterized y the rewrd-ssocited color visul resources re llocted to this tsk-irrelevnt oject. Processing of rewrd feedck elicits n ERP component known s the MFN, nd we find tht the mgnitude of the MFN elicited y positive feedck predicts the ehviorl impct of rewrd on visul serch on per-prticipnt sis. The MFN is thought to reflect neurl processing involved in the evlution of the motivtionl impct of n event (Gehring nd Willoughy, 2002) nd hs een linked to other mediofrontl ERP components elicited y the commission of errors or feedck indicting erroneous performnce (Holroyd et l., 2002). In generl, these mediofrontl components re thought to reflect corticl processing in system tht involves midrin dopmine neurons (Holroyd nd Coles, 2002). Our experimenttion ws motivted y theoreticl interprettions of the role of dopmine in niml pproch ehvior tht suggest tht rewrd-relted mesolimic dopmine cts to fcilitte perceptul nd ttentionl processing of stimuli with rewrd-conditioned fetures (Schultz et l., 1997; Berridge nd Roinson, 1998; Ikemoto nd Pnksepp, 1999; Redgrve et l., 1999; Wise, 2004; Alcro et l., 2007). Motivted y the ide tht the MFN might constitute n indirect index of ctivity in this system, we elieve tht the correltion etween nterior cingulte ctivity nd the impct of rewrd on performnce reflects n underlying reltionship etween mesencephlic rewrd processing nd ctivity in visul cortex. According to this, rewrd-relted ctivity in the dopmine system initites series of events one stge of which involves the nterior cingulte tht eventully leds to chnges in sensory representtion. There is sustntil circumstntil evidence for this ide in the connectivity nd ehvior of dopminergic neurons. Dopminergic nuclei such s the sustnti nigr nd ventrl tegmentl re project diffusely to the sl gngli nd cortex; in primtes, the gretest density of corticl terminls re in medil frontl cortex, including the superior frontl gyrus nd nterior cingulte (Willims nd Goldmn-Rkic, 1993). Anterior cingulte nd surrounding cortex is known to e fundmentlly involved in the control of ttention nd processing of ttended stimuli (Mesulm, 1999; Hopfinger et l., 2000). Midrin dopmine neurons themselves show pttern of ctivity consistent with the cretion of ssocitions etween stimuli nd outcome: cells ecome ctive when unexpected rewrd is encountered ut do not respond to expected rewrd nd ecome less ctive thn norml when n expected rewrd fils to mterilize (Schultz et l., 1997; Schultz, 2002). Finlly, recent results hve demonstrted tht rewrd expecttion is represented in the ctivity of individul cells in primry visul cortex in the rt (Schuler nd Ber, 2006), suggesting the existence of neurl rchitecture necessry for the trnsltion of rewrd processing to sensory modultion in lowlevel visul cortex. Results from the current study demonstrte tht humn vision opertes ccording to principles tht re strikingly similr to those tht underlie pproch ehvior in non-primte nimls (for review, see Berridge nd Roinson, 1998; Ikemoto nd Pnksepp, 1999). Even ee tht hs found flower tht is rich with nectr will spend the dy serching for flowers of the sme color (Menzel nd Muller, 1996). This close correspondence cross species suggests tht the rin structures involved should e phylogeneticlly old nd thus present in the rins of very different nimls. The reltionship of the impct of rewrd on ttention to the nterior cingulte cortex is consistent with this; cingulte cortex is thought to hve developed either efore neocortex or shortly therefter (Allmn et l., 2001). The dopminergic structures in the midrin, which we suggest underlie the oserved ctivity in cingulte cortex, re even older, long predting neocortex (Mrín et l., 1998). We emphsize the phylogenetic ge of the rin res responsile for the impct of rewrd on ttention ecuse we elieve tht the pttern identified in the present study reflects the ction of very old cognitive mechnism. Erly in the development of the rin, visul is towrd stimuli chrcterized y rewrd-conditioned fetures likely constituted the sole source of ttentionl control. This is my continue to ply primry role in ttentionl control in nimls with rins less complex thn those of primtes. This hs cler dptive enefits; environmentl stimuli tht hve grnered primry rewrds such s food re very likely to do so gin in the future, nd thus ttending to them mkes sense. Humns hve cquired the ility to select stimuli in the sence of immedite externl reinforcement, ut the development of this ility does not necessrily preclude the continuing influence of the older system. The current study complements nd significntly extends growing literture exmining the impct of rewrd on the immedite deployment of ttention nd ttentionl lerning in humns. In previous study, lingering effects of ttentionl suppression of distrcting stimulus, known s negtive priming, were only found fter high rewrds, indicting tht persisting inhiition of visul representtions is olished y poor outcomes (Dell Lier nd Chelzzi, 2006). In susequent study designed to explore long-term effects of rewrds, humn prticipnts ecme more efficient t selecting trgets consistently ssocited with high-mgnitude rewrd ut reltively inefficient t ignoring the sme stimuli when shown s distrctors (Dell Lier nd Chelzzi, 2009). Interestingly, the ility to ignore given distrctor lso improved when this ws consistently followed y high (s opposed to low) rewrds, wheres the ility to select the sme items s trgets ecme reltively impired. Finlly, the consistent ssocition of stimuli with rewrd cn improve their detectility, rendering them reltively immune to the ttentionl link (Rymond nd O Brien, 2009). As noted in Introduction, studies investigting the effects of rewrd expectncy on ttentionl deployment re rdiclly different from the present work ecuse this mnipultion does not llow for the dissocition of strtegic nd utomtic effects re-

8 Hickey et l. Rewrd Chnges Visul Slience J. Neurosci., August 18, (33): lted to rewrd (Munsell, 2004). Nonetheless, previous work found n enhnced nd erlier N2pc the ERP component used s n index of selective ttention in the present study when elicited y trgets ssocited to the expecttion of high rewrds (Kiss et l., 2009). Animl electrophysiology is just eginning to explore modultions of visul processing tht result from controlled stimulus rewrd ssocitions (Schuler nd Ber, 2006; Peck et l., 2009; Frnkó et l., 2010), ut the exct link to the present findings is still uncler. Despite the recent surge of interest, nd s noted in Introduction, the impct of rewrd on ttention is not prominent fctor in models of visul serch nd ttention (with the notle exception of Nvlpkkm et l., 2009). It is cler tht, in future refinements of ttentionl theory, the role of rewrd will need to e given full considertion. In summry, the present results provide evidence tht rewrd hs direct impct on humn vision tht is independent of its role in strtegy nd endogenous ttentionl set. Our results suggest tht the nterior cingulte cortex corticl expression of the mesolimic dopmine system plys crucil role in this source of ttentionl control. References Alcro A, Huer R, Pnksepp J (2007) Behviorl functions of the mesolimic dopminergic system: n ffective neuroethologicl perspective. Brin Res Rev 56: Allmn JM, Hkeem A, Erwin JM, Nimchinsky E, Hof P (2001) The nterior cingulte cortex. Ann NY Acd Sci 935: Bell AJ, Sejnowski TJ (1995) An informtion-mximiztion pproch to lind seprtion nd lind deconvolution. Neurl Comput 7: Berridge KC, Roinson TE (1998) Wht is the role of dopmine in rewrd: hedonic impct, rewrd lerning, or incentive slience? Brin Res Rev 28: Bromerg-Mrtin ES, Hikosk O (2009) Midrin dopmine neurons signl preference for dvnce informtion out upcoming rewrds. Neuron 63: Cousineu D (2005) Confidence intervls in within-suject designs: simpler solution to Loftus nd Msson s method. Tut Qunt Meth Psychol 1: Dell Lier C, Chelzzi L (2006) Visul selective ttention nd the effects of monetry rewrds. Psychol Sci 17: Dell Lier C, Chelzzi L (2009) Lerning to ttend nd to ignore is mtter of gins nd losses. Psychol Sci 20: Delorme A, Mkeig S (2004) EEGLAB: An open source toolox for nlysis of single-tril EEG dynmics including independent component nlysis. J Neurosci Methods 134:9 21. Frnk MJ, Seeerger LC, O Reilly RC (2004) By crrot or y stick: Cognitive reinforcement lerning in prkinsonism. Science 306: Frnkó E, Seitz AR, Vogels R (2010) Dissocile neurl effects of long-term stimulus-rewrd piring in mcque visul cortex. J Cogn Neurosci 22: Gehring WJ, Willoughy AR (2002) The medil frontl cortex nd the rpid processing of monetry gins nd losses. Science 295: Hrter MR, Aine CJ (1984) Brin mechnisms of visul selective ttention. In: Vrieties of ttention (Prsurmn R, Dvies R, eds), pp London: Acdemic. Hickey C, McDonld JJ, Theeuwes J (2006) Electrophysiologicl evidence of the cpture of visul ttention. J Cogn Neurosci 18: Hickey C, Di Lollo V, McDonld JJ (2009) Electrophysiologicl indices of trget nd distrctor processing in visul serch. J Cogn Neurosci 21: Hillyrd SA, Vogel EK, Luck SJ (1998) Sensory gin control (mplifiction) s mechnism of selective ttention: electrophysiologicl nd neuroimging evidence. Philos Trns R Soc Lond B Biol Sci 353: Holroyd CB, Coles MG (2002) The neurl sis of humn error processing: Reinforcement lerning, dopmine, nd the error-relted negtivity. Psychol Rev 109: Holroyd CB, Coles MG, Nieuwenhuis S (2002) Medil prefrontl cortex nd error potentils. Science 296: Hopf JM, Boelmns K, Schoenfeld MA, Luck SJ, Heinze HJ (2004) Attention to fetures precedes ttention loctions in visul serch: evidence from electromgnetic rin responses in humns. J Neurosci 24: Hopfinger JB, Buonocore MH, Mngun GR (2000) The neurl mechnisms of top-down ttentionl control. Nt Neurosci 3: Iked T, Hikosk O (2003) Rewrd-dependent gin nd is of visul responses in primte superior colliculus. Neuron 39: Ikemoto S, Pnksepp J (1999) The role of nucleus ccumens dopmine in motivted ehvior: unifying interprettion with specil reference to rewrd-seeking. Brin Res Rev 31:6 41. Kiss M, Driver J, Eimer M (2009) Rewrd priority of visul trget singletons modultes ERP signtures of ttentionl selection. Psychol Sci 20: Luck SJ (2005) An introduction to the event-relted potentil technique. Cmridge, MA: Msschusetts Institutes of Technology. Luck SJ, Hillyrd SA (1994) Sptil filtering during visul serch: evidence from humn electrophysiology. J Exp Psychol Hum Percept Perform 20: Luck SJ, Hillyrd SA (1994) Electrophysiologicl correltes of feture nlysis during visul serch. Psychophysiology 31: Mrín O, Smeets WJ, González A (1998) Evolution of the sl gngli in tetrpods: new perspective sed on recent studies in mphiins. Trends Neurosci 21: Munsell JH (2004) Neuronl representtions of cognitive stte: rewrd or ttention? Trends Cogn Sci 8: Menzel R, Muller U (1996) Lerning nd memory in honeyees: From ehviour to neurl sustrtes. Annu Rev Neurosci 19: Mesulm MM (1999) Sptil ttention nd neglect: Prietl, frontl nd cingultecontriutionstothementlrepresenttionndttentionltrgetingofslient extrpersonl events. Philos Trns R Soc Lond B Biol Sci 354: Nvlpkkm V, Koch C, Peron P (2009) Homo economicus in visul serch. J Vis 9:31. Peck CJ, Jngrw DC, Suzuki M, Efem R, Gottlie J (2009) Rewrd modultes ttention independently of ction vlue in posterior prietl cortex. J Neurosci 29: Perrin F, Pernier J, Bertrnd O, Echllier JF (1989) Sphericl splines for sclp potentil nd current density mpping. Electroencephlogr Clin Neurophysiol 72: Posner MI, Cohen Y (1984) Components of visul orienting. In: Attention nd performnce X (Boum H, Bouwhuis DG, eds), pp Hillsdle, NJ: Erlum. Rymond JE, O Brien JL (2009) Selective visul ttention nd motivtion: the consequences of vlue lerning in n ttentionl link tsk. Psychol Sci 20: Redgrve P, Prescott TJ, Gurney K (1999) Is the short-ltency dopmine response too short to signl rewrd error? Trends Neurosci 22: Roinson TE, Berridge KC (1993) The neurl sis of drug crving: n incentive-sensitiztion theory of ddiction. Brin Res Rev 18: Rolls ET (2000) The oritofrontl cortex nd rewrd. Cere Cortex 10: Rugg MD, Coles MGH (1995) Electrophysiology of mind: event-relted rin potentils nd cognition. London: Oxford UP. Scherg M (1992) Functionl imging nd locliztion of electromgnetic rin ctivity. Brin Topogr 5: Schultz W (2002) Getting forml with dopmine nd rewrd. Neuron 36: Schultz W, Dyn P, Montgue PR (1997) A neurl sustrte of prediction nd rewrd. Science 275: Shuler MG, Ber MF (2006) Rewrd timing in the primry visul cortex. Science 311: Tlrich J, Tournoux P (1988) Co-plnr stereotxic tls of the humn rin. New York: Thieme Medicl Pulishers. Theeuwes J (1991) Cross-dimensionl perceptul selectivity. Percept Psychophys 50: Treismn AM, Gelde G (1980) A feture integrtion theory of ttention. Cogn Psychol 12: Willims SM, Goldmn-Rkic PS (1993) Chrcteriztion of the dopminergic innervtion of the primte frontl cortex using dopminespecific ntiody. Cere Cortex 3: Wise RA (2004) Dopmine, lerning nd motivtion. Nt Rev Neurosci 5: Wolfe JM, Cve KR, Frnzel SL (1989) Guided serch: n lterntive to the feture integrtion model for visul serch. J Exp Psychol Hum Percept Perform 15: Woodmn GF, Luck SJ (1999) Electrophysiologicl mesurement of rpid shifts of ttention during visul serch. Nture 400: