Positive and Negative Modulation of Word Learning by Reward Anticipation

Transcription

1 Human Bain Mapping 29: (2008) Positive and Negative Modulation of Wod Leaning by Rewad Anticipation Daniel E. Callan, 1,2 * and Nicolas Schweighofe 3 1 Computational Neuoscience Laboatoies, ATR, Kyoto, Japan 2 National Institute of Infomation and Communication Technology, Kyoto, Japan 3 Depatment of Biokinesiology and Physical Theapy, Univesity of Southen Califonia, Los Angeles Abstact: Recent evidence fom neuoscience indicates that the anticipation of extenal ewads may enhance declaative memoy consolidation by inceasing dopaminegic-modulated plasticity in the hippocampus. A numbe of studies in psychology, howeve, have shown that extenal ewads may have null, o even negative, effects on leaning. To shed light on this issue, we developed a novel task, in which native Japanese speakes wee ewaded to lean unknown English wods inside a functional MRI scanne. Rewads had no effect on ecall pefomance unless we used a ating of ewad-induced anxiety as a covaiate. In this case, fo highly ewaded wods, we found a negative coelation between ecall pefomance and anxiety atings. Fo those wods, high ecall pefomance and low anxiety atings wee associated with enhanced activity in the midbain dopaminegic centes, the hippocampus, and the amygdala. On the othe hand, low ecall pefomance and high anxiety atings wee associated with enhanced activity in the anteio cingulate and middle fontal gyus, bain egions that have been shown to be involved with anxiety and divided attention, espectively. A connectivity analysis indicated positive functional connectivity between the midbain dopaminegic centes and both the hippocampus and the amygdala, as well as negative connectivity between the anteio cingulate and the amygdala. Thus, both ou behavioal and imaging esults suggest that the anticipation of ewads can, depending on the individual level of ewad-induced anxiety, have eithe a beneficial effect o a negative effect on wod leaning. Hum Bain Mapp 29: , VC 2007 Wiley-Liss, Inc. Key wods: ewad; leaning; dopamine; hippocampus; declaative; semantic; fmri; anxiety; amygdale; wod INTRODUCTION It is uncetain whethe the anticipation of extenal ewads has a negative o positive effect on leaning, as two Contact gant sponso: NSF; Contact gant numbe: IIS ; Contact gant sponso: NIH; Contact gant numbe: P20 RR *Coespondence to: Daniel Callan, ATR, Hikaidai, Seika-cho, Soaku-gun, Kyoto , Japan. dcallan@at.jp Received fo publication 12 Octobe 2006; Accepted 26 Januay 2007 DOI: /hbm Published online 27 Mach 2007 in Wiley InteScience (www. intescience.wiley.com). seemingly opposing views have been poposed. On one hand, a numbe of studies in neuoscience have shown that ewad and memoy systems ae stongly inteconnected in the bain, indicating a positive effect of ewad anticipation in modulating encoding. Specifically, dopamine neuons in the substantia niga pas compacta and vental tegmental aea (SN/VTA), which ae known to be citically involved in ewad pediction [Kisch et al., 2003; Schultz, 1998], send substantial pojections to the CA1 pat of the hippocampus [Thiey et al.,2000]. Dopamine contols a majo souce of sensoy inputs to the CA1, and enhances long-tem potentiation and pevents depotentiation [Benado and Pince, 1982; Fey et al., 1989; 1990; 1991; Gibkoff and Ashe, 1984; Li et al., 2003; Otmakhova and Lisman, 1999]. In ats, selective lesion of the dopaminegic neuons that innevate the hip- VC 2007 Wiley-Liss, Inc.

2 Callan and Schweighofe pocampus seiously impais spatial memoy [Gasbai et al., 1997]. In humans, dopaminegic midbain stuctues and the hippocampus show geate activation fo ewaded vesus neutal items that wee late ecognized [Adcock et al., 2006; Wittmann et al., 2005]. Thus, thee is good evidence at the neual level that ewad anticipation inceases dopamine elease in the hippocampus, which in tun enhances plasticity and encoding. On the othe hand, eseach in psychology on the benefit of extinsic ewads on leaning has been inconclusive. Unewaded ats lean to get out of a maze as well as do ewaded ats [Tolman and Honzik, 1930]. Pefomance dependent ewads can enhance subjects attention duing encoding o eheasal fo ewad-associated items when compaed with neutal items [Loftus, 1972], but when these item-specific attention biases ae minimized, the effects of ewads ae small [Nilsson, 1987]. Futhemoe, although this is a debated subject [Cameon and Piece, 1994; Sansone and Haackiewicz, 2000], a lage numbe of human studies have found a geneal null, o even detimental, effect of extenal ewads on pefomance and leaning see [Deci et al., 1999; Sansone and Haackiewicz, 2000] fo eview. A possible explanation to this effect has been poposed as the competing esponse model [Reiss and Sushinsky, 1975], accoding to which the anticipation of pefomance-dependent ewads leads to divided attention, which in tun educes encoding. Futhemoe, the possibility of a lack of ewad acquisition contingent to failue can be peceived as a theat that can geneate anxiety [Haackiewicz and Sansone, 2000], which futhe divide attention [Mogg and Badley, 1998], and educes pefomance. Studies have epoted that deceased ecall pefomance is coelated with inceased anxiety [Andeoletti et al., 2006; the findings wee fo middle-aged and olde adults] and convesely that inceased woking memoy pefomance is coelated with educed anxiety [Hudetz et al., 2004]. Thus, fom a behavioal standpoint, we hypothesized that ewad anticipation has a beneficial modulatoy effect on encoding, and that this beneficial effect can be canceled by ewad-induced anxiety. Fom a neual standpoint, we hypothesized that ewad anticipation enhances midbain (SN/VTA) dopaminegic neuons activity via modulation fom amygdala activity, esulting in enhanced hippocampal encoding. We futhe hypothesized that, in individuals who expeience high levels of ewad-induced anxiety, an antagonistic mechanism is simultaneously at wok that educes hippocampal encoding. In suppot of ou hypothesis, memoy fomation has been linked to inceased dopamine elease in the amygdala [Fied et al., 2001], inceased SN/VTA activity [Heckes et al., 2002; Schott et al., 2004], and inceased hippocampal activity [Meltze and Constable, 2005; Schott et al., 2004; Wittmann et al., 2005]. Amygdala activity has been found to be elated to encoding [Kensinge and Schacte, 2006; Phelps, 2004] and to pedict woking memoy pefomance [Schaefe et al., 2006]. Futhemoe, the amygdala is in pat involved with mediating the function of positive ewads in eliciting goal-diected behavio [Elliott et al., 2004; Gottfied et al., 2003]. Convesely, the amygdala also shows a decease in activity to negative emotions [Enst et al., 2005; Etkin et al., 2006]. Pojections fom the Amygdala to the SN/VTA [Fudge and Habe, 2000] may modulate the dopaminegic pojections between SN/VTA and the hippocampus. We put fowad hee that in cases of low anxiety and ewad anticipation, the amygdala facilitates dopaminegic elease in SN/VTA that modulates (enhances) encoding in the hippocampus. Convesely, in the case of high anxiety thee is a decease of activity in the amygdala inhibiting dopaminegic elease in SN/VTA that diminishes the modulatoy effect it has on encoding in the hippocampus. The modulatoy effect of the amygdala may be mediated by activity fom the anteio cingulate cotex (ACC). The ACC has been associated with anticipatoy anxiety [Chua et al., 1999], egulating cognitive and emotional pocessing [Bush et al., 2000], as well as elating actions to thei consequences [Rushwoth et al., 2004] and has an inhibitoy influence on amygdala activity [Etkin et al., 2006]. To test these hypotheses, we developed a novel task, in which native Japanese speakes wee diffeentially ewaded (300, 100, and 0 yen) to lean English wods inside an functional MRI (fmri) scanne (see Mateial and Methods fo details). The expeiment consisted of a petest taken outside the fmri scanne, followed by an encoding phase inside the scanne, then a distacte task inside the scanne, and a delayed ecall posttest outside the scanne. Then, to quantify the degee of anxiety associated with each ewad condition (ewad-induced anxiety), qualitative atings of ewad-induced anxiety wee obtained sepaately fo each of the conditions. Subjects wee then awaded a monetay sum coesponding to thei pefomance on the post-test. MATERIALS AND METHODS Subjects Fifteen 23- to 34-yea-old (mean, 26 yeas, Std. 3.9 yeas) ight-handed female native Japanese speakes with some English ability (at least 6 yeas of classes in junio and senio high school) fom a pedominantly all female school paticipated in this study. Only female subjects wee available fo this study theefoe the esults may elate to females only. Futhe eseach needs to be conducted to detemine whethe the esults epoted hee genealize to males and/ o whethe thee ae gende diffeences. Only subjects who had a qualifying scoe on the petest wee included in the study. In addition to a vaiable ewad amount based on leaning pefomance (see late), subjects eceived a fixed amount fo thei paticipation, and gave witten infomed consent fo expeimental pocedues, appoved by the ATR Human Subject Review Committee. The subjects wee instucted at the beginning of the expeiment that they could ean a maximum of 9,840 yen and a minimum of 5,000 yen fo thei paticipation. 238

3 Wod Leaning by Rewad Anticipation Pocedue and Stimuli The expeiment consisted of a vocabulay petest taken outside the fmri scanne, an encoding phase inside the scanne, a distacte task inside the scanne, and a posttest outside the scanne. The petest involved sequential pesentation (using Matlab 6.0) of 82 English wods (Appendix). Fo each wod, the subject needed to type the coesponding Japanese wod and a familiaity ating anging fom 0 to 5 (0 meaning no familiaity and 5 meaning highly familia). Subjects wee told to espond quickly and wee infomed that they could ean 10 yen fo each coect answe on the petest. A native Japanese speake poficient in English immediately coected the petest. Only subjects who had at least 11 coect esponses on the petest with familiaity atings of 5 and at least 33 incoect esponses with familiaity atings of 1 o 0 wee included in the study. Responses on which the subject took longe than 30 s to answe wee discaded. Based on the esults of the petest, fou goups of 10 wods wee ceated fo the fou conditions: 300 yen ewad unknown wods, 100 yen ewad unknown wods, 0 yen ewad unknown wod, and unewaded known wods. An attempt was made to balance vaious aspects of the stimuli in expeimental conditions. A native Japanese speake fluent in English chose and classified the 82 wods, in 30 vey easy wods, and 52 difficult wods. All wods used in the expeiment wee concete nouns. The wods selected fo each of the expeimental conditions fo each subject wee based on thei own petest esults. The English wods anged fom 4 to 7 lettes (mean ¼ 5.95; Std. ¼ 0.88) and wee 1 to 3 syllables long (mean ¼ 2.01; Std. ¼ 0.61). The coesponding Japanese tanslation wods anged fom 2 to 3 Kanji chaactes (othewise known as Chinese chaactes), o hiagana (one of the two Japanese phonetics system) when appopiate, (mean ¼ 2.25; Std. ¼ 0.44) and 2 to 5 moa sounds (mean ¼ 3.81; Std. ¼ 0.80). The numbe of English lettes and syllables as well as the numbe of Kanji and hiagana in the coesponding Japanese tanslation wee balanced fo easy and difficult wods (as well as acoss the diffeent ewad conditions) duing composition of the wod lists fo each condition and subject. Post hoc analysis eveals that the complexity of the Kanji chaactes (defined by the numbe of stokes needed to wite the chaacte) did not significantly diffe between easy and difficult wods (as well as acoss the diffeent ewad conditions) fo all subjects except one (P < 0.05). Thee was no significant coelation between behavioal pefomance and complexity of the kanji chaactes composing the wods. The English wods wee pesented in lowe case in all phases of the expeiment. Japanese wods wee pesented only duing the encoding phase and wee mostly in Kanji with some hiagana whee appopiate. Following the petest, subjects wee given the instuctions fo the encoding phase of the expeiment. Subjects wee infomed of the ewad associated with getting a coect answe on the post-test fo each of the fou conditions (unknown wods 300, 100, 0 yen; known wods ¼ 0 yen). The diffeent potential ewad values fo leaning an English wod and its Japanese tanslation was epesented by a diffeent coloed suounding squae fame (gold ¼ 300 yen; silve ¼ 100 yen; geen ¼ 0 yen). To incease subjects motivation and ensue that subjects pay attention to the nonewaded wods, incoect answes wee penalized by 100 yen. The encoding phase consisted of thee sessions of about 5 and a half minutes each. In each session, the 40 English wods and thei coesponding Japanese tanslation wee pesented once. To avoid excess eye movement subjects wee instucted to fixate thei eyes on a small fixation coss in the cente of the sceen. Each English wod and its coesponding tanslation was pesented within the ewadcoded coloed suounding squae fame fo 6 s duing which subjects wee instucted to subvocally ehease the association between the English wod and Japanese tanslation. The English wod was pesented just above the fixation point and the Japanese tanslation was pesented just below the fixation coss. The onset of pesentation was synchonized with fmri scanning, using Neuobehavioal System s Pesentation softwae. Following the six-second stimulus pesentation, only the fixation coss emained on the sceen fo 1 s, afte which the next stimulus was pesented (intestimulus inteval ¼ 7 s). The fmri expeiment was an event elated design, in which the vaious items wee pesented pseudo-andomly once pe session, such that subjects wee not able to pedict the condition of the next stimulus. A diffeent event ode was detemined fo each of the thee sessions fo each subject. Afte the encoding phase, a distacte task, which took place inside of the fmri scanne, was given to ensue that subjects would not ehease wods fom the expeiment until the time of the post-test. The distacte task involved vebally poducing visually pesented English syllables beginning with a //, /l/, o a vowel alone. Because English // and /l/ ae vey difficult fo native Japanese speakes to poduce and peceive, and because stimuli wee pesented at a apid pace (evey 2.25 s), this task left no time to ehease wods fom the encoding task. In the 10-min delayed post-test, conducted outside of the fmri scanne, the 40 English wods wee pesented (using Matlab 6.0) sequentially. Subjects wee equied to type the coesponding Japanese wod and then indicate the condition to which the stimuli belonged (300, 100, 0, known). A native Japanese speake poficient in English immediately coected the post-test. Afte the posttest, subjects wee asked to give a qualitative ating (0 5) of thei degee of anxiety associated with each condition (0 ¼ low anxiety; 5 ¼ high anxiety). Specifically they wee asked the following (tanslated fom the Japanese): Did you expeience any anxiety while leaning the English wods? Please indicate the level of anxiety by a numbe (0-low anxiety, 5-high anxiety) fo each Gold (300 yen woth), Silve (100 yen woth), Geen (0 yen woth) and known wods. The expeiment was then ove, and subjects wee given the monetay ewad coesponding to thei pefomance. 239

4 Callan and Schweighofe fmri Data Collection, Pepocessing, and Analysis Fo functional bain imaging, a Shimadzu Maconi s Magnex Eclipse 1.5T PD250 scanne, located at the ATR Bain Activity Imaging Cente, was used. Functional T2* weighted images wee acquied using a gadient echoplana imaging sequence (echo time 48 ms; epetition time 2,000 ms; flip angle 908). A total of 20 contiguous axial slices wee acquied with a mm 3 voxel esolution coveing the ceebum and the top pat of the ceebellum (TR ¼ 2,000 ms). A total of 162 scans wee taken fo a single session. The fist 6 scans wee discaded. Images wee pepocessed using pogams within SPM2 (Wellcome Depatment of Cognitive Neuology, Univesity College, London). Diffeences in acquisition time between slices wee accounted fo, images wee ealigned-unwaped and spatially nomalized to a standad space (default) using a template EPI image (2 2 2mm 3 voxels), and wee smoothed using a mm 3 FWHM Gaussian kenel. The data was assessed (SPM2) using a geneal linea model employing a boxca function convolved with a haemodynamic esponse function (with time and dispesion deivatives). High pass filteing (cutoff peiod equal to twice the maximum diffeence in seconds between two occuences of the same condition) was caied out to educe the effects of extaneous vaiables (scanne dift, low fequency noise, etc.). Auto-egession was used to coect fo seial coelations. Fixed effect analyses wee conducted fo each subject sepaately fo the contasts of inteest (300 yen ewad vesus the 0 yen ewad condition). A andom effect one-sample t- test was conducted using as data the contast estimate (300 yen vesus 0 yen condition) fo each subject. Additionally, a andom effects multiple egession analysis was conducted using each subjects pefomance and anxiety scoes fo the high ewad (300 yen) condition as pedictos and each subjects contast estimates fo the 300 yen vesus 0 yen condition as obsevations. Region of Inteest Analysis The egions of inteest wee defined by MNI (Montéal Neuological Institute) coodinates given in pevious eseach aticles: Hippocampus (encoding) [Meltze and Constable, 2005], (26, 24, 18) seach adius 5 mm; Amygdala (ewad based goal-diected behavio) [Elliott et al., 2004], ( 30, 2, 20) seach adius 5 mm; SN/VTA (ewad anticipation) [Kisch et al., 2003], ( 8, 18, 16) seach adius 5 mm; Anteio cingulate (anticipatoy anxiety) [Chua et al., 1999], (12, 26, 30) seach adius 10 mm; and the ight middle fontal gyus (MFG) (divided attention) [Iidaka et al., 2000], (28, 28, 24) seach adius 5 mm. Functional Connectivity Analysis Functional connectivity among bain egions of inteest was investigated using a psycho-physiologic inteaction (PPI) analysis [Fiston et al., 1997; Gitelman et al., 2003] (SPM2). Seed activity within the egions of inteest upon which the PPI analysis was conducted was detemined by a PCA of active voxels (P < 0.05) in a 5-mm adius centeed at the peak voxel based on the analysis given in Table II. Regions of inteest included the hippocampus, amygdala, and ACC. Afte contast images of the PPI fo each subject was detemined fo the high ewad when compaed with no ewad condition, a andom effects multiple egession analysis was conducted using the same contast as in the egession analysis. RESULTS Behavioal Results Thee was no main effect of ewads on post-test ecall pefomance (1 way ANOVA; F(1,14) ¼ 0.28; P > 0.1; paied two-tailed t-tests: 300 yen > 0 yen: T ¼ 0.63, P > 0.1; 300 yen > 100 yen: T ¼ 0.70, P > 0.1; 100 yen > 0 yen: T ¼ 0.01, P > 0.1) see Figue 1A. Howeve, qualitative anxiety atings showed significant diffeences between the ewad conditions and the no ewad condition (1 way ANOVA; F(1,14) ¼ 5.5, P < 0.05, using the Geenhouse Geisse coection fo nonspheicity; paied two-tailed t-tests: 300 yen > 0 yen: T ¼ 2.5, P < 0.05; 100 yen > 0 yen: T ¼ 2.3, P < 0.05), but no significant diffeence between the ewaded conditions (300 yen > 100 yen: T ¼ 0.69, P > 0.1) see Figue 1B. An analysis of covaiance of ecall pefomance using anxiety ating as a covaiate indicated a significant diffeence between the high ewad condition and the no ewad conditions (300 yen > 0 yen; F(1,14) ¼ 9.21; P < 0.01), but no significant (n.s.) diffeence between the high ewad and low ewad conditions (300 yen > 100 yen; F(1,14) ¼ 1.4; P > 0.1 n.s.), and no significant diffeence between the low ewad and no ewad conditions (100 yen > 0 yen; F(1,14) ¼ 1.9; P > 0.1 n.s.). The 300- yen condition showed a significant negative coelation between ecall pefomance and anxiety ating ( ¼ 0.65, P < 0.01) see Figue 1C. Coelations wee not significant fo the othe two conditions (100 yen: ¼ 0.25, P > 0.1 n.s.; 0 yen: ¼ 0.29, P > 0.1 n.s.). Coelation between anxiety ating fo the diffeent ewad amounts ae as follows: (300 yen and 100 yen: ¼ 0.79, P < 0.001); (100 yen and 0 yen: ¼ 0.27, P > 0.1 n.s.); (300 yen and 0 yen: ¼ 0.12, P > 0.1 n.s.). Bain Imaging Results: One Sample t-test We then assessed the neual coelates of ewads. In line with ou behavioal esults, we found no significant diffeence (even at P < 0.05 one-tailed uncoected) in the hippocampus in a diect compaison between the high ewad goup and the no ewad goup. Only the left SN/VTA showed a tend towad activation fo the diect compaison between the high ewad and no ewad conditions out of all of ou egions of inteest (P < 0.05 one-tailed uncoected). 240

5 Wod Leaning by Rewad Anticipation Figue 1. Behavioal esults. (A) Pecent coect ecall pefomance fo the 300, 100, and 0 yen ewad conditions (standad eos ae plotted above each ba). No significant diffeence was found between the conditions. (B) Qualitative ating scoe of ewad-induced anxiety fo the 300, 100, and 0 ewad conditions (standad eos ae plotted above each ba). The 0 yen ewad condition (*) showed significantly lowe anxiety ating than both ewad conditions. (C) Analysis of covaiance of ecall pefomance using anxiety as a covaiate. The 300 yen condition showed a significant diffeence in ecall pefomance when accounting fo anxiety ating when compaed with the 0 yen condition. The fitted linea plots of ecall pefomance by anxiety ating ae shown fo the 300, 100, and 0 yen conditions. The 300 yen condition (*) showed a significant negative coelation between pefomance and anxiety ating ( ¼ 0.65, P < 0.01). Bain Imaging Results: Multiple Regession To study the neual coelates of the obseved negative coelation between pefomance and ewad-induced anxiety (Fig. 1) in the high ewad condition but not in the no ewad conditon, we pefomed a andom effect multiple egession analysis (SPM2). The contast estimate of the 300 yen minus the 0 yen condition fo each subject was enteed as obsevations into the multiple egession analyses with each subject s pefomance and anxiety scoes fo the 300 yen condition used as pedicto vaiables. The use of the 300 yen minus the 0 yen contast estimate in the multiple egession analysis contols fo many within subject confounds, such as shot tem memoy, woking memoy, pocesses elated to eading, vebal pocessing, etc., so that the neual pocesses involved with ewad elated encoding can be specifically assessed. The multiple egession analysis in SPM2 allows fo the pedicto vaiables to be weighted fo the analysis. To investigate the behavioal negative coelation between pefomance and ewad-induced anxiety we weighted the ecall pefomance vaiable þ1 and the anxiety vaiable 1. As can be seen in Table I and Figues 2 and 4A, thee was significantly geate activity in the left and ight SN/VTA, ight hippocampus, left amygdala, left and ight supeio tempoal gyus/sulcus, left and ight middle tempoal gyus, ight tempoal pole, ight tempoal occipital junction, left middle occipital gyus, left and ight infeio paietal lobule, and ight postcental gyus (P < onetailed uncoected; spatial extent theshold ¼ 50 voxels; in small egions such as the amygdala and midbain aeas a spatial extent theshold of 15 voxels was used. The only additional cluste of activity that was pesent was in the left paahipocampal gyus). A small volume coection analysis fo multiple compaisons, using the false discovey ate (FDR) pocedue [Genovese et al., 2002], showed significant diffeential activity (FDR P < 0.05 one-tailed) in the left SN/ VTA, the ight hippocampus, and the left and ight amygdala (Figs. 2 and 4A, Table II). We then pefomed a second andom effect multiple egession analysis fo the high ewad condition (300 yen) elative to the no ewad condition (0 yen), in which we weighted the pedicto vaiables 1 fo ecall pefomance and þ1 fo anxiety level. As can be seen in Table I and TABLE I. Multiple egession fo the high ewad vs. no ewad condition Bain egion Contast a Contast b Hippocampus Right 28, 26, 6 Right 22, 22, 16 Paahippocampus Right 20, 40, 12 Right 16, 30, 12 Amygdala Left 32, 1, 17 SN/VTA Left 6, 16, 14 Right 10, 24, 11 Anteio cingulated Right 14, 28, 24 Posteio cingulated Left 4, 34, 18 Supeio tempoal Left 56, 22, 8 gyus/sulcus Right 50, 40, 8 Middle tempoal gyus Left 58, 36, 0 Tempoal pole Right 50, 4, 40 Tempoal occipital junction Right 52, 68, 2 Middle occipital gyus Left 18, 88, 10 Infeio paietal lobule Left 48, 30, 48 Right 58, 36, 40 Right 46, 34, 48 Postcental gyus Left 28, 32, 42 Right 60, 24, 48 Right 62, 8, 18 Middle fontal gyus Right 30, 24, 26 The MNI (x,y,z) coodinate denotes the peak voxel in the egion of inteest upon which the fitted linea esponses wee deived. a þ, pefomance;, anxiety. b, pefomance; þ, anxiety. 241

6 Callan and Schweighofe Figue 2. Significant diffeential activity (P < uncoected; spatial extent theshold ¼ 50 voxels) identified by andom-effects multiple egession analysis of the 300 yen elative to the 0 yen condition (weighted pedicto vaiables: þ pefomance, anxiety fo the 300 yen condition). Top: Fo each of the thee egions of inteest, SN/VTA, hippocampus, and amygdale, a coonal o sagittal slice is shown. Bottom: the fitted linea esponses of the contast estimate show how these bain egions exhibit inceased activity fo high pefomance and low anxiety. Hoizontal slices coveing the entie bain ae given in Figue 4. Figues 3 and 4B, thee was significantly geate activity in the ight anteio cingulate (ACC), left posteio cingulate, as well as ight MFG (P < one-tailed uncoected; spatial extent theshold ¼ 50 voxels). A small volume coection analysis fo multiple compaisons showed significant diffeential activity (FDR P < 0.05 one-tailed) fo the ight ACC and the ight MFG (Figs. 3 and 4B, Table II). To futhe illustate the opposite effects of ecall pefomance and ewad-induced anxiety levels in the ROI fo the high ewad condition, we plotted the fitted esponse TABLE II. Multiple egession fo the high ewad vs. no ewad condition: Small Volume Coection Analysis Bain egion Contast a Contast b T pfdr x,y,z T pfdr x,y,z Hippocampus Right 22, 22, 16 SN/VTA Left 8, 16, 18 Amygdala Left 32, 0, Right 34, 2, 18 ACC Right 14, 28, 26 MFG Right 24, 26, 22 The MNI (x,y,z) coodinate denotes the peak voxel in the egion of inteest upon which the fitted linea esponses wee deived. a þ, pefomance;, anxiety. b, pefomance; þ, anxiety. 242

7 Wod Leaning by Rewad Anticipation Significant diffeential activity (P < uncoected; spatial extent theshold ¼ 50 voxels) identified by andom-effects multiple egession analysis of the 300 yen elative to the 0 yen condition (weighted pedicto vaiables: pefomance, þ anxiety fo the 300 yen condition). Top: activity in the two egions of inteest: Figue 3. anteio cingulate and middle fontal gyus. Bottom: the fitted linea esponses of the contast estimate show how these bain egions exhibit inceased activity fo highe anxiety and lowe pefomance. Hoizontal slices coveing the entie bain ae given in Figue 4. obtained as a function of ecall pefomance o anxiety level atings fo each subject. Figue 2 shows the esults fo the left VTA/SN, ight hippocampus, and left amygdala. Figue 3 shows the esults fo the ight ACC and MFG. The left VTA/SN, the ight hippocampus, and left amygdala showed significant positive coelation with pefomance and negative coelation with anxiety ating (Pefomance: left VTA/SN ¼ 0.78, P [lt ]0.001; ight hippocampus: ¼ 0.76, P < 0.005; left amygdala: ¼ 0.77, P < Anxiety ating: VTA/SN ¼ 0.71, P < 0.005; ight hippocampus: ¼ 0.69, P < 0.005; left amygdala: ¼ 0.70, P < 0.005). Convesely, the ight ACC and MFG showed positive coelation with anxiety ating and negative coelation with pefomance (Pefomance: ACC: ¼ 0.81, P < ; MFG: ¼ 0.66, P < 0.01; anxiety: ACC: ¼ 0.73, P < 0.005; MFG: ¼ 0.73, P < 0.005). Bain Imaging Results: Psycho-Physiological Inteaction Analysis We finally investigated functional connectivity among bain egions of inteest, using a PPI analysis [Fiston et al., 1997; Gitelman et al., 2003] (SPM2 see Mateial and Methods). The left SN/VTA showed significant positive functional connectivity (FDR P < 0.05 one-tailed) with the ight hippocampus and the left amygdala associated with highe pefomance and lowe anxiety ating (pedicto vaiable weighting: þ pefomance, anxiety) fo the high ewad (300 yen) vesus low ewad (0 yen) condition (Fig. 5). Convesely, the ight ACC showed a tend towads negative connectivity (P < 0.05 one-tailed uncoected) with the left amygdala associated with lowe pefomance and highe anxiety ating (pedicto vaiable weighting: þ pefomance, anxiety) fo the high ewad (300 yen) vesus low ewad (0 yen) condition (Fig. 5). DISCUSSION Ou behavioal and bain imaging esults confimed ou hypotheses that ewad anticipation has a beneficial modulatoy effect on wod leaning, but only when ewad-geneated anxiety levels ae not too high. Specifically, ou behavioal esults did not show a significant diffeence in pefomance between the ewaded and unewaded items, unless 243

8 (A) Hoizontal slices coveing activity found thoughout the bain fo the same analysis given in Figue 2: Significant diffeential activity (P < uncoected; spatial extent theshold ¼ 50 voxels) identified by andom-effects multiple egession analysis of the 300 yen elative to the 0 yen condition (weighted pedicto vaiables: þ pefomance, anxiety fo the 300 yen condition). (B) Hoizontal slices coveing activity found thoughout the bain fo the same analysis given in Figue 4. Figue 3: Significant diffeential activity (P < uncoected; spatial extent theshold ¼ 50 voxels) identified by andom-effects multiple egession analysis of the 300 yen elative to the 0 yen condition (weighted pedicto vaiables: pefomance, þ anxiety fo the 300 yen condition). MNI Z-axis coodinates ae given below the hoizontal slices.

9 Wod Leaning by Rewad Anticipation Results of the functional connectivity analysis between the egions of inteest. Both the hippocampus and the amygdala show significant (P < 0.05 FDR coected) positive connectivity (geen) with the VTA/SN, but not with each othe fo high ecall pefomance Figue 5. and low anxiety. The anteio cingulate shows a tend (P < 0.05 uncoected) of inhibitoy connectivity (ed) with the amygdala fo low ecall pefomance and high anxiety. measues of the subjects ewad-induced anxiety levels wee taken into account: in this case, fo highly ewaded wods, we found a negative coelation between ecall pefomance and qualitative anxiety ating. Thus, subjects less pone to ewad-induced anxiety tended to ecalled items leaned in the high ewad condition bette than those leaned in the no ewad condition. Fo those subjects moe pone to ewad-induced anxiety, high ewads tended to be detimental fo encoding. Ou bain imaging esults ae also consistent with ou hypothesis: fo the high ewad vesus the no ewad condition, high pefomance and low anxiety atings coelated with activity in the SN/VTA, the hippocampus, and the amygdala (Figs. 2 and 4A, Tables I and II). Pevious studies have shown involvement of these bain egions duing encoding (SN/VTA and inceased dopaminegic levels: [Adcock et al., 2006; Fied et al., 2001; Schott et al., 2004, 2006; Wittmann et al., 2005]; Hippocampus: [Adcock et al., 2006; Casasanto et al., 2002; Geicius et al., 2003; Halsband et al., 2002; Maguie and Fith, 2004; Meltze and Constable, 2005; Menon et al., 2000; Schott et al., 2004; Weis et al., 2004; Wittmann et al., 2005]; Amygdala: [Kensinge and Schacte, 2006; Phelps, 2004]). These same egions have also been implicated duing pocessing of anticipatoy ewad (SN/ VTA: [Adcock et al., 2006; Kisch et al., 2003; Schultz, 1998; Wittmann et al., 2005]; Hippocampus: [Adcock et al., 2006; Wittmann et al., 2005]; Amygdala: [Elliott et al., 2004; Gottfied et al., 2003]). The amygdala in paticula in some situations also shows a decease in activity to ewad omission [Enst et al., 2005] and emotional conflict esolution 245

10 Callan and Schweighofe [Etkin et al., 2006]. In ou expeiment, the possibility of not eceiving a high ewad (ewad omission) may esult in a decease in amygdala activity duing high anxiety. Ou esults futhe suppot the hypothesis accoding to which high ewads can induce anxiety, which leads to divided attention and educed encoding. The lack of an objective and on-line measue of ewad-induced anxiety levels associated with each wod duing fmri scanning, such as galvanic skin esponse is a potential shotcoming of ou study. Howeve, a cooboation of the validity of ou qualitative anxiety ating aises fom ou finding of significant activation in the egion of the ight ACC associated with anxiety [Chua et al., 1999] fo the multiple egession analysis of high ewad (300 yen) elative to no ewad (0 yen) weighting the pedicto vaiables 1 fo ecall pefomance and þ1 fo anxiety (Figs. 3 and 4B, Tables I and II). Futhemoe, in this same compaison, diffeential activity was also pesent in the egion of the MFG (Figs. 3 and 4B, Tables I and II) associated with divided attention [Iidaka et al., 2000]. Thus, ewad-induced anxiety may divide attention between the ewad cue and the item to be leaned, and may cause less efficient encoding. This explanation is consistent with the competing esponse model [Reiss and Sushinsky, 1975], accoding to which the anticipation of pefomancedependent ewads leads to divided attention. Consistent with the view that anxiety induced divided attention educes pefomance unde the high ewad condition, the esults of the multiple egession analysis weighting the pedicto vaiable pefomance þ1 and anxiety 1 (Figs. 2 and 4A, Table I) eveals tends in diffeential activity in bain egions involved with semantic pocessing (MTG: [Menon et al., 2000]; tempoal pole; [Patteson et al., 2006]; infeio paietal lobe, including SMG, [Binde et al., 2003]), auditoy/phonological pocessing (STG/MTG; [Lee, 2004; Scott, 2005], and othogaphic phonological pocessing (occipital-tempoal junction; [Thuy et al., 2004]). Enhanced encoding by elatively geate use of pocessing the semantic elationship between the Japanese wod and coesponding English wod is facilitated by ewad when attention is not divided as a esult of ewad-induced anxiety. A diect compaison between the high-ewad with the no-ewad conditions, i.e., without consideing pefomance and anxiety atings, showed no inceased activity in the hippocampus; only a tend in diffeential activity between conditions was pesent in the left SN/VTA. Thus, ou esults may seem at odds with those of Wittmann et al. [2005] and those of Adcock et al. [2006], who found geate activation in both hippocampus and SN/VTA fo high ewaded vesus neutal o low ewaded items that wee late ecognized. An impotant diffeence in ou study and the Wittmann et al. [2005] study is the dissociation of the ewad and the item to be implicitly leaned in the Wittmann et al. [2005] study, and the diect elationship between anticipatoy ewad and the item to be explicitly leaned in ou study. The detimental effect of anxiety on modulation of encoding by anticipatoy ewad (as found in ou study) may not be pesent when the task that is ewaded is not diectly elated to the encoding of the item to be leaned (as in [Wittmann et al., 2005]. Howeve, Adcock et al. [2006] found pefomance elated changes in the hippocampus and SN/VTA in an explicit leaning task such as ous. A majo diffeence between the two studies howeve is the timing of ewad cue and items to be leaned. In ou study the ewad cue is given simultaneously with the item to be leaned, while in the Adcock et al. [2006] study the ewad cue is pesented seveal seconds befoe the item to be leaned. Simultaneous pesentation may esult in geate anxiety elated divided attention, which can have a detimental effect on pefomance accoding to the competing esponse model [Reiss and Sushinsky, 1975]. Additionally, pesentation of the ewad cue befoe the item to be leaned may esult in geate item-specific attention biases that mediate enhanced pefomance [Loftus, 1972; Nilsson, 1987]. Anothe diffeence between ou study and that of Adcock et al. [2006] and Wittmann et al. [2005] is the use of all female subjects in ou study wheeas the othe studies used pedominantly males. It is possible (howeve we believe this to be unlikely) that the divegence in ou findings fom theis is that the inteaction of pefomance with ewad anxiety exists only fo females. The connectivity analysis evealed that the SN/VTA showed significant functional connectivity to both the hippocampus (consistent with findings by [Adcock et al., 2006]) and the amygdala, fo the high ewad vesus (300 yen) the no ewad (0 yen) condition weighting the pedicto vaiables þ1 fo ecall pefomance and 1 fo anxiety. Although one cannot detemine the ecipocality o causal influence that one bain egion has ove anothe with functional connectivity analysis, this patten of connectivity suggest that the pojections fom the amygdala to the SN/VTA [Fudge and Habe, 2000] may modulate the dopaminegic pojections between SN/VTA and the hippocampus [Gasbai et al., 1996] and facilitate encoding. Futhe, the amygdala may be modulated by anxiety, as we found a tend in inhibitoy connectivity between the ACC and the amygdala, but not to the othe egions (Fig. 5). This is consistent with the known anatomical connectivity between the anteio cingulate and the amygdala [Cunningham et al., 2002]. Thus, ou esults suggest that ewad-geneated anxiety, may not only esult in dividing attention, but also may diectly educe the value of the anticipated ewad, as coded by the dopaminegic neuons. In this way, it would diectly educe the beneficial effect of ewad anticipation on hippocampal encoding. It is inteesting to point out that connectivity between the ostal ACC and the amygdala ae impotant in esolving emotional conflict [Etkin et al., 2006]. Conflict-elated Rostal ACC activity is coelated with a simultaneous eduction of amygdala activity. Activity in the amygdala has also been shown to be implicated with encoding [Kensinge and Schacte, 2006; Phelps 2004] and woking memoy pefomance [Schaefe et al., 2006]. It is easonable to conjectue that anxiety induced activity in the ACC found in ou study may inhibit amygdala ewad elated activity that is impotant 246

11 Wod Leaning by Rewad Anticipation fo facilitating dopamine elease in SN/VTA leading to enhanced hippocampal encoding. Ou study gives an account to the discepancies obseved in neuoscience expeiments and in psychological behavioal expeiments on the effects of ewads on declaative memoy encoding. On one hand, mounting evidence suggests that ewads enhance declaative leaning via dopaminegic elease in the hippocampus via pathways with the VTA/SN [Adcock et al., 2006; Schott et al., 2004; Wittmann et al., 2005]. On the othe hand, these esults ae somewhat at odds with a lage numbe of behavioal studies that show null o even detimental effects of ewads on pefomance and leaning [Deci et al., 1999; Sansone and Haackiewicz, 2000]. Consistent with the competing esponse model [Reiss and Sushinsky, 1975], ou esults suggest that unde conditions of high ewads and low anxiety (less divided attention), inceased SN/VTA activity, via inceased amygdala activity, enhances dopaminegic elease in the hippocampus. Howeve, unde conditions of high anxiety, the activity in the amygdala is suppessed by inhibitoy connections fom the ACC, in tun educing SN/VTA activation, and thus educing the dopaminegic elease in the hippocampus. In sum, both ou behavioal and imaging esults suggest that, in leaning situations, pefomance-based ewads should be used pasimoniously, as ewads can geneate anxiety that can cancel the potential benefit of ewads on encoding. ACKNOWLEDGMENTS We wish to thank Akiko Callan fo he help with compute pogamming and Saoi Tanaka, Haold Hill, and Rebecca Lewthwaite fo helpful comments on pevious vesions of this manuscipt. REFERENCES Adcock RA, Thangavel A, Whitfield-Gabieli S, Knutson B, Gabieli JD (2006): Rewad-motivated leaning: Mesolimbic activation pecedes memoy fomation. Neuon 50: Andeoletti C, Veatti BW, Lachman ME (2006): Age diffeences in the elationship between anxiety and ecall. Aging Ment Health 10: Benado LS, Pince DA (1982): Dopamine action on hippocampal pyamidal cells. J Neuosci 2: Binde JR, McKienan KA, Pasons ME, Westbuy CF, Possing ET, Kaufman JN, Buchanan L (2003): Neual coelates of lexical access duing visual wod ecognition. J Cogn Neuosci 15: Bush G, Luu P, Posne MI (2000): Cognitive and emotional influences in anteio cingulate cotex. Tends Cogn Sci 4: Cameon J, Piece WD (1994): Reinfocement, ewad, and intinsic motivation: A meta-analysis. Rev Educ Res 64: Casasanto DJ, Killgoe WD, Maldjian JA, Glosse G, Alsop DC, Cooke AM, Gossman M, Dete JA (2002): Neual coelates of successful and unsuccessful vebal memoy encoding. Bain Lang 80: Chua P, Kams M, Toni I, Passingham R, Dolan R (1999): A functional anatomy of anticipatoy anxiety. Neuoimage 9 (6, Pat 1): Cunningham MG, Bhattachayya S, Benes FM (2002): Amygdalocotical spouting continues into ealy adulthood: Implications fo the development of nomal and abnomal function duing adolescence. J Comp Neuol 453: Deci EL, Koestne R, Ryan RM (1999): A meta-analytic eview of expeiments examining the effects of extinsic ewads on intinsic motivation. Psychol Bull 125: Discussion Elliott R, Newman JL, Longe OA, William Deakin JF (2004): Instumental esponding fo ewads is associated with enhanced neuonal esponse in subcotical ewad systems. Neuoimage 21: Enst M, Nelson EE, Jazbec S, McClue EB, Monk CS, Leibenluft E, Blai J, Pine DS (2005): Amygdala and nucleus accumbens in esponses to eceipt and omission of gains in adults and adolescents. Neuoimage 25: Etkin A, Egne T, Peaza DM, Kandel ER, Hisch J (2006): Resolving emotional conflict: A ole fo the ostal anteio cingulage cotex in modulating activity in the amygdala. Neuon 51: Fey U, Hatmann S, Matthies H (1989): Dompeidone, an inhibito of the D2-ecepto, blocks a late phase of an electically induced long-tem potentiation in the CA1-egion in ats. Biomed Biochim Acta 48: Fey U, Schoede H, Matthies H (1990): Dopaminegic antagonists pevent long-tem maintenance of posttetanic LTP in the CA1 egion of at hippocampal slices. Bain Res 522: Fey U, Matthies H, Reymann KG, Matthies H (1991): The effect of dopaminegic D1 ecepto blockade duing tetanization on the expession of long-tem potentiation in the at CA1 egion in vito. Neuosci Lett 129: Fied I, Wilson CL, Moow JW, Cameon KA, Behnke ED, Ackeson LC, Maidment NT (2001): Inceased dopamine elease in the human amygdala duing pefomance of cognitive tasks. Nat Neuosci 4: Fiston KJ, Buechel C, Fink GR, Mois J, Rolls E, Dolan RJ (1997): Psychophysiological and modulatoy inteactions in neuoimaging. Neuoimage 6: Fudge JL, Habe SN (2000): The cental nucleus of the amygdala pojection to dopamine subpopulations in pimates. Neuoscience 97: Gasbai A, Sulli A, Innocenzi R, Pacitti C, Bioni JD (1996): Spatial memoy impaiment induced by lesion of the mesohippocampal dopaminegic system in the at. Neuoscience 74: Gasbai A, Sulli A, Packad MG (1997): The dopaminegic mesencephalic pojections to the hippocampal fomation in the at. Pog Neuopsychophamacol Biol Psychiaty 21:1 22. Genovese CR, Laza NA, Nichols T (2002): Thesholding of statistical maps in functional neuoimaging using the false discovey ate. Neuoimage 15: Gitelman DR, Penny WD, Ashbune J, Fiston KJ (2003): Modeling egional and psychophysiologic inteactions in fmri: The impotance of hemodynamic deconvolution. Neuoimage 19: Gottfied JA, O Dohety J, Dolan RJ (2003): Encoding pedictive ewad value in human amygdala and obitofontal cotex. Science 301: Geicius MD, Kasnow B, Boyett-Andeson JM, Eliez S, Schatzbeg AF, Reiss AL, Menon V (2003): Regional analysis of hippocampal activation duing memoy encoding and etieval: fmri Study. Hippocampus 13: Gibkoff VK, Ashe JH (1984): Modulation by dopamine of population esponses and cell membane popeties of hippocampal CA1 neuons in vito. Bain Res 292:

12 Callan and Schweighofe Halsband U, Kause BJ, Sipila H, Teas M, Laihinen A (2002): PET studies on the memoy pocessing of wod pais in bilingual Finnish-English subjects. Behav Bain Res 132: Haackiewicz JM, Sansone C (2000): Rewading competence: The impotance of goals in the study of intinsic motivation. In: Haackiewicz JM, Sansone C, editos. Intinsic and Extinsic Motivation: The Seach fo Optimal Motivation and Pefomance. New Yok: Academic Pess. P Heckes S, Weiss AP, Alpet NM, Schacte DL (2002): Hippocampal and bain stim activation duing wod etieval afte epeated and semantic encoding. Ceeb Cotex 12: Hudetz JA, Hudetz AG, Reddy DM (2004): Effect of elaxation on woking memoy and the bispectal index of the EEG. Psychol Rep 95: Iidaka T, Andeson ND, Kapu S, Cabeza R, Caik FI (2000): The effect of divided attention on encoding and etieval in episodic memoy evealed by positon emission tomogaphy. J Cogn Neuosci 12: Kensinge EA, Schacte DL (2006): Amygdala activity is associated with the successful encoding of item, but not souce, infomation fo positive and negative stimuli. J Neuosci 26: Kisch P, Schienle A, Stak R, Samme G, Blecke C, Walte B, Ott U, Bukat J, Vaitl D (2003): Anticipation of ewad in a nonavesive diffeential conditioning paadigm and the bain ewad system: an event-elated fmri study. Neuoimage 20: Lee KM (2004): Functional MRI compaison between eading ideogaphic and phonogaphic scipts of one language. Bain Lang 91: Li S, Cullen WK, Anwyl R, Rowan MJ (2003): Dopamine-dependent facilitation of LTP induction in hippocampal CA1 by exposue to spatial novelty. Nat Neuosci 6: Loftus G (1972): Eye fixations and ecognition memoy fo pictues. Cogn Psychol 3: Maguie EA, Fith CD (2004): The bain netwok associated with acquiing semantic knowledge. Neuoimage 22: Meltze JA, Constable RT (2005): Activation of human hippocampal fomation eflects success in both encoding and cued ecall of paied associates. Neuoimage 24: Menon V, White CD, Eliez S, Glove GH, Reiss AL (2000): Analysis of a distibuted neual system involved in spatial infomation, novelty, and memoy pocessing. Hum Bain Mapp 11: Mogg K, Badley BP (1998): A cognitive motivational analysis of anxiety. Behav Res The 36: Nilsson L (1987): Motivated memoy: Dissociation between pefomance data and subjective epots. Psychol Res 49: Otmakhova NA, Lisman JE (1999): Dopamine selectively inhibits the diect cotical pathway to the CA1 hippocampal egion. J Neuosci 19: Patteson K, Lambon Ralph MA, Jeffeies E, Woollams A, Jones R, Hodges JR, Roges TT (2006): Pesemantic cognition in semantic dementia: Six deficits in seach of an explanation. J Cogn Neuosci 18: Phelps EA (2004): Human emotion and memoy: Inteactions of the amygdala and hippocampal complex. Cu Opin Neuobiol 14: Reiss S, Sushinsky LW (1975): Ovejustification, competing esponses, and the acquisition of intinsic inteest. J Pesonal Soc Psychol 31: Rushwoth MFS, Walton ME, Kenneley SW, Banneman DM (2004): Action sets and decisions in the medial fontal cotex. Tends Cogn Sci 8: Sansone C, Haackiewicz JM (2000): Intinsic and Extinsic Motivation: The Seach fo Optimal Motivation and Pefomance. New Yok: Academic Pess. Schaefe A, Bave TS, Reynolds JR, Bugess GC, Yakoni T, Gay JR (2006): Individual diffeences in amygdala activity pedict esponse speed duing woking memoy. J Neuosci 26: Schott BH, Sellne DB, Laue CJ, Habib R, Fey JU, Gudeian S, Heinze HJ, Duzel E (2004): Activation of midbain stuctues by associative novelty and the fomation of explicit memoy in humans. Lean Mem 11: Schott BH, Seidenbeche CI, Fenke DB, Laue CJ, Bunzeck N, Benstein HG, Tischmeye W, Gundelfinge ED, Heinze HJ, Duzel E (2006): The dopaminegic midbain paticipates in human episodic memoy fomation: Evidence fom genetic imaging. J Neuosci 26: Schultz W (1998): Pedictive ewad signal of dopamine neuons. J Neuophysiol 80:1 27. Scott SK (2005): Auditoy pocessing speech, space and auditoy objects. Cu Opin Neuobiol 15: Thiey AM, Gioanni Y, Degenetais E, Glowinski J (2000): Hippocampo-pefontal cotex pathway: Anatomical and electophysiological chaacteistics. Hippocampus 10: Thuy DH, Matsuo K, Nakamua K, Toma K, Oga T, Nakai T, Shibasaki H, Fukuyama H (2004): Implicit and explicit pocessing of kanji and kana wods and non-wods studied with fmri. Neuoimage 23: Tolman EC, Honzik CH (1930): Intoduction and emoval of ewad, and maze pefomance in ats. Univ Calif Public Psychol 4: Weis S, Klave P, Reul J, Elge CE, Fenandez G (2004): Tempoal and ceebella bain egions that suppot both declaative memoy fomation and etieval. Ceeb Cotex 14(3), Wittmann BC, Schott BH, Gudeian S, Fey JU, Heinze HJ, Duzel E (2005): Rewad-elated FMRI activation of dopaminegic midbain is associated with enhanced hippocampus-dependent long-tem memoy fomation. Neuon 45:

13 Wod Leaning by Rewad Anticipation APPENDIX: WORD LIST 249