NeuroImage. Decoded fmri neurofeedback can induce bidirectional confidence changes within single participants

Transcription

1 NeuroImage 149 (2017) Contents lsts avalable at ScenceDrect NeuroImage journal homepage: Decoded fmri neurofeedback can nduce bdrectonal confdence changes wthn sngle partcpants Aurelo Cortese a,b,c,d,, Kaoru Amano c, A Kozum a,c, Hakwan Lau d,e,, Mtsuo Kawato a,b,c, a Department of Decoded Neurofeedback, ATR Computatonal Neuroscence Laboratores, Kyoto, Japan b Faculty of Informaton Scence, Nara Insttute of Scence and Technology, Nara, Japan c Center for Informaton and Neural Networks (CNet), NICT, Osaka, Japan d Department of Psychology, UCLA, Los Angeles, USA e Bran Research Insttute, UCLA, Los Angeles, USA ARTICLE INFO Keywords: fmri neurofeedback MVPA Nonlnear modelng Confdence ABSTRACT Neurofeedback studes usng real-tme functonal magnetc resonance magng (rt-fmri) have recently ncorporated the mult-voxel pattern decodng approach, allowng for fmri to serve as a tool to manpulate fne-graned neural actvty embedded n voxel patterns. Because of ts tremendous potental for clncal applcatons, certan questons regardng decoded neurofeedback (DecNef) must be addressed. Specfcally, can the same partcpants learn to nduce neural patterns n opposte drectons n dfferent sessons? If so, how does prevous learnng affect subsequent nducton effectveness? These questons are crtcal because neurofeedback effects can last for months, but the short- to md-term dynamcs of such effects are unknown. Here we employed a wthn-subjects desgn, where partcpants underwent two DecNef tranng sessons to nduce behavoural changes of opposng drectonalty (up or down regulaton of perceptual confdence n a vsual dscrmnaton task), wth the order of tranng counterbalanced across partcpants. Behavoral results ndcated that the manpulaton was strongly nfluenced by the order and the drectonalty of neurofeedback tranng. We appled nonlnear mathematcal modelng to parametrze four man consequences of DecNef: man effect of change n confdence, strength of down-regulaton of confdence relatve to up-regulaton, mantenance of learnng effects, and anterograde learnng nterference. Modelng results revealed that DecNef successfully nduced bdrectonal confdence changes n dfferent sessons wthn sngle partcpants. Furthermore, the effect of up- compared to down-regulaton was more promnent, and confdence changes (regardless of the drecton) were largely preserved even after a week-long nterval. Lastly, the effect of the second sesson was markedly dmnshed as compared to the effect of the frst sesson, ndcatng strong anterograde learnng nterference. These results are nterpreted n the framework of renforcement learnng and provde mportant mplcatons for ts applcaton to basc neuroscence, to occupatonal and sports tranng, and to therapy. Introducton Real-tme functonal magnetc resonance magng (rt-fmri) neurofeedback has enjoyed a consderable rse n nterest n recent years, both as a tool for addressng basc neurobologcal questons as well as for potental clncal applcatons (decharms, 2008; Sulzer et al., 2013; Km and Brbaumer, 2014; Staram et al., 2016). Whereas most prevous studes have manly focused on partcpants learnng to self-regulate a unvarate blood-oxygen-dependent-level (BOLD) sgnal n specfc bran areas (Weskopf et al., 2003; decharms et al., 2004; Brbaumer et al., 2013; Sulzer et al., 2013), recently rt-fmri neurofeedback has ncorporated connectvty-based approaches (Sulzer et al., 2013; Koush et al., 2013, 2015; Megum et al., 2015) and mult-voxel pattern analyss (MVPA) [or decodng analyss, (Kamtan and Tong, 2005)], openng a new range of possbltes (LaConte et al., 2007; LaConte, 2011; Shbata et al., 2011; debettencourt et al., 2015). Here we nvestgate some propertes of the relatvely new procedure called decoded neurofeedback (DecNef), specfcally focusng on: bdrectonal behavoral changes, the magntude of each tranng drecton, and the effects of order of tranng, such as the mantenance of tranng effects over tme and nterference between tranng sessons. vously, up-and-down regulaton of unvarate BOLD sgnal wthn a sngle partcpant n nterleaved block desgns of rt-fmri neurofeedback has been shown to be possble (Weskopf et al., 2004; Correspondng authors at: Department of Decoded Neurofeedback, ATR Computatonal Neuroscence Laboratores, Kyoto, Japan. Correspondng author at: Department of Psychology, UCLA, Los Angeles, USA. E-mal addresses: cortese.aurelo@gmal.com (A. Cortese), hakwan@gmal.com (H. Lau), kawato@atr.jp (M. Kawato). Receved 8 November 2016; Accepted 28 January 2017 Avalable onlne 03 February / 2017 The Author(s). Publshed by Elsever Inc. Ths s an open access artcle under the CC BY-NC-ND lcense (

2 decharms, 2008). Subsequently, functonal connectvty between regons, as well as unvarate actvty wthn specfc areas, was shown to be ncreased or decreased upon tranng (Schenost et al., 2013; Vet et al., 2012). Usng a multvarate neurofeedback approach Shbata and colleagues (Shbata et al., 2016) smlarly traned partcpants to actvate or deactvate mult-voxel patterns assocated wth facal attractveness. In a between-subjects desgn, dfferent groups of partcpants learned to actvate or deactvate the relevant neural representaton, and these led to subsequent ncreases or decreases n facal-preference ratngs (Shbata et al., 2016). In many of these prevous studes partcpants were gven explct strateges for neural nducton (e.g., whch neural loc to manpulate) through verbal nstructons. In contrast, some other rt-fmri neurofeedback studes succeeded n tranng subjects to actvate or deactvate (patterns of) neural actvty or connectvty covertly wthout explct nstructon about the target of nducton or the purpose of the tranng (Bray et al., 2007; Megum et al., 2015; Shbata et al., 2016; Ramot et al., 2016). In these studes partcpants were smply told to focus somehow on ther mental actvty n each tral, after whch a feedback sgnal was gven that ndcated the extent of the reward. Although both approaches have prevously been used, there s currently an open debate n the feld on whch strategy - explct vs. mplct nstructon - s preferable and s most effectve (Sulzer et al., 2013; Staram et al., 2016). Nevertheless, snce explct nstructons do not seem to be necessary, fmri neurofeedback can be treated as a neural operant condtonng or renforcement learnng process (Brbaumer et al., 2013; Koralek et al., 2012). From ths perspectve t s worth notng that so far, to the best of our knowledge, no study has successfully demonstrated opposng behavoral changes for dfferent neurofeedback manpulatons wthn sngle partcpants. Ths open queston s mportant for both practcal as well as scentfc reasons. Scentfcally, as n optogenetcs studes n rodents (Desseroth, 2010, 2015) and bran stmulaton approaches n humans (Wagner et al., 2007), to rgorously prove the causal relatonshps between bran actvty and behavors t s desrable to be able to nduce and suppress the same pattern of bran actvtes. Importantly, t s necessary to confrm that these ndeed lead to opposte behavoral effects wthn the same partcpants. Practcally, these consderatons are also very mportant for future studes n basc neuroscence, for therapy and clncal applcatons of DecNef, as well as f ths s to be used for occupatonal and sport tranng. Consderng neurofeedback from the perspectve of renforcement learnng hghlghts why manpulatng actvty n both drectons wthn the same partcpants may pose specfc challenges. Interference of learnng across tranng sessons s expected when we attempt opposte behavoral manpulatons n successon. These effects of nterference have been studed extensvely n motor and vsuomotor learnng, wth varous elegant studes showng that prevous learnng can hnder or nterfere wth the subsequent practce of a second task (Brashers-Krug et al., 1996; Krakauer et al., 1999; Osu et al., 2004). Interference can be retrograde or anterograde dependng on the drecton n tme of the learnng/memory effects. In a classc A 1 BA 2 paradgm, partcpants are nstructed to sequentally learn Task A, Task B, and then Task A agan. Retrograde effects reflect how learnng of Task B affects the memory mantenance of Task A 1, whle anterograde effects reflect how the memory of Task A 1 affects the learnng of Task B (Sng and Smth, 2010). Anterograde nterference has receved less attenton n the lterature (Sng and Smth, 2010). Interference effects have also been shown to be present n perceptual learnng (Setz et al., 2005; Yotsumoto et al., 2009). Hence, anterograde nterference of learnng resultng from the bdrectonal use of DecNef manpulatons wthn partcpants may prove two aspects: (1) behavoral changes are the result of a true learnng process and, (2) behavoral effects should be long lastng. We therefore amed to address the followng three questons n ths research. Frst, f some behavoral change s nduced by a neurofeedback manpulaton, s t possble to develop another neurofeedback manpulaton to cancel out the frst behavoral change? Second, how long s the behavoral change mantaned after neurofeedback manpulaton? Thrd, how much nterference occurs when two dfferent neurofeedback manpulatons are conducted n sngle partcpants? The frst queston s mportant because t s desrable to have the ablty to cancel out negatve sde effects, should these occur. The second queston s related to the effcency of neurofeedback as a therapeutc method. Megum et al. (2015) showed that 4 days of functonal connectvty neurofeedback (FCNef) changed restng-state functonal connectvty, and these effects lasted more than two months. Amano et al. (2016) showed that 3 days of DecNef nduced assocatve learnng between color and orentaton lastng for 3 5 months. However, there s no quanttatve study examnng md-term effects; for example, to what extent are neurofeedback effects mantaned one week after manpulaton? The thrd pont s ethcally mportant n consderng cross-over desgns as canddate paradgms for randomzed control trals to show statstcal effectveness of neurofeedback therapy. If two dfferent neurofeedback manpulatons nterfere severely wthn sngle patents, a cross-over desgn s not a sutable opton. To address these questons, here we used DecNef to manpulate a specfc cogntve representaton - perceptual confdence - bdrectonally,.e., up (ncrease confdence), and down (decrease confdence). Perceptual confdence can be best nterpreted as the degree of certanty n one's own perceptual decsons. Several studes have lnked frontoparetal areas wth the computaton of perceptual confdence, n humans (Huettel et al., 2005; Flemng et al., 2010; Rouns et al., 2010; Smons et al., 2010; De Martno et al., 2012; Zzlsperger et al., 2014; Rahnev et al., 2016) as well as n prmates (Kan and Shadlen, 2009; Fetsch et al., 2014) and rats (Kepecs et al., 2008; Lak et al., 2014). Specfcally, dorsolateral prefrontal cortex (dlpfc) as well as nferor paretal cortex seem to play a crucal role. For example, gray matter thckness n the dlpfc correlated wth metacogntve abltes of partcpants (Flemng et al., 2010), and transcranal magnetc stmulatons (TMS) appled over loc wthn the dlpfc have been shown to selectvely dsrupt confdence judgements (Rouns et al., 2010). A recent study has hghlghted the temporal structure of confdence processng n cortcal areas, demonstratng that electrophysologcal correlates of decson confdence can be detected n the left paretal cortex (Zzlsperger et al., 2014). We frst constructed a classfer for hgh vs. low confdence by utlzng MVPA n four blateral, anatomcally defned regons of nterest (ROIs): the nferor paretal lobule (IPL), and three subregons of the dlpfc - the nferor frontal sulcus (IFS), mddle frontal sulcus (MFS) and mddle frontal gyrus (MFG). Next, wth a wthn-subjects desgn, partcpants learned to mplctly nduce mult-voxel actvaton patterns reflectng hgh and low confdence (Up- and Down-DecNef, respectvely) over two weeks. In both weeks, nducton sessons took place across two consecutve days, and confdence changes were measured wth a - and -Test, mmedately before and after the fmri nducton sesson. Partcpants were randomly assgned to one of two groups, defnng the order of nducton (Up- then Down- DecNef or vce versa). The second DecNef sesson was carred out one week after the frst, n order to measure whether effects would survve after a one-week nterval. To best capture the dfferental effects of DecNef on confdence judgements, we utlzed nonlnear equaton modelng. Materals and methods Partcpants and expermental desgn All experments and data analyses were conducted at the Advanced Telecommuncatons Research Insttute Internatonal (ATR). The study was approved by the Insttutonal Revew Board of ATR. All partcpants gave pror wrtten nformed consent. A companon paper 324

3 MVPA sesson -test DecNef 1 Low Confdence Day 2 Day 3 -test Week-long nterval -test DecNef 2 Hgh Confdence Day 4 Day 5 -test Day 1 Group D-U Group U-D -test DecNef 1 Hgh Confdence Day 2 Day 3 -test Week-long nterval -test DecNef 2 Low Confdence Day 4 Day 5 -test Fxaton 1 s Nose 1 s Stmulus 2 s Delay 4 s Decson RIGHT LEFT 2 s Confdence s ITI 6 s I P L IFS M F S M F G Inducton Rest Feedback ITI 6s 2s 6s 2s 6s Fg. 1. Experment tmelne and desgn. (a) The entre experment conssted of 6 fmri scannng days grouped nto 4 sessons. The frst two days were dentcal for all partcpants: a retnotopy sesson to functonally defne vsual areas, followed by an MVPA sesson, durng whch partcpants of both groups performed n a 2-forced choce dscrmnaton task wth confdence ratng. Partcpants were then randomly assgned to ether group D-U or group U-D. For the subsequent DecNef sessons (day 3 4 and 5 6), group U-D frst underwent Hgh Confdence followed by Low Confdence DecNef, whle group D-U dd the reverse sequence. In order to examne the change n confdence due to DecNef, each DecNef sesson was preceded and followed by a - and -test (on the same days), a psychophyscal assessment usng the same behavoral task employed n the MVPA sesson. (b) In the MVPA sesson, and n - and -Tests, each tral started wth a fxaton cross, followed by a nose random dot moton (RDM). The stmulus was then presented, consstng of a coherent RDM wth ether rghtward or leftward moton. After a 4 s delay, partcpants were requred to report the drecton of moton (left or rght) and ther confdence n ther decson of drecton durng a fxed tme wndow. A tral ended wth a 6 s ITI. Three TRs, startng at stmulus presentaton onset, were averaged and used for the actual MVPA. (c) Blateral frontoparetal ROIs used for MVPA and DecNef, from a representatve partcpant. Each ROI s depcted on an nflated cortcal surface for both the rght and left hemspheres. IPL: nferor paretal lobule, IFS: nferor frontal sulcus, MFS: mddle frontal sulcus, MFG: mddle frontal gyrus. (d) A neurofeedback tral commenced wth a vsual cue ndcatng the nducton perod, durng whch partcpants were asked to manpulate, change ther bran actvty n order to maxmze the sze of the feedback dsc and the reward. Importantly, the dsc for feedback after the nducton perod equally needed to be maxmzed both for up- and down- regulaton. Inducton was followed by a rest perod, then the feedback dsc was presented for 2 s and a tral ended wth a 6 s ITI. Group D-U: Down- then Up-DecNef, group U-D: Up- then Down-DecNef; ITI: ntertral nterval. 325

4 (Cortese et al., 2016) was publshed elsewhere, whch dscussed mplcatons of the results for neural mechansms of metacogntve functons, especally perceptual confdence and conscous awareness. The expermental data used here s exactly the same, but the research objectves of the two manuscrpts are dfferent. The entre experment was subdvded nto 4 fmri sessons, spannng over 6 scannng days. In the frst sesson, partcpants took part n a retnotopy scan, followed by an MVPA sesson, on separate days (Fg. 1A, the retnotopy data was used to functonally defne vsual areas but was not used for any of the analyses pertanng to ths manuscrpt, and thus not shown n the fgure). Durng the MVPA sesson partcpants performed a 2-alternatve forced choce dscrmnaton task wth confdence judgements whle n the fmri scanner. Ths allowed us to decode the actvaton patterns correspondng to certan levels of perceptual confdence, whch were to be manpulated wth DecNef. After MVPA, partcpants undergong DecNef tranng were randomly assgned to one of two groups, wth dfferent orders of tranng: Down-Up group (amng to nduce Low confdence, then Hgh confdence, D-U throughout the manuscrpt) or Up-Down group (amng to nduce Hgh confdence, then Low confdence; U-D throughout the manuscrpt). The expermenter was not blnded to group assgnment, whle partcpant were unaware of the ams of tranng nor of group assgnment. The confdence change between before vs. after DecNef, defned as changes n confdence n the psychophyscal tasks, s our prmary dependent varable of nterest. Eghteen partcpants (23.7 ± years old; 4 female) wth normal or corrected-to-normal vson were enrolled n the frst part of the study (MVPA sesson). One partcpant was removed due to corrupted data. Partcpants were screened out ether f they could not or declned to come back for the DecNef sessons, or due to low decodng accuracy ( < 55% n more than two ROIs, based on prevous work (Amano et al., 2016)). Ten partcpants performed the full DecNef experments (24.2 ± 3.2 years old, 3 female). For transparency, Table 1 reports the decodng results of the 17 subjects for whch the decodng analyss was run. Stmul, behavoral task and DecNef desgns Behavoral task The behavoral task was the same for both MVPA, and -/- Tests. In behavoral - and -Tests, partcpants performed the task on a standard computer and gave responses wth a standard keyboard outsde of the fmri scanner. In the MVPA sesson, partcpants gave ther responses va a 4-buttons pad whle n the scanner. The behavoral task was a moton dscrmnaton task wth twoalternatve forced choce of moton drecton and confdence ratng usng random dot moton (RDM, Fg. 1B). Partcpants were nstructed to ndcate the perceved drecton of moton (left or rght) after a short delay followng stmulus presentaton, and rate the level of confdence about ther perceptual decson (4-pont scale). Ths knd of scale s commonly used n confdence studes as t allows greater senstvty for partcpants tral-by-tral confdence judgements n ambguous perceptual decsons compared wth a smpler two-optons ratng (such as hgh or low) (Flemng et al., 2015). The majorty of trals (6% of the total number) had threshold coherence of moton, that s, moton coherence that lead to the targeted psychologcal threshold, mdway between floor and celng performance. The rest of the trals were dvded between catch trals (no coherence, 25% of the total number of trals) and hgh coherence trals (80% coherence, 1% of the total number of trals). The order was randomzed wthn and across runs. Ths confguraton of moton coherences was selected n order to prevent slow drfts n bas (ndvdual crteron n respondng) across runs. For all analyses (both behavoral and wth fmri data), only trals at threshold moton coherence were utlzed. Thus, for these trals the moton drecton was ambguous, and performance -.e., task accuracy - n judgng the moton drecton was not sgnfcantly dfferent from the targeted psychologcal threshold level of 75% correct (76.6 ± %, t (16) =1.475, P=0.16) durng the MVPA scannng sesson. Vsual stmul All stmul were created and presented wth Matlab (Mathworks) usng the Psychophyscs Toolbox extensons Psychtoolbox 3 (Branard, 1997). Vsual stmul were presented on an LCD dsplay ( resoluton, 60 Hz refresh rate) durng ttraton and the - and - Test stages, and va an LCD projector ( resoluton, 60 Hz refresh rate) durng fmri measurements n a dm room. Stmul were shown on a black background and conssted of RDM. We used the Movshon-Newsome (MN) RDM algorthm (Shadlen and Newsome, 2001). The stmulus was created n a square regon of 20 2, but only the regon wthn a crcular annulus was vsble (outer radus: 10, nner radus: 0.85 ). Dot densty was (contrast 100%), wth a speed of 9 /s and sze of Sgnal dots all moved n the same drecton (left or rght, non-cardnal drectons of 20 and 200 ) whereas nose dots were randomly replotted. Dots leavng the square regon were replaced wth a dot along one of the edges opposte to the drecton of moton, and dots leavng the annulus were faded out to mnmze edge effects. Table 1 Classfcaton of confdence (hgh vs. low) wth permutaton testng (n=1000) for statstcal sgnfcance evaluaton. Columns (4) represent the four ROIs of nterest, chosen a pror based on prevous studes hghlghtng ther mportance n the computaton of confdence judgements. Rows represent ndvdual partcpant's data, wth the frst 10 havng partcpated n the full DecNef tranng program. IPL IFS MFS MFG 69.5 ± 8.3, P= ± 8.2, P= ± 8.4, P= ± 6.5, P= ± 7.3, P= ± 5.3, P= ± 7.9, P= ± 7.5, P= ± 3.1, P= ± 5.6, P= ± 3.6, P= ± 4.9, P= ± 6.3, P= ± 5.5, P= ± 7.0, P= ± 5.7, P= ± 6.7, P= ± 2.9, P= ± 4.6, P= ± 6.3, P= ± 7.5, P= ± 6.5, P= ± 7.5, P= ± 5.0, P= ± 7.0, P= ± 4.4, P= ± 6.5, P= ± 4.4, P= ± 6.1, P= ± 4.9, P= ± 4.8, P= ± 5.1, P= ± 5.8, P= ± 3.4, P= ± 3.0, P= ± 6.1, P= ± 6.4, P= ± 5.4, P= ± 3.3, P= ± 3.7, P= ± 6.5, P= ± 2.7, P= ± 4.6, P= ± 4.0, P= ± 5.0, P= ± 4.9, P= ± 6.7, P= ± 5.3, P= ± 5.0, P= ± 7.2, P= ± 5.3, P= ± 7.1, P= ± 5.5, P= ± 4.0, P= ± 4.2, P= ± 5.4, P= ± 5.2, P= ± 3.6, P= ± 6.4, P= ± 5.1, P= ± 5.6, P= ± 6.3, P= ± 4.1, P= ± 3.5, P= ± 9.7, P= ± 6.5, P= ± 7.6, P= ± 5.3, P=

5 fmri scans for MVPA The purpose of the fmri scans n the MVPA sesson was to obtan the fmri sgnals correspondng to hgh and low confdence levels. These confdence measures would then be used as labels to compute the parameters for the decoders used n the MVPA and DecNef sessons (Shbata et al., 2011). Durng the MVPA sesson, partcpants performed a perceptual dscrmnaton task wth a confdence ratng n the fmri scanner (see above, subsecton Behavoral task and Fg. 1B). Throughout the fmri runs, partcpants were asked to fxate on a whte cross (sze 0.5 ) presented at the center of the dsplay. A bref break perod was provded after each run upon partcpant's request. Each fmri run conssted of 16 task trals (1 tral=18 s; Fg. 1B), wth a 20 s fxaton perod before the frst tral (1 run=308 s). The entre sesson conssted of 12 runs. The fmri data for the ntal 20 s of each run were dscarded due to possble unsaturated T1 effects. Durng the response perod, partcpants were nstructed to use ther domnant hand to press the button on a response pad. Concordance between responses and buttons was ndcated on the screen and, mportantly, randomly changed across trals to avod motor preparaton confounds (.e., assocatng a gven response wth a specfc button press). fmri scans preprocessng The fmri sgnals n natve space were preprocessed usng custom software (mrvsta software package for MATLAB, freely avalable at The mrvsta package uses functons and algorthms from the SPM sute (freely avalable at All functonal mages underwent 3D moton correcton. Hem-lateral ROIs were anatomcally defned through cortcal reconstructon and volumetrc segmentaton usng the Freesurfer mage analyss sute, whch s documented and freely avalable for download onlne ( We used the standard Destreux cortcal atlas, based on a parcellaton scheme that dvdes the cortex nto gyral and sulcal regons (Destreux et al., 2010) n each hemsphere. Blateral ROIs were then created by mergng the correspondng sngle left and rght hem-lateral ROIs. Once the ROIs were dentfed (see Fg. 1C), tme-courses of BOLD sgnal ntenstes were extracted from each voxel n each ROI and shfted by 6 s to account for the hemodynamc delay usng the Matlab software. A lnear trend was removed from the tme-course, and the tme-course was z-score normalzed for each voxel n each run to mnmze baselne dfferences across runs. The data samples for computng the MVPA were created by averagng the BOLD sgnal ntenstes of each voxel for 3 volumes, correspondng to the 6 s from stmulus onset to response onset. MVP Analyses Algorthm. We used sparse logstc regresson (SLR) (Yamashta et al., 2008), whch automatcally selects the relevant voxels n the ROIs for MVPA, to construct ndvdual bnary classfers based on the man behavoral varable of nterest: confdence (hgh vs. low, correct trals only to ncrease the S/N rato). Although confdence was rated wth a 4- pont scale, we used SLR to classfy confdence, as opposed to sparse lnear regresson (SLR), because the am was to nvestgate the bdrectonalty of a neurofeedback manpulaton wthn sngle partcpants. As such, t was desrable to have a bnary classfer, leadng to two expermental tranng sessons, such as nducton of Hgh- and Low-Confdence. Datasets and cross-valdaton. For each partcpant's MVPA we performed a k-fold cross-valdaton, where the entre data set s repeatedly subdvded nto a tranng set and a test set. The two data sets can be seen as ndependent snce they were used to ft the parameters of a model (decoder) and evaluate the predctve power of the traned (ftted) model, respectvely. For each partcpant, the number of folds was automatcally adjusted between k=9 and k=11 n order to approxmately equate the number of samples between the data sets. Thus, the number of folds n each cross-valdaton procedure was ~10, a typcal value for k-fold cross-valdaton procedures (Tong and Pratte, 2012). Furthermore, the classfcaton by SLR was optmzed wth an teratve approach (-SLR of Hrose et al., 2015). That s, n each fold of the cross-valdaton, the process was repeated 10 tmes. On each teraton, the selected features were removed from the pattern vectors, and only the features wth unassgned weghts were used for the next teraton. At the end of the k-fold cross-valdaton, the test accuracy was averaged for each teraton across folds, n order to evaluate the accuracy at each teraton. The optmal number of SLRs (number of teratons) was then chosen and used for the fnal computaton of the decoder used n the neurofeedback tranng procedure (Supplementary Note 1). Because confdence was rated on a 4-pont scale, we assgned the ntermedate ratngs (2, 3) to the lowand hgh-confdence classes, respectvely, n order to collapse the 4 ntal confdence levels to 2, and equate the number of trals n each class. For each partcpant, frst we merged one ntermedate level wth the hgh- (level 4) or low-confdence (level 1) class dependng on the total number of trals. Then, to equate the number of trals, we randomly sampled trals from the left-out ntermedate confdence level to the class now havng a lower total number of trals. Ths rebalancng was based on the confdence ratng response dstrbuton, and on the fnal number of trals, and was repeated n tmes (n=10, due to the low number of resampled trals). In order to drectly compare the nformaton contaned n multvoxel patterns pertanng to the confdence dmenson across the four ROIs, the best sample set was voted by k-fold cross-valdaton mean accuracy, gven the a pror assumpton that these areas are crtcal for generatng confdence. Therefore, once a sample set was selected, the cross-valdaton mean accuracy for that partcular set was used for each ROI, thus ensurng that exactly the same nformaton was used. Importantly, the calculaton of weghts used for DecNef was done by usng the entre data set of correct trals (wthout cross-valdaton) and the optmal number of teratons (as descrbed above). To evaluate the statstcal sgnfcance of decodng accuraces we used permutaton testng. For each subject and each ROI, we performed n=1000 random permutatons, where the labels of low vs. hgh confdence n the prevously selected sample set were each tme randomly shuffled. For each permutaton the accuracy was averaged across CV runs and SLR teratons, and we thus obtaned a dstrbuton of n=1000 accuraces. Statstcal sgnfcance was computed as P = sum ( Dperm > Dobs+1)/( nperm +1), or smply the sum of the permuted accuraces greater than the observed accuracy dvded by the number of permutatons. ROIs and voxels for classfcaton. We constructed four decoders n frontoparetal areas, correspondng to the followng blateral anatomcal ROIs: nferor paretal lobule (IPL), and three subregons generally regarded as beng part of the dorsolateral prefrontal cortex (dlpfc), namely the nferor frontal sulcus (IFS), mddle frontal sulcus (MFS), and the mddle frontal gyrus (MFG) (Fg. 1C). These areas have been prevously lnked to confdence judgements n perceptual decsons (Kan and Shadlen, 2009; Flemng et al., 2010; Rouns et al., 2010; Smons et al., 2010; Rahnev et al., 2016). Addtonally, we also examned four ROIs n the vsual processng stream (V12, V3A, hmt and the fusform gyrus), because we were evaluatng perceptual confdence wth vsual stmul n a dfferent study. As reported n Cortese et al. (2016), confdence classfcaton was at chance for these ROIs n the vsual areas. Thus, for the DecNef tranng, we selected only the four frontoparetal ROIs that showed hgher decodng accuracy, 327

6 and the decodng results of these four ROIs are presented n the Results secton. Each bnary decoder was traned to classfy a pattern of BOLD sgnals nto ether low or hgh confdence, usng data samples obtaned from up to 120 trals collected n up to twelve fmri runs. As a result, the nputs to the decoders were the partcpants moment-to-moment bran actvatons, whle the outputs of the decoders represented the calculated lkelhood of each confdence measure. The mean ( ± s.e.m) number of voxels for decodng was 1802 ± 63 for IPL, 490 ± 19 for IFS, 412 ± 6 for MFS, and 1350 ± 42 for MFG. The mean ( ± s.e.m) number of voxels selected by -SLR for each ROI was 62 ± 9 for IPL, 66 ± 10 for IFS, 64 ± 8 for MFS, and 91 ± 13 for MFG. The selected voxels were then used for the DecNef tranng. We opted for four separate classfers nstead of one large ROI to account for possble ndvdual dfferences n confdence representaton loc at the regonal level (Rahnev et al., 2016). DecNef tranng and testng Once ndvdual confdence classfers were constructed, ten selected partcpants completed a two-day DecNef tranng n each sesson (amng to up- or down-regulate confdence level), and each sesson was separated by at least one week (see Fg. 1A). Each partcpant went through both Up- and Down DecNef tranng, and the order was counterbalanced across partcpants (.e., Up then Down vs. Down then Up). The neurofeedback task tself s llustrated n Fg. 1D: partcpants were asked to manpulate, modulate or change ther bran actvty n order to make the feedback dsc presented at the end of each tral as large as possble. The expermenters provded no further nstructons nor strateges. Importantly, the dsc for feedback after the nducton perod was maxmzed both for nducng the actvaton patterns for hgh confdence and for low confdence, n the up- and down- regulaton sessons, respectvely. Partcpants receved monetary reward proportonal to ther nducton success (ablty to nduce the selected actvaton pattern). Only the expermenter knew whch sesson took place on a certan day, whle partcpants were unaware. After each tranng day, partcpants were asked to descrbe ther strateges n makng the dsc sze larger. Answers vared from I was countng to I was focusng on the dsc tself to I was thnkng about food. When partcpants were asked to report whch group they thought they were assgned to at the end of the experments (n=5, 2 months later, and n=4, 5 months later 1 partcpant could not be reached), ther answers were at chance at the group level (57% correct, Ch-square test, χ 2 =0.225, P=4). On each day of a gven DecNef sesson, partcpants were traned n up to 11 fmri runs. The mean ( ± s.e.m) number of runs per day was 10 ± 0.1 across days and partcpants. Each fmri run conssted of 16 trals (1 tral=20 s) preceded by a 30-s fxaton perod (1 run=350 s). The fmri data for the ntal 10 s were dscarded to avod unsaturated T1 effects. Each tral started wth a vsual cue (three concentrc dsks, whte, gray and green) sgnalng the nducton perod (Fg. 1D). The nducton perod lasted for 6 s, and was followed by a 6 s rest perod. The rest perod was followed by the neurofeedback dsk (a whte rng) presented on the gray screen for up to 2 s. Fnally, a tral ended wth a 6 s ntertral nterval (ITI). Ether durng the nducton perod, or at the begnnng of the rest perod (pseudo-random onsets: 2, 4, 6, or 8 s from tral start) a 2 s nose RDM was also presented. Pseudo-random onsets were desgned n order to mnmze potental nterference of the RDM onset on the nducton of bran actvty. Durng the rest perod, partcpants were asked to smply fxate on the central pont and rest. Ths perod was nserted between the nducton and the feedback perods to account for the hemodynamc delay, assumed to last 6 s. The sze of the dsc represented how much the BOLD sgnal patterns obtaned from the nducton perod corresponded to actvaton patterns of the target confdence level (hgh or low). The whte dsc was always enclosed n a larger whte concentrc crcle (5 radus), whch ndcated the dsc's maxmum possble sze. The sze of the dsc presented durng the feedback perod was computed at the end of the rest perod accordng to the followng steps. Frst, measured functonal mages durng the nducton perod underwent 3D moton correcton usng Turbo BranVoyager (Bran Innovaton). The subsequent steps were performed usng the Matlab software. Second, tme-courses of BOLD sgnal ntenstes were extracted from each of the voxels dentfed n the MVPA sesson, for each of the four frontoparetal ROIs used (IPL, IFS, MFS, and MFG), and were shfted by 6 s to account for the hemodynamc delay. Thrd, a lnear trend was removed from the tme-course, and the BOLD sgnal tme-course was z-score normalzed for each voxel usng BOLD sgnal ntenstes measured for 20 s startng from 10 s after the onset of each fmri run. Fourth, the data sample to calculate the sze of the dsc was created by averagng the BOLD sgnal ntenstes of each voxel for 6 s n the nducton perod. Fnally, the lkelhood of each confdence state was calculated from the data sample usng the confdence decoder computed n the MVPA sesson. The sze of the dsc was proportonal to the averaged lkelhood from the four dfferent frontoparetal ROIs (rangng from 0 to 100%) of the target confdence assgned to each partcpant on a gven DecNef block. Importantly, partcpants were unaware of the relatonshp between ther actvaton patterns nducton and the sze of the dsk tself. The target confdence was the same throughout a DecNef block. In addton to a fxed compensaton for partcpaton n the experment, a bonus of up to 3000 JPY was pad to the partcpants based on the mean sze of the dsc on each day. Mathematcal modelng and model comparson Nonlnear global model We constructed a mathematcal model to objectvely examne the effects of Up and Down DecNef, a decay of learnng (of DecNef effect) due to one-weak lapse, and an anterograde nterference of learnng from the frst to second week. The model was ft to 4 confdence measurements at the 4 tme ponts for each partcpant and possesses 4 model parameters. X j are the expermentally measured confdence values, whle Xˆj are the estmated confdence values for each partcpant (wth =1:10) (.e., confdence outcome of DecNef effects), at each tme pont (wth j = 1: 4; we derved n = 30 : 3 ponts of measurement at n=10 subjects, as the frst tme pont was assumed to be fxed for the global model). The man 4-parameter nonlnear model to descrbe DecNef effects s formally outlned as follows. Xˆ 1 = B (1) for = 1 : 5, group D U Xˆ 2 = B + εδ down (2) Xˆ 3 = B + εδα Up (3) Xˆ 4 = B + εδα + γδ Up (4) for = 6 : 10, group U D Xˆ 2 = B + Δ Up (5) Xˆ 3 = B + Δα Down (6) Xˆ 4 = B + Δα + εγδ Down (7) Error = ΣΣ j( X Xˆ ) = F ( α, ε,ɣ, Δ) j 2 j Where B s the ntal baselne on the frst day of the DecNef procedure n the frst week (0 - after realgnment to a common level for each group). Δ, the confdence change by Up-DecNef n the frst week whch putatvely s the only true result. A Δ value of 0 would ndcate no (8) 328

7 Table 2 AICc analyss. The most negatve AICc value ndcates the best model ft. Δ AICc <2 ndcate that the current model has a hgh lkelhood of beng the best model under dfferent crcumstances,.e., a new dataset. Model (est. parms) Fxed parms AICc Δ w Const. Confdence mean ( X1 ) k Const. Confdence [mean k ( X2, X3, X4 )] Wthn-week const. confdence k 1 k Polynomal 1st deg. (α 1, α 2 ) k Polynomal 2nd deg. (α 1, β 1, α 2, k 1 k β 2 ) Sub-model (Δ) α=1 ε=0 ɣ= Sub-model (Δ, ε) α=1 ɣ= Sub-model (Δ, ε, α) ɣ= Sub-model (Δ, ε, ɣ) α= Sub-model (Δ, ɣ, α) ε= Sub-model (Δ, ε, ɣ) α= Sub-model (Δ, ε, ɣ, α) Sub-model (Δ, ε, α) ɣ= DecNef effects on confdence, whle a Δ value of 1 would ndcate a 100% confdence ncrease. ɛ, the rato of Down-DecNef effect normalzed by Up-DecNef effect, explaned as the fact that Down-DecNef effect s of reduced magntude compared to Up-DecNef n both nstances - week 1 for group D-U and week 2 for group U-D. A ɛ value of 0 would ndcate no down effects, whle a ɛ value of 1 would ndcate maxmum effects (dentcal magntude to Up-DecNef effects). ɣ, the anterograde learnng nterference resultng n a reduced second week DecNef effect, namely the fact that n the second week there s only lmted DecNef effect. A ɣ value of 0 would ndcate that there was full anterograde learnng nterference thus there exsted no learnng effect n the second week, whle a ɣ value of 1 would ndcate absence of anterograde learnng nterference. Last, ɑ, the learnng persstence (1 - [decay of learnng]) across the one-week nterval, the remarkable result that the confdence level attaned at the end of the frst week s preserved at least untl the begnnng of the next sesson. An ɑ value of 0 would ndcate no learnng persstence, whle an ɑ value of 1 would ndcate perfect memory mantenance,.e. no memory loss. Submodels and alternatve smpler models Submodels are defned by settng dfferent parameters to zero or one, one at a tme or concomtantly, followng the ratonale of prortzng aganst model complexty. Ths gves rse to a herarchcal group of models, from smpler to most complex (capturng sngle or ncreasngly more aspects of DecNef effects on the confdence measure). The frst model s the smplest and only estmates Δ, wth the other parameters settng as ɛ=0, ɣ=1, ɑ=1. The second model, by complexty order, assumes Up and Down-DecNef effects, estmates Δ and ɛ, whle ɣ=1, ɑ=1. The thrd model estmates Δ, ɛ and ɣ, wth ɑ=1. Further DecNef-based models estmate Δ, ɛ and ɑ, wth ɣ=1; or estmate Δ, ɛ and ɑ, wth ɣ=0; or estmate Δ, ɛ and ɣ, wth ɑ=0; or fnally, estmate Δ, ɣ, and ɑ, wth ɛ=0. We consdered alternatve models that do not take nto account DecNef drecton assumptons or a pror conceptons. These are a 1- parameter constant confdence model (the confdence measure does not change, s constant throughout the experment), wth two versons: k = X 1, or k = mean ( X2, X3, X 4). Other free models are a wthn-week constant confdence, and wthn-week ft by frst and second degree polynomal. Model comparson For model comparson, we used the second-order or corrected Akake Informaton Crteron (AICc) (Burnham and Anderson, 2002), whch s a corrected verson of the Akake Informaton Crteron (AIC) (Akake, 1974). Raw AIC s computed accordng to the followng equaton: AIC = nlog (ˆ σ 2 ) + 2k (9) Resdual Sum of Squares where σˆ 2 =, n s the sample sze and k the number of n parameters n the model. In our set of global models, n =30, and k vared from 1 to 4. In the modelng reported for small sample szes (.e., / k 40), the second-order or corrected Akake Informaton Crteron (AICc) should be used nstead. Although the AICc formula assumes a fxed-effects lnear model wth normal errors and constant resdual varances, whle our model s nonlnear, the standard AICc formulaton s recommended unless a more exact small-sample correcton to AIC s known (Burnham and Anderson, 2002): 2 ( k k+1) AICc = AIC + ( n k 1) (10) For model comparson, two useful metrcs are Δ AICc and Akake weghts (w ). Δ AICc s a measure of the dstance of each model relatve to the best model (the model wth the most negatve, or lowest, AIC value), and s calculated as: AICc Δ = AICc mn ( AICc) (11) As ndcated n Burnham and Anderson (2002), Δ AICc <2 suggests substantal evdence for the th model, whle Δ AICc >10 ndcates that the model s very unlkely (mplausble). Akake weghts (w ) provde a second measure of the strength of evdence for each model, s drectly related to Δ AICc, and s computed as: exp ( ΔAICc /2) w = R exp ( ΔAICc /2) =1 (12) AICc, Δ AICc, and w are reported n Table 2. Model averagng and cross-valdaton In cases such as ours, where a hgh degree of model selecton uncertanty exsts (the best AIC model s not strongly weghted), a formal soluton s to compute parameter estmates through modelaveragng. For ths approach, two procedures may be used, dependng on the results. The frst approach makes use of only a lmted subset of models that are closest to the current best model (Δ AICc < 2), whle the second approach wll consder all models (n fact, ths accounts to consder all models wth w 0). We adopted the frst approach, selectng only models wth hgh lkelhood to keep the parameter estmates wthn the same scale as the orgnal sngle models. Parameters are estmated accordng to the equaton: R w β βˆ ˆ = =1 R w =1 (13) where βˆ s the estmate for the predctor n a gven model, and w s the Akake weght of that model. Uncondtonal error, necessary to compute the uncondtonal confdence nterval for a model-averaged estmate, can be calculated accordng to the followng equaton: R =1 se ˆ ( β ˆ )= w var ˆ ( β ˆ) + (ˆ β β ˆ ) 2 (14) where var ˆ ( β ˆ) s the varance of the parameter estmate n model, and βˆ and βˆ are as defned above. The confdence nterval s then smply gven by the end ponts: βˆ± z se ˆ ( βˆ) 1 α/2 (15) For a 95% confdence nterval (CI), z1 α/2=1.96. Note that the uncondtonal varance comprses two terms, the frst one local (nternal varance of model ), whle the second one global, n that t represents the varance between the common estmated parameter and the true value n model. In order to assess the nternal varance of the models, as well as the 329

8 generalzablty and to prevent overfttng, we run a leave-two-out (one for each group, D-U and U-D) cross-valdaton (CV) model averagng procedure. The procedure was repeated for each CV test-par: wth 10 partcpants, dvded nto 2 groups, we thus obtaned 25 CV runs (all possble combnatons of test pars). For each CV run, the left-out partcpant data acted as test set, whle the tranng data set was hence composed of the remanng partcpants (four n each group). In each CV run, the three best models prevously selected by AICc (ndcated n bold n Table 2) were ftted on the tranng data, Akake weghts were calculated, and fnally model averagng was performed. After modelaveragng, as a measure of the goodness of ft, the predcted confdences were correlated wth the observed confdences from the current CV test run. In the man text we report the mean ± std value of correlaton values. Furthermore, we also run a correlaton of estmated vs. observed confdence changes by poolng all test sets together, to assess sgnfcance at the group level. To do so, for each partcpant we averaged the estmated values from the CV runs (5) n whch that specfc partcpant acted as test data. Lastly, the fnal parameter estmates were computed as follows. Frst, for each parameter we took the average across the 25 CV runs. As mentoned above, the uncondtonal error s composed of two terms. The varance of the parameter estmate n model was taken as the varance n the 25 CV runs model estmaton processes. The global varance s computed between the mean fnal estmate and the mean estmate n each sngle model. Gven the two sources of varance we then evaluated the 95% CI. See Supplementary Fg. 1 for a graphcal account of the entre process. Nonlnear mxed (global - local) model In the second part of the modelng approach, we consder all data ponts, and model populaton's ndvdual fts wth nonlnear equatons wth global and local parameters (we thus ft the model to all n = 40 : 4 ponts of measurement at n=10 subjects, as all tme ponts were consdered for the mxture model). The equatons determnng the model are thus very smlar to those gven above: Xˆ 1 = B (16) for =1: 5, group D-U Xˆ 2 = B + εδ Down (17) Xˆ 3 = B + εδ α Up (18) Xˆ 4 = B + εδ α+ γδ Up (19) for = 6: 10, group U-D Xˆ 2 = B + Δ Up (20) Xˆ 3 = B + Δ α Down (21) Xˆ 4 = B + Δ α+ εγδ Down (22) Error = ΣΣ j( X Xˆ ) = F ( α, ε,ɣ, Δ, B ) j 2 j (23) Compared wth the global-parameter model (Eqs. (1) (8)), n ths ndvdualzed model B (the ntal startng pont) s now optmzed ndvdually, as well as Δ, the confdence change nduced by DecNef. For ths model, n=40 (data ponts), and k=23 (parameters). To avod overfttng and evaluate the nternal varance of the model, ths fnal model was estmated wth a leave-two-out CV, wth a smlar approach as for the global model. For each CV run, the model parameters were estmated wth the tranng set, and then evaluated on the left-out test set by correlatng observed confdences (two partcpants data n the test data) and predcted confdences. The fnal parameter estmates are reported as the cross-valdated mean ± 95% CI. Results at the group level are reported by correlatng all observed confdences wth ndvdual predcted confdences. Importantly, because each partcpant on a gven group was tested n 5 dfferent CV runs (due to t beng pared once wth all other partcpants n the opposng group), ts fnal estmated change was computed as the mean of the 5 CV runs. See Supplementary Fg. 2 for a graphcal account of the process. Analyss, statstcs and model-solvng routnes All analyses were performed wth Matlab (Mathworks) versons 2011b and 2014a wth custom made scrpts. Addtonal statstcal analyss such as ANOVA were performed wth SPSS 22 (IBM statstcs). We employed Matlab optmzaton routnes to solve the systems of nonlnear equatons wth a nonlnear programmng solver, under leastsquare mnmzaton. The Matlab solver was fmncon, wth the followng optmzaton optons. A sequental quadratc problem (SQP) method was used; specfcally, the SQP algorthm. Ths algorthm s a medum-scale method, whch nternally creates full matrces and uses dense lnear algebra, thus allowng addtonal constrant types and better performance for the nonlnear problems outlned n the prevous secton. As compared wth the default fmncon nteror-pont algorthm, the SQP algorthm also has the advantage of takng every teratve step n the regon constraned by bounds, whch are not strct (a step can exst exactly on a boundary). Furthermore, the SQP algorthm can attempt to take steps that fal, n whch case t wll take a smaller step n the next teraton, allowng greater flexblty. We set bounded constrants to allow only certan values n the parameter space to be taken by the estmates, reflectng the bologcal dmenson they were explanng. As such, boundares were set as: Δ [0 1], ɛ [ 1 0], ɣ [0 1], and ɑ [0 1], and for the mxed model B [1 4]. The functon tolerance was set at 10 20, the maxmum number of teratons at 10 6 and the maxmum number of functon evaluatons at Statstcal results nvolvng multple comparsons are reported n both the corrected and uncorrected forms. The ratonale behnd ths decson s that these comparsons can be nterpreted as multple comparsons of one hypothess across dfferent medums, or smply as dfferent hypothess, n whch case no multple comparsons should be consdered. For multple comparsons, we used the Holm-Bonferron procedure, where the P-values of nterest are ranked from the smallest to the largest, and the sgnfcance level α s sequentally adjusted based α on the formula for the th smallest P-values. MRI parameters ( n + 1) The partcpants were scanned n a 3T MR scanner (Semens, Tro) wth a head col n the ATR Bran Actvaton Imagng Center. Functonal MR mages for retnotopy, the MVPA sesson, and DecNef stages were acqured usng gradent EPI sequences for measurement of BOLD sgnals. In all fmri experments, 33 contguous slces (TR=2 s, TE=26 ms, flp angle=80, voxel sze= mm 3, 0 mm slce gap) orented parallel to the AC-PC plane were acqured, coverng the entre bran. For an nflated format of the cortex used for retnotopc mappng and an automated parcellaton method (Freesurfer), T1-weghted MR mages (MP-RAGE; 256 slces, TR=2 s, TE=26 ms, flp angle=80, voxel sze=1 1 1 mm 3, 0 mm slce gap) were also acqured durng the fmri scans for the MVPA. Results MVPA for confdence was performed wth the data from all ntal 17 subjects, as reported n Table 1. Sgnfcance was assessed through permutaton testng (randomly shufflng the labels of hgh and low confdence). Results were consstent wth our a pror hypothess and prevous work that confdence representatons are found n frontoparetal cortces (Kan and Shadlen, 2009; Flemng et al., 2010; Rouns et al., 2010; Smons et al., 2010; Rahnev et al., 2016). Furthermore, as reported here, ndvdual dfferences n confdence decodablty were 330

9 a Down-DecNef, All ROIs (mean nducton) Day 1 Day 2 b Day 1 Down-DecNef, best ROI Day 2 Success rate Success rate c Day 1 Up-DecNef, All ROIs (mean nducton) Day 2 d Day 1 Up-DecNef, best ROI Day 2 Success rate Success rate Fg. 2. Indvdual partcpant's success rates for each DecNef sesson (total, 4). (a-b) Success rate for Down-DecNef, wth (a) representng the overall nducton performance (all 4 ROIs averaged), as n the expermental sesson, whle (b) s selectve for the best ROI (sngle ROI wth hghest nducton lkelhood on a gven tral). (c d) Success rate for Up-DecNef, descrpton same as above for Down-DecNef. Indvdual colors represent sngle partcpants. Dots represent the rato of successful trals/total trals n each run. also apparent. Indeed, for many partcpants one ROI (dfferent across partcpants) had much lower decodng ablty. Ths outcome llustrates our ratonale for usng 4 separate ROIs for the DecNef tranng, nstead of selectng a sngle best one (whch would have been dfferent across partcpants) or a sngle large unfed one. DecNef can be essentally assumed as a neural operant condtonng and/or renforcement learnng paradgm, wth whch a mult-voxel pattern correspondng to a specfc pece of bran nformaton can be nduced wthout explct knowledge from the partcpants. Throughout the manuscrpt we refer to Up-DecNef for Hgh-Confdence DecNef and Down-DecNef for Low-Confdence DecNef. Snce there were two groups (for DecNef order counterbalancng), we wll often refer to these as D-U (frst sesson s Down-DecNef whle the second s Up- DecNef) and U-D (the reverse, Up-DecNef frst, Down-DecNef then) throughout the results secton. The ROIs selected for ths study, asde from beng mportant for metacognton, are also part of large-scale functonal bran networks assocated, for example, wth attenton and especally top-down modulaton of vsual attenton (Fox et al., 2005; Bressler and Menon, 2010; Rosenberg et al., 2016). vous work showed that rtms to the DLPFC resulted n lower levels of vsblty for stmul whch was pronounced for correct trals (Rouns et al., 2010). We have addressed these crucal concerns n a companon manuscrpt, whch amed at elucdatng the relatonshp between confdence and perceptual evdence (Cortese et al., 2016). We reported that DecNef effects on confdence were specfc and unlkely to result from crteron shfts, mood changes or an attentonal modulaton (Cortese et al., 2016), thus supportng the vew that confdence alone was manpulated. Indeed, task accuracy dd not change between - and -Tests: two-way repeated measures ANOVA, factors of neurofeedback (Down - Up) and tme ( - ), showed nonsgnfcant nteracton (F 1,9 =0.030, P=0.867) as well as non-sgnfcant man effects of tme (F 1,9 =0, P=94) and neurofeedback (F 1,9 =1.854, P=0.206). Furthermore, smlarly as n Scharnowsk et al. (2015), any unspecfc effects that are related to task demands should only allow to ether ncrease or decrease the confdence level, but t would be dffcult to allow bdrectonal control (Scharnowsk et al., 2015). The current paper ams at explorng and explanng the dynamcs of DecNef neurofeedback tranng on confdence, therefore we wll focus on these dynamcal aspects. Before explorng the DecNef effects on confdence, t s also mportant to analyze the ablty of partcpants to nduce the target actvaton patterns n the selected bran regons. The overall nducton performance, measured as the rato of successful trals (trals wth nducton lkelhood >.5 over total trals), for all ROIs averaged dd not show a marked learnng curve over runs or days of tranng (Fg. 2A, C). Durng the DecNef tranng tself, the feedback sgnal was also gven as the average across all four ROIs. Gven the complcated nature of such nducton, t s perhaps unsurprsng that, overall, there was a mxed success rate n the nducton lkelhood at the group level (although ndvdual varablty seemed to be szeable). Nevertheless, when lookng at the best performng ROI (that s, the sngle best performng ROI n each tral, that also had nducton lkelhood >.5), all subjects showed very hgh success rates (Fg. 2B, D). Ths n turn ndcates that on any gven tral, at least one ROI was successfully nducng the target pattern. Although the monetary sgnal fed back to partcpants was based on the average of the four ROIs, because the underlyng pattern actvaton was very successful n at least one of the ROIs, probably ths had a much larger effect n the learnng and therefore on the fnal measurable confdence changes. Behavoral data from the - and -Tests show that confdence was dfferentally manpulated by DecNef (Fg. 3). Importantly, the resultng changes n confdence could not be attrbuted to a smple week order effect (two-way ANOVA wth repeated measures, nonsgnfcant effects of neurofeedback, F 1,9 =70, P=0.558, and tme, F 1,9 =2.834, P=0.127, and non-sgnfcant nteracton, F 1,9 =0.844, P=82, Fg. 4A bottom part - week average). As dsplayed n Fg. 3A, the confdence change was larger for Up- DecNef than Down-DecNef, but mportantly, the level attaned at the end of the frst week was almost entrely preserved untl the begnnng of the second week, n the second sesson. Lastly, the second week effect seemed present but reduced as compared to the frst week effect. Thus, order of DecNef (Up then Down, or Down then Up) had a large nfluence on how confdence was manpulated. A mxed-effects ANOVA wth repeated measures, wth wthn-subjects factor tme, and betweensubjects factors neurofeedback and order, clarfed ths fndng, as t resulted n a sgnfcant nteracton between the three factors (F 1,16 =4.769, P=0.044). Furthermore, the factor tme (F 1,16 =4.623, P=0.047) and the nteracton between tme and neurofeedback (F 1,16 =18.050, P=0.001) both had sgnfcant effect on the dependent 331