N-back Tranng Task Performance: Analyss and Model J. Isaah Harbson (jharb@umd.edu) Center for Advanced Study of Language and Department of Psychology, Unversty of Maryland 7005 52 nd Avenue, College Park, MD 27642 USA Sharona M. Atkns (satkns@psyc.umd.edu) Neuroscence & Cogntve Scence Program Department of Psychology, Unversty of Maryland Bology/Psychology Buldng, College Park, MD 27642 USA Mchael R. Dougherty (mdougherty@psyc.umd.edu) Department of Psychology and Center for Advanced Study of Language, Unversty of Maryland Bology/Psychology Buldng, College Park, MD 27642 USA Abstract Despte the n-back task s apparent effectveness as a workng memory (WM) tranng task, ts status as a WM assessment s questonable. We analyzed the accuracy and reacton tme data of partcpants performng of an adaptve n-back tranng task and developed a computatonal model to descrbe ths performance. Applcaton of our model to n-back tranng data suggests that performance s consstent wth a two-stage, famlarty and recollecton account. Furthermore, our results suggest that nterference resoluton s an mportant determnng factor for task accuracy, especally when respondng to targets. Keywords: workng memory; executve functonng; workng memory tranng; n-back; contnuous performance task; computatonal model. N-back and Workng Memory The n-back task has often been used as a workng memory (WM) assessment (Owen et al., 2005) and has recently become popular as a WM tranng task (Jaegg et al., 2008). Performance gans on n-back tranng transfer to tasks that are heavly relant on WM. Nevertheless, pror work questons the valdty of n-back as a measure of WM ablty (Jaegg et al. 2010; Kane et al., 2007) and n-back performance gans do not appear to transfer to complex WM span tasks (Jaegg et al., 2008; L et al., 2008). Understandng how n-back s performed s mportant both for the purpose of evaluatng the ts valdty as a measure of WM and for solatng the mechansms that mprove over the course of WM tranng. The present study provdes an analyss of performance on an adaptve n-back tranng task and a model of n-back performance. The N-back Task In the n-back task, partcpants are presented wth a sequence of stmul (e.g., letters) one at a tme and asked to compare the current stmulus to one presented n tems pror n the sequence. When performng 2-back, the current stmulus s a target when t matches the stmulus presented two stmul ago. So n the letter sequence P-F-D-C, the partcpant should respond match f the 5 th letter n the sequence were a D because t would match the one occurrng two pror, but respond no match otherwse. The nter-relatonshps wthn a sequence of stmul appear to be an mportant factor n determnng how the task s performed. In partcular, stmul (.e., lures) that match n locatons n+1 or n-1 can change how the n-back task s performed (Kane et al, 2007). For example, f the 5 th letter n the aforementoned sequence were an F, t would be consdered a lure because t occurred n+1 stmul ago, and the correct response s non match. Lures are more dffcult to reject than other non-lure/non-targets stmul; partcpants are less accurate and take longer to respond to lures than to other non-targets (Gray, Chabrs, & Braver, 2003; Kane et al, 2007; McCabe & Hartman, 2008; Oberauer, 2005). Arguably, the presence of lures changes how partcpants perform the n-back task (Kane et al., 2007). Wthout lures, t would be possble to use famlarty alone as the bass for a correct response. Any stmulus re-occurrng somewhat recently would be a target. However, when lures are ncluded n the sequence recent re-occurrence s not enough to dstngush targets from non-targets. Instead, t s necessary to recollect ether what stmulus occurred n tems back or have a fne-graned estmate of when a famlar stmulus last appeared. Gven the suggested mportance of lures, the current analyss focuses on comparng partcpant performance on targets, lures, and other non-targets. Experment: Tranng Data Ffty-sx partcpants completed ten sessons of an adaptve, n-back tranng task as part of a larger workng memory tranng battery. Ths battery ncluded a tranng verson of runnng-span, letter-number sequencng, and block span (Atkns et al., 2009) tasks as well as four tasks provded by Post Scence nc. (Bran Ftness Program, Verson 2.1; Insght, Verson 1.1). For the present purposes, we wll only note that many partcpants mproved ther performance on the tranng tasks, and specfcally on the n-back tranng task. Furthermore, performance gans on the n-back tranng task correlated wth gans n several remote tasks, ncludng sentence ambguty resoluton (Novck et al., submtted). 120
N-back Tranng Task Desgn Smlar to other tranng versons of n-back, our verson adapted n dffculty based on partcpant performance. Two factors were manpulated to change the task dffculty. The frst was the lure level. There were three levels of lures. The easest level (level 0) conssted of no lures. At the next dffculty level (level 1) lures appeared n poston n+1. In the most dffcult lure level (level 2) lures appeared both n poston n+1 and n-1. In addton to adaptng lure level to partcpant performance, we also adapted dffculty by changng the value of n. N could range from 1 to 8. Partcpants were presented 25-tem sequences. In each sequence there were 5 targets, 0 or 5 lures and the rest were other non-targets (.e., letters that had last occurred more than 10 letters pror). Partcpant performance on each sequence was used to determne whether and how the task dffculty should adapt on the subsequent sequence of 25. When partcpants were correct at least 85% of the tme the task got more dffcult; when they were correct less than or equal to 65% of the tme, the task got easer. Otherwse, the task remaned at the same dffculty level. The dffculty level changed by frst changng the lure level. If the dffculty needed to be ncreased and the lure level was less than 2, the lure level would ncrease. Once at the maxmal lure level, n would ncrease and the lure level would be reset at zero. Smlarly when the task needed to be made easer and the lure level was greater than 0, the lure level would be decreased by one level. If the lure level was already 0, then n would be decreased by one and the lure level would be reset to two. All partcpants started at 2- back wth no lures (.e., lure level of zero). Fgure 1: Mean Dffculty level reached by partcpants by tranng sesson. General Fndngs On average, partcpants showed marked mprovement over the course of tranng. Fgure 1 shows the mean dffculty level reached by partcpants across tranng sessons, where dffculty level s defned as the value of n reached plus 1/3 of the lure level or LureLevel D = n +. Eq.1 3 Dffculty level can be taken as an ndcator of overall performance, but t does not shed lght on what cogntve processes were used to complete the task.. For that purpose we turn to accuracy and reacton tmes on the target, lures, and other non-targets ndvdually. Accuracy Fgure 2 shows the percent correct when the target, lure, and other non-target trals were shown n the thrd through 25 th seral postons. Partcpants demonstrated pronounced and consstent prmacy on target trals across seral postons. Lttle or no prmacy was found for lures and other nontarget trals. Fgure 2: Mean Accuracy for Targets, Lures and Other non-targets across seral poston n the stmulus sequence. When accuracy s examned separately for each level of n, the same basc relatonshp s found. There s an ntal drop n target performance down to an asymptote; the lowest level of the asymptote s negatvely correlated wth n. The top panel of Fgure 3 shows representatve results from the 4-back task. Reacton Tmes Partcpants responded correctly to both lures and targets sgnfcantly more slowly than to other non-targets. As shown n Fgure 4, the mean correct reacton tme (RT) to targets and lures were both approxmately 380 ms (380.5 and 379.8 respectvely). The RT to other non-targets was 343.4, sgnfcantly qucker than both other trals types as determned by wthn partcpant t-tests (p s < 0.001, note that other sgnfcance values are also from wthn partcpant t-tests). Ths same pattern s found when analyses are performed separately for each level of n. The target and lure RTs dd not dffer sgnfcantly for any value of n. In contrast, for all n values except 8 other non-targets were responded to more quckly than lures and for all n values except 2 other non-targets were responded to more quckly than targets (p s < 0.05). A dfferent pattern was found for ncorrect response RTs. Partcpants were sgnfcantly faster at respondng ncorrectly to targets than to lures (p < 0.05) and other nontargets (p < 0.01). When examned at each level of n, the 121
results are largely consstent. For n s of three through eght, ncorrect target responses were qucker than ncorrect lure and ncorrect other non-target responses. However, lkely due to the small number of ncorrect lure and other nontarget responses, these dfferences were only sgnfcant four tmes. Comparng correct to ncorrect response RTs, no sgnfcant dfference was found for targets. However, correct responses were sgnfcantly qucker than ncorrect responses for both lures (p < 0.01) and other non-targets (p < 0.001). Summary of Results The RT results are consstent wth prevous research. Lures were expected to take longer to reject than other non-targets. Smlarly, responses to lures were expected to be less accurate than responses to other non-targets. However, the prmacy found n targets trals was surprsng. The number of tems that t s necessary to track, namely n, s constant across the entre sequence. Despte ths, the accuracy for early targets n the sequence s greater than for later targets. Follow-up analyses ndcated that the obtaned prmacy was not due to a decrease n the probablty of respondng match due to the number of pror match responses. The probablty of respondng match to a target dd not vary wthn a sequence, and remaned constant at about 58%. One explanaton for the observed prmacy s that partcpants were less than perfect at removng stmul from consderaton that were not longer relevant. Irrelevant stmul, stmul that occurred greater than n postons pror, may have been mantaned n addton to and potentally at the expense of the relevant stmul. Removal of rrelevant nformaton has prevously been ndcated as mportant to performance n the n-back task (Oberauer, 2005). Fgure 3: Partcpant (Panel A) and Model (Panel B) Accuracy across seral postons for 4-back. Fgure 5: Partcpant Reacton tme data (Panel A) and Model predctons for 4-back. Fgure 4: Mean Reacton Tme for Targets, Lures and Other non-targets for Correct and Incorrect Trals. Modelng n-back Performance A computatonal model of n-back performance was developed based on pror work descrbng n-back performance. Specfcally, the model mplemented a twostage decson process, whch ncludes a famlarty and a recollecton process. It also mplemented mperfect removal 122
of rrelevant nformaton from the set actvely mantaned n WM. Both of these assumptons were based on Oberauer s (2005) account of n-back performance. In addton, to allow the rrelevant nformaton mantaned n WM to mpact performance, we mplemented forgettng as due to nterference between tems actvely mantaned n WM (Oberauer & Lewandowsky, 2008). Model Implementaton These theoretcal assumptons were mplemented wthn an exstng model of famlarty/probablty judgment and recall/recollecton, HyGene (Thomas et al., 2008). Whle ths model has prevously only been appled to hypothess generaton and judgment, t s based on a model of recognton memory, Mnerva2 (Hntzman, 1988) and s therefore well equpped to handle famlarty judgments. It also utlzes samplng and retreval dynamcs based on successful models of recall, makng t capable of recollecton as well. To apply HyGene to the n-back task t was necessary to: (1) Elaborate on ts WM processes, (2) Add a mult-stage recognton process, and (3) Represent tme. WM Processes We assumed that whle performng the n- back task, partcpants try to mantan the last n tems n an actve subset of memory. Once the tem s more than n stmul old, the model attempts to remove that tem from the actve subset. The probablty of successfully removng the no longer relevant tem on each tme step s determned by a new parameter n the model, premove. In addton, tems n the actve subset compete wth one another. Each feature can only be mantaned by one tem n the actve subset (Oberauer & Lewandowsky, 2008), therefore the competton for features between actve tems causes nterference. Recognton Process The model completes up to three processes when respondng n the n-back task. The ntal step s determnng the famlarty of the current stmulus. If the stmulus s not suffcently famlar, then the current stmulus s judged as a non-match and no further processng steps are taken. However, f the current stmulus s suffcently famlar, an attempt to recall or recollect the n-th back tem s made. If the retreved tem matches the current stmulus, the response s match. If the retreved tem does not match the current stmulus, then the response s nonmatch. If retreval fals, that s the actvaton of the to-beretreved tems s less than a threshold tretreval, then the model guesses whether or not that stmulus s a target. The RT predctons from the present smulatons are based on the smplfyng assumpton that each process (famlarty judgment, recollecton, and guessng) takes a sngle unt of tme. Tme Contextual drft was used to represent tme. Wth each tme step the representaton of the current context was modfed wth probablty pdrft. Ths allowed the model to search for the n-th back stmulus by probng memory wth the n-th back context. However, we assumed that the n-th back stmulus s only probablstcally renstated. Specfcally, each tem of the n-th back context s renstated wth probablty prenstate. The current, modfed verson of HyGene does not use any of the standard HyGene parameters (L, A C, Act MnH, TMAX). Instead, as ndcated n the model modfcaton descrpton t ntroduces four new parameters. These parameters and ther values for the reported smulatons are shown n Table 1. Table 1. Parameters Name Sm. Value premove.15 pdrft.33 prenstate.75 tretreval.10 Model Detals There are three components used n the modfed model: the probe, the actve subset of memory, and semantc memory. Each stmulus n the actve subset of memory s represented as a trace, a combnaton of an tem (e.g., letter) and the context n whch the tem appeared. Each tem s represented as a unque, randomly generated vector of 1 s, - 1 s, and 0 s. Ones represent the presence and negatve ones represent the absence of some abstract feature. A zero ndcates that the presence or absence of a feature s unknown or lost. For each smulaton run, a new randomly generated vector s created for each of the letters used n the experment. The collecton of unque letter vectors consttutes the semantc memory of the model. Whle the ntal context vector s generated randomly, lke the tem vectors, each subsequent context was generated based on the prevous context vector and a random drft factor. Each element n a new context s the same as each element n the prevous context wth probablty (1-pDrft). Wth pdrft, that element s set to a random value (.e., -1, 0, 1). As each stmulus s processed, a vector representng that stmulus and the vector representng the current context are stored as a trace n the actve subset of memory. Once the actve subset has more than n traces, the model attempts to remove the traces of the tems that occurred more than n stmul pror from the actve subset. The probablty of removng the extra traces at each tme step s premove. The mantenance of tems n the actve subset has a cost. Specfcally, every trace competes wth every other trace for each of ts shared features. When a new tem enters the actve subset, there s a 50% chance that t loses each feature t shares wth an tem already n the actve subset and a 50% chance that t keeps that feature and that the tem already n the actve subset loses t. 123
Famlarty s accessed by probng the actve subset wth the tem porton of the current vector. To determne famlarty, the frst step s to calculate the smlarty of the current tem to the tems n the actve subset by S M j= P T N j j = 1, Eq. 2 where P j s jth element n probe P and T j s the jth element n memory trace. N s the number of elements that are non-zero n ether the probe or the trace. M s the number of traces n the actve subset. The actvaton of each trace, A, s the cube of ts smlarty value. The echo ntensty of the actve subset to the probe s the sum of all these actvatons: I = M A = 1, Eq. 3 where M s the number of traces n the actve subset. If the I s greater than 0, then the stmulus s consdered famlar. Otherwse, the response s non-match. If the tem s famlar then the recollecton or recall process s ntated to determne f the current stmulus matches the stmulus n-back. Ths requres the n-th back context be renstated. Each element n the current context s converted to the n-th back context wth probablty prenstate. The renstated context s used to probe the actve subset by agan cubng the results from Equaton 2. Ths tme, however, the context s used as the probe and actvatons are not used to determne the echo ntensty but nstead the echo content by C = M = 1 A T j. Eq. 4 The echo content s a nosy verson of the tems most actvated by the renstated context. C wll not be an exact match of any partcular tem. Therefore, C s dsambguated followng the procedure used to dsambguate hypotheses n HyGene. Ths s done by recallng tems from semantc memory based on ther actvaton to C. Semantc memory s the collecton of the vectors representng each of the tems used as stmul. C s frst normalzed and then t s used to probe semantc memory. Once more Equaton 2 s used to determne the actvaton but ths tme of semantc memory nstead of the actve subset. Retreval from semantc memory s based on the actvaton of each tem vector. The probablty of samplng semantc vector s P = W A j= 1 A j, Eq. 5 where W s the number of vectors n semantc memory. The frst tem sampled from semantc memory s consdered the n-th back stmulus. However, to be successfully retreved the actvaton of the to-be-retreved vector must be greater than the retreval threshold, tretreval, otherwse retreval fals and the model guesses whether or not the stmulus s a target. The probablty of the model guessng target s set to the actual probablty of targets n the sequence, 0.2 n the current experment. If retreval s successful then the retreved tem s compared wth the current stmulus. If the current stmulus matches the retreved tem, then the response s match. If the retreved tem does not match the current stmulus, then the response s non-match. Famlarty, recollecton, and guessng each take tme. Here we assume that each take a sngle unt of tme. Therefore, the RT predctons are completely determned by the average number of processes requred to correctly and ncorrectly respond to the targets, lures and other nontargets. Smulatons Results The model was run once on each stmulus sequence gven to partcpants at each level of n. The second panel of Fgure 3 shows smulaton results for 4-back. The model produces prmacy, especally for targets. It also shows the same pattern of RT results as shown by partcpants, as shown n the second panel of Fgure 5. Specfcally, correct responses are made to targets and lures at approxmately the same speed but responses to other non-targets are faster. Incorrect responses to other non-targets and lures are slower than ncorrect responses to targets. Whle the detaled results are only shown for 4-back, the model predctons, lke partcpant performance, s consstent across levels of n. The only change beng that as n ncreases, the asymptotc level of accuracy for targets decreases for both partcpants and the model. Prmacy s predcted by the model due to the nterference between the tems mantaned n the actve subset of memory. Specfcally, t s due to the number of other tems that any gven tem must compete wth before that tem can be used to make a response. For example, when performng 4-back, the frst tem of the sequence only competes wth the three tems added after t. After the thrd subsequent tem s added, the frst tem wll be the n-th back stmulus to be used to make the next response. However, the fourth tem n the sequence competes wth at least the three tems that preceded t nto the actve subset and the three tems that followed t. The amount of nterference s ncreased when tems that are no longer relevant reman n the actve subset. However, even wth perfect removal of rrelevant tems some degree of prmacy s found. As mentoned above, the RT predctons are completely drven by the number of processes used to make a response. For example, normally two processes are necessary to make a correct or ncorrect response to a target: famlarty and recollecton. Correct responses to other non-targets are qucker because they can usually be dentfed as nonmatches by the results of the famlarty process alone. In contrast, ncorrect responses to other non-targets occur prmarly when the stmulus s judged as famlar but recall fals and an ncorrect guess of match s made. Lke 124
targets, correct lure responses often nvolve both famlarty and recollecton, but ncorrect lure responses are sometmes the result of false recollecton and sometmes the result of guessng. General Dscusson A detaled examnaton of n-back performance supports the clam that lures are necessary for makng the task more than a famlarty judgment task (Kane et al., 2007). However, the dfference n RTs between other non-targets and the two tral types n whch recollecton s necessary, targets and lures, ndcated that the presence of lures n a stmulus sequence does not necessarly change how partcpants respond to the other non-target trals. The present model accounts for ths RT data by assumng that the famlarty of a stmulus determnes whether or not a recollecton s attempted. If a stmulus s not suffcently famlar, then the stmulus s mmedately labeled a non-target. Therefore, accordng to the present model, correct responses on nontarget trals can be accounted for exclusvely by famlarty whether or not the stmulus sequence also contans lures. Only lures and targets, the tral types lkely to be famlar due to ther occurrence approxmately n stmul ago are lkely to trgger recollecton. Other non-targets make up at least 50% of the trals n most applcatons of n-back, so an overall n-back score could mostly reflect the ablty to dscrmnate famlar tems. Therefore, accordng to the present analyss the score does not prmarly reflect a partcpant s ablty to recognze the reoccurrence of the n-th back tem, but nstead famlarty judgment. Ths s one potental reason for the low correlaton between the n-back task and standard workng memory assessments (e.g., operaton span and readng span) n whch recall s necessary. WM s often conceptualzed as havng a capacty or span component as well as an executve functon or attentonal control component. The present modelng effort suggests that the span component of WM s not necessary to account for n-back performance, as ths aspect of WM s not mplemented wthn the model. Instead the executve functon or attentonal control aspect alone mght be suffcent. Attentonal control was mplemented here as the ablty to remove rrelevant nformaton from attenton (premove) and the ablty to conduct controlled memory search (prenstate). Ths mght also dfferentate n-back from other WM assessments, as the other tasks mght rely more heavly on capacty or span. Acknowledgments Ths research was supported by the Unversty of Maryland Center for Advanced Study of Language wth fundng from the Department of Defense. The authors thank Mchael Buntng, Jared Novck, Scott Weems, Erka Hussey, Susan Teubner-Rhodes, and Barbara Forsyth for ther contrbutons to the desgn and mplementaton of the experment. References Atkns, S. M., Harbson, J. I., Buntng, M. F., Teubner- Rhodes, S., & Dougherty, M. R. (2009, November). Measurng workng memory wth automated block span and automated letter-number sequencng. Poster presented at the 50 th Annual Meetn of the Psychonomc Socety. Gray, J. R., Chabrs, C. F., & Braver, T. S. (2003). Neural mechansms of general flud ntellgence. Nature Neuroscence, 6, 316-322. Hntzman, D. L. (1988). Judgments of frequency and recognton memry n a multple-trace memory model. Psychologcal Revew, 96, 528-551. Jaegg, S. M., Buschkuehl, M., Jondes, J., & Perrg, W. J. (2008). Improvng flud ntellgence wth tranng on workng memory. Proceedngs of the Natonal Academy of Scences of the Unted States of Amerca, 105, 6829-6833. Jaegg, S. M., Buschkuehl, M., Perrg, W. J., Meer, B. (2010). The concurrent valdty of the N-back task as a workng memory measure. Memory, 18, 394-412. Kane, M. J., Conway, A. R. A., Mura, T. K., & Colflesh, G. J. H., (2007). Workng memory, attenton control, and the n-back task: A queston of construct valdty. Journal of Expermental Psychology: Learnng, Memory, and Cognton, 33, 615-622. L, S. C., Schmedek, F., Huxhold, O., Röcke, C., Smth, J., & Lndenberger, U. (2008). Workng memory plastcty n old age: Practce gan, transfer, and mantenance. Psychology of Agng. 23, 731-742. McCabe, J., & Hartman, M. (2008). Workng memory for tem and temporal nformaton n younger and older adults. Agng, Neuropsychology, and Cognton, 15, 754-600. Novck, J. M., Hussey, E., Teubner-Rhodes, S., Dougherty, M. R., Harbson, J. I., & Buntng, M. F. (submtted). Clearng the garden path: Improvng sentences processng through executve tranng. Oberauer, K. (2005). Bndng and nhbton n workng memory: Indvdual and age dfferences n short-term recognton. Journal of Expermental Psychology: General, 134, 368-387. Oberauer, K., & Lewandowsky, S. (2008). Forgettng n mmedate seral recall: Decay, temporal dstnctveness, or nterference? Psychologcal Revew, 115, 544-576. Owen, A. M., McMllan, K. M., Lard, A. R., & Bullmore, E. (2005). N-back workng memory paradgm: A metaanalyss of normatve functonal neuromagng studes. Human Bran Mappng, 25,46-59. Thomas, R. P., Dougherty, M. R., Sprenger, A., & Harbson, J. I., (2008). Dagnostc hypothess generaton and human judgment. Psychologcal Revew, 115, 155-185. 125