ARTICLE IN PRESS Neuropsychologia xxx (2010) xxx xxx

Transcription

1 Neuropsychologa xxx (200) xxx xxx Contents lsts avalable at ScenceDrect Neuropsychologa journal homepage: Storage and bndng of object features n vsual workng memory Paul M. Bays, Emma Y. Wu, Masud Husan UCL Insttute of Cogntve Neuroscence & Insttute of Neurology, Queen Square, London, UK artcle nfo abstract Artcle hstory: Receved 30 November 200 Accepted December 200 Avalable onlne xxx Keywords: Workng memory Vsual attenton Bndng Jont dstrbuton Mxture model An nfluental concepton of vsual workng memory s of a small number of dscrete memory slots, each storng an ntegrated representaton of a sngle vsual object, ncludng all ts component features. When a scene contans more objects than there are slots, vsual attenton controls whch objects gan access to memory. A key predcton of such a model s that the absolute error n recallng multple features of the same object wll be correlated, because features belongng to an attended object are all stored, bound together. Here, we tested partcpants ablty to reproduce from memory both the color and orentaton of an object ndcated by a locaton cue. We observed strong ndependence of errors between feature dmensons even for large memory arrays (6 tems), nconsstent wth an upper lmt on the number of objects held n memory. Examnng the pattern of responses n each dmenson revealed a gaussan dstrbuton of error centered on the target value that ncreased n wdth under hgher memory loads. For large arrays, a subset of responses were not centered on the target but nstead predomnantly corresponded to mstakenly reproducng one of the other features held n memory. These msreportng responses agan occurred ndependently n each feature dmenson, consstent wth msbndng due to errors n mantanng the bndng nformaton that assgns features to objects. The results support a shared-resource model of workng memory, n whch ncreasng memory load ncrementally degrades storage of vsual nformaton, reducng the fdelty wth whch both object features and feature bndngs are mantaned. 200 Elsever Ltd. All rghts reserved.. Introducton What lmts the vsual nformaton that can be mantaned n short-term memory? Hstorcally, ths queston has been addressed by examnng the frequency of recall errors as memory load s manpulated, ether n studes of partal report (Irwn, 99, 992; Irwn & Andrews, 996; Sperlng, 960) or change detecton (Luck & Vogel, 997; Pashler, 988; Phllps, 974; Rouder et al., 2008; Todd & Maros, 2004; Vogel, McCollough, & Machzawa, 2005; Vogel, Woodman, & Luck, 200). The results of these studes have commonly been nterpreted as supportng a lmt on the number of objects that can be smultaneously represented n workng memory. In one nfluental verson of ths model, the objects present n a vsual scene compete for storage n a small number of ndependent memory slots. Each slot mantans a representaton of a sngle ntegrated object (ncorporatng all ts features, bound together) wth hgh fdelty, and the allocaton of vsual attenton determnes whch objects gan access to a slot (Cowan, 200; Hollngworth Correspondng author at: UCL Insttute of Neurology, Sobell Department, 33 Queen Square, London WCN 3BG, UK. E-mal address: p.bays@on.ucl.ac.uk (P.M. Bays). & Henderson, 2002; Irwn & Andrews, 996; Luck & Vogel, 997). Recently, ths concepton of workng memory has been challenged by studes examnng how recall errors are dstrbuted n the space of possble responses, based on dscrmnaton (Bays & Husan, 2008, 2009; Palmer, 990) or reproducton tasks (Bays, Catalao, & Husan, 2009; Wlken & Ma, 2004; Zhang & Luck, 2008, 2009). These studes have revealed strct lmts on the fdelty wth whch multple vsual objects can be mantaned: the precson wth whch each vsual feature s stored declnes rapdly as the total number of tems n memory ncreases. Ths fndng s dffcult to reconcle wth the concept of storage n ndependent slots, and has led to the development of an alternatve, shared-resource account of workng memory (Bays & Husan, 2008; Wlken & Ma, 2004). Accordng to ths proposal, a sngle memory resource s flexbly dstrbuted between the elements of a vsual scene. As more tems are stored, less resource s avalable per tem, wth the result that the features of each tem are stored wth ncreasng varablty ( nose ). Vsual attenton provdes flexble control over dstrbuton of ths resource, such that salent or goal-relevant tems are stored wth enhanced resoluton (Bays & Husan, 2008). Importantly, n contrast to the slot model, ths resource-based account does not predct a fxed upper lmt on the number of /$ see front matter 200 Elsever Ltd. All rghts reserved.

2 2 P.M. Bays et al. / Neuropsychologa xxx (200) xxx xxx objects that can be mantaned. Indeed a mathematcal model based on shared resources (Bays & Husan, 2008) predcts the appearance of such a capacty lmt n change detecton tasks, prevously consdered evdence n favor of a fxed slot model. Nonetheless, a number of attempts have been made to fnd a compromse poston between the two models, n whch varyng resoluton of storage co-exsts wth a fxed lmt on the number of objects that can be stored (Alvarez & Cavanagh, 2004; Awh, Barton, & Vogel, 2007; Zhang & Luck, 2008). In partcular, recent studes by Luck and colleagues (Zhang & Luck, 2008, 2009) have presented results from a color reproducton task whch appear to provde support for such a hybrd model. In these studes, partcpants were presented wth a memory array of colored squares. After a bref retenton nterval, one array locaton was ndcated and partcpants were requred to report the color they recalled at that locaton by clckng on a color wheel. The authors analysed the dstrbuton of responses on the color wheel as a mxture of two components: a gaussan dstrbuton centered on the correct color of the probed tem, and a unform dstrbuton spread equally over all possble responses. The gaussan component ndcates varablty n the stored representatons of the colors n the memory array. Consstent wth a resource-model account, the varablty wth whch each tem was stored depended on the total number of tems n memory, as ndcated by an ncrease n the gaussan wdth wth ncreasng memory load. In addton, however, Zhang and Luck proposed that the unform component corresponds to a proporton of trals on whch subjects choose a response at random. As n a slot model, ths mght occur f no nformaton was stored about the probed object as the result of exceedng an upper lmt on the number of objects that can smultaneously be mantaned n workng memory. Here we put ths nterpretaton to the test, by examnng the jont dstrbuton of errors when subjects are requred to reproduce from memory two dfferent features (color and orentaton) belongng to the same probed object. If only a subset of objects n an array can be stored, the absolute error n reportng color and orentaton should be correlated, and the jont dstrbuton of errors n the dual-feature task should consst of two components: one n whch the object s stored and both features are recalled (wth gaussan varablty), and one n whch the object s not stored and both responses are random. Nether result was observed: nstead our results revealed that both the absolute error and the occurrence of unform responses were strongly ndependent across feature dmensons. Ths fndng s nconsstent wth the hypothess of a fxed upper lmt on the number of objects stored n memory. Instead these results support the proposal of Wheeler and Tresman (2002) that vsual features n dfferent dmensons are mantaned n ndependent memory stores. These authors conclusons were based n part on the observaton of bndng errors n a change detecton task: errors caused by ncorrectly combnng n memory features that belong to dfferent objects (Allen, Baddeley, & Htch, 2006; Robertson, 2003; Tresman, 998; Tresman & Schmdt, 982; Wheeler & Tresman, 2002; Wolfe & Cave, 999). We have prevously proposed (Bays et al., 2009) that the unformly dstrbuted responses nterpreted by Zhang and Luck (2008) as random guesses may nstead correspond to mstakenly reportng the features of one of the other tems held n memory. Here, by analysng the frequency of these msreportng errors wthn and across feature dmensons, we confrm that they are the result of msbndng features held n ndependent memory stores, consstent wth the storage of vsual features n separate sensory representatons (Pasternak & Greenlee, 2005). These results have mportant mplcatons for the nature of vsual workng memory representatons and the locus of bndng of separate features belongng to an object. 2. Materals and methods 2.. Expermental protocol Ten subjects (seven male, three female; age 22 26) partcpated n the study after gvng nformed consent. All had normal or corrected-to-normal vsual acuty; none reported any dffculty n makng color dscrmnatons. Stmul were dsplayed on a 2-n. CRT montor at a vewng dstance of 60 cm. Eye poston was montored onlne at 000 Hz usng a frame-mounted nfra-red eye tracker (Eyelnk 000, SR Research Ltd., Canada). Each tral began wth the presentaton of a central fxaton cross (whte, dameter 0.75 of vsual angle) aganst a black background. Once a stable fxaton was recorded on the cross, a memory array was presented, consstng of a number of colored orented bars ( ) randomly dstrbuted around fxaton at eccentrctes n the range 6 0, wth a mnmum center-to-center separaton of 6 between tems (example n Fg. a). The color and orentaton of each tem were ndependently chosen at random from two crcular parameter spaces. The orentaton parameter space corresponded to the range of angles 0 80 (.e. the full range of possble bar orentatons). For color, the parameter space was defned by a crcle n CIE L*a*b* coordnates wth constant lumnance (L* = 50), center at a*=b* = 20, and radus 60. The memory array was presented for 2 s, followed by a pattern mask for 00 ms and then a blank retenton nterval (900 ms). The pattern mask was ncluded to ensure conc memory dd not contrbute to performance. A sngle (probe) tem was then presented at one randomly chosen locaton from the precedng memory array. Subjects were nstructed to adjust the orentaton and color of the probe tem to match the features of the tem that had been presented at the same locaton n the memory array (the target). The probe s features were adjusted usng two nput dals (PowerMate USB Multmeda controller, Grffn Technology, USA) one operated wth each hand (randomly assgned). Turnng one dal caused the probe to rotate through the range of possble orentatons (Fg. b, top); turnng the other dal caused the probe s color to cycle through the space of possble colors (Fg. b, bottom). The probe s ntal features were randomly assgned. Subjects could adjust the two dals n any order or smultaneously, and ndcated adjustment was complete by depressng the center of ether dal. Accuracy was stressed, and responses were not tmed. Each subject completed 300 trals n total: a block of 250 trals wth sx-tem memory arrays (hgh-load), and a block of 50 trals wth just one tem n each array (low-load). Fewer trals were requred n the low-load condton because there was no possblty of msreportng a non-target tem, greatly smplfyng the datantensve modelng component of the analyss (Secton 2.2.3). Hgh- and low-load blocks were completed n a counter-balanced order. Any tral on whch gaze devated more than 2 from the central cross durng presentaton of the memory array was aborted and restarted wth new feature values. Ths constrant prevented subjects fxatng ndvdual memory array tems, whch otherwse mght bas storage towards partcular objects n the array (Bays & Husan, 2008) Analyss Wthn-dmenson errors Our ntal analyss examned errors n recall of color and orentaton separately. Responses n each dmenson were analysed n terms of the crcular parameter space of possble feature values (range to radans; Fg. b). For each tral, a measure of recall error n each dmenson was obtaned by calculatng the angular devaton between the feature value reported by the subject and the feature value of the target tem n the memory array. To obtan measures of performance comparable across feature dmensons, we calculated the recall bas, defned as the mean of the recall error, and precson, defned as the recprocal of the standard devaton of error. As n a prevous study (Bays et al., 2009), we used the defnton of mean and standard devaton for crcular data gven by Fsher (993), and subtracted from the precson estmate the value expected by chance (.e. f the subject had responded at random on each tral). To nvestgate the source of recall errors, we frst examned the dstrbuton of error n each feature dmenson wth respect to a probablstc model of memory performance descrbed by Zhang and Luck (2008). Ths model proposes that errors n a reproducton task arse from two sources: gaussan varablty n memory for the target feature, and a fxed probablty on each tral of guessng at random. The dstrbuton of responses s therefore descrbed by a mxture model (McLachlan & Peel, 2000) of general form: p(ˆ) = k kp k where ˆ s the reported feature value, k s the probablty that a response comes from the kth component ( k = ), and p k s the probablty densty functon descrbng the dstrbuton of responses under that component. Zhang and Luck s model has two components (k = 2) whose probablty densty functons are gven n the frst two rows of Table. The target component (T) corresponds to nosy recall of the target feature, resultng n a dstrbuton of responses drawn from a crcular gaussan (von Mses) dstrbuton centered on the true feature value of the target;

3 P.M. Bays et al. / Neuropsychologa xxx (200) xxx xxx 3 Fg.. Precson of workng memory n a dual-feature reproducton task. (a) Subjects were presented wth an array of colored, orented bars, followed by a pattern mask. After a blank retenton nterval, a probe appeared and subjects used two response dals to adjust ts color and orentaton to match the tem at the correspondng locaton n the memory array (the target). (b and c) Turnng each dal cycled the probe through a crcular parameter space of possble colors or orentatons. Some examples of orentatons (b) and colors (c) are shown correspondng to dfferent ponts n each response space. (d) Recall precson as a functon of memory load: object (low) versus 6 objects (hgh). Precson s defned as the recprocal of the standard devaton of error n subjects responses: zero ndcates chance performance. the unform component (U) corresponds to random guessng, producng a unform dstrbuton of responses across all possble feature values. Maxmum lkelhood estmates of the mxture parameters { T, U} and, the standard devaton of the von Mses dstrbuton (Jammalamadaka & Sengupta, 200), were obtaned separately for each feature dmenson, subject, and array sze n MATLAB usng a custom-wrtten expectaton-maxmzaton algorthm (Blmes, 998; Dhllon & Sra, 2003; see also Lawrence, 200). The optmzaton procedure was repeated from a range of dfferent ntal parameter values to ensure that global maxma were obtaned Jont dstrbuton of color and orentaton errors In order to model the jont dstrbuton descrbng responses n both feature dmensons, we extended the mxture model to nclude both color and orentaton responses: p(ˆ O, ˆ C ) = k kp k where ˆ O s the reported orentaton value and ˆ C s the reported color value. On each tral, the response for each feature dmenson could come from target or unform dstrbutons, resultng n four possble combnatons of color and orentaton response dstrbutons (Table 2, rows 4). Maxmum lkelhood estmates were obtaned as above for mxture parameters { TT, UT, TU, UU} and O and C, the standard dev- Table Mxture model components descrbng the dstrbuton of responses wthn a sngle feature dmenson. k Mxture component Response type Probablty densty (p k ) T Target (ˆ ) 2 U Unform 3 N Non-target 2 m (ˆ ϕ ) s the true feature value of the target tem, and ˆ the feature value reported by the subject (range < ). ϕ s the true feature value of the th non-target tem (m n total). s the von Mses dstrbuton wth mean zero and standard devaton. Dagrams ndcate how the response probabltes contrbuted by each component are dstrbuted around target (T) and non-target (N) feature values (for llustraton, only two non-targets are shown). The model proposed by Zhang and Luck (2008) s descrbed by rows 2 (target and unform components). Bays et al. (2009) added a thrd component (row 3) to dstngush non-target from unform responses. atons of the von Mses dstrbutons descrbng varablty n orentaton and color recall, respectvely. We examned two opposng hypotheses regardng the relatonshp between errors n the two feature dmensons. Under the hypothess that color and orentaton errors are fully ndependent, the jont response dstrbuton should comprse a mxture of all four response combnatons, occurrng at the frequences predcted by the product of the margnal probabltes obtaned separately for each dmenson (Secton 2.2.). That s, the parameters descrbng the frequences of each combnaton of response types can be predcted drectly from the parameters O and C obtaned by fttng orentaton and color responses separately: TT = C O T T, TU = O C, T U UT = O, and U C T UU = O U C U. Under the opposng hypothess that color and orentaton errors are fully correlated, the jont response dstrbuton should comprse a mxture of only two components: trals on whch both the color and orentaton of the target are reported, and trals on whch both responses are random. Hence mxture parameters UT and TU (correspondng to trals n whch one response s from the target dstrbuton and one from the unform) have predcted values of zero. Ths hypothess predcts mxture parameters for target target and unform unform responses should equal the correspondng parameters obtaned for orentaton and color separately,.e. TT = O T = C and T UU = O U = C. Because the margnal values of U these parameters obtaned n Secton 2.2. were not exactly dentcal, for modelng purposes we averaged across feature dmensons: TT = ( O T + C )/2, T UU = ( O U + C U )/2. As nether hypothess was fully consstent wth our results, we quantfed the strength of correlaton between feature dmensons by calculatng 2, the equvalent for bnary varables of the coeffcent of determnaton r 2 : ( TT UU UT TU)2 2 = T U U T where T = TT + TU, etc. Lke r 2, ths measure falls n the range 0 2, where 2 = 0 ndcates complete ndependence between feature dmensons, and 2 = ndcates full correlaton Msreportng errors Bays et al. (2009) proposed that the majorty of responses captured by the unform component of the Zhang and Luck (2008) model are not a result of random guessng. Instead they are nstances of subjects mstakenly reportng the feature value of one of the non-target tems n the memory array (msreportng errors). As n our prevous study, we assessed the frequency of these errors by addng a thrd component to the mxture model descrbng responses n each feature dmenson (Table, row 3). Responses due to ths non-target component (N) are drawn wth equal probablty from von Mses dstrbutons centered on the feature values of each of the non-target tems n the memory array. Because the target tem s unknown at the tme of storage, the standard devatons of target and non-target von Mses dstrbutons are equal n ths model. Maxmum lkelhood estmates for ths threecomponent model were obtaned separately for color and orentaton responses, as above. To capture the jont dstrbuton of color and orentaton responses under ths expanded model requres ten components (Table 2, rows 0). On each tral, the response for each feature dmenson can come from target, non-target, or unform dstrbutons, resultng n nne possble combnatons of color and orentaton response types. In addton, on trals where both responses reflect non-target fea-

4 4 P.M. Bays et al. / Neuropsychologa xxx (200) xxx xxx Table 2 Mxture model components descrbng the jont dstrbuton of responses n two feature dmensons. k Mxture component Response type Probablty densty (p k ) Orentaton Color TT Target Target O (ˆ O O ) C (ˆ C C ) ( ) 2 2 UU Unform Unform 2 3 TU Target Unform O (ˆ O O ) 2 4 UT Unform Target 5 N = N Same non-target 2 C (ˆ C C ) m O (ˆ O ϕ O ) C (ˆ C ϕ C ) 6 N /= N Dfferent non-targets m(m ) O (ˆ O ϕ O ) C (ˆ C ϕ O j ) 7 NT Non-target Target m j /= O (ˆ O ϕ O ) C (ˆ C C ) 8 TN Target Non-target O (ˆ O O ) m C (ˆ C ϕ C ) 9 NU Non-target Unform 0 UN Unform Non-target m 2 m O (ˆ O ϕ O ) 2 C (ˆ C ϕ C ) O and C are the true orentaton and color values, respectvely, of the target tem (range < ). ˆ O and ˆ C are the correspondng feature values reported by the subject. ϕ O and ϕ C are true feature values of the th non-target tem (m n total). O and C are von Mses dstrbutons wth mean zero and standard devatons O and C, correspondng to varablty n recall of orentaton and color, respectvely. Rows 4 descrbe the extenson to two feature dmensons of Zhang and Luck s (2008) model, ncludng only target and unform components; rows 0 descrbe the correspondng expanson of the Bays et al. (2009) model, dstngushng non-target from unform responses. tures, they can be due to reportng the orentaton and color of the same non-target (N = N), or the orentaton of one non-target and the color of a dfferent non-target (N /= N). We agan compared the predctons of two opposng hypotheses regardng the relatonshp between msreportng errors n the two feature dmensons. Under the hypothess of full correlaton, non-target responses occur when the subject msdentfes whch tem from the memory array has been probed, resultng n color and orentaton responses centered on the feature values of one of the non-targets. The frequency of these same non-target responses should therefore be equal to ther margnal frequences obtaned separately for orentaton and color responses: for modelng purposes we averaged across feature dmensons as above, N=N = ( O N + C )/2. Other combnatons nvolvng nontarget responses have zero probablty under ths hypothess: NT =0, TN = 0 and N N /= N =0. Under the alternatve ndependence hypothess, non-target responses occur fully ndependently n each feature dmenson: hence, NT = O, TN = O, N /= N = ((m )/m) O N C N and N=N = (/m) O N C N N C C T T N (where m s the number of non-targets) Hypothess testng Data from each ndvdual subject was analysed separately, and then pared sample t-tests used to make statstcal comparsons at the group level. An arcsne transformaton was used for tests on proportonal data, ncludng mxture parameters. 3. Results 3.. Bas and precson The recall task s llustrated n Fg. a. On each tral, a subject was presented wth an array of colored orented bars surroundng a central fxaton pont. After a blank retenton nterval, one array locaton was ndcated and the subject had to reproduce from memory both the color and the orentaton of the tem prevously dsplayed at that locaton (the target tem). For each feature dmenson, the recall error was defned as the devaton wthn the space of possble responses (Fg. b) between the reported and actual feature value of the target. The fdelty of reproducton n each feature dmenson can be characterzed by two parameters, bas and precson. Bas ndcates a systematc tendency to devate from the correct target value n the same drecton from tral to tral. No sgnfcant bas was observed n color or orentaton responses (t <.7, p > 0.0). Precson measures the degree to whch responses cluster around the correct feature value: a precson of zero ndcates that responses are randomly dstrbuted relatve to the target. Consstent wth prevous studes, the recall precson vared substantally wth changes n memory load (Fg. c). When only one tem was present n the memory array (low-load), subjects recalled both color and orentaton wth consderable precson (orentaton, 2.9 ± 0.3 rad ; color, 2.6 ± 0.2 rad ; mean ± S.E.; Fg. c), comparable wth performance n a prevous study n whch recall of just one feature (color) was requred (3.4 rad ; Bays et al., 2009). The precson of recall dd not sgnfcantly dffer between feature dmensons (t =.4, p = 0.20). When the number of tems n the memory array was ncreased (hgh-load, 6 tems), recall precson decreased sgnfcantly n both feature dmensons (orentaton, 0.64 ± 0.08 rad ; color, 0.70 ± 0.09 rad ; t > 8.2, p < 0.00). Precson agan dd not dffer sgnfcantly between dmensons (t = 0.8, p = 0.44) and was smlar to that prevously observed for color recall wth sx tem arrays (0.5 rad n Bays et al., 2009; but see also Fougne, Asplund, & Maros, 200). Subjects could choose to reproduce the orentaton and color of the target n any order, or adjust both smultaneously. Whether a feature was adjusted frst or second had no sgnfcant effect on recall precson under ether load condton (t < 0.32, p > 0.75).

5 P.M. Bays et al. / Neuropsychologa xxx (200) xxx xxx Dstrbuton of errors Fg. 2a plots the dstrbuton of responses relatve to the target feature value for color (top) and orentaton (rght) on trals wth just one tem n the memory array (low-load). In both feature dmensons, the dstrbuton of errors was accurately descrbed by a (crcular) gaussan centered on the target feature value (red curves; C = 0.30 ± 0.0; O = 0.29 ± 0.03). The jont hstogram of errors n both dmensons s shown by the heat map n Fg. 2a. The magntudes of error n color and orentaton on each tral were uncorrelated (r 2 < 0.0). The dstrbuton of errors n each feature dmenson n the hghload condton (6 tems) s shown n Fg. 2b (top and rght). As observed n prevous studes (Bays et al., 2009; Zhang & Luck, 2008), the pattern of responses was not consstent wth a solely gaussan dstrbuton of error n ether feature dmenson. Instead, the overall declne n precson (Fg. c) appeared to result from ncreases n two sources of error. Frst, unlke n the low-load condton, a sgnfcant proporton of responses n the hgh-load condton (orentaton, 29 ± 6%; color, 32 ± 5%) were statstcally unrelated to the true feature value of the target tem (.e. unformly dstrbuted). Second, the varablty of those responses that were centered on the target feature value ncreased compared to the low-load condton ( C = 0.46 ± 0.03; O = 0.60 ± 0.04, t > 6.9, p < 0.00), ndcatng that each feature was stored wth ncreased nose. The mxture of gaussan and unform components that best ft the observed dstrbuton of errors for each feature dmenson are shown by the red curves n Fg. 2b Unformly dstrbuted errors The unform response component has been nterpreted as ndcatng a proporton of trals on whch the probed object s not stored n memory. Ths hypothess predcts that unformly dstrbuted responses wll be fully correlated across feature dmensons: a unform color response wll always concde wth a unform orentaton response, and vce versa. Based on the model parameters obtaned from fttng orentaton and color responses separately, we can predct the jont dstrbuton of errors we would expect under ths hypothess: the predcton s shown by the heat map n Fg. 2c (top). The magntudes of color and orentaton errors are correlated n ths dstrbuton, wth r 2 = We also consdered an alternatve hypothess n whch unform responses occur fully ndependently n the two feature dmensons: the predcton of ths model s shown n Fg. 2c (bottom). The ndependence hypothess predcts a concentraton of responses along horzontal and vertcal axes of the jont dstrbuton, correspondng to trals on whch the color response comes from the gaussan dstrbuton and the orentaton response comes from the unform dstrbuton, and vce versa. The magntudes of color and orentaton errors are uncorrelated under ths hypothess. The jont hstogram of observed errors n the hgh-load condton s shown n Fg. 2b (heat map). A concentraton of responses along the axes s clearly vsble along vertcal and horzontal axes, as predcted under ndependence. Neglgble correlaton was observed between error magntudes n color and orentaton (r 2 = 0.02), also consstent wth the ndependence hypothess. To examne n more detal the frequences of unform and target-centered responses, we ft a probablstc model to subject s responses n whch trals could fall nto four categores: those on whch both color and orentaton responses were centered on target values (TT), those on whch both responses were unrelated to the target and drawn from a unform dstrbuton (UU), those on whch the orentaton response was centered on the target orentaton and the color response was from the unform dstrbuton (TU), and vce versa (UT). The ftted parameter values are shown n Fg. 2d, along wth the predctons under correlated and ndependent unform responses. Inconsstent wth the correlaton hypothess, whch predcts that every tral wll fall ether nto category TT or category UU, a hghly sgnfcant proporton of trals were descrbed by categores UT or TU (28 ± 3%; t = 4, p < 0.00). Overall, the observed parameter values ndcated strong ndependence of unformly dstrbuted responses n color and orentaton dmensons ( 2 = 0.) Msreportng errors In a prevous study (Bays et al., 2009) we proposed that a mxture model comprsng only target and unform components may be nsuffcent to fully descrbe the pattern of responses on reproducton tasks. We suggested that a thrd source of errors needed to be consdered: nstances of mstakenly reportng a feature value belongng to one of the other (non-target) tems held n memory. Such msreportng errors appear unformly dstrbuted when responses are plotted relatve to the target feature value (as n Fg. 2 and Zhang & Luck, 2008), and hence may be ncorrectly attrbuted to random guessng. However, these errors appear as a sgnfcant concentraton of the response dstrbuton around zero when responses are plotted relatve to each of the feature values of the non-target tems n the memory array. These non-target dstrbutons are plotted n Fg. 3a, for color (top) and orentaton responses (rght) n the hgh-load condton. Whereas the guessng nterpretaton predcts that these dstrbutons should be unform, responses n each dmenson centered on the feature values of the non-targets were sgnfcantly more frequent than expected by chance (color: t = 7.7, p < 0.00; orentaton: t = 4.7, p = 0.00). Followng Bays et al. (2009), we ft a three-component model to the data from each dmenson, n whch responses could come from a dstrbuton centered on the target, a unform dstrbuton, or a dstrbuton shared equally between each of the non-targets. The resultng parameter estmates ndcated that msreportng errors formed a sgnfcant proporton of responses n each dmenson (color: 7 ± 4%; orentaton: 6 ± 6%; t > 4.7, p = 0.00). As a result, the proporton of responses attrbuted to the unform component was sgnfcantly reduced n comparson to the two-component model (color: 5% versus 32%; orentaton: 4% versus 29%; t > 2.3, p < 0.05). We consdered two possble sources of msreportng errors. One hypothess was that they were caused by errors n storng the locatons of tems n the memory array; the alternatve was that they were caused by msbndng of features durng storage or mantenance n memory. Agan the crtcal test s the degree of ndependence between responses on each tral. The locaton-error hypothess predcts that msreportng errors wll be correlated across feature dmensons: mstakenly reportng the color of a non-target wll always concde wth mstakenly reportng the orentaton of the same non-target. In contrast, the msbndng hypothess predcts that msreportng errors wll occur ndependently n each feature dmenson. The jont dstrbuton of responses relatve to each non-target s feature values s shown by the heat map n Fg. 3a. The magntude of devaton of responses from non-target feature values was uncorrelated across feature dmensons (r 2 < 0.0) consstent wth ndependence of non-target errors. As before, we ft a probablstc model to the jont response data allowng for all possble combnatons of target, non-target and unform responses. The crtcal parameters that dstngush the two hypotheses are those nvolvng combnatons of target and non-target responses, shown n Fg. 3b. These parameters ndcate the frequency of four categores of tral: those on whch the subject responds wth the orentaton of the target but the color

6 6 P.M. Bays et al. / Neuropsychologa xxx (200) xxx xxx Fg. 2. Dstrbuton of errors relatve to target feature values. (a) Frequency of response as a functon of the devaton between reported and target feature values n the low-load ( object) condton: for color (top), orentaton (rght), and conjuncton of both features (heat map). Colored lnes ndcate the response probabltes predcted by a mxture model combnng a gaussan dstrbuton centered on the target value and a unform dstrbuton spread equally across the response space. (b) Error dstrbutons plotted as n (a) for the hgh-load (6 object) condton. (c) Predcted dstrbutons of color and orentaton responses under condtons of full correlaton (top) and full ndependence (bottom) between feature dmensons. Compare wth the observed dstrbuton of errors shown n (b). (d) Proporton of trals on whch both responses are centered on target values (TT); both are unformly dstrbuted (UU); the orentaton response s centered on the target and the color response s unform (TU); and vce versa (UT). Estmates obtaned by fttng a mxture model to observed responses are shown along wth predctons under full correlaton and full ndependence models. of a non-target (TN), or vce versa (NT); trals where the subject responds wth both the color and the orentaton of a sngle nontarget (N = N); and those where they respond wth the color of one non-target and the orentaton of another (N /= N). Fg. 3b shows the observed frequences of responses n each category. In addton, t dsplays the predcted frequences under correlated- and ndependent-msreportng hypotheses, based on the model parameters obtaned by fttng color and orentaton responses separately. The correlated-msreportng hypothess predcts that non-target responses wll occur only when the subject mstakenly responds wth both the color and orentaton of a sngle non-target (N = N). Inconsstent wth ths hypothess, sgnfcant proportons of trals corresponded to one target feature and one non-target feature (NT or TN; 4 ± 3%; t = 0, p < 0.00), or responses to the color and orentaton of two dfferent non-targets (N /= N; 3 ± %; t = 2.8, p = 0.02). The ndependent-msreportng hypothess, n contrast, predcts that non-target responses wll occur ndependently for color and orentaton dmensons. All non-target components estmated from the data were consstent wth ths full-ndependence model (t <.8, p > 0.). 4. Dscusson The resoluton wth whch vsual features are stored n workng memory s hghly dependent on total memory load, and begns to declne as soon as the number of tems n memory exceeds one (Bays & Husan, 2008; Palmer, 990; Wlken & Ma, 2004; Zhang &

7 P.M. Bays et al. / Neuropsychologa xxx (200) xxx xxx 7 Fg. 3. Dstrbuton of errors relatve to non-target feature values. (a) Frequency of response as a functon of the devaton between reported feature values and those of each non-target (unprobed) tem, n the hgh-load (6 object) condton, for color (top), orentaton (rght), and conjuncton of both features (heat map). Note the strong central tendency n each dstrbuton, ndcatng that subjects frequently mstakenly report the features of non-target tems. (b) Proporton of trals on whch color and orentaton responses are both centered on the same non-target (N = N); each s centered on a dfferent non-target (N /= N); the orentaton response s centered on the target and the color response on a non-target (TN); and vce versa (NT). Estmates obtaned by fttng a mxture model to observed responses are shown along wth predctons of correlated-msreportng and ndependent-msreportng models. Luck, 2008). However, t remans controversal whether ths loss of fdelty alone accounts for all errors n recall, or whether t coexsts wth a fxed upper lmt on the number of objects that can be smultaneously mantaned (Alvarez & Cavanagh, 2004; Awh et al., 2007; Bays et al., 2009; Bays & Husan, 2009; Cowan & Rouder, 2009; Zhang & Luck, 2008, 2009). Recent attempts to address ths queston have examned recall of tems varyng n a sngle feature dmenson (typcally color). Here, we presented arrays of objects that vared n two dmensons, color and orentaton: partcpants were requred to reproduce from memory both features of a sngle object, ndcated by locaton. As n prevous studes, the precson wth whch subjects were able to reproduce each feature dffered substantally between condtons of low ( tem) and hgh (6 tems) memory load. Despte the substantal qualtatve dfferences between feature dmensons, precson (calculated wth respect to the range of possble feature values) was comparable for color and orentaton responses. Precson values were also smlar to those obtaned n a prevous study employng a very dfferent response methodology (a mouse clck on a wheel of color values n Bays et al., 2009, versus adjustng a dal to cycle through possble feature values n the present study). Ths provdes an mportant valdaton of the precson measure as a reproducble and general measurement of recall performance. Sgnfcantly, the smlarty of precson measures across dmensons dd not reflect a smple trade-off n performance between color and orentaton dmensons, as ths would predct a negatve correlaton n the magntudes of error n each dmenson whch was not observed. Whle t s clear that ncreasng the number of tems held n memory makes recall of each one less precse, the mechansms underlyng ths ncrease n uncertanty are contentous. A crtcal focus of debate s the manner n whch errors are dstrbuted wthn the space of possble responses. In the current study, when only one object was stored n memory, the dstrbuton of responses relatve to the target feature value ndcated that the stored representatons of color and orentaton were each ndependently corrupted by gaussan nose. When the task requred that a larger number of objects be stored, the dstrbuton of recall errors agan ndcated the presence of gaussan nose, but wth a substantal ncrease n varablty compared to the one-tem condton. These results are consstent wth a shared-resource model of workng memory n whch the varablty of storage s determned by the fracton of total memory resources avalable per tem (Bays & Husan, 2008; Wlken & Ma, 2004). However, whereas performance n the low-load condton was accurately captured by gaussan varablty alone, ths provded only a partal descrpton of the dstrbuton of errors when memory load was ncreased. In ths case, as n a prevous analyss (Zhang & Luck, 2008), a better ft was obtaned by a mxture model that also ncluded a second, unform dstrbuton (spread equally across all possble responses). The correct nterpretaton of ths addtonal, non-gaussan component s one of the key ssues the present study sought to address. 4.. Independence of memory stores for dfferent vsual features The presence of the unform error component under condtons of hgh memory load has been nterpreted as supportng a hybrd model of workng memory, n whch recall performance s lmted both by a declne n resoluton wth ncreasng memory load, and also by an upper lmt on the number of objects that can be stored (Zhang & Luck, 2008, 2009). Accordng to ths account, when the number of tems n the memory array exceeds the maxmum capacty, only a subset of objects s selected for storage. Hence, f the probe corresponds to an object that was not selected, the subject wll guess randomly n both feature dmensons. Ths model predcts that the occurrence of unform responses wll be fully correlated across feature dmensons: a unform color response wll always concde wth a unform orentaton response, and vce versa. Whle estmates of the proposed (object) capacty lmt vary, they have typcally fallen n the range two to four: a largescale study of 70 undergraduates usng a change-detecton task obtaned a mean capacty of 2.9 tems (reported n Vogel & Awh, 2008), whle the estmate obtaned from Zhang and Luck s (2008) data was 2.3 tems. In the present study we tested recall of 6 tem arrays, comfortably exceedng these proposed lmts, and mplyng that at least some objects should not have ganed access to memory under the predctons of a hybrd model. Nonetheless, absolute errors n color and orentaton were strongly ndependent (r 2 = 0.02), and a detaled analyss of the jont response dstrbuton revealed neglgble correlaton ( 2 = 0.) between unform response components n recall of the color and orentaton of the probed object.

8 8 P.M. Bays et al. / Neuropsychologa xxx (200) xxx xxx These results are nconsstent wth an upper lmt on the number of ntegrated objects stored n workng memory. The observed ndependence n errors across feature dmensons nstead mples that the multple vsual features from dfferent dmensons that make up an object may be mantaned separately, n ndependent memory stores. A smlar concluson was reached prevously by Wheeler and Tresman (2002), based on analyss of a varant of the change detecton task. These authors demonstrated that error rates were determned by the total number of features that needed to be remembered wthn each dmenson (e.g. how many colors were n a memory array) rather than the number of separate objects those features were dstrbuted between. Whle ths outcome was contrary to a prevous result obtaned by Luck and Vogel (997),thas been replcated n several subsequent studes (Olson & Jang, 2002; Xu, 2002). Instead of a sngle capacty-lmted memory store mantanng ntegrated object representatons, Wheeler and Tresman proposed parallel memory stores for each feature dmenson, wth ndependent capactes. In ths account, the nformaton requred to combne the features nto ntegrated objects (the bndng nformaton) s mantaned separately and ndependently from the features themselves. Such an account would also be consstent wth a large range of fndngs that demonstrate that sensory representatons, for example for dfferent vsual features, are assocated wth ndependent workng memory representatons (Pasternak & Greenlee, 2005). Whle the ntegrated-object hypothess predcts that an object s features wll always be remembered together, the ndependentstores account allows for the possblty that one feature of an object could gan access to memory whle another feature does not. However, the strong ndependence of storage between feature dmensons observed here s stll unexpected. Ths s because the ndependent-stores hypothess also assumes a fxed upper lmt on memory capacty, although now reflectng a maxmum number of features that can be stored per feature dmenson rather than a maxmum number of bound object representatons (Wheeler & Tresman, 2002). Assumng that selecton of features for storage s governed by the allocaton of vsual attenton to an object or locaton (Desmone & Duncan, 995; Duncan, 984; Posner & Cohen, 984; Tresman & Gelade, 980), ths model stll makes the predcton that storage of features belongng to the same object wll be strongly correlated. The absence of such a correlaton n the present analyss leads to one of two conclusons: ether selecton of a lmted number of features for storage occurred ndependently n each feature dmenson (nconsstent wth most current models of attentonal selecton), or all the features n each array were stored n memory Msbndng of object features In a prevous study (Bays et al., 2009), we proposed an alternatve explanaton for the unform component observed by Zhang and Luck (2008) that does not requre a lmt on the number of tems stored. In these studes, subjects reported the color of one tem from an array, ndcated by a locaton cue. Whle superfcally a smple test of memory for color, ths task also requres memory for locaton: to respond accurately, subjects must not only remember the colors n the array, but also whch color appeared where. Errors n recallng whch color corresponds to the probed locaton wll result n subjects mstakenly reportng the color of one of the other objects held n memory. Consstent wth ths hypothess, when we examned the dstrbuton of responses relatve to these non-target colors, we found that the responses captured by the unform component n Zhang and Luck s (2008) analyss were not n fact dstrbuted equally across the response space, but nstead predomnantly clustered around colors belongng to other objects n the memory array. Ths was not apparent n prevous analyses (Zhang & Luck, 2008, 2009) because errors were only consdered n relaton to the color of the probed tem, and the other, non-target colors n the array were randomly dstrbuted relatve to ths target color (see also Bays, 200). The frequency of these msreportng errors n the present study was assessed by addng a thrd component to the mxture model, a gaussan component dstrbuted equally between non-target color values. We confrmed that, as n the prevous study (Bays et al., 2009), a substantal proporton of responses that Zhang and Luck s model nterpreted as random guesses were n fact nstances of mstakenly reportng a feature value belongng to one of the other, non-probed objects held n memory. Prevously, we proposed that msreportng responses mght be a consequence of varablty ether n memory for locaton or msbndng of object features durng mantenance or recall from memory (Bays et al., 2009). Subjects were requred to report the color of an object matchng a partcular probed locaton n the dsplay: error n memory for locatons could therefore result n a subject mstakenly reportng the color of one of the other, nonprobed objects. Alternatvely, both colors and locatons n the array may have been stored accurately, but the nformaton ndcatng whch color belonged wth whch locaton may have become dsrupted (Allen et al., 2006; Robertson, 2003; Tresman, 998; Tresman & Schmdt, 982; Wheeler & Tresman, 2002; Wolfe & Cave, 999). Because two feature dmensons other than locaton were tested, these hypotheses can be dscrmnated n the present study. The locaton-error hypothess predcts that non-target responses wll occur smultaneously n both dmensons: the subject wll mstake whch object s locaton corresponds to the probe, and report both the color and orentaton of an tem at a dfferent locaton n the memory array. In contrast, we predcted that msbndng would occur ndependently for each feature, so one response mght accurately reflect the target feature value whereas the other corresponds to a non-target; or color and orentaton responses could correspond to the features of two dfferent non-targets. The present results ndcate that nontarget responses occur ndependently n each feature dmenson (r 2 < 0.0), consstent wth the msbndng hypothess. Whle locaton errors are a predctable consequence of varablty n storage of spatal nformaton, t appears they dd not contrbute sgnfcantly to responses n the current study. Ths could be a consequence of the mnmum separaton mantaned between objects n each memory array (6 of vsual angle), whch may have preserved accurate dentfcaton of the probed object even under consderable varablty n recall of ts locaton. The observaton of msbndng errors n the present study s consstent wth prevous results showng an ncrease n errors when recall s tested for feature conjunctons (e.g. color-shape pars) as opposed to ndvdual features (Brown & Brockmole, 200; Fougne & Maros, 2009; Wheeler & Tresman, 2002). A corollary of ndependent storage across feature dmensons s that a further mechansm s requred to mantan the bndng nformaton that groups features nto objects. A shared-resource account of workng memory mples that the fdelty wth whch ths bndng nformaton s mantaned wll declne monotoncally wth ncreasng memory load, as occurs for ndvdual feature values. Bays et al. (2009) examned how the dstrbuton of recall errors vared wth both the number of tems held n memory and the presentaton duraton of the memory array. The frequency of msreportng responses was not sgnfcantly affected by duraton of presentaton, but ncreased rapdly wth memory load. Interpreted n lght of the

9 P.M. Bays et al. / Neuropsychologa xxx (200) xxx xxx 9 present results, ths confrms that msbndng occurs wth ncreasng frequency as total memory load ncreases. A number of authors (Rensnk, 200; Wheeler & Tresman, 2002; Wolfe, 999) have proposed that encodng and mantenance of bndng nformaton may be partcularly dependent on the allocaton of vsual attenton. In the present study, the demands of the task would have encouraged an equal dstrbuton of attenton between all tems n the memory array; however, we have prevously shown that drawng vsual attenton to one array tem wth an exogenous cue (a bref flash) results n an advantage for that tem n terms of recall precson (Bays & Husan, 2008). Ths mples a role for attenton n determnng how lmted memory resources are dstrbuted wthn a vsual scene. If, as the present results suggest, mantenance of bndng nformaton s smlarly resource-lmted, we would expect attentonal manpulatons to have smlar effects on the frequency of msbndng Encodng lmtatons Whle msreportng errors were not nfluenced by exposure duraton n Bays et al. (2009), the proporton of trals attrbuted to the unform response component dramatcally declned as presentaton duraton ncreased, suggestng that these responses may reflect lmtatons on the speed wth whch vsual nformaton can be encoded rather than memory capacty. A prevous estmate based on performance n a change detecton task estmated the rate of encodng nto memory at 50 ms per tem (Vogel, Woodman, & Luck, 2006), or 300 ms for a 6 tem array. However, detectng changes n color of the magntude used n that task would have requred only a coarse representaton n memory. Our analyss of error dstrbutons n the color report task demonstrated that encodng of a sx tem array was stll n progress after 500 ms, and a small unform component was stll present even after 2 s (Bays et al., 2009). Consstent wth ths latter result, n the present study a small proporton of responses (<5%) was explaned nether by gaussan recall varablty nor by msreportng of non-target features. Crtcally, analyss of the jont dstrbuton revealed that these responses occurred separately n each feature dmenson, wth less than 4% of trals correspondng to unform responses for both color and orentaton (.e. UU n the full model descrbed n Table 2). If the unform response component s nterpreted as reflectng nstances where no nformaton s stored about the target feature (as n Zhang & Luck, 2008), these results mply that at least one feature was stored for 5.8 of the 6 memory array tems, and 5. out of 6 features were stored n each dmenson. Accommodatng these results wthn a slot or hybrd model of workng memory would therefore requre a substantal upward revson of the capacty lmt compared to typcal estmates (<3 tems, see Secton 4.), n addton to whatever modfcaton would allow ndependent allocaton of ths capacty n dfferent feature dmensons. Alternatvely, the small proporton of responses attrbuted to the unform component of the mxture model may reflect relatvely mnor sources of error not captured by the other (gaussan and msreportng) components of the model, e.g. ncomplete encodng, lapses of attenton, bases towards average or canoncal feature values, or devatons from a strct gaussan dstrbuton n recall varablty. Incomplete encodng may be one reason why a prevous study nvestgatng object ntegraton came to dfferent conclusons about ndependence of feature dmensons (Gajewsk & Brockmole, 2006). These authors nstructed subjects to explctly report whch conjuncton of color and shape they had observed at a probed locaton, out of a canoncal set of features. Trals on whch both responses were correct or both ncorrect were more frequent than expected under a smple assumpton of uncorrelated error across feature dmensons. However, unlke n the present study, t s not possble usng ths partal-report methodology to dstngush errors caused by nosy recall of a stored feature from those caused by guessng or msbndng. Furthermore, the memory array was presented very brefly (<200 ms) so these results may smply have reflected a falure to encode all the objects n the tme avalable. An mportant queston for shared-resource models of workng memory, not drectly addressed by the present study, s to what extent vsual features from dfferent dmensons tap nto the same memory resource. Prevous studes based on change detecton have typcally observed lttle or no performance cost when addtonal features are added to a memory array f they belong to a dfferent feature dmenson (Luck & Vogel, 997; Olson & Jang, 2002; Vogel et al., 200; Wheeler & Tresman, 2002), e.g. three colors and three shapes can be remembered as accurately as three colors alone (although there may be substantal errors n rememberng whch color belongs wth whch shape). These results have led to the concluson that dfferent feature dmensons recrut dfferent storage capactes. However, change detecton performance may be relatvely nsenstve to changes n recall precson (Bays & Husan, 2008; Wlken & Ma, 2004). A recent study (Fougne et al., 200) has reexamned ths queston usng a mxture model approach, as n the present study. These authors observed a sgnfcant, though modest (mean 2 ), ncrease n the standard devaton of the gaussan component of the model when a second set of features from a dfferent dmenson was added to the memory load. Addtonally, a small (mean 9%, equvalent to recall of 5.5 out of 6 tems) unform component was observed when features were dstrbuted between dfferent objects. These results suggest there may be some cost assocated wth mantanng multple feature dmensons, overlooked by prevous studes due to the use of change detecton methodology. However, the very small sze of these effects, despte doublng the total number of features n memory, does not appear consstent wth a model n whch features from dfferent dmensons share a sngle resource Conclusons The present results are dffcult to reconcle wth models of workng memory n whch only a subset of nformaton n each array s selected for storage (Alvarez & Cavanagh, 2004; Awh et al., 2007; Cowan, 200; Luck & Vogel, 997; Zhang & Luck, 2008). However, our fndngs can be straghtforwardly accommodated wthn resource-based accounts of vsual workng memory (Bays & Husan, 2008; Wlken & Ma, 2004). Because a shared memory resource can be dstrbuted equally between all the tems n an array, ths model does not predct that errors n dfferent feature dmensons must be correlated. Instead, as the total memory load ncreases, the fdelty of storage declnes. One consequence s that, as prevously demonstrated, ndvdual vsual features (e.g. color, orentaton, locaton) are recalled wth ncreasng varablty (Bays & Husan, 2008; Palmer, 990; Wlken & Ma, 2004; Zhang & Luck, 2008). The present results suggest that the bndng nformaton that groups features nto objects also becomes degraded wth ncreasng memory load, resultng n a systematc ncrease n the frequency wth whch ndependently stored features are ncorrectly combned. Acknowledgements Ths research was supported by the Wellcome Trust and the Natonal Insttute for Health Research Clncal Bomedcal Centre at Unversty College London Hosptals/Unversty College London.