Different states in visual working memory: when it guides attention and when it does not

Transcription

1 Review Different states in visual working memory: when it guides attention and when it does not Christian N.L. Olivers 1, Judith Peters 2,3, Roos Houtkamp 4 and Pieter R. Roelfsema 3,5 1 Department of Cognitive Psychology, Vrije Universiteit Amsterdam, Van der Boechorststraat 1, 1081 BT Amsterdam, The Netherlands 2 Department of Cognitive Neuroscience, Maastricht University, P.O. Box 616, 6200 MD, The Netherlands 3 Department of Vision and Cognition, Netherlands Institute for Neuroscience, an institute of the Royal Netherlands Academy of Arts and Sciences (KNAW), Meibergdreef 47, 1105 BA Amsterdam, The Netherlands 4 Department of Cognitive Biology, Otto-von-Guericke Universität, Leipzigerstrasse 44, Haus 91, Magdeburg, Germany 5 Department of Integrative Neurophysiology, Centre for Neurogenomics and Cognitive Research, Vrije Universiteit Amsterdam, The Netherlands Recent studies have revealed a strong relationship between visual working memory and selective attention, such that attention is biased by what is currently on our mind. However, other data show that not all memorized items influence the deployment of attention, thus calling for a distinction within working memory: whereas active memory items function as an attentional template and directly affect perception, other, accessory items do not. We review recent evidence that items compete for the status of attentional template that contains only one object at a time. Neurophysiological results provide insight into these different memory states by revealing a more intricate organization of working memory than was previously thought. Different states of mind Workingmemory(WM)(seeGlossary) is the cognitive ability that enables us to retain and manipulate relevant information for the near future. Its everyday use is endless, ranging from temporarily retaining an address until we have entered it in the in-car navigation system, to trying to bring to mind the colors of our home interior while looking at the different paint charts in the Do-It- Yourself store. Its capacity is limited, with estimates pointing towards a maximum of approximately four representations [1]. It has been proposed that not all WM representations have the same status [1 3]. On the one hand there seems to be information that is applied to the current task. This type of representation is active, within the focus of central executive processes, and is therefore directly available. On the other hand there seems to be information that is merely stored for later use. This type of representation is temporarily peripheral to the current mental manipulations but is held in WM to be available for later use. For example, people can keep two separate calculations in mind without interference. However, switching tabs takes Corresponding authors: Olivers, C.N.L. (cnl.olivers@psy.vu.nl); Roelfsema, P.R. (p.roelfsema@nin.knaw.nl) time suggesting that focusing executive attention on one calculation at least partially deactivates the other [2,4]. In this article we review behavioral and neurophysiological evidence that the distinction between active and accessory WM states also has consequences for perception. When the active WM representation is used not for cognitive tasks such as arithmetic but for a visual task instead, it becomes an attentional template that drives the selection of relevant visual information from the sensory input. In other words, whereas executive processes determine which memory representation is currently active, here we focus on how active memory representations determine which perceptual objects are selected. We will review evidence on the guidance of selective attention by active visual memory representations, whereas accessory memory items (AMIs), although remembered, have relatively little influence on visual selection. We will then address the distinct neural codes responsible for the difference between active templates versus accessory memory representations. Taken together, the evidence suggests that although WM can store multiple objects, observers can actively look for only one at a time. Working memory and visual search There are numerous situations in daily life in which WM and selective attention interact. We look for shoes that Glossary Attention: the act or state of selective concentration on a particular aspect of the environment. Accessory memory item (AMI): item in WM that is not relevant to the task at hand. Biased competition: model proposing that visual input competes for selective processing. Competition is biased in favor of task-relevant information matching task-related WM content. Orthogonal coding: representing different sources of information in uncorrelated (i.e. independent) coding patterns across the same population of neurons. Search template: WM representation of the target in a visual search task to which incoming information is matched. Working memory (WM): storage and manipulation of a limited amount of information for cognitive tasks /$ see front matter ß 2011 Elsevier Ltd. All rights reserved. doi: /j.tics Trends in Cognitive Sciences, July 2011, Vol. 15, No

2 Box 1. Guidance of search by the search template: neuronal mechanisms Models of visual search hold that a visual object stored in memory biases neuronal activity in visual cortex such that it promotes the selection of matching visual items during search [5,8,42]. For example, neurons in the prefrontal cortex carry signals related to the cue stimulus that needs to be memorized as the target for a later search as a persistent increase in their firing rate [35]. Lesions in the prefrontal cortex impair visual search if the search target varies across trials [26]. Neurons in the prefrontal cortex could feed back to visual cortex to also enhance the activity of the neurons that code the taskrelevant features in the episode preceding the presentation of the search array, enriching the representation of the search template in memory [43,44]. After the presentation of a search array, neurons in the visual cortex initially represent the features of the items present. This first visually driven response phase does not yet distinguish between target and distractors but it is followed by a later attentional selection epoch [45]. In the selection phase, the template representation interacts with the visual representation so that the neurons that code visual information that matches with the template enhance their response [43,46,47]. These neurons then also provide additional information about the target of search. For example, neurons in early visual areas enhance their response if their small receptive fields fall on the search target, and these cells therefore code the location of the search target with high precision [48,49]. These spatial selection signals can be read out in areas involved in the selection of motor programs including areas that are involved in the selection of an eye movement towards the target [50]. match a specific belt. We switch from channel to channel to find our favorite TV network. Such situations are collectively referred to as visual search. In the typical laboratory version, observers are instructed to look for a specific target object (as defined by a feature such as shape or color) among a number of irrelevant distractor objects [5]. Although visual search has traditionally been regarded as an attention task, some form of memory representation is required to specify what we are looking for. This representation is the search template (also called the attentional set) [5,6]. Extant theories of visual search have either explicitly or implicitly assumed that WM provides the store for representing the template, and that a WM representation is in principle sufficient to affect the deployment of attention [7 9] (Box 1). Recent studies have examined this assumption by requiring participants to hold information in memory while engaged in an unrelated visual search task. Figure 1a shows an example trial (from [10]). Observers were asked to remember a color and then search for a target shape among a number of distractor shapes. Crucially, one of the distractors in the search display could carry the remembered color. When it did, search was slowed indicating that it indeed attracted attention. Additional studies revealed that the memory-matching object attracts eye movements, can trigger relatively early lateralized electroencephalography (EEG) signals that have been associated with shifts of selective attention (i.e. the N2pc, Figure 1c [11,12]), and selectively engage frontothalamic circuits that can mediate attentional shifts [13] (see [14] for an overview). Taken together, these findings support the view that WM representations bias attention. However, other studies have failed to find memorydriven interference in visual search even though they used almost identical tasks [15 17]. In two of these studies [15,16], observers were asked to remember two objects on each trial (Figure 1b). One object, the search target, was immediately relevant for the subsequent visual search task, and the other accessory item had to be remembered for a later task. This time there was no additional interference from distractors that matched the AMI. Moreover, a subsequent version of the task (in which targets and distractors were dispersed in time rather than in space) revealed that whereas the search target generated strong EEG components indicative of attentional processing, a distractor matching the AMI generated an EEG pattern that was indistinguishable from nonmatching distractors [18] (Figure 1d). These findings suggest that a WM representation does not always induce an attentional bias. Different states of items in visual working memory Although this might seem trivial, it is worth noting that even when AMIs induce attentional biases these biases are not as strong as when observers are actually looking for the item [17,19,20]. Furthermore, when performing both a memory and a search task observers hardly ever confuse the two. During search, they do not generate more false alarms in response to a memory-matching search object even when it is drawn from the same category as the target [15,16,18]. This already indicates that looking for something involves more than memorizing it. So why do some items in WM bias attention but others only weakly or not at all? We believe that the findings, taken together, reflect a fundamental division in WM between different representational states. Figure 2 illustrates a model in which WM representations adopt different states (after [15 17,20,21]). When an item is necessary for an imminent visual task, it receives the status of search template. This means that it gains full access to the sensory input, through feedback connections to lower visual areas, and biases selection towards matching objects. Importantly, although WM as a whole can represent multiple items, the number of templates that can be active at a time is limited to only one. Consequently, accessory memory items remain in a dormant state that nevertheless supports accurate memory. We propose that memory items interact, such that the stronger the template, the weaker the influence of the accessory items on selection because the search template pushes them into a state with little access to the visual representations. This blocking of the accessory item might not always be complete, for example when the target representation is weak or the accessory item is particularly strong [22]. Our proposal is compatible with previous theories of the role of WM in visual search such as biased competition theory [8] and guided search [5] (Box 1) if the additional assumption is made that the top-down bias for search is limited to the template but is weak or nonexistent for accessory items. It is also consistent with reports that even below the capacity limit of four items, memory resources are not distributed evenly and that the next target of selection receives a privileged status [23]. The model explains why studies requiring participants to remember two new objects on each trial one search target and one accessory item found effects of AMIs to be weak or absent. The fact that the target is new on every trial means that observers have to keep a strongly active 328

3 [(Figure_1)TD$FIG] (a) Memory item(s) Search task Memory test Results Remember this Memory-matching distractor Look for diamond Which is the memory item? Mean search RT (ms) Unrelated Memory-matching Distractor type (b) (c) Look for this Remember this Memory-matching distractor Search target present? μv nm 2 4 AMI contra N2pc AMI ipsi Is this the memory item? Mean search RT (ms) Unrelated Memory-matching Distractor type (d) - 5 μv Regular distractor -200 ms 800 ms OSN AMI 5 μv Search target TRENDS in Cognitive Sciences Figure 1. Combined visual WM and visual search paradigms and the typical results. (a) Example trial from Olivers and colleagues [10,21] in which observers were first asked to remember a color and then search for a diamond. This was followed by a memory test. The remembered color could return as a distractor in the search display. When it did, response times were delayed relative to an equally unique but unrelated distractor. Reproduced, with permission, from [10] (data from their Experiment 1). Similar procedures were used by Soto and colleagues with the same results [14]. (b) An example trial from Downing and Dodds [15]. Here too a memory task was combined with a search task but now the search target changed from trial to trial. This meant that two objects had to be remembered on each trial: the search target and the item for the later memory test. Now the memory item did not slow search. Reproduced, with permission, from [15] (data averaged across their Experiments 1 and 2). Houtkamp and Roelfsema [16] used similar procedures with the same result. (c) EEG findings for the type of task illustrated in (a) with display items that matched the AMI and were either presented in the left or right visual field. There is a significant N2pc component contralateral to the memory-matching distractor (grey region) that is indicative of attentional capture. Reproduced, with permission, from [12]. (d) EEG findings for the type of task illustrated in (b) with search objects presented serially at fixation. Note that only the search target evoked EEG markers of attentional selection (occipital selection negativity) and memory template matching (P3b, not shown) whereas the EEG response evoked by distractors matching the AMI was indistinguishable from unrelated distractors. Reproduced, with permission, from [18]. template for it in WM that results in the suppression of the accessory items. By contrast, studies that observed stronger effects of the memory items on search consistently used the same search target from trial to trial. Under these conditions, search is likely to become automated. Automation involves the transition from explicit, effortful WM activity to more implicit and cognitively less demanding memory representations [24,25], such that with practice the search template can be at least partially offloaded to other systems [26]. This reduces the blockade of the other items in WM enabling them to affect search. Consistent with this hypothesis, it has recently been found that a memory-matching distractor interferes with search when the target remains unchanged from trial to trial but the same distractor fails to affect search when the target varies from trial to trial (and thus has to be remembered in addition to the accessory item [21]). Merely increasing the memory load from one to two items per se has no such effects [27]. Apparently, only when one of the items is given the clear status of template is the other item put at a low ebb. One search template at a time The fact that AMIs have at most a minor influence on visual selection does not necessarily entail that observers can only maintain one template. After all, the accessory item is typically irrelevant to the immediate task and hence there is no reason to maintain a template for it. The limit in the number of templates therefore needs to be tested in conditions where it would be beneficial to maintain more than one. Several studies have done this. In visual search, performance is slower and less accurate when observers are asked to simultaneously look for 329

4 [(Figure_2)TD$FIG] Visual working memory T = Target (template) A = Accessory memory item D = Visual distractor Retinotopic representation TRENDS in Cognitive Sciences Figure 2. Different states in visual WM. A visual WM model in which memory items can adopt different states [15,16,20]. When assuming the unique state of search template (T), of which there can be only one, memory items have direct influence on the representation of the visual input through feedback connections to visual cortex. The active template is thought to block other memory items from affecting selection by either suppressing them or causing them to be recoded. These accessory items (A) adopt a dormant state with little to no influence on sensory areas (here shown by only a weak interaction with a distractor (D) in the retinotopic representation of the visual search display. two targets than when people look for either target alone unless both targets share a distinctive property [28,29]. One study has directly estimated the number of templates that can be maintained if subjects store new templates on every trial [30] (if the same template is used over and over it can be delegated to systems other than WM). Participants searched a rapid stream of objects at central fixation for the potential presence of for example a red or a green object, or a shoe or a basket (with either none or one of these objects present in the stream). Performance was systematically worse than in a condition in which they only had to look for one specific object. The data were fitted to various models assuming zero, one or two target templates using signal detection theory to estimate the expected number of correct trials and false alarms given different template numbers. Across a number of experiments, the estimated number of templates ranged between 0.9 and 1.1 regardless of whether the subjects were looking for objects defined within the same dimension (e.g. a red or a green object) or with completely different defining properties (e.g. a red object or a shoe). The limit of one template also emerges from attentional capture studies. When observers are asked to look for a red target their attention is captured by red distractors but not by green or blue distractors [31,32]. Thus, observers employ a specific attentional set for red. However, when asked to look for a red or green target, then not only the red and green but also the blue distractors capture attention. Apparently, instead of maintaining two templates one for each of the relevant colors a single template for any color has to be employed. Neuronal mechanisms for the maintenance of the search template and the accessory items The results presented in previous sections converge to a straightforward conclusion: only one memory representation can serve as a search template, and this representation blocks attentional guidance by other memory representations. Which neuronal mechanisms are responsible for the storage of the search template and AMIs, and why does the accessory item not guide attention? Prefrontal areas probably play a pivotal role in maintaining and partitioning the task-relevant information in WM from information that is stored for later use [33]. Lesions in the prefrontal cortex impair performance if the target of a visual search varies from trial to trial but the search for a constant target is relatively spared [26]. One way to segregate items in memory would be to maintain them in separate divisions of frontal cortex but there are also different possibilities that rely on storage by changed synapses or in different phases of oscillation (Box 2). Most of the neurophysiological evidence pertains to the storage of information by mnemonic responses of neurons that maintain a trace of the previous stimuli in their sustained firing rate. Task-relevant and task-irrelevant items might for example be represented in different prefrontal areas or by different neuronal populations within the same area [34]. The neuronal population that represents the search template should be in a special position to provide feedback to lower visual areas, biasing the attentional selection to relevant information and providing a template to be matched to the incoming information [35] (Box 1). This feedback excites feature selective neurons in 330

5 [(Figure_3)TD$FIG] (a) Search template Accessory memory item Key: Inactive cell Cell with mnemonic activity Feedback connection to visual cortex Orthogonal (b) Fixation Cue 1 One-object delay Cue 2 Two-object delay Report 2 1 Key: Trace of cue1 Trace of cue2 Combined trace (c) 25 Fixation First cue One-object delay Second cue Two-object delay Firing rate (HZ) First object Key: A B C D Times (ms) (d) Average normalized response Best object Cue 1 One object delay Worst object (e) Best object Report Two-object delay Worst object TRENDS in Cognitive Sciences Figure 3. Orthogonal coding of working memories in prefrontal cortex. (a) The search template is represented by a set of active neurons (blue circles) that provides feedback to visual cortex (arrows). When the same item is stored as an AMI it is represented by an orthogonal set of neurons. Thus, the activity of a neuron in the search template representation of a particular object does not predict the activity of the same cell during the representation of the same object as accessory item (note that 50% of the active cells become silent and 50% of the silent cells become active during the switch between representations). (b) Behavioral task of Warden and Miller [39]. The monkey saw a sequence of two cues and had to report this sequence by making eye movements to the first and second cue in a search array. The panels illustrate the sequence of events observed in the frontal cortex. The encoding of the second cue pushed the first cue in an orthogonal accessory state (the purple cell illustrates that the representations of the two cues are overlapping). In the two object delay, both representations are simultaneously active. At the end of the delay the monkey prepares to search for the first cue and the original template for cue 1 is restored, whereas the second cue is now represented in an orthogonal accessory state. Now only the first cue representation provides feedback to the visual cortex, without interference of the second cue (reproduced, with permission, from [38]). (c) Example of single unit activity in the prefrontal cortex of a monkey (cell with orange box in B). The different traces show trials where the first item to be memorized was A, B, C or D. During the one-object delay the cell prefers cue D but the preferred first cue changes in the second delay period when an additional cue is stored in memory. (d) For all cells the cues were sorted according to the strength of the response evoked by first cue and then activity was averaged. Thus, by definition, cue activity (black) decreases with stimulus rank (x-axis). Note that activity in the first delay (cyan) also decreases with rank implying that neuronal activity during the first delay is predicted by cue-evoked activity. (e) The activity is again sorted according to the responses during the presentation of the first cue. Activity in the two-object delay is not predicted well by the activity evoked by cue 1 (purple line) but the original mnemonic code reappears when the monkey has to report cue 1 (cyan) by making an eye movement to this cue. Panels B, D and E were reproduced, with permission, from [39] and panel C from [38]. visual cortex causing these cells to also exhibit mnemonic activity and thereby enriching the representation of the template [36,37]. The prefrontal neurons that represent the accessory items might not provide feedback to visual cortex, so that these representations have comparatively little influence on visual representations and on selective attention (Figures 2 and 3a). To investigate the representation of multiple items in memory, Warden and Miller [38,39] required monkeys to remember two object cues (cue 1 and cue 2) that were 331

6 Box 2. Alternative neuronal mechanisms for the segregation of multiple items in memory In addition to representation by different neuronal populations, two other neural mechanisms have been suggested for the independent storage of multiple items in WM. These mechanisms are not mutually exclusive, and it is possible that the brain uses more than one mechanism for the important task of maintaining information that will become relevant at a later point in time while avoiding interference with currently relevant items in memory. Synaptic trace hypothesis: it is generally accepted that long-term memories are stored as altered patterns of synaptic weights. Recent work [51] indicates that items can also be maintained in synaptic traces for a short period of time (one or a few seconds) by increased intracellular calcium levels at a specific subset of synapses. Because there is no need for persistent spiking activity, these short-term synaptic memories do not interfere with other information that is stored by continuous firing or by reverberations in a neural circuit. In this way, accessory items could be maintained in a passive state that does not interfere with the search template that would then be represented by persistent firing. And yet, the representation of the accessory items differs from other items in long-term memory because these short-term synaptic traces can be converted back into an appropriate pattern of persistently firing neurons if the memory becomes relevant at a later point in time [52,53]. Oscillation hypothesis: it has been suggested that neurons can also encode multiple representations in different phases of a cortical oscillation [54,55]. Multiple memories might be stored in separate gamma-oscillation subcycles (40 Hz) that take place at different phases of a lower-frequency (5 12 Hz) theta oscillation [56]. That is, memory item A is represented by neurons firing in gamma cycle 1, memory item B by neurons firing in gamma cycle 2 and so on, and this sequential memory reactivation would repeat with every theta cycle. At present, it is not clear if memories are indeed coded at different phases of a theta oscillation. Remarkably, a recent study [57] reported that different memories occurred at slightly different phases of the high-frequency gamma oscillation. It is presently unknown if there are neuronal mechanisms that could use this phase shift to prevent interference between different items in WM. presented sequentially and separated by a delay. This was followed by a response of the animal as to which two items it had seen (for example a sequence of two eye movements to these items) (Figure 3b). Figure 3c shows an example Box 3. Neural dynamics of changing states in WM How do memorized items switch from passive to active states and back? Neuroimaging studies suggest that attentional mechanisms reactivate items in WM [3,58]. Bringing a memory item into the focus of attention elicits activity in the intraparietal sulcus [59]: an area known to play an important role in visual WM [60] as well as spatial and object-based attention [61]. Moreover, reactivating memory items that were recently relevant elicits more activity in IT (compared to items in a deeper state of dormancy) that in turn shows stronger connectivity with prefrontal and parietal areas [62]. Activity of frontoparietal networks has also been observed when items in WM are replaced or refreshed [63] or when attention is switched between items [64]. We recently observed changes in neuronal activity that occur when an AMI receives the status of attentional template during search (Peters, Roelfsema and Goebel, unpublished). Observers had to remember two objects (a house and a face) one for each of two consecutive search tasks. During the first search, the second object maintained in WM was accessory but it subsequently was turned into a template for the second search. This switch engaged frontal and parietal areas together with an activity change in corresponding facespecific and house-specific visual areas. The overlap in neuronal activity, observed when states in WM change in different tasks, indicates the control by a rather general and highly adaptive executive mechanism that adjusts representations of goal-related information upon changing task demands [65,66]. Box 4. Questions for future research How do accessory items switch into an active search template and back? Which neuronal mechanisms control these transitions and how do they work? How is interference from AMIs prevented? We discussed the role of different types of persistent activity, synaptic memory and oscillations but what is their relative importance? Changing the target from trial to trial is probably not the only way of pushing the accessory items into a dormant state. Can observers strategically decide to keep the target template active in WM to suppress other memory representations when search is expected to be difficult, for example because of complex displays [17,67] or brief presentation durations [11]? Some studies reported that under some conditions AMIs can even be rejected more efficiently during visual search [15,17]. Can these items be stored as a template for rejection? What is the time course of switching an accessory item into a search template and back? Some studies have shown that more time between the encoding of an AMI and the search task weakens its effect on attentional deployment [67,68]. Does this mean that more time to switch template allows for a more efficient block of the accessory item? The original distinction between central (active) and peripheral (passive) WM representations stems from experiments on mental arithmetic and counting tasks [2,4] in which people switch from one calculation to another without the need to look for specific information in the environment. Is the distinction between a search template and an AMI equivalent? Cowan [69] has recently argued that there is a focus of attention in WM that can contain more than a single item. Does this attentional focus map onto the sum of the search template and the accessory items? What is the contribution of visual cortical areas to the search template and the accessory items? Are the accessory items confined to frontal cortex and does persistent activity in visual cortex enrich the representation of only the search template? Does the present framework apply to tasks other than the visual search tasks that are the focus here? Does selection of a visual search template have consequences for memory representations outside the visual modality? neuron that was recorded in the prefrontal cortex during this task. The neuron was activated strongly by cue D and it maintained a trace of this cue in its persistent firing during the first delay. Remarkably, the mnemonic tuning of the cell changed after the appearance of the second cue because it now was more active in trials where A or B had been presented as cue 1. This dramatic change in mnemonic activity was typical across the population of prefrontal cells. The neuronal activity evoked by cue 1 was a good predictor of activity in the first memory interval (Figure 3d, black and cyan lines) but a poor predictor when cue 2 was also stored in memory (Figure 3e, purple line) as if the second cue pushed the representation of cue 1 into a different orthogonal form that coexisted with the memory representation of the second cue (see schematics in Figure 3a,b). Importantly, the information about the cues was not lost because it was still possible to decode both stimuli from the population of neuronal responses. These results demonstrate that the same memory item can be stored in prefrontal cortex in at least two orthogonal states (see also [40]). Albeit speculative, we propose that one of these states (the search template) exerts control over visual cortical representations (arrows in Figure 3a), whereas the other state (the accessory item) is without such an influence. One prediction of such an orthogonal 332

7 coding scheme would be that the transformation from the active into the passive state is reversed when the accessory item becomes relevant again, so that it can interact with the visual representations. This is precisely what was observed by Warden and Miller [39]; the original memory code for cue 1 re-emerged when this cue had to be selected in the search display as a target of an eye movement (Figure 3e, cyan line). Concluding remarks We conclude that WM items can be in different states. The techniques available to monitor brain activity (such as single cell recording, functional magnetic resonance imaging and EEG), in combination with available paradigms that can distinguish between WM states, might prove to be a stepping stone for a deeper understanding of this crucial cognitive ability. There are many remaining questions (Box 4). One obvious challenge for the future is to unravel the mechanisms that determine what is relevant and what is not, and that allow items to change between active and passive states. Traditionally, such operations have been attributed to central executive processes [41], and most probably the prefrontal cortex is again involved probably in combination with posterior networks as is discussed in Box 3. What we have argued here is that such processes make use of different representational states of WM. The proposed distinction between a search template and accessory items might prove to be only one of a rich variety of divisions yet to be made with respect to WM representations. Acknowledgements This work was supported by a NWO-VIDI grant ( ) to CNLO and a HFSP and a NWO-VICI grant ( ) awarded to PRR. The authors acknowledge Joel Reithler for useful comments on an earlier draft of the manuscript. References 1 Cowan, N. (2001) The magical number 4 in short-term memory: a reconsideration of mental storage capacity. Behav. Brain Sci. 24, Oberauer, K. (2002) Access to information in working memory: exploring the focus of attention. J. Exp. Psychol.: Learn. Mem. Cogn. 28, Nee, D.E. and Jonides, J. (2011) Dissociable contributions of prefrontal cortex and the hippocampus to short-term memory: evidence for a 3- state model of memory. NeuroImage 54, Garavan, H. (1998) Serial attention within working memory. Mem. Cognit. 26, Wolfe, J.M. (1994) Guided search 2.0. A revised model of visual search. Psychon. Bull. Rev. 1, Duncan, J. and Humphreys, G.W. (1989) Visual search and stimulus similarity. Psychol. Rev. 96, Pillsbury, W.B. (1908) Attention, Swan, Sonnenschein 8 Desimone, R. and Duncan, J. (1995) Neural mechanisms of selective visual attention. Annu. Rev. Neurosci. 18, Pashler, H. and Shiu, L.P. (1999) Do images involuntarily trigger search?. A test of Pillsbury s hypothesis. Psychon. Bull. Rev. 6, Olivers, C.N.L. et al. (2006) Feature-based memory-driven attentional capture: visual working memory content affects visual attention. J. Exp. Psychol.: Hum. Percept. Perform. 32, Dalvit, S. and Eimer, M. (2011) Memory-driven attentional capture is modulated by temporal task demands. Vis. Cogn. 19, Kumar, S. et al. (2009) Electrophysiological evidence for attentional guidance by the contents of working memory. Eur. J. Neurosci. 30, Soto, D. et al. (2007) Dissociating the neural mechanisms of memorybased guidance of visual selection. Proc. Natl. Acad. Sci. U.S.A. 104, Soto, D. et al. (2008) Automatic guidance of attention from working memory. Trends Cogn. Sci. 12, Downing, P.E. and Dodds, C.M. (2004) Competition in visual working memory for control of search. Vis. Cogn. 11, Houtkamp, R. and Roelfsema, P.R. (2006) The effect of items in working memory on the deployment of attention and the eyes during visual search. J. Exp. Psychol.: Hum. Percept. Perform. 32, Woodman, G.F. and Luck, S.J. (2007) Do the contents of visual working memory automatically influence attentional selection during visual search? J. Exp. Psychol.: Hum. Percept. Perform. 33, Peters, J. et al. (2009) Remembered but unused: the accessory items in working memory that do not guide attention. J. Cogn. Neurosci. 21, Carlisle, N.B. and Woodman, G.F. (2011) Automatic and strategic effects in the guidance of attention by working memory representations. Acta Psychol. 137, Olivers, C.N.L. and Eimer, M. (2011) On the difference between working memory and attentional set. Neuropsychologia 49, Olivers, C.N.L. (2009) What drives memory-driven attentional capture? The effects of memory type, display type, and search type. J. Exp. Psychol.: Hum. Percept. Perform. 35, Zhang, B. et al. (2010) Guidance of visual attention from working memory contents depends on stimulus attributes. Neurosci. Lett. 486, Bays, P.M. and Husain, M. (2008) Dynamic shifts of limited working memory resources in human vision. Science 321, Schneider, W. and Shiffrin, R.M. (1977) Controlled and automatic human information processing: I. Detection, search, and attention. Psychol. Rev. 84, Woodman, G.F. et al. (2007) The role of working memory representations in the control of attention. Cereb. Cortex 17, i118 i Rossi, A.F. et al. (2007) Top-down attentional deficits in macaques with lesions of lateral prefrontal cortex. J. Neurosci. 27, Soto, D. and Humphreys, G.W. (2008) Stressing the mind: the effect of cognitive load and articulatory suppression on attentional guidance from working memory. Percept. Psychophys. 70, Huang, L. and Pashler, H. (2007) A Boolean map theory of visual attention. Psychol. Rev. 114, Menneer, T. et al. (2007) Costs in searching for two targets: Dividing search across target types could improve airport security screening. Appl. Cogn. Psychol. 21, Houtkamp, R. and Roelfsema, P.R. (2009) Matching of visual input to only one item at any one time. Psychol. Res. 73, Folk, C.L. and Anderson, B.A. (2010) Target-uncertainty effects in attentional capture: color-singleton set or multiple attentional control settings? Psychon. Bull. Rev. 17, Eimer, M. and Kiss, M. (2010) Top-down search strategies determine attentional capture in visual search: behavioral and electrophysiological evidence. Atten. Percept. Psychol. 72, Soto, D. et al. (2006) Dividing the mind: the necessary role of the frontal lobes in separating memory from search. Neuropsychologia 44, Genovesio, A. et al. (2006) Representation of future and previous spatial goals by separate neural populations in prefrontal cortex. J. Neurosci. 26, Miller, E.K. et al. (1996) Neural mechanisms of visual working memory in prefrontal cortex of the macaque. J. Neurosci. 16, Ranganath, C. and D Esposito, M. (2005) Directing the mind s eye: prefrontal, inferior and medial temporal mechanisms for visual working memory. Curr. Opin. Neurobiol. 15, Pasternak, T. and Greenlee, M.W. (2005) Working memory in primate sensory systems. Nat. Rev. Neurosci. 6, Warden, M.R. and Miller, E.K. (2007) The representation of multiple objects in prefrontal neuronal delay activity. Cereb. Cortex 17, I41 I50 39 Warden, M.R. and Miller, E.K. (2010) Task-dependent changes in shortterm memory in the prefrontal cortex. J. Neurosci. 30, Sigala, N. et al. (2008) Hierarchical coding for sequential task events in the monkey prefrontal cortex. Proc. Natl. Acad. Sci. U.S.A. 105,

8 41 Baddeley, A. (1986) Working Memory, Oxford University Press 42 Hamker, F.H. (2005) The reentry hypothesis: the putative interaction of the frontal eye field, ventrolateral prefrontal cortex, and areas V4, IT for attention and eye movement. Cereb. Cortex 15, Chelazzi, L. et al. (1993) A neural basis for visual search in inferior temporal cortex. Nature 363, Tomita, H. et al. (1999) Top-down signal from prefrontal cortex in executive control of memory retrieval. Nature 401, Lamme, V.A.F. and Roelfsema, P.R. (2000) The distinct modes of vision offered by feedforward and recurrent processing. Trends Neurosci. 23, Bichot, N.P. et al. (2005) Parallel and serial neural mechanisms for visual search in macaque area V4. Science 308, Jiang, Y. et al. (2000) Complementary neural mechanisms for tracking items in human working memory. Science 287, Motter, B.C. (1994) Neural correlates of attentive selection for color or luminance in extrastriate area V4. J. Neurosci. 14, Moro, S.I. et al. (2010) Neuronal activity in the visual cortex reveals the temporal order of cognitive operations. J. Neurosci. 30, Schall, J.D. and Hanes, D.P. (1993) Neural basis of saccade target selection in frontal eye field during visual-search. Nature 366, Mongillo, G. et al. (2008) Synaptic theory of working memory. Science 319, Barak, O. et al. (2010) Neuronal population coding of parametric working memory. J. Neurosci. 30, Deco, G. et al. (2010) Synaptic dynamics and decision making. Proc. Natl. Acad. Sci. U.S.A. 107, Fell, J. and Axmacher, N. (2011) The role of phase synchronization in memory processes. Nat. Rev. Neurosci. 12, Panzeri, S. et al. (2010) Sensory neural codes using multiplexed temporal scales. Trends Neurosci. 33, Lisman, J.E. and Idiart, M.A.P. (1995) Storage of 7+/-2 short-term memories in oscillatory subcycles. Science 267, Siegel, M. et al. (2009) Phase-dependent neuronal coding of objects in short-term memory. Proc. Natl. Acad. Sci. U.S.A. 106, Lepsien, J. and Nobre, A.C. (2006) Cognitive control of attention in the human brain: Insights from orienting attention to mental representations. Brain Res. 1105, Nobre, A.C. et al. (2004) Orienting attention to locations in perceptual versus mental representations. J. Cogn. Neurosci. 16, Xu, Y.D. and Chun, M.M. (2006) Dissociable neural mechanisms supporting visual short-term memory for objects. Nature 440, Serences, J.T. et al. (2004) Control of object-based attention in human cortex. Cereb. Cortex 14, Nee, D.E. and Jonides, J. (2008) Neural correlates of access to shortterm memory. Proc. Natl. Acad. Sci. U.S.A. 105, Roth, J.K. et al. (2009) Similar and dissociable mechanisms for attention to internal versus external information. NeuroImage 48, Garavan, H. et al. (2000) A parametric manipulation of central executive functioning. Cereb. Cortex 10, Rao, S.C. et al. (1997) Integration of what and where in the primate prefrontal cortex. Science 276, Duncan, J. (2001) An adaptive coding model of neural function in prefrontal cortex. Nat. Rev. Neurosci. 2, Han, S.W. and Kim, M.S. (2009) Do the contents of working memory capture attention? Yes, but cognitive control matters. J. Exp. Psychol.: Hum. Percept. Perform. 35, Dombrowe, I. et al. (2010) The time course of working memory effects on visual attention. Vis. Cogn Cowan, N. (2011) The focus of attention as observed in visual working memory tasks: making sense of competing claims. Neuropsychologia 49,