SPECIAL ISSUE SEEKING THE NEURAL SUBSTRATES OF VISUAL WORKING MEMORY STORAGE

Transcription

1 SPECIAL ISSUE SEEKING THE NEURAL SUBSTRATES OF VISUAL WORKING MEMORY STORAGE Bradley R. Postle 1 *, T. Jason Druzgal 2, Mark D Esposito 3 ( 1 Department of Psychology, University of Wisconsin, Madison, USA; 2 Program in Neuroscience, University of Pennsylvania Medical School, Philadelphia, USA; 3 Wills Neuroscience Institute and Department of Psychology, University of California, Berkeley, USA) ABSTRACT It is widely assumed that the prefrontal cortex (PFC) is a critical site of working memory storage in monkeys and humans. Recent reviews of the human lesion literature and recent neuroimaging results, however, challenge this view. To test these alternatives, we used event-related fmri to trace the retention of working memory representation of target faces across three delay periods that were interposed between the presentation of each of four stimuli. Across subjects, only posterior fusiform gyrus demonstrated reliable retention of target-specific activity across all delay periods. Our results suggest that no part of frontal cortex, including PFC, stores mnemonic representation of faces reliably across distracted delay periods. Rather, working memory storage of faces is mediated by a domainspecific network in posterior cortex. Key words: working memory, prefrontal cortex, fusiform gyrm, short-term memory, fmri THE MULTIPLE MEANINGS OF WORKING MEMORY The concept of working memory in primate cognition is broadly construed as the capacity to retain information no longer present in the environment, to manipulate and/or transform this information, and to use it to guide behavior. Within different traditions of the behavioral sciences, however, the term working memory can have different specific meanings. In neuroscience, the modern study of working memory was arguably launched by Jacobsen (1935; 1936), who described a deficit on a test of delayed response following large bilateral frontal lobe lesions. The profound and enduring influence of Jacobsen s work is manifested in at least three ways. One has been the adoption of the delayed-response task and its variants e.g., delayed alternation, delayed recognition, reversal learning as the gold standard procedure for neuroscientific investigations of working memory. The basic procedure consists of first presenting a target stimulus, then imposing a delay during which the subject does not receive sensory information about the target, then prompting a response through which the subject reveals whether or not critical information about the target stimulus was retained across the delay period. In a properly controlled experiment, wrote Jacobsen (1936), information about the Cortex, (2003) 39,

2 928 Bradley R. Postle and Others target stimulus must be supplied by the subject either through some sustained activity during the period of delay or by recall from past experience (p. 12). A second influence of Jacobsen s work has been the assumption that this task requires, and thus serves as an index of, what he, in the parlance of the day, termed immediate memory (Jacobsen, 1936). (In neuroscience, the use of this term to refer to the temporary retention of information was superceded by the 1960s with the term short-term memory, and in the 1990s by the term working memory. 1 ) A third influence of Jacobsen s work is the idea that the frontal lobes, particularly the prefrontal cortex (PFC), are critical to working memory function (e.g., Stuss and Knight, 2002; Warren and Akert, 1964). (It is the details of this third idea that will be explored in this paper.) A quantum leap in the neuroscientific investigation of working memory occurred in the early 1970s when neurophysiologists recording from individual PFC neurons observed sustained activity throughout the delay period (Fuster and Alexander, 1971) that resembled the neural correlate of immediate memory that had been predicted by Jacobsen (1936) (as well as by Hebb (1949)). Independent of these developments in neuroscience, Baddeley and Hitch, in 1974, proposed a multiple-component model of working memory that has been vastly influential within cognitive psychology (Baddeley and Hitch, 1974). This model comprised, in simplified outline, two independent buffers for the storage of verbal and of visuospatial information and a Central Executive to control attention and the management of information in the buffers (Baddeley, 1986). Human working memory is widely viewed as a fundamental cognitive capacity that contributes critically to such high level cognitive functions as learning, reasoning, and language comprehension (Baddeley, 1992; Jonides, 1995). Since the introduction of this multiple-component model of human working memory, many cognitive psychologists have proposed alternative models that employ a wide variety of mechanisms to produce working memory behavior (e.g., Miyake and Shah, 1999). Beginning in the 1980s, Goldman-Rakic has suggested that the sustained delay-period activity studied by neuroscientists and the multiple-component system proposed by Baddeley and colleagues were cross-species manifestations of the same fundamental mental phenomenon (Goldman-Rakic, 1987, 1990) 2. This assumption has been widely accepted up through the present day, although precise specification of the most useful points of comparison awaits further maturation in both fields. The advent of neuroimaging in the 1990s gave cognitive neuroscientists the opportunity to investigate the neural bases of working memory functions. Some of these neuroimaging studies have 1 In the neuroscientific tradition, the term working memory was introduced in the early 1960s by Pribram (Miller et al., 1960; Pribram et al., 1964), who drew an analogy between the inferred mechanism that was disrupted by PFC lesions and the working memory built into contemporary computer simulations of human problem solving (Feigenbaum and Simon, 1961; Newell et al., 1958). However, this term didn t enter the common lexicon of the neuroscience of primate cognition (see footnote #2) until the 1980s (e.g., Goldman-Rakic, 1987; Petrides, 1989; Petrides and Milner, 1982). 2 Working memory is used in yet a different way in a third field within the behavioral sciences rodent learning and memory but this falls outside the scope of this brief review. For a helpful comparison of this third use of the term vs. its use in cognitive psychology, (see Becker and Morris, 1999).

3 Visual working memory storage 929 been motivated explicitly by models of human working memory. For example, Paulesu and colleagues (1993) and Awh and colleagues (1996) have tested predictions of the phonological loop model of verbal working memory (Baddeley, 1986) with positron emission tomography (PET). Other neuroimaging studies are best viewed as extensions to the human species of the neuroscientific study of sustained neural activity during delay tasks. For example, neuroimaging explorations of the what vs. where organization of human visual working memory (e.g., Courtney et al., 1997, 1998; Postle and D Esposito, 1999, 2000) have been directly influenced by single unit electrophysiological studies of sustained delay-period activity in monkeys (Rao et al., 1997; Wilson et al., 1993). The present study falls into this latter category. THE CONCEPT OF WORKING MEMORY STORAGE Although the neuroscientific study of working memory has emphasized sustained delay-period activity as the presumptive neural correlate of the temporary retention of information, considerable empirical and theoretical work has made clear that working memory function is supported by multiple distinct cognitive processes. In addition to the temporary retention of information, human working memory also permits the processing of this information via, for example, the shifting of attention among items held in working memory (Garavan, 1998; McElree, 1998), the inhibition of prepotent responses (Diamond, 1990), the mediation of proactive interference (D Esposito et al., 1999; Jonides et al., 1998), the updating of mnemonic representations (Kiss et al., 1998; Morris and Jones, 1990; Postle et al., 2001; Salmon et al., 1996), and coordination of multiple task performance (D Esposito et al., 1995). Such executive control processes, from the perspective of the multiple-component model of working memory, are the purview of the Central Executive (Baddeley, 1998; Baddeley and Logie, 1999). As discussed earlier, it has long been accepted by neuroscientists that most working memory behavior depends importantly on the PFC (e.g., Fuster, 1997; Goldman-Rakic, 1987; Stuss and Knight, 2002; Warren and Akert, 1964). An important question for contemporary cognitive neuroscience, however, is to determine which of the many cognitive functions that can contribute to working memory performance are supported by PFC. The present study focused on the neural basis of the temporary retention of visual representations of stimuli by humans. We will refer to this hypothesized function as working memory storage, in keeping with our earlier use of this terminology (D Esposito and Postle, 1999, 2000; Postle et al., 1999). This function, presumed to be homologous to the sustained delay-period activity recorded from the neurons of monkeys, might also correspond to specific elements of many theoretical models of working memory in contemporary cognitive psychology and cognitive neuroscience, including: the visual cache subcomponent of visuospatial working memory in the multiple-component model (Baddeley and Logie, 1999; Logie, 1995); the visual store represented in

4 930 Bradley R. Postle and Others the EPIC architecture (Kieras et al., 1999); or the activation of subsets of longterm memory representations as it is variously implemented by the embeddedprocesses model (Cowan, 1988, 1999), the controlled-attention model (Engle et al., 1999), the ACT-R cognitive architecture (Lovett et al., 1999), and the biologically based computational model of O Reilly and colleagues (O Reilly et al., 1999). THE NEURAL BASES OF WORKING MEMORY STORAGE In the neuroscience tradition of working memory research there are two dominant views about the neuroanatomical basis of working memory storage. According to one class of models, PFC supports working memory storage during delay periods (Courtney et al., 1997; Goldman-Rakic, 1987; Smith and Jonides, 1999). A second class of models posits that stimulus representations may be stored across delay periods in posterior cortical regions, whereas PFC s working memory-related functions support extramnemonic executive control operations, such as maintaining task set, manipulating or transforming mnemonic representations, and using and the information held in working memory to organize behavior (D Esposito et al., 1999; Petrides, 1994, 2000). One strategy for determining which brain regions make a necessary contribution to working memory storage is to assess the retention of items in working memory across delay periods that are interrupted by distracting, taskirrelevant stimuli. The logic is that, whereas delay-period activity associated with simple delayed-recognition tasks is commonly observed in many brain regions, much of it likely reflects the operation of processes either redundant with or unrelated to the storage operations that are necessary for successful task performance. Distracting events inserted between target and probe, however, can reveal which of these regions make a critical contribution to memory storage, because only in these regions will delay-period activity be sustained despite the presentation of the distractors. (A loss of delay-period activity that is attributable to a distracting stimulus is assumed to reflect disruption of the memory trace (Miller and Desimone, 1994).) For example, important evidence has come from electrophysiological studies in which monkeys performed ABBA or ABA variants of the delayed-recognition working memory task, in which distracting stimuli ( B ) were interposed between target (first A ) and probe (second A ) stimuli. Experiments employing pictures of common objects and recording in inferotemporal cortex (IT, Miller and Desimone, 1994), and employing spatial memoranda and recording in posterior parietal cortex (PPC, Constantinides and Steinmetz, 1996), found that neurons in these posterior regions that demonstrated sustained activity across single, undistracted delay periods lost this sustained activity upon the presentation of interfering stimuli on distraction trials. With the same object working memory procedure employed by Miller and Desimone (1994), however, PFC units have been identified whose sustained activity across delay periods was not disrupted by the interposition of distracting items (Miller et al., 1996). Comparable properties have also been reported in PFC neurons of animals performing a spatial working memory task (di Pellegrino and Wise,

5 Visual working memory storage ). These data have been interpreted as evidence that, whereas working memory representations may be retained across broadly distributed cortical networks on simple tests of delayed-recognition, only PFC is capable of maintaining these representations in the presence of distraction. They therefore represent important support for the view that PFC is critical for working memory storage. Recently, however, data inconsistent with this model of PFC function in the monkey have been presented by Petrides (2000), who demonstrated a doubledissociation of working memory functions attributable to PFC vs. anterior IT cortex: Lesions of PFC did not impair memory for the selection of one among two object stimuli across long (90 and 120 sec) delay periods, but did disrupt memory for one from among a set of three, four, or five items across shorter (10 sec) delays; and lesions of anterior IT cortex had the converse effect. These results are consistent with the alternative view that object working memory storage depends on IT cortex, whereas control functions like the monitoring of multiple mnemonic representations are supported by PFC. Thus, the literature on the working memory functions of PFC in the monkey currently presents an equivocal picture. The human literature, too, presents a mixed picture with respect to these two models. Consistent with the view that frontal cortex 3 (FC) is important for working memory storage are the results of the fmri studies of Courtney and colleagues (1998, 1997), which identify delay period-spanning sustained activity only in FC. A study that employed a variant of the ABBA design has also highlighted an important role for FC in detecting target memoranda among equally familiar distractors (Jiang et al., 2000), although its design did not permit an analysis of storage-specific mechanisms. Consistent with the alternative view, in contrast, are the results of two fmri studies finding that delay-specific load-sensitive activity for letter stimuli (characteristic of the operation of a storage function) is localized reliably in left posterior perisylvian cortex, but not in PFC (Postle et al., 1999; Rypma and D Esposito, 1999), and of two fmri studies indicating that the function of PFC during the delay period may be one of response selection, not of working memory storage (Pochon et al., 2001; Rowe et al., 2000). Additionally, a meta analysis has indicated that patients with PFC lesions typically are not impaired on tests of digit span and spatial span (D Esposito and Postle, 1999), suggesting that working memory storage functions are independent of PFC integrity. THE PRESENT STUDY To adjudicate between these two alternative views we conducted an eventrelated fmri study of ABBA performance in five human subjects, the design of which incorporated the critical features of the monkey electrophysiological 3 Many neuroimaging studies of human working memory have identified activity associated with particular stimulusand trial types only in non-pfc regions of FC (e.g., Courtney et al., 1997, 1998; Jiang et al., 2000). This was also our experience in the present study, and so our analyses incorporated all of FC.

6 932 Bradley R. Postle and Others Fig. 1 Top panel. A schematic representation of a four-stimulus trial of the behavioral task. On two- and three-stimulus trials the End message appeared at times 12 sec and 20 sec, respectively. Bottom panel. A schematic representation of the analysis model corresponding to this trial. Short bars represent unmodeled epochs of the task; tall bars represent impulse (or stick function ) covariates that were convolved with the HRF to yield the final covariate set. An HRF was derived empirically for each subject and this figure illustrates a representative HRF; HRFs depicted in thicker lines correspond to delay-specific covariates for Delay 1, Delay 2, and Delay 3. studies (Constantinides and Steinmetz, 1996; Miller et al., 1996), including the ability to isolate delay-specific activity (Figure 1; also see Materials and Methods). The logic of our study was that, whereas neuroimaging studies of delay tasks routinely identify delay-period activity in multiple brain areas, the insertion of distracting stimuli into a trial might serve to weed out activity that is not critical to the retention of the target stimulus. It is worth noting that we did not design our task so that performance would vary greatly across conditions. Rather, we intended that performance be relatively high in all conditions so as to simplify comparisons of brain activity associated with each. We chose to use faces as stimuli in our delayed-recognition task because visual perception of faces reliably activates a region of the inferior temporal cortex, the fusiform gyrus (FG), reliably (Kanwisher et al., 1997), and would thus facilitate the identification of a stimulus-sensitive region of interest (ROI) in posterior cortex 4. Working memory for face stimuli is also commonly associated with activity in FC: in PFC (Courtney et al., 1996), and in adjacent regions of premotor (PMC) and insular cortex (Jiang et al., 2000). Thus, in this experiment we tested two hypotheses: 1) working memory storage of face information is mediated in FC; and 2) working memory storage of face information is mediated in FG. 4 Posterior FG in humans, in the vicinity of the temporooccipital junction, may be homologous to IT cortex in the monkey (Kirchhoff et al., 2000), in that it is supports comparable object recognition functions. Our use of this experimental strategy did not require us to make assumptions about whether face-evoked activity in the FG is related to face perception per se (Kanwisher et al., 1997) or rather to mental operations associated with subordinate classification of an object within a category with which the participant has expertise (Gauthier et al., 1999).

7 Visual working memory storage 933 MATERIALS AND METHODS Subjects Five healthy adults (two women; mean age = 21.5) recruited from the undergraduate and medical campuses of the University of Pennsylvania participated in this experiment. Stimulus Materials Stimuli were gray scale, full-face head shots of adults. Each stimulus was cropped so that only a small portion of the neck (but no clothing) was visible, along with face and hair (Figure 1). Stimuli were controlled such that only faces representing the same ethnicity, sex, and age group (defined subjectively) appeared on the same trial. No stimulus appeared on more than one trial. Nonrepetition of stimuli was important for our experimental design because neuropsychological (Milner, 1964) and neuroimaging (Stern et al., 2001) results suggest a more important role for PFC in tasks in which stimuli repeat than in which they don t. This pattern suggests a role for PFC in suppressing or otherwise mediating the potentially interfering effects of stimulus repetition (Milner, 1964; Stern et al., 2001), a control function whose operation could provide a confounding source of activity in our results. Behavioral Task At the beginning of each delayed face-recognition trial subjects viewed a target stimulus (1 sec) followed by a delay period (Delay 1; 7 sec) followed by a second stimulus (1 sec). On two-stimulus trials this second stimulus served as the probe. On three-stimulus trials the second stimulus was followed by a second delay period and a third stimulus (which served as the probe); on four-stimulus trials the third stimulus was followed by a third delay period and a fourth stimulus (Figure 1, top panel). Subjects responded to every post-target stimulus with a button press indicating whether the stimulus was a match (right thumb) or a nonmatch (left thumb) with the target. The end of each trial was indicated by the word End (1 sec), which appeared 3 sec after the offset of the probe. A total of 96 trials were presented to each subject. Trial type (two-, three-, and four-stimulus) occurred unpredictably and equiprobably. Half of the threestimulus trials featured identical faces on Stimulus 2 and Stimulus 3. Subjects were trained to respond nonmatch to Stimulus 3 in these instances, because it did not match the target (i.e., they were trained to ignore repetition of non-target stimuli). MRI Data Acquisition fmri scanning was conducted with a 1.5T scanner equipped with a fast gradient system for echo-planar imaging. High resolution sagittal and axial T1- weighted images were obtained in every subject, and a gradient echo, echoplanar

8 934 Bradley R. Postle and Others sequence (TR = 2000 ms, TE = 50 ms) was used to acquire data sensitive to the blood oxygen level-dependent (BOLD) signal. Scans of the working memory task were preceded by a scan in which we derived the hemodynamic response function (HRF) for each subject (Aguirre et al., 1998). General fmri Data Processing Raw fmri data preprocessing included the following steps, in order: reconstruction; sync interpolation in time to correct for the slice acquisition sequence; and motion correction with the six parameter, rigid body routine from SPM96b. Note that unlike PET data, which features a high degree of spatial coherency, or smoothness ( global flow, Friston et al., 1990), fmri data do not have inherently high spatial coherency (Zarahn et al., 1997a), and can thus be analyzed without imposing a higher degree of spatial smoothness on the data via exogenous smoothing. A noteworthy feature of unsmoothed fmri data sets is that the analyses can be performed in a massively parallel univariate manner, such that inferential statistical analyses of fmri time series (in the case of the present report, with the modified general linear model (GLM)) are performed independently at each voxel in the data set. In this way, the activity of individual voxels can be assessed for statistical significance, and can be interpreted in the same way as would be a significant local maximum in a data set to which exogenous smoothing has been applied. This approach highlights the importance of taking into account the number of statistical tests performed in the analysis in order to avoid inflation of the false positive rate of the resultant statistical map. We controlled α at p.05 in all t-maps with Bonferroni correction, which has been demonstrated to control false-positive rates at the level of.05 when applied to unsmoothed data analyzed with the method described in this report (Zarahn et al., 1997a). In this approach, Bonferroni correction cannot be viewed as too stringent (Postle et al., 2000), and the application of a cluster or regional extent criterion is not needed. The HRF, which characterizes the fmri response resulting from a brief impulse of neural activity (Boynton et al., 1996), was used to convolve independent variables entered into the modified general linear model (GLM, Worsley and Friston, 1995) that we used to analyze the results of the scans of our working memory task. The fmri time series data were filtered and adjusted as described previously (Postle et al., 2000). The principle of the fmri time series analysis was to model the fmri signal changes evoked by each stimulus presentation epoch with covariates comprised of BOLD HRFs (i.e., delta functions) spaced at 4 sec intervals along the reference function (corresponding to the 8 sec stimulus onset asynchrony (SOA), Postle et al., 2000; Zarahn et al., 1997b, Figure 1, bottom panel). The 8 sec SOA was selected because different event types in our experiment occurred in a temporally dependent manner (i.e., delay periods were always preceded and followed by a stimulus presentation epoch). The smoothness of the fmri response to neural activity allows fmri evoked responses that arise from temporally dependent events to be resolved on the order of 4 sec (Zarahn et al., 1997b). Thus, in our data, the parameter estimates associated with a delay-period covariate would not be contaminated by

9 Visual working memory storage 935 variance in the fmri time series attributable to either of the stimuluspresentation events that bracketed it. (An alternative approach, modeling the delay period with a boxcar or a train of HRFs, would not yield data appropriate for our hypothesis test, because the parameter estimates associated with such delay-period covariates would be sensitive to stimulus-evoked activity, and thus would not index delay-specific effects. Such an event-related fmri design would only be appropriate if different epochs within our task could be perfectly randomized (Dale and Buckner, 1997), which they cannot be in delayedrecognition tasks.) The least-squares solution of the GLM of the fmri time series data yielded parameter estimates associated with each covariate of interest. Differences in fmri signal (either between conditions or vs. baseline) were tested by computing t-statistics resulting from linear combinations of the covariates in question. All contrasts performed on the data of these subjects featured in excess of 1400 effective degrees of freedom. HYPOTHESIS TESTS Principal Hypothesis Test The principal hypothesis testing analyses treated the data from each subject as a case study (as contrasted with a group analysis, Postle et al., 2000). Because the contrasts performed on each subject s data featured greater than 1400 effective degrees of freedom they were sufficiently sensitive for this approach. Once individual subject data were analyzed, we planned to compare across cases in order to assess the reliability (i.e., the replicability) or variability of effects of interest. This approach is analogous to that used in electrophysiological studies in monkeys. The principal hypothesis test was performed as three nested contrasts. In the first, we identified Delay 1-specific activity across the entire brain with the contrast [Delay 1 baseline]. In the second, we identified Delay 2-specific activity that was restricted to Delay 1- specific voxels by computing the mapwise Delay 2 t-map with the contrast [Delay 2 baseline], and masking out all voxels that had not been identified with the Delay 1 contrast. (Note that the Bonferroni correction for this second t- map yielded a lower threshold for t-values corresponding to p.05, because the Delay 1 map contained many fewer voxels than the whole brain map, Figure 2, top panel.) In the third, we identified Delay 3 activity from trials in which the second and third stimuli did not match (i.e., ABCA trials) that was restricted to Delay 1- and Delay 2-specific voxels by computing the mapwise Delay 3 t- map with the contrast [Delay 3 baseline], and masking out all voxels that had not been identified with the second contrast. (Note that the threshold for significant t-values was still lower for this third contrast.) Thus the voxels surviving these three nested contrasts were voxels that demonstrated significant delay-period activity across all three delay periods. This principal hypothesis test was a conservative test, because the probability of identifying voxels by chance with this nested three-contrast procedure was ( ).

10 936 Bradley R. Postle and Others Fig. 2 Results of the principal analysis in FC and FG. The Delay 1 column illustrates the statistical maps identifying voxels in FC and FG with Delay 1-specific activity ([Delay 1 baseline]). Circles identify the voxels comprising the FC and FG ROIs used in the first follow-up random effects group analysis. Delay 1 ROIs in FC were located in: right hemisphere middle frontal gyrus (MFG, Brodmann s area (BA) 9/46) and inferior frontal gyrus (IFG, BA 44) in subject 1; left MFG (BA 8) in subject 2; left MFG (BA 9/46) in subject 3; right superior frontal sulcus (BA 8) in subject 4; and IFG (BA 44) and premotor cortex (BA 6) in subject 5. FG Delay 1 ROIs were located bilaterally in subjects 1, 3, 4, and 5, and in right hemisphere in subject 2. The Delay 2 column illustrates the statistical maps identifying voxels in FC and FG with Delay 2-specific activity ([Delay 2 baseline]) within the search space (translucent blue voxels) comprising the voxels that had been identified in Panel A. The Delay 3 column illustrates the statistical maps identifying voxels in FC and FG with Delay 3-specific activity ([Delay 3 baseline]) within the search space (translucent blue voxels) comprising the voxels that had been identified in the Delay 2 column. This Delay 3 activity can be interpreted as a candidate neural correlate of storage of the working memory representation of face stimuli. Such activity, identified with arrows, was seen in FC only in subject 1 (right MFG (BA 9/46)), and in FG bilaterally in subjects 1 and 5, in right hemisphere in subjects 2 and 3, and in left hemisphere in subject 4.

11 Visual working memory storage 937 Therefore we planned two follow-up analyses in the event that the principal hypothesis yielded positive results. First Confirmatory Analysis This was performed as a random-effects group analysis in which we assessed activity in each of three ROIs: FC, FG, and PPC. These ROIs were defined for each subject as the voxels within FC 5 (Brodmann s areas 9, 46, 45, 47, 8, and 6), FG (Brodmann s areas 19 and 37), and PPC (Brodmann s area 7) that were identified with the [Delay 1 baseline] contrast (Fig. 2, Delay 1 column). We extracted the spatially averaged fmri time series data from the voxels within each of these ROIs that exceeded the mapwise threshold for the [Delay 1 baseline] contrast, and calculated from them Delay 1, Delay 2, and Delay 3 effects. The metric of the resultant parameter estimates (or beta values) was mean percentage signal change. We also converted these data to t-values by dividing the parameter estimates by the residual error term from the GLM. Data in both metrics were then assessed with two-factor (ROI, Delay period) ANOVAs. In an analysis using a model-fitting approach, such as the present study, the effect size, rather than the raw data, is of principal interest. T-values derived as in the preceding paragraph can be used as normalized indices of effect size because the residual error term that makes up the denominator of the t-value is positively, linearly related to the same scaling factor (or gain effect ) that characterizes differences in overall BOLD signal intensity across scanning sessions (i.e., across subjects). Indeed, t-values may account for more unexplained intersubject variance than do percentage signal change measures (Postle et al., 2000), thereby increasing the sensitivity of random-effects group analyses. Here, we report results in both measures as a check on the reliability of the analyses. In order to assess whether quantitative differences in delay-evoked activity between ROIs was specific to the delay period, and therefore interpretable for our hypothesis test, or characteristic of all epochs of the trial, which would render this delay-specific analysis uninformative, we assessed Probe-evoked activity in the same Delay 1-evoked ROIs used in the first confirmatory analysis. The Probe epoch of the task contained both face-perception and motor-response components, and thus could be considered an assay of activity levels in the two ROIs in a condition that was independent of the delay-specific activity that was of principal theoretical interest in our experiment. Second Confirmatory Analysis This contrast was implemented to test for delay period activity that may have been stable across all three delay periods, but subthreshold in one or more of the contrasts assessing individual delay periods. It was performed as case studies. A 5 The FC ROI was this large of necessity because, as can be seen in the Delay 1 column of Figure 2, no one subregion of FC was reliably activated across all participants.

12 938 Bradley R. Postle and Others TABLE I Mean behavioral performance by trial type Accuracy (Proportion correct [SE]) Reaction Time (Msec [SE]) 2-stimulus 3-stimulus 4-stimulus 2-stimulus 3-stimulus 4-stimulus.929 [.026].821 [.026].827 [.044] [59.6] [47.2] [48.7] contrast that represented the effect of delay-period activity across all trials of the study [(Delay 1 baseline) + (Delay 2 baseline) + (Delay 3 baseline)] was generated for each subject, and thresholded at a mapwise a of p.05. (The + denotes mathematical addition, not the logical AND operator.) As with the Principal Hypothesis Test, the resultant t-maps were assessed for across-subject reliability. RESULTS Behavior. Analysis of the accuracy data (Table I) indicated that multi-delay trials were more difficult than single-delay trials, with the ANOVA revealing a main effect of trial type [F (2, 8) = 16.4; p < 0.005], and a planned contrast revealing a significant difference between the two-stimulus and four-stimulus conditions [F (1, 4) = 22.5; p < 0.01]. Accuracy on three- and four-stimulus trials, however, did not differ [F (1, 4) =.004; n.s.]. Reaction times (Table I) did not vary reliably as a function of trial type, the ANOVA indicating an absence of a main effect [F (2, 8) = 1.70; n.s.]. Most importantly, the relatively high accuracy in all three conditions assured us that we could compare the fmri data across different conditions without the concern that these measures might be confounded by grossly disparate levels of difficulty. fmri. All five subjects demonstrated reliable activity sustained across all delay-periods only in FG. To determine this, we first identified Delay 1-specific activity in each subject, noting in particular foci of activity in FC, and FG, as well as in PPC, which we selected as a posterior control region. The location of Delay 1-specific activity in FC (as well as in PPC) was highly variable across subjects (Figure 2, Delay 1 column). We reasoned that working memory storage of the target stimulus representation that was resistant to interference from Stimulus 2 would be represented as Delay 2-specific activity within a subset of the voxels that were active during Delay 1 (Figure 2, Delay 2 column), and further, that working memory storage of the target stimulus representation that was also resistant to interference from Stimulus 3 would be represented as Delay 3-specific activity that was restricted to a subset of the voxels that had been active during Delay 1 and Delay 2 (Figure 2, Delay 3 column). Five of five subjects demonstrated Delay 3 activity in FG that could be interpreted as a neural correlate of the memory trace of the target stimulus, an outcome whose binomial p =.03. In contrast, such activity was observed in FC in only one subject, and in PPC in none (Figure 2, Delay 3 column).

13 Visual working memory storage 939 Fig. 3 Results of the ROI-based confirmatory group analysis. Panel A. illustrates the mean delay-specific effects, calculated within each Delay 1-evoked ROI (see Figure 2), for each of the three delay periods. Error bars represent standard error of the mean, illustrating the difference between delay-evoked activity in FG and FC. Note that the pattern of greater delay-evoked activity in FG than in FC is the opposite of that seen with probe-evoked activity (see Results). Panel B. presents the same data, but with error bars representing 95% confidence intervals, illustrating that delay-evoked activity in the FC ROIs was not statistically different from 0 during Delay 2 and Delay 3. To confirm that these results were not arrived at spuriously due to low sensitivity (i.e., Type II error), we performed two follow-up analyses of these data. The first, an ROI-based random-effects group analysis, used the voxels identified by the Delay 1 statistical maps in each subject to define functionally derived ROIs in FC, FG, and PPC (Figure 2, Delay 1 column). We extracted from these Delay 1-specific voxels estimates of delay-specific activity for each of the three delay periods (Figure 3a). We extracted these estimates of activity in two formats: the magnitude of the parameter estimates associated with the Delay 1, Delay 2, and Delay 3 covariates in our statistical model (measured in mean percentage signal change); and these same parameter estimate values scaled by the residual error estimate from the same statistical model (measured in t-values). The results from analysis of the mean percentage signal change data confirmed that only the FG ROI demonstrated reliably elevated activity across all delay periods. Although delay-specific activity was significant in each ROI during Delay 1, activity in the FC ROIs was not reliably different from 0 during Delay 2 and Delay 3; nor was PPC activity different from 0 during Delay 3 (Figure 3b). An ANOVA performed on these data revealed main effects of ROI [F (2, 8) = 10.06; p < 0.01] and of delay [F (2, 8) = 5.67; p < 0.05], but no interaction [F (4, 16) = 1.78; n.s.]. Analysis of the t-value data, perhaps better suited for such random-effects analyses (see Experimental Procedures, and Postle et al., 2000), revealed qualitatively comparable results, with the exception that the ROI Delay interaction was significant in this analysis [F (4, 16) = 3.21; p < 0.05]. We interpret the main effect of ROI in the preceding analysis as evidence that delay activity in FG plays a functionally more important role than does

14 940 Bradley R. Postle and Others Fig. 4 Statistical map of the [(Delay 1 baseline) + (Delay 2 baseline) + (Delay 3 baseline)] contrast in two representative slices of each subject s data. This contrast represents a more lenient way of defining working memory storage that is reliable across the three delay periods. Voxels identified by this contrast, highlighted with arrows, occurred reliably only in right FG (5 of 5 subjects), and in left FG in all but subject 2. In FC, they were found in right MFG (BA 9/46) in subjects 1 and 3, also left MFG (9/46) in subject 3, left IFG (BA 47) in subject 4, and right IFG (BA 45) in subject 5. No FC voxels were sensitive to this contrast in subject 2. delay activity in FC. One possible caveat for this interpretation, however, is that disparities in the inherent signal-to-noise characteristics of these brain regions (that were independent of our experimental manipulations) may have contributed to the main effect of ROI. That is, if FG simply displays higher levels of evoked activity than FC in response to any experimental condition, then our results wouldn t tell us anything interesting about delay-period activity per se. This, however, was not the case: A comparison of probe-evoked activity in the same ROIs used in the preceding analysis revealed that FC activity was greater than FG activity (1.1 vs. 0.8 as measured in mean percentage signal change, 5.28 vs as measured by t-values). Therefore, it cannot be argued that the delayperiod results were due to a general disparity in the level of activity in these two regions. This disparity is only observed during the delay period, and can thus be interpreted as indicative of a functional difference in the delay-period activity of these two regions. The second follow-up analysis tested the possibility that voxels in FC other than those identified by the previous analyses may have demonstrated reliably above-baseline activity across the three delays, even though this activity was below statistical threshold of Delay 1-identifying contrasts, and thus that these voxels had been excluded from the subsequent analyses. Such a pattern of activity, for which we tested with the contrast [(Delay 1 baseline) + (Delay 2

15 Visual working memory storage 941 TABLE II Loci of FC activity in each delay period Subject Delay 1 Delay 2 Delay 3 1 right MFG (9/46) right MFG (9/46)# right MFG (9/46)# right IFG (44/45) right IFG (44/45) right PMA (6) right IFGpo (45) left MGF (9/46) 2 left MFG (9/46) left MFG (6) left MFG (6)# 3 right MFG (9/46) right MFG (9/46) right MFG (9/46) right PMA (6) left SFG (8) left MFG (9/46) left MFG (9/46) left PMA (6) 4 right MFG (9/46) right MFG (9/46) medial SFG (9) left IFG (47) 5 right IFGpo (45) right IFGpo (45)# right IFGpo (45) right PMA (6) left SFG (8) left SFG (8)# left PMA (6) left IFGpo (45) left PCG (6) # Indicates same location as the previous delay period; IFG, inferior frontal gyrus, IFGpo, inferior frontal gyrus, pars opercularis; MFG, middle frontal gyrus; PCG, precentral gyrus; PMA, premotor area; SFG, superior frontal gyrus. baseline) + (Delay 3 baseline)], could also be plausibly interpreted as a neural correlate of stable working memory storage across all three delay periods. Voxels identified by this contrast were located reliably only in right FG (5 of 5 subjects, binomial p =.03). They were also located in left FG in 4 of 5 subjects. The locus of FC activity, in contrast, was highly variable across four subjects and absent in the fifth (Figure 4). It is unlikely that the same mental process (i.e., working memory storage of faces) would be represented with such a high degree of heterogeneity across subjects. Finally, we assessed the pattern of delay-period activity in FC within the thresholded, unmasked statistical map associated with each delay period. The results, summarized in Table II, did not reveal any patterns across delay periods (e.g., no consistent shifts of activity between Delay 1 and subsequent delays). DISCUSSION The results of this study suggest that FC does not store mnemonic representations of faces across distracted delay periods. Rather, they suggest that this function is performed by FG. The model of working memory storage functions that emerges from the results of this and other human neuroimaging experiments is that they are mediated in a domain-specific way by discrete, segregated networks in posterior cortex. In contrast to faces, for example, working memory storage of verbal material is supported by left posterior perisylvian regions associated with language comprehension functions (Awh et al., 1996; Paulesu et al., 1993; Postle et al., 1999; Rypma and D Esposito, 1999), and visual working memory storage of spatial and object features of

16 942 Bradley R. Postle and Others stimuli is supported by posterior regions of the dorsal and ventral visual processing streams, respectively (Postle and D Esposito, 1999; Postle et al., 2000). At least three candidate models might explain this accumulating pattern of data: Working memory storage may be accomplished by 1) sustained activity in the same networks that process perceptual information about the to-beremembered stimulus; 2) the operation of domain-specific short-term memory buffers located proximally to these sensory networks; or 3) temporary activation (e.g., by attention) of the long-term memory representations that correspond to the memoranda (Awh et al., 1998; Cowan, 1988, 1999; Engle et al., 1999; Lovett et al., 1999; O Reilly et al., 1999) 6. Additional psychological and neuroimaging research is required to test these alternatives. How might the present results be reconciled with those from earlier physiological studies in monkeys and humans that have provided the empirical basis for the storage-in-pfc model? One possibility, which has been proposed elsewhere (Courtney et al., 1998; Ungerleider et al., 1998), is that working memory function is supported differently in the brain of the human than in the brain of the monkey. A second is that the idea that the function of delay-period PFC activity (as seen, e.g., in di Pellegrino and Wise, 1993; Miller et al., 1996) is one of storage or maintenance of mnemonic representations may need to be reconsidered. That monkeys with PFC lesions can retain object memories across 120 sec delay periods (Petrides, 2000) is logically inconsistent with the idea that PFC is a necessary neural substrate for working memory storage. And the possibility that posterior areas may support working memory storage in the monkey, as well as in humans, is allowed by the fact that spatial delay-period activity is qualitatively and quantitatively equivalent in PPC and in FC (Chafee and Goldman-Rakic, 1998). Several alternative explanations of the functional significance of PFC delay-period activity have been proposed recently, and are discussed below. In humans, too, neuropsychological studies suggest that PFC is not necessary for working memory storage (D Esposito and Postle, 1999). Additionally, the claim that only PFC demonstrates sustained activity during the delay periods of working memory tasks (Courtney et al., 1997) can be questioned on the methodological basis that the delay-period regressors employed in the multiple regression analysis of data from this study, once convolved with a model of the HRF (Courtney et al., 1997), were susceptible to contamination with stimulusevoked variance in the fmri signal. (See Materials and Methods, General fmri Data Processing, for a discussion of the use of discrete impulse functions vs. boxcar covariates to model the delay period of delayed-recognition tasks). Finally, many human neuroimaging studies, broadly consistent with the monkey literature (e.g., Chafee and Goldman-Rakic, 1998; Colby and Goldberg, 1999; Gnadt and Andersen, 1988; Sereno and Maunsell, 1998), have found evidence that activity in posterior cortex is sustained across delay periods in working 6 FG extends from the temporal pole into occipital cortex, and doubtlessly supports a broad heterogeneity of information processing functions. We do not intend to suggest that any portion of FG, including the temporooccipital portion that is the focus of this report, is specific, or dedicated to working memory storage of face stimuli. The fallacy of such one brain region equals one function logic has been discussed elsewhere (D Esposito et al., 1998; Sarter et al., 1996).

17 Visual working memory storage 943 memory tasks (e.g., Postle et al., 1999; Postle and D Esposito, 1999; Rypma and D Esposito, 1999; Zarahn et al., 1999). Our finding that FC (including PFC) does not govern working memory storage is consistent with an emerging view that this region s contribution to working memory function is to control task-related behavior via functions operating at a level that is abstracted from the processing of individual stimuli. Examples of these functions, none of which are stimulus specific, include control of attention (de Fockert et al., 2001; Knight et al., 1999), transformation of mnemonic representations from their encoded state (D Esposito et al., 1999; Petrides, 1994); abstraction across trials of patterns and regularities with which to guide behavioral set (Miller, 2000); response selection (Pochon et al., 2001; Rowe et al., 2000), and mediation of the effects of interference in working memory (D Esposito et al., 1999; Jonides et al., 1998; Milner, 1964; Stern et al., 2001). Acknowledgements. Supported by: American Federation for Aging Research and NIH grants NS01762 and AG B.R.P. received support from NIH grant AG00255 (to Virginia M.-Y. Lee, University of Pennsylvania Medical School) and a Vilas Young Investigator Award (University of Wisconsin-Madison). We thank John Curtin for helpful discussion of statistical analyses. REFERENCES AGUIRRE GK, ZARAHN E and D ESPOSITO M. The variability of human, BOLD hemodynamic responses. NeuroImage, 8: , AWH E, JONIDES J and REUTER-LORENZ PA. Rehearsal in spatial working memory. Journal of Experimental Psychology: Human Perception and Performance, 24: , AWH E, JONIDES J, SMITH EE, SCHUMACHER EH, KOEPPE RA and KATZ S. Dissociation of storage and rehearsal in verbal working memory: Evidence from positron emission tomography. Psychological Science, 7: 25-31, BADDELEY AD. Working Memory, London: Oxford University Press, BADDELEY AD. Working memory. Science, 255: , BADDELEY AD. The central executive: A concept and some misconceptions. Journal of the International Neuropsychological Society, 4: , BADDELEY AD and HITCH GJ. Working Memory. In GA Bower (Ed), The Psychology of Learning and Motivation, Vol. 8. New York: Academic Press, 1974, Ch. 2, pp BADDELEY AD and LOGIE RH. Working memory: the multiple-component model. In A Miyake and P Shah (Eds), Models of Working Memory. Cambridge, UK: Cambridge University Press, 1999, Ch. 2, pp BECKER JT and MORRIS RG. Working memory(s). Brain and Cognition, 41: 1-8, BOYNTON GM, ENGEL SA, GLOVER GH and HEEGER DJ. Linear systems analysis of functional magnetic resonance imaging in human V1. The Journal of Neuroscience, 16: , CHAFEE MV and GOLDMAN-RAKIC PS. Matching patterns of activity in primate prefrontal area 8a and parietal area 7ip neurons during a spatial working memory task. Journal of Neurophysiology, 79: , COLBY CL and GOLDBERG ME. Space and attention in parietal cortex. Annual Review of Neuroscience, 22: , CONSTANTINIDES C and STEINMETZ MA. Neuronal activity in posterior parietal area 7a during the delay periods of a spatial memory task. Journal of Neurophysiology, 76: 1352, COURTNEY SM, PETIT L, MAISOG JM, UNGERLEIDER LG and HAXBY JV. An area specialized for spatial working memory in human frontal cortex. Science, 279: , COURTNEY SM, UNGERLEIDER LG, KEIL K and HAXBY J. Object and spatial visual working memory activate separate neural systems in human cortex. Cerebral Cortex, 6: 39-49, COURTNEY SM, UNGERLEIDER LG, KEIL K and HAXBY JV. Transient and sustained activity in a distributed neural system for human working memory. Nature, 386: , COWAN N. Evolving conceptions of memory storage, selective attention, and their mutual constraints within the human information processing system. Psychological Bulletin, 104: , COWAN N. An embedded-processes model of working memory. In A Miyake and P Shah (Eds), Models of Working Memory. Cambridge, UK: Cambridge University Press, 1999, Ch. 3, pp