D. E. Meyer, S. T. Mueller, T. L. Seymour, D. E. Kieras. University of Michigan. Sponsored by the U.S. Office of Naval Research

Transcription

1 BRAIN LOCI OF TEMPORAL CODING AND SERIAL-ORDER CONTROL FOR VERBAL WORKING MEMORY REVEALED BY COMPUTATIONAL MODELING AND FOCAL LESION ANALYSIS OF MEMORY-SPAN PERFORMANCE* D. E. Meyer, S. T. Mueller, T. L. Seymour, D. E. Kieras University of Michigan Sponsored by the U.S. Office of Naval Research *Poster presented at the meeting of the Cognitive Neuroscience Society, San Francisco, Abstract published in the Cognitive Neuroscience Society Annual Meeting Program 2000 (p. 147), a supplement of the Journal of Cognitive Neuroscience. 1

2 Introduction Baddeley's (1986, 1992) tripartite model (Figure 1, left panel) of human verbal working memory (WM) has inspired many pioneering studies by cognitive neuroscientists (e.g., Awh et al., 1996; Cohen et al., 1994; D'Esposito et al., 1999; Grasby et al., 1993; Jonides et al., 1997, 1998; Paulesu et al., 1999; Petrides et al., 1993; Postle et al., 1999; Schumacher et al., 1996; Smith et al., 1998; Smith & Jonides, 1999). From results of PET and fmri neuroimaging, these studies have reached several tentative conclusions, as embodied in the right panel of Figure 1: The inferior posterior region of the left parietal lobe, Brodmann's area 40, implements a phonological buffer for temporary "pure" storage of coded verbal items. The anterior ventral lateral regions of the left frontal lobe, Brodmann's areas 44 and 6, implement a mechanism for (sub)vocal rehearsal. The anterior dorsolateral regions of the left frontal lobe, Brodmann's areas 9 and 46, implement executive control processes for managing verbal information. 2

3 Current Neurocognitive Theory of Verbal Working Memory Central Executive Executive Control Phonological Storage Buffer (Sub)vocal Articulatory Rehearsal Phonological Storage Buffer (Sub)vocal Rehearsal Figure 1. Left Panel--Baddeley s (1986, 1992) tripartite model of verbal WM with components for storage, rehearsal, and executive control. Right Panel--Hypothesized sites where components of the tripartite model of verbal WM are putatively implemented in the brain. 3

4 Some Functions of Executive Control for Verbal WM The hypothesized functions of executive control by dorsolateral prefrontal cortex (DLPFC) include temporal coding of items' serial order, manipulating verbal information, updating the contents of WM, and focusing attention on successive items (D'Esposito et al., 1999; Jonides et al., 1997, 1998; Postle et al., 1999; Smith et al., 1998; Smith & Jonides, 1999). Images of brain activity recorded during performance of n-back, item recognition, and some other verbal WM tasks appear consistent with these hypotheses. Yet despite the prominence and elegance of this neurocognitive theory, there are compelling reasons to suspect that it requires substantial revision and elaboration. Specifically, we claim that some control functions previously attributed to DLPFC are implemented instead by inferior posterior parietal cortex (IPPC) under some basic conditions. Our claims, as summarized below, have two complementary bases: detailed computational modeling of performance in the serial memory-span task, a fundamental paradigm of verbal WM (Baddeley, 1986); and careful analyses of empirical data obtained from patients with focal brain lesions in IPPC, DLPFC, and ventrolateral prefrontal cortex (VLPFC) during the memory-span and other serial-ordering tasks. 4

5 Summary of Present Claims The verbal serial memory-span task is NOT a "pure storage" task. Contrary to the prevailing neurocognitive theory based on the tripartite WM model (Figure 1), performance of the memory-span task is NOT typically mediated by processes analogous to the automatic recording and playback of speech items on an audio tape loop. Complex executive control processes manage the construction, linking, and updating of item and serial-order information during rehearsal and recall for the memory-span task. In right-handed individuals, these control processes are implemented by left inferior parietal cortex (IPPC, Brodmann's area 40), not by DLPFC. The left IPPC likewise implements some executive control functions for other verbal WM paradigms that entail serial list learning, such as the n-back task. The left IPPC is NOT simply a passive phonological buffer. 5

6 Neuropsychological Studies of DLPFC Functions in Memory-Span Tasks Some support for the present claims comes from neuropsychological studies of performance on the verbal serial memory-span task by patients who have focal lesions or other types of dysfunction in DLPFC. D'Esposito and Postle (1999) conducted a meta-analysis of eight neuropsychological studies that measured performance on the forward digit-span task of the WAIS-R. Across 115 patients who had damage in left DLPFC and/or other regions of prefrontal cortex (Figure 2), no study yielded a reliable decrement in the memory-span scores of such patients versus normal controls. D'Esposito and Postle (2000) have shown that Parkinson's Disease, which disrupts functions of DLPFC, leaves patients' forward digit spans essentially intact (Figure 2). Similarly, McDowell et al. (1998) have shown that bromocriptine, a D2 dopamine receptor agonist that facilitates DLPFC activity in brain-damaged patients, does not affect their forward digit spans significantly. 6

7 Implications of Spared Memory Span Despite DLPFC Damage Taken together, the essentially null effects of DLPFC damage and dopamine agonist modulation on verbal memory span have important implications. Insofar as the serial memory-span task engages executive control processes, they presumably are not implemented in DLPFC or other regions of prefrontal cortex, but rather in some other posterior brain region(s). 7 7 Forward Digit Span Normal Controls PFC- Lesion Patients Forward Digit Span Normal Controls Parkinson's Patients Figure 2. Left Panel--Composite map of PFC brain damage in patients who have virtually normal verbal serial memory spans (from D Esposito & Postle, 1999). Middle Panel--Mean forward digit spans of normal controls with traumatic brain injury to PFC (from D Esposito & Postle, 2000). Right Panel--Mean forward digit spans of normal controls and patients with Parkinson s Disease (from D Esposito & Postle, 2000) 7

8 Neuropsychological Studies of IPPC Functions in Memory-Span and other Serial Ordering Tasks Further support for our claims comes from neuropsychological studies of performance on the verbal memory-span and other basic serial-ordering tasks by patients who have lesions in inferior posterior parietal cortex. Shallice (1988) has surveyed eight single-case studies of damage to IPPC. In each case, the patient's verbal memory span is greatly reduced (Figure 3), whereas many other mental functions are intact. For example, one such case, K.F., has a memory span of 2.3 digits but normal long-term memory (Warrington & Shallice, 1969). More generally, Kimura (1982) and colleagues (Kimura & Watson, 1989; Mateer & Kimura, 1977) have found that patients with IPPC lesions perform poorly on various oral and manual serial-ordering tasks. They are slower and more error prone in reproducing short syllable sequences (e.g., "badaga"), short sequences of non-verbal oral movements (e.g., mouth opening, lip pursing, teeth clicking), and complex multisegmented manual movements (Figure 3). In contrast, their reproduction of single oral movements and repetitive single syllables (e.g., "bababa") is relatively intact. 8

9 Implications of Selective Deficits Caused By IPPC Lesions Taken together, the detrimental effects of IPPC lesions on performance of verbal memory-span and nonverbal serial-ordering tasks have important implications. Insofar as these tasks all require executive control for temporal coding and management of order information, it appears likely that these processes are implemented by IPPC Oral Movements Forward Digit Span Normal Controls IPPC- Lesion Patients Syllables per Second Normal Controls IPPC- Lesion Patients Percent Correct Normal Controls Manual Movements IPPC- Lesion Patients Figure 3. Left Panel -- Mean forward digit spans of normal controls and patients with IPPC lesions (based on Shallice, 1988). Middle Panel -- Mean syllable rates for reproductions of "badaga" by normal controls and patients with IPPC lesions (from Kimura & Watson, 1989). Right Panel -- Mean percent correct for reproductions of sequential nonverbal oral and complex segmented manual movements by normal controls and patients with IPPC lesions (from Kimura & Watson, 1989.) 9

10 Theoretical Interpretation of The Functional Roles Played by Inferior Posterior Parietal Cortex in Verbal WM To further understand how processes in inferior posterior parietal cortex may mediate verbal WM, memory-span performance, and other serial-ordering behaviors, our research has taken several additional steps: We have formulated computational models of verbal working memory and performance in the serial memory-span task (Kieras et al., 1999). Our models have incorporated complex executive control for temporal encoding, updating of verbal WM, and supervision of rehearsal and recall, which are all needed to perform memory-span tasks correctly. Through simulations based on our models, accurate qualitative and quantitative accounts have been obtained for observed memory-span and serial-position data in normal subjects. Deficient performance of the memory-span task by patients with IPPC lesions also can be approximated closely in at least some major respects with our models. The success of our models supports the present claims that, in all likelihood, the types of executive control assumed by them are implemented in IPPC. 10

11 Our computational models for the serial memory-span task are based on modeling with the Executive-Process Interactive Control (EPIC) architecture shown here. EPIC implements algorithmic information processing in an interactive brain-like manner. Like the brain, EPIC has multiple types of working memory stores, and distinct modules for perceptual, cognitive, and motor operations. EPIC's cognitive processor uses the contents of WM to control perception and action through production rules that "fire" in parallel like neural networks do, enabling performance of many tasks to be modeled realistically. The EPIC Architecture Schematic diagram of the EPIC architecture (Meyer & Kieras, 1999). 11

12 EPIC Model of Verbal WM in the Memory-Span Task With the realistic components of EPIC, we have formulated precise computational models of verbal WM that emulate a phonological-loop mechanism (cf. Baddeley & Hitch, 1974; Schweickert & Boruff, 1986; Waugh & Norman, 1965). Our models account quantitatively for performance of representative WM tasks (Kieras, Meyer, Mueller, & Seymour, 1999). For example, one prototypical case with which we have dealt especially is the serial memory-span task (see next page). In what follows, a generic version of this task is considered more fully, and empirical results from it are fit with simulated outputs from one of our EPIC models. To achieve this fit, the present model includes sets of production rules that execute a performance strategy with three complementary functions: storage, rehearsal, and recall. We have discovered that for these functions to succeed, they must be coordinated by a highly elaborate executive control process. Without such control, it is impossible to perform the memory-span task properly. This realization, and the empirical good fit of our model, have important implications for identifying and describing the functions of different brain regions that modulate verbal WM. 12

13 Generic Verbal Serial Memory-Span Task The verbal serial memory-span task modeled here has the following design: On each trial, 3 to 9 words are presented auditorily at a constant moderate rate. After the final word of a trial, participants hear a signal (BEEP) that prompts them to recall the presented word sequence in its original serial order. Ample time is allowed for recall, after which a new trial starts. Word sequences are constructed randomly from a small pool of words. No word is used more than once per trial, but words occur repeatedly across trials.... river...ticket...crayon BEEP river river ticket river,ticket,crayon 13

14 Memory-Span Performance Strategy Consistent with EPIC, storage and perceptual-motor processes for the memory-span task are assumed to involve auditory stimulus and articulatory response codes. Auditorily perceived stimuli are held in a phonological WM buffer. Individual phonological features of items decay randomly in an all-or-none fashion from this buffer, so subvocal articulation is used to reactivate the auditory perceptual processor, yielding fresh (covert) copies of the stored verbal information. For this purpose, EPIC s vocal motor and auditory perceptual processors serve as components of a strategic programmable phonological loop. Operation of the phonological loop is coordinated by an executive control process whose production rules use available auditory and articulatory mechanisms for storage, rehearsal, and recall of word sequences (see flowchart on next page). Auditory Processor Phonological Working Memory Executive Control Processes Vocal Motor Processor 14

15 Executive Control Processes in EPIC Model of Verbal WM Trial Start If first external item, tag as stimulus chain start Wait for external stimulus Tag as new If add chain empty, tag as add chain start If last external item, wait for recall signal Wait for recall signal and rehearsal complete Wait for covert input Tag as new If start item, tag as new rehearsal chain start Flowchart of the executive control processes and overall task strategy used by the EPIC model of verbal WM for performing the serial memoryspan task. The task strategy includes concurrent processes for rehearsal-chain and addchain construction, subvocal rehearsal, and final recall. Start recall process Wait for rehearsal complete Wait for recall complete If item in add chain, start rehearsal process Terminate all threads Trial done 15

16 Rehearsal Process in the EPIC Model Phonological codes for words are maintained in WM through rehearsal. Rehearsal is a cyclic process coordinated by executive control. The rehearsal process is implemented by a set of production rules that cause specific words to be articulated. The production rules for rehearsal keep track of two chains of items in phonological (auditory) WM: the rehearsal-chain and the add-chain. When a new stimulus item is perceived and stored, tags are generated that put a link to this item at the end of the add-chain (see flowchart on next page). Concurrently with construction of the add-chain, the rehearsal-chain is continuously cycled and rebuilt through covert articulation. A new rehearsal-chain is built by articulating the current rehearsal-chain followed by the current add-chain. 16

17 Rehearsal Process in EPIC Model of Verbal WM Rehearsal Start no Very first item? yes Subvocalize external code Subvocalize first item in rehearsal chain no More in rehearsalchain? yes Subvocalize next item no More inadd chain? yes Flowchart of the rehearsal process used by the present EPIC model of verbal WM for performing the serial memory-span task. Represented here are the operations performed during one cycle of rehearsal. These operations include, when need be, subvocalizing the first stimulus item on a trial, subvocalizing each item of the current rehearsal-chain in auditory WM, and then subvocalizing each item in the current add-chain. On trials with articulatory suppression, no rehearsal process would occur. Subvocalize next add chain item Rehearsal Done If required auditory item is missing at any point, guess intelligently, or clean up and exit 17

18 Recall Process in the EPIC Model On each trial, after the recall cue has been perceived and the current rehearsal cycle has been completed, our EPIC model attempts to overtly recall the items in the rehearsal-chain. These items are transferred individually and serially from phonological WM to EPIC s vocal motor processor (see flowchart on next page). Recall errors occur when phonological features or serial-order tags of items decay from WM before they can be recalled. Under the current task strategy, the recall process makes sophisticated guesses when it attempts to recall an item whose phonological features or serial-order tags have already decayed from WM. The sophisticated guessing involves a redintegration of features based on phonological similarity. The identities of missing serial-order tags are guessed on the basis of remaining item subsequences in the rehearsal-chain. 18

19 Final Recall Process in EPIC Model of Verbal WM Recall Start - Rehearsal Used Recall Start - No Rehearsal Say first item in rehearsal chain Say first item in stimulus chain More in rehearsalchain? no More in stimulus chain? no yes yes Say next item Say next item Recall Done Recall Done If required auditory item is missing at any point, guess intelligently, or clean up and exit Flowchart of the final recall process used by the EPIC model of verbal WM for performing the serial memory-span task. The left panel shows steps in recall after prior rehearsal has occurred on a trial. The right panel shows steps in recall if there has been no prior rehearsal (e.g., if articulatory suppression has precluded rehearsal). 19

20 Other Parsimonious Assumptions Of The Model The serial order of items is represented by supplementary tags that form implicit linked-list structures in the rehearsal-chains and add-chains. Phonologically similar items have more features in common. No inherent fixed limit exists on the number of items stored in WM. Limitations in phonological WM capacity stem from time-based decay. Distinct codes are used for items from external (overt) and internal (covert) sources. The decays of distinct phonological features and serial-order tags from WM are independent, all-or-none stochastic processes. Decay times have a log-normal distribution with two parameters: M, the median, and s, the spread. The values of M and s may differ for phonological features and serial-order tags, depending on their source (external presentation or internal articulation). 20

21 Simulation of Results from Past Memory-Span Studies Using our EPIC model of verbal WM, we have simulated results from several past studies with the serial memory-span task, including Baddeley (1966, Exps. 1-3; 1968, Exps. 1-3) and Longoni et al. (1993, Exps.1-2). Together, they reported 20 distinct observed percentages of correct sequence recall as a function of list length, word duration, and phonological similarity (see graph on right). Our model accurately predicts Proportion Correct (Observed) Observed vs Predicted Data Proportion Correct (Predicted) this large data set (R 2 = 0.90) on the basis of a few parameter values corresponding to a priori variations of word duration and phonological similarity across experiments. The model's goodness-of-fit confirms that its assumptions about WM storage, rehearsal, and executive control should be taken seriously. 21

22 New Constraints on Neurocognitive Theories of Verbal WM Our EPIC computational model for the serial memory-span task puts strong new constraints on neurocognitive theories of verbal WM. Basic assumptions of our model, which enable its good quantitative fit to data, suggest that executive control of temporal coding, serial ordering, and verbal WM updating is crucial in performing the memory-span task. Given their crucial nature, it is plausible that these control processes are actually implemented somewhere in the brain. Because prefrontal lesions have little, if any, effect on serial memory span, DLPFC is unlikely to be the site of this implementation, contrary to current neurocognitive theory (cf. Figure 1). Instead, a much greater possibility is that inferior posterior parietal cortex implements the executive control suggested by our model. If so, then this yields a novel account of why and how IPPC lesions drastically reduce serial memory span; that is, they disrupt certain specific types of executive control. 22

23 Prediction of Data from IPPC Patients Based on EPIC Model Pursuing the latter possibility further, we will apply our EPIC model of verbal WM to predict serial memory-span data from patients with IPPC lesions. This prediction entails the following steps: Our original normal EPIC model, which successfully fit results from healthy subjects, will be "lesioned" to disrupt its rehearsal and recall of serial-order information for the memory-span task, but not its retention of item information per se. Simulations with the lesioned and normal EPIC models will produce predicted functions of percent correct sequence recall versus list length, as well as predicted functions of percent correct item recall versus serial position in lists. Results from these simulations will be compared to those from IPPC patients who have performed the verbal serial memory-span task under representative conditions. The following graphs illustrate the types of prediction about patients memoryspan performance that our EPIC model will produce. More studies with IPPC patients must be done to test these predictions fully. 23

24 Simulated Memory-Span Data for IPPC Patients 1 1 Probability of Correct Recall Normal Lesioned Probability Correct Serial Position List Length Probability Correct Serial Position Left Panel -- Simulated percent correct serial recall of item sequences by normal and lesioned EPIC models. Right Panel -- Corresponding percent correct item recall versus serial position for the normal (top) and lesioned (bottom) models. 24

25 Conclusions A good fit between observed data from IPPC patients and simulated data from our "lesioned" EPIC computational model may further support the present theoretical claims. According to these claims, we can conclude that: The current neurocognitive theory of verbal working memory requires substantial revision and elaboration. In future elaborations of this theory, a major role must be assigned to inferior posterior parietal cortex for implementing executive control of temporal coding, serial ordering, and verbal WM updating. This additional specific role of IPPC must be understood as contributing significantly to performance of memory-span, n-back, and other verbal WM tasks that require on-line acquisition, rehearsal, and recall of serial lists. More research is needed to characterize other executive control processes and neural mechanisms that are implemented by prefrontal cortex and complement those in inferior posterior parietal cortex. 25

26 References Awh, E., Jonides, J., Smith, E, E., Schumacher, E. H., Koeppe, R., & Katz, S. (1996). Dissociation of storage and rehearsal in verbal working memory: Evidence from PET. Psychological Science, 7, Baddeley, A. D. (1966). Short-term memory for word sequences as a function of acoustic, semantic, and formal similarity. Quarterly Journal of Experimental Psychology, 18, Baddeley, A. D. (1968). How does acoustic similarity influence short-term memory. Quarterly Journal of Experimental Psychology, 20, Baddeley, A. D., & Hitch, G. J. (1974). Working memory. In G. Bower (Ed.), Recent advances in learning and motivation, Vol. VIII (pp ). New York: Academic Press. Baddeley, A. D. (1986). Working memory. Oxford, UK: Oxford University Press. Baddeley, A. D. (1992). Working memory. Science, 255, Cohen, J. D., Froman, S. D., Braver, T. S., Casey, B. J., Servan-Schreiber, D., & Noll, D. (1994). Activation of the prefrontal cortex in a nonspatial working memory task with functional MRI. Human Brain Mapping, 1, D'Esposito, M., & Postle, B. R. (1999). The dependence of span and delayedresponse performance on prefrontal cortex. Neuropsychologia, 37, D'Esposito, M., & Postle, B. R. (2000). Neural correlates of processes contributing to working-memory function: Evidence from neuropsychological and pharmacological studies. In S. Monsell & J. Driver (Eds.), Control of cognitive processes: Attention and Performance XVIII. Cambridge, MA: M.I.T. Press. D'Esposito, M., Postle, B. R., Ballard, D., & Lease, J. (1999). Maintenance versus manipulation of information held in working memory: An event-related fmri study. Brain and Cognition, 41, Grasby, P. M., Frith, C. D., Friston, K. J., Bench, C., Frackowiak, R. S. J., & Dolan, R. J. (1993). Functional mapping of brain areas implicated in auditory-verbal memory function. Brain, 116, Jonides, J., Schumacher, E. S., Smith, E. E., Koeppe, R. A., Awh, E., Reuter-Lorenz, P. A., Marshuetz, C., & Willis, C. R. (1998). The role of parietal cortex in verbal working memory. Journal of Neuroscience, 18, Jonides, J., Schumacher, E. S., Smith, E. E., Lauber, E. J., Awh, E.,Minoshima, S., &, Koeppe, R. A. (1997). Verbal working memory load affects regional brain activation as measured by PET. Journal of Cognitive Neuroscience, 9, Kieras, D. E., Meyer, D. E., Mueller, S., & Seymour, T. (1999). Insights into working memory from the perspective of the EPIC architecture for modeling skilled perceptual-motor and cognitive human performance. In A. Miyake & P. Shah (Eds.), Models of working memory: Mechanisms of active maintenance and executive control (pp ). New York: Cambridge University Press. Kimura, D. (1982). Left-hemisphere control of oral and brachial movements and their relation to communication. Philosophical Transactions of the Royal Society of London, B298, Kimura, D., & Watson, N. (1989). The relation between oral movement control and speech. Brain and Language, 37, Longoni, A. M., Richardson, A. T. E., & Aiello, A. (1993). Articulatory rehearsal and phonological storage in working memory. Memory & Cognition, 21, Mateer, C., & Kimura, D. (1977). Impairment of nonverbal oral movements in aphasia. Brain and Language, 4, McDowell, S., Whyte, J., & D'Esposito, M. (1998). Differential effect of a dopaminergic agonist on prefrontal function in traumatic brain injury patients. Brain, 121, Meyer, D. E., & Kieras, D. E. (1999). Precis to a practical unified theory of cognition and action: Some lessons from computational modeling of human multiple-task performance. In D. Gopher & A. Koriat (Eds.), Attention and performance XVII (pp ). Cambridge, MA: M.I.T. Press. Paulesu, E., Frith, C. D., & Frackowiak, R. S. J. (1993). The neural correlates of the verbal component of working memory. Nature, 362, Petrides, M., Alivisatos, B., Meyer, E., & Evans, A. C. (1993). Functional activation of human frontal cortex during the performance of verbal working memory tasks. Proceedings of The National Academy of Sciences, 90, Postle, B., Berger, J. S., & D'Esposito, M. (1999). Functional neuroanatomical double dissociation of mnemonic and executive control processes contributing to working memory performance. PNAS, 96, Schumacher, E. H., Lauber, E., Awh, E., Jonides, J., Smith, E. E., & Koeppe, R. A. (1996). PET evidence for an amodal verbal working memory system. Neuroimage, 3, Schweickert, R., & Boruff, B. (1986). Short-term memory capacity Magic number or magic spell? Journal of Experimental Psychology Learning, Memory, and Cognition, 12, Shallice, T. (1988). From neuropsychology to mental structure. New York: Cambridge University Press. Smith, E. E., & Jonides, J. (1999). Storage and executive processes in the frontal lobes. Science, 283, Smith, E. E., Jonides, J., Marshuetz, & Koeppe, R. A. (1998). Components of verbal working memory: Evidence from neuroimaging. Proceedings of The National Academy of Sciences, 95, Warrington, E. K., & Shallice, T. (1969). The selective impairment of auditory verbal short-term memory. Brain, 92, Waugh, N. C., & Norman, D. A. (1965). Primary memory. Psychological Review, 72,