COGNITION: MIND AND BRAIN. Chapter 5: Working Memory. Todd S. Braver

Transcription

1 COGNITION: MIND AND BRAIN Chapter 5: Working Memory Todd S. Braver REVISED DRAFT, JULY 2003: Please do not circulate or distribute without permission Please address correspondence to: Todd S. Braver Department of Psychology Campus Box 1125 Washington University Saint Louis, MO Phone: ,

2 NOTES: Still to Add: Key Terms, Study Questions, O-Span Demonstration box IMPORTANT STILL TO ADD: FIGURES!! I wasn t sure about how to approach this, and so just left a pointer with a brief description of what figure will have. I ll need guidance as to how to pick / find figs Questions that I had about cross-referencing and other issues are denoted like this: [?? QUESTION: ] 2

3 Introduction You re at the party and engaged in a lively conversation with some friends about a current movie that all of you have seen. Your friend Joe makes a point about the movie, and why he felt that one of the stars was not convincing in the role. You somewhat disagree with this point, and want to make your case. However, before you begin speaking, your other friend Mary makes a different point, indicating why she doesn t like the actor to begin with. You feel that your point is a good one, and so want to jump into the conversation. Yet you also know that if you interpret Mary before she s done with her point she will get offended. Moreover, you find yourself contemplating the things that she is saying, being drawn in (possibly because she triggers some other knowledge regarding the actor that you have stored in long-term memory, e.g., Chapter 4). Importantly, you also need to keep paying attention to what Mary is saying, because you don t want to be repeating some of the same points. So, your challenge is to keep in mind the points you want to make while at the same time listening and comprehending to what Mary is saying. Boy, is your working memory system being taxed! Hurry up, Mary, and finish talking so that you can jump in before you completely lose track of your point! As the above scenario indicates, there are many occasions in every day life when we need to keep critical information active in mind briefly, storing it until it can be used in an upcoming situation. Often in these situations, not only do we need to keep certain bits of information accessible in mind, but also need to perform cognitive operations on it, mulling it over, manipulating or transforming it. These types of short-term mental storage and manipulation operations are collectively termed working memory. A common way of conceptualizing 3

4 working memory is as a mental blackboard a workspace that provides a temporary holding store so that relevant information is highly accessible and available for inspection and computation. When such tasks are accomplished, the information can be easily erased so that the whole process can begin again. Working memory in everyday cognitive activities The prevalence and critical nature of such a working memory system to everyday cognition can be easily be appreciated by considering the diverse situations in which such a system seems to be important: Getting a phone number from information, then needing to dial that number after hanging up (especially if you are a cheapskate who doesn t want to pay for the automatic dialing service!) and having no pen to write the information down. You may find yourself, repeating the numbers internally as a way of making sure you don t forget. Getting lost while en route to a new destination, and receiving verbal directions from a helpful stranger. The directions are fairly complex, so you may try to form a mental map to encode them more reliably ( Take the first left, go straight for one mile, past the 7-11, then bear right, stop at the 4-way intersection, and pull into the driveway at the 3 rd building on left ). Reading text and trying to parse a fairly complex sentence. This may require mentally breaking the sentence into subparts in order to achieve full comprehension ( The wide receiver the quarterback was looking for was tripped by the safety. Who was tripped? ). 4

5 Playing chess or some other game of strategy. Deciding what move to make often involves a process of looking ahead imagining what the game situation would be like following a particular move, and how your opponent is likely to react in such a situation. Figuring out a tip at a restaurant. You want to tip at least 15%, but are too poor to afford much more than that. A strategy combining short-cuts, breaking the problem into parts, and basic mental arithmetic seems to do the trick. Yet a key problem is to be able to store the answers to intermediate computations and not confuse these with other types of information (the bill is $28.37, which is close to $30; 10% of 30 is $3 and half of that is $1.50; add them together and you get about $4.50). A computer metaphor for working memory A natural, and intuitively appealing model for how to think about the nature and structure of working memory is to start with the standard computer metaphor that dominates much of cognitive psychology. In the standard desktop or laptop computer, there are two means by which information is stored, the hard-disk and the RAM (actually, this is a simplification that ignores certain subtleties, such as graphics memory and other forms of hard-wired information that are part of the computer s hardware). In the computer-based model of cognition, the information stored in the hard-disk might be analogous to long-term memory, whereas the RAM might be analogous to working memory. As in long-term memory, the hard-disk is the means by which information is stored permanently in a stable and reliable form. All software programs and the operating system of the computer get stored on the hard disk. Yet when this stored information needs to be utilized, in order to run the software, the program must be retrieved from the hard- 5

6 disk and loaded into RAM (the same happens for the programming code that runs the operating system, such as Windows). The notion of working memory being like a temporary work-space is accommodated nicely in this analogy, since RAM is completely flexible and reset when the task being executed by the program is finished, or when the software program is closed. Interestingly, the analogy to computers also suggests two further characteristics of working memory. First, RAM is completely flexible with regard to content. That is, there is no fixed mapping between the location (address) or part of RAM and the program that utilizes it; any software program can access any particular part of RAM. Second, the more RAM a computer has, the more complex and sophisticated programs can be run on it, and the more programs that can be running simultaneously (indeed, this is something that has fueled the ever accelerating pace of RAM chip development, and computer manufacturers zeal to get consumers to continually buy the latest technology). Thus, if we adopt the computer-based metaphor of working memory, we might postulate that storage in working memory involves a content-free flexible buffer, and that cognitive abilities will be dependent upon the size of the buffer. Yet it is not clear that these characteristics of computer-like working memory fit well with actual human working memory structure and function. In this chapter, you will be able to survey some of the evidence and decide for yourself. However, as you will also discover, the answers to the questions raised here are just beginning to emerge, so we will not promise you an easily resolution. Nevertheless, the study of working memory is at an exciting phase at this point in time, and as you will see cognitive neuroscience approaches have in many ways revolutionized the types of questions that can be asked, and the insights that have been gained about how working memory works. 6

7 [?? INSERT FIGURE 1 HERE: THIS WILL BE A DIAGRAM OF THE COMPUTER METAPHOR OF WORKING MEMORY SHOWING ANALOGY BETWEEN MEMORY AND COMPUTER COMPONENTS Real world implications of working memory It is also worth mentioning here that a better understanding of the nature of human working memory will not just add to our general basic knowledge of human cognitive psychology, but it also may have very important implications for understanding why individuals differ in cognitive skills and ability, and their success or failure in accomplishing real-world goals. Research into working memory has suggested that there are large variations among individuals in working memory capacity (Daneman & Carpenter 1980), and that such individual differences predict things like general intelligence (as measured by standard IQ tests), verbal SAT scores, and even the speed in which a new mentally demanding skill such as computer programming is acquired (Kane & Engle 2002, Kyllonen & Christal 1990). In this type of research, individuals are given simple tests to determine their working memory span (?? See BOX for demonstration). Based on these simple tests, researchers have found that individuals with higher working memory spans also tend to score better on IQ tests and other general ability measures, to solve novel problems more effectively, and to learn new skills more quickly. Thus, having a good working memory appears to increase the ease with which you achieve success in obtaining real-world goals, such as being admitted to a good university or acquiring new cognitive skills. As our examples above indicate, such a relationship between working 7

8 memory and everyday cognitive ability may not be surprising given how pervasively working memory affects such a wide range of complex cognitive tasks. Yet the question still remains, why do individuals differ so widely in working memory capacity, and where exactly do the differences lie? If we understood more precisely the components of working memory, and which part is the most critical for real-world cognitive success, we might be able to develop methods to train and exercise our working memory in a manner that could improve its function, and consequently enhance our cognitive repertoire. On that note, we begin our discussion of current research in working memory. However, in order to understand the current state of research in this area, it is first important to gain a perspective on how current conceptions of working memory have evolved from earlier ideas in cognitive psychology. Such a perspective will enable us to better contextualize the theoretical framework of working memory that currently dominates the field. We will describe this framework in detail, and the evidence supporting it. From there we will move to see how cognitive neuroscience approaches have helped to better elucidate the underlying mechanisms of working memory that is, how working memory works. From short-term memory to working memory: A brief history Ideas regarding the nature and function of short-term storage have evolved considerably over the last hundred years within cognitive psychology. We can see that the terms for this storage system have also changed over the years, from primary memory to short-term memory to working memory. Let s take a brief trip through this history, in order to better understood how and why conceptualizations have changed. 8

9 William James: Primary Memory and Consciousness The idea that there is a distinct form of memory that stores information temporarily in the service of on-going cognition is not a new one. Indeed, the first discussion of a distinction between short-term and long-term storage systems was put forth by the pioneering American psychologist William James in the late 19 th century. James labeled these two forms of memory primary memory and secondary memory (James 1890). For James, these terms indicated the relationship of the information storage to consciousness. According to James, primary memory was the initial repository by which current perceptual experiences could be stored and made immediately available to conscious attention, inspection, and introspection. As such, James suggested that such information would be continually accessible. In his words, an object of primary memory is thus not brought back; it never was lost. This contrasts with the long-term storage system, for which stored information cannot be accessed without an active retrieval process being initiated. The link between working memory and consciousness remains a central part of most current thinking. Yet the answer to the question of whether we are conscious of the entire contents of working memory is still open to debate. Conversely, it may also not be true that everything we are conscious of should be considered to be part of the contents of working memory. As we will discuss later, some current models suggest that only a subset of working memory is consciously experienced (Cowan 1995). Early Studies: The characteristics of short-term memory Despite this early theorizing by James regarding the cognitive system for short-term information storage, experimental studies of the characteristics of this system did not take place until the 1950s. [?? ADD AS FOOTNOTE: Mention that this gap was because of dominance of behaviorist views in early 20 th century. Focus away from cognitively based theories]. 9

10 George Miller (Miller 1956) provided detailed evidence that the capacity for short-term information storage is limited. In what has to be one of the most provocative and entertaining opening paragraphs of a cognitive psychology paper, Miller declared My problem is that I have been persecuted by an integer this number has followed me around, has intruded in my most private data, and has assaulted me from the pages of our most public journals. The number assumes a variety of disguises, being sometimes a little larger and sometimes a little smaller than usual, but never changing so much as to be unrecognizable. The title of the paper was the The Magical Number Seven, Plus or Minus Two. Miller suggested that individuals can only keep around 7 items active in short-term storage, and that this limitation influences their performance on a wide range of mental tasks. What data supported Miller s claim? Tests of short-term memorization such as repeating back series of digits (a digit span test) show that regardless of how long the series, the correct recall of digits appears to plateau at about 7 items (Guildford & Dallenbach 1925), though for some individuals this number is lower and some it is higher (we return to discuss these individual differences at the end of the chapter). However, a critical point that Miller made is that although there is a limitation in the number of items that can be simultaneously held in short-term storage, what defines an item is highly flexible, and subject to strategic influences. Specifically, Miller suggested that single items can be grouped together into chunks higher-level units of organization. Thus, 3 single digits could be chunked together into a 3-digit unit: 3, 1 and 4 becomes 314. So what determines how much can be chunked together? Miller suggested that chunking may be governed by meaningfulness. For example, if the numbers 314 are your local area code (as they are for me), it is a very natural process to store them together as a chunk. These processes also seem to be ubiquitous in language, where we effortlessly group 10

11 letters into word-chunks and words into phrase-chunks. Indeed, this may be why our ability to maintain verbal information in short-term storage may be better than for other types of information. Most importantly, the key notion of Miller s chunk idea is that short-term storage, though possibly subject to certain constraints, is still very much open to strategic influences. This notion is still very much present in current ideas of working memory, as we discuss later in the chapter. As a side note, although the notion of a magic number is still very much a part of current ideas regarding short-term storage capacity, recent work has suggested that this number might not actually be 7 +/- 2, as Miller suggested, but instead maybe much less: 3 +/- 1. This revised estimate comes from a review of studies suggesting that storage capacity is much lower than 7, when individuals are prevented from using chunking or other strategies, such as rehearsal (Cowan 2001). The work of Miller drew a great deal of attention to the concept of short-term memory and its functional characteristics. A key piece of evidence suggesting the functional uniqueness of short-term memory came from studies of amnesics, such as the famous H.M. discussed in [?? Chapter??] who showed grossly impaired long-term memory but relatively intact performance on immediate recall tasks, such as digit span (Baddeley & Warrington 1970, Scoville & Milner 1957). As a result of this, and other findings, a common view emerged that short-term memory was structurally and functionally distinct from long-term memory, and as such could be independently studied to better understand its specific characteristics. In particular, it seemed that short-term memory could be uniquely defined in terms of its short duration and high level of accessibility (i.e., that information seems to be held on-line ). Thus, during the 1950s and 1960s much research was devoted to examining these characteristics. 11

12 A central idea regarding short-term memory was that information would only be available for a very brief period if it were not consciously rehearsed. An experimental technique called the Brown-Petersen task was developed to explicitly test that idea (Brown 1958, Peterson & Peterson 1959). In a typical procedure, participants would be given a string of 3 consonants to memorize and then prevented from engaging in active memorization via a task distraction (e.g., being asked to count from 100 backwards by 3). After a set delay the participant would be asked to recall the strong. By measuring recall accuracy in relation to delay interval, the time-course of forgetting could be revealed. What was found is that after a delay as short as 6 seconds, recall accuracy declined to about 50%, and by about 18 seconds recall was close to zero. These findings suggested that short-term storage was truly short in duration. However, in work that followed, a controversy arose as to whether the forgetting of information was truly due to a passive decay over time, or rather due to interference from other stored information (termed proactive interference, since the it was due to the forward acting interference effect of the earlier trials). This argument was bolstered by the fact that participants recall performance tended to be much better in the first few trials of the task (when proactive interference had not yet built up). Moreover, if a trial was inserted that tested memory for a different type of information than the previous trials (e.g., switching from consonants to letters), participants recall performance greatly increased (Wickens et al 1976). The argument was that recall performance improved on such trials because the sources of proactive interference had been released. The debate over whether information is lost from short-term memory because of decay or interference is one that has never fully resolved, and is still studied today (Nairne 2002)

13 [?? INSERT FIGURE 2 HERE: THIS WILL SHOW DATA FROM BROWN-PETERSON TASK ILLUSTRATING DECLINE IN RECALL WITH DURATION A second key idea regarding short-term memory was that information stored there could be rapidly accessed. A classic set of studies conducted by Sternberg provided strong support for this idea (Sternberg 1966, Sternberg 1969). In Sternberg s experimental procedure, a variable number of items (the memory set), such as digits, were presented briefly at the beginning of a trial and then removed for a short delay. Following the delay, a probe item appeared and the participant had to respond as to whether the probe matched an item in the memory set or not. The time required to respond should reflect the sum of four quantities: 1) the time required to perceptually process the probe item; 2) the time required to access and compare an item in short-term memory against the probe item; 3) the time required to make a binary response decision (match/nonmatch); and 4) the time required to execute the proper motor response. Sternberg hypothesized that as the number of items in the memory set increased, quantity (2) should increase (because an additional item would need to be accessed), but the other three quantities should remain constant. Thus, Sternberg hypothesized when the reaction time was plotted as a function of the number of memory set items, this plot should reveal a linear function. Moreover, the slope of that function should reveal the average time needed to access an item held in short-term memory. The results were as predicted the data formed an almost perfect straight line, with a reaction time slope of approximately 40 msec (??Figure 3). Sternberg suggested that this value was an estimate of the scanning rate for shortterm memory. The results definitely bore out the hypothesis that information held in short-term 13

14 memory could be accessed at high speed! It should be noted, however, that more recent work has called into question the fundamental assumption that short-term memory scanning proceeds sequentially (i.e., one item at a time) that Sternberg relied upon to interpret the linear relationship of reaction time to memory set size. In particular, through the use of sophisticated mathematical modeling techniques, it has been claimed that the same types of linear functions could be found from a parallel scanning process (i.e., one that accesses all items at the same time) whose efficiency declines with the number of items held in short-term memory (McElree & Dosher 1989, Townsend & Ashby 1983) ?? INSERT FIGURE 3 HERE: THIS WILL BE A DIAGRAM OF THE STERNBERG DATA SHOWING THE LINEAR RELATIONSHIP BETWEEN MEMORY SET SIZE AND RT The Atkinson-Shiffrin model: The relationship of short-term and long-term memory If short-term and long-term memory were posited to be distinct modes of storing clearly articulated in the model proposed by Atkinson & Shiffrin (Atkinson & Shiffrin 1968). In their model, short-term memory served as the gateway by which information could gain access to long-term memory. The function of short-term memory was to provide a means by which to control and enhance which information made it into long-term memory, via rehearsal and coding strategies. The Atkinson-Shiffrin model was highly influential, in that it laid out a comprehensive view of information processing and its various stages. So much so, that even today, it is still referred to as the modal model of memory

15 ?? INSERT FIGURE 4 HERE: THIS WILL BE A DIAGRAM OF THE ATKINSON-SHIFFRIN MEMORY MODEL SHOWING THE STRUCTURAL DISTINCTIONS BETWEEN MEMORY STORES Yet today, the modal model does not have the influence it once had, and most psychologists favor a different conceptualization of the function of short-term storage that is not so exclusively focused on its relationship to long-term storage. Thus, as we indicated in the beginning of the chapter, currently the more commonly used term to refer to short-term storage is working memory, since this term is felt to capture better the notion that a temporary storage system might provide a useful workplace in which to engage complex cognitive activities. What caused this shift in perspective? One factor was the accumulating problems found with the ability of the Atkinson-Shiffrin model to account for how information gets into long-term memory. Neuropsychological data contributed strongly to this issue. Shallice & Warrington (Shallice & Warrington 1970) reported case studies of brain-damaged patients (typically suffering from damage to the parietal lobe) who showed drastic impairments in short-term memory, such as a digit span of only 1 or 2 items, and very quick forgetting on the Brown- Peterson test. Nevertheless, these same patients were able to both learn new information in a fashion comparable to healthy individuals. This finding suggested that information can gain access to the long-term memory system even when the short-term memory system was dramatically impaired. The Atkinson-Shiffrin model could not predict such a finding, since in that model information must pass through short-term memory before entering long-term 15

16 memory. With a poorly functioning short-term memory, according to Atkinson & Shiffrin, long-term storage should also be impaired. A final strand of evidence suggested that there is not a single system for short-term storage but multiple ones. In this work, Baddeley & Hitch (Baddeley & Hitch 1974) had participants perform a reasoning task that involved making simple true-false decisions about spatially arrayed letters, such as answering the question B does not follow A for the array B A. Critically, before each trial, the participants were also given a string of digits to hold in short-term memory and then repeat back immediately after each trial was over. Typically subjects were given strings of around 6-8 digits in length, which according to Miller, should fill the capacity of short-term memory. If this memory store is critical for performing complex cognitive tasks and there is only one short-term store available, then performance on the reasoning task should drastically decline with the addition of the additional requirement of digit memorization. However, this was not the case. The participants took slightly longer to answer questions but made no more errors when also holding digit strings in short-term memory. Baddeley & Hitch argued from their data that there were multiple systems available for shortterm storage and that these storage systems are coordinated by the actions of a central control system that flexibly handles memory allocation and the balance between processing and storage. The Baddeley & Hitch working memory model The model of working memory put forth by Baddeley & Hitch emphasized three important characteristics that differentiated from the Atkinson & Shiffrin model. First, the function of short-term storage is not primarily as a way-station for information to reside en route to long-term memory. Instead, the primary function of this storage system is to enable complex cognitive activities which require the integration, coordination and manipulation of multiple bits 16

17 of internally represented information. Thus, in the reasoning problem described above, working memory is required to: a) hold an internal trace of the two letters B and A and their spatial relationship; b) provide a workspace for parsing and unpacking the verbal statement (e.g., deciding that the statement B does not follow A implies that A follows B); and c) enable comparison of the two sources of information. A second characteristic of the Baddeley model of working memory is that there is an integral relationship between control systems that govern when information is deposited and removed from short-term storage and the storage buffers themselves. This tight level of interaction is what enables the short-term stores to serve as effective workplaces for cognition. A third characteristic of the Baddeley & Hitch model is that there are at least two distinct short-term memories: one for holding verbal information and the other for holding visuospatial information. Because these short-term stores are independent, there is greater flexibility in memory storage. For example, even if one buffer is engaged in storing information, the other can still be utilized to full effectiveness. Likewise, the supervision of these system by control, or executive processes suggests that information can be rapidly shuttled between the two stores, and also coordinated across them. Thus, in the reasoning task described above, the explanation as to why the participants performance did not suffer too much is that while the verbal store was occupied with digit storage, the visuospatial store was utilized to do much of the cognitive work involved in the reading task (which primarily involved evaluating spatial relationships). At those points in which the verbal store was truly needed for the reasoning task (e.g., when the sentence information was being properly parsed), the control system might be able to shuffle the information into the visuospatial store (e.g., by transformation of the information into a mental image). 17

18 To summarize, Baddeley & Hitch proposed a model of working memory in which there were three components: two short-term stores - termed the phonological loop (for verbal information) and the visuospatial scratchpad (for visuospatial information) and a control system, termed the central executive. These three components interact together to provide a comprehensive workspace for cognitive activity. The Baddeley & Hitch model was a major departure from previous theorizing about short-term memory, by emphasizing not its duration or relationship to long-term memory but instead its flexibility and critical importance to ongoing cognition. In the years since first describing the model, Alan Baddeley has been a major figure in working memory research, by continuing to elaborate on the initial conception of the working memory model, and by providing a great deal of support for its validity and usefulness. Baddeley s conceptualization of working memory is currently still highly influential, and serves as a source of a great deal of research. Consequently, in the next section, we will describe the model in greater detail, and describe some of the empirical support for it ?? INSERT FIGURE 5 HERE: THIS WILL BE A DIAGRAM OF THE BADDELEY AND HITCH MODEL ILLUSTRATING THE THREE COMPONENTS OF WM Understanding the working memory model Let s explore in detail the three components of Baddeley s working memory model. As we will see, much research has been devoted to studying this model and testing ideas about its characteristics. Although the initial conceptualization of a central controller interacting with 18

19 dual short-term memory buffers has been retained over the years, certain aspects of the model have become further elaborated by subsequent research and the contributions of other investigators. In particular, there has been an intense focus of research on storage within verbal working memory, since so much of everyday cognition seems to rely on this cognitive function. Indeed, as a student, your verbal working memory system is frequently taxed, as you focus much of your daily energies on reading and comprehending long sentences such as this one! So, we begin with studies of the phonological loop. The phonological loop Let s try a demonstration to begin thinking about the properties of the phonological loop. In the next line is a series of digits. Please read these quickly and then immediately close your eyes. Keep silent, but try to remember the digits. Then, after a few seconds repeat them back out loud to yourself: Did you try it? How did you do in recalling the numbers accurately? It s no coincidence that I picked 7 digits. The demonstration was meant to mimic something you may do frequently, which is getting a phone number from information. How did you accomplish the task? Most people, in this situation report that when reading the digits they hear them in their head (with the sounds being of their own voice). Then when their eyes are closed they rehearse the sounds, but silently (if you followed the instructions!). What does it mean to rehearse? The subjective experience seems to be of speaking the digits in your mind. Does this fit with your own experience? 19

20 The idea that verbal working memory seems to involve both an inner ear and inner voice is central to current thinking regarding the function of the phonological loop. Baddeley and colleagues (Baddeley 1986) have proposed that the phonological loop system involves two subcomponents: a phonological storage buffer and articulatory rehearsal process. The idea is that when encoding visually presented verbal information, a first stage is to transform the information into a sound-based or auditory-phonological code. This sound-based code is something of an internal echo-box, a repository of sounds that reverberates briefly before fading away. In order to prevent complete decay, and active process must intercede to refresh the information. The active refreshment comes in the form of subvocal articulation that is, internally voicing the sounds that you internally heard. The process seems to be much like our ability to shadow, quickly repeating back something that we hear. This is not coincidental, and will figure in our discussion below, regarding the natural function of the phonological loop. Once the verbal information is spoken by the inner voice (the subvocal rehearsal process) it can then be again heard by the inner ear (the phonological store). This is the process by which the refresh occurs, and enables a cycling or continuous loop to play for as long as the verbal material needs to be maintained in working memory. A corollary of this model is that the first step of the process translation into a phonological code is only necessary when viewing visually presented material. For auditory information, such as speech, initial access to the phonological store is automatic, and in fact, obligatory. This idea sounds intuitive and that has been part of its appeal. But more importantly, the model just described outlines a number of important characteristics about the nature of verbal working memory that should be testable. First, it suggests that verbal working memory capabilities should depend upon the ease or difficulty of both phonological processing 20

21 (translating verbal information into a sound-based code) and articulatory processing (translating verbal information into a speech-based code). Second, it suggests that because working memory is flexible, with the possibility of storage via alternative, but possibly less effective means (i.e., utilizing the central executive and visuospatial scratchpad), performance on verbal working memory tasks will not be catastrophically disrupted when the phonological loop component is unusable. But under these conditions, the dependence of working memory will no longer be related to phonological or articulatory factors. Third, the phonological loop model suggests that the two primary components of verbal working memory phonological storage and articulatory rehearsal are subserved by functionally independent cognitive modules, and hence should be dissociable. These claims hold up under experimental scrutiny. Behavioral studies have suggested that phonological and articulatory factors do significantly affect verbal working memory performance. One example of this is the phonological similarity effect. Working memory performance (when items have to be serially recalled) is significantly poorer if the items to be maintained are phonologically similar, than if they are dissimilar (Conrad & Hull 1964). This can easily be appreciated by imagining trying to hold either of the following two strings of letters in working memory: D B C T P G vs. K F Y L R Q. In the first string, the letters all have the ee sound, whereas in the second list all the letters are have distinct sounds. In these tasks, it has further been observed that when participants make errors, they tend to be from substituting another item into the string that sounds phonologically similar to the actual item to be maintained, such as substituting a V for the G in the first string. The effects of articulation on working memory can be seen in the word length effect. Performance is much worse when the items to be maintained are long words, such as university, 21

22 individual and operation than short words, such as yield, item and brake. The key factor seems not to be the number of syllables per se, but rather the time it takes to pronounce them. Thus, even when words are matched on syllable number (as well as other variables such as meaning and frequency), performance is worse for words that have long vowel sounds, such as harpoon and baby, than words with short vowel sounds such as bishop and picnic (Baddeley et al 1975). The word length effect is explained in the phonological loop model by the assumption that pronunciation time impacts the speed of subvocal rehearsal (which requires speech-based processing). The longer it takes to rehearse a set of items in working memory the more likely it is that those items will have decayed from the phonological store. The relationship between pronunciation time and working memory performance was further tested in a compelling study involving children from Wales (Ellis & Hennelly 1980). In the Welsh language, digit names, although having the same number of vowels as English, tend to have longer vowel sounds and consequently take longer to say. As predicted, when performing digit span tests in Welsh, the childrens scores were significantly below average norms. However, when these same children performed the tests again in English (they were bilingual) their scores were perfectly normal. Follow-up studies have suggested that the faster an individual s speech rate, the higher their digit span (Cowan et al 1992) ?? INSERT FIGURE 6 HERE: THIS WILL BE A DIAGRAM OF THE RELATIONSHIP BETWEEN READING RATE AND DIGIT SPAN

23 What happens when the normal operation of the phonological loop is disrupted? Under these conditions the Baddeley model suggests that auxiliary systems such as the central executive and visuospatial scratchpad will take over. Once these systems are in effect, storage of verbal information in working memory will no longer show sensitivity to phonological and articulatory factors (e.g., phonological similarity and word length effects). One test of this claim involves the classic cognitive psychology technique of dual-task interference. The idea is that one can prevent the normal operation of the phonological loop by requiring participants to produce overt and irrelevant speech (the dual-task) while at the same time maintaining information in working memory. This type of interference manipulation has been termed articulatory suppression. In a typical paradigm, the participant is asked to repeat the word such as the out loud over and over. Under such conditions, the articulatory rehearsal system will be overloaded, and consequently will be unable to refresh information held in the phonological store. As expected, performance is significantly, though not catastrophically impaired. More importantly, under these conditions, both the phonological similarity and word length effects are no longer present (Baddeley 1986, Baddeley et al 1984). Neuropsychological studies have provided another means by which to test the claims of the phonological loop model. One famous set of studies involves the patient P.V., a 28-year old woman who suffered from a stroke that damaged a large extent of her left hemisphere, but especially the cortical regions thought to be involved in language processing (Basso et al 1982, Vallar & Baddeley 1984, Vallar & Papagno 1986). Despite this large degree of brain damage, P.V. s language processing abilities were remarkably intact (i.e., she could clearly perceive and comprehend spoken speech). Nevertheless, P.V. suffered from a dramatic decline in her performance on verbal working memory tasks, especially when these involved auditorily 23

24 presented information. Indeed, rather than the normal span length of around 7 items, P.V. s span was only about 2 items under auditory presentation. P.V.s poor auditory verbal working memory might be expected if her brain damage selectively targeted the phonological loop, making her more reliant on the visuospatial scratchpad to perform verbal working memory tasks. When verbal information is presented visually, the scratchpad might be more easily engaged than when information is presented auditorily (and must first be processed phonologically, before being translated to a visuospatial code). More importantly, when performing verbal working memory tasks, P.V. showed no effect of word length regardless of how the information was presented, and no phonological similarity effect for visually presented items. Likewise, when asked to perform the task under articulatory suppression conditions, she showed no further decline in performance (although it might be argued that her performance was so poor already, that no further decline could be observed). Since the case of P.V., a number of other patients with selective auditory-verbal short-term memory deficits have been identified. Based on their common pattern of deficits and the neuroanatomical locus of damage, it has been suggested that these patients have damage to the phonological store component of verbal working memory, and that this component is localized within the left inferior parietal cortex (Vallar & Papagno 1995). An important characteristic of the phonological loop model is that the phonological store and articulatory rehearsal processes are functionally independent components. How can this idea be tested? Behavioral studies suggest that if two experimental factors affect the same cognitive process, then when both factors are jointly present they should have an even larger impact on performance (i.e., they should interact statistically). In contrast, if the factors target functionally independent processes, then the presence of one factor should not influence the 24

25 effect of the other they should not interact. This is known as additive factors logic (Sternberg 1969). With respect to verbal working memory, if the word length and phonological similarity effects target independent components of the phonological loop (the storage and rehearsal systems) then manipulations of word length and phonological similarity should not interact with each other. Precisely, this effect was obtained (Longoni et al 1993). The magnitude of the phonological similarity effect on performance was not influenced by word length, and vice versa ?? INSERT FIGURE 7 HERE: THIS WILL BE A PLOT OF LONGONI DATA SHOWING INDEPENDENT EFFECTS OF WORD LENGTH AND PHONOLOGICAL SIMILARITY However, behavioral data only provide indirect evidence for functional independence. Neuropsychological findings can provide a stronger case that separate brain systems support phonological storage and rehearsal. As described previously there is fairly strong consensus in the literature that damage to left inferior parietal cortex impairs phonological storage. In contrast, impairments in rehearsal related functions have been typically found in patients with lesions within the left inferior frontal lobe (Fiez 2001). These patients commonly have problems with overt speech production, and as such are labeled as non-fluent or Broca s aphasics. Similarly, the brain damaged area is often referred to as Broca s area. As might be expected, patients with these disorders such no word length effects, and no effects of articulatory suppression (Belleville et al 1992, Goerlich et al 1995). 25

26 In neuropsychological patients, there seems to be a relationship between left inferior parietal damage and phonological storage impairments on the one hand, and left inferior frontal cortex (Broca s area) damage and articulatory rehearsal impairments on the other. Neuroimaging studies have provided a means to examine these relationships in healthy subjects. This is obviously critical, since it is important to demonstrate whether these same regions are also engaged under normal processing conditions. In a classic study conducted by Paulesu and colleagues (Paulesu et al 1993), participants were asked to memorize a series of six visually presented letters in order to respond appropriately to a probe letter (did it match one of the originally presented letters, yes or no?) following a short delay. Two conditions were performed, one with English letters and one with Korean characters. The assumption was that the phonological loop system would be engaged to maintain the English letters, but not utilized for the Korean characters (since the participants were not Korean speakers). Using PET, the investigators observed increased blood flow in both left inferior parietal cortex and left inferior frontal cortex. Additionally activation was observed in other motor-related brain structures such as the supplementary motor area (SMA), insula, and cerebellum. In a second experiment, the investigators attempted to dissociate regions associated with phonological storage from those subserving articulatory rehearsal. To do this, subjects were asked to perform rhyme judgments on the letters (decide if each letter rhymed with B). The assumption was that the rhyme task would only engage articulatory rehearsal processes but not phonological storage. When contrasting the first and second experiments only the left parietal cortex was found to show significantly reduced activation. Results highly consistent with these were also found in a similar neuroimaging study by Awh and colleagues that nevertheless employed a different working memory task (the n-back, which we will discuss further later) and control conditions 26

27 (Awh et al 1996). Thus, when taken together, the neuroimaging results converge to further establish the dissociability of the storage and rehearsal components of verbal working memory ?? INSERT FIGURE 8 HERE: THIS WILL SHOW THE PAULESU IMAGING DATA ILLUSTRATING INCREASED ACTIVITY IN LEFT PARIETAL AND LEFT FRONTAL CORTEX More recent neuroimaging studies have also shed light on additional characteristics of verbal working memory, but also suggest a more complex picture. For example, Julie Fiez and colleagues have found different timecourses of activity within distinct subregions of Broca s area during the delay period of a working memory task. Based on their data they hypothesized that the more dorsal region is involved in the initial assembly of a new articulatory rehearsal program, whereas the more ventral region is thought to be involved with the repetitive execution of this program during the maintenance period (Chein & Fiez 2001). Neuroimaging studies are clearly playing an important role in refining and reshaping the verbal working memory model. While reading this section, you may have been thinking to yourself, what is the true function of the phonological loop in cognition? How did evolve to become a part of our cognitive apparatus? Surely it did not arise just to help us retain telephone numbers! It seems intuitive that the phonological loop would have to play some role in language processing, since it is so clearly integrated with language comprehension and production systems. An hypothesis that seems to be consistent with the experimental data is that working memory is not critical for comprehension of familiar language, but is critical for new language learning (Baddeley et al 27

28 1998). That is, in children learning their first language, and in adults learning a second language or acquiring new vocabulary, the phonological loop plays an essential role. It may be the case that evolution has imbued us with a specific expertise in repeating back what we hear, even if we don t initially understand it. This form of imitation is something that even young infants can do, and it may provide a means for scaffolding us into learning new words via a linkage of sound and meaning. Developmental data strongly support this claim, by showing children s ability to repeat back nonwords strongly predicts the size of their vocabulary one year later (Gathercole & Baddeley 1989). Likewise, in a vocabularly learning study, patient P.V. was found to be completely unable to learn the Russian equivalent of any words in her native language, despite extensive practice with them (Baddeley et al 1988). Thus, the phonological loop may have a primary function as a language learning device. Yet that same functionality can be put into service into a wide-range of working memory tasks that we encounter during everyday cognitive activities. The visuospatial scratchpad The phonological loop system is proposed to mediate short-term storage in tasks that involve verbal information. For tasks that involve visuospatial information, the Baddeley model suggests that a different buffer system is involved. To get a feel for when visuospatial working memory might be needed and how it might operate, try the answering the following question: What objects are on the walls of your bedroom? Name them in order, starting from your bedroom door and moving clockwise from wall to wall (Note: if you re currently in your bedroom as you read this, close your eyes first before answering; don t cheat!). Do you think you named all of the objects correctly? Check yourself when you get the opportunity. More importantly, how did you approach the task? Most people report that they 28