In press: Psychology of Learning and Motivation

Transcription

1 In press: Psychology of Learning and Motivation An Exemplar-Retrieval Model of Short-Term Memory Search: Linking Categorization and Probe Recognition Robert M. Nosofsky Indiana University Bloomington Robert Nosofsky Department of Psychological and Brain Sciences 1101 E. Tenth Street Indiana University Bloomington, IN (812) Running Head: Memory Search 1

2 Table of Contents 0. Abstract 1. Introduction and Background 1.1. Introduction 1.2 Background 2. The Core Version of the Formal Model 3. Short-Term Probe Recognition in a Continuous-Dimension Similarity Space 4. Short-Term Probe Recognition of Discrete Stimuli 5. A Power-Law of Memory Strength 6. Bridging Short-Term and Long-Term Probe Recognition and Incorporating the Role of Previous Memory Sets 6.1. Review of Empirical Findings 6.2. The Extended EBRW Model: Conceptual Description 6.3. The Extended EBRW Model: Formal Description 6.4. Modeling Application 7. Evidence for a Joint Role of Categorization and Familiarity Processes 8. Summary and Conclusions 2

3 Abstract Exemplar-retrieval models such as the exemplar-based random-walk (EBRW) model (Nosofsky & Palmeri, 1997) have provided good accounts of response-time (RT) and choice-probability data in a wide variety of categorization paradigms. In this chapter I review recent work showing that the model also accounts accurately for RT and choiceprobability data in a wide variety of probe-recognition short-term memory-search paradigms. According to the model, observers store items from study lists as individual exemplars in memory. When a test probe is presented, it causes the exemplars to be retrieved. The exemplars that are most readily retrieved are those that are highly similar to the test probe and that have the greatest memory strengths. The retrieved exemplars drive a familiarity-based evidence-accumulation process that determines the speed and accuracy of old-new recognition decisions. The model accounts for effects of memoryset size, old-new status of test probe, and study-test lag; effects of the detailed similarity structure of the memory set; and the role of the history of previously experienced memory sets on performance. Applications of the model reveal a quantitative law of how memory strength varies with the retention interval. In addition, the model provides a unified account of how probe recognition operates in cases involving short and long study lists. Furthermore, it provides an account of the classic distinction between controlled versus automatic processing depending on the types of memory-search practice in which observers engage. In short, the model brings together and extends prior research and theory on categorization, attention and automaticity, short- and long-term memory, and evidence-accumulation models of choice RT to move the field closer to a unified account of diverse forms of memory search. 3

4 1. Introduction and Background 1.1. Introduction. A fundamental issue in cognitive science concerns the mental representations and processes that underlie memory search and retrieval. A major approach to investigating the nature of memory search is to measure both accuracies and response times (RTs) in tasks of probe recognition. In such tasks, observers are presented with a list of to-be-remembered items (the memory set ) followed by a test probe. The task is judge, as rapidly as possible while minimizing errors, whether the test probe is old (a member of the memory set) or new. In this chapter I provide a review of recent and ongoing work in which I have applied an extension of an exemplar-retrieval model of categorization (Nosofsky & Palmeri, 1997) to account for probe-recognition memory search (Nosofsky, Little, Donkin, & Fific, 2011; Nosofsky, Cox, Cao, & Shiffrin, 2014). As I will describe, the model builds upon and extends classic theories in the domains of categorization and memory and ties them together with evidence-accumulation models of decision making. I will argue that the model provides a coherent account of a highly diverse set of memory-search results, moving the field in the direction of a unified account of categorization and probe recognition. To anticipate, I will provide evidence showing that the model accounts for: i) classic effects of memory-set size, old-new status of the probe, and study-test lag on recognition RTs and accuracies; ii) effects of the detailed similarity structure of the list of to-be-remembered items; and iii) the role of the history of previous memory sets on judgments involving the present set. Furthermore, applications of the model will reveal an intriguing quantitative law of how memory strength varies with the recency with 4

5 which study items were presented. The model will also allow for a unified account of how probe recognition operates in cases involving both short and long lists. In addition, the model will provide accounts of the basis for the classic distinction between controlled versus automatic processing depending on the types of memory-search practice in which observers engage (e.g., Shiffrin & Schneider, 1977) Background. In the seminal memory-scanning paradigm introduced by Sternberg (1966, 1969), observers maintain short lists of items in memory and are then presented with a test probe. The observers task is to classify the probe as old or new as rapidly as possible while minimizing errors. Under Sternberg s conditions of testing, the result was that mean RTs for both old and new probes were linearly increasing functions of the size of the memory set. Furthermore, the RT functions for the old and new probes were parallel to one another. These results led Sternberg to formulate his classic serial-exhaustive model of memory search. Since that time, a wide variety of other information-processing models have also been developed to account for performance in the task (for reviews and analysis, see Reed, 1973 and Townsend and Ashby, 1983). One modern formal model of short-term probe recognition is the exemplar-based random walk (EBRW) model (Nosofsky et al., 2011). According to this model, shortterm probe recognition is governed by the same principles of global-familiarity and exemplar-based similarity that are theorized to underlie long-term recognition and forms of categorization (Clark & Gronlund, 1996; Gillund & Shiffin, 1984; Hintzman, 1988; Kahana & Sekuler, 2002; Medin & Schaffer, 1978; Murdock, 1985; Nosofsky, 1986, 5

6 1991; Nosofsky & Palmeri, 1997; Shiffrin & Steyvers, 1997). The model assumes that each item of a memory set is stored as an individual exemplar in memory. When a test probe is presented, it causes the individual exemplars to be retrieved. The exemplars that are most readily retrieved are those that are highly similar to the test probe and that have the greatest memory strengths. The retrieved exemplars drive a familiarity-based evidence-accumulation process that determines the speed and the accuracy of old-new recognition decisions. I start this chapter by providing a formal statement of a core version of the model. The core version adopts the simplifying assumption that the observer s performance depends only on the current memory set being tested, without influence from previous sets. Later in the chapter, I present an extended version of the model in which I attempt to capture how the observer s experience with previous memory sets influences performance on the current set. 2. The Core Version of the Formal Model A schematic illustration of the workings of the EBRW model as applied to probe recognition is presented in Figure 1. The model assumes that each item of a study list is stored as a unique exemplar in memory. The exemplars are represented as points in a multidimensional psychological space. In the baseline model, the distance between exemplars i and j is given by 1 K d w x - x, ij k ik jk (1) k 1 6

7 where x ik is the value of exemplar i on psychological dimension k; K is the number of dimensions that define the space; ρ defines the distance metric of the space; and w k (0 < w k, w 1 ) is the weight given to dimension k in computing distance. In situations k involving the recognition of holistic or integral-dimension stimuli (Garner, 1974), which will be the main focus of the present work, ρ is set equal to 2, which yields the familiar Euclidean distance metric. The dimension weights w k are free parameters that reflect the degree of "attention" that subjects give to each dimension in making their recognition judgments. In situations in which some dimensions are more relevant than others in allowing subjects to discriminate between old versus new items, the attention-weight parameters may play a significant role (e.g., Nosofsky, 1991). In the experimental situations considered in the present work, however, all dimensions tend to be relevant and the attention weights will turn out to play a minor role. The similarity of test item i to exemplar j is an exponentially decreasing function of their psychological distance (Shepard, 1987), s exp - c d, (2) ij j ij where c j is the sensitivity associated with exemplar j. The sensitivity governs the rate at which similarity declines with distance in the space. When sensitivity is high, the similarity gradient is steep, so even objects that are close together in the space may be highly discriminable. By contrast, when sensitivity is low, the similarity gradient is shallow, and objects are hard to discriminate. In most previous tests of the EBRW 7

8 model, a single global level of sensitivity was assumed that applied to all exemplar traces stored in long-term memory. In application to the present short-term recognition paradigms, however, allowance is made for forms of exemplar-specific sensitivity. For example, in situations involving high-similarity stimuli, an observer's ability to discriminate between test item i and exemplar-trace j will almost certainly depend on the recency with which exemplar j was presented: Discrimination is presumably much easier if an exemplar was just presented, rather than if it was presented earlier on the study list (due to factors such as interference and decay). Each exemplar j from the memory set is stored in memory with memory-strength m j. As is the case for the sensitivities, the memory-strengths are exemplar-specific (with the detailed assumptions stated later). Almost certainly, for example, exemplars presented more recently will have greater strengths. When applied to old-new recognition, the EBRW model presumes that abstract elements termed criterion elements are part of the cognitive processing system. The strength of the criterion elements, which we hypothesize is at least partially under the control of the observer, helps guide the decision about whether to respond "old" or "new". In particular, as will be explained below, the strength setting of the criterion elements influences the direction and rate of drift of the exemplar-based random-walk process. Other well-known sequential-sampling models include analogous criterionrelated parameters for generating drift rates, although the conceptual underpinnings of the models are different from those in the EBRW model (e.g., Ratcliff, 1985, pp ; Ratcliff, Van Zandt, & McKoon, 1999, p. 289). 8

9 Presentation of a test item causes the old exemplars and the criterion elements to be activated. The degree of activation for exemplar j, given presentation of test item i, is given by a m s. (3) ij j ij Thus, the exemplars that are most strongly activated are those with high memory strengths and that are highly similar to test item i. The degree of activation of the criterion elements (C) is independent of the test item that is presented. Instead, criterionelement activation functions as a fixed standard against which exemplar-based activation can be evaluated. As discussed later in this chapter, however, criterion-element activation may be influenced by factors such as the size and structure of the memory set, because observers may adjust their criterion settings when such factors are varied. Upon presentation of the test item, the activated stored exemplars and criterion elements race to be retrieved (Logan, 1988). The greater the degree of activation, the faster the rate at which the individual races take place. On each step, the exemplar (or criterion element) that wins the race is retrieved. Whereas in Logan s (1988) model the response is based on only the first retrieved exemplar, in the EBRW model the retrieved exemplars drive a random-walk process. First, there is a random-walk counter with initial setting zero. The observer establishes response thresholds, R old and R new, that determine that amount of evidence needed for making each decision. On each step of the process, if an old exemplar is retrieved, then the random-walk counter is incremented by unit value towards the R old threshold; whereas if a criterion element is retrieved, the 9

10 counter is decremented by unit value towards the R new threshold. If either threshold is reached, then the appropriate recognition response is made. Otherwise, a new race is initiated, another exemplar or criterion element is retrieved (possibly the same one as on the previous step), and the process continues. The recognition decision time is determined by the total number of steps required to complete the random walk. It should be noted that the concept of a "criterion" appears in two different locations in the model. First, as explained above, the strength setting of the criterion elements influences the direction and rate of drift of the random walk. Second, the magnitude of the R old and R new thresholds determine how much evidence is needed before an old or a new response is made. Again, other well-known sequential-sampling models include analogous criterionrelated parameters at these same two locations (for extensive discussion, see, e.g., Ratcliff, 1985). Given the detailed assumptions in the EBRW model regarding the race process (see Nosofsky & Palmeri, 1997, p. 268), it turns out that, on each step of the random walk, the probability (p) that the counter is incremented towards the R old threshold is given by p i = A i / (A i + C), (4) where A i is the summed activation of all of the old exemplars (given presentation of item i), and C is the summed activation of the criterion elements. (The probability that the random walk steps toward the R new threshold is given by q i = 1- p i.) In general, therefore, test items that match recently presented exemplars (with high memory strengths) will 10

11 cause high exemplar-based activations, leading the random walk to march quickly to the R old threshold and resulting in fast OLD RTs. By contrast, test items that are highly dissimilar to the memory-set items will not activate the stored exemplars, so only criterion elements will be retrieved. In this case, the random walk will march quickly to the R new threshold, leading to fast NEW RTs. Through experience in the task, the observer is presumed to learn an appropriate setting of criterion-element activation (C) such that summed activation (A i ) tends to exceed C when the test probe is old, but tends to be less than C when the test probe is new. In this way, the random walk will tend to drift to the appropriate response thresholds for old versus new lists. In most applications, for simplicity, I assume the criterion-element activation is linearly related to memory set size. (Because summed-activation of exemplars, A i, tends to increase with memory-set size, the observer needs to adopt a stricter criterion as memory-set size increases.) Given these processing assumptions and the computed values of p i (Equation 4), it is then straightforward to derive analytic predictions of recognition choice probabilities and mean RTs for any given test probe and memory set. The relevant equations are summarized by Nosofsky and Palmeri (1997, pp , ). Simulation methods are used when the model is applied to predict fine-grained RT-distribution data. In sum, having outlined the general form of the model, I now review specific applications of the model to predicting RTs and accuracies in different variants of the short-term probe-recognition paradigm. 11

12 3. Short-Term Probe Recognition in a Continuous-Dimension Similarity Space In Nosofsky et al. s (2011) initial experiment for testing the model, the stimuli were a set of 27 Munsell colors that varied along the dimensions of hue, brightness, and saturation. Similarity-scaling procedures were used to derive a precise multidimensionalscaling (MDS) solution for the colors (Shepard, 1980). The MDS solution provides the x ik coordinate values for the exemplars (Equation 1) and is used in combination with the EBRW model to predict the results from the probe-recognition experiment (cf. Nosofsky, 1992). The design of the experiment involved a broad sampling of different list structures to provide a comprehensive test of the model. There were 360 lists in total. The size of the memory set on each trial was 1, 2, 3 or 4 items, with an equal number of lists at each set size. For each set size, half the test probes were old and half were new. In the case of old probes, the matching item from the memory set occupied each serial position equally often. To create the lists, items were randomly sampled from the full set of stimuli, subject to the constraints described above. Thus, a highly diverse set of lists was constructed, varying not only in set size, old/new status of the probe, and serial position of old probes, but also in the similarity structure of the lists. Because the goal was to predict performance at the individual-subject level, three subjects were each tested for approximately 20 one-hour sessions, with each of the 360 lists presented once per session. As it turned out, each subject showed extremely similar patterns of performance, and the fits of the EBRW model yielded similar parameter estimates for the three subjects. Therefore, for simplicity, and to reduce noise in the data, I report the results from the analysis of the averaged subject data. 12

13 In the top panels of Figure 2 I report summary results from the experiment. The top-right panel reports the observed correct mean RTs plotted as a function of: i) set size, ii) whether the probe was old or new (i.e., a lure), and iii) the lag with which old probes appeared in the memory set. (Lag is counted backwards from the end of the list.) For old probes, there was a big effect of lag: In general, the more recently a probe appeared on the study list, the shorter was the mean RT. Indeed, once one takes lag into account, there is little remaining effect of set size on the RTs for the old probes. That is, as can be seen, the different set size functions are nearly overlapping (cf. McElree & Dosher, 1989; Monsell, 1978). The main exception is a persistent primacy effect, in which the mean RT for the item at the longest lag for each set size is pulled down. (The item at the longest lag occupies the first serial position of the list.) By contrast, for the lures, there is a big effect of set size, with longer mean RTs as set size increases. The mean proportions of errors for the different types of lists, shown in the top-left panel of Figure 2, mirror the mean RT data just described. The goal of the EBRW modeling, however, was not simply to account for these summary trends. Instead, the goal was to predict the choice probabilities and mean RTs observed for each of the individual lists. Because there were 360 unique lists in the experiment, this goal entailed simultaneously predicting 360 choice probabilities and 360 mean RTs. The results of that model-fitting goal are shown in the top and bottom panels of Figure 3. The top panel plots, for each individual list, the observed probability that the subjects judged the probe to be old against the predicted probability from the model. The bottom panel does the same for the mean RTs. Although there are a few outliers in the plots, overall the model achieves a good fit to both data sets, accounting for 13

14 96.5% of the variance in the choice probabilities and for 83.4% of the variance in the mean RTs. The summary-trend predictions that result from these global fits are shown in the bottom panels of Figure 2. It is evident from inspection that the EBRW does of good job of capturing these summary results. For the old probes, it predicts the big effect of lag on the mean RTs and the nearly overlapping set-size functions. Likewise, it predicts with good quantitative accuracy the big effect of set size on the lure RTs. The errorproportion data (left panels of Figure 2) are generally also well predicted. The explanation of these results in terms of the EBRW model is straightforward. According to the best-fitting parameters from the model (see Nosofsky et al., 2011, Table 2), more recently presented exemplars had greater memory strengths and sensitivities than did less recently presented exemplars. From a psychological perspective, this pattern seems highly plausible. For example, presumably, the more recently an exemplar was presented, the greater should be its strength in memory. Thus, if an old test probe matches the recently presented exemplar, it will give rise to greater overall activation, leading to shorter mean old RTs. In the case of a lure, as set size increases, the overall summed activation yielded by the lure will also tend to increase. This pattern arises both because a greater number of exemplars will contribute to the sum, and because the greater the set size, the higher is the probability that it least one exemplar from the memory set will be highly similar to the lure. As summed activation yielded by the lures increases, the probability that the random walk takes correct steps towards the R new threshold decreases, so mean RTs for the lures get longer. 14

15 Beyond accounting well for these summary trends, inspection of the detailed scatterplots in Figure 3 reveals that the model accounts for fine-grained changes in choice probabilities and mean RTs depending on the fine-grained similarity structure of the lists. For example, consider the choice-probability plot (Figure 3, top panel) and the Lure-Size- 4 items (open diamonds). Whereas performance for those items is summarized by a single point on the summary-trend figure (Figure 2), the full scatterplot reveals extreme variability in results across different tokens of the Lure-Size-4 lists. In some cases the false-alarm rates associated with these lists are very low, in other cases moderate, and in still other case the false-alarm rates exceed the hit rates associated with old lists. The EBRW captures well this variability in false-alarm rates. In some cases, the lure might not be similar to any of the memory-set items, resulting in a low false-alarm rate; whereas in other cases the lure might be highly similar to some of the memory-set items, resulting in a high false-alarm rate. 4. Short-Term Probe Recognition of Discrete Stimuli The application in the previous section involved short-term probe recognition in a continuous-dimension similarity space. A natural question, however, is how the EBRW model might fare in a more standard version of the Sternberg paradigm, in which discrete alphanumeric characters are used. To the extent that things work out in a simple, natural fashion, the applications of the EBRW model to the standard paradigm should be essentially the same as in the just-presented application, except they would involve a highly simplified model of similarity. That is, instead of incorporating detailed assumptions about similarity relations in a continuous multidimensional space, we apply 15

16 a simplified version of the EBRW that is appropriate for highly discriminable, discrete stimuli. Specifically, in the simplified model, I assume that the similarity between an item and itself is equal to one; whereas the similarity between two distinct items is equal to a free parameter s (0 < s < 1). Presumably, the best-fitting value of s will be small, because the discrete alphanumeric characters used in the standard paradigm are not highly confusable with one another. Note that the simplified model makes no use of the dimensional attention-weight parameters or the lag-dependent sensitivity parameters. All other aspects of the model were the same, so we estimated the lag-dependent memory-strengths, random walk thresholds, and criterion-element parameters. Here, I illustrate an application of the simplified EBRW model to a well-known data set collected by Monsell (1978; Experiment 1, immediate condition). In brief, Monsell (1978) tested 8 subjects for an extended period in the standard Sternberg paradigm, using visually presented consonants as stimuli. The design was basically the same as the one described in the previous section of this chapter, except that the similarity structure of the lists was not varied. A key aspect of Monsell s procedure was that individual stimulus presentations were fairly rapid, and the test probe was presented either immediately or with brief delay. Critically, the purpose of this procedure was to discourage subjects from rehearsing the individual consonants of the memory set. If rehearsal takes place, then the psychological recency of the individual memory-set items is unknown, because it will vary depending on each subject s rehearsal strategy. By discouraging rehearsal, the psychological recency of each memory set item should be a systematic function of its lag. (Another important aspect of Monsell s design, which I 16

17 consider later in this review, is that he varied whether or not lures were presented on recent lists. The present applications are to data that are collapsed across this variable.) The mean RTs and error rates observed by Monsell (1978) in the immediate condition are reproduced in the top panel of Figure 4. (The results obtained in the briefdelay condition showed a similar pattern.) Inspection of Monsell's RT data reveals a pattern that is very similar to the one we observed in the previous section after averaging across the individual tokens of the main types of lists (i.e., compare to the observed-rt panel of Figure 2). In particular, the mean old RTs vary systematically as a function of lag, with shorter RTs associated with more recently presented probes. Once lag is taken into account, there is little if any remaining influence of memory-set-size on old-item RTs. For new items, however, there is a big effect of memory-set-size on mean RT, with longer RTs associated with larger set sizes. Because of the non-confusable nature of the consonant stimuli, error rates are very low; however, what errors there are tend to mirror the RTs. Another perspective on the observed data is provided in Figure 5, which plots mean RTs for old and new items as a function of memory-set-size, with the old RTs averaged across the differing lags. This plot shows roughly linear increases in mean RTs as a function of memory-set-size, with the positive and negative functions being roughly parallel to one another. (The main exception to that overall pattern is the fast mean RT associated with positive probes to 1-item lists.) This overall pattern shown in Figure 5 is, of course, extremely commonly observed in the Sternberg memory-scanning paradigm. Nosofsky et al. (2011) fitted the EBRW model to the Figure-4 data by using a weighted least-squares criterion (see the original article for details). The predicted mean RTs and error probabilities from the model are shown graphically in the bottom panel of 17

18 Figure 4. Comparison of the top and bottom panels of the figure reveals that the EBRW model does an excellent job of capturing the performance patterns in Monsell's (1978) study. Mean RTs for old patterns get systematically longer with increasing lag, and there is little further effect of memory-set-size once lag is taken into account. Mean RTs for lures are predicted correctly to get longer with increases in memory-set size. (The model is also in the right ballpark for the error proportions, although in most conditions the errors are near floor.) Figure 5 shows the EBRW model's predictions of mean RTs for both old and new probes as a function of memory-set size (averaged across differing lags), and the model captures the data from this perspective as well. Beyond accounting for the major qualitative trends in performance, the EBRW model provides an excellent quantitative fit to the complete set of data. The best-fitting parameters from the model (see Nosofsky et al., 2011, Table 3) were highly systematic and easy to interpret. As expected, the memory-strength parameters decreased systematically with lag, reproducing the pattern seen in the fits to the data from the previous section. The best-fitting value of the similarity-mismatch parameter (s =.050) reflected the low confusability of the consonant stimuli from Monsell s experiment. The conceptual explanation of the model s predictions is essentially the same as already provided in the previous section. In sum, without embellishment, the EBRW model appears to provide a natural account of the major patterns of performance in the standard version of the Sternberg paradigm in which discrete alphanumeric characters are used, at least in cases in which the procedure discourages rehearsal and where item recency exerts a major impact. In addition, I should note that although the present chapter focuses on predictions and 18

19 results at the level of mean RTs, the exemplar model has also been shown to provide successful quantitative accounts of probe-recognition performance at the level of complete RT distributions. Examples of such applications are provided by Nosofsky et al. (2011), Donkin and Nosofsky (2012b), and Nosofsky, Cao, Cox and Shiffrin (2014). 5. A Power-Law of Memory Strength Applications of the EBRW model to the probe-recognition paradigm have led to the discovery of an interesting regularity involving memory strength. As described in the previous sections, in the initial tests of the model, separate memory strength parameters were estimated corresponding to each individual lag on the study list. It turns out, however, that the estimated memory strengths follow almost a perfect power function of this lag. The power-function relation has been observed in both studies reviewed in the previous sections, but is brought out in most convincing fashion in an experiment reported by Donkin and Nosofsky (2012a). In this experiment, participants studied 12- item lists consisting of either letters or words, followed by a test probe. Separate RTdistribution data for hits and misses for positive probes were collected at each study lag. (RT-distribution data for false alarms and correct rejections for negative probes were collected as well.) In line with the results reported in the previous sections, mean RTs and error probabilities increased in regular fashion with increases in study lag (for detailed plots, see Donkin & Nosofsky, 2012a, Figures 1-3). The EBRW model provided an excellent quantitative account of the complete sets of detailed RTdistribution and choice-probability data. 1 19

20 The discovery that resulted from the application of the model is illustrated graphically in Figure 6. The figure plots, for each of four individual participants who were tested, the estimated memory-strength parameters against lag. As shown in the figure, the magnitudes of the memory strengths are extremely well captured by a simple power function. Indeed, a special case of the model that imposed a power-function relation on the memory strengths provided a more parsimonious fit to the data than did the full version of the model that allowed all individual memory-strength parameters to vary freely (for detailed analyses, see Donkin & Nosofsky, 2012a). Furthermore, a variety of alternative quantitative functions, including exponential, hyperbolic, linear, and logarithmic functions, failed to provide an adequate account of the data. Interestingly, other researchers have previously reported that a variety of empirical forgetting curves are well described as power functions (e.g., Anderson & Schooler, 1991; Wickelgren, 1974; Wixted & Ebbesen, 1991). For example, Wixted and Ebbesen (1991) reported that diverse measures of forgetting, including proportion correct of free recall of word lists, recognition judgments of faces, and savings in relearning lists of nonsense syllables, were well described as power functions of the retention interval. Wixted (2004) considered a variety of possible reasons for the emergence of these empirical power-function relations and concluded that the best explanation was that the strength of the memory traces themselves may exhibit power-function decay. The model-based results from Donkin and Nosofsky (2012a) lend support to Wixted s suggestion and now motivate the new research goal of unpacking the detailed psychological and neurological mechanisms that give rise to this discovered power law of memory strength. 20

21 6. Bridging Short-Term and Long-Term Probe Recognition and Incorporating the Role of Previous Memory Sets 6.1. Review of Empirical Findings. The hypotheses that global familiarity and exemplar-based similarity govern long-term recognition and categorization have been central ones in the field of cognitive psychology for decades (e.g., Gillund & Shiffrin, 1984; Hintzman, 1986, 1988; Medin & Schaffer, 1978; Nosofsky, 1986). The idea that those very same principles may underlie short-term probe recognition is less widely held; however, as just reviewed in the previous sections of this chapter, evidence in favor of that hypothesis has been mounting in recent years (e.g., Donkin & Nosofsky, 2012a,b; Nosofsky et al., 2011; see also Kahana & Sekuler, 2002). More rigorous support for the idea would arise, however, if one could show that an exemplar-familiarity model accounted parsimoniously for probe recognition involving both short and long lists within the same experimental paradigm. Nosofsky, Cox, Cao and Shiffrin (2014) pursued that aim by further testing the EBRW model in a memory-search paradigm in which memoryset size took on values 1, 2, 4, 8 and 16 across trials. This aim of bridging short-term and long-term probe recognition with the EBRW model was a timely one, given intriguing results reported recently by Wolfe (2012). Following some of the early hybrid memory and visual search paradigms of Schneider and Shiffrin (1977) and Shiffrin and Schneider (1977), Wolfe conducted experiments in which observers maintained lists of items in memory, and then searched through visual arrays to locate whether a member of the memory set was present. Extending Shiffrin and Schneider s investigations, however, Wolfe tested not only memory sets that 21

22 included a small number of items, but ones that contained 8 or 16 items (and, in an extended paradigm, 100 items). Under his conditions of testing, he found that mean RTs were extremely well described as a logarithmic function of memory set size. In a related earlier investigation, Burrows and Okada (1975) examined memory search performance in cases involving memory sets composed of 2 through 20 items. Mean RT was well described as either a logarithmic or bilinear function of memory set size. One of Nosofsky, Cox et al. s (2014) goals was to explore the hypothesis that the principles of exemplar-based retrieval and global familiarity formalized within the EBRW model might provide an account of the curvilinear relation between mean RT and set size observed in probe-recognition paradigms that include longer list lengths. A second major goal of Nosofsky, Cox et al. s (2014) study was to investigate from a model-based perspective how relations between targets and foils across trials influence the process of probe recognition. Thus, we examined how relations between previously experienced memory sets and current sets impact performance. As I briefly review below, this issue is a classic one in the cognitive psychology of memory and attention, with some of the most famous results in the field aimed at understanding the issue (Schneider & Shiffrin, 1977; Shiffrin & Schneider, 1977). As will be seen, the current modeling involving the EBRW model sheds new light on how the history of experience with previous memory sets impacts performance on current sets. Nosofsky, Cox et al. (2014) tested subjects in three conditions. Following the language from Shiffrin and Schneider (1977), in the varied-mapping (VM) condition, items that served as positive probes (old targets) on some trials might serve as negative probes (foils) on other trials, and vice versa. In the consistent-mapping (CM) 22

23 condition, one set of items always served as positive probes, and a second set always served as negative probes. Finally, in an all-new (AN) condition, on each trial, a completely new set of items formed the memory set (see also Banks & Atkinson, 1974). The VM condition places the greatest demands on the current list context by forcing the observer to discriminate whether a given item occurred on the current list rather than previous ones. The AN condition requires the observer to remember the current list, but requires less contextual discrimination than VM because no target or foil had been presented on earlier lists. The CM condition allows (but does not require) the observer to rely solely on long-term memory and to ignore the current-list context. Schneider and Shiffrin (1977) and Shiffrin and Schneider (1977) demonstrated dramatic differences in patterns of performance across VM and CM conditions in their hybrid memory-visual search paradigms. VM conditions showed the usual pattern that performance depended on memory-set size, and this pattern remained as practice continued. However, in CM conditions performance tended to become invariant with set size as practice continued. As reviewed below, Nosofsky, Cox et al. (2014) observed similar patterns in their study (with AN performance intermediate between CM and VM). The contrasting patterns of performance across VM and CM conditions in visual/memory search is among the most fundamental empirical results reported in the field of cognitive psychology, and provides valuable information concerning how different forms of practice and experience influence controlled versus automatic human information processing. Yet, although Shiffrin and Schneider provided a conceptual theoretical account of the performance patterns in their VM and CM conditions, they did not develop a formal quantitative model. Nosofsky, Cox et al. s (2014) aim was to begin 23

24 to make headway towards developing a unified formal-modeling account of memorysearch performance across VM, CM, and AN conditions. As will be developed below, a successful model would bring together prior research and theory on attention and automatism, visual and memory search, short and long term memory retrieval, and categorization. In the experiment reported by Nosofsky, Cox et al. (2014), there were 50 subjects in each of the VM, CM and AN conditions. The stimuli were 2400 unique object images used in the long-term memory study of Brady, Konkle, Alvarez, and Oliva (2008). In the AN condition, a new set of stimuli was randomly sampled from the complete set of 2400 images on each individual trial. No stimulus was used more than once in the experiment (unless it was an old test probe for the current list). In both the VM and CM conditions, for each individual subject, a set of 32 stimuli was randomly sampled from the 2400 images and served as that subject s stimulus set for the entire experiment. In the VM condition, on each trial, the memory set was randomly sampled from those 32 stimuli. If the test probe was a foil, it was randomly sampled from the remaining members of the 32-stimulus set. In the CM condition, for each individual subject, 16 stimuli were randomly sampled that served as the positive set, and the remaining 16 stimuli served as the negative set. On each trial, the memory set was randomly sampled from the positive set. If the test probe was a foil, it was randomly sampled from the negative set. The memory-set sizes were 1, 2, 4, 8 and 16. The size of the memory set was chosen randomly on each individual trial. The status of the test probe (old or new) was chosen randomly on each individual trial. If the test probe was old, its serial position on the study list was chosen randomly on each trial. Each subject participated for 5 blocks 24

25 of 25 trials each. Further details regarding the procedure are provided by Nosofsky, Cox et al. (2014). The mean correct RTs are displayed as a function of conditions (VM, AN, CM), set size 2, and probe type (old vs. new) in Figure 7 (top panel). The mean proportions of errors are displayed as a function of these variables in Figure 8 (top panel). Mirroring the results from Wolfe (2012) and Burrows and Okada (1975), the mean RTs in the VM and AN conditions get substantially longer as set size increases and the increase is curvilinear in form. That is, the lengthening in RTs occurs at a decreasing rate as set size increases. This pattern is roughly the same for the old and new probes. The lengthening in RTs is much smaller in the CM condition and may be limited to the old probes. Unlike Wolfe s and Burrows and Okada s data, there are substantial proportions of errors in most of the conditions (Figure 8). The overall pattern of error data is similar to the mean RTs, the main exception being a pronounced increase in errors for new items in the VM condition at set-size 16. In addition, across all set sizes and for both old and new probes, mean RTs are longer and error rates are higher in the VM condition than in the AN condition. Clearly, mean RTs are much shorter and error rates are much lower in the CM condition than in the other two conditions. As was the case for the memory-search experiments reported in the earlier sections, we also analyzed the data for the old probes as a joint function of set size and lag. The functions showed the same form as reported earlier in this chapter, suggesting that nearly all the effects of set size on old-item performance was due to the differential lags with which old items were tested. 25

26 6.2. The Extended EBRW Model: Conceptual Description. To account for the role of the history of previous lists on memory-search performance of current lists, we extend the EBRW in straightforward fashion. The key idea is that exemplars from previous lists are not erased from memory with the presentation of each current list. Instead, all exemplars from previous trials of the experiment are stored, albeit with decreased memory strengths and sensitivities due to having been presented in the distant past. In recent years, the precise assumptions I have made concerning the nature of these previous-list traces have been evolving (and will likely continue to evolve as new experiments are conducted). Here, I present a fairly general version of the extended model to date, which is being developed in collaboration with Rui Cao and Richard Shiffrin (Nosofsky, Cao, & Shiffrin, in preparation). The basic idea in the extended model is that when a test probe is presented, it causes the retrieval of not only the old exemplars on the current list, but also exemplars from previous lists in the experiment. These long-term-memory (LTM) exemplars enter the evidence-accumulation process of the random walk in the same manner as the current-list exemplars. The probability with which an LTM exemplar is retrieved depends jointly on its memory strength, its similarity to the test probe, and the extent to which any context elements associated with the LTM exemplar match or mismatch the current list context (cf. Raaijmakers & Shiffrin, 1981; Howard & Kahana, 2002). An important open question, however, concerns that extent to which the old and new labels that are associated with the LTM exemplars are themselves stored with 26

27 those exemplars. In the case of what I will term a familiarity-only version of the model, the response labels are not part of the exemplar traces. Instead, if an LTM exemplar is retrieved, it always causes the random walk to take a step toward the OLD threshold. An alternative version of the extended model, which I will term a labeling model, assumes that the old and new labels associated with the test probes on previous trials are stored along with the exemplars themselves. So, for example, it testprobe T was old on a given trial, then a representation of T-old would be stored in memory; whereas if T was a new test probe on that trial, then a representation of T-new would be stored in memory. In making an old-new decision for the current list, if an LTM exemplar is retrieved that has an old label, then the random walk takes a step in the direction of the OLD threshold. Crucially, however, if the LTM exemplar that is retrieved has a new label, then the random walk steps toward the NEW response threshold. Note that this labeling version of the model is basically a type of exemplarbased categorization model (as originally formalized by Nosofsky & Palmeri, 1997), with the categories being old versus new. The most straightforward assumption is that the labeling/categorization strategy does indeed apply in the consistent-mapping (CM) version of the task. Indeed, one of the key hypotheses advanced by Shiffrin and Schneider (1977) is that a major component in the development of automatic processing in CM memory-search tasks involves the use of categorization (for more extended discussion and debate, see, e.g., Cheng, 1985; Logan & Stadler, 1991; Schneider & Shiffrin, 1985). Note that under CM conditions, there is one fixed set of items (the positive category) that always receives an old 27

28 response and a second fixed set of items (the negative category) that always receives a new response. In principle, the observer does not need to pay any attention to the current memory set in order to perform the task: If the test probe belongs to the positive category then the observer can respond old, and likewise for the negative category. It seems plausible that, following sufficient practice, the observer can develop these longterm categories and use them effectively for performing CM search. In the case of VM and AN search, however, the situation is not as clear-cut. Because assignment of previously presented exemplars to old and new responses is not diagnostic in VM and AN memory search, it seems that the best strategy would be to try to ignore the previous trials and focus solely on the current list. However, the recording of previous exemplars (along, perhaps, with their response labels) may simply be an automatic component of the memory system (e.g., Logan, 1988), so that previous exemplars (and their associated response labels) may enter into the evidenceaccumulation process in VM and AN search as well The Extended EBRW Model: Formal Description. Given the conceptual development provided above, the formal extension of the EBRW model to accounting for the role of previous list history is as follows. First, the core mechanisms in the model (formalized in Equations 1-4) continue to operate in all conditions. That is, presentation of the test probe leads to the probabilistic retrieval of the exemplars and criterion elements associated with the current list, and the retrieval of these exemplars and criterion elements drives the random-walk process in the same manner as already described. As explained earlier in this section, however, the LTM exemplars may also 28

29 be retrieved during the memory-search process and will contribute to the direction and rate of drift of the random walk. We presume that by directing different context cues towards the retrieval process (e.g., Raaijmakers & Shiffrin, 1981), the observer can potentially give differential weight to the information from the current list and from LTM in the memory-search process. Let w List (k) (0 < w List (k) < 1) denote the weight that the observer gives to the current list in each main memory-search condition k (k = AN, VM, CM); and let w LTM (k) = 1 - w List (k) denote the weight that the observer gives to the LTM exemplar traces. Furthermore, let Old-O(k) denote the activation of all old LTM exemplars given presentation of an old (O) test probe in condition k; Old-N(k) denote the activation of all old LTM exemplars given a new (N) test probe; and analogously for New-O(k) and New-N(k). Then, for an old test probe, the probability that each individual random-walk step moves toward the R OLD response threshold is given by p old = [(w List A i + w LTM Old-O)]/ ([ (w List A i + w LTM Old-O) + (w List C + w LTM New-O)]. (5) For example, the random walk steps toward the R OLD threshold anytime an old exemplar from the current memory set is retrieved (measured by w List A i ), or anytime an old exemplar from LTM is retrieved (measured by w LTM Old-O). Conversely, the random walk steps toward the R new threshold anytime that a criterion-element is retrieved (measured by w List C), or anytime an exemplar from LTM is retrieved that is associated with the NEW category. (Note that the probability that the random-walk moves toward the R new threshold on any given step is simply q old = 1-p old.) 29

30 Analogously, for new test probes, the probability that each individual randomwalk step is toward the R OLD response threshold is given by p new = [(w List A i + w LTM Old-N)]/ ([ (w List A i + w LTM Old-N) + (w List C + w LTM New-N)], (6a) whereas the probability that the random walk steps toward the R new threshold is simply q new = 1- p new = [(w List C + w LTM New-N)]/ ([ (w List A i + w LTM Old-N) + (w List C + w LTM New-N)]. (6b) (It should be emphasized that, in this notation, the probability of taking steps toward the R old and R new thresholds is denoted by p and q, respectively; whereas the type of test probe [old vs. new] is denoted by the subscript on p and q.) Thus, Equation 6b formalizes the idea that, for new test probes, the random walk correctly steps toward the NEW threshold anytime that a criterion element is retrieved or anytime that an LTM exemplar is retrieved that is associated with the NEW category label. Different versions of the model arise depending on the parameter settings given to the LTM exemplars. For example, setting the New-O and New-N parameters equal to zero yields a pure familiarity -based model, in which retrieval of exemplars from LTM always moves the random walk towards the OLD threshold. Indeed, Nosofsky, Cox et al. (2014) found that a version of such a pure familiarity-based model yielded excellent 30