The attention cascade model and attentional blink q

Transcription

1 Cognitive Psychology 56 (2008) The attention cascade model and attentional blink q Shui-I Shih * School of Psychology, University of Southampton, Highfield SO17 1BJ, UK Accepted 7 June 2007 Available online 10 July 2007 Abstract An attention cascade model is proposed to account for attentional blinks in rapid serial visual presentation (RSVP) of stimuli. Data were collected using single characters in a single RSVP stream at 10 Hz [Shih, S., & Reeves, A. (2007). Attentional capture in rapid serial visual presentation. Spatial Vision, 20(4), ], and single words, in both single and dual RSVP streams at 19 Hz [Potter, M. C., Staub, A., & O Connor, D. H. (2002). The time course of competition for attention: Attention is initially labile. Journal of Experimental Psychology: Human Perception and Performance, 28(5), ]. The model adopts similar architecture of the cognitive accounts of attentional blinks and employs computational details from theories of attention gating. The model has elaborated working memory and attention control mechanism. Both bottom-up and top-down salience are explicit in the model. Quantitative fits are good and the model parameters have plausible values. The model handles stimulus competition, lag 1 sparing, intrusion errors, and magnitude of the dip; it also accounts for commonly observed effects such as stimulus similarities (local and global), target+1 blank, and stimulus salience. Ó 2007 Elsevier Inc. All rights reserved. Keywords: Attentional blink; Attention gating; Rapid serial visual presentation 1. Introduction Rapid serial visual presentation (Potter & Levy, 1969) has been a useful tool in investigating the temporal dynamics of visual selective attention. In RSVP, visual events suc- q I thank Nick Donnelly for his support for the research in his role as the Deputy Head (Research) and then the Head of the School. I thank Adam Reeves for suggesting the name attention cascade for the model, for his criticisms on previous drafts, and for his comments on the article. I also thank Steven Glautier for comments on the article. * Fax: address: sis@soton.ac.uk /$ - see front matter Ó 2007 Elsevier Inc. All rights reserved. doi: /j.cogpsych

2 Shui-I Shih / Cognitive Psychology 56 (2008) ceed one another at the same spatial location at a rapid rate (e.g., 8 14 events per second). A robust phenomenon called the attentional blink or AB (Raymond, Shapiro, & Arnell, 1992) emerges when observers search for two (or more) targets in RSVP streams: report of the second target (T2) is impaired when T2 lags the first target (T1) by ms or so. There are various cognitive accounts for AB. These include the inhibition model (Raymond et al., 1992), interference model (Shapiro, Raymond, & Arnell, 1994; see also Isaak, Shapiro, & Martin, 1999), two-stage model (Chun & Potter, 1995; see also Giesbrecht & Di Lollo, 1998), two-stage competition model (Potter, Staub, & O Connor, 2002), central interference model (Jolicoeur, 1999), hypothesis of attentional dwell time (Duncan, Ward, & Shapiro, 1994), and hypothesis of temporal loss of control (Di Lollo, Kawahara, Ghorashi, & Enns, 2005). These accounts all capture the essential point that processing T1 temporarily prevents adequate processing of T2, but do so by postulating a variety of different structural mechanisms. Recently several connectionist models of AB have been published (e.g., Bowman & Wyble, 2007; Chartier, Cousineau, & Charbonneau, 2004; Dehaene, Sergent, & Changeux, 2003; Fragopanagos, Kockelkoren, & Taylor, 2005; Nieuwenhuis, Gilzenrat, Holmes, & Cohen, 2006). These models are all connectionist networks, involving numerous parameters to capture complex interactions (e.g., excitation or inhibition) within and between layers of artificial neurons. Although such networks admirably capture details of the AB phenomena, they do so by permitting local, idiosyncratic influences, and they abjure the constraints provided in a broader cognitive framework. In contrast, the structural models are less able to capture all the details, and may not map neatly onto the physiology, but they have the merit of employing cognitive mechanisms such as attention and working memory that are already well established. The main purpose of the present work, therefore, is to improve on the current structural models in order to capture more detail concerning the AB without losing sight of the bigger picture. In this article, I propose a mathematical model, the attention cascade model, which uses a relatively small number of parameters to provide a fairly detailed characterization of the system that processes RSVP streams. The model consists of several processing stages each of which is described by a simple mathematical function that may, or may not, reflect the net output of an ensemble of neurons (i.e., no commitment to a particular physiological implementation is made). The attention cascade model uses an architecture similar to those seen in cognitive accounts of AB and employs computational details from theories of attention gating (Reeves & Sperling, 1986; Shih & Sperling, 2002; Sperling & Weichselgartner, 1995). The following presents a brief summary of cognitive accounts of AB and an overview of the attention cascade model. Next, the functional forms, equations, and parameters of the model are illustrated while applying the model to the data of Shih and Reeves (2007). The model is also applied to the data of Potter et al. (2002) whose experimental parameters are substantially different from most AB studies. Finally, a number of theoretical implications are addressed in the General discussion. 2. Cognitive accounts of the attentional blink (AB) There are two classes of cognitive theories of AB (McLaughlin, Shore, & Klein, 2001): interference models (e.g., Isaak et al., 1999; Shapiro et al., 1994), and bottleneck models (e.g., Chun & Potter, 1995; Giesbrecht & Di Lollo, 1998; Jolicoeur, 1999; Potter et al.,

3 212 Shui-I Shih / Cognitive Psychology 56 (2008) ). For simplicity, one exemplar of each class will be described: the interference model of Shapiro et al. (1994) and the two-stage model of Chun and Potter (1995). These models have five common assumptions. First, each RSVP item that receives sufficient sensory processing activates its trace in long-term memory (LTM). The activated traces give rise to preliminary representations, which may not be complete but which are sufficient to identify items within the class of possible items in the RSVP stream. Second, these preliminary representations may be fleetingly accessible to awareness but are not reportable later unless they have been transferred to and consolidated in working memory. Third, transfer is initiated by any item that sufficiently matches the closest target template (consisting of, for example, perceptual and conceptual features that define the targets). The degree of similarity between a stimulus and the target templates defines the top-down salience of the stimulus. T1 and T2 are transferred into working memory because of their high top-down salience, whereas distractors (in general) are not. Fourth, the item trailing a target (i.e., T1+1 and T2+1) may also enter working memory, if the target is transferred and if the presentation rate is fast, an imprecision in selection due to inherently limited temporal resolution. Fifth, the AB effect arises from limitations in working memory, although the limitation is construed somewhat differently in the interference and two-stage models. Within the interference model, the working memory limitation that produces the AB effect is based on two functions of top-down salience (Shapiro et al., 1994; see also Isaak et al., 1999). Top-down salience dictates whether entry into working memory is triggered and how much working memory resource is assigned to each admitted representation, subject to availability (see also Duncan & Humphreys, 1989). The interference model holds that working memory requires about ms (the attentional dwell time, Duncan et al., 1994) to process stimuli and that the AB arises when a large fraction of the resources has been allocated to T1 (and T1+1), leaving insufficient resources to T2 which arrives in working memory before T1 processing has completed. That is, the AB reflects the competition of at least four items: T1, T1+1, T2, and T2+1. On the other hand, the working memory limitation producing AB in the two-stage model is based on its partition of information processing in RSVP into two stages (Chun & Potter, 1995). Stage 1 detects potential targets according to a pre-set criterion of topdown salience and the detection of a potential target triggers Stage 2 processing. However, the model assumes that Stage 2 is resource-limited and operates at a processing speed much lower than the presentation rate of RSVP so that it admits no more inputs before the current processing is completed. If T2 arrives before Stage 2 becomes disengaged from processing T1 (and T1+1) it will suffer decay or loss before being identified as a target, reducing the probability of T2 report and yielding the AB. 3. Attention cascade model The attention cascade model makes the same five assumptions as the existing AB theories. A schematic diagram of the model is displayed in Fig. 1. The core of the model is the attention control mechanism that governs and interacts with the remaining components according to instructions, task demands, observer s strategy, inherent limitations, and stimulus characteristics. The model is thus called attention cascade. After entering the sensory processor, Fig. 1 shows that a stimulus may be processed along one or two pathways: one mandatory and the other contingent on the bottom-up

4 Shui-I Shih / Cognitive Psychology 56 (2008) Fig. 1. A diagram of the attention cascade model. ACM, attention control mechanism. salience of the stimulus (see Fig. 1). The mandatory pathway leads to the activation of LTM traces of the stimulus (symbolized as a gradient spot in LTM) and then to the formation of a preliminary representation of the stimulus that is temporarily available in the peripheral buffer. The buffer is equivalent to Potter s (1993) conceptual short-term memory and the two-stage model s Stage 1 (post-categorical) buffer (Potter, Chun, Banks, & Muchenhoupt, 1998). The amount of information in a preliminary representation transferred to the buffer of working memory is determined by the onset and duration of the attention window. There are two ways to initiate the window and the initiation is independent of whether working memory is engaged. The attention window can be triggered by a preliminary representation that exceeds a response criterion of top-down salience, which is preconfigured in the attention control mechanism according to instructions, task demand, and stimulus characteristics. The transfer contingent on the top-down salience of a stimulus is called controlled attention gating (Reeves & Sperling, 1986; Shih, 2000; Shih & Sperling, 2002; Sperling & Weichselgartner, 1995), which involves four processing stages (sensory, LTM, peripheral buffer, and attention control mechanism) in the attention cascade model. Another way to initiate the attention window is contingent on the bottom-up salience of a stimulus the degree of physical differences between the stimulus and its leading stimuli in time. The model uniquely assumes that sensory processing of a sufficiently bottom-up salient stimulus can directly signal the attention control mechanism to initiate the attention window, shortening the delay in transferring its preliminary representation to the buffer of working memory (see also Shih, 2000; Weichselgartner & Sperling, 1987). This automatic mode (involving two stages: sensory and attention control mechanism)

5 214 Shui-I Shih / Cognitive Psychology 56 (2008) is independent of the distribution of top-down salience of stimuli on a particular trial, although the threshold for automatically signaling the attention control mechanism might need optimization across trial blocks in a manner depending on both types of salience. The model assumes that the width of the attention window depend on the presentation rate, task demand, and inherent limitations. For example, the width for a typical AB task would be narrower than that for a task requiring participants to report 3 or 4 consecutive items after detecting a prescribed cue (e.g., Di Lollo et al., 2005; Nieuwenstein & Potter, 2006; Weichselgartner & Sperling, 1987). However, the model assumes that the width remains constant given invariable presentation rate and task demand in a block of trials. Any attention window needs to be triggered, and it is clear that triggering times will vary from trial to trial depending on random as well as systematic influences. While AB theories only consider systematic (cross-condition) variations in triggering times, the attention cascade model also incorporates a random component. In principle, the width of the attention window should also be subject to random influences. However, all the models, including the present one, take the window width to be constant, primarily to make calculations tractable. Thus on a trial in which the trigger is relatively late, the entire window is moved forward in time, rather than being shortened. The model characterizes each item in the working memory buffer by a strength value: the integral of the product between the item s perceptual availability and the attention window (Shih & Sperling, 2002). If the strength is stronger than the response threshold (e.g., due to long exposure duration), the item is passed to the response buffer ready for output; otherwise it needs to be consolidated. The strength of a representation is subject to decay or interference if its admission to the consolidation processor in working memory is delayed. Once the processor becomes available, it admits representations in the buffer at that moment as a batch. Similar to the interference model, the processor is assumed to allocate resources to inputs according to their top-down salience subject to resource availability. Because of possible imprecision in stimulus selection at the stage of attention gating, consolidation processing begins with item screening. For example, each item strength may be weighted by the item s top-down salience. Like the two-stage and central interference models, the attention cascade model assumes that consolidation is time consuming and forms the bottleneck of the system: once the consolidation processor is engaged, it takes no further inputs until the current processing is completed or aborted except when a specific processing strategy is engaged (e.g., to report three or more consecutive items in RSVP, Nieuwenstein & Potter, 2006). The processor interacts with LTM so that LTM can select relevant memory traces and reinforce their consolidation in working memory. Unlike existing AB models which do not quantify the consolidation process, the attention cascade model specifies that an item s strength grows during consolidation processing and, at the end of the processing, the item is identified if its final strength is greater than the current noise in the processor, otherwise a guess (if required) is produced. The identified item or guess is stored in the response buffer until it is requested. As in Shih and Sperling (2002) and Reeves and Sperling (1986), the model further assumes that the perceived temporal order of items processed in the same batch depends on their relative strengths. This property is not specifically modelled in the present application because T1 and T2 were sampled from different sets in Shih and Reeves (2007) and no related data is available for Potter et al. (2002). By incorporating internal noises into the processor, the model stipulates how item strengths reflect the probability of target report.

6 Shui-I Shih / Cognitive Psychology 56 (2008) Application to data of Shih and Reeves (2007) This section applies the attention cascade model to quantitatively predict the data of two experiments from Shih and Reeves (2007) to illustrate the functional form (or equation) for each component and the meaning of the model s parameters. The experiments are described first, and followed by illustrations of the model application Methods and results The two experiments used identical parameters in stimuli and presentation. Stimuli were colored single digits and capital letters. Colors were photometrically equiluminant so that stimuli of different colors were processed at approximately the same rate and subject to equivalent forward and backward masking. The bottom-up salience of stimuli was thus defined by color. For example, among yellow items in RSVP, a red item is highly bottom-up salient, while a yellow item is not. The stimulus onset asynchrony (SOA) was 100 ms. In the first Experiment, the T1 salience, T2 salience, and target onset asynchrony (TOA) were independently varied. The dependent measures were P(T1), the identification accuracy of T1, P(T2), the identification accuracy of T2, and P(T2 T1), the identification Fig. 2. Experiment 1, Shih and Reeves (2007). Panels on the left display the data, panels on the right the model predictions. The curve parameters are indicated in the legend and in (a). Vertical lines depict standard errors of the means.

7 216 Shui-I Shih / Cognitive Psychology 56 (2008) accuracy of T2 given correct T1 identification. The results are presented in Fig. 2, showing three main findings. First, bottom-up salient targets were more accurately identified than non-salient targets (the target saliency effect). Second, T1 salience has little influence on the AB, while T2 salience appears to moderate the AB. (Note: If T2 performance had been kept under ceiling, one would expect an upward curve shift from non-salient to salient T2 conditions.) Third, at the TOA of 100 ms, P(T2) was greater than P(T1) (T2 superiority) when T1 was non-salient. In the second experiment, two items (S1 and S2) were made bottom-up salient in each RSVP. S2 was a distractor and T2 was non-salient. Varied between blocks was the task relevance of bottom-up saliency (relevant or irrelevant). In the relevant condition, S1 was T1 (i.e., T1 was salient). In the irrelevant condition, S1 was a distractor and T1 non-salient (i.e., no target was ever salient). A critical manipulation was the S2 T2 lag: S2 either led or trailed T2 (i.e., as T2 1 or T2+1 item). The S2-as-T2+1 condition was used as the baseline because the performance in this condition did not differ from the condition without S2 (Shih, 2004). The results are presented in Figs. 3 and 4, which again reveal the target saliency effect and T2 superiority at 100 ms TOA condition given nonsalient T1. The critical finding is that a salient distractor (S2) leading T2 improves P(T2) and P(T2 T1). Fig. 3. Salience-relevant condition, Experiment 2, Shih and Reeves (2007). In this condition, T1 is salient and T2 non-salient. T1 is the first salient item in RSVP, and S2 (a distractor) the second. Panels on the left display the data, panels on the right the model predictions. The curve parameters are indicated in the legend. Vertical lines depict standard errors of the means.

8 Shui-I Shih / Cognitive Psychology 56 (2008) Fig. 4. Salience-irrelevant condition, Experiment 2, Shih and Reeves (2007). In this condition, no target is salient, and both salient items in RSVP are distractors. Panels on the left display the data, panels on the right the model predictions. The curve parameters are indicated in the legend. Vertical lines depict standard errors of the means Application of the attention cascade model As Shih and Sperling (2002), the present modeling used 100 ms as one unit of time wherever the time dimension is concerned. This implementation is to bring values of parameters and variables into a reasonable range. For example, 30 and 280 ms were, respectively, implemented as 0.3 and 2.8 time units. However, for convenience and meaningfulness, descriptions concerning the time are referred in terms of milliseconds whenever appropriate From stimuli to peripheral buffer Because (a) the contrast of stimuli is high and remains constant for all conditions and (b) all stimuli are displayed at the fixation, it is sufficient to characterize a stimulus in the sensory processor by a constant when the stimulus is perceptually available (i.e., including physical stimulation and visual persistence, Shih & Sperling, 2002). Because the stimuli (digits and letters) are highly familiar, it is assumed that their LTM traces are automatically activated to an equivalent degree. Thus, the availability of the preliminary representation of a stimulus in the peripheral buffer can be simply characterized by a rectangular function, which takes the value 1 when the stimulus is perceptually available and 0 otherwise. See Fig. 5a for an illustration.

9 218 Shui-I Shih / Cognitive Psychology 56 (2008) Fig. 5. Examples of functions implemented in the attention cascade model. The x-axis represents the time with one time unit equivalent to 100 ms. The y-axis in (a) and (b) is dimensionless. (a) Depicts two rectangular functions, one representing the preliminary representation (PR) for Item i and the other the attention window (AW). The PR is perceptually available (i.e., y = 1) for the duration of SOA (100 ms = 1 time unit) and the AW operates from a i to a i + w. The overlap between the two functions (i.e., the dotted area) indicates the initial memory strength of Item i. (b) Second- and fourth-order gamma density functions (with a time constant of 12 ms), respectively, representing automatic and controlled triggering time distributions of the attention window. The numbers in parentheses indicate (Mean, SD) of the associated function. (c) Portrays two strength decay functions (dotted lines) over the duration queuing for the consolidation processor and two strength growth functions (solid lines) over the consolidation duration Attention window The window is interfaced between the peripheral buffer and working memory buffer to accept certain stimuli for advanced processing. The attention window is also assumed to be a rectangular function, 1 with the width parameter w configured according to the task demand and presentation particulars but subject to inherent limitations (Fig. 5a). For example, given an SOA of 100 ms in RSVP, a task requiring a report of three consecutive items would prescribe a wider window (e.g., 300 ms) than a typical AB task (e.g., 90 1 Sperling and Weichselgartner (1995) and Shih and Sperling (2002) characterize an attention window as transitions between different attentional episodes an observer s readiness to react to stimuli as a function of spatial location. Applied to search in RSVP, three episodes can be defined for each target: a pre-target, target, and post-target episodes. The pre- and post-target episodes can be represented by 0 indicating no attention is allocated for the transfer, while the target episode by 1 indicating maximum attention for transferring is allocated. A rectangular function for the attention window assumes instantaneous transition between episodes. Although such assumption is unrealistic, it is adopted here because the estimated transition time constant was small: about 5 10 ms in Shih and Sperling (2002).

10 100 ms). This is because a typical AB task has a variable number of distractors between the targets, a wide window (e.g., 300 ms), if used, would admit more distractors into working memory, increasing processing difficulty Initiation of the attention window The attention window is initiated in one of two modes: automatic or controlled. The automatic mode is followed when the sensory processor detects a sufficiently bottom-up salient stimulus: the processor signals the attention control mechanism to open the attention window. The initiation follows a controlled mode if a stimulus lacks bottom-up saliency: the attention window is initiated after the attention control mechanism has deemed the top-down salience of the stimulus acceptable (i.e., sufficient match between the preliminary representation and any one of the target templates). The model assumes that the random variables of the processing times in the four stages (sensory processor, LTM activation, peripheral buffer, and attention control mechanism) are independently and identically distributed as an exponential probability density function (ke kt ) (i.e., one-stage RC-circuit or first-order gamma function). Conceivably, the time constant (b = 1/k) may vary from stage to stage. However, quantitative fits are good when the same time constant is assumed for the four stages. The present implementation therefore applied the same time constant b to these four stages. (This simplification may be an accident due to the RSVP method and may not be generalized to experiments using longer stimulus exposure durations.) Consequently, relative to the onset of Stimulus i, the onset (or triggering time) of the attention window is a random variable determined by the sum of two (automatic) or four (controlled) exponential functions. The resulting distribution is called second- or fourth-order gamma density function: f ðtþ ¼ Shui-I Shih / Cognitive Psychology 56 (2008) b a ða 1Þ! ta 1 e t b a ¼ 2 or 4 ð1þ where a (the order parameter) corresponds to the number of processing stages. From Eq. (1) it follows that the mean and variance of f(t) are, respectively, given by ab and ab 2. Thus, although the triggering time varies from trial to trial, it is generally faster and less varied in the automatic mode than the controlled mode (see Fig. 5b) Item strength in the working memory buffer The working memory buffer accumulates (mathematically integrates) the preliminary representation passing through the attention window. The amount of information accumulated for Stimulus i defines its item strength s i, which is the overlapped area between the two rectangular functions one for the preliminary representation and one for the attention window (see Fig. 5a). Using 100 ms as one time unit, s i takes a value between 0 and f, the duration of the item s perceptual availability, inclusive. The f is typically the SOA (e.g., 1 time unit in Shih & Reeves (2007)) unless it is varied for a particular item; for example, the f of T1 is lengthened by replacing the T1+1 with a blank. If s i is stronger than the response threshold h, the item is passed to the response buffer ready for output; otherwise, it needs to be consolidated. The parameter h cannot be estimated independently, and modeling the present study does not require it. However, this parameter plays a role in the effect of +1 blank (e.g., Raymond et al., 1992) and it will be addressed in the General discussion. When advancing items from the working memory buffer to the consolidation processor is delayed, the attention cascade model assumes that s i decays

11 220 Shui-I Shih / Cognitive Psychology 56 (2008) exponentially with the decay rate equal to the quantity (1 s i ) so that the higher the initial strength is, the slower the decay (see Fig. 5c). 2 Thus, at the end of queuing for a duration d (d 0), the remaining strength q i for Stimulus i is: q i ¼ s i e dð1 siþ ð2þ Consolidation processor Item strengths are changed along with the successive operations in the consolidation processor. First, the processor weighs each item strength with the item s top-down salience because the distractor(s) trailing a potential target may pass through the attention window. In the present implementation, a target/digit is weighted by 1 and a distractor/letter 0 (i.e., reset q i to 0). Second, if the sum of the weighted item strengths ( P q i ) exceeds the limited capacity C of the processor, a new strength value r i is defined by multiplying q i with the ratio C= P q i ; otherwise, r i = q i. That is, ( q i PC ; if C < P q r i ¼ q k k k k ð3þ q i ; otherwise Thus, if the processor is nearing its capacity limit, all items are weakened just enough that the limit is not breached. Third, during consolidation processing, r i grows as a cumulative density function of an exponential distribution with the growth rate equal to the quantity r i so that the higher the r i is the more rapidly its strength grows (see Fig. 5c). 2 Thus, after a processing duration p, the resulting strength v i of Stimulus i is: v i ¼ r i þ r i ð1 e rip Þ At the end of the processing, the output will be the identity of Stimulus i if its resulting strength v i is greater than the internal noise at that moment; otherwise a guess will be made. A Gaussian distribution with a mean l n and a standard deviation r n is assumed for the internal noise. Because the internal noise reflects the trial to trial variation, the current application assumes constant processing duration p Estimating model parameters The functional forms rectangular function for preliminary representations, Gamma density function for the triggering time of the attention window, rectangular function for the attention window, Gaussian for internal noise are assumed and not formally estimated. There are six free parameters: the time constant b of pre-attention-window processing stages, the width w of the attention window, the capacity C of the consolidation processor, the duration p of consolidation processing, the mean l n and standard deviation r n of the internal noises. They are assumed invariant between conditions. ð4þ 2 This assumption is empirically motivated. The fitting with a fixed rate failed. When including several rate parameters to accommodate different conditions, the quantitative fits were also disappointing. The poor fits should not be surprising because the item strength varies from trial to trial due to the variability in the onset of the attention window. Using the item strength to derive its rate of decay or growth can accommodate the range of variability that experimentally defined conditions cannot.

12 Shui-I Shih / Cognitive Psychology 56 (2008) Table 1 Parameter estimates regarding Shih and Reeves (2007) and Potter et al. (2002) Parameter Shih and Reeves Potter et al. Experiments 1 and 2 Experiment 1 Experiments 2 and 3 b, Time constant (ms) w, Width of attention window (ms) C, Capacity of ICP p, Duration of ICP (ms) l n, Mean of ICP noise (ms) r n, SD of ICP noise (ms) Mean R (0.79) a Note. Mean R 2 is an average of 100 R 2, each computed from a Monte Carlo simulation with 1000 trials per condition; each R 2 is adjusted for the number of free parameters. a The mean R 2 without adjusting for the number of free parameters. The parameters are estimated to simultaneously predict the group data of Shih and Reeves (2007) on P(T1), P(T2), and P(T2 T1) in 50 conditions across the two experiments. There are 28 conditions in Experiment 1: 2 T1 Salience 2 T2 Salience 7 TOA, and 22 conditions in Experiment 2: 12 in the salience relevant condition (2 S2 T2 Lag 6 TOA) and 10 in the salience irrelevant condition (2 S2 T2 Lag 5 TOA). The details of finding the optimum parameters are described in Appendix A. Table 1 lists the optimum parameter estimates. The best fitting curves are presented in Figs The model parameters have plausible values and the model reasonably accounts for the data. I defer the discussions after the next section, showing how the model accounts for data observed from experiments that used presentation parameters quite different from conventional AB research. The critical point here is that the methods of Shih and Reeves (2007) were parametric but fairly conventional, and these data are representative of those from a large variety of other studies, so the ability of the model to fit them well is a critical test of generalizability. 5. Application to data of Potter et al. (2002) This section applies the attention cascade model to quantitatively predict the data of Experiments 1 3 from Potter et al. (2002). The study used much faster presentation rate and shorter TOAs than most AB experiments. It therefore represents a test of generalizability beyond the conventional data exemplified by Shih and Reeves (2007) Method and results The SOA was 53 ms in each RSVP stream. T1 and T2 were words, distractors rows of ampersands or percentage signs. Thus, words appeared distinct in RSVP streams. On each trial, a single RSVP stream was displayed in Experiment 1, while two simultaneous (but unsynchronized) RSVP streams (above and below the fixation) in Experiments 2 and 3. By varying the onset asynchrony between the two RSVP streams in a trial, it is possible to produce a TOA that is shorter than the SOA. The set of TOA (in ms) was {53, 107, and 213} in Experiment 1, {0, 40, 107, and 213} in Experiment 2, and {0, 13, 27, and 40} in Experiment 3. The dependent measures were P(T1) and P(T2). The results are

13 222 Shui-I Shih / Cognitive Psychology 56 (2008) presented in Fig. 6, which shows T2 superiority at the TOA between 13 and 53 ms (and 107 ms in Experiment 2), and T1 superiority at the TOA of 213 ms Applying the model to the data Because the appearance of words is sufficiently different from that of rows of ampersands and percentage signs (Chun & Potter, 1995), the attention window is assumed to be initiated automatically by targets. The simulation requires new parameters for Experiments 2 and 3 to accommodate off-fixation stimulation, potential perceptual interference from unsynchronized RSVP streams, and a potential increase in the number of items entering the working memory buffer. Because the number of data points is small and characterizing these factors is not critical to the present purpose, the current application absorbs these factors into the internal noise. Consequently, two sets of mean l n and standard deviation r n of the Gaussian noise are estimated: one set for Experiment 1 and the other Experiments 2 and 3. The remaining parameters (pre-attention-window processing time constant b, width of attention window w, consolidation capacity C, and consolidation duration p) were estimated using the data of all three experiments simultaneously. Table 1 lists the optimum parameter estimates. The best fitting curves are presented in Fig. 6. Again, the model parameters have plausible values and are similar to those Fig. 6. Experiments 1 3, Potter et al. (2002). Panels on the left display the data, panels on the right the model predictions. The curve parameters are indicated in the legend. Vertical lines depict standard errors of the means.

14 estimated for the data of Shih and Reeves (2007). Although less than 80% of variance in the data is accounted for by the model, Fig. 6 shows that the model predictions replicate the pattern of the results. Given the small number of data points and substantial differences among experiments, the model did a reasonable job in this application. 6. General discussion The present study applies the attention cascade model to simulate the results of two studies that employed very different experimental parameters. Shih and Reeves (2007) closely followed conventional AB procedure, using an SOA of 100 ms in single RSVP, varying the TOA between 100 and 800 ms, and using single characters as stimuli. On the other hand, Potter et al. (2002) employed an SOA of 53 ms, used both single and dual RSVP, densely sampled the TOA between 0 and 213 ms, and used 4- or 5-letter words as targets. Comparing to the performance of neural network models (e.g., Bowman & Wyble, 2007; Chartier et al., 2004), the attention cascade model does a reasonably good job in quantitative predictions (Figs. 2 4 and 6) Interpreting the parameters Shui-I Shih / Cognitive Psychology 56 (2008) Pre-attention-window processing The model specifies the attention window as the means to admitting stimuli into working memory, and assumes two routes to initiate the window. A physically salient item initiates the fast, automatic route, resulting in a shortened delay of the window onset relative to the onset of the item s preliminary representation. It is assumed that the four stages prior to the window have the same processing rate. The assumption is consistent with the finding that the same b value works well for both automatic and controlled routes for Shih and Reeves (2007). Furthermore, despite the numerous differences in methods between Potter et al. (2002) and Shih and Reeves, similar b estimates (12 14 ms) are obtained for the two studies, suggesting the estimate may be characteristic of similar populations (university students) Attention window The estimated width w of the attention window for Shih and Reeves (2007) is much wider than that for Potter et al. (2002) (148 vs. 80 ms). The discrepancy is a fair reflection of the different presentation rates used in the two studies (SOA: 100 vs. 53 ms). The ratio of about 3:2 between w and SOA was similar to that observed in Shih and Sperling (2002). Whether the similar relationship is a coincidence remains to be examined. The window width adapts to not only actual but also illusory presentation rate (Akyürek, Riddell, Toffanin, & Hommel, 2007). For a given SOA (e.g., 100 ms), changing the ratio of actual stimulus duration to blank duration affects the perceived speed of RSVP 30 ms stimulus to 70 ms blank (30/70) appears faster than 70 ms stimulus to 30 ms blank (70/30). Akyürek et al. presented participants illusory fast (30/70) or slow (70/30) RSVP in most trials and found that the slow group made more order errors at lag 1 than the fast group. They also found that the event-related potentials indicated one processing episode per trial in the slow group, but two processing episodes per trial in the fast group. These findings suggested that the slow group adopted a wider attention window than the fast group, despite the fact that the actual presentation rate was identical for the two groups.

15 224 Shui-I Shih / Cognitive Psychology 56 (2008) Capacity C of the consolidation processor The estimate for the capacity C of the consolidation processor is very different between the two studies (103 vs. 38). The difference however appears reasonable once the capacity C is interpreted with reference to the SOA. One can conceptualize the quantity C/SOA as the number of items the available resources can process. The C of 103 for Shih and Reeves (2007) corresponds to about one item (i.e., single digit), while the C of 38 for Potter et al. (2002) 0.7 item (i.e., 4- or 5-letter word). Although a word has more information to be processed than a digit or letter, the letters within a word may help each other to recover the word (McClelland & Rumelhart, 1981). Thus, both estimated values appear plausible. Of course, a much wider range of conditions would need to be tested to discover if this relationship holds true in general Duration p of consolidation processing The model produces similar values of p for both studies, suggesting that consolidating a single digit or letter takes nearly as much time as a word. The value (about 500 ms) agrees with that hypothesized by existing AB theories as an attentional dwell time (Duncan et al., 1994; see also Ghorashi, Zuvic, Visser, & Di Lollo, 2003). The value however seems at odds with the estimate of a consolidation rate as fast as about 50 ms per item (Vogel, Woodman, & Luck, 2006). This discrepancy is addressed next Consolidation duration Vogel et al. (2006, Experiment 2) presented on each trial three arrays in succession memory (100 ms), mask (200 ms), and test (2 s). The memory and test arrays each contained four colored squares; the two arrays were identical on half of the trials, but were different in the color of one of the squares on the remaining trials. The task was to indicate whether the two arrays were identical on each trial. They reasoned that the task would require the participants to consolidate the four items (regarding the color and location) in the memory array for later comparison. The critical manipulation was the memory-mask SOA and it was finely sampled between 117 and 317 ms. The dependent measure was K: the number of items worth of information stored in working memory. The results showed that K increased linearly over the first 200 ms SOA and then leveled off at about 2.5 between 200 and 300 ms. The slope of the best fitting function for the linear part of the results was about 0.02 K per millisecond of SOA. They thus concluded that the consolidation rate was about 50 ms per item. According to the attention cascade model, the manipulation of memory-mask SOA in Vogel et al. affects not the consolidation duration, but the perceptual availability of stimuli and, hence, their initial strengths in working memory. Otherwise, it would have been suggested that the consolidation duration for T1 is less than 100 ms. Furthermore, the measure of a processing rate does not necessarily reflect that of a processing duration. In the attention cascade model, the consolidation duration is relatively constant for a given class of stimuli (e.g., 500 ms or so for alphanumeric characters and words); however, the number of stimuli to be consolidated in the same batch may vary. Therefore, the disagreement in the estimate of consolidation rate/duration may be a result of different conceptualizations and can only be resolved through formal modeling.

16 Shui-I Shih / Cognitive Psychology 56 (2008) Some effects This section contrasts the attention cascade model with the other AB models regarding some effects associated with the AB paradigm T2 Superiority, lag 1 sparing, order inversion, and +1 intrusion In AB experiments, P(T1) is generally higher than P(T2) T1 superiority. However, when the TOA is short (e.g., <100 ms or so), T2 superiority may emerge, especially when T1 is non-salient. Lag 1 sparing (Potter et al., 1998) is a special case of T2 superiority; it is so termed because it occurs in experiments using an SOA of about 100 ms so that T2 superiority is present when T2 lags T1 by one item. The results of Potter et al. (2002) show that T2 superiority may emerge even when there are intervening distractors between the two targets (i.e., lag > 1) so long as the TOAs are short enough. The predictions of the attention cascade model are in agreement with these findings (see Figs. 2 4 and 6). The model predicts that the effect emerges when T2 strength is greater than T1 strength while they are processed together in the consolidation processor. The present modeling predicts this would happen in the following conditions: the TOA of 100 ms condition given non-salient T1 in Shih and Reeves (2007), and the TOAs between 13 and 53 (inclusive) in Potter et al. (2002). See Appendix B for detailed illustrations. Lag 1 sparing is often accompanied by the T1 T2 order inversion: When T1 and T2 are sampled from the same stimulus set, participants tend to report T2 before T1 when the TOA is short, especially at lag 1 condition (e.g., Chun & Potter, 1995; Hommel & Akyürek, 2005; Potter et al., 2005). When targets and distractors are sampled from the same stimulus set (e.g., letters) but distinguished by other feature (e.g., colored targets vs. black distractors), the majority of errors in T1 report involve the +1 intrusion (e.g., mistaken T1+1 as T1) rather than T1 misses (e.g., Broadbent & Broadbent, 1987; Raymond et al., 1992). There is a built-in mechanism in the attention cascade model to account for the order inversion and +1 intrusion, although it is not formally implemented in the present application due to data availability. This mechanism is identical to the decision computation implemented in Reeves and Sperling (1986) and Shih and Sperling (2002). That is, at the output of consolidation, the temporal order of stimuli within the same batch of processing is judged by their relative strengths: the onset of a stronger stimulus is perceived as earlier than a weaker one. Thus, in the model, the order inversion reflects that T2 is greater than T1 in strength, and the +1 intrusion the +1 item is greater than the target in strength. Other AB theories also explain these effects. In the interference model, during the attentional dwell time, the T1+1 item has priority next to T1 in receiving attentional resources than its subsequent items; hence T2 is less disadvantaged if it is the T1+1 item, producing lag 1 sparing. Because the interference model allocates resources on a first-come-first-serve basis, it cannot explain T2 superiority unless T2 has higher top-down salience than T1. T2 superiority is particularly addressed in the two-stage competition model (Potter et al., 2002), which suggests that two attentional stages are involved to enable a correct target report. Stage 1 begins with detection of a potential target and ends with its identification. Once a target has been identified begins Stage 2 to consolidate the target. According to the model, attention in Stage 1 is labile resources allocated to one potential target may be reallocated to a new input, while attention in Stage 2 is fixed, which consolidates

17 226 Shui-I Shih / Cognitive Psychology 56 (2008) one target at a time over duration of ms. When the TOA is short enough, T1 and T2 compete for attention in Stage 1 and either may become the first one identified to monopolize Stage 2. The shorter the TOA, the less stable the resources has attached to T1 and, hence, increases the likelihood that T2 may win over T1, resulting in T2 superiority. However, it is unclear how attentional resources are distributed in Stage 1 (e.g., top-down salience). Within what interval would Stage 1 favor newcomers? To what extent can resources attached to T1 be re-distributed to T2? How does Stage 1 processing duration relate to the amount of resources allocated to a potential target? Effect of +1 blank Replacing the item trailing T1 (i.e., T1+1 item) with a blank often eliminates the AB (e.g., Raymond et al., 1992). In the interference model, blank T1+1 does not draw attentional weightings to deprive T2 processing, resulting in attenuated AB. In the two-stage model, blank T1+1 enables faster identification and consolidation of T1, reducing the queuing time for T2 and, thus, reducing the AB. However, because consolidation processing is mandatory in the two-stage model and two-stage competition model, both models would still expect small deficit in the T1+1 blank condition. In the attention cascade model, perceptual availability of T1 is lengthened by T1+1 blank so that on entering working memory its strength may be no less than the strength of a consolidated T1 in a non-blank +1 condition. It is thus plausible to assume that an item can be readily reported if its strength is above certain value (i.e., the response threshold h) (Appendix B). Similarly, replacing T2+1 with a blank or leaving T2 unmasked results in abating of the AB. (Of course, the ability to add an extra parameter to account for the effect of the blank would not discriminate in favor of the attention cascade model unless it was impossible to add such an extra parameter to one of the other models for a like effect.) Effects of similarity between T1 and T1+1 Reduced featural or spatial similarity between T1 and T1+1 distractor increases accuracy in T1 report [P(T1)] and attenuates the AB (Chun & Potter, 1995; Raymond, Shapiro, & Arnell, 1995; but see Ward, Duncan, & Shapiro, 1997). In the interference model, the amount of attentional resources taken up by T1+1 depends on its similarity to targets (i.e., top-down salience); the more similar it is the more resources it attracts, leaving less for T2 and yielding greater AB. However, the T1 T1+1 similarity does not affect the top-down salience of T1 and hence the attentional resources allocated to T1; the interference model would predict P(T1) indifferent to the similarity. In the two-stage model, the duration of Stage 2 processing of T1 depends on the similarity between T1 and T1+1: the more similar they are the more interference and longer processing duration; the variation in the interference accounts for the similarity effect on P(T1) and that in processing duration affects the queuing time for T2 and hence AB. In the attention cascade model, such local similarity affects the amount of resources allocated to T1: the greater the similarity is the fewer resources are allocated to T1 (see Eq. (3)), yielding lower P(T1). The attention cascade model does not explicitly address the modulation effect of T1 T1+1 similarity on the AB; it may however account for the effect via the internal noise: the greater the top-down salience of the T1+1 distractor the greater the internal noise, which would reduce the chance of correctly consolidating not only T1 but also T2.

18 Shui-I Shih / Cognitive Psychology 56 (2008) Effects of global similarity Global similarity refers to the overall discriminability between targets and distractors (e.g., Chun & Potter, 1995). For example, given letter targets, a condition with symbol distractors has lower global similarity than one with digit distractors; among black distractors, colored targets are more discriminable than black targets. They found that both targets were reported more accurately and the AB was much reduced in globally dissimilar conditions (see also Maki, Bussard, Lopez, & Digby, 2003). The interference model would account for these effects in the same way as it does the effects of T1 T1+1 similarity. However, the two-stage model, in addition to the variations in local interference and processing duration in Stage 2, proposes that global similarity affects the target detection criterion: the lower the similarity is the fewer target-relevant features must be detected before triggering Stage 2 processing, yielding earlier start and finish of Stage 2 processing of T1 and, hence, less T2 decay and less AB. The attention cascade model deals with the effect of global similarity in the context of stimulus salience: a stimulus (target or distractor) that is sufficiently bottom-up salient initiates the attention window in an automatic mode; however a non-salient target in a controlled mode. Thus, the attention cascade model suggests a computational model of stimulus salience in the temporal domain Effects of stimulus salience Stimulus saliency has been modeled elegantly in the spatial domain, for example in visual search with multiple objects presented in a single frame (e.g., Itti & Koch, 2000; Kock & Ullman, 1985; Wolfe, Cave, & Franzel, 1989), with the models based on a wide variety of search data (e.g., Bacon & Egeth, 1994; Jonides & Yantis, 1988; Theeuwes & Burger, 1998; Treisman & Gelade, 1980). Standard spatial models consist of a two-dimensional spatial saliency map that encodes the saliency at each location due to both bottomup and top-down factors, coupled with (for example) a winner-take-all network such that the most salient location in the map is selected for subsequent attentional processing. The spatial model cannot be applied directly to RSVP, because, given a single spatial location, the winner-take-all algorithm would ensure that all the RSVP items are to be selected for further processing. An alternative is to compute relative saliency over some time interval. This may be done by assuming a single monitoring window that computes momentary, relative saliency for items falling within the window. The attention cascade model assumes that the sensory processors serve this function irrespective of top-down salience, the sensory processors signal the attention control mechanism to initiate the attention window if it has discovered a bottom-up salient stimulus, otherwise, the window can only be initiated via passing a threshold of top-down salience as determined prior to working memory admission. Although the bottom-up salience of a stimulus may determine whether the attention window is initiated automatically, it does not govern the resource distribution in working memory. Instead, the resource distribution is governed by the top-down salience of the stimulus. The above distinction can be illustrated in the findings of Maki and Mebane (2006). They asked participants to search for a black word (nominally T2) among distractors, which were black false-font strings. On some trials, one distractor (nominally T1) was replaced with a red false-font string, a red consonant string, a red digit string, or a red word. They found that only those trials with a red word or a red consonant string produced the AB. The results are consistent with the notion that a bottom-up salient