Visual attention and manual response selection: Distinct mechanisms operating on the same codes

Transcription

1 VISUAL COGNITION, 2002, 9 (4/5), Visual attention and manual response selection: Distinct mechanisms operating on the same codes Bernhard Hommel Section of Cognitive Psychology, University of Leiden, The Netherlands and Max Planck Institute for Psychological Research, München, Germany Werner X. Schneider Department of Experimental Psychology, Ludwig-Maximilians-Universität München, Germany In four experiments, participants made a speeded manual response to a tone and concurrently selected a cued visual target from a masked display for later unspeeded report. In contrast to a previous study of H. Pashler (1991), systematic interactions between the two tasks were obtained. First, accuracy in both tasks decreased with decreasing stimulus (tone-display) onset asynchrony (SOA) presumably due to a conflict between stimulus and response coding. Second, spatial correspondence between manual response and visual target produced better performance in the visual task and, with short SOAs, in the tone task, too presumably due to the overlap of the spatial codes used by stimulus- and response-selection processes. Third, manual responding slowed down with increasing SOA reflecting either a functional bottleneck or strategic queuing of target selection and response selection. Results suggest that visual stimulus selection and manual response selection are distinct mechanisms that operate on common representations. The present paper deals with the relationship and possible interactions between two human control processes, one concerned with the selection of environmental stimulus information and the other with the voluntary selection of Please address all correspondenc e to B. Hommel, University of Leiden, Section of Cognitive Psychology, Postbus 9555, 2300 RB Leiden, The Netherlands. hommel@fsw.leidenuniv.n l We wish to thank Benjamin Beyer, Irmgard Hagen, Nike Hucke, Nicola Korherr, Albrecht Schnabel, and Alexandra Tins for collecting the data; and John Duncan, Pierre Jolicoeur, Hal Pashler, Mark Van Selst, and Rob Ward for comments on previous versions of this paper. Ó 2002 Psychology Press Ltd DOI: /

2 VISUAL ATTENTION AND RESPONSE SELECTION 393 action. The first one, commonly called attention, has been thoroughly investigated for decades in experimental psychology and, more recently, also in the neurosciences (for recent overviews, see Allport, 1983; Desimone & Duncan, 1995; Posner & Petersen, 1990). Most research in this area, the present work included, has been concentrated on visual attention, hence, the selection of visual stimuli or objects. Although many questions concerning the details of attentional selection are still unsettled (Allport, 1993), most researchers agree in that visual attention is in some sense capacity-limited, dealing with only one object (or few objects) at a time. In the process of selecting a given visual object, spatial stimulus information seems to play a major role, presumably because object features are cross-linked and integrated by referencing their common location in space (e.g., LaBerge & Brown, 1989; Schneider, 1999; Treisman, 1988; Van der Heijden, 1992; Wolfe, 1994). The second control process of interest, commonly called response selection, is assumed to be invoked whenever people are choosing among several possible actions. The response selection mechanism seems to share some characteristics with visual attention. As with stimulus selection, response-selection processes are believed to be strictly limited in capacity, dealing with only one response at a time (for recent overviews, see Pashler, 1993, 1994). Furthermore, studies on phenomena of stimulus response compatibly (for a recent overview, see Hommel & Prinz, 1997) have repeatedly shown evidence of strong, spontaneous interactions between spatial stimulus and response codes, suggesting that spatial information plays an important role not only in stimulus selection, but in action control as well (e.g., Hommel, 1993a; Proctor, Reeve, & Van Zandt, 1992; Wallace, 1971). How are visual attention and response selection, or the processes subserving them, related to one another? Despite some obvious similarities and some arguments for a more integrated view (Allport, 1993; Deubel, Schneider, & Paprotta, 1998; Rizzolatti, Riggio, & Sheliga, 1994; Schneider, 1995), mechanisms of stimulus and response selection have been traditionally treated as separate and independent and, in fact, it may not seem obvious why it should be otherwise. Support for the traditional (although usually implicit) independence assumption comes from a study of Pashler (1991), the first one that tackled this issue directly. In Experiment 1 of that study, participants were presented with a visual attention task while performing a manual binary-choice task (see Figure 1 for our own, somewhat simplified, design version). The manual task required a speeded left or right keypress response (R1) to the pitch of a tone (S1). In the visual attention task, a brief masked letter array was presented. One of the letters was cued as target (S2) that was to be reported at leisure (R2) after the manual response. The stimulus onset asynchrony (SOA) between tone and visual array and, hence, the temporal overlap of the two tasks, varied randomly between 50, 150, and 650 ms. Pashler reasoned that if both manual response (R1) selection and visual stimulus (S2) selection would require the same

3 394 HOMMEL AND SCHNEIDER Figure 1. A schematic illustration of the procedure in Experiment 1. Task 1 requires a speeded left right keypressing response to the pitch of a tone. Task 2 requires an unspeeded judgement of the marked target of a brief, masked four-letter display. capacity-limited mechanism, then visual stimulus selection should suffer the more the greater the temporal overlap with manual response selection. Hence, the accuracy of S2 report should get worse the shorter the SOA. However, the results did not show any dependence of report accuracy on SOA, which led Pashler to conclude that visual attention and manual response selection do not rely on the same mechanism but are independent. In our view, this conclusion might be premature for two reasons. The first reason has to do with Pashler s (1991) emphasis on SOA main effects as an indicator of interdependence between stimulus and response selection. Focusing on main effects makes sense from a bottleneck view, hence if we assume that interdependence necessarily shows up as a non-specific queueing of selection processes. However, interdependence may also produce specific interactions between stimulus and response selection, interactions that do not need to express themselves in SOA main effects. As already mentioned, both the selection of visual stimuli and of manual responses can be assumed to rely and operate on spatial representations. If one adds the assumption of a structural overlap between spatial stimulus and response codes (Hommel, 1997; Prinz, 1990) and/or the processes operating on them (Duncan, 1996; Rizzolatti et al., 1994; Schneider, 1995), one might expect rather specific patterns of spatially mediated interactions between stimulus and response selection. If, for instance, a left-hand keypress response is selected in the manual task, this might facilitate shifting attention to, or selecting, a left-side target stimulus in the visual task. As a consequence, one would expect spatial

4 VISUAL ATTENTION AND RESPONSE SELECTION 395 correspondence effects between (manual-)response location and (visual-)target location. Although such a correspondence effect might vary in size with SOA, thus producing a correspondence-by-soa interaction, it would not yield an SOA main effect. A second reason for us to hesitate accepting Pashler s (1991) negative conclusion is that it is based on investigating only three particular levels of SOA. If both tasks were processing in parallel, these values may not have produced (sufficient) temporal overlap of stimulus-selection processes in task 2 and response-selection processes in task 1 (cf., Hommel, 1993b). Moreover, both particular SOA values and the range of SOA levels have been shown to affect the strategies subjects employ under dual-task conditions, especially with combinations of an easy, speeded manual-choice task and a difficult, unspeeded visual-attention task (De Jong & Sweet, 1994). Thus, it may be that choosing other SOAs increases the temporal overlap of selection processes and/or induces different strategies in ways that do reveal interactions between stimulus and response selection. In summary, we feel that Pashler s (1991) evidence for claiming independence of visual attention and response selection is inconclusive and possibly premature. In the present Experiments 1 and 2 we tested whether focusing on non-specific queueing effects in the original study might have concealed the view on specific interactions between stimulus- and response-selection processes. To do so, we replicated Pashler s first experiment and considered in our analyses not only effects of SOA, but also those of the spatial correspondence between R1, the manual response, and S2, the to-be-selected visual target stimulus. In Experiments 3 and 4 we investigated the role of temporal overlap of stimulus and response selection by using SOAs between 200 and 400 ms, a range that might be expected to increase overlap. EXPERIMENT 1 Experiment 1 had two aims. First, we wished to replicate the most important finding of Pashler (1991, Exp. 1) by using a slightly simplified search task (i.e., four rather than eight letters, see Figure 1), but the same levels of SOA. In the visual attention task, Pashler found no effect of SOA, although he did find one in the manual task, where RTs decreased significantly from 650 to 150 ms SOA. This latter effect was not interpreted as resulting from temporal overlap of stimulus and response selection, but rather as a kind of warning effect an issue to be treated in more detail in Experiment 4 and in the General Discussion. More relevant for the moment, however, is the question of whether the theoretically important null effect of SOA on the attention task can be reproduced. Second, we did not only look for main effects of SOA, but also included analyses of the impact of spatial correspondence between visual target and manual response. If our previously mentioned considerations concerning

5 396 HOMMEL AND SCHNEIDER specific interactions between spatial stimulus and response selection were correct, one would expect target response correspondence to affect performance, whether a main effect of SOA occurs or not. Method Participants. Twenty-four adults were paid to participate in single sessions of about 50 min. They reported having normal or corrected-to-normal vision and audition, and were not familiar with the purpose of the experiment. Apparatus and stimuli. The experiment was controlled by a Hewlett Packard Vectra QS20 computer, attached to an Eizo 9070S monitor via an Eizo MD-B11 graphics adaptor for stimulus presentation, and interfaced with a D/A card (Data Translation 2821) for auditory output. Tone judgements in the manual task were given by pressing the left or right of two microswitches mounted side by side on a slightly ascending wooden plate. Letter responses in the visual-attention task were given by pressing one of four horizontally arranged keys of the computer keyboard (function keys F1 F4, accordingly labelled as A, B, C, and D). Each participant operated the microswitches with the index and middle finger of the right hand and the computer keys with four fingers of the left hand. Auditory stimuli (S1) were sinusoidal tones of 200 and 800 Hz. Each tone was presented simultaneously through two loudspeakers located about 50 left and right of the median plane. Visual stimuli were all taken from the standard text mode font and were presented in white on a black screen. A plus sign served as fixation cross, a vertical line as marker, the uppercase letters A, B, C, and D as stimuli for the visual task (S2), and four Xs as mask. From a viewing distance of about 60 cm, each letter measured 0.3 in width and 0.4 in height. The fixation cross always appeared at screen centre. The four stimulus letters, as well as the four Xs replacing them, were centred at the four stimulus positions 0.6 to the left and right and 0.4 above and below screen centre. The target was indicated by the bar marker appearing 0.3 (edge-to-edge) above the (upper) target or below the (lower) targe (see Figure 1). Design. The experiment consisted of five blocks of 96 randomly ordered trials each, preceded by 40 randomly determined practice trials. The trials in each block resulted from the possible combinations of two tone stimuli or responses (left vs right key), three tone-letter SOAs (50, 150, or 650 ms), four letter targets (A, B, C, or D), two vertical target locations (upper vs lower row), and two horizontal target locations (left vs right). Procedure. The verbal instruction stressed that tone responses should be speeded (within a reasonable accuracy range), whereas letter responses were to

6 VISUAL ATTENTION AND RESPONSE SELECTION 397 be given at leisure. Following an intertrial interval of 1300 ms, each trial began with the presentation of the fixation mark for 1000 ms and a further blank interval of 500 ms. The tone was then presented for 100 ms and the letter display appeared at a variable time after tone onset (50, 150, or 650 ms, depending on the SOA, see Figure 1). Each letter display consisted of the four letters A, B, C, and D, distributed across the four stimulus positions, with the target indicated by the bar marker. The identity and the location of the target letter depended on the respective condition and were thus balanced across trials, whereas the locations of the remaining three nontarget letters were determined randomly in each trial. After a variable exposure duration (see later), the marker was deleted and the reaction stimuli were replaced by the mask, which stayed on until the letter response was given. Following tone onset, the program waited 1000 ms for the tone response. In case of a missing or incorrect tone response, or with anticipations (RT < 150 ms), auditory error feedback was given. The respective trial was recorded and repeated at some random position in the remainder of the block. In order to discourage speeded letter responses, the program did not accept those responses earlier than 700 ms after mask onset. There was no upper temporal limit for letter responses. Exposure duration of the visual display was set to 400 ms during practice trials and then reduced to 200 ms when the first block started. At the end of each block, the duration was individually adjusted according to the error rate in the letter task since the last adjustment: It was reduced by 42 ms (corresponding to three screen-refresh cycles) with an error rate below 20%, but increased by 42 ms with a rate above 30%. After each block, participants were given feedback about their average RT in the tone task and their accuracy in the letter task. On that occasion, they could pause as long as they wished. Results Trials with missing tone responses or anticipations accounted for 2.9% and 0.1% of the data, respectively, and were excluded from analyses. For each participant, mean RTs and proportions of (choice) error (PEs) in the manual task, and PEs for letter responses in the visual task were computed as a function of R1 S2 correspondence and SOA. Tone task. An ANOVA of RTs revealed that reactions were quicker with the two shorter than the longest SOA (462, 461, and 481 ms, respectively), F(2, 46) = 5.50, p <.01. For choice errors all three effects were significant. The main effect of SOA, F(2, 46) = 12.38, p <.001, indicated a monotonous decrease of error rates with increasing SOA (5.7%, 4.4%, and 2.6%, respectively), and the correspondence effect, F(1, 23) = 8.85, p <.01, was due to less errors being made with correspondence (3.5%) as compared with

7 398 HOMMEL AND SCHNEIDER Figure 2. Experiment 1: Reaction times (RTs, in ms) on Task 1 and proportions of errors (PEs, in %) on Task 2 as a function of stimulus onset asynchrony (SOA) and spatial compatibility between primary response and secondary target stimulus. non-correspondenc e (4.9%). As indicated by the interaction, F(2, 46) = 3.76, p <.05, the correspondence effect was restricted to the two short SOAs (effect sizes: 2.2%, 1.9%, and 0.1%). Letter task. The result pattern more or less mirrored the pattern of choice errors in the tone task. The SOA effect, F(2, 46) = 11.50, p <.01, reflected a substantial decrease of error rates from the shortest to the two longer SOAs (26.0%, 18.0%, and 18.9%, respectively), whereas the correspondence effect, F(1, 23) = 7.05, p <.05, was due to fewer errors with correspondence (20.0%) than non-correspondence (21.8%). Again, however, these two effects interacted, F(2, 46) = 53.65, p <.05, indicating that reliable correspondence effect were only obtained for the two shorter SOAs (see Figure 2). Discussion Experiment 1 yielded three relevant results. First, manual RTs did not increase with increasing task overlap, but there was a positive correlation between RTs and SOA instead (i.e., a decrease with increasing overlap). The finding of a positive relationship replicates the results of Pashler (1991) and will be discussed in more detail in Experiment 4 (along with the accompanying negative relationship in manual PEs). More important for now is the absence of a negative relationship, which indicates that manual response selection was more or less (if we ignore the effect in manual PEs for a moment) unimpaired by task overlap. Hence, if there was queuing of stimulus and response selection, it must have been the former that was delayed until the latter was completed. Indeed, the second important result is that, in contrast to Pashler (1991), we did

8 VISUAL ATTENTION AND RESPONSE SELECTION 399 obtain a main effect of SOA in the visual attention task. Following Pashler s reasoning this might indicate the queuing of (manual) response selection and (visual) stimulus selection. That is, stimulus selection might have had, or was scheduled, to await the completion of response selection. Third, however, the picture is complicated by substantial effects of spatial target response correspondence on errors in either task, which points to some interaction between stimulus and response selection. In our view, a simple queuing story is insufficient to account for such an outcome pattern, for the following reasons. One implication of our findings is that subjects seem to have some choice about when to start selecting the visual target. Obviously, correspondence effects on response selection as found in PEs presuppose that the visual target (or its marker) has already been located, which implies that stimulus selection must have at least started in some trials before the manual response was selected. According to the same reasoning, correspondence effects on the visual task indicate that other trials must have been associated with the reverse sequence, that is, target selection must have followed manual response selection. Moreover, correspondence affected manual errors at SOA levels that were not associated with any RT effect. This means that if in some trials target selection did start before the manual response was selected, this occurred without a RT cost. Put differently, working on target selection does not delay response selection. As this also implies the reverse no delay of target selection by response selection it raises two questions: First, why then were selection processes sequenced at all and, second, why did task overlap impair visual performance? To address these questions let us consider how and on what kinds of codes stimulus and response selection processes may operate. A working model One way to conceive this is illustrated in Figure 3A, where we slightly simplified matters by assuming that only two letters (A and B) appear in each visual display. Before a target is selected, so our explanation goes, its elements are read into and temporarily stored in some kind of working memory (cf., Schneider, 1999). As the target letter is not yet selected (for reasons discussed in Experiment 2), all the elements need to be spatially coded otherwise later attempts to select the target letter according to a spatial criterion (i.e., whether or not it occupied the location indicated by the marker) would necessarily fail. This means that letter identities are bound to location codes, and vice versa, just as indicated on the left-hand part of Figure 3A. Responses are also spatially coded, which requires the binding of response identities to location codes (Stoet & Hommel, 1999; Wallace, 1971) as illustrated on the right-hand part of Figure 3A. Let us assume that R1 selection typically precedes S2 selection as suggested by the asymmetry of SOA effects on manual RTs and visual PEs. If in the process of selecting a response one of the spatial codes is activated the

9 Figure 3. Simplified network of bindings between stimulus-identity (i.e., letter-related) codes and location codes on the one hand, and between response-identity (i.e., finger-related) codes and location codes on the other. The example is based on a two-letter display. Panel A shows the basic architecture, assuming that the letters A and B appear on a left and right location, respectively, and that a left and right response are operated by the index and middle finger of the right hand. Panel B illustrates how selecting a (right) response affects stimulus selection, while Panel C shows how selecting a (right) stimulus affects response selection.

10 VISUAL ATTENTION AND RESPONSE SELECTION 401 RIGHT code, say the stored code of the spatially corresponding target (here, the letter B) is also activated via the common link to the RIGHT code (see Figure 3B). Increased activation of a letter representation implies a higher probability of retrieving it from working memory when stimulus selection proper takes place, which supports target selection if this letter actually is the target but makes selection errors more likely if it is not (see the example for non-correspondenc e in Figure 3C). In other words, selecting a response primes the selection of (codes of) spatially corresponding stimuli. This model can nicely account for our correspondence effects. As the stronger correspondence effects on the second than the first task suggest, response selection commonly preceded target selection, so that the outcome of the former influenced the latter according to the scenario in Figure 3B C. However, as soon as the manual response was selected and executed there was no need to maintain the activation of the corresponding response-related codes any further and, in fact, there is evidence for a quick decay of response codes after use (Stoet & Hommel, 1999). Accordingly, correspondence effects only appeared when target selection immediately followed response selection but not at the longest SOA. Yet, even if targets are eventually selected only after response selection is completed, it may be possible to already prepare their selection to some degree by marking the target location (e.g., by activating the spatial code associated with the bar marker, which we have omitted in Figure 3). This would allow for a correspondence effect on manual RTs, which may not have appeared here for reasons addressed in Experiment 2, and on manual choice errors, which we did obtain in Experiment 1. There are two possibilities to account for the SOA main effects we obtained in the visual task. One is to assume that selecting a response requires the same mechanism or resources needed for stimulus encoding. In a recent series of studies, Jolicoeur, Dell Acqua, and colleagues (e.g., Jolicoeur & Dell Acqua, 1998, 1999; for a recent summary, see Jolicoeur, Tombu, Oriet, & Stevanovski, in press) have provided strong evidence for interactions between stimulus encoding and response selection. They argue that processing a stimulus for later report requires consolidation of its code in working memory, a process that is assumed not to allow for concurrent response selection. According to this view, the SOA effects we obtained may reflect the effect of delayed consolidation: Selecting the manual response deferred consolidating the visual stimulus, the more so the greater the task overlap, so that information was lost with short SOA. Alternatively, not the encoding of the visual items may have been deferred until (or, sometimes, carried out before) manual response selection but only the selection of the target from those encodings. That is, codes of all letter may have been bound to codes of their current location before, or concurrently with, manual response selection, but the actual target selection may have taken place only after response selection was completed. If so, the stored bindings may decay over time, so that increasing task overlap would lead to a loss of

11 402 HOMMEL AND SCHNEIDER information about identity location relationships. As stimulus selection relies on this information, the quality of its outcome will decrease with decreasing SOA. Although our data will not allow a final conclusion, we find the latter scenario more plausible than the assumption of delayed consolidation. Consider visual performance with short SOAs: Even though error rates were increased with the shortest SOA, accuracy was relatively good at all SOAs and in no case approached chance level. Although this is partly a consequence of our adaptive presentation procedure, it is difficult to see how that reasonable performance was possible if any encoding had been suspended for several hundred milliseconds after the actual stimuli disappeared. Moreover, if consolidation is as time and resource consuming as the findings of Jolicoeur and colleagues suggest, it would seem odd to consolidate traces of the whole stimulus array before or even instead of selecting the target. Rather, it made more sense either to select the target on presentation what does not seem to have happened or to encode the items for later selection. This suggests that storing the whole display is somehow easier and/or goes quicker than selecting the target. Thus, our first guess regarding the impact of SOA on visual search in our design is that task overlap delays target selection, not visual encoding as such. 1 This leaves open our first question, namely, why was temporal overlap of stimulus and response selection avoided at all? In our view, a possible, though tentative answer to this is suggested by recent observations of Müsseler and Hommel (1997a, b). They found that preparing a left or right keypress impairs both detection and identification of a feature-overlapping, masked stimulus, a pattern that Hommel and Müsseler (2002) were able to replicate with combinations of verbal actions and words. As proposed by Hommel, Müsseler, Aschersleben, and Prinz (in press) and Stoet and Hommel (1999), this may be due to that a feature code that is integrated into an action plan is, in a sense, temporarily occupied and thus less available for representing a stimulus possessing the same feature. Now, given that in our task the manual response and the 1 Note that our preference, and the arguments supporting it, does not necessarily question the general assumption of Jolicoeur and colleagues that memory consolidatio n can constitute a real bottleneck. As Jolicoeur and Dell Acqua (2000) have pointed out, consolidatio n costs may be much reduced if the display conditions are favourable, stimulus presentatio n is relatively long, and/or masking is not severe. Given our adaptive timing procedure, such conditions may have been met in our study. Moreover, the tasks employed by Pashler (1991) and us differ from the tasks showing clear demonstration s of consolidatio n costs. Whereas Jolicoeur and Dell Acqua (1998, 1999) used tasks combining non-trivial demands on identificatio n (identifying randomly presented, masked letters or symbols) with virtually none on spatial selection, our task had exactly opposite characteristics. Thus, all our subjects had to do when encoding and storing the visual stimulus was to bind known characters to new locations a process that may not pose severe capacity problems on visual encoding and short-term storage.

12 VISUAL ATTENTION AND RESPONSE SELECTION 403 visual target were spatially defined, there was always the possibility of feature overlap between response and visual target. To avoid any complication and confusion arising from that and, in particular, to prevent spatial codes needed for target selection from being occupied by manual action planning, any overlap between stimulus and response selection may have been circumvented. Hence, the sequencing of selection processes may have been a strategy to reduce crosstalk between stimulus- and response-feature codes in order to avoid selection errors. We get back to this consideration in Experiment 2. In sum, Experiment 1 shows that Pashler s (1991, Exp. 1) main finding, the statistical independence between search performance and SOA, did not replicate. Instead, we find a substantial impairment in search performance with increasing task overlap. Moreover, search performance is better with spatial correspondence than non-correspondence between the search target and a concurrently performed manual response, also suggesting that stimulus and response selection processes are interdependent. EXPERIMENT 2 Although we saw an effect of response target correspondence in Experiment 1, it was asymmetric and only occurred in task 2. According to our explanation, this was because participants attempted to minimize interference of and from the manual task by merely encoding the visual display upon presentation and delaying target identification until the manual response was selected. Obviously, manual correspondence effects can only be produced by selecting the target, not by merely storing it, because it is the selection process that uses spatial codes to access and determine the target s identity via location-identity bindings (see Figure 3). As target selection followed response selection, it could not affect its outcome. One might object that at least the shorter SOAs in Experiment 1 (50 and 150 ms) should have allowed for selecting the target before the manual response. However, recall that these short SOAs were intermixed with a rather long one (650 ms), so that preparing to select the target before the response would not have paid for about one-third of the trials. Therefore, although participants could have often been able to select the target upon presentation, they may have been reluctant to do so. Instead, they may have favoured an all-purpose late-selection strategy they could apply to both short and long SOAs (for similar considerations, cf., De Jong & Sweet, 1994; Rogers & Monsell, 1995). Admittedly, our story is post hoc, so that we conducted Experiment 2 to seek for independent evidence supporting it. One important and testable implication of our interpretation is that manual correspondence effects should show up if participants would only be given sufficient time to always select the visual target before the manual response. In Experiment 2 we restricted the range of possible SOAs to very small values only by replacing the 650 ms SOA of

13 404 HOMMEL AND SCHNEIDER Experiment 1 by one of 100 ms. That is, we used SOAs of 50, 100, and 150 ms, intervals that should provide participants with sufficient time to determine the visual target letter before starting to select their manual response. If so, and if participants would accept this offer, response selection would follow target selection and should therefore be affected by spatial target response correspondence. Method Participants. Nineteen adults were paid to participate in single sessions. They fulfilled the same criteria as in Experiment 1. Apparatus, stimuli, design, and procedure. These were as in Experiment 1, except that the three SOAs were 50, 100, and 150 ms, and that the error feedback was visual. Results Trials with missing tone responses (1.4%) and anticipations (0.1%) were excluded from analyses. The remaining data were treated as in Experiment 1. Tone task. The RT analysis revealed two significant main effects (see Figure 4): RTs increased with SOA (443, 440, and 450 ms, respectively), F(2, 36) = 3.93, p <.05, and correspondence produced faster responses than Figure 4. Experiment 2: Reaction times (RTs, in ms) on Task 1 and proportions of errors (PEs, in %) on Task 2 as a function of stimulus onset asynchrony (SOA) and spatial compatibility between primary response and secondary target stimulus.

14 VISUAL ATTENTION AND RESPONSE SELECTION 405 non-correspondence (438 vs 451 ms), F(1, 18) = 19.34, p <.001. The interaction approached the 10% level. The choice-error analysis yielded a significant main effect of correspondence, F(2, 18) = 7.91, p <.05, produced by smaller error rates with correspondence than non-correspondence (3.6% vs 5.2%). Letter task. Both main effects were highly significant, SOA, F(2, 36) = 17.53, p <.001, and correspondence, F(1, 18) = 27.82, p <.001. As shown in Figure 4, error rates decreased with increasing SOA (25.6%, 18.8%, and 15.9%, respectively), and less errors were made with correspondence than with non-correspondence (18.1% vs 22.1%). Discussion The most important outcome of Experiment 2 is that, as expected, manual responses were clearly affected by spatial target response correspondence. This strongly suggests that, in contrast to Experiment 1, the target was selected before manual response selection set in, or at least before it was completed. Selecting the target must have required, or been associated with, the activation of spatial codes representing its location and these codes must have affected the process of selecting the manual response. The interaction of correspondence and SOA fits well into this picture. Because with increasing SOA the likelihood increases that response selection begins before the visual target is identified, correspondence effects on manual responses become less and less likely. Comparing Experiments 1 and 2 it is interesting to note that, in accordance with our speculation, exchanging just one SOA value (100 for 650 ms) was sufficient to change the whole outcome pattern, at least in the manual task. This is most obvious for the shortest SOA, which was associated with a substantial correspondence effect in Experiment 2 but not in Experiment 1. If the 50 ms SOA allowed for early target selection in Experiment 2 it should have done so in Experiment 1, too, which leads us to conclude that our participants were able to adopt rather different ways to coordinate their two sub-tasks in the two experiments. Hence, if the task context suggests that target selection is manageable before response selection, the visual display is analysed and the target is selected right after presentation. In contrast, the display is merely encoded if the task context suggests that target and response selection might often interfere with each other. That is, people seem to be able to configure their own cognitive system in order to meet external constraints on dual-task performance to quite a remarkable degree, a fact emphasized by De Jong and Sweet (1994) and Meyer and Kieras (1997). Apart from its expected effect on manual responses, correspondence also affected letter-report performance. If we assume that correspondence effects in the search task reflect the impact of manual response selection on visual target selection, and if it is true that in Experiment 2 target selection took place before

15 406 HOMMEL AND SCHNEIDER the manual response was selected, how can we account for the presence of such an effect? In our view, an effect of correspondence on target selection is surprising only if one assumes that selecting a stimulus object is a discrete, separable processing step of a fixed, short duration. Indeed, if the target letter would have been identified and selectively stored right after letter-display presentation, it would be difficult to see how this process could ever be affected by responseselection processes some 100 ms later. But assume target selection in the present context consisted in only priming the target letter for later report, hence increasing the activation level of the target-letter code to a greater extent than that of its competitors. To take the example from Figure 3, the target letter A may be selected by activating its identity code, as well as its associated location code (here: the left code), to a larger degree than the competing B code. Naturally, this would prime the corresponding manual response (here: the index code), thus producing manual correspondence effects of the kind observed in Experiment 2. As soon as the manual response is selected, including the activation of the response-related location code, activation of this latter code would spread to the associated letter-identity code. In the example, selecting the index-finger response would prime the letter code of A, whereas selecting the middle-finger response would prime the letter code of B. If priming through target selection and priming through response selection converge on the same identity code, the respective letter will be reported with a high likelihood, hence, accuracy will be high. However, if the target is primed by stimulus-selection processes, but one of its competitors is primed by responseselection processes, participants may sometimes erroneously report the competing letter, which impairs report performance. In other words, response target correspondence can well affect target report even if target selection takes place before the manual response is selected. The observation of a higher degree of interaction between the two tasks in the context of a rather small SOA range may suggest that these conditions tempted subjects to conjoin tasks, that is, to effectively transform the two tasks into one. Accordingly, one may doubt whether the outcome of Experiment 2 allows for conclusions that are applicable to standard dual-task situations. On the one hand, it is clear that blocking short SOAs did affect the way subjects scheduled stimulus- and response-related processes, thereby allowing more crosstalk between the tasks. On the other hand, though, there is no evidence for a strict temporal locking of processes across the two tasks. For instance, consider RTs and PEs for the two shortest SOAs in Figure 4. If the two tasks were really treated as one we would expect that manual response selection awaits stimulus processing in the visual task, which should delay RT with increasing SOA. Yet, there is not the slightest increase from 50 ms to 100 ms SOA, hence, no evidence of any grouping or synchronization of selection or other processes. For the same SOAs strong correspondence effects were observed in both tasks, which suggests that the underlying processes do not depend on task conjoining.

16 VISUAL ATTENTION AND RESPONSE SELECTION 407 In sum, Experiment 2 shows that correspondence effects in manual responses, which were absent in Experiment 1, can be obtained if both the length and the range of tone-display SOAs is chosen to allow for selecting the visual target before manual response selection begins. This demonstrates that manual response selection can be affected by visual target selection in very much the same way as target selection is influenced by response selection, which provides further support for our assumption that stimulus and response selection are not entirely independent. EXPERIMENT 3 Experiment 3 focused on a possible role of the degree of temporal overlap between stimulus and response selection. As the hitherto used SOA levels of 50, 150, and 650 ms do not fully cover the temporal range of possible interactions between the two selection processes, we chose to investigate SOAs of 200, 300, and 400 ms in Experiment 3, which in all other respects replicated Experiments 1 and 2. If Pashler s (1991) claim of independence between stimulus and response selection is correct, this modification should have no effects on the results, which amounts to predicting a null effect of SOA in either task. If, however, the degree of temporal overlap of the selection processes does play a critical role, an SOA main effect might show up in at least one of the two tasks. Method Participants. Twenty-four adults were paid to participate in single sessions. They fulfilled the same criteria as in Experiment 1. Apparatus, stimuli, design, and procedure. These were as in Experiment 1, except that the three SOAs were 200, 300, and 400 ms. Results Anticipations (0.1%) and trials with missing tone responses (2.0%) were excluded from analyses. The remaining data were treated as in Experiment 1. Tone task. In the RT analysis, only the main effect of SOA was significant, F(2, 46) = 31.16, p <.001, due to an increase of RTs with SOA (483, 501, and 530 ms, respectively). An SOA main effect was also obtained in the choice error analysis, F(2, 46) = 5.55, p <.01, but here performance was worse with the shortest than the longer SOAs (2.9%, 2.0%, and 2.4%, respectively). Letter task. The main effect of correspondence was significant, F(1, 23) = 7.47, p <.05, as was the interaction of correspondence and SOA, F(2, 46) =

17 408 HOMMEL AND SCHNEIDER Figure 5. Experiment 3: Reaction times (RTs, in ms) on Task 1 and proportions of errors (PEs, in %) on Task 2 as a function of stimulus onset asynchrony (SOA) and spatial compatibility between primary response and secondary target stimulus. 3.91, p <.05, whereas the SOA effect was not reliable (p =.13). As shown in Figure 5, substantial (and significant) correspondence effects were obtained with SOAs of 200 and 300 ms but not with 400 ms. Discussion First of all, we were able to replicate the correspondence effects on search performance for the two shorter SOAs. This completes the picture from Experiments 1 and 2 in indicating continuous decrease of correspondence effects with increasing SOA. According to the model sketched in Figure 3, such effects represent the impact of preceding response selection processes on target selection. And, indeed, this impact should get weaker as the delay of target selection to response selection gets longer. Even more important in the present context, comparing the results of Experiments 1 3 reveals that our variation of the temporal overlap between manual response selection and visual stimulus processing led to a considerable change in the data pattern of Experiment 3. In particular, the effect of SOA on performance in the tone task became more pronounced with longer SOAs. Considering that increasing the SOA (within the range of manual RTs) is likely to increase the to-be-expected (but obviously avoided) temporal overlap of manual response selection and visual stimulus selection, this may be another indication of the strategic queuing of stimulus- and response-selection

18 VISUAL ATTENTION AND RESPONSE SELECTION 409 processes. However, there are three problems with this interpretation: (1) An easy one that can be rejected on grounds of the present data, (2) a not so easy one that we tested in Experiment 4, and (3) a crucial one that we will address in the General Discussion. The easy problem is that the increase of RTs with SOA was accompanied by a decrease of choice errors, which might indicate a speed accuracy tradeoff. Yet, there are two reasons to doubt that such a tradeoff was responsible for the SOA effect on RTs. One is, that choice errors in the tone task decreased with SOA in all four experiments of the present study often to a larger extent than in Experiment 3, whatever SOA values were used and whether an SOA effect on RTs was obtained or not (see Figure 7). This mirrors the similar findings of Pashler (1991), who consistently observed negative correlations between choice errors and SOA across a number of task variations. Obviously, the error effect does not go with the RT effect but seems to be a standard byproduct of the basic task design used here (see General Discussion for some considerations on underlying causes). Another reason to doubt that our RT effect was due to a speed accuracy tradeoff becomes evident if we compare the tone-task data from the two longest SOAs only (i.e., 300 and 400 ms). Here, RTs increased markedly from 501 to 530 ms, and so did the error rates (2.0% and 2.4%, respectively). Hence, a positive SOA effect on RTs can be obtained in the absence of a negative effect on errors, a finding that runs counter to a tradeoff account. A perhaps more serious objection to interpret the SOA effect on RTs as a queuing effect has already been considered by Pashler (1991). The original idea of varying the SOA between the tone and the visual display was to manipulate the onset of stimulus-selection processes in the search task relative to response-selection processes in the tone task. If SOA has an effect, so the reasoning goes, this can be attributed to the varying degree of temporal overlap between stimulus- and response-related processes. However, note that this kind of manipulation confounds the variation of the onset of stimulus-directed processes with the variation of the onset of the visual stimulus itself. Therefore, we cannot be sure whether an effect of SOA really reflects characteristics of the attentional process we wish to index or only those of the attention-drawing visual stimuli. In fact, from studies on sensory facilitation we know that RT to a stimulus in one modality can be reduced by presenting a stimulus in another modality, even if this latter stimulus is irrelevant to the task (see Nickerson, 1973, for an overview). That is, responses in our tone task may have been quicker with short than with long SOAs not because visual stimulus processing hampered manual response selection with long SOAs, but because the mere presence of the visual stimulus display facilitated the manual response with short SOAs. As the validity of this interpretation cannot be evaluated on the basis of the present data, we conducted Experiment 4 to provide an empirical test.

19 410 HOMMEL AND SCHNEIDER EXPERIMENT 4 The aim of Experiment 4 was to find out whether the positive correlation between manual RT and SOA observed in Experiments 1 3 might have been due to sensory facilitation, hence, to an unspecific facilitation effect of the visual display. We tested the sensory-facilitation account by comparing the effect of the mere presentation of a letter-task display with the effect of requiring active processing of the display. We did that by using the same task and SOA range as in Experiment 3, but in addition to the search blocks, where participants again searched for a cued letter target, we also ran a non-search block, where the search display was presented but no search was to be performed. The predictions were straightforward. Clearly, we expected to replicate the increase of manual RT with SOA in the search block (apart from the correspondence effect on search performance). According to the sensory-facilitation hypothesis, the non-search block should give rise to the same positive relationship between RT and SOA after all, the stimulus conditions were virtually identical in search and non-search blocks. In contrast, if visual-attentional processes were responsible for the increase of manual RT with SOA in Experiment 3, no such an effect should occur in the non-search block. Method Participants. Twenty adults were paid to participate in single sessions. They fulfilled the same criteria as in Experiment 1. Apparatus, stimuli, design, and procedure. These were as in Experiment 3, with the following exceptions. One of the five experimental 96-trial blocks was replaced by a non-search block of about the same length (96 plus 9 additional catch trials, see later). During this block, the display conditions were basically the same as in the search blocks, including the presentation of the briefly masked four-letter displays at the three SOAs. However, the participants performed the tone task only but did not search for target letters. Accordingly, no target marker was presented and no letter was to be reported. In order to require the participants to monitor the visual events none the less, nine catch trials were randomly distributed across the block. In these catch trials a bright yellow frame appeared for 300 ms, which was to be responded to by pressing the space bar with 2000 ms after the tone response. Each frame appeared with one of nine randomly ordered frame tone onset asynchronies, ranging from 200 ms (frame before tone) to 600 ms (frame after tone), in steps of 100 ms. The frame outline measured with an edge-to-edge distance between frame and (area of) letter display of 0.3 horizontally and 0.4 vertically. The non-search block was run after the first, second, third, or fourth (i.e., last)

20 VISUAL ATTENTION AND RESPONSE SELECTION 411 search block. Its position was balanced across participants, so that each of the four possible positions was realized with two participants. Results Each participant responded correctly to all catch trials with no exception. Trials with missing tone responses (1.3%) and anticipations (0.02%) were excluded from analyses. The remaining data from the standard search blocks were treated as in Experiment 1. From non-search blocks, the data from catch trial were excluded and mean RTs and PEs for tone responses were computed as a function of SOA. Tone task in search vs non-search blocks. A 2 3 (Task context SOA) ANOVA was run on the RTs to tones from search and non-search blocks. Apart from a significant effect of SOA, F(2, 38) = 9.05, p <.001, responses were quicker in the non-search block than in search blocks (397 vs 497 ms), F(1, 19) = 61.56, p <.001. More important, the interaction was significant, F(2, 38) = 16.67, p <.001. As shown in Figure 6 and confirmed by separate analysis, this was due to that RT increased with SOA in the search blocks, but not in the non-search block. The errors analysis did not produce significant results. The SOA main effect approached the significance criterion (p =.051), but the underlying pattern was opposite to the RT pattern (5.3%, 3.6%, and 3.8% in search blocks, and 3.9%, 3.8%, and 2.8% in non-search blocks). Figure 6. Experiment 4: Reaction times (RTs, in ms) for search blocks (straight lines) and non-search blocks (dotted line) on Task 1 and proportions of errors (PEs, in %) on Task 2 as a function of stimulus onset asynchrony (SOA) and spatial compatibility between primary response and secondary target stimulus.