Feature saliency affects delayed matching of an attended feature

Transcription

1 This article was downloaded by: [National Cheng Kung University] On: 01 October 2012, At: 00:50 Publisher: Psychology Press Informa Ltd Registered in England and Wales Registered Number: Registered office: Mortimer House, Mortimer Street, London W1T 3JH, UK Journal of Cognitive Psychology Publication details, including instructions for authors and subscription information: Feature saliency affects delayed matching of an attended feature Cheng-Ta Yang a, Yu-Chin Chiu b & Yei-Yu Yeh c a Department of Psychology, National Cheng Kung University, Tainan, Taiwan b Department of Psychology, University of California, San Diego, CA, USA c Department of Psychology, National Taiwan University, Taipei, Taiwan Version of record first published: 04 Jul To cite this article: Cheng-Ta Yang, Yu-Chin Chiu & Yei-Yu Yeh (2012): Feature saliency affects delayed matching of an attended feature, Journal of Cognitive Psychology, 24:6, To link to this article: PLEASE SCROLL DOWN FOR ARTICLE Full terms and conditions of use: This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae, and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand, or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.

2 JOURNAL OF COGNITIVE PSYCHOLOGY, 2012, 24 (6), Feature saliency affects delayed matching of an attended feature Cheng-Ta Yang 1, Yu-Chin Chiu 2, and Yei-Yu Yeh 3 Downloaded by [National Cheng Kung University] at 00:50 01 October Department of Psychology, National Cheng Kung University, Tainan, Taiwan 2 Department of Psychology, University of California, San Diego, CA, USA 3 Department of Psychology, National Taiwan University, Taipei, Taiwan We examined how top-down attentional modulation and bottom-up stimulus saliency interact with feature memory. Experiment 1 used a delayed-matching-to-sample (DMS) task to examine the relative saliency between features by observing the relative accuracy of recognition at different stimulus durations. Feature salience decreased according to the following order: colour, form, and texture. In a modified DMS task (Experiments 2 and 3), participants were required to attend to one of three features and ignore the others. After a delay, they were required to choose which of the two test stimuli matched the reference stimulus on the attended feature, disregarding other task-irrelevant features. The target was either identical to the reference stimulus or mismatched the reference stimulus on one of the irrelevant features. The results showed that colour matching was affected neither by a form change nor by a texture change. Form matching was affected by a colour change, and texture matching was affected by a colour or form change. These results are consistent with the relative saliency hypothesis. Even when all features of an attended object are maintained, a relatively more salient task-irrelevant feature can interfere with the delayed recognition of a less salient feature. Keywords: Attentional modulation; Delayed-matching-to-sample task; Feature saliency. Our environment is complex, and a visual scene comprises of multiple objects, which contain various features, including colour, luminance, orientation, form, texture, and size. At any given moment, only a small portion of the visual information can be selected and identified for conscious processing due to the limited capacity of the human information processing system (Baddeley, 1981; Broadbent, 1965). To fulfil behavioural goals, selective attention plays an important role in selecting relevant information for further visual processing and inhibiting irrelevant distractors (Desimone & Duncan, 1995; Yantis, 2008). Both top-down attentional control and bottom-up stimulus saliency influence the deployment of selective attention. Much research demonstrates that the content of visual working memory biases attention (Downing, 2000; Olivers, Meijer, & Theeuwes, 2006; Woodman & Luck, 2007), suggesting that top-down attentional control plays a critical role in biasing attention. On the other hand, previous studies showed that Correspondence should be addressed to Cheng-Ta Yang, Department of Psychology, National Cheng Kung University, No. 1, University Rd., Tainan, Taiwan yangct@mail.ncku.edu.tw This work was supported by grants from National Science Council to C.-T. Yang (NSC C H and NSC H MY2) and to Y.-Y. Yeh (NSC H MY3). We thank S.-H. Lin and Y.-J. Chen for their assistance in stimulus generation and C.-J. Wu for her assistance in figure production. Parts of the results were presented at the 15th meeting of Object, Perception, Attention, and Memory in Long Beach, CA. # 2012 Psychology Press, an imprint of the Taylor & Francis Group, an Informa business

3 visually salient but task-irrelevant distractors attract attention and impair visual search performance (Theeuwes, 2004, 2010). These studies suggested that stimulus saliency biases attention during visual search. Nonetheless, recently a separate line of studies further demonstrated that not all salient distractors impair visual search performance; instead, only visually salient distractors that share one or more features with the search target capture attention (Folk & Remington, 2010; Folk, Remington, & Johnston, 1992; Folk, Remington, & Wright, 1994). In short, the top-down and bottom-up factors do not act independently; their interactions determine the visual search performance. The interaction between a top-down attentional set and bottom-up saliency during perception also has a direct impact on memory; however, few studies have addressed the question of how the two factors interact in feature memory. Therefore, the goal of the present study was to investigate the extent to which the memory of a simple feature, as defined by the task set, is affected by the processing of other task-irrelevant features within the object. We were especially interested in the case where only one feature of an object that consists of multiple features is relevant for the later visual decision because this case allows us to investigate the selectivity of visual attention for the subsequent memory test. Specifically, we tested whether memory judgement based on a feature is affected by any change in other task-irrelevant features during testing. RELATIVE SALIENCY HYPOTHESIS According to the object-based selection model (Egly, Driver, & Rafal, 1994; O Craven, Downing, & Kanwisher, 1999; Roelfsema, Lamme, & Spekreijse, 1998), all features of an attended object are coactivated regardless of task relevancy. When an object is attended, all features of the object are simultaneously encoded (Brockdorff & Lamberts, 2000; Logan, 1988; Rensink, 2000) and retrieved for memory comparison and decision making during testing. However, in the real world, not all features are equally salient and important for visual decision. This concept of relative saliency has been widely used in the literature of visual search (Itti & FEATURE MEMORY 715 Koch, 2000; Nothdurft, 2006; Turatto & Galfano, 2000), classification (Wallsten & Barton, 1982), and change detection (Yang, 2011; Yang, Hsu, Huang, & Yeh, 2011). In the current study, we define relative saliency as the relative processing speed of features during the encoding phase. Assuming that all features of an attended object are simultaneously encoded, the rate of encoding may vary between different features. Within a given time window, a more salient feature is sampled at a higher rate (Lamberts, 2002) and thus is stored with more memory traces. Lowlevel features stored in visual working memory are of high fidelity (Magnussen & Dyrnes, 1994; Magnussen, Greenlee, Aslaksen, & Kildebo, 2003), with the salient feature representations being stronger than the less salient feature representations. Previous studies have demonstrated the relative processing speeds between features. For example, Viviani and Aymoz (2001) manipulated a change between colour (red/green), form (circle/square), and motion (fixed/moving) and asked participants to report which change had occurred first. The response probabilities at different stimulusonset asynchronies (SOAs) were used to estimate the perception time of each feature change. The study found that changes in colour and form were perceived almost simultaneously, and both of these changes were perceived more quickly than motion change. In Vogel et al. s studies (Vogel, Woodman, & Luck, 2006; Woodman & Vogel, 2005, 2008), participants performed a change detection task for colour squares, orientation bars, and geometric shapes. Shortly after the presentation of the memory array, pattern masks were presented at the locations of each item to disrupt items that had not been consolidated completely. Five levels of SOAs of the memory array were manipulated to estimate the consolidation rate of each feature. The results showed that change detection accuracy and memory capacity (k) increased as a function of SOA. More importantly, the estimated consolidation rate of colour was higher than that of orientation and shape. Moreover, Lamberts et al. used a stochastic feature sampling model (Lamberts, Brockdorff, & Heit, 2002; Lamberts & Kent, 2008) to estimate the perceptual rates and retrieval rates of features in an object (i.e., a hot air balloon with three binary features: balloon

4 716 YANG, CHIU, YEH colour, rope orientation, and gondola shape) from the perceptual matching and recognition memory data. Their behavioural results revealed that colour was better memorised than shape, which, in turn, was better memorised than orientation. 1 Moreover, the modelling results suggested that differences in performance were attributed to differences in sampling rates. Therefore, the relative saliency of features influences the encoding and consolidation speed (i.e., the feature sampling rate), which subsequently affects memory strength and decision. Based on the fact that relative feature saliency affects the strength of feature representations, which in turn affects visual decision, we proposed a relative saliency hypothesis to explain how the top-down and bottom-up factors interact with the memory judgement of an attended feature. According to this hypothesis, a change of a taskirrelevant feature may or may not interfere with the memory judgement of a task-relevant feature depending on the relative saliency between the task-relevant and task-irrelevant feature. That is, a memory judgement based on a salient feature would not be affected by less salient taskirrelevant features. In contrast, a salient taskirrelevant feature would interfere with the memory judgement of a less salient task-relevant feature. THE PRESENT STUDY Few studies have investigated how changes in taskirrelevant features at test time influence memory judgement of an attended feature (Baumann, Endestad, Magnussen, & Greenlee, 2008; Magnussen, Idås, & Myhre, 1998). Magnussen et al. (Baumann et al., 2008; Magnussen et al., 1998) conducted a series of studies using a delayed-matching-to-sample (DMS) task to test the participants memory of the orientation and spatial frequency of a grating. Specifically, participants were asked to match either the orientation or the spatial frequency of a test grating to the 1 Vogel et al. (2006; Woodman & Vogel, 2008) found that orientation was better memorised than shape, whereas Lamberts et al. (2002; Lamberts & Kent, 2008) found the opposite result. These inconsistent findings may have occurred because different stimuli were used to test the memory of features. The former studies used orientation bars and geometric shapes to test participants memory of different features, whereas the latter studies used a real object (i.e., hot air balloon). The difficulty in delayed recognition of orientation and shape is different across the two series of studies. reference grating. They found that a change in spatial frequency at test time prolonged the delayed discrimination of orientation, and vice versa, suggesting that a change in a task-irrelevant feature interferes with the memory judgement of a task-relevant feature. However, the role of relative saliency between task-relevant and taskirrelevant features in delayed matching to an attended feature was not addressed in Magnussen et al. s study. It is perhaps because the orientation and spatial frequency of a grating are equally salient for a visual decision such that delayed matching to one of the features is affected by a change in the other. The aim of this study was to investigate how top-down attentional control and bottom-up feature saliency interact with memory judgement based on a simple feature (i.e., colour, form, or texture 2 ). In Experiment 1, we used a DMS task in which participants were asked to select one of the test stimuli that matched to the whole reference stimulus. The duration of the reference stimulus was manipulated to investigate the degree to which a feature is encoded at a given time point. The distractor differed from the reference stimulus by one of the three features. We can then infer the relative saliency between features by observing the relative accuracy of delayed matching. In Experiments 2 and 3, we used a modified DMS task. Participants were required to pay attention to a specific feature of the reference stimulus. During the test phase, participants had to choose the test stimulus that matched the value of the attended feature to the reference stimulus. The target was either identical to the reference stimulus or mismatched the reference stimulus on one of the task-irrelevant features. This modified DMS task allowed us to investigate whether any change of a task-irrelevant feature interferes with the memory judgement of an attended feature. According to the relative saliency hypothesis, we predict that only a change of a more salient task-irrelevant feature will interfere with the memory judgement based on a less salient task-relevant feature, and judgement based on a salient feature will not 2 Although memory of colour, form, orientation, and motion have been extensively investigated, texture, surprisingly, has not been explored, even though texture is an important visual cue regarding surface property; in addition, texture discrimination contributes to object identification and scene perception (Bergen & Landy, 1991; Julesz, 1981). Thus, this study investigated memory of the colour, form, and texture of an object.

5 be interfered with by a less salient task-irrelevant feature. EXPERIMENT 1 We first examined the relative saliency between features with different stimulus durations. Participants were asked to precisely match the entire stimulus with the reference stimulus. We manipulated the duration of the reference stimulus and changed the distractor by one feature. The amount of a feature that is encoded increases as a function of stimulus duration. When the stimulus duration increases, the memory traces of a feature increases and the matching accuracy increases as well. If the processing of a feature is finished by a certain time point, the participants can almost 100% accurately choose the target, and the distractor is not mistaken as the target. However, if the feature processing is not finished, we can observe error in delayed matching. We therefore can infer the relative saliency between features by examining the relative accuracy of memory judgement at different stimulus durations. Method Participants. Twenty-five undergraduate students at National Taiwan University were randomly assigned to one of the two experiments and received different manipulations of stimulus durations. These participants received a bonus credit for their participation. Their ages ranged from 18 to 22 years old (M19.75, SD1.23). All participants had normal or corrected-to-normal vision, and they signed an informed consent form prior to the experiment. Equipment. A personal computer with a 667 MHz Intel Pentium III processor controlled stimulus displays and recorded responses. The experimental paradigm was programmed and presented with E-prime 1.1 (Schneider, Eschman, & Zuccolotto, 2002). The visual events were presented on a 15-inch Sampo colour monitor with a vertical refresh rate of 85 Hz. Stimuli. From a viewing distance of 60 cm, each stimulus subtended 38 in width and 38 in height on the display against a dark grey (x.31, y.30; luminance3.8 cd/m 2 ) background. The stimuli were geometric shapes (a rectangle, dumbbell, diamond, and half-circle), and each consisted of two additional surface features (i.e., colour and texture). The stimuli were coloured in one of the four colours, red (x 0.62, y 0.34; luminance 8.8 cd/m 2 ), green (x 0.30, y 0.59; luminance8.8 cd/m 2 ), yellow (x 0.47, y 0.47; luminance8.9 cd/m 2 ), or blue (x 0.15, y 0.07; luminance8.7 cd/m 2 ), with approximately equal luminance ( cd/ m 2 ). Four texture patterns were created by the black line on the surface of the stimuli (see Figure 1 for examples). There were a total of 64 (4 forms4 colours 4 textures) different stimuli. Design. There were two within-subject variables (distractor type in three levels and stimulus duration in three levels). The distractor differed from the reference stimulus in one of the three features: colour (C), form (F), and texture (T). To infer the relative saliency between features, we manipulated the stimulus duration. The duration of the reference stimulus was 25, 50, or 150 ms in Experiment 1A, and it was 200, 300, or 400 ms in Experiment 1B. Twenty-four observations were included in each condition, and all of the trials were randomly presented in four blocks. At the beginning of the experiment, 32 practice trials were used to confirm that the participants understood the instructions. Procedure. Each trial began with a white fixation cross for 500 ms. A memory display containing a reference stimulus was presented at the centre of the display for a variable duration. After the memory display, a random dot image was presented for 300 ms to mask the memory display. Following the mask, a test display was presented until a response was provided. The test display consisted of a target and a distractor placed at the centre with an edge-to-edge distance of The participants were asked to accurately choose the target identical to the reference stimulus. If the stimulus on the left matched the reference stimulus, the participant pressed the left key, and if the stimulus on the right matched the reference stimulus, he or she pressed the right key. The interval between trials was 1000 ms. The experiment lasted approximately 30 min. See Figure 2 for the trial procedure. Results FEATURE MEMORY 717 Accuracy data in Experiments 1A and 1B were separately analysed with a 3 (distractor type:

6 718 YANG, CHIU, YEH Figure 1. Examples of the stimuli in Experiment 1. Geometric shapes were defined by three types of features, including colour, form, and texture. There were four levels for each feature dimension. [To view this figure in colour, please visit the online version of this Journal.] Downloaded by [National Cheng Kung University] at 00:50 01 October 2012 colour, form, and texture) 3 (stimulus duration: 25, 50, 150 in Experiment 1A; 200, 300, 400 in Experiment 1B) repeated measures analysis of variance (ANOVA). Table 1 shows the mean performance data. Our results showed that the main effects of distractor type were significant in both Experiment 1A, F(2, 22) 36.56, MSE0.306, p B.05, g 2 p.77, and Experiment 1B, F(2, 24) 21.56, MSE0.114, p B.05, g 2 p.36. In both experiments, the accuracy was higher when the distractor types were colour and shape than when the distractor type was texture. The main effects of stimulus duration were also significant in both Experiment 1A, F(2, 22) 22.41, MSE0.188, p B.05, g 2 p.67, and Experiment 1B, F(2, 24) 7.86, MSE0.082, p B.05, g 2 p.40. Matching accuracy increased as a function of stimulus duration. The interaction was significant in Experiment 1A, F(4, 44) 7.62, MSE0.246, p B.05, g 2 p.41, but was not significant in Experiment 1B (p.25). These results suggested that perceptual rates of different features varied and that the relative saliency between features varied as a function of stimulus duration. To further infer the relative saliency between features at different time points, we conducted post hoc analyses. Our results showed that when stimulus duration was 25 ms, accuracy decreased in the following order of distractor type: colour (0.94), form (0.81), and texture (0.63). For the other stimulus durations, accuracy was higher when the distractor types were colour and shape than when the distractor type was texture. Although the stimulus duration was 400 ms (i.e., long enough for the encoding of a complex scene; Figure 2. An illustration of the procedure used in Experiment 1. The target, which was identical to the reference stimulus, was on the left side, whereas the distractor, which differed from the reference stimulus in colour, was on the right side. [To view this figure in colour, please visit the online version of this Journal.]

7 TABLE 1 Mean and standard deviation in parentheses of accuracy data in Experiments 1A and 1B Potter, 1976), texture memory was worse than the memory of colour and shape, suggesting that texture was the least salient feature. Discussion Experiment 1A (N 12) We found that accuracy increased as a function of the duration of the reference stimulus. These results suggest that feature information is accumulated over time (Kent & Lamberts, 2006; Lamberts, 2000; Lamberts & Kent, 2008; Vogel et al., 2006; Woodman & Vogel, 2005). With longer encoding time, more feature information is perceived and retained for the subsequent memory comparison and decision. The abovechance level (0.5) performance suggests that all features are encoded and retained in memory. Moreover, we found a significant interaction between the stimulus duration and the distractor type in Experiment 1A, though there was no significant interaction in Experiment 1B. These results suggest that the perceptual rates of the three features vary. These features are encoded in the following order: colour (fastest), followed by form and texture (slowest). Therefore, colour is the most salient, form is less salient, and texture is the least salient. These results provided empirical evidence for the relative saliency between features at different time points. EXPERIMENT 2 Experiment 1B (N 13) Colour (0.07) (0.04) (0.02) (0.04) (0.01) (0.02) Form (0.07) (0.04) (0.03) (0.06) (0.04) (0.04) Texture (0.16) (0.16) (0.13) (0.10) (0.07) (0.06) Three types of distractors (colour, form, and texture), and three stimulus durations (Experiment 1A: 25, 50, and 150 ms; Experiment 1B: 200, 300, and 400 ms) were manipulated. A modified DMS task was used to test the relative saliency hypothesis. Specifically, we examined the extent to which the matching of a task-relevant feature to the reference stimulus is affected by a change in a task-irrelevant feature in a target at test time. The participants were required to match a particular feature (e.g., colour) to the reference stimulus and ignore the other features. The target, by definition, shared the value of the attended feature (e.g., red) with the reference stimulus, and the distractor differed from the reference stimulus in the value of the attended feature (e.g., green). Moreover, we orthogonally manipulated the change of a taskirrelevant feature in both the target and the distractor. Manipulating the target type allowed us to examine whether the change of a taskirrelevant feature in a target would influence the target selection during memory decision. Manipulating the distractor type may discourage participants from attending to the task-irrelevant feature because if the distractor always matched the task-irrelevant feature to the reference stimulus, participants could simply chose the test stimulus that differed from the reference stimulus in the task-irrelevant feature, which constituted two-thirds of the experimental trials. We expect to observe high matching accuracy when the target is identical to the reference stimulus and lower matching accuracy when a more salient taskirrelevant feature changes in the target than when a less salient task-irrelevant feature changes. Method FEATURE MEMORY 719 Participants. Twelve undergraduate students from National Taiwan University participated. Their ages ranged from 18 to 22 years old, and all had normal or corrected-to-normal vision; they received a bonus course credit for their participation in the experiment, which lasted for 30 min. All participants signed an informed consent form prior to the experiment. Stimuli and design. The stimuli were the same as those in Experiment 1. There were two withinsubject variables as follows: attended feature (three levels) and target type (three levels). The participants were required to attend to one of the three features (i.e., colour, form, and texture) and ignore the other task-irrelevant features. The target type containing three levels was nested to the attended-feature factor. The target was either identical to the reference

8 720 YANG, CHIU, YEH Downloaded by [National Cheng Kung University] at 00:50 01 October 2012 Figure 3. An illustration of the target type and possible distractors when delayed matching is based on colour (i.e., red). In the C-I condition, the target is identical to the reference stimulus. One task-irrelevant feature is changed in the other two conditions, e.g., changes in texture (C-T) and form (C-F). There are two possible distractors that are mismatched with the reference stimulus in colour (i.e., green) and another feature, such as texture or form. [To view this figure in colour, please visit the online version of this Journal.] stimulus or shared a task-relevant feature and a task-irrelevant feature with the reference stimulus while differing in another task-irrelevant feature. For example, if colour was the attended feature, the three target types were either identical (C-I 3 ), identical except for texture (i.e., change in texture, C-T) or identical except for form (i.e., change in form, C-F) (see Figure 3 for an example). The distractor did not share the attended feature with the reference stimulus. Moreover, there was a change in one of the task-irrelevant features. In this example, the distractor differed not only in colour (reference stimulus: red; distractor: green) but also differed in form (change in form) or texture (change in texture). The orthogonal manipulation of the target type and distractor type was designed to 3 We used the following abbreviation rule to simplify the names of the conditions; the first letter of the alphabet denoted the attended feature dimension (i.e., C: colour; F: form; and T: texture) and the second letter of the alphabet denoted the target type (i.e., I: identical; C: change in colour; F: change in form; and T: change in texture). discourage participants from adopting any strategies in memory judgement. The experiment was divided into three sessions, during which participants were instructed to attend to one of the three features. The order of the sessions was counterbalanced across participants. In each session, 36 observations were included in each condition, and all trials were randomly presented in four blocks. Prior to each session, 28 trials were used for practice. Procedure. The procedure was the same as that of Experiment 1, except that participants were asked to choose a target that matched the value of the attended feature to the reference stimulus rather than the whole stimulus. In the beginning of each session, participants were instructed to attend to a particular feature. When the test display was shown, the participants were instructed to accurately and rapidly choose the target whose feature value in the attended feature matched that of the reference stimulus. The participants clicked either the left or right button of the mouse to choose which test

9 stimulus s feature value matched that of the reference stimulus. Results Because the target type was nested to the attended-feature factor, accuracy and median reaction time (RT) of correct responses were analysed with three separate repeated measures ANOVAs, according to the attended-feature factor. The data from different distractor types were combined for the analyses. Table 2 shows the results. Accuracy. When colour was attended, the main effect of target type was not significant (p.1), suggesting that a change in any task-irrelevant feature did not influence colour matching. However, the main effect of target type was significant when matching form, F(2, 22) 5.84, MSE0.001, p B.05, g 2 p.36, and texture, F(2, 22) 6.11, MSE0.002, p B.05, g 2 p.35. Tukey post hoc comparisons were conducted to TABLE 2 Mean and standard deviation of accuracy and median reaction time (RT) of correct responses in Experiment 2 Target type C-I C-F C-T Matching colour Accuracy M SD RT M SD F-I F-C F-T Matching form Accuracy M SD RT M SD T-I T-C T-F Matching texture Accuracy M SD RT M SD The attended feature (colour, form, and texture) was manipulated. The target type (in three levels) was nested to the attended-feature factor where the first alphabet denoted the attended feature dimension (i.e., C: colour; F: form; and T: texture), and the second alphabet denoted the target type (i.e., I: identical; C: change in colour; F: change in form; and T: change in texture). investigate the interference produced by a mismatch of a task-irrelevant feature on memory judgement of a task-relevant feature. We found that colour change interfered with form matching, as demonstrated by a significantly lower accuracy for F-C (0.95) than for F-I (0.99) and F-T (0.98) (psb.05). Both form and colour changes interfered with texture matching; T-F (0.93) and T-C (0.94) were significantly lower than T-I (0.98) (psb.05). Reaction time. Median RTs of correct responses were analysed. We found a similar pattern of results regarding accuracy. The main effect of target type was not significant when colour was attended (p.1), but the effect was significant when form, F(2, 22) 17.35, MSE , p B.05, g 2 p.61, and texture, F(2, 22) 28.50, MSE , p B.05, g 2 p.72 were attended. Tukey post hoc comparisons showed that colour change interfered with form matching as F-C ( ms) was significantly slower than F-I ( ms) and F-T ( ms) (psb.05). Both the colour and form changes interfered with texture matching, as T-C ( ms) and T-F ( ms) were significantly slower than T-I ( ms) (psb.05). Discussion FEATURE MEMORY 721 Our results showed that colour matching was not affected by any change in the task-irrelevant features. Form matching was affected by colour change but was not affected by texture change. Texture matching was affected by changes in both colour and form. Together with the results from Experiment 1, when the stimulus duration was 150 ms, matching accuracy decreased as a function of distractor type: colour (1.00), form (0.98), and texture (0.85). We can thus infer that colour was the most salient feature, texture was the least salient feature, and form was of moderate saliency. As a result, the current findings supported the relative saliency hypothesis suggesting that the memory judgement of a task-relevant feature is only affected by a more salient task-irrelevant feature. EXPERIMENT 3 The results of Experiment 2 showed that a relatively salient task-irrelevant feature interferes

10 722 YANG, CHIU, YEH with memory judgement based on a less salient task-relevant feature. Both changes of colour and changes of form interfered with texture matching when stimulus duration was 150 ms. However, it is possible that this effect was due to the poor perceptual discriminability on texture, i.e., the perceptual limitation account, rather than due to attention or memory processes. Perhaps participants had a difficult time perceiving texture compared to form or colour when the stimulus duration was short. As a result, we tested the perceptual limitation account in Experiment 3 by increasing the stimulus duration to 400 ms, such that participants had more time to perceive and encode texture. However, we predicted the same pattern of results as Experiment 2, i.e., changes in both colour and form interfere with texture matching. In other words, we predicted that the results are consistent with the relative saliency account rather than the perceptual limitation account. Method Participants. Thirteen undergraduate students at National Taiwan University participated in this experiment; they received a bonus credit for participating in this experiment, which lasted for 30 min. Their ages ranged from 18 to 22 years old (M 19.58, SD1.01). All participants had normal or corrected-to-normal vision. All participants signed an informed consent form prior to the experiment. Stimuli, design, and procedure. The stimuli, design, and procedure were identical to Experiment 2, except that participants were only required to match texture to the reference stimulus and the duration of the reference stimulus was increased to 400 ms so that participants had enough time to encode texture. Results The accuracy and median RTs of correct responses were analysed with separate one-way repeated measures ANOVA of target type. Table 3 summarises the mean performance data. Accuracy. The main effect of the target type was significant, F(2, 24) 8.00, MSE0.004, p B.05, g 2 p.40. Tukey post hoc comparison showed that the accuracy was higher when the target type was T-I (0.97) than when the target types were T-F (0.89) and T-C (0.89) (psb.05). This finding suggested that colour and form interfered with texture matching. Reaction time. Similarly, the main effect of the target type was also significant, F(2, 24) 35.80, MSE , p B.05, g 2 p.75. The Tukey post hoc comparison showed that RT was faster when the target type was T-I ( ms) than when the target types were T-F ( ms) and T-C ( ms) (psb.05). These results suggested that texture matching was affected by changes of colour and form. Discussion Both accuracy and RTs showed a similar pattern of results. Changes of the task-irrelevant but salient features (i.e., colour and form) interfered with the memory judgement of texture, despite the fact that the duration of the reference stimulus was increased to 400 ms. Again, these results supported the relative saliency hypothesis suggesting that a change of a more salient taskirrelevant feature interfered with the memory judgement of a less salient task-relevant feature. It is unlikely that the results were due to the poor perceptual discriminability of texture because the encoding time was long enough for the perceiving of texture. GENERAL DISCUSSION This study examined how top-down attentional modulation and bottom-up stimulus saliency interact with the recognition memory of a simple TABLE 3 Mean and standard deviation of accuracy and median reaction time (RT) of correct responses in Experiment 3 Target type Matching texture T-I T-C T-F Accuracy M SD RT M SD Only texture was attended. The target type was manipulated at three levels.

11 feature, i.e., colour, form, or texture in a DMS task. In Experiment 1, top-down attentional setting was not manipulated; thus, participants were likely to attend to all three features. The target was always identical to the reference stimulus, whereas the distractor differed from the reference stimulus by only one feature. We examined the relative saliency between features by observing the relative accuracy of recognition at different stimulus durations. The results of matching accuracy revealed that colour was encoded at the fastest speed, form was encoded at a slower speed, and texture was encoded at the slowest speed. These results suggested that salience decreased according to the following order: colour, form, and texture. In Experiments 2 and 3, we manipulated the attentional setting in a modified DMS task to further test the relative saliency hypothesis. The participants were required to attend to one of the three features and to determine which of the two test stimuli matched the value of the attended feature to that of the reference stimulus. Both accuracy and RTs showed similar patterns of results, supporting the relative saliency hypothesis. When participants matched colour, changes of any task-irrelevant feature did not interfere with the matching performance. When participants matched form, the performance decreased when the target mismatched on the salient colour feature dimension. When participants matched texture, performance decreased when the target mismatched on either colour or form dimension. These results cannot be explained by a perceptual limitation account, which states that the results of Experiment 2 are due to the poor perceptual discriminability on texture rather than due to attention or memory processes. It is perhaps because of the difficulty of perceiving texture compared to form or colour when the duration was 150 ms, texture matching is interfered by a form or colour change. In Experiment 3, despite the stimulus duration was increased to 400 ms, which allowed the participants to have sufficient time to encode texture, matching texture was still affected by colour and form changes. Based on the results of Experiment 1, showing the rank order of feature salience, these results are consistent with the relative saliency hypothesis. According to this hypothesis, all features of an attended object are obligatorily encoded into the memory (Logan, 1988), as suggested by an object-based selection model (Egly et al., 1994; O Craven et al., 1999; Roelfsema et al., 1998) and FEATURE MEMORY 723 a biased-competition attention model (Desimone, 1998; Desimone & Duncan, 1995) regardless of task relevancy. It is assumed that different features are perceived at different speeds, depending on their relative saliency (Lamberts & Kent, 2008; Viviani & Aymoz, 2001; Woodman & Vogel, 2005, 2008). Within a certain time window, a more salient feature is sampled at a higher rate (Lamberts, 2002) and, thus, is stored with more memory traces. The salient feature representations are assumed to be stronger than the less salient feature representations. When the test display is shown, the obligatory retrieval of a previously viewed object (Logan, 1988) reactivates all feature representations concurrently with the processing of test stimuli. Participants must compare the test stimuli with the retrieved feature representations for accurate memory recognition. Memory judgement based on a more salient and stronger feature representation is more accurate and less likely to be affected by the change of a less salient taskirrelevant feature in the target at test time. However, when a memory judgement is based on a weaker feature representation, a change of the more salient feature interferes with the memory judgement. Given that colour is the most salient feature, matching colour is not affected by any change of other task-irrelevant features. In contrast, matching texture, the least salient feature, was affected by changes in both colour and form. These results converge to suggest that the relative feature saliency between task-relevant and task-irrelevant features determines whether a task-irrelevant feature influences memory judgement on the task-relevant feature. Top-down attentional setting cannot always override the influence of bottom-up stimulus salience on the memory judgement of a task-relevant feature. It is important to note that the current results revealed the role of top-down attentional modulation on delayed recognition of a feature. When attending to all three features for 150 ms (Experiment 1A), texture memory was not processed to its full extent. The mean hit rate based on texture discrimination was only In contrast, participants were able to accurately judge the texture pattern in the presence of a mismatch salient colour (mean hit rate of 0.93) when they were only required to attend to the texture (Experiment 2). These results are consistent with the previous findings showing that attention to a task-relevant feature selectively enhances the

12 724 YANG, CHIU, YEH neural representation of the attended feature (see Maunsell & Treue, 2006, for a review). The feature-based attentional modulation is independent of space and objecthood, which has been demonstrated in psychophysical (Rossi & Paradiso, 1995; Wegener, Ehn, Aurich, Galashan, & Kreiter, 2008), neurophysiological (Treue & Martinez Trujillo, 1999), electrophysiological (Nobre, Rao, & Chelazzi, 2006), and neuroimaging studies (Saenz, Buracas, & Boynton, 2002). More importantly, our results revealed that top-down attentional modulation cannot fully override the influence of bottom-up stimulus saliency on matching a task-relevant feature. Instead, our data suggest that both top-down and bottom-up factors influence delayed recognition of a simple feature. A change in a salient task-irrelevant feature can interfere with the target selection. However, it is possible that the interference by a salient but task-irreleavant feature was casued by automatic attentional capture at test time (rather than at the time of encoding) as many previous studies showed that a salient feature captures attention despite it is not task-relevant, especially at the early stage of visual processing (see Theeuwes, 2010, for a review). Once captured, the matching on this salient feature would start and likely to be completed earlier than the matching on a less salient feature. As a result, one might argue that our results could be explained by attentional capture of the salient task-irrelevant features. However, we think that this account is unlikely. In two-thirds of trials, both target and distractor stimuli differed in the task-irrelevant feature from the reference stimulus at test time. The salient task-irrelevant feature could elicit automatic attentional processing, which would interfere with the matching performance on the less-salient feature of the target. In the rest of the trials (one-third), the salient task-irrelevant feature of the target matched that of the reference stimulus, and would counterintuitively facilitate the matching performance on the less salient feature. Although combining these two effects, the attentional capture by salient task-irrelevant feature would produce more interference than facilitation; this account provided a rather convoluted way to explain our data. In contrast, the relative saliency hypothesis applied to the encoding phase provides a more parsimonious account of our data. Nevertheless, it is possible that the participants in our experiment maintained all features so that they could use redundant feature information in the DMS task. This strategy might be partly because the task-relevant feature and one of the task-irrelevant features of the target remained the same as in the reference, but the third feature was different. In other words, our paradigm may encourage participants to develop a goal for maintaining the task-irrelevant features of the reference stimulus. In previous studies using a change detection task (Luck & Vogel, 1997), participants did not know which feature would be changed, and, thus, they might have encoded and maintained all of the stimulus features simultaneously. Further investigation is required to understand the context in which visual working memory is feature based or object based. In conclusion, when all features are maintained in the visual working memory, it is difficult to ignore a salient task-irrelevant feature when the memory of a less salient feature is tested. Magnussen et al. (Baumann et al., 2008; Magnussen et al., 1998) found that task-irrelvant spatial frequency influenced orientation judgement, and vice versa. These results may be attributed to the evidence indicating that these two features were equally salient in a grating pattern. The results from these studies and the current study taken together suggest that individual features of an object are conjointly selected into the visual memory when the task-irrelevant feature varies in the test probe. We further argue that independent comparison is made in parallel across different features. A more salient feature interferes with the memory decision of a less salient feature. The top-down task set and the bottom-up feature saliency interactively determine the memory decision of the attended feature. Original manuscript received November 2011 Revised manuscript received March 2012 First published online July 2012 REFERENCES Baddeley, A. (1981). The concept of working memory: A view of its current state and probable future development. Cognition, 10, Baumann, O., Endestad, T., Magnussen, S., & Greenlee, M. W. (2008). Delayed discrimination of spatial frequency for gratings of different orientation: Behavioral and fmri evidence for low-level

13 perceptual memory stores in early visual cortex. Experimental Brain Research, 188, Bergen, J. R., & Landy, M. S. (1991). Computational modeling of visual texture segregation. In M. S. Landy & J. A. Movshon (Eds.), Computational models of visual processing (pp ). Cambridge, MA: The MIT Press. Broadbent, D. E. (1965). Information processing in the nervous system. Science, 150, Brockdorff, N., & Lamberts, K. (2000). A featuresampling account of the time course of old-new recognition judgments. Journal of Experimental Psychology: Learning, Memory and Cognition, 26, Desimone, R. (1998). Visual attention mediated by biased competition in extrastriate visual cortex. Philosophical Transactions of the Royal Society: Biological Sciences, 353B, Desimone, R., & Duncan, J. (1995). Neural mechanisms of selective visual attention. Annual Review of Neuroscience, 18, Downing, P. E. (2000). Interactions between visual working memory and selective attention. Psychological Science, 11, Egly, R., Driver, J., & Rafal, R. D. (1994). Shifting visual attention between objects and locations: Evidence from normal and parietal lesion subjects. Journal of Experimental Psychology: General, 123, Folk, C. L., & Remington, R. (2010). A critical evaluation of the disengagement hypothesis. Acta Psychologica, 135, Folk, C. L., Remington, R. W., & Johnston, J. C. (1992). Involuntary covert orienting is contingent on attentional control settings. Journal of Experimental Psychology: Human Perception and Performance, 18, Folk, C. L., Remington, R. W., & Wright, J. H. (1994). The structure of attentional control: Contingent attentional capture by apparent motion, abrupt onset, and color. Journal of Experimental Psychology: Human Perception and Performance, 20, Itti, L., & Koch, C. (2000). A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Research, 40, Julesz, B. (1981). Textons, the elements of texture perception, and their interactions. Nature, 290, Kent, C., & Lamberts, K. (2006). The time course of perception and retrieval in matching and recognition. Journal of Experimental Psychology: Human Perception and Performance, 32, Lamberts, K. (2000). Information-accumulation theory of speeded categorization. Psychological Review, 107, Lamberts, K. (2002). Feature sampling in categorization and recognition of objects. Quarterly Journal of Experimental Psychology: Human Experimental Psychology, 1A, Lamberts, K., Brockdorff, N., & Heit, E. (2002). Perceptual processes in matching and recognition of complex pictures. Journal of Experimental FEATURE MEMORY 725 Psychology: Human Perception and Performance, 28, Lamberts, K., & Kent, C. (2008). The time course of object-feature retrieval in recognition. Journal of Experimental Psychology: Learning, Memory and Cognition, 34, Logan, G. D. (1988). Toward an instance theory of automatization. Psychological Review, 95, Luck, S. J., & Vogel, E. K. (1997). The capacity of visual working memory for features and conjunctions. Nature, 390, Magnussen, S., & Dyrnes, S. (1994). High-fidelity perceptual long-term memory. Psychological Science, 5, Magnussen, S., Greenlee, M. W., Aslaksen, P. M., & Kildebo, O. Ø. (2003). High-fidelity perceptual long-term memory revisited*and confirmed. Psychological Science, 14, Magnussen, S., Idås, E., & Myhre, S. H. (1998). Representation of orientation and spatial frequency in perception and memory: A choice reaction-time analysis. Journal of Experimental Psychology: Human Perception and Performance, 24, Maunsell, J. H. R., & Treue, S. (2006). Feature-based attention in visual cortex. Trends in Neurosciences, 29, Nobre, A. C., Rao, A., & Chelazzi, L. (2006). Selective attention to specific features within objects: Behavioral and electrophysiological evidence. Journal of Cognitive Neuroscience, 18, Nothdurft, H.-C. (2006). Salience and target selection in visual search. Visual Cognition, 14, O Craven, K. M., Downing, P. E., & Kanwisher, N. (1999). fmri evidence for objects as the units of attentional selection. Nature, 401, Olivers, C. N., Meijer, F., & Theeuwes, J. (2006). Feature-based memory-driven attentional capture: Visual working memory content affects visual attention. Journal of Experimental Psychology: Human Perception and Performance, 32, Potter, M. C. (1976). Short-term conceptual memory for pictures. Journal of Experimental Psychology: Human Learning and Memory, 2, Rensink, R. A. (2000). The dynamic representation of scenes. Visual Cognition, 7, Roelfsema, P. R., Lamme, V. A., & Spekreijse, H. (1998). Object-based attention in the primary visual cortex of the macaque monkey. Nature, 395, Rossi, A. F., & Paradiso, M. A. (1995). Feature-specific effects of selective visual attention. Vision Research, 35, Saenz, M., Buracas, G. T., & Boynton, G. M. (2002). Global effects of feature-based attention in human visual cortex. Nature Neuroscience, 5, Schneider, W., Eschman, A., & Zuccolotto, A. (2002). E-prime: User s guide. Psychology Software Inc. Theeuwes, J. (2004). Top-down search strategies cannot override attentional capture. Psychonomic Bulletin and Review, 11, Theeuwes, J. (2010). Top-down and bottom-up control of visual selection. Acta Psychologica, 135, 7799.