Cognition 146 (2015) Contents lists available at ScienceDirect. Cognition. journal homepage:

Transcription

1 Cognition 146 (2015) Contents lists available at ScienceDirect Cognition journal homepage: Organization principles in visual working memory: Evidence from sequential stimulus display Zaifeng Gao, Qiyang Gao, Ning Tang, Rende Shui, Mowei Shen Department of Psychology, Zhejiang University, Hangzhou, China article info abstract Article history: Received 3 September 2014 Revised 13 August 2015 Accepted 11 October 2015 Keywords: Gestalt principle Visual working memory Perception Although the mechanisms of visual working memory (VWM) have been studied extensively in recent years, the active property of VWM has received less attention. In the current study, we examined how VWM integrates sequentially presented stimuli by focusing on the role of Gestalt principles, which are important organizing principles in perceptual integration. We manipulated the level of Gestalt cues among three or four sequentially presented objects that were memorized. The Gestalt principle could not emerge unless all the objects appeared together. We distinguished two hypotheses: a perceptionalike hypothesis and an encoding-specificity hypothesis. The former predicts that the Gestalt cue will play a role in information integration within VWM; the latter predicts that the Gestalt cue will not operate within VWM. In four experiments, we demonstrated that collinearity (Experiment 1) and closure (Experiment 2) cues significantly improved VWM performance, and this facilitation was not affected by the testing manner (Experiment 3) or by adding extra colors to the memorized objects (Experiment 4). Finally, we re-established the Gestalt cue benefit with similarity cues (Experiment 5). These findings together suggest that VWM realizes and uses potential Gestalt principles within the stored representations, supporting a perception-alike hypothesis. Ó 2015 Elsevier B.V. All rights reserved. 1. Introduction Visual working memory (VWM) is an online storage buffer actively involved in maintaining and manipulating a limited set of visual information (Baddeley, 1998; Luck & Vogel, 2013). Since VWM capacity predicts a broad range of high-level human abilities (e.g., fluid intelligence, academic achievement) (e.g., Cowan et al., 2005; Unsworth, Fukuda, Awh, & Vogel, 2014; Vogel, McCollough, & Machizawa, 2005), there has been strong interest in characterizing the nature of this capacity limit (for reviews see Baddeley, 2012; Luck & Vogel, 2013; Ma, Husain, & Bays, 2014; Orhan & Jacobs, 2014; Suchow, Fougnie, Brady, & Alvarez, 2014). The majority of the current VWM studies have focused on the representation unit of VWM (e.g., Brady, Konkle, & Alvarez, 2011; Fougnie & Alvarez, 2011; Huang & Pashler, 2007; Shen, Yu, Xu, & Gao, 2013; Wheeler & Treisman, 2002; Xu, 2002; Zhang, Johnson, Woodman, & Luck, 2012), the nature of VWM resources (e.g., Bays & Husain, 2008; Huang, 2010; Van den Berg, Shin, Chou, George, & Ma, 2012; Zhang & Luck, 2008), and the mechanisms of distributing VWM resources (e.g., Griffin & Nobre, 2003; Rerko, Corresponding author at: Department of Psychology and Behavioral Sciences, Xixi Campus, Zhejiang University, Hangzhou , China. address: mwshen@zju.edu.cn (M. Shen). Souza, & Oberauer, 2014; Schmidt, Vogel, Woodman, & Luck, 2002; Williams & Woodman, 2012). However, the active property of VWM has received less attention. For instance, to have a full view of a natural scene during traveling, sequential saccades are conducted to sample the visual information at distinct locations. It has been suggested that VWM is involved in integrating the sampled discrete information into a coherent visual percept (Hollingworth, Richard, & Luck, 2008; Richard, Luck, & Hollingworth, 2008). However, how this integration takes place in VWM for the sequentially processed information remains unknown. In the current study, we investigated this issue by studying whether the information-organizing principles employed in visual perception also operate in VWM for the sequentially memorized objects. Although we do not know much about the organization principle in VWM for sequentially presented objects, there is a considerable amount of research on organization principles for simultaneously presented information in both visual perception and VWM. The Gestalt grouping principles (e.g., connectedness, proximity, similarity) are among the most important principles in grouping multiple pieces of isolated information into an integrated unit (for reviews see Todorovic, 2008; Wagemans et al., 2012). Interestingly, these Gestalt grouping principles seem to operate also in a post-perceptual stage in VWM. For example, it /Ó 2015 Elsevier B.V. All rights reserved.

2 278 Z. Gao et al. / Cognition 146 (2015) has been demonstrated that proximity and connectedness affect VWM performance (Woodman, Vecera, & Luck, 2003; Xu, 2006, 2008). They influence which objects are extracted into VWM and improve VWM performance; displays with Gestalt principles yield better performance than displays without Gestalt cues. Functional magnetic resonance imaging (fmri) (Xu, 2008; Xu & Chun, 2007) and event-related potentials (ERP) (Luria & Vogel, 2011) studies further revealed that isolated objects or features can be grouped into one object via these Gestalt grouping principles. Xu (2006) thus suggested that the proximity and connectedness cues were among essential elements in defining an integrated object representation in VWM. Recent behavioral and ERP studies further show that common fate (Luria & Vogel, 2014), collinearity (Anderson, Vogel, & Awh, 2013), similarity (Gao, Xu, et al., 2011; Peterson & Berryhill, 2013; Shen et al., 2013), and symmetry (Rossi-Arnaud, Pieroni, Spataro, & Baddeley, 2012) also significantly facilitate VWM performance. It is striking to note that all the aforementioned VWM studies adopted a one-shot change-detection task (see Luck & Vogel, 2013 for a review). This task mimics a situation where the visual system extracts information from a scene containing simultaneous presentations of objects in a brief exposure time, and is now the most commonly used paradigm in the VWM field. However, in everyday life, there are more situations where the to-beprocessed information is presented in a serial manner. For instance, our naturalistic perception usually involves constantly changing scenes, such as when making eye movements, which are inherently sequential (e.g., Richard et al., 2008). Although recent studies suggest that simultaneous and sequential displays are likely to share similar mechanisms in VWM (e.g., Allen, Baddeley, & Hitch, 2014; Ricker & Cowan, 2014; Shen, Gao, Ding, Zhou, & Huang, 2014; Xu & Chun, 2006), there are fundamental processing differences between the two (e.g., Frick, 1985; Kesner, Hopkind, & Fineman, 1994; Lecerf & Ribaupierre, 2005; Mammarella et al., 2006; Shulman, 1971; Zarahn, Aguirre, & D Esposito, 2000). For instance, for the sequential stimulus presentation, the to-be-remembered items are in sequence, and VWM has to update and refresh its contents from time to time. Only one item is processed at a time, and the possible relationship between single items cannot be extracted into VWM during the encoding phase. In contrast, for the simultaneous display, all the to-be-remembered items are presented in a single, time-limited scene. This presentation mode offers more opportunities to organize the items into meaningful groups or chunks (Lecerf & Ribaupierre, 2005). Indeed, it has been suggested that beyond the to-be-remembered items, the relationship between the items (e.g., a global pattern formed by items) could also be established (e.g., Jiang, Chun, & Olson, 2004; Jiang, Olson, & Chun, 2000). Although the simultaneous display allows more information to be encoded into VWM, it also has drawbacks. For instance, it often leaves us unable to draw clear distinctions between perception and VWM for an observed effect, since the product of visual perception is usually directly consolidated and stored into VWM (Gao, Gao, Li, Sun, & Shen, 2011; Orhan & Jacobs, 2013). Although the previous one-shot VWM studies have thrown light on the organization principles in VWM, it is difficult to distinguish whether the revealed facilitation effect is attributed to the grouping occurring at the perceptual stage or at the maintenance stage. A recent ERP experiment indeed demonstrated grouping effects in both the perceptual individuation stage and the VWM maintenance phase in one study (Anderson et al., 2013). Moreover, considering that VWM and perception have a very tight relationship, it is quite possible that the result of perceptual grouping is extracted into VWM and thus indirectly affects VWM performance. Corroborating this possibility, most of the aforementioned VWM studies on Gestalt grouping actually explored how the single items in a simultaneous display are perceptually grouped and whether such grouping is transferred from perception to VWM (e.g., Jiang et al., 2004; Peterson & Berryhill, 2013; Woodman et al., 2003; Xu, 2006). To understand the information organization principles specially operating in VWM, a viable alternative is to employ a sequential stimulus presentation manner. On the other hand, due to the aforementioned processing differences, the manipulation of the sequential vs. simultaneous presentation has been an important variable in previous studies of verbal working memory (in which the verbal stimuli were show visually) (e.g., Aivio & Csapot, 1971; Frick, 1985; Pazzaglia, 1999; Penney, 1975) and spatial working memory (exploring information integration of sequential displays) (e.g., Brockmole, Irwin, & Wang, 2003; Brockmole & Wang, 2003; Brockmole, Wang, & Irwin, 2002; Jiang, 2004; Jiang & Kumar, 2004; Jiang, Kumar, & Vickery, 2005). In contrast, although many VWM studies have been conducted using the sequential presentation manner, all these studies predominately focused on mechanisms such as VWM capacity (including crossdomain capacity, e.g., Farrell & Oberauer, 2014) (e.g., Cohen, Sreenivasan, & D Esposito, 2014; Fischer, 2001; Kumar & Jiang, 2005; Wood, 2007; Xu & Chun, 2006), representation retention and decay (e.g., Ricker & Cowan, 2014; Salmela, Moisala, & Alho, 2014), and VWM updating mechanisms (e.g., Crone, Wendelken, Donohue, van Leijenhorst, & Bunge, 2006; Federico, Delogu, & Raffone, 2014). To our knowledge, no study has addressed the potential principles of organizing sequentially presented items in VWM. The current study also aims at shedding light on this issue. To this end, the current study presented objects (orientated disks) sequentially at distinct locations, while manipulating whether regularities (i.e., Gestalt principles) existed among the objects to be remembered. Any regularities between the stimuli would not emerge unless all the sequentially presented stimuli were combined within VWM (i.e., within VWM it would appear as a simultaneous display). To probe the effects of regularity on VWM performance, we had to ensure that most of the remembered stimuli were already in VWM so that the regularity could potentially emerge. Therefore, we constrained the memory load to within the VWM capacity of normal adults by presenting 3 or 4 objects (but see the current Experiment 5; Cowan, 2001; Gao et al., 2009; Luck & Vogel, 1997; Xu & Chun, 2006). For the current study, we tested two distinct hypotheses. The first is the perception-alike hypothesis, which proposes that perception and VWM share analogous processing mechanisms, which has received evidence from several lines of recent VWM studies (e.g., Gao, Gao, et al., 2011; Gao, Li, Yin, & Shen, 2010; Gao, Xu, et al., 2011; Harrison & Tong, 2009; Mayer et al., 2007; Serences, Ester, Vogel, & Awh, 2009). The perception-alike hypothesis predicts that VWM can realize and effectively use the potential regularities of the Gestalt principles, and VWM performance will be significantly higher for with-regularity than for no-regularity conditions (referred to as regularity benefit). The second hypothesis is the encoding-specificity hypothesis, which claims that what is stored is determined by what is perceived and how it is encoded (Tulving & Thomson, 1973). The encoding-specificity hypothesis predicts that a previously observed grouping facilitation takes place at the perceptual stage when a simultaneous display is employed. However, since no relationship between the single items is available and extracted into VWM in the sequential display condition, there will be no difference between withregularity and no-regularity conditions. 2. Experiment 1: Detecting collinearity cues in VWM In a previous one-shot change detection task, Anderson et al. (2013) presented the participants pairs of oriented solid disks with

3 Z. Gao et al. / Cognition 146 (2015) rectangular gaps, and demonstrated that when the memorized disks were collinear, VWM performance (both the number of stored objects and representation precision) was significantly improved. They concluded that VWM treated the two isolated disks as one object. In Experiment 1, we examined whether the collinearity cue could improve VWM performance for sequentially presented disks. If VWM can detect and use collinearity cues, then VWM will treat the two isolated disks as one, hence exhibiting a significant regularity benefit. The current experiment contained two sub-experiments. In Experiment 1a the probability for with-regularity condition and no-regularity condition was balanced. As detailed below, we revealed a significant regularity benefit with the collinearity cue. However, the results may be not due to that VWM could detect and use the collinearity cue, but was resulted from a guessing strategy. Particularly, because there were 1/2 of trials having collinear pairs of disks overall in the memory array, 1 therefore, when participants remembered some but not all disks in a trial, they intended to guess that an unremembered disk was collinear with one of the known disk. To examine this guessing-strategy alternative, we added an Experiment 1b by setting the probability of noregularity condition as 2/3 (hence 1/3 of trials overall contained the collinearity cue). If the guessing-strategy was correct, then no regularity benefit would be observed Methods Participants There were 16 (10 male; age years) and 16 participants (8 male; age years) in Experiment 1a, and 1b, respectively. The participants were students at Zhejiang University, and were volunteered to participate in the experiment with signed consent forms. All had normal color vision and normal or corrected-tonormal visual acuity Stimuli & apparatus The experiment was programmed via Presentation Ò (Neurobehavioral Systems Inc, Berkeley, USA). The objects were presented within a radius of 5 of visual angle around screen center, against a gray (128, 128, 128, RGB) background on a 17-in. CRT monitor (refresh rate 100 Hz). The participants were required to remember the orientation of solid disks (see Fig. 1; 0.75 of visual angle) that contained a rectangular gap (0.325 of visual angle). Eight different orientations were used: 0, 45, 90, 135, 180, 225, 270, and 315. The distance between each two disks was of visual angle Procedure Participants were required to memorize the four orientations of the disks. The four disks were divided into two parts; each part contained a pair of disks that were not collinear. Each trial began by presenting two red digits at the screen center for 500 ms, and the participants were required to repeatedly vocalize the two digits separately (e.g., 2 and 1, not 21 ) throughout the trial to prevent them from verbally rehearsing the orientation. After a blank interval of 200 ms, two disks (memory array 1) were presented for 250 ms (see Fig. 1A). To ensure that the participants cannot integrate the first two disks with the subsequently presented stimulus at a perceptual stage and allow a sufficient consolidation time of memory array 1 (cf. Jiang, 2004), the 1 As detailed below, there were actually three memory conditions in Experiment 1a: (1) With-regularity condition with four disks in two collinear cues, (2) halfregularity condition with two disks in a collinear cue and two disks not, and (3) noregularity condition with four disks without any collinear cues. Therefore, still there were 1/2 of collinear pairs of disks across all conditions. first two disks were backward masked for 100 ms immediately after their disappearance, followed by a 500-ms blank interval. Therefore, the representation of memory array 1 was available in VWM when memory array 2 was displayed. Another two disks (memory array 2) were then presented for 250 ms and backward masked for 100 ms. Finally, after a 1000-ms blank interval, a test array containing four disk stimuli was shown for a maximum of 3000 ms, with their locations being kept constant between memory and test arrays. In 1/2 of trials, the orientation of one of the disks in the test array was changed to a new value that not used in the memorized disks. The participants were required to judge whether there was any change in orientation relative to the previously memorized orientations. Participants were instructed to press F on the keyboard when no change occurred and J ifa new orientation appeared in the test array. Participants were encouraged to respond accurately and their response accuracy was analyzed Design & analysis We manipulated the degree of collinearity between the four orientated disks when they appeared simultaneously by adjusting the locations of the gaps. In the critical condition, the two pairs of disks could form two virtual rectangles when their oriented gaps were aligned according to the collinearity principle (With-regularity condition; see Fig. 1B). Two rectangles hence could be observed when no change took place between the memory and test arrays (1/2 of trials of With-regularity condition); whereas only one virtual rectangle could be observed in the test array when a change occurred (1/2 of trials of With-regularity condition). In the baseline condition, the disks were randomly selected and no collinearity existed between the disks. 2 Consequently, no subjective rectangles could be formed (No-regularity condition; see Fig. 1D) and observed in the test array when no change took place between memory and test arrays (1/2 of trials of No-regularity condition); however, a rectangle would be observed if a change occurred (1/2 of trials of Noregularity condition). Because under the No-change trials of Withregularity and No-regularity conditions, there were two and zero rectangles, respectively, the participants may make a judgment by simply counting the number of subjective rectangles in the test array. To prevent this possible strategy, we added a third condition in which one pair of disks formed a subjective rectangle while the other pair did not (Half-regularity condition; see Fig. 1C). For the test array of the half-regularity condition, when a change took place the test array could show two subjective rectangles (the new item and the old one formed a second rectangle; 1/4 of trials of Halfregularity condition), or no subject rectangles (an item in the previous rectangle was replaced with a new one; 1/4 of trials of Halfregularity condition); when no change took place, only one subjective rectangle was displayed (1/2 of trials of Half-regularity condition). The current experiment adopted a one-factor within-subjects design (Regularity-condition: With-regularity, Half-regularity, and No-regularity). In Experiment 1a, the probability for With-regularity, Half-regularity, and No-regularity condition was equalized (i.e., 1/3). In Experiment 1b, 1/6 of overall trials were With-regulation condition, 1/6 of overall trials were Halfregularity condition, and 2/3 of overall trials were No-regularity condition. 3 There were 64 trials in each condition of Experiment 1a, resulting in a total of 192 trials; whereas in Experiment 1b, there 2 Since the disks were randomly selected, there were cases that the same stimuli as in the With-regularity condition were selected. However, in this situation, there was no collinearity cue between them. 3 Across all conditions, there were 50% (1/3 in With-regularity +1/6 in Halfregularity) of collinear pairs of disks in Experiment 1a, and 25% (1/6 in Withregularity +1/12 in Half-regularity) in Experiment 1b.

4 280 Z. Gao et al. / Cognition 146 (2015) Fig. 1. (A) A schematic illustration of a single trial in Experiment 1. In this sample trial, the two sequentially presented pairs of disks can be grouped into two objects containing virtual elongated rectangles occluding black disks via collinearity cues, and no change takes place in the test array. (B), (C), and (D) demonstrate the three regularity conditions; (B) both disk pairs are collinear, (C) one pair (top) is collinear, and (D) none of the pairs is collinear. were 64 trials in both With- and Half-regularity conditions, and 256 trials in the No-regularity condition, resulting in a total of 384 trials. All the three conditions were fully randomized and presented. There was a 5-min break every 64 trials. The entire experiment lasted approximately 20 min in Experiment 1a, and 60 min in Experiment 1b. Prior to the experimental blocks, participants were trained with at least 48 practice trials. A one-way repeated Analysis of Variance (ANOVA) with Regularity-condition as the within-subjects factor was conducted on accuracy. Accuracy was determined in terms of signal detection theory, which allowed us to disentangle the sensitivity (d 0 ) to the change from the response bias (criterion). Significant main effect of Regularity-condition was followed up by post hoc Bonferronicorrected contrasts. The descriptive data of the hit rates and false alarms were shown in Table Results The result profiles of Experiments 1a and 1b were similar. Both showed that the d 0 was highest under the With-regularity condition, and lowest under the No-regularity condition Experiment 1a The one-way AONVA on d 0 (Fig. 2A) revealed a significant main effect of Regularity-condition, F(2,30) = 33.11, p <.001, g 2 p =.69. Post hoc contrasts revealed that the d 0 was significantly higher for With-regularity than for Half-regularity (p =.001) and Noregularity (p <.001) conditions, and Half-regularity was significantly higher than No-regularity condition (p =.016). The one-way AONVA on criterion (Fig. 2B) revealed that the main effect of Regularity-condition did not reach significance, F (2,30) = 1.36, p =.27, g 2 p =.08, suggesting that the regularity conditions did not affect the participants criterion Experiment 1b The one-way AONVA on d 0 (Fig. 2C) revealed a significant main effect of Regularity-condition, F(2,30) = 7.735, p =.007, g 2 p =.34. Post hoc contrasts revealed that the d 0 was significantly higher for With-regularity than for No-regularity (p =.011) condition. The difference between With-regularity and Half-regularity condition was marginally significant (p =.077). There was no difference between Half-regularity and No-regularity condition (p =.96). The one-way AONVA on criterion (Fig. 2D) revealed that the main effect of Regularity-condition did not reach significance, F (2,30) =.005, p =.99, g 2 p = 0, suggesting that the regularity conditions did not affect the participants criterion. Table 1 Means (S.E.) for hit rates and false alarms in all conditions of Experiments 1 5. Index Experiment 1a 1b Hit rates With-regularity.88(.02).83(.03).92(.01).70 (.03).68 (.03).63 (.02) False/halfregularity.79(.03).78(.03).80(.04) No-regularity.77(.03).78(.02).81(.03).62 (.01).18 (.02).57 (.03) False alarms With-regularity.27(.04).39(.04).23(.04).23 (.04).67 (.02).19 (.02) False/halfregularity.35(.04).45(.03).32(.03) No-regularity.48(.03).47(.02).33(.03).70 (.01) 2.3. Discussion.25 (.02).19 (.02) In line with Anderson et al. (2013), the results of Experiments 1a and 1b consistently demonstrated that the VWM performance was significantly improved when clear collinearity cues existed (i.e., regularity benefit). Critically, we revealed the regularity benefit with the sequentially presented stimuli, suggesting that VWM can detect and use collinearity cues after the sequentially presented stimuli have been extracted into VWM, even when most of the trials were without regularity. Moreover, our further analysis implied that the Gestalt regularity began to play a role from the beginning of the formal experiment. Particularly, we examined the first 25% and the last 25% of the trials in Experiment 1a, in which all the three regularity conditions were equally tested. We found significant regularity benefit in both stages in terms of d 0 : F(2,30) = 11.85, p <.001, g 2 p =.441 (With-regularity = 1.52, Half-regularity = 1.16, and No-regularity = 0.56) for the first 25% of trials, and F(2,30) = 9.813, p =.001, g 2 p =.395 (With-regularity = 1.94, Half-regularity = 1.30, and Noregularity = 1.13) for the last 25% of trials. Moreover, the regularity benefit seems to be more obvious for the first 25% of trials than the last 25% of trials, because d 0 was significantly higher under Half-regularity than under No-regularity for the first 25% of trials (p =.017), but not for the last 25% of trials (p =.377). Additionally, we found that the d 0 under With-regularity and Half-regularity conditions was dropped in Experiment 1b relative to that in Experiment 1a, while the d 0 under No-regularity condition was a little bit raised in Experiment 1b. These results could be due to the practicing effect or task set configuration formed

5 Z. Gao et al. / Cognition 146 (2015) Fig. 2. The results of d 0 (A), criterion (B) in Experiment 1a, and d 0 (C), criterion (D) in Experiment 1b. Error bars stands for S.E. when completing the extensive trials of No-regularity condition in Experiment 1b. We need to point out that there was an unbalanced probability of a change, given a regular test display, a half-regular test display, or an irregular test display in Experiment 1. Particularly, the P (change regular display) = P(change irregular display) = 0.33, and P(change half-regular display) = 0.66 for Experiment 1a; and P (change regular display) = 0.33, P(change irregular display) = 0.11, and P(change half-regular) = 0.83 for Experiment 1b. This imbalance (or statistical regularity) may affect the criterion in that the participants were more inclined to judge change when they saw a half-regular display than for any other displays. 4 Moreover, the participants may intend to guess that any forgotten orientation was collinear with the one being retained, which would be better than chance if the forgotten orientation was tested. This guessingstrategy alternative could explain why the d 0 was higher under regular than under irregular displays. Critically, the guessing-strategy alternative predicted a significant difference on criterion between the three memorized conditions at least in Experiment 1b, where a massive difference existed between the aforementioned three conditional probabilities of change. Contrary to this prediction, there was no difference on criterion between the three memorized conditions in both Experiment 1a and 1b. Additionally, the guessing-strategy alternative predicted a better performance for half-regular than for irregular displays. Although this prediction was supported by Experiment 1a, a non-significant difference was found in Experiment 1b. These results together suggested that the participants did not make much use of the statistical regularities of the memorized stimuli in the present experiments. 4 We thanked an anonymous reviewer for pointing this out to us. 3. Experiment 2: Detecting closure cue in VWM Experiment 2 examined whether the regularity benefit in Experiment 1 could be generalized to other Gestalt principles. Here, we used the Kanizsa triangle as the stimulus of interest. In line with Experiment 1, it also forms a virtual contour constructed by an additional closure cue. If sequentially presented stimuli could form a Kanizsa triangle in VWM, then a similar finding as in Experiment 1 would be observed. Additionally, to have a more strict examination of the guessingstrategy alternative, in Experiment 2 we used the same stimuli for both With-regularity and No-regularity conditions. The guessingstrategy would predict a non-significant difference between the two conditions Methods Participants, stimuli & apparatus Sixteen undergraduate or graduate students (8 male; age years) at Zhejiang University volunteered to participate in the experiment and signed consent forms. A new type of disk (0.75 of visual angle from a viewing distance of 57 cm) was used (see Fig. 3). The participants were required to memorize disks orientations. Twelve different orientations were used: 0,30,60,90, 120, 150, 180, 210, 240, 270, 300, and 330 relative to the horizontal line. The other aspects were the same to Experiment Procedure & design Participants were required to memorize the orientations of three sequentially presented disks. Each disk was presented for 150 ms and was backward masked for 100 ms. A 600-ms blank

6 282 Z. Gao et al. / Cognition 146 (2015) Fig. 3. (A) A schematic illustration of a single trial in Experiment 2. In this sample trial, the three sequentially presented memory items could form a Kanizsa triangle in VWM, and n change takes place in the test array. (B), (C) and (D) demonstrate the three regularity conditions; (B) clear closure cue exists, (C) part of the closure cue (the bottom two), and (D) no closure cue exists. interval was inserted before the next disk was presented, so that the same stimulus onset asynchrony (SOA) between the sequentially memorized stimuli was used as in Experiment 1. The other aspects were the same as in Experiment 1. Three conditions were tested. In the first condition, the three disks with notches formed a Kanizsa triangle (With-regularity condition; see Fig. 3B). In the second condition, the same stimuli were used but the locations of the notches were adjusted so that no Kanizsa triangle could be formed (No-regularity condition; see Fig. 3D). Finally, to prevent the participants from using certain other strategies (e.g., memorizing the first two stimuli and making a judgment by simply checking whether an intact Kanizsa triangle existed) in the With-regularity condition, the third condition consisted of two elements of a Kanizsa triangle and a disk not belonging to the Kanizsa triangle (False-regularity condition; see Fig. 3C). As for the test array, when a change (1/2 of trials) occurred a new disk not used in the memory array was displayed. It is important to note that in the With-regularity condition, a Kanizsa triangle appeared under the no-change condition, while under the change condition, two elements of a Kanizsa triangle and a new disk not belonging to the memorized objects were displayed (i.e., similar to the memorized stimuli in the False-regularity condition). For the False-regularity condition, a reversed scenario was used. For the No-regularity condition, there was no sign of Kanizsa triangle (i.e., no False-regularity display appeared) regardless of the change. Therefore, the current experiment adopted a one-factor (Regularity-condition: With-regularity, False-regularity, and Noregularity) within-subjects design. There were 72 trials in each condition, with a total of 216 trials, which were fully randomized and presented in three blocks of 72 trials each. A break of 5 min was allowed between blocks, and the entire experiment lasted approximately 30 min. Prior to the experimental blocks, participants were trained with at least 48 practice trials, including 16 trials from each experimental condition. A one-way repeated ANOVA with Regularity-condition as within-subjects factors was conducted on d 0 and criterion. The other aspects were the same to Experiment Results The one-way AONVA on d 0 (Fig. 4A) revealed a significant main effect of Regularity-condition, F(2,30) = 22.06, p <.001, g 2 p =.60. Post hoc contrasts revealed that the d 0 was significantly higher for With-regularity than for No-regularity (p <.001) and Falseregularity (p <.001) conditions, yet there was no difference between the latter two (p = 1.0). The one-way AONVA on criterion (Fig. 4B) revealed that the main effect of Regularity-condition did not reach significance, F (2,30) =.56, p =.58, g p 2 =.04, suggesting that the regularity conditions did not affect the participants criterion Discussion By probing a new Gestalt principle of closure, we replicated the findings of Experiment 1; that is, regularity significantly enhanced the VWM performance relative to the condition without regularity. It is of note that the probability of a change, given a regular test display, a false-regular test display, or an irregular test display was balanced (0.5) in Experiment 2, further suggesting that the findings of Experiment 1 were not resulted from an unbalanced statistical regularity. Moreover, Experiment 2 provides further evidence suggesting that the guessing-strategy alternative could not explain the revealed regularity benefit. Because the same stimuli were used in With-regularity and No-regularity conditions, hence if participants used a guessing-strategy, there should be no difference between With-regularity and No-regularity. Moreover, the performance under No-regularity should be better than that under Falseregularity. However, in contrast to these predictions, we found that the performance was significantly better for With-regularity than for No-regularity and False-regularity conditions, and no difference between the latter two. Therefore, VWM uses potential regularities (Gestalt principles) that exist among the stimuli, instead of using guessing-strategy, to improve the performance. In the next two experiments, we further examined the robustness of the revealed regularity benefit, by examining the influence of probing manner (Experiment 3) and the influence of taskirrelevant features such as color (Experiment 4) on the regularity benefit. 4. Experiment 3: Probing manner cannot erase the regularity benefit Both Experiments 1 and 2 used a whole-probe method, in which all the tested stimuli could form a global configuration. It is possible that this testing method primed the participants to process the memorized stimuli in a global manner, which constrained the generality of the revealed regularity benefit. Moreover, during the test phase the participants could see intact virtual objects, which may also prime the participants to check the potential regularity among the stored objects. To rule out these alternatives, we used a partialprobe manner in Experiment 3, in which only one disk was presented at the screen center for judgment while zero or two virtual

7 Z. Gao et al. / Cognition 146 (2015) Fig. 4. The results of d 0 (A), criterion (B) in Experiment 2. Error bars stands for S.E. rectangles could be formed in VWM. The participants were to judge whether the probe matched any of the retained disks. We specifically examined whether the collinearity cue still played a role in VWM when a partial-probe method was adopted. If the regularity benefit was limited to the whole-probe manner, then no regularity benefit would be observed in Experiment 3. Otherwise, a similar regularity benefit would be obtained as in Experiments 1 and 2. disks and requiring the participants to memorize only the orientations of the disks and ignore the colors. Our previous studies have provided consistent evidence suggesting that irrelevant colors can be involuntarily extracted into VWM (e.g., Gao, Gao, et al., 2011; Gao et al., 2010). We examined whether the regularity benefit could still be observed when the task-irrelevant colors were processed into VWM and rendered the similarity cue unavailable Methods Seventeen undergraduate or graduate students (6 male; age years) at Zhejiang University volunteered to participate in the experiment and signed consent forms. One participant was removed due to chance level of performance. The other ones were the same to Experiment 1, except for the following aspects. A partial-probe method was adopted, in which only one disk was presented at the screen center. In addition, a benefit of using the partial-probe method was that it made the possible strategy addressed by the Half-regularity condition ineffective; we therefore tested only the With-regularity and Noregularity conditions. A one-way repeated ANOVA with Regularity-condition (With-regularity vs. No-regularity) as the within-subjects factor was conducted on d 0 and criterion Results & discussion The one-way AONVA on d 0 (Fig. 5A) revealed that the performance under With-regularity condition was significantly higher than No-regularity condition, F(1,15) = 13.56, p =.002, g p 2 =.48. In addition, the main effect of Regularity-condition also reached significance on Criterion, F(1,15) = 5.38, p =.035, g p 2 =.26, suggesting that the participants were more prone to press no change under With-regularity relative to No-regularity conditions. Using the partial-probe method, we replicated the main finding from Experiments 1 and 2, confirming that regularity significantly improved VWM performance. Therefore, regardless of the probing manner, VWM detects and uses potential Gestalt principles that exist among the sequentially presented items. 5. Experiment 4: Task-irrelevant colors cannot erase the regularity benefit In Experiments 1 3, all the memorized stimuli had the same color (black), which may be considered as a similarity cue that could contribute to the observed regularity benefit (see Experiment 5 for example). Thus, we tested whether one pure Gestalt principle could lead to the regularity benefit, by presenting colored 5.1. Methods Sixteen undergraduate or graduate students (5 male; age years) at Zhejiang University volunteered to participate in the experiment and signed consent forms. Because Experiment 3 revealed that probing manner did not affect the regularity benefit, we used a similar design as in Experiment 3 to make the experiment more concise. However, different from Experiment 3, all the memorized stimuli were colored disks. In each trial, four colors were randomly selected from the following six colors: red (255, 0, 0; in RGB value), green (0, 255, 0), blue (0, 0, 255), yellow (255, 255, 0), cyan (0, 255, 255) and magenta (255, 0, 255). However, the probe was always in black. In addition, we compared the regularity benefit between Experiments 3 and 4 to examine the effect of colors. We first computed the difference between With-regularity and No-regularity conditions for Experiments 3 and 4, respectively. Then an independent t-test was conducted between the two experiments Results & discussion In line with the finding of Experiment 3, the one-way AONVA on d 0 (Fig. 6A) revealed that the performance under With-regularity condition was significantly higher than No-regularity condition, F (1,15) = 6.76, p =.02, g p 2 =.31. The main effect of Regularitycondition did not reach significance on Criterion, F(1, 15) = 3.34, p =.09, g p 2 =.18, although a similar trend as in Experiment 3 was revealed. Additional comparison between Experiments 3 and 4 on the degree of regularity benefit suggested that the difference between Experiment 3 and 4 did not reach significance in either d 0, t(30) = 1.06, p =.30, g p 2 =.04, or criterion, t(30) =.74, p =.46, g p 2 =.02. These results suggested that adding extra colors did not impair the regularity benefit. By presenting participants colored disks from which irrelevant colors could be extracted into VWM, we replicated the regularity benefit revealed in Experiments 1 3, suggesting that the irrelevant colors did not impair the regularity benefit.

8 284 Z. Gao et al. / Cognition 146 (2015) Fig. 5. The results of d 0 (A), criterion (B) in Experiment 3. Error bars stands for S.E. Fig. 6. The results of d 0 (A), criterion (B) in Experiment 4. Error bars stands for S.E. 6. Experiment 5: Detecting similarity cues in VWM Experiments 1 4 tested the effect of collinearity and closure cues on the sequentially presented disks. The final products of both cues were virtual contours, the properties of which may constrain the generality of our conclusion. First, the virtual contour can group the separated objects as one virtual object (cf. Anderson et al., 2013), from which the elements orientations could be deduced without intentionally retaining the individual values of each. The revealed regularity benefit hence may be limited to the cues forming virtual contours. Second, the virtual contour contains a hole. Previous studies suggested that hole plays a special role in visual perception, which is a key factor in determining the topological nature of a visual object (Chen, 1982, 2005). Therefore, the current finding may be limited to representations involving holes. Experiment 5 examined the generality of the revealed regularity benefit by testing a new Gestalt cue similarity, by which no virtual contours could be formed. Recent studies demonstrated that the VWM performance was significantly improved when color similarity cues existed in the memorized objects in a one-shot change detection task (Gao, Xu, et al., 2011; Peterson & Berryhill, 2013; Shen et al., 2013). For instance, Peterson and Berryhill (2013) manipulated whether a color value was shared by two of the to-be-memorized four or six colors (i.e., similarity cue). They demonstrated that the VWM performance was considerably improved when the similarity cue existed than when it was absent; however, the two objects should be spatially near each other to make the similarity cue take place. Experiment 5 examined whether the same regularity benefit could be observed when the two shared-value colors were sequentially presented in two distinct memory arrays (see Fig. 7A for an illustration). To avoid the possible guessing-strategy (cf. Experiment 1b), we required the participants to memorize six colors in total, which were equally to be probed. This resulted in 1/6 of overall trials testing the samevalue colors. In addition, to avoid participants just remembering one of the two shared-value colors, different from Experiments 3 and 4, the probe was appeared at one of the previously occupied locations instead of at the screen center. One additional advantage of this setting was that no statistical regularity would be obtained from the test display (cf. the discussion in Experiment 1). If the regularity benefit was due to the specialty of Gestalt cues examined in Experiments 1 4, then no regularity benefit would be observed in Experiment 5. Otherwise, a similar regularity benefit would be obtained as in Experiments Methods Twenty-third 5 undergraduate or graduate students (10 male; age years) at Zhejiang University volunteered to participate in the experiment and signed consent forms. One female participant was removed from analysis because her overall accuracy was out of two standard deviations of the average accuracy of tested group, which resulted in 22 valid participants for further analysis. The stimuli were colored 6 dots (see Fig. 7; 0.75 of visual angle). In each trial, four or six colors were randomly selected from the fol- 5 We pre-set the number of participants as 16. But the experimenter collected 23 participants without noticing. Here we reported all the available data. 6 For interpretation of color in Fig. 7, the reader is referred to the web version of this article.

9 Z. Gao et al. / Cognition 146 (2015) Fig. 7. (A) A schematic illustration of a single trial in Experiment 5. In this sample trial, a similarity cue of colors existed between two sequentially presented memory arrays, and no change takes place in the test array. (B) and (C) demonstrate the two regularity conditions; (B) similarity cue exists, (C) no similarity cue exists. lowing eight colors: red (255, 0, 0), green (0, 255, 0), blue (0, 0, 255), yellow (255, 255, 0), magenta (255, 0, 255), cyan (0, 255, 255), black (0, 0, 0), and white (255, 255, 255). The locations of the to-bememorized colors were randomly selected from six possible locations in the circular configuration with a radius of 5 (see Fig. 7B and C). The current experiment adopted a one-factor (Regularitycondition: With-regularity vs. No-regularity) within-subjects design. The other aspects of the procedure (Fig. 7A) were the same to Experiment 4, except for the following aspects. We replaced the colored disks in Experiment 4 with colored dots. Each memory array contained three distinct colors (see Fig. 7A), the locations of which were randomly selected with the constraint that at most two locations could be spatially adjacent (i.e., no extra location between them). In the No-regularity condition (see Fig. 7C for example), the six to-be-memorized dots had distinct colors. In the With-regularity condition, two of the six dots shared one color (see Fig. 7B for example), and each memory array contained one of the two same-color dots (see Fig. 7A). To ensure the similarity cue was effective, the locations of two same-color dots were adjacent to each other if they were in a scene (see Fig. 7B; cf. Peterson & Berryhill, 2013). Following Peterson and Berryhill (2013), we set the duration of memory array as 200 ms. For the probe, a single color appeared at one of the six locations, such that the participants could not just remember one color of the same value in the With-regularity condition. In 1/2 of overall trials the probe matched the memorized color at that location. All the six color dots were equally to be probed (64 trials each). The whole experiment lasted about 30 min Results & discussion The ANOVA on d 0 (Fig. 8A) revealed that the performance under With-regularity condition was significantly higher than Noregularity condition, F(1,21) = 4.72, p =.041, g p 2 =.183. For the criterion (Fig. 8B), the ANOVA did not reveal a significant main effect of Regularity-condition, F(1,21) = 1.901, p =.182, g p 2 =.083, suggesting that the regularity conditions did not affect participants criterion. 7 Using the similarity cues, we replicated the main findings from Experiments 1 4, suggesting that the revealed regularity benefit was not limited to the Gestalt cues which could form virtual contours. 7 It is of note that we replicated these findings by testing 10 new participants after 6 months: d 0 was significantly higher under with-regularity condition (1.20) than under no-regularity condition (0.86), F(1,9) = 22.16, p =.001, g p 2 =.711. The main effect of regularity was not significant on criterion, F < General discussion The current study examined the underlying principles of VWM when integrating sequentially memorized stimuli. We manipulated the presence of Gestalt cues that were obvious when the single stimulus elements were presented simultaneously. However, when the stimulus elements were presented sequentially, these cues were not apparent until stored within VWM. Unlike previous VWM studies (e.g., Jiang et al., 2004; Peterson & Berryhill, 2013; Shen et al., 2013; Woodman et al., 2003; Xu, 2006), in which the source of the revealed grouping effects was largely attributed to perceptual grouping, in five experiments we demonstrated converging evidence that VWM can detect and use Gestalt cues (e.g., collinearity, closure, and similarity) among the stored contents to improve VWM performance. These findings support a perceptionalike hypothesis in contrast to the encoding-specificity hypothesis. Thus, to form a coherent visual experience of the environment, VWM employs the Gestalt principles to help organize the sampled sequential information into meaningful groups. Forming meaningful groups in VWM based on sequentially presented stimuli is also supported by one recent illusion study from our laboratory (Shen, Zhou, Xu, & Gao, 2015). In that study, we presented the elements of a Ponzo illusion sequentially: we first presented two horizontal lines and a backward mask; then, after a 900-ms blank interval, we presented another two intersected lines and a backward mask; finally, after a blank interval of 1000 ms, the participants had to reproduce a line to match the length of one of the two horizontal lines displayed before. In this setting, the participants could not form a Ponzo illusion stimulus until all the elements were stored in VWM. In line with the current findings, the participants showed significant illusion effects by reproducing a significantly longer bottom line than upper line, a typical result that has been revealed in visual perception (e.g., Leibowitz & Judisch, 1967). Therefore, we argue that it is safe to claim that meaningful groups can be formed in VWM with the help of Gestalt cues. The considerably higher performance for the With-regularity than for the No-regularity condition suggests that VWM can realize and use Gestalt cues to compress the stored information in VWM. Researchers have realized a paradox of VWM: the capacity of VWM is limited, only storing 3 4 simple objects at most; however, VWM can handle many on-line processes efficiently (see Orhan, Sims, Jacobs, & Knill, 2014 for a review). How does our brain accomplish this achievement? One of the potential solutions is to adopt an effective algorithm to store the information. Brady and colleagues showed the first elegant evidence that VWM can use the regularity among simultaneously presented colors (pairs of colors were regularly associated with each other across trials) to compress the

10 286 Z. Gao et al. / Cognition 146 (2015) Fig. 8. The results of d 0 (A), criterion (B) in Experiment 5. Error bars stands for S.E. memorized information such that more external information could be retained in VWM (Brady, Konkle, & Alvarez, 2009). Using a sequential presentation manner, the current study suggests novel manners to compress the stored information in VWM, and offers clear evidence that VWM has the ability to realize potential regularities even after the stimuli have been stored in VWM. However, the precise mechanisms by which VWM detects and utilizes these regularities need to be explored in future studies. The current study provides further evidence for the following three aspects of VWM mechanisms. First, Jiang and Kumar (2004) demonstrated the two sequentially-presented dot-arrays were integrated as one unit at the perceptual/encoding stage if the blank interval between them was shorter than 500 ms; however, they would be stored in separate representations if the blank interval were 500 ms or even longer. In the current study, a 500 ms (Experiments 1, 3, 4, and 5) or 600 ms (Experiment 2) blank interval with a 100 ms mask was adopted between memory array 1 and 2. Therefore, the current settings did not enable a temporal integration of the two memory arrays at the perceptual/encoding stage. However, we consistently demonstrated a regularity benefit. This finding is congruent with previous findings that the information in VWM is not stored in a separate (Zhang & Luck, 2008) but rather a dependent manner (Brady & Tenenbaum, 2013; Jiang et al., 2000). If the information were stored in a separate manner, then the current regularity benefit would not exist. Second, the current study adds new evidence supporting the perception-alike hypothesis. Several studies have consistently demonstrated that visual perception and VWM share many similarities (e.g., Gao, Bentin, & Shen, 2014; Gao, Gao, et al., 2011; Gao, Xu, et al., 2011; Gao et al., 2010; Harrison & Tong, 2009; Mayer et al., 2007; Serences et al., 2009). For instance, the visual cortex in charge of processing orientation is also involved in holding orientation in VWM (Ester, Serences, & Awh, 2009; Harrison & Tong, 2009). All these studies mainly focused on information extraction and storage of VWM. The current study further suggests that for the active information organization, VWM and perception use similar Gestalt principles (at least for the currently tested three ones). Finally, the current study, together with previous studies, suggests that the Gestalt principles operate in both simultaneous and sequential displays in VWM tasks. It is worth noting that the current finding is congruent with the recently proposed probabilistic clustering theory (PCT) of VWM (Orhan & Jacobs, 2013). PCT addresses the organization of information in VWM, particularly for the simultaneous display situation. It claims that VWM represents multiple to-be-remembered items in terms of a probability distribution over all possible clusterings of visual items. Moreover, by adopting standard probabilistic inference (Bayes rule), VWM automatically determines the appropriate clusterings for the tested items, as well as the weights that should be allocated to each cluster. However, as their study mainly focused on the simultaneous display condition, they could not tell whether the cluster was built at the encoding (perceptual) or the maintenance (VWM) phase of the task. The current study shows clear evidence that clusters can be built within VWM, thereby supporting and extending the PCT; moreover, it suggests that the Gestalt principles play a critical role during the formation of the cluster. However, whether the grouping takes place in VWM in an automatic way or if it is a resource-consuming process needs further elaboration. In the current study we demonstrated that grouping takes place within VWM; however, it remains unclear as to whether the previously encoded items still exist in VWM after the group is formed or they are erased from VWM. One possibility is that a two-level hierarchical object is stored in VWM: the top level is the grouped object, and the bottom level contains the single items (Brady et al., 2011; Shen et al., 2013). However, Anderson et al. (2013) demonstrated in an ERP experiment that when the to-beremembered stimuli were presented simultaneously, the VWM stored the two collinear disks as one object without the constituent elements. We are currently conducting ERP studies aimed at elucidating this point. To conclude, in five experiments we presented consistent evidence demonstrating that VWM can detect and adopt potential Gestalt cues among the sequentially presented stimuli. Therefore, the Gestalt principles also operate in VWM to guide information integration in everyday life. Acknowledgements We thank three anonymous reviewers for their comments on this study. This research is supported by the National Natural Science Foundation of China (No , , , and ), the Research Fund of Department of Education of Zhejiang Province (No. Y ), SRF for ROCS, SEM, and Fundamental Research Funds for the Central Universities. Reference Aivio, A. P., & Csapot, K. (1971). Short-term sequential memory for pictures and words. Psychonomic Science, 24(2), Allen, R. J., Baddeley, A. D., & Hitch, G. J. (2014). Evidence for two attentional components in visual working memory. Journal of Experimental Psychology: Learning, Memory, and Cognition. Anderson, D. E., Vogel, E. K., & Awh, E. (2013). Selection and storage of perceptual groups is constrained by a discrete resource in working memory. Journal of Experimental Psychology: Human Perception and Performance, 39(3),