Early information processing contributions to object individuation revealed by perception of illusory figures

Transcription

1 J Neurophysiol 6: 53 5, 6. First published September 7, 6; doi:.5/jn.8.6. Early information processing contributions to object individuation revealed by perception of illusory figures Claire K. Naughtin, Jason B. Mattingley,, and Paul E. Dux School of Psychology, The University of Queensland, St Lucia, Queensland, Australia; and Queensland Brain Institute, The University of Queensland, St Lucia, Queensland, Australia Submitted 9 January 6; accepted in final form 4 September 6 Naughtin CK, Mattingley JB, Dux PE. Early information processing contributions to object individuation revealed by perception of illusory figures. J Neurophysiol 6: 53 5, 6. First published September 7, 6; doi:.5/jn.8.6. To isolate multiple coherent objects from their surrounds, each object must be represented as a stable perceptual entity across both time and space. Recent theoretical and empirical work has proposed that this process of object individuation is a mid-level operation that emerges around 3 ms after stimulus onset. However, this hypothesis is based on paradigms that have potentially obscured earlier effects. Furthermore, no study to date has directly assessed whether object individuation occurs for task-irrelevant objects. In the present study we used electroencephalography (EEG) to measure the time course of individuation, for stimuli both within and outside the focus of attention, to assess the information processing stage at which object individuation arises for both types of objects. We developed a novel paradigm involving items defined by illusory contours, which allowed us to vary the number of to-be-individuated objects while holding the physical elements of the display constant (a design characteristic not present in earlier work). As early as ms after stimulus onset, event-related potentials tracked the number of objects in the attended hemifield, but not those in the unattended hemifield. By contrast, both attended and unattended objects could be individuated at a later stage. Our findings challenge recent conceptualizations of the time course of object individuation and suggest that this process arises earlier for attended than unattended items, implying that voluntary spatial attention influences the time course of this operation. object individuation; selective attention; illusory contours; ERP; P; Npc NEW & NOTEWORTHY We present findings that challenge current conceptualizations of the processing stage at which object individuation arises and expand our understanding of the nature of when individuation emerges for objects both within and outside the focus of attention. We do so using electroencephalography in conjunction with a novel illusory enumeration paradigm that allowed us to manipulate the number of to-be-individuated objects without introducing other extraneous visual confounds present in previous designs. TO ISOLATE OBJECTS in a cluttered visual environment, individuals must first register, or individuate, each item as a distinct perceptual entity on the basis of its spatiotemporal properties. Representations generated at this stage of information processing are coarse and are thought to precede analyses necessary Address for reprint requests and other correspondence: C. K. Naughtin, School of Psychology, McElwain Bldg., The Univ. of Queensland, St Lucia, Queensland 47, Australia ( claire.naughtin@gmail.com). for identification (Kahneman et al. 99; Pylyshyn 994). Functional magnetic resonance imaging (fmri) studies on the neural substrates of object individuation have implicated a distributed network of occipital, parietal, and frontal brain regions (Naughtin et al., in press; Xu 9). Currently, however, there is no clear picture regarding the temporal dynamics of object individuation, or whether this operation is influenced by selective attention (Kahneman et al. 99; Pylyshyn 994; Xu and Chun 9). In the present study we used electroencephalography (EEG) to determine the time point at which object individuation arises for both attended and unattended items. Previous EEG investigations into the temporal dynamics of object individuation have manipulated the number of distinct items using cues such as color (e.g., a variable number of red shapes among green distractors; Mazza and Caramazza ; Mazza et al. 3). These studies have typically shown that the number of individuated stimuli in an attended hemifield modulates a lateralized, negative-going event-related potential (ERP) that begins 3 ms after stimulus onset (the Npc component; Ester et al. ; Revkin et al. 8). Importantly, Npc amplitude increases with the number of targets for sets of four items or less, but not for larger set sizes (Anderson et al. 4; Ester et al. ; Pagano et al. 4). This component therefore appears to track one s ability to rapidly and accurately enumerate a set of distinct objects up until the point where individuation reaches its capacity limit (Piazza et al. ; Trick and Pylyshyn 994; Xu and Chun 9). However, as we describe in detail below, because of some specific characteristics of previous experiments on individuation, we cannot currently infer whether object individuation arises earlier than the Npc or whether its time course is modulated by selective spatial attention. Hyde and Spelke (9, ) have suggested that individuation might arise at time points earlier than the Npc. In their study, participants passively viewed a variable number of dots within a small (,, 3) or large set size (8, 6, 4). The authors observed a negative-going potential that peaked ms after stimulus onset (an N) and that increased with target numerosity for small set sizes but not for large set sizes. Although Hyde and Spelke s findings suggest that object individuation occurs within the first ms after stimulus onset, their manipulations of individuation load also varied with the total number of visual elements. This conclusion was directly challenged by Mazza et al. (3), who presented small sets of red targets among green distractors, which effectively equated the total number of visual elements across numerosities. They found no effect of numerosity for small set sizes at time points /6 Copyright 6 the American Physiological Society 53

2 54 OBJECT INDIVIDUATION earlier than the Npc when targets appeared among distractors (see also Gebuis and Reynvoet ; Libertus et al. 7), suggesting that the N effect may not reflect the exact temporal locus for object individuation. In a recent review, Mazza and Caramazza (5) outlined three key conditions that need to be met to infer evidence of object individuation. First, because it is assumed that multiple objects are indexed simultaneously during individuation (e.g., Pylyshyn ), an EEG marker of individuation must be modulated by the number of concurrently presented objects. Second, because participants can typically only individuate up to four objects at once (Kaufman et al. 949; Mandler and Shebo 98), an EEG marker of individuation should asymptote at three or four objects. Third, because observers can individuate objects even when they are presented among distractors (e.g., Trick and Pylyshyn 993), an EEG marker of individuation should be evident under conditions in which participants must individuate targets presented alone or among distracting objects. On the basis of these assumptions, Mazza and Caramazza (5) proposed that there are early, mid-level, and late processing stages to object individuation and that each stage is indexed by distinct ERP components. A pre-individuation early response is reflected by the N, where a perceptual representation is created for all visual elements (both targets and distractors; see Mazza et al. 3), but processing at this stage is not specific to the task-relevant objects that need to be individuated. Individuation is argued to emerge at the midlevel stage reflected by the Npc, because the response of this component meets all core assumptions: Npc activity is modulated by the number of simultaneously presented objects, and it asymptotes at set sizes of four (Drew and Vogel 8; Mazza and Caramazza ; Pagano et al. 4). Critically, these effects emerge regardless of whether there are also irrelevant elements in the display (Ester et al. ; Mazza and Caramazza ). The post-individuation late response is reflected by the contralateral delay activity (CDA), which is a prolonged, negative-going EEG response that onsets around 3 4 ms after the stimulus and has been associated with the maintenance of representations in working memory (Vogel and Machizawa 4). Although the CDA response mirrors that of the Npc (Mazza and Caramazza ; Pagano et al. 4), thus meeting the requirements for an EEG marker of individuation, Mazza and Caramazza (5) argue that the CDA is likely a by-product of individuation; specifically, the maintenance of individuated items in working memory. A common limitation of all previous electrophysiological studies of object individuation is that they have used displays in which manipulations of individuation load are confounded with other low-level visual properties of the display (e.g., hue, eccentricity, or number of visual elements). Such visual confounds could have concealed any early effects of individuation in these previous studies. For example, early ERPs are likely to be sensitive to these visual factors, and the increased noise introduced by them might have reduced the chances of an effect of individuation being observed. To conclusively determine the temporal locus of object individuation, an ideal paradigm would manipulate the number of simultaneously presented items (among distractors) while controlling for all other physical elements of the display. This was the first goal of the current study. The second goal of our study was to determine the role of attention in individuation. Although individuation was initially thought to emerge preattentively (Pylyshyn 989), it is now generally accepted that this operation draws on selective attention mechanisms (Kahneman et al. 99; Mazza and Caramazza 5; Xu and Chun 9). No study to date, however, has directly assessed whether objects outside the focus of attention are individuated. At the behavioral level, participants are slower to individuate objects if these are presented among distractors rather than in isolation (Mazza et al. 3), suggesting possible interference from the individuated, task-irrelevant objects. Behavioral findings are limited in that they cannot tell us whether or not observers individuated the unattended items, because many processes (in addition to individuation) occur between the stimulus appearing and a response being made. The effects of attention could influence any number of stages of information processing, not just individuation. Fortunately, we can use EEG to identify neural markers of individuation for attended and unattended stimuli with millisecond precision. Selective attention might modulate individuation in an all-or-none manner, whereby only items within the focus of attention are individuated. Such an effect is shown in studies of salience-driven attentional capture, whereby capture by task-irrelevant distractor items (as indicated by the Npc component) is eliminated when they do not match top-down featural or temporal control settings (e.g., Kiss et al. ; Lien et al. 8; Sawaki and Luck ). When the distractors share features with the target object, however, the extent to which distractors are processed is instead reduced relative to targets but is not eliminated entirely (Kiss et al. 3; Matusz and Eimer 3). To address these two goals, we developed a novel enumeration paradigm that controlled stimulus characteristics between conditions and could be used with EEG. Specifically, we manipulated the number of to-be-individuated items at cued and uncued locations within the left and right visual fields (see Fig. ). Each target object was an illusory square produced when a quarter segment was briefly removed from each one of a set of four black-disk inducers (see Fig. A). Up to four illusory squares could appear in each of the cued and uncued hemifields, and the participants task was to report how many illusory targets appeared on the cued side only. Unlike previous paradigms in which displays confounded manipulations of target numerosity with other low-level visual variables, the physical stimuli (i.e., the number of objects, the number of vertical, horizontal, and curved edges, the overall contrast values, and the eccentricities) were identical across all our numerosity conditions. The only aspect that differed across numerosities was the orientation of the individual inducers with respect to one another. Behaviorally, because performance typically declines with increasing individuation load (Ester et al. ; Mazza et al. 3; Pagano et al. 4), we predicted that participants would produce more errors with each additional object they had to enumerate. In addition, we expected reaction times to show a quadratic effect of target set size; specifically, we predicted that reaction times would increase with each additional to-beindividuated object up to the largest set size, at which point reaction times should decrease (i.e., the end effect; Trick and Pylyshyn 993). In the EEG data, we used Mazza and Cara-

3 OBJECT INDIVIDUATION 55 mazza s (5) definitions to identify neural signatures of individuation. We reasoned that EEG markers of individuation should be evident if activity increased with the number of simultaneously presented objects, up to a set size of three or four, even under our display conditions in which targets and distractors were presented concurrently. In addition, we assessed the time course of object individuation for task-relevant and task-irrelevant items to examine the role of selective attention in the temporal dynamics of this operation. If object individuation does not depend on top-down attentional control settings (Pylyshyn 989), we should observe similar time courses for attended and unattended objects. That is, the amplitude of an EEG marker of individuation should increase with the number of illusory objects, regardless of whether those objects appear in the attended or unattended hemifield. Conversely, if individuation draws on selective attention mechanisms (Kahneman et al. 99; Mazza and Caramazza 5; Xu and Chun 9), there should be no EEG markers of individuation for unattended objects (i.e., no effect of nontarget numerosity on ERP amplitude). METHODS AND MATERIALS Participants The study consisted of two experiments. There were 5 participants in experiment (8 women), the aim of which was to validate the behavioral protocol, and a further 6 in experiment (5 women), which employed EEG to measure the time course of object individuation processes. The mean age of participants in the two experiments was.8 yr (SD 5. yr) and 5.5 yr (SD 4.4 yr), respectively. Data from participants with poor task accuracy (3 participants in experiment, n ; participant in experiment, n 5) were excluded. We excluded two participants from experiment because their performance was close to chance ( 7% errors, where chance was 75% errors) and another participant who made % errors on zero-target trials. The excluded participant from experiment made 96% errors in 4-item trials, which left an insufficient number of correct trials for the ERP analysis. All participants had normal or corrected-to-normal vision, gave written, informed consent, and were financially reimbursed for their time ( h for experiment and h for experiment ). The University of Queensland Human Research Ethics Committee approved the experimental protocol. Design and Stimuli Placeholder and inducer stimuli consisted of solid black disks (radius.5 of visual angle) superimposed on a light gray background. Inducer disks also contained a gap in one of four quadrants. These stimuli were presented in groups of four in a square configuration, with a center-to-center distance of.5 between each disk. Four such configurations were presented in each hemifield (left and right), and the center of each was 6. from fixation. In illusory contour configurations, all four inducers had their gap oriented toward the center of the configuration. This arrangement created an additional illusory square that was apparent on top of the four black disks (Kanizsa 955). These illusory squares served as the objects that participants had to detect. There were also nonillusory configurations in which the inducers were oriented randomly such that the gap could appear in any one of the quadrants, with the only constraint being that no two adjacent inducers ever formed an illusory edge. On any given trial, either side of the array could contain,,, 3, or 4 new objects (illusory squares), depending on the configuration of the black inducer disks. All displays were physically identical in terms of the total number of inducer elements, the number of vertical, horizontal, and curved edges, overall contrast values, and eccentricities. We only varied the orientation of the inducers relative to one another to alter the number of (illusory) objects. We also manipulated covert spatial attention on each trial by cueing participants to one of the two visual hemifields (left or right). In each experiment, there were three within-participant factors: target numerosity (,,, 3, or 4 items), nontarget numerosity (,,, 3, or 4 items), and cued hemifield (left or right). Target numerosity was the number of illusory squares in the cued hemifield, and nontarget numerosity was the number of illusory squares in the uncued hemifield. The cued hemifield was indicated by an arrow cue ( or ) presented in the center of the screen ( ). Stimuli were generated in Photoshop, and the experiments were programmed in MATLAB using the Psychophysics toolbox (Brainard 997; Pelli 997). Experiment Procedure. The purpose of experiment was to first validate our novel enumeration task for manipulating individuation load (Fig. A). The goal was to employ a paradigm that was similar to standard enumeration tasks in which participants have to individuate a variable number of target items (Ester et al. ; e.g., Mazza et al. 3; Pagano and Mazza ), while low-level visual features, such as the number of visual inducers, the number of vertical, horizontal, and curved edges, hue, contrast, and eccentricity were controlled to ensure the visual displays were physically identical across all conditions. The only factor differentiating between displays across conditions was whether the inducers in each set of four were oriented to form an illusory square. Each trial began with a fixation square (5 ms), followed by the presentation of four placeholder configurations in the left and right hemifields (5 ms), each consisting of four black disks. An arrow cue then appeared in the center of the screen to indicate the visual hemifield participants should attend to. After 5 ms, each group of placeholder disks then changed to inducer disks by removal of a quarter segment from each. Each configuration within the cued and uncued hemifield formed an illusory square object or a nonillusory configuration (i.e., no illusory square). This target display remained on the screen for 3 ms, and participants task was to determine the number of illusory squares that appeared within the cued hemifield. A response screen appeared after offset of the target display, and participants indicated the number of targets via key press. We emphasized response accuracy over speed, and no response deadline was imposed. On any given trial, there could be zero, one, two, three, or four targets in the cued hemifield and the same possible number of ignored nontargets in the uncued hemifield. The next trial began after a 5-ms interval. Participants completed practice trials, followed by 6 test trials (split across 6 blocks). Response feedback was provided on practice trials only. Trial types were presented in a pseudorandomized order, and the selection of target/nontarget configurations was randomized across trials. Participants were tested in a dimly lit laboratory to minimize distraction from other visual stimuli. Experiment Procedure. After validating our paradigm in experiment, in experiment we recorded EEG while participants completed the illusory enumeration task. In this experiment the aim was to isolate the time point in information processing at which object individuation arises for attended and unattended items. The enumeration task was identical to that described for experiment, with the following exceptions. Each trial began with the cue display for a variable duration of 4 6 ms (the exact duration was randomly determined for each trial), followed by the target display for 3 ms. Cue display duration was jittered to minimize the extent to which any cue-related

4 56 OBJECT INDIVIDUATION A How many squares? Fig.. Paradigm and behavioral results for experiment. A: schematic representation of a single trial of the illusory enumeration task. Each trial began with the presentation of 4 placeholder groups (black disks) in each hemifield, followed by a central arrow cue. The placeholders then changed into inducers (disks with quarter segments removed), where some of the inducer groups were arranged to form an additional illusory square (targets). When prompted, participants reported the number of targets in the attended hemifield. B: error rates as a function of the target and nontarget numerosity conditions. Error bars represent SE. C: reaction times (in ms) as a function of the target and nontarget numerosity conditions. Error bars represent SE. B Errors (%) ms (fixation) 5 ms (placeholders) Target Numerosity: 5 ms (arrow cue) C 3 ms (sample display) Reaction Time (ms) Until response (response) Non Target Numerosity 45 Non Target Numerosity activity would be present in the ERPs in response to the target display. A response prompt then appeared for,7 ms, during which participants had to indicate the number of targets they detected in the cued hemifield. We imposed a response time limit to be consistent with previous ERP investigations into object individuation (e.g., Ester et al. ; Mazza et al. 3; Pagano and Mazza ). To ensure that participants responses were uniformly distributed over the keyboard, we also assigned arbitrary response keys for each target numerosity ( D for target, F for targets, J for 3 targets, K for 4 targets, and spacebar for targets). Participants responded with the following fingers: left middle, D; left index, F; right index, J; right middle, K; right thumb, spacebar. After the response period, there was a delay of,5 ms before the next trial began. Participants completed 8 test trials (split over 8 blocks) and 5 practice trials, and were fitted with a 64-electrode head cap during the practice block. We instructed participants to minimize eye, head, and body movements during the experiment and to take breaks in between each of the testing blocks. EEG recording. Continuous EEG data were acquired using a BioSemi ActiveTwo system (BioSemi, Amsterdam, The Netherlands), digitized at a,4-hz sampling rate with 4-bit analog-todigital conversion. We recorded from 64 active scalp electrodes mounted on a nylon cap, and these electrodes were arranged according to the International Standard - system. Electrodes were referenced online to the standard BioSemi reference electrodes. We also recorded eye movements from bipolar horizontal electrooculogram (EOG) using electrodes placed to the outer canthi of each eye, and from the bipolar vertical EOG using electrodes placed above and below the left eye. EEG analysis. EEG data were analyzed offline using Brain Electrical Source Analysis (BESA 5.3; MEGIS Software, Gräfeling, Germany). Each scalp electrode was referenced offline to the average of all 64 scalp electrodes and subjected to a.-hz low-pass and a 45-Hz high-pass digital filter. EOG electrodes were referenced offline into bipolar vertical and horizontal EOG channels. Any noisy scalp electrodes identified via visual inspection were replaced by a spherical spline interpolation of voltages recorded at all other scalp electrodes (the maximum number of interpolated electrodes for any given participant was 3). The data were segmented into epochs from ms before to 4 ms after the target display onset, and the average voltage recorded ms before stimulus onset served as the baseline measurement. Incorrect trials were removed from the ERP analyses (8.% of trials) because of the unknown source of the error on these trials. We also identified and removed any trials with blinks, eye movements, or other movement artifacts (a further.8% of trials). The remaining epochs were averaged together, separately for each participant, target numerosity, and nontarget numerosity condition (,,, 3, or 4 objects). Even though we jittered the duration of the arrow cue over a -ms range to cancel out any cue-related activity in the target display ERPs, some residual overlap from the cue ERPs might still have remained. To ensure there was no remaining cue-related activity in the target ERPs, we subtracted the ERP evoked by zeroitem trials (i.e., those in which no illusory squares were present) from all other remaining numerosities (Busse and Woldorff 3). The zero-item trials were identical to the other trial types in terms of their physical stimuli elements (as specified earlier), trial structure, task demands, and frequency, except that the inducers in the cued and uncued hemifields were oriented so that they did not form any illusory squares. This subtraction method effectively removed any activity in the continuous EEG data that were not time-locked to the target display (Talsma and Woldorff 5). This procedure was undertaken separately for attended and unattended objects. Specifically, activity from zero-target trials was subtracted from activity associated each of the other target numerosities, and vice versa for the nontarget numerosities. We analyzed activity for targets and nontargets separately to ensure there were sufficient trial numbers within each cell of the analysis. We analyzed data for two components which have previously been associated with early and mid-level processing stages of object individuation, the N and the Npc (see Fig. 3; Mazza and Caramazza 5). In addition, to assess for evidence of object individuation at even earlier stages of processing, we also analyzed the P component, a positive visually evoked potential that peaks around ms after stimulus onset and is argued to reflect initial perceptual processing in extrastriate cortex (Heinze et al. 994; Hillyard et al. 998). The P amplitude is typically greater for attended than unattended stimuli

5 OBJECT INDIVIDUATION 57 (Luck and Hillyard 994) and is more responsive to targets than distractors in tracking scenarios that require the simultaneous processing of multiple moving objects (Drew et al. 9; Stormer et al. 3). The electrodes and time windows defined for each ERP component were based on previous studies that have used comparable paradigms. The early nonlateralized components were P ( 5 ms, PO7/ PO8, PO3/PO4, CP/CP, C/C; Stormer et al. 3) and N(39 9 ms, PO7/PO8, PO3/PO4, O/O; Gebuis and Reynvoet ; Hyde and Spelke 9). The lateralized Npc component was defined as the difference in activity 9 3 ms after stimulus onset between ipsilateral posterior sites (PO7/PO3/O for left targets/nontargets, PO8/PO4/O for right targets/nontargets) and contralateral sites (PO8/ PO4/O for left targets/nontargets, PO7/PO3/O for right targets/ nontargets; Ester et al. ; Mazza et al. 3). The mean amplitudes and peak latencies, collapsed across the corresponding electrodes for each component, were then subjected to separate one-way repeatedmeasures analyses of variance (ANOVAs) for the factors of target numerosity (,, 3, 4; collapsed across all nontarget conditions) and nontarget numerosity (,, 3, 4; collapsed across all target conditions). For all behavioral and ERP analyses, the Greenhouse-Geisser correction was applied for violations of the sphericity assumption. RESULTS Experiment To confirm that there was a linear effect of the number of to-be-individuated targets on behavioral performance, we ran a target numerosity (,,, 3, 4) by nontarget numerosity (,,, 3, 4) repeated-measures ANOVA on the mean proportion of errors and mean reaction times. As expected, we found a significant main effect of target numerosity on error rates, F(, 8) 5.6, P.4, p.9, including a significant positive linear pattern such that errors increased with larger target numerosities, F(, ) 6.7, P., p. (Fig. B). There was no significant effect of nontarget numerosity on error rates, and no significant target by nontarget numerosity interaction, F values.9, P values.37, p values.5. A similar pattern of results was seen for reaction times (see Fig. C). There was a significant main effect of target numerosity, F(, 4) 4.67, P., p., with a marginal quadratic pattern, such that reaction times increased up to the largest target numerosity, at which point they decreased, F(, 8) 3.7, P.7, p.7. No other main effects or interactions were significant, F values.35, P values.6, p values.7. These results confirm that the illusory squares produced similar behavioral performance to that observed in previous studies (e.g., Ester et al. ; Mazza et al. 3; Pagano and Mazza ) and demonstrate that our novel paradigm was effective in manipulating individuation load. Experiment Behavioral performance. Consistent with experiment, a repeated-measures ANOVA on the mean proportion errors with factors of target vs. nontarget numerosity (levels of,,, 3, and 4 objects for each) revealed a significant main effect of target numerosity, F(, 47) 8.77, P., p.7, including a significant positive linear pattern, F(, 4).4, P., p.3 (see Fig. A). There was also a significant target by nontarget numerosity interaction, F(6, 364).85, P.4, p.7. We followed up this interaction with separate one-way target numerosity ANOVAs conducted on data at each level of nontarget numerosity but still found a significant effect of target numerosity for all nontarget numerosity ANOVAs, F values 3.87, P values.6, p values.4, including a significant positive linear pattern, F values 5.5, P values.8, p values.9. This finding suggested that although error rates might have differed slightly across some target non-target cells, the overall linear effect of target load still emerged regardless of the number of nontarget items. The reaction time results showed a comparable pattern to the accuracy data (see Fig. B). There was a significant main effect of target numerosity, F(3, 75) 33.7, P., p.58, with a significant quadratic pattern (the end effect ), F(, 4) 76.7, P., p.76. There was also a significant main effect of nontarget numerosity, F(4, 96) 3.3, P., p., with a significant linear pattern, F(, 4) 8.6, P.7, p.6, and a significant target by nontarget numerosity interaction, F(6, 384).73, P.39, p.7. Follow-up one-way target numerosity ANOVAs at each level of nontarget numerosity, as per the accuracy analysis, produced a significant main effect, F(4, 96) 9.8, P., p.45, and significant quadratic pattern, F(, 4) 3.78, P., p.57. This result replicates the behavioral performance observed in experiment, again demonstrating a linear decline in performance with each additional object to be individuated. Although experiment also yielded reliable effects of nontarget numerosity and a significant interaction for accuracy and reaction times, the A 4 Target Numerosity: B 75 Errors (%) 3 Reaction Time (ms) Fig.. Behavioral results from experiment. A: error rates as a function of the target and nontarget numerosity conditions. Error bars represent SE. B: reaction times (in ms) as a function of the target and nontarget numerosity conditions. Error bars represent SE. Non Target Numerosity 45 Non Target Numerosity

6 58 OBJECT INDIVIDUATION effect of target numerosity was maintained under all nontarget numerosity conditions. ERP results. We first compared P, N, and Npc peaks across the target and nontarget numerosity conditions. A oneway repeated-measures ANOVA with the factor of target numerosity (,, 3, 4) on mean P amplitudes revealed a significant main effect, F(, 5) 3.69, P.8, p.3, including a significant quadratic pattern, F(, 4) 3.89, P., p.37 (see Fig. 3). There was no significant main effect when an identical ANOVA was applied to nontarget numerosity mean P amplitudes, F (see Fig. 4). In addition, similar effects were observed for P peak latency measures, with a significant effect for target numerosity, F(3, 7) 3.34, P.4, p., but no significant quadratic pattern and no significant effect of non-target numerosity, F As usefully pointed out by a reviewer, the nonillusory square configurations could still potentially include two diagonally opposite inducers that were oriented to face each other, forming two corners of an illusory square. These half illusory objects might have been individuated as partial objects and interfered with performance. To assess whether this was the case, we compared error rates between trials in which there was a half object at any of the distractor locations in the cued or uncued hemifield and trials in which there were no half objects present at all. There was no difference in error rates between these trials, t, suggesting that the presence of these half objects did not impact participants performance. values. Visual inspection of the grand-averaged waveforms for nontarget numerosities, however, suggests that the P time window used by Stormer et al. (3) was somewhat later than the peak in our data (see Fig. 4A, top). To ensure we did not miss an early effect of individuation for unattended objects, we also analyzed activity between 6 and 85 ms after stimulus onset. Once again, there was no significant effect of nontarget numerosity on the amplitude or latency at this earlier P time window, F values.46, P values.33, p values.6. Together, these results suggest that only the number of illusory squares to be individuated in the attended hemifield modulated activity 5 ms after stimulus onset, thus providing the first evidence for object individuation at this early stage of visual information processing. The pattern shown for P amplitudes was also observed for the N component. A one-way repeated-measures ANOVA on mean N amplitude for target numerosity (,, 3, 4) revealed a significant main effect, F(, 54) 4.7, P., p.37, with a significant positive linear pattern, F(, 4) 3.96, P., p.5 (see Fig. 3). Although a one-way repeated measures ANOVA for nontarget numerosity yielded a marginal main effect, F(3, 7).35, P.79, p.9, there was no significant linear or quadratic pattern, F values.49, P values.8, p values.9. There were no Fig. 3. Event-related potentials (ERPs), shown separately for P, N, and Npc, as a function of target numerosity conditions in experiment. A: grand-averaged ERP waveforms obtained 4 ms after stimulus onset across a selection of posterior central, occipital, and parietal electrodes: P (PO7/PO8, PO3/PO4, CP/CP, C/C), N (PO7/PO8, PO3/PO4, O/O), and Npc (lateralized difference wave between PO7/PO8, PO3/PO4, O/O). These ERP waveforms were normalized relative to zerotarget trials to remove any cue-related activity. B: spline-interpolated isocontour voltage topographies at the time corresponding to each peak used in the ERP analysis: P ( 5 ms), N (39 9 ms), and Npc (9 3 ms). These topography maps are averaged across all numerosity conditions, separately for targets appearing within the left and right hemifield. We also averaged across hemifields for the ERP analysis. Black and white circles indicate which electrodes were used in each peak analysis. C: ERP peaks as a function of the 4 target numerosities (,, 3, 4). Error bars indicate SE. A B C P N Npc Time (ms) - μv Numerosity

7 OBJECT INDIVIDUATION 59 A B C P N Npc Fig. 4. Event-related potential (ERPs), shown separately for P, N, and Npc, as a function of nontarget numerosity conditions in experiment. A: grand-averaged ERP waveforms obtained 4 ms after stimulus onset across a selection of posterior central, occipital, and parietal electrodes: P (PO7/PO8, PO3/PO4, CP/CP, C/C), N (PO7/PO8, PO3/PO4, O/O), and Npc (lateralized difference wave between PO7/PO8, PO3/PO4, O/O). These ERP waveforms were normalized relative to zero-nontarget trials to remove any cue-related activity. B: spline-interpolated isocontour voltage topographies at the time corresponding to each peak used in the ERP analysis: P (6 85 ms is displayed, but 5 ms was also analyzed), N ( 8 ms is displayed, but 39 9 ms was also analyzed), and Npc (9 3 ms). These topography maps are averaged across all numerosity conditions, separately for nontargets appearing within the left and right hemifield. We also averaged across hemifields for the ERP analysis. Black and white circles indicate which electrodes were used in each peak analysis. C: ERP peaks as a function of the 4 nontarget numerosities (,, 3, 4). Error bars indicate SE. 3 4 Time (ms) - μv Numerosity significant N latency effects either, F values.79, P values.58, p values.7. As with the P analysis of nontarget numerosities, visual inspection of the grand-averaged waveforms indicated that a slightly earlier time window might better capture the N component in our data. We therefore ran an additional analysis with the N time window used by Mazza et al. (3), 8 ms after stimulus onset, but found no significant main effect of nontarget numerosity for amplitude or latency, F values.38, P values.57, p values.5 (see Fig. 4). We thus found no evidence for individuation of unattended objects in the N, despite strong attentional modulation of attended objects at this early time point. To assess for later modulations of brain activity, which arguably reflect the temporal locus of individuation (Mazza and Caramazza 5), we also conducted separate one-way repeated-measures ANOVAs on mean Npc amplitudes for target numerosity (,, 3, 4) and nontarget numerosity (,, 3, 4). As was observed for the earlier nonlateralized components, there was a significant main effect of target numerosity, F(3, 7) 3.34, P.4, p., including a significant positive linear pattern, F(, 4) 6.8, P.5, p. (see Fig. 3). There was also a significant main effect of nontarget numerosity, F(, 55) 6.9, P., p., as well as a significant negative linear pattern, F(, 4).6, P., p.34 (see Fig. 4). The effect of target numerosity on the Npc is consistent with prior investigations into object individuation (e.g., Mazza et al. 7, 3; Pagano et al. 4), but we provide the first evidence comparing this with ERP markers of task-irrelevant object individuation. It appears that objects in unattended locations are individuated, but the extent of processing declines with each additional object in the display (as reflected by a linear decrease in Npc negativity with nontarget numerosity). We elaborate on this finding in the DISCUSSION. The analyses presented thus far included trials in which a correct response was made. It might be argued that ERPs observed under the larger set size conditions could be noisier than those observed with smaller set sizes, because participants made more errors with larger set sizes (thus yielding fewer correct trials for analysis than with smaller set sizes). To assess whether this was the case, we also analyzed the ERP data using both correct and incorrect trials. The same pattern of results was observed when all trials were included: the P and N components were modulated by target numerosity, F values 6.5, P values., p values. (with a significant linear pattern, F values 7.44, P values.), but not nontarget numerosity, F values.35, P values.8, p

8 5 OBJECT INDIVIDUATION values.9. There was also a significant effect of both target and nontarget numerosities for the Npc component, F values 4.55, P values.6, p values.6. These findings suggest that the results from the analyses of correcttrial data cannot be attributed to noisier ERPs for larger vs. smaller set sizes. Furthermore, any null results are likely not due to insufficient power, because they did not benefit from this substantial increase in trial numbers. To further assess whether unbalanced trial numbers across conditions had an impact on our results, we balanced the number of trials in each condition. This was done by randomly selecting trials from the easier conditions (which yielded more correct trials) to match the number of trials in the most difficult conditions (which yielded the fewest correct trials). Analyses using these trial-balanced conditions replicated our original results: there was a significant effect of target numerosity on ERP amplitude at the P [quadratic effect, F(, 4).9, MSE, P.4, p.3], N [linear effect, F(, 4) 6.33, MSE, P., p.4], and Npc [linear effect, F(, 4) 6.69, MSE, P.6, p.]. On the other hand, the effect of nontarget numerosity was only evident at the level of the Npc [linear effect, F(, 4) 4.89, MSE, P.37, p.7]. All other effects were nonsignificant (P values.399). These findings corroborate the original analyses that included all (correct and incorrect) trials and suggest that these effects (with balanced, correct-trial conditions) were not driven by nosier ERPs at larger set sizes. DISCUSSION In the present study we examined the temporal dynamics of individuation for attended and unattended objects. We employed a novel enumeration paradigm in which inducers (black disks with quarter segments removed) could form illusory squares. Critically, this design kept visual elements constant across numerosity conditions ( to 4 illusory squares), which allowed us to explore early cortical contributions to object individuation in the absence of other confounding physical variables (e.g., changes in hue or luminance as a function of numerosity). We observed that the number of attended targets modulated EEG activity as early as ms after stimulus onset. Critically, this pattern of results could not simply reflect a pop-out effect, whereby attention was automatically drawn to the illusory figure, because the number of targets (rather than the absolute presence or absence of any illusory figures) modulated the amplitude of the ERPs. It might be argued that the effect of target numerosity we observed for P amplitude reflects some sort of motor preparatory process, because a similar pattern of results was seen in the reaction time data. If this were the case, we should have seen the same quadratic pattern in the P latencies and in response to nontarget numerosities, but neither effect was observed. Furthermore, the P component likely corresponds to a processing stage that precedes any motor preparation operations. Indeed, such processes are typically reflected by later, slow-wave ERP components (e.g., Poljac and Yeung 4). The results are also unlikely to reflect a deliberate counting strategy for two reasons. First, the target displays in our paradigm were effectively masked, so there was no iconic memory trace that could have been used after target offset (Spencer 969). Second, the numerosity range we employed ( 4 objects) was within that typically associated with subitizing rather than counting (e.g., Trick and Pylyshyn 994). Instead, contrary to Mazza and Caramazza s (5) recent proposal, we find that object individuation for task-relevant items occurs early in the visual information processing hierarchy (Johannes et al. 995). This finding is in line with recent work showing that numerosity information can be rapidly extracted as early as 75 ms after stimulus onset (Park et al. 6). Our findings show, for the first time, that individuation of attended objects emerges earlier in the visual processing stream than the mid-level stage reflected by the Npc. Contrary to previous studies that failed to find effects of task-relevant numerosity on the P (Hyde and Spelke 9) or N (Mazza et al. 3) under conditions in which targets and distractors appeared together, we found reliable modulations for both components. In our paradigm, targets under all numerosity conditions had to be individuated from distractors that were present on the attended and/or unattended side of the display. Previous claims for early encoding of numerosity (Hyde and Spelke 9, ) have been discounted on the basis of the assumption that they arose from a confound in the number of target and distractor items in the display (see Mazza et al. 3). In the present study, we tested for an early neural signature of object individuation using a novel paradigm in which physical properties were balanced across target conditions, and in which objects were defined solely by the orientations of the inducing elements. We can thus be confident that the number of physical items (or other low-level visual factors) in the display did not drive the set-size modulation we observed for our early ERP components. We also found evidence that unattended, task-irrelevant items are individuated, but that this operation has a later temporal signature than that for attended objects. Specifically, whereas the number of unattended nontargets modulated the Npc component, this factor did not affect the P or N. The direction of Npc effect differed for targets and nontargets, however, whereby Npc amplitude decreased with nontarget numerosity, whereas it increased with target numerosity. The latter result is consistent with previous ERP studies of individuation that have also found a positive relationship between Npc amplitude and attended set size (e.g., Ester et al. ; Mazza et al. 3) and further supports the idea that the Npc reflects the process of object individuation (Mazza and Caramazza 5). In addition, we found that distractor objects are indeed individuated to some extent, but not at the same time point or in the same manner as attended targets. This suggests a role for attention in the time course of operations involved in the registration of stimuli as distinct perceptual objects. Although previous theoretical and empirical studies have suggested that individuation draws on selective attention, we show that attention works to modulate the extent or the timing of individuation, rather than prevent task-irrelevant individuation altogether (for similar effects of attention, see Kiss et al. 3; Matusz and Eimer 3). Although our study was not designed to assess interactions between the individuation of targets and distractors, the differing Npc patterns observed for each might indicate that distractor individuation depends on the number of targets to be individuated concurrently. This interpretation is consistent with this operation being capacity limited (Piazza et al. ; Trick and Pylyshyn 994; Xu and Chun 9). A recent fmri study by Jeong and Xu (3) found evidence of distractor individuation under low, but not high, target encoding loads in a visual short-term memory paradigm, suggesting that taskirrelevant items are only individuated when task-relevant indi-

9 OBJECT INDIVIDUATION 5 viduation is not demanding and thus does not use all available perceptual resources (Lavie 5). Future research could explore whether Npc amplitude shows a similar interaction between task relevance and encoding load. More broadly speaking, our findings have implications for research into the relationship between spatial and object-based attention. Previous work has suggested that these two forms of attention interact such that participants are better at detecting targets at an uncued spatial location when attention is cued to another location on that same object, relative to when it is cued to a completely different object (Egly et al. 994; Martinez et al. 6). This behavioral effect is complemented by an enhanced N response for uncued spatial locations on the attended vs. unattended object, even when the objects are formed by illusory contours (Gogler et al. 6 Martinez et al. 7a, 7b). In the present study we observed similar interactions between spatial attention and individuation. Specifically, individuation was evident earlier in the visual processing stream for objects that appeared at attended locations, relative to those that appeared at unattended locations. An interesting avenue for future research would be to explore whether individuation of a specific part of an object is impacted by the deployment of object-based, rather than spatial, attention. In conclusion, we have shown that object individuation of attended items arises ms after stimulus onset, suggesting that this operation occurs earlier in the visual information processing hierarchy than previously thought. We also found that unattended items are individuated at a later time point following stimulus onset, suggesting that selective attention modulates the temporal dynamics of object individuation and the extent to which task-irrelevant objects are individuated. Together, these findings challenge current conceptualizations of the processing stage at which object individuation arises and expand our understanding of the nature of when individuation emerges for objects both within and outside the focus of attention. ACKNOWLEDGMENTS We thank Oscar Jacoby and Amy Taylor, who assisted with data collection. GRANTS This research was supported by Australian Research Council (ARC) Discovery Grants DP95 (to P. E. Dux), DP466 (to P. E. Dux), and DP95 (J. B. Mattingley), ARC-Special Research Initiatives Science of Learning Research Centre Grant SR35, and ARC Centre of Excellence for Integrative Brain Function Grant CE47. P. E. Dux was supported by ARC Future Fellowship FT33, and J. B. Mattingley by ARC Australian Laureate Fellowship FL3. DISCLOSURES No conflicts of interest, financial or otherwise, are declared by the authors. AUTHOR CONTRIBUTIONS C.K.N., J.B.M., and P.E.D. conception and design of research; C.K.N. performed experiments; C.K.N. analyzed data; C.K.N., J.B.M., and P.E.D. interpreted results of experiments; C.K.N. prepared figures; C.K.N., J.B.M., and P.E.D. drafted manuscript; C.K.N., J.B.M., and P.E.D. edited and revised manuscript; C.K.N., J.B.M., and P.E.D. approved final version of manuscript. REFERENCES Anderson DE, Vogel EK, Awh E. A neural measure of item individuation. In: Cognitive Electrophysiology of Attention: Signals of the Mind, edited by Mangun GR. San Diego, CA, Academic, 4, p Brainard DH. The Psychophysics toolbox. Spat Vis : , 997. Busse L, Woldorff MG. The ERP omitted stimulus response to no-stim events and its implications for fast-rate event-related fmri designs. Neuroimage 8: , 3. Drew T, McCollough AW, Horowitz TS, Vogel EK. Attentional enhancement during multiple-object tracking. Psychon Bull Rev 6: 4 47, 9. Drew T, Vogel EK. Neural measures of individual differences in selecting and tracking multiple moving objects. J Neurosci 8: , 8. Egly R, Driver J, Rafal RD. Shifting visual attention between objects and locations: evidence from normal and parietal lesion subjects. J Exp Psychol Gen 3: 6 77, 994. Ester EF, Drew T, Klee D, Vogel EK, Awh E. Neural measures reveal a fixed item limit in subitizing. J Neurosci 3: ,. Gebuis T, Reynvoet B. Continuous visual properties explain neural responses to nonsymbolic number. Psychophysiology 49: ,. Gogler N, Finke K, Keller I, Muller HJ, Conci M. Object integration requires attention: visual search for Kanizsa figures in parietal extinction. Neuropsychologia. In press. Heinze HJ, Mangun GR, Burchert W, Hinrichs H, Scholz M, Munte TF, Gos A, Scherg M, Johannes S, Hundeshagen H, Gazzaniga MS, Hillyard SA. Combined spatial and temporal imaging of brain activity during visual selective attention in humans. Nature 37: , 994. Hillyard SA, Vogel EK, Luck SJ. Sensory gain control (amplification) as a mechanism of selective attention: Electrophysiological and neuroimaging evidence. Philos Trans R Soc Lond B Biol Sci 353: 57 7, 998. Hyde DC, Spelke ES. All numbers are not equal: an electrophysiological investigation of small and large number representations. J Cogn Neurosci : 39 53, 9. Hyde DC, Spelke ES. Spatiotemporal dynamics of processing nonsymbolic number: An event-related potential source localization study. Hum Brain Mapp 33: 89 3,. Jeong SK, Xu Y. Neural representation of targets and distractors during object individuation and identification. J Cogn Neurosci 5: 7 6, 3. Johannes S, Münte T, Heinze H, Mangun G. Luminance and spatial attention effects on early visual processing. Cogn Brain Res : 89 5, 995. Kahneman D, Treisman A, Gibbs BJ. The reviewing of object files: object-specific integration of information. Cogn Psychol 4: 75 9, 99. Kanizsa G. Quasiperceptual margins in homogeneously stimulated fields. In: The Perception of Illusory Contours, edited by Petry S and Meyer GE. New York: Springer Science, 955, p Kaufman EL, Lord MW, Reese TW, Volkmann J. The discrimination of visual number. Am J Psychol 6: , 949. Kiss M, Grubert A, Eimer M. Top-down task sets for combined features: behavioral and electrophysiological evidence for two stages in attentional object selection. Atten Percept Psychophys 75: 6 8, 3. Kiss M, Grubert A, Petersen A, Eimer M. Attentional capture by salient distractors during visual search is determined by temporal task demands. J Cogn Neurosci 4: ,. Lavie N. Distracted and confused? Selective attention under load. Trends Cogn Sci 9: 75 8, 5. Libertus ME, Woldorff MG, Brannon EM. Electrophysiological evidence for notation independence in numerical processing. Behav Brain Funct 3: 5, 7. Lien MC, Ruthruff E, Goodin Z, Remington RW. Contingent attentional capture by top-down control settings: converging evidence from eventrelated potentials. J Exp Psychol Hum Percept Perform 34: 59 53, 8. Luck SJ, Hillyard SA. Spatial filtering during visual search: evidence from human electrophysiology. J Exp Psychol Hum Percept Perform : 4, 994. Mandler G, Shebo BJ. Subitizing: an analysis of its component processes. J Exp Psychol Gen :, 98. Martinez A, Ramanathan DS, Foxe JJ, Javitt DC, Hillyard SA. The role of spatial attention in the selection of real and illusory objects. J Neurosci 7: , 7a. Martínez A, Teder-Salejarvi W, Hillyard SA. Spatial attention facilitates selection of illusory objects: evidence from event-related brain potentials. Brain Res 39: 43 5, 7b.