A dissociation between visual and auditory hemi-inattention: Evidence from temporal order judgements

Transcription

1 Neuropsychologia 45 (2007) A dissociation between visual and auditory hemi-inattention: Evidence from temporal order judgements Scott Sinnett a,, Montserrat Juncadella b, Robert Rafal c, Elena Azañón a, Salvador Soto-Faraco d a Parc Científic de Barcelona, Universitat de Barcelona, Spain b Ciutat Sanitaria i Universitaria de Bellvitge, Spain c University of Wales, Bangor, United Kingdom d ICREA & Parc Científic de Barcelona, Universitat de Barcelona, Spain Received 26 August 2005; received in revised form 31 January 2006; accepted 3 March 2006 Available online 9 May 2006 Abstract Patients with right hemisphere brain lesions often suffer from deficits in spatial attention that can be manifested in different sensory modalities. It has recently been claimed that a relationship (i.e., association) could exist between symptoms of hemi-inattention in different modalities, based on correlations between the results of visual and auditory clinical tests of neglect or extinction. However, it should be noted that the visual and auditory tasks varied greatly both in response type and level of sensitivity. Here, we have examined cross-modal associations in spatial attention deficits using a temporal order judgment task (TOJ) in which patients were required to identify which of two visual or auditory objects had appeared first. When compared to age and education matched control participants, the patients needed, on average, the contralesional stimulus to lead the ipsilesional stimulus to achieve the point of subjective simultaneity (PSS). No association between the degree of visual and auditory hemi-inattention was observed amongst the patients, suggesting that there is a certain degree of independence between the mechanisms subserving spatial attention across sensory modalities Elsevier Ltd. All rights reserved. Keywords: Right hemisphere patients; Attention; Crossmodal; Vision; Audition 1. Introduction Neurological patients with lesions affecting the temporoparietal (TPL) region of the right hemisphere often suffer deficits in spatial attention. Typically, these patients can fail to detect, or to be consciously aware of objects, or parts of objects, presented in the contralesional side of space (i.e., the side opposite of the lesion), a syndrome commonly referred to as spatial hemineglect. One manifestation of this neurological syndrome, that can be present in the absence of other symptoms of neglect, is extinction. In extinction, contralesional stimuli presented in isolation can usually be detected; but when presented concurrently with a competing ipsilesional stimulus, the patient often Corresponding author at: Departament de Psicologia Bàsica, Universitat de Barcelona, Pg. Vall d Hebrón, 171, Barcelona, Spain. Tel.: ; fax: address: ssinnett@ub.edu (S. Sinnett). fails to consciously perceive the contralesional target. That is, orienting attention to the ipsilesional stimulus causes the contralesional stimulus, which would normally be perceived, to be extinguished from awareness. This deficit appears to be based on a failure at the level of attentional mechanisms; that is, the deficits seen in spatial processing can occur in the absence of primary sensory difficulties (i.e., hemianopsia), and are modulated with manipulations of attention (Posner, Walker, Friedrich & Rafal, 1984). The attentional and perceptual deficits observed following TPL lesions have been reported in several sensory modalities (i.e., vision, somatosensation, and audition; see Di Pellegrino, Basso, & Frassinetti, 1997; Guerrini, Berlucchi, Bricolo, & Aglioti, 2003; Pavani, Làdavas, & Driver, 2003 for examples), thereby offering researchers an opportunity to investigate the mechanisms of crossmodal attention. Indeed, crossmodal extinction has been observed in some of these patients when a stimulus in one sensory modality (i.e., a visual stimulus on the right) has the effect of extinguishing the perception of a stimulus in a separate modality (i.e., a /$ see front matter 2006 Elsevier Ltd. All rights reserved. doi: /j.neuropsychologia

2 S. Sinnett et al. / Neuropsychologia 45 (2007) touch on the left; see for example, Di Pellegrino, Làdavas, & Farnè, 1997; Làdavas, Pavani & Farnè, 2001; Maravita, Spence, Clarke, Husain & Driver, 2000; Làdavas et al., 2004). The current investigation examines the extent to which symptoms of auditory and visual hemi-inattention are associated in individual patients. This issue has important implications when considering multisensory spatial processing. An association could indicate that the deficits of hemi-inattention affect a general sensory processing system (i.e., multimodal spatial map). That is, if symptoms of visual and auditory hemi-inattention were associated, this would suggest that individual sensory modalities share, to some extent, attentional resources for spatial processing (see Driver & Spence, 2004 for a related example). However, dissociation would suggest a certain degree of independence in the spatial representation of different sensory modalities. This would converge with the idea that different sensory modalities are able to draw from separate reservoirs of attentional resources (see Wickens, 1984; Treisman & Davies, 1973; Duncan, Martens & Ward, 1997; Soto-Faraco & Spence, 2002; Sinnett, Costa & Soto-Faraco, in press), or that spatial attention mechanisms in different sensory modalities can be somewhat decoupled (see Eimer & Driver, 2000; Eimer & Van Velzen, 2002; Soto-Faraco, Morein-Zamir & Kingstone, 2005). The attentional disturbances seen in neglect patients offer an opportunity to further investigate any possible segregation between sensory modalities. A few earlier studies have investigated the levels of association between visual and auditory neglect with mixed results. Bisiach, Cornacchia, Sterzi and Vallar (1984) studied 107 right and left brain lesioned patients (in addition to healthy participants) in an auditory task that required participants to manually point to the perceived direction of a sound delivered via headphones. 1 The patients were divided into four different groups (left versus right hemisphere lesioned patients, and then by the absence or presence of visual defects based on confrontation tasks). The right brain damaged patients showing signs of visual defects (n = 25) had the poorest performance on the sound localization task, misperceiving the origin of the sound further in space ipsilesionally (to the right) than any of the other groups. Of interest, 15 of the patients in this group (right brain damaged patients with visual defects) exhibited visual hemineglect on cancellation tests. The auditory and visual scores of these 15 patients (right brain damaged patients exhibiting visual hemineglect) were analysed in order to determine if performance was correlated in the two modalities. The magnitude of the errors in the auditory task was generally larger when compared with patients not suffering from visual hemineglect, leading the authors to claim that the magnitude 1 It should be noted that the manual pointing task in itself is not the most adequate task to measure amounts of auditory hemi-inattention, since it is possible that visual and motor biases could affect the manual responses. Indeed, it has been reported that healthy, neurobiologically intact participants achieve better auditory localization scores when blindfolded than when being allowed to see (Warren, 1970; Platt & Warren, 1972). This same trend has been seen with neglect patients with improved scores while blindfolded (Soroker, Calamaro, Glicksohn, & Myslobodsky, 1997; Pavani, Farne, & Làdavas, 2003). of visual and auditory neglect was associated. However, it should be noted that the correlation failed to reach statistical significance even when restricting the analysis to the 15 patients who exhibited visual hemineglect based on cancellation tasks. That is, including the other 10 patients, who were originally part of the group of right brain damaged patients showing visual defects, would surely nullify any numerical trend indicating an association between auditory and visual hemi-inattention deficits, as the auditory localization scores for these patients were not biased to the right to the same extent as the 15 patients classified as exhibiting visual hemineglect. Zimmer, Lewald and Karnath (2003) investigated the ability of hemineglect patients to lateralize sounds (i.e., to decide if a sound, presented via headphones, was positioned to the left or right of midline). Their patients were divided into two subgroups, one (n = 7) showing erratic lateralization judgements (i.e., spatial localization errors to the left and right of the original sound source) and the other (n = 8) consistently perceiving the sounds as slightly shifted to the right (i.e., into ipsilesional space). The authors concluded that the inability to localize sound (the former group) was associated with the strength of clinical visual neglect tests (i.e., they performed worse on classic clinical visual neglect tests). That is, the group that consistently perceived the sounds as shifted to the right failed to show an association between their results for visual and auditory testing. If spatial hemi-inattention deficits were based on a breakdown of some kind of multimodal map of space, one would expect the group who consistently misperceived sound location towards the right (a more classical symptom of auditory hemineglect, see Pavani, Farne, & Làdavas, 2003), to exhibit stronger symptoms of clinical neglect (i.e., letter cancellation and copying tests) when compared with the group who exhibited erratic auditory localization judgements. Thus, Zimmer et al. s (2003) results seem more consistent with a dissociation between auditory and visual hemi-inattention symptoms. Bellman, Meuli and Clarke (2001) suggested that two types of auditory neglect might exist based on how a group of patients (n = 4) performed in two different tests measuring deficits of auditory spatial attention. In the first task, pairs of words were presented to the left and right ears (via headphones) and the patients were required to repeat what they heard at both ears. The second task required patients to locate the perceived source of sounds (presented via headphones) occurring at five different locations, one central and four lateral (two in each hemispace). Half of the patients were impaired when recalling the word presented to the left ear (i.e., extinction, test one as described above) while they did not show any spatial bias when localizing sound. The other half showed the reverse, that is, they were able to correctly repeat the words, but sound localization judgements were severely biased towards the right. The authors interpreted the results as evidence for two separate types of auditory hemineglect and explained the dissociation by the location of their respective lesions. The group who suffered from extinction-like symptoms in the word identification task, but was able to localize sound, had lesions affecting the basal ganglia, whereas, the group displaying a spatial bias in localization suffered from fronto-temporo-parietal lesions. These results suggest that the

3 554 S. Sinnett et al. / Neuropsychologia 45 (2007) observed auditory deficits in neglect patients could be a result of a left ear extinction (i.e., an extinction of stimuli presented to the left ear), as observed with the group that was not able to correctly localize sound but was able to repeat words; or, could possibly be rooted in higher level attentional mechanisms, as seen in the group who failed to repeat words, but could localize sound. Directly addressing the question of whether the deficits of hemi-inattention reflect a general problem in spatial processing more or less independent of sensory modalities, or if this attentional deficit can selectively affect different modalities, it has recently been proposed that auditory hemineglect is frequently associated with visual hemineglect (Pavani, Husain, Làdavas, & Driver, 2004; see also Pavani, Farne, & Làdavas et al., 2003; Pavani, Làdavas, & Driver et al., 2003 for a review of auditory neglect). Pavani et al. (2004) performed a meta-analysis of four different studies (Pavani & Làdavas, 1999; Pavani, Meneghello, & Làdavas, 2001; Pavani, Làdavas & Driver, 2002; Pavani, Husain, Malhotra, & Driver, in preparation) using correlation tests between measures of auditory hemineglect and clinical measures of visual neglect. Pavani et al. (2004) concluded that there was a strong correlation between the deficits observed in the auditory and visual modalities, thus arguing for the existence of a general disturbance in multisensory spatial processing possibly accounting for the co-occurrence of visual and auditory hemineglect in their patients. Yet, as pointed out by Pavani et al. (2004), there still remain rarer dissociations between modalities in some patients calling for further discussion (see for example, Bartolomeo & Chokron, 1999, for a related dissociation between visual and tactile neglect symptoms). Moreover, and perhaps most importantly, it would be extremely beneficial to use a standardised task that is comparable at the signal level, the response demands, and the difficulty across modalities. For instance, it is important to point out that the auditory tasks used in the studies that were included in Pavani et al. s meta-analysis were different than the visual tasks both at the level of response type and task sensitivity. Visual tasks were either, for example, a simple cancellation task or a line bisection task, 2 while the auditory tasks involved, for example, the discrimination of vertical sounds (i.e., excluding laterality judgements), determination of whether successive sounds occurred simultaneously or not, or the manual pointing towards the origin of sounds. The importance of using comparable tasks across modalities was also noted by Bisiach et al. (1984), who suggested that the visual and auditory tasks of their study might have different sensitivities, thereby skewing the level of association or dissociation between hemi-inattention symptoms across modalities. It is possible that the more sophisticated auditory tasks usually employed are more likely to accurately measure levels of auditory hemi-inattention than the rather simpler visual cancellation and confrontation tasks, which register only the most blatant and severely affected patients. 2 The cancellation tasks consisted of different combinations of the Bell (Gauthier, Dehaut, & Joannette, 1989), Letter (Diller & Weinberg, 1977), Mesulam (1985) and Star (Wilson, Cockburn, & Halligan, 1987) tasks. Wilson et al. (1987) were also responsible for the line bisection task. In a study investigating the presence of auditory hemineglect in right brain damaged patients, Soroker et al. (1997), tested patients who exhibited, and patients who did not exhibit, visual hemineglect. These authors found that both types of brain damaged patients (i.e., with or without symptoms of hemineglect) extinguished left-sided sound stimuli However, the group exhibiting visual neglect showed an advantage in localization for sounds presented on the right side, but at the same time was inferior, compared to normal individuals, at localizing unilaterally presented sounds to the contralesional side, thereby apparently supporting the notion that symptoms of visual and auditory hemineglect are associated. In the present study we evaluated levels of association between auditory and visual hemi-inattention deficits in seven right hemisphere brain damaged patients using a similar paradigm in both the auditory and visual modalities; the temporal order judgement task (TOJ). This task requires participants to determine which of two different stimuli, presented in succession to left and right hemispace at different stimulus onset asynchronies (SOA s), had been presented first. The use of the TOJ task has been used previously to explore several aspects of spatial inattention deficits in a number of studies. In a case study presented by Di Pellegrino et al. (1997), their patient suffered the greatest amount of extinction (i.e., the failure to report the letter on the left side when presented with two lateralized letters) when the stimuli were simultaneously presented (i.e., 0 ms SOA), but still failed to correctly report the left stimulus even when presented 300 ms before, or after, the right stimulus. The authors explained this result by an imbalance in the amount of attention directed towards the ipsilesional side. That is, ipsilesional stimuli appear to be selected first for processing even if the contralesional stimulus leads by up to 300 ms. Following on the idea that the attentional deficits of these patients may reflect on the temporal aspects of stimulus processing, many researchers have attempted to determine the amount of time contralesional stimuli need to be presented before ipsilesional stimuli for the perception of simultaneity to be achieved. These studies have used the TOJ paradigm as it allows for the calculation of the point of subjective simultaneity (PSS), that is, the average temporal asynchrony needed between two events for the observer to perceive them as occurring simultaneously (see for example, Rorden, Mattingley, Karnath, & Driver, 1997; Spence, Shore, & Klein, 2001). Thus, as measured in extinction patients, shifts in PSS can reflect the degree to which the processing of stimuli presented on the affected side of space (arguably unattended) is inhibited or delayed. In addition to the PSS, the just noticeable difference (JND) can also be calculated from the data collected in a TOJ task. The JND refers to the smallest temporal interval between two stimuli needed for an observer to correctly judge which stimulus had been presented first on 75% of the trials. Thus, JND represents a measure of the precision in temporal discrimination. The measures derived from the TOJ task are commonly used in patients whose spatial hemi-inattention symptoms are not severe, so they can eventually detect the stimulus on the left side if given sufficient time (i.e., extinction syndrome). Several of these studies have demonstrated that, in patients with

4 S. Sinnett et al. / Neuropsychologia 45 (2007) hemi-inattention deficits, visual stimuli appearing on the left side of space need to be presented before stimuli on the right side of space by at least 200 ms for the PSS to be achieved (Rorden et al., 1997; Baylis, Simon, Baylis, & Rorden, 2002; Cate & Behrmann, 2002). Additionally, it has been shown that the deficits of these patients in judging the temporal order of targets are not confined to the visual modality, as Karnath, Zimmer and Lewald (2002) reported similar PSS shifts for the auditory modality, and Guerrini et al. (2003) observed similar findings in the tactile modality. These findings seemingly indicate that an overall disturbance in attention across sensory modalities may be a possibility after TPL lesions, rather than a focalised problem within a specific sensory modality. However, without directly testing whether the symptoms of visual and auditory hemi-inattention observed in these patients are associated, it is difficult to arrive at such a conclusion. We have adapted the traditional TOJ paradigm and applied it to hemi-inattention patients. However, rather than requiring responses tied to the nomenclature of physical space, such as left and right, which have typically been utilized in previous studies (see for example, Baylis et al., 2002; Rorden et al., 1997; Karnath et al., 2002), in the present study patients were required to identify which of the different objects had been presented first, irrespective of location. This modification was used because responses confined to the naming of the side of space that the stimulus had appeared on, for example, either left first (or right) or left-then-right, could introduce severe biases on the data. That is, it is possible that the word right, or the action of pointing to the right, may be favoured by these patients, consequently, they will respond right more often than left, especially when confused or unsure of the correct response. Although the identification task used in our study may not provide a complete control for this potential confound, we believe that our modification can help to alleviate some of the potential output bias that the patients may have. 2. Method 2.1. Patients Seven patients were recruited from Bellvitge Hospital, Barcelona, Spain (n = 5) and from Ysbyty Gwynedd Hospital, Bangor, Wales (n = 2). All had unilateral lesions, due to stroke, involving the right inferior parietal lobule (see Table 1 for summary of other brain regions involved in each patient). Each patient had, at the time of testing, or at some point in the course of their illness, symptoms of spatial hemi-inattention such as extinction or neglect, as measured by standard pencil and paper tasks. All patients were reliably able to perceive unilaterally presented visual stimuli (confrontation tasks). The mean time since stroke was 35 months (range 96 3 months). Their average age was 55 years (range 74 36) and average amount of education was 10.4 years (see Table 1 for a summary of lesion locations for each patient). It is important to note that we did not want to create a dichotomy between groups (i.e., no-hemi-inattention versus hemi-inattention ), as recovery issues could prove problematic to the study. That is, a patient who at one point had exhibited symptoms of hemi-inattention may not exhibit the same symptoms at a later time, thereby potentially placing them in the incorrect testing group. As discussed in more detail later, this is important as the degree of visual or auditory hemi-inattention seen in the TOJ tasks will reflect a higher degree of sensitivity when compared to the traditional visual neglect tasks. The visual and auditory versions of the experiment were also performed by a group of 18 age (51 years, range 80 38) and education (11.2 years) matched control participants Materials and apparatus Stimulus presentation during the experiment was controlled and programmed using DMDX software (created by J. Forster, see u.arixona.edu/ jforster/dmdx.htm). All tests were presented using an Acer laptop computer, with the visual stimuli presented on a 15 LCD screen, and the auditory stimuli presented through headphones (Sennheiser, HD 435). Two geometrical shapes, a black diamond and a black square, were used as visual stimuli. The square measured 50 square pixels, and by rotating the square 45 the diamond was created. For auditory testing, the bark of a dog and the caw of a crow were used as sounds (available at The sound files were manipulated using Cool Edit software (Syntrillium Software Corp.) in order to achieve two sounds of equivalent duration (350 ms) and average amplitude Procedure The TOJ tests were performed in a quiet clinical room at Bellvitge Hospital in Barcelona, and in quiet testing rooms at the University of Wales, Bangor or the Ysbyty Gwynedd Hospital in Bangor. Informed consent was obtained from all participants before testing. All patients were competent to give informed consent. In the visual task, patients sat approximately 60 cm away from the computer screen. At the beginning of each trial, a fixation cross in the middle of the screen was presented for 750 ms. Patients were instructed to focus on the cross and the experimenter ensured compliance with this instruction by frequently reminding the patients throughout the test. Pairs of visual stimuli appeared 500 ms after the fixation cross was removed, at one of the 13 the different SOA s ( 1000, 500, 300, 200, 100, 50, 0, 50, 100, 200, 300, 500, and 1000 ms; negative values indicate left side first). Visual presentation of each stimuli lasted 350 ms (i.e., the same length as the sounds; see Fig. 1A). The distance between the left and right stimulus was 20 cm (i.e., 10 cm from the center of the screen). Table 1 Patient information including age, time since lesion and sex as well as areas affected by the lesion Patient Age/sex Time since lesion (months) Areas affected Clinical deficit: at time of admittance and testing TOJ deficit MA 50/F 14 IPL, F, BG, STG, HG +, + A IG 36/F 40 IPL, STG, HG +, A AS 39/M 96 IPL, STG +, VAE EP 43/M 71 IPL, STG, HG, F, BG +, A, VAE MV 74/M 3 IPL +, + V, AAE TR 73/M 13 IPL, HG, STG +, + V ET 70/F 8 IPL, STG +, + V IPL: inferior parietal lobe; STG: superior temporal gyrus; HG: Heschl s gyrus; BG: basal ganglia; F: lateral frontal cortex; clinical deficits: hemi-inattention (based on pencil and paper): +: deficit present, : deficit absent. TOJ deficit: A auditory, V visual, VAE visual anti-extinction, AAE auditory anti-extinction.

5 556 S. Sinnett et al. / Neuropsychologia 45 (2007) Fig. 1. Schematic representation of the Visual TOJ task (A) and the Auditory (B) TOJ task. The fixation cross was presented for 750 ms in the visual condition. Then, the two visual stimuli would appear, one to each side of the screen, at one of the pre-defined SOAs. Stimuli were presented on the screen for 350 ms and removed. In the Auditory condition a central beep signaled the start of the trial. After the beep there was a period of silence of 1 s before the two auditory stimuli were played, one to each ear, at one of the pre-defined SOAs. Each stimulus lasted 350 ms. In the auditory task a short beep lasting 50 ms, rather than a fixation point, indicated the beginning of each trial. One second after the beep, the stimuli (crow and dog sounds) were played at the selected SOA s (same SOA s as the visual experiment; the sounds lasted 350 ms; see Fig. 1B). After the stimulus presentation, patients were asked to identify which of the two objects (square versus diamond in the visual task, or dog versus crow in the auditory task) had been presented first and verbally tell the experimenter. The experimenter then coded the response in the computer so as to prevent any error by the patients while attempting to correctly press the required key. When the patient was ready, the experimenter initiated the next trial. All patients were informed about the nature of the tasks and performed a minimum of four blocks in each of the sensory modalities for a total of eight left first, and eight right first trials of each SOA, and 16 simultaneously presented trials in both the auditory and visual versions of the experiment. The order and side of presentation of each stimulus (square or diamond; crow or dog) was equiprobable throughout each block and chosen at random. The eight blocks were completed in either one single session, or split between two sessions (depending on patient fatigue). Patients were presented with a number of training trials before the first block of each task (visual or auditory TOJ) in order to acquaint them with the stimuli and the task. The experimenter pseudo-randomly chose the succession of blocks in each sensory modality (i.e., blocks could alternate between modalities, but could also be presented successively). Healthy controls were tested following exactly the same protocol, except that testing always took place in one session and in the participants own home. The experimenter took care to ensure that the experiment was conducted in a quiet room without distractions. 3. Results The percentage of identification scores regarding which stimulus had been presented first was transformed to represent the percentage of left first responses (whether it be the diamond or square in the visual task, or the dog or the crow in the auditory task). These raw proportion scores of left first responses as a function of SOA where transformed to their equivalent Z-scores using probit analysis assuming a normal cumulative distribution (see Finney, 1964). In short, transformed Z-scores are obtained by applying the inverse of the standard normal distribution function to the raw proportion scores. 3 This transformation, commonly used in studies using the TOJ task (i.e., Zampini, Shore & Spence, 2003), allows one to perform a linear regression on the transformed data and, from the slope and intercept of the fitted line, derive the PSS and the JND. The PSS, that is, the SOA point that corresponded to the point in the function at which left and right responses were equally probable (p = 0.5), can be calculated from the slope and intercept of the best-fitted 3 In order to be able to apply the inverse standard normal distribution, raw proportions of 1 and 0 were replaced by 0.99 and 0.01, respectively.

6 S. Sinnett et al. / Neuropsychologia 45 (2007) line of the Z-scores (PSS = slope/intercept). Similarly, the JND can be calculated by dividing the slope by as the ±0.675 point corresponds to the 75% and 25% points on the cumulative normal distribution. That is, the average SOA at which participants were correct on 75% of the trials. These calculations were carried out for each participant (patients and controls) and sensory modality independently. The R 2 values corresponding to the linear regressions of the individual data averaged 0.55 (all p < 0.02) and 0.60 (all p < 0.005) for the patients and the controls (respectively) for the visual data, and 0.63 (all p < 0.01) and 0.62 (all p < 0.02), respectively, for the auditory data Visual temporal order judgements For the visual modality, patients required, on average, the stimulus on the left (whether it be the diamond or square) to be presented 193 ms (S.D. = 471 ms) before the stimulus on the right for PSS to be achieved (see Fig. 2A). Control participants were very accurate, achieving an average PSS of 21.5 ms (S.D. = 54; see Fig. 2B). The JND (minimum time needed to be correct on 75% of the trials) for patients was on average 275 ms (S.D. = 88 ms) and 195 ms (S.D. = 7 ms) for the age matched controls. A t-test indicated that the PSS for the patients shifted towards the left (i.e., the left stimulus had to significantly lead the right stimulus for subjective simultaneity to be achieved) when compared to the controls, but only marginally (t(23) = 1.6, p = 0.06, one-tail). It is likely that the great variability in patient scores explains the lack of significance (see Section 4 below). In terms of JND, patients required more time than controls to be correct on 75% of the trials (t(23) = 4.0, p = 0.001) Auditory temporal order judgements In the auditory modality patients required the left stimulus to be presented an average of 69 ms (S.D. = 91 ms) before the right stimulus (see Fig. 3A) for PSS to be achieved. Control subjects were more accurate, with an average PSS of 3 ms (S.D. = 58 ms, (see Fig. 3B). The auditory JND for the patient group was on average 235 ms (S.D. = 60 ms), and 221 ms (S.D. = 67 ms) for the age matched controls. A t-test showed that the PSS for patients was statistically different than the PSS of the controls (t(23) = 2.2, p = 0.02, one-tailed), again with a considerable shift to the left. Patients and controls did not differ on JND (i.e., the amount of time required to be correct on 75% of the trials; t(23) = 0.5, p = 0.63). Fig. 2. Data from patients (A) and controls (B) in the Visual TOJ task, fitted with logistic curves. Thin gray lines represent individual curves for each participant, while the thick solid black line corresponds to the group average. Note the shift in PSS (the SOA where the curve crosses the 0.5 mark) for the patient group. Fig. 3. Data from patients (A) and controls (B) in the Auditory TOJ task, fitted with logistic curves. Thin gray lines represent individual curves for each participant, while the thick solid black line corresponds to the group average. Note the shift in PSS (the SOA where the curve crosses the 0.5 mark) for the patient group.

7 558 S. Sinnett et al. / Neuropsychologia 45 (2007) Discussion Fig. 4. Auditory (y-axis) and visual (x-axis) PSS scores in the TOJ task for each patient (initials) and control (black dots). The area defined by the dotted vertical lines represents the 99% confidence interval for normal performance in the visual task based on the control data. The area defined by the dotted horizontal lines represents the 99% confidence intervals for the normal performance in the auditory task based on the control data. Note how 5 out of the 7 individual patient scores fall within the confidence interval for one modality while falling outside of the confidence interval of the other modality (dissociation) Correlation between visual and auditory temporal order judgements A correlation was not found between the magnitude of visual PSS and auditory PSS scores (r(7), Pearson correlation = 0.399, p = 0.376) for the patient group. As shown graphically in Fig. 4 there was no association between auditory and visual hemi-inattention as assessed by temporal order judgements. This figure depicts individual patient scores for each task compared to the 99% confidence intervals based on the control data in each modality. The confidence interval based on the PSS scores for the control participants in the visual task ranged from 15 to 58 ms. Three patients fell outside this range with visual PSS values of 249 ms (MV), 174 ms (TR) and 1198 ms (ET), indicating that the contralesional stimulus had to lead the ipsilesional stimulus in order to achieve PSS. It should be noted that although patient ET s PSS is much larger than the other patients, she was able to identify unilaterally presented visual stimuli (i.e., she did not suffer from hemianopia). Two patients (AS and EP) exhibited what could be described as anti-extinction, requiring the ipsilesional stimulus to lead the contralesional stimulus by 106 and 228 ms, respectively (see Goodrich & Ward, 1997; Humphreys, Riddoch, Nys, & Heinke, 2002, for examples of anti-extinction). This observation will be addressed in the discussion. The confidence intervals based on the PSS scores of the control participants in the auditory task ranged from 37 to 43 ms. Three of the patients fell outside this range with auditory PSS values of 140 ms (MA), 158 ms (IG), and 183 ms (EP). One patient (MV) appeared to exhibit auditory anti-extinction requiring the ipsilesional sound stimulus to be lead the contralesional sound stimulus by 63 ms. The present study is the first to address the deficits of temporal processing in hemi-inattention patients using a comparable task across different sensory modalities. This task allowed us to more precisely compare the extent of visual and auditory spatial hemi-inattention deficits in a group of right hemisphere damaged patients. Regarding each sensory modality in isolation, the results of our study are in the same trend as previous findings with right hemisphere lesioned patients in spatial TOJ tasks. In particular, we found that in both the auditory and visual modalities, patients, on average, required the contralesional stimulus to be presented before the ipsilesional stimulus by a greater amount of time than needed by controls for simultaneity to be perceived. Although our results fell short of significance in the visual modality, arguably due to the high variability of the patients results, some individual patients required, on average, that the stimulus on the left lead the stimulus on the right by much more time than needed by the controls (i.e., 193 ms versus 22 ms, visual PSS, and 69 ms versus 3 ms, auditory PSS). The variability in the degree of hemi-inattention for each sensory modality, combined with the small sample of patients, can explain the non-significance of the PSS shift when compared to the controls. Furthermore, this lack of significance is completely in accord with the nature of the dissociation being reported. That is, as some of the patients in the group suffer from visual hemi-inattention, but others do not, it is conceivable that the group average will fall closer to that of the control group (and with a large variance). Of more importance, when looking at the individual patients scores, the PSS values of the patients who show visual hemi-inattention deficits, but do not exhibit signs of auditory hemi-inattention (MV, ET, and TR), are very much beyond the upper limit of the confidence interval based on control performance. Indeed, patient MV required contralesional visual stimuli to be presented in advance of ipsilesional visual stimuli (249 ms), while exhibiting the opposite for the auditory modality (i.e., anti-extinction; the ipsilesional auditory stimulus needed to lead the contralesional stimulus by 63 ms to achieve PSS). The same can be said for those patients who show auditory hemi-inattention without signs of visual hemiinattention (MA, IG and EP). Yet, contrary to what one would expect according to recent proposals that auditory extinction is associated with visual extinction (Pavani et al., 2004), our patients do not exhibit an association. Indeed, a strong dissociation was observed in our group of patients. In fact, only two of the patients did not show a dissociation (AS and MV), with patient AS showing consistent signs of hemi-inattention in the auditory modality, but failing to show signs of hemi-inattention in the visual modality, instead achieving a PSS reflecting visual anti-extinction. Patient MV exhibited similar scores, but in the opposite direction (visual hemi-inattention and auditory antiextinction). The dissociation observed in our patients suggests that the attention mechanisms that are at failure in our right brain lesion patients are somewhat segregated in terms of the sensory modality that they serve. That is, the deficits seen in hemiinattention (i.e., neglect) patients after right hemisphere brain

8 S. Sinnett et al. / Neuropsychologia 45 (2007) damage do not necessarily affect a general multimodal map of spatial representation/processing. While it is certainly possible that more than one sensory modality could be affected following brain damage to the right TPL, this association does not seem to be a necessary consequence of these type of attention disorder. Rather, it appears that patients suffering from hemiinattention deficits can have just one sensory modality affected (i.e., a dissociation between auditory and visual hemi-inattention symptoms). Our results support other behavioural research, not based on spatial attention, suggesting the idea that different sensory modalities draw from separate attentional reservoirs (i.e., Soto-Faraco & Spence, 2002; Sinnett et al., in press; Wickens, 1984). What could be the potential cause for the dissociation observed in the current study, and the discrepancy with previous studies that reported an association? In previous experimentation the potential correlation between the magnitude of visual and auditory hemi-inattention symptoms was always addressed using standard clinical visual tests and more precise auditory measures. However, it is apparent that the two tests measuring hemi-inattention in each modality are very different and could thereby skew the results of the correlation. Our contribution is that we have assessed levels of hemi-inattention (as indicated by the PSS) using a measure that provides a comparable index of spatial attention deficits across both sensory modalities. A possible explanation for the dissociation between auditory and visual hemi-inattention seen in our results could be the anatomical location of the brain lesion. It has been argued that the posterior parietal cortex could be an area important for multisensory spatial processing (i.e., Macaluso & Driver, 2004), therefore, it could be the case that a lesion affecting this area could have consequences in spatial processing across sensory modalities. If a common multisensory area were damaged, an association between auditory and visual hemi-inattention symptoms would be observed. This was not the case in the patients studied in the current investigation, even though all had lesions affecting the posterior parietal cortex. Lesion analysis, summarised in Table 1, did not reveal an obvious neuroanatomical distinction between patients impaired on the visual TOJ task and those who were not. All patients had lesions of the inferior parietal lobule that included the angular gyrus. In 6 of the 7 patients the lesion also involved mid and posterior parts of the superior temporal gyrus extending to the temporo-parietal junction. Yet, 3 of these patients displayed a significant shift on visual TOJ while two performed within the normal range on this task (the 3rd patient exhibited anti-extinction). Table 1 does suggest a potential neuroanatomical determinant of auditory hemi-inattention as measured by our TOJ task. Indeed, the 3 patients having the largest shifts in the auditory PSS (MA, 140 ms; IG, 158 ms; and EP, 183 ms) all had lesions involving Heschl s gyrus (auditory cortex). However, another patient in the group with lesion involvement of Heschl s gyrus (TR) did not exhibit signs of auditory hemi-inattention on the TOJ task. Moreover, only 2 of the 3 patients exhibiting auditory hemi-inattention had damage extending into the basal ganglia, in addition to the lesion involving the Heschl s gyrus. Recall that Bellman et al. (2001) reported auditory extinction, in a dichotic listening task, for a group of patients with brain lesions affecting the basal ganglia. One could argue that the auditory hemi-inattention observed in the patients with Heschl s gyrus lesions is due to a sensory problem and not to a disturbance in spatial processing. However, it should be noted that these patients performed at normal levels if the stimulus heard on the left side was presented more than 200 ms before the right stimulus. Interestingly, none of the 3 patients exhibiting high levels of auditory hemiinattention showed any signs of visual hemi-inattention. In fact, patients AS and EP showed what could be considered antiextinction in the visual modality, as their PSS scores were 106 and 228 ms (i.e., the right stimulus needed to lead the left for PSS to be achieved; see Goodrich & Ward, 1997; Humphreys et al., 2002). Yet, all three patients who suffered from auditory hemi-inattention also had lesions involving posterior areas of the brain (i.e., towards the visual cortex), not different in this respect from those patients who exhibited visual hemi-inattention. In most of the previous TOJ studies with hemi-inattention patients, the use of a spatial task (see for example, Di Pellegrino, Basso, & Frassinetti, 1997; Cate & Behrmann, 2002) may have introduced a response bias due to the natural tendency of these patients to prefer right (in manual pointing or the verbalization of the word right ). Here, we have attempted to neutralise this spatial component by making it orthogonal to the task (which stimulus came first, independent of location). It should be acknowledged, however, that patients could still preferentially respond to the stimulus that is presented on the right when confused. Nevertheless, the identification task should help to alleviate this problem to some degree and future research should consider the influence that different response types might have on these neurological tests. For instance, a task such as simply stating if the stimuli were presented simultaneously or not might, in principle, overcome possible criterion shifts based on the location of the objects in space. However, as it has been shown by Guerrini et al. (2003); see also Jaskowski (1991), the simultaneity task may involve slightly different task demands or attentional mechanisms and, therefore, could be less sensitive to temporal judgements of spatial laterality. In conclusion, the present results suggest that auditory and visual hemi-inattention deficits reflecting extinction are not necessarily as closely associated as has been suggested by previous studies. Thus, while it still remains possible that some manifestations of hemi-inattention can affect multisensory maps of space thereby producing associations, it is clear that the symptoms of heminegelct can arise in separate sensory modalities as well. Future research should concentrate on using equivalent measures (i.e., the TOJ task used in our experiment) to explore any possible associations between neglect and/or extinction symptoms and different sensory modalities, crucially, attempting to determine how lesion locations can help identify potential manifestations of hemi-inattention in unimodal, as well as more general, multimodal conditions.

9 560 S. Sinnett et al. / Neuropsychologia 45 (2007) Acknowledgement This research was supported by grants from Nissan Motors Co., Ministerio de Ciencia y Tecnologia (TIN C03-02) and the James S. McDonnell Foundation (JSMF ). References Bartolomeo, P., & Chokron, S. (1999). Egocentric frame of reference: its role in spatial bias after right hemisphere lesions. Neuropsychologia, 37, Baylis, G. C., Simon, S. L., Baylis, L. L., & Rorden, C. (2002). Visual extinction with double simultaneous stimulation: What is simultaneous? Neuropsychologia, 40, Bellman, A., Meuli, R., & Clarke, S. (2001). Two types of auditory neglect. Brain, 124, Bisiach, E., Cornacchia, L., Sterzi, R., & Vallar, G. (1984). Disorders of perceived auditory lateralization after lesions of the right hemisphere. Brain, 107, Cate, A., & Behrmann, M. (2002). Spatial and temporal influences on extinction. Neuropsychologia, 40, Diller, L., & Weinberg, J (1977). Hemi-inattention in rehabilitation: The evolution of a rational remediation program. In E. A. Weinstein, R. P. Friedland (Eds.), Hemi-inattention and hemisphere specialization. Advances in Neurology (pp ), vol. 18. New York: Raven Press. Di Pellegrino, G., Basso, G., & Frassinetti, F. (1997). Spatial extinction on double asynchronous stimulation. Neuropsychologia, 35, Di Pellegrino, G., Làdavas, E., & Farné, A. (1997). Seeing where your hands are. Nature, 388, 730. Driver, J., & Spence, C. (2004). Crossmodal spatial attention: Evidence from human performance. In C. Spence & J. Driver (Eds.), Crossmodal space and crossmodal attention. Oxford: Oxford University Press. Duncan, J., Martens, S., & Ward, R. (1997). Restricted attentional capacity within but not between sensory modalities. Nature, 387, Eimer, M., & Driver, J. (2000). An event-related brain potential study of crossmodal links in spatial attention between vision and touch. Psychophysiology, 37, Eimer, M., & Van Velzen, J. (2002). Crossmodal links in spatial attention are mediated by supramodal control processes: Evidence from event-related brain potentials. Psychophysiology, 39, Finney, D. J. (1964). Probit analysis: Statistical treatment of the sigmoid curve. London: Cambridge University Press. Gauthier, L., Dehaut, F., & Joannette, Y. (1989). The bells test: A quantitative and qualitative test for visual neglect. International Journal of Clinical Neuropyschology, 11, Goodrich, S. J., & Ward, R. (1997). Anti-extinction following unilateral parietal damage. Cognitive Neuropsychology, 14, Guerrini, C., Berlucchi, G., Bricolo, E., & Aglioti, S. (2003). Temporal modulation of spatial tactile extinction in right brain-damaged patients. Journal of Cognitive Neuroscience, 15, Humphreys, G. W., Riddoch, M. J., Nys, G., & Heinke, D. (2002). Transient binding by time: Neuropsychological evidence from anti-extinction. Cognitive Neuropsychology, 19, Jaskowski, P. (1991). Two-stage model for order discrimination. Perception & Psychophysics, 50, Karnath, H., Zimmer, U., & Lewald, J. (2002). Impaired perception of temporal order in auditory extinction. Neuropsychologia, 40, Làdavas, E., & Farnè, A. (2004). Neuropsychological evidence for multimodal representations of space near specific body parts. In C. Spence & J. Driver (Eds.), Crossmodal space and crossmodal attention (pp ). Oxford: Oxford University Press. Làdavas, E., Pavani, F., & Farnè, A. (2001). Auditory peripersonal space in humans: a case of auditory-tactile extinction. Neurocase, 7, Macaluso, E., & Driver, J. (2004). Spatial representations and crossmodal interactions in the human brain. In C. Spence, G. Calvert, & B. E. Stein (Eds.), The handbook of multisensory processes (pp ). Cambridge, USA: MIT Press. Maravita, A., Spence, C., Clarke, K., Husain, M., & Driver, J. (2000). Vision and touch through the looking glass in a case of crossmodal extinction. Neuroreport, 11(16), Mesulam, M. M. (1985). Principles of behavioural neurology. Philadelphia: Davis. Navarra, J., Vatakis, A., Zampini, M., Soto-Faraco, S., Humphreys, W., & Spence, C. (2005). Exposure to asynchronous audiovisual speech extends the temporal window for audiovisual integration. Cognitive Brain Research, 25, Pavani, F., Farne, A., & Làdavas, E. (2003). Task-dependent visual coding of sound position in visuospatial neglect patients. Neuroreport, 14, Pavani, F., Husain, M., Làdavas, E., & Driver, J. (2004). Auditory deficits in visuospatial neglect patients. Cortex, 40, Pavani, F., Husain, M., Malhotra, P., & Driver, J. (in preparation). Spatial and non-spatial auditory deficits in right-hemisphere patients are related to visual neglect. Pavani, F., & Làdavas, E. (1999). Functional integration of visual, auditory and proprioceptive spatial maps. In Proceedings of the 31st annual general meeting of the European brain and behavioural society. Pavani, F., Làdavas, E., & Driver, J. (2002). Selective deficit of auditory localization in patients with visuospatial neglect. Neuropsychologia, 40, Pavani, F., Làdavas, E., & Driver, J. (2003). Auditory and multisensory aspects of visuospatial neglect. Trends in Cognitive Sciences, 7, 9. Pavani, F., Meneghello, F., & Làdavas, E. (2001). Deficit of auditory space perception in patients with visuospatial neglect. Neuropsychologia, 39, Platt, B. B., & Warren, D. H. (1972). Auditory localization: The importance of eye movements and a textured visual environment. Perception and Psychophysics, 12, Posner, M., Walker, J., Friedrich, F., & Rafal, R. D. (1984). Effects of parietal injury on covert orienting of attention. Journal of Neuroscience, 4, Rorden, C., Mattingley, J. B., Karnath, H., & Driver, J. (1997). Visual extinction and prior entry: Impaired perception of temporal order with intact motion perception after unilateral parietal damage. Neuropsychologia, 35, Sinnett, S., Costa, A., & Soto-Faraco, S. (in press). Manipulating inattentional blindness within and across sensory modalities. Quarterly Journal of Experimental Psychology (A). Spence, C., Shore, D. I., & Klein, R. M. (2001). Multisensory prior entry. Journal of Experimental Psychology: General, 130, Soroker, N., Calamaro, N., Glicksohn, J., & Myslobodsky, M. S. (1997). Auditory inattention in right-hemisphere-damaged patients with and without visual neglect. Neuropsychologia, 35, Soto-Faraco, S., Morein-Zamir, S., & Kingstone, A. (2005). On audiovisual spatial synergy: The fragility of the phenomenon. Perception and Psychophysics, 67, Soto-Faraco, S., & Spence, C. (2002). Modality-specific auditory and visual temporal processing deficits. Quarterly Journal of Experimental Psychology, 55A, Treisman, A., & Davies, A. (1973). Dividing attention to ear and eye. In S. Kornblum (Ed.), Attention and performance IV (pp ). NY: Academic Press. Warren, D. H. (1970). Intermodality interactions in spatial localization. Cognitive Psychology, 1, Wickens, C. D. (1984). Processing resources in attention. In R. Parasuraman & D. R. Daves (Eds.), Varieties of attention (pp ). Orlando: Academic Press. Wilson, B., Cockburn, J., & Halligan, P. (1987). Development of a behavioural test of visuospatial neglect. Archives of Physical Medicine and Rehabilitation, 68, Zampini, M., Shore, D. I., & Spence, C. (2003). Multisensory temporal order judgments: the role of hemispheric redundancy. International Journal of Psychophysiology, 50, Zimmer, U., Lewald, J., & Karnath, H.-O. (2003). Disturbed sound lateralization in patients with spatial neglect. Journal of Cognitive Neuroscience, 15,