Neuromagnetic recordings reveal the temporal dynamics of auditory spatial processing in the human cortex

Transcription

1 Neuroscience Letters 396 (2006) Neuromagnetic recordings reveal the temporal dynamics of auditory spatial processing in the human cortex Hannu Tiitinen a,b,, Nelli H. Salminen a, Kalle J. Palomäki a,c, Ville T. Mäkinen a,b, Paavo Alku c, Patrick J.C. May a,b a Apperception & Cortical Dynamics (ACD), Cognitive Science, Department of Psychology, P.O. Box 9, FIN University of Helsinki, Finland b BioMag Laboratory, Engineering Centre, Helsinki University Central Hospital, Helsinki, Finland c Laboratory of Acoustics and Audio Signal Processing, Helsinki University of Technology, Finland Received 1 July 2005; received in revised form 1 November 2005; accepted 4 November 2005 Abstract In an attempt to delineate the assumed what and where processing streams, we studied the processing of spatial sound in the human cortex by using magnetoencephalography in the passive and active recording conditions and two kinds of spatial stimuli: individually constructed, highly realistic spatial (3D) stimuli and stimuli containing interaural time difference (ITD) cues only. The auditory P1m, N1m, and P2m responses of the event-related field were found to be sensitive to the direction of sound source in the azimuthal plane. In general, the right-hemispheric responses to spatial sounds were more prominent than the left-hemispheric ones. The right-hemispheric P1m and N1m responses peaked earlier for sound sources in the contralateral than for sources in the ipsilateral hemifield and the peak amplitudes of all responses reached their maxima for contralateral sound sources. The amplitude of the right-hemispheric P2m response reflected the degree of spatiality of sound, being twice as large for the 3D than ITD stimuli. The results indicate that the right hemisphere is specialized in the processing of spatial cues in the passive recording condition. Minimum current estimate (MCE) localization revealed that temporal areas were activated both in the active and passive condition. This initial activation, taking place at around 100 ms, was followed by parietal and frontal activity at 180 and 200 ms, respectively. The latter activations, however, were specific to attentional engagement and motor responding. This suggests that parietal activation reflects active responding to a spatial sound rather than auditory spatial processing as such Elsevier Ireland Ltd. All rights reserved. Keywords: Magnetoencephalography (MEG); Auditory; Spatial; P1m; N1m; P2m; Attention In natural auditory environments, sounds are endowed with spatial cues because they are emitted from sources situated in the three-dimensional (3D) space surrounding the listener. Recently, the processing of auditory stimuli with spatial properties has been studied using event-related potentials (ERPs) and fields (ERFs) measured in the electro- (EEG) and magnetoencephalography (MEG), respectively, which provide temporally precise measures of cortical activity [1,6,12,13,14]. These efforts have focused on the auditory N1m, a prominent cortical response peaking some 100 ms after stimulus onset. Its amplitude in both the left and right hemisphere has been found to reflect sound source direction, and displays maxima and minima for contraand ipsilateral sound sources, respectively [12 14]. Also, both Corresponding author. Tel.: ; fax: address: hannu.tiitinen@helsinki.fi (H. Tiitinen). the overall amplitude of the N1m and its variation according to sound source direction are larger in the right hemisphere. A right-hemispheric preponderance is also indicated by hemodynamic studies revealing activation in the right-hemispheric planum temporale both with stationary [22] and moving [2,15] spatial sound. Complementing these findings, behavioral studies have shown that localization is more accurate for stimuli in the left hemifield [5] and when listening with the left ear only [7]. The study of auditory spatial processing in humans has largely neglected the temporal dynamics of the underlying brain activity. This has been due to the limited temporal resolution of hemodynamic methods, whereas researchers using EEG and MEG usually restrict their studies to a particular deflection of the ERP/ERF and thus have not taken full advantage of the high temporal resolution of these methods. The ERP/ERF comprises a series of deflections which together could be used to study the temporal dynamics of auditory spatial processing. For transient /$ see front matter 2005 Elsevier Ireland Ltd. All rights reserved. doi: /j.neulet

2 18 H. Tiitinen et al. / Neuroscience Letters 396 (2006) stimuli, many spatial cues occur only after the first 100 ms poststimulus [4]. Therefore, the processing of these cues cannot be reflected by the N1(m) but might, rather, be captured by the P2(m) response occurring at 200 ms post-stimulus. The P2m has been found to reflect the processing of complex sound features [8,18]. As it is generated partly in the planum temporale [8],an area which hemodynamic studies have identified as taking part in auditory spatial processing [2,15,20,22], it might turn out to be useful for studying the processing of spatial cues in auditory stimulation. Hemodynamic studies have reported activation in the superior temporal gyrus and in posterior parietal areas during auditory spatial tasks [10,15,16,20,22], although the role of the parietal areas in these tasks has remained unclear [3,11]. One interpretation is that parietal areas are a part of a where processing stream analyzing sound location as opposed to a what stream dedicated to the processing of auditory object identity [1,10,16]. A recent PET experiment, however, suggests that the parietal activation occurs only when active responding is required, whereas auditory spatial stimulation in itself activates only auditory areas in the temporal cortex [22]. Parietal activation during an auditory spatial task was also detected in an EEG study [1] which suggests that temporally precise EEG and MEG methods might be useful in elucidating the functional role of parietal cortex. The ability to localize sounds is based on the utilization of multiple localization cues embedded in the sound signal: the interaural time and level difference (ITD and ILD, respectively) and the spectral distortions produced by the pinnae, the head and the body [4]. Although often used as stimulus material in cognitive neuroscience [2,10], the ITD and ILD, when utilized alone, only represent a subset of spatial sound cues and lead to the subject perceiving sound as originating inside the head rather than in the surrounding space. For creating genuine 3D environments, loudspeakers can be used in certain conditions [6,22] but this results in an auditory environment which is specific to the measurement room and cannot be replicated elsewhere. Additionally, loudspeakers cannot be used in MEG due to their magnetic interference. Recently, head-related transfer functions (HRTFs) [21], which replicate the spectral distortions produced by the pinnae, the head, and the body, have provided realistic spatial sound stimuli for brain research purposes [12 14]. An even higher level of realism can be achieved by presenting stimuli through loudspeakers and recording them with miniature microphones placed inside the ear canals of each individual subject, and thereafter using these recordings as subject-specific stimulus material in earphone stimulation. In sum, the study of auditory processing in human cortex has utilized stimulus material whose spectral detail, unfortunately, does not correspond to natural, spatial environments. Although research on the processing of auditory spatial information has gained momentum we still lack an adequate description of the temporal and spatial evolution of the underlying neural activity (e.g. in terms of the putative what and where processing streams). To address these issues we designed three interlinked experiments utilizing subject-specific, realistic spatial sound stimulation and high-resolution MEG recordings. For an improved signal-to-noise ratio, the number of repetitions of each stimulus was elevated at the expense of the number of subjects. The aim of the first experiment was to find out how the direction of the source of a realistic 3D sound is reflected in the P1m, N1m, and P2m responses. In the second experiment, we aimed to reveal processing specific to the spectral detail characteristic of a realistic spatial sound by comparing brain responses elicited by spatially impoverished ITD stimuli and realistic 3D sounds. In the third experiment, we aimed to clarify how temporal and parietal areas take part in auditory spatial processing during passive and active recording conditions, with the hypothesis that parietal activation only occurs in conditions where attentional engagement is required [1,22]. The stimulus set comprised noise bursts with a bandwidth of 10 khz presented at an onset-to-onset interstimulus interval of 1000 ms. Realistic spatial sound stimuli (3D sounds) were obtained by individual recordings for each subject. Microphones were placed in the subject s ear canals blocked with silicon paste. 50-ms noise bursts were sequentially presented from loudspeakers placed at eight locations around the subject at a distance of 1.5 m (Fig. 1A). The recordings of 200-ms epochs took place in a slightly reverberant room adhering to the IEC standard for listening rooms. Lateralized sounds (perceived as originating inside the head) from the right and left hemifields, corresponding to direction angles 90 and 270 (Fig. 1A), were produced by applying ITDs. In order to ensure their correspondence with the 3D stimuli, the ITD sounds were filtered to match the smoothed grand-averaged spectrum of the 3D sound originating from the 0 direction. Five volunteers (all male, right-handed, aged 30 42) took part in the study with informed consent and with the approval of the Ethical Committee of Helsinki University Central Hospital. The stimuli were delivered through a custom-made tubephone system (frequency range 100 Hz 10 khz). Magnetic responses were recorded with a 306-channel whole-head magnetometer (Vectorview 4D, Neuromag Oy, Finland) using a sampling rate of 600 Hz (passband Hz). Horizontal and vertical eye movement sensors were used for removing epochs with eyemovement-related artefacts defined as absolute displacements exceeding 150 V. The subject sat in a reclining chair and in Experiments I and II, was under instruction not to pay attention to the auditory stimuli and to concentrate on watching a selfselected silent film. In Experiment I, 3D sounds were presented from eight equidistantly spaced direction angles separated by 45 steps (Fig. 1A). In Experiment II, the stimuli consisted of 3D and ITD sounds presented from the right and the left hemifields (i.e. from angles 90 and 270 ). In Experiment III, the stimuli of Experiment II were presented to the subject who was under instruction to attend to a target stimulus and to press a button with the right index finger when he detected the target. The target was either the 3D sound from the right (90 ) or the left (270 ) hemifield. Only brain responses accompanied by a correct behavioral response were included in the analyses. The stimuli were presented in a random order, counterbalanced across subjects. For each stimulus, >500 responses in Experiments I and II and >166 responses in Experiment III were collected.

3 H. Tiitinen et al. / Neuroscience Letters 396 (2006) Fig. 1. The P1m, N1m, and P2m responses as indicators of the cortical processing of sound source direction. In Experiment I, realistic spatial sound was presented from eight equally spaced sound source directions in the azimuthal plane. (A) Grand-averaged MEG responses. Sounds from each source direction elicited a response complex comprising the P1m, N1m, and P2m. The right-hemispheric P1m and N1m peaked earlier for sound sources contralateral to the hemisphere. The amplitudes of the P1m and N1m in both hemispheres and the right-hemispheric P2m varied according to the sound source direction. Overall, sound sources in the contralateral hemisphere resulted in larger peak amplitudes, and the right-hemispheric responses were larger in amplitude than the left-hemispheric ones. (B) Grand-averaged MCEs obtained at the latencies of the P1m, N1m, and P2m to the 3D sound from the direction angle 90. Activation restricted to the vicinity of the auditory cortex was observed at each latency. The MEG responses were averaged over a 500-ms poststimulus period (filter passband 2 30 Hz, baseline-correction with respect to a 100-ms pre-stimulus period). For each temporal lobe, data from the sensor displaying the largest N1m amplitude were selected for further analyses. The peak amplitudes and latencies of the P1m, N1m, and P2m were identified. Response sources were localized by using the minimum current estimation (MCE)-technique [19] which, unlike the ECD method, has the advantage that it can describe several local or distributed sources. The activations were mapped to a realistic head model which was divided into a grid of 1425 points. Six regions of interest (ROIs) were defined in the temporal, parietal, and frontal cortices of the two hemispheres. From the summed activity within each ROI, the amplitude and latency of maximal activity were identified. Statistical significances of the differences in response peak latencies and amplitudes were tested with a multivariate analysis of variance (MANOVA). Newman Keuls post hoc tests were performed when appropriate. For statistical analyses, direction angles 225, 270 and 315 represented left-hemifield stimulation and angles 45,90, and 135 right-hemifield stimulation. When analyzing the effect of attention, the MEG responses to the 3D target sound (attended) were compared to those elicited by the same sound in the listening condition where the 3D sound in the opposite hemifield was the target (unattended). The 3D sounds, presented from eight sound source directions in the azimuthal plane, elicited a prominent response complex consisting of the P1m, N1m, and P2m in both hemispheres (see Fig. 1A and Table 1). Response amplitudes and latencies to sound sources in the left and right hemifields are summarized in Table 2. In general, the right-hemispheric responses had larger amplitudes than the left-hemispheric ones (F[3,18] = 4.90, p < 0.01). Response peak latencies were affected by sound source direction in the right (F[3,18] = 21.14, p < 0.001) but not in the left hemisphere (F[3,18] = 2.09, p = n.s.). In the right hemisphere, sound sources in the left hemifield elicited an earlier P1m and N1m than did right-hemifield ones. In both hemispheres, contralateral sound sources elicited larger response amplitudes than ipsilateral ones (F[3,18] = 11.9, F[3,18] = 38.61, for the left and right hemisphere, respectively, p < for both). With the exception of the left-hemispheric P2m (whose amplitude Table 1 Mean amplitudes and latencies Amplitudes (ft/cm) Latencies (ms) P1m N1m P2m P1m N1m P2m Experiment I Left hemisphere Right hemisphere Experiment II Left hemisphere Right hemisphere Experiment III Left hemisphere Right hemisphere

4 20 H. Tiitinen et al. / Neuroscience Letters 396 (2006) Table 2 Mean amplitudes and latencies (Experiment I) Amplitudes (ft/cm) Latencies (ms) Left hemifield Right hemifield Left hemifield Right hemifield Left hemisphere P1m p < p = n.s. N1m p < p = n.s. P2m p = n.s p = n.s. Right hemisphere P1m p < p < N1m p < p < P2m p < p = n.s. was unaffected by sound source direction), maximal responses were consistently elicited by sound sources in the contralateral hemifield. MCE localization (Fig. 1B) revealed large areas of activation in the vicinity of the auditory cortex at the peak latencies of the P1m, N1m, and P2m in both hemispheres. This activation was more pronounced in the right hemisphere, especially at the peak latencies of the N1m and P2m. The subjects were presented with realistic 3D sounds and spatially impoverished ITD sounds from either the left or right hemifield. As shown in Fig. 2A and Table 3, both the 3D and ITD sounds elicited prominent P1m, N1m, and P2m responses. While there were no differences in response latencies (F[3,8] = 1.56, p = n.s., F[3,8] = 1.81, p = n.s., for the left and right hemisphere, respectively), the 3D sound resulted in 115% larger peak amplitudes of the right-hemispheric P2m than did the ITD sounds (F[3,8] = 1.93, p = n.s. and F[3,8] = 36.04, p < for the left and right hemisphere, respectively; see Table 3). MCEs obtained at the peak latency of the P2m are shown in Fig. 2B, revealing how the 3D sounds resulted in more pronounced activation than did the ITD sounds, especially in the right hemisphere. The stimulus set consisted of 3D and ITD sounds originating from the left and right and the subject s task was to attend to the stimuli and respond to a target (3D) sound. The P1m, N1m, and P2m elicited by the attended and unattended 3D sounds are shown in Fig. 3A (see also Table 1). The amplitude of the N1m in the left and right hemisphere was 23% and 30% larger, respectively, for the attended than for the unattended sounds, although these effects did not reach statistical significance. As revealed by MCE analyses, however, attention strongly affected the activations taking place in the parietal and frontal areas: while unattended stimuli activated temporal brain areas only, attended stimuli led to activity spreading from temporal to parietal and frontal areas. Specifically, as shown in Fig. 3B, the activity elicited by unattended sound reached a maximum in the temporal areas at 105 ms (11.9 and 16.4 na m for the left and right hemisphere, respectively), in the latency range of the N1m. A secondary peak was observed in the P2m latency range, Table 3 Mean amplitudes (Experiment II) Amplitudes (ft/cm) 3D ITD Left hemisphere P1m p = n.s. N1m p = n.s. P2m p = n.s. Right hemisphere P1m p = n.s. N1m p = n.s. P2m p < Fig. 2. Brain responses to 3D and ITD sounds. In Experiment II, the subjects were presented with realistic spatial 3D sounds and spatially impoverished ITD sounds in the left and right hemifield (angles 90 and 270 ). (A) Grand-averaged MEG responses to 3D (solid line) and ITD (dashed line) sounds from the left and right hemifield. While the P1m and N1m to the 3D and ITD sounds did not differ, the right-hemispheric P2m elicited by the 3D sound was considerably larger in amplitude than that elicited by the ITD sound. (B) Grand-averaged MCEs obtained at the peak latency of the P2m to 3D and ITD sounds originating from the right hemifield. Interhemispheric differences were visible in the MCEs as more pronounced activation in the right hemisphere. The results support a righthemispheric preponderance in auditory spatial processing and indicate that the P2m might reflect processing specific to the spectral cues of spatial sounds.

5 H. Tiitinen et al. / Neuroscience Letters 396 (2006) Fig. 3. The effect of an active task on auditory spatial processing. In Experiment III the stimulus set was identical to that of Experiment II and the subject s task was to respond by pressing a response key when a target stimulus was detected. The target sound was the 3D sound originating from either the left or the right hemifield. (A) Grand-averaged MEG responses to attended (solid line) and unattended (dashed line) 3D sounds. (B) Grand-averaged MCEs derived from the responses to the attended and unattended left-hemifield 3D sound. Attention had a marked effect on cortical activation: At around the 80-ms latency of the N1m response, the activation is similar to that observed in the passive recording condition of Experiment I. However, following this initial activity, at around 160 ms, parietal activation specific to attentional engagement and motor responding emerges. Further, at around the 180-ms peak latency of the P2m, the attended stimulus activates a frontal area. These results suggest that the parietal activation reflects active responding rather than auditory spatial processing in itself. at around 180 ms. The activity in the parietal (4.3 and 4.7 na m) and frontal (5.2 and 6.3 na m) brain areas remained low during the processing of the unattended sound. Importantly, only when the stimulus was attended was a more extensive network of cortical areas activated: In the left hemisphere, the initial activation in the temporal areas at 107 ms was followed by activity in the parietal and frontal areas at 181 and 203 ms, respectively (F[2,8] = 9.86, p < 0.01). The parietal (11.2 na m, p < 0.01) and frontal (9.9 na m, p < 0.01) activity was significantly stronger for the attended than for the unattended sounds (F[2,8] = 5.6, p < 0.05). The increase in parietal and frontal activation was observed only in the left hemisphere, possibly due to the subjects carrying out the motor response with their right-hand index finger. Here, we studied the temporal dynamics of auditory spatial processing in the human cortex and found that each element of the auditory response complex comprising the P1m, N1m, and P2m reflected the location of the sound source in the azimuthal plane. Of these three, the P2m appeared to be especially sensitive to the presence of spatial cues in the auditory stimuli and displayed twice as large amplitudes to the natural 3D sounds than to their spatially impoverished ITD counterparts even though the two sound types were carefully matched for their spectrotemporal complexity. This, of course, does not exclude the possibility that the P2m is also sensitive to variations in spectrotemporal detail unrelated to the spatial quality in stimulation [8,18]. In general, the 3D sounds resulted in more prominent brain activity in the right than the left hemisphere, indicating that the spatial cues embedded in auditory stimuli might predominantly be processed in the right hemisphere. The engagement of attentional processes had a major effect on the global dynamics of cortex, with parietal and frontal activation emerging only in conditions where stimuli were attended and responded to. In contrast, the brain activity in the passive recording condition and that observed in the processing of unattended stimuli was restricted to temporal areas only. Thus, the spatial and temporal evolution of neural activity associated with the processing of spatial cues can be accessed with MEG. Corroborating previous results [12 14], the amplitude of the N1m varied according to sound source direction. However, expanding the analysis to include brain responses before and after the N1m revealed that the processing of spatial cues is also reflected by the P1m and P2m responses. The current results indicate that the N1m and P1m responses are similarly sensitive to spatial cues, and that differences in the way realistic and spatially impoverished sounds are processed become especially apparent in the right hemisphere at the latency of the P2m response. These amplitude variations might be explained by intracortical results showing that neurons responding strongest to contralateral sound sources are in the majority in both hemispheres, with only a few neurons preferring sound sources in the front or rear [9,17]. This distribution of preferential responding could account for the amplitude variation of the P1m, N1m, and P2m responses if one assumes that MEG response amplitude reflects the number of responding neurons. In this study, MCEs were utilized in describing how the temporal succession and spatial distribution of cortical activation depends on attentional engagement. Corroborating previous findings utilizing PET [22], our analyses revealed parietal activation only during an active auditory spatial task but not during passive stimulation. Further, during the active stimulation condition, the parietal activity only emerged when the subject detected

6 22 H. Tiitinen et al. / Neuroscience Letters 396 (2006) and responded to a target stimulus, roughly 100 ms after the onset of activation in temporal areas. Thus, parietal areas might not participate in auditory spatial processing per se but, rather, might only take part in active responding to spatial sound stimuli. These results would then place significant parietal parts of the claimed where processing stream outside the processing of the spatial sound itself, into a cortical area and processing stage associated with active motor responding. Acknowledgement This study was supported by the Academy of Finland (Proj. no , ). References [1] C. Alain, S.R. Arnott, S. Hevenor, S. Graham, C.L. Grady, What and where in the human auditory system, PNAS 98 (2001) [2] F. Baumgart, B. Gaschler-Markefski, M.G. Woldorff, H.J. Heinze, H. Scheich, A movement-sensitive area in auditory cortex, Nature 400 (1999) [3] P. Belin, R.J. Zatorre, What, where and how in auditory cortex, Nat. Neurosci. 3 (2000) [4] J. Blauert, Spatial Hearing, MIT Press, Cambridge, [5] K.A. Burke, A. Letsos, R.A. Butler, Asymmetric performances in binaural localization of sound in space, Neuropsychologia 32 (1994) [6] R.A. Butler, The influence of spatial separation of sound sources on the auditory evoked response, Neuropsychologia 10 (1972) [7] R.A. Butler, Asymmetric performances in monaural localization of sound in space, Neuropsychologia 32 (1994) [8] K.E. Crowley, I.M. Colrain, A review of the evidence for P2 being an independent component process: age, sleep and modality, Clin. Neurophysiol. 115 (2004) [9] S. Furukawa, L. Xu, J.C. Middlebrooks, Coding of sound-source location by ensembles of cortical neurons, J. Neurosci. 20 (2000) [10] P.P. Maeder, R.A. Meuli, M. Adriani, A. Bellmann, E. Fornari, J.P. Thiran, A. Pittet, S. Clarke, Distinct pathways involved in sound recognition and localization: a human fmri study, NeuroImage 14 (2001) [11] J.C. Middlebrooks, Auditory space processing: here, there or everywhere? Nat. Neurosci. 5 (2002) [12] K. Palomäki, P. Alku, V. Mäkinen, P. May, H. Tiitinen, Sound localization in the human brain: neuromagnetic observations, NeuroReport 11 (2000) [13] K. Palomäki, H. Tiitinen, V. Mäkinen, P. May, P. Alku, Cortical processing of speech sounds and their analogues in a spatial auditory environment, Cogn. Brain Res. 14 (2002) [14] K.J. Palomäki, H. Tiitinen, V. Mäkinen, P.J.C. May, P. Alku, Spatial processing in human auditory cortex: the effects of 3D, ITD, and ILD stimulation techniques, Cogn. Brain Res. 24 (2005) [15] F. Pavani, E. Macaluso, J.D. Warren, J. Driver, T.D. Griffiths, A common cortical substrate activated by horizontal and vertical movement in the human brain, Curr. Biol. 12 (2002) [16] J.P. Rauschecker, B. Tian, Mechanisms and streams for processing of what and where in auditory cortex, PNAS 97 (2000) [17] G.C. Stecker, I.A. Harrington, J.C. Middlebrooks, Location coding by opponent neural populations in the auditory cortex, PLoS Biol. 3 (2005) e78. [18] H. Tiitinen, P. Sivonen, P. Alku, J. Virtanen, R. Näätänen, Electromagnetic recordings reveal latency differences in speech and tone processing in humans, Cogn. Brain Res. 8 (1999) [19] K. Uutela, M. Hämäläinen, E. Somersalo, Visualization of magnetoencephalographic data using minimum current estimates, NeuroImage 10 (1999) [20] J.D. Warren, B.A. Zielinski, G.G.R. Green, J.P. Rauschecker, T.D. Griffiths, Perception of sound-source motion by the human brain, Neuron 34 (2002) [21] F.L. Wrightman, D.J. Kistler, Headphone simulation of free-field listening. I: Stimulus synthesis, II: Psychophysical validation, J. Acoust. Soc. Am. 85 (1989) [22] R.J. Zatorre, M. Bouffard, P. Ahad, P. Belin, Where is where in the human auditory cortex? Nat. Neurosci. 5 (2002)