Event-related brain activity associated with auditory pattern processing

Size: px

Start display at page:

Download "Event-related brain activity associated with auditory pattern processing"

Christiana Ward
5 years ago
Views:

1 Cognitive Neuroscience p Website publication November NeuroReport, () ONE of the basic properties of the auditory system is the ability to analyse complex temporal patterns. Here, we investigated the neural activity associated with auditory pattern processing using event-related brain potentials. Participants were presented with a continuously repeating sequence of four tones with rare changes in either the frequency or timing of one of the tones. Both frequency- and time-deviant sounds generated mismatch negativity (MMN) waves that peaked at midline central electrode sites and inverted in polarity at inferior temporal and occipital sites, consistent with generators in the supratemporal plane. The MMN scalp topography was similar for the frequency- and time-deviant stimuli, suggesting that both spectral and temporal relations among elements of an auditory pattern are encoded in a unified memory trace. NeuroReport : Lippincott Williams & Wilkins. Key words: Auditory; Cortex; Event-related potentials; Frequency; Mismatch negativity; Pattern; Perception; Sensory memory Introduction The perception of complex auditory events such as speech and music depends upon a listener s ability to extract temporal patterns from the ongoing acoustic input. Auditory pattern perception is influenced by stimulus-related factors such as the rate of presentation and frequency separation among elements composing the pattern, - and more subjective factors such as the listener s knowledge and prior experience with the pattern. Selective attention can also influence pattern perception by enabling the listener to focus on the whole pattern or on discrete elements within the pattern. In recent years, many studies have been concerned with the neural bases of auditory pattern processing. Single-unit recordings in cats, and non-human primates, have shown that primary and secondary auditory cortices contribute to pattern perception. In human subjects, the neural activity associated with auditory pattern processing can be investigated using the mismatch negativity (MMN) wave of the eventrelated brain potentials (ERPs). The MMN wave is typically elicited by infrequent deviant stimuli occurring in sequences of standard stimuli. The deviant stimuli may differ in such dimensions as pitch, intensity, duration, spatial location, or stimulus onset asynchrony. Deviations from simple auditory patterns such as sequences of tones that alternate or decrease regularly in frequency can also generate an MMN. This MMN wave to pattern deviant stimuli Event-related brain activity associated with auditory pattern processing Claude Alain,,,CA Filomeno Cortese and Terence W. Picton, Rotman Research Institute, Baycrest Centre for Geriatric Care and Department of Psychology, University of Toronto, 0 Bathurst Street, Toronto, Ontario MA E, Canada CA, Corresponding Author and Address may reflect a neural mismatch between the incoming stimulus and the expected stimulus based upon the organization of the previous stimuli., In other words, the pattern of stimuli lead the listener to expect one signal and not another, and the MMN occurs when an incoming sound violates that expectancy. Such an account predicts that the MMN amplitude would vary with the magnitude of deviation in frequency and/or timing between two consecutive elements composing the pattern. In the current study, we examined the effects of small and large decrements in spectral or temporal transitions between two consecutive stimuli in a repeating pattern of four tones. If the MMN to pattern deviance reflects a neural mismatch between the incoming sound and the expected one, then the MMN amplitude should vary with change in the expected spectral and/or temporal transition between two consecutive stimuli. Another important aspect of the study was to evaluate the nature of auditory memory underlying MMN generation. For instance, the MMN to patterndeviant sounds may depend on representations of the elements composing the pattern as well as their order and/or the representation of the pattern as a whole. Recent scalp distribution analyses have revealed different topographies for MMNs elicited by frequency-, intensity- and duration-deviant stimuli, suggesting that different stimulus attributes are encoded into separate memory traces. However, there is also evidence that MMN generation can depend 0- Lippincott Williams & Wilkins Vol No October

2 C. Alain, F. Cortese and T. W. Picton p on an integrated (Gestalt) representation of the standard stimulus rather than separate memory traces for each individual feature. In the current study, we recorded the MMN from a large array of electrodes to examine the scalp topography of the MMN elicited by frequency- and time-deviant stimuli embedded in a four-tone pattern. If the MMN relies on separate memory traces, then the MMN amplitude distribution should vary as a function of the deviant type. Materials and Methods Twelve participants (six men; aged years) were presented with a standard pattern consisting of four tones (000, 000, 00, 00 Hz, see Fig. ). The stimuli were 0 ms in duration ( ms rise/fall time) and were presented binaurally, with an interstimulus interval (ISI) of 0 ms through insert earphones at 0 db SPL. Two types of deviations occurred: () frequency, in which the frequency transition between two consecutive tones was reduced by or % (e.g. for the standard 000 Hz tone which normally follows the 00 Hz tone, the deviant was 00 and 00 Hz respectively) and () time, in which the standard 0 ms ISI between two consecutive tones was reduced by or % (i.e. to 0 and 0 ms ISI, respectively). The sequences were constructed such that a deviant stimulus occurred in % of all stimuli, with a minimum interval between deviants of standard tones. Of that %, each of the four possible deviants occurred with equal probability (i.e. % each). Participants were presented with eight sequences of 00 tones including 0 deviant tones. In total, 0 standards and 0 deviants (0 per deviant type) were presented to each participant. Participants were instructed to ignore the auditory stimuli and to perform a concomitant visuo-motor task. The visual stimuli were five lights on a response box. On each trial, one light turned on and participants turned it off by pressing the button corresponding to the light. The next light turned on 00 ms after this response. This serial-choice reaction time task provided a continuous monitoring of performance and allowed us to examine whether the participants behavior was affected by the auditory stimuli. To evaluate the potential distracting effects of sound on performance, participants also performed the visual task without the tones presented in the background. The ability to detect changes in auditory patterns was assessed behaviorally in a second experiment held on a different day without EEG recording. Eight participants from the ERP study participated in the behavioral auditory discrimination task. Participants were asked to listen to the same sequences employed to generate the MMN wave and to press a button as quickly as possible on the response box when they heard a sound that deviated from the ongoing pattern. The reaction times (RTs) were analysed for correct trials only, i.e. those trials with RTs occurring between 00 and 000 ms after a deviant sound was presented. Responses at other times were classified as false alarms (FAs). The electrical brain activity was amplified and filtered (bandpass 0. 0 Hz; 0 Hz sampling rate) via NeuroScan SynAmps from an array of channels that included electrodes on the neck and face. Vertical and horizontal eye movements were monitored with electrodes placed at the outer canthi and at the superior and inferior orbit. All electrodes were referenced to an electrode placed on the tip of the nose during the recording and converted to an average reference off-line. ERPs were averaged separately for each site and stimulus type. The epoch included 00 ms pre-stimulus activity and 00 ms post-stimulus activity. Trials contaminated by excessive peak-to-peak deflection (± 0 V) were excluded from the ERP average. In FIG.. Schemata of the stimuli used. Participants listened to a repeating pattern of four tones with occasional changes in the frequency and timing of one of the tones. The potential deviant stimulus is shown by an asterisk. Stimulus onset asynchrony = 00 ms. Vol No October

3 p Auditory pattern and mismatch negativity each individual average, the ocular artifacts (e.g. blinks and lateral movements) not removed by the artifact rejection criteria were corrected using ocular source components through brain electrical source analysis (BESA) software. This involved a decimation of the original 0 data points to 0 points reducing the effective sampling rate to 0 Hz. The MMNs for both deviant types (frequency and time) and both magnitudes (small and large) were identified in the difference wave between the standard and the deviant stimuli. The MMN peak latency was defined as the maximum negativity between 00 and 00 ms following deviant onset. The MMN mean amplitude was computed for the 0 0 ms interval and measured relative to the mean amplitude of the 00 ms pre-stimulus activity for each deviant-type (frequency and time) and magnitude (small and large). The MMN peak latencies and amplitudes were then submitted to an analysis of variance for repeated measures with deviant type, magnitude, and electrode as factors. Scalp topographies were statistically analysed after voltage normalization and included electrode sites. The electrodes from the face and neck were omitted from the analysis. Type I errors associated with inhomogeneity of variance were controlled by decreasing the degrees of freedom using the Greenhouse-Geisser epsilon. Results Participants responded quickly (mean ± s.d. RT = 0 ± ms) and accurately to visual stimuli, making < % errors during the serial-choice reaction time task. The participants pace was not affected by the presence of sounds (mean RT = 0 ± ). We also examined the RTs to visual stimuli following up to s after either standards or a deviant auditory stimulus. The RTs were not affected by the type of auditory stimuli presented in the background (Table ). Figure shows the MMN for both deviant types and both magnitudes recorded over the frontal, central and temporal regions while participants performed the visuo-motor task. Because the stimuli were presented at short and fixed ISIs, the MMN to time-deviant stimuli contained pre-stimulus activity elicited by previous tones (i.e. the P-N-P response to the preceding stimulus would occur at different Table. Response time to visual stimuli as a function of the auditory stimulus occurring in the preceding 000 ms. Preceded by Reaction time (ms) Standards 0 ± Frequency deviant % 0 ± Frequency deviant % 0 ± Time deviant % 0 ± Time deviant % 0 ± FIG.. Group means MMN for frequency (pitch) and time-deviant sounds recorded over the right hemisphere. The stimulus onset is shown by the vertical bar and negativity is plotted upward. Fz, middle frontal; FC, superior right frontal-central; C, superior right central; CP, midcentral-parietal; TP0, right mastoid. times and would not cancel in the subtraction). MMNs to both frequency- and time-deviants peaked at about 00 ms post-stimulus. At the midline frontal site (i.e. Fz), the MMN peaked earlier for large than small deviant sounds (F(,) =., p < 0.0). The interaction between deviant-type and deviation magnitude was marginally significant (F(,) =., p = 0.0), due to the magnitude effect on MMN latency being larger for the time-deviant than for the frequency deviant. The effect of magnitude and deviant-type on the MMN amplitude was quantified by comparing the mean voltage obtained at the midline frontal site (Fz). The analysis of variance with magnitude and deviant-type yielded a significant effect of magnitude (F(,) =., p < 0.00). The MMN amplitude was increased by about. times for large frequencydeviants whereas it was 0 times larger for large time-deviants, producing a significant deviant-type magnitude interaction (F(,) =.0, p < 0.0). Figure shows the MMN amplitude distribution as a function of deviant type and magnitude. Both frequency- and time-deviant stimuli generated an MMN that was maximum over frontal regions and was larger over the right hemisphere (F vs F, Vol No October

0 0 0 0 0 p FIG.. Normalized isopotential gray-scale maps of the MMN elicited by frequency-deviant (top) and time-deviant (bottom) stimuli in the four-tone pattern.

4 p FIG.. Normalized isopotential gray-scale maps of the MMN elicited by frequency-deviant (top) and time-deviant (bottom) stimuli in the four-tone pattern. The original data ( scalp sites) were interpolated with a spherical spline algorithm. F(,) =., p < 0.0). Because the MMN latency varied with deviant type and magnitude, the MMN amplitude distributions were examined over the particular 0 ms interval where the MMN was observed to be maximal at Fz for each deviant type. The electrode magnitude interaction was significant (F(,0) =., p < 0.00, = 0.). However, no significant difference in amplitude distribution was found between the MMN to frequency and time-deviant stimuli (F(,0) =.) nor was the electrode type magnitude interaction significant (F(,0) =.). Similar results were obtained when all measurements were taken in the 0 0 ms interval. Table summarizes the group mean performance of the eight participants who took part in the behavioral auditory discrimination task. The repeated measures ANOVA on response latency revealed a main effect of deviant type (F(,) =., p < 0.0), magnitude (F(,) =.0, p < 0.00), and a significant interaction between the magnitude and the deviant type (F(,) =., p < 0.00). That is, participants were faster in detecting large timedeviants than large frequency- deviants, whereas they showed similar response times in detecting small frequency- and time-deviant stimuli. There were significant correlations between both accuracy and response time obtained at the auditory discrimination task and the MMN amplitude recorded at Fz during the visuo-motor task (r = 0. for both accuracy-mmn and RT-MMN, p < 0.0 in both cases) indicating that good auditory discrimination was associated with larger MMN responses. Discussion C. Alain, F. Cortese and T. W. Picton Table. Group mean performance (± s.d.) in detecting frequency- and time-deviant stimuli Deviant Magnitude Accuracy Reaction type (%) time (ms) Pitch Small ± ± Large ± ± Time Small ± ± Large ± ± For all participants, deviations in spectral or temporal transition in a four-tone pattern elicited an MMN wave. Our results replicate and extend previous findings showing an MMN to pattern-deviant stimuli using simpler tone patterns.,, The MMN amplitude and latency varied with the magnitude of deviation, which is consistent with many studies showing a decrease in the amplitude and an increase in the latency of the MMN with reduced discriminability between standards and deviant stimuli., This finding is also consistent with the proposal that the MMN to pattern-deviant sounds reflects a violation of expectancy established by the preceding stimuli. This result cannot be accounted for by simple differences along a physical dimension because the large frequency-deviant stimuli were more similar to the immediately preceding stimulus than the expected standard stimulus at that particular position. The MMN to pattern-deviant stimuli therefore reflects a neural mismatch between the incoming sounds and a representation of a pattern that includes the spectral and temporal relationships among the elements of the pattern. The similarity in MMN topography for frequency- and time-deviant stimuli suggests that auditory patterns may be encoded as a Gestalt defined 0 Vol No October

5 p Auditory pattern and mismatch negativity by the relations among the elements that comprise the pattern rather than by the properties of individual elements of the pattern., This similarity in the MMN for frequency- and time-deviant stimuli differs from previous results. Levanen and colleagues 0 observed different topographies for the MMNm (the magnetic counterpart of the MMN) to frequency- and time-deviant stimuli occurring in a train of identical standard stimuli. The discrepancy between our findings and those of Levanen et al. may be due to the fact that our standard stimuli were not all the same but varied according to a set pattern. Levanen s paradigm may have eased the separate encoding of a specific frequency and a specific timing whereas our study facilitated the encoding of a four stimulus pattern. The MMN wave may reflect the activity of a distributed neural network, including generators located in the auditory and prefrontal cortices. In the current study, the polarity inversion of the MMN recorded over frontal and inferior temporal regions suggests that the superior surface of the temporal lobe generates the MMN wave to pattern-deviant stimuli. Single-cell recordings in cats and non-human primates,, along with neuropsychological studies and scalp recording of event-related brain potentials in humans have shown that the auditory cortex plays a critical role in the discrimination of both auditory features (e.g. pitch) and temporal order. Coupled with our findings, this suggests that auditory pattern memory, upon which the MMN generation depends, may be located in auditory cortices within the Sylvian fissure. A greater magnitude of deviation was associated with both an enhanced amplitude and a more frontal distribution. This finding occurred with both types of deviance. Physiologically, this could represent increased activity in the supratemporal plane with some changes in the orientation of equivalent dipoles or some additional activation in the frontal lobes.,, The frontal activity would occur primarily with large frequency- and time-deviant stimuli because they were more salient than small deviant sounds, as suggested by the better performance during the auditory discrimination task (Table ). The enhanced frontal activity could be related to involuntary attention switching to rare deviant sounds., However, the fact that performance at the visuo-motor task was affected by neither the auditory stimuli nor the occurrence of deviant sounds (Table ) suggests that this call to attention was unheeded. This finding is also consistent with the proposal that the MMN indexes an automatic process that detects changes from what is expected and that this automatic expectancy can be based on complex patterns as well as simple stimulus features. Conclusion Infrequent changes in spectral and temporal transitions between two consecutive stimuli from an ongoing four-tone auditory pattern elicited a reliable MMN. This MMN to a pattern-deviant stimuli is consistent with the proposal that listeners can automatically extract an auditory pattern and encode it into memory. The MMN scalp topographies are consistent with generators in the auditory cortex within the Sylvian fissure. The similarities in scalp topography for the MMN to frequency- and timedeviant stimuli suggests that sequential patterns are encoded into a unified auditory event, regardless of the properties that made up the pattern. References. Bregman AS. Auditory Scene Analysis. The Perceptual Organization of Sounds. London: MIT Press, 0.. Espinoza-Varas B and Watson CS. Perception of complex auditory patterns by humans. In Dooling RJ and Pulse SH, eds. The Comparative Psychology of Audition. Perceiving Complex Sounds. Hillsdale, NJ: Erlbaum, :.. Watson CS, Foyle DC and Kidd GR. J Acoust Soc Am, (0).. McKenna TM, Weinberger NM and Diamond DM. Brain Res, ().. Robin DA, Abbas PJ and Hug LN. J Acoust Soc Am, (0).. Rauschecker JP. Acta Otolaryngol, ().. Wang X, Merzenich MM, Beitel R and Schreiner CE. J Neurophysiol, 0 ().. Näätänen R. Attention and Brain Function Hillsdale, NJ: Erlbaum,.. Alain C, Woods DL and Ogawa KH. NeuroReport, 0 (). 0. Alain C and Woods DL. Psychophysiology, ().. Nordby H, Roth WT and Pfefferbaum A. Psychophysiology, ().. Tervaniemi M, Saarinen J, Paavilainen P et al. Biol Psychology, ().. Tervaniemi M, Maury S and Näätänen R. Neuroreport, ().. Giard MH, Lavikainen J, Reinikainen K. et al. J Cogn Neurosci, ().. Gomes H, Bermstein R, Ritter W et al. Psychophysiology, ().. Berg P and Scherg M. Electroencephalogr Clin Neurophysiol 0, ().. McCarthy G and Wood CC. Electroencephalogr Clin Neurophysiol, 0 0 ().. Tiitinen H, May P, Reinikainen K and Näätänen R. Nature, 0 ().. Warren RM and Bashford JA, Jr. Percept Psychophys, (). 0. Levanen S, Ahonen A, Hari R et al. Cerebr Cortex, ().. Giard MH Perrin F, Echallier JF. et al. Electroencephalogr Clin Neurophysiol, ().. Alain C, Woods DL and Knight RT. Brain Res (In press).. Alho K, Woods DL, Algazi A et al. Electroencephalogr Clin Neurophysiol, ().. Hari R, Hamalainen M, Ilmoniemi R. et al. Neurosci Lett 0, ().. Divenyi PL and Robinson AJ. Brain Lang, 0 ().. Giard M-H, Perrin F, Pernier J and Peronnet F. Psychophysiology, 0 (0).. Perrin F, Pernier F, Bertrand O and Echallier JF. Electroencephalogr Clin Neurophysiol, (). ACKNOWLEDGEMENTS: This study was supported by grants from the National Alliance for Research on Schizophrenia and Depression, the Medical Research Council of Canada and the Natural Sciences and Engineering Research Council of Canada. Thanks to Lori Bernstein, Kimberly Kane, Leun Otten, and Robert West for careful reading of an earlier version of this manuscript. Received August ; accepted August Vol No October

Separate memory-related processing for auditory frequency and patterns

Psychophysiology, 36 ~1999!, 737 744. Cambridge University Press. Printed in the USA. Copyright 1999 Society for Psychophysiological Research Separate memory-related processing for auditory frequency and