Frequency and periodicity are represented in orthogonal maps in the human auditory cortex: evidence from magnetoencephalography

Transcription

1 J Comp Physiol A (1997) 181: 665±676 Ó Springer-Verlag 1997 ORIGINAL PAPER G. Langner á M. Sams á P. Heil á H. Schulze Frequency and periodicity are represented in orthogonal maps in the human auditory cortex: evidence from magnetoencephalography Accepted: 25 July 1997 Abstract Timbre and pitch are two independent perceptual qualities of sounds closely related to the spectral envelope and to the fundamental frequency of periodic temporal envelope uctuations, respectively. To a rst approximation, the spectral and temporal tuning properties of neurons in the auditory midbrain of various animals are independent, with layouts of these tuning properties in approximately orthogonal tonotopic and periodotopic maps. For the rst time we demonstrate by means of magnetoencephalography a periodotopic organization of the human auditory cortex and analyse its spatial relationship to the tonotopic organization by using a range of stimuli with di erent temporal envelope uctuations and spectra and a magnetometer providing high spatial resolution. We demonstrate an orthogonal arrangement of tonotopic and periodotopic gradients. Our results are in line with the organization of such maps in animals and closely match the perceptual orthogonality of timbre and pitch in humans. Key words Periodicity pitch á Tonotopy á Auditory cortex á Magnetoencephalography á Neuronal map Abbreviations AC auditory cortex á BMF best modulation frequency á CF characteristic frequency á ECD equivalent current dipole á f 0 fundamental frequency á MEG magnetoencephalography á SF sustained eld G. Langner (&) á P. Heil á H. Schulze Institute of Zoology, Technical University Darmstadt, Schnittspahnstr. 3, D Darmstadt, Germany Fax: /164808, gl@neuro.bio.tu-darmstadt.de M. Sams Low Temperature Laboratory, Helsinki University of Technology, Finland Introduction The relationships between frequency, periodicity, pitch, and timbre Acoustic signals used by humans and animals for communication are often harmonic in structure. A harmonic signal is characterized by a temporal envelope which uctuates periodically with a period given by the reciprocal of the fundamental frequency (f 0 ), and a spectrum composed of frequencies which are integer multiples of f 0. Any portion of the spectrum which contains at least two adjacent harmonics has the same temporal envelope period and, in a human listener, elicits the same pitch as the fundamental component alone. This percept is referred to as Ôperiodicity pitch' or, because the fundamental component is not required to elicit that pitch, the percept of the Ômissing fundamental' (Schouten 1970). The pitch of a harmonic signal can be varied, by varying its temporal envelope period without varying its spectral envelope or bandwidth ± two parameters essential for the timbre of a sound (Fig. 1). Conversely, timbre can be varied by varying the spectral envelope without varying the temporal envelope period, i.e. without varying pitch (Fig. 1). For example, the timbre of a pure tone di ers from the timbre of a harmonic sound, but it elicits the same pitch provided it has the same period as the temporal envelope of the harmonic sound (Fig. 1). Likewise, di erent musical instruments (or voices) can di er in timbre when producing the same note. In that sense pitch and timbre are largely independent and may be considered as orthogonal perceptual parameters related to temporal envelope periodicity and spectral content, respectively. The independence of pitch and timbre was demonstrated in psychophysical experiments (Plomp and Steeneken 1971; Krumhansl and Iverson 1992), and pitch and timbre were shown to compete in streaming experiments (Singh 1987).

2 666 coincidence units are activated (Langner 1981, 1983, 1992). Moreover, as found in the midbrain of cat and chinchilla, neurons with di erent BMFs are spatially mapped in an orderly fashion, such that the gradients of BMF (periodotopy) and CF (tonotopy) are roughly orthogonal to each other (Schreiner and Langner 1988; Langner et al. 1992). Representation of frequency and periodicity in the auditory cortex Fig. 1 Pitch and timbre are largely independent percepts. Signals with the same temporal envelope period have the same pitch, but they may have di erent spectral envelopes and therefore di erent timbres. Signals with di erent periods elicit a di erent pitch, while their spectral envelopes (and their timbre) may be similar Representation of frequency and periodicity in the auditory brainstem The independence of pitch and timbre is in line with response properties of neurons in the brainstem of various animals. In addition to their frequency tuning neurons are often tuned to particular modulation frequencies of temporal envelopes (for review see Langner 1992). Neurons which are tuned to a certain characteristic frequency (CF) may prefer di erent temporal envelope modulations, e.g. they may have di erent best modulation frequency (BMF). To a rst approximation their spectral and temporal tuning properties are independent. In the cochlear nucleus of various animals, the tuning of neurons to temporal envelope modulations is evident mainly with respect to the synchronization of their discharges to the modulation (Frisina et al. 1990; Rhode and Greenberg 1994). In the auditory midbrain, synchronization is relatively poor, especially to high modulation frequencies (>100 Hz, corresponding to modulation periods<10 ms). However, neurons can be tuned to such high modulation frequencies with respect to their average discharge rate (Langner 1981, 1983, 1992; Rees and Mùller 1983; Rose and Capranica 1985; Langner and Schreiner 1988; Heil et al. 1995). In a theory of temporal correlation analysis neurons in the auditory midbrain are considered as coincidence detectors. Action potentials at their inputs represent delayed and undelayed neuronal responses to the envelope of periodic signals. When these di erent neuronal delays are compensated by the period of the signal the Orthogonality of neuronal representations of frequency and envelope periodicity has been described in the auditory cortex (AC) analogue of the Mynah bird (Hose et al. 1987). In contrast, Pantev et al. (1989), using magnetoencephalography (MEG) in humans, found that the locations of the magnetic eld generators within the supratemporal cortex that were activated with a harmonic sound of missing fundamental and with a pure tone of a frequency corresponding to that fundamental were in close register. These authors concluded that ``... the tonotopic organization of the primary auditory cortex re ects the pitch rather than the frequency of the stimulus''. This conclusion appears unsatisfactory, as it is di cult to reconcile such an organization with the perceptual orthogonality of pitch and timbre, as outlined above. Such an organization would also be at variance with the quasiorthogonal arrangement of periodotopy and tonotopy found in animals. However, the stimulus set used by Pantev et al. (1989), viz. only two pure tones and one harmonic sound, may have been too restricted to reveal the spatial representation of frequency and temporal envelope period. We therefore attempted to reinvestigate this issue using a more extensive set of stimuli, viz. up to six pure tones and six harmonic sounds, and a magnetometer with high spatial resolution, hoping that these conditions would be suited to resolve the discrepancy between the nearly orthogonal arrangement of tonotopy and periodotopy in the auditory midbrain and forebrain of animals and their proposed parallel arrangement in the human AC. Materials and methods Acoustic stimuli Acoustic stimuli were pure tones and harmonic sounds. Their spectra are schematically illustrated in Fig. 2. Pure-tone frequencies were 50 Hz, 100 Hz, 200 Hz, 400 Hz, and, in some experiments, also 800 Hz and 1600 Hz. In the gures they are referred to as s50, s100, etc. (Fig. 2, left column). Harmonic sounds (Fig. 2, centre and right columns) were composed of harmonics of 50 Hz, 100 Hz, 200 Hz, and 400 Hz and thus elicited a pitch corresponding to these fundamental frequencies. All harmonic sounds had an upper cut-o frequency of 5 khz, but the lower cut-o frequency was either 400 Hz (Fig. 2, centre) or 800 Hz (Fig. 2, right). Signals with 400 Hz cut-o are referred to as p50, p100, etc., and those with 800 Hz cut-o as h100 and h200 (h50 and h400 were

3 667 Fig. 2 Schematic spectra of pure tones and harmonic signals used as stimuli. The high cut-o frequency was 5000 Hz for all harmonic sounds, while the low cut-o frequency was 400 Hz for p50, p100, p200, and p400 and 800 Hz for h100 and h200. The only harmonic stimulus of this set which includes its fundamental frequency is p400. The spectra do not show the roll-o of the sound transmission not used). Signals with the same cut-o frequency have very similar spectral envelopes. Thus, pure tones and harmonic signals labelled with the same number (signals within a given row in Fig. 2) have the same temporal envelope period and elicit the same pitch. Signals labelled with the same letter (signals within a given column in Fig. 2) have the same spectral envelope and elicit a similar timbre. All stimuli had durations of 500 ms, including linear rise and fall times of 10 ms, and were presented with interstimulus intervals of 1.2 s. Stimulus presentation Six subjects (one female, ve males) participated in the experiments. In early experiments, pure tones (four subjects) and harmonic sounds (six subjects) were presented in separate experimental sessions. Within each session stimuli were presented in random order. In later experiments (four subjects), each harmonic sound was followed by a pure tone with the same pitch, and stimulus pairs of di erent pitch were presented in random order. The sounds were produced by equipment located outside the magnetically shielded room inside which the subjects were seated during the measurements. Sounds were led to the subjects' ears via plastic tubes of several meters in length and specially designed ear pieces. The transfer function of this system was at from 1 khz to 2 khz and fell o by 15 db/octave below 1 khz and by 20 db/octave above 2 khz. Before an experiment, each subject adjusted loudness levels of all stimuli to be equal (60 db HL). Prior to magnetoencephalographic recording subjects were asked to remember the highest number of consecutive signals (or signal pairs) with the same pitch in order to keep their attention high during the recording session. One to ve independent measurements were performed per subject and within each session each stimulus was repeated about 200 times. The duration of a session was about 45 min. Magnetoencephalographic recording Auditory-evoked magnetic elds were recorded with a 122-channel magnetometer, covering the whole head (Ahonen et al. 1993; HaÈ - maè laè inen et al. 1993; MaÈ kelaè et al. 1993). Location of the head with respect to the SQUID sensors was determined by measuring the magnetic elds produced by small currents delivered to three coils attached to the scalp. The location of the coils on the preauricular points and the nasion was measured with a 3D-digitizer. The recording bandpass was 0.03±100 Hz (3-dB points; high-pass roll-o 35 db/decade, low-pass over 80 db/decade) with a sampling rate of 0.4 khz. Trials contaminated by eye blinks were rejected. Data analysis After digital low-pass ltering at 40 Hz equivalent current dipoles (ECDs) that best described the measured magnetic eld at a given latency were found by least-squares t in a spherical volume conductor (Kaukoranta et al. 1986). We used a one-dipole model, separately for the left and right hemisphere, with a subset of channels over each hemisphere. Such a one-dipole approximation is only a rst approximation, since in each hemisphere multiple sources may be expected to be active at most times (for example in di erent cytoarchitectonic areas of the supratemporal AC). However, there is no reasonable estimate of how many sources should be assumed to be active at a given time. Because it has previously been shown that the one-dipole model provides a very good approximation of the measured magnetic eld during short time segments (for review see Hari 1990), we concentrated our analysis of ECD locations on those short-time segments during which the measured magnetic eld was best approximated by the one-dipole model. In order to make our interpretation of the data as demonstrated in the gures more comprehensible we added ellipses (Fig. 5), polygons (Fig. 6) and arrows or dotted lines (Figs. 5±10), all of them matched by eye. Results Basic observations Typically, the computed time-course of the ECD magnitude and the corresponding goodness-of- t of the onedipole model showed peaks around 100 ms and 200 ms after stimulus onset followed by sustained plateaus. These de ections are referred to as M100, M200, and SF (sustained eld), respectively. Figure 3 illustrates an example with additional peaks around 60 ms (M60) and 150 ms (M150) which were seen in some subjects. The precise latency of the prominent de ections was stimulus dependent (see also Forss et al. 1993; Roberts and Poeppel 1996). Figure 4 shows the latency of the peak of M100 from the right hemisphere of all subjects which were stimulated with harmonic sounds of 400 Hz cut-o plotted against the envelope period. The contin-

4 668 Fig. 3 Time-course of the magnitude of an equivalent current dipole (ECD), computed for consecutive 2-ms intervals from the magnetic elds measured over the left hemisphere of one subject. The stimulus was a harmonic signal (p100) with a duration of 500 ms which started at t = 10 ms. Acoustic delays have been subtracted. The goodness-of- t (g) between the eld predicted by the one-dipole model and the original data is also shown. Typically, peaks of the ECD magnitude and g were observed at latencies corresponding to peak de ections of the averaged original MEG signal uous line without symbols in Fig. 4 represents the result of a linear regression analysis: latency M100 = (89 3) ms + ( ) á period (ms); r = and P < The location of the ECD in each hemisphere was not stable as a function of time. Two representative examples from di erent hemispheres of two subjects are shown in Fig. 5a, b. The data in this gure and most of the following gures are presented as viewed from a lateral perspective. Three-dimensional data analyses are presented in Figs. 11, 12, and 13. During certain time Fig. 4 Latency of the peak of M100 plotted as a function of temporal envelope period of harmonic signals with a cut-o frequency of 400 Hz. Results were obtained from the right hemisphere of six subjects stimulated through the contralateral ear. The results of a linear regression analysis (continuous line without symbols) are also shown Fig. 5a, b ECD location as a function of time after stimulus onset (stimulus: p400; time steps: 5 ms) and as seen from a lateral perspective. Data are from the right (a) and left auditory cortices (b) of two subjects. Note that di erent regions (elliptical boundaries) seem to be active at di erent delays with the momentary maximum of activity jumping rapidly between those regions (indicated by arrows)

5 669 segments after stimulus onset the ECD is located within rather restricted cortical areas (elliptical boundaries in Fig. 5) and the ECD location changes relatively little as a function of time. At other times, however, there appear to be rapid (i.e. often within <5 ms) and pronounced changes in ECD location from one restricted cortical area to another or back and forth between two such areas. However, the goodness-of- t during such rapid changes in ECD location was very small, preventing a functional interpretation. Figure 6 shows for the right hemisphere of one subject the ECD locations during short time segments around M100 and M200 and during SF where the goodness-of- t was maximal (80±98%), i.e. around the peaks of the ECD magnitude. Each data point represents 2 ms. The ECD trajectories are viewed from a lateral perspective and their approximate position in relation to the whole head is indicated by the inset in Fig. 6a, b Lateral view of ECD trajectories in the right hemisphere of one subject for stimulation with pure tones (a) and harmonic sounds (b). The labels next to the trajectories identify the stimuli (see Materials and methods and cf. Fig. 2). Each data point designates the location of the ECD during a 2-ms interval within time segments of maximal goodness-of- t. The selected segments had durations of 10 ms for M100, of 20 ms for M200, and of 30 ms for SF. The arrows indicate the systematic spatial progressions of the ECD trajectories with frequency for pure tones and with fundamental frequency for harmonic sounds during M100 and during SF. A systematic progression during M200 is questionable. The positions of M100, M200, and SF relative to one another were similar in most subjects and in both hemispheres. For a better comprehension, their positions in relation to a whole head are schematically indicated in the insert in a Fig. 6a. The stimuli were four pure tones (a) and four harmonic sounds (b). Note that the ECD trajectories of M100, M200, and SF are spatially segregated. The ECD trajectories of M200 are located most anteriorly, as has been described earlier (e.g. Pantev et al. 1989; Hari 1990), and those of SF are located slightly more anterior than those of M100, corroborating more recent results of Pantev et al. (1994). Also note that for each de ection (M100, M200, SF) the ECDs obtained with pure tones and with harmonic sounds are localized in approximately equivalent cortical areas. However, there are systematic detailed di erences which are addressed further below. Tonotopy Previous MEG studies have demonstrated that the tonotopic gradient obtained for M100 and for SF runs mainly along the lateromedial axis, thus approximately perpendicular to the projection plane of Figs. 6±10, with more lateral ECD locations for low frequencies and more medial locations for high frequencies (e.g. Romani et al. 1982; Pantev et al. 1988, 1989, 1994, 1995; Cansino et al. 1994). However, the tonotopic gradient for M100 has a signi cant component also in the tangential plane, so that a gradient should also be visible from a lateral perspective. Indeed, in several cases a tonotopic order of ECD trajectories obtained with pure tones of di erent frequencies was evident in a lateral view. In the case illustrated in Fig. 6a, for example, the ECD trajectories during M100 shifted with increasing frequency from a more posterior and superior location to a more anterior and inferior location (arrow in M100 box in Fig. 6a; see also Fig. 10), a shift in line with results of Pantev et al. (1994, 1995). A similar shift of ECD trajectories with frequency, in this case from posterior-inferior to anterior-superior, was seen during SF (arrow in SF box in Fig. 6a). While during M200 the positional changes of ECD locations for s100, s200, and s400 seem to indicate a gradient mainly from superior to anterior, this gradient is not corroborated by the ECD position for s50. The three-dimensional orientation of the tonotopic gradient during M100 was analysed in the following way and the results are demonstrated in Figs. 12 and 13: for each 10-ms ECD trajectory with a maximal goodness-of- t the centre of gravity was rst computed. Then, the distances in three-dimensional space of the centres for pure-tone stimuli of neighbouring frequencies were computed. Figure 12a shows, for all four subjects for which data were available, cumulative distances of these centres of gravity in the left hemisphere plotted against pure-tone frequency. Note that the frequency axis in Fig. 12a is logarithmic and that the cumulative distances increase roughly linearly with the logarithm of stimulus frequency. The thick solid line in Fig. 12a represents the result of a linear regression analysis between the logarithm to the base 2 of stimulus frequency and the cumulative distance. This function indicates an ap-

6 670 proximately equal spacing of octaves in the human AC, a result also in line with those of previous studies (e.g. Romani et al. 1982; Pantev et al. 1989, 1994, 1995; Cansino et al. 1994). The thick broken line in Fig. 12a represents the component of the average cumulative distance only along the lateromedial or depth coordinate. The proximity of this line to the individual data and their t reveals that the depth coordinate is a major contributor to the cumulative three-dimensional distances of the data. The thin broken line in Fig. 12a represents the component of the average cumulative distance within the tangential or lateral surface plane. This line is not quite as close to the data as the thick broken line. Nevertheless, the contribution of the tangential plane to the cumulative three-dimensional distances is signi cant. In other words, frequency is represented along the depth coordinate, with low frequencies more lateral and high frequencies more medial, and also within the tangential dimension, with low frequencies more superior and posterior and high frequencies more anterior and inferior. The average three-dimensional orientation of the tonotopic gradient, from low to high frequencies, and the spatial frequency resolution that emerged from this analysis, are represented by the orientation and length of a vector in Fig. 13. The top and middle panels present a lateral and a top view, respectively. The lateral view can conveniently be compared with the raw data (e.g. Figs. 6a, 10). Periodotopy The ECD trajectories during M100 and during SF obtained with harmonic sounds of di erent fundamental frequency also showed a topographic order that was evident in a lateral perspective. Examples of periodotopic organizations during M100 are illustrated in Figs. 6b, 8, and 10, and during SF in Fig. 7. In each case, the approximate course of the periodotopic gradient is indicated by an arrow. The periodotopic gradients obtained in di erent subjects di er somewhat, but in general seem to run along a posterior and inferior to anterior and superior axis during M100 (Figs. 6b, 8, 10), and along a posterior and more superior to anterior and more inferior axis during SF (Fig. 7). The reproducibility of the measurements is indicated by Fig. 7a, b, which shows results from two measurements made on the same subject (PH) several months apart. Note that the obtained periodotopic gradients were quite similar, viz. were oriented from posteriorsuperior to anterior-inferior (see arrows). Furthermore, absolute coordinates obtained in di erent measurements and subjects de ned from external head coordinates are quite similar. The average distance of the centres of gravity of ECD trajectories for corresponding stimuli in the three di erent experiments illustrated in Fig. 7 is mm, only about twice the average distance Fig. 7a±c Lateral view of the ECD locations during SF (~380 ms). For each stimulus, the ECD trajectory represents a 20-ms segment (time interval: 2 ms). The results were obtained from the right hemisphere of two subjects with (a) being a repetition of (b) several months later. The approximate orientations of the periodotopic gradients are indicated by arrows between centres of gravity for neighbouring stimuli obtained in the same experiment (viz., mm). Orthogonality of periodotopy and tonotopy In a given subject there are obvious di erences in the orientations of the tonotopic and periodotopic gradients for corresponding de ections, as seen from the lateral perspective (cf. Fig. 6a and 6b). To investigate this issue further, ve subjects were stimulated with harmonic signals di ering in f 0 (hence in pitch) and in the low cuto frequency of the spectrum. The lower cut-o frequency was 400 Hz for p50, p100, p200, and p400, and 800 Hz for h100 and h200 (Fig. 2). This di erence in

7 671 Fig. 9 Lateral view of ECD trajectories during M60 (10-ms segments; time intervals of 2 ms) in the left hemisphere of subject HS. Note the di erent orientations of the tonotopic and periodotopic gradients (dashed lines). Also note that the trajectories for the two harmonic stimuli with a cut-o frequency of 800 Hz (viz. h100 and h200) are displaced relative to those with a cut-o frequency of 400 Hz (viz. p50, p100, p200, p400) in the direction of the tonotopic gradient. Note the grid-like regular spacing of ECD trajectories, especially for harmonic signals Fig. 8a±c Lateral view of the ECD trajectories during M100 in the left hemisphere of two subjects (a and c) and in the right hemisphere of a third subject (b). Subjects were stimulated with six harmonic sounds di ering in fundamental frequency and in low cut-o frequency which was 400 Hz for p50, p100, p200, and p400, and 800 Hz for h100 and h200 (see also Fig. 2 for key to the stimuli). For each stimulus, the ECD trajectory represents a 10-ms segment (time interval: 2 ms). The approximate orientations of the periodotopic gradients are indicated by arrows with continuous lines while the approximate displacements of h100 and h200 (owing to their higher cut-o frequency) are indicated by arrows with dotted lines the lower spectral boundary also constitutes a shift of the centre of gravity of the stimulus spectrum along the frequency axis. We therefore expected to obtain di erences in the ECD locations between p- and h-stimuli that should indicate the tonotopic gradient. In addition, stimuli with identical cut-o frequency, but with di erent fundamental frequency, should reveal the periodotopic gradient. Figure 8 shows the results obtained for M100 from the left hemisphere of two subjects (Fig. 8a, c) and from the right hemisphere of a third subject (Fig. 8b). As noted above, the periodotopic gradients, revealed by stimuli of di erent fundamental frequency and similar spectral envelope, are oriented roughly from posterior to anterior (as suggested by the arrows with continuous lines). ECD trajectories for stimuli of di erent cut-o frequency are displaced. In other words, stimuli eliciting the same pitch are not colocalized. As suggested by the arrows with dotted lines, the displacement is roughly along the superior to inferior axis and within, approximately orthogonal to the periodotopic gradient (see also Fig. 10). A similar result was obtained for M60 in subjects with a prominent de ection at this latency. Data from one subject (HS) are illustrated in Fig. 9. Note the gridlike arrangement of ECDs suggesting a regular representation of signal parameters along two axes as well as equal distances for ECDs elicited by harmonic stimuli separated by octaves. Note also that during M60 the periodotopic gradient and the displacement of p- and h-stimuli are reversed compared to those during M100 (cf. Figs. 8 and 9). The reliability of these measurements is again emphasized by the similarities in the absolute coordinates of ECD trajectories. The average di erences in threedimensional space between the centres of gravity of ECD trajectories for corresponding stimuli in di erent subjects was mm, less than twice the average distance between neighbouring stimuli in the same subject (viz., mm).

8 672 It may be noteworthy that in some cases the ECD trajectories for p400, the only harmonic stimulus whose spectrum included the f 0, showed peculiarities. For example, in subject GL (Fig. 8a) the position of the ECD trajectory for p400 would seem more appropriate for h400, a stimulus which was not tested. Since p400 is a combination of h400 and the fundamental frequency of 400 Hz (see Materials and methods), p400 would be expected to elicit the same response as h400, provided the subject did not hear or was able to Ôignore or suppress' the fundamental component. In subject PH (Fig. 7a, b; Fig. 6b) the orientation of the ECD trajectory for p400 is markedly di erent from the orientation of the trajectories for the other stimuli. Evidence for orthogonality from experiments using harmonic and pure tones in the same session As noted above, the relative displacement of ECD trajectories for stimuli of di erent cut-o frequency was roughly orthogonal to the periodotopic gradient. More direct evidence for an approximately orthogonal organization of tonotopy and periodotopy comes from the analysis of the locations of ECD trajectories for harmonic and pure tone stimuli which were presented to four subjects as stimulus pairs in the same experimental session. Figure 9 presents such data for M60. In this lateral view the tonotopic gradient, revealed by the ECD trajectories for pure-tone stimuli of di erent frequency, runs approximately from inferior to superior, while the periodotopic gradient, revealed by the ECD trajectories for harmonic stimuli of di erent f 0 runs approximately from anterior to posterior. While in this lateral view tonotopic and periodotopic gradients (dotted lines in Fig. 9) are not orthogonal to each other, it is important to stress that they are clearly not parallel. The displacement to more superior loci of ECD trajectories for harmonic stimuli of higher (h100, h200) relative to those of lower cut-o frequency (p50, p100, etc.) is approximately in the direction that is expected from the orientation of the tonotopic gradient. Note that the ECD trajectories for the harmonic stimuli with a cut-o frequency of 400 Hz (viz. p50, p100, etc.) are located between the ECD trajectories for pure-tone stimuli of 400 and 800 Hz, while harmonic stimuli with a cut-o frequency of 800 Hz (h100 and h200) are located between pure tones of 800 and 1600 Hz. For harmonic stimuli of di erent fundamental frequencies the ECD trajectories are regularly spaced with an average of 6 mm/octave and for pure tones of di erent frequencies with an average of 8 mm/octave. The cut-o frequencies of p100 and h100, as well as of p200 and h200 and h200 (400 Hz and 800 Hz, respectively) are also separated by one octave, however, in this case along the frequency axis. Accordingly, the corresponding ECDs are separated by about 10 mm in the direction of the tonotopic gradient. Corresponding data for M100, from a di erent subject and hemisphere, are illustrated in Fig. 10. In this case and in this lateral view, tonotopic and periodotopic gradients are roughly orthogonal and, as during M60, the displacement of ECD trajectories for harmonic stimuli of higher (h100, h200) relative to those of lower cut-o frequency (p50, p100, etc.) is again approximately in the direction expected from the orientation of the tonotopic gradient. Note that the tonotopic and periodotopic gradients run in directions opposite to those of the corresponding gradients during M60 (cf. Figs. 9 and 10). Three-dimensional analysis of ECD positions The most direct evidence for an orthogonal organization of tonotopy and periodotopy comes from analysis of the three-dimensional spatial relationships between ECD trajectories for pure-tone and harmonic stimuli in all four subjects which were stimulated with both types of signals in the same experimental session. The analysis was performed in the following way. We rst calculated the centre of gravity of the ECD trajectory for each stimulus in a given subject and hemisphere during M100 and, where possible, during M60. Next, we obtained the vector between centres of gravity for pure-tone and for harmonic stimuli that are neighbours along the dimen- Fig. 10 Lateral view of ECD trajectories during M100 (10-ms segments; time intervals of 2 ms) in the right hemisphere of subject PH. Note the quasi-orthogonal orientations of the tonotopic and periodotopic gradients (dashed lines). Also note that the trajectories for the two harmonic stimuli with a cut-o frequency of 800 Hz (viz. h100 and h200) are displaced relative to those with a cut-o frequency of 400 Hz (viz. p50, p100, p200, p400) approximately in the direction of the tonotopic gradient

9 673 sion of frequency and fundamental frequency, respectively, i.e. the vector between s50 and s100 (s50 s100), between s100 and s200 (s100 s200), etc., between h100 and h200 (h100 h200), between p50 and p100 (p50 p100), between p100 and p200 (p100 p200), and between p200 and p400 (p200 p400). Finally, we calculated the angles formed by all vectors between two neighbouring pure-tone stimuli with all vectors between two neighbouring harmonic stimuli (e.g. the angles formed by s200 s400 with p50 p100, with p100 p200, etc.). Figure 11 shows the frequency distributions of these angles, with a bin width of 15 degrees, for M100 in the left and right hemispheres and for M60 in the left hemisphere. The number of M60 data with su cient goodness of t for the right hemisphere was too limited for an evaluation. The angles were not evenly distributed. Rather, the distributions peak near 90 degrees, as identi ed in Fig. 11. These values emphasize that, on average, tonotopic and periodotopic gradients are oriented nearly orthogonally to one other. Another independent demonstration of the orthogonality of tonotopy and periodotopy comes from a comparison of the average tonotopic and periodotopic vectors in three-dimensional space. Figure 12b plots the cumulative three-dimensional distances of the centres of gravity of ECD trajectories during M100 for neighbouring harmonic stimuli, in a fashion analogous to that described above for pure-tone stimuli (see Fig. 12a). The periodotopic data in Fig. 12b are from the same subjects and hemispheres that provided the tonotopic data in Fig. 12a and the angle distribution of Fig. 11a, and, in addition, from two subjects (SA and SL) who were stimulated with harmonic signals only. Although the cumulative distances of the centres of gravity for harmonic signals scatter more widely among subjects than those for pure tones (cf. Figs. 12a and 12b), they also increase roughly linearly with the logarithm of f 0 (thick solid line in Fig. 12b). Thus, octaves of f 0 are approximately equally spaced in the human AC. However, the spatial resolution for fundamental frequency (viz. 5.5 mm/octave) is less than that for pure tone frequency (viz. 7.1 mm/octave). The thick broken line in Fig. 12b represents the component of the average cumulative distance along the lateromedial or depth coordinate only. Note that this line deviates considerably from the individual data, thus the depth coordinate is not the major contributor to the cumulative three-dimensional distances of the periodotopic data. In contrast, the thin broken line, which represents the component of the average cumulative distance within the tangential or lateral surface plane, is in rather close register with the data. Thus, the tangential plane contributes most to the cumulative threedimensional distances of the ECD centres of gravity for harmonic stimuli. To allow the comparison with the tonotopic vector, the average three-dimensional orientation of the periodotopic gradient, from low to high fundamental frequencies, and the spatial frequency resolution that Fig. 11a±c Distributions of angles (binwidth: 15 degrees) formed by tonotopic and periodotopic gradients during M100 in the left (a) and right hemisphere (b) and during M60 in the left hemisphere (c). Each entry is the angle formed by a vector between the centres of gravity of ECD trajectories for two pure-tone stimuli of neighbouring frequency and a vector between the centres of gravity of ECD trajectories for two harmonic stimuli of neighbouring fundamental frequency, as described in detail in the Results. The mean angles, the standard deviations, the vector strengths, and the likelihoods of the distributions to be random, as obtained by circular statistics, are also identi ed emerged from this analysis, are also represented by the orientation and length of a vector in Fig. 13. The two vectors form an angle of 95 degrees when viewed from the lateral perspective (Fig. 13a), and an angle of 89 degrees when viewed from the top perspective (Fig. 13b). However, to derive the average spatial organization, the two vectors should be viewed from an

10 674 Fig. 12a, b Cumulative three-dimensional distances in the left hemisphere between the centres of gravity of ECD trajectories during M100 as a function of frequency for pure-tone stimuli (a) and of fundamental frequency for harmonic stimuli (b). Di erent symbols represent di erent subjects. The thick continuous lines represent the results of linear regression analyses between cumulative distance (cd) and the logarithm to the base 2 of frequency and fundamental frequency (1d f). These results suggest a 30% greater spacing for tonotopy than for periodotopy (7.1 versus 5.5 mm/octave). The thick broken lines represent the components of the distances along the lateromedial or depth coordinate and the thin broken lines within the tangential or surface plane only optimal perspective, i.e. by looking down onto that plane in which both vectors have their maximal length. In this plane, the tonotopic vector has a length corresponding to a frequency resolution of 7.1 mm/octave and the periodotopic vector a length corresponding to a resolution of 5.5 mm/octave, i.e. the values derived from the cumulative three-dimensional distances of the ECD centres of gravity (cf. Fig. 12). This optimal plane is tilted relative to a horizontal plane and points downward in the anterior and medial directions. Figure 13c shows that in this optimal plane, periodotopic and tonotopic vectors form an angle of 85 degrees, i.e. an angle again close to 90 degrees. Fig. 13 Average orientation and spatial resolution of tonotopic and periodotopic gradients during M100 represented by the orientation and length of two vectors. The radius of each circle represents 7.1 mm/octave. The data are shown from a lateral view (top), the same perspective as used for Figs. 5±10, from a top view (middle), and from an optimal view (bottom), i.e. by looking down onto that plane in which both vectors have their maximal length. This optimal plane is tilted relative to a horizontal plane and points downward in the anterior and medial directions. The angle formed by the average tonotopic and periodotopic vectors is 85 degrees (bottom panel), but appears slightly di erent when viewed from a lateral or top perspective, viz. 95 (top) and 89 degrees (middle), respectively Discussion The present study has demonstrated with the MEG technique the existence of a tonotopic and a periodotopic organization of the human AC and, most importantly, an approximately orthogonal arrangement of tonotopic and periodotopic gradients.

11 675 The tonotopic organization, which was analysed here in most detail for the generators of M100 (Figs. 6a, 10, 12a, 13), compares well with that reported in previous studies (e.g. Romani et al. 1982; Pantev et al. 1989, 1994, 1995; Lie geois-chauvel et al. 1994; Cansino et al. 1994). In agreement with our results, the tonotopic gradient during M100 has been reported to be oriented along the depth coordinate with low frequencies being represented more laterally and high frequencies more medially (Romani et al. 1982; Pantev et al. 1989, 1994, 1995). These studies also document a component of the tonotopic gradient in the tangential plane with low frequencies being represented more posteriorly and high frequencies more anteriorly. Likewise, the above studies and ours agree in that there is an approximately equal spacing of octaves (Figs. 6a, 10, 12a). Our cumulative distances suggest an average spatial resolution of 7.1 mm/octave. The values that can be derived from other studies vary considerably. The data of Cansino et al. (1994; their Fig. 9) reveal three-dimensional resolutions of about 4±7 mm/octave in di erent individuals, those of Romani et al. (1982; their Fig. 4) average resolutions of about 4 mm/octave, and those of Pantev et al. (1994; their Fig. 5) some 2±3 mm/octave. Whatever the source of this variability, it is important to note that these studies, while they disagree with respect to absolute distances, do agree with respect to the relative spatial relationships of ECD locations for di erent stimuli. Furthermore, they seem to agree with respect to the relative spatial relationships of ECD locations for di erent de ections of the response, e.g. the anterior location of M200 relative to M100, of SF relative to M100, etc. (see e.g. Pantev et al. 1989; Hari 1990; McEvoy et al. 1994). The periodotopic organization of the human AC, demonstrated here for M60 (Figs. 9, 13c) SF (Figs. 6a, 10, 12a), and in most detail for M100, con rms our preliminary results reported elsewhere (Langner et al. 1994). The nearly orthogonal arrangement of the tonotopic and the periodotopic gradients, again documented here in most detail for M100 (Figs. 6, 8, 10, 12, 13), constitutes the central nding of the present study. This nding is radically opposed to the conclusion drawn by Pantev et al. (1989) from their MEG study in humans, viz. that ``... the spatial arrangement of the generators of the magnetic wave M100 re ects the pitch rather than the spectral contents of the stimulus'', i.e. in e ect the conclusion that tonotopy and periodotopy are organized in parallel. We believe that these opposing conclusions originate from the design and selection of stimuli. Pantev et al. (1989) used only two pure-tone stimuli with frequencies of 250 and 1000 Hz and only one harmonic signal, composed of the fourth to eighth harmonic of the 250- Hz f 0, i.e. of 1000 Hz, 1250 Hz, 1500 Hz, and 1750 Hz. Clearly, the selection of only a single harmonic stimulus prevents the demonstration of any periodotopic gradient. Furthermore, this harmonic signal was presented together with a (continuous) narrow-band noise centred at the f 0. This may explain why that the ECD evoked by this stimulus was located closer to the ECD evoked by the 250-Hz tone than to that evoked by the 1000-Hz tone. The demonstration of an orthogonal arrangement of tonotopic and periodotopic gradients in the present study was achieved by selecting stimuli that allowed the variation of temporal envelope period without varying the spectral envelope and vice versa (see Fig. 2). The orthogonality could easily be missed by the use of stimuli that are designed such that temporal envelope period and spectral envelope vary concomitantly. The orthogonal arrangement agrees well with results of single- and multi-unit mapping studies of tonotopy and periodotopy in the auditory midbrain and forebrain of mammals and birds, which have demonstrated nearly orthogonal maps of CF and BMF (Hose et al. 1987; Schreiner and Langner 1988; Langner et al. 1992). It was shown that a temporal analysis of periodicity analysis is underlying the spatial representation of periodicity orthogonal to the tonotopic map in the midbrain (Langner and Schreiner 1988). The similarity of representations in the midbrain of animals and in the human cortex may suggest also similar mechanisms of pitch perception. The conclusion would be that one dimension of the cortical map represents the result of the cochlear frequency analysis and corresponds to the perceptual quality of timbre, while the other dimension, orthogonal to the frequency axis, represents the result of the temporal periodicity analysis in the brainstem and corresponds to the perceptual quality of pitch. As a result, signals with the same timbre (e.g. two vowels [a:]) are represented at di erent locations in this map, provided they di er in pitch (e.g. when uttered by a male subject and a female subject). The orthogonality of tonotopy and periodotopy, as well as their relative spatial resolutions, obtained from our results in the human AC, are also in line with the perceptual independence of pitch and timbre as described by Plomp and Steeneken (1971). In fact, the correspondence is most remarkable. These authors presented subjects with stimuli of ltered periodic pulses. Stimuli di ered in repetition rate (200±640 Hz), i.e. in pitch, and in the centre frequency of the 1/3-octave bandpass lter (2000±6400 Hz). Subjects were asked to judge and rank the similarity of signals presented in triads. The multi-dimensional scaling analysis of their data revealed an orthogonal organization of pulse repetition rate and centre frequency, i.e. a perceptual orthogonality of pitch and timbre. Furthermore, the perceptual distances of stimuli di ering in centre frequency were about 1.6 times as large as those of stimuli di ering in repetition rate. This compares remarkably well with the ratio of 1.3 between the spatial resolutions for frequency and fundamental frequency (viz. 7.1 mm/ octave divided by 5.5 mm/octave) obtained in the present study. This close correspondence may constitute another example of the match of behavioural discriminability of a stimulus parameter and the scaling of its

12 676 cortical representation, well known from the somatosensory and visual systems, as well as from the auditory system, particularly of echolocating bats (Suga et al. 1987). Acknowledgements This study was supported by the Academy of Finland, the Deutsche Forschungsgemeinschaft, and the Human Frontier Science Program Organization. We are grateful to Prof. R. Hari and many colleagues at the Low Temperature Laboratory of the Helsinki University of Technology for making these experiments possible. References Ahonen A, HaÈ maè laè inen M, Kajola MJ, Knuutila JET, Laine PP, Lounasmaa OV, Parkkonen LT, Simola JT, Tesche CD (1993) 122-Channel SQUID instrument for investigating the magnetic signals from the human brain. Physica Script T49: 198±205 Cansino S, Williamson SJ, Karron D (1994) Tonotopic organization of human auditory association cortex. Brain Res 663: 38± 50 Forss N, MaÈ kelaè JP, McEvoy L, Hari R (1993) Temporal integration and oscillatory responses of the human auditory cortex revealed by evoked magnetic elds to click trains. Hear Res 68: 89±96 Frisina RD, Smith RL, Chamberlain SC (1990) Encoding of amplitude modulation of the gerbil cochlear nucleus: I. A hierarchy of enhancement. Hear Res 44: 99±122 HaÈ maè laè inen M, Hari R, Ilmoniemi RJ, Knuutila J, Lounasmaa OV (1993) Magnetoencephalography ± theory, instrumentation, and applications to noninvasive studies of the working human brain. Rev Mod Phys 65: 413±497 Hari R (1990) The neuromagnetic method in the study of the human auditory cortex. In: Grandori F, Hoke M, Romani GL (eds) Auditory-evoked magnetic elds and electric potentials. Advances in Audiology, vol 6. Karger, Basel, pp 222±282 Heil P, Schulze H, Langner G (1995) Ontogenetic development of periodicity coding in the inferior colliculus of the mongolian gerbil. Aud Neurosci 1: 363±383 Hose B, Langner G, Scheich H (1987) Topographic representation of periodicities in the forebrain of the Mynah bird: one map for pitch and rhythm? Brain Res 422: 367±373 Kaukoranta E, HaÈ maè laè inen M, Sarvas J, Hari R (1986) Mixed and sensory nerve stimulation activate di erent cytoarchitectonic areas in the human primary somatosensory cortex SI. Neuromagnetic recordings and statistical considerations. Exp Brain Res 63: 60±66 Krumhansl CL, Iverson P (1992) Perceptual interactions between musical pitch and timbre. J Exp Psychol 18: 739±751 Langner G (1981) Neuronal mechanisms for pitch analysis in the time domain. Exp Brain Res 44: 450±454 Langner G (1983) Evidence for neuronal periodicity detection in the auditory system of the guinea fowl: implications for pitch analysis in the time domain. Exp Brain Res 52: 333±355 Langner G (1992) Periodicity coding in the auditory system. Hear Res 60: 115±142 Langner G, Sams M, Heil P, Schulze H, Hari R (1994) Orthogonal representation of frequency and periodicity in the human auditory cortex: evidence from magneto-encephalography. Soc Neurosci 20 (part1): 325 (abstract) Langner G, Schreiner CE (1988) Periodicity coding in the inferior colliculus of the cat. I. Neuronal mechanisms. J Neurophysiol 60: 1799±1822 Langner G, Schreiner CE, Albert M (1992) Tonotopy and periodotopy in the auditory midbrain of cat and guinea fowl. In: Cazals Y, Horner K, Demany L (eds) Auditory physiology and perception. Pergamon Press, Oxford, pp: 241±248 Lie geois-chauvel C, Musolini A, Badier JM, Marquis P, Chauvel P (1994) Evoked potentials recorded form the auditory cortex in man: evaluation and topography of the middle latency components. Electroencephalogr Clin Neurophysiol 92: 204±214 MaÈ kelaè JP, Ahonen A, HaÈ maè laè inen M, Hari R, Ilmoniemi R, Kajola M, Knuutila J, Lounasmaa OV, McEvoy L, Salmelin R, Salonen O, Sams M, Simola J, Tesche C, Vasama J-P (1993) Functional di erences between auditory cortices of the two hemispheres revealed by whole-head neuromagnetic recordings. Human Brain Mapping 1: 48±56 McEvoy L, Hari R, Imada T, Sams M (1994) Human auditory cortical mechanisms of sound lateralization: I Interaural time di erences at sound onset. Hear Res 67: 98±109 Pantev C, Hoke M, Lehnertz K, LuÈ tkenhoè ner B, Anogianakis G, Wittkowski W (1988) Tonotopic organization of the human auditory cortex revealed by transient auditory evoked magnetic elds. Electroencephalogr Clin Neurophysiol 69: 160±170 Pantev C, Hoke M, LuÈ tkenhoè ner B, Lehnertz K (1989) Tonotopic organization of the auditory cortex: pitch versus frequency representation. Science 246: 486±488 Pantev C, Eulitz C, Elbert T, Hoke M (1994) The auditory evoked sustained eld: origin and frequency dependence. Electroencephalogr Clin Neurophysiol 90: 82±90 Pantev C, Bertrand O, Eulitz C, Verkindt C, Hampson S, Schuierer G, Elbert T (1995) Speci c tonotopic organizations of di erent areas of the human auditory cortex revealed by simultaneous magnetic and electric recordings. Electroencephalogr Clin Neurophysiol 94: 26±40 Plomp R, Steeneken HJM (1971) Pitch versus timbre. Seventh International Congress on Acoustics, Budapest, pp 387±380 Rees A, Mùller AR (1983) Responses of neurons in the inferior colliculus of the rat to AM and FM tones. Hear Res 10: 301± 330 Rhode WS, Greenberg S (1994) Encoding of amplitude modulation in the cochlear nucleus of the cat. J Neurophysiol 71: 1797±1825 Roberts TPL, Poeppel D (1996) Latency of auditory evoked M100 as a function tone frequency. Neuroreport 7: 1138±1140 Romani GL, Williamson SJ, Kaufman L (1982) Tonotopic organization of the human auditory cortex. Science 216: 1339±1340 Rose GJ, Capranica RR (1985) Sensitivity to amplitude modulated sounds in the anuran auditory nervous system. J Neurophysiol 53: 446±465 Schouten JF (1970) The residue revisited. In: Plomp R, Smoorenburg GF (eds) Frequency analysis and periodicity detection in hearing. Sijthof, Leiden, pp 41±54 Schreiner CE, Langner G (1988) Periodicity coding in the inferior colliculus of the cat. II. Topographical organization. J Neurophysiol 60: 1823±1840 Singh PG (1987) Perceptual organization of complex-tone sequences ± a tradeo between pitch and timber. J Acoust Soc Am 82: 886±899 Suga N, Niwa H, Taniguchi I, Margoliash D (1987) The personalized auditory cortex of the mustached bat ± adaptation for echolocation. J Neurophysiol 58: 643±654