19 th INTERNATIONAL CONGRESS ON ACOUSTICS MADRID, 2-7 SEPTEMBER 2007

Transcription

1 19 th INTERNATIONAL CONGRESS ON ACOUSTICS MADRID, 2-7 SEPTEMBER 2007 ASSESSMENTS OF BONE-CONDUCTED ULTRASONIC HEARING-AID (BCUHA): FREQUENCY-DISCRIMINATION, ARTICULATION AND INTELLIGIBILITY TESTS PACS: Ts Nakagawa, Seiji; Okamoto, Yosuke; Fujimoto, Kiyoshi National Institute of Advanced Industrial Science and Technology (AIST), Midorigaoka, Ikeda, Osaka , Japan; ABSTRACT Bone-conducted ultrasounds (BCUs) can be perceived by the profoundly deaf who hardly sense sounds even with conventional hearing-aids, as well as normal-hearing subjects. We developed a bone-conducted ultrasonic hearing-aid (BCUHA) for the profoundly deaf in which ultrasounds at about 30 khz are amplitude-modulated by speech sounds and presented to the user s mastoid by a vibrator. Users perceive demodulated speech sounds. Psychoacoustical tests were undertaken to investigate the characteristics of the BCUHA and assess its practicability. Hearing tests in profoundly deaf subjects showed that 42% were able to perceive sounds and 17% were able to recognize words using the BCUHA prototype. Psychophysical measurements in normal-hearing subjects showed that: (1) difference limens for frequency (DLFs) for BCU are larger than that of air-conduted sound (AC) below khz and above 6.0 khz, however, BCU and AC showed almost same DLFs between 0.25 and 4.0 khz, (2) articulations for Japanese monosyllables were about 60% in normal-hearing subjects, and no major differences were observed between confusion matrices of BCU and AC, and (3) intelligibility for familiar 4-mora Japanese words reached 85%. These results indicate the practicability of BCUHAs, and provide some useful information to estimate the mechanisms of BCU perception. INTRODUCTION Severely hearing-impaired people cannot hear even with the use of a conventional hearing-aid. Although cochlear implants, implanted into the temporal cranial bone, stimulate the cochlea nerve electrically and can restore hearing ability, their performance is not necessarily satisfactory. There are no commercially available hearing-aids that can recover sufficient sensation of hearing for the profoundly deaf. On the other hand, although the upper frequency limit of human hearing is believed to be no higher than about 24,000 Hz, several studies have reported that bone-conducted ultrasounds (BCUs) are audible [1-4]. Indeed, BCU hearing in humans has been demonstrated under various auditory pathological conditions, including sensorineural hearing loss and middle ear disorders [3]. BCUs are perceived even by profoundly deaf subjects who can hardly sense sounds even with conventional hearing-aids [5]. In 1991, Lenhardt et al. reported that BCU modulated by speech sounds were intelligible to some extent [5]; suggesting the possibility to develop a novel hearing-aid based on BCU perception. However, Dobie disputed Lenhardt s results obtained from subjective psychological experiments [6]. Ever since, there has been an ongoing controversy. Lenhardt s argument was recently objectively supported by magnetoencephalography (MEG) [7, 8] and positron emission tomography (PET) [9]. As well, a bone-conducted ultrasonic hearing-aid (BCUHA) for the profoundly deaf has indeed been developed [10]. A BCUHA is far easier to attach than a cochlear implant. This removes the mental and physical burden experienced by cochlear implant users. Moreover, such a hearing-aid could also be used to treat tinnitus in severely hearing-impaired people [11, 12]. Thus, many clinical applications are expected for BCUHAs. In this study, to assess the practicability of the BCUHA, hearing tests were carried out in deaf subjects. To obtain detailed information about the characteristics of hearing using a BCUHA, frequency-resolution, articulations for Japanese monosyllables, and intelligibilities of Japanese words were investigated in normal-hearing subjects.

2 BONE-CONDUCTED ULTRASONIC HEARING-AID (BCUHA) Figure 1 shows a schema of the BCUHA. Ultrasounds are amplitude-modulated by speech and presented to the mastoid or the sternocleidomastoid by a vibrator [10]. The amplitude-modulated signal is given by the following expression: U(t) = ( 1 + m f(t) ) * g(t) (Eq. 1) where f(t), g(t), and m represent a modulator signal (speech), carrier signal, and a modulationdepth, respectively. With the BCUHA, both the speech and pitch of the carrier signal, equals to ten-odd khz, are perceived simultaneously. Figure 2 shows a BCUHA. The basic parameters for the BCUHA were determined according to the results of previous studies into the physiological and psychoacoustic characteristics of BCU perception: subjective pitch [13], dynamic range of loudness [14], and the optimal carrier frequency for perception [15]. HEARING TESTS IN DEAF SUBJECTS TO EVALUATE THE PRACTICABILITY OF BCUHAs To test the utility of the BCUHA, psychoacoustical measurements were carried out in 40 normal-hearing, 37 midrange deaf, and 24 profoundly deaf subjects [10]. Test I BCU tone bursts (25-32 khz, duration: 250 ms, rise/fall: 10 ms) were presented by a BCUHA. Subjects were requested to answer whether they were able to sense sounds or not. Test II 250 Hz, 500 Hz, 1000 Hz, 2000 Hz, and 4000 Hz tone bursts (duration: 250 ms, rise/fall: 10 ms) were presented in a paired comparison procedure via a BCUHA. The stimuli intensity was set at the most clearly perceived level. In each trial, subjects were requested to indicate which of two tones had the higher pitch. Test III Japanese numbers (ex: /ichi/ (one), /ni/ (two), /san/ (three)) were presented to subjects via a BCUHA. In a trial, each stimulus was presented 10 times at intervals of 2 s. After the presentation of the stimulus, the subjects were informed which number was presented by a computer display placed in front of the subject. 5 trails were carried out for each subject using 5 speech sound vibrator microphone amplitude-modulated ultrasound Figure 1.- A schema of the bone-conducted ultrasonic hearing-aid (BCUHA). Ultrasounds are amplitude-modulated by speech or environmental sounds that are detected by a microphone and presented to the mastoid or the sternocleidomastoid muscle by a vibrator. Figure 2.- A prototype of a BCUHA. Body size: 64 * 118 * 24 mm, Weight: 178 g. Some parameters (carrier frequency, carrier amplitude, input signal amplitude, modulation depth, output signal amplitude) can be set on the liquid crystal display. Other parameters can be set from a personal computer via an RS-232C interface. 2

3 different Japanese numbers. Results In experiment I, 100% of normal-hearing, 100% of midrange-deaf and 42% of profoundly-deaf subjects were able to obtain a sound sensation. In experiment II, 95% of normal hearing, 57% of midrange deaf and 21% of profoundly deaf subjects were able to correctly discriminate frequencies. Also, 95% of normal hearing, 73% of midrange deaf and 17% of profoundly deaf subjects were able to recognize the words in experiment III. FREQUENCY-DISCRIMINATION TEST To investigate the psychoacoustic characteristics and underlying mechanisms of BCU hearing, difference limens for frequency (DLFs) for pure tones modulated onto ultrasonic carriers were measured [16]. Methods Five Japanese normal-hearing subjects (20-33 years old) participated. The experiments were conducted individually in a fully anechoic and soundproof room. The tonal signals to be discriminated had center frequencies (CFs) of 0.125, 0.25, 0.5, 1, 2, 4, 6 and 8 khz, respectively. Pure sinusoidal tones were presented under an air-conducted (AC) conditions. Under the BCU conditions, the tone signals were modulated onto the ultrasonic carriers. There were two types of carriers: a 30 khz sine wave and a bandpass Gaussian noise with a rectangular window of 30 ± 4 khz. The modulation depth was set at 0.9. Stimulus duration was 200 ms with rising/falling ramps of 50 ms, each shaped with a half-cycle with a raised-cosine function. The silent interval between the two tones in each trial was 300 ms. Intensities were always above 15 db SL. DLFs were measured using a two-alternative forced choice adaptive procedure with a decision rule that estimated the 70.7% correct point on the psychometric function. In each trial, subjects were asked to indicate which of the two tones, equally spaced in linear frequency on either side of the nominal CF, had the higher frequency. The initial frequency differences of each run were 3% of the CFs. The frequency difference was decreased by a factor of 1.4 after two consecutive correct responses and increased by the same factor after each incorrect response. Each run consisted of six reversals, and the threshold estimate was taken as the geometric mean of the frequency differences of the last four reversals. Three estimates were obtained for each of the CF conditions. The geometric mean of the estimates represented the DLFs for each subject. Results Fig. 3 illustrates the DLFs as a proportion of the CFs under each condition. These modulation conditions yielded similar DLF functions. The proportional DLFs were less than 1% between 1 and 4 khz and increased at below 0.25 khz and above 6 khz. These elevations were greater under the ultrasonic conditions and more with the noise carrier. Two-way ANOVA with repeated measures indicated a significant main effects of the modulation type (p < 0.01) and of the CF (p < 0.001). Multiple comparisons showed significant differences among all signal conditions for the and 8 khz conditions, and between noise carrier and AC for the 6 khz condition (P < 0.05 by Tukey s HSD tests). The largest individual difference was shown at 6 BCU (sinusoid) BCU (bandpass noise) AC CF (khz) Figure 3.- Difference limens for frequency (DLFs) expressed as a proportion of the center frequency (CF) and plotted as a function of it. The mean across five subjects are shown. 3

4 0.125 khz. Discussions Generally, similar patterns were obtained under both BCU and AC conditions. These results indicated that the pitch generated by amplitude-modulated BCU corresponded to that of AC pure tones. Such a pitch was essentially identical between the sinusoidal and noise carriers. For middle (0.125 ~ 6.0 khz) CFs, no significant differences were observed in DLFs between BCU and AC. On the other hand, for low (< khz) and high (> 6.0 khz) CFs, the DLFs were larger under BCU conditions than for AC. Masking by the ultrasonic carrier was implausible because sinusoidal BCU little masked AC of below 8 khz [14]. Some physical explanations might explain the results for the high frequencies. First, undersampling might have occurred due to the small amounts of frequency differences between the tones and carrier. Second, our vibrator had a resonance of 30 khz and outputs were reduced when the frequency moved away from the resonance. This well accounts for the larger DLFs of the noise carrier, which had a wider amplitude fluctuation and broader bandwidth. However, these physical explanations are not applicable to the low center frequency. From the large individual differences at the lowest center frequency, biological explanations seem more plausible. One possible explanation is low-cut filtering in a pathway conducting demodulated signals to the cochlea. However, this cannot fully explain the difference between sinusoidal and noise carriers. Another theoretical line wolud be that demodulation develops after ultrasounds are modulated to an 8 16 khz range by a process like brain resonance [11]. This implies pitch perception as if from sounds of below half the original frequencies. However, it deviates from our impression that subjective pitch is similar between BCU and AC. It also cannot explain the difference between sinusoidal and noise carriers. At present, we have no supporting evidence to explain the low discriminability at the lower frequencies with BCU hearing. ARTICULATION AND INTELLIGIBILITY TESTS To obtain more detailed information about the characteristics of speech-hearing using a BCUHA, monosyllable-articulation and word-intelligibility tests were conducted in 10 Japanese normal-hearing subjects (21-34 years old) [17]. The experiments were carried out in an anechoic room. Confusion matrices, which show information as to what kind of errors occurred in hearing, were produced based on the results of the monosyllable-articulation test and compared between BCU and AC. Articulation tests Articulations of 100 Japanese monosyllables, recorded in a female voice from a commercially available database [18, 19], were measured using a BCUHA. Monosyllablearticulations of AC speech were also measured. Intensities of amplitude-modulated BCUs were set at 20, 25, 30 db greater than the threshold. Intensities of AC were set at 20, 25, 30 db (A). Intelligibility tests The word-intelligibility of BCU speech was investigated. In the experiment, 4-mora Japanese words recorded in a female voice were selected from a database [18, 19] in which Japanese words are classified into four levels using familiarity as a parameter to control the degree of word difficulty: familiarity , familiarity , familiarity , and familiarity The concept of familiarity represents the degree of people s familiarity with each word. 50 words from each level were selected, so 200 words were used in the experiment. Intensities of amplitude-modulated BCUs were set at 20, 25, 30 db greater than the threshold. Confusion matrices To examine the speech perception tendencies of BCUHAs, confusion matrices were composed for both BCU and AC. The results of the monosyllable-articulation tests with BCU and AC were used. The phonemes were classified into vowels, unvoiced consonants, voiced plosive and fricative consonants and other voiced consonants. Results Figure 4 shows the results of the monosyllable-articulation and word-intelligibility tests. Although the scores of the articulation of BCU were lower than that of AC, 60% was achieved. Further, word intelligibility for BCU showed scores of more than 85% for words with highfamiliarities. 4

5 Figure 5 shows the respective confusion matrices for BCU and AC. Some differences between BCU and AC were found; two-way ANOVA regarding the number of misidentified phonemes with spoken phonemes and sound types showed that the number perceived as /j/ was significantly larger for BCU than for AC when palatalized sound was presented (p<0.05), and the number perceived as /r/ or /rj/ was significantly larger for AC than for BCU when voiced plosives and fricatives were presented (p<0.01) (related blocks are denoted by circles and crosses, respectively, in the figure 5). However, as a whole, these matrices show similar patterns. Discussion The similar patterns of both confusion matrices for BCU and AC indicate that there is no major difference in the ways of speech recognition for the types of sounds, i.e., the same pitch is perceived by amplitude-modulated BCU and AC. Therefore, it seems probable that signal processing schemes aimed at enabling better intelligibility of AC speech can also be applied to BCU speech. Some differences observed between amplitude-modulated BCU and AC may depend on the demodulation-mechanism of BCU. Also the confusion patterns themselves may provide some clues as to the sensory mechanism of BCU perception. The word-intelligibility scores obtained at sound levels of 25 and 30 db were significantly higher than those at 20 db, but there was no significant difference between the scores at 25 db and 30 db. Also monosyllable-articulation tests under BCU conditions showed no significant effect between the scores at 20 and 30 db, whereas the scores under AC showed a significant effect among all sound levels. These results indicated that the intelligibility of BCU Sound pressure level [db(a)] Sensation level [db] Familiarity Sensation level [db] Figure 4.- Left: Average scores of the monosyllable-articulation tests for BCU. Right: Average scores of the word-intelligibility tests for BCU and AC. Error bars indicate the SEM. ( * p < 0.05, ** p < 0.01) (1) (2) (3) (4) Perceived phoneme (1) (2) (3) (4) (1) Vowels (2) Unvoiced consonants (3) Voiced plosive and fricative consonants (4) Other voiced consonants AC 0 100% Figure 5.- Confusion matrices based on the results of monosyllable-articulation tests with AC and BCU. The phonemes were classified into four groups; (1) vowels, (2) unvoiced consonants, (3) voiced plosive and fricative consonants, and (4) other voiced consonants. Blocks with larger grey values indicate higher appearance frequencies for those pairs. BCU 5

6 does not necessarily increase with the sound level. This may be due to the carrier signal s pitch [13] and the narrow dynamic range of BCU hearing [14]. Further investigation will be useful regarding the methods of amplitude-modulation of ultrasound by a speech signal. CONCLUSION The BCUHA, a novel hearing-aid using BCU hearing, was assessed by psychoacoustical tests. The hearing tests showed more than 40% of the profoundly deaf subjects examined experienced clear sound sensations, and some subjects recognized words when using a BCUHA. No significant differences were observed in frequency-resolution in the middle frequency range (0.25 ~ 6.0 khz). As well, word-intelligibility for familiar words reached 85%. These results point to the practicability of BCUHAs. The present results are also useful for the continuing development of BCUHAs. Because pitch perception is similar between BCU and AC hearing, several signal-processing methods that are effective for conventional hearing-aids may be applicable to BCUHAs. According to the results of the frequency-discrimination test, improvements are possible through the amplification of modulation signals in the low and high frequency ranges. Learning might also be important. In the current experiments, the more often subjects experienced BCU hearing, the better their performances. Some profoundly deaf subjects, who have frequently participated in our hearing experiments, can even understand everyday speech. Further studies will be needed to clarify the details outlined in this report. Acknowledgments This work was supported by the Industrial Technology Research Grant Program of the New Energy and Industrial Technology Development Organization (NEDO), Japan, and a Research Grant from the Strategic Information and Communications R&D Promotion Programme, Ministry of Internal Affairs and Communication, Japan for SN. References: [1] V. Gavreau V: Audubillite de sons de frequence elevee. Comput Rendu. 226 (1948) [2] R. J. Pumphrey: Upper limit of frequency for human hearing. Nature. 166 (1950) 571 [3] R.J. Bellucci, and D.E. Schneider: Some observations on ultrasonic perception in man. Ann. Otol. Rhinol. Laryngol. 71 (1962) [4] J. F. Corso: Bone-conduction thresholds for sonic and ultrasonic frequencies. J. Acoust. Soc. Am. 35(1963) [5] M. L. Lenhardt, R. Skellett, P. Wang, and A. M. Clarke: Human ultrasonic speech perception. Science 253 (1991) [6] R. A. Dobie, M. L. Wiederhold: Ultrasonic hearing. Science 255 (1992) , [7] H. Hosoi, S. Imaizumi, T. Sakaguchi, M. Tonoike, K. Murata: Activation of the auditory cortex by ultrasound. The Lancet 351 (1999) [8] S. Nakagawa, T. Sakaguchi, M. Yamaguchi, M. Tonoike, S. Imaizumi, H. Hosoi, Y. Watanabe: Characteristics of auditory perception of bone-conducted ultrasound. Tech Rep IEICE. WIT99-15 (1999) [9] S. Imaizumi, H. Hoso, T. Sakaguchi, Y. Watanabe, N. Sadato, S. Nakamura, A. Waki, Y. Yonekura: Ultrasound activates the auditory cortex of profoundly deaf subjects. NeuroReport 12 (2001) [10] S. Nakagawa, Y. Okamoto, Y. Fujisaka: Development of a Bone-conducted Ultrasonic Hearing Aid for the Profoundly Sensorineural Deaf. Trans. Jpn. Soc. Med. Biol. Eng. 44 (2006) [11] M. L. Lenhardt: Ultrasonic hearing in humans: applications for tinnitus treatment. Int. Tinnitus J. 9 (2003) [12] T. Koizumi, S. Nakagawa, T. Nishimura, H. Hosoi: Auditory adaptation by pseudo-tinnitus: A MEG study. Audiology Japan 47 (2004) [13] S. Nakagawa, M. Tonoike: Measurement of brain magnetic fields evoked by bone-conducted ultrasounds: Effect of frequencies. Unveiling the Mystery of the Brain: International Congress Series, Elsevier 1278 (2005) [14] T. Nishimura, S. Nakagawa, T. Sakaguchi, H. Hosoi: Ultrasonic masker crarifies ultrasonic perception in man. Hearing Research 175 (2003) [15] Nakagawa S, Yamaguchi M, Tonoike M, Watanabe Y, Hosoi H, Imaizumi S: Characteristics of auditory perception of bone-conducted ultrasound in humans revealed by magnetoencephalography. NeuroImage 11 (2000) s746 [16] Fujimoto K, Nakagawa S: Non-linear ultrasonic perception. Hearing Research 204: , [17] Y. Okamoto, S. Nakagawa, K. Fujimoto, M. Tonoike: Intelligibility of bone-conducted ultrasonic speech. Hearing Research 208 (2005) [18] Amano S, Kondo T: Modality dependency of familiarity ratings of Japanese words. Perception & Psychophysics 57 (1995) [19] Sakamoto S, Suzuki Y, Ozawa K, Kondo T, Sone T: New lists for word familiarity and phonetic balance. J Acoust Soc Jpn. 54 (199)