Development and evaluation of the Turkish matrix sentence test

Transcription

1 International Journal of Audiology ISSN: (Print) (Online) Journal homepage: Development and evaluation of the Turkish matrix sentence test Melanie A. Zokoll, Dilek Fidan, Didem Türkyılmaz, Sabine Hochmuth, İclâl Ergenç, Gonca Sennaroğlu & Birger Kollmeier To cite this article: Melanie A. Zokoll, Dilek Fidan, Didem Türkyılmaz, Sabine Hochmuth, İclâl Ergenç, Gonca Sennaroğlu & Birger Kollmeier (2015): Development and evaluation of the Turkish matrix sentence test, International Journal of Audiology, DOI: / To link to this article: View supplementary material Published online: 07 Oct Submit your article to this journal Article views: 6 View related articles View Crossmark data Full Terms & Conditions of access and use can be found at Download by: [Kocaeli Universitesi] Date: 16 October 2015, At: 23:54

2 International Journal of Audiology 2015; Early Online: 1 11 Original Article Development and evaluation of the Turkish matrix sentence test Melanie A. Zokoll, Dilek Fidan, Didem Türkyılmaz, Sabine Hochmuth, clâl Ergenç #, Gonca Sennarolu & Birger Kollmeier,$,^ Medizinische Physik and Cluster of Excellence Hearing4all, Carl von Ossietzky Universität Oldenburg, Germany, Department of Turkish Language Teaching, Kocaeli University, Kocaeli, Turkey, Audiology Department Health Sciences Faculty, Hacettepe University, Ankara, Turkey, # Department of Linguistics, Ankara University, Ankara, Turkey, $ Hörzentrum Oldenburg GmbH, Oldenburg, Germany, and ^HörTech ggmbh, Oldenburg, Germany Abstract Objectives: The Turkish matrix sentence test, TURMatrix, was developed for precise, internationally comparable speech intelligibility testing. Design: The TURMatrix comprises a base matrix of ten well-known Turkish names, numbers, adjectives, objects, verbs, from which syntactically fixed sentences were randomly composed. Test conduction may be in an open-set (standard), or closed-set response format. Homogeneity in intelligibility of the test material was optimized by applying level adaptations (maximal 3 db) based on word-specific speech reception thresholds (SRTs). Test list equivalence was verified and reference values were determined. Study sample: Thirty-eight native listeners of Turkish with normal hearing. Results: After training, mean SRT and slope of the final test lists were db SNR and %/dB, respectively (fixed SNR measurements; inter-list variability). For adaptive measurements, average across listeners was db SNR in the open-set and db SNR in the closed-set response format. Mean SRT for adaptive measurements in the open-set response format in quiet was db. Individual SRTs in quiet correlated more closely with audiograms than with SRTs in noise. Conclusions: The TURMatrix was developed according to European standards and provides reliable speech intelligibility measurements in noise and quiet. Key Words: Speech audiometry; speech recognition in noise; signal-to-noise ratio (SNR); speech reception threshold (SRT); matrix sentence test, Turkish The difficulty of understanding speech in noisy environments is not very well reflected by pure-tone audiogram findings (e.g. Plomp, 1978; Smoorenburg, 1992). To better understand a patient s problem, it is necessary to assess his or her communication ability in noisy environments. This might also help to assess possible supra-threshold distortions that occur in the auditory system as a result of hearing impairment (Kollmeier, 1998), independent from the sensitivity loss assessed by the tone audiogram or speech audiometry in quiet. Hence, speech recognition tests in noise have become more and more important in audiological diagnostics in recent years. Applications for these tests include hearing rehabilitation, hearing research, room acoustics, and speech transmission systems. A requirement for speech audiometry is that it needs to be performed in the listener s native language. In particular, if tests are measured in noise, which creates a more authentic listening situation, non-native listeners typically perform more poorly than native listeners with the same hearing ability (e.g. van Wijngaard et al, 2002; Warzybok et al, 2015). Thus, to test a person s speech recognition in a valid way, it is essential that the test be available in that person s native language. This test should also offer a format that facilitates its practical application in host countries in order to test people who do not speak the country s majority language. However, immigrant-language-specific materials are not necessarily clinically applicable for audiologists in the respective host country since they are difficult to administer and score if the audiologist is not a native speaker (e.g. Ramkissoon, 2001). Hence, a closed-set test design is advantageous, where the patient reports his or her response not verbally, but by choosing his or her response from a list of different response alternatives. Appropriate test designs for multilingual speech recognition tests have been recommended by the International Collegium for Rehabilitative Audiology (ICRA, see Akeroyd et al, 2015). The current paper therefore aims at establishing one of these formats for the Turkish language, namely the matrix sentence test. Turkish is the largest language of the Turkish language family (25 written and a number of non-written languages; Menz, 2011) with almost 70 million native speakers (Göksel & Kerslake, 2005; Menz, 2011; Lewis et al, 2014). It is spoken predominantly in the Turkish Republic but also in European countries (nearly four million people), Australia and North America (Göksel & Kerslake, 2005). In Germany, for example, Turkish is the most important immigrant Correspondence: Melanie A. Zokoll-van der Laan, Medizinische Physik, Carl von Ossietzky Universität Oldenburg, Oldenburg, Germany. melanie.zokoll@ uni-oldenburg.de (Received 11 February 2015; accepted 15 July 2015) ISSN print/issn online 2015 British Society of Audiology, International Society of Audiology, and Nordic Audiological Society DOI: /

3 S2 M. A. Zokoll et al. language, with about two million speakers (Lewis et al, 2014). Some of its most important structural properties are listed in Supplementary Appendix 1 to be found online at doi/abs/ / For the Turkish language, the most common speech audiometric tests in clinical use consist of isolated mono- and multi-syllabic words presented in quiet (i.e. words and numbers, e.g. the Tek-test developed for native listeners of Turkish in Germany; Tek, 2007). Recently, a further monosyllabic word recognition test was added by Durankaya and colleagues (2014). The motivation for this additional test was to overcome some of the shortcomings of the older speech materials, including not completely phonemically balanced, homogenous, or familiar test lists (Durankaya et al, 2014). For assessing communication ability in a noisy environment, however, all of these tests are less appropriate than sentence recognition tests, for the following reasons: Abbreviations ANOVA HINT LTASS RMS SII SRT SNR TURMatrix Analysis of variance Hearing in noise test Long-term average speech spectrum Root mean square Speech intelligibility index Speech reception threshold Signal-to-noise ratio Turkish matrix sentence test Speech recognition tests consisting of single syllables or words in noise usually require an announcement phrase or cueing stimulus or some fixed timing to direct the listeners attention to the moment when each test item is presented. This requires additional measurement time without obtaining additional information from the listener. Sentences, on the other hand, provide their internal, natural sequence of cues without the need for an additional announcement phrase. Hence, they provide many more test items (i.e. words) per unit of time, which makes the test very time-efficient. Speech reception thresholds (SRTs) for single syllables or words vary considerably across items, spanning a range of up to 20 db (see Kollmeier, 1990) thus yielding very shallow speech intelligibility functions with slopes typically below 10%/dB. This leads to a low precision in the threshold estimate and therefore a low efficiency of the measurement procedure (c.f. Kollmeier et al, 2015). Speech recognition tests consisting of sentences, on the other hand, can achieve a very low variability across sentence-specific SRTs, leading to very steep intelligibility functions and high efficiency (e.g. precision of the threshold estimate within a given amount of measurement time). This makes the test very accurate. In addition, speech recognition tests using sentences better reflect situations in which everyday conversation takes place than do tests using single words. To the authors knowledge, the only sentence recognition test in noise for the Turkish language is the hearing in noise test (HINT; Cekic & Sennaroglu, 2008). The HINT makes use of short, meaningful, everyday sentences and was initially introduced by Nilsson and colleagues (1994) for the English language. Its speech material resembles Plomp-type sentences (Plomp & Mimpen, 1979). In the standard HINT protocol, the effects of loss of audibility on speech intelligibility are assessed with the speech reception threshold in quiet, and the effects of suprathreshold distortion are assessed in three conditions (noise from the front, NF, 0; noise from the right, NR, 90; and noise from the left, NL, 270) by measuring SRTs in noise (Soli & Wong, 2008). The level of the noise is fixed at 65 dba, and the level of the speech is adapted to find the threshold. Usually, HINT thresholds of the standard protocol and the range of normal HINT performance are given (Soli & Wong, 2008). HINT scores of a certain listener can be interpreted in relation to this reference normal-hearing range. Even though the Plomp-type sentences and the HINT tests in different languages are efficient for diagnostics due to their usually accurate SRT estimation, they show a high degree of semantic context and a very limited sentence corpus: i.e. only a few hundred sentences are available (240 for the Turkish HINT, Cekic & Sennaroglu, 2008). This clearly limits their use in settings in which frequent retesting is required, such as research and rehabilitation applications (Wagener et al, 1999c). Additionally, these tests can only be conducted in the standard open-set response format, where the listeners repeat what they understood. As a consequence of this format, the audiometrist or audiologist needs a certain level of competence in the respective test language. This contrasts with the matrix sentence test design first introduced by Hagerman (1982) (see Kollmeier et al, 2015 for a review), which aims at avoiding some of the shortcomings mentioned above for the HINT and Plomp-type sentences. It is based on syntactically fixed sentences (for the English language, e.g. name, verb, numeral, adjective, object) selected from a base matrix of 10 sentences with five words each. Despite featuring only 50 words in total, the possibility to freely combine the words between the different word groups leads to different possible sentences. This practically unlimited number of sentences makes it impossible to learn the matrix sentences by heart and makes the test also suitable for applications for which repeated testing is required. Matrix sentence test SRTs are interpreted in relation to a normal-hearing threshold and the range of normal performance usually obtained for monaural headphone presentations of speech in noise. In the standard protocol for SRT estimation in noise, the level of the noise is fixed at 65 db or higher (so that it is audible), and the level of the speech is adapted to find the threshold; the starting level is 0 db SNR or higher (i.e. above threshold). As a key feature, matrix sentence tests can be conducted in a closed-set response format. In this response format, listeners select the words that they understood out of the base matrix of 50 words displayed on a screen. This makes the matrix sentence tests suitable for use with patients of a different native language than the audiometrist or audiologist (e.g. immigrants), but also for automatic test conduction. It is known from the literature, however, that the different response formats usually lead to different SRTs: on average, the closed-set response format yields lower SRTs than the open-set response format (e.g. Kollmeier et al, 2015) which may call for separate reference values for the two response formats. One reported disadvantage of matrix sentence tests is the significant training effect (e.g. Wagener et al, 1999b; Hochmuth et al, 2012; Jansen et al, 2012). To reduce its impact on the SRT, it is usually recommended to start each examination with two training lists of 20 sentences each (e.g. Kollmeier et al, 2015). The present study aims at developing a Matrix sentence test for the Turkish language: the TURMatrix. During the development of the TURMatrix, the construction principles used (outlined in the first section, Construction of the TURMatrix) were those established by the EU project HearCom, with the goal of providing a test that is

4 Turkish matrix sentence test S3 comparable to those developed for other languages (see Kollmeier et al, 2015, for an overview and Akeroyd et al, 2015 for the recommended principles). The research questions examined in the successive optimization of the TURMatrix (outlined in the second section, Optimization of the TURMatrix) and its evaluation with listeners with normal hearing (outlined in the third section, Evaluation of the TURMatrix) were as follows: Do the characteristics of the TURMatrix (average SRT, variability across lists and listeners, average slope of the discrimination function, magnitude of the training effect and difference between the open-set and closed-set response format) coincide well with comparable matrix sentence tests in other languages? Are the TURMatrix test results in quiet more closely related to the respective test results in noise or to the individual audiogram? Is a prediction of the SRT in quiet based on the shape of the audiogram more appropriate than a simple prediction based only on the average audiogram? How do the characteristics of the TURMatrix compare to the Turkish HINT test, both in quiet and in background noise, especially with respect to test efficiency? Construction of the TURMatrix The speech material for the TURMatrix was selected and developed by linguists and co-authors D.F. and I.E. The design and selection of the speech material followed the same structure as the Swedish test of Hagerman (Hagerman, 1982) and the design criteria recommended by the ICRA consortium (Akeroyd et al, 2015). Table 1 shows the base matrix of ten sentences of five words each with the same sentence structure. By pseudo-randomly selecting words within the single word groups of this base matrix new sentences could be composed. The words were arranged in the Turkish word order: name numeral adjective object verb. The words included in the matrix were chosen among the most commonly used words in everyday spoken Turkish. Since a word frequency corpus for Turkish was not available, linguistic experts were asked their opinions about word frequency. Special attention was paid to the semantic neutrality and the familiarity of the words, also for different age groups. Abstract words were excluded from the inventory. Additionally, the authors excluded words with a soft g (), because the soft g is not a sound in Turkish, but only a letter with the function of lengthening the vowels or creating a diphthong in the language (for more information, see Fidan, 2011). Phoneme frequency [%] Phoneme Figure 1. Phoneme distribution of the TURMatrix in percent: i.e. how often each of the phonemes listed here occurs in the test corpus divided by the total number of all phonemes within the test. Phonemes are sorted in the order of their frequency. As a consequence of using only common words, words with [] were also not included in the database, since their frequency is generally low. The resulting phoneme distribution of the TURMatrix is shown in Figure 1. Unfortunately, no reference phoneme distribution was available for the Turkish language, rendering a comparison of the test phoneme distribution with the assumed phoneme distribution of the Turkish language impossible. However, based on the experience from matrix sentence tests in other languages, and as pointed out by the ICRA recommendations (Akeroyd et al, 2015), due to the size of the base matrix and the usage of common words in Turkish, it can be assumed that the underlying phoneme distribution is represented to a sufficient degree. The Turkish linguistic features of the final inventory are as follows: Nouns (including names and objects) comprise only twosyllable words that start and end with a consonant in order to prevent junctures/co-articulations. No suffixes were used. Further, all the vowels in the Turkish language were included. The distribution of manner of articulation was taken into consideration (the same applies to the word groups of adjectives and verbs mentioned below). Adjectives comprise only words with two syllables and starting with a consonant. Six of them also end with a consonant; the remaining four with a vowel. No suffixes were used. Further, all the vowels Table 1. The speech material of the Turkish matrix sentence test with the 50-word base matrix consists of ten sentences with the same syntactical structure. The words in bold are randomly selected to form one of the test sentences. Name Numeral Adjective Object Verb English translation Gönül yedi mavi sepet haketmi Gönül deserved seven blue baskets Zuhal bir yeni kilim verdi Zuhal gave a single new rug Fırat sekiz beyaz yatak satmı Fırat sold seven white beds Hikmet üç küçük çatal getirdi Hikmet brought three small forks Tuncay altı yeil cımbız bulmu Tuncay found six green tweezers Nurşen be temiz gömlek çizdi Nuren drew five clean shirts Poyraz dokuz renkli balon fırlatmı Poyraz threw nine colourful balloons Seyhan on bordo minder gördü Seyhan saw ten maroon cushions Meltem iki güzel terlik kazanmış Meltem won two nice slippers Dilek dört siyah fincan yolladı Dilek sent four black cups

5 S4 M. A. Zokoll et al. in the Turkish language were used. Numerals comprise five words with two syllables, and five words with one syllable. Only numbers, preferably between 1 and 10 were used in this category avoiding words like a few, some, any. Again, no suffixes were used. Verbs comprise five words with two syllables and five words with three syllables. Within this word group, all words began with a consonant and all vowels of Turkish were included. Additionally, the coherence between verbs and nouns and objects was considered. Third person singular and past tense as a tense marker was preferred. There are two past tense suffixes in Turkish. One is -mi, which indicates the indefiniteness, and the second is -DI, which indicates the definiteness. Methods and results (construction of the TURMatrix) RECORDING THE SPEECH MATERIAL The speech material was recorded in an approximately m (length width height) double-walled, sound-attenuated booth in the House of Hearing in Oldenburg, Germany, a special building that offers its institutions quality technical and acoustic resources. For the recordings, a Neumann U89i microphone with omnidirectional characteristics (Georg Neumann GmbH, Berlin, Germany) and a Macbook Pro (Apple Distribution International, Cork, Ireland) with a USB Soundcard RME Fireface 400 (Distribution Audio AG, Haimhausen, Germany), and ProTools software version 8 (Avid Technology Inc., Burlington, USA) with a sampling rate of 44.1 khz and 24 bit resolution were used. The recording room had reverberation times (T30) of less than 0.5 s and provided a signal-to-noise ratio exceeding 40 db SPL for all frequencies between and 8 khz, thus fulfilling the requirements of ISO (2012). The signal-to-noise ratio of the final recordings was better than 40 db. The speaker, a linguist and current co-author, was instructed to keep the same speech level and distance from the microphone during the recordings. Additionally, the speaker was asked to use natural speech effort, speech rate and intonation for all sentences and to speak all words clearly. The compliance with these instructions was continuously verified by three listeners, one of them also a Turkish linguist, outside the recording booth. To produce more natural sounding sentences, the concept of preserving co-articulation, like for the Oldenburg Sentence Test, was used (see Wagener et al, 1999c). Accordingly, 100 sentences, including all possible combinations of two consecutive words, were recorded four times. The recorded material was post-processed, including high-pass filtering at 0.05 khz, in order to remove any potential low-frequency humming and equalization of the sentences in terms of root mean square level in order to adjust for potential loudness differences of the speaker during the recording session. All recorded sentences were reviewed with respect to speech rate, loudness, intonation, artifacts, and clarity in order to select the 100 best sentences for further processing. TEST LISTS FOR THE TURMATRIX The test lists used for the TURMatrix were derived from 30 base test lists containing 10 sentences each, fulfilling the requirements of appearing purely random and containing each word of the base matrix exactly once. Instead of the theoretically possible sentences, the 30 base test lists contained only 300 sentences. The random nature of the sentences makes it virtually impossible for the listeners to memorize them. CUTTING THE SPEECH MATERIAL AND RESYNTHESIZING THE SENTENCES In order to generate the sentences from the base test lists, the 100 recorded and selected sentences were cut into individual words. Cool Edit-Pro (Syntrillium, Scottsdale, USA) and Praat (open source software for phonetic analysis, Boersma & Weenink, 2001) software were used in this process. Following the procedure used by Wagener and colleagues (1999c), sentences were cut into single words, preserving the co-articulation to the consecutive word at the end of the cut word. This resulted in 10 realizations of each word of the base matrix containing the co-articulation to each of the 10 possible following words (500 word realizations in total). The individual words were then edited to remove recording artifacts, if necessary. Finally, 478 of the realizations were concatenated to new sentences according to the 30 base test lists (by chance, 22 realizations were not included in the new sentences). For the concatenation, two consecutive sound files with appropriate co-articulation were placed together using an overlap of 15 ms in order to obtain transitions as natural as possible for the majority of the new sentences. If additional editing was necessary, this was documented for each sentence in order to be able to regenerate the sentences in later processing steps. The newly synthesized sentences were reviewed by two native speakers and a phonetician (including two of the authors). DEVELOPMENT OF THE TEST-SPECIFIC MASKING NOISE As was done for the German test (see Wagener et al, 1999c, 2003 for more details), the masking noise was generated from the resynthesized speech material by 30-fold superposition of all individual sentences, creating a quasi-stationary noise. The resulting test-specific noise exhibited the same long-term average spectrum as the speech signal (see Figure 2). Thereby, optimal spectral masking and a steep discrimination function can be achieved. Although female, the speaker had a deep voice. Thus, the long-term average spectrum in the lower frequencies resembled the male long-term average speech spectrum (LTASS) published by Byrne and colleagues (1994). Optimization of the TURMatrix The purpose of the optimization procedure was to homogenize the TURMatrix words with respect to speech recognition. This is Figure 2. Long-term spectrum of the TURMatrix sentences and TURMatrix noise generated by superimposing the speech material. The spectrum is given as root mean square (rms) levels in onethird octave bands. The male and female LTASS spectra (Byrne et al, 1994) are also given.

6 necessary in order to obtain a steep discrimination function for the test (e.g. Brand & Kollmeier, 2002). The optimization was done as described by, for example, Hochmuth et al (2012). In the first step, the word-specific intelligibility functions were determined. On the basis of the parameters describing the function, i.e. SRT and slope, word realizations were selected and level adjustments were conducted in order to equalize the intelligibilities of all word realizations as much as possible. The theoretical background for this procedure has been described in earlier publications (e.g. Wagener et al, 1999a, 2003). Material and methods (Optimization of the TURMatrix) The optimization measurements were performed in a custom-made sound-attenuated booth fulfilling the requirements of ISO (2012) at the House of Hearing (Oldenburg, Germany). Measurements were done using free-field-equalized Sennheiser HDA 200 headphones (Sennheiser Electronics, Wedemark-Wennebostel, Germany), a standard personal computer with an RME Digi96/8 PAD soundcard, Tucker Davies Technologies HB 7 headphone buffer, touch screen for recording the responses, and the HörTech Oldenburg Measurement Applications software (OMA, HörTech, Oldenburg, Germany). The headphones were free-field equalized (ISO, 2004) using a finite impulse response filter with 801 coefficients. The measurement setup was calibrated to db SPL with a Brüel & Kjær (B&K) 4153 artificial ear, a B&K inch microphone, a B&K 2669 preamplifier, and a B&K 2610 measuring amplifier. The calibration of the whole system used the test-specific noise generated from the speech material (see above). The speech signal and the noise were calibrated digitally to the same overall level. In total, 12 (four female and eight male) native speakers of Turkish were recruited, aged between 22 and 36 years (mean 28.8 years). The listeners had normal hearing, confirmed by pure-tone audiometry at the beginning of the session (better ear pure-tone thresholds 15 db HL for octave frequencies of khz up to 8 khz, mean PTA kHz db HL). The listeners were born in Turkey and native speakers of Turkish. All had lived in Germany for no more than five years. Measurements were performed monaurally on the better ear. Each listener was tested with 10 test lists of 30 sentences (i.e. three subsequent base test lists) at ten different SNR values (from 22 to 5.5 db) at a constant noise level of 65 db SPL to determine the speech intelligibility function of the recorded words. The order of test lists and SNRs was randomized in such a way that each list was measured at different levels for different listeners. Two training lists of 30 sentences each were presented prior to the actual measurements at 0 db SNR and 4 db SNR to familiarize the test listener with the test concept and the speech material. The masking noise started 500 ms before the sentences and ended 500 ms after the sentences. The test listener repeated the words that he or she understood (openset response format) and the tester recorded the correctly repeated words on a touch screen via word scoring (i.e. every word was counted separately). Results (Optimization of the TURMatrix) The word scores obtained in the optimization measurements were used to determine speech intelligibility functions for each individual word realization. This was performed by fitting a logistic function (Equation 1, e.g. Wagener et al, 2003) to the combined intelligibility Turkish matrix sentence test S5 data of all listeners for each individual word realization. Thereby, the SRT and slope describing the function for each word can be obtained: SI( SNR) 1 4s ( SRT SNR) 1 e 50 (1) where SRT is the mean SRT and s 50 the slope at the SRT. Using this approach, a mean word-specific SRT of db SNR and a median word-specific slope of the intelligibility function of 17.8%/dB for the 478 word realizations was obtained (see above). The median slopes of the logistic functions did not differ strongly between the word positions within a sentence. The mean SRT of the verbs, db SNR, was somewhat higher than the mean SRTs of the other word groups (name: db SNR; numeral: db SNR; adjective: db SNR; and object: db SNR). In the next step, each word realization was adjusted in level in order to shift its SRT as close as possible to the average SRT of the entire speech material. The amount of this level correction per word was limited in order to avoid unnatural sounding sentences: the maximum level correction was set to 3 db (see also Hochmuth et al, 2012). The level adjustment to each word was determined based on the difference between the word s SRT and the mean SRT of the entire speech material ( 9.4 db SNR). After level adjustments, sample sentences containing individual words with especially high differences in loudness (theoretical maximum of 6 db between two consecutive words) were reviewed by a native listener to ensure that they sounded natural. Reasonable optimizations could not be achieved for all 478 word realizations. There was one word realization whose SRT, even after the maximum level correction of 3 db, was still more than 4 db away from the target SRT. For another 25 word realizations, a reliable estimate of SRT and slope could not be obtained based on the available data from the optimization measurements. Any base list that contained words that could not be optimized was discarded from the total pool of base lists. Thus, from the original 30 base lists, 13 lists remained in the test for further evaluation. One of those remaining 13 base lists was discarded after a native speaker listened to them and heard noticeable problems arising from the optimization and resynthesis of the optimized realizations. The final 12 base lists were combined to six test lists of 20 sentences each. After level corrections, a mean SRT of db SNR was computed as the expected value based on the data obtained so far for the remaining 452 realizations. The expected SRTs and standard deviations for the different word groups were db SNR (name), db SNR (numeral), db SNR (adjective), db SNR (object), and db SNR (verb). Evaluation of the TURMatrix The evaluation measurements aimed at verifying that the test lists (i.e. the twelve base lists that remained after optimization) of the TURMatrix are equivalent with respect to speech intelligibility in noise. The evaluation measurements also provide reference values for further applications, such as SRT threshold and the range of normal performance for speech in noise (presented monaurally via headphone). Since the matrix test is also suitable for testing in quiet, and thus for assessing the effects of loss of audibility on speech intelligibility, the SRT threshold and range of normal performance in quiet was also obtained. For the evaluation measurements, the twelve base lists were combined to six 20-sentence lists.

7 S6 M. A. Zokoll et al. Material and methods (Evaluation of the TURMatrix) The data for this project was collected as part of a multi-center study. Two centers participated: the Department of Audiology and Speech Pathology, Hacettepe University, Sıhhiye Ankara, Turkey and the Medical Physics Group, Carl von Ossietzky University Oldenburg, Germany. At the University of Oldenburg, the listeners were tested with the same equipment and set-up as for the optimization measurements. At the Hacettepe University, the listeners were tested in a standard sound-attenuated booth (IAC Ltd., Winchester, UK) using free-field-equalized Sennheiser HDA200 headphones (Sennheiser Electronics), a Packard Bell EasyNote S8 with Win XP and Echo indigo io soundcard (Echo Audio Corporation, Santa Barbara, USA), and the HörTech Oldenburg Measurement Applications software (OMA, HörTech ggmbh). The headphones were free-field equalized (ISO, 2004) using a finite impulse response filter with 801 coefficients. The measurement setups were calibrated in the same way as described for the optimization measurements. In total, 26 native speakers of Turkish were recruited (N 15, 12 females and three males, in Ankara, and N 11, seven females and four males, in Oldenburg), aged between 19 and 42 years (mean 25.4 years). The listeners all had normal hearing (better ear pure-tone thresholds 20 db HL for octave frequencies of khz up to 8 khz, mean PTA kHz db HL, range of PTA kHz 11.7 db HL, for Ankara, and mean PTA kHz db HL, range of PTA kHz 8.6 db HL, for Oldenburg). All measurements during evaluation were performed monaurally on the better ear. To investigate the training effect, SRTs of all 20-sentence lists were measured consecutively prior to the actual measurements in either the open- (Ankara) or closed-set response format (Oldenburg). In the closed-set response format, listeners selected the words that they understood out of the base matrix displayed on a touch screen. The scoring method for both formats was word scoring. For SRT estimation, the adaptive procedure A1, proposed by Brand & Kollmeier (2002), was used with noise level fixed at 65 db SPL. The subsequent evaluation measurements to verify test list equivalence were performed at both sites with the standard open-set response format at constant SNRs ( 11.0 db, 8.5 db, 6.0 db) with a noise level of 65 db SPL. The SNRs were selected to yield approximately 20% and 80% speech intelligibility (pair of compromise, Brand & Kollmeier, 2002), as well as 50% speech intelligibility, ensuring adequate estimation of SRT and slope of the list-specific intelligibility functions. The orders of the test lists and the SNRs were randomized. In all measurements, the masking noise started 500 ms before the sentence and ended 500 ms after the sentence. Finally, to obtain reference values for normal performance in quiet, the SRT in quiet was measured adaptively at both sites with the open-set response format and two consecutive 20 sentence lists. Results (Evaluation of the TURMatrix) TEST-LIST EQUIVALENCE Evaluation measurements at fixed SNRs in the open-set response format were conducted at both sites with all 26 listeners to obtain the list-specific speech intelligibility functions. This was done by fitting the logistic function (Equation 1) to the results of all listeners for each base list. Table 2 summarizes the mean intelligibilities for the different SNRs, as well as the SRTs and slopes describing the functions calculated for the 12 base lists. The targeted intelligibilities (20%, 50%, and 80%) were met quite well with the selected SNRs. The mean SRT and slope of the 12 base lists were db SNR and %/dB, respectively (Table 2). Not shown in the table are the inter-individual differences between listeners. Those were determined by fitting the logistic function (Equation 1) to the data for all base lists obtained for an individual listener. The mean SRT and slope of the listeners was db SNR and %/dB, respectively. Furthermore, test list equivalence was verified through statistical analysis. For this purpose, mixed design ANOVAs (general linear model repeated-measures procedure; IBM SPSS Statistics software, version 22, IBM, Armonk, USA) were evaluated on the intelligibilities at each of the three fixed SNRs with the base list as the within-listener factor and site (i.e. Ankara or Oldenburg) as between-listeners factor. The latter was included to investigate whether the different response formats during training affected performance at fixed SNRs with the open-set format: In Ankara, listeners were trained with the open-set Table 2. Speech intelligibility (SI) per test list from 26 listeners with normal hearing at three signalto-noise ratios ( 6, 8.5, and 11 db SNR). The parameters SRT and slope for each test list were determined by fitting a logistic model function to the individual SIs. Mean and SD across test lists are also shown. SI [%] at Calculated 10-item test list no. 6 db SNR 8.5 db SNR 11 db SNR SRT [db SNR] Slope [%/db] Mean SD

8 Turkish matrix sentence test S7 response format, while the listeners in Oldenburg were trained with the closed-set format. The mixed design ANOVA excluded listeners with incomplete data (due, e.g. to confusion of test lists). Thus, data from only 17 (of 26) listeners for 8.5 db SNR, and from 22 (of 26) listeners for 6, and 11 db SNR, were included in the analyses. For the factor base list no significant differences were found for data obtained with 6, 8.5, and 11.0 db SNR (F 5.761, , Greenhouse-Geisser correction, r 0.13; F 11, , r 0.09; and F 11, , r 0.09, respectively, all p 0.05). There was no significant interaction of base list with site. There was a significant main effect of site for the data obtained with 6 and 11 db SNR (F , p 0.043, r 0.44; F 1, , p 0.027, r 0.47). For 8.5 db SNR, there was no such effect (F 1, , p 0.178, r 0.34). Nonetheless, this indicates that listener scores obtained in Ankara, where listeners were trained with the open-set response format, were worse (between 5.6% and 8.1%) than those obtained in Oldenburg, where listeners were trained with the closed-set response format. TRAINING EFFECT RESULTS To investigate the training effect, listeners were trained with six consecutive 20-sentence lists prior to the evaluation measurements in either the open-set (Ankara) or closed-set response format (Oldenburg). SRTs were obtained adaptively. On average, from the first to the sixth test list measured, listeners improved by 2.0 db SNR for the open-set and 1.4 db SNR for the closed-set response format (see Figure 3). The major contribution to the training effect was found within the first two measurements (mean: 1.1 db SNR for the open-set, and 0.6 db SNR for the closed-set response format). The differences between the first and the third measurements were 1.7 db SNR, and 1.0 db SNR, respectively. From the third measurement on, improvement was marginal: the difference between the third and sixth measurement was only 0.3 db SNR for the open-set and 0.4 db SNR for the closed-set response format. Pooling SRTs of the third until the sixth measurement, an SRT of db SNR was found for the open-set response format, and db SNR for the closed-set response format. A statistical analysis of the SRTs obtained for the consecutive adaptive measurements in the open- and closed-set response formats, with a mixed design ANOVA (using measurement number, i.e. measurement one to six, as the within-listeners factor, and response format as the between-listeners factor) was performed in order to test for significant differences between consecutive measurements or/ and between formats, respectively. The ANOVA revealed significant main effects for both factors (measurement number: F 5, , p 0.001, response format: F 1, , p 0.001, r 0.59), but no significant interaction (F 5, , p 0.213, r 0.24), with the latter two findings indicating that listeners tested in the closedset response format performed significantly better than those tested in the open-set response format (mean difference 0.8 db SNR) and that the effect of measurement number was in general the same in the two formats. In a follow-up analysis using a standard contrast offered by the IBM SPSS Statistics software, the simple contrast, the measurement number six was compared to all other measurements. The analysis revealed that SRTs for the sixth measurements were significantly better (i.e. p 0.05) than for the first (F 1, , r 0.91), second (F 1, , r 0.86) and also third measurements (F 1, , r 0.69), but not for the fourth (F 1, , r 0.26) and fifth measurements (F 1,24 1, r 0.12). ADAPTIVE MEASUREMENTS IN QUIET SRTs in quiet were obtained for all 26 listeners in two consecutive measurements. Different SRTs were found for the listeners in Ankara and Oldenburg. While listeners tested in Ankara showed mean SRTs of and db SPL for the first and second test lists in quiet, respectively, listeners tested in Oldenburg had SRTs of and db SPL. A mixed design ANOVA was performed on the SRTs with the measurement number (i.e. first or second measurement in quiet) as the within-listeners factor and site (Ankara or Oldenburg) as between-listeners factor. The analyses confirmed that the difference of about 6 db SPL between the two sites was significant (F 1, ; p 0.001; r 0.77). The analysis also revealed a main effect of the within-listeners factor measurements number (F 1, ; p 0.044, r 0.40). Mean SRTs for the second measurement in quiet were about 0.5 db SPL better than for the first measurement. Figure 3. Results of the six subsequent training measurements with 20-sentence lists using the adaptive procedure and an openset response format (closed diamonds), and or closed-set response format (open squares). SRTs (means and standard deviations) are plotted as a function of measurement number (i.e. temporal order). The light and dark grey areas show the regions 1 and 2 standard deviations around the mean SRT obtained for the open-set response format when pooling all adaptive training measurements from the third measurement on. Discussion The TURMatrix adds a new sentence recognition test to the speech audiometric tests available for the Turkish language. It was developed according to the same principles as the already existing tests of the same structure (e.g. Wagener et al, 1999a,b,c, 2003; Jansen et al, 2012, Hochmuth et al, 2012, Dietz et al, 2014). This speech recognition test in noise can be used for assessing a patient s communication ability in a noisy environment. It could also help to assess any supra-threshold distortions occurring in the auditory system that result from hearing impairment but are independent of the sensitivity loss. Unlike the Turkish HINT (Cekic & Sennaroglu, 2008), the Turkish matrix sentence test shows a low degree of semantic context, making it much more suitable for settings in which frequent retesting is required, such as research and rehabilitation applications. Although it uses syntactically fixed sentences concatenated from a small number of words that were cut out from the recordings, matrix sentence tests retain a high degree of naturalness in the spoken sentences by preserving the co-articulation between successive words

9 S8 M. A. Zokoll et al. within a certain sentence; this procedure was introduced by Wagener and colleagues (1999c). The naturalness of the spoken sentences is also maintained during the optimization by limiting the level adaptations of the speech material (e.g. Hochmuth et al, 2012). The limit is set with the help of native listeners of the respective language, such that the naturalness of the spoken sentences is not destroyed; in the current study, the limit was set at 3 db SPL. This is the same limit as set for the Finnish (Dietz et al, 2014), Polish (Oziemek et al, 2010), and Spanish (Hochmuth et al, 2012) tests. A maximum of 4 db SPL was chosen for the French (Jansen et al, 2012) and Russian matrix sentence tests (Warzybok et al, 2015), while for the German matrix sentence test, only 2 db SPL was allowed (Wagener et al, 1999a). Realizations of the Turkish words which could not be adjusted adequately in level (and thus intelligibility) were excluded. Thus, only 120 sentences of the originally re-synthesized 300 sentences are represented in the final test version. This is only 20 sentences fewer than reported for the Finnish matrix sentence test (Dietz et al, 2014), and resulted from the thorough optimization procedure. As a result of adhering to the same principles, the TURMatrix was expected to be comparable to other matrix sentence tests with respect to the mean list-specific intelligibility function and intelligibility across listeners (see e.g. Table 1 and Figure 6 from Kollmeier et al, 2015). The optimization procedure should lead to a steep slope for the test-specific intelligibility function, which facilitates an accurate SRT estimation (Kollmeier, 1990). The results for the TURMatrix verification reported in the third section (Evaluation of the TURMatrix) indicate that the expectation was met: The average SRT was determined to be 8.3 db SNR, with a variability across lists of 0.2 db and across listeners of 0.7 db. These values lie well in the range of 10.1 to 6 db SNR reported by Kollmeier et al (2015; see below for further discussion). In addition, a final slope of 14.1 %/db was achieved with the TURMatrix test. This is similar to the slopes of the Italian (Puglisi et al, 2015, unpublished), French (Jansen et al, 2012), Norwegian (Øygarden, 2009), and Russian matrix sentence tests (Warzybok et al, 2015), and lies between those obtained for the other languages, ranging from 10.2%/dB for Dutch (Houben et al, 2014) to 17.1%/dB for German and Polish (Wagener et al, 1999b; Ozimek et al, 2010; for an overview see Kollmeier et al, 2015). The similarity to other matrix sentence tests may be due to the common developmental standards applied to the matrix sentence tests and bodes well for future common multi-center and multilingual studies with this kind of test. The evaluation measurements described in the third section (Evaluation of the TURMatrix), further demonstrated that the 12 remaining 10-sentence base lists were equivalent on the basis of the intelligibility scores for the three SNRs considered, thus contributing to the validity of the test. The target intelligibilities of approximately 20%, 50% and 80% speech intelligibility were approximately met by the selected SNRs (Table 2), providing a valid basis for the calculation of SRT and slope across test lists and listeners (Brand & Kollmeier, 2002). The standard deviation of the calculated SRTs between base lists was 0.2 db SNR, which is in the same range as found for the German and Spanish matrix sentence tests (German: 0.16 db SNR, Wagener et al, 1999b; Spanish: 0.2 db SNR, Hochmuth et al, 2012), and is less than that found for the original Hagerman test (0.3 db SNR, Hagerman, 1982). Various 20- and 30-sentence lists can be combined out of the final 12 base lists of 10 sentences. This compensates for the loss of sentences during optimization. The seemingly random nature of the sentences and the randomized order of presentation during the measurement phase make it impossible for the listener to memorize them. For the TURMatrix, a strong training effect was observed between administration of the first and the second test lists. This is similar to the observation for other matrix sentence tests (Kollmeier et al, 2015). In the present study, the mean improvement between the third and sixth 20-sentence test list was less than 0.5 db SNR for both the closed-set (0.4 db SNR) and open-set response formats (0.3 db SNR). Although this small difference was significant according to the statistical analysis, the size of the effect seems to be negligible with regard to the use in practice. The statistical significance was primarily due to the comparatively small standard deviations across listeners. Hence, a small but significant difference between these conditions was detected, which can not normally be found for tests with a poorer accuracy. The standard deviations after the second 20-sentence test list were 0.73 and 0.71 db SNR for the open- and the closed-set response formats, respectively, and compare well with values for other matrix tests in other languages with very similar conditions (between 0.6 db and 1.0 db; Kollmeier et al, 2015). Assuming normally-distributed test results (SRTs) in adaptive measurements with the TURMatrix for listeners with normal hearing, the 95% confidence interval for the test result is in a range of about 1.4 db around the average SRT (see grey area in Figure 3 for the openset response format). Concluding from this, the recommendation for the TURMatrix is to conduct two 20-sentence training lists prior to testing, as recommended for the other matrix sentence tests (e.g. Wagener et al, 1999b; Hochmuth et al, 2012; Dietz et al, 2014). Independent of the training effect, a significant difference of about 0.8 db SNR was found between the open- and closed-set response formats. Listeners with normal hearing performed better when the basic word matrix was provided as a visual cue (i.e. in the closedset response format). The same effect was observed for the Italian, Russian, and Spanish matrix sentence tests, but not for the Polish or German matrix sentence tests (see e.g. Kollmeier et al, 2015 for an overview). Hochmuth and colleagues (2012) suggested that the absence of this effect for the Polish test resulted from the extensive training before the actual evaluation measurements. This effect of format could also have caused the difference found for 6 and 11 db SNR in the subsequent measurements with fixed SNRs; although the testing here was all done in the open-set response format, listener scores obtained in Ankara were significantly lower than those obtained in Oldenburg. The addition of a visual cue during training in Oldenburg could have led to a more stable memorization of the words within the base matrix, leading to less confusion when matching the presented word to the memorized possible words, and thus higher intelligibility scores at least for the most extreme SNRs. The hearing abilities of the listeners recruited in Ankara and Oldenburg may also have added marginally to the effect (see below). SRTs obtained in quiet were about 6 db lower for the listeners tested in Ankara than for those tested in Oldenburg. This may have resulted from small differences in the hearing abilities of the two normal-hearing listener groups (mean PTA kHz db HL for Ankara, and PTA kHz db HL for Oldenburg). This can also be seen in Figure 4: The SRTs in quiet are highly correlated with the audiogram, expressed as the PTA kHz (R , F 1, , p 0.001) and the listeners from Ankara (filled symbols) exhibit higher PTA kHz values and SRTs in quiet than those from Oldenburg. To compensate for the individual frequency-dependent audiogram in relation to the SRT in quiet, the right panel of Figure 4 shows the same SRT data in quiet plotted against the individual speech-intelligibility-index (SII)-based prediction of the SRT (ANSI 1997, SII standard applied similar to Hochmuth et al, submitted). Even though the SII is expected to be