Speech habilitation of hard of hearing adolescents using electropalatography and ultrasound as evaluated by trained listeners

Transcription

1 CLINICAL LINGUISTICS & PHONETICS, 2003, VOL. 17, NO. 3, Speech habilitation of hard of hearing adolescents using electropalatography and ultrasound as evaluated by trained listeners BARBARA BERNHARDT, BRYAN GICK, PENELOPE BACSFALVI and JULIE ASHDOWN School of Audiology and Speech Sciences, University of British Columbia, Canada Department of Linguistics, University of British Columbia, Canada Ministry of Health, British Columbia, Canada (Received 30 October 2002; accepted 20 November 2002) Abstract Four adolescents with moderate to severe sensorineural hearing losses and moderately unintelligible speech participated in a 14-week speech therapy study using two dynamic visual feedback technologies, electropalatography and ultrasound imaging. Electropalatography provides information about tongue-hard palate contact points. Ultrasound displays images of tongue shape and movement in two dimensions from the tip to the root. Treatment targets for all participants included a sibilant place contrast (/s/ versus /s/), liquids /l/ and /r/, and the tense-lax vowel contrast with the high vowels. Trained listener evaluations of pre- and post-treatment transcripts are reported in this paper. Significant improvements in speech production were noted across students and targets. Treatment targets improved significantly more than non-treatment test targets overall. Students showed greatest gains on consonants that were absent or marginal in their speech pre-treatment. No particular advantage of one technology over the other was evident in this sample. Keywords: Electropalatography, ultrasound, hearing impaired. Introduction Advances in articulatory visual feedback technology have provided new opportunities for treatment of speech disorders. In the present study, four adolescents with hearing and speech impairments participated in speech therapy utilizing two dynamic visual feedback systems, electropalatography and two-dimensional ultrasound imaging. Results of the study are presented in terms of trained listener transcriptions of pre- and post-treatment single-word data. The following introduction describes Address correspondence to: Barbara Bernhardt, 5804 Fairview Avenue, Vancouver BC Canada, V6T 1Z3. bb@audiospeech.ubc.ca Clinical Linguistics & Phonetics ISSN print/issn online 2003 Taylor & Francis Ltd DOI: /

2 200 B. Bernhardt et al. speech characteristics of the hearing impaired, discusses previous research using similar visual feedback technology, and outlines predictions for the current study. Congenital sensorineural hearing impairment can affect speech development. The following characteristics have been observed in speakers with such impairments: (a) suprasegmental aberrations in pitch, loudness, rate, intonation, and/or resonance; (b) omission of speech sounds in contexts with lower intensity and pitch, e.g., unstressed syllables, syllable-final position, or consonant clusters; (c) substitution of stops for other sound classes, particularly fricatives, and/or a lack of clear contrast between stops and fricatives, or stops and nasals; (d) a predominance of front consonants, particularly labials; (e) voicing mismatches for obstruents; and (f ) substitution of central vowels for other vowels (Smith, 1975; Tobey, Pancamo, Staller, Brimacombe and Beiter, 1991; Dagenais, 1992; Tye-Murray, Spencer and Woodworth, 1995; Spencer, Tye-Murray and Tomblin, 1998). Overall, speech sounds with high frequency components and/or weak intensity tend to be more difficult to acquire, and visible speech sounds with lower frequencies and higher intensity tend to be less difficult to acquire. Speakers with congenital sensorineural hearing impairment vary considerably in their speech production, with some showing severe impairment, and some showing near-normal speech. For example, in Spencer et al. s (1998) study, 13 children with hearing aids had an average per cent match with adult consonant targets of 45.6%, a severe level of speech impairment according to Shriberg and Kwiatkowski s (1982) severity index for articulation. This compared with an average match with adult consonant targets of 54.2% for 25 children with cochlear implants in the same study (a moderate level of speech impairment). Tye-Murray et al. s (1995) study of 28 children with cochlear implants showed a similar average per cent match with adult targets (53%). Children wearing implants varied in both studies, however, showing a range of 14% to 93% match for consonants. A number of factors can influence development of speech in persons with hearing impairment. These include individual factors, e.g., the aetiology and severity of the hearing impairment, the individual s other abilities (cognitive, linguistic, motor, perceptual ), and/or the individual s interest in oral communication. They also include intervention factors, e.g., the type and adequacy of assistive hearing technology (implant, hearing aid or other assistive device), the age of detection, the amount of intervention, and/or the approaches used in language and speech habilitation. Individual factors such as aetiology, severity and other abilities are immutable. Age of detection cannot be changed after the fact. However, intervention can be modified, particularly as new technologies become available. In the following section, previous research employing articulatory visual feedback technology (the focus for this paper) is discussed. Clinically, the major tool used to display tongue movement patterns has been the electropalatograph, and thus the discussion focuses on that technology. (Fletcher and Hasegawa [1983] and Dagenais [1992] have also reported on the use of glossometry, which measures and displays the space between tongue and palate.) A number of researchers have reported significant changes in speech production following use of EPG with persons with hearing impairment (e.g., Fletcher, Hasegawa, McCutcheon and Gilliom, 1980; Fletcher, Dagenais and Critz-Crosby, 1991; Dagenais, 1992; Crawford, 1995; Williams and Bernhardt, 1998; Bernhardt, Fuller, Loyst and Williams, 2000; Derry and Bernhardt, 2000). Most of the EPG studies have addressed the same type of phonetic targets, and results are fairly similar. Thus, only a few are described below.

3 V isual feedback 201 Typically, EPG studies with persons with hearing impairment have few subjects, and are relatively short-term. In Fletcher et al. s (1991) 1-month EPG study, there were five girls with profound hearing loss aged years. These participants showed more rapid gains in learning lingual consonants in that one month than they had in the previous years of traditional speech therapy. Four learned the alveolar/velar contrast, and four the contrast between /s/ and /s/. Only minimal gains were observed for nasality and voicing (which EPG does not address) and for phonetic targets that were close approximations to the adult targets pre-treatment. Overall, those with the most severe impairments made the greatest gains. In Crawford s (1995) 2-month study, two 10-year-olds with profound hearing loss showed significant improvement in velar production after 2 months of EPG training, targets which they had not been able to master with traditional therapy. Williams and Bernhardt (1998) reported similar positive gains for sibilants, velars and liquids for a 16-week EPG study that included five subjects with moderate to profound hearing loss (including two adults). Unlike in the Fletcher et al. (1991) study, pretreatment severity was not a strong predictor of speech production outcomes in the Williams and Bernhardt (1998) study. Dagenais (1992) reported results from more extensive group studies in which EPG was used with deaf children. In the first study, two groups of nine participants each participated in either EPG or traditional aural/oral speech therapy. Participants in the EPG condition showed greater immediate post-treatment gains for velars, /æ/, /a/, and /u/, and had improved tongue-palatal contact for sibilants. Children receiving traditional therapy (i.e., without visual feedback) showed greater gains for /i/, and in terms of general speech intelligibility. Six months later, children in both groups had regressed significantly, although the EPG group had not regressed to the same degree. A follow-up 2-year study was conducted with four 10-year-olds, using both EPG and traditional treatment. The EPG appeared to be most effective in the first year of the 2-year study. Participants learned to produce speech sounds that they had not mastered previously with traditional speech therapy. After 12 months, a learning plateau appeared to be reached with EPG, although intelligibility and post-alveolar fricatives did continue to improve. In the second year, traditional communicative speech therapy approaches had more of an impact on intelligibility. These studies suggest that EPG and other such laboratory-based technologies may be most useful in early stages of learning, as a means of helping speakers acquire new articulatory patterns. The study reported in this paper follows in the tradition of other studies using visual feedback technology, which assumes that visual information on tongue movement and contacts will result in the learning of new phones. Predictions and questions for this study were at several levels. 1. At the most general level, all participants were predicted to demonstrate significant improvement in speech production over the study, replicating other studies in the literature. 2. The question was asked whether severity would be predictive of degree of change after treatment. The literature was contradictory in this regard. 3. In terms of specific phonemic targets, the following three predictions were made: (a) Treated targets would show greater gain than untreated targets. (b) Untreated targets that were phonetically similar to the treated targets would, however, show some influence of training through generalization

4 202 B. Bernhardt et al. (as is often found in studies of phonological intervention, e.g., Bernhardt, 1992; Elbert and Gierut, 1986). (c) Targets that were absent or marginal in the participants speech pre-treatment would show greater gain than those that were more well-established, replicating Fletcher et al. (1991). 4. In terms of the type of visual feedback tool, it was predicted that results would vary depending on the tool as follows: (a) Because the exact contact points for alveolar and post-alveolar sibilants are observable with EPG, it was predicted that EPG training would result in greater gains for sibilant place contrasts than ultrasound training, which shows the general position and shape of the tongue, but no contact points. (b) Because ultrasound shows tongue shape and movement patterns as far back as the root, it was predicted that ultrasound training would result in greater gains for vowels and liquids than EPG training. The tongue dorsum and root components of vowels and liquids are observable on ultrasound, but not on EPG (although EPG does show some tonguepalate contact for those categories). Method Participants Four high school students ages participated in the study, three young men (pseudonyms Peran, Purdy and Palmer) and one young woman (pseudonym Pamela). Research has previously concentrated primarily on younger children. However, it has been shown that adults with hearing and speech impairments can benefit from EPG training (Derry and Bernhardt, 2000; Williams and Bernhardt; 1998). Like adults, adolescents have near-mature oral mechanisms, making it possible to rule out maturation as a significant extraneous variable during a training study. In addition, like adults, teens have had years of traditional speech therapy and education. The long unofficial baseline increases the probability that changes that occur with the technology-based treatment are more likely to be a direct result of that treatment. Audiological reports from the year prior to the study reveal the following about the hearing of the participants. The three young men have bilateral severe-toprofound sloping sensorineural losses, with Purdy having slightly better hearing in one ear. When using bilateral hearing aids, their hearing levels are in the moderate (Palmer) or moderate-to-severe range (Purdy, Peran). Pamela has a fluctuating and progressive sensorineural hearing loss due to Large Vestibular Aqueduct Syndrome. In the left ear, she has a sloping moderate-to-severe hearing loss through 3000 Hz and in the right ear, a severe-to-profound loss. With hearing aids, her thresholds slope downwards from normal to the mild loss range in the better ear up to 2000 Hz, and above 2000 Hz, from the moderate to severe loss range. In terms of frequency level, aided conversational speech can be detected up to 1000 Hz for Purdy, up to 1500 Hz for Palmer, and up to 2000 Hz for Peran and Pamela. All students find single word recognition difficult and fare better given sentence or paragraph context. At the time of this study, grade levels ranged from 10 (Peran, Purdy, Pamela) to 12 (Palmer). The students were in an oral high school programme for deaf and

5 V isual feedback 203 hard of hearing, with partial mainstreaming. All students received instruction from teachers of the deaf and hard of hearing and their speech-language pathologist, the third author of this paper. On the Rhode Island Test of Language Structure (1983, normed for hearing impaired), participants scored within or above average range. Purdy and Palmer worked part-time in hearing settings outside of school. To communicate, the students use oral language with hearing conversational partners, and a pidginized sign with each other, often with mouthed silent speech. Three of the students have languages other than English in the home (Peran, Tagalog; Purdy, Portuguese; and Pamela, Cantonese). Pamela speaks Cantonese more than English, which is her second language. Purdy s and Peran s use of their home language is not known, although according to self-report, English was their primary language of use. Speech at pre-study screening Five weeks prior to the study, a screening sample was audio-recorded by the third author in the high school setting, using a 164-word list from Bernhardt (1990) as a basis for the single word elicitation. The tapes were transcribed by the first author of this paper, an experienced transcriber of disordered speech. The transcriptions were confirmed informally by the students speech-language pathologist (the third author for this paper). On the screening list, the participants Percent Consonants Match (PCM) with the adult targets was as follows: Pamela, 60.4%; Peran, 53.1%; Purdy, 37.2%; Palmer, 34.8%. Percent Vowel Match (PVM) showed divergent results: Purdy, 58.1%; Pamela, 43.2%; Palmer; 42.9%; Peran, 29.6%. Combining match scores for consonants and vowels, scores by participant were: Pamela, 51.8%; Purdy, 47.7%; Peran, 41.35%; Palmer, 38.9%. Investigators in this study were able to communicate orally with all participants. Pamela and Purdy were easiest to understand (which fits with their PVM and PCM scores). The students speech production was typical of those with significant sensorineural hearing loss. All participants had mild hypernasality and difficulty with loudness regulation. In addition, Peran and Palmer had difficulty regulating pitch and coordinating breath support for speech, and Peran had a fairly rapid rate of articulation. Sibilants and liquids were the least well-established consonant categories across participants, although all categories showed some deviations. According to the pretreatment screening, singleton /l/ was relatively accurate syllable initially for all but Palmer, although not always in clusters. Palmer showed the least accuracy overall for sibilants and liquids. Peran used /r/ and /s/ most of the time (although not always across all word positions or in clusters), and did use /6/ once. The tense-lax distinction for vowels was weakly established across participants as were vocalic /6/ and diphthongs. (See tables 1 8 for substitution and match analyses for consonants and vowels by participant. Only those phonemes targeted in the study as treatment or test targets are included, due to space limitations.) Substitution and deletion patterns noted at the screening were typical of speakers with congenital hearing impairment. Consonant deletion was uncommon, although was observed in clusters, and also occasionally in syllable-final positions. In terms of voicing, obstruents showed devoicing, particularly word finally; in addition, wordinitial aspirated stops were sometimes produced without aspiration. In terms of manner, fricatives were sometimes produced as stops or were weakly articulated. Stops (particularly /t/) were sometimes produced as fricatives or affricates (Palmer).

6 204 Table 1. B. Bernhardt et al. Pamela s treatment and non-treatment test consonants at screening assessment Word-initial Intervocalic Word-final Total Tx C Matches Subst. Matches Subst. Matches Subst. Match Tx s 4/7 T, s, sf 1/2 ts 1/3 ts 6/13 Tx s 1/2 ts 1/1 s 0/2 2/4 Tx r 0/1 w 0/2 w 0/2 o 0/4 Tx l 1/1 2/2 0/3 vĩai 3/6 Non-Tx t 2/2 1/2 t 1/1 4/6 Non-Tx da 1/1 No data Non-Tx ts 2/2 1/1 0/1 s 3/4 Note. Only singleton consonants were targeted in treatment. Subst.=substitution. ano final /d/ in screening sample. However, /d/ showed devoicing word finally in the EPG/US samples, as did other obstruents. Table 2. Pamela s treatment and non-treatment test vowels at screening assessment Tx V Match Proportion Substitutions Tx i 2/8 I, i- Tx I 2/4 i, u Tx u 1/4 o Tx o 2/2 Non-Tx ei 1/5 E, ai, æ Non-Tx E 2/4 a, I Non-Tx 6/k 0/4 o Table 3. Purdy s treatment and non-treatment test consonants at screening assessment Word-initial Intervocalic Word-final Total Tx C Matches Subst. Matches Subst. Matches Subst. Match Tx s 0/4 s,th 0/3 th,d,c 0/1 C 0/8 Tx s 1/2 ts 2/2 3/3 6/7 Tx r 0/1 rw 0/3 w 0/1 oo 0/5 Tx l 2/4 lw 2/3 j 0/1 c 4/8 Non-Tx t 2/2 2/3a t 1/1 5/6 Non-Tx d 1/1 1/1a 0/2 Del. 2/4 Non-Tx ts 1/4 s 0/1 C 0/1 s 1/6 Note. Only singleton consonants were targeted in treatment. Subst.=substitution. athe matching alveolar stops were flaps. The liquids were often produced as the glide [w] prevocalically, and as vowels postvocalically. In terms of place of articulation, sibilants typically showed lack of accurate placement and tongue grooving. The alveolars were sometimes dentalized. Sometimes, consonants that were not present in the target words were inserted, often a nasal or [h]. The velar nasal was typically produced as an [n] or [ng]. In addition, velar stops were sometimes produced as uvulars or glottals (Peran). The vowels showed a tendency for centralization or lowering. Tense vowels were laxed, and lax vowels tensed some of the time. Diphthongs were generally produced as monophthongs.

7 V isual feedback 205 Table 4. Purdy s treatment and non-treatment test vowels at screening assessment Tx V Match Proportion Substitutions Tx i 3/4 I Tx I 2/4 ei, i Tx u 4/4 Tx o 1/1 Non-Tx ei 1/1 Non-Tx E 0/2 Non-Tx 6/k 0/1 o Table 5. Peran s treatment and non-treatment test consonants at screening assessment Word-initial Intervocalic Word-final Total Tx C Matches Subst. Matches Subst. Matches Subst. Match Tx s 0/1 z j 1/2 sf 1/2 C 2/5 Tx s No data 2/2 1/1 3/3 Tx r 3/3 2/3 w 1/1 6/7 Tx l 1/1 1/1 0/1 Del. 2/3 Non-Tx t 1/2 tf 1/2a &f 2/2 4/6 Non-Tx d 0/1 df 2/2a 1/1 Non-Tx ts 1/3 dz 0/1 z 0/1 C 1/4 Note. Only singleton consonants were targeted in treatment. Subst.=substitution. athe matching alveolar stops were flaps. Table 6. Peran s treatment and non-treatment test vowels at screening assessment Tx V Match Proportion Substitutions Tx i 3/7 I Tx I 0/1 i Tx u 1/2 c Tx o 1/2 v Non-Tx ei 1/3 I, E Non-Tx E 1/2 v Non-Tx 6/k 1/1 Table 7. Palmer s treatment and non-treatment test consonants at screening assessment Word-initial Intervocalic Word-final Total Tx C Matches Subst. Matches Subst. Matches Subst. Match Tx s 0/5 tf, T, sf 1/2 sf 1/2 sf 2/7 Tx s 0/2 tf, sf 0/1 ç 0/2 sf, C 0/5 Tx r No data 0/2 j,w 0/1 Del. 0/3 Tx l 0/1 l f 0/1 o 0/3 Del., e 0/5 Non-Tx t 1/2 tf 0/2 t 1/3 Del., ts 2/7 Non-Tx d 1/3 t 0/1 d w 1/1 2/5 Non-Tx ts 0/1 4 0/1 C 0/1 tc 0/3 Note. Only singleton consonants were targeted in treatment. Subst.=substitution.

8 206 B. Bernhardt et al. Table 8. Palmer s treatment and non-treatment test vowels at screening assessment Tx V Match Proportion Substitutions Tx i 4/5 I Tx I 1/3 i, ai Tx u 1/1 Tx o 1/2 v Non-Tx ei 1/2 ci Non-Tx E 1/6 oo,i,v,a,e Non-Tx 6/k 0/2 e, E Procedures The technologies For this study, the updated WIN-EPG (2000) was used. The participants wore custom-made artificial palates with 66 embedded electrodes. Tongue-palate contact points during speech were displayed in real-time on a computer screen. Both client and clinician productions were available for display. Both palatograms and accompanying sound files were digitally stored on the computer. ( For further technical details on EPG, see Hardcastle, Jones, Knight, Trudgeon and Calder, 1989.) The other technology used was an Aloka SSD-900 portable ultrasound with a 3.5 MHz convex intercostal transducer probe (as in figure 1). The speaker held the transducer under the chin, just above the larynx. Echo patterns from sound waves returning from the tongue surface were converted to moving images. Mid-sagittal, coronal, or transverse views of the tongue shape and movement patterns were Figure 1. Ultrasound as used in speech research.

9 V isual feedback 207 displayed on a screen. Articulatory data were recorded to VHS tape at 30 frames per second from the ultrasound machine (JVC Super VHS ET Professional Series, SR-VS20). Acoustic data were simultaneously recorded on audiotape using a Pro- Sound YU34 unidirectional microphone amplified through the built-in pre-amplifier in a Tascam cassette deck, then simultaneously recorded to the same VHS tape as the video signal. For further details on use of ultrasound, see Gick (in press). The intervention study While waiting for the arrival of the artificial palates, students continued to receive weekly traditional speech therapy in the schools for 4 weeks after the speech screening. In this period, focus was on initial /l/ and/or a sibilant. The formal intervention study contained two treatment blocks; the first had nine weekly sessions and the second, five weekly sessions. Treatment goals addressed the major needs noted at the screening and confirmed at the pre-treatment assessment (described below). They were the same for all participants: coronal fricatives /s/ and /s/, liquids /l/ and /r/ and the tense-lax vowel contrast for the high front and back vowels (/i/-/i/, /u/-/o/). ( Voiced sibilants were not addressed because obstruent voicing was an additional challenge for the participants, and could not be easily displayed by the technologies used.) For consonants, three phonemes were considered non-treatment test targets: /ts/, /t/, and /d/. Generalization from /s/ to the palatoalveolar affricate was considered possible. If there was dentalization of alveolars, there was potential generalization for alveolar placement for stops /t/ and /d/ from the training on grooved sibilants, particularly /s/. If the primary difficulty for stops was voicing, little generalization was expected for /t/ and /d/, because neither technology addressed voicing. The untreated tense-lax control/generalization pair for vowels was /ei/-/e/. The other test vowel category was the syllabic /6/ (and /k/) which could potentially show generalization from consonant training with /r/, but would not be expected as a result of tense-lax vowel training. The students came as a group to the university and each received individual therapy for about 30 minutes from one of three speech-language pathologists (first, third, and fourth authors) randomly assigned. Students were assigned to one of two treatment conditions for the first nine sessions: one pair received six sessions of ultrasound training (Purdy and Peran) and three sessions of EPG training, and the other pair received six sessions of EPG training (Pamela and Palmer) and three of ultrasound training. Targets were counterbalanced across equipment and in order, although liquids were second in sequence for both pairs. Goals, equipment and participants were as follows: Pamela and Palmer: Purdy and Peran: 1. EPG sibilants /s/ versus /s/ 2. Ultrasound liquids /l/ and /r/ 3. EPG vowels /i/ versus /I/; /u/ versus /o/ 1. Ultrasound vowels /i/ versus /I/; /u/ versus /o/ 2. EPG liquids /l/ and /r/ 3. Ultrasound sibilants /s/ versus /s/ In the final five weekly sessions, both types of equipment were used in each session for all participants, and some time was spent without equipment, practising the targets at sentence level.

10 208 B. Bernhardt et al. During therapy, the treatment targets were demonstrated by the speech-language pathologists, and practiced as isolated segments and in monosyllables CV, VC, and CVC (as single words and in short phrases). With the ultrasound, the mid-sagittal display was the most commonly used display, although coronal and transverse displays were also used on occasion to demonstrate tongue grooving or vowel height. Pre-post treatment stimuli for analysis The screening sample gave a general idea of areas of difficulty for the students, and gave a basis for setting up treatment goals. However, there were insufficient exemplars of each of the treatment and non-treatment test phonemes for a detailed quantitative analysis or to assess potential variability. Thus, a more in-depth focused sample for the test phonemes was collected at the beginning, middle and end of the EPG and ultrasound treatment study. Ten nonsense word stimuli were recorded for each of the test consonants and vowels on the EPG and on the ultrasound machine. The students read each word in its sentence frame 10 times before going on to the next one. If they were not sure of the pronunciation of the test word, they were given an imitative model by the experimenter. Word-initial consonants were elicited using printed words in CV syllables with the vowel /a/ (e.g., taw, pronounced with /a/ in the target dialect). Word-final consonants were elicited in CVC syllables starting with /ha/ (e.g., hoss ). The /h/ and the low back vowel have no contact with the palate, and thus are less likely to obscure tongue patterns specific to the targets. Vowels were elicited in the CVC syllable /h b/. Again, /h/ and /b/ do not contact the palate and thus have minimal influence on lingual consonants. The final /b/ was a challenging target phonetically because the participants often devoiced final stops. However, it was considered that, in accordance with English vowel lengthening patterns, participants might produce longer vowels if they knew the target was /b/, even if they did not produce voiced stops. Given more time to produce vowels, they might be more likely to produce the tense-lax contrast. The full set of vowel syllables evaluated for this study was /hib, hib, hub, hob, heib, heb, h{6/k}b/. Consonant target syllables were /ta, hat, da,had, sa,has, sa,has,tsa,hats,la,hal, ra,har/. (Additional vowel and consonant targets were collected but not analysed, due to time constraints.) The sentence frame for the stimuli was I m a. The schwa of the article a was considered to have minimal influence on the following test syllable. Data analysis In the current study, focus was on speech production outcomes as evaluated by trained listener transcriptions. This represents an impairment level evaluation according to the World Health Organization s (2000) ICIDH-2 model of (dis)ability. Outcomes studies at the impairment level are the most common in speech therapy research and represent a first, although not the ultimate, step in determining the relative effectiveness of a particular type of intervention ( Frattali, 1998). Previous research with EPG has suggested that visual feedback may be most useful in early stages of learning, helping speakers acquire new articulatory patterns. Thus, any changes, even minimal ones, should be first observable in the laboratory to listeners familiar with disordered speech and transcription. (A number of other outcomes evaluation studies are in progress, focusing on tongue movement and contact patterns and everyday listener and participant evaluations.)

11 V isual feedback 209 Arriving at agreement for transcriptions of the speech samples was a challenging process for these data. Two research assistants with phonetic transcription training did a first pass transcription of all the data. Reliability was well below acceptable levels of agreement, however (<50% agreement for phonemic targets), probably because only one of the two had experience transcribing speech of the hearing impaired. Consequently, researchers with more experience and training in transcription (the first and second authors of this paper) took on the task, first listening independently to subsets of each sample (i.e., pre-treatment consonants for Purdy, or post-treatment vowels for Pamela), then comparing results, and arriving at agreement item by item within the subset. Details of pronunciation were observed, but the main criterion for assigning a phone to a phonemic category was whether it was an acceptable variant of its category, distinct from other phonemes close to that category. The general criteria used for consensus-building were as follows. To be scored as a match with the English targets, an attempted /s/, /s/, and /ts/ had to be voiceless, accurate in terms of manner (fricative/affricate; stridency) and general place (alveolar versus palatoalveolar regions). Degree of stridency and exact tongue placement could differ from that of hearing speakers, but the two transcribers had to agree that the particular token was an unmistakable variant within the target sibilant category. For an /l/ to be scored as a match with the adult target, it had to be identifiably lateral and accurate in place (alveolar prevocalically or velarized postvocalically). For an /r/ to be scored as a match with the adult target, it had to have an audible English rhotic quality. An /l/ or /r/ token with excessive labialization, pharyngealization or derhotacization (in the case of the rhotic) was considered unacceptable. Tense and lax vowel pairs had to be noticeably distinct from one another in tenseness, and accurate in backness and height. Degree of tenseness/ laxness might differ from that of hearing speakers, but there had to be an audible contrast. The stops /t/ and /d/ had to be alveolar stops and distinct from each other in terms of voicing. Again, degree of voicing might differ from that of hearing speakers, but there had to be an audible contrast. Agreement between the consensus transcriptions and the closest of the original transcriptions was only 63% for actual phonemic targets (ranging from 55% for Palmer s transcripts, to 75% for Peran s). However, the consensus transcriptions and the closest of the original transcriptions agreed 84% of the time on direction of change in accuracy between pre- and post-treatment samples for each of the speakers (i.e., the target improved, did not improve, or regressed). Because the main questions for the present study involved direction of change over time, this level of external reliability was considered reasonable. The low agreement with actual transcriptions by less trained listeners has motivated further study of the speech outcomes by untrained listeners (a study which is currently underway.) A Wilcoxon s statistical test showed results of the transcriptions from the ultrasound tapes and the EPG sound files were not significantly different across speakers. Thus, the data from the two sets of sound files were combined (giving 20 items per target at both pre- and post-treatment points). Because of the small numbers of participants and tokens per category, nonparametric statistical analyses were performed for only a few between-subject variables. Results Results for the treatment study are displayed in figures 2 to 8. Figure 2 shows overall per cent change from pre- to post-treatment by participant for treated and untreated

12 210 B. Bernhardt et al. Figure 2. Average % change from pre- to post-treatment for treated and untreated segmental categories by participant. Note. Untreated targets include selected consonants and vowels only. targets. Figures 3 to 6 show per cent match for individual targets by participant. Figures 7 and 8 show per cent change for individual treatment targets, both by participant and by phoneme. As predicted, the participants showed a significantly higher proportion of matches with the phonemic targets post-treatment than pre-treatment overall ( Wilcoxon Signed Ranks Test, p=0.000). For all test targets, the average pre-treatment match was 37.5%, and the average post-treatment match, 62.5%. The average gain for all test consonants (in terms of match with the adult targets) was 32.3% and for vowels, 20.6%, a non-significant difference (Mann-Whitney U, p=0.701). The treatment targets showed an average gain of 36%, and the control targets, an average gain of 15%, a significant difference as predicted (prediction 3a; Wilcoxon s Signed Ranks, p=0.004, 2-tailed). Examining individual results (see figures 2, 7 and 8), for treatment target consonants, Purdy, Palmer, and Pamela showed similar average gains (58.3%, 53.8%, 46.9% respectively), and Peran a 28.8% gain (figure 2). This compared with an improvement Figure 3. Pamela s treated and untreated test targets before and after treatment. Note. Word-initial position is indicated by a # prior to the consonant, and wordfinal position by a # after the consonant for figures 3 to 6. Untreated consonants: the average % match for /d/ and /ts/ in word-initial and -final positions. Untreated vowels: the average % match for /E/ and /6/. Other tested targets showed a 100% match pre-treatment.

13 V isual feedback 211 Figure 4. Purdy s treated and untreated test targets before and after treatment. Note. Untreated consonants: the average % match for word-initial and -final /ts/ and word-final /d/. Untreated vowels: the average % match for /E/, /ei/, and /6/. Other test targets showed a 100% match pre-treatment. Figure 5. Palmer s treated and untreated test targets before and after treatment. Note. Untreated consonants: the average % match for word-initial and -final /ts/, /t/, and /d/. Untreated vowels: the average % match for /ei/, /E/ and /6/. Figure 6. Peran s treated and untreated test targets showing non-matches pre-treatment. Note. Untreated consonants: the average % match for word-initial and -final /ts/, /t/, and /d/. Untreated vowels: the average % match for /ei/, /E/ and /6/. of 11.7% for untreated test consonants /ts/, /t/ and /d/ for Purdy, and a regression for untreated consonants of 11%, 23.5% and 13% for Palmer, Pamela and Peran. For treated tense-lax high vowels, Purdy and Palmer showed similar gains (30%, 25%), Pamela showed a 16.9% gain, and Peran a regression of 1%. This compared with gains for untreated vowels /ei/, /E/, and /6/ (and /k/) of 47.5% (Pamela), 35%

14 212 B. Bernhardt et al. Figure 7. Percent average change from pre- to post-treatment for individual treated consonants. Note. Purdy showed 100% accuracy for /s/ pre-treatment. Figure 8. Percent average change from pre- to post-treatment for individual treated vowels. (Palmer), 23% (Purdy) and 15% (Peran). In terms of segments (as seen particularly in figures 7 and 8), the consonant showing the greatest average gain across participants was /r/ (62.5%), followed by /s/ (57.5%), /l/ (primarily postvocalic, 49.4%) and /s/ (18.1%). The vowel showing the greatest average gain across participants was the lax back vowel /o/ (34.4%) followed by /i/ and /u/ (16.3%), and /I/ (10%). The question was asked whether severity might be predictive of treatment outcomes. In the present study, PCM did not appear to be a predictor of gain. Pamela and Peran both had a higher PCM than the others pre-treatment yet showed different degrees of gain. However, PVM and conversational comprehensibility may have been predictive of gain. Pre-treatment, Peran had a lower PVM than the other participants and was somewhat difficult to understand in conversation; post-treatment he showed a more modest gain than the others. Additional predictions for phonemic targets concerned similarity of treatment and non-treatment targets and degree of establishment pre-treatment. Some untreated targets that were phonetically similar to the treatment targets showed gains through generalization as predicted, and others did not. All but Purdy showed gains for untreated vocalic /6/ (and /k/), potential generalization targets from treatment for consonantal /r/. Gains for /6/ (and /k/) ranged from 10% (Peran) to 90% (Pamela). The tense-lax contrast generalization targets /ei/ and /E/ showed a 23.5% average gain across participants, with all participants showing positive gains. Consonants showed divergent and variable results. Coronal generalization targets /ts/ versus /t/ and /d/ had a similar average level of regression across participants ( 13%), although there was individual difference in that regard. Palmer showed a

15 V isual feedback % gain for /ts/, and Peran, Purdy and Palmer showed a slight improvement on either /t/ or /d/. It was predicted that targets that were absent or marginal pretreatment would show greater gains than those that were more well-established. For consonants, this prediction appeared to be upheld. The postvocalic /l/ and /s/ were absent or marginal in the pre-treatment speech of Pamela, Peran, and Palmer and those consonants showed greater gains than /s/, which had a 67.5%, 72.5% and 27.5% match by participant respectively prior to treatment. Purdy did not use [s] pre-treatment, yet showed a 55% match for /s/ after treatment. Palmer and Pamela did not use /r/ pre-treatment, but showed an 85% to 90% match for /r/ posttreatment. Purdy and Peran showed more modest gains for /r/, which they used at least some of the time pre-treatment (Peran in fact had a 100% match for wordfinal [r] pre-treatment). For vowels, results were less clear. The prediction was upheld for the lax back vowel /o/. For the three male participants, /o/ was marginal pretreatment, and showed greater gain than the other vowels; Pamela had a 55% match for /o/ pre-treatment, and showed only a 7% gain for that vowel. However, for the other back vowel /u/ (which had been relatively well-established across participants pre-treatment), there were inconsistent results across participants. Palmer and Purdy (who had the lowest pre-treatment match for /u/) showed a slight improvement, and Pamela and Peran a slight regression. With respect to the front vowels, Pamela had only a 5% match for /i/ pre-treatment, and showed a 90% match after treatment, following the prediction. Palmer, Peran and Purdy had some use of /i/ prior to treatment (80% 70%, 60% matches respectively) and showed minor changes after treatment, with Palmer showing a slight improvement, and Peran and Purdy a slight regression. The lax vowel /I/ was marginal pre-treatment across participants, but showed inconsistent results, with improvement for Palmer and Purdy, and regression for Pamela and Peran. Finally, in terms of the technologies used, changes were noted in contact and movement patterns in the direction of the adult targets (which are currently being analysed). Overall results for the participants who spent more time on EPG did not differ significantly from those of the participants who spent more time on ultrasound, contrary to prediction 4 (Mann-Whitney U, p=0.240). In terms of individual sound classes, Pamela and Peran showed the least gain in vowels, and had been in different vowel conditions by technology (Pamela using EPG, and Peran ultrasound). Purdy and Palmer showed the greatest gains in consonants, and had been in different consonant conditions by technology. It had been predicted (4a) that EPG training would be more effective for sibilants, yet all participants improved to a similar degree on their least well-established sibilant. Ultrasound did not appear to be more effective for liquids or vowels as had been predicted (4b). Results for postvocalic /l/ were equivalent across technologies across participants. Pamela and Palmer, who had been in the EPG condition for liquids, showed greater gains for /r/ than Purdy, who was in the ultrasound condition for liquids (although, as mentioned above, the former pair had less pre-treatment knowledge of the targets). Vowel results crossed technologies, as noted above. Discussion This study set out to investigate relative effects of ultrasound and EPG in speech habilitation with four adolescents with sensorineural hearing impairment. At the most general level, all of these adolescent participants demonstrated significant

16 214 B. Bernhardt et al. improvement in speech production over the study, replicating previous research with EPG and other visual feedback but with an adolescent age group. The participants had received speech training for many years and had not yet learned several of the target contrasts and phonemes prior to this study. In the weeks between the screening assessment and the beginning of the visual feedback training, the students had received continuing traditional speech therapy for /l/ and sibilants, yet those targets remained treatment needs at the outset of this study. Furthermore, /s/ was one of the least-improving consonants in both time periods. The rapid increase in segmental accuracy during the visual feedback study, especially for treatment targets overall, suggests that the technologies were facilitative. (One of the study speech-language pathologists was the students school speech-language pathologist, thus minimizing clinician difference between prior treatment and the present study.) Severity of speech impairment may have had some influence on results. Pretreatment, Peran had the lowest PVM, and was the least comprehensible in conversation, and he showed more modest gains across targets. This is contrary to the findings of Fletcher et al. (1991), in which the most severely impaired made the greatest gains, and Williams and Bernhardt (1998), in which results were more random in terms of severity. Because all of the studies with visual feedback have small numbers of participants, the influence of severity on improvement cannot yet be determined. In terms of phonemic targets, those that were absent or marginal in the participants speech pre-treatment (one or more of the sibilants, /r/, postvocalic /l/, and /o/), tended to show greater gains than those that were more well-established pretreatment. This finding replicates Fletcher et al. (1991) and Dagenais (1992), and reinforces Dagenais s interpretation that visual feedback may be most useful in facilitating the establishment of new articulatory patterns. Although treatment targets showed significantly greater gains than non-treatment test targets, there was one deviation from this pattern. Untreated vowels showed greater improvement than some treated vowels for three participants. However, included among untreated vowels were the rhotic vowels /6/ and /k/. The improvement in this vowel could probably be attributed to treatment for the consonantal /r/. Modelling for consonantal /r/ often involved prolongation of /r/, creating a vocalic [k]. Generalization was in fact one of the predicted effects of treatment. Untreated targets that were close to the treatment target were expected to show gains, even if the gains were expected to be less than those for the treated targets. The mid vowels /ei/-/e/ were not addressed in treatment, and thus, the improvement for this pair does suggest generalization from tense-lax training. Overall, if the treatment vowel tense-lax pairs are considered together, each of the participants showed gains in at least one tense-lax treatment pair, providing a basis for generalization. Purdy and Palmer showed a mean gain for both front and back high vowel contrasts. Pamela showed a mean gain for the front tense-lax pair, and Peran for the back tense-lax pair. Generalization in the consonant category was less than in the vowel category, possibly because of greater differences between generalization and treatment targets in the consonant category. Voicing was not expected to change as a result of treatment. It was not addressed with the technology, and there were no voicing contrasts introduced in treatment. Post-treatment, voicing contrasts for the stops remained variable within and between participants, with some improvement, and some regression. The lack of change for voicing was therefore as predicted, and

17 V isual feedback 215 replicated Fletcher et al. (1991). With respect to place of articulation, Palmer improved on the affricate /ts/, but the other participants made only minimal gain, or showed regression on this phoneme, i.e. there was no transfer from /s/ treatment. ( In subsequent intervention with the students, timing of the affricate was found to be more challenging than the placement.) With respect to alveolar placement for /t/ and /d/, students who dentalized stops pre-treatment showed less of a tendency to do so after treatment. This may have generalized from treatment for the alveolar /s/. Finally, results did not appear to be dependent on a particular technology. Changes were observed for the same consonant and vowel targets across the two technologies. Both pairs of students did have time on both types of technologies in the final five sessions, which could have masked earlier, specific effects of the technologies. However, results of an intervening test probe at the end of nine sessions (not reported here) were similar to those reported here, suggesting that the type of technology was not as important as the fact that visual feedback was made available on articulatory patterns. Conclusion The present study continues to support the perspective that articulatory visual feedback facilitates speech habilitation for hearing impaired speakers across a variety of sound classes by providing information about tongue contact, movement, and shape. This study focused on specific phonemic targets and based evaluation of outcomes on transcription by trained listeners. Further research is in progress to examine the articulatory patterns as viewed with ultrasound and EPG, and to address the effects of treatment on comprehensibility, and changes in speaker activity and participation levels, as per the World Health Organization s (2000) ICIDH-2 model of (dis)ability. More research is needed with larger more diverse groups to substantiate further what types and levels of impairment are best addressed with visual feedback, what types of visual feedback might best most useful for specific disorders, specific sound classes, and different languages, and what modifications in such technologies might be suggested. Acknowledgements The authors would like to acknowledge the support of the Canadian Foundation for Innovation, the federal government organization that funded the equipment in the Interdisciplinary Speech Research Laboratory. We would also like to thank the Variety Club for funding some of the artificial palates, Satoru Tsuiki for providing the dental casts, and Dara Kanwischer for assistance in the laboratory. Finally, we give special thanks to the students and the Burnaby school district for their enthusiastic participation in this study. References BERNHARDT, B., 1990, Application of nonlinear phonological theory to intervention with six phonologically disordered children ( Unpublished doctoral dissertation, University of British Columbia, Vancouver, Canada). BERNHARDT, B., 1992, The application of nonlinear phonological theory to intervention with one phonologically disordered child. Clinical Linguistics and Phonetics, 6,

18 216 V isual feedback BERNHARDT, B., FULLER, K., LOYST, D. and WILLIAMS, R., 2000, Speech production outcomes before and after palatometry for a child with a cochlear implant. Journal of the Association of Rehabilitative Audiology, 23, Crawford, R., 1995, Teaching voiced velar stops to profoundly deaf children using electropalatography in two case studies. Clinical Linguistics and Phonetics, 9, DAGENAIS, P., 1992, Speech training with glossometry and palatometry with profoundly hearing impaired children. The Volta Review, 94, DERRY, K. and BERNHARDT, B., 2000, Palatometry intervention in relation to body (structure and function), activity, and participation (Poster presented at the VIIIth meeting of the Intl. Clin. Phonetics and Linguistics Assoc., August , Edinburgh, Scotland). ELBERT, M. and GIERUT, J., 1986, Handbook of Clinical Phonology (San Diego, CA: College- Hill Press). ENGEN, E. andengen, T., 1983, Rhode Island Test of Language Structure (Hillsboro, OR: Butte Publications Inc.) FLETCHER, S., DAGENAIS, P. and CRITZ-CROSBY, P., 1991, Teaching consonants to profoundly hearing-impaired speakers using palatometry. Journal of Speech and Hearing Research, 34, FLETCHER, S. and HASEGAWA, A., 1983, Speech modification by a deaf child through dynamic orometric modeling and feedback. Journal of Speech and Hearing Disorders, 48, FLETCHER, S., HASEGAWA, A., MCCUTCHEON, M. and GILLIOM, J., 1980, Use of lingua-palatal contact patterns to modify articulation in a deaf adult. In D. L. McPherson and M. Schwab ( Eds), Advances in Prosthetic Devices for the Deaf: A Technical Workshop (Rochester, MD: NTID Press), pp FRATTALI, C., 1998, Measuring Outcomes in Speech-Language Pathology (New York: Thieme). GICK, B., in press, The use of ultrasound for linguistic phonetic fieldwork. Journal of the International Phonetic Association. HARDCASTLE, W., JONES, W., KNIGHT, C., TRUDGEON,A.andCALDER, G., 1989, New developments in EPG. A state of the art report. Clinical Linguistics and Phonetics, 3, SHRIBERG, L. and KWIATKOWSKI, J., 1982, Phonological disorders III: A procedure for assessing severity of involvement. Journal of Speech and Hearing Disorders, 47, SMITH, C., 1975, Residual hearing and speech production in deaf children. Journal of Speech and Hearing Research, 18, SPENCER, L., TYE-MURRAY, N. and TOMBLIN, B., 1998, The production of English inflectional morphology, speech production and listening performance in children with cochlear implants. Ear and Hearing, 19, TOBEY, E., PANCAMO, S., STALLER, S., BRIMACOMBE, J. and BEITER, A., 1991, Consonant production in children receiving a multichannel cochlear implant. Ear and Hearing, 12, TYE-MURRAY, N., SPENCER, L. and WOODWORTH, G., 1995, Acquisition of speech by children who have prolonged cochlear implant experience. Journal of Speech and Hearing Research, 38, WILLIAMS, R. and BERNHARDT, B., 1998, Speech treatment outcomes as determined by unfamiliar listeners. Poster presented at the ASHA convention, November, 1998, San Antonio, TX. ASHA Abstracts, p WORLD HEALTH ORGANIZATION, 2000, ICIDH-2: International classification of Impairments, Activities and Participation. A manual of dimensions of disablement and functioning, Beta-1 draft for field trials (Geneva, Switzerland: Author).