Cued Speech Transliteration: Effects of Speaking Rate and Lag Time on Production Accuracy

Transcription

1 Journal of Deaf Studies and Deaf Education, 2016, doi: /deafed/enw034 Advance Access publication May 24, 2016 Empirical Manuscript empirical manuscript Cued Speech Transliteration: Effects of Speaking Rate and Lag Time on Production Accuracy Jean C. Krause and Morgan P. Tessler University of South Florida Correspondence should be sent to Jean C. Krause, Department of Communication Sciences and Disorders, University of South Florida, 4202 E. Fowler Avenue, PCD 1017, Tampa, FL ( Abstract Many deaf and hard-of-hearing children rely on interpreters to access classroom communication. Although the exact level of access provided by interpreters in these settings is unknown, it is likely to depend heavily on interpreter accuracy (portion of message correctly produced by the interpreter) and the factors that govern interpreter accuracy. In this study, the accuracy of 12 Cued Speech (CS) transliterators with varying degrees of experience was examined at three different speaking rates (slow, normal, fast). Accuracy was measured with a high-resolution, objective metric in order to facilitate quantitative analyses of the effect of each factor on accuracy. Results showed that speaking rate had a large negative effect on accuracy, caused primarily by an increase in omitted cues, whereas the effect of lag time on accuracy, also negative, was quite small and explained just 3% of the variance. Increased experience level was generally associated with increased accuracy; however, high levels of experience did not guarantee high levels of accuracy. Finally, the overall accuracy of the 12 transliterators, 54% on average across all three factors, was low enough to raise serious concerns about the quality of CS transliteration services that (at least some) children receive in educational settings. Since P.L was passed in 1975, deaf and hard-of-hearing (DHH) children in the United States have moved in increasingly large numbers from residential programs to local public schools (Moores, 1992). As a result, it is estimated that more than 80% of K-12 students with hearing loss are now educated in the public school setting, and the majority of these students spend at least 40% of their day in regular classrooms alongside hearing students (U.S. Department of Education, 1999). In this setting, many DHH children rely on interpreters to gain access to classroom communication. For these students, access depends largely on the clarity of the visual signal 1 received from the interpreter. That is, a necessary prerequisite to understanding the message (which ultimately depends on how the language is presented and the student s language processing abilities) is that deaf students receive a clear visual signal from interpreters (just as others in the classroom need a clear auditory, i.e., speech, signal in order to process the message). The clarity of the visual signal, in turn, depends on two channels in the communication pathway: (a) accuracy, the percentage of the message correctly produced by the interpreter and (b) intelligibility, the percentage of the message that can be correctly received by deaf persons who are proficient in the language and communication mode used by the interpreter. In this article, we examine accuracy in interpreters who use a communication mode known as Cued Speech (CS). Cued Speech Developed by Cornett (1967), CS is a system of manual cues that are produced in synchrony with speech, or the visual mouth movements of speech, and designed to disambiguate phonemes confusable through speechreading alone. The cues consist of handshapes, representing groups of visually distinct consonants, combined with placements, representing groups of visually distinct vowels. As shown in Figure 1, the American English system uses eight handshapes to distinguish among 25 consonants and six placements to distinguish among 15 vowels. Handshapes and placements each represent multiple phonemes, but the set of phonemes assigned to a particular cue Received September 8, 2015; revisions received May 2, 2016; accepted May 4, 2016 The Author Published by Oxford University Press. All rights reserved. For Permissions, please journals.permissions@oup.com 373

2 374 Journal of Deaf Studies and Deaf Education, 2016, Vol. 21, No. 4 Figure 1. Assignment of consonants to handshapes and vowels to hand placements in Cued Speech for American English. Reprinted from Speech Communication, Vol. 3, Jean C. Krause, Katherine A. Pelley-Lopez, and Morgan P. Tessler, A method for transcribing the manual components of Cued Speech, pp , Copyright 2011, with permission from Elsevier. group are, by design, easily distinguishable via speechreading. As a result, experienced users of CS can obtain near-perfect reception of words (Nicholls & Ling, 1982) and everyday sentences (Uchanski et al., 1994). Although the exact number of transliterators 2 using CS is unknown, a national survey conducted annually by the American Annals of the Deaf (Anonymous, 2013) reports that over 10% of educational programs for deaf students offered a CS component in 1994, 2003, and Taken together with reports from the National Cued Speech Association indicating that the vast majority of deaf children using CS are mainstreamed, relying on a CS transliterator for access to the auditory environment, it seems reasonable to conclude that the number of transliterators using CS is small but sizable and has remained steady over the last few decades. For the purposes of investigating interpreter accuracy, CS is an attractive candidate for initial study because there is a direct mapping from the phoneme sequence in the spoken message to the cue sequence in the transliterated message; that is, only one cue sequence can correctly represent a given pronunciation. Thus, as long as the cues produced by the transliterator can be reliably transcribed, it is relatively easy to identify and tabulate the number of errors in a transliterated message. The result is quantitative data that can describe transliterator accuracy in various conditions with a high degree of resolution. This level of resolution makes it possible to measure the extent to which various factors, such as speaking rate and lag time, affect accuracy. Accuracy Although the factors that affect interpreter accuracy are relatively unexplored, accuracy itself is fairly well understood. The Registry for Interpreters of the Deaf has assessed sign language interpreting skills, including accuracy, since the inception of

3 Krause et al. 375 their certification program in 1972 (Krause, Schick, & Kegl, 2010). Not long thereafter, Strong and Rudser (1985, 1986) developed two instruments with high interrater reliability that could be used by interpreter trainers, researchers, and sign language interpreters themselves to assess accuracy. As of today, much of what is known about interpreter accuracy comes from the Educational Interpreter Performance Assessment (Schick & Williams, 1994), or EIPA, which is designed to assess the performance of interpreters who use American Sign Language (ASL), Manually Coded English systems, Pidgin Signed English, and more recently, CS (EIPA-CS; Krause, Kegl, & Schick, 2008). The EIPA examines a large number of specific interpreting skills (organized into four skill domains) using a Likert scale ranging from 0 (no skill) to 5 (advanced skill). It has proven to be a valid and reliable (Shick, Williams, & Bolster, 1999) research tool for examining the quality of educational interpreters, with studies using the EIPA having reported data for more than 2,000 educational interpreters nationwide (e.g., Schick, Williams, & Kupermintz, 2006). Although such data provide highly valuable descriptive information regarding accuracy and other aspects of interpreter performance, it is not an ideal tool for investigating the effects of various factors on accuracy. For such analyses, it is more convenient to have a metric focused solely on accuracy, preferably with a higher degree of resolution than that offered by Likert scale ratings (in order to ensure that small effects can be detected). Therefore, a percent-correct measure of accuracy is used in this study to assess the effects of factors such as speaking rate and lag time on CS transliterator accuracy. Speaking rate One aspect of the visual signal that may be a governing factor in transliterator accuracy is speaking rate. For example, it would not be surprising to find that transliterators exhibit decreases in accuracy with increases in speaking rate due to the increased task demand. On the other hand, it is also possible that these two variables could be largely independent of each other for a range of speaking rates, given that a presumed goal of transliterator training is for transliterators to maintain a consistently high degree of accuracy across a wide range of speaking rates (i.e., the range that would typically be encountered in interpreting situations). Whether at least some transliterators have this ability, however, is unknown; no studies to our knowledge have examined the effect of speaking rate on communication mediated by a CS transliterator or by any other type of interpreting professional. Instead, the limited studies that are available in the visual modality have focused on the effect of rate in situations involving direct communication. Such studies have shown, for example, that reception of artificially accelerated fingerspelling decreases systematically for compression factors greater than 2 (Reed, Delhorne, Durlach, & Fischer, 1990), whereas intelligibility of isolated words in ASL begins to decrease when the compression factor is 2, with a breakdown in processing occurring at times the normal rate for ASL (Fischer, Delhorne, & Reed, 1999). Although it seems reasonable to expect similar breakdown points for individuals receiving information via an interpreter (English to ASL), it cannot be assumed that such breakdown points also apply to CS or other communication modalities. Moreover, because the rate alterations in these studies were typically achieved through artificial means, even the ASL breakdown points described above are based on individuals receiving a 100% accurate visual signal for processing. Whether such a high level of accuracy is possible in naturally produced messages remains unknown. To answer this question, the effect of speaking rate on accuracy of production must be examined. Lag time Another factor that may affect transliterator production accuracy is lag time or the average delay in seconds between the spoken message and the transliterated message. For ASL interpreters, at least one study has found an inverse relationship between error rates and lag time. Cokely (1986) derived this result by analyzing videotaped performances of four interpreters from a national conference, all of whom had equal interpreting experience. Of the four interpreters, two had a lag time of approximately 2 s, and two had lag time of approximately 4 s. The interpreters with a 4-s lag time made fewer mistakes. Cokely suggested that the ability to chunk the information into meaningful units resulted in the interpreter relaying the most accurate underlying meaning. Whether this relationship between lag time and accuracy applies for CS transliteration, however, is unknown. Given that CS transliteration relies on processing the source message at the phoneme level and very little processing of meaning, it is possible that optimal lag times for CS transliterators may be significantly shorter than for ASL interpreters. Therefore, research is needed to determine the nature of the relationship between accuracy and lag time for CS transliterators. Current Study In this article, 12 CS transliterators of varying experience levels were asked to transliterate an audio source presented at three different speaking rates. Two characteristics of the visual signal were obtained from transliterated messages: (a) accuracy, measured as a percent-correct score based on the target cue sequence and (b) lag time (in seconds), reported as the average delay between the spoken message and the transliterated message. The goal of the experiment was to determine the effect of speaking rate and lag time on accuracy levels. Method Participants Twelve CS transliterators (CST01 CST12) participated in the study. All were female and worked as CS transliterators (at least part-time) at the time of the study. Each completed a survey regarding level of education, relevant certifications, and experience (in years) as a CS transliterator. Based on their responses to the written survey questions, the transliterators were classified into one of three experience categories, according to the following definitions: 1. novice: minimal certification or no certification, with work experience of less than the equivalent of one full-time year; 2. experienced: minimal (e.g., state level) certification with less than the equivalent of three full-time years of work experience, or no certification with 3 5 years of experience; 3. veteran: highest level of certification (e.g., national level) and/or more than 5 years of experience. Table 1 summarizes general background information and the corresponding experience-level classification for each of the 12 participants. Materials A lecture about plants, drawn from a 25-min educational film entitled Life Cycle of Plants (Films for the Humanities, 1989), served as the stimulus materials for the transliterators. Designed for use in a high school setting, the film presents a general introduction

4 376 Journal of Deaf Studies and Deaf Education, 2016, Vol. 21, No. 4 to plant growth and reproduction and includes some specialized vocabulary pertaining to plants (e.g., names of plant species). To create a lecture version of this material, the audio narration from the film was re-recorded; a male talker read a transcript of the film at a deliberate speaking rate, simulating a lecture-style delivery, and inserted sizable pauses (1 3 s) at every sentence (or major phrase) boundary. The resulting recorded audio lecture was divided into three segments of roughly equal length, excluding a short beginning segment (~1 min) of the lecture that was reserved for use as warm-up material. Two additional versions of the lecture materials were then produced at alternate speaking rates. Using a feature of the computer program Praat (Boersma & Weenink, 2012) that manipulates duration of speech signals without affecting pitch or phonetic qualities, the lecture was slowed by an expansion factor of 1.25 to obtain a slow-conversational speaking rate for one version; for the other version, the lecture was sped up by a compression factor of 0.8 to obtain a fast-conversational speaking rate. Thus, all lecture materials (warm-up segment and three test segments) were available for the experiment at three different speaking rates: a normal- conversational rate (i.e., the original speaking rate) of 109 words-per-minute (wpm); a slow-conversational rate of 88 wpm; and a fast-conversational rate of 137 wpm. Recording Sessions All transliterators participated individually in a single test session, roughly 1.5 hr in length. Lecture (warm-up and test) materials were presented from a portable stereo via high-quality speakers. Each participant s transliteration of the lecture was filmed using a digital video camera and transferred to a computer hard disk for later analysis. The camera was positioned six to eight feet in front of the transliterator, who could choose to sit or stand. The camera was centered and zoomed such that a view of the transliterator from the chest and up was achieved. At the beginning of the session, each transliterator was provided with a transcript of the lecture and a list of key vocabulary words and was given up to 30 min to prepare. When ready (or when 30 min had elapsed), the transliterator was presented with the entire lecture in three segments; each segment was presented at a different speaking rate. As shown in Table 2, the three speaking rates were always presented from slowest to fastest (the first segment was presented at the slow-conversational rate, the second segment at the normal-conversational rate, and the final segment at the fast-conversational rate), but the order of segments was counterbalanced across transliterators to minimize any effects due to differences in difficulty of the segments. Breaks were offered between lecture segments, and a 5-min warm-up period was provided prior to each of the segments. During this period, the warm-up material was presented at the upcoming speaking rate, and the transliterator was given the opportunity to practice transliterating at this rate and to adjust the volume of the speakers as needed to ensure a comfortable, audible presentation level. Video Analysis Procedures The video recordings of each transliteration were viewed in slow motion using Adobe Premiere Pro 1.5. All cues produced by the transliterators were transcribed following the method described previously by Krause, Pelley-Lopez, and Tessler (2011). This method has been shown to produce highly reliable (i.e., >90%) transcriptions, even when the cuer s hand gestures are heavily co-articulated (Krause et al., 2011). To analyze the accuracy of the cues produced, each transcription was compared to a Table 1. Participant background in CS transliteration and experience-level classification Experience (years) Current use (hours/week) Education Certification Classification CST Some college a State level Novice CST Some college a State level Veteran CST Some college a State level Novice CST Some college State level Veteran CST High school State level Veteran CST Some college a State level Veteran CST Master s degree National level: TSC b Veteran CST Some college None Experienced CST College grad None Veteran CST College grad None Veteran CST College grad None Veteran CST College grad National level: TSC b Veteran Note. CS = Cued Speech. a College coursework included one or more courses in the field of interpreting or transliterating. b Transliteration Skills Certificate, issued by TECUnit, the national certifying body for interpreters who use CS. Table 2. Presentation order of lecture segments for each transliterator Presentation order (rate) Transliterator First (slow-conversational) Second (normal-conversational) Third (fast-conversational) CST01, CST07 Segment 1 Segment 2 Segment 3 CST02, CST08 Segment 1 Segment 3 Segment 2 CST03, CST09 Segment 2 Segment 1 Segment 3 CST04, CST10 Segment 2 Segment 3 Segment 1 CST05, CST11 Segment 3 Segment 1 Segment 2 CST06, CST12 Segment 3 Segment 2 Segment 1

5 Krause et al. 377 sequence of target cues (i.e., the expected cue sequence, based on a phonetic transcription of the narrator s speech). Based on this comparison, each cue was categorized as a correct production, a substitution error, or an insertion error, whereas any target cue that was not produced by the transliterator at all was categorized as an omission. The number of cues in each category was then tabulated, and percentage scores for each category were calculated by taking the number of cues in the category and dividing it by the total number of target cues for the passage. Lag time was recorded at the phrase level, where phrases were defined as any utterance separated by the sizable pauses inserted by the narrator, as described above. For each phrase, the difference in time was measured between the beginning of the spoken word and the beginning of its visual representation in the transliterated message, at three points: the first syllable in the phrase, the middle syllable in the phrase, and the final syllable in the phrase. The lag time for the phrase was defined as the average of these three measurements. Results Table 3 summarizes the overall frequency of occurrence, across all transliterators and speaking rates, for each type of cue production category. As expected, correct cues occurred most frequently; however, the average proportion of correct cues was only 54%. Omissions were the most frequent type of error (24%), followed closely by substitutions (22%); insertions occurred relatively infrequently, accounting for only an additional 5% of cues beyond the target cue sequence. A repeated-measures, one-way analysis of variance performed (after an arcsine transformation to reduce any inequalities in variance) on the frequency of occurrence data for the three error production categories (omissions, substitutions, and insertions) showed that the main effect of error category was significant (F(2, 22) = 9.3, p <.001). Post hoc tests with Bonferroni corrections confirmed that omissions and substitutions both occurred more frequently than insertions (p <.01). The difference between frequency of omissions and substitutions was not significant (p >.05). Paired t tests with Bonferroni corrections confirmed that correct cues occurred more frequently than any type of error (p <.05). Data for individual transliterators, shown in Figure 2, revealed two different error patterns among participants; two transliterators (CST01 and CST03) showed a large proportion of omissions but very few substitutions, whereas the remaining transliterators all exhibited a much higher proportion of substitutions and, with one exception (CST04), relatively fewer omissions. Notably, these two different error patterns also coincided with differences in experience level. That is, veteran transliterators produced 24% substitutions on average, many more substitutions than novices, who produced just 3%. However, they produced half as many insertions on average (4% vs. 8%) and generally exhibited substantially higher accuracy than novice transliterators (on average, 58% vs. 45%). The accuracy advantage of veterans over novices was a result of many fewer omissions veterans omitted 19% of cues on average, whereas novices omitted a staggering 53%. Of the 10 transliterators classified at the higher experience levels (9 veteran, 1 experienced), only CST04 exhibited such a large proportion of omissions. A veteran, CST04, also exhibited a sizable proportion of substitutions; as a result, she had the lowest accuracy (29%) of any transliterator in the study, demonstrating that experience alone is not a guarantee that adequate transliteration skills are developed. Effect of Speaking Rate The effect of speaking rate on each of the cue production categories is shown in Figure 3. For correct cues, a repeated-measures, one-way analyses of variance indicated that the frequency of correct cues (F(1.22, 13.41) = 102, p <.001) differed significantly across speaking rates, and post hoc testing showed that the proportion of correct cues decreased significantly with each increase in rate (p <.001 after Bonferroni corrections). For errors, the effect of speaking rate was analyzed by examining the interaction between speaking rate and error production category using a 3 3 analysis of variance with two repeated measures (error production category and speaking rate); the interaction was significant (F(1.83, 20.17) = 32, p <.001), indicating that the frequency of errors produced in one or more of the error production categories was affected by speaking rate. To examine this interaction more closely, three repeated-measures, one-way analyses of variance (one for each error production category, with speaking rate as a within-subjects factor) were performed. Results indicated that omissions (F(1.46, 16.03) = 67, p <.001) differed significantly across speaking rates, with the proportion of omissions increasing significantly with each increase in rate (p <.001 after Bonferroni corrections). Thus, omissions exhibited Table 3. Average frequency of occurrence for cue production categories and average lag time by individual transliterator Cue production category Transliterator Correct cues (%) Omissions (%) Substitutions (%) Insertions (%) Lag time (seconds) CST CST CST CST CST CST CST CST CST CST CST CST Average SE Note. SE = standard error.

6 378 Journal of Deaf Studies and Deaf Education, 2016, Vol. 21, No. 4 Figure 2. Frequency of occurrence for each cue production category (correct cues, omissions, substitutions, insertions) for the individual transliterators at each of the three experience levels. Panel D). Thus, the effect of speaking rate on cue production was very robust. As speaking rate increased, every transliterator s accuracy decreased, and the decrease was primarily a result of increased omissions. Lastly, the data in Figure 4 suggest that experience may also have had a role in how the transliterators in this study were affected by speaking rate. As speaking rate increased from the slow rate to the fast rate, frequency of correct cues declined more steeply for novice transliterators ( 0.6% per wpm, on average) than for either the experienced ( 0.4% per wpm) or veteran transliterators ( 0.3% per wpm, on average). Similarly, frequency of omissions increased with speaking rate more steeply for novices (0.6% per wpm, on average) than for transliterators with more experience (0.3% per wpm for both the experienced transliterator and the veteran transliterators, on average). The slight increase in average frequency of substitutions with speaking rate was due to the experienced transliterator and all but one of the veteran transliterators who produced a slightly higher proportion of substitutions as speaking rate increased from the slow to the fast rate (0.11% per wpm, on average); in contrast, the novice transliterators and CST04, the veteran who performed much like a novice, produced a slightly lower proportion of substitutions over this same range of speaking rates ( 0.04% per wpm, on average). Effect of Lag Time Figure 3. Frequency of occurrence for each cue production category (correct cues, omissions, substitutions, insertions) at each of the three speaking rates, averaged across all transliterators. a positive relationship with speaking rate (slow rate: 16%, normal rate: 23%, fast rate: 32%), whereas correct cues showed a negative relationship (slow rate: 64%, normal rate: 55%, fast rate: 44%). Although the proportion of substitutions showed a weak positive relationship with speaking rate (increasing from 19.7% at the slow rate to 23.1% at the fast rate), the overall difference in frequency of substitutions across speaking rate did not reach statistical significance (F(1.26, 13.84) = 4, p >.05). Insertions were generally unaffected by rate (F(1.57, 17.23) = 3, p >.05). Data for individual transliterators were consistent with these patterns. Panel A of Figure 4 shows that frequency of correct cues decreased with each increase in speaking rate for all individual transliterators; similarly, every transliterator s frequency of omissions increased with each increase in speaking rate (Figure 4, Panel B). Although 8 of the 12 transliterators also showed some increase in substitutions (Figure 4, Panel C), the amount of increase in substitutions from the slow rate to the fast rate was smaller than that exhibited for omissions in all but one of these transliterators (CST12). In her case, the amount of increase was similar for both cue production categories and relatively small (5 percentage point increase for substitutions vs. 4 percentage point increase for omissions from the slow rate to the fast rate). Moreover, no transliterator showed substantial changes in frequency of insertions with speaking rate (Figure 4, Table 3 shows that the average amount of lag time (across all three speaking rates) used by the individual transliterators varied considerably, ranging from 0.79 s for CST10 to 3.62 s for CST03 and CST04. Despite this wide variation, there was no relationship between a transliterator s average lag time and average accuracy, that is, the average frequency of correct cues produced by the transliterator across all three speaking rates (p >.05). At the phrase level, however, a weak negative relationship was detected (Pearson s r =.164, p <.000). Figure 5 shows the effect of lag time on accuracy at the phrase level, across all transliterators. Transliterators used lag times ranging from 0.34 to 7.31 s to cue individual phrases, but most phrases (75%) were produced with lag times between 0.75 and 2 s, and the vast majority (95%) of phrases were produced with lag times of 4 s or less. As Figure 5 shows, however, lag time was not strongly associated with accuracy. That is, the accuracy of phrases cued with any particular lag time varied widely (from 20% or less to 100%). Thus, the tendency for accuracy to decrease as lag time increased was very weak; although statistically significant, it accounted for just 3% of the variance in accuracy at the phrase level. Individual transliterator data at the phrase level are consistent with an overall weak relationship between lag time and accuracy. Table 4 shows that whenever a relationship at the phrase level could be observed in an individual (in individual data, less statistical power is available to observe relationships, due to the smaller number of data points), it was in the expected (negative) direction. Specifically, correlations at the phrase level reached statistical significance for three transliterators (CST01, CST02, and CST04), with all three showing a negative relationship between accuracy and lag time. Three additional transliterators (CST06, CST08, and CST10) showed trends consistent with a negative relationship that did not reach statistical significance. In the remaining six cases, no relationship was detected (i.e., less than 5% change in accuracy for 1-s change in lag time). Finally, it is worth noting that there was a large difference in lag time between transliterators at different experience levels: the average lag time for novices was 3.59 s, whereas the average lag

7 Krause et al. 379 Figure 4. Frequency of occurrence for each cue production category (correct cues, omissions, substitutions, insertions) at each of the three speaking rates, for each of the 12 transliterators. Experience categories of individual transliterators are denoted by line style: dotted lines for novices, dashed line for experienced, and solid lines for veterans. Table 4. Phrase-level correlations between accuracy and lag time by individual transliterator (asterisks indicate statistically significant Pearson s r values) Direction of relationship Strength of relationship (Pearson s r) CST01 Negative.505** CST02 Negative.320* CST03 Flat.083 CST04 Negative.288** CST05 Flat.009 CST06 Negative.186 CST07 Flat.031 CST08 Negative.249 CST09 Flat.080 CST10 Negative.113 CST11 Flat.133 CST12 Flat.063 Figure 5. Relationship between accuracy, or percentage of cues correctly produced, and lag time at the phrase level. Each data point represents one phrase produced by one transliterator. The best-fitting line indicates a weak, negative relationship between the two variables. time for the experienced and veteran transliterators was 1.51 s. Although this difference could have occurred by chance (given the small numbers of transliterators in each experience group), the size of the difference and consistency across individuals are consistent with a possible relationship between experience and lag time. Discussion The primary purpose of this study was to examine the effects of speaking rate and lag time on the accuracy of messages produced by CS transliterators. Of these two factors, lag time had the least effect on accuracy. Despite a wide variation in lag times, there was no relationship between a transliterator s average lag time and average accuracy. At the phrase level, increased Note. *p <.05; **p <.01. lag time was associated with lower accuracy, but the relationship was weak and explained only 3% of the variance in accuracy of phrases. In contrast, speaking rate had a fairly large negative effect on accuracy. On average, CS transliterators were 20 percentage points less accurate at the fast speaking rate than at the slow speaking rate. This decline was caused primarily by an increase in omitted cues (16 percentage points, on average) over the same range of speaking rates. These effects were exhibited by every individual transliterator, although they were strongest in the transliterators with the least experience; the average decline in accuracy was 18 percentage points for experienced and veteran transliterators and 29 points for novice transliterators. Generally, increased experience level was associated with increased accuracy, regardless of speaking rate. However, high

8 380 Journal of Deaf Studies and Deaf Education, 2016, Vol. 21, No. 4 levels of experience did not guarantee high levels of accuracy. Three of nine veteran transliterators had average accuracy levels less than 50%, with one in particular, CST04, exhibiting very low accuracy (29%) the lowest of all 12 transliterators examined, including both novices who had far less experience. Even if CST04 is assumed atypical and excluded, the veteran transliterators exhibited a wide range of accuracy levels from 42.5% (CST10) to 84.4% (CST12). In other words, despite a general trend that transliterators with more experience are more accurate (six of nine veterans in this study were more accurate than the three less-experienced transliterators), it cannot be assumed that individuals with a high level of experience are, in fact, highly accurate transliterators. Rather, their accuracy can vary widely. The reasons for this wide variation are unknown and could stem from differences not only in transliterator background (e.g., education, quality and amount of training, credentials, etc.) but also innate transliterator characteristics, such as working memory or manual dexterity. Until these factors can be adequately understood, it is important that all transliterators be monitored for accuracy (through informal and formal assessments), regardless of experience level. It is also worth noting that the nature of errors made by transliterators differed with experience level. Novice transliterators exhibited a high proportion of omissions and very few substitutions, whereas veteran transliterators produced a larger proportion of substitutions than omissions. Moreover, it was observed that the majority of omissions made by novice transliterators occurred when large chunks of the message were dropped (from the mouth as well as the hands), in order to keep pace with the speaker. Apart from these omissions, cues produced by the novice transliterators were very accurate 94% of attempted cues (i.e., disregarding omissions) were produced correctly. Omissions made by veteran transliterators, on the other hand, tended to be confined to individual cues within words or at most, short sequences of words that were not essential to the meaning of the message. Their higher substitution rate reduced overall accuracy of attempted cues to 71%, but the vast majority of the message was transliterated. Interestingly, the transliterator with the lowest accuracy of attempted cues was the experienced transliterator (i.e., the transliterator with an experience level intermediate to the novice and veteran groups) just 52% of the cues she attempted were produced correctly. Although no generalizations regarding the role of experience can be drawn from this limited data set, the trend in these findings suggests a possible developmental trajectory that warrants further investigation. Namely, an evolution of skill from transliterators at the novice level, who tend to cue correctly but lag far behind the talker, omitting large portions of the message; to more experienced transliterators, who are more likely to keep pace with the talker but may do so at the expense of large numbers of incorrect cues (i.e., substitutions); to veteran transliterators, who are typically able to keep pace with the talker with fewer substitutions and a higher degree of accuracy. Despite this evolution of improving accuracy with experience, the data in this study suggest that the accuracy of many CS transliterators, even those with a great deal of experience, falls well below 100%. Accuracy This study was not designed to characterize the average accuracy of typical CS transliterators but rather to examine if and how factors such as speaking rate and lag time are related to accuracy. That said, it cannot be ignored that the overall accuracy, 54% on average across all three factors, was low enough to raise serious concerns about the quality of CS transliteration services that (at least some) children receive in educational settings. Although not a large or nationally comprehensive sample, the educational CS transliterators in this study were randomly selected from two distinct regions of the country and are thus representative of at least some segment of typical transliterators working in the schools. Consequently, the data suggest an urgent need for increased transliterator training and professional development opportunities. At present, however, transliterators have little opportunity for either. None of the interpreter training programs in the country provide training specifically for CS transliterators, and professional development opportunities are limited, at best. One tool that could help to a small extent is the EIPA-CS (Krause et al., 2008). An extension of the EIPA (Schick & Williams, 1992) specifically for CS transliterators, the EIPA-CS, is a valid and reliable assessment tool that provides individuals with a comprehensive evaluation of strengths and weaknesses as well as an overall numerical rating of skill level. This information could be used by individuals (or by school districts and/or states with multiple CS transliterators) to identify what types of professional development are most needed in order to improve their skills. Of course, knowing what type of training is needed does not guarantee that it exists or is easily accessible to those who would benefit from it. Therefore, it is incumbent upon everyone in and around the CS transliteration community to ensure that appropriate training gets to those who need it; doing so will likely involve making more frequent trainings on a wider variety of topics available locally as well as providing transliterators with the means to travel to obtain training at remote locations. In addition to increasing access to professional development opportunities on topics specific to CS transliteration, increasing access to other transliterator training opportunities (e.g., oral transliteration, sign-based transliteration, etc.) may also be of benefit for developing many of the modality-general skills required of transliterators. That said, it is worth pointing out that although the overall accuracy of transliterators in this study was just 54%, this number is only an average, and higher accuracy was observed in a number of different circumstances. At the slow speaking rate, for example, average accuracy was 64%, and all individual transliterators exhibited higher accuracy at the slow rate than at the normal or fast rates. Presumably, accuracy could have been higher yet if the message were presented at an even slower rate. Although a slower rate is not likely to be a typical rate for conversational communication, it is possible that it may nonetheless be an appropriate representation of a deliberate pace used by some classroom teachers when presenting complex material. Thus, for future studies, it may be helpful to measure speaking rates of teachers in actual classrooms in order to determine what range of speaking rates is most appropriate to evaluate. In addition to speaking rate, experience was another factor associated with increased accuracy. On average, the accuracy of veteran transliterators was 58%, and this increased to 66% at the slow speaking rate. Of course, as described earlier, not every veteran transliterator achieved higher accuracy than every novice transliterator; however, the two veteran transliterators who held national certification in CS transliteration did so easily: CST07 s accuracy was 81% on average and 84% at the slow rate, whereas CST12 s accuracy was 84% on average and 89% at the slow rate. These performance levels are much more encouraging and underscore the importance of ongoing assessment and credentialing of all working transliterators.

9 Krause et al. 381 Intelligibility Although the overall accuracy of transliterators in this study was lower than expected, one possible explanation might lie in how accuracy (percentage of cues correctly produced by a transliterator) is related to intelligibility (percentage of speaker s words correctly received by a deaf consumer with fluent English skills who is proficient in CS). For a number of reasons, it is likely that intelligibility may be somewhat higher than accuracy. One reason is that the nature of CS requires that a transliterator mouth the message while producing the accompanying hand cues. Thus, even if some percentage of the cues produced within a particular word are incorrect or missing, the deaf consumer may still be able to recognize the word via a combination of speechreading and of extracting information from any correct cues that are present. Another possibility is that transliterators could have preserved the message content via the use of paraphrase or by producing content words, which carry the majority of the message s meaning, with higher accuracy than function words. Either of these strategies would preserve intelligibility of the original message but have a negative effect on accuracy because they involve deviations from the target (expected) cue sequence. Finally, it may be that some errors in accuracy are not particularly disruptive to transmission of the cued message; if the errors are predictable and relatively consistent, the deaf consumer may be able to compensate for or adjust to a transliterator s style of cueing, much like a listener can adjust to a talker s accent. Conclusions Although further research is needed, the results of the present study are an essential first step toward quantifying factors that affect the accuracy of CS transliterators. By (a) examining transliterators of varying experience levels and lag times and (b) systematically manipulating the speaking rate of the materials they were asked to transliterate, three important findings have been established. First, lag time has only a very small effect on accuracy, with longer lag times weakly associated with lower accuracy. Second, high levels of experience do not guarantee high levels of accuracy, although it appears that transliterators with more experience are likely to be more accurate than transliterators with minimal experience. In order to confirm this preliminary finding and gain insight into which transliterator factors are the best predictors of transliterator accuracy, more research is needed to examine additional transliterators at every experience level, from a wide variety of backgrounds (i.e., various credentials, amounts and types of training, etc.), for the relationship between transliterator characteristics (e.g., working memory, manual dexterity, etc.) and transliterator accuracy. Third, increases in speaking rate are associated with reductions in transliterator accuracy, primarily as a result of increased omissions; in novice transliterators, large chunks of the message tend to be dropped, whereas transliterators with more experience tend to omit shorter phrases as well as cues within words. Regardless of the nature of the omissions, it is easy to see why increased omissions would be a coping strategy for transliterators facing increased speaking rates; however, it is unknown how these changes affect a deaf child s ability to receive the message. Therefore, it is important that future research investigate transliterator intelligibility as a function of speaking rate and accuracy. Moreover, analogous accuracy and intelligibility experiments are needed for transliterators who use other communication modes with deaf children in classrooms (e.g., ASL, Conceptually Accurate Signed English, Signing Exact English, etc.). Such research would provide valuable information about what accuracy levels are needed from interpreters/transliterators in each modality at each speaking rate in order to ensure accessibility to the teacher s message and other classroom communication. Notes 1. The term visual signal used here is a shortened version of Battison s (1978) observation that sign language is a manually produced, visually received signal, whereas speech is an orally produced, auditorily received signal. 2. Whereas the function of an interpreter is to translate between two languages (e.g., spoken English and American Sign Language), the function of a transliterator is to transfer information between two modes of the same language (e.g., spoken English and either signed English or cued English). Thus, the term transliterator is used (most typically, Cued Speech transliterator or cued language transliterator) for interpreting professionals who transfer spoken English information to/from Cued Speech. Conflicts of Interest No conflicts of interest were reported. Funding National Institute on Deafness and Other Communication Disorders/National Institutes of Health (5 R03 DC ). Acknowledgments The authors wish to thank Dana Herrington, Katherine Lopez, and Jessica Vick for assistance with accuracy measurements and analyses, Wendy Fuchs for assistance with lag time measurements, and John Lum for assistance in stimulus creation. We also thank Joe Frisbie for the cue chart used in Figure 1. References Anonymous. (2013). U.S. Program and services chart. American Annals of the Deaf, 158, doi: /aad Battison, R. (1978). Lexical borrowing in American Sign Language. Silver Spring, MD: Linstok Press. Boersma, P., & Weenink, D. (2012). Praat: Doing phonetics by computer [computer program]. Version Retrieved July 10, 2012, from Cokely, D. (1986). The effects of lag time on interpreter errors. Sign Language Studies, 53, doi: /sls Cornett, R. O. (1967). Cued Speech. American Annals of the Deaf, 112, Films for the Humanities (Producer). (1989). The life cycle of plants [Film]. Available from Films Media Group, P.O. Box 2053, Princeton, NJ Fischer, S. D., Delhorne, L. A., & Reed, C. M. (1999). Effects of rate of presentation on the reception of American Sign Language. Journal of Speech and Hearing Research, 42, doi: / jslhr Krause, J. C., Kegl, J. A., & Schick B. (2008). Toward extending the Educational Interpreter Performance Assessment to Cued Speech. Journal of Deaf Studies and Deaf Education, 13, doi: /deafed/enm059

10 382 Journal of Deaf Studies and Deaf Education, 2016, Vol. 21, No. 4 Krause, J. C., Pelley-Lopez, K. A., & Tessler, M. P. (2011). A method for transcribing the manual components of Cued Speech. Speech Communication, 53, doi: /j.specom Krause, J. C., Schick, B., & Kegl, J. A. (2010). A version of the Educational Interpreter Performance Assessment for Cued Speech transliterators: Prospects and significance. In C. J. LaSasso, K. L. Crain, & J. Leybaert (Eds.), Cued Speech and cued language development of deaf students (pp ). San Diego, CA: Plural Publishing. Moores, D. E. (1992). A historical perspective on school placement. In T. N. Kluwin, D. E. Moores, & M. G. Gaustad (Eds.), Toward effective public school programs for deaf students: Context, process, and outcomes (pp. 7 29). New York, NY: Teachers College, Columbia University. Nicholls, G., & Ling, D. (1982). Cued Speech and the reception of spoken language. Journal of Speech and Hearing Research, 25, doi: /jshr Reed, C. M., Delhorne, L. A., Durlach, N. I., & Fischer, S. D. (1990). A study of the tactual and visual reception of fingerspelling. Journal of Speech and Hearing Research, 33, doi: / jshr Schick, B., & Williams, K. (1992). The educational interpreter performance assessment: A tool to evaluate classroom performance. Paper presented at the issues in Language and deafness: The use of sign language in educational settings: Current concepts and controversies, Omaha, NE. Schick, B., & Williams, K. (1994). The evaluation of educational interpreters. In B. Schick & M. P. Moeller (Eds.), Sign language in the schools: Current issues and controversies (pp ). Omaha, NE: Boys Town Press. Schick, B., Williams, K., & Bolster, L. (1999). Skill levels of educational interpreters working in public schools. Journal of Deaf Studies and Deaf Education, 4, doi: / deafed/ Schick, B., Williams, K., & Kupermintz, H. (2006). Look who s being left behind: Educational interpreters and access to education for deaf and hard-of-hearing students. Journal of Deaf Studies and Deaf Education, 11, doi: /deafed/ enj007 Strong, M., & Rudser, S. F. (1985). An assessment instrument for sign language interpreters. Sign Language Studies, 49, doi: /sls Strong, M., & Rudser, S. F. (1986). The subjective assessment of sign language interpreters. Sign Language Studies, 53, doi: /sls Uchanski, R. M., Delhorne, L. A., Dix, A. K., Braida, L. D., Reed, C. M., & Durlach, N. I. (1994). Automatic speech recognition to aid the hearing impaired: Prospects for the automatic generation of Cued Speech. Journal of Rehabilitation Research and Development, 31, U.S. Department of Education. (1999). To assure the free appropriate public education of all children with disabilities: Twenty-first annual report to Congress on the implementing of the Individuals with Disabilities Act. Washington, DC: U.S. Department of Education.