A New Stroboscopy Rating Form

Transcription

1 Journal of Voice Vol. 13, No. 3, pp Singular Publishing Group, Inc. A New Stroboscopy Rating Form Bruce J. Poburka Department of Communication Disorders Minnesota State University, Mankato, Minnesota, U.S.A. Summary: A newly developed stroboscopy rating form is presented. The evolution of the new form is discussed by focusing on how problems with another commonly used form led to its development. Interjudge agreement and intrajudge reliability data for the new rating form reveal that the parameters of phase closure, phase symmetry, and phase regularity remain difficult to rate even with the revised rating form. Key Words: Videostroboscopy--Rating form Visuoperceptual--Stroboscopy--Vocal fold function. In this article, a new stroboscopy rating form, the Stroboscopy Examination Rating Form (SERF), is introduced (Figure 1, A and B). The SERF was developed to help resolve selected stroboscopy rating problems that were identified in a previous study about stroboscopy rating training.~ The SERF is a relatively simple rating form that is intended for clinical use and/or research in which rating the basic stroboscopy parameters is necessary. The form is not designed for making more detailed observations that may be necessary in some research protocols. 2 Information is presented about how the new rating form was developed to improve on a previous stroboscopy rating form that is in widespread use. Finally, preliminary agreement and reliability data are presented. Accepted for publication October 1, Address correspondence and reprint requests to Bruce J. Poburka, PhD, CCC-SLP, Department of Communication Disorders, MSU 77, RO. Box 8400, Mankato State University, Mankato, MN , U.S.A. Presented at the annual meeting of the American Speech- Language-Hearing Association, November 1997, Boston, Massachusetts, U.S.A. DIFFICULTIES WITH PERCEPTUAL JUDGMENTS Stroboscopy allows dynamic assessment of laryngeal behavior. Clinicians use it to assess laryngeal function, guide treatment, and to evaluate the efficacy of treatment. Although recent advances in equipment have facilitated objective measurement from stroboscopy, 3 quantitative measurements are still somewhat difficult to obtain. 4 For clinical purposes, making visuo-perceptual judgments continues to be the most common use of stroboscopy. If visuo-perceptual judgments are to be used effectively in clinical situations, it is critical to achieve satisfactory levels of interjudge agreement and intrajudge reliability. Unfortunately, this is not always the case. Recent studies reporting interjudge agreement and intrajudge reliability have revealed less than satisfactory values. Ramos, Bless, Harmon, and Ford 5 reported interjudge reliability ranging from.20 to.85 and an overall interjudge reliability value of.72. Teitler 6 found intrajudge reliability values ranging from.31 to.97. Interjudge reliability ranged from.75 to.98. Poburka and Bless 1 found interjudge agreement to range from.21 to

2 404 BRUCE J. POBURKA Stroboscopy Evaluation Rating Form (SERF) Rater: Client: Date: Bruce J. Poburka, Ph.r Amplitude normal pitch & loudness) Mucosal Wave normal pitch & loudness) Right: % Left: % Right: % Left: % Fo: Fo: Non-vibrating Portion (shade in affected areas) Supraglottic Activity (Ignore voice onsets) A FIG. 1. A. The Stroboscopy Evaluation Rating Form (SERF). (continued) Journal of Voice, %l. 13, No. 3, 1999

3 A NEW STROBOSCOPY RATING FORM smooth Right Fold 2 3 Vocal Fold Edge Smoothness Left Fold 0 circle on rough smooth rough Vocal Fold Edge Straightness 0 1 straight Right Fold 2 3 Ocircle on Left Fold irregular straight 4 5 irregular pitch & loudness Vertical Level ~ircle one o on-plane.y: ~ off-plane Phase Closure point of contact % of time open closed Breath~ +90% <10% Q _~ 66% 33%.~ Norma ~, 33% 66% <10% +90% Pressed / Fr Frame count: open phase: Closed phase: Phase Symmetry point of contact % of time symmetrical Always assymmetrical O% 20% 40% ~, 60% 80% 100% Always symmetrical I Regularity % of time regular Always irregular 0% 20% 40% 60% 80% 100% Always regular Method(s) used: stop phase running phase Glottal Closure Hourglass Complete Posterior Incomplete ~ ~ Ga~ Anterior A Spindle Variable If closure pattern is variable, indicate the predominant closure pattern: Summary~Additional Comments: B FIG. 1 (continued) B. The Stroboscopy Evaluation Rating Form (SERF).

4 406 BRUCE J. POBURKA As with all perceptual evaluation procedures, the reasons for reduced agreement and reliability of stroboscopy ratings are varied. Hirano and Bless 7 identified a number of factors that can influence the process of obtaining and interpreting stroboscopic images. They included the observer's (1) knowledge of how vocal fold vibration relates to sound production; (2) knowledge of normal anatomy and physiology; (3) skill with the stroboscopic technique; and (4) skill in interpretation of the recorded image. Bless, Hirano, and Feder 8 indicated that, although basic skills come with relatively little training, some aspects of stroboscopy interpretation are complicated. Recognizing the need for improved reliability, a number 0f investigators have developed procedures for making stroboscopy observations more objective and reliable. These procedures include using consensus rating procedures and various image measuring techniques. 3,9,10 THE IMPETUS FOR A NEW STROBOSCOPY RATING FORM Poburka and Bless recently completed a: stroboscopy rating: training study. 1 During this study, a panel of experienced stroboscopy raters illuminated certain stroboscopy rating problems. Some of the rating difficulties appeared to be related to the rating form, the Stroboscopic Assessment of Voice (SAV), which is widely used (Figure 2, A and B). It was felt that agreement and reliability of ratings could be improved if the rating form was modified. Although it was not expected that a revised rating form will solve all problems, it may help resolve some of the issues that the raters felt might have interfered with the rating process. The problems that served as the impetus for the new rating form and their solutions are summarized below according to the stroboscopy parameter to which they pertain. General refinements The SERF form utilizes a "visual" approach to rating several of the stroboscopic parameters. The rater uses a superior view image of the larynx upon which to mark his or her judgments of selected parameters. The rater simply marks the laryngeal image to correspond with what he or she observes, The rater does not have to transform his or her observation into a scalar number or make an assessment of normalcy. This approach was intended to make rating somewhat more concrete. Additionally, several clarifying instructions (eg, ignore voice onsets for supraglottic activity, focus on point of contact) are included to facilitate rating. Supraglottic activity Several aspects of this parameter make rating difficult when using the SAV protocol. First, supraglottic activity can be observed in two directions. The first is medio-lateral, in which the ventricular folds are forced medially toward the midline. Severe mediolateral (M-L) compression can result in the true vocal folds being obscured from view'by the ventricular folds that lie just above the true folds. The second type of supraglottic activity is in the antero-posterior (A-P) direction. This type of compression is characterized by shortening of the aryepiglottic folds. This results in movement of the epiglottis posteriorly toward the arytenoids. Currently, the SAV form does not provide a place to rate the A-P type of compression (Figure 2A). Inexperienced raters may ignore the presence of A-P compression and rate only the M-L dimension that is outlined on the rating form. Experienced raters may be influenced by the A-P compression and their ratings may reflect their perception of either type of compression. This difference in rating technique may account for some of the lack of agreement that was observed in the earlier study. The SERF form features a laryngeal image with concentric circles superimposed (See Figure 1A). The rater evaluates M-L and A-P constriction separately by choosing which numbered circle best corresponds to the observed degree of constriction. Additionally, a recommendation to ignore voice onsets is included. Vocal fold edge On the SAV form, the parameter of vocal fold edge combines 2 phenomena on the same scale. The descriptive term on the normal end of the scale is labeled as "smooth/straight" and the other (pathological) end of the scale is labeled as "rough/irregular" (Figure 2A). The problem with this scale is that it is possible for a vocal fold edge to be smooth and not straight. For example, in the case of an intracordal

5 A NEW STROBOSCOPY RATING FORM 407 Glottic Closure Scale for rating glottic closure on the SAV form. There is no provision for rating a variable glottal closure Activity Slight com- Dysphonia pression of plica ventventricular ricularis V. folds folds not visible Scale for rating supraglottic activity on the SAV form. There is no means of differentiating antero-posterior (A-P) from medio-lateral (M-L) compression. @ of VF Approximation Glottic Plane Qff Vocal Fold Smooth Rough Edge Straight Irregular Right (~ (~) Scale for rating vocal fold edge on the SAV form. The scale combines two phenomena; smoothness and straightness which ideally should be rated separately. A Amplitude,~.,::b Slightly Moderately Severely Barely Normal Decreased Decreased Decreased Perceptible ~,~.t FIG. 2. A. Diagram illustrating the design of the SAV form with comments about selected parameters. (continued) No Visible Movement

6 408 BRUCE J. POBURKA LeftMuc sal Right Slightly Moderately Severely Barely Decreased Decreased Decreased Perceptible Absent Non-vibrating Portion None 20% 40o/o 60o/o 80o/o Phase Open phase predominates Normal Closed phase predominates (Glottal fryextreme Symmetry Symmetrical asymmetrical asymmetrical asymmetrical generally always during begin during during 50%+ asymmetrical asymmetrical or end tasks extremes of pitch or loud Scale for rating phase symmetry on the SAV form. The scale is a mix of percentages of time and descriptive statements. The lack of a uniform scale may confuse Regular irregular irregular irregular generally always during begin during during 50%+ irregular irregular or end tasks extremes of 75%+ pitch or loud Scale for rating regularity on the SAV form. The scale is a mix of percentages of time and descriptive statements. The lack of a uniform scale may confuse raters. B FIG. 2 (continued) B. Diagram illustrating the design of the SAV form with comments about selected parameters.

7 A NEW STROBOSCOPY RATING FORM 409 cyst, the pathology typically develops into a smooth bulge along the vocal fold. The fold would be smooth, but the bulge would impair the straightness of the edge. The rater is therefore forced to decide which feature to rate, the smoothness or the straightness. This created problems for raters in the earlier study) When rating a cyst, one of the judges rated the edge to be normal because he was attending to the smoothness while others assigned ratings consistent with impairment because they were attending to the straightness of the fold edge. The SERF features separate scales for smoothness and straightness ( See Figure 1B). This revision should help reduce confusion about which aspect to rate and should improve agreement among raters. Phase closure When evaluating phase closure, the rater must assess the relative durations of the open and closed phases of the vibratory cycle. Under normal vibratory conditions, the vocal cycle can be divided into 3 parts: (1) the opening part of the open phase; (2) the closing part of the open phase; and (3) the closed phase. The open phase is any portion of the cycle when there is a glottal space, whether the vocal folds are opening (moving laterally) or closing (moving medially). The closed phase is any time the glottis is closed. Rating phase closure can be a difficult task because the rater must in effect transform the pattern of vibration observed in the strobe examination into a mental image of a glottal waveform (Figure 3). The scale for rating phase closure on the SAV form (see J~ o 0 Open Phase -Closed Phase- FIG. 3. Glottal waveform showing the open and closed phases of the phonatory cycle. Figure 2B) ranges from a -5 (open phase predominates, eg, whisper dysphonia) to a +5 (closed phase predominates, eg, glottal fry or extreme hyperadduction). On the SERF, the rater chooses a pair of numbers (% open and closed) best representing the relative durations of the open and closed phases. It also provides a place to record numbers of frames involving the open and closed phases (see Figure 1B). In this procedure, the examiner counts the number of video frames comprising the open and closed phases of the phonatory cycle. This "stop frame" procedure should be regarded as an estimate or gross measure of the relative durations of the open and closed phases because strobe images are averaged images that are "sampled" from many phonatory cycles. An assumption is made that the strobe-averaged image accurately represents the actual vibratory behavior. The stop-frame procedure provides the rater with some objective information to help inform his or her rating. In the previous study, 1 the panel of raters presented an additional issue that further complicates the process of evaluating phase closure. There was a question of whether phase closure should be evaluated by observing overall motion of the vocal folds or only motion at the point of contact between the folds. This becomes an issue in conditions where there is incomplete closure along the glottis except for a single point of contact between 2 localized points (eg, an hourglass closure pattern) like the closure that is sometimes seen with bilateral prominent vocal fold masses. If the rater is considering the overall motion of the vocal folds, he or she may observe that closure is not accomplished along most of the glottal length and assign a rating that indicates a predominant open phase. However, if another rater considers only the point of contact, she may assign a more normal rating, based on the fact that the nodules axe sometimes contacting each other (during the closed phase) and not at other times (during the open phase). To help alleviate confusion over this matter, the SERF includes an instruction to rate at the point of vocal fold contact. Phase symmetry When rating phase symmetry, the rater evaluates the amount of time that the vocal folds are vibrating symmetrically. On the SAV, the rating scale for phase symmetry is a 0-5 scale with descriptors for each

8 410 BRUCE J. POBURKA point along the scale (see Figure 2B). The problem is that the descriptors are a mixture of percentages (eg, asymmetrical during 50%+) and descriptive statements (eg, asymmetrical during beginning or ending tasks; asymmetrical during extremes of pitch and loudness). Raters in the stroboscopy training study reported that it is confusing to use a scale where some points correspond to a percentage and others do not. The raters were uncertain if they should interpret the points on the scale that have descriptors, (eg, asymmetry at beginning/end of task) as simply a smaller percentage (eg, 25%) of time that vibration was asymmetrical. As with phase closure, an additional source of confusion was the question of whether the rater should focus on the point of vocal fold contact or on the motion of the entire vocal folds. The SERF has instructions to rate the percentage of time that vibration is symmetrical and all points on the scale are along a single continuum (% of time vibration is symmetrical) so that each point has relevance to the others. The revised form also indicates that the focus should be on the point of vocal fold contact (See Figure 1B). Regularity In the previous training study, 1 regularity was among the most problematic parameters to rate. The judges discussed several problems with rating regularity that may explain the lower levels of agreement and reliability. Specifically, there was discussion about the best method to use when evaluating this parameter. Regularity of vibration is defined as the degree to which one phonatory cycle is like the next. Regularity can be evaluated in 2 ways: with the stop phase or the running phase. These phases are controlled by user-selected settings on the stroboscopy unit. In the running phase, the rate of strobe light illumination is timed to be approximately 2 Hz different from the rate of vocal fold vibration. If the light is flashed at a slightly different rate than the rate of vocal fold vibration, each flash of the light will illuminate the vocal folds in a slightly different part of their vibratory cycle (the vocal folds cannot be seen when the strobe light is off). Conversely, during stop phase operation, the strobe light is timed to be in exact synchrony with the fundamental frequency. In this case, if vibration is regular, each flash will itlu- minate the vocal folds while they are in the same part of their vibratory cycle and the vocal folds will appear to stand still. If vibration is irregular, one vibratory cycle may have a slightly different duration than the next and the folds will be illuminated in a slightly different part of the cycle and then appear to quiver or shake. It is the degree of quivering or shaking that indicates the degree of irregularity of vibration. The conventional way to assess regularity is by using the stop phase. Because of time constraints, only a small portion of the stroboscopic examination may be completed using the stop phase. This is because examining all other strobe parameters requires the running phase. It may be desirable to examine regularity in a variety of phonatory tasks (eg, pitch or loudness changes) that are normally performed only using the running phase. Individuals who have strobosqopy rating experience can detect irregularity of vibrfiti0n even while viewing in the running phase. If vibration becomes irregular under the running phase, the strobe unit cannot adequately track the fundamental frequency and adjust the illumination rate. As a result, the image temporarily shimmers or quivers somewhat the way irregular vibration appears in the stop phase.this is useful information for the experienced rater and can be used in evaluating regularity. A problem that emerged during the previous study was whether regularity should only be rated with use of the stop phase. Some judges reported that their ratings were based on observations of laryngeal behavior using the only stop phase. Others made their judgments based on both the running and stop phases. As a result, it is likely that judgments were made using different parts of the examination and under different viewing conditions. Both of these factors may have reduced agreement among judges. An additional problem is that, like phase symmetry, the scale was a mix of percentages of time that vibration was irregular and descriptive statements requiring the rater to interpret how to use the scale (see Figure 2B). The SERF features a place where the rater can indicate which mode of operation (running phase or stop phase) was used in making his or her rating of regularity. This information may be useful in resolving differences in ratings between judges or in making follow-up observations after treatment Additionally, the form has a uniform scale of percentage of time that vibration was regular (see Figure 1B).

9 A NEW STROBOSCOPY RATING FORM 411 Glottic closure In the initial study, 1 the parameter of glottic closure did not have favorable agreement among raters even after training (67% agreement). In some ways, this was surprising because glottal closure does not require an assessment of any complex movement patterns. Some of the differences in rating may have related to a limitation in the rating protocol that was used. The SAV rating form does not provide a means for the rater to indicate if a variable glottal closure pattern is observed (see Figure 2A). When rating pathological phonatory patterns, the glottic closure pattern may change from one part of the examination to the next. If a rater makes an observation of a certain glottal closure pattern, his or her rating may be accurate only for a portion of the strobe examination because of pattern variability. A different rater may rate glottal closure during a different part of the examination and his or her rating would differ. A variable glottal closure pattern may be valuable diagnostically since it may indicate problems with phonatory control. The SERF has been designed to include variable as a glottal closure pattern choice. It also provides a space to indicate the closure pattern that predominated during the examination (see Figure 1B). PRELIMINARY DATA ON AGREEMENT AND RELIABILITY USING THE SERF Three judges who were speech-language pathologists specializing in voice with an average of approximately 9 years of stroboscopy rating experience rated the same 42 video samples that were used in the earlier study) The samples ranged from normal phonation to severe phonatory dysfunction. The ratings were carried out on a self-paced basis. Video equipment and rating conditions were not controlled. Tables 1 and 2 summarize the agreement and reliability data for the ratings. Table 1 shows the percentage of exact agreement (when all 3 judges rated identically) and the percentage of ratings where 2 of the 3 judges rated identically (consensus). Although the percentages of exact agreement are not particularly impressive, the consensus data are encouraging. The most problematic parameters for agreement were mucosal wave, phase closure, phase symmetry, and phase regularity. The high levels of agreement and reliability for the parameters of vertical level and nonvibrating portion are somewhat misleading. These values are high in part because there were only a few video samples in which there were nonvibrating portions or where the vertical level was not normal. Table 2 shows intrajudge reliability for 2 judges (reliability was not calculated for judge 3 who was the principal investigator and was aware that some samples were repeated for assessing reliability). Although the judges achieved mostly high levels of reliability, the parameters of phase closure, phase symmetry, and phase regularity were once again the most problematic to rate. The agreement and reliability data indicate that phase closure, phase symmetry, and phase regularity TABLE 1. Interjudge Agreement by Stroboscopy Parameter for 3 Judges Interjudge Agreement (3 Judges) Parameter % Exact % Consensus* Glottal Closure Amplitude Left Right Mucosal Wave Left Right 2 68 Non-vib. portion Left Right Supraglottic Activity AP ML Edge-Smoothnessft

10 412 BRUCE J. POBURKA TABLE 2. Intrajudge Reliability by Stroboscopy Parameter for 2 Judges Intrajudge Reliability Judge 1 Judge 2 Parameter % Exact %+/-1 %> +/-1 %Exact %+/-1 %>+/-1 Glottal Closure 100 NA Amplitude Left Right Mucosal Wave Left Right Non-vib. portion Left Right Supraglottic Activity AP ML Edge-Smoothness Left Right Edge-Straightness Left Right Vertical Level Phase Closure Phase Symmetry Regularity NA continue to be stumbling blocks in the stroboscopy rating process. The modifications in the rating form did not sufficiently resolve the problems with rating these parameters and they will most likely require special emphasis in any stroboscopy rating instruction. SUMMARY AND DISCUSSION The SERF form was developed in an effort to resolve problems that were revealed in a previous stroboscopy rating study using the SAV form. In the previous study, 3 judges using the SAV form reached consensus on 86% of all ratings. Using the SERF, the judges reached consensus on 83% of all ratings. While the outcomes are essentially the same, there are some differences between the 2 studies. First, the judges were not the same in the 2 studies. Only 1 of the 3 judges participated in both studies. In the earlier study, the 3 judges were all in the same room so the video equipment and rating conditions were identical. These factors were not controlled in the present study. In general, the judges had a favorable response to the SERE They reported a number of advantages including faster rating times, less hesitation in making the ratings, and less confusion over left and right orientations on the form. All of the judges reported that, on average, they spent 4 to 5 minutes per case when using the SERF to complete their ratings. Some of these advantages may add to the form's utility as a stroboscopy rating tool for clinical and research purposes.

11 A NEW STROBOSCOPY RATING FORM 413 After completing the ratings for this project, the judges and other professionals provided suggestions for further modification of the rating form. These modifications included changing the vertical level parameter to a dichotomous rating of on plane/off plane and changing the nonvibrating portion grid to include only the anterior ~ of the vocal folds. These suggestions are reflected in the rating form presented in this article (see Figure 1) but were not implemented at the time of the ratings. Copies of the SERF can be obtained by contacting the author. REFERENCES 1. Poburka B J, Bless DM. A multi-media, computer-based method for stroboscopy rating training. J Voice. In press. 2. Khidr A, Ramos C, Bless DM, Heisy D. Resolving the battle between internal and external standards for visual perceptual ratings of laryngeal images: an essential step towards reliable research protocol. Paper presented at: the meeting of the American Speech-Language-Hearing Association; November 1997; Boston, Mass. 3. Sercarz J, Berke G, Arnstein D, Gerratt B, Natividad M. A new technique for quantitative measurement of laryngeal videostroboscopic images. Ear Nose Throat J. 1991;117: Wendler J. Stroboscopy. J Voice. 1992;6: Ramos C, Bless DM, Harmon R, Ford C. The mucosal wave as a prognostic sign in vocal paralysis. Paper presented at the meeting of the American Speech-Language- Hearing Association; November 1993; Anaheim, Calif. 6. Teitler N. Examiner Bias: Influence of Patient History on Perceptual Ratings of Videostroboscopy [master's thesis]. Madison, Wisc: University of Wisconsin-Madison; Hirano M, Bless DM: Videostroboscopic Examination of the Larynx. San Diego, Calif: Singular Publishing Group Inc; Bless DM, Hirano M, Feder R. Videostroboscopic evaluation of the larynx. Ear Nose Throat J. 1987;66(7): Peppard R. Effects of Aging on Selected Vocal Characteristics of Female Singers and Non-singers [Dissertation]. Madison, Wisc: University of Wisconsin-Madison; Peppard R, Bless DM. A method for improving measurement reliability in laryngeal videostroboscopy. J Voice. 1990;4: