Assessment of Pediatric Obstructive Sleep Apnea Using a Drug-Induced Sleep Endoscopy Rating Scale

The Laryngoscope VC 2016 The American Laryngological, Rhinological and Otological Society, Inc. Assessment of Pediatric Obstructive Sleep Apnea Using a Drug-Induced Sleep Endoscopy Rating Scale Derek J. Lam, MD, MPH; Edward M. Weaver, MD, MPH; Carol J. Macarthur, MD; Henry A. Milczuk, MD; Eleni O Neill, MPH; Timothy L. Smith, MD, MPH; Thuan Nguyen, MD, MS, PhD; Steven A. Shea, PhD Objectives/Hypothesis: Assess the reliability of a Sleep Endoscopy Rating Scale (SERS) and its relationship with pediatric obstructive sleep apnea (OSA) severity. Study Design: Retrospective case series of pediatric patients who underwent drug-induced sleep endoscopy (DISE) at the time of surgery for OSA from January 1, 2013 to May 1, 2014. Methods: Three blinded otolaryngologists scored obstruction on DISE recordings as absent (0), partial (11), or complete (12) at six anatomic levels: nasal airway, nasopharynx, velopharynx, oropharynx, hypopharynx, and arytenoids. Ratings were summed for a SERS total score (range, 0 12). Reliability was calculated using a j statistic with linear weighting. SERS ratings and obstructive apnea-hypopnea index (OAHI) were compared using Spearman correlation. A receiver operating characteristic (ROC) analysis determined the ability of the SERS total score to predict severe OSA (OAHI >10). Results: Thirty-nine patients were included (mean age, 8.3 6 5.1 years; 36% obese; mean OAHI, 19.1 6 23.7). Intrarater and inter-rater reliability was substantial-to-excellent (j 5 0.61-0.83) and fair-to-substantial (j 5 0.33-0.76), respectively. Ratings correlated best with OAHI for the oropharynx (r 5 0.54, P 5.02), hypopharynx (r 5 0.48, P 5.04), and SERS total score (r 5 0.75, P 5.002). In ROC analysis, a SERS total score 6 demonstrated sensitivity/specificity of 81.8%/87.5%, respectively, and correctly classified 84% of patients. Conclusions: The SERS can be applied reliably in children undergoing DISE for OSA. Ratings of the oropharynx, hypopharynx, and SERS total score demonstrated significant correlation with OSA severity. A SERS total score 6 was an accurate predictor of severe OSA. Key Words: Pediatric sleep apnea, obstructive sleep apnea, drug-induced sleep endoscopy, adenotonsillectomy. Level of Evidence: 4. Laryngoscope, 126:1492 1498, 2016 INTRODUCTION Recent evidence has demonstrated that up to 30% of children undergoing adenotonsillectomy (AT) for treatment of pediatric obstructive sleep apnea (OSA) will have significant residual disease, 1 4 likely due to obstruction at locations besides the tonsils or adenoids. 5,6 Drug-induced sleep endoscopy (DISE) is a method of evaluating the upper airway using a flexible fiberoptic endoscope inserted From the Department of Otolaryngology Head and Neck Surgery (D.J.L., C.J.M., H.A.M., E.O., T.L.S.), Oregon Health and Science University, Portland, Oregon; Department of Otolaryngology Head and Neck Surgery (E.M.W.), University of Washington, Seattle, Washington; Department of Public Health and Preventive Medicine, Division of Biostatistics (T.N.), Oregon Health and Science University, Portland, Oregon; and the Oregon Institute of Occupational Health Sciences (S.A.S.), Oregon Health and Science University, Portland, Oregon, U.S.A. Editor s Note: This Manuscript was accepted for publication December 1, 2015. Presented at the American Academy of Otolaryngology Head and Neck Surgery Annual Meeting, Orlando, Florida, U.S.A., September 21 24, 2014. All work was performed at the Oregon Health and Science University, Portland, Oregon, U.S.A. The authors have no funding, financial relationships, or conflicts of interest to disclose. Send correspondence to Derek J. Lam, MD, Department of Otolaryngology Head and Neck Surgery, PV-01, Oregon Health and Science University, 3181 SW Sam Jackson Park Rd., Portland OR 97221. E-mail: lamde@ohsu.edu DOI: 10.1002/lary.25842 1492 transnasally during light sedation and spontaneous breathing that attempts to recreate natural sleep. 7 DISE allows direct observation of dynamic airway collapse throughout the pharynx. This technique has previously been shown to be reliable in both adults and children and has been shown to be able to discriminate snorers from nonsnorers. 6,8 11 Several scoring systems for standardizing the findings during DISE have been published. 6,10,12 However, none has attempted to propose a scale with a total score indicating overall obstruction that includes the nasal airway as part of the scoring. Because nasal obstruction has been implicated as an important potential contributor to upper airway obstruction and OSA, 13,14 we sought to include the nasal airway and further refine and simplify existing scoring systems. The objectives of this study were to 1) propose a Sleep Endoscopy Rating Scale (SERS) assessing the airway from the nose to the supraglottis and evaluate its reliability, and 2) assess the relationship between baseline severity of OSA and the sleep endoscopy findings, both at individual anatomic levels and for the SERS total score. MATERIALS AND METHODS Study Design and Subjects This was a retrospective case series of 39 consecutive patients who underwent DISE as part of their evaluation for

Fig. 1. Sleep Endoscopy Rating Scale scoring scheme. [Color figure can be viewed in the online issue, which is available at www.laryngoscope. com.] OSA between January 1, 2013 and May 1, 2014 at Doernbecher Children s Hospital, a tertiary care pediatric hospital. OSA was diagnosed either by preoperative polysomnography with obstructive apnea-hypopnea index (OAHI) 2 or by clinical evaluation including positive screen on the Michigan Pediatric Sleep Questionnaire (PSQ). 15 In surgically na ıve patients, DISE was performed either for patients with small tonsils during initial clinical exam or for those considered high risk for residual OSA after AT (e.g., patients with severe baseline OSA defined as OAHI >10, obesity, Down syndrome, craniofacial anomalies, 1493

or neuromuscular disorders). 3,4,16 19 Baseline measures of OSA severity included polysomnography parameters (e.g., OAHI), sleep apnea quality of life (OSA-18), 20 and PSQ scores. Electronic medical records were reviewed for all clinical data including the previously mentioned questionnaires, which are routinely administered in the clinic and scanned into the electronic medical record. This study was approved by the Oregon Health and Science University Institutional Review Board. Sleep Endoscopy Rating Scale DISE was performed using a flexible fiberoptic endoscope advanced transnasally under sedation using a combination of dexmedetomidine (bolused at 1 2 lg/kg over 5 10 minutes, followed by infusion at 0.5 1 lg/kg/hr) and ketamine (1 mg/kg bolus up to 40 mg after starting dexmedetomidine infusion, supplemented with 0.25-0.5 mg/kg boluses to maintain adequate sedation). Endoscopies were digitally recorded and later scored anonymously and independently by three pediatric otolaryngologists (D.J.L., C.J.M., H.A.M.). Ratings of the same videos were repeated approximately 1 month after the initial ratings were done. Endoscopic examinations were scored according to the SERS shown in Figure 1, with only the pictorial guides and without prior training. The degree of obstruction was scored as absent (0), partial (1), or complete (2) at the following six anatomic levels: nasal airway, nasopharynx (including adenoids), velopharynx, oropharynx (including palatine tonsils if present), hypopharynx (including the epiglottis), and arytenoids. The text descriptions included in Figure 1 were added to the figure after all ratings were completed by all raters using only the pictorial guide. These clarifications were later included to help address areas of uncertainty among the three raters in order to help improve on future reliability. Patients were kept supine with the head in a neutral midline position without neck extension. In situations where upper airway obstruction obscured the end of the endoscope, chin lift or jaw thrust were employed to help establish an adequate view at each level, and thereafter the patient s airway was allowed to relax until the dynamic patterns of collapse could be observed. For the purposes of simplification, ratings were based on the maximal degree of closure observed at each anatomic level. Because we were not aware of any well-validated assessments of nasal endoscopic findings, we chose to develop our own endoscopic scoring scale for the nasal airway as described in Figure 1. Ratings of the nasal airway were assessed without the use of nasal decongestion. Ratings of velopharyngeal obstruction were often difficult to separate from the palatine tonsils due to their close anatomic relationship, but the decision was made to consider any dynamic closure of the velopharyngeal port as part of this anatomic site, even if it appeared at least partially attributable to the palatine tonsils. Fluttering of the soft palate noted with active snoring was also scored as partial obstruction. In ratings of the hypopharynx, epiglottic retroflexion and effacement were included in this category, as it is often glossoptosis or lingual tonsil hypertrophy that causes the epiglottic retroflexion. In addition to the ratings at individual anatomic sites, the scores at each site were also summed to yield a SERS total score representing the overall degree of obstruction throughout the upper airway, ranging from 0 to 12. Statistical Analysis Inter-rater and intrarater reliability of the SERS scores were assessed using the j statistic with linear weighting, adapted for use with multiple raters. 21 Because ratings of all 39 subjects could not be completed by one of the raters, scores from all three raters were only available for the first 20 subjects; scores for all 39 subjects were available for only two of the raters (D.J.L., C.J.M.). Intrarater reliability was calculated using all available rating data for each rater. To determine whether the limitation of inter-rater reliability analysis to only two raters negatively impacted or biased the reliability results, sensitivity analysis was performed by calculating inter-rater reliability for all three raters using data from only the first 20 subjects and comparing this to analysis using data from only two raters (D.J.L., C.J.M.) for all 39 patients. The results were not substantially different: the difference in j statistic at any level was 0.04 with a slightly higher j at the nasopharynx, oropharynx, and supraglottis, and a slightly lower j at the nasal airway, velopharynx, and hypopharynx when scores for all three raters were included. Based on this sensitivity analysis, only data for all 39 subjects from two raters (D.J.L., C.J.M.) were presented. These scores were then averaged, and Spearman correlation coefficients were calculated between the OAHI and ratings at individual anatomic sites as well as for the SERS total score. To quantify the relative contributions of the different anatomic sites to overall disease severity, univariate linear regression was used to determine the association between mean SERS ratings at the individual anatomic sites and OAHI. A scatter plot of OAHI versus SERS ratings with linear fits and analysis of regression residual statistics were performed to identify potential outliers. Mann-Whitney tests were used to compare OAHI distributions among subgroups defined by the number of sites of obstruction. To further characterize the relationship between the SERS total score and potentially improve its clinical utility, a receiver operating characteristic (ROC) analysis was performed to determine what value of the SERS total score would best predict the presence of severe OSA, defined as OAHI >10 or 10. The area under the ROC curve was calculated to determine the overall ability of the SERS total score to differentiate patients with severe OSA. RESULTS Thirty-nine patients underwent DISE at the time of surgery for OSA. Of these, 24 patients underwent polysomnography prior to surgery. Twenty-nine patients underwent AT, five patients underwent tonsillectomy alone, two patients underwent supraglottoplasty and expansion pharyngoplasty, and two patients underwent diagnostic airway evaluation only. The cohort included children aged 4 months to 17 years, with a mean age of 8 years. Body mass index covered a very broad range (1st percentile to 99th percentile) with a median z score of 1.1 (50th percentile) and a mean of 2.8 (99th percentile). Just over one-third of the cohort was considered morbidly obese (>99th percentile). Nearly all patients had a positive PSQ screen for OSA, and the majority of patients had severe pediatric OSA. There was a fairly broad distribution of SERS scores and tonsil size (Table I). Reliability of SERS Ratings at Specific Anatomic Sites and Relationship With OSA Severity Intrarater reliability ranged from 0.61 to 0.83 at individual anatomic sites (Table II). Inter-rater reliability ranged from 0.33 to 0.76, with moderate-tosubstantial reliability (j 5 0.41 0.60 and 0.61 0.80, defined as moderate and substantial, respectively) 22 observed for ratings of the nasal airway, oropharynx, and hypopharynx. Among these sites, the mean SERS 1494

Characteristic TABLE I. Baseline Characteristics (N 5 39). Value Demographics Age at time of surgery, yr Mean 6 SD 8.3 6 5.1 Median (range) 8.3 (0.25 to 17.4) BMI z score Mean 6 SD 2.7 6 4.3 Median (range) 1.1 (28.22 to 8.22) Obese, N (%) 14 (36) Male, N (%) 25 (64) Comorbidities, N (%) Down syndrome 3 (8) Neuromuscular disorder 3 (8) Craniofacial anomaly 4 (10) Other* 7 (18) Tonsil size, N (%) Surgically absent 1 (3) 11 5 (13) 21 8 (21) 31 20 (53) 41 4 (11) Polysomnography OAHI, events/hr, N (%) 24 (62) Mean 6 SD 19.1 6 23.7 Median (range) 10.7 (0.4 to 102.4) SpO 2 nadir, %, mean 6 SD 83.3 6 9.0 ODI, events/hr, mean 6 SD 17.6 6 21.4 Questionnaire scores Michigan SRBD domain of PSQ Mean score 6 SD 0.57 6 0.20 Positive screen, score 37 (95) >0.33, N (%) OSA-18 Mean 6 SD 70 6 19 Score <60, small impact 11 (28) on HRQL, N (%) Score 60, 80, moderate 14 (36) impact on HRQL, N (%) Score >80, large impact on 8 (21) HRQL, N (%) Procedures at time of DISE, N AT 29 Tonsillectomy only 5 Supraglottoplasty 1 Expansion pharyngoplasty 1 Microlaryngoscopy/bronchoscopy 2 *Includes asthma, acrodysostosis, congenital heart disease. PSQ score 5 number of positive responses/number of positive and negative responses. AT 5 adenotonsillectomy; BMI 5 body mass index; DISE 5 druginduced sleep endoscopy; HRQL 5 Health Related Quality of Life; OAHI 5 obstructive apnea-hypopnea index; ODI 5 oxygen desaturation index; OSA-18 5 Obstructive Sleep Apnea Quality of Life Questionnaire; PSQ 5 Pediatric Sleep Questionnaire; SD 5 standard deviation; SpO 2 5 blood oxygen saturation; SRBD 5 sleep-related breathing disorder. ratings most strongly correlated with OAHI were the oropharynx (r 5 0.54, P 5.02) and hypopharynx (r 5 0.48, P 5.04). In linear regression analysis, ratings of these same sites demonstrated significant associations with OAHI (b 5 10.6, 95% confidence interval [CI]: 0.8 to 20.4 and 7.9, 95% CI: 0.02 to 15.8, respectively) (Table III). Relationship of Overall Obstruction and OSA Severity When we examined the severity of OSA based on the number of sites of complete obstruction, we found significantly worse baseline OAHI in those patients with two or more sites of complete obstruction compared to those with only partial obstruction or a single site of complete obstruction (OAHI 5 31.2 6 29.1 vs. 7.0 6 3.9, respectively, P 5.03) (Fig. 2). The strongest correlation with OAHI was observed for the SERS total score (r 5 0.75, P 5.002) (Table III). Examination of the scatter plot of OAHI versus SERS total score demonstrated a single outlier with OAHI >100 that was confirmed to be a significant outlier based on analysis of residual statistics (Fig. 3). When excluding this outlier, a strong correlation remained and a linear regression line fit to the plot showed a significant association between increasing SERS total score and increasing OAHI (b 5 4.9, 95% CI: 1.2 to 8.5) (Table III). In ROC analysis, using a dichotomous outcome of OAHI >10 or 10, we found that a SERS total score of 6 or greater demonstrated sensitivity and specificity of 81.8% and 87.5%, respectively, and would correctly classify 84% of patients. In general, the SERS total score had an area under the curve of 0.90 (95% CI: 0.77 to 1.00), indicating an excellent accuracy in predicting the presence of severe OSA (Fig. 4). DISCUSSION Although the majority of children undergoing AT for OSA show significant improvement, and a substantial proportion are cured, several studies have demonstrated that at least 20% of children undergoing AT will have residual OSA after surgery. 3,4,23 Specific populations of children have demonstrated even greater prevalence of OSA after AT (Down syndrome, 24,25 obese children, 3 5 African American children 3 ). However, in a more general population, or even within these high-risk TABLE II. Sleep Endoscopy Rating Scale Reliability (N 5 39). Intrarater Reliability, j (95% CI) Inter-rater Reliability, j (95% CI) Nasal airway 0.61 (0.40 to 0.84) 0.64 (0.42 to 0.88) Nasopharynx 0.75 (0.57 to 0.94) 0.36 (0.14 to 0.60) Velopharynx 0.83 (0.67 to 0.99) 0.35 (0.11 to 0.60) Oropharynx 0.71 (0.52 to 0.92) 0.50 (0.29 to 0.73) Hypopharynx 0.83 (0.70 to 0.97) 0.76 (0.60 to 0.93) Supraglottis 0.69 (0.41 to 0.98) 0.33 (0.11 to 0.57) CI 5 confidence interval. 1495

TABLE III. SERS Ratings Correlation and Association With OAHI (N 5 18). SERS Score, Mean 6 SD Spearman Correlation r (P Value) Unadjusted Association, b (95% CI)* Nasal airway 0.83 6 0.60 0.32 (.19) 21.25 (213.20 to 10.69) Nasopharynx 0.95 6 0.57 0.31 (.19) 10.11 (21.65 to 21.87) Velopharynx 1.21 6 0.61 0.30 (.22) 5.60 (26.97 to 18.18) Oropharynx 1.26 6 0.61 0.54 (.02) 10.57 (0.79 to 20.35) Hypopharynx 0.76 6 0.79 0.48 (.04) 7.91 (0.02 to 15.80) Supraglottis 0.39 6 0.45 0.35 (.15) 3.27 (214.14 to 20.68) Total score 5.62 6 1.62 0.75 (.002) 4.86 (1.23 to 8.48) SERS scores ranged from 0 to 2 for individual levels, 0 to 12 for total score. *Based on univariate linear regression analysis, excluding outlier with OAHI >100. Statistically significant result with P <.05. CI 5 confidence interval; OAHI 5 obstructive apnea-hypopnea index; R 5 correlation coefficient; SD 5 standard deviation; SERS 5 Sleep Endoscopy Rating Scale. populations, it remains unclear who is most likely to fail AT and require further workup and treatment. There is a growing body of literature investigating the use of DISE to help identify anatomic risk factors for AT failure in children. The occurrence of occult or sleep-state dependent laryngomalacia has been well documented as a source of pediatric OSA. 26 29 Lingual tonsil hypertrophy and glossoptosis have also been implicated as common causes of obstruction in patients with residual OSA after AT. 30 32 This study was undertaken to determine whether a simple SERS can reliably identify patterns of obstruction that are associated with more severe OSA and therefore may be at greater risk for failure of AT. We developed a subjective SERS with three degrees of obstruction, similar to those used in previous reports. 6,10,12,33 Though the rating scale described in this study is a modification of the VOTE (Velum, Oropharynx, Tongue Base, Epiglottis) scale originally proposed by Kezirian et al., 12 it is important to distinguish between the current rating scale and the semiquantitative VOTE rating system, which describes the obstructive pattern at individual anatomic sites without an overall scoring paradigm. The VOTE scale was originally conceived as a surgical planning tool in adults with OSA to facilitate standardized characterization of the pattern of dynamic collapse observed during drug-induced sleep. However, it was not intended to be used as an assessment of disease severity or as a prognostic tool and was not validated for this purpose. A similar scoring system described by Chan et al. was developed with a goal of standardizing the DISE evaluation in children. 10 The current SERS further builds on their work by adding an assessment of the nasal airway and simplifies its implementation by focusing on only maximal closure with three degrees of obstruction rather than four. In this study, the inter-rater and intrarater reliability were at least fair and substantial, respectively, similar to the reliability of DISE demonstrated in other studies. In the report by Chan et al., 10 both intrarater and inter-rater reliability were similar to what we found, despite using a different statistical method for evaluating reliability. We chose to use a linear-weighted j statistic adapted for multiple raters, which is the preferred statistic for evaluating categorical or ordinal data as was Fig. 2. Severity of obstructive sleep apnea based on sites of obstruction during drug-induced sleep endoscopy. [Color figure can be viewed in the online issue, which is available at www.laryngoscope. com.] Fig. 3. Obstructive sleep apnea severity by Sleep Endoscopy Rating Scale total score. Arrow indicates outlier. [Color figure can be viewed in the online issue, which is available at www.laryngoscope. com.] 1496

Fig. 4. Receiver operating characteristic (ROC) curve of Sleep Endoscopy Rating Scale total score and severe obstructive sleep apnea (obstructive apnea-hypopnea index >10 or 10). [Color figure can be viewed in the online issue, which is available at www.laryngoscope.com.] the case here. The intraclass correlation utilized by Chan et al. is also a common method for analyzing the reliability of ordinal data, and in certain situations is equivalent to a quadratic-weighted j analysis. When we repeated our reliability analysis using the intraclass correlation, our reliability results were slightly improved across all anatomic sites (data not shown), but overall the reliability was comparable. We chose to present the linear-weighted j analysis, which we felt to be a more appropriate approach for our data. When we evaluated how well the SERS ratings reflected the severity of OSA, we found significantly worse severity of baseline OSA in patients with multiple levels of complete obstruction compared to those with one or fewer levels of complete obstruction. This finding makes intuitive sense and is in keeping with previous investigations of DISE in both adult and pediatric OSA populations; however, these findings are not universal and may depend on the scoring algorithms used. 34 Investigation of the relationship between specific anatomic levels and OSA severity demonstrated significant correlation between the OAHI and the degree of obstruction at the oropharynx, hypopharynx, and the SERS total score. These results suggest that obstruction at the level of the oropharynx (including tonsils) and the hypopharynx (primarily tongue base) are likely the most important contributors to the severity of baseline OSA. Correlations at other levels of the airway, although nonsignificant in this analysis, were all in the positive direction, and it is possible that with greater numbers these might achieve statistical significance. Based on these findings, we hypothesize that patients with multilevel obstruction, in particular those with significant hypopharyngeal or base of tongue obstruction, would be more likely to have significant residual OSA after AT. This hypothesis is consistent with previous reports using DISE to assess the pattern of obstruction in patients who have failed previous AT. Truong et al. 33 as well as Durr et al. 35 have reported that a majority of their pediatric OSA patients who underwent DISE were noted to have multilevel obstruction, and the base of tongue was the most common single site identified. It is worth noting that there did not appear to be a significant association between ratings of the nasal airway and OSA severity, which is one of the distinguishing features of our scoring system. However, this may be at least in part because of the small sample size and the fact that this was a selected population that included a high proportion of morbidly obese patients who were surgically na ıve and undergoing AT. In this population, it is possible that the nasal airway is a relatively small contributor to overall obstruction, and there may be a stronger association with severity of obstruction in patients who have had an AT and still have residual OSA or who are not obese. Regardless, we feel that inclusion of ratings of the nasal airway offers a more comprehensive assessment of the upper airway that requires little additional effort to perform. Prospective studies with larger sample sizes are planned to address this question. The proposed SERS has been designed to assess the entire upper airway from the nose to the larynx while maintaining sufficient simplicity to allow its use by general or pediatric otolaryngologists in the course of routine practice. It is intended to facilitate prognostication of surgical outcome, but we expect that individual surgeons will augment the current rating scale with more descriptive patterns of collapse similar to the original description of the VOTE rating system to further inform surgical decision making. The SERS total score is a gross overall measure of obstruction that is easy to calculate and simplifies the findings during DISE. Because the SERS total score showed a strong correlation and significant association with baseline OSA severity, it can potentially be used to help identify patients who may be at risk for residual OSA after AT and therefore warrant closer postoperative follow-up, including the need for postoperative polysomnography. The ROC analysis demonstrated that a SERS total score of 6 or greater correctly identifies patients with severe OSA (OAHI >10) 84% of the time. Because severe baseline OSA is one of the primary risk factors for residual OSA after AT, we hypothesize that this value may be useful in identifying patients who should be followed postoperatively, especially if they did not undergo pretonsillectomy polysomnography. Once these patients are identified, the specific findings during DISE can potentially help to direct the surgical approach should the need for further surgery for OSA be necessary. Strengths of this study include the use of videos that were independently and repeatedly reviewed by pediatric otolaryngology staff blinded to the patient identity or treatment course as well as the broad range of baseline OSA severity that improves generalizability. However, there are important limitations as well. The study included a relatively small number of patients that limited the statistical power and analyses that could be performed. In addition, we found that even with the visual aid shown in Figure 1 to help standardize the subjective SERS ratings, there was relatively poor inter-rater reliability of ratings of the nasopharynx, 1497

velum, and supraglottis. This may be partly attributable to using ratings from only two raters for our calculation of the inter-rater reliability. However, the sensitivity analysis we performed using data from only the first 20 subjects that included ratings from all three raters yielded results that were not substantially different, suggesting minimal bias due to the small number of raters. To further improve upon the SERS reliability, we discussed specific subjective discrepancies in the ratings at these levels and refined our definitions of partial and complete obstruction. The refined definitions were added to Figure 1 subsequent to the results presented here. The current reliability results are based on an initial set of ratings made without specific instruction beyond the example photographs presented in Figure 1, which was initially developed and used without the additional text clarifications shown in the figure. We are currently collecting data on subsequent ratings to test the reliability with these refinements. In addition, we focused our analysis of the relationship between the SERS ratings and OSA severity on the OAHI and did not account for other sleep study parameters. We acknowledge that the OAHI does not capture the totality of an individual patient s sleep disturbance or disease severity; however, this parameter remains the primary one upon which most treatment decisions are based, so we felt this was a reasonable compromise. Future analysis will also test the association between the SERS and other measures of OSA burden, such as the OSA-18. Finally, our analysis assessed the relationship of the SERS ratings and baseline OSA severity, but does not directly address the question of whether these ratings assessed prior to surgery can accurately predict the response to treatment. CONCLUSION The SERS described in this study can be applied reliably in children undergoing DISE for OSA. We were able to demonstrate significant correlation between baseline OSA severity and obstruction observed in the oropharynx and hypopharynx as well as the SERS total score. We further demonstrated that a SERS total score of 6 or greater was an accurate predictor of severe OSA. The potential to prognosticate the outcome of AT using the SERS has important implications in identifying patients who would benefit from post-tonsillectomy polysomnography and possibly further treatment for residual OSA. BIBLIOGRAPHY 1. Mitchell RB. Adenotonsillectomy for obstructive sleep apnea in children: outcome evaluated by pre- and postoperative polysomnography. Laryngoscope 2007;117:1844 1854. 2. Tauman R, Gulliver TE, Krishna J, et al. Persistence of obstructive sleep apnea syndrome in children after adenotonsillectomy. J Pediatr 2006; 149:803 808. 3. Marcus CL, Moore RH, Rosen CL, et al. A randomized trial of adenotonsillectomy for childhood sleep apnea. N Engl J Med 2013;368:2366 2376. 4. Bhattacharjee R, Kheirandish-Gozal L, Spruyt K, et al. Adenotonsillectomy outcomes in treatment of obstructive sleep apnea in children: a multicenter retrospective study. Am J Respir Crit Care Med 2010;182:676 683. 5. Costa DJ, Mitchell R. Adenotonsillectomy for obstructive sleep apnea in obese children: a meta-analysis. Otolaryngol Head Neck Surg 2009;140: 455 460. 6. Fishman G, Zemel M, DeRowe A, Sadot E, Sivan Y, Koltai PJ. Fiber-optic sleep endoscopy in children with persistent obstructive sleep apnea: inter-observer correlation and comparison with awake endoscopy. Int J Pediatr Otorhinolaryngol 2013;77:752 755. 7. Croft CB, Pringle M. Sleep nasendoscopy: a technique of assessment in snoring and obstructive sleep apnoea. Clin Otolaryngol Allied Sci 1991; 16:504 509. 8. Kezirian EJ, White DP, Malhotra A, Ma W, McCulloch CE, Goldberg AN. Interrater reliability of drug-induced sleep endoscopy. Arch Otolaryngol Head Neck Surg 2010;136:393 397. 9. Rodriguez-Bruno K, Goldberg AN, McCulloch CE, Kezirian EJ. Test-retest reliability of drug-induced sleep endoscopy. Otolaryngol Head Neck Surg 2009;140:646 651. 10. Chan DK, Liming BJ, Horn DL, Parikh SR. A new scoring system for upper airway pediatric sleep endoscopy. JAMA Otolaryngol Head Neck Surg 2014;140:595 602. 11. Berry S, Roblin G, Williams A, Watkins A, Whittet HB. Validity of sleep nasendoscopy in the investigation of sleep related breathing disorders. Laryngoscope 2005;115:538 540. 12. Kezirian EJ, Hohenhorst W, de Vries N. Drug-induced sleep endoscopy: the VOTE classification. Eur Arch Otorhinolaryngol 2011;268:1233 1236. 13. Sin S, Wootton DM, McDonough JM, Nandalike K, Arens R. Anterior nasal resistance in obese children with obstructive sleep apnea syndrome. Laryngoscope 2014;124:2640 2644. 14. Ishii L, Roxbury C, Godoy A, Ishman S, Ishii M. Does Nasal Surgery Improve OSA in Patients with Nasal Obstruction and OSA? A Metaanalysis. Otolaryngol Head Neck Surg 2015;153:326 333. 15. Chervin RD, Hedger K, Dillon JE, Pituch KJ. Pediatric sleep questionnaire (PSQ): validity and reliability of scales for sleep-disordered breathing, snoring, sleepiness, and behavioral problems. Sleep Med 2000;1:21 32. 16. Lal C, White DR, Joseph JE, van Bakergem K, LaRosa A. Sleep-disordered breathing in Down syndrome. Chest 2015;147:570 579. 17. Thottam PJ, Choi S, Simons JP, Kitsko DJ. Effect of adenotonsillectomy on central and obstructive sleep apnea in children with Down syndrome. Otolaryngol Head Neck Surg 2015;153:644 648. 18. Mitchell RB, Boss EF. Pediatric obstructive sleep apnea in obese and normal-weight children: impact of adenotonsillectomy on quality-of-life and behavior. Dev Neuropsychol 2009;34:650 661. 19. Marcus CL, Brooks LJ, Draper KA, et al. Diagnosis and management of childhood obstructive sleep apnea syndrome. Pediatrics 2012;130:e714 e755. 20. Franco RA Jr, Rosenfeld RM, Rao M. First place resident clinical science award 1999. Quality of life for children with obstructive sleep apnea. Otolaryngol Head Neck Surg 2000;123(1 pt 1):9 16. 21. Hallgren KA. Computing inter-rater reliability for observational data: an overview and tutorial. Tutor Quant Methods Psychol 2012;8:23 34. 22. Viera AJ, Garrett JM. Understanding interobserver agreement: the kappa statistic. Fam Med 2005;37:360 363. 23. Brietzke SE, Gallagher D. The effectiveness of tonsillectomy and adenoidectomy in the treatment of pediatric obstructive sleep apnea/hypopnea syndrome: a meta-analysis. Otolaryngol Head Neck Surg 2006;134:979 984. 24. Shete MM, Stocks RM, Sebelik ME, Schoumacher RA. Effects of adenotonsillectomy on polysomnography patterns in Down syndrome children with obstructive sleep apnea: a comparative study with children without Down syndrome. Int J Pediatr Otorhinolaryngol 2010;74:241 244. 25. Donnelly LF, Shott SR, LaRose CR, Chini BA, Amin RS. Causes of persistent obstructive sleep apnea despite previous tonsillectomy and adenoidectomy in children with down syndrome as depicted on static and dynamic cine MRI. AJR Am J Roentgenol 2004;183:175 181. 26. Chan DK, Truong MT, Koltai PJ. Supraglottoplasty for occult laryngomalacia to improve obstructive sleep apnea syndrome. Arch Otolaryngol Head Neck Surg 2012;138:50 54. 27. Digoy GP, Shukry M, Stoner JA. Sleep apnea in children with laryngomalacia: diagnosis via sedated endoscopy and objective outcomes after supraglottoplasty. Otolaryngol Head Neck Surg 2012;147:544 550. 28. Thevasagayam M, Rodger K, Cave D, Witmans M, El-Hakim H. Prevalence of laryngomalacia in children presenting with sleep-disordered breathing. Laryngoscope 2010;120:1662 1666. 29. Mase CA, Chen ML, Horn DL, Parikh SR. Supraglottoplasty for sleep endoscopy diagnosed sleep dependent laryngomalacia. Int J Pediatr Otorhinolaryngol 2015;79:511 515. 30. Abdel-Aziz M, Ibrahim N, Ahmed A, El-Hamamsy M, Abdel-Khalik MI, El-Hoshy H. Lingual tonsils hypertrophy; a cause of obstructive sleep apnea in children after adenotonsillectomy: operative problems and management. Int J Pediatr Otorhinolaryngol 2011;75:1127 1131. 31. Fricke BL, Donnelly LF, Shott SR, et al. Comparison of lingual tonsil size as depicted on MR imaging between children with obstructive sleep apnea despite previous tonsillectomy and adenoidectomy and normal controls. Pediatr Radiol 2006;36:518 523. 32. Lin AC, Koltai PJ. Persistent pediatric obstructive sleep apnea and lingual tonsillectomy. Otolaryngol Head Neck Surg 2009;141:81 85. 33. Truong MT, Woo VG, Koltai PJ. Sleep endoscopy as a diagnostic tool in pediatric obstructive sleep apnea. Int J Pediatr Otorhinolaryngol 2012; 76:722 727. 34. Dedhia RC, Weaver EM. Association between drug-induced sleep endoscopy and measures of sleep apnea burden. Otolaryngol Head Neck Surg 2015;153:875 880. 35. Durr ML, Meyer AK, Kezirian EJ, Rosbe KW. Drug-induced sleep endoscopy in persistent pediatric sleep-disordered breathing after adenotonsillectomy. Arch Otolaryngol Head Neck Surg 2012;138:638 643. 1498