Validation of the Snore Outcomes Survey for Patients With Sleep-Disordered Breathing Richard E. Gliklich, MD; Pa-Chun Wang, MD, MSc ORIGINAL ARTICLE Objective: To develop and validate a self-reported outcomes measure for patients with sleep-disordered breathing the Snore Outcomes Survey. Design: Item areas of the SOS were developed by an expert panel. Consecutive patients were enrolled into the study in a prospective manner. Patients received the SOS, the Epworth Sleepiness Scale, the Pittsburgh Sleep Quality Index, the Medical Outcomes Study 36-Item Short- Form Health Survey, and standard overnight polysomnography at baseline and after 4 months of continuous positive airway pressure therapy. Setting: A tertiary care, academic otolaryngology and sleep disorders referral center. Patients: One hundred fifty-six adult patients presenting with a chief complaint of snoring or sleep-disordered breathing to the Massachusetts Eye and Ear Infirmary, Boston. Main Outcome Measures: Test-retest reliability, intrasurvey reliability, internal consistency, validity, and standardized response means of the SOS. Results: Overall, reliability of the SOS was excellent (testretest reliability r=0.86; P.001; Cronbach coefficient, 0.85). The SOS index significantly correlated with the Epworth Sleepiness Scale (r= 0.42; P.001) and the global Pittsburgh Sleep Quality Index score (r= 0.38; P.001), as well as with the number of recorded arterial oxygen saturation levels below 85% (r= 0.46; P=.02). The SOS index was sensitive to clinical changes after intervention (standardized response mean, 0.57). Conclusion: The SOS is a reliable and valid instrument for assessing sleep-related health status for patients with snoring and sleep-disordered breathing and for measuring change in health status following therapy. Arch Otolaryngol Head Neck Surg. 2002;128:819-824 From the Department of Otolaryngology and the Clinical Outcomes Research Unit, Massachusetts Eye and Ear Infirmary, and the Department of Otology and Laryngology, Harvard Medical School, Boston, Mass. Dr Wang is now with the Department of Otolaryngology, Cathay General Hospital, and the Department of Public Health, China Medical College, Taiwan. SLEEP-DISORDERED breathing (SDB) represents a continuum of sleep disorders from simple snoring to severe obstructive sleep apnea syndrome. While 2% to 4% of the middleaged workforce are reportedly affected by obstructive sleep apnea syndrome, SDB affects 3 to 6 times this number. 1-5 Recent advances in treatment of SDB in patients with and without obstructive sleep apnea syndrome have raised questions regarding treatment efficacy and how to determine the best practice. 6-11 While the polysomnogram (PSG) provides reliable data on respiratory behavior during sleep, it does not fully address the problem of SDB from either a patient s or a spouse s perspective and is prohibitively expensive and burdensome to be used for long-term, multiple follow-ups in these patients. 12,13 In view of this, we developed the Snore Outcomes Survey (SOS) to be a patient-based measure for the full range of SDB patients in whom snoring is a primary symptom, 14-16 where the goal is to measure the snoring component of SDB. Approaches to develop and validate qualityof-life measures as performed in this study have been previously well described. 17 RESULTS STUDY POPULATION There were 130 men and 26 women (mean±sd age, 46.2±11.6 years). The mean Respiratory Distress Index was 32.66±28.34. The mean awake oxygen saturation was 94.97%±2.4%, and the minimum oxygen saturation during sleep was 77.85%±16.04%. RELIABILITY OF SOS 1. Test-Retest Reliability: mean test and retest total scores varied from 27.54±15.89 819
PARTICIPANTS AND METHODS CONSTRUCTION OF SOS The item areas for the SOS were developed by an expert panel. The initial questionnaires were piloted with open-ended responses. Actual survey items were constructed according to a Likert scaling model. The SOS (Figure) contains 8 items that evaluate the duration, severity, frequency, and consequences of problems associated with SDB, snoring in particular. Because of the impact of SDB on others, a separate Spouse/Bed Partner Survey (SBPS) containing 3 Likert-type items was also developed. Scores on the SOS and SBPS are normalized on a scale ranging from 0 (worst) to 100 (best), similar to other measures such as the Medical Outcomes Study 36-Item Short-Form Health Survey (SF-36). STUDY POPULATION One hundred fifty-six consecutive patients with chief complaints of snoring and SDB who were evaluated by otolaryngologists at the Sleep and Snoring Center of Massachusetts Eye and Ear Infirmary, Boston, were enrolled. All patients underwent overnight PSG. SLEEP STUDY Measurements of height and weight were obtained to calculate the body mass index as weight in kilograms divided by the square of height in meters. All patients underwent standard overnight PSG to record the pulse oxygen saturation, chest and abdominal wall movement, electroencephalogram, oronasal airflow, electrocardiogram, electrooculogram, and submental electromyogram. Respiratory Distress Index was defined as the sum of total apnea and hypopnea episodes per hour of sleep. An apnea episode is defined as cessation of airflow lasting longer than 10 seconds, whereas a hypopnea episode is defined as a 33% or greater reduction in combined oral and nasal flow lasting longer than 10 seconds. SURVEY FORMS At entry, patients were administered the SOS, the Epworth Sleepiness Scale 15,18 (ESS), the Pittsburgh Sleep Quality Index 19 (PSQI), and the SF-36. 20,21 Permission to use these forms were obtained in each case. The 8-item ESS evaluates daytime somnolence and generates a total score ranging from 0 (best) to 24 (worst). The PSQI is a 19-item, selfreported, global sleep questionnaire that evaluates 7 dimensions of sleep quality. Each domain is scored on a scale from 0 (best) to 3 (worst), and these subscores are summed to yield a total score ranging from 0 (best) to 21 (worst). The SF-36 is a widely used generic quality-of-life measure that divides general health into 8 domains. The subscales and definition of SF-36 and PSQI are listed in Table 1. RELIABILITY AND VALIDITY OF SOS Fifty-nine patients who were deemed clinically stable were retested after a 2- to 4-week interval. This cohort received no medical or surgical intervention during this interval. Testretest, intersurvey, and intrasurvey reliability were calculated using Spearman rank order correlation coefficients for individual items and for total survey score. Internal consistency of the SOS was calculated using Cronbach correlation coefficients. In addition to its face validity, the SOS was assessed for convergent validity through correlations to concurrent PSG data. In addition, scores on the SOS were compared with 3 previously validated measures that were completed by the patients at the same time as the SOS. These measures were the ESS, PSQI, and SF-36. One hundred fortynine patients completed all 3 surveys. REPONSIVENESS OF SOS Instrument responsiveness or sensitivity to clinical change was assessed in the following manner. Of the 149 patients used in the validity analysis, 21 were retested with the SOS after being treated for 4 months with continuous positive airway pressure. These patients were evaluated on an intention-to-treat basis. Actual in-home monitoring of continuous positive airway pressure compliance was not performed. However, all patients reported that continuous positive airway pressure resolved their snoring when it was used. Longitudinal sensitivity to clinical change was calculated as the standardized response mean, according to method described by Liang et al. 22 STATISTICAL ANALYSIS All data were stored in an Access 7.0 database (Microsoft, Redmond, Wash). Analyses were conducted using the SAS software package (SAS Institute, Cary, NC). on the first administration to 27.59±17.44 on the second administration. The test-retest correlation coefficients for individual items (r=0.54-0.88; P.001) and total score (r=0.86; P<.001) were statistically significant (Table 2). 2. Intersurvey Reliability: the correlation between SOS and SBPS was not significant (r=0.18; P=.07). 3. Intrasurvey Reliability: the item-item, item-total correlations for SOS are presented in Table 3. The itemitem correlations for SBPS were very high. 4. Internal Consistency: Cronbach correlation coefficients were calculated as 0.85 for the SOS and 0.96 for the SBPS. VALIDITY OF SOS 1. Correlation With PSG Data: the correlations between the SOS and the PSG parameters are shown in Table 4. Four Respiratory Distress Index elements, awake oxygen saturation, number of arterial oxygen saturation values less than 85%, and minimum arterial oxygen saturation were collected from a computerized PSG database. The number of desaturation values to less than 85% was most strongly correlated with the SOS (r= 0.46; P=.02). 2. Correlation With ESS: a significant correlation was observed between the SOS and the ESS (r = 0.42; 820
The Snore Outcomes Survey. (Printed with permission of Outcome Sciences Inc, Boston, Mass.) P.001). The correlation between the SBPS and ESS was not significant (Table 5). 3. Correlation With SF-36: the 8 subscale scores of the SF-36 were compared with the SOS total score, and the correlation coefficient was derived. The SOS demonstrated significant correlations with all SF-36 subscales except for the mental health subscale. The SBPS did not correlate well with any SF-36 subscale (Table 5). 4. Correlation With PSQI: component and total scores were calculated according to published guidelines. 19 The PSQI evaluates 7 dimensions of sleep quality: subjective quality, latency, duration, efficiency, disturbance, use of sleep medication, and daytime dysfunction (Table 1). The SOS was found to be significantly (P.01) correlated with the total PSQI score, subjective sleep quality, habitual sleep efficiency, sleep disturbance, and daytime dysfunction (Table 5). RESPONSIVENESS OF SOS The standardized response mean for the total SOS score was 0.57, indicating that the SOS is responsive to clinical change. The SBPS, on the other hand, was not sensitive to change. COMMENT Sleep-disordered breathing 1 is a condition characterized by repeated pauses in breathing during sleep, which lead to the fragmentation of sleep and a decrease in arterial oxygenation. The spectrum of SDB includes simple snoring, obstructive sleep apnea, and upper airway resistance syndrome. Several studies have shown that patients with SDB carry higher risks of daytime somnolence and cardiovascular morbidities. 2,3 The objective evaluation of SDB has relied on standard PSG, which requires a patient to sleep in a laboratory overnight. From a cost and burden perspective, overnight PSG is not an efficient means to track patients over time. Although snoring is only one component of SDB, a valid and reliable patient-based measure for snoring could contribute to the tracking of patients with SDB and certainly those whose SDB is limited to snoring alone. Use of patient-based questionnaires to assess patients with SDB has become a widely accepted ap- 821
Table 1. SF-36 and PSQI Subscales* SF-36 Subscales and Definition Physical functioning: limitation on physical activities such as walking, bathing, and strenuous sports Role-physical: problems with work or other daily activities as a result of physical health Bodily pain: intensity of bodily pain or limitation due to pain General health: perception of current health and health outlook Vitality: level of energy Social functioning: extent health interferes with normal social activities Role-emotional: problems with daily activities as result of emotional issues Mental health: mental health screening PSQI Components Subjective sleep quality Sleep latency Sleep duration Habitual sleep efficiency Sleep disturbances Use of sleeping medication Daytime dysfunction Total score *SF-36 indicates Medical Outcomes Study 36-Item Short-Form Health Survey; PSQI, Pittsburgh Sleep Quality Index. Table 2. Spearman Correlations for Test-Retest Reliability of SOS* Individual Items Spearman r 1 0.62 2 0.88 3 0.88 4 0.66 5 0.71 6 0.71 7 0.75 8 0.54 Total score 0.86 *SOS indicates Snore Outcomes Survey. P.001 for all items. proach. 14-16,18,19 In this study, we introduce the SOS as a reliable, valid, and sensitive disease-specific outcomes measure that adds another dimension to our understanding of the impact of SDB to patients quality of life. While other measures focus on sleep quality and somnolence, the SOS focuses on snoring with the idea that it is the presenting complaint for many patients. In terms of reliability, the SOS demonstrated good test-retest reliability for individual test questions and for total score. Improvement in wording may further enhance the reliability of items 1, 4, and 8. The measured item and total score test-retest reliabilities ranging from 0.54 to 0.88 were comparable with those of wellstudied SF-36, which had correlations ranging from 0.6 to 0.81 for its subscales. 23 The poor correlation between the SOS and SBPS show a significant discrepancy between patient and bed-partner evaluations in terms of the patient s snoring. This result was not expected. The Cronbach correlation coefficients of 0.85 and 0.96 for SOS and SBPS, respectively, exceed the commonly accepted threshold (0.7) for a reliable measure. 17 The overall comparison between the SOS and the PSG data suggests that snoring may be more strongly related to oxygen desaturation than previously assumed. Previous studies have also shown oxygen desaturation to be more closely related to quality of life than the Respiratory Distress Index. 24 Although the SOS and PSG were Table 4. Spearman Correlations Between SOS and Polysomnogram Data* r (P Value) Polysomnogram Data SOS SBPS Respiratory Distress Index 0.21 (.26) 0.22 (.31) Awake oxygen saturation 0.04 (.83) 0.52 (.01) No arterial oxygen saturation 0.46 (.02) 0.1 (.46) values 85% Minimum oxygen saturation 0.35 (.07) 0.16 (.48) *SOS indicates Snore Outcomes Survey; SBPS, Spouse/Bed Partner Survey. Table 3. Spearman Correlations for Intrasurvey Reliability of SOS and SBPS* Items, r (P Value) 2 3 4 5 6 7 8 SOS Intrasurvey Reliability SOS items 1 0.49 (.001) 0.52 (.001) 0.6 (.001) 0.38 (.001) 0.47 (.001) 0.51 (.001) 0.65 (.001) 2 0.32 (.001) 0.47 (.001) 0.29 (.004) 0.37 (.001) 0.57 (.001) 0.51 (.001) 3 0.82 (.001) 0.11 (.27) 0.27 (.006) 0.31 (.002) 0.35 (.001) 4 0.12 (.23) 0.43 (.001) 0.33 (.001) 0.46 (.001) 5 0.25 (.01) 0.52 (.001) 0.35 (.001) 6 0.35 (.001) 0.79 (.001) 7 0.53 (.001) SBPS Intrasurvey Reliability SBPS items 1 0.88 (.001) 0.88 (.001) 2 0.9 (.001) *SOS indicates Snore Outcomes Survey; SBPS, Spouse/Bed Partner Survey. 822
Table 5. Spearman Correlations Between SOS With ESS, SF-36, and PSQI* r (P Value) Data SOS SBPS Correlations Between SOS and ESS ESS data ESS total score 0.42 (.001) 0.11 (.23) Correlations Between SOS and SF-36 SF-36 data Physical functioning 0.23 (.006) 0.04 (.68) Role-physical 0.33 (.001) 0.14 (.16) Bodily pain 0.18 (.03) 0.14 (.30) Vitality 0.37 (.001) 0.11 (.29) Role-emotional 0.23 (.006) 0.14 (.71) Social functioning 0.34 (.001) 0.01 (.91) General health 0.18 (.04) 0.04 (.67) Mental health 0.15 (.08) 0.0005 (.96) Correlations Between SOS and PSQI PSQI data Global PSQI total score 0.38 (.001) 0.09 (.35) Subjective sleep quality 0.34 (.001) 0.03 (.76) Sleep latency 0.049 (.56) 0.1 (.30) Sleep duration 0.033 (.68) 0.21 (.04) Habitual sleep efficiency 0.22 (.007) 0.07 (.45) Sleep disturbances 0.31 (.001) 0.08 (.40) Use of sleep medication 0.128 (.11) 0.08 (.40) Daytime dysfunction 0.47 (.001) 0.12 (.21) *SOS indicates Snores Outcomes Survey; ESS, Epworth Sleepiness Scale; SF-36, Medical Outcomes Study 36-Item Short-Form Health Survey; PSQI, Pittsburgh Sleep Quality Index; and SBPS, Spouse/Bed Partner Survey. correlated, the correlations were moderate, again suggesting that they are measuring different aspects of sleep disturbance. The SOS was rigorously compared with other sleep-specific questionnaires such as the ESS and PSQI. The correlation coefficient of 0.42 between the SOS and ESS was similar to that between the SOS and PSQI daytime dysfunction component (r= 0.47) and confirms the predicted convergent validity. The weak but statistically significant correlation between the SOS and several SF-36 subscales again supports convergence and suggests that SDB may have an impact on general health. 24 Responsiveness, or sensitivity to longitudinal change, is the ability of a health measure to detect clinical change over time. The standardized response mean measures this sensitivity. The SOS (standardized response mean, 0.57) demonstrates moderate responsivenesss. 22,25 Hence, the SOS can be used as an evaluative instrument in clinical studies of interventions for snoring and other forms of SDB when the principal complaint is snoring. The SOS is short, easy to understand, and easy to administer in a busy clinical setting without sacrificing its evaluative power. In addition to providing a qualityof-life dimension to the overall assessment of SDB, the SOS can be used to follow-up patients with SDB over time, with and without intervention. To summarize, the SOS is a patient-based, diseasespecific outcomes measure for SDB. It is valid, reliable, and sensitive to clinical change. The SOS also provides an inexpensive and accurate measure to follow up patients with SDB, especially when snoring is the primary symptom. Accepted for publication November 21, 2001. Corresponding author and reprints: Richard E. Gliklich, MD, Department of Otolaryngology and Clinical Outcomes Research Unit, Massachusetts Eye and Ear Infirmary, 243 Charles St, Boston, MA 02114. REFERENCES 1. Young T, Palta M, Dempsey J, Skatrud J, Weber S, Badr S. The occurrence of sleep-disordered breathing among middle-aged adults. N Engl J Med. 1993;328: 1230-1235. 2. Briones B, Adams N, Strauss M, et al. Relationship between sleepiness and general health status. Sleep. 1996;19:583-588. 3. Lavie P. Incidence of sleep apnea in a presumably healthy working population: a significant relationship with excessive daytime sleepiness. Sleep. 1983;6:312-318. 4. Young T, Evans L, Finn L, Palta M. Estimation of the clinically diagnosed proportion of sleep apnea syndrome in middle-aged men and women. Sleep. 1990; 20:705-706. 5. Young T, Blustein J, Finn L, Palta M. Sleep-disordered breathing and motor vehicle accidents in a population-based sample of employed adults. Sleep. 1997; 20:608-613. 6. Fujita A, Conway W, Zorick F, et al. Surgical correction of anatomic abnormalities of obstructive sleep apnea syndrome: uvulopalatopharyngoplasty. Otolaryngol Head Neck Surg. 1981;89:923-934. 7. Mickelson SA. Laser-assisted uvulopalatoplasty for obstructive sleep apnea. Laryngoscope. 1996;106:10-13. 8. Powell NB, Riley RW, Guilleminault C. Radiofrequency tongue base reduction in sleep-disordered breathing: a pilot study. Otolaryngol Head Neck Surg. 1999; 120:656-664. 823
9. Li KK, Powell NB, Riley RW, et al. Radiofrequency volumetric reduction of the palate. Otolaryngol Head Neck Surg. 2000;122:410-414. 10. Woodson BT, Derowe A, Hawe M, et al. Pharyngeal suspension suture with repose bone screw for obstructive sleep apnea. Otolaryngol Head Neck Surg. 2000; 122:395-401. 11. Conradt R, Hochban W, Heitmann J, et al. Sleep fragmentation and daytime vigilance in patients with OSA treated by surgical maxillomandibular advancement compared to CPAP therapy. J Sleep Res. 1998;7:217-223. 12. Li KK, Riley RW, Powell NB, Gervacio L, Troell RJ, Guilleminault C. Obstructive sleep apnea surgery: patient perspective and polysomnographic results. Otolaryngol Head Neck Surg. 2000;123:572-575. 13. Pradhan PS, Gliklich RE, Winkelman J. Screening for obstructive sleep apnea in patients presenting for snoring surgery. Laryngoscope. 1996;106:1393-1397. 14. Piccirillo JF, Gates GA, White DL, Schectman KB. Obstructive sleep apnea treatment outcomes pilot study. Otolaryngol Head Neck Surg. 1998;118:833-844. 15. Johns MW. A new method for measuring daytime sleepiness: the Epworth Sleepiness Scale. Sleep. 1991;14:540-545. 16. Flemons WW, Reimer MA. Development of a disease-specific health related quality of life questionnaire for sleep apnea. Am J Respir Crit Care Med. 1998;158: 494-503. 17. Nunnally JC. Psychometric Theory. 2nd ed. New York, NY: McGraw-Hill; 1978. 18. Johns MW. Daytime sleepiness, snoring, and obstructive sleep apnea. Chest. 1993;103:30-36. 19. Buysse DJ, Reynolds CF III, Monk TH, Bermen SR, Kupfer DJ. The Pittsburgh Sleep Quality Index: a new instrument for psychiatric practice and research. Psychiatry Res. 1989;28:192-213. 20. Mchorney CA, Ware JE, Raczek AE. The MOS 36-Item Short-Form Health Survey (SF-36), II: psychometric and clinical tests of validity in measuring physical and mental health constructs. Med Care. 1993;31:247-263. 21. Ware JE. Validity: norm-based interpretation. In: Ware JE, ed. SF-36 Health Survey Manual and Interpretation Guide. Boston, Mass: Nimrod Press; 1993:10:1-10:38. 22. Liang MH, Fossel AH, Larson MG. Comparisons of five health status instruments for orthopedic evaluation. Med Care. 1990;28:632-642. 23. Brazier JE, Harper R, Jones NMB, et al. Validating the SF-36 health survey questionnaire: new outcome measure for primary care. BMJ. 1992;305:160-164. 24. Gliklich RE, Taghizadeh F, Winkelman JW. Health status in patients with disturbed sleep and obstructive sleep apnea. Otolaryngol Head Neck Surg. 2000; 122:542-546. 25. Cohen J. Statistical Power Analyses for the Behavioral Sciences. 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates; 1988. CME Announcement CME Hiatus: July Through December 2002 CME from JAMA/Archives will be suspended between July and December 2002. Beginning in early 2003, we will offer a new online CME program that will provide many enhancements: Article-specific questions Hypertext links from questions to the relevant content Online CME questionnaire Printable CME certificates and ability to access total CME credits We apologize for the interruption in CME and hope that you will enjoy the improved online features that will be available in early 2003. 824