Sleep, 19(10):S243-S247 1996 American Sleep Disorders Association and Sleep Research Society Methods of Diagnosing Sleep Apnea J The Diagnosis of Sleep Apnea: Questionnaires and Home Studies W. Ward Flemons and John E. Remmers Faculty of Medicine, University of Calgary, and *The Alberta Lung Association Sleep Centre, Foothills Hospital, Calgary, Alberta, Canada.::, Summary: Alternatives to the standard method of diagnosing sleep apnea (SA) are becoming increasingly popular due to the expense and/or, in some cases, the limited availability of polysomnography (PSG). The most common diagnostic alternatives have been clinical prediction rules and portable monitoring. Most portable monitors record one or more signals such as oxygen saturation, heart rate, airflow, or ribcage and abdominal movements. To date, most published studies of these monitors have had serious methodologic problems that have limited the acceptance of this technology. We have developed a two-step diagnostic approach for SA based on a clinical prediction rule and the results from a simple, but reliable and accurate, portable monitor that records oxygen saturation, snoring, and body position. The preliminary results indicate that such a strategy is very useful in a population of outpatients suspected of having SA and would preclude the need for PSG to investigate for this possibility in the majority of patients. However, before portable monitoring becomes widely adopted, each system should be more thoroughly tested, and increased attention should be directed at the design of the study so that the results are more generalizeable to other sleep clinic populations. Key Words: Sleep apnea syndromes-raye's Theorem-Likelihood ratios Diagnosis-Respiratory system-medical history taking. Sleep apnea (SA) has been shown to be exceptionally common in adults (1) and is the most common disorder that is diagnosed after a full night of polysomnography (PSG) in most sleep labs. PSG is a labor intensive, specialized test that is expensive. To try and reduce costs and increase the number of patients that can be evaluated, alternative diagnostic strategies that rely on clinical predictors and simplified home (portable) monitoring have gained increasing recognition over the past six years. Until recently, most clinical studies of portable monitors have had serious methodologic flaws, thus limiting the acceptance of a non PSG approach to the diagnosis of sleep apnea (2). We have developed a two-step diagnostic approach for SA based on using simple clinical predictors (3) and a simple oximetry-based portable monitor that has been validated in a group of more than 200 patients selected randomly from a consecutive series of patients referred to the Alberta Lung Association (ALA) Sleep Centre at the Foothills Hospital. Our preliminary re- Accepted for publication September 1996. Address correspondence and reprint requests to W. Ward Flemons, M.D., F.R.c.P.c., Foothills Hospital, 1403-29 St., N.W., Calgary, Alberta T2N 2T9, Canada. S243 suits indicate that, in the appropriate setting, the probability that a particular patient has SA can be accurately estimated and can be used to guide an experienced clinician's decision making. For the majority of patients who are suspected of having obstructive sleep apnea (OS A) and who are not suffering from serious underlying illnesses, PSG will not be necessary. Further research is required to improve the quality of studies that evaluate non-psg methods of diagnosing OSA to ensure that correct diagnostic decisions, based on their results, are made. Diagnostic decision making A simple method of analyzing how clinicians make diagnostic decisions is the "threshold approach" (4). When a test that is not considered the "gold standard" is being performed to diagnose a particular condition, the results are often best thought of in terms of how they modify the probability of disease for that particular patient. At a low probability of disease, a clinician will decide that the patient does not have the disorder and will abandon further testing. Alternatively, when
S244 w. W. FLEMONS AND 1. E. REMMERS the probability of disease is quite high, the appropriate decision is to proceed with treatment for the disorder. Only when the probability of disease is intermediate, i.e. somewhere between the low and high thresholds, is it reasonable to proceed with further diagnostic tests. The decision about what probability levels the low and high threshold limits should be set at can be made arbitrarily by experienced clinicians. Alternatively, they can be established mathematically; however, this requires a reasonable estimation of the various outcomes (utilities) of treated versus untreated sleep apnea. This information is currently not available. However, knowing how clinical predictors and/or how the results of portable monitoring influence the probability of SA, an experienced clinician can make appropriate clinical decisions such as concluding that a patient does not have SA, that further testing, such as PSG is indicated, or that treatment for SA should be commenced. According to Bayes' Theorem, the calculation of the probability of a particular disease requires an a priori estimate of the probability of disease (the prevalence of disease could be used) and the operating characteristics (sensitivity and specificity) of the diagnostic test (5). Clinical predictors can be used to generate an accurate estimate of the probability of SA and can in tum be used as an a priori estimate of SA for portable monitoring. Although the diagnostic performance of portable monitors when compared with PSG is often summarized as a single estimate of sensitivity and specificity, a large amount of diagnostic information is lost with this approach. As the cutoff is changed for what constitutes a "positive" result from portable monitoring, the sensitivity and specificity will change. A better method of summarizing how diagnostic performance changes when the cutoff for the definition of a positive test is changed is the calculation of likelihood ratios (LR) (4). A LR contrasts the proportion of patients with and without a disorder who have a given level of a diagnostic test result (4). A LR of one indicates that the post-test probability of disease is not different from the pre-test probability. A LR < 1 indicates the post-test is lower than the pre-test probability of disease and conversely a LR > 1 indicates the post-test is greater than the pre-test probability of disease. Clinical prediction of sleep apnea We have previously demonstrated how clinical predictors can be analyzed and compared with PSG results to determine which ones independently predict SA in a sample of consecutively referred patients to our sleep clinic (3). These independent predictors can be statistically combined using multiple regression techniques to formulate a clinical prediction rule (CPR) for SA. Several investigators have created CPRs (6,7) that we have previously validated, thus demonstrating that the probability estimates for SA from CPRs are generalizeable to other sleep centers. The problem faced by most clinicians attempting to use these rules is that they involve complicated mathematical formulas that limit their usefulness. We generated a four-item clinical prediction rule that included 1) neck circumference, 2) hypertension, 3) habitual snoring, and 4) partner reports of frequent choking and gasping during sleep. We simplified the model into a table that can be used to determine a patient's sleep apnea clinical score (SACS). The higher the SACS the more likely it is that a patient has SA, which we defined as an apnea-hypopnea index> 1 O. LRs for SACS <5,5-10, 10-15, and >15 ranged from 0.25 to 5.17. We have described how to use these LRs to generate estimates of the probability that a patient has SA. To calculate the probability of SA using LRs, it is necessary to convert probabilities into odds ratios. This process can be simplified using a nomogram (3,4,8). In a second random sample of consecutively referred patients to our sleep clinic, we have validated this SA CPR by comparing receiver operating characteristic curves from the first and second samples of patients, demonstrating that there is almost complete overlap. The probability estimates of SA generated by CPRs are usually not low enough or high enough to exceed the low and high thresholds, respectively, for diagnostic decision making. However, these estimates can be used as a priori probabilities for subsequent diagnostic testing with portable monitors. Portable monitors for diagnosing sleep apneathe need for improved methodology There have been numerous studies published that have evaluated non-psg methods for diagnosing SA. In general, most of these suffer from some serious methodologic flaws. In 1994 the American Sleep Disorders Association (ASDA) published standards and a review of the available literature on these devices (2). It was determined that over 30 portable systems (level II and III) at that point had been marketed yet there were only validation studies published in peer-reviewed journals for nine of these systems. Even more concerning was the finding that only 330 patients in total had been studied and only 30 of these were studied at home unattended, i.e. in the setting in which they were intended to be used. Standards for the conduct of studies that investigate new diagnostic systems have previously been published (4,9,10). A brief synopsis of those standards is listed below in point form. There should be: Sleep. Vol. 19, No. 10, 1996
DIAGNOSIS OF SA S245 ) 1. an independent, blind comparison with a reference standard. 2. an appropriate spectrum of patients, for example, males and females, mild, moderate, and severe cases. 3. avoidance of work-up bias. (The results of the diagnostic test should not influence the decision to have patients undergo the reference standard test.) 4. methods for performing the test described in detail to allow for duplication of the study and to allow clinicians to use the same methodology. 5. an adequate description of the study population that allows readers to decide for themselves whether their particular population of patients is similar enough to the study population for them to use the results in their own practice. 6. adequate sample size. (A conservative estimate is that there should be 2::200 patients in the study, approximately equally divided between those with and those without SA. This would allow for confidence intervals of approximately ± 10% for estimates of sensitivity and specificity.) 7. avoidance of selection bias. (This is usually avoided by including a consecutive sample of patients referred to a sleep clinic. The study should not just include patients referred to a sleep lab, since there is almost always some decision by an experienced clinician about which patients are referred to the sleep lab. This "filtering" of patients should not occur since there is usually no way to describe on what basis the decision was made to refer the patient for a PSG. Thus, these results may not be generalizeable to other populations.) 8. an adequate description of the study setting and an appropriate setting for the study. (For example, basic descriptors of the study could include whether it was a tertiary referral sleep clinic or a community sleep clinic; the types of physicians referring to the sleep clinic; the population base, and the types of patients referred [ethnicity, financial status, insured vs. uninsured], etc. Since most portable monitors are intended for use outside of a sleep lab, this is the setting in which they should be studied.) To date, many studies that have been reported on portable monitors follow the first four criteria listed above. However, most investigators have failed to give an adequate description of their study population, have an inadequate sample size, have introduced a selection bias, and in almost all studies have failed to describe their study setting adequately and, more importantly, have failed to study patients in a non-sleep lab setting. At this time, there are two notable exceptions to the generalities stated above. The study by Douglas et al. (11), although technically not a study of a portable monitor, did meet the most important criteria listed above. Although none of their patients were studied at home the purpose of the study was not to develop a home-based monitoring system. A study of home oximetry in 240 patients compared to sleep lab PSG, reported by Series et al. (12), conforms most closely with the criteria above. Portable monitors for diagnosing sleep apneathe SnoreS at monitor The initial report of the SnoreSat monitor (SSat) by Issa et al. that recorded snoring and oxygen saturation demonstrated high sensitivities and specificities in a population of sleep-lab patients who were undergoing PSG (l3). Subsequently, the monitor has been modified so that it also records body position. The analogue oxygen saturation value is converted to digital and stored at a frequency of 1 Hz on a controller board. Subsequent software analysis reports a respiratory disturbance (RD) when the oxygen saturation drops 2::4% from baseline, with baseline defined as a moving average of the top 15% of the Sa0 2 values over the previous five minutes. The time that the probe is attached to the patient is defined as those periods when the Sa0 2 is 2::50%. The respiratory disturbance index (RDI) is defined as the number of RDslhour that the probe is on. In a followup study, the at-home performance of the SSat monitor has been evaluated over a period of three years in a study population selected randomly from consecutive patients referred to the ALA Sleep Centre, a tertiary referral center for southern Alberta, servicing a population base of 1.2 million people. Approximately 70% of patients are referred from general practitioners and 17% from otolaryngologists. Patients used the SSat monitor for at least two nights at home and underwent a full night of PSG that included simultaneous SSat monitoring. Preliminary results are available from 245 patients including 176 that have had simultaneous SSat monitoring. To compare the results between SSat and PSG we have chosen four methods; each has advantages and limitations that are beyond the scope of this article to discuss. First, the RDIs from two nights of home SSat monitoring were found to be highly correlated (r = 0.92). The correlation between home SSat RDI and the PSG derived AHI (r = 0.83) was similar to the correlation between home SSat RDI and simultaneous SSat RDI (r = 0.82). Simultaneous SSat RDI was highly correlated with AHI (r = 0.97). The results indicated a systematic bias-the home SSat RDI being less than the simultaneous RDI or AHI that we suspect relates to a number of factors. We have found that patients spend more time supine in the sleep lab than at home. Sleep, Vol. 19, No. 10, 1996
S246 w. W. FLEMONS AND 1. E. REMMERS Also, we suspect that there may be a difference in RDI if ear probes are used compared to the use of finger probes since there is a difference in circulation times, a possibility we are currently studying. Finally, there will be a decrease in RDI compared to AHI since we do not have a measurement of sleep time at home. However, this would not explain the discrepancy between home SSat and simultaneous SSat results. Overall, we believe these results suggest that the simultaneous SSat RDI provides an extremely close estimation of AHI and that the difference between home and lab RDI is likely a function of body position and, possibly, less sleep disruption at home, resulting in fewer sleep onset hypopneas. These sleep onset hypopneas, in some patients, may result in an overestimate of their true AHI. Second, we compared SSat and PSG results using the methods comparison approach of Bland and Altman (14). The mean difference between PSG AHI and home SSat RDI was 7.6, similar to the mean difference between simultaneous SSat RDI and home SSat RDI of 9.3. The limits of agreement were also very similar (38.6 and -23.4 for PSG AHI vs. home SSat RD!, 40.8 and -22.2 for simultaneous SSat RDI vs. home SSat RDI). There was obviously much closer agreement between PSG AHI and simultaneous SSat RDI, the mean difference was -1.1 and the limits of agreement were 11.76 and -13.96. Third, we have calculated LRs for home SSat and simultaneous SSat RDIs using the PSG AHI > 10 as the reference standard for defining SA. For home SSat, LRs ranged from 0.12 for an RDI <5 to 21.70 for an RDI >20. For simultaneous SSat, LRs ranged from 0 for an RDI <5 to 54.04 for an RDI >20. Finally, we have adapted the diagnostic agreement analysis first proposed by White et al. (15) to define rates of agreement, rates of overestimation, and rates of underestimation. Agreement was defined as the PSG AHI and the SSat RDI both 2::30 or the PSG <30 and the difference between the AHI and RDI <10. Overestimation was defined as the PSG AHI <30 and a SSat RDI - PSG AHI difference> 10. Underestimation was defined as the PSG AHI <30 and a PSG AHI - SSat RDI difference> 10. The rates of agreement, overestimation, and underestimation for the home SSat were 73%, 3%, and 24%, respectively. The rates for simultaneous SSat were 99%, 1 %, and 0%, respectively. CONCLUSIONS Most studies to date have suffered from methodologic flaws that have prevented a non-sleep lab-based approach to the diagnosis of sleep apnea from being widely adopted. Our preliminary results indicate that Sleep, Vol. 19, No. 10, 1996 Symptomatic Patients Asymptomatic Patients CPR & Hx of other possible sleep disorders Low Probability Polysomnography (Symptomatic Patients) Stop Testing for SA (Asymptomatic Patients) Moderate to High Probability Portable Monitoring / '\ NoOSA OSA Polysomnography (Symptomatic Patients) Stop Testing for SA (Asymptomatic Patients) Treatment FIG. 1. An algorithm for how a CPR for SA and a history (hx) of other possible sleep disorders, plus portable monitoring, can be used in conjunction with PSG in a diagnostic strategy for SA. there is much more home versus sleep-lab variability in results of portable monitoring than there is in simultaneous recordings, emphasizing the importance of validating these monitoring systems in the setting in which they are intended to be used. We believe we have found systematic differences between home versus sleep lab monitoring that cannot be accounted for simply by the failure of portable monitors to record sleep. Further studies are ongoing to further delineate this; however, we propose that it raises some serious questions about how "gold" PSG is as a reference standard for the diagnosis of SA. We strongly suspect that portable monitoring will prove to be more cost effective than PSG, but this assumption needs to be validated in a proper randomized trial. We believe that a portable monitor like the SSat can be successfully incorporated into a diagnostic algorithm that relies on the estimation of the probability that a patient suffers from SA. The estimation of this probability can be enhanced by an accurate clinical assessment, and we have shown the utility of a CPR in this regard. Likelihood ratios are the best summary of diagnostic performance and can be combined in a complementary diagnostic strategy that includes both a CPR and portable monitoring. Ultimately, portable monitoring will become more widely used since SA is so common, and PSG facilities are relatively scarce. It is a strategy that requires common sense and a clear understanding of what the results indicate, thus it is best left to an experienced clinician who also has access to PSG facilities. In such a case,..
DIAGNOSIS OF SA S247 If, we propose an algorithm (see Fig. 1) that depends on the details and the severity of symptoms that a patient presents with. Acknowledgements: The authors thank the Alberta Lung Association Sleep Centre staff for arranging investigations and data management and Dr. W. A. Whitelaw for reviewing the manuscript. Research was supported by the HSRIF of Alberta, Medical Research Council of Canada, and the AHFMR. REFERENCES 1. Young T, et al. The occurrence of sleep disordered breathing among middle-aged adults. N Engl J Med 1993;328:1230-5. 2. Ferber R. et al. Portable recording in the assessment of obstructive sleep apnea. Sleep 1994; 17(4):378-92. 3. Flemons WW, Whitelaw WA, Brant R, Remmers JE. Likelihood ratios for a sleep apnea clinical prediction rule. Am J Respir Crit Care Med 1994;150:1279-85. 4. Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinical epidemiology: a basic science for clinical medicine. Boston: Little, Brown, 1991. 5. Sox HC Jr. Probability theory in the use of diagnostic tests. An introduction to critical study of the literature. Ann Intern Med 1986; I 04(1 ):60-6. 6. Viner S, Szalai Jp, Hoffstein V. Are history and physical examination a good screening test for sleep apnea? Ann Intern Med 1991;115:356-9. 7. Crocker BD, et al. Estimation of the probability of disturbed breathing during sleep before a sleep study. Am Rev Respir Dis 1990;142:14-8. 8. Fagan TJ. Nomogram for Bayes' Theorem. N Engl J Med 1975;293:257. 9. Jaeschke R, Guyatt G, Sackett D. Users' guide to the medical literature. lama 1994;271 :389-91. to. Guyatt G, Tugwell P, Feeny D, Haynes RB, Drummond M. A framework for clinical evaluation of diagnostic technologies. Can Med Assoc J 1986;134:587-94. 11. Douglas NJ, Thomas S, Jan M. Clinical value of polysomnography. Lancet 1992;339:347-50. 12. Series F, Marc I, Connier Y, La Forge J. Utility of nocturnal home oximetry for case finding in patients with suspected sleep apnea hypopnea syndrome. Ann Intern Med 1993; 119:449-53. 13. Issa FG, Morrison D, Hadjuk E, Iyer A, Feroah T, Remmers JE. Digital monitoring of sleep-disordered breathing using snoring sound and arterial oxygen saturation. Am Rev Respir Dis 1993; 148: 1023-9. 14. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;307-10. 15. White DP, Gibb TJ, Wall 1M, Westbrook PRo Assessment of accuracy and analysis time of a novel device to monitor sleep and breathing in the home. Sleep 1995; 18(2): 115-26. Sleep. Vol. 19, No. 10, 1996