Occupational screening for obstructive sleep apnea in commercial drivers

Transcription

1 AJRCCM Articles in Press. Published on May 13, 2004 as doi: /rccm oc Occupational screening for obstructive sleep apnea in commercial drivers Indira Gurubhagavatula MD, MPH, Greg Maislin MS, MA, Jonathan E. Nkwuo, PhD, Allan I. Pack, MBChB, PhD Center for Sleep and Respiratory Neurobiology, Division of Sleep Medicine, Department of Medicine, University of Pennsylvania Medical Center, Philadelphia, Pennsylvania Pulmonary and Critical Care and Sleep Section, Philadelphia VA Medical Center, Philadelphia, Pennsylvania Address Correspondence to: Indira Gurubhagavatula, MD, MPH Center for Sleep and Respiratory Neurobiology Hospital of the University of Pennsylvania 9th Floor, Maloney Building 3600 Spruce Street Philadelphia, PA Phone: (215) FAX: (215) This article has an online data supplement, which is accessible from this issue's table of content online at This work was supported by Trucking Research Institute contract DTFH61-93-C funded by the Federal Highway Administration (FHA) (now Federal Motor Carriers Safety Administration). The Trucking Research Institute is part of the American Trucking Association. Also supported by NIH grants 3-M01-RR00040, P01-HL-60287, and K23 RR As part of the contract, both the Trucking Research Institute and Federal Highway Administration could comment on the manuscript but could not mandate change. Abbreviated title: Sleep Apnea in Commercial Drivers Subject Code: 110. Sleep-disordered breathing: diagnostic methods Copyright (C) 2004 by the American Thoracic Society.

2 Abstract Excluding the presence of obstructive sleep apnea in commercial drivers is valuable, as the syndrome may increase their risk of sleepiness-related accidents. Using polysomnography as the criterion standard, we prospectively compared accuracies of five strategies in excluding the presence of severe sleep apnea and secondarily, any sleep apnea among 406 commercial drivers. These were: 1) symptoms, 2) body mass index 3) symptoms plus body mass index, 4) a two-stage approach with symptoms plus body mass index in everyone, followed by oximetry in a subset, and 5) oximetry in all. For excluding severe apnea, the two-stage strategy was highly successful, with 91% sensitivity and specificity, and a negative likelihood ratio of This strategy was comparable in accuracy to oximetry, which had a negative likelihood ratio of 0.12, and was 88% sensitive and 95% specific. If we avoided oximetry altogether, then symptoms together with body mass index were 81% sensitive and 73% specific, with a negative likelihood ratio of On the other hand, excluding any apnea could not be done with reasonable accuracy unless oximetry was used. We conclude that two-stage screening is likely to be a viable means of excluding severe sleep apnea among commercial drivers. Key words: polysomnography, nocturnal pulse oximetry, questionnaire 1

3 Introduction Obstructive sleep apnea (OSA) with daytime sleepiness affects 2-4% of Americans (1). Untreated OSA may lead to decreased cognitive function (2), psychomotor impairment (3), decrement in driving skills (4), including increased off-road deviations in driving simulators (4) and increased risk of vehicular accidents (5-7). In the occupational setting, then, OSA is a salient issue for commercial drivers. Among commercial drivers, severe apnea, defined as having an apnea-hypopnea index (AHI) of 30 events/hour (8) on a sleep study, may lead to marked sleepiness and impaired task performance (9). Moreover, treating severe sleep apnea with positive airway pressure (10) improves alertness (11), crash risk (12) and performance assessed by a driving simulator (13), benefits that are not realized after administering placebo (13). However, data for treating mild to moderate sleep apnea, with AHI between 5 and 30 events/hour (8), are less compelling (14-18). Thus, identifying and then treating severe OSA in commercial drivers is of particular interest. Identification of sleep apnea has long relied on in-laboratory polysomnography as the diagnostic standard (19). However, polysomnography is expensive (20), not easily accessible, and remains unsuitable for occupational screening. Other case-identification strategies are readily available and avoid reliance on a specialized laboratory, but have never been evaluated in this occupational setting. We evaluated several such strategies alone and in various combinations, each with increasing -complexity. The simplest strategy we explored depended on response to questions about three apnearelated symptoms. We also chose BMI, since obesity is a major OSA risk factor (20). Additionally, we looked at a risk score that combined information about these symptoms with BMI as well as age and gender, in a tool we developed and called the multivariable apnea 2

4 prediction index (21). Another strategy we chose combined multivariable prediction with nocturnal oximetry in two stages, limiting oximetry to a subset of drivers. The final strategy used oximetry for all drivers, counting the desaturation frequency as a measure of sleep-disordered breathing. In a large cohort of commercial drivers, we compared how well these strategies could exclude individuals with severe sleep apnea. Secondarily, we determined how well they exclude any sleep apnea, defined by AHI 5 events/hour. We have presented results of this study in abstracts (9, 22-25), and a manuscript describing results of performance tests in this sample is in review (26). Methods (Word Count 824) The online supplement contains additional detail regarding subject selection, diagnostic studies and data analysis. Subject Selection As part of a study of determinants of OSA and neurobehavioral consequences, we mailed a questionnaire that asked about age, gender, height, weight, and apnea symptom frequency to 4286 randomly-selected commercial driver s license holders in Pennsylvania, within Philadelphia and its 50-mile radius. We enrolled individuals from among 1329 responders after sorting them into two strata, high or low risk for apnea, to enrich the sample for presence of apnea (1, 27). We performed oximetry and polysomnography in 44.8% (247) of the 551 in the higher-risk stratum and 20.4% (159) of the 778 in the lower-risk stratum. When describing data pooled over risk groups, we computed summary statistics as population-weighted averages of stratum-specific values. The online supplement details our sampling strategy and weighting method. 3

5 Multivariable prediction We determined each respondent s symptom-frequency score for apnea (range 0-4) (21). The multivariable prediction (21) (range 0-1) combines this score with BMI, age and gender, with 0 representing lowest risk and 1 representing highest risk of having apnea. (See online supplement for details). Pulse oximetry Without knowing polysomnography results, a single technician computed the oximetry desaturation index (ODI) from 379 studies as number of desaturations of 3% magnitude divided by test duration in hours. A second re-scored a randomly-chosen 10% of traces; intraclass correlation coefficients were computed to assess reliability. The online supplement contains details regarding oximetry. Polysomnography and Mitigating Bias in the Relationships between Predictive Variables and Apnea Status Prospectively (after questionnaire administration), technical staff monitored electroencephalograms, eye, chin and pre-tibial muscle activity, electrocardiography, oximetry, respiratory effort, and airflow by thermistor(32). Blinded to questionnaire results, staff scored polysomnograms (28) and computed the AHI as the number of apneas plus hypopneas divided by hours of sleep time. An apnea was 10 seconds of airflow cessation, and a hypopnea 50% airflow reduction for 10 seconds, associated with 3% fall in oxyhemoglobin saturation or an arousal. While this 3% decline was part of the definition of a hypopnea and also for an event on oximetry, a desaturation was not a requirement for the definition of a hypopnea. AHI 30 events/hour defined severe apnea, and AHI 5 events/hour defined at least mild apnea (8). 4

6 We assessed the comparability of survey responders, non-responders and in-lab participants to evaluate our degree of success in mitigating participation bias. We compared age, gender, and zip code (a surrogate for ethnicity and socioeconomic status); the role of these variables in OSA has been reviewed previously (29). We could not compare body mass index, as these data were unavailable at the Pennsylvania Driver Licensing Bureau. The Two-Stage Strategy We sorted in-lab subjects based on high, intermediate or low multivariable predictions (30). Two parameters were defined to categorize participants scores into three groups: upper bound separated the high predictions from the intermediate, while lower bound separated the intermediate from the low. High scorers were predicted to have OSA, with subsequent review of their polysomnogram to assess this prediction. Intermediate scorers would undergo oximetry; if the oxygen desaturation index (ODI) a third parameter value (called oxygen desaturation index threshold ), they would be predicted to have OSA and undergo polysomnography. Those with low multivariable prediction or ODI < ODI threshold would be predicted to not have OSA (Figure 1). Determination and comparisons of the optimal cutpoints for the single-stage strategies and the optimal parameter set for the two-stage strategy We determined the discriminatory characteristics of symptoms, BMI, multivariable prediction, and oximetry (30), by computing Area-Under-the-Curve (AUC) (31, 32) (37) for receiver operating characteristic (ROC) curves. These were constructed by computing sensitivity and specificity at various cutpoints using ROCKIT (Chicago, Ill) (33). The optimal sensitivity specificity, and the associated cutpoint were derived by extrapolating from the ROC curve at the 5

7 point where the slope equaled one (34). This slope was selected upon assigning relative weights to false-positive and false-negative diagnoses (see online repository). Negative likelihood ratios were computed as (1-sensitivity)/specificity at this optimal cutpoint. For the two-stage method, we used SAS programming (Cary, NC) to compute sensitivity and specificity for each of 180 combinations of upper bound, lower bound and ODI threshold (30). To do this, for each combination of lower bound, upper bound and ODI threshold, we compared the binary prediction of the screening strategy against the AHI value of each subject s PSG. We computed the total number of false-positive and false-negative predictions. Sensitivity was 1 false negative rate, while specificity equaled 1 false positive rate. We plotted these values of sensitivity against 1-specificity, and computed AUC using SigmaPlot (Rockware, Inc., Golden, CO) (see online repository for details). We determined an optimal parameter set for the two-stage strategies using a procedure analogous to that applied for the one-stage strategies. We found one parameter set for excluding severe apnea, and a different set of values for any apnea Using bootstrap resampling (35), we computed non-parametric 95% confidence intervals around the sensitivity, specificity, negative likelihood ratio and AUC (see online repository). Results Demographics and Apnea Occurrence Among respondents, 93.5% were male, with average ± SD age 44.4 ± 11.2 years. The sample contained 85% Caucasians, 12.5% African-Americans, and 1.9% Hispanics. The data are summarized in Table 1, for 247 high-risk subjects, 159 low-risk subjects, the weighted average of both groups, and the 1329 respondents. The proportion of OSA in the weighted sample was 28.1% using AHI 5/hour to define any apnea, and 4.7% using AHI 30/hour to define severe 6

8 apnea. The weighted averages were computed as (0.42*higher risk mean) + (0.59*lower risk mean). Distribution of BMI Figure 2 shows the BMI frequency distribution of the weighted sample for the following categories: BMI<25 kg/m 2, kg/m 2 (overweight), kg/m 2 (obese, Class I), kg/m 2 (obese, Class II), and 40 kg/m 2 (extremely obese). Approximately half were obese, with BMI 30 kg/m 2, another 38% were overweight, with BMI kg/m 2. The Two-Stage Strategy: Determining Optimal Parameters For identifying severe apnea, upper bound = 0.9, lower bound = 0.3, and desaturation threshold = 10 events/hour are the optimal parameters. For identifying any apnea, these optima are: upper bound = 0.9, lower bound = 0.2, and desaturation threshold = 5 events/hour. Discriminatory Power of Screening Strategies Table 2 shows the relative discriminatory power of symptoms, BMI, multivariable prediction, the two-stage strategy and oximetry for identification of severe apnea expressed as AUC for ROC curves, sensitivity, specificity and negative likelihood ratios. The ROC curves are shown in Figure 3. For severe sleep apnea, the negative likelihood ratio is highest (0.62) when symptoms alone are used. The ratio improved and decreased further with the complexity of the strategy: it was 0.33 for BMI, 0.26 for multivariable prediction, 0.10 for the two-stage strategy and 0.12 for oximetry. We looked at whether increasingly complex strategies increased discriminatory power by increasing sensitivity or specificity. We report these results for single cutpoints, which were specifically chosen using the optimization strategy we described (see online supplement). This 7

9 strategy takes into account two salient criteria: OSA prevalence, and a ratio of false positive to false negative diagnoses. These ratios were assigned so that missing a case was considered more important than wrongly labeling a normal driver as having apnea, particularly so in the case of severe apnea. Using these optimized cutpoints, symptoms alone were least sensitive and specific, BMI was more sensitive and specific than symptoms alone. Multivariable prediction augmented sensitivity offered by BMI alone, from 77% to 81%, and the two-stage strategy raised sensitivity and specificity to 91%. Oximetry enhanced specificity, raising it to 95%, with sensitivity similar to that of the two-stage strategy. The two-stage strategy missed few cases of severe apnea, preserved specificity, and had the lowest negative likelihood ratio. All strategies identified any apnea less accurately than severe apnea (see Table 3). Again, symptoms alone were not particularly useful, with negative likelihood ratio of This ratio again improved, i.e. decreased as complexity of the strategy increased, with the two-stage strategy and oximetry having better discriminatory power (negative likelihood 0.29). No strategy, however, excluded drivers with AHI>=5/hour with acceptably high sensitivity. Discussion Identifying commercial drivers with severe apnea, defined by AHI >= 30/hour, was our primary goal because this cutpoint is associated with marked sleepiness and impaired task performance in our studies (9). Additionally, patients with severe sleep apnea derive benefits from positive airway pressure therapy (10) including reduction in crashes (12) and improved driving performance as measured by a driving simulator (13). In this regard, our analysis shows that the two-stage strategy that combines symptoms plus BMI with oximetry performed very well, with 91% sensitivity and specificity, and yielded 8

10 the best negative likelihood ratio, Applying a standard Bayesian nomogram (36), if this strategy predicts low risk for severe apnea, then given our pretest probability of 4.7%, the likelihood of having severe apnea is below 0.5%. This excludes severe apnea with high confidence (37). This result is particularly useful, since confirmatory PSG testing is expensive and often inaccessible. Additionally, the strategy predicted that oximetry is not necessary in 31% of our sample, nor polysomnography in 86%. The false positive rate was 8.9%, the false negative rate 0.5%, and the negative predictive value 99%. Oximetry applied to every subject maintained negative predictive value at 99%, but is more expensive and less convenient than the two-stage strategy, and did not improve the negative likelihood ratio. Thus, to exclude severe apnea, the two-stage strategy that we proposed previously (30) is optimal for this sample.(42) All of our strategies performed relatively worse in the prediction of any apnea, compared with severe apnea (see Table 3). Again, symptoms alone had little value in this population with negative likelihood 0.75, and oximetry offered minimal additional predictive advantage to the two-stage strategy, with negative likelihood For our two-stage strategy, we chose multivariable prediction, which combined symptoms with BMI, rather than using BMI alone. While BMI<25 kg/m 2 was successful in excluding all but three cases of apnea, only 46 subjects (20% of the weighted sample) met this criterion, whereas 88 (38%) drivers had multivariable prediction <0.3. Thus, addition of symptoms was more useful in this setting than use of BMI alone, since more subjects could potentially be prevented from requiring oximetry. Evaluating these strategies in this group of commercial drivers is likely to be important for public health reasons, as United States accident reports indicate that in 2001, large trucks were involved in 429,000 crashes. Nearly 5,000 of these crashes were fatal ones, responsible for 9

11 12 percent of all traffic deaths, while an additional 130,000 victims suffered non-fatal injuries. Commercial crashes are also expensive, costing on average $75,637 per crash and $3.54 million per fatal crash (38). Sleepiness has been shown to account for 31-41% of major crashes of commercial vehicles (39, 40). While we know little about the role of OSA in crashes in commercial vehicles, studies in passenger cars have shown increased crash risk in drivers with apnea (5-7) and that effective treatment of OSA reduces such risk (12). Our strategies assess the risk of OSA, rather than OSA syndrome, in which the presence of sleep-disordered breathing is accompanied by subjective sleepiness. We did not limit our analysis to those drivers who reported sleepiness in our study. Such sleepiness may itself be subject to reporting bias, particularly in an occupational setting. Therefore, we propose that enrollment of drivers regardless of the report of sleepiness remains one of the study s strengths. Without taking sleepiness into account, the proportion of OSA in our population was similar to that reported by Young et al, who reported that 24% of middle-aged men had AHI5/hour and 9.1% had AHI15/hour (1). Stoohs et al (41) reported a much higher value of 78% with oxyhemoglobin desaturation index5/hour, and 10% with oxyhemoglobin desaturation index 30/hour. In our population, we note that 29% of the weighted sample had oxyhemoglobin desaturation index 5/hour, and 3% had oxyhemoglobin desaturation index30/hour. Diagnostic ascertainment methodologies differed between these studies; similar to Young et al, we used full sleep study data, while Stoohs et al used a less rigorous definition of snoring with 3% desaturation. Since electroencephalogram and airflow data were not recorded during this study, desaturations without airflow limitation could get scored and raise the reported prevalence considerably (42), particularly since this group had a high prevalence (44%) of smoking. This rate is higher than the smoking rate we report in our sample (see Table 1). Pulmonary function 10

12 data, were it available, might explain a high prevalence of desaturation. Additionally, the subjects were recruited from a single employer, rather than being drawn from a more representative, community-based population. We address whether this difference in proportion could be an indication of differential participation in our study not merely on the basis of a given predictor, but rather on both the predictor and apnea status simultaneously (e.g., greater participation among obese subjects who also have OSA, versus obese subjects without OSA). While this bias may limit generalizability, more serious is the potential threat to internal validity. To mitigate this bias, then, given the value of any predictor, participation must be independent of apnea status. Specific design strengths limit, though do not eliminate the likelihood of such bias in our study: prospective data collection, objective disease ascertainment, and blinded scoring of results. We also conducted a non-responder analysis to assess the likelihood of such bias, were it to occur. Because we performed sleep studies only after questionnaire administration, reporting symptoms on the basis of a priori knowledge of apnea status was unlikely. Additionally, an overnight sleep study is the objective standard for apnea diagnosis. Moreover, technicians who scored the sleep studies had no knowledge of any other screening test result. Finally, our comparison of age, gender and socio-demographic factors as assessed by zip code between responders, non-responders and in-lab subjects yielded no statistically significant differences. Indeed, the age and gender distributions were nearly identical (data not shown). While this comparability does not guarantee the absence of participation bias, we are encouraged that responders and non-responders were similar in the variables we assessed. The availability of BMI data in non-responders would have further strengthened our analysis. Despite these 11

13 limitations, this is the first and largest-scale study to address screening for OSA in any high-risk population, where public safety remains a distinct concern. We also note that we validated our strategy in the same cohort in which it was derived, which may artificially inflate its predictive value, a phenomenon known as regression towards the mean (43). Developing the strategy in a subset of our population and validating it in another would have been a stronger approach. However, this split-sample approach is not viable in this study because of insufficient numbers of subjects with severe apnea. Oximetry was conducted concurrently with polysomnography in our study, raising the question of whether the two tests could be scored independently. However, our scorer of oximetry data had no prior knowledge of sleep study data. Additionally, re-scoring of a random 10% sample of oximetry tracings by a second as well as by the original interpreter showed no significant differences in scores. Test-retest and inter-rater reliabilities were high, with intraclass correlation coefficients of 99% and 97%, respectively. Performing oximetry and sleep studies together in the laboratory also assured that the driver was sleeping in an identical position and was in an identical stage of sleep for both studies. Future studies need to evaluate oximetry done independently, and also at home. While we chose oximetry for stage two of our two-stage strategy, we also consider the selection of multivariable prediction for the first stage of this strategy, rather than BMI. While BMI may be a simpler alternative, the multivariable prediction incorporates other information in addition to BMI including age, gender, and symptom frequency. The predictive utility of these additional variables becomes most important in situations when the subject is not obese (21). Thus, we expect our algorithm to have incremental value among populations with lower prevalence of obesity compared with using BMI as the Stage 1 screen. Even in this relatively 12

14 obese population, however, the multivariable apnea prediction provided substantial improvement in discriminatory power over BMI alone. Moreover, the optimal cutpoints we selected for BMI and multivariable prediction, which take into account prevalence and misclassification rates, show that multivariable prediction excluded a much larger proportion of drivers from requiring further testing as compared with BMI alone. Our study raises the question of whether commercial drivers should be screened routinely for severe OSA, perhaps during pre-employment physical examination. Our study is the first step towards addressing this question; the two-stage screening strategy we propose is optimal and highly accurate in excluding severe apnea. Before instituting routine screening, however, we propose that additional data be acquired. A case-control study similar to that done among drivers of passenger cars (7) should be conducted first to evaluate the role of OSA as a risk factor for crashes of commercial vehicles, particularly those involving injury or death. A finding that AHI30/hour is a risk factor for crashes would strengthen our conclusion that the two-stage strategy would be a reasonable approach to screen commercial drivers for this disorder. However, if further studies indicate that an AHI below this level is a more appropriate cutoff, then the screening strategy to be adopted will require further refinement. Second, controlled, randomized trials should assess whether drivers identified by screening will use and benefit from therapy. The costs of screening need to be weighed against the costs of outcomes without screening. Confirming treatment benefits at acceptable economic costs would provide justification for institution of a routine screening program in this occupational setting. We conclude that among our community-based sample of commercial drivers, a twostage screening algorithm that incorporates questionnaire data in all followed by oximetry in a 13

15 subset is useful in excluding drivers with severe sleep apnea, a group who may be at risk for fallasleep accidents judged by off-road deviations in driving simulators (4). Acknowledgements This work was supported by Trucking Research Institute contract DTFH61-93-C funded by the Federal Highway Administration (FHA) (now Federal Motor Carriers Safety Administration). The Trucking Research Institute is part of the American Trucking Association. As part of the contract, both the Trucking Research Institute and Federal Highway Administration could comment on the manuscript but could not mandate change. The work is also supported by NIH grants 3-M01-RR00040, P01-HL-60287, and K23 RR Nellcor, Inc. and Ohmeda, Inc. provided partial support for the study. 14

16 REFERENCES 1. Young T, Palta M, Dempsey J, Skatrud J, Weber S, and Badr S. The occurrence of sleepdisordered breathing among middle-aged adults. New Engl J Med 1993;328: Mitler M. Daytime sleepiness and cognitive functioning in sleep apnea. Sleep 1993;16:S Kim H, Young T, Matthews C, Weber S, Woodward A, and Palta M. Sleep-disordered breathing and neuropsychological deficits. A population-based study. Am J Respir Crit Care Med 1997;156: George CF, Boudreau AC, and Smiley A. Simulated driving performance in patients with obstructive sleep apnea. Am J Respir Crit Care Med 1996;154(1): American Thoracic Society. Sleep apnea, sleepiness and driving risk. Am J Respir Crit Care Med 1994;150: Young T, Blustein J, Finn L, and Palta M. Sleep-disordered breathing and motor vehicle accidents in a population-based sample of employed adults. Sleep 1997; 20: Teran-Santos J, Jimenez-Gomez A, and Cordero-Guevara J. The association between sleep apnea and the risk of traffic accidents. Cooperative Group Burgos-Santander. New Engl J Med 1999;340(11): Sleep-Related Breathing Disorders in Adults: Recommendations for Syndrome Definition and Measurement Techniques in Clinical Research. The Report of an American Academy of Sleep Medicine Task Force. Sleep 1999;22(5): Pack AI, Maislin G, Staley B, George C, Pack FM, and Dinges DF. Factors associated with daytime sleepiness and performance in a sample of commercial drivers [abstract]. Sleep 2001;24(suppl):A

17 10. Sullivan C, Issa F, Berthon-Jones M, and Eves L. Reversal of obstructive sleep apnea by continuous positive airways pressure applied through the nares. Lancet 1982;1: Jenkinson C, Davies RJ, Mullins R, and Stradling JR. Comparison of therapeutic and subtherapeutic continuous positive airway pressure for obstructive sleep apnoea: a randomised prospective parallel trial. Lancet 1999;353(9170): George CF. Reduction in motor vehicle collisions following treatment of sleep apnoea with nasal CPAP. Thorax 2001;56(7): Hack M, Davies RJ, Mullins R, Choi SJ, Ramdassingh-Dow S, Jenkinson C, and Stradling JR. Randomised prospective parallel trial of therapeutic versus subtherapeutic nasal continuous positive airway pressure on simulated steering performance in patients with obstructive sleep apnoea. Thorax 2000;55(3): Barnes M, Houston D, Worsnop C, Neill A, Mykytyn I, Kay A, Trinder J, Saunders N, McEvoy D, and Pierce R. A Randomized Controlled Trial of Continuous Positive Airway Pressure in Mild Obstructive Sleep Apnea. Am J Respir Crit Care Med 2002;165: Engleman H, Martin S, Deary I, and Douglas N. Effect of CPAP therapy on daytime function in patients with mild sleep apnoea/hypopnoea syndrome. Thorax 1997;52: Engleman H, Kingshott R, Wraith P, Mackay T, Deary I, and Douglas N. Randomized placebo-controlled crossover trial of continuous positive airway pressure for mild sleep apnea/hypopnea syndrome. Am J Respir Crit Care Med 1999;159(2): Monasterio C, Vidal S, Duran J, Ferrer M, Carmona C, Barbe F, Mayos M, Gonzalez- Mangado N, Juncadella M, Navarro A, Barreira R, Capote F, Mayoralas L, Peces-barba G, Alonso J, and Montserrat J. Effectiveness of Continuous Positive Airway Pressure in Mild Sleep Apnea-Hypopnea Syndrome. Am J Respir Crit Care Med 2001;164:

18 18. Redline S, Adams N, Strauss M, Roebuck T, Winters M, and Rosenberg C. Improvement of mild sleep-disordered breathing with CPAP compared with conservative therapy. Am J Respir Crit Care Med 1998;157: Block AJ, Cohn MA, Conway WA, Hudgel DW, Powles AC, Sanders MH, and Smith PL. Indications and standards for cardiopulmonary sleep studies. Sleep. 1985;8(4): Pack A. Obstructive sleep apnea. Adv Int Med 1994;39: Maislin G, Pack A, Kribbs N, Smith P, Schwartz A, Kline L, Schwab R, and Dinges D. A survey screen for prediction of apnea. Sleep 1995;18: Pack AI, Maislin G, Staley B, Pack FM, and Dinges DF. Determinants of prevalence of sleep apnea in commercial drivers [abstract]. Am J Respir Crit Care Med 2001;163:A Gurubhagavatula I, Maislin G, and Pack AI. Economics of obstructive sleep apnea (OSA) screening in commercial drivers [abstract]. Am J Respir Crit Care Med 2000;161:A Maislin G, Gurubhagavatula I, Hachadoorian R, Pack F, O'Brien E, Bogage A, Staley B, Dinges DF, and Pack AI. Operating characteristics of the multivariable apnea prediction index in non-clinic populations [abstract]. Sleep 2003;26:A Pack AI, Maislin G, Staley B, Pack FM, and Dinges DF. Obstructive sleep apnea (OSA) in commercial drivers [abstract]. Am J Respir Crit Care Med 2000;161:A Pack AI, Staley B, Pack FM, Rogers WC, George CFP, Dinges DF, and Maislin G. Impaired Performance in Commercial Drivers: Role of Short Sleep Durations and Sleep Apnea (under review). Ann Intern Med Gislason T, Almqvist M, Eriksson G, Taube A, and Boman G. Prevalence of sleep apnea syndrome among Swedish men--an epidemiological study. J Clin Epidemiol 1988; 41(6):

19 28. Rechtschaffen A, and Kales A A manual of standardized terminology techniques and scoring system for sleep stages of human subjects. National Institute of Health, Washington, DC: US Government Printing Office Young T, Peppard PE, and Gottlieb DJ. Epidemiology of obstructive sleep apnea: a population health perspective. Am J Respir Crit Care Med 2002;165(9): Gurubhagavatula I, Maislin G, and Pack AI. An algorithm to stratify sleep apnea risk in a sleep disorders clinic population. Am J Respir Crit Care Med 2001;164(10 Pt 1): Hanley J, and McNeil B. The meaning and use of area under a receiver operating characteristic (ROC) curve. Radiology 1982;143: Harrell F, Lee K, and Mark D. Tutorial in biostatistics: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 1996;15: ROCKIT Software. Department of Radiology, Biological Sciences Division. University of Chicago Hospitals and Clinics. Chicago, Illinois. Available ftp at random.bsd.uchicago.edu McNeil B, Keeler E, and Adelstein S. Primer on certain elements of medical decision making. New Engl J Med 1975;293(5): Efron B, and Tibshirani R. Bootstrap Methods for Standard Errors, Confidence Intervals, and Other Measures of Statistical Accuracy. Stat Sci 1986;1: Fagan T. Letter: Nomogram for Bayes Theorem. New Engl J Med 1975;293: Jaeschke R, Gordon H, Guyatt G, and DL. S. Users' guides to the medical literature. III. How to use an article about a diagnostic test. B. what are the results and will they help me in caring for my patients? JAMA 1994;271(9):

20 Traffic Safety Facts - Large Trucks. National Center for Statistics and Analysis, U.S. Department of Transportation, National Highway Traffic Safety Administration. 39. National Transportation Safety Board Safety Study. Fatigue, alcohol, other drugs, and medical factors in fatal-to-the-driver heavy truck crashes. Volume A Report on the determination and evaluation of the role of fatigue in heavy truck accidents. Prepared for AAA Foundation for Traffic Safety in cooperation with the Commercial Vehicle Safety Alliance by Transportation Research and Marketing Stoohs R, Bingham L, Itoi A, Guilleminault C, and Dement W. Sleep and sleep-disordered breathing in commercial long-haul truck drivers. Chest 1995;107: Levy P, Pepin JL, Deschaux-Blanc C, Paramelle B, and Brambilla C. Accuracy of oximetry for detection of respiratory disturbances in sleep apnea syndrome. Chest. 1996;109(2): Bland J, and Altman D. Regression towards the mean. Brit Med J 1994;308:

21 Word count: 3368 Abstract word count: 197 Figure Legends word count: 221 Tables: 3 Figures: 3 20

22 Figure Legends Figure 1. The two-stage strategy for prediction of apnea, which combines the use of the multivariable prediction index, and oximetry. Stage I consists of administering the symptom questionnaire and calculating the multivariable apnea prediction based on a symptom score, BMI, age and gender. Based on the multivariable apnea prediction, drivers undergo a sleep study if the score is high, or no further testing if the score is low. The intermediate group undergoes oximetry, with subsequent overnight sleep study if the desaturation index is high, and no further testing if the desaturation index is low. Figure 2. Distribution of body mass index among 406 subjects studied in the laboratory. At least 50% of the subjects were obese, while an additional 38% were overweight. Only 11% had BMI values below 25 kg/m 2. Figure 3. Receiver operating characteristic curves with associated area-under-curve for symptoms, BMI, multivariable prediction, two-stage screening and oximetry. Polysomnography was used as the criterion standard. The discriminatory power increases as more administratively complex strategies are used to identify severe sleep apnea. Symptoms alone (thin black solid line) had the lowest AUC, BMI (thin black dashed line) had higher AUC, multivariable prediction (gray solid line) had still higher AUC, the two-stage algorithm (heavy black solid line) had higher AUC than multivariable prediction and oximetry (heavy black dashed line) had the highest AUC. 1

23 Table 1. Characteristics of Study Sample Stratified by Risk Group Variable High Risk (n=247) Studied in Laboratory Total Sample Low Risk Weighted (n = 1329) (n=159) Average Mean age in years (SD) 49.3 (11.6) 42.6 (9.8) 45.4 (7.5) 44.4 (11.2) % men Mean multivariable apnea prediction (SD) 0.64 (0.16) 0.26 (0.14) 0.41 (0.15) 0.49 (0.21) Mean BMI (SD) 33.0 (5.5) 27.7 (3.65) 29.9 (4.86) 28.4 (4.85) Systolic blood pressure (SD) (17.1) (17.2) 132 (0.92) --- Diastolic blood pressure (SD) 80.8 (9.3) 75.8 (11.3) 77.9 (0.58) --- Current smoking (%)* 30.7 (3.0) 31.0 (3.7) 30.9 (2.5) --- Any smoking (%) 63.9 (3.1) 58.9 (3.9) 61.0 (2.6) --- Proportion of Subjects with OSA (SE) No OSA 48.2% (3.2) 88.7% (2.5) 71.9% (2.0) --- At least mild OSA 51.8% (3.2) 11.3% (2.5) 28.1% (2.0) --- At least moderate OSA 23.5% (2.7) 1.3% (0.9) 10.5% (1.2) --- Severe OSA 11.3% (2.0) 0.00% 4.7% (0.8) --- Weighted average computed as (0.415*Higher risk mean) + (0.585*Lower risk mean), and weighted standard error (SE) computed as the square root of (0.415) 2 * (Higher risk SE) 2 + (0.585) 2 * (Lower risk SE) 2 *smoked at least one cigarette during the preceding month

24 Table 2. Discriminatory power of five strategies for predicting severe sleep apnea. AUC is area under receiver-operating-characteristic curve. Sensitivity and specificity are associated with the point on the curve where the slope of the tangent line equals 1. "Cutpoint" is the value of the predictor variable associated with these values of sensitivity and specificity. The reported values of AUC, sensitivity and specificity are bias-corrected means from 1000 bootstrap re-samples (see online repository for details). Predictor Cutpoint AUC 95% CI Sensitivity 95% CI Specificity 95% CI LR NEG 95% CI Symptoms ( ) ( ) ( ) ( ) BMI ( ) ( ) ( ) ( ) Multivariable Prediction Multivariable Prediction + Oximetry ( ) ( ) ( ) ( ) (0.9, 0.3, 10)* ( ) ( ) ( ) ( ) Oximetry ( ) ( ) ( ) ( ) confidence interval likelihood ratio of a negative prediction from a screening test *upper bound, lower bound, desaturation threshold

25 Table 3. Discriminatory power of five strategies for predicting any sleep apnea. AUC is area under receiver-operating-characteristic curve. Sensitivity and specificity are associated with the point on the curve where the slope of the tangent line equals 1. "Cutpoint" is the value of the predictor variable associated with these values of sensitivity and specificity. The reported values of AUC, sensitivity and specificity are bias-corrected means from 1000 bootstrap re-samples (see online repository for details). Predictor Cutpoint AUC 95% CI Sensitivity 95% CI Specificity 95% CI LR NEG 95% CI Symptoms ( ) ( ) ( ) ( ) BMI ( ) ( ) ( ) ( ) Multivariable Prediction Multivariable Prediction + Oximetry ( ) ( ) ( ) ( ) (0.9, 0.2, 5)* ( ) ( ) ( ) ( ) Oximetry ( ) ( ) ( ) ( ) confidence interval likelihood ratio of a negative prediction from a screening test *upper bound, lower bound, desaturation threshold

26 Figure 1 Stage I Multivariable Apnea Prediction Questionnaire Low score n=1329 Intermediate Score High Score No further testing Low Desaturation Index Stage II Oximetry n=379 High Desaturation Index Sleep study to confirm OSA n=406

27 Figure 2 Proportion Studied in Laboratory % 30.0% 13.8% 11.3% 6.4% <25 [25-30) [30-35) [35-40) >= 40 BMI Interval (kg/m2)

28 Figure 3 1 SENSITIVITY SPECIFICITY

29 Occupational screening for obstructive sleep apnea in commercial drivers Methods Section for Online Repository Only Indira Gurubhagavatula MD, MPH, Greg Maislin MS, MA, Jonathan E. Nkwuo, PhD, Allan I. Pack, MBChB, PhD.

30 All subjects provided signed, informed consent before enrollment. The University of Pennsylvania Institutional Review Board approved the protocol. Subject Selection for Questionnaires The state of Pennsylvania provided the names of subjects to whom we mailed our questionnaires. This was a random sample of holders of commercial drivers licenses in Pennsylvania within 50 miles of our Sleep Center. Of 4286 questionnaires mailed, we received 1486 (33.0%) responses. Of these, 95 responses were unusable because the CDL holder had died or relocated, so we evaluated our screening strategies among the remaining 1391 (31.5%). Of these, 1329 had complete data to permit calculation of likelihood of apnea [E1] that was used to stratify the sample.. Calculation of Multivariable prediction Our mailed questionnaire asked: During the past month, have you had, or have you been told about, the following symptom: 1) snorting or gasping, 2) loud snoring, and 3) breathing stops, choking or struggling for breath. Respondents rated symptom occurrence as: Never (0); Rarely, less than once/week (1); Once or twice/week (2); Three or four times/week (3); Five to seven times/week (4); Don t know. We computed the mean symptom-frequency score, with a potential range 0 to 4. When all symptoms were present 5 to 7 times/week, the symptom score was 4. This was the simplest strategy we assessed. We combined this symptom score with BMI, age and gender to determine the multivariable prediction [E1], which ranged 0 to 1, with 0 representing no risk and 1 representing maximal risk for OSA. Construction of the In-Lab Sample We used a two-tiered stratified sampling technique [E2, E3] to select respondents for oximetry and polysomnography. We sought to recruit 250 from the 500 respondents at the

31 highest risk for apnea, and 160 from the remaining, lower-risk respondents, to enrich our final sample studies in the laboratory for the presence of OSA. Our selected sample sizes ensured that the stratum-weighted, pooled proportion with least moderate OSA (AHI 15 events/hour) could be estimated with a margin-of-error of ± 3%, assuming rates of 15% in the higher-risk group and 7% in the lower-risk group. We first sorted the 1329 respondents by the multivariable apnea prediction scores, which has a scale between zero and one (see above) [E1], and invited respondents with the top 500 scores, plus an additional 51 before reaching our recruitment goal. When we reached the driver with the 551 st largest score (apnea prediction = 0.436), we had recruited 247 drivers (44.8%). From the remaining 778, we recruited 159 (20.4%) respondents in random order. Thus, 406 subjects (247 high-risk and 159 low-risk) completed in-laboratory studies. We had usable oximetry data on 379 of these. The average ± SD recording time on oximetry was 6 hours and 47 minutes ± 47 minutes. Calculation of Pooled Summary Statistics for In-Lab Sample When describing the in-lab sample pooled over risk groups, summary statistics were computed as weighted averages of stratum-specific values with standard errors and confidence intervals computed using conventional methods for stratified random sampling. Stratum weights for the higher and lower risk group weights were (=551/1329) and (=778/1329), respectively. These weights reflected our estimates of the proportions of commercial driver s license holders at high and low risk in our population, where risk was assigned as high or low depending on whether the multivariable prediction was above or below , respectively. Other statistical analyses, such as the ones characterizing the associations between predictive strategy and sleep study data, were already conditional on sample characteristics, and so we did not use sample weights in those analyses.

32 (25)Scoring of Overnight Oximetry Continuous, transcutaneous oximetry data were recorded during polysomnography using the N-200 oximeter (Nellcor Inc., Pleasanton, CA) by finger probe. Sampling frequency was 3 Hz, and paper speed 15 cm/hour. We received usable data on 379 studies. The average ± SD of the duration of the recordings was 7 hours and 32 minutes ± 47 minutes. A single observer counted desaturations without knowledge of polysomnography results. A desaturation occurred if the saturation trace dropped by 3% below the immediately preceding baseline. Desaturation ended when the level rose by 2% above the nadir. Finger movement or probe dislodgment artifacts were not counted; these were 1) steep, vertical falls in the trace, followed by steep recoveries, or 2) short, uniform vertical marks 2-4% in magnitude, identifying low quality signal. The oximetry desaturation index (ODI) was the number of desaturations of 3% magnitude divided by test duration in hours. This first observer and a second re-scored a randomly chosen 10% of traces; intra-class correlation coefficients were computed to assess scoring reliability. (32)(12)Definition of the Two-Stage Strategy In our strategy to combine the multivariable prediction and oximetry, we separated in-lab subjects into three groups: those with high, intermediate and low multivariable predictions [E4]. The cutpoints (or parameters) that separated the three groups were variables. We called the value of the multivariable apnea prediction score that separated the high group from the intermediate the upper bound, while the score separating the intermediate from the lower group was the lower bound. Those with scores in the high range were predicted to have OSA, with subsequent review of their sleep study to assess this prediction. Those with scores in the lower range would be predicted to be free of OSA and would not be further studied. We assessed the result of their sleep study to determine whether our prediction of no sleep apnea was correct.

33 Those with scores in the intermediate range would, in this strategy, undergo oximetry; this group s desaturation indices were compared against a threshold value, a variable called the ODI threshold. Those with desaturation indices exceeding this threshold would be predicted to have OSA and hence would undergo polysomnography, while the rest would be predicted to be free of OSA (see Figure 1). We would assess their sleep study results to determine the correctness of our prediction. We determined the optimal values for these three variables (upper bound, lower bound, desaturation threshold) for predicting severe apnea [E5], and as a secondary objective, a different set of values for any apnea [E5]. Determination of Discriminatory Power We performed Area Under the Curve (AUC) analysis for receiver operating characteristic (ROC) curves [E6, E7], using ROCKIT (Chicago, IL) [E8]. This analysis determined the relative discriminatory power of symptoms, BMI, multivariable prediction, and oximetry (34). ROC curves were also constructed for the two-stage strategy, by plotting sensitivity against 1-specificity (see below). Based on published methods [E9], optimal sensitivity and specificity were identified by taking the values associated with the point on the ROC where the tangent line s slope equaled [(1-p)/p][FP/FN], where p = apnea prevalence, FP = false positive rate, and FN = false negative rate. We believed that missing cases of severe apnea was much more costly than a false positive, a reasonable assumption for screening applications with public safety implications, and so we weighted FP/FN at 1:20. For predicting any apnea, we weighted this ratio at 1:3, assuming that the cost of missing a case of any apnea was less serious. The estimated proportion of any apnea in our population was and the proportion of severe apnea was Thus, for AHI 30/hour, the slope = ( )/(0.047)(1/20) ~ 1 and for AHI 5/hour, the slope = ( )/(0.281)(1/3) ~ 1.

34 Building the ROC Curve for the Two-Stage Screen For each of our two objectives, we varied upper bound of the multivariable prediction score from 0.2 to 0.9, in 0.1 unit increments. We varied lower bound from 0.1 to the value of upper bound minus 0.1. Thus, when upper bound was 0.9, there were 8 values of lower bound (0.1 to 0.8, in 0.1-unit increments). When upper bound was 0.8, there were 7 values of lower bound, and so on. We varied threshold ODI from 5 to 25 events/hour in increments of 5 events/hour, as described previously [E4]. Using SAS programming (Cary, NC), for each of the [( )X5]=180 combinations of these three variables, we computed sensitivity and specificity for predicting severe apnea, and secondarily, at least mild apnea. We generated a receiver-operating-characteristic curve for the two-stage strategy by plotting sensitivity against 1-specificity. We note that several sensitivity values could be associated with a unique specificity. To address this issue, we rank-ordered specificity, then selected the highest corresponding cutpoint value associated with the non-unique sensitivities associated with that specificity value. We plotted sensitivity against 1-specificity, and computed AUC using SAS. We determined the optimal sensitivity and specificity as above, by selecting the sensitivity and specificity associated with the point on the ROC curve whose tangent line gave unit slope. The parameter combination associated with this value of sensitivity and specificity was the optimum parameter set. We reported this sensitivity and specificity in Table 2, and we computed and reported negative likelihood ratios as (1-sensitivity)/specificity. Calculation of Confidence Intervals for the Single-Stage Strategies Using the SAS jackboot macro, we performed bootstrap re-sampling [E10] to generate non-parametric 95% confidence intervals around the estimates of AUC, sensitivity, specificity, and negative likelihood ratios shown in Table 2.

35 For our single-stage strategies, we re-sampled the in-lab data on 406 participants regarding symptoms, BMI, multivariable predictions, and oxyhemoglobin desaturation indices. We computed AUC s as the c-statistic generated from a SAS logistic regression on 1000 boostrap re-samples. We used the optimal cutpoint for each strategy to compute sensitivity, specificity, and negative likelihood ratios on each of 1000 bootstrap re-samples, creating a bootstrap distribution of values for each of these estimates. We then computed nonparametric 95% lower and upper confidence limits using the 2.5 th and 97.5 th percentile values of each of these three (sensitivity, specificity and negative likelihood ratio) distributions. Calculation of Confidence Intervals for the Two-Stage Strategies: AUC For the two-stage strategy, we constructed confidence intervals around AUC in the following way. First, we re-sampled from the in-lab data (multivariable prediction, oximetry and AHI values), and from each, we applied our 180 sets of upper bound, lower bound and ODI threshold. Doing so provided 180 values of sensitivity and 1-specificity for each re-sample, for each of our two AHI criteria (>= 5/hour and >= 30/hour), from which we built 1000 ROC curves. The AUC was computed using the SAS area macro for each of these curves. As for the singlestage strategies, we determined nonparametric 95% lower and upper confidence limits using the 2.5 th and 97.5 th percentile values of AUC. Calculation of Confidence Intervals for the Two-Stage Strategies: Sensitivity, Specificity and Negative Likelihood To compute confidence intervals around sensitivity, specificity and negative likelihood for the two-stage strategy, we first selected the optimum cutpoints for ODI threshold, upper bound and lower bound, as described above. We re-sampled values of BMI, multivariable prediction and ODI threshold, and applied these cutpoints to each set of re-sampled data. Doing