* Cedars Sinai Medical Center, Los Angeles, California, U.S.A.

Transcription

1 Sleep. 18(2): American Sleep Disorders Association and Sleep Research Society Home Monitoring-Actimetry Assessment of Accuracy and Analysis Time of a Novel Device to Monitor Sleep and Breathing in the Home *tdavid P. White, *tthomas J. Gibb, *John M. Wall and *Philip R. Westbrook * The National Jewish Center, Denver V.A. Medical Center; t University of Colorado Pulmonary Division, Denver; and * Cedars Sinai Medical Center, Los Angeles, California, U.S.A. Summary: Obstructive sleep apnea is increasingly recognized as a common and debilitating disorder. As a result, a variety of diagnostic technologies have evolved to potentially decrease cost and improve access and ease of assessment. In this study we compared the Healthdyne NightWatch (NW) System (a home sleep diagnostic methodology) to standard polysomnography (PSG) in two sleep centers. Two separate studies were completed. NW was compared to a simultaneously obtained PSG in 30 patients (IN-LAB study). Seventy additional patients were studied in both the home with NW and in the laboratory with PSG (HOME-LAB study). The NW system records eye movement, leg movement, Sa0 2, nasal-oral airflow, chest and abdominal wall motion, body position and heart rate on a solid state recorder, which permits sleep staging based on body and eye movement and standard respiratory assessment. For the PSG, standard paper recording techniques were used. The IN-LAB study revealed a correlation between NW and PSG for total sleep time of r = 0.72, with NW tending to score some awake time as nonrapid eye movement sleep. The correlation for apnea-hypopnea index (AHI) was r = 0.94 between systems, with a sensitivity of 100% and specificity of 63.6% at an AHI threshold of 10. The HOME-LAB study demonstrated understandably poor correlations between NW and PSG for most measures of sleep, which is likely a product of night-to-night variability in sleep, home versus laboratory effects and the differences in sleep staging methodology. However, the correlation for AHI was r = 0.92, with a sensitivity of 90.7% and a specificity of 70.4% at an AHI threshold of 10. Using a new methodology to assess agreement between diagnostic systems, we observed 78.6% diagnostic agreement between NW and PSG in the HOME-LAB study, with NW underestimating AHI 4.3% of the time and overestimating it in 17.1 % of cases. This may relate to night-to-night variability in AHI or greater NW computer sensitivity to subtle hypopneas. We conclude that NW provides an accurate determination of AHI in both the home and laboratory, using limited instrumentation. The analysis time for NW is also reduced compared to PSG, and patients generally prefer the NW evaluation. Key Words: Sleep-Apnea-Home monitoring-polysomnography. Obstructive sleep apnea is increasingly recognized as a common disorder with important morbidity and potential mortality. The most recent comprehensive epidemiological study suggests that 4% of adult men and 2% of adult women have symptomatic sleep apnea (1), and an even greater number of individuals demonstrate more apneas and hypopneas than is commonly considered normal. The potential sequelae of sleep apnea are substantial and include sleep disruption with waking hypersomnolence and poor daytime performance (2,3) and the consequences of recurrent hyp- Accepted for publication September Address correspondence and reprint requests to David P. White, M.D., Respiratory Care (IliA), Denver V.A. Medical Center, 1055 Clermont St., Denver, CO 80220, U.S.A. 115 oxia and hypercapnia, including systemic and pulmonary hypertension (4), arrhythmias (5) and possibly decreased survival (6). Thus, identification and treatment of these patients is important. To date, most diagnostic evaluations of potential sleep apnea patients include an in-laboratory polysomnogram during which a complete montage of sleep and respiratory variables are monitored throughout the night. However, such studies are time consuming and expensive. To reduce cost and potentially improve patient access to sleep evaluation, a number of devices have been developed to either screen or diagnose patients with sleep apnea in the home. Such devices have been as simple as an oximeter (7,8) or as complex as a portable multichannel complete recording system to monitor sleep and breathing (9). Each such device has

2 116 D. P. WHITE ET AL. its own potential advantages and disadvantages, although very few have been carefully validated for accuracy in both the sleep laboratory and the home. Most home sleep monitoring systems suffer from one of several potential problems. First, the majority (10-15) do not actually monitor sleep itself, making it difficult to determine the frequency of apneas and hypopneas per hour of sleep (apnea-hypopnea index, AHI). As this AHI is the standard measure of sleepdisordered respiration, its absence from most home sleep reports makes interpretation of their results potentially difficult. However, at least one study suggests that sleep staging even in the laboratory does not substantially improve diagnostic accuracy (16). As a result, this issue is controversial. Second, signal quality during home evaluation is virtually never assessed during data acquisition, but only during subsequent analysis. As a result, if an important signal is lost or is of poor quality, the study must be repeated. Previously, studies indicated such failure rates to be between 9 and 33% (10-15) when devices measuring more than oximetry alone were assessed. Thus, important potential problems do exist with most such systems. In this study we evaluated, at two sleep centers, a novel system to monitor sleep and breathing in the home. The potential advantages of the system evaluated are two. First, sleep-linked behavior (body and eye movement) as a measure of sleep are continuously monitored such that AHIs can be determined. Second, near real-time data can be intermittently transmitted from the home to the laboratory by modem, so that transducer function can be verified and corrected if needed. This system was evaluated during simultaneous standard polysomnography in the sleep laboratory in one group of patients, with a second patient group undergoing both polysomnography in the laboratory and sleep recording in the home. METHODS This assessment consisted of two separate evaluations of this home monitoring system [NightWatch (NW)]: an in-laboratory study (IN-LAB) where NW results were compared to simultaneously obtained standard paper polysomnography (PSG) and a homelaboratory (HOME-LAB) comparison, during which PSG results obtained in the laboratory were compared to NW data acquired in the home. Both components of this study were conducted in two separate laboratories: the National Jewish/University of Colorado Sleep Center (Denver, CO, U.S.A.) and The Cedars Sinai Sleep Disorders Center (Los Angeles, CA, U.S.A.). The results from the two centers were grouped for data analysis as there was little systematic difference in the observations of the two laboratories. The methodologies for these studies will be described separately. IN-LAB study Thirty patients were studied, 15 in each sleep center. Each patient had been seen by a physician who ordered a standard polysomnogram to determine if obstructive sleep apnea was present. The expertise of these referring physicians was likely somewhat variable and thus a broad range of pretest probabilities for sleep apnea likely existed in these patients. However, no formal pretest probability evaluation was undertaken. Therefore this patient population is likely typical for a sleep laboratory. All patients signed an informed consent to participate in this study, which had the approval of each institution's Human Studies Committee. Each participant spent 1 entire night (about 7 hours) in the sleep laboratory, during which the standard PSG and the NW were simultaneously recorded on paper and on a solid state recorder, respectively. Any problems encountered with either system were immediately corrected during the night by a technician. For polysomnography the following signals were recorded on paper: three channels of electroencephalogram (EEG), two channels of electrooculogram, the submental electromyogram (EMG), nasal-oral airflow (thermistor), chest plus abdominal wall motion [either inductance plethysmography (NIMS, Miami, FL, U.S.A.) or piezo electrodes (Pro-Tech Services, Woodinville, W A, U.S.A.)], arterial oxygen saturation [Ohmeda oximeter set on fast (3-second) response mode with pen liz amp frequency response set at 15], anterior tibialis EMG and electrocardiography. The patient's sleeping position was observed visually (video camera) and recorded on the paper. All signals were recorded on a Grass Model 78E polygraph (Grass Instruments, Waltham, MA, U.S.A.). The NightWatch System records the following signals on a compact solid state recorder (Fig. 1) with a sampling and storage rate of 4 Hz for each signal: eye movement (one channel, piezo electrode), leg movement (one channel, piezo electrode), arterial oxygen saturation (Nonin Medical Inc., Plymouth, MN, U.S.A.), nasal-oral airflow (thermistor), chest plus abdominal wall motion (piezo electrodes), body position and movement (mercury gauge placed on the chest), and heart rate. Completely separate recording electrodes were used to obtain NW signals from those used for the PSG described above. The NW System has the ability to send to the laboratory analysis station, by modem, 2-minute portions of the complete recording so that signal quality can be assessed and transducer function corrected if needed (i.e. oximeter probe ad- Sleep. vol. 18. No

3 HOME MONITOR OF SLEEP AND BREATHING 117 FIG. 1. Depicted is an individual with all NightWatch (NW) monitoring equipment attached. The NW system monitors eye movement, leg movement, arterial oxygen saturation, nasal-oral airflow, chest plus abdominal wall motion, body position and movement, and heart rate. justed, chest-abdominal motion bands tightened, etc.). The frequency of such transmission can be set prior to beginning the study and adjusted during the study. During this IN-LAB study, data were transmitted from the sleep room to the monitoring station outside the sleep room at least every 30 minutes. If signal problems were identified, the technician would enter the room and correct the problem. Sleep was staged on the PSG using standard Rechtschaff en and Kales criteria (17). The NW System stages sleep in a manner not requiring EEG using the following general algorithm: eye movement + body movement = wakefulness, no eye movement or body movement = nonrapid eye movement (NREM) sleep, eye movement + no body movement = rapid eye movement (REM) sleep. Respiratory abnormalities were defined similarly for both recording systems. Apneas were defined as ~ 10-second pauses in respiration and were defined as central if there was no associated effort, obstructive if respiration was present throughout the apnea, and as mixed if a central respiratory pause was followed by obstructed ventilatory efforts. Hypopneas were defined as a decrement in airflow (generally> 50%) associated with either an arousal from sleep at termination or a fall In arterial oxygen saturation (4% in Denver and 2% in Los Angeles). These different oxygen desaturation requirements were selected due to the lower barometric pressure (about 630 mm Hg) in Denver and thus the greater likelihood of oxygen desaturation in Denver with small decrements in ventilation. An arousal was defined for PSG as 3 seconds of alpha EEG or a clear frequency shift, whereas for NW it was defined as 5 seconds of leg or eye movement, usually with an obvious increase in heart rate although no specific criteria for heart rate changes were defined. If neither arousal nor desaturation were observed, the event was not scored. Hypopneas were not scored as central or obstructive. Finally, arterial oxygen saturation is recorded continuously on NW, and thus the mean and lowest sleeping oxygen saturation values can be easily computer determined. On paper, the lowest value can be readily assessed. However, the mean sleeping value is more difficult to calculate. To accomplish this, the technician estimated the mean Sa0 2 levei on every 30-second epoch, with mean values for the night being determined from these estimates. HOME-LAB study Seventy patients were studied, 50 in Los Angeles and 20 in Denver. These participants were different patients from those included in the IN-LAB protocol. Each was referred to the sleep center for polysomnography to determine if obstructive sleep apnea was present. Therefore, this group represented a similar population to that described above for the IN-LAB study. Each signed an informed consent to participate in the study, which had the approval of each institution's Human Studies Committee. Each participant spent 1 complete night (about 7 hours) in the sleep laboratory undergoing standard polysomnography. During this study all signals were recorded only on paper, using the PSG methodology described above. These individuals were also studied for I complete night in the home, using the NW System monitoring the signals described above. To complete this home study each patient reported to the sleep lab- Sleep. Vol. 18. No

4 118 D. P. WHITE ET AL. oratory on the afternoon or evening of the day they were to be studied. At that time each patient was thoroughly instructed in how to apply the NW equipment and how to record the data. Depending on the time of day and distance to the patient's home, it was common for some of the equipment to be attached in the sleep laboratory and worn home. The amount of equipment attached at the laboratory varied, with most patients having the majority, if not all, of the equipment attached in the laboratory; however, it was common for the patient to attach the eye sensor at home. The patients were then instructed on how to connect the sensors to the recorder and how to set up the recorder and modem. At bedtime, after all equipment was attached and the system was set to record, a 2-minute data set was sent by modem to the laboratory to ensure that all transducers were functioning properly. If not, the patient was called, transducers were adjusted and another data set was sent. This procedure continued until all signals were satisfactory. At that time the patient was allowed to sleep. Throughout the night, 2-minute data sets were transmitted by modem at least every 30 minutes to the sleep laboratory and, if signal problems were detected, the patient was called by phone and the transducers corrected. In the home, the patients were allowed to sleep as late as they wished and returned the equipment to the laboratory the next day. We attempted to randomize the order of these studies (home vs. laboratory) and accomplished this in many cases. However, for scheduling reasons, a number of patients had to be studied in the laboratory prior to the home, and thus a true complete randomization was not accomplished, with more patients being studied in the laboratory first. In every case, however, both studies were completed within 10 days of each other, thus avoiding the confounding influence of a long time delay between studies. In addition, we attempted as much as possible to begin both studies at approximately the same time in the evening. However, the laboratory study generally ended after about hours due to technician constraints, whereas in the home the patients were allowed to sleep in the morning as long as they wished. We also instructed patients to be as consistent as possible on the 2 study nights in terms of alcohol consumption, food intake, etc. Questionnaire data indicate that most complied. Finally, each patient filled out a questionnaire after both studies had been completed regarding their preference for recording technique (home vs. laboratory). This questionnaire included a linear analog scale addressing the patient's satisfaction with the study. This satisfaction value was compared between the two study techniques (home vs.laboratory) and the technique given the higher score was recorded as the preferred sleep study methodology. Data analysis All studies (NW and PSG) at each sleep center (Denver and Los Angeles) were analyzed by a single sleep technician (T.G. or l.w.). That technician, in every case (PSG and NW), was blinded to the identity ofthe patient's record being scored. In the case of the PSG, each page of the paper record was scored by hand (as described above) and the results were simply mathematically tabulated. The NW system comes with a computer analysis package, which stages sleep using the algorithm described above. In addition, the parameters by which apneas and hypopneas are defined can be programmed into the system and the record will be analyzed for respiratory abnormalities. For each NW record, the technician set all parameters appropriately and then allowed the computer to score the record. However, once the computer analysis was completed, the technician subsequently evaluated the entire record in 2-minute epochs to ensure that the computer analysis was correct. When apparent errors were identified, they were corrected. The results were then mathematically tabulated by the computer. All records were scored, as described above, for sleep stages (wake, NREM and REM), respiratory abnormalities and arterial oxygen saturation. In addition, we determined both patient satisfaction with each study technique (questionnaire) and technician record analysis time. In comparing the two systems, PSG versus NW (both IN-LAB and HOME-LAB), we focused on certain variables. For sleep itselfwe assessed total sleep time, sleep efficiency [(time asleep/time in bed) 100], sleep latency (time to sleep onset), REM latency, percent of sleep time that was NREM (% NREM) and percent of sleep time that was REM (% REM). For respiration we focused on three variables: apnea/hypopnea index [the number of apneas plus hypopneas/hours of sleep (AHI)], the minimal Sa02 (arterial oxygen saturation) observed during the night and the mean sleeping Sa02 (mean Sa02)' As there is little clinical reason to distinguish apneas from hypopneas, we grouped them together in the AHI. Also, as the vast majority of apneas were obstructive or mixed, and as there was little difference between systems in defining apnea type, all types of apnea were again grouped in the AHI. We also compared the periodic leg movement index (number of PLMs per hour of sleep) and the analysis time for a single study. The data from the two studies (IN-LAB and HOME LAB) were analyzed separately. In each case, the results of the PSG were directly compared to the NW using three approaches. First, a simple, paired, two-tailed t test was applied to the data for each variable to determine if systematic differences between the systems Sleep. Vol. 18. No

5 Gender Age (years) Height (cm) Weight (kg) BMI (kglm2) TABLE 1. Patient characteristics IN-LAB study (n = 30) 23 male, 7 female 51.1± ± ± HOME MONITOR OF SLEEP AND BREATHING 119 HOME-LAB study (n = 70) 52 male, 18 female 47.9 ± ± ± Values for age, height, weight and BMI are expressed as means + SE. - existed. Second, a least squares linear correlation was determined for each variable between the PSG and the NW to assess how closely NW data related to PSG results. Third, we determined the frequency of diagnostic errors encountered with NW (assuming the PSG results to be the gold standard) based on standard sensitivity and specificity determinations at two AHI thresholds (10 and 20 events per hour). Positive and negative predictive values were also calculated at these thresholds. However, we believe such specificity and sensitivity determinations to be somewhat flawed as a tool in assessing the validity of a new diagnostic approach to sleep apnea such as NW. This is the case for two reasons. First, there is no well-defined "cutoff' above which sleep apnea is diagnosed and below which it is excluded. Second, small differences in AHI between systems (one to two events per hour), which are clinically unimportant, may be interpreted as total diagnostic errors during sensitivity and specificity assessment. As a result, the following new approach was also developed for determination of diagnostic errors: Diagnostic Agreement: AHI ~ 40 on both systems or, if AHI < 40 on PSG, the AHI was within 10 events/ hour on both systems; Overestimate of A HI: AHI was 10 events/hour greater on NW than PSG (both < 40/ hour); Underestimate of AHI: AHI was 10 events/hour less on NW than PSG (both < 40/hour). Variable All values will be reported as the mean ± the standard error of the mean (SE), with statistical significance being accepted when p < RESULTS The characteristics of the subjects included in the two studies are listed in Table 1. Again, all were referred to the laboratory primarily for a diagnosis of obstructive sleep apnea. IN-LAB study TABLE 2. IN-LAB data comparison (n = 30) PSG (mean ± SE) Total Sleep Time (minutes) 335 ± 17 %NREM 86 ± 1 % REM 14 ± 1 Sleep efficiency (%) 76 ± 2* Sleep latency (minutes) 15 ± 3 REM latency (minutes) 122 ± 11* AHI (no.lhour) 31 ± 6 Minimum Sa02 (%) 81 ± 2 Mean Sa02 (%) 92 ± 1* PLM index (no.lhour) 9 ± 5 Analysis time (hours) * * p < 0.05, PSG different from NW. ** P < 0.05, statistically significant relationship. As can be seen in Table 2, there was a general similarity of sleep-staging results between the two systems when data were collected simultaneously. However, the NW system tended to score more total sleep time (364 vs. 335 minutes, p = 0.06) and therefore a greater sleep efficiency (83% vs. 76%). There was also a small, but statistically longer REM onset latency on the PSG (122 vs. 116 minutes). All other sleep variables were similar when the entire group is considered. When individual results were correlated (r values), there were always statistically significant relationships between variables (Table 2). However, as can be seen in Fig. 2 (where this relationship for total sleep time is demonstrated for the two systems) there was some variability between NW and PSG in determining if the patient was asleep. As stated above, NW tended to score some awake time as NREM sleep. When respiration is considered, there was quite good agreement between systems, as can be seen in Table 2 and Fig. 3. The AHI correlated quite well between NW and PSG, as did the various measures of Sa0 2 The PLM index did not correlate particularly well between the systems (r = 0.34), with NW tending (with one exception) to demonstrate fewer PLMs than the PSG. Fi- NW (mean + SE) r Value 364 ± ** 87 ± ** 13 ± ** 83 ± ** 14 ± ** 116 ± ** 32 ± ** 81 ± ** 93 ± ** 7± ** 95% confidence interval for r value Low High

6 120 D. P. WHITE ET AL. 500 IN LAB STUDY 150 IN LAB STUDY.. ' Sleep Time 300 NW (min.) r=.72 p<.ooi 50=t----:~-r----,r r Sleep Time PSG (min.) FIG. 2. The relationship between total sleep time measured by ~ versus PSG during the IN-LAB study is shown. The dotted hnes denote the 95% confidence interval for individual observations. As can be seen, NW tended to score more sleep than the PSG although a highly significant relationship was observed between th~ systems. nally, the analysis time for NW was always substantially less than for the PSG. The sensitivity and specificity of the NW system in diagnosing sleep apnea (when data were acquired simultaneously with PSG) are described in Table 3. At an AHI threshold of 10, NW was 100% sensitive. However, there were four false positives (63.6% specificity). In three cases the AHI on NW was between 10 and 15 and thus was only slightly above the threshold. In the fourth, clear cycling hypopneas were evident on NW where condensed respiratory signals could be exam~ in~d, but were subtle and were missed on paper. Thus, this may actually represent a false negative on the PSG. At an AHI threshold of 20, the NW sensitivity fell to 76.9% with three false negatives. Again, in two cases, the AHI on NW was between 10 and 20 and thus was quite close to the threshold value. In the third case, the same total number of disordered breathing events (apneas plus hypopneas) was observed on both systems, but NW overscored total sleep time, yielding a reduced AHI. The specificity at the AHI = 20 threshold AlII NW (Events per Hour) AlII PSG (Events per Hour) FIG. 3.. The relationship between AHI measured by NW versus PSG dunng the IN-~AB study is shown. The dotted lines denote the 95~ confidence Interval for individual observations. As can be seen, WIth rare exceptions, a tight linear correlation was observed. was 88.2% with two false positives. One of these false positives was the same case described above, where NW scored subtle hypopneas that were apparently missed on the PSG. The other was a case where NW scored an AHI of25, which was only minimally above the threshold.. Finall~, when our new methodology for assessing diagnostlc agreement between systems was utilized (Table 4), NW agreed with PSG 86.7% of the time. In two cases (6.7%) NW overestimated AHI. In both cases, as discussed above, subtle but clearly present hypopneas were detected on NW and missed on paper, with both potentially representing PSG false negatives. In two cases (6.7%) NW underestimated AHI. In one case, PSG-detected hypopneas were missed on NW and in the other, as discussed above, NW simply overscored sleep, yielding a reduced AHI. HOME-LAB study.in assessing the HOME-LAB results, one must reahze that at least three independent variables can in- TABLE 3. Sensitivity, specificity and predictive values of NW IN-LAB study (n - 30) HOME-LAB study (n = 70) AHI> 10 AHI> 20 AHI > 10 AHI > Sensitivity (%) Specificity (%) Positive predictive value (%) Negative predictive value (%)

7 TABLE 4. Diagnostic agreement Diagnostic agreement (%) Overestimate of AHI (%) Underestimate of AHI (%) HOME MONITOR OF SLEEP AND BREATHING 121 HOME-LAB IN-LAB study study (n = 30) (n = 70) fluence the relationship of NW to PSG data. These include night-to-night variability in sleep and respiration, the influence of sleeping in the home versus the laboratory, and the difference in monitoring techniques (NW vs. PSG). The methodologies used in this study do not allow us to meaningfully separate these possibilities. The characteristics of the 70 patients studied are listed in Table 1. There was a total of two failures of the NW system in the home (2.9%) and adequate data were not obtained. These patients were obviously not included in data analysis. However, two additional patients were studied, so that the total number of participants with analyzable data was 70. In 13 of these patients, the time required for "hook up" for both the PSG and the NW was recorded. These times were quite similar at 36.2 ± 3.1 minutes for PSG and 39.2 ± 1.8 minutes for NW. During the NW studies in the home, after the initial call or calls were completed to verify signal quality, the technician recorded the number of additional calls required to correct equipment or signal problems over the course of the night. When all 70 studies are considered, a total of 57 calls were needed (0.8 calls per study). Almost half (47.1%) of the NW studies required no additional calls, whereas five studies (7.1%) required three such calls. The majority of these calls occurred early in the night, with 50% occurring in the first 50 minutes of the study and 75% Variable TABLE 5. HOME-LAB data comparison (n = 70) PSG (mean ± SE) Total recording time (minutes) 450 ± 4* Total sleep time (minutes) 364 ± 10* %NREM 84 ± 1* % REM 16 ± 1* Sleep efficiency (%) 80 ± 2 Sleep latency (minutes 14 ± 3* REM latency (minutes) 120 ± 9* AHI (no.lhour) 28 ± 4* Minimum Sa0 2 (%) 81 ± 2 Mean Sa0 2 (%) 93 ± 1* PLM index (no.lhour) 3 ± I Analysis time (hours) 2.5 ± 0.3 * p < 0.05, PSG different from NW. ** p < 0.05, statistically significant relationship. within the first 2 hours. The most common problem requiring a call related to a poor signal from the respiratory effort transducer (38.6% of calls). In addition, 36.8% of the calls were a response by the laboratory technician to a failure to receive a modem transmission when expected. This generally was a product of phone use by a family member in the patient's home or a failure of the patient to properly connect the modem and never represented a transducer problem. Difficulty with the arterial oxygen saturation (15.8%) and airflow (14.0%) signals were the next most common problems. Thus, sleep was not interrupted particularly frequently due to transducer problems. As can be seen in Table 5, the recording time was somewhat longer in the home on NW than in the laboratory, as was total sleep time (Fig. 4). However, sleep efficiency was similar between the two systems, with a larger percent of REM sleep in the home on NW and NREM sleep in the laboratory (PSG). Surprisingly, the sleep latency was significantly shorter in the laboratory (PSG) than at home (NW), with opposite results for REM latency. Again, as stated above, the full explanation for these differences and the relatively low r values (Table 5) cannot be discerned from this study due to the multiple independent influences on these sleep variables. However, when the IN-LAB results are compared with the HOME-LAB study, they suggest that total sleep time is probably similar in the home (NW) and laboratory (PSG), despite a longer time in bed in the home. Thus, the sleep efficiency may be slightly lower at home. The differences in percent NREM and REM sleep and the various latencies (sleep and REM) observed on NW in the home are likely real, as there were no systematic differences in these variables when they were acquired simultaneously (Table 2). NW (mean ± SE) r Value 469 ± ± ** 78 ± I 0.29** 22 ± ** 83 ± ** 27 ± ** 90 ± ** 31 ± ** 82 ± ** 95 ± I 0.93** 22 ± ** 1.0 ± ** 95% confidence interval for r value Low High

8 122 D. P. WHITE ET AL. HOME-LAB STUDY '.. JI-"'-.. ' HOME-LAB srudy 400 Sleep Time 350 NW (min.) ' '.. '.. ' ' r= ' ' p=.oo6 150 I Sleep Time PSG (min.) FIG. 4. The relationship between total sleep time measured by NW versus PSG during the HOME-LAB study is demonstrated. The dotted lines denote the 95% confidence interval for individual observations. The relatively poor correlation between these variables can be seen. This is likely the result of several independent influences on sleep time (see text). The home NW study was a good predictor of AHI and oxygenation, as can be seen in Table 5 and in Fig. 5. Although the AHI on the PSG was slightly lower than on NW, the two correlated well (r = 0.92), as did the various measures of oxygen saturation. The PLM index was substantially and significantly higher on NW (22 ± 7) than on PSG (3 ± 1). The explanation for this is unclear, as there was little relationship between the PLM index obtained by NW and PSG when obtained simultaneously. Thus, HOME-LAB conclusions for this variable are difficult. The sensitivity and specificity ofthe NW system for the diagnosis of obstructive sleep apnea are given in Table 3. At an apnea threshold of 10, the sensitivity was 90.7%, with a total offour false negatives. In every case (each false negative) the AHI on the PSG was < 20, and thus the differences between the NW and PSG were small. The specificity at this threshold (AHI = 10) was 70.4%, with eight false positives. In five of these false positives the AHI on NW was < 20, and thus the values were close to the threshold. In the remaining three cases, substantial differences in AHI were observed with clear, frequent events being observed on NW at home, but not in the laboratory (PSG). In all three cases, the patient spent more time supine in the home (NW) than in the laboratory (PSG), which may explain these differences. At an apnea threshold of 20, the sensitivity fell to 86.2%, with four false negatives. In two cases the AHI on PSG was <25 and AHINW 80 (Events 60 per Hour) AHI PSG (Events per Hour) r=.92 p<.ooi FIG. S. The relationship between AHI measured by NW versus PSG during the HOME-LAB study is depicted. The dotted lines denote the 95% confidence interval for individual observations. A highly significant relationship for this variable was observed between evaluation systems. thus quite close to the NW value. In the other two false negatives, the explanation for differences between systems was unclear. The specificity at this threshold (AHI = 20) rose to 82.9%, with a total of seven false positives. In five of these cases, the AHI on NW was < 30 and thus close to the PSG value. In the two remaining false positives, substantially more sleep time occurred supine at home (NW) than in the laboratory (PSG), which may explain the higher AHI on NW. The level of diagnostic agreement between systems was 78.6% in the HOME-LAB study (Table 4). Therefore, in 55 of70 cases the two systems agreed regarding AHI. In 12 cases (17.1 %), the AHI was overestimated by NW. All cases were subsequently reviewed on NW and clear, unambiguous respiratory events were occurring in the home. In four cases, the higher AHI on NW could potentially be attributed to a considerably higher percentage of sleep time in the supine posture at home. In four additional cases, substantially more REM sleep was observed in the home than the laboratory, with all four individuals demonstrating a greater REM than NREM AHI. In the final four cases, no clear explanation could be discerned. In three cases (4.3%) NW underestimated the AHI as determined by PSG in the laboratory. Based on the results of the IN LAB study, this may be a product of NW overestimating sleep, thus yielding a lower AHI. As can be seen in Table 5, the analysis time was significantly less on NW than PSG (1.0 ± 0.1 vs. 2.5

9 HOME MONITOR OF SLEEP AND BREATHING 123 ± 0.1 hours, p < 0.01). Finally, each participant was asked, by questionnaire, which sleep evaluation system they preferred. Of the 63 individuals completing the questionnaire, 56% preferred NW, 17% preferred the PSG, and 27% were without a preference. DISCUSSION This study attempted to evaluate the diagnostic accuracy of the NightWatch (NW) sleep evaluation system when compared to simultaneously obtained paper PSG in the laboratory and when NW was used in the home. The results of the IN-LAB study suggest: (a) that NW tends to occasionally score wakefulness as NREM sleep, yielding slightly greater total sleep times and sleep efficiencies than PSG; (b) that there is generally good agreement between NW and PSG in quantifying apnea plus hypopnea frequency with acceptable sensitivity, specificity and diagnostic agreement between systems; and (c) that analysis time is substantially less on NW than PSG. The results of the HOME LAB study are quite similar and indicate: (a) that there are rare technical failures of the NW system in the home (2.9%), (b) that there is good agreement between systems in the diagnosis of obstructive sleep apnea and (c) that less time is required to analyze NW studies than the PSG. There are a number of inherent difficulties in trying to compare one sleep diagnostic system to another. The three most prominent problems are the lack of a true gold standard in assessing respiration during sleep and thus difficulties in detecting apneas and hypopneas, the absence of a well-accepted cutoff in apneahypopnea frequency in the diagnosis of obstructive sleep apnea, and the night-to-night variability in measures of sleep and respiration that makes home assessment versus laboratory evaluation difficult (12,18). These issues will be addressed separately as they apply to this study. Detecting apneas and hypopneas during sleep studies has always been difficult because virtually all transducers used to assess respiration are semiquantitative at best (19). As a result, the judgment and experience of the individual analyzing the record is always important. In this study, quite similar transducers (thermistors, oximeters, piezo electrodes, inductance plethysmography) were used for both NW and PSG. However, these respiratory signals were displayed quite differently, with NW utilizing a computer screen with a condensed time axis (2-minute epochs) and PSG using standard paper recording techniques with a long time axis (3D-second epochs). Whether different display methodologies such as these can lead to quantitative differences in the assessment of apneas and hypopneas has never been systematically addressed. However, it was our experience in this study that subtle apneas and hypopneas were frequently more visible and therefore more easily detected on the condensed computer screen than on paper. This was found to be the case when the records from IN-LAB study patients demonstrating considerable discrepancies in AHI between analysis systems were reexamined. Often, fairly obvious apneas or hypopneas could be detected on the computer screen but were less clearly visible on paper. This could reflect inaccurate signals on NW. However, this seems unlikely, as changes in arterial oxygen saturation, flow and effort were moving in a pattern highly consistent with obstructive apneas and hypopneas. Therefore it could be argued that the false-positive patients identified by NW are really falsenegative patients on PSG. We have certainly not presented the data in this fashion (Tables 3 and 4) because the PSG is considered the gold standard; however, this possibility should be kept in mind. The absence of a clear cutoff in the AHI by which sleep apnea can be diagnosed presents obvious problems in sensitivity and specificity determinations for the two systems. At an AHI diagnostic threshold of 20, a patient with an AHI of 21 on PSG and 19 on NW will be considered an NW false negative, whereas clinically the two results are virtually identical. As a result we developed our own methodology for the determination of "diagnostic accuracy" (Table 4), which we believe represents a better way to approach this problem, as it takes into consideration the clinical uncertainty that AHI thresholds present. However, to make our results as clear as possible, both approaches (sensitivity/specificity and diagnostic agreement) were utilized. The night-to-night variability in measures of sleep and respiration (13,18) was almost certainly a confounding variable in the HOME-LAB study, particularly because the patients were studied in two different environments. Although most studies assessing such night-to-night variability in respiratory parameters suggest relatively good agreement (13,18,20), there is commonly an 8-10% error in sleep apnea diagnosis from one night to the next when various AHI thresholds are used (13,18). Therefore, it seems likely that a substantial percentage of the false positives and false negatives encountered in the HOME-LAB study related more to night-to-night variability than any diagnostic inaccuracy in the NW system. When the three problems described above (no gold standard for respiration assessment, no AHI cutoff in apnea diagnosis and night-to-night variability) are considered, we conclude that the NW system is producing reliable, clinically interpretable data. A number of home sleep diagnostic systems are currently available. However, relatively few have been

10 124 D. P. WHITE ET AL. carefully evaluated. These vary from simple oximetry to complex systems that fully monitor sleep and respiration in the home. The oximetry studies have been highly variable, with some reporting excellent results (7) and others quite poor and inconsistent data (8,21). As a result, the role of oximetry is controversial. It would seem that a clearly positive study with continuously cycling oxygen saturation in a patient with a compatible history is highly indicative of sleep apnea. On the other hand, if the study shows no cycling pattern and the oxygen saturation is maintained above 90% throughout the night in a patient with a low pretest clinical probability of sleep apnea, then this diagnosis can be essentially excluded. However, results in between these extremes may be difficult to interpret (22). A two-channel system that monitors only oximetry and snoring (SNORESA T) has recently been described with excellent sensitivity and specificity in the laboratory (10). However, it has not been tested in the home. Therefore, this system certainly has potential. Another two-channel system (MESAM 2) monitored only heart rate and snoring in the assessment of sleep apnea and was reported to have reasonable sensitivity and specificity when tested in the laboratory (11). However, it has been subsequently upgraded to a four-channel unit. This MESAM 4 system monitors oxygen saturation, heart rate, snoring and body position. This system was evaluated in 56 patients being directly compared to simultaneous PSG (12). Although an excellent sensitivity and specificity was reported at an AHI of 10, these values were determined almost exclusively from the oximetry signal (oxygen desaturation index), with little additional information being obtained from the other three signals. In addition, this system has never been evaluated in the home. Therefore, it is unclear how well the MESAM 4 works outside the laboratory and whether it provides more useful information than oximetry alone. The Edentec four-channel sleep monitoring system, which records flow (thermistor), effort (impedance), Sa0 2 and heart rate, has been evaluated on several occasions. The first study (14) suggested excellent sensitivity and specificity for detecting an AHI > 5 when used simultaneously with PSG in the laboratory. However, no other comparisons were provided and it was not studied in the home. The second evaluation (13) again relied primarily on an in-laboratory comparison, with only five patients being studied in both the home and the laboratory. However, there was only a 9% failure rate in the home, and excellent correlations between PSG and this home monitor were reported. Once again, this system relies heavily on sleep diaries to determine sleep time, which can be erroneous. This device certainly holds promise but needs more evaluation in the home. Sleep. Vol. 18. No A Medilog four-channel system (actigraphy, leg EMG and chest-abdominal wall motion) was evaluated in 36 individuals (23). This system had a substantial failure rate in the home (33%), yet demonstrated a reasonable correlation with laboratory-determined apnea index when it did function properly. Specificity was not defined. However, arterial oxygen saturation was not measured in either the home or the laboratory and distinguishing central from obstructive apneas is likely to be quite difficult with this system. Also hypopneas were not assessed, which would probably be difficult without a flow signal. Thus, there are potential problems with this device. Finally, the Vitalog three-channel system (chest-abdominal wall motion and oxygen saturation) was evaluated in 14 patients studied simultaneously with PSG in the laboratory (15). There was a 14% failure rate in the laboratory and a relatively low positive predictive value for sleep apnea. Thus, further development and evaluation will likely be required with this device. NightWatch offers several potential advantages over the systems described above. First, sleep versus wakefulness can be reasonably well detected, with NW making the determination of AHI possible. Although NW tended to overscore sleep, the differences were generally small and thus there were only three (4.3%) underestimates of AHI (Table 4). Whether monitoring sleep is necessary or helpful in the diagnosis of sleep apnea is certainly debatable at this time. The study by Douglas et al. (16) suggested that monitoring sleep is oflittle assistance in this process. However, the Douglas study was conducted in the laboratory and not the home. As a result, a careful comparison of home monitoring methodologies with and without some measure of sleep is certainly needed. Second, NW has the ability to send portions of the recording to the sleep laboratory by modem, so data quality can be verified and problems corrected if necessary. Using this methodology led to a much lower frequency of study failure than has been encountered with other systems. There are several potential disadvantages to the NW system as well. The principal one relates to operating expense. To take advantage of the modem feature for transducer verification, a technician must be available all night to receive these transmissions and correct equipment problems should they occur. However, it was our experience that the NW transducer verification methodology was quite useful in insuring that all signals were initially of good quality and remained useful over the first few hours of sleep as occasional problems were encountered. Thereafter, the patients were rarely awakened to correct transducer problems. It could be argued, therefore, that technician services are not required all night and that after several hours the technician could either go home or work on other projects,

11 HOME MONITOR OF SLEEP AND BREATHING 125 thereby reducing cost. The fact that almost half the studies were completed without any need for modem interaction after signal quality was initially verified suggests this could frequently be accomplished. In addition, over 35% of calls to the patients were due to modem problems and thus do not indicate transducer malfunction. As a result, the NW system generally requires minimal interaction after the initial set-up, allowing technicians to direct their attention to other pursuits, thereby reducing costs. It is also certainly possible that a single technician could conduct multiple NW studies at a time, although we never attempted more than two at one time. This would certainly be the case after the initial 1-2 hours of data collection. Neither ofthese approaches, however, were evaluated in this study and, as a result, their effectiveness remains speculative. The issue of when home monitoring is appropriate and when a formal in-laboratory PSG is required in the diagnosis of obstructive sleep apnea is highly controversial. Such a decision must certainly be based on the sensitivity and specificity of the diagnostic equipment, the relative costs of the different approaches, the comfort and desires of the patient and, potentially, the availability ofthe recording systems. All these considerations must be weighed with each diagnostic system and with each patient group. How NW fits into this diagnostic algorithm for sleep apnea could be argued. NightWatch's sensitivity, specificity and diagnostic agreement with PSG seem quite reasonable when NW is used in the home. This is particularly the case when night-to-night variability in AHI is recognized. NW underestimated AHI in only three patients (4.7%, Table 4) and overestimated AHI in 12 (17.1 %, Table 4). However, some of the overestimates certainly relate to night-to-night variability and PSG false negatives. Therefore, the equipment seems to function adequately. The cost is almost certainly less than that ofa standard PSG, although not as much less as is likely to be the case with some of the other home diagnostic systems. However, NW costs could possibly be reduced further ifmore than two patients at a time were studied or the technician did not spend the entire night in the laboratory. Finally, patients seem to prefer NW in the home to PSG in the laboratory, although the differences were not huge. The diagnostic algorithm into which NW best fits in the diagnosis of sleep apnea is certainly arguable at this time. However, we would contend that symptomatic (hypersomnolent) patients with a high probability of sleep apnea could certainly be studied with NW in the home. If sleep apnea is observed, therapy could be instituted, although an additional titration night would be required if continuous positive airway pressure (CPAP) was the therapeutic choice. If sleep apnea was not discerned on the NW study, a full PSG in the laboratory would likely be necessary in the sleepy patient. However, we believe this would rarely be required. In the minimally symptomatic snorer, an NW study might also be appropriate. In this case, a positive study would certainly lead to therapy as outlined above. On the other hand, a patient with a negative study could simply be followed by the physician with no further evaluation. Therefore, any patient in whom there exists a reasonable suspicion of sleep apnea could be studied with NW. All patients diagnosed with sleep apnea by NW obviously would be treated. It is the negative study that presents a potential problem and should lead to a full PSG in symptomatic patients and careful follow-up in minimally or asymptomatic ones. Finally, it could be argued that split-night studies are less expensive than any home monitoring algorithm that requires a subsequent CPAP titration night. However, this argument assumes that the majority of patients scheduled for split-night studies will have sleep apnea. A negative (no sleep apnea) split-night study (thus a regular full-night PSG) is likely to be more expensive than a negative home study. A split-night study also may not allow adequate time for CPAP titration, therefore not discerning the ideal pressure to abolish sleep apnea. As a result, with the multiple approaches to the diagnosis and treatment of sleep apnea currently available, each laboratory will have to design their own algorithm that leads to accurate but hopefully inexpensive diagnosis and treatment of patients with sleep apnea. Two technical limitations are worthy of discussion. First, we did not meaningfully assess the NW computer analysis program. We simply allowed the technicians to go through the computer-scored record and make corrections as necessary. As a result, how well the analysis program would have functioned alone cannot be discerned in a quantitative manner. In addition, the technicians did not record how frequently analysis editing was required. However, it was the clear impression of both technicians that frequent changes in computer-scored sleep stages (awake, NREM, REM) and relatively frequent editing of respiratory events was necessary. Even with such technician interaction, NW records were scored in a mean time of only about 1 hour (Tables 2 and 5). Second, it became evident as the study progressed that the NW system does not consistently delineate brief arousals that may occur at the termination of an apnea or hypopnea. As a result, hypopneas with only arousal (no quantifiable oxygen de saturation) could be missed on NW. However, this did not seem to occur commonly, as the number of apneas and hypopneas were generally quite similar between systems. In conclusion, this study indicates that the NW sleep

12 126 D. P. WHITE ET AL. evaluation system functions well in the home, with a low failure rate, good to excellent sensitivity and moderate to good specificity considering the vagaries of diagnosing sleep apnea, improved patient acceptance over the standard PSG, and decreased technician analysis time when compared with the PSG. How such a system as NW compares to other home sleep assessment units or how it will ultimately fit into the algorithm for the detection of sleep apnea cannot be discerned from this study. Acknowledgements: The authors thank Judy Tisdale for secretarial assistance in the production of the manuscript. This study was supported by Healthdyne Technologies, Inc. REFERENCES I. Young T, Palta M, Dempsey J, Skatrud J, Weber S, Badr S. The occurrence of sleep-disordered breathing among middle-aged adults. N Engl J Med 1993;328: Roehrs T, Zorich F, Wettig R, Conway W, Roth T. Predictors of objective daytime sleepiness in patients with sleep-related breathing disorders. Chest 1989;95: Findley LJ, Unverzagt ME, Suratt PM. Automobile accidents involving patients with obstructive sleep apnea. Am Rev Respir Dis 1988; 138: Fletcher EC, DeBehnker RD, Lovoi MS, et al. Undiagnosed sleep apnea in patients with essential hypertension. Ann Int Med 1985; I 03: Shepard JW, Garrison MW, Guther DA, Dolan GF. Relationship of ventricular ectopy to oxyhemaglobin desaturation in patients with obstructive sleep apnea. Chest 1985;83: He J, Kryger M, Zorick FJ, Conway W, Roth T. Mortality and apnea index in obstructive sleep apnea. Experience in 385 male patients. Chest 1988;94: Series F, Marc I, Cormier Y, LaForge J. Utility of nocturnal home oximetry for case finding in patients with suspected sleep apnea hypopnea syndrome. Ann Int Med 1993;119: Williams AJ, Yu G, Santiago S, Stein M. Screening for sleep apnea using pulse oximetry and a clinical score. Chest 1991; 100: Hoelscher TJ, McCall WV, Powell J, Marsh GR, Erwin CWo Two methods of scoring sleep with the Oxford Medilog 9000: comparison to conventional paper scoring. Sleep 1989; 12: Issa FG, Morrison D, Hadjuk E, Iyer A, Feroah T, Remmers JE. Digital monitoring of sleep-disordered breathing using snoring sound and arterial oxygen saturation. Am Rev Respir Dis 1993;148: II. Stoohs R, Guilleminault C. Investigations of an automaticscreening device (MESAM) for obstructive sleep apnea. Eur Respir J 1990;3: Stoohs R, Guilleminault C. MESAM 4: an ambulatory device for the detection of patients at risk for obstructive sleep apnea. Chest 1992;101: Redline S, Tosteson T, Boucher MA, Millman RP. Measurement of sleep-related breathing disturbances in epidemiological studies: assessment ofthe validity and reproducibility of a portable monitoring device. Chest 1991; 100: Emsellam HA, Corson WA, Rappaport BA, Hackett S, Smith LG, Hausfeld IN. Verification of sleep apnea using a portable sleep apnea screening device. South Med J 1990;83: Gyulay S, Gould D, Sawyer B, Pond D, Mant A, Saunders N. Evaluation of a microprocessor-based portable home monitoring system to measure breathing during sleep. Sleep 1987; 10: Douglas NJ, Thomas S, Jan MA. Clinical value of polysomnography. Lancet 1992;339: Rechtschaffen A, Kales A, eds. A manual of standardized terminology and scoring system for sleep stages of human subjects. Los Angeles: Brain Information Services/Brain Research Institute, University of California at Los Angeles, Wittig RM, Romaker A, Zorick FJ, Roehrs TA, Conway WA, Roth T. Night-to-night consistency of apnea during sleep. Am Rev Respir Dis 1984; 129: Martin RJ, Block AJ, Cohn MA, et al. Indications and standards for cardiopulmonary sleep studies. Sleep 1985;8: Block AJ, Wynne JW, Boysen PG, Lindsey S, Martin C, Cantor B. Menopause, medroxyprogesterone and breathing during sleep. Chest 1981;70: Allan MA, Fitzpatrick MF, Malloy J, Douglas NJ. Correlation between oximetry and polysomnography in patients with sleep apnealhypopnea syndrome. Am Rev Respir Dis 1990; 141 :A858 (abstract). 22. Indications and standards for cardiopulmonary sleep studies. Am Rev Respir Dis 1989;139: Ancoli-Israel S, Kirpke D, Mason W, Messin S. Comparisons of home sleep recordings and polysomnograms in older adults with sleep disorders. Sleep 1981 ;4: