Performance of a New Portable Wireless Sleep Monitor

Transcription

1 pii: jc SCIENTIFIC INVESTIGATIONS Performance of a New Portable Wireless Sleep Monitor Magdy Younes, MD, FRCPC, PhD 1,2 ; Marc Soiferman, MSc 2 ; Wayne Thompson, RPSGT 1 ; Eleni Giannouli, MD, FRCPC 1 1 Sleep Disorders Centre, Winnipeg, MB, Canada; 2 YRT Ltd, Winnipeg, MB, Canada Study Objectives: To determine if signals generated by a new sleep monitor (Prodigy) are comparable to signals generated during in-laboratory polysomnography (PSG). Methods: Fifty-nine patients with various sleep disorders (25 with moderate/severe sleep apnea) were studied. Full PSG was performed using standard acquisition systems. Prodigy was attached to the forehead with four disposable snap electrodes. Four additional electrodes were attached to monitor eye movements and muscle activity, and to serve as reference (mastoid). One frontal EEG signal was outputted in real time from the monitor and stored in the PSG record along with the other PSG signals. PSG was scored for sleep variables manually, and monitor records were scored by a validated automatic system (MSS) (MSS-Prodigy). MSS-Prodigy was briefly edited following suggestions of an Editing Helper feature of MSS. Results: Technical failures resulted in one study being unusable and another with data for only 3 hours. Prodigy EEG signal stored in the PSG record was visually indistinguishable from the PSG-derived EEG signals. Important differences between manual scores and unedited MSS-Prodigy were seen in a few patients in some sleep variables (notably onset latencies and REM time). Editing Helper issued 2.1 ± 0.8 suggestions/file. Only these suggestions were pursued during editing. Intraclass correlation coefficients for manual vs. edited MSS-Prodigy were > 0.83 for all sleep variables except for stages N1 and N3 (0.57 and 0.58). Conclusions: When scored with MSS, and with only very minor editing, the monitor s results show excellent agreement with manual scoring of polysomnography data, even in patients with severe sleep disorders. Keywords: home sleep testing, ORP, odds ratio product, Prodigy Citation: Younes M, Soiferman M, Thompson W, Giannouli E. Performance of a new portable wireless sleep monitor. J Clin Sleep Med. 2017;13(2): INTRODUCTION Availability of monitors that accurately and economically evaluate quantity and quality of sleep at home can offer many advantages in the investigation and management of sleep disorders. Although several commercial devices have long been available for home monitoring of the electroencephalogram (EEG), and were used in several research studies, 1 5 their wide use in clinical sleep medicine has been hampered by the need for technologists to apply conventional EEG electrodes at home and manually score the collected records after the study. The difficulty related to the time-consuming, expensive manual scoring may have been overcome. Several automatic systems have been validated in patients with sleep disorders and are commercially available Validation was, however, performed using polysomnography (PSG) records from in-laboratory studies or from home studies that employed conventional montages. The main issue that remains, therefore, is whether adequate signals can be obtained and correctly auto-scored from simplified montages that can be self-applied at home. One of the currently available automatic scoring systems, Michele Sleep Scoring (MSS), requires only one central electrode, along with ocular and muscle electrodes, to perform comprehensive sleep staging. In a totally arms-length multicenter study agreement between its unedited sleep scoring and the average score of ten academic technologists, it was BRIEF SUMMARY Current Knowledge/Study Rationale: The current need for investigating sleep disorders is much greater than can be met by the standard in-laboratory sleep studies. We compared the results of a new portable, wireless, forehead-mounted monitor that utilizes easy-to-apply frontal electrodes with results of in-laboratory full polysomnography in 59 patients with a range of sleep disorders including severe sleep apnea. Study Impact: We show that EEG signals generated by this monitor are visually indistinguishable from polysomnography signals and, when scored with its automatic system that requires minimal guided editing, produce results in good agreement with manual scoring, and, in addition, provides a continuous index of sleep depth throughout the night (Odds-Ratio-Product). This monitor may make it possible to obtain high quality inexpensive sleep evaluation in the home. comparable to agreement between individual scorers. 9 In another in-laboratory study that included many patients with moderate and severe sleep apnea, unedited MSS also showed excellent agreement with manual scoring. 11 However, since manual editing is required before automatic scores can be accepted, we recently evaluated the editing actions taken by technologists editing the results of MSS and found that the vast majority of edits have little impact on clinically relevant summary results. 10 Furthermore, when editing was limited to areas identified by an Editing Helper feature within MSS, 245

2 Figure 1 Photos of the device. (A) The head-mounted unit (HMU) attached to the forehead showing the location of the various electrodes. (B,C) Front and back of the HMU showing the battery compartment and the snaps on the back. (D,E) Front and side views of the table-top unit (TTU). editing time was reduced from 59 ± 26 to 6 ± 7 minutes, while summary results were not affected in any clinically significant way. 10 MSS was also accurate when frontal derivations were mapped instead of central derivations. 12 Thus, we believe that MSS is likely to provide economical, accurate scoring when facial electrodes, which can be self-applied, are used. What remains to be determined is whether technically adequate signals can be obtained from a home monitor that utilizes a limited number of strictly facial disposable electrodes. A commercial portable sleep monitor that utilizes frontal electrodes currently exists, Sleep Profiler (Advanced Brain Monitoring). Although some data exist showing accuracy of its scoring algorithms when applied to conventional PSG signals in normal subjects, and some abstracts appeared describing its use in home studies, we are not aware of studies documenting accuracy of the commercial system (Sleep Profiler) when tested against PSG data. Also, Fietze et al. 20 used the auxiliary inputs of the Somnowatch (SOW; Somnomedics, Germany) to monitor a single frontal EEG channel (F4-M1) with and without monitoring of chin EMG and eye movements. The Somnowatch was placed on the patient s chest, and scoring of sleep was performed manually on the signals generated by the unit. The authors reported good agreement with manual scoring of full polysomnography in patients with sleep apnea. A new portable wireless sleep monitor was recently introduced, the Prodigy Sleep Monitor (YRT Ltd; distributed in Canada by Cerebra Health, Winnipeg). It is attached to the forehead with disposable electrodes. Additional electrodes are placed for recording eye movements and muscle activity. The data collected is scored by MSS and edited using suggestions by the Editing Helper within MSS. The objective of this study was to compare auto-scoring results of Prodigy signals with manual- and auto-scored results of full PSG records obtained simultaneously in patients with assorted pathologies. METHODS Patients Fifty-nine patients referred to the sleep center for suspected sleep disorders were studied. There were no exclusion criteria other than patients who were unwilling to participate or unable to give consent and those who belonged to a vulnerable group. Participants signed a consent form. The study was approved by the University of Manitoba Research Ethics Board. Device The device is a small plastic enclosure (Head Mounted Unit, HMU) cm that weighs 70 g (including 2 AAA batteries) (Figure 1). It is attached to the forehead by 4 disposable snap electrodes. We used 2 Neuroline S electrodes (Ambu, Copenhagen, Denmark) for the corner frontal electrodes (Figure 1A) and a Nicolet disposable electrode Duo (NT , Natus, Pleasanton, CA) for the 2 central electrodes (ground and bias). Four additional electrodes were attached to the unit through touch-proof connectors located around the perimeter of the unit (Figure 1C). We used one 10-cm Neuroline K for the left eye and cm Neuroline 10-K electrodes that were manually cut to appropriate lengths for the other 3 locations (right eye, chin, and left mastoid; Figure 1A). All signals are transmitted wirelessly at 120 Hz to a tabletop unit (TTU; Figure 1D). The TTU is powered by a 9-volt DC Medical Power Supply. It has a user interface that guides the user through start-up and shut-down. It also has 4 output ports (Figure 1E) through which 2 EEG signals and a 3-sec and 30-sec moving average of the odds-ratio-product (ORP) can be outputted in real time to be recorded, if desired, by an auxiliary recording system (e.g., a PSG system). ORP is a continuous index of sleep depth that ranges from 0 (deep sleep) to 2.5 (full wakefulness). 21 There is also a USB port (not shown) for downloading the stored sleep data. The stored data is viewed, scored (Auto or manual) and edited on the MSS viewer. The viewer is similar to others available with common PSG systems (Figure 2). Clicking an icon opens a text file with the Editing Helper recommendations for what to edit. 10 After editing, a report is generated containing numerical and graphic summaries of the findings (Figure 3). Procedure Patients were first prepared for conventional PSG. The skin where Prodigy electrodes would be attached was then cleaned with abrasive swabs. The HMU was then attached to the forehead with the 4 snap electrodes. The 2 EOG electrodes were placed diagonally from the standard EOG electrodes of the PSG. The mastoid electrode was placed below the left mastoid cup-electrode of the PSG. The chin electrode was placed close to the right chin electrode of the PSG or, for bearded patients, on the anterior border of the masseter muscle. One frontal EEG signal from the TTU was connected to the auxiliary box of the PSG system (Sandman or Embla; Natus, Pleasanton, CA) for simultaneous recording on the PSG file. Just before lights-out, HMU and TTU were turned on and the study began. The PSG signals were monitored by the 246

3 Figure 2 (A) An example of data collected by Prodigy as displayed on the Prodigy viewer. The top section shows one 30-sec epoch in NREM sleep stage 2 with an arousal in the latter half of the epoch, while the lower section is the summary of the entire study. (B) An epoch in REM sleep in the same patient. EEG = electroencephalogram, EMG = electromyogram, EOG = electro-oculogram, R = REM sleep. attending technologists on the PSG monitor as usual. The technologists had access only to the single EEG signal from Prodigy, as there was no real time display on the TTU. They did not investigate or interfere in any way with the Prodigy setup even if the signal was poor. As per our center s protocol, in patients deemed by the technologist to have moderate or severe sleep apnea the study was terminated in order to initiate continuous positive pressure breathing (CPAP); the masks used here cannot be applied while the HMU is attached. Analysis The PSG record was scored manually by a technologist on scoring duty. Eleven technologists participated in the manual scoring (5.3 ± 1.3 PSGs/technologist). The PSG record was also exported in the European data format (EDF). The EDF file was opened in the MSS viewer and sent for automatic scoring by YRT s servers, once after mapping the central EEG signals (Auto-Cs) and once after mapping the frontal signals of the PSG (Auto-Fs). Prodigy data recorded in the TTU (EDF file) was downloaded to a computer, opened in the viewer software and sent for scoring by MSS. 35 Hz low pass filter and 0.33 Hz high pass filter were applied to the EEG and EOG signals. No filters are applied to the EMG signal since this signal is processed (rectified and averaged) in the head unit before being transmitted to the TTU at 120 Hz. A report was generated for the manual and each of the 3 automatic scores. The report included average ORP in different stages (Figure 3). Because ORP is directly related to the power spectrum of the EEG, 21 similarity between ORP from Prodigy signals and the PSG signals would indicate that Prodigy EEG signals are substantially similar to those collected by the PSG system. The PSG record was inspected to visually compare the real time Prodigy EEG signal stored in the PSG record with the central and frontal EEG signals collected by the PSG system. In addition, 10 consecutive records (#34-#43) were selected at random to perform an initial comparison between ORP values obtained from the PSG signals and the Prodigy signals. This step was taken to identify any systematic difference in EEG between the two acquisition systems. This preliminary comparison indicated that whole-record ORP from Prodigy is systematically higher than that obtained after scoring the PSG s signals, resulting in relatively more stage W. Investigation of the reasons for this difference in a sample of 4 from the

4 Figure 3 Sample numerical and graphical report after automatic scoring and editing of Prodigy data. ORP during stage awake is close to 2.5 and during N3 it is close to zero. ORP during stage N2 is highly variable and ORP rises during REM periods. The ORP values reported in this patient for average ORP in N2 (0.58) and during all sleep (0.70) are typical of very good sleep. Average ORP for the whole recording is slightly elevated (0.96) because of some increase in awake time. All arousals are classified as spontaneous since the Prodigy data do not contain respiratory or movement signals unless the study included recording of these variables by an independent level 3 system. EEG = electroencephalogram, N1, N2, N3 = stages 1, 2 and 3 of NREM sleep, ORP = odds-ratio-product in different sleep stages, in all sleep time (Sleep), in wake after sleep onset (WASO) and in the total recording (All stages), REM = rapid eye movement. records (those showing a large difference in ORP) identified 2 reasons, neither of which relates to signal quality, per se: 1. Input impedance of PSG hardware (to be distinguished from electrode impedance) was found to be considerably higher than input impedance of Prodigy. Thus, relative to PSG data, Prodigy signals contained more high frequency power, resulting in a higher ORP. Additional processing was applied to the Prodigy signals to match the extra impedance of the PSG hardware. This consisted of a Fourier multiplier, the values of which were determined empirically by analyzing the differences between the impulse responses for the different systems. 2. The EEG often contains cardiac artifacts. These are removed during auto-scoring of PSG files by use of the EKG signal in such files. Without an EKG signal in Prodigy records these were not filtered out, resulting in an artificial increase in high-frequency power and overestimation of ORP. A new algorithm was developed to identify and remove cardiac artifacts in the absence of an EKG signal. The algorithm consists of scanning the point-by-point derivative of the EEG 248

5 Table 1 Patient demographics and main polysomnographic abnormalities. Disorder n Sex Age BMI A/AW Index AHI PLM Index Minimum SpO 2 (%) % SpO 2 < 90% SE OSA 38 9F/29M 50.8 ± ± ± ± ± ± ± ± 18.1 PLM 7 3F/4M 50.1 ± ± ± ± ± ± ± ± 18.2 Insomnia 6 3F/3M 53.5 ± ± ± ± ± ± ± ± 9.3 No pathology 8 5F/3M 40.1 ± ± ± ± ± ± ± ± 6.8 All 59 20F/39M 49.6 ± ± ± ± ± ± ± ± 17.0 Values presented as mean ± SD. BMI = body mass index (kg/m 2 ), A/AW index = arousal and awakening index (events/h), AHI = apnea-hypopnea index (events/h), OSA = obstructive sleep apnea, PLM = periodic limb movements (events/h), SD = standard deviation, SE = sleep efficiency (%), SpO 2 = oxygen saturation. Table 2 Editing Helper suggestions. Suggestion # Cited # File Errors Found Confirm sleep onset (always issued) Check missing REM in the following sections: 24 9 (REM is < 10% of TST, or REM onset > 3 h, or > 3 h between two REM periods) Check awake periods (awake time is > 30% of recording time) 23 2 Check N3 periods (N3 is > 25% of TST, or ORP in N3 > 0.8) 12 0 Confirm REM periods (> 3 h of REM, or REM > 30% of TST) 2 2 Confirm REM onset (REM onset latency is < 30 min) 3 2 Text in parenthesis describes conditions under which each suggestion is issued. The sections chosen for checking possible missing REM sleep are identified during scoring from epochs that just fall short of meeting REM criteria. REM = rapid eye movement sleep, TST = total sleep time, N3 = stage 3 of NREM sleep, ORP = odds ratio product. looking for sharp, brief positive transients followed, within < 35 ms by sharp negative transients (positive transients), or the opposite (negative transients). A 30-sec epoch contains cardiac artifacts if there are between 20 and 70 positive transients (extreme range of heart rate) with a median interval of 0.45 to 1.50 s (extreme range of R-R interval) and there are < 5 negative transients, or if there are negative transients meeting the same conditions with < 5 positive transients. If artifacts are confirmed, the EEG data is replaced by interpolated values between the start of the first component and the end of the second component. Correction of these 2 issues reduced the difference between whole record PSG ORP and Prodigy ORP in the 4 test files from 0.37 ± 0.17 to 0.04 ± Final agreement between ORP values obtained from the PSG and Prodigy for all files and all sleep stages is provided in Results. After correcting these 2 problems, all TTU files were scored with an MSS version that incorporated the 2 new algorithms. In addition, the text file containing Editing Helper recommendations 10 was opened. This file is generated in the viewer after the scoring is complete by software that scans the summary results and identifies potential errors of clinical importance (See Discussion). A specific suggestion is added to a text file when certain conditions 10 are met in the summary data (see Editing Helper Suggestions in Results). The scoring was reviewed on the MSS viewer by one of the authors (MY). The Editing Helper suggestions were followed and any errors found were corrected manually. No other edits were performed. This step resulted in an additional summary report (Edited Prodigy) for a total of 5 reports (Manual, Auto-Cs, Auto Fs, Unedited Prodigy, and Edited Prodigy). The following statistical comparisons were then performed: 1. Analysis of variance for repeated measures and Tukey test for multiple comparisons to identify differences between averages of manual scores and the 4 autoscores in different conventional sleep variables (see Results for a listing of these variables). 2. Intra-class correlation (ICC) for the relations between values of sleep variables obtained: (a) manually vs. those obtained with each of the 4 auto scores, (b) from Auto-Cs vs. Auto-Fs, (c) from Auto-Cs vs. unedited Prodigy, and (d) From unedited vs. edited Prodigy. 3. Intra-class correlation (ICC) for the relation between ORP values obtained from Prodigy signals vs. those obtained from PSG signals. RESULTS Patients Patients were 20 females and 39 males aged 49.6 ± 12.5 years with a body mass index of 32.6 ± 6.5 kg/m 2 (Table 1). Thirty-eight patients had obstructive apnea (OSA) (apneahypopnea index [AHI] 26.4 ± 29.5 events/h; range events/h), 7 had periodic limb movement (PLM) > 10 events/h (27.6 ± 18.0 events/h; range events/h), 6 had insomnia (sleep efficiency 61.2% ± 9.3%; range 38% to 69% with no 249

6 M Younes, M Soiferman, W Thompson, et al. Performance of a New Portable Sleep Monitor Figure 4 Comparison of electroencephalography signals (EEG) obtained by the polysomnogram (PSG) equipment (C3/M2 and F3/M2) and by Prodigy. The Prodigy signal was exported from Prodigy in real time and recorded simultaneously (with a 0.5-sec delay) in the PSG file. (A) An epoch staged awake. (B) An epoch scored stage 1. (C,D) Two epochs staged as NREM stage 2. (E) An epoch staged as NREM stage 3. Note the visual similarity between the Prodigy signal and the other 2 signals in all 4 panels. Also note the marked difference between the ORP in the 2 stage 2 panels. ORP = odds-ratio-product Visual Inspection of Prodigy EEG Signal other pathology), and 8 had normal sleep. Table 1 gives other PSG details in these 4 patient groups. Twenty-five of the OSA patients (AHI 33 ± 34 events/h) were placed on CPAP after 233 ± 86 minutes of diagnostic observation. The Prodigy EEG signal exported to the PSG record was very similar visually to the EEG signals of the PSG. Figure 4 illustrates 5 examples with different EEG patterns. Editing Helper Suggestions Technical and Patient-Related Incidents One type 1 diabetic patient developed a hypoglycemic reaction 2.5 h after lights-out and was taken to the emergency department. One of the no pathology subjects requested that the HMU be removed during a brief awake period following 4 h of excellent sleep. Otherwise, there were no complaints. In one patient, all Prodigy signals were zeroed by the TTU throughout the study because of high impedance of the ground electrode. This study was not included in the analysis. In another study the same happened after 3 hours of recording. In 4 studies the mastoid electrode was zeroed by the TTU because of high impedance. These records were scored mapping the contralateral EOG signals, instead of the mastoid, as reference for the EEG electrodes and are included in the results. Therefore, in total, technical failures resulted in one study being unusable and another with data for only 3 hours. The number of suggestions ranged from 1 (n = 12 files) to 4 (n = 4) with the average being 2.1 ± 0.8 suggestions per file. Table 2 shows the number of files in which different suggestions were issued and the number of times these resulted in actual editing. Confirm sleep onset is issued routinely (n = 58) because errors in sleep onset latency (SOL) are fairly common.10 Fifteen errors were found resulting in shifting SOL 22 ± 16 min (range 2 to 63 min). In 4 cases the early erroneous sleep onset was because the software failed to delete blinks despite the presence of robust blinkdetection algorithms (the blink pattern was atypical) (Figure 5A). This produces very high theta power in the EEG and, by consequence, a low ORP, in the sleep range.21 One case was due to erroneous REM scoring throughout the PSG (see below). In the remaining 10 PSGs some early epochs were transitional between awake and sleep (Figure 5B). In such epochs ORP is intermediate between awake and sleep levels 250

7 M Younes, M Soiferman, W Thompson, et al. Performance of a New Portable Sleep Monitor Figure 5 Examples of scoring errors identified by following the suggestions of the Editing Helper. The stages showing in the tracings are those assigned after manual editing. (A) An awake epoch wrongly scored initially as stage 2 early in the PSG resulting in erroneous sleep onset latency. The blinks were missed by the software because their configuration was outside the limits for blink detection. (B) Another epoch scored stage 2 and corrected manually to awake. In transitional epochs such as this, where ORP is between 1.0 and 2.0 the software tends score sleep if other features of sleep are identified. In this case, the software identified spindles (arrows) that were not confirmed visually. (C) A 30-sec epoch scored initially as stage 2 within a 7-min period of missed REM sleep. The rapid eye movement in this epoch (arrows) was the only one in the entire period and it was missed because the amplitude was too small in the right EOG. The Editing Helper flags period as suspected REM if they meet other REM criteria even if no REMs are identified. (D) An awake epoch scored initially as REM despite high level of EMG because the software erroneously attenuated the EMG to below REM sleep threshold. This error occurs when there is no true REM sleep in the entire PSG, which makes it difficult to select a specific EMG level as the REM threshold. EEG = electroencephalogram, EMG = chin electromyogram, EOG = electrooculogram. Figure 2 shows representative tracings from the Prodigy file (stored in the TTU) in a patient with PLM, as displayed on the MSS viewer, and Figure 3 shows the summary report generated after autoscoring for that patient. The report includes conventional sleep variables as well as average ORP values in different sleep stages, in total sleep time and in total recording time. In addition, the hypnogram, arousals, and time course of ORP are displayed graphically. The results presented below were obtained from similar reports generated from central and frontal PSG signals and from unedited and edited Prodigy signals. and hence inconclusive. The final score assigned by the software depends on other ancillary findings that favor sleep or wakefulness. At times, it errs on the side of sleep although the epoch may visually be classified as awake. Check for missing REM appeared in 24 files and resulted in finding missed REM in nine (15 ± 10 min; range 4 39 min) (e.g., Figure 5C). Check awake periods was suggested in 22 files and resulted in converting some stage W to NREM sleep in 2 files. No errors were found in N3 scoring although the Check N3 periods suggestion appeared in 12 files because of excessive N3 time.10 In 2 files, REM scoring was completely erroneous with very short REM onset latency and extensive REM time. There was in reality no REM sleep throughout the 2 studies. An error (bug) was discovered in the software that caused the EMG calibration algorithm in such cases (no REM throughout) to force all EMG values to very low levels. As a result, REM was scored whenever there were rapid eye movements and ORP was < 2.0 (Figure 5D). In these 2 files, editing resulted in conversion of all REM periods to stage W. Comparison of Sleep Variables Determined Manually and with Digital Analysis: One record was removed from analysis because, other than in periods of clear wakefulness, the EEG signals, both in the PSG and in Prodigy, never reached a convincing pattern of sleep. Visual assessment by one of the authors (MY), as well as the ORP values, indicated that EEG was in a transitional 251

8 Table 3 Average values of sleep variables with different scoring approaches. Variable Manual Auto-Cs Unedited Auto-Fs Unedited Auto-TTU Unedited Auto-TTU Edited F p Total sleep time (min) 219 ± ± 93 a 206 ± 93 a 217 ± 93 b,c 216 ± 94 b,c 7.6 < Sleep efficiency (%) 71 ± ± 18 a 67 ± 18 a 70 ± 16 b,c 70 ± 17 b,c 6.3 < Sleep onset latency (min) 20 ± ± ± ± 14 a,b,c 22 ± 17 d 4.0 < REM onset latency (min)* 175 ± ± ± ± ± NS Stage awake (min) 90 ± ± 72 a 102 ± 71 a 90 ± 60 b,c 94 ± 64 b,c 8.3 < NREM stage 1 (min) 28 ± ± 22 a 32 ± 20 a,b 32 ± 21 a,b 32 ± 20 a,b 3.2 < 0.02 NREM stage 2 (min) 148 ± ± 64 a 119 ± 59 a 130 ± 66 a,b,c 130 ± 64 a,b,c 25.6 < NREM stage 3 (min) 17 ± ± 25 a 33 ± 30 a,b 30 ± 31 a,b,c 30 ± 31 a,b,c 26.1 < Total NREM (min) 192 ± ± 78 a 183 ± 78 a,b 191 ± 84 b,c 192 ± 82 b,c 6.8 < REM sleep (min) 27 ± ± ± ± ± NS Values presented as mean ± SD. * = Measured from lights-out. Superscripts a,b,c,d = significant difference from Manual, Auto-Cs, Auto-Fs and unedited Auto-TTU, respectively by Tukey test for multiple comparisons. Auto-Cs = auto-scoring using central EEG electrodes, Auto-Fs = auto-scoring using frontal EEG electrodes, Auto-TTU = auto-scoring using Prodigy EEG signals, F = F statistic from analysis of variance for repeated measures, NREM = non-rapid eye movement, REM = rapid eye movement, SD = standard deviation. Table 4 Intraclass correlations between manual and various automatic scores of sleep variables. Comparison TST SE SOL ROL W N1 N2 N3 REM Manual vs. Cs Manual vs. Fs Manual vs. Unedited Prodigy Manual vs. Edited Prodigy Cs vs. Fs Cs vs. Unedited Prodigy Fs vs. Unedited Prodigy Prodigy Edited vs. Unedited Comparison of Manual vs. Edited Prodigy is the most relevant comparison; these results are shaded. Cs = auto-scoring using central electrodes in the polysomnogram, Fs = auto-scoring using frontal electrodes of the polysomnogram, N1, N2, N3 = stages 1, 2, 3 of NREM sleep, NREM = non-rapid eye movement, REM = rapid eye movement, ROL = REM onset latency measured from lights out, SE = sleep efficiency, SOL = sleep onset latency, TST = total sleep time, W = awake. state through most of the study and could not be scored with any confidence. There were major differences in this record between manual and Auto scoring of awake time. Table 3 shows average results (n = 57) of sleep variables reported after manual scoring and after the 4 automatic scores (Auto-Cs, Auto-Fs from the PSG, and unedited and edited Prodigy files). Scoring with the frontal electrodes of the PSG produced very similar results to scoring from central electrodes except for a 10-min increase in N3, a 5-min increase in NREM time, and a 3-min decrease in N1 time. Likewise, other than an increase in average SOL by 7 min, there were no differences between the edited and unedited Prodigy scores. Relative to manual scoring, unedited Auto-scoring from PSG electrodes produced less sleep time, primarily because of less N2 while N3 time was longer with frontal EEG scoring. Results of scoring with Prodigy data were closer to manual scoring than were the results of scoring with PSG electrodes. In fact, there were no significant differences between manual and prodigy results except for a lesser N2 time and a higher N1 and N3 times. Although there were many significant differences between the various approaches, the absolute differences were very small. Table 4 shows agreement (ICCs) between individual results obtained with manual scoring and the various automatic scores. Agreement between manual and the various auto-scores are illustrated graphically for several sleep variables in Figures 6 and 7. Agreement between manual and unedited scoring from central electrodes (top panels) was very good (ICC > 0.86) for all variables except SOL. Agreement between manual and unedited Prodigy was not as good, especially for REM duration and REM onset time. All ICCs improved substantially following the brief editing and became similar to or higher than agreement between manual and central electrode scoring. The only notable exception was a much shorter REM duration with Prodigy scoring in 2 patients (Figure 6, bottom right panel). In one case, the chin EMG in Prodigy was, inexplicably, well above baseline during the single period of REM sleep. This period was, accordingly, not identified by the Editing Helper as potential missing REM sleep. In the other case, most rapid eye movements in Prodigy recordings did not meet the detection threshold and only 6 such movements were identified during 75 minutes of manually scored REM sleep. By contrast, when scoring from PSG eye channels, the software identified 34 eye movements during manually scored REM sleep, and all REM sleep was scored correctly. 252

9 Figure 6 Scatter plots of individual results of different sleep variables obtained by manual scoring (abscissa) and the values obtained with three different automatic scores. Diagonal line is the line of identity. ICC = intraclass correlation coefficient, NREM = sum of times in all NREM stages, PSG = central electroencephalography (EEG) signals from the polysomnogram, REM = rapid eye movement, TST = total sleep time. There was excellent agreement between results of auto-scoring using central and frontal electrodes (Cs vs. Fs, Table 4). Agreement between unedited Prodigy and unedited results of scoring using the PSG electrodes (Cs and Fs) was poor for SOL, REM onset time and REM time. Otherwise agreement was very good to excellent. Comparison of Odds Ratio Product Determined from Different EEG Signals Figure 8 shows scatter plots comparing individual results of ORP when calculated for different sleep stages by automatic scoring using central, frontal or unedited Prodigy EEG signals. There was excellent agreement among the results of the 3 signals. DISCUSSION The current study has demonstrated that the new Prodigy monitor, which monitors frontal derivations using disposable electrodes, is well tolerated and provides EEG signals that are comparable to those obtained from conventional central and frontal cup electrodes during full polysomnography. Furthermore, when combined with automatic scoring by MSS and brief editing according to suggestions of MSS Editing Helper software, Prodigy can provide fast, reliable sleep staging and sleep depth evaluation. Comparison with Other Portable Sleep Monitors that Utilize Frontal Electrodes We are aware of only two such systems, the Sleep Profiler (Advanced Brain Monitoring) and the modified Somnowatch (Compumedics). 20 Like Prodigy, the Sleep profiler is mounted on the forehead and the signals are scored by a proprietary automatic system. However, it is capable of recording only one EEG channel and there is no detailed information about the performance of the commercial system vs. manually scored clinical PSGs. The modified Somnowatch is not placed on the forehead and, accordingly, requires long electrodes to reach 253

10 Figure 7 Scatter plots of individual results of different sleep variables obtained by manual scoring (abscissa) and the values obtained with three different automatic scores. Diagonal line is the line of identity. ICC = intraclass correlation coefficient, PSG = central electroencephalography (EEG) signals from the polysomnogram, ROL = onset rapid eye movement sleep measured from lights-out, SOL = sleep onset latency. the appropriate locations on the face. Furthermore, it appears to require manual scoring of the sleep signals. Prodigy EEG Signals vs. Polysomnography EEG Signals An important issue addressed here was whether, given differences in hardware and electrodes (disposable vs. cup electrodes), the EEG signals generated by Prodigy can be scored by MSS, which was developed from PSG-acquired signals. Scoring sleep by MSS is largely dependent on ORP. 21 Thus, for MSS to be compatible with Prodigy data, ORP must be similar in the two sets of signals. The left Prodigy signal that was inputted in real time, and stored, in the PSG file was visually very similar to the PSG-acquired EEG signals (e.g., Figure 4). Since ORP describes the powers in different EEG frequency ranges relative to each other, 21 which is a function of EEG pattern, we were surprised to find later that ORP of the signals stored in Prodigy was systematically higher than ORP of the PSG-stored EEG signals, including the exported Prodigy signal. In the sample files we tested for similarity in ORP, we found the difference was due to higher beta power in the stored Prodigy files. This suggested that the reason was in the input impedance of the two hardware systems. By applying a standard input to both hardware systems (PSG and Prodigy) and comparing the power spectrum of the data stored in each, we found a steeper decline in power vs. frequency plots in the PSG data, indicating higher input impedance. The difference in impedance characteristics of the two systems was compensated for by extra impedance added by software to the Prodigy signals before scoring. Thereafter, Prodigy ORP agreed very well with values obtained from the PSG signals (Figure 8). Accuracy of Automatic Sleep Scoring of Prodigy Signals Notwithstanding agreement between ORP in signals generated by the two systems (Figure 8), there remained some uncertainties as to whether the results of scoring Prodigy data with MSS would be similar to those obtained after scoring PSG data. First, MSS was developed based on central EEG patterns while Prodigy records contain frontal EEG. There are differences between frontal and central EEG patterns particularly in spindle characteristics, K-complex prominence, and delta 254

11 Figure 8 Scatter plots of individual odds-ratio-product (ORP) values measured in different sleep stages from different electroencephalography signals. Each dot is the average ORP in the indicated stage (see legend in left panel) in one patient. Large symbols are the average values of all patients for the indicated stage. Diagonal line is the line of identity. Note that average ORP (large symbols) decreases progressively as stage progress from stage awake through the different stages of NREM sleep but that there is a wide range of ORP values in different patients in any given stage except for N3. Note also that ORP in REM sleep (x symbols) is highly variable between patients with an average (open triangle) similar to that of NREM stage 1 (open square). N1, N2, N3 = stages 1, 2, 3 of NREM sleep, REM = rapid eye movement. wave amplitude. 22 These differences could possibly degrade the scoring of different stages of NREM sleep. This possibility was unlikely in view of the recent demonstration that, with the exception of a slight increase in N3, results of using PSG-acquired frontal and central EEG signals are virtually identical. 12 However, the frontal derivation used in Prodigy is different from the derivations used in the PSG in the aforementioned study 12 and the electrodes were also different. So some uncertainty remained. Second, MSS relies on several signals in the PSG to filter out artifacts from the EEG (e.g., EKG artifacts) and refine the scoring in ambiguous epochs (2.00 > ORP > 1.00). These auxiliary signals are missing from the Prodigy files. Absence of the EKG was clearly a problem and additional algorithms had to be developed to remove the EKG artifacts in the absence of an EKG signal. But the impact of other missing signals remained uncertain. Third, it was uncertain whether signals from Prodigy s EOG, chin, and mastoid electrodes would be comparable to the corresponding PSG signals given the use of disposable electrodes and a single mastoid reference. Table 3 shows that the net impact of the technical differences mentioned above was minimal with respect to average values in different sleep stages. In fact, the Prodigy results were closer to manual scoring than the automatic scores of the PSG signals (Table 3). For individual results, agreement between manual scoring and unedited Prodigy scoring was somewhat inferior to agreement between manual and unedited scoring of frontal and central signals in a few patients, particularly in scoring REM latency and REM time (Figures 5 and 6 and Table 4). Most of these discrepancies were identified by the Editing Helper and were largely corrected after very brief editing (Figures 5 and 6 and Table 4). In the final analysis, agreement between edited Prodigy scoring and manual scoring of PSG data (highlighted row, Table 4) was well within the reported level of agreement between experienced scorers for all sleep variables The ability of MSS Editing Helper to identify errors of clinical importance was established previously. 10 The current study provides further confirmation of the utility of this instrument. However, the Helper can be rendered useless economically if the person approving the auto-scoring decides to investigate the scoring accuracy outside the recommended (by the Helper) areas. Once this starts, the scorer will inevitably disagree with some of the auto-scores; scorers even modify many of their own scores if asked to edit them. 11 Once some errors/differences are found, the scorers, not knowing how many of these lie ahead, are compelled to continue modifying every score with which they disagree. If they were to look back after completing this editing, they would realize that very few clinically significant changes were made. This kind of retrospective evaluation of the value of epoch-by-epoch editing is what led to the development of the Editing Helper. The Helper is not concerned with epoch-by-epoch accuracy. Rather, it looks only for substantial errors that, from clinical experience, would affect clinical decisions. Thus, although differences exist between the manual scores and the auto-scores edited by the Helper guidance (Figures 6 and 7), and some of these differences may be true errors, these differences do not affect clinical decisions. ORP as a Measure of Sleep Depth/Quality Another advantage of this hardware/software system is that it provides the odds-ratio-product in different sleep stages and in the whole record (Figure 3). As seen in Figure 8, and as documented previously, 12,21 average and range of ORP values decrease progressively as stage moves from W to N1 to N2 to N3, with ORP in N3 being very low (0.43 ± 0.16 for PSG signals and 0.45 ± 0.17 for Prodigy signals; Figure 8). However, as reported previously, 12,21 sleep depth and ORP vary considerably among patients within stage 2 (Figure 8; also compare Figure 4C and 4D). These differences are not captured by Rechtschaffen and Kales classification. Average ORP over the entire study is the time-weighted average of ORP in different 255

12 stages. Accordingly, it reflects the distribution of different NREM stages as well as the depth of sleep in stage 2. Thus, by providing ORP data within individual stages and in the whole PSG, Prodigy makes available information about sleep depth/ quality that is not captured by conventional scoring. Potential Applications 1. To supplement level 3 studies for home diagnosis of sleep apnea: Addition of sleep data would considerably enhance the value of these studies 12 : (A) Availability of TST provides a more accurate denominator for the respiratory disturbance index. (B) The impact of sleep apnea on sleep quality would be identified. Daytime symptoms (fatigue, excessive somnolence) in the absence of frequent desaturation may thus be explained. Without such information the study would otherwise be considered negative or inconclusive, requiring further investigation (e.g., level 1 study). (C) Sleep information may identify co-existing sleep pathology (e.g. insomnia, poor sleep quality unrelated to OSA) that may be the main reason for the patient s symptoms. (D) Number of technically inadequate level 3 studies would be reduced To investigate sleep disorders unrelated to OSA: Insomnia and non-restorative sleep are very prevalent. 32 Poor sleep is a risk factor for many disorders including diabetes mellitus, 33,34 mood disorders, 35,36 weight gain, 37,38 cognitive disorders, 39,40 and overall mortality. 41,42 Investigation of these disorders requires sleep monitoring. The limited resources and cost of in-laboratory tests are major deterrents and the cause of the patient s symptoms remains unknown in most patients. A simple, reliable inexpensive home sleep monitor would make it possible to perform repeated home studies to identify possible associations with poor sleep. In addition, it would be possible to distinguish between patients who have excessive awake time but have good sleep quality (low ORP) when they are asleep and others in whom sleep depth is poor through most of the study. The former type is likely to reflect life style causes whereas the latter would point to an organic cause or abnormality in sleep regulation Relatively inexpensive home studies can be repeated frequently to determine efficacy of mechanical (e.g., CPAP or mandibular devices) or pharmacological (hypnotics, antidepressants) interventions on sleep quality. 4. Multiple Sleep Latency Tests can be performed at home. 5. The real time ORP in Prodigy may be useful in monitoring depth of sleep in intensive care units and optimal adjustment of sedative administration. Limitations 1. Ease of self-application and failure rate after selfapplication of the device have not been formally evaluated. Informal testing identified a number of programming errors that resulted in signal failure in a number of studies. These have been corrected. It should be noted that with the proposed montage (Figure 1) there is redundancy in EEG monitoring. For EEG to be unscorable, mastoid reference must fail together with failure of both frontal or both EOG electrodes. MSS requires only one valid EEG derivation. The mastoid electrode alone is adequate since it reports activity in the ground electrode on the forehead. When the mastoid fails either frontal electrode referenced to either EOG electrode produces a signal that can be scored by MSS. 2. The system was not tested in patients with REM behavior disorder, seizures and other neurological disorders. It is also possible that artifacts not encountered in the current study may occur and degrade its performance. Further studies are needed to evaluate its performance in a larger more diverse population. CONCLUSIONS The current results indicate that, after adjustments to match system impedance to that of commercial PSG equipment and including an algorithm to remove the cardiac artifact in the absence of an EKG signal, the portable, wireless Prodigy sleep monitor, which utilizes disposable forehead electrodes, generates EEG signals that are virtually indistinguishable, visually and in spectral features, from signals generated during full polysomnography with cup electrodes placed on central and frontal locations. Furthermore, when Prodigy signals are scored by MSS and briefly edited using only the suggestions of the Editing Helper feature of MSS, scoring of sleep shows excellent agreement with manual scoring of the full PSG. ABBREVIATIONS AHI, apnea-hypopnea index Auto-Cs, automatic scoring of central EEG signals in the polysomnogram Auto-Fs, automatic scoring of frontal EEG signals in the polysomnogram CPAP, continuous positive airway pressure EDF, European data format EEG, electroencephalogram EKG, electrocardiogram EMG, electromyogram EOG, electrooculogram HMU, head-mounted unit ICC, intraclass correlation coefficient MSS, Michele Sleep Scoring N1, stage 1 of NREM sleep N2, stage 2 of NREM sleep N3, stage 3 of NREM sleep NREM, non-rapid eye movement ORP, odds ratio product 256

13 OSA, obstructive sleep apnea PLM, periodic limb movement PSG, polysomnography REM, rapid eye movement SOL, sleep onset latency Stage W, stage awake TST, total sleep time TTU, table-top unit YRT, Younes Respiratory Technologies REFERENCES 1. Young T, Shahar E, Nieto FJ, et al. Predictors of sleep disordered breathing in community-dwelling adults: the Sleep Heart Health Study. Arch Intern Med. 2002;162(8): Young T, Palta M, Dempsey J, Skatrud J, Weber S, Badr S. The occurrence of sleep-disordered breathing among middle-aged adults. N Engl J Med. 1993;328(17): Spira AP, Blackwell T, Stone KL, et al. Sleep-disordered breathing and cognition in older women. J Am Geriatr Soc. 2008;56(1): Foster GD, Sanders MH, Millman R, et al. Obstructive sleep apnea among obese patients with type 2 diabetes. Diabetes Care. 2009;32(6): Kuna ST, Benca R, Kushida CA, et al. Agreement in computer-assisted manual scoring of polysomnograms across sleep centers. Sleep. 2013;36(4): Pittman SD, MacDonald MM, Fogel RB, et al. Assessment of automated scoring of polysomnographic recordings in a population with suspected sleepdisordered breathing. Sleep. 2004;27(7): Anderer P, Gruber G, Parapatics S, et al. An E-health solution for automatic sleep classification according to Rechtschaffen and Kales: validation study of the Somnolyzer 24 x 7 utilizing the Siesta database. Neuropsychobiology. 2005;51(3): Punjabi NM, Shifa N, Doffner G, Patil S, Pien G, Aurora RN. Computerassisted automated scoring of polysomnograms using the Somnolyzer system. Sleep. 2015;38(10): Malhotra A, Younes M, Kuna ST, et al. Performance of an automated polysomnography scoring system versus computer-assisted manual scoring. Sleep. 2013;36(4): Younes M, Thompson W, Leslie C, Egan T, Giannouli E. Utility of technologist editing of polysomnography scoring performed by a validated automatic System. Ann Am Thorac Soc. 2015;12(8): Younes M, Raneri J, Hanly P. Staging sleep in polysomnograms: analysis of inter-scorer variability. J Clin Sleep Med. 2016;12: Younes M, Younes M, Giannouli E. Accuracy of automatic polysomnography scoring using frontal electrodes. J Clin Sleep Med. 2015;12(5): Levendowski DJ, Popovic D, Berka C, Westbrook PR. Retrospective crossvalidation of automated sleep staging using electroocular recording in patients with and without sleep disordered breathing. Int Arch Med. 2012;5(1): Stepnowsky C, Levendowski D, Popovic D, Ayappa I, Rapoport DM. Scoring accuracy of automated sleep staging from a bipolar electroocular recording compared to manual scoring by multiple raters. Sleep Med. 2013;14(11): Popovic D, Khoo M, Westbrook P. Automatic scoring of sleep stages and cortical arousals using two electrodes on the forehead: validation in healthy adults. J Sleep Res. 2014;23(2): Westbrook P, Levendowski D, Cetel M, et al. Night to night variability in sleep architecture and continuity in patients evaluated for chronic insomnia. Sleep. 2014;37(Abstract Suppl):A Levendowski D, Cetel M, Rosenberg R, et al. Night to night variability in subjective and objective sleep quality metrics in patients evaluated for chronic insomnia. Sleep. 2014;37(Abstract Suppl):A Cetel M, Rosenberg R, Levendowski D, et al. Comparison of objective and subjective measures of awakenings in patients evaluated for chronic insomnia. Sleep. 2014;37(Abstract Suppl):A Levendowski D, Cetel M, Rosenberg R, et al. Differences in heart rate variability during REM and NREM sleep, a biomarker for depression? Sleep. 2014;37(Abstract Suppl):A Fietze I, Penzel T, Partinen M, et al. Actigraphy combined with EEG compared to polysomnography in sleep apnea patients. Physiol Meas. 2015;36(3): Younes M, Ostrowski M, Soiferman M, et al. Odds ratio product of sleep EEG as a continuous measure of sleep state. Sleep. 2015;38(4): Silber MH, Ancoli-Israel S, Bonnet MH, et al. The visual scoring of sleep in adults. J Clin Sleep Med. 2007;3(2): Norman RG, Pal I, Stewart C, Walsleben JA, Rapoport DM. Interobserver agreement among sleep scorers from different centers in a large dataset. Sleep. 2000;23(7): Collop NA. Scoring variability between polysomnography technologists in different sleep laboratories. Sleep Med. 2002;3(1): Danker-Hopfe H, Kunz D, Gruber G, et al. Interrater reliability between scorers from eight European sleep laboratories in subjects with different sleep disorders. J Sleep Res. 2004;13(1): Pittman SD, MacDonald MM, Fogel RB, et al. Assessment of automated scoring of polysomnographic recordings in a population with suspected sleepdisordered breathing. Sleep. 2004;27(7): Magalang UJ, Chen NH, Cistulli PA, et al. Agreement in the scoring of respiratory events and sleep among international sleep centers. Sleep. 2013;36(4): Malhotra A, Younes M, Kuna ST, et al. Performance of an automated polysomnography scoring system versus computer-assisted manual scoring. Sleep. 2013;36(4): Zhang X, Dong X, Kantelhardt JW, et al. Process and outcome for international reliability in sleep scoring. Sleep Breath. 2015;19(1): Rosenberg RS, Van Hout S. The American Academy of Sleep Medicine inter-scorer reliability program: sleep stage scoring. J Clin Sleep Med. 2013;9(1): Berry RB, Brooks R, Gemaldo CE, Harding SM, Marcus CL, Vaughn BV for the American Academy of Sleep Medicine. The AASM Manual for the Scoring of Sleep and Associated Events: Rules, Terminology and Technical Specifications. Version 2.0. Darien, IL: American Academy of Sleep Medicine; Ohayon M, Reynolds CF III. Epidemiological and clinical relevance of insomnia diagnoses algorithms according to the DSM-IV and the International Classification of Sleep Disorders (ICSD). Sleep Med. 2009;10(9): Nilsson PM, Röst M, Engström G, et al. Incidence of diabetes in middle-aged men is related to sleep disturbances. Diabetes Care. 2004;27(10): Mallon L, Broman JE, Hetta J. High incidence of diabetes in men with sleep complaints or short sleep duration: a 12-year follow-up study of a middle-aged population. Diabetes Care. 2005;28(11): Riemann D, Voderholzer U. Primary insomnia: a risk factor to develop depression? J Affect Disord. 2003;76(1-3): Baglioni C, Spiegelhalder K, Lombardo C. et al. Sleep and emotions: focus on insomnia. Sleep Med Rev. 2010;14(4): Patel S, Hu FB. Short sleep duration and weight gain: a systematic review. Obesity. 2008;16(3): Spiegel K, Tasali E, Leproult R, et al. Effects of poor and short sleep on glucose metabolism and obesity risk. Nat Rev Endocr. 2009;5(5): Altena A, van der Werf YD, Strijers RL, et al. Sleep loss affects vigilance: effects of chronic insomnia and sleep therapy. J Sleep Res. 2008;17(3): Nissen C, Kloepfer C, Nofzinger EA, et al. Impaired sleep-related memory consolidation in primary insomnia. Sleep. 2006;29(8): Gallicchio L, Kalesan B. Sleep duration and mortality: a systematic review and meta-analysis. J Sleep Res. 2009;18(2): Cappuccio FP, D Elia L, Strazzullo P, Miller MA. Sleep duration and all-cause mortality: a systematic review and meta-analysis of prospective studies. Sleep. 2010;33(5): Younes M, Hanly PJ. Immediate postarousal sleep dynamics: an important determinant of sleep stability in obstructive sleep apnea. J Appl Physiol. 2016;120(7):