Can an automated sleep detection algorithm for waist worn accelerometry replace sleep logs? Journal: Manuscript ID apnm-2017-0860.r1 Manuscript Type: Article Date Submitted by the Author: 17-Mar-2018 Complete List of Authors: Barreira, Tiago; Syracuse University, Exercise Science Redmond, Jessica; Utica College Brutsaert, Tom; Syracuse University, Exercise Science Schuna, John; Oregon State University, School of Biological and Population Health Sciences Mire, Emily; Pennington Biomedical Research Center, Population Science Katzmarzyk, Peter; Pennington Biomedical Research Center Tudor-Locke, Catrine; University of Massachusetts Keyword: accelerometer, objectively measured, nocturnal Is the invited manuscript for consideration in a Special Issue? : N/A
Page 1 of 18 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 Title Can an automated sleep detection algorithm for waist worn accelerometry replace sleep logs? Tiago V. Barreira a, Jessica G. Redmond b, Tom D. Brutsaert a, John M. Schuna Jr. c, Emily F. Mire d, Peter T. Katzmarzyk d, Catrine Tudor-Locke e a School of Education, Syracuse University, 820 Comstock Ave, Syracuse, New York, 13244, USA b Biology Department, Utica College, 1600 Burrstone Rd, Utica, New York, 13505, USA c College of Public Health and Human Sciences, Oregon State University, 1500 SW Jefferson St., Corvallis, Oregon, 97331, USA d Pennington Biomedical Research Center, 6400 Perkins Rd, Baton Rouge, Louisiana, 70808, USA e Department of Kinesiology, University of Massachusetts Amherst, 30 Eastman Ln, Amherst, Massachusetts, 01002, USA Corresponding Author: Tiago Barreira, PhD, Assistant Professor, Exercise Science Department, Syracuse University, 820 Comstock Ave, Syracuse, NY 13210 Phone (315) 443-5588 Email tvbarrei@syr.edu 35 36 37 38
Page 2 of 18 2 39 40 41 42 43 44 45 46 47 Abstract The purpose of this study was to test whether estimates of bedtime, wake time, and sleep period time (SPT) were comparable between an automated algorithm (ALG) applied to waist-worn accelerometry data and a sleep log (LOG), in an adult sample. A total of 104 participants were asked to log evening bedtime and morning wake time and wear an ActiGraph wgt3x-bt accelerometer at their waist 24 h/days for 7 consecutive days. Mean difference (MD) and mean absolute difference (MAD) were computed. Pearson correlations and dependent sample t-tests were used to compare LOG-based and ALG-based sleep variables. Effect sizes were calculated for variables with 48 49 significant mean differences. A total of 84 participants provided 2+ days of valid accelerometer and LOG data for a total of 368 days. There was no mean difference 50 51 52 53 54 55 56 57 58 59 (p=.47) between LOG 472±59 min and ALG SPT 475±66 min (MAD=31±23 min, r=.81). There was no significant mean difference between bedtime (11:48 pm and 11:53 pm for LOG and ALG, respectively, p=.14, MAD=25±21 min, r=.92). However, there was a significant mean difference between LOG (7:41 am) and ALG (7:49 am) wake times (p=.01, d=0.11, MAD=24±21 min, r=.92). The LOG and ALG data were highly correlated and relatively small differences were present. The significant mean difference in wake time might not be practically meaningful (Cohen s d=0.11) making the ALG useful for sample estimates. MAD, which gives a better estimate of the expected differences at the individual level, also demonstrated good evidence supporting ALG individual estimates. 60 Key words accelerometer; objectively measured; nocturnal; 61
Page 3 of 18 3 62 63 64 65 66 67 68 69 70 Introduction Accelerometers were used to estimate sleep (Kripke et al. 1978; Sadeh and Acebo 2002; Webster et al. 1982) prior to their introduction for physical activity (PA) measurement (Freedson et al. 1998; Janz 1994; Melanson and Freedson 1995); however, accelerometry-based sleep and physical activity measurement have historically been performed in independently of each another. Until recently, research protocols typically required participants to wear the devices exclusively during the nighttime to estimate sleep or during wake time to estimate PA. However, this recently changed as the National Health and Nutrition Examination Survey (NHANES) adopted a 71 72 24 h/day accelerometer monitoring protocol (Troiano et al. 2014). In response, other researchers have also moved to 24 h/day accelerometer-based data collection. This 73 74 75 76 77 78 79 80 81 82 83 84 shift was prompted for 2 primary reasons: 1) the belief that participants would wear the devices for more hours during the day (Troiano et al. 2014), and 2) to study the influence of both sleep and PA on health parameters. Therefore, we chose to use a 24h/day protocol with an accelerometer attached at the waist for the International Study of Childhood Obesity Lifestyle and the Environment (ISCOLE) (Chaput et al. 2015; Katzmarzyk et al. 2013; Katzmarzyk et al. 2015). The waist attachment was chosen because PA was our focus and at the time there was no consensus on the methodology to identify PA intensity using wrist-worn devices. The 24 h/day waist protocol we employed was helpful in increasing wear time compliance (Tudor-Locke et al. 2015). However, there was no methodology available to automatically identify sleep from a waist-worn accelerometer when worn 24 h/day, so we created an automated algorithm (ALG) to identify sleep with waist-worn accelerometry and tested its accuracy (Barreira
Page 4 of 18 4 85 86 87 88 89 90 91 92 93 et al. 2015; Tudor-Locke et al. 2014). The correct identification of sleep is important for the further identification and separation of sedentary time and non-wear because those can have similar acceleration signals. While we developed and tested the ALG accuracy with children aged 9-11 years (Barreira et al. 2015), other researchers (McVeigh et al. 2016) have tested the accuracy of our initial ALG (Tudor-Locke et al. 2014) but to our knowledge, the final version of the ALG (Barreira et al. 2015) has not been tested in other samples. Thus, the purpose of this study was to test whether estimates of bedtime, wake time, and sleep period time (SPT) were comparable between an automated algorithm (ALG) applied to waist-worn accelerometry data and a sleep log 94 95 (LOG), in an adult sample. Material and Methods 96 97 98 99 100 101 102 103 104 105 106 107 As part of a larger study, 104 participants were invited to take part in this additional data collection. Written informed consent was obtained from each participant and the Syracuse University Institutional Review Board approved the study protocol and procedures prior to any recruitment. Each participant s height and body mass were measured using a portable stadiometer (Seca 213; Seca GmbH & Co. KG., Hamburg, Germany) and a digital scale (SC-240; Tanita Corporation, Tokyo, Japan), respectively. Body mass index (BMI) was calculated as kg/m 2. Participant age was self-reported. Accelerometer data were collected using the ActiGraph wgt3x-bt (ActiGraph, LLC, Pensacola, FL), which is a triaxial accelerometer capable of collecting raw acceleration data at sampling frequencies up to 100 Hz over a dynamic range of ±8g. To initialize and download data from the wgt3x-bt ActiLife software (ActiGraph LLC) was used. It
Page 5 of 18 5 108 109 110 111 112 113 114 115 116 is possible to output a variety of data from ActiLife software, including raw acceleration and activity counts for each axis, vector magnitude from all three axes, steps, inclinometer position ( off, lying, sitting, or standing ), and lux for different epochs. For this study, data were collected at 80 Hz and processed in 60 sec epochs using the low-frequency extension filter to increase sensitivity of detecting low-magnitude accelerations, as suggested by Hjorth et al. (2012) Identical to our previous work (Barreira et al. 2015; Tudor-Locke et al. 2014), participants were asked to wear an ActiGraph wgt3x-bt accelerometer at their waist 24 h/days for 7 consecutive days, unless coming into contact with water (swimming, bathing, etc.). 117 118 For the LOG, participants were asked to write down the time they went to bed (bedtime) each night and the time they got out of bed (wake time) each morning for the same days 119 120 121 122 123 124 125 126 127 128 129 130 of accelerometer monitoring. Participants were also asked to record details about any night wakening episodes and/or accelerometer removal. LOG was used to obtain self-reported bedtime and wake time. LOG sleep period time (SPT) was calculated as the elapsed time between self-reported bedtime and wake time. We implemented our previously published (Barreira et al. 2015) and freely available algorithm (SAS syntax available at http://www.pbrc.edu/pdf/pbrcsleepepisodetimemacrocode.pdf) using SAS 9.4 without any modifications. Briefly, the ALG begins by using the Sadeh algorithm (Sadeh et al. 1994) to classify each minute as wake or sleep, followed by changing any wake minute to sleep if the accelerometer was in the off position. Once each minute was classified, a set of rules were applied to identify bedtime and wake time. SPT was
Page 6 of 18 6 131 132 133 134 135 136 137 138 139 calculated as the elapsed between bedtime and wake time. Although multiple SPTs were possible, no instance of multiples were included in the final analysis. Lastly, the SPT was examined for non-wear and if more than 90% of the minutes included in the SPT were classified as non-wear, the whole SPT was considered invalid and reclassified as non-wear. We analyzed the data two ways: 1) complete data and 2) screened data. For the complete data we used all 466 days averaged between 95 participants. For the screened data, we paired all the valid LOG and ALG nights (466) and flagged every difference of 150+ minutes for bedtime, wake time, and SPT for closer examination. In 140 141 total, 62 bed times, 32 wake times, and 83 SPT were examined. The examination included visual inspection of the 60 sec epoch accelerometer data and the LOG 142 143 144 145 146 147 148 149 150 151 152 153 information. In most instances, a large SPT difference was indicative of a large difference in bed or wake time. If it was clear that the participant had made a mistake in LOG data entry all variables associated with that night were excluded from the analyses. For example, as shown in Figure 1, the ALG estimated wake time at 4:07 am while the participant log entry had a wake time at 7:03 am. Looking at steps/min, it is clear that the participant got out of bed close to 4:10 am and even went for a 30 min run starting at 4:52 am (additional examples provided supplementary figures S1 and S2 figures). In total 98 nights were excluded from the analysis, 77 were deleted due to dissimilar SPT, 7 due to different bed times, 3 due to different wake times, 1 due to different bed and wake times, and an additional 10 nights were excluded because of inconsistences in most entries by the participant or just only a single valid day remained for a participant once all exclusions were made. If the difference was not due to a clear
Page 7 of 18 7 154 155 156 157 158 159 160 161 162 mistake in log entry, data were retained (see Figure 2 for example). In this case, it appears again that the ALG has a better estimate of bedtime but it is possible that the participant went to bed around the indicated time in the LOG, had some restless sleep and briefly arose (possibly to go to the bathroom). In addition, participants data were only included if they had 2+ days of valid data. In total 368 days were included. Participants data were averaged across valid days, providing just one value per participant for each of the variables described above. In total, 85 participants had valid data. Similar to our previous publications (Barreira et al. 2015; Tudor-Locke et al. 2014), a number of statistical procedures were used to compare the multiple outputs by the 2 163 164 measurement tools. Pearson product moment correlations were calculated to assess the magnitude of associations, and previously published standards (Safrit and Wood 165 166 167 168 169 170 171 172 173 174 175 176 1995) were used to classify the associations. MD and MAD were calculated to determine differences between methods. Paired t-tests were used to compare mean LOG SPT and mean ALG SPT. Cohen s d effect size was computed when significant mean differences were present. In addition, Bland Altman plots were prepared. The same calculations were conducted to compare LOG bedtime and wake time to ALG variables. All statistical analyses were conducted using IBM SPSS Statistics (version 24; SPSS, Chicago, IL), and the level of significance was defined as P <.05. Results In total for the screened data, 84 participants (75% females, BMI 23.6±3.8 kg/m 2, age 23.8±6.0 years), provided 2+ days of valid accelerometer and LOG data for a total of 368 days. The results from both the screened and completed data are presented in Tables 1 and 2. For the screened data, there was no mean difference (t(83)=0.73,
Page 8 of 18 8 177 178 179 180 181 182 183 184 185 p=.47) between LOG SPT min and ALG SPT.. The Bland Altman plot is presented in Figure 3. Only three data points were outside the 95% limits of agreement (±1.96 SD). There was no significant mean difference for bedtime However, there was a small (Cohen s d=0.11) but significant mean difference in wake time (t(83)=2.51, p=.01). For the complete data set, there were no significant mean differences (all p>.18) in any of the comparisons. However, absolute differences were 2-3 times as large as the absolute differences in the screened data, 25±21 vs 63±55, 24±21 vs 45±33, and 31±23 vs 85±63 for bed time, wake time, and SPT from screened and complete data respectively. 186 187 Discussion We compared estimates of bedtime, wake time, and SPT between a LOG and the 188 189 190 191 192 193 194 195 196 197 automated ALG and believe that waist worn accelerometry is a good alternative to LOG for estimation of those variables. LOG and ALG data were highly correlated and relatively small differences were present. Although a significant between-method mean difference of 8 min was found for wake time, this difference might not be meaningful in practice due to its small effect size. MAD, which gives a better estimate of the expected differences at the individual level, provided good validity evidence for the ALG use of individuals estimates. It is also important to note that during visual inspection of the data that showed large discrepancies between LOG and ALG, 20% of the LOG data were deemed erroneous. We presented both screened and complete data results but we believe that this clearly demonstrates one of the LOG limitations. 198 199 When compared to the results in our previous study with children (Barreira et al. 2015), the results in this study were positive. Overall, the correlations were higher and
Page 9 of 18 9 200 201 202 203 204 differences were smaller. We showed a smaller MD (3 vs 20 min) but a similar MAD (31 vs 34 min) for SPT. The estimated LOG and ALG bedtimes were over 2 h later in this sample, and the wake time was 1 h later leading to a 1 h difference in SPT between the 2 samples. This adds to the validity of the ALG because young adults are known to sleep for shorter durations than children (Hayley et al. 2015). 205 206 207 208 Due to the limited research on the implementation of automated algorithms to detect sleep related estimates, comparison to other studies is difficult. To our knowledge, McVeigh et al. (2016) is the only study that has evaluated the accuracy of automated sleep algorithms in adults. However, in their study (McVeigh et al. 2016) the 209 210 comparisons were made against visual inspection of the data and researcher determined in bed and out-of-bed times and not a comparison to a LOG like in this 211 212 213 214 215 study. Due to the different data analysis between the studies, direct comparison is difficult; however in this study, we showed tighter limits of agreement than McVeigh et al.(2016) For example, SPT 95% limits of agreement in our study was -78-72 min while in-bed wear for McVeigh et al. was -309-228 min. This is likely caused by the different methods used to test the accuracy of the algorithm, visual inspection vs LOG. 216 217 218 219 220 221 222 As previously mentioned, there is a general belief that 24 h/day protocols can improve wear compliance, however there is a lack of data to either support or refute this notion. In our previous study with children, the 24 h/day waist accelerometry protocol was helpful in increasing wear time compliance (Tudor-Locke et al. 2015). Although it was not the purpose of this study, we did compute wake wear time. Once SPT was extracted and non-wear was estimated (Troiano et al. 2008), wake wear was estimated at 15.5±1.3 h/day. That was 1.3 h/day higher than the US representative sample from
Page 10 of 18 10 223 224 NHANES (Troiano et al. 2008). This suggests that the 24 h/day wear protocol has the potential to improve wear time. 225 226 227 228 229 230 231 Although carefully conducted, this study had some limitations. Approximately 20% of the LOG-ALG pairing days were not included in the screened analysis because their estimates differed by more than 150 min. However, the pairings were only excluded when it was clear that the LOG information was not plausible based on careful examination of the accelerometer data. Additional contact with participants to check LOG information could have been helpful in improving the quality of the LOGs and overall retention rate of data. The accuracy of the ALG was tested against a LOG and 232 233 not polysomnography which is the gold standard for sleep assessment. However, the intention of the ALG is to replace the LOG and make it easier for researchers 234 235 236 conducting large scale/epidemiological investigations where both sleep and physical activity are behaviors of interest. The ALG is not intended to replace clinical measures of sleep duration and sleep quality. 237 238 239 240 In conclusion, we found the ALG to be a valid substitute for a sleep LOG. Using a waistworn accelerometer to estimate bedtime, wake time, and sleep period time has the potential to increase wear compliance, can lead to high wake wear time, and decrease the burden to the researcher and participant by eliminating the use of the sleep LOG. 241 242 243 244
Page 11 of 18 11 245 246 247 248 249 250 251 252 253 Conflict of Interest The authors have no conflicts of interest to report. References Barreira, T.V., Schuna, J.M., Jr., Mire, E.F., Katzmarzyk, P.T., Chaput, J.P., Leduc, G. et al. 2015. Identifying children's nocturnal sleep using 24-h waist accelerometry. Med. Sci. Sports Exerc. 47(5): 937-943. doi:10.1249/mss.0000000000000486. Chaput, J.P., Katzmarzyk, P.T., LeBlanc, A.G., Tremblay, M.S., Barreira, T.V., Broyles, S.T. et al. 2015. Associations between sleep patterns and lifestyle behaviors in children: an international comparison. Int. J. Obes. Suppl. 5(Suppl 2): S59-65. 254 255 doi:10.1038/ijosup.2015.21. PMID:Pmc4850622 Freedson, P.S., Melanson, E., and Sirard, J. 1998. Calibration of the Computer Science 256 257 258 259 260 261 262 263 264 265 and Applications, Inc. accelerometer. Med. Sci. Sports Exerc. 30(5): 777-781. Hayley, A.C., Skogen, J.C., Øverland, S., Wold, B., Williams, L.J., Kennedy, G.A. et al. 2015. Trajectories and stability of self-reported short sleep duration from adolescence to adulthood. J. Sleep Res. 24(6): 621-628. doi:10.1111/jsr.12316. Hjorth, M.F., Chaput, J.-P., Damsgaard, C.T., Dalskov, S.-M., Michaelsen, K.F., Tetens, I. et al. 2012. Measure of sleep and physical activity by a single accelerometer: Can a waist-worn Actigraph adequately measure sleep in children? Sleep Biol. Rhythms, 10(4): 328-335. doi:10.1111/j.1479-8425.2012.00578.x. Janz, K.F. 1994. Validation of the CSA accelerometer for assessing children's physical activity. Med. Sci. Sports Exerc. 26(3): 369-375.
Page 12 of 18 12 266 267 268 269 270 271 272 273 274 Katzmarzyk, P.T., Barreira, T.V., Broyles, S.T., Champagne, C.M., Chaput, J.-P., Fogelholm, M. et al. 2013. The International Study of Childhood Obesity, Lifestyle and the Environment (ISCOLE): Design and methods. BMC Public Health, 13(1): 900. Katzmarzyk, P.T., Barreira, T.V., Broyles, S.T., Champagne, C.M., Chaput, J.P., Fogelholm, M. et al. 2015. Physical activity, sedentary time, and obesity in an International sample of children. Med. Sci. Sports Exerc. 47(10): 2062-2069. doi:10.1249/mss.0000000000000649. Kripke, D.F., Mullaney, D.J., Messin, S., and Wyborney, V.G. 1978. Wrist actigraphic measures of sleep and rhythms. Electroencephalogr Clin. Neurophysiol. 44(5): 674-676. 275 276 McVeigh, J.A., Winkler, E.A., Healy, G.N., Slater, J., Eastwood, P.R., and Straker, L.M. 2016. Validity of an automated algorithm to identify waking and in-bed wear time in hip- 277 278 279 280 281 282 283 284 285 286 worn accelerometer data collected with a 24 h wear protocol in young adults. Physiol. Meas. 37(10): 1636-1652. doi:10.1088/0967-3334/37/10/1636. Melanson, E.L., Jr., and Freedson, P.S. 1995. Validity of the Computer Science and Applications, Inc. (CSA) activity monitor. Med. Sci. Sports Exerc. 27(6): 934-940. Sadeh, A., and Acebo, C. 2002. The role of actigraphy in sleep medicine. Sleep Med. Rev. 6(2): 113-124. Sadeh, A., Sharkey, K.M., and Carskadon, M.A. 1994. Activity-based sleep-wake identification: an empirical test of methodological issues. Sleep. 17(3): 201-207. Safrit, M.J., and Wood, T.M. 1995. Introduction to measurement in physical education and exercise science. St. Louis: Mosby.
Page 13 of 18 13 287 288 289 290 291 292 293 294 295 Troiano, R.P., Berrigan, D., Dodd, K.W., Masse, L.C., Tilert, T., and McDowell, M. 2008. Physical activity in the United States measured by accelerometer. Med. Sci. Sports Exerc. 40(1): 181-188. doi:10.1249/mss.0b013e31815a51b3. Troiano, R.P., McClain, J.J., Brychta, R.J., and Chen, K.Y. 2014. Evolution of accelerometer methods for physical activity research. Br. J. Sports Med. 48(13): 1019-1023. doi:10.1136/bjsports-2014-093546. PMID:PMC4141534 Tudor-Locke, C., Barreira, T.V., Schuna, J.M., Jr., Mire, E.F., Chaput, J.P., Fogelholm, M. et al. 2015. Improving wear time compliance with a 24-hour waist-worn accelerometer protocol in the International Study of Childhood Obesity, Lifestyle and the 296 297 Environment (ISCOLE). Int. J. Behav. Nutr. Phys. Act. 12: 11. doi:10.1186/s12966-015- 0172-x. PMID:Pmc4328595 298 299 300 301 302 303 Tudor-Locke, C., Barreira, T.V., Schuna, J.M., Jr., Mire, E.F., and Katzmarzyk, P.T. 2014. Fully automated waist-worn accelerometer algorithm for detecting children's sleep-period time separate from 24-h physical activity or sedentary behaviors. Appl. Physiol. Nutr. Metab. 39(1): 53-57. doi:10.1139/apnm-2013-0173. Webster, J.B., Kripke, D.F., Messin, S., Mullaney, D.J., and Wyborney, G. 1982. An activity-based sleep monitor system for ambulatory use. Sleep. 5(4): 389-399. 304 305 306 307 308
Page 14 of 18 14 309 Table 1. Bedtime, wake time, and sleep period time for screened and complete data set. 310 Screened Data Complete Data Bedtime Wake Time Sleep Period Time (min) Algorithm 11:53 PM 7:49 AM 475±66 Log 11:48 PM 7:41 AM 472±59 Algorithm 11:48 PM 7:56 AM 488±79 Log 11:48 PM 7:50 AM 481±62 311 Table 2. Analysis results for the screened and complete data sets. 312 Screened Data Complete Data Sleep Variable Difference (min) Correlation t p Bedtime 5±33 0.92 1.50 0.15 Wake Time 8±30 0.92 2.51 0.01 Sleep Period Time 3±38 0.81 0.73 0.47 Bedtime 0±66 0.66 0.01 0.92 Wake Time 6±41 0.86 1.40 0.18 Sleep Period Time 6±82 0.34 0.75 0.46 313 314 315 316 317 318 319 320 321
Page 15 of 18 15 322 Figure Legends 323 324 Figure 1 Example of a participant's self-recorded and accelerometer determined sleep related behaviors that was excluded from the analysis. 325 326 Figure 2 Example of a participant's self-recorded and accelerometer determined sleep related behaviors that was retained in the analysis. 327 Figure 3 Bland-Altman plot of Log and Algorithm sleep estimates.
Page 16 of 18 Figure 1 Example of a participant's self-recorded and accelerometer determined sleep related behaviors that was excluded from the analysis. 254x190mm (300 x 300 DPI)
Page 17 of 18 Figure 2 Example of a participant's self-recorded and accelerometer determined sleep related behaviors that was retained in the analysis. 254x190mm (300 x 300 DPI)
Page 18 of 18 Figure 3 Bland-Altman plot of Log and Algorithm sleep estimates. 254x190mm (300 x 300 DPI)