Fall 2015 Meeting October 2-3, 2015 Actigraphy: Validity, Reliability, and Clinical Utility Cindy Nichols, PhD, DABSM, FAASM, CBSM Munson Sleep Disorders Center Traverse City, MI
Conflict of Interest Disclosures for Speakers 1. I do not have any relationships with any entities producing, marketing, re-selling, or distributing health care goods or services consumed by, or used on, patients, OR 2. I have the following relationships with entities producing, marketing, re-selling, or distributing health care goods or services consumed by, or used on, patients: Type of Potential Conflict Details of Potential Conflict Grant/Research Support Consultant Speakers Bureaus Financial support Other 1. 2. 3. 3. The material presented in this lecture has no relationship with any of these potential conflicts, OR 4. This talk presents material that is related to one or more of these potential conflicts, and the following objective references are provided as support for this lecture:
Objectives Describe the basic technology underlying actigraphy. Provide an overview of the reliability and validity of actigraphy in detection of sleep and wake, in normal and abnormal populations. Discuss the clinical utility of actigraphy in evaluation of sleep disorders.
Basic technology of actigraphy Measures frequency, duration, and intensity of physical activity. Movement is a vector sum of movement in 3 directions (side to side, head to toe, up and down). Components Piezoelectric accelerometer -pass filter (filters everything except the 2-3 Hz band) Timer Memory Interface
Basic technology of accelerometer Measures rate of change of velocity (i.e., gforce ). Piezoelectric component converts the mechanical motion into an electrical signal. When pressure is placed on the spring, the piezoelectric material is compressed. When acceleration occurs, the pressure changes on the piezoelectric material release energy, generating a voltage.
Many current uses for actigraphy Sleep/circadian patterns Activity monitoring in obesity research Scratching behavior in children with atopic dermatitis Movement in ADHD research Safety monitoring in the elderly Sprint speeds in football and soccer Tonic-clonic seizure detection Gait analysis and tremor in Parkinson s Disease Fetal movement
Examples of actigraphy devices Prices range from about $400-$3,000 depending on features (memory, +/- light sensor, interfaces, etc.)
Accelerometer placement Wrist Leg Hip Waist Jaw Ankle Toe
CPT codes 95803: actigraph testing, recording, analysis, interpretation, and report (72 hours-14 consecutive days) 95803-26: actigraph data interpretation and report only 95803-TC: actigraph testing without interpretation and report
Background concepts Reliability (consistency, precision): Do different actigraphy devices, placements, etc., agree with each other? Validity (meaningfulness): Does actigraphy measure what we are intending to measure (sleep, movement, PLMs, etc.)? Sensitivity: What is the agreement between minutes of sleep scored by actigraphy vs. PSG? Specificity: What is the agreement between minutes of wake scored by actigraphy vs. PSG?
Reliability and Validity 1) Which target best describes actigraphy? 2) What are we really aiming at? Quality of sleep? Quantity of sleep? Consistency of sleep? Pattern of sleep across time?
Why is it important to understand reliability in actigraphy? Reliability is necessary but not sufficient to establish validity. The ability to obtain multiple samples over time improves test reliability by reducing the probability of sampling error caused by night-tonight variability. Actigraphy may be reliable in one demographic, condition, or severity but not in another.
Sensitivity and specificity applied to actigraphy Sensitivity = true positive If the person is really sleeping, does the actigraph indicate sleep? Specificity = true negative If the person is not sleeping, does the actigraph indicate wake? Actigraph sleep Actigraph wake PSG sleep true positive false negative PSG wake false positive true negative Sensitivity=true positives/(true positives+false negatives) Specificity=true negatives/(false positives+true negatives)
Sensitivity and specificity characteristics of actigraphy Actigraphy generally has High sensitivity (sleep detection): 90-97% Low specificity (wake detection):25-60% 1. Sadeh A. The role and validity of actigraphy in sleep medicine: an update. Sleep Medicine Reviews 2011;15:259-267.
What is the question? Is it more important to know if the person is asleep in bed (sensitivity) or if they are awake in bed (specificity)? Analogy: Is it more important to detect the number of people in a pool who are swimming well or to detect the ones who are struggling? If you intervene with a swimmer who seems to be struggling but is really OK, will it cause harm, incur costs, or distract you from helping someone who is drowning?
Understanding actigraphy specificity and sensitivity Assuming 90% sleep efficiency, if all epochs are scored as sleep, the sensitivity of actigraphy will be 100% and specificity will be 0%. If the device is less able to detect small movements, test specificity is reduced. If the device is more able to detect small movement, test sensitivity is reduced. More severely disordered sleep results in less diagnostic utility for actigraphy, particularly in insomnia 1. 1. Paquet J, Kawinska A, Carrier J. Wake detection capacity of actigraphy during sleep. Sleep 2007;30:1362-9.
Scoring algorithms Most are based on a combination of linear compilations (of predefined windows) of activity levels and various logical decisions (smoothing, decision trees, neural networks) 1. Use of the high sensitivity setting does not appear to improve detection of wake (55% wake detection) 2. Minutes of immobility for determination of sleep onset vary across age and specific device 3. 1. Tilmanne J, Urbain J, Kothare M, et al. Algorithms for sleep-wake identification using actigraphy: a comparative study and new results. J Sleep Res 2009;18:85-98. 2. van de Wouw E, Evenhuis H, Echteld M. Comparison of two types of Actiwatch with polysomnography in older adults with intellectual disability. J Intellect Dev Disabil 2013;38:265-73. 3. Meltzer L, Walsh, Peightal A. Comparison of actigraphy immobility rules with polysomnographic sleep onset latency in children and adolescents. Sleep & Breathing 2015; epub ahead of print
Actigraphy example
Validity data on sleep apps Consumer devices lack objective evidence to support claims 1. Sleep Time tm app for iphone was similar to actigraphy in detecting sleep (86% sensitivity) but appears to have no correlation with sleep efficiency or other sleep parameters 2. 1. Lee J, Finkelstein J. Activity trackers: a critical review. Studies in health technology & informatics 2014;205:558-62. 2. Bhat S, Ferraris A, Gupta D, et al. Is there a clinical role for smartphone sleep apps? J Clin Sleep Med 2015;11:709-715.
Validity data on commercially available devices The Fitbit Ultra (sensitive setting) demonstrated reasonable specificity (.79) but inadequate sensitivity/accuracy (.70), overestimating sleep time. Similar to actigraphy, the Fitbit Ultra (normal mode) demonstrated good sensitivity (.86) but poor specificity (.52) 1. 1. Meltzer L, Hiruma L, Avis K, et al. Comparison of a commercial accelerometer with polysomnography and actigraphy in children and adolescents. Sleep 2015;38:1323-1330.
Comparing actigraphy, caregiver observations, and self-report Caregivers overestimate sleep time in both the elderly 1 and infants 2. Actigraphy may be more accurate than sleep diaries for individuals with depression 3 and PTSD 4. 1. Hoekert M, Riemersma-van der Lek R, Swaab D, et al. Comparison between informant-observed and actigraphic assessments of sleep-wake rhythm disturbances in demented residents of homes for the elderly. Am J Geriatr Psychiatry 2006;14:104-11. 2. So K, Adamson T, Horne R. The use of actigraphy for assessment of the development of sleep/wake patterns in infants during the first 12 months of life. J Sleep Res 2007;16:181-7. 3. McCall C, McCall W. Comparison of actigraphy with polysomnography and sleep logs in depressed insomniacs. J Sleep Res 2012;21:122-7. 4. Westermeyer J, Sutherland R, Freerks M, et al. Reliability of sleep log data versus actigraphy in veterans with sleep disturbance and PTSD. J Anxiety Disord 2007;21:966-75.
Reliability data Comparison of the AMI and Phillips devices demonstrated poor agreement, with >1 hr differences between devices in both WASO and TST 1. When compared to PSG, accuracy is better for wrist placement than hip-based measurement 2. 1. Meltzer L, Walsh, Traylor J et. al. Direct comparison of two new actigraphs and polysomnography in children and adolescents. Sleep 2012;35:159-166. 2. Zinkhan M, Berger K, Hense S, et al. Agreement of different methods for assessing sleep characteristics: a comparison of two actigraphs, wrist and hip placement, and self-report with polysomnography. Sleep Med 2014;15:1107-14.
Clinical utility of actigraphy in insomnia Advantage of actigraphy is the ability to document patterns across many days. Since clinical interventions are usually based on an extended period of time, actigraphy may be a good fit for the evaluation of interventions for insomnia. Actigraphy typically underestimates wake, but sleep diaries over-estimate wake in insomnia patients. Accuracy of actigraphy declines as sleep efficiency declines 1. 1. Taibi D, Landis C, Vitiello M. Concordance of polysomnographic and actigraphic measurement of sleep and wake in older women with insomnia. J Clin Sleep Med 2013;15:217-25.
Clinical utility of actigraphy in hypersomnias May be helpful in determining average daily sleep time and stability during the week prior to evaluation with PSG and MSLT 1. Morgenthaler T, Alessi C, Friedman L, et al. Practice parameters for the use of actigraphy in the assessment of sleep and sleep disorders: an update for 2007. Sleep 2007;30:519-529.
Clinical utility of actigraphy in SDB May slightly improve accuracy of AHI in HSAT, although the improvement is minor 1. When compared to patients with insomnia or even normals, actigraphy may have a higher correlation with WASO in untreated OSA patients vs. OSA patients without other comorbid sleep disorders 2. 1. Garcia-Diaz E, Quintana-Gallego E, Ruiz A, et al. Respiratory polygraphy with actigraphy in the diagnosis of sleep apnea-hypopnea syndrome. Chest 2007;131:725-32. 2. Dick R, Penzel T, Fietze I, et al. AASM standards of practice compliant validation of actigraphic sleep analysis from SOMNOwatch versus polysomographic sleep. Physiol Meas 2010;31:1623-33.
Clinical utility of actigraphy in shift work and circadian rhythm sleep disorders Potential use for evaluating time in bed and relationship to safety measures 1. Ability of actigraphy to evaluate patterns of rest and activity is also useful in evaluation of circadian rhythm disorders 2. 1. Kawada T, Shimizu T, Kuratomi Y, et al. Monitoring of the sleep patterns of shift workers in the automotive industry. Work 2011;38:163-7. 2. Morgenthaler T, Lee-Chiong T, Alessi C, et al. Practice parameters for the clinical evaluation and treatment of circadian rhythm sleep disorders. Sleep 2007;30:1445-59.
Clinical utility of actigraphy in PLMD Agreement between PSG and two different actigraphy devices attached to both legs is poor (+/- 30/hr) 1. Detection of PLMs improves when sensor is moved to the ankle or base of the big toe 2. 1. Gschliesser V, Frauscher B, Brandauer E, et. al. PLM detection by actigraphy compared to polysomnography: a validation and comparison of two actigraphs. Sleep Med 2009;10:306-11. 2. Kemlink D, Pretl M, Sonka K, et al. A comparison of polysomnographic and actigraphic evaluation of periodic limb movements in sleep. Neurol Res 2008;30:234-8.
Conclusions Actigraphy underestimates SOL and WASO in comparison to PSG. Poorer sleep is associated with poorer accuracy of actigraphy, therefore clinical utility for diagnosis of many sleep disorders is limited. Actigraphy data do not correlate well with measures of sleep quality.
Conclusions Actigraphy is particularly useful in documenting patterns across many days, and in the evaluation of interventions for circadian rhythm disorders and insomnia. Actigraphy, subjective sleep descriptions, and PSG all measure aspects of sleep.
Questions?