Department of Economics Working Paper

Transcription

1 Department of Economics Working Paper Number February 2016 A Practical Validation Study of a Commercial Accelerometer Using Good and Poor Sleepers David L. Dickinson Appalachian State University Joseph Cazier Appalachian State University Thomas Cech Appalachian State University Department of Economics Appalachian State University Boone, NC Phone: (828) Fax: (828)

2 Summary Apracticalvalidationstudyofacommercialaccelerometerusinggoodandpoorsleepers. DavidL.Dickinson JosephCazier ThomasCech WevalidatedtheperformanceofFitbitsleeptrackingdevicesagainstresearchDgradeactigraphyacross fourdays/nightson38youngadultgoodandpoorsleepers.fitbitdevicesunderestimatedchangesin nightlysleepcomparedtostandardactigraphy.nevertheless,weestimatedthefitbitcaptures88% (poorsleepers)to98%(goodsleepers)ofactigraphyestimatedsleeptimechanges,whichmaystillbe usefulforqualitativesleepanalysisovertime. Keywords:Sleeptracking,Fitbit,validation,actigraphy 1

3 INTRODUCTION TheusefulnessandvalidityofresearchDgradeactigraphydevicesarewellknown(seeSadeh,2011,and referencestherein).theriseininterestregardingconsumersleeptrackingdevicesforresearchimplies theneedfortestingsuchdevicesagainstacceptedsleepmonitoringtechnologies.thispaperreports resultsfromavalidationstudyofthefitbitsleeptrackingdeviceagainststandardactigraphy.fitbitisa leadingmakerofdevicesthatclaimtotracksleep,thoughrecentvalidationattemptshaveproduced mixedresults(meltzeret#al.,2015;montgomeryddownset#al.,2012;evensonet#al.,2015). 1 Asummary oftheclaimsandvalidityofnumerousconsumersleepmonitorsisfoundinrussoet#al.(2015),witha focusonthequestionoftheirpossibleusefulnessevenabsentclinicaldleveldatavalidity.ourstudy intendstocontributetothisdebate.wefindthatthefitbit,whilegenerallyproducingbiasedpoint estimatesofsleeptimeandefficiency,accuratelytracksdirectionalchangesofnightlysleepovertime, eveninpoorsleepers.thisimpliescontextswherethefitbitmaybeuseful,suchasintestingtheimpact ofasleepinterventiononagivensubject. MATERIALSANDMETHODS Werecruited38adultsubjects(23females,15males;26.05±7.99yearsold)whoeachsimultaneously woreacommonlyutilizedresearchdgradeactigraph(actiwatchspectrumplus)andafitbitchargehr devicefor4weekdays/nights.boththeactiwatchandfitbitweresettosampledataat30dsecond epochs,andthefitbitwassetto normal mode.weusethepittsburgsleepqualityindex(psqi)to identifybothgood(psqi 5;n=20)andpoorsleepers(PSQI>5).Subjectskeptsleepdiaries,andwe reportbothrawanddiarydadjustedfitbitdataontotalsleeptimeandefficiency. 2 Subjectswere compensated$50forparticipationandprocedureswereapprovedbytheinstitutionalreviewboardat theauthors university Onthefirstday,subjectsvisitedourlabandprovidedwritteninformedconsent,completedthePSQI, receiveddeviceinstructions,andwereassignedbothanactiwatchandafitbitdevice.beforedeparting, subjectswereinstructedtoreturntothelabeachdayforapproximatelytwentydminutes.duringthis time,theycompletedsleepdiariesonlineandlabtechnicianssyncedfitbitdeviceswithlabcomputers anddownloadedsubjects FitbitandActiwatchdatafromthepreviousday. StatisticalAnalysis Wecompareeachsubject snightlysleepmeasuretotheanalogousactigraphydproducedmeasure:timed indbed(tib),totalsleeptime(tst),sleepefficiency(asautomaticallydevicedscored),andtst/tib(which wecallquasidefficiency).actigraphydataarescoredusingvalidatedprocedures,andweexaminefitbit 1 Otherconsumersleeptrackershavebeenthesubjectofvalidationtests.Forexample,similarmixedresultshave beenfoundinvalidationstudiesofthejawboneupdevice(toonet#al.,2015;dezombottiet#al.,2015;evensonet# al.,2015). 2 TheproceduresusedfordiaryDaidedscoringoftheFitbitdataweresimilartovalidatedactigraphyprocedures (e.g.,goldman#et#al.,2007).becausesubjectssimultaneouslyworebothdevices,thisassuredthatthediarydaided scoringofbothfitbitandactigraphydatautlizedtheexactsamesleepdiaryrecord. 2

4 measuresoftstusingbothrawanddiarydadjusteddata.toourknowledge,existingvalidationstudies ofconsumermonitoringdevicesdonotalwaysadjustdevicedatawithinputfromsleepdiaries,even thoughthisiscommoninmanyresearchstudies. 3 Foreachoutcomemeasure,M,weestimatethefollowing: (1)########Fitbit(M)#=#α + β Actigraphy(M)#+#ε Whereεisarandomeffectserrortermaccountingforthemultipleobservations(n=4)persubject.The nullhypothesesthatbothα=0andβ=1impliesfitbitoutcomesarestatisticallynodifferentthan Actiwatchoutcomes.Rejectionofα=0reflectsageneralover/underestimationbyFitbitofthe actigraphydbasedmeasure.rejectionofβ=1indicateshypodorhyperdsensitivityofthefitbittochanges intheoutcomemeasure,comparedtoactigraphy. RESULTS Figures1,2,andTable1summarizethekeyresults.Figure1showsthescatterplotFitbitdatameasures (TSTandEfficiency)comparedtotheanalogousActigraphymeasure,withthelinearregressionestimate ofequation(1)superimposed.table1showsthefullestimationresultsoftst,sleepefficiency(shown infig.1),timedindbed(tib),andquasidefficiency(notshowninfig.1)aswellasestimatesforthe separatesubsamplesofgoodandpoorsleepers.inmostinstances,table1indicatesthatthefitbit generallyoverestimatestib,tst,andefficiencyrelativetotheactigraphymeasure(i.e.,rejectionofα=0 infavorofα>0).theresultsmostcloselyapproximateα=0andβ=1forthesubsampleofgoodsleepers, forwhomweestimatethatthefitbitmeasureof(diarydadjusted)tstisstatisticallyindistinguishable fromtheactigraphytstmeasure. FIGS1&2HERE Figure1andTable1resultsarebasedondiaryDadjusted(i.e., scored )Fitbitmeasures,asistypically donewithactigraphydata.figure2reproducesequation(1)estimatesfortstusingrawdataasdirectly providedtotheconsumerbythefitbitdevice.thescatterplotshowsthereductioninfitbitmeasure varianceresultingfromdiarydaidedscoring.thoughnotreportedintable1,estimatesofequation(1) usingfitbitrawdatafailtorejectα=0and β=1.however,weattributethistothehighervarianceinthe rawfitbitdata(seefig.2).tofurtherhighlighttheimportanceofdiarydaidedscoredoffitbitdata,we notethatthecorrelationbetweentheactiwatchrawversusscoreddatais.9582,comparedto.6327 betweenfitbitrawversusscoreddata. TABLE1HERE 3 Somedevicesrequireuseractivationof sleepmode,whichmayserveasadiarydtypemeasure.thefitbit ChargeHRdoesnotrequiresuchuseractivation.Also,somevalidationstudiesinvolveconcurrentPSGdata acquisition,butitisnotalwaysclearwhetherconsumerdevicedataisadjustedaspartofscoring. 3

5 DISCUSSION GiventheprevalentuseofactigraphyforextraDlabmonitoringofsubjectsleeplevels,weaimedto assessthepracticalusefulnessofthefitbitdeviceasanalternativetoactigraphyincertaincontexts. TheNationalSleepFoundationplacessignificantemphasisonsleepleveltargetsandguidelines,and theyroutinelyidentifysleepdeficitsbycomparingnightlysleepguidelinestoselfdreportmeasures.one useoflowdcostsleepmonitoringdevicesmaybetohelpassesswithindsubjectsleeptrendsinsettings whereclinicalaccuracyisnotnecessary.inotherwords,consumersleeptrackingdevicesmaystillbe qualitativelyusefulforpersonalgoaltrackingorevensomeappliedresearchpurposes(e.g.,did interventionxsignificantlyincreasejohndoe snightlysleep?). OurstatisticalanalysisfindsthatdiaryDadjustedFitbitdatashowfairlygoodquantitativepredictionson thekeytstvariableforgoodsleepers,andatleastqualitativelyusefuldataonpoorsleepers.the statisticalfitbetweenactigraphyandfitbitsleepefficiency(anditisunclearhowthatisdefinedwith Fitbit)ispoor,whichsuggeststheuseofthequasiDefficiencymeasure,TST/TIB.Thisisnotsurprising giventhemanualconstructionoftst/tibbenefitsfromdiarydaidedscoring.insum,whilecaution shouldbeexercisedtoproperlyscorefitbitdata,wefinditcanbeausefulandinformativemeasureof subjectsleepmeasuresincertaincontexts. ACKNOWLEDGEMENTS:WegratefullyacknowledgefundingfromtheXXXXXUniversityYYYYY Program.Valuableresearchassistancewiththedatacollectionandpreparationwasalsoprovidedby WWWWWandBBBBBBB. CONFLICTSOFINTERESTS:Theauthorsreportnoconflictofinterests.Actigraphydeviceswere purchasedforanearlierprojectfundedthroughnsfgrantbcsd tozzzzzz.fitbitdevices purchaseddirectlythroughamazon.com. AUTHORCONTRIBUTIONS:DLDandJCdesignedthestudy;DLDandJCwrotetheprotocol;JCandTC collectedthedata;dldconductedtheanalysis;tcscoredthedata;dldinterpretedthedata;allauthors contributedtowriting.allauthorshaveapprovedthefinalmanuscript. 4

6 FIGURE1:FitBitv.Actigraphy(OLSlinefitshown) TOTALSLEEPTIME(Nightly)COMPARISONSLEEPEFFICIENCYCOMPARISON Actigraphy TST Actigraphy Sleep Efficiency FIGURE2:FitBitscoredv.rawnightlyTSTdatacomparisontoactigraphy(OLSlinefitshown) TSTcomparisonusingscoredFitbitdataTSTcomparisonusingrawFitbitdata Actigraphy TST Actigraphy TST Note:LeftpanelofFig.2reproducestheleftpanelofFig.1withaxisrescaledforcomparabilitywithrawFitbitdata

7 TABLE1:FitBitoutcomemeasuresregressedagainstActigraphymeasures. TIB Variable Constant α ActigraphyTIB (1) AllSubjects (n=152) (22.035)***.854 (.048)*** DependentVariable=FitBitTimeinBed (2) GoodSleepers (PSQI 5;n=72) (27.908)***.912 (.053)*** (3) PoorSleeperss (PSQI>5;n=80) (31.93)***.803 (.072)*** β R-squared Testofβ=1 Χ 2 (1)=9.32*** Χ 2 (1) = 2.77* Χ 2 (1) = 7.48*** TST DependentVariable=FitBitTotalSleepTime Variable AllSubjects (n=152) GoodSleepers (PSQI 5;n=72) PoorSleepers (PSQI>5;n=80) Constant α (18.649)*** (22.013) (25.247)*** ActigraphyTST β.917 (.050)***.974 (.056)***.879 (.071)*** R-squared Testofβ=1 Χ 2 (1)=2.69* Χ 2 (1)=.21 Χ 2 (1)=2.92* TST/TIB DependentVariable=FitBitQuasiPEfficiency(TST/TIB) Variable AllSubjects (n=152) GoodSleepers (PSQI 5;n=72) PoorSleepers (PSQI>5;n=80) Constant α (13.175) (10.845) (22.440) ActigraphyTST/TIB β.742 (.140)***.840 (.117)***.685 (.238)*** R-squared Testofβ=1 Χ 2 (1)=3.39* Χ 2 (1)=1.87 Χ 2 (1)=1.75 Efficiency DependentVariable=FitBitEfficiency(devicedefined) Variable AllSubjects (n=152) GoodSleepers (PSQI 5;n=72) PoorSleepers (PSQI>5;n=80) Constant α (5.432)*** (1.767)*** (10.140)*** ActigraphyEfficiency β.207 (.061)***.105 (.021)***.279 (.115)** R-squared Testofβ=1 Χ 2 (1)=168.72*** Χ 2 (1)= *** Χ 2 (1)=39.61*** Notes:Randomeffectsregressionmodelswitherrorsclusteredbysubject(4observationspersubject).Robust standarderrorsshowninparenthesis.*,**,***indicatesignificanceatthe.10,.05,and.01levels,respectively, forthe2dtailedtest.statisticalequivalencebetweenactigraphyandfitbitoutcomevariableimpliesα=0, β=1. 6

8 REFERENCES Buysse,D.J.,Reynolds,C.F.III.,Monk,T.H.,Berman,S.R.,&Kupfer,D.J.(1989).ThePittsburghSleep QualityIndex:anewinstrumentforpsychiatricpracticeandresearch.Psychiatry#Research,28(2), 193D213. dezambotti,m.,claudatos,s.,inkelis,s.,colrain,i.m.,&baker,f.c.(2015).evaluationofaconsumer fitnessdtrackingdevicetoassesssleepinadults.chronobiology#international,32(7),1024d1028. Evenson,K.R.,Goto,M.M.,&Furberg,R.D.(2015).Systematicreviewofthevalidityandreliabilityof consumerdwearableactivitytrackers.international#journal#of#behavioral#nutrition#and#physical# Activity,12(1),1D22. Goldman,S.E.,Stone,K.L.,AncoliDIsrael,S.,Blackwell,T.,Ewing,S.K.,Boudreau,R.,...&Newman,A.B. (2007).Poorsleepisassociatedwithpoorerphysicalperformanceandgreaterfunctional limitationsinolderwomen.sleep,30(10),1317d1326. Meltzer,L.J.,Hiruma,L.S.,Avis,K.,MontgomeryDDowns,H.,&Valentin,J.(2014).Comparisonofa CommercialAccelerometerwithPolysomnographyandActigraphyinChildrenand Adolescents.Sleep,38(8),1323D1330. MontgomeryDDowns,H.E.,Insana,S.P.,&Bond,J.A.(2012).Movementtowardanovelactivity monitoringdevice.sleep#and#breathing,16(3),913d917. Russo,K.,Goparaju,B.,&Bianchi,M.T.(2015).Consumersleepmonitors:isthereababyinthe bathwater?nature#and#science#of#sleep,7,147d157. Sadeh,A.(2011).Theroleandvalidityofactigraphyinsleepmedicine:anupdate.Sleep#medicine# reviews,15(4),259d267. Toon,E.,Davey,M.J.,Hollis,S.L.,Nixon,G.M.,Horne,R.S.,&Biggs,S.N.(2015).Comparisonof CommercialWristDBasedandSmartphoneAccelerometers,Actigraphy,andPSGinaClinical CohortofChildrenandAdolescents.Journal#of#clinical#sleep#medicine:#JCSM:#official#publication# of#the#american#academy#of#sleep#medicine.(edpubaheadofprint) 7