SMARTPHONE-BASED USER ACTIVITY RECOGNITION METHOD FOR HEALTH REMOTE MONITORING APPLICATIONS

SMARTPHONE-BASED USER ACTIVITY RECOGNITION METHOD FOR HEALTH REMOTE MONITORING APPLICATIONS Igo Bisio Fabio Lavagetto Maio Machese Andea Sciaone Univesity of Genoa DYNATECH {igo.bisio fabio.lavagetto maio.machese andea.sciaone}@unige.it Keywods: Abstact: Remote Monitoing: Activity Recognition: Acceleomete: Decision Tees: Windowed Decision: Andoid Smatphones. In the famewok of health emote monitoing applications fo individuals with disabilities o paticula pathologies quantity and type of physical activity pefomed by an individual/patient constitute impotant infomation. On the othe hand the technological evolution of Smatphones combined with thei inceasing diffusion gives mobile netwok povides the oppotunity to offe eal-time sevices based on captued eal wold knowledge and events. This pape pesents a Smatphone-based Activity Recognition (AR) method based on decision tee classification of acceleomete signals to classify the use s activity as Sitting Standing Walking o Running. The main contibution of the wok is a method employing a novel windowing technique which educes the ate of acceleomete eadings while maintaining high ecognition accuacy by combining two singleclassification weighting policies. The poposed method has been implemented on Andoid OS smatphones and expeimental tests have poduced satisfying esults. It epesents a useful solution in the afoementioned health emote applications such as the Heat Failue (HF) patients monitoing mentioned below. І INTRODUCTION In the famewok of health emote monitoing technologies fo individuals with disabilities o paticula pathologies quantity and type of physical activity pefomed by an/a individual/patient constitute impotant infomation fo the medical staff that monitos its state of health. An impotant case is epesented by people suffeing fom HF: continuous monitoing of biometic paametes such as body weight and the physical activity eally pefomed allow defining specific theapies that can significantly impove the quality of life. On the othe hand the technological evolution of smatphones combined with thei inceasing diffusion gives mobile netwok povides the oppotunity to offe moe advanced and innovative sevices. Among these ae the so-called context-awae sevices. Examples of context-awae sevices ae use pofile changes as a esult of context changes use poximity-based advetising o media content tagging etc. In ode to povide context-awae sevices a desciption of the smatphone envionment must be obtained by acquiing and combining context data fom diffeent souces and sensos both extenal (e.g. cell IDs GPS coodinates) and intenal (e.g. battey powe acceleomete measuements). An example of context-awae applications is (Keally 2011) fo emote health cae sevices and (Boyle 2006) fo monitoing patients affected by chonic diseases. In geneal the monitoing of the physical activity epesents a vey useful tool to develop effective theapies. Fo this eason the poposed AR method is designed to distinguish fou diffeent use activities by peiodically classifying acceleomete signals fames using a decision tee appoach. The method employs a novel efficient windowing technique which educes the signals fame acquisition ate and goups sets of consecutive fames in windows epesenting use state. In ode to maintain high ecognition accuacy the educed fequency of acceleomete eadings is compensated

by weighting each single-fame classification with a combination of two diffeent sets of weights which takes into account each fame s instant of occuence and classification confidence. Expeimental tests show accuate esults while peseving battey life. The contibutions of this position pape will be futhe developed. In fact the application of the poposed appoach equies an extensive expeimentation in coopeation with a medical staff and a sample goup of patients suffeing fom HF. Moeove befoe the eal deployment of such a smatphone-based use activity ecognition method the ecognized movements set should be eniched with othe typical activities such as climbing/down the stais and indoo/outdoo cycling teadmill. ІІ RELATED WORK In ecent yeas a significant amount of wok has been poposed concening context-awae sevices elying on acceleomete data. Application fields ae divese and include emote health-cae (Ryde 2009) social netwoking and Activity Recognition (Miluzzo 2008). Diffeently fom the appoach poposed in this pape some of the poposed methods ae designed to wok with ad hoc sensos won by the uses (Keally 2011). Othe methods ae developed fo commone devices such as smatphones. Fo example Nokia s N95 is a popula choice (e.g. (Miluzzo 2008) (Wang 2009) and (Ryde 2009)) have been implemented on it) but newe devices have eceived some attention as well: an iphone vesion of (Miluzzo 2008) has been developed and (Ryde 2009) has also been implemented on Andoid smatphones. Othe methods stand in between equiing both custom hadwae and off-the-shelf devices. Fo example the method descibed in (Keally 2011) employs an Andoid smatphone and ad-hoc weaable sensos. A Pape Contibution The poposed method is implemented on an Andoid smatphone and it takes into account the limitations of mobile devices. Even though pocessing powe and battey capacity ae impoving they still emain limited and valuable and must not be employed excessively fo backgound added-value sevices giving pioity to voice calls. Othe pojects have also tackled such poblem: (Wang 2009) poposes a system that manages senso duty cycles and educes enegy consumption by shutting down unnecessay sensos. The poposed method follows a simila appoach but it also adds a window-based mechanism which epesents the main contibution of the pape. ІІІ APPLICATIVE SCENARIO: HEART FAILURE PATIENT MONITORING The Heat Failue (HF) is a chonic disease that altenates intense and weak phases and equies epeated and fequent hospital teatments. The use of automatic instuments fo a emote and ubiquitous monitoing of biological paametes elevant with espect to the HF pathophysiology offes new pespectives to impove the patients life quality and the efficacy of the applied clinical teatment. In moe details HF is a disease epesented by the limited capacity of the heat to povide a sufficient blood flow needed to meet all the body s necessities. HF usually causes a significant quantity of symptoms such as shotness of beath weight gain due to excessive fluids leg swelling and execise intoleance. This illness condition can be diagnosed with echocadiogaphy and blood tests and the consequent teatment commonly consists of continuous lifestyle measues dug theapy o in vey citical cases sugey. Cuently in medical pactice well-known patients management models ae focused on manually handled emote monitoing appoaches: nuses daily inteogate though a phone call patients about thei weight and the physical activity they have done. The achieved esults of this pactice show that this continuous emote monitoing appoach impoves the quality of life of these patients pevents the pogession to HF advanced stages and educes the use of hospitalization. The pesented smatphone-based AR method associates moden context-awae capability of the ecent smatphone platfoms obtained by implementing specific algoithmic solutions with the any-time and any-whee communication capability commonly offeed by them. This joint usage will allow educing the patients involvement in the monitoing pocess without impacting its effectiveness. In paticula the method poposed is going to be applied in an eal expeimental campaign in coopeation with a medical staff.

ІV THE PROPOSED ACTIVITY RECOGNITION METHOD The poposed activity ecognition method is designed to distinguish fou diffeent use activities: Sitting Standing Walking and Running. The algoithm peiodically collects aw signals fom the smatphone acceleomete. Sensed signal consists of a sequence of tiples epesenting acceleation measuements along thee othogonal axes (poduced at a vaiable ate). F (fame duation) seconds woth of signal is acquied evey T [s] (fame acquisition peiod) i.e. the acceleomete is switched off fo T F seconds in ode to educe the oveall enegy consumption. With espect to constant acceleomete signals acquisition a decease in signals acquisition ate may lead to a less pecise knowledge of the activity. Theefoe a windowing technique based on a single-fame classification weighting mechanism is employed as descibed in Section IV.C. Fo evey fame a set of distinctive featues is computed. Such featue vecto is used by a decision tee classifie to classify the fame as Sitting Standing Walking and Running. A goups of W consecutive fames ae oganized in windows with consecutive windows ovelapped by O fames. Evey completed window is assigned to one of the fou consideed classes based on a decision policy that takes into account each fame s instant of occuence and classification confidence. A Raw Acceleomete Signal The smatphone employed in this wok is an HTC Deam which mounts a 3-axial acceleomete built by Asahi Kasei Copoation. Each signal sample (also called data in the following) poduced by such integated chip epesents the acceleation (in m/s 2 ) measued on thee othogonal axes. In moe detail facing the phone display the oigin is in the lowe-left cone of the sceen with the x axis hoizontal and pointing ight the y axis vetical and pointing up and the z axis pointing outside the font face of the sceen. B Fame Classification In ode to be classified a featue vecto is associated to each individual fame composed by M samples. As in (Miluzzo 2008) the featues employed fo single-fame classification ae the mean ( µ ) standad deviation ( σ ) and numbe of peaks of the measuements (P computed as epoted in eq. 1) along the thee axes x y and z of the acceleomete. M Pj = ρm whee m= 1 (1) 1 if ( jm+ 1 jm )( jm jm 1 ) < 0 jm ε ρm = 0 othewise j is a geneic vaiable epesenting the acceleomete signal along the thee axes and j is m the m -th sample of the fame. ε in equation (1) is a theshold employed to define a signal peak. Thus the µ µ µ σ σ σ. featue vecto is { x y z x y z Px Py Pz } Once a featue vecto has been computed fo a given fame it is used by a classifie in ode to associate the coesponding fame to one of the classes descibed at the beginning of Section IV. The employed classifie is a decision tee (Ross 1993) a commonly used classifie in simila AR woks such as (Miluzzo 2008) (Ryde 2009). Using the Weka wokbench (Hall 2009) seveal decision tees wee designed and compaed based on thei ecognition accuacy. A decision tee was tained fo evey combination of two and thee of the uses employed in the dataset ceation (see Section V.A). In ode to evaluate the classifies pefomance a sepaate test set was used fo each combination. C Windowed Decision The ate of acceleomete eadings must be compatible with the enegy esouces of the smatphone. Windowed decisions defined below guaantee satisfactoy pefomance while saving the enegy esouce. In details goups of W consecutive fames ae oganized in windows. The window size (i.e. the numbe of fames in each window) affects the state associated with the use and must be set caefully. Small windows ensue a quicke eaction to actual activity changes but ae moe vulneable to occasionally misclassified fames. On the othe hand lage windows eact moe slowly to activity changes but povide bette potection against misclassified fames. Consecutive windows ae ovelapped by O fames. Employing heavily-ovelapped windows povides a bette knowledge of the activity but may also imply consecutive windows beaing edundant infomation while using slightly-ovelapped

windows could lead to signal sections epesenting meaningful data falling acoss consecutive windows. The paametes employed ae: minimum time that must elapse between consecutive windowed decisions; W numbe of peiods in each window; O numbe of peiods shaed by consecutive windows; N pause between two consecutive signal acquisitions expessed in numbe of fames. In pactice it is equivalent to consideing a fame-acquisition peiod T = ( N + 1) F whee F is the fame duation expessed in seconds. Such paametes ae tied by the following expession: ( W O) T and epesented in Fig 1. Figue 1. Diagam epesenting the aw acceleomete signal acquisition. Each windowed decision assigns the cuent window to one of the fou consideed classes based on a given decision policies. Fou diffeent decision policies wee poposed evaluated and compaed as detailed in the following. 1) Majoity Decision. The simplest windowed decision policy is a majoity-ule decision: the window is associated to the class with the most fames in the window. Such decision mechanism is employed in some ealie wok on AR e.g. (Toth 2008). While it is clealy simple to implement and computationally inexpensive the majoity-ule windowed decision teats all fames within a window in the same way without consideing when the fames occued o the single fame classifications eliability. 2) Time-Weighted Decision. A fist altenative to the majoity-ule decision is the time-weighted decision. In a nutshell it implies giving diffeent weights to a window s fames based solely on thei position in the window and assigning a window to the class with the highest total weight. This way a fame will have a geate weight the close it is to the end of the window unde the assumption that moe ecent classifications should be moe useful to detemine the cuent use activity. In ode to detemine what weight to give to fames Ω t was designed accoding a weighting function ( ) to the citeia that ( 0) 1 Ω = and ( t ) ( t ) Ω Ω fo 1 2 any t1 t2. It is woth noticing that t is nonnegative and t = 0 epesents the time at which the most ecent fame occued. If T f is the instant associated with a fame and T d is the instant at which the windowed decision is made then the fame will be assigned a weight equal to ( Td Tf ) Ω. Two diffeent weighting functions wee compaed: a gaussian ( ) e negative exponential Ω e ( t) = e k t. i ( ) ω 2 t 2 2k g Ω g t = e and a Fo each function type five diffeent functions wee compaed by choosing k g and k e based on efeence instant T and focing Ω T = i = g e whee ω is one of five linealy-spaced values between 0 and 1. 3) Scoe-Weighted Decision. A second kind of windowed decision policy equies assigning to each fame a scoe epesenting how eliable its classification is. In ou wok we implemented the method poposed in (Toth 2008) not epoted fo the sake of bevity and we applied it to the windowed based AR method. The basic idea is that the close a fame s featue vecto is to the decision bounday the moe uneliable the fame s classification will be unde the hypothesis that the majoity of badly classified samples lie nea the decision bounday. The distance of a featue vecto fom the decision bounday is given by the shotest distance to the leaves with class label diffeent fom the label associated to the featue vecto. The distance between a featue vecto and a leaf is obtained by solving a constained quadatic pogam. Using sepaate taining data fo each leaf an estimate of the coect classification pobability conditional to the distance fom the decision bounday is poduced. Such estimate is computed by using the leaf s pobability of coectly and incoectly classifying taining set samples (obtained in tems of elative fequency) and pobability density of the distance fom the decision bounday conditional to coect and false classification (obtained though kenel density estimation). The classification scoe is finally given by the lowe bound of the 95% confidence inteval fo the estimate of the coect classification pobability conditional to the distance fom the decision bounday. The confidence inteval lowe bound is

used instead of the coect classification pobability conditional to the distance fom the decision bounday estimate because the latte may emain close to 1 even fo lage distances. Howeve a lage distances may not imply a eliable classification pobably due to an unknown sample located in a egion of the featue space insufficiently epesented in the taining set. On the contay passed a cetain distance (which vaies with evey leaf) the confidence inteval lowe bound deceases apidly. 4) Joint Scoe-/Time-Weighted Decision. Anothe windowed decision policy is given by combining the tempoal weights and the classification scoes into a single joint time-andscoe weight. Fusion is obtained simply by multiplying the coesponding time weight and classification scoe since both ae between 0 and 1. TABLE I. EMPLOYED DATASET. Sitting Standing Walking Running Fames 3702 3981 3822 3711 Duation [min] 246.8 265.4 254.8 247.4 all fou consideed use activities. Activities ae pefomed in andom ode and thei labels ae used as gound tuth. At fist single-fame classification is pefomed on the sequence poducing ecognizedclass labels and classification scoes. Afte that windowed decision accuacy is evaluated (as descibed in Section V.C) fo all admissible combinations of W O and N i.e. paamete values especting the equations in Section IV.C. was set to 60 [s] and W O and N wee evaluated W O [ 0 W 1] N [ 014] fixed in an empiical way. Theefoe in the following intevals: [ 39] 411 diffeent { W O N } tiples wee evaluated. C Results Consideing the single-fame esults of all the evaluated classifies the one with the best accuacy poduced a 98% coect test set classification aveage. In the following the elated confusion matix (with pecentages) has been epoted. V PERFORMANCE INVESTIGATION TABLE II. CONFUSION MATRIX IN CASE OF SINGLE- FRAME CLASSIFICATION (%). A Dataset The dataset employed in the expeiments was acquied by fou voluntees. Each voluntee acquied about 1 hou of data fo each of the classes descibed in Section IV poducing a total of almost 17 hous of data as shown in Table I. The phone was kept in the use s font o ea pants pocket (the last one is not used fo the Sitting class since uses will not keep the smatphone in a back pocket while sitting) as suggested in (Bao 2004) and taining data was acquied accodingly. Futhemoe the acquisition of taining data was pefomed keeping the smatphone with the display facing towads the use o away fom him and keeping the smatphone itself pointing up o down. Fo evey combination of two and thee uses the dataset was then divided into a taining set fo classifie taining and a distinct test set fo pefomance evaluation puposes. B Paametes Setting In ode to detemine the best values fo paametes W O N and ω an additional ad hoc sequence not included in the dataset used fo classifie taining and testing was acquied by a fifth voluntee. Such sequence is made of just ove an hou of aw acceleomete signal and it is efeed to Sitting Standing Walking Running Sitting 99 0 1 0 Standing 0.27 98.68 0.82 0.23 Walking 0 0.05 98.85 1.1 Running 0.28 0.1 4.4 95.22 As descibed in Section V.B windowed decision was applied to an ad hoc sequence using 411 diffeent paamete configuations. Futhemoe all six decision policies descibed in Section IV.C (and also listed in Table III) wee compaed fo each paamete configuation using five diffeent values fo ω (as descibed in Section IV.C) and two diffeent values fo T (i.e. 60 s and 120 s) fo each policy. The esults can be summed up in Tables III. It epots the Recognition Accuacy (%) defined as the aveage coect detection ove all consideed classes and the Reading Time (%) which is the pecentage of time dedicated to acceleomete signal eading with espect to the continuos eading (stictly elated to the enegy consumpion). Fom Table II the Recognition Accuacy is eally outstanding in case of fame-based appoach. Concening the windowed appoaches which allow to save enegy the time-based fame classification weighting doesn t seem to impove pefomance compaed to the majoity decision significantly

while employing classification-scoe weighting by itself o combined with time-weighting led to significant impovements in windowed decision accuacy. Oveall the best paamete configuation led to the mentioned 88.24% windowed decision accuacy: it was obtained W = 5 O = 1 and N = 7 joint scoe/time fame weighting using a Gaussian function and T = 120 [s]. Such decision policy led to an 8.24% incease in windowed decision accuacy compaed to the classical majoity-ule decision. Futhemoe using N = 7 allows educing the Reading Time by 87.5% with espect to the case of constant acceleomete signal acquisition (N = 0) thus educing enegy consumption while maintaining a satisfying ecognition accuacy. TABLE III. PERFORMANCE OF THE PROPOSED WINDOWED DECISION APPROACHES. Fame Based Window Based Decision Appoach Single-fame classification RA(%) Reading Time (%) 98 100 Majoity 80 12.5 Time Weighted (Gaussian / Exponential) 80 / 8 4.62 12.5 / 20 Scoe Weighted 88.24 9.09 Joint Scoe-/Time- Weighted (Gaussian / Exponential) 88.24 / 88.24 9.09 / 9.09 TABLE IV. PERFORMANCE OF ALTERNATIVE SMARTPHONE-BASED APPROACHES IN THE LITERATURE Refeence (Miluzzo 2008) (Wang 2009) (Ryde 2009) RA (%) 79 90 Recognized Classes Sitting Standing Walking Running StillVehicle Walking Running Senso(s) Acceleomete Acceleomete GPS 96 Outdoo Activities Acceleomete CONCLUSIONS In this pape a smatphone-based activity ecognition method designed to distinguish fou diffeent use activities is descibed. It epesents a useful solution fo health emote monitoing applications in paticula in case of patients affected by Heath Failue. It is based on the classification of acceleomete signal fames using a decision tee mechanism. In ode to limit the device enegy consumption the poposed method employs a windowing technique which educes the fame acquisition ate and goups sets of consecutive fames in windows epesenting the use state. The poposed AR method has a good level of accuacy. Its ecognized movements set will be eniched with othe typical activities such as climbing/down the stais and indoo/outdoo cycling teadmill soon. Afte that it will be applied in an expeimental campaign in coopeation with a medical staff to measue the quantity and the type of physical activity of patients affected by Heat Failue. REFERENCES [1] M. Keally G. Zhou G. Xing J. Wu and A. Pyles 2011. PBN: Towads Pactical Activity Recognition Using Smatphone-Based Body Senso Netwoks SenSys 11 Novembe 1 4 2011 Seattle WA USA. [2] J. Boyle M. Kaunanithi T. Wak W. Chan and C. Colavitti 2006. Quantifying functional mobility pogess fo chonic disease management Engineeing in Medicine and Biology Society. EMBS'06. 28th Annual Intenational Confeence of the IEEE 5916 5919. [3] E. Miluzzo et al. 2008. Sensing meets mobile social netwoks: the design implementation and evaluation of the CenceMe application in SenSys 08: In Poc. of the 6th ACM confeence on Embedded netwok senso systems. ACM Novembe 5-7 pp. 337 350. [4] Y. Wang J. Lin M. Annavaam Q. A. Jacobson J. Hong B. Kishnamachai and N. Sadeh 2009. A famewok of enegy efficient mobile sensing fo automatic use state ecognition In Poceedings of the 7th intenational Confeence on Mobile Systems Applications and Sevices (Kaków Poland June 22-25 2009). MobiSys '09. ACM New Yok NY 179-192. [5] J. Ryde B. Longstaff S. Reddy and D. Estin 2009. Ambulation: a tool fo monitoing mobility pattens ove time using mobile phones UC Los Angeles: Cente fo Embedded Netwok Sensing. [6] L. Bao and S. S. Intille 2004. Activity ecognition fom use-annotated acceleation data In 2nd Intenational Confeence PERVASIVE 04 Vienna Austia Apil 21-23. [7] J. Ross Quinlan 1993. C4.5: pogams fo machine leaning Mogan Kaufmann Publishes Inc. [8] M. Hall et al. 2009. The WEKA data mining softwae: an update SIGKDD Exploations Volume 11 Issue 1. [9] N. Toth and B. Pataki 2008. Classification confidence weighted majoity voting using decision tee classifies Intenational Jounal of Intelligent Computing and Cybenetics vol. 1 no. 2 Apil.