Real-Time 3D TUG Test Movement Analysis for Balance Assessment Using Microsoft Kinect

Transcription

1 Real-Time 3D TUG Test Movement Analysis for Balance Assessment Using Microsoft Kinect Asma Hassani 1, Alexandre Kubicki 2,3, Vincent Brost 1, and Fan Yang 1 1 Laboratoire LE2I CNRS 6306, université de Bourgogne, Dijon, France {asma.hassani, vincent.brost, fanyang}@u-bourgogne.fr 2 INSERM U1093, cognition action et plasticité sensori-motrice, campus universitaire, université de Bourgogne, BP 27877, Dijon, France 3 Centre Hospitalier Universitaire de Dijon, Hôpital de Champmaillot, 2 rue Jules Violle, Dijon, France alexandre.kubicki@u-bourgogne.fr Abstract. In this paper, a real-time 3D computer vision system for balance assessment based on the Kinect sensor is presented. It is intended for rehabilitation of the frail elderly in home-based environment. The system runs in real-time and is capable of extracting nine spatio-temporal parameters from recorded joint positions by 3D skeletal sequence processing. In this study, ten healthy young subjects and ten elderly subjects were asked to perform the TUG (Timed Up and Go) test in order to compare the results between the 2 groups. Through the TUG movements, the parameters were calculated and we mainly analyze the transfer from sitting-to-standing and back-to-sitting that represent two of the most commonly executed human movements. The results showed good estimation with important precision. In addition, we observed that the trunk angle is the most important parameter that allow to show the effect of aging. Keywords: Geriatric health autonomy assessment Kinect spatio-temporal movement parameter extraction 3D real-time video processing 1 Introduction The proportion of older people in the population is increasing, hence the need for balance assessment in the frail elderly. This allows to evaluate the motor abilities of the frail elderly so health professionals could be alerted in case of deterioration. Thus, to deal with the social consequences of aging, several methods for balance assessment in elderly were proposed. Most of these methods are not suitable for the home environment and imply complex operations to perform [1]. In addition, there is a lack of physiotherapy at home. Recently, Kinect sensor has been used in prototype systems for fall detection [2] and assessment of postural control [3]. In [4], the authors developed a Kinectbased rehabilitation system to assist therapists in their work with students who had motor disabilities. Indeed, Kinect is a low-cost and portable device that combines an RGB camera, a depth sensor and a multi-array microphone. It offers

2 2 inexpensive depth sensing for a wide variety of emerging applications in computer vision, augmented reality and robotics. In addition, it allows to perform marker-less human 3D tracking of the body movements in home environments. In [5], the study showed that results (segment lengths and angle estimation) for shoulder are accurate with Kinect and that it could be used at home for rehabilitation exercises. Furthermore, gait velocity [2], hand and elbow movements [6] can be assessed by means of Kinect. Moreover, it allows to evaluate anatomical landmark displacement and trunk angle during commonly performed clinical tests of postural control that represent favorable results when compared to some existing 3D motion analysis systems [3]. In addition, in [7], Kinect has been shown to be able to create a 3D human model with similar accuracy to expensive and complex 3D body scanning systems. In this paper, we present a low-cost, robust and home-based system for realtime balance assessment in the frail elderly. In fact, the earlier balance problem detection and the precocity of rehabilitation could allow, through a primary or secondary prevention, prolonging functional independence of elderly. Our system can accurately measure several parameters that have been shown relevant for balance assessment. It automatically and accurately extracts shoulder kinematics and TUG duration by the use of the 3D skeleton, with robustness to environmental changes and variations in the placement of the Kinect sensor. We present the results obtained from 10 healthy young subjects and 10 elderly subjects and a comparison between the results of the two groups was carried out in an attempt to highlight motor behavior disorders. This article is organized as follows: section 2 presents the proposed real-time system from data acquisition to feature parameter extraction after a brief description of the TUG test. Section 3 illustrates experiment tests and the analysis of the results. Finally, we discuss and conclude this study in section 4. 2 System and Protocol Description 2.1 Timed Up and Go Test The Timed Up and Go test is a simple clinical balance test that requires no special equipment or training. It has been shown to predict falls risk in the elderly and assess functional mobility. The TUG test consists of a sequence of sit-to-stand (STS), walking a distance of 3 m, turning and back-to-sit (BTS) [8]. The total time taken to complete the TUG test allows predicting the risk of falling. A score of 12 s is considered as the upper limit of normal mobility. Abnormal mobility was defined as having a TUG score of >= 20 s. The TUG includes two actions that are commonly executed throughout the different stages of the human lifetime: STS and BTS. The shoulder moves in the sagittal plane with a forward-upward (STS) and downward-backward (BTS) displacement. These movements allow estimating some parameters that were identified in the literature as relevant for balance assessment [9]. These parameters are the following: a) movement duration, b) shoulder path curvature, c)

3 3 trunk angle, and d) ratio which matches the vertical phase duration divided by the horizontal phase duration. 2.2 Experimental Setup and Data Acquisition The sensor was placed at a height of cm from the ground, and the chair was placed at a distance of m from the camera (see Fig. 1). The subject directly faced the Kinect sensor. The Kinect does not require any calibration; whenever a subject is in the camera s field of view his stick figure skeleton is automatically created. This skeleton information allows determining in real-time spatio temporal parameters which are relevant for the balance assessment. To extract parameters for balance assessment, we capture Kinect skeleton recordings of the TUG movement realized by the subject. Therefore, we use the data acquired to compute the parameters set. The Kinect sensor and its SDK produce a 3D virtual skeleton to establish the positions of 20 skeleton joints on a human form. For example, skeleton tracking determines where a user s head, hands, knees, and center of mass are. For each of these skeleton joints, X, Y, and Z values are reported. Kinect provides approximately 30 skeleton frames per second. Fig. 1. Overview of the experimental setup. 2.3 Sitting Posture Recognition Using the SVM Classifier The sitting posture corresponds to both the initial and final phases of the TUG test. In order to automatically recognize the sitting posture, we based on the Support Vector Machine applied to the depth positions of the skeleton joints. The Support Vector Machine (SVM) [10] is a powerful classification method that showed impressive classification performance in pattern recognition applications, including action recognition [11 14]. The linear SVM allows to separate

4 4 two sets of d-dimensional samples, i.e, it determines the hyperplane w.x+b where w, x and b are a weight vector, the vector of features and a constant, respectively. The purpose consists to find the optimal decision function that maximizes the distance between the nearest point x i and the hyperplane. Regarding the nonlinear SVM, it is used to transform the training data within a higher dimension into a new dimension. In our study, we use SVM with linear kernel to recognize the sitting posture (see Fig. 2). The head joint is considered as the origin of the skeleton so that our system is independent of the position of the person. A training set of skeleton joints is collected that corresponds to subjects performing the STS and BTS transfers, which include the sitting posture, and the walking action. The classification method takes input vectors of the depth positions of the 20 body joints as both training and testing data. The training data contain 8314 input vectors. A vector of the training set classification labels is given. It classifies the data in two classes: 1 for the positive class, and -1 for the negative class. Concerning the test data, each frame is treated separately, i.e, for each frame, a vector of test data is inputted and processed. Fig. 2. The proposed method of the sitting posture recognition. However, the sensor sometimes produces erroneous data. To overcome this problem, we compared the data of each frame to those of the ten frames that precede it according to some criteria such as the shoulder position, the trunk angle value and the distance between hip and knee. Thus, if the difference is greater than a certain threshold, for each criterion of comparison, the data of this frame will be removed. 2.4 Kinematic Parameters Extraction Process The calculation of the parameters starts (ends) when the sitting posture is recognized by means of the SVM classifier. Extracted features correspond to the kinematics of shoulder displacement during STS and BTS and the TUG duration. STS is composed of two main phases: the first phase is the forward flexion and the second, the extension phase, starts on lift-off of the buttocks from the

5 5 seat and ends when maximum hip, trunk and knee extension and maximum head flexion velocity are reached. BTS is characterized by a movement in the opposite direction than STS: downward and backward. The parameters are the following [1]: Shoulder path curvature. Shoulder paths during forward and backward displacements are similar and almost straight, therefore the curvature of path for upward and downward displacements were only calculated. Curvature is defined as follows: cur = D max (1) L where L corresponds to a straight line passing between the initial and the final position of shoulder displacement and D max means the maximal perpendicular distance measured from the actual path to the straight line. Fig. 3. Shoulder path during STS transfer. Curvature of paths is given by the ratio Dmax/L (adapted from Mourey et al [9]). Ratio. The ratio is computed as follows: ratio = D vph D hph (2) where D vph and D hph represent the vertical and the horizontal movement duration, respectively. Trunk angle. It refers to the angle θ between the trunk and the vertical plane passing through the center of mass of the body.

6 6 Movement duration. The total movement duration of shoulder motion, during STS, corresponds to the time interval between the moment when the shoulder depth component exceeded 8.5% of its initial position, and the moment when the head vertical component reaches or exceeds 94% of the size of the person. The thresholds are experimentally determined. In BTS, it is defined as the time interval between the moment when the shoulder vertical component drops its peak value, and the moment when the vertical components of the hips reach their minimum values and the trunk angle reaches its limit. TUG duration. TUG duration, measured in seconds, means the total time taken to perform all TUG tasks. It corresponds to the time interval between the moment when the forward phase starts and the moment when the backward phase ends. 3 Experiments and Results 3.1 Experimental protocol Ten elderly subjects and ten healthy young subjects participated in the present study. Among the elderly subjects, there is a person possessing a cane and a little mobility in the cervical spine, another subject with mild to moderate Alzheimer s disease. Subjects were asked to complete three trials of the TUG test: to rise from a chair, walk 3 m, cross a mark on the floor, turn around, walk back, and sit down. 3.2 Result Analysis Figure 4 shows the results of the trunk angle during STS and BTS for both of the healthy young subjects and the elderly subjects. During the STS transfer, averages of trunk angle for elderly and young subjects were ± 9.44 and ± 9.18 respectively. During the BTS transfer, they were ± and ± Thus, elderly subjects showed an altered trunk angle: angle in young group was more than twice angle in elderly group, and this can be explained by the motor strategies changes (less trunk flexion) associated to aging and frailty. Indeed, the trunk angle is important for the maintenance and control of posture: the trunk flexion allows shifting the center of mass towards the future basis of support in this dynamic balance task. It corresponds to a postural anticipation needed to perform the transfer, and directly allowed by the motor planning process. As regards the ratio, the values were between 1.70 and in the elderly group and between 1.03 and 4.18 in the young group during STS. This parameter is strongly linked with the trunk angles. An increase of the ratio can be interpreted as both an increase in vertical phase duration and a decrease in horizontal phase duration.

7 7 (a) STS Trunk Angle ( ) (b) BTS Trunk Angle ( ) Fig. 4. Distribution of the trunk angles during STS and BTS for 10 elderly subjects and 10 young subjects. Concerning the curvature, it was ± 0.13 in the elderly group and 0.07 ± 0.03 in the young group during STS. During BTS, it was 0.18 ± 0.11 in the elderly group and 0.12 ± 0.05 in the young group. For both groups, duration of STS was shorter than duration of BTS, except for a few cases, which is proved in the work of Manckoundia [9]. In addition, results showed that elderly subjects executed BTS motions significantly more slowly than young subjects. Concerning the TUG duration, for the young subjects, the values were between 8.21 s and s which refer to healthy subjects as the limit score is not exceeded. For the elderly subjects, the values were between s and s. Thus, there are several levels of functional abilities, including the frail elderly (TUG duration > 30 s). A one-way analysis of variance (one-way ANOVA) was performed whose factor is Group that included young and elderly. The parameters showing a main effect of the factor Group were the trunk angle in both STS and BTS transfers and the STS ratio. Results from the one-way ANOVA showed that there were significant differences between groups at STS trunk angle [F (1, 18) = 23.12, p = ], BTS trunk angle [F (1, 18) = 17.56, p = ] and STS ratio [F (1, 18) = 6.07, p = 0.024]. These parameters were significantly linked to functional abilities. The Table 1 presents the parameters values of the BTS transfers given by the two persons shown in Fig. 5. The first person (a) performed a sub-optimal BTS transfer and the second (b) performed a non-optimal BTS transfer. Based on

8 8 (a) Sub-optimal BTS transfer (b) Non-optimal BTS transfer Fig. 5. BTS transfer. these results, we can suggest that our system produces good measurement estimation specially regarding the trunk angle estimation. Furthermore, the TUG is widely used and well validated for the quantitative aspect (duration). However, this test requires the execution of various complex movements whose quality is not taken into consideration in the final result of the test. Our system allows to calculate parameters that help to evaluate the quality of these movements (ratio, angle). Table 1. The results obtained from the BTS transfers shown in Figure 5. Figure Ratio Curvature Trunk angle Duration a b Conclusions In this work, we have presented a novel movement analysis system for realtime balance assessment in the frail elderly, in home-based environment, using the Kinect sensor. Kinect allows to track the TUG test movements of subject

9 9 in real-time. Using the 3D virtual skeleton, our system analyzes the TUG test movement and nine spatio-temporal parameters are automatically extracted for STS and BTS transfers by 3D real-time video processing. Results of the preliminary experiments obtained with ten healthy young subjects and ten elderly subjects are consistent with previous studies [15, 16] and show good estimation with important precision. Moreover, our system points out that the trunk angle is an interesting parameter that allows to show the effect of aging. Currently, we are collecting more data from elderly subjects. Our future works consist to classify data so that the system can give the result that older people have balance problems or not and to verify the reliability of this classification. References 1. Hassani, A., Kubicki, A., Brost, V., Yang, F.: Preliminary study on the design of a low-cost movement analysis system: reliability measurement of timed up and go test. In: 9th International Conference on Computer Vision Theory and Applications (VISAPP). (2014) Stone, E., Skubic, M.: Evaluation of an inexpensive depth camera for passive in-home fall risk assessment. In: th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth). (2011) Clark, R.A., Pua, Y.H., Fortin, K., Ritchie, C., Webster, K.E., Denehy, L., Bryant, A.L.: Validity of the microsoft kinect for assessment of postural control. Gait & Posture 36 (2012) Chang, Y.J., Chen, S.F., Huang, J.D.: A kinect-based system for physical rehabilitation: A pilot study for young adults with motor disabilities. Research in Developmental Disabilities 32 (2011) Bonnechère, B., Jansen, B., Salvia, P., Bouzahouene, H., Omelina, L., Cornelis, J., Rooze, M., VAN SINT JAN, S.: What are the current limits of the kinect sensor? Proc. 9th Intl Conf. Disability, Virtual Reality & Associated Technologies (ICDVRAT) (2012) Chang, C.Y., Lange, B., Zhang, M., Koenig, S., Requejo, P., Somboon, N., Sawchuk, A., Rizzo, A.: Towards pervasive physical rehabilitation using microsoft kinect. In: th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth). (2012) Weiss, A., Hirshberg, D., Black, M.: Home 3D body scans from noisy image and range data. In: 2011 IEEE International Conference on Computer Vision (ICCV). (2011) Dubost, V., Beauchet, O., Manckoundia, P., Herrmann, F., Mourey, F.: Decreased trunk angular displacement during sitting down: an early feature of aging. Physical therapy 85 (2005) Manckoundia, P., Mourey, F., Pfitzenmeyer, P., Papaxanthis, C.: Comparison of motor strategies in sit-to-stand and back-to-sit motions between healthy and alzheimer s disease elderly subjects. Neuroscience 137 (2006) Cortes, C., Vapnik, V.: Support-vector networks. Machine Learning 20 (1995) Oshita, M., Matsunaga, T.: Automatic learning of gesture recognition model using SOM and SVM. In: Proceedings of the 6th International Conference on Advances in Visual Computing - Volume Part I. ISVC 10, Berlin, Heidelberg (2010)

10 Kim, T.K., Cipolla, R.: Gesture recognition under small sample size. In: Proceedings of the 8th Asian Conference on Computer Vision - Volume Part I. ACCV 07, Berlin, Heidelberg (2007) Danafar, S., Gheissari, N.: Action recognition for surveillance applications using optic flow and SVM. In: Proceedings of the 8th Asian Conference on Computer Vision - Volume Part II. ACCV 07, Berlin, Heidelberg (2007) Munsell, B.C., Temlyakov, A., Qu, C., Wang, S.: Person identification using fullbody motion and anthropometric biometrics from kinect videos. In: Proceedings of the 12th International Conference on Computer Vision - Volume Part III. ECCV 12, Berlin, Heidelberg (2012) Mourey, F., Grishin, A., d Athis, P., Pozzo, T., Stapley, P.: Standing up from a chair as a dynamic equilibrium task: a comparison between young and elderly subjects. The journals of gerontology. Series A, Biological sciences and medical sciences 55 (2000) B Mourey, F., Pozzo, T., Rouhier-Marcer, I., Didier, J.P.: A kinematic comparison between elderly and young subjects standing up from and sitting down in a chair. Age and ageing 27 (1998)