Fall Recognition Approach Based on Human Skeleton Information

Transcription

1 2016 Sixth International Conference on Instrumentation & Measurement, Computer, Communication and Control Fall Recognition Approach Based on Human Skeleton Information LUO Kai Intelligent Information Processing Lab., Dept. of Computer Science & Technology Yanbian University Yanji, China JIN Xiao-feng Intelligent Information Processing Lab., Dept. of Computer Science & Technology Yanbian University Yanji, China Abstract In this paper, we present a fast approach for fall recognition. This approach according to human-body skeleton information which was obtained from Kinect sensor. First, following the falls defined by FICSIT, head and center joints, and their relative distance are considered as feature to describe the behavior. Second, applying the slide-window method and threshold for behavior action stage, motion feature vector was extracted. In the end, falls were trained and recognized by Optimal Levenberg-Marquardt BP classification algorithm. Experimental results show that the approach is high-efficiency, and rate of accuracy reaches 90%. Keywords-component; behavior recognition; fall recognition; Kinect; skeleton information I. INTRODUCTION Human behavior recognition is becoming one of the most active and extensive research topics in artificial intelligence and cognitive sciences. To face the population aging question in world, the significance of the aged daily behavior research becomes more important today [1, 2]. In daily life, accidental falls, especially for empty nester, without in time rescue may lead to serious consequences. The traditional methods on fall recognition can be divided into two categories: one is based on wearable sensor, the other is based on video [3, 4]. The method based on wearable sensor system is mature, but it is high costs and inconvenience. Videobased method, mounting one or several cameras to extract outline of human body, which cannot be met with various changes of illumination and view angle. In contrast with wearable sensors system, method of the human body skeleton information extracted from Kinect sensor is more effective and costless. Therefore, in recent years, human behavior analysis based on Kinect sensor has become a research hotspot. Such as DTW-based human body recognition method via skeleton information [5]. Gesture track recognition based on HMM [6]. ZHENG [7] proposed an approach for motion and expressions capture using Kinect. In this paper, we present a fast fall recognition method based on the human body skeleton information. In the start phase of fall, only two skeletal nodes (head and center of gravity) and distance between them are chosen to represent motion features, and then combined with BP neural network to further recognize fall behavior. Our study improves fall recognition accuracy and performance, the work is also valuable for empty nester daily life surveillance systems. II. FALL BEHAVIOR FEATURE EXTRACTION A. Fall behavior comparative analysis FICSIT (Frailty and Injuries Cooperative Studies of Intervention Techniques) defined falls as: an unexpected event in which the participants come to rest on the ground, floor, or lower level. Not including leaning on furniture or walls. Falls, therefore, can be regarded as a kind of behavior out of consciousness control, and it should emerge some obvious characteristics if comparing with daily life movements [8]. FICSIT pointed out that major features of falls are: Significant posture changes; Head descends fast, and pretty much straight down in some stages; Height difference between head and center of gravity will be decreased until body touch down on ground level. In this paper, with 20 skeletal nodes extracted from Kinect. On the basis of falls definition, significant changes on heights (head node and center of gravity node) and height difference between them were selected to describe fall behavior, and respectively marked as: H h, H c, H hc. Through the further analysis and comparison with lying, picking and sitting, H h and H c will drop sharply, height difference H hc will decrease while falls. Four sets of behaviors on picking, lying, sitting and falls were randomly selected, in contrast to the curve of H h, H c, H hc (see Fig.1). In Fig.1, the abscissa axis is frame, the ordinate axis is the amplitude, (a), and (b), (c) is H h, H c, H hc changing curve. Supported by Science & technology Department of Jilin Province of China under Grant No JC /16 $ IEEE DOI /IMCCC

2 For Tab.1, falls is apparently different with sitting and picking but similar to lying down. However, Fig.1 declares that H h, H c, H hc of lying down decline slower than falls, its slope or gradient are quiet different. In addition, the H h of the four behaviors show on same trend. To compare with [4] which used the head movement track to recognize falls, the feature proposed in this paper has the better separability. (a) H h changing curve (b) H c changing curve B. Confirm the Action stage Intercepting the action stage from the whole on-going behavior procedure needs to figure out the beginning and the end precisely. In order to remove noise, it is supposed to smooth the curves of H h, H c, H hc, then to compute their second order difference. Taking head node for example, Fig.2(a) presents second order difference curves, and throughout action occurring stage the curve is jittering fiercely [9]. This paper used slide-window method to pick up start point and end point in action stage. Fig.2 (b) is variance curve of Fig.2 (a). Start point corresponds to position on variance curve, of which value great than mean value in first time (denotes A in Fig.2 (b)). End point corresponds to maximum value on variance curve (denotes B in Fig.2 (b)). Extracted action stage consequence presents in Fig.2(c), in which the non-action stages are zeroed. C. The feature of falls Action stage features of falls, the H h, H c, H hc are sharp decrease and short duration, so the amplitude and duration can be measured by mean value of first-order difference d. 1 d h h N 1 N 1 ( i1 i1 i) Fig 1. Four different behaviors Hh Hc Hhc changing curve. Assumed that all activities schema is action stable recovery. Obviously, the action stage is faster than the other two stages. Therefore, consider the action stage as the key of the fall recognition. Tab.1 listed difference of the action stage in four behaviors. TABLE I. (c) H hc changing curve COMPARISON HH,HC,HHC OF FOUR DIFFERENT BEHAVIOR Falls Lying Sitting Picking H h H c H hc is decline, - is unchanged. where, N is the number of sustained frames during action stage. And first-order difference reflects the degree of rising or decreasing on curve. d<0 if curve is monotonic decreasing, or d>0 when monotonic increasing. By (1), the mean value of first-order difference can be extracted. For nodes head, center of gravity, and the relative distance between them, respectively marked with d H, d C and d HC. Therefore, feature vector F for an action w can be represented by (2). F d d d w w w w H C HC where, F w is the feature vector for wth action. 708

3 be taken into account in BP design [11]. Number of hidden nodes commonly depends on experience or trial-and-error method, which can be estimated by (3). h ioa (a) Second order difference curve (head node) where, h is the number of hidden nodes, i is the number of input nodes, o is the number of output nodes, a is a constant (from 1 to 10). Considering i = 3, o = 1 in this paper, so h value range in [3,12]. Optimal h was determined via a comparative experiment. In experiment, limits MMSE to 0.01 and number of training times less than Experimental results showed in Tab.II. TABLE II. COMPARISON HH,HC,HHC OF FOUR DIFFERENT BEHAVIOR Fig 2. Process of behavior initiation extraction. III. (b) Jitter in the window (c) Consequence of the action stage FALL BEHAVIOR RECOGNITION METHOD A. Parameter of BP neural network In this paper, BP neural network is applied to fall recognition classifier. BP neural network is a kind of multilayer feed-forward artificial neural network, which is mainly applied to resolve problems of approximation function, pattern classification, data compression and model prediction, and etc. It is one of the most widely applied approach in engineering field by far [10]. To improve performance, the BP network model should be designed reasonably according to the features of sample data. And parameters such as hidden nodes, initial weights, threshold and appropriate learning method also should h MMSE h MMSE As Tab.II shows, MMSE keeps stable after h=8, and h=10 is the best. If strictly constrained initial value of BP network weights and threshold to [-1, 1], the training process will be converged too fast for reaching gradient lower limit, and MMSE dose not meet with preset value. So NW initialization method is introduced into initialization for avoiding over fitting issue in training process. Focused content of standard BP network algorithm is to finding out optimal weights and the threshold to make MMSE decline along with gradient direction. But standard BP network has two shortages [12]: Slower convergence; Trapped into a local minimum. Improved BP algorithms typically include SCG, elastic BP and optimal Levenberg-Marquardt. SCG algorithm is suitable for large-scale of neural network. In this paper, i=3 and o=1, that mentioned above means small scale. Elastic BP algorithm, although converges rapidly, but its performance goes worse when MMSE decreasing. Optimal Levenberg-Marquardt algorithm is the fastest one on convergence, and highly precise [13]. The comparative results are shown in Tab.III. TABLE III. THREE LEARNING METHOD COMPARATIVE RESULTS (H=10) Learning method Training times MMSE SCG elastic BP Optimal L-M The comparative results show that, optimal Levenberg- Marquardt algorithm performed best, its MMSE reached when h=

4 B. Fall behavior recognition algorithm based on human skeleton information On the basis of analysis above, this paper presents fall behavior recognition algorithm based on human-body skeleton information, the algorithm is described as followed: Step1: Acquire human-body skeleton information from the Kinect sensor, and extract the H h, H c, H hc. Step2: Use method of second order difference and sliding windows to action stage determination. Step3: Compute first difference mean by (1), and use (2) to extract human behavior feature vector F. Step4: Use optimal L-M BP neural network to training and recognition processes, obtain classified result of fall behaviors. IV. EXPERIMENTAL RESULTS AND DISCUSSION All experiments were implemented in MATLAB 2014a. Sample set is acquired from 5 persons, and the set is consisted of 4 different behavior groups, 90 samples of fall, and 20 samples of lying, sitting and picking respectively. To prove the feature F validity and recognition accuracy, 2 experiments were designed and implemented. A. Verifying the efficiency of the feature vector In order to verify the efficiency of the feature vector discussed in (1). Every 5 samples were selected randomly from each behavior groups, and extract feature vector F. Tab.IV listed value of F for each single behavior. TABLE IV. COMPARISON F FEATURE VECTOR OF EACH BEHAVIOR behavior F 1 F 2 F 3 F 4 F 5 falls [-0.62, -0.67, -0.56] [-0.68, -0.57, -0.57] [-0.70, -0.67, -0.57] [-0.71, -0.58, -0.68] [-0.53, -0.64, -0.74] lying Sitting picking [-0.24, -0.34, -0.34] [-0.54, -0.09, -0.49] [-0.66, -0.40, -0.09] [-0.31, -0.40, -0.41] [-0.40, -0.19, -0.39] [-0.73, -0.52, -0.01] [-0.32, -0.47, -0.25] [-0.44, -0.08, -0.38] [-0.58, -0.52, -0.12] [-0.15, -0.39, -0.40] [-0.44, -0.27, -0.37] [-0.60, -0.44, -0.24] [-0.11, -0.36, -0.36] [-0.68, -0.25, -0.45] [-0.61, -0.48, -0.09] In Tab.IV, all of the fall behavior present exceeding 0.5 on absolute value for each component. But all of lying behavior are less than 0.5. In addition, d c components of sitting behavior are smaller than others, d hc components of picking behavior are smaller than others but d c components are greater than 0.5. So in other words, F feature distributed in similar regularity inner same behavior, and presented dissimilarity for various behavior, that is proved the feature vector F has strong ability to discriminate falls and other behavior. B. Fall behavior recognition experiment By using randomized and crossover operated training and testing sets selection way, keeps training set including 50 samples of falls, and every 12 samples of others remaining behaviors respectively. And keeps testing set with 40 samples of falls, every 8 samples from others. Recognition experiment result based on optimal L-M BP neural network is shown in Tab.V. TABLE V. BEHAVIORS RECOGNITION RESULT No. Testing set Recognize Accuracy Falls Others as Falls (%) Average accuracy 90.0 In Tab.V, accuracy ranged in 85% to 97.5%, average accuracy reached 90%. Misrecognition may be caused by few of acquired sample data does not reflect real behavior itself. Some of participators performed falls behavior under control of conscious self-protection. Seemingly slow-motion of fall behavior is pretty similar to lying. The feature of action stage can be more apparent, that well have the better result. V. CONCLUSIONS This paper present an approach of fall recognition based on the human-body skeleton information. According to the FICSIT s definition of the fall behavior, for fall behavior recognition only three features (head and center of gravity nodes, and distance between them) are taken into account. Applied optimal L-M BP algorithm for training and testing processes, and result of recognition is validity and accurate. In addition, constricting steps on less feature parameters and vertical direction in action stage, the approach performs robustly and efficiently. However, the extracted features only concerned two nodes and their relationship, which means the approach is limit to the few of behaviors recognition. ACKNOWLEDGMENT Our work was supported by Science & Technology Department of Jilin Province of China under Grant No JC. REFERENCES [1] B. Kwolek, M. Kepski, Fuzzy inference-based fall detection using kinect and body-worn accelerometer, Applied Soft Computing, vol. 40, pp , December [2] R.Haas, T. P. Haines, Twelve month follow up of a falls prevention program in older adults from diverse populations in Australia: A qualitative study, Archives of Gerontology and Geriatrics, vol. 58, no. 2, pp , April [3] LIDong, S. Liang, Design of fall detection device for elderly people based on accelerometer", Transducer and Microsystem Technologies, vol. 27, no. 9, 2008, pp [4] F. Yang, J. Xie, Y. Zhou, ZQ. Wang, Fall detection system based on head moving trajectory and 3D vision, Modern Electronics Technique, vol. 35, no. 2, pp , January [5] F. LIU, Human action recognition based on Depth Image, Computer Engineering, vol. 40,no. 8, pp , August

5 [6] Y. Zhang, S. Zhang, Y. Luo, XD. Xu, Gesture track recongition based on Kinect depth image information and its applications, Application Research of Computers, vol. 20, no. 9, pp , September [7] LG. Zheng, JL. Luo, Xu.Ge, Implementation on mocap system based on Kinect, Journal of Jilin University (Engineering and Technology Edition), vol. 43, pp , March [8] DM. Buchner, MC. Hornbrook, NG. Kutner, ME. Tinetti, MG. Ory, CD. Mulrow, et al., Development of the common data base for the FICSIT trials, Journal of the American Geriatrics Society, vol. 41, no. 3, 1993, pp [9] Y. Han, C. Sheng-Luen, Y. Jeng-Sheng, QJ. Chen, Real-time skeletonbased indoor activity recognition, Control Conference(CCC), nd Chinese IEEE, Xi an.china, pp , July [10] XL. Lu, X. Xu, Human activity recognition based on acceleration and HGA-BP neural network, Computer Engineering, vol. 41, no. 9, pp , September [11] QCLv, A poselet-based approach for fall detection, IT in Medicine and Education(ITME), 2011 International Symposium on IEEE, vol. 2 Cuangzhou. China, pp December [12] R. Cucchiara, C. Grana, A. Prati, R. Vezzani, Probabilistic posture classification for human-behavior analysis, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, vol. 35, no. 1, pp , January [13] BMF. Moller, A scaled conjugate gradient algorithm for fast supervised learning, Neural networks, vol. 6, no. 4, 1993, pp