Evaluating the accuracy of markerless body tracking for digital ergonomic assessment using consumer electronics

Transcription

1 Evaluating the accuracy of markerless body tracking for digital ergonomic assessment using consumer electronics Dominik Bonin a, Sascha Wischniewski a a Unit Human Factors, Ergonomics, Federal Institute for Occupational Safety and Health (BAuA), Dortmund, NRW, GERMANY 1. Introduction Ergonomic assessment tools help to design ergonomic products as well as healthy and safe socio-technical work systems. Digital ergonomic assessment tools underwent significant development within the last years, but in most cases the procedure is still time consuming and expensive. The aim of the present study was to figure out if low budget markerless tracking, which is mainly used in gaming industry, might be suitable for ergonomic assessment. Since August 2014 the second generation of the Microsoft Kinect sensor (the v2 sensor) is available with improved tech specs compared to the first generation. As a basis for subsequent posture analysis, the accuracy of the v2 sensor is compared to the Kinect sensor under different circumstances like changing distance, light, clothes, view angles and velocity. Based on these findings, their suitability for ergonomic assessment is rated. 2. Methods In total 7 different trials were conducted, containing four static and two dynamic trials. For the static trials Microsoft Kinect sensor was compared to the v2 sensor. A foam dummy (height: 185cm) was used as reference object. Each trial was recorded with 30 frames per second, 1800 frames were captured for each trial. In the first trial the distance of the camera was changed (2.3m, 2.5m, 3m, 3.5m, 3.7m, 4m), the second trial contained different light conditions (1Lux, 690Lux, 2400Lux), the third trial was conducted with different angles (frontal plane of the dummy to z-axis of the Kinect Sensor: 0, 30, 60, 90 ) and within the last static trial different clothing (white skintight, blue workwear) was applied. As evaluation criteria the positional joint data of the right shoulder, right elbow and right hand were saved to a comma separated file. Mean values and standard deviations were examined with spreadsheet calculation afterwards. The dynamic trials were conducted by a human test person, performing two sets of maximal range elbow flexion/extension cycles. The distance to the sensor was set to 3m. The movement task was recorded with different velocities and different angles to the sensor (0, 60 ). To ensure the exact velocity of the movement task, a metronome was used, providing an acoustic signal for the reverse points of the movement (metronome beats at: 40bpm, 60bpm, 80bpm). The Kinect sensors were running simultaneously to a goniometer (SG110, Captiv System, TEA). The dynamic elbow angles of the Kinect sensors were calculated using 3D vector analysis in a self-written c# application, with a *.csv raw data export. Afterwards the elbow angles at the reverse point values were extracted. For a comparison between the systems, the data streams were synchronized. Afterwards, the average and standard deviations of min and max values were compared. As a reference, the range of motion (ROM) of the test person was measured manually by using a mechanical goniometer. 3. Results In the static trials the v2 sensor showed an improved accuracy over the Kinect, especially in extended ranges > 3m. The standard deviation (SD) range spread from 0.000m at 2.3m up to m at 3.7m. Kinect range was from SD 0,000m at 2.3m up to m at 3.7m (see tables 1-4). The v2 sensor was able to provide reliable data even at 4m distance. The largest improvements were found in standard deviations of the z-component, representing the depth measurement, especially with increasing angles of the dummy to the sensor (see table 3). 1

2 Table 1. Comparison of standard deviations, 3m distance, trial changing distances Kinect 1 SD[m], n=1800 frames Kinect 2 SD [m], n=1800 frames 2.3 m 2.5 m 3.0 m 3.5 m 3.7 m 2.3 m 2.5 m 3.0 m 3.5 m 3.7 m 4.0 m Shoulder-x Shoulder-y Shoulder-z Elbow-x Elbow-y Elbow-z Hand-x Hand-y Hand-z Table 2. Comparison of standard deviations, 3m distance, trial different light conditions 1 Lux 690 Lux 2400 Lux 1Lux 690 Lux 2400 Lux Shoulder-x Shoulder-x Shoulder-y Shoulder-y Shoulder-z Shoulder-z Elbow-x Elbow-x Elbow-y Elbow-y Elbow-z Elbow-z Hand-x Hand-x Hand-y Hand-y Hand-z Hand-z Table 3. Comparison of standard deviations, 3m distance, trial different angles Shoulder-x Shoulder-x Shoulder-y Shoulder-y Shoulder-z Shoulder-z Elbow-x Elbow-x Elbow-y Elbow-y Elbow-z Elbow-z Hand-x Hand-x Hand-y Hand-y Hand-z Hand-z

3 Table 4. Comparison of standard deviations, 3m distance, trial different clothing white skintight blue workwear white skintight blue workwear Shoulder-x Shoulder-x Shoulder-y Shoulder-y Shoulder-z Shoulder-z Elbow-x Elbow-x Elbow-y Elbow-y Elbow-z Elbow-z Hand-x Hand-x Hand-y Hand-y Hand-z Hand-z The dynamic trials showed a good body tracking ability from both sensors. Each sensor provided reproducible deviations from the Captiv goniometer with advantages of the v2 sensor in tracking stability. The local minima and maxima diverge among the three systems. Extension maxima were detected with less deviation compared to the min (flexion) values. The v2 sensor showed the best accuracy within the flexion values (see tables 5-6). The Captiv goniometer showed the biggest differences to the manual ROM measurement in flexion with deviations up to 28 (see figure 1) Degrees [ ] Frame Number Kinect [ ] Kinect2 [ ] Captiv [ ] Figure 1. Comparison between Microsoft Kinect, Kinect v2 sensor and Captiv goniometer. Example for elbow movement, trial 3m distance, 40bpm, 0 rotation 3

4 Table 5. Comparison of mean values and standard deviations at different velocities, trial 0 rotation 3m_40bpm_0 Extension[ ] SD [m] Flexion [ ] SD [m] Deviation Ex [%] Deviation Flex [%] Kinect Kinect Captiv m_60bpm_0 Kinect Kinect Captiv m_80bpm_0 Kinect Kinect Captiv Man. Gonio. 171,00 31,00 Table 6. Comparison of mean values and standard deviations, at different velocities, trial 60 rotation 3m_40bpm_60 Extension[ ] SD [m] Flexion [ ] SD [m] Deviation Ex [%] Deviation Flex [%] Kinect Kinect Captiv m_60bpm_60 Kinect Kinect Captiv m_80bpm_60 Kinect Kinect Captiv Man. Gonio Discussion As a basis for subsequent posture analysis, the static trials were analyzed to verify the measurement accuracy of the proprietary internal Kinect skeleton algorithm. Within the static trials the v2 sensor delivered the best overall accuracy, especially in extended ranges and different view angles. Both sensors delivered standard deviations in mm range which seemed to be stable enough for further investigations. Building on these achievements, the dynamic trials were conducted. Both sensors showed good body tracking abilities. The high deviations of the Captiv goniometer at the flexion reverse point can be explained with skin- and muscle movement. Some of the higher deviations of the Kinect sensors can be explained with unreliable performed motion tasks by the human test person. Nevertheless, the standard deviations compared between the three synchronized systems give a good indication about the tracking stability. Previous investigations about tracking accuracy of the Kinect 1 sensor demanded accuracy improvements for further investigations (e.g. Dutta 2012, Choppin & Whead 2012). The present study now showed improved accuracy from the v2 4

5 sensor over the Kinect 1 sensor, especially in dynamic motions. From technical view, the v2 sensor seems to be suitable as data supplier for ergonomic posture assessment tools, that work with optical body pose classification, e.g. the Ovako Working Posture Analysing System (Karhu et al. 1977) or similar. Regardless the above, there are still known problems with optical systems in general, e.g. occlusions, which have to be considered. 5. Summary The aim of the present study was to obtain reliable data about technical measurement accuracy of markerless body tracking under standardized conditions. The results show a good quality of data that seem to be accurate enough for posture analysis in ergonomic assessment processes. Based on these findings, further investigations are ongoing to test the accuracy and usability of such systems in realistic working environments with partially automated ergonomic posture classification and additional sensors to counteract incorrect measurements, due to optical masking effects, occlusions or artifacts. Acknowledgements The presented research was partly funded by the German Federal Ministry of Education and Research (BMBF, Funding number: 16SV6144) References Choppin, S. and J. Wheat. 2012, Markeless tracking of human movement using Microsoft Kinect. In Proceedings of the 30 International Conference on Biomechanics in Sports, Melbourne, Australia Dutta, T Evaluation of the Kinect sensor for 3-D kinematic measurement in the workplace. Applied Ergonomics, 43(4), Karhu, O., Kansi, P. and Kuorinka, I Correcting working postures in industry: a practical method for analysis. Applied Ergonomics, 8, Kinect for Windows sensor components and specifications, Accessed 26 March 2015, Kinect for Windows v2 sensor technical documentation, Accessed 26 March 2015, TEA T-Sens Gonio & Torsion datasheet, Accessed 26 March 2015, 5