Accuracy and validity of Kinetisense joint measures for cardinal movements, compared to current experimental and clinical gold standards.

Size: px

Start display at page:

Download "Accuracy and validity of Kinetisense joint measures for cardinal movements, compared to current experimental and clinical gold standards."

Gertrude Bradley
5 years ago
Views:

1 Accuracy and validity of Kinetisense joint measures for cardinal movements, compared to current experimental and clinical gold standards. Prepared by Engineering and Human Performance Lab Department of Kinesiology and Physical Education University of Lethbridge Principal Investigator: Dr Jon Doan, P.Eng.

2 OVERVIEW New marker-less motion tracking systems use skeletal tracking, gesture recognition, and predictive modelling to provide high fidelity, low cost, plug and play motion capture in real-world contexts. Given these features, this technology has the potential to revolutionize clinical assessment, where subjective evaluations plus simple empirical measures have previously been the norm. The proposed study will examine a novel entry into this marketplace, namely the Kinetisense, specifically examining the validity of measures of basic anatomical postures from the new system against current experimental and clinical gold standards, namely three dimensional motion analysis from a passive marker infrared light system (VICON Peak) and analog goniometer respectively. Research questions: 1) validity of Kinetisense joint measures for cardinal movements in the sagittal and frontal planes, compared to current experimental and clinical gold standards; 2) repeatability of Kinetisense motion assessments. Methods: 24 healthy young adults performed 8 different actions, each to two different levels (specific normal range deflection, maximal deflection) and at one of two different

3 clinically relevant camera-subject distances, inside the shared calibration volume of the Kinetisense and VICON Peak motion capture systems. Data analysis and comparison: Bland-Altman agreement analysis will be used to compare perceived and maximum joint angles from the Kinetisense, the VICON Peak, and the clinical tools (goniometer, inclinometer).

4 Figures 1 through 7. Clinical comparisons. The primary purpose for the Kinetisense unit is as a clinical measurement tool, useful in measuring, charting, and demonstrating joint postures for clients. In that capacity, then, the Kinetisense (Kinect sensor plus Kinetisense software [as of Novemeber ]) has been validated in these comparisons against the current clinical standard, which is hand goniometer. Goniometry here was done by a registered clinician (athletic therapist), and sensor to participant distances were maintained at typical clinic ranges (1.5 m and 2.5 m). It should be noted that motions in the third dimension (specifically rotations in the transverse plane, such as neck rotation and trunk rotation) were not included in the study. Common clinical measurement of these postures typically involves some compromised position (ex. Fully prone for neck rotation measurement) that we had not included in our experimental protocol (thus not in our ethics approval) due to timing issues, and the need to reconfigure VICON markers for prone or trunk fully flexed postures. The Kinetisense is fully capable of capturing these rotations in normal (and gravity-loaded) upright posture, and our simple examination of this function as a stand-alone suggested Kinetisense provided quick recognition and good accuracy.

6 Example for Bland-Altman (B-A) analysis. B-A is a well-established technique for assessing agreement between clinical measurement methods (Bland & Altman, 1986.; subsequent citations in peer-reviewed literature) that avoids statistical issues with correlation, while providing a quick visual representation of the full assessment set. Paired assessments are plotted as the mean difference between paired measures (y-axis) and mean value across paired measures (x-axis). Mean plus/minus 2 standard deviation provides thresholds for clinically important differences that is, all measures between those thresholds are clinically similar.

7 Figure 1A. Neck lateral flexion to perceived 10º, measured in the frontal plane at 1.5 and 2.5 metres from the KINECT sensor. Kinetisense compares well with clinical measure for neck lateral flexion in both directions, with average difference between measures (MEAN) close to zero, no comparisons outside +/- 2 SD, and no difference between left and right measures.

9 Figure 1B. Neck lateral flexion to participant maximum deflection, measured in the frontal plane at 1.5 and 2.5 metres from the KINECT sensor. Continued good accuracy here only two out of 40 comparisons above +2 SD.

10 Figure 2A. Shoulder abduction to perceived 45º, measured in the frontal plane at 1.5 and 2.5 metres from the KINECT sensor. Kinetisense compares well with clinical measure for shoulder abduction, with average difference between measures (MEAN) close to zero, and three of 40 comparisons outside +/- 2 SD. Some asymmetry here (all inaccurate comparisons are on left) suggests a possible lighting

11 contrast issue in lab environment that disadvantaged larger motions (shoulder abduction) to left side.

12 Figure 2B. Shoulder abduction to participant maximum deflection, measured in the frontal plane at 1.5 and 2.5 metres from the KINECT sensor. Continued good accuracy here only two out of 40 comparisons outside +/- 2 SD, one each to the left and right. Mean difference between assessments less than 6% of mean average measure of deflection.

13 Figure 3A. Trunk lateral flexion to perceived 10º, measured in the frontal plane at 1.5 and 2.5 metres from the KINECT sensor. Kinetisense compares well with clinical measure for trunk lateral flexion, with average difference between measures (MEAN) close to zero (approximately 1 degree), and two of 40 comparisons outside +/- 2 SD. No asymmetry in errors here.

15 Figure 3B. Trunk lateral flexion to participant maximum deflection, measured in the frontal plane at 1.5 and 2.5 metres from the KINECT sensor. Continued good accuracy here one comparisons above +2 SD, mean difference between paired measures < 1/3 value of average joint measure.

16 Figure 4A. Hip abduction to perceived 45º, measured in the frontal plane at 1.5 or 2.5 metres from the KINECT sensor.

17 Kinetisense compares well with clinical measure for hip abduction, with average difference between measures (MEAN) extremely close to zero, and two of 40 comparisons outside +/- 2 SD.

18 Figure 4B. Hip abduction to participant maximum deflection, measured in the frontal plane at 1.5 or 2.5 metres from the sensor. Kinetisense compares well with clinical measure for hip abduction, with average difference between measures (MEAN) extremely close to zero, and one of 40 comparisons outside +/- 2 SD.

19 Figure 5A. Shoulder flexion to perceived 45 degrees, measured in the sagittal plane with participants 1.5 m or 2.5 m from the KINECT sensor. Kinetisense compares well with clinical measure for shoulder flexion, with average difference between 39 measurement pairs (MEAN) extremely close to zero. Single outlier in right flexion reflects a single failed recognition by the Kinetisense.

21 Figure 5B. Shoulder flexion to perceived 45 degrees, measured in the sagittal plane with participants 1.5 m or 2.5 m from the KINECT sensor (outlier removed). Same data as Figure 5A, with outlier removed. Three pairs outside accuracy threshold, but most comparisons centred around small mean difference (difference < 2% of average).

22 Figure 5C. Shoulder flexion to participant maximum, measured in the sagittal plane with participants 1.5 m or 2.5 m from the KINECT sensor. Kinetisense compares well with clinical measure for shoulder flexion, with average difference between 39 measurement pairs (MEAN) extremely close to zero. Two outliers reflect failed recognitions by the Kinetisense.

24 Figure 5D. Shoulder flexion to participant maximum, measured in the sagittal plane with participants 1.5 m or 2.5 m from the KINECT sensor (outliers removed). Same data as Figure 5C, with outlier removed. Three pairs outside accuracy threshold, but most comparisons centred around small mean difference (difference < 5% of average).

25 Figure 6A. Trunk forward flexion to 10 degrees, measured in the sagittal plane with participants 1.5 m or 2.5 m from the Kinect sensor. Single outlier here is failed recognition by the Kinetisense. Some smart anatomy in the programming that automatically rejected unrealistic values then provided an operator alert and prepared a re-measurement would be a possible simple solution for some postures.

27 Figure 6B. Trunk forward flexion to 10 degrees, measured in the sagittal plane with participants 1.5 m or 2.5 m from the KINECT sensor (outlier removed). Same data as Figure 6A, with outlier removed. Two pairs outside accuracy threshold, but most comparisons centred around small mean difference.

28 Figure 6C. Trunk forward flexion to maximum deflection, measured in the sagittal plane with participants 1.5 m or 2.5 m from the Kinect sensor. Three outliers, all on the right side. It is possible that lab lighting asymmetry influenced the contrast (and recognition) in this quadrant.

30 Figure 6D. Trunk forward flexion, measured in the sagittal plane with participants 1.5 m or 2.5 m from the KINECT sensor. Same data as Figure 6C, with outliers removed. One measurement pair outside accuracy threshold, but most comparisons centred around small mean difference.

31 Figure 7A. Knee flexion to perceived 135 degrees (contained angle), measured in the sagittal plane with participants 1.5 m or 2.5 m from the KINECT sensor. Four of forty measurement pairs outside the accuracy thresholds, but continued good accuracy between measures and mean measure difference < 5% average value.

33 Figure 7B. Knee flexion to participant minimum (contained angle), measured in the sagittal plane with participants 1.5 m or 2.5 m from the KINECT sensor. Small mean difference here, and good accuracy with only two measurement pairs outside accuracy threshold.

34 Summary. Clinical comparisons. The Kinetisense provides excellent accuracy compared to the common clinical measurement technique mean differences between the measurement techniques averaged <5% of the actual measurement across all 14 positions (7 postures at two magnitudes), and just 37 measurement pairs (~6%) fell outside the clinicallyrelevant accuracy thresholds established for the multiple measures. There were 7 Kinetisense measurement outliers (included in the 37 above), out of more than 550 total paired measurements. These sparse and spurious outliers could be reliably fixed through the inclusion of some logical anatomical limits (plus operator alert and recapture) in the software. This study does not make any examination of inter- and intra-rater reliability for the use of goniometry itself. Previous research would suggest this current clinical standard can have good intra- and inter-rater reliability (Carter et al., J HAND SURG), but that reliability is decreased with increasing practitioners across multiple sites working with real clients (La Stayo & Wheeler, PHYS THER). It could be suggested that Kinetisense will avoid these limitations, bringing an objectivity and consistency across multiple therapists and multiple sites as required.

35 Figures 8 through 14. Experimental comparisons. The Kinetisense unit is not an experimental motion capture tool, as the software does not support interval-based whole body capture, analysis, and export. Even without these functions, however, the system does operate on current best practices and technology in marker-less motion capture. In that interest, and to further establish measurement validity, the Kinetisense (Kinect sensor plus Kinetisense software [as of Novemeber ]) has been validated against a current experimental standard, namely passive marker infrared three dimensional motion capture using a VICON-Peak Motus system and conventional whole body passive marker set-up. Analysis in the experimental comparisons are restricted to 16 samples at each of seven joint motions, each to prescribed and maximal joint deflections, with sensor to participant distances at 2.5 m. The restriction to 2.5 m, and subsequent restriction to 16 rather than 40 subjects, was a delimitation of the VICON set-up because we were dual-tasking the VICON, we could not reconfigure the capture volume top make the 1.5 m Kinect camera distance work with the VICON cameras. In fact, Kinetisense was much quicker to set up and more flexible in use than the VICON unit, which requires specific calibration, highly fixed camera

36 positions (for the duration of any single capture session or experiment), and extensive post-processing.

37 Figure 8A. Neck lateral flexion to perceived 10 degrees, measured in the frontal plane with participants 2.5 m from the Kinect sensor.

38 Small mean difference here (~4% of average), and good accuracy with zero measurement pairs outside clinical accuracy threshold.

39 Figure 8B. Neck lateral flexion to maximal deflection, measured in the frontal plane with participants 2.5 m from the Kinect sensor. Continued good accuracy here - small mean difference (~10% of average), and zero measurement pairs outside clinical accuracy threshold. This comparison is our smallest number of paired measures (9 pairs).

40 Figure 9A. Shoulder abduction to perceived 45 degrees, measured in the frontal plane with participants 2.5 m from the Kinect sensor. Small mean difference here (~4% of average), and good accuracy with one of 18 measurement pairs outside clinical accuracy threshold.

42 Figure 9B. Shoulder abduction to maximum deflection, measured in the frontal plane with participants 2.5 m from the Kinect sensor. Continued good accuracy here - small mean difference (~5% of average), and two measurement pairs outside clinical accuracy threshold. Both extreme measures are from same participant, suggesting that VICON skin surface marker accuracy may be the issue with these two measure pairs.

43 Figure 10A. Trunk lateral flexion to perceived 10 degrees, measured in the frontal plane with participants 2.5 m from the Kinect sensor. Small mean difference here (~8% of average), and good accuracy with zero measurement pairs outside clinical accuracy threshold.

44 Figure 10B. Trunk lateral flexion to maximum deflection, measured in the frontal plane with participants 2.5 m from the Kinect sensor. Continued good accuracy here - small mean difference (~3% of average), and zero measurement pairs outside clinical accuracy threshold.

46 Figure 10A. Hip abduction to perceived 10 degrees, measured in the frontal plane with participants 2.5 m from the Kinect sensor. Small mean difference here (~12% of average), and good accuracy with zero measurement pairs outside clinical accuracy threshold.

47 Figure 10B. Hip abduction to maximal deflection, measured in the frontal plane with participants 2.5 m from the Kinect sensor. Both outliers here are failed recognition by the Kinetisense. Some smart anatomy in the programming that automatically rejected unrealistic values,then provided an operator alert and prepared a re-measurement would be a possible simple solution for some postures.

48 Figure 10C. Hip abduction to maximal deflection, measured in the frontal plane with participants 2.5 m from the Kinect sensor (outliers removed). Same data as Figure 10B, with outliers removed. Zero measurement pairs outside accuracy threshold, and most comparisons centred around small mean difference (about 12% of average measure).

49 Figure 11A. Shoulder flexion to perceived 45 degrees, measured in the sagittal plane with participants 2.5 m from the Kinect sensor. Single outlier here is failed recognition by the Kinetisense.

50 Figure 11B. Shoulder flexion to perceived 45 degrees, measured in the sagittal plane with participants 2.5 m from the Kinect sensor (outlier removed). Same data as Figure 11A, with outliers removed. Zero measurement pairs outside accuracy threshold, and most comparisons centred around small mean difference (less than 2% of average measure).

51 Figure 11C. Shoulder flexion to maximal deflection, measured in the sagittal plane with participants 2.5 m from the Kinect sensor. Small mean difference here (~1% of average), and good accuracy with one measurement pair outside clinical accuracy threshold. Possibly some asymmetry here, with all right measures greater for Kinetisense, all left measures (except clinically inaccurate) favouring VICON. This asymmetry could be a lab lighting issue, or a VICON camera distribution issue.

52 Figure 12A. Trunk forward flexion to perceived 10 degrees, measured in the sagittal plane with participants 2.5 m from the Kinect sensor. Small mean difference here (~4% of average), and good accuracy with zero measurement pairs outside clinical accuracy threshold.

53 Figure 12B. Trunk forward flexion to maximal deflection, measured in the sagittal plane with participants 2.5 m from the Kinect sensor. Continued good accuracy here - small mean difference (~5% of average), and one measurement pair out of 13 outside clinical accuracy threshold.

54 Figure 13A. Knee flexion to perceived 135 degrees, measured in the sagittal plane with participants 2.5 m from the Kinect sensor. Small mean difference here (<1% of average), and good accuracy with one measurement pair below the clinical accuracy threshold (suggesting an error in the VICON data point).

56 Figure 13B. Knee flexion to participant minimum (contained angle), measured in the sagittal plane with participants 2.5 m from the Kinect sensor. This is a difficult measurement for both systems because of the depth of the posture. Despite that, suitable clinical accuracy and just one pair outside threshold.

57 Summary. Experimental comparisons. Kinetisense measures are valid compared to VICON-Peak measures, based on Bland-Altman agreement analysis. Differences that do exist may be a function of different segmental models (skin surface for the VICON, simplified skeleton for the Kinetisense), different data capture methods (time-based three-dimensional position sample for the VICON, instantaneous angular interpolation for the Kinetisense), and real differences in measurement. Conclusion The Kinetisense appears to be a highly viable clinical tool, accurate in comparison to current clinical and experimental gold standards, and possessing numerous advantages (cost, set up and collection speed, inter-user reliability, rapid data recording and patient progress reporting, ease of use) over those gold standards as well.

KINETISENSE 3D CONCUSSION WHITE PAPER

KINETISENSE 3D CONCUSSION WHITE PAPER Concussion is a significant problem, whether it is received through a high impact sport, a car crash, or other means. Difficult in detection and proper management,