Gesture Control in a Virtual Environment. Presenter: Zishuo Cheng (u ) Supervisors: Prof. Tom Gedeon and Mr. Martin Henschke

Transcription

1 Gesture Control in a Virtual Environment Presenter: Zishuo Cheng (u ) Supervisors: Prof. Tom Gedeon and Mr. Martin Henschke

2 2 Outline Background Motivation Methodology Result & Discussion Conclusion

3 3 Background NUIs are increasingly becoming an essential part of the contemporary HCI. The concept of NUI was developed by Steve Mann in 1990s [1]. NUI enables human-machine interaction via the people s common behaviours such as gesture, voice, facial expression, eye movement, etc.

4 4 Background (continued) Electromyography (EMG) signal recognition and vision-based gesture recognition plays important roles in NUI.

5 5 Motivation Identify the limitation and defects of EMG and Vision-based gesture recognition Explain the dominance of traditional input devices (i.e. keyboard and mouse) Summarise some lessons in HCI

6 6 Methodology Select two devices as the typical examples to be analysed

7 7 Methodology (continued) Assessment on general aspects Training time for novice users User-friendliness Assessment on gesture control in virtual environment Navigation Precise Manipulation Experiment 1 Experiment 2 Experiment 3

8 8 Experiment 1 Demo video for using MYO armband and Kinect sensor is shown to each subject. Ask the subject to perform all the gestures. Evaluate their degree of proficiency. - Only if the degree of proficiency is acceptable, the subject is allowed to do the next two experiments.

9 9 Data collected in Experiment 1 Training time Error rate Subjective impression of MYO armband and Kinect sensor

10 10 Experiment 2 Map each gesture in MYO and Kinect into the interaction events. Ask subjects to reach a specific destination in the virtual maze. Test the performance of MYO, Kinect, and keyboard.

11 11 Data collected in Experiment 2 Time Error rate Incorrect operation (human error) Recognition problem (device limitation) Times of changing gestures

12 12 Experiment 3 Map each gesture in MYO and Kinect into the interaction events. Ask subjects to grab the keys and use them to open the doors. Test the performance of MYO, Kinect, and mouse.

13 13 Data collected in Experiment 3 Time Error rate Incorrect operation (human error) Recognition problem (device limitation) Times of changing gestures Movement range of subject s arm Calculate Euler angles in 3D Euclidean space

14 14 Result & Discussion Experiment 1 MYO has much longer training time than Kinect. Subjects rate higher mark for Kinect.

15 Result & Discussion (continued) 15

16 Result & Discussion (continued) 16

17 17 Conclusion MYO is less user-friendly than Kinect, but performs better in precise manipulation. Kinect is much simpler than MYO, but it requires larger body movement. This leads to the low accuracy in precise manipulation. Traditional input devices are far more reliable than EMG and Vision-based devices.

18 18 Conclusion MYO is less user-friendly than Kinect, but performs better in precise manipulation. Kinect is much simpler than MYO, but it requires larger body movement. This leads to the low accuracy in precise manipulation. Traditional input devices are far more reliable than EMG and Vision-based devices. Any Questions?

19 19 Reference [1] Mann,S, 2001, Intelligent image processing. John Wiley & Sons, Inc. [2] Petrovski, A, Investigation of natural user interfaces and their application in gesture-driven human-computer interaction. In Information and Communication Technology, Electronics and Microelectronics (MIPRO), th International Convention on. Opatija, May IEEE. pg [3] Gary, K, Research Methods in Biomechanics. 2nd ed. Champaign, IL: Human Kinetics Publ. [4] Doumanoglou, A, Asteriadis, S, Alexiadis, D, Zarpalas, D and Daras, P, A Dataset of Kinect-based 3D Scans. IVMSP Workshop, 2013 IEEE 11th, /IVMSPW , pg [5] Microsoft Corporation. (2014) Kinect for Windows SDK. [Online]. Available: [6] Tao, G, Archambault, PS and Levin, MF, Evaluation of Kinect Skeletal Tracking in a Virtual Reality Rehabilitation System for Upper Limb Hemiparesis. Virtual Rehabilitation (ICVR), 2013 International Conference, /ICVR , pg

20 Thank You! 20