HAND GESTURE RECOGNITION FOR HUMAN COMPUTER INTERACTION

Transcription

1 e-issn Volume 2 Issue 5, May 2016 pp Scientific Journal Impact Factor : HAND GESTURE RECOGNITION FOR HUMAN COMPUTER INTERACTION KUNIKA S. BARAI 1, PROF. SANTHOSH BANOTH 2 1,2 WCOE, RTMNU, NAGPUR, MAHARASHTRA, INDIA Abstract Hand gesture is one of the best methods for non-verbal communication like sign language. It is difficult for deaf and dumb people to speak or communicate with normal people. In this paper we give one helping hand to the deaf people to interact easily with community or with machine by using speaking system available in it. In this paper we first recognize the gesture of hand and convert the signal into equivalent operation, stored into the programming. Gesture recognition remains a very challenging task in the field of computer vision and human computer interaction. The suggested solutions in the literature for sign language recognition are very expensive for day to day use. Though a lot of work has been done in this area previously but most of the approaches rely on intrusive hardware in the form of wired or colored gloves or are specific language/dialect dependent for accurate sign language interpretation. In our proposed system there are 4 modules: real time hand tracking, hand segmentation, feature extraction and gesture recognition. During the research available human computer interaction approaches in posture recognition were tested and evaluated. A series of image processing techniques with Hu-moment classification was identified as the best approach. Due to recent advances in sensing technologies, such as time-offlight and structured light cameras; there are new data sources available, which make hand gesture recognition more feasible. Keywords Gesture recognition I. INTRODUCTION Currently, communication between human and computer plays an important role in our daily life. We are always looking for more convenient ways of interaction to transfer data to the machines or commanding them faster and easier. The only means of communication method for the hearing impaired community is the use of sign language. The hearing impaired community has developed their own culture and methods to communicate among themselves and with ordinary person by using sign gestures. A person who can talk and hear properly cannot communicate with a mute person unless he is familiar with sign language. Lately, a lot of work in static and dynamic gesture and pose recognition appeared. Most of the approaches for sign language recognition can be roughly divided into two groups: pose estimation is performed and the parameters of the pose are used to determine a gesture; gesture recognition is performed directly on raw image. The idea of our project is to design a gesture recognition system that will automatically capture, recognize and translate the alphabets of Indian Sign Language into corresponding text and voice in a vision based setup. In our project we propose to recognize single handed as well as double handed gestures accurately with a single normal webcam using bare human hands. The aim of our project is to recognize the gestures with highest accuracy and in least possible time. To recognize the gestures, our system consists of 4 modules: Hand Tracking and Segmentation, Feature Extraction, Gesture Recognition, voice recognition, speaker implementation, Application All rights Reserved 241

2 II. LITRATURE REVIEW Attempts to recognize sign language automatically began to appear in the 90s. Many researchers are trying to develop automatic sign language recognition system in various sign languages. Various works have been carried out previously on various sign language recognition techniques. In [1] This paper discusses about continuous hand gesture recognition. It reports a robust and efficient hand tracking as well as segmentation algorithm where a new method, based on wearing glove on hand is utilized. We have also focused on another tracking algorithm, which is based on skin colour of the palm part of the hand i.e. free hand tracking. A comparative study between two tracking methods is presented in this paper. A finger tip can be segmented for proper tracking in spite of the full hand part. Waving goodbye is a gesture. Pressing a key on a keyboard is not a gesture because the motion of a finger on its way to hitting a key is neither observed nor significant. So the gloved hand is tracked and the value stored is active to recognition. In [2] To overcome this problem technology can act as an intermediate flexible medium for speech impaired people to communicate amongst themselves and with other individuals as well as to enhance their level of learning / education. The objective of this research is to identify a low cost, affordable method that can facilitate hearing and speech impaired people to communicate with the world in more comfortable way where they can easily get what they need from the society and also can contribute to the well-being of the society. Another expectation is to use the research outcome as a learning tool of sign language where learners can practice signs. In this paper Sign Language Recognition is done and then Contour matching is performed. As soon as contour is matched the recognition completed. Here Hu-moments and YCrCb color space is part of contour matching. In [3] In this paper, they present a face and gesture recognition based human-computer interaction (HCI) system using a single video camera. Different from the conventional communication methods between users and machines, they combine head pose and hand gesture to control the equipment. They can identify the position of the eyes and mouth, and use the facial center to estimate the pose of the head. Two new methods are presented in this paper: automatic gesture area segmentation and orientation normalization of the hand gesture. It is not mandatory for the user to keep gestures in upright position, the system segments and normalizes the gestures automatically. The experiment shows this method is very accurate with gesture recognition rate of 63.6%. The user can control multiple devices, including robots simultaneously through a wireless network. In[4] In this paper, they introduce a hand gesture recognition system to recognize the alphabets of Indian Sign Language. Cam shift method and Hue, Saturation, Intensity (HSV) color model are used for hand tracking and segmentation. For gesture recognition, Genetic Algorithm is used. We propose an easy-to-use and inexpensive approach to recognize single handed as well as double handed gestures accurately. This system can definitely help millions of deaf people to communicate with other normal people. III. PROPOSED SCHEME The Gesture Recognition System takes the input hand gestures through the in-built web camera. Hand tracking is done using Cam shift method. Then the Segmentation of hands is carried out All rights Reserved 242

3 using HSV color model. The segmented hand image is represented using certain features. These features are used for gesture recognition using the Genetic Algorithm which gives optimized results. The final result obtained is converted into corresponding text and voice. Our system consists of 4 modules: Hand Tracking, Segmentation, Feature Extraction and Gesture Recognition. Image Segmentation: During the images segmentation stage the skin color detection and region segmentation is carried out. The skin color detection techniques are broadly classified as cbcr color space, RGB color Space, HS (Hue, Saturation value) color space, Normalized RGB & HSV(Hue,Saturation Value). The RGB color space that defines skin region rule and gives the boundary of skin cluster is used in this work. Based on the values of the RGB (Red, Green and Blue) in the image frame the skin color is detected. We have observed that the R/G ratio stays within a narrow band of values for skin pixels, whereas it is much more variable for non-skin pixels. Therefore, we could use this ratio to decide whether a pixel is likely to belong to the hand region or not. In particular, we empirically observe that the following two thresholds successfully capture hand-like intensities: 1.05 < R / G < 4.00 The skin region detection algorithm is as follows: Step1: Acquire RGB image frame Step2: Separating R, G and B components. Step3: if R/G>=1.04 and R/G<4 then do Step4 else it is not skin. Step4: Convert the skin pixels into white and rest of them into black i.e. assign the value 255 to skin pixels and 0 to the rest of the pixels. Feature Extraction: In feature extraction several general purpose features are extracted and the relationship between features and classes is inferred by an appropriate classifier. We then find the Center of Region (COR) of the hand region as well the farthest point from the COR. Ideally the uninterrupted white portions of this signal correspond to the fingers or the wrist. By tracking the circle, we keep the track of the transition i.e. either from white-to-black or black-to-white. We store the co-ordinates of the transition points and for visual convenience we plot the points on the circle. The transition points are then joined serially in such a way that every point is connected with its neighboring points. Thus we get a bunch of line segments near the circle. Boundary Extracted Image Extremas and Convex All rights Reserved 243

4 Classification Of Gestures: International Journal of Current Trends in Engineering & Research (IJCTER) The mid points thus obtained are of immense importance. Each of these midpoints are joined with the centroid forming an individual line segment with the centroid. Take the wrist segment as the reference segment as wrist is the only part which is outstretched every time in every image. As observed each finger when outstretched properly and separately makes angle with the wrist. Thus we assigned a range of angle to every finger so as to detect which finger is outstretched on the basis of angle made by the unknown finger. So calculate the angle of each individual segment with respect to the wrist line. Depending on angle made the fingers are identified. While the number of midpoint plotted gives us the count of the fingers outstretched. Once the fingers are identified, the gesture made by the set of obtained fingers can also be recognized. Hence the gesture is said to be classified. Zero Padded Image Noise removed Image Image After Closing Boundary Extracted Image Speech Playback: To play the corresponding speech a database of pre-recorded speech tracks is made. As soon as the gesture is recognized corresponding speech track is fetched from the database and played in the MATLAB with the help of speakers. IV. CONCLUSIONS In this paper a real time hand gesture recognition system is proposed by means of Cam shift method, HSV color model and Genetic algorithm. The proposed gesture recognition system can handle different types of hand gestures in a common vision based platform. The system is suitable for both single handed and double handed gestures. The system makes use of bare hands for interacting with the computer and is inexpensive, therefore facilitating the deaf-dumb people to use it. The proposed system is effective in recognizing the alphabets of Indian Sign Language. This system can definitely help millions of deaf people to communicate with other normal people. REFERENCE [1] Dharani Mazumdar, Anjan Kumar Talukdar And Kandarpa Kumar Sarma Dept. of Electronics & Communication Technology, Gauhati University Guwahati-14, Assam, India Gloved and Free Hand Tracking based Hand Gesture Recognition, ICETACS [2] Matheesha Fernando, Janaka Wijayanayaka Low cost approach for Real Time Sign Language Recognition 2013 IEEE 8th International Conference on Industrial and Information Systems, ICIIS 2013, Aug , 2013, Sri Lanka. [3] Yo-Jen Tu, Chung-Chieh Kao, Huei-Yung Lin Department of Electrical Engineering, National Chung Cheng University, Chia-Yi 621, Taiwan, R.O.C. Human Computer Interaction Using Face and Gesture All rights Reserved 244

5 [4] Archana S. Ghotkar, Rucha Khatal, Sanjana Khupase, Surbhi Asati & Mithila Hadap Department of Computer Engineering, Pune Institute of Computer Technology, Pune, India International Conference on Computer Communication and Informatics (ICCCI -2012), Jan , 2012, Coimbatore, INDIA. Hand Gesture Recognition for Indian Sign Language [5] G. Tofi ghi, S.A. Monadjemi, and N. Ghasem-Aghaee. Rapid hand posture recognition using adaptive histogram template of skin and hand edge contour. In Machine Vision and Image Processing (MVlP), th Iranian, pages I - 5, oct [6] Eng-Jon Ong, Richard Bowden, "A Boosted Classifier Tree for Hand Shape Detection, pp.889, Sixth IEEE International Conference on Automatic Face and Gesture Recognition (FG'04), 2004 [7] W. Zeng and X. Lu, Region-based nonlocal means algorithm for noise removal, Electronics Letters, vol. 47, pp , [8] W. Zeng and X. Lu, A generalized DAMRF image modeling for superresulution of license plates, IEEE Trans. Intelli. Transp. Systems, vol. 13, pp , [9] W. L. Zeng and X. B. Lu, A robust variational appraoch to superresolution with nonlocal TV regularisation term, Imaging Science Journal, vol. 61, pp , All rights Reserved 245