Volume 114 No. 7 2017, 111-121 ISSN: 1311-8080 (printed version); ISSN: 1314-3395 (on-line version) url: http://www.ijpam.eu ijpam.eu Hand Gesture Recognition In Real Time Using IR Sensor Rohith H R 1, Shiva Gowtham 2, Sharath Chandra A S 3 1,2,3R V College Of Engineering Bengaluru, India Electonics Department April 14,15-2017 Abstract Communication is a major challenge to people with hearing loss and people who are aphonic. Due to these disabilities there exists a communication gap between the hearing population and those with the disability. A wearable glove based system fitted with IR sensors can be used in bridging this gap in communication. The gloves are used to recognize the hand gestures of the disabled people and convert those gestures into meaningful messages in real time. With the amount of bend made by the fingers continuous data stream is obtained as output from Infrared (IR) sensor. IR sensors change the output voltage depending on the strength of the received signal. Change in bend is converted to change in the electrical voltage by IR sensors. The output from IR sensors is processed by a microcontroller and a corresponding message is displayed. The hardware components used in this project are Microcontroller (Norton), IR sensors, LCD display, switch, buzzer and power supply. The software components are Embedded C, Kiel U vision. The main purpose of this device is to bridge the gap in communication and help the people with disability to overcome the challenge and communicate more effectively. 111
Keywords IR sensor ;an machine interfaces; Gestures recognition, wearable sensors; motion detection. 1. Introduction According to Albert Mehrabian's book Silent Messages, based on extensive research, communication on a daily basis is made up of 93% non-verbal communication and only 7% verbal communication. But realistically non-verbal communication can vary anywhere between 60 to 90 percent. Therefore irrespective of having a disability or not, gestures play a major role in communication. People with hearing loss and people who are aphonic communicate with the help of gestures called Sign language. Sign language chiefly uses manual communication like hand signs and gestures to mediate a message. There are many different sign languages. Every country have their own sign language, and few countries have more than one. People without the disability don t feel the need to learn the sign language until a need arises, and hence majority of the people are not aware of communicating through sign language. So it becomes hard to communicate effectively with people having hearing loss. Hence there is a need for a system which can bridge this gap in communication. There are 2 approaches. which are used in recognizing. and interpreting the gestures. vision.based glove based systems. Fig1: Flex sensor attached to glove in the hand is used to perform Vision based systems provide more.non contact and natural solution. to obtain information. of its surroundings. and hand 112
gestures. Among many hand recognition phases the 3 main among them are tracking, detection and recognition. Due to its complexity, system. requires. an high level operating. algorithm., and hence making the design difficult. Even if the system is feasible an enclosure in which measures are taken to provide a environment that meets certain requirements, it lacks. processing. speed and accuracy.. Glove based gestures recognition systems is less complex and provides more accuracy. Glove based systems make use of sensors. fixed to the gloves. to capture the movement. of fingers.. This paper proposes a wearable glove based system that can interpret the hand gestures of people with disability and convert those gestures into meaningful messages that can be displayed on a screen. 2. Methodology Hand Talk [3] a glove equipped with IR sensor which translates finger and hand movement into words or sentence. Gloves equipped with IR sensor recognise hand movements and then transmit data to either phone or display on an LCD or it could either be input to a speaker. The IR sensor data are either converted to text or to voice. Person with no knowledge about sign language can read from the LCD display or listen from the cell phone what the dumb or deaf person is saying in sign language. The main advantage of Hand Talk design was its cost effectiveness, long life and simplicity to create. interactive glove which helps to improve life of deaf and dumb. Fig 2: Block diagram of Hand-Talk gloves[2] 113
To start with research, once we obtain a electrical voltage depending on hand movement. Different hand gestures are interpreted by finger. Components used in Hand Talk system [1] are: 1.Microcontroller 2 Motion sensor(ir Sensor )3. LCD display 4. Smart Phone 5. Speaker. Motion Sensors could be either an, flex sensor or an IR sensor.in a Hand Talk gloves equipped with accelerometer based on the amount of bent of the finger the signals are interpreted into words. Where as in flex sensor depending upon the change in resistance due to bent the signals are interpreted. Infrared sensor [2] is a that emits to sense things around it. IR measures heat of objects and also detect the motion. These types of sensors are capable of measuring only infrared.radiation,. In infrared sensor an IR LED could be used as emitter.and IR photodiode as detector. But in IR sensor the detector has to be sensitive to wavelength emitted by emiter When.IR light falls on the photodiode., Resistances and output voltages changes with change in magnitude infrared light received. Figure 3 explains [7] the working of IR sensor with no obstacle in front of it sender. continuously sends infrared rays and since there is no obstacle to reflect back the infrared rays, sensor do not detect infrared rays. When a gloves is equipped with IR sensor the emitter is placed in the back of the finger and detector of the sensor is placed at front or at the nails part of gloves. When the emitter of IR sensor sends infrared radiation it is received by the detector placed in front of gloves. In case the person is wearing the gloves and bends the finger to communicate with other person then the infrared radiation emitted would not be received the detector of the IR sensor equipped on the gloves. Fig 3: Working of IR sensor with no obstacle in front[7] And Figure 4 explains the working of IR sensor with obstacle in front of it. Sender continuously sends infrared rays and since and when there is an obstacle to reflect back the infrared rays, sensor detects infrared rays. 114
Figure 4: Working of IR sensor with no obstacle in front[7] In the context of this project,when a dumb/deaf person wears Hand Talk gloves and doesn t bend his fingers then there would be no obstacle to interrupt the infrared rays. But when the person bends his finger the infrared rays would not reach the receiver due to interference by obstacle. It is programmed in microcontroller such a way that when the receiver receives infrared rays it is read as binary bit 1 and when receiver do not receive infrared rays it is read as binary bit 0. Innovati s LCD 2x16 connections are very simple, and is used in many LCD applications it could be controlled Innovati s basic commander. LCD 2x16 has 2 lines where each line can display upto 16 characters. In this LCD cursor control command is used to display on the screen.[1] 3. Working Of System Hand Talk gloves is worn by deaf/dumb person. The IR sensors are fixed on each of the finger of the gloves. The IR sensors mounted on gloves can detect hand gesture made by t person wearing gloves. 5V supply is required for system to operate hence not much of a problem to worry about getting a shock. And then the IR sensors convert these movements ofand into electrical signal. A ADC is used which converts analog to digital signal. The voltage for every movement of hand is recorded and is programmed in such a way in microcontroller as a predefined value. The IR sensor is given as input to microcontroller(norton) to compare to predefined value. After the comparison by microcontroller, the voltage difference measured obtained would be within a fixed range as programmed and dumped in the microcontroller. 115
Fig 5:Working and flowchart of Hand-Talk gloves with IR sensor as switch[2] And in the microcontroller using Arduino software it would be programmed in such a way that for every voltage there would be a sentence that is to be displayed on LCD. Microcontroller(Norton) compares the voltage produced due to hand movement with predefined voltage and displays the sentence in that voltage range. 4. Output Software protous simulation is used LCD and keypad interfacing. When a finger is bent by a deaf/ dumb or a normal person then corresponding word stored in microcontroller for communication would be displayed in LCD so that the deaf/dumb people can convey their messages to others. Fig 6:LCD with keypad interfacing[5] The Compilation for the above design using keil software is shown in Figure 7 116
Fig 7:Compilation output of LCD With Keypad Interfacing using keil software In figure 8 it shows that for various bends of the IR sensor the various voltage will be produced which is measured using oscilloscope and given as input data for microcontroller. Figure 8: IR sensor output analysis for various bends If a deaf/dumb or normal person, wants to communicate some message with the normal person he/she has to bend a finger so that corresponding word or sentence would be displayed on LCD screen and the message will be conveyed to normal person when he reads it from LCD. From the Figure 9 we can see that when a finger is bent corresponding word or sentence stored in microcontroller ie I Need Doctor is displayed on LCD display. This output normal person can read the message in LCD. 117
Fig 9: LCD display displaying output based on finger bent The entire system of Handtalk gloves equipped with IR sensor. And IR sensor is connected to microcontroller which is programmed using the Arduino software to display words or sentence on the LCD display. Te whole system is shown in Figure 10. Fig 10 : Complete model of Hand Talk gloves equipped with IR sensor 5. Applications This Hand Talk gloves system [2] is cheap, cost efficient and portable. The system uses simple techniques.it helps deaf and dumb people in public sectors, working areas, marking areas for communicating with others[5]. This gloves plays major role in various fields such as Biometrics, Robotics, Musical instrument by replacing physical buttons with hand gestures, Automatic control in industries. It can also be used for computer gaming as a simulator. It is also loaded with touch sensors on the thumb, the fingers and the palm which allows the wearer to use the 118
keyboard. Another application can be found in the field of fire extinguishing by fire extinguishers when members of a team can t even see each other person, these gloves will be able to transmit signals through simple hand gestures. The Hand Talk gloves have symbols and icons on them that light up according to the signal received. 6. Conclusion Dumb and deaf person rely majorly on interpreters and sign language for communication. However, in everyday life dumb/deaf people find it difficult in finding interpreters for communication as they are not available everywhere, also high costs and also scheduling qualified interpreters. Also this gloves would help improving quality of life of dumb or deaf significantly. The data obtained from glove equipped with IR sensor was used to extract word for every ASL. System provides an identity and moral support to dumb and deaf people. It can be extended to display any number of words, which requires further modification of the gloves. The Hand Talk will be further extended by developing a smart phone app and speech recognition(voice apr9600) IC which would enable reliable communication between Hard of Hearing and the Visually Impaired people. 7. Acknowledgement Our sincere thanks to our guide Mr Kiran V and Mrs Rohini S Hallikar (Assistant Professor) Department of Electronics and Communication for all the help and support the project hand gesture recognition system using IR sensor from January 2017 to April 2017. 8. Reference [1] K Anetha, Rejina, Parvin J, PG Scholar, Dept. of Embedded Systems, Dr N G.P.,Institute, of Technology, Coimbatore, India, International Journal of Innovative Research in Computer and Comunication Engineering, (An ISO : 3297; 2007 Certified Organization) Vol no 2, : 3, July 2014 [2] A.A.Markad, A.L.Borkar, Student, Asst Professor,.Department of Electronics and, Telecommunication, M S S C E T., Jalna, Maharashtra,, International journal of Electrical and Electronic Engineers,Volume. No 8, Issue 01., Janaury-June 2016 119
[3] Ajinkya Raut,,Vineeta Singh,,Vikrant Rajput,,Ruchika Mahale PES MCOE, PUNE, The International Journal of,engineering and Science Volume No 1,,Issue 2,, Pages 19-25., 2012,.ISSN: 2319 1813,, ISBN: 2319 1805 [4] Angelo M. Sabatini,LauraDipietro,. Senior Member at IEEE, and Paol Dario., Fellow., IEEE transactions on systems.,man and cybernetics, part c:.applications.and reviews.,volume No.38,N) 4,July,2008 [5] J. Thilagavathy Assistant professor, A. Jeyapaul Murugan, Department of ECE Dr.Sivanthi Aditanar College of Engineering, International. journal Of engineering research and technology., ISSN:. 2378-0171, Volume No: 03 Issue 2, March : 2016 [6] J. Segen and S. Kumar, Human-computer Interaction using gesture recognition and 3D hand tracking, Proc. of ICIP 98, vol. 3, Chicago,Oct.1998, pp. 188-192 [7]D. Kim, S. Lee, and J. Paik, Active shape model-based gait recognition using infrared images,.international Journal of Signal Processing, Image Processing and Pattern Recognition, vol. 2, no.4, Dec..2009 [8]Z.Zhang., Z.Ren. and.j. Meng, ROBUST part-:based Hand Gesture Recognition. Using. Kinect. Sensor., in IEEE. Transactions. on multimedia, vol 15., no.15., pp/.1110-1120, August. 2013. [9] Kalpattu S Abhishek,Physics Department and material science,hong Kong uni,hong. Kong., 2016 IEEE InternationalConference. on Electron. Devices. and Solid. StateCircuits. [10]M. Elmahgiubi, M. Ennajar, N. Drawil, M. S. Elbuni,"Sign. Language Translator. and Gesture. Recognition",computer and Information. technology 201 Global. Summit. on Sousse,1-6, 2016 120
121
122