International Journal of Science, Engineering and Technology Research (IJSETR), Volume 3, Issue 5, May 2014

Transcription

1 Automatic Emotion Recognizer for Low Functioning Persons with Autism in Virtual Reality Environment G.Aparna 1, S.Srinivasan 2 1 Department of Electronics and Communication, P.B. College of Engineering, Chennai. 2 Professor, Dept of Information and Technology, P.B. College of Engineering, Chennai. Abstract Autism spectrum disorders are a group of developmental disabilities can cause significant social communication and behavioral challenges over lifetime. The theory of causation of autism and the other autism spectrum disorder is incomplete but many number of persons suffering from this disorder. People with autism have issues in non-verbal communication particularly low functioning individual with autism. They might be completely non-verbal and cognitively impaired, lack of understanding and responding to non-verbal cues and communication. Several augmentative systems were developed for low functioning people but they act only as a learning environment and often failed to operate in conjunction with real world situations. This system must also require a manual intervention and assessment reports of an individual autistic person. So we proposed a new therapeutic system called automatic emotion recognizer that recognizes Virtual Reality based facial expressions there by finding different kinds of emotions i.e. neutral, smile, anger, sad, surprise of an autistic person. Index Terms autism spectrum disorder, facial expression, low functioning people, virtual reality. I. INTRODUCTION Autism Spectrum Disorder is highly variable neuro developmental disorder which is characterized by great difficulty in communicating with others, abnormal behavior patterns and forming relationships. People with ASD have deficits in three major areas: verbal and non-verbal communication, social awareness and interaction, imaginative play and cognitive inflexibility. Diagnosis of autism is based on mainly behavioral assessment, parental report and clinical report. In some cases, Diagnostician uses tools such as Childhood Autism Rating Scale (CARS) and Autism Diagnostic Observation Scale (ADOS) to measure the severity range of an individual autistic person.. Most of the autistic people face difficulties in communicating with others and also lacking in understanding social cues and convention. They are unable to properly express non-verbal communication and body language. Low Functioning People with Autism might be completely non-verbal, cognitively impaired; they does not communicate even with closest of the family members.these inabilities hinder them from understanding verbal and non-verbal communications, as well as reading human facial expressions effectively. The ability to identify and determine one s emotions can serve as an empowerment for the field of artificial intelligence and gave rise to smarter, more powerful machines that understands the intention of users. An intelligent machine with emotional awareness can achieve the shortcomings of autistic people. With that emotional awareness, the machine is capable of teaching and guiding autistic people on how to respond appropriately when the person that he or she is communicating with is expressing various emotions [1]. We developed an automatic emotion recognizer for low functioning persons with autism using Desktop VR tool. This tool breaks the dependency of an autistic person as such it does not need any manual intervention and also operate flexibly in conjunction with real world scenarios. II. EARLY INTERVENTON SYSTEM An innovative VR-based facial emotional expression presentation system was developed that allows monitoring of eye gaze and physiological signals related to emotion identification to explore new efficient therapeutic paradigms. The eye tracking and physiological data were analyzed to determine intergroup and intergroup variations of gaze and physiological patterns. Performance data, physiological signals and eye tracking indices indicated that there were differences in the way adolescents with ASD process and recognize emotional faces compared to their typically developing peers. However, VR systems applied in the context of autism therapy focus on performance or explicit user feedback as primary means of evaluation and thus lack adaptability Traditional behavioral intervention is not accessible to the vast majority of ASD population due to lack of trained therapists as well as intervention costs and often failed in identifying complex expressions as well as required more prompts and more time to facial emotional expression understanding tasks. It uses immersive head mounted displays (HMD) as VR tool that rated as heavy and discomfort. The main problem found in our existing system as, classification systems designed to output one emotion label per input utterance may perform poorly if the expressions cannot be well captured by a single emotional label and multiple algorithm need for finding the Human-emotion. Among the fundamental social impairments in the ASD population are challenges in appropriately recognizing and responding to facial expressions. Innovative technology promises alternative paradigm in increasing intervention accessibility that recognizes VR-based facial expressions recognition system in a synchronous manner. Innovative technology promises alternative or assistive therapeutic paradigms in increasing decreasing assessment efforts, increasing intervention accessibility, reducing the cost of treatment, promoting intervention and ultimately skill generalization. We believe that such ability will provide insight to the emotion recognition process of the children with ASD and eventually help in designing new intervention paradigms to address the emotion recognition vulnerabilities. III. DIAGNOSING TOOL: ADOS The Autism Diagnostic Observation Scale (ADOS) is a standardized, semi-structured which is a combination of the 1434

2 two earlier instruments called ICD-10 and DSM-IV diagnostic criteria used by professional for diagnosing autism. It assesses communication, social interaction and imaginative use of materials for individuals who may have an autism spectrum disorder. The ADOS consist of four modules, each based on language and age I- Preverbal / Single Words II- Phase Speech III- Fluent Speech, Child/Adolescent IV- Fluent Speech, Adolescent/ Adult Each module contains standard activities and materials that are presented by examiners in order to elicit behaviors that have been used for identifying and also diagnosis of an ASD at that age and language level i.e. eye contact, conversation, use of speech and language, unusual sensory interests and shared enjoyment. The aim of the activities is to structure the interactions so that the child (or) adult being assessed is sufficiently interacted those subjects want to participate socially. In 2002 lord, rutter, dilavore & risi reported ADOS manual, intraclasss correlations are as follows: test-retest reliability from.59 to.73 and interrater reliability from.82 to.93.cronbach s alphas for internal consistency were consistently highest for the communication-social interaction total score from.91 to.94 and lowest for stereotyped behaviors and restricted interests score (.63 to.65 for modules 2 &1 and.47 to.56 for modules 4 and 3). Each module can be administrated in minutes and notes are taken by the examiner during its administration. The overall ratings are completed immediately after the administration, which are then used to formulate a diagnosis through the use of the diagnostic algorithm provided for each module. Scoring made at the end of the module is similar across modules with some identical items, but others are relevant only for the module being used. In this study, their results indicated substantial interrater and test-retest reliability for individual items, excellent interrater reliability within domains and external internal consistency. We considered only two autistic samples under the age group of 31 and 34 with moderate autism. IV. SYSTEM DESIGN A. Non-Verbal Communication Low functioning individual with autism are completely non-verbal; they express their feelings and share their own emotions via facial expression, body postures, gestures and eye gaze. The purpose of this study was to conduct a systematic examination of nonverbal communicative behaviors using multiple perspectives (both trained observers and instructors familiar with children's idiosyncratic communication) and to explore instructors' responsively to these subtle but important cues. More specifically, the objectives in this study were as follows: Fig.1 Autistic Persons with different kinds of Facial Expression To compare the frequency of potentially communicative behaviors displayed by a group of students with autism, as identified by trained observers and familiar instructors; To identify the communicative functions perceived by familiar instructors; To determine the proportion of communicative behaviors to which instructors responded and determine/identify different kinds of facial expressions and emotions of an individual who is suffered from ASD.Further, a more elaborate approach to responsively coding, including a broader range of instructors' responses and sequential analysis, would provide a richer understanding of social interactions. B. Experimental Setup An experiment was performed in a laboratory with two rooms separated by one-way glass windows for caregiver observation. In the inner room, the subject sat in front of the task computer. The caregivers sat in the outside room. A therapist was present in the inner room to monitor the process. The task computer monitor was also routed to the outer room for caregiver observation during training session. The session was video recorded for the whole duration of participation. C. Architectural Design Initially an image is captured by web camera; face detection is done using skin tone detection by finding skin-colored pixels and regions in an image then converting the image into the HSV color space and classifies to either skin or non-skin those results in much stretched skin color cluster image. The face clipped image takes as input for feature extraction that reduces dimensionality as well as extracts both transient and intransient features i.e. eyes, eyebrows and mouth to perform the desired task. 1435

3 Camera Skin Tone Detection Extraction Filtering Selection Set Analysis Face Expression Fig. 2 VR-based Automatic Emotion Recognizer System In this module Local Binary Pattern (LBP) ) exploits textual nature of human face as well as relationship between component features via eyes, nose and mouth to detect face patterns and reduce the dimension of the feature space.. After applying a textual descriptor, facial features become darker. For expression analysis, intransient features are selected for the final emotion-specific feature set. This feature set was reduced using information gain on per emotion class basics and permit as ranking of the features was implemented using modified fuzzy C means clustering. Then filtering has been applied to find the edges of intransient features with the help Canny s edge detector. Filtered feature set is analyzed and classified to maintain trained dataset.finally, VR-based facial expressions are recognized there by emotions are identified for accessing and controlling in the modern environment. V. SYSTEM IMPLEMENTATION Accessing & Controlling Process When user login has been done in face emotion detection system, an image of autistic person is taken as input by clicking browse button Fig. 4 Skin Tone Analysis After recognizing a given image is human then connected button is clicked to apply RGB color space which is used to isolate the presence of arms, face and gestures as well as eliminate the illumination condition to best extent. Though it is a face emotion detection system, remove regions that are unlikely to represent faces and then face acquisition is done by examining common features on face. Fig. 5 Applying RGB color space Apply Local Binary Pattern by clicking binary button. Face image can be viewed as texture pattern it provides powerful image representation and feature recognition for further image processing Fig. 3 Human Detection After that by clicking skin button, a skin detector typically transforms image in to skin or non- skin pixel and then skin classifier is used to label the pixels for decision boundary to detect whether it is an human or not. Fig.6 Extraction By clicking a face button, the end result of extraction task is shown called feature vector that contains both transient and intransient features. 1436

4 Fig.7 Facial Analysis After feature extraction, selection helps to reduce the feature vector by removing irrevalant, redundant and noisy features. It selects only intransient features by clicking eye-lip button.. Fig.10 Trained Dataset By clicking emotion button, Automatic emotion recognizer system automatically detects frontal faces from the captured image and codes them with respect to 5 dimensions in real time: neutral, anger, smile, sadness, surprise. Emotions are analyzed from training dataset based on classification technique called decision tree. Fig. 8 Selection using Clustering Algorithm Canny s filtering is applied to detect edges of intransient features.these edges are to be maintained in an edge pixel database for further processing. Fig. 11 Emotion Detection Fig. 9 filtering using Canny s edge detector After feature filtering, trained dataset has been created for each autistic person according to 5 different kinds of emotions. In the "Person table, it stores the name of autistic people and their index of 5 kinds of emotion. In the Position table, for each index, there are 6 control points for lip curve, 5 control points for left eye curve, 6 control points for right Bezier curve. VI. CONCLUSION We have developed a VR-based automatic emotion recognizer system that was able to collect facial features data while the subjects were involved in an emotion recognition tasks. Specifically, we developed controllable levels of facial expressions of emotion based on long standing research documenting certain universal expression patterns as well as a desktop virtual reality presentation to avoid issues and sensitivities individuals with ASD.This system must be more useful for low functioning individual with autism and flexibly operate in real world scenarios. Mainly, it reduces assessment efforts such as clinical reports, parental feedback, psychological testing etc. ACKNOWLEDGMENT We thank Aadhuraa Special School, Kanchipuram for providing support for this work 1437

5 REFERENCES [1] Emily mower, A framework for automatic human emotion classification using emotion profiles,ieee transactions on audio, speech and language processing, vol-19, pp , July [2] H.C.Vijaylakshmi and S.patil kulkurni, face detection for skin tone images using wavelet and texture features, TECHNIA international journal of computing science and technologies, pp ,jan [3] Ihsan ullah,muhammad hussain,ghulam Muhammad,hatim Aboalsamh, George Bebis, and Anwar M. Mirza, gender recognition from facial images with local WLD descriptor, national plans for science and technology, pp ,2012. [4] Krishna Mohan Kediri, Abas said, and M.Yuntis, emotion detection using sub-image based features through human facial expressions, international conference on computer and information science, pp , [5] Maringanti hima bindu, Priya gupta, and U.S.Tiwary, cognitive model based emotion recognition from facial expressions for live human computer interaction, proceedings of the IEEE symposium on computational in image and signal processing, pp , 2007 [6] Ming-chuan hung and don-ling yang, An efficient fuzzy c means clustering algorithm, information science, pp ,2001 [7] Ryszard S. choras, image feature extraction techniques and their applications for CBIR and biometrics system, international journal of biology and biomedical engineering, pp 6-16, 2007 [8] Teik toe, shi-min, siu-yeung and yok-yen, emotional advisor to help children with autism in social communication, intelligent decision technologies, vol-3, pp , [9] Vasantha.M, DR. V. subbiah bharathi, R.dhamodharan, medical image feature extraction, selection and classification, international journal of engineering science and technology, vol 2(6), pp , [10] Vineet saini, Rajnish garg, A comparative analysis on edge detection techniques used in image processing, IOSR journal of electronics and communication engineering (IOSRJECE), vol 1, pp 56-59, June