Available online at www.sciencedirect.com Procedia Engineering 41 (2012 ) 1441 1447 International Symposium on Robotics and Intelligent Sensors 2012 (IRIS 2012) Robot-based Intervention Program for Autistic Children with Humanoid Robot NAO: Initial Response in Stereotyped Behavior Luthffi Idzhar Ismail a, *, Syamimi Shamsudin a,b, Hanafiah Yussof a, Fazah Akhtar Hanapiah c and Nur Ismarrubie Zahari d a Center Of Excellence for Humanoid Robots & Bio-sensing (HuRoBs), Faculty of Mechanical Engineering, Universiti Teknologi MARA (UiTM), 40450 Shah Alam, Selangor, Malaysia b Faculty of Manufacturing Engineering, Universiti Teknikal Malaysia Melaka, Hang Tuah Jaya, 76100 Durian Tunggal, Melaka, Malaysia. c Faculty of Medicine,Universiti Teknologi MARA (UiTM) Shah Alam, 40000, Malaysia d Department of Mechanical and Manufacturing, Faculty of Engineering, Universiti Putra Malaysia, Serdang 43400, Malaysia Abstract The development and research on Human-Robot Interaction (HRI) between the humanoid robot and autistic children is new and innovative. This paper presents the initial response of stereotyped behavior in HRI between Humanoid Robot NAO and children with Autism Spectrum Disorder (ASD) during the Robot-based Intervention Program (RBIP) and normal class session. The presence of stereotyped behavior in children with ASD is being evaluated during the RBIP interaction and normal class session interaction. Humanoid Robot NAO is being utilized for the interaction in RBIP. The relationship between initial response of stereotyped behavior and the intelligence level of ASD children were analyzed during two interaction set-up, which were RBIP and normal classroom interaction, from which these findings are discussed in this paper. Our focus of discussion in this research is the initial response of autistic children exhibiting stereotyped behavior in RBIP and normal classroom session 2012 The Authors. Published by Elsevier Ltd. Selection and/or peer-review under responsibility of the Centre of Humanoid Robots and Bio-Sensor (HuRoBs), Faculty of Mechanical Engineering, Universiti Teknologi MARA. Open access under CC BY-NC-ND license. Keywords: Rehabilitation Robotics, Human-Robot Interaction, Humanoid Robot NAO, Autism Spectrum Disorder, Robot-based Intervention Program 1. Introduction Autism Spectrum Disorder (ASD) is a brain developmental disorder that affects the growth of social and communication skills of an individual [1]. Number of children that suffered from Autism Spectrum Disorder is increasing. The disorder is noticeable by social functioning that is characterized by impairment in the use of nonverbal behaviors, failure to develop proper peer relationships and lack of social and emotional connections [2]. Autism is a complex developmental disability that typically appears during the first three years of life and is a result of a neurological disorder that affects the normal operation of the brain, impacting development in the areas of social interaction and communication skills. Many researches and practitioner have developed interventions to teach play activities to children with autism served through early intervention [3]. New generation of therapy involving HRI and robotics elements is believe to produce a good improvement of behavior to the ASD children. Human-Robot Interaction (HRI) is a new and promising field that has been gaining an increasing amount of interest by researchers in the field of autonomous robotics, as well as those in human-computer interaction [4]. Service robot applications such as delivering hospital meals, mowing lawns, or vacuuming floors, bring autonomous robots into environments shared with people [5]. However, human-robot interaction functions in these tasks are still at minimal level. Human-robot social interaction plays an important role in spreading the use of the robot in human daily life. In the course of effective social interaction, robots will be able to perform many tasks in the human society. These tasks may include, but * Corresponding author. Luthffi Idzhar Ismail Tel: +60 3 5543 5161 ; fax: +60 3 5543 5160. E-mail address: Luthffi@yahoo.com 1877-7058 2012 Published by Elsevier Ltd. doi:10.1016/j.proeng.2012.07.333 Open access under CC BY-NC-ND license.
1442 Luthffi Idzhar Ismail et al. / Procedia Engineering 41 ( 2012 ) 1441 1447 not restricted to, handling various house duties, providing medical care for elderly people, assisting people with motor or cognitive disabilities, educational entertainment, personal assistance, giving directions at information points in public places, and many more [6]. Stereotyped and repetitive patterns of behavior, interests, and activities have been considered central to autistic disorder [7]. Based on study by Turner [8], stereotyped behavior is divided into lower- level behaviors characterized by repetition of movement including dyskinesias, repetitive manipulation of objects, stereotyped movements and repetitive forms of selfinjurious behavior, and higher-level behaviors including object attachments, insistence on sameness, repetitive language, and limited interests. These stereotyped behaviors are characterized typically by their cyclical and rhythmic environment [9] and have been acknowledged and studied in other human populations such as nursing home patients and drug addicts [10]. Humanoid Robot NAO is an autonomous humanoid robot developed by French Company named Aldebaran-Robotics [11]. This small size of humanoid robots is very smart and attractive for the interaction with ASD Children. It can be programmed autonomously and could perform verbal and non-verbal interaction during the Robot-based Intervention Program (RBIP). The Humanoid Robot NAO is being used in this RBIP because of his specification and promising capabilities [12]. 2. Robot-based Intervention Program Robot-based Intervention Program is an innovative intervention robotic program that will facilitate and augmentate the therapy for children with autism spectrum disorder. The intervention program utilizes the Humanoid Robot NAO as a main tool for the robot-based interaction. The Humanoid Robot NAO is 573.2mm height, 4.5kg weight and equip with 2 cameras, voice synthesis, voice recognition and powered by LiPo Battery. NAO has 25 degrees of freedom from his head to his feet and each joint is equipped with position sensor. The 2 VGA CMOS camera (640x480) and four microphones are equipped in the NAO s head. The inertial unit consists of 2 gyro meters and 3 accelerometers and 4 Force Sensitive Resistor under each foot of NAO makes NAO able to estimate his current state. Inertial, bumper, foot contact, sonar, tactile touch and battery are the list if sensor that equipped Robot NAO. 2.1. Experiment Setup and Layout (a)
Luthffi Idzhar Ismail et al. / Procedia Engineering 41 ( 2012 ) 1441 1447 1443 (b) (c) Fig. 1. Illustration of experiment setup for (a) RBIP Interaction Setup Area (b) layout of RBIP (c) Control Room during RBIP Figure 1(a) illustrates the RBIP Interaction Setup Area, (b) layout of RBIP and (c) Control Room during RBIP in National Autism Society of Malaysia (NASOM). The experiments is equipped with 1 unit of Humanoid Robot NAO, 2 units of HD Camera, 1 unit of mini camera mounted at NAO s chest, personal computer (PC), table and chairs. Selected 12 Autistic Children and their respective teacher is involve in the Robot-based Intervention Program. The interaction between the autistic children and the Humanoid Robot NAO is being observed and monitored by their respective teacher. The interaction time is limited to 14 minutes and 30 seconds for five different modules. There is a 30 second interval break between the modules. 2.2. Experiment Protocol: Module (a)
1444 Luthffi Idzhar Ismail et al. / Procedia Engineering 41 ( 2012 ) 1441 1447 (b) Fig. 2. Illustration of experiment protocol for (a) Robot-based Intervention Program (RBIP) (b) 5 Modules algorithm in choregraphe GUI Figure 2(a) illustrates the experiment protocol for Robot-based Intervention Program. There are 5 modules that has been designed and autonomously programmed to the Humanoid Robot NAO. Figure 2(b) illustrates the Module Algorithm that has been programmed in the Graphical User Interface (GUI) choregraphe and uploaded to the Humanoid Robot NAO for autonomously run. Module 1 is known as introductory rapport module. The first section begins with a 45-seconds period where there will be no movement, speech nor music executed by the robot. This module aims to observe the child s reaction to the presence of the NAO robot. The robot shall turn its head to the right, left and then back to facing the child. The head turning cycle will be repeated continuously for 90 seconds. This is followed by a 15-seconds break. Finally, in the third section the robot s eyes LED (2 x 8 LED of RGB full color) will be blinking continuously with random red, green and blue colors for 40 seconds. After a 10 seconds break, the blinking sequence starts again for another 40 seconds. Module 2 is known as NAO Talks. The module starts with the robot saying greeting the child to the Robot Based Intervention Program. NAO shall continue asking the questions such as; a)how are you? b)what is your name? c)where do you live? d)how old are you? e)are you happy? f)do you like going to school? g)what is your favourite colour? Module 3 is known as NAO Arm Movement. The humanoid robot NAO will extend its right hand halfway and wave to the child. After 10 seconds break, the robot waves with its left hand. This sequence repeats for a timed duration of 6 cycles. Moreover, module 4 is known as NAO Song Play and Eyes Blink. The humanoid robot NAO will be playing a children nursery rhyme; Twinkle twinkle little star through its audio loudspeakers. After a 30 seconds, NAO will play another children song; Humpty dumpty. Module 5 is known as NAO Song Play and Hand Movement. The humanoid robot NAO will repeat the hand movement sequence (as in Module 4) whilst simultaneously playing a children song; ABC through its audio loudspeakers continuously for 6 cycles. In the event of emergency, there are conditions to abort the procedures. These conditions include situations where the child becomes restless and uncooperative. And the second condition is if the child s parents were to request such action to abort the interaction, then the procedure is aborted.
Luthffi Idzhar Ismail et al. / Procedia Engineering 41 ( 2012 ) 1441 1447 1445 3. Classroom Behavior Observation The interaction between the child and their teacher is also being observed in their normal classroom during their school time. The observation is limited to 14 minutes 30 seconds of direct interaction between the teacher and autistic child, which is same duration of interaction as in RBIP. The evaluation for the observation is focuses on the stereotyped behavior response from the ASD Children. Figure 3 shows the illustration of interaction in the normal classroom. 4. Results and Discussion Fig. 3. Illustration of interaction in normal classroom for human-human interaction The initial response of Stereotyped Behavior is being observed and evaluated in the behavior score sheet for RBIP and classroom interaction is referred on widely used autism screening tools GARS-2 (Gilliam Autism Rating Scale-2nd Edition)[13]. There are many criteria that have been evaluated in the behavior score sheet under the subscale score of stereotyped behavior. The child is being evaluated on the fundamentals of: a) Avoids establishing eye contact, described as the child is looks away when eye contact is made by Humanoid Robot NAO. b) Stares at hands, objects, or items in the environment for at least 5 seconds. c) Flicks fingers rapidly in front of eyes for periods of 5 seconds or more. d) Whirls, turns in circle. e) Rocks back and forth while seated or standing. f) Makes rapid lunging, darting movements when moving from place to place. g) Flaps hands or fingers in front of face or at sides. h) Makes high pitch sounds or other vocalization for self-stimulation. The evaluation is based on given score direction. There are as follows: 0-Never Observed (evaluate) 1-Observed (evaluate) 8-Child does not speak at all during the module (not able to evaluate since the child did not speak) NA-Not Applicable The evaluation is carried out by certified occupational therapist, clinical physician and certified psychologist based on interaction in RBIP as well as interaction in normal class session. Table 1 shows the number of children with their respective FSIQ and their Stereotype Behavior Score in RBIP and Classroom.
1446 Luthffi Idzhar Ismail et al. / Procedia Engineering 41 ( 2012 ) 1441 1447 Table 1. Number of child with their respective FSIQ and their Stereotype Behavior Score in RBIP and Classroom Child Number Full Scale IQ Score Stereotype Behavior Score observed in Robot-based Intervention Program (RBIP) (%) Stereotype Behavior Score observed in Classroom (%) 1 52 12.5 0 2 78 0 25 3 66 7.5 12.5 4 96 2.5 25 5 44 12.5 25 6 107 2.5 25 Fig. 4. Illustration of initial response of stereotype behavior for Child Number VS Stereotyped Behavior Figure 4 shows the initial response of stereotype behavior for Robot-based Intervention Program (RBIP) and normal human-human interaction in class. In this figure, it is clearly illustrate that the stereotyped behavior were expressed more in the normal class interaction compared to the human-robot interaction in Robot-based Intervention Program. However, there was 1 case that exhibited more stereotype behavior in RBIP as compared to classroom interaction. This maybe due to the limitation of limited interaction time in Robot-based Intervention Program and unfamiliarity to the experiment setup as compared to the more familiar classroom set up. Fig. 5. Illustration of initial response of stereotype behavior for Full Scale IQ VS Stereotyped Behavior
Luthffi Idzhar Ismail et al. / Procedia Engineering 41 ( 2012 ) 1441 1447 1447 Moreover, figure 5 shows the relationship between score for Full-Scale Intelligence Quotient (FSIQ) score with the response of stereotyped behavior in the children with autism spectrum disorder. The pattern of the graphs shows that autistic children with Full Scale Intelligent Quotient (FSIQ) more than 65 percent exhibited lower stereotyped behavior score in Robot Based Intervention Program (RBIP) compared to the normal classroom interaction. The ASD children are considered mildly impaired (MI) for the FSIQ range score of 55 to 69 based on IQ test. So, for ASD children with a score that above 65, they are actually ready to be exposed in the robotics element in their therapy and they are more positive when interacting with the humanoid robot. 5. Conclusion As far as the initial response of stereotyped behavior is concern, the ASD children with higher score of Full-Scale Intelligence Quotient (FSIQ) response with less stereotyped behavior presence in Robot-based Intervention Program (RBIP) compared to the normal human-human interaction in normal classroom session. Based on the current results of Robot-based Intervention Program, the responses from the Children with Autism Spectrum Disorder is very promising and valuable for future work. Further repetitive exposure maybe needed for the lower FSIQ children to express favourable responses. Overall, children with ASD present more stereotype behavior in the classroom interaction and showed less stereotype behavior during the interaction in RBIP. Stereotyped behavior can be reduced and improvement can be made with more structured modules in Robot-based Intervention Program in the future work. Acknowledgements The authors gratefully acknowledge the Ministry of Higher Education Malaysia (MOHE), Universiti Teknologi MARA (UiTM) Shah Alam, Selangor and The National Autism Society of Malaysia (NASOM) for their support. A part of this project is supported by the Ministry of Higher Education Malaysia (MOHE) under the Fundamental Research Grant Scheme (FRGS) (600-RMI/FRGS 5/3/Fst/ (31/2011)). References [1] K. C. Welch, U. Lahiri, Z. Warren, N. Sarkar, "An Approach to the Design of Socially Acceptable Robots for Children with Autism Spectrum Disorders," International Journal of Social Robotics, vol. 2, pp. 391-403, 2010 [2] Dickstein-Fischer, L.; Alexander, E.; Xiaoan Yan; Hao Su; Harrington, K.; Fischer, G.S.;, "An affordable compact humanoid robot for autism spectrum disorder interventions in children," Engineering in Medicine and Biology Society,EMBC, 2011 Annual International Conference of the IEEE, vol., no., pp.5319-5322, Aug. 30 2011-Sept. 3 2011 [3] K. Lifter, J. Ellis, B. Cannon, and S. Anderson, Developmental specificity in targeting and teaching play activities to children with pervasive developmental disorders, Journal of Early Intervention, vol. 27, no. 4, p. 247, 2005. [4] C. Breazeal;, "Social interactions in HRI: the robot view," Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on, vol.34, no.2, pp.181-186, May 2004 [5] D. Wilkes, A. Alford, R. Pack, R. Rogers, R. Peters, and K. Kawamura, Toward socially intelligent service robots, Appl. Artif. Intell. J., vol. 12, pp. 729 766, 1997. [6] Salichs, M.A.; Barber, R.; Khamis, A.M.; Malfaz, M.; Gorostiza, J.F.; Pacheco, R.; Rivas, R.; Corrales, A.; Delgado, E.; Garcia, D.;, "Maggie: A Robotic Platform for Human-Robot Social Interaction," Robotics, Automation and Mechatronics, 2006 IEEE Conference on, vol., no., pp.1-7, 1-3 June 2006 [7] Watt. N, Wetherby.A, Barber, A, Morgan, L, Repetitive and Stereotyped Behaviors in Children with Autism Spectrum Disorders in the Second Year of Life, Journal of Autism and Developmental Disorders, Vol. 38, pp. 1518-1533, 2008 [8] Turner M. Annotation: Repetitive behavior in autism: A review of psychological research. Journal of Child Psychology and Psychiatry 1999;40:839 849. [9] Berkson, G. Repetitive stereotyped behaviors. American Journal of Mental Deficiency, Vol. 88, pp.239-246., 1983. [10] Lewis, M. H., Baumeister, A. A., & Mailman, R. B. A neurobiological alternative to the perceptual reinforcement hypothesis of stereotyped behavior: A commentary on Self-stimulatory behavior and perceptual reinforcement. Journal of Applied Behavior Analysis, Vol. 20, pp. 253-258. 1987 [11] D. Gouaillier, V. Hugel, P. Blazevic, C. Kilner, J. Monceaux, P. Lafourcade, B. Marnier, J. Serre, B. Maisonnier, "Mechatronic design of NAO humanoid," in IEEE Int. Conf. on Robotics and Automation, Kobe Int. Conf. Center, Kobe, Japan, May, 2009, pp. 769-774 [12] Ismail, L.; Shamsuddin, S.; Yussof, H.; Hashim, H.; Bahari, S.; Jaafar, A.; Zahari, I.;, "Face detection technique of Humanoid Robot NAO for application in robotic assistive therapy," Control System, Computing and Engineering (ICCSCE), 2011 IEEE International Conference on, vol., no., pp.517-521, 25-27 Nov. 2011 [13] James E. G, Gilliam Autism Rating Scale 2 nd Edition, Examiner s Manual.