Humanoid Robots for children with Autism: Experiences with Robosapien and NAO

Transcription

1 Humanoid Robots for children with Autism: Experiences with Robosapien and NAO Anas Tahir Department of Electrical Engineering KINDI Research Center, Qatar University Doha, Qatar Uvais Qidwai Department of Computer Science and Engineering KINDI Research Center, Qatar University Doha, Qatar Abstract Autism Spectrum Disorder (ASD) impacts many families in the gulf region, especially in Qatar (prevalence rate is not officially specified yet). This paper presents a brief review of different existing robotic or computer based solutions for therapy of Autistic children and focuses on the use of NAO and Robosapien robots in this context in Qatar. Many solutions are present in the market but most renowned and popular solutions are summarized in this paper in order to have insight of the work done for the therapy of Autistic patients. The main objective of this review paper is to give a quick and brief overview of existing technologies with emphasis on NAO and Robosapien robots in order for new researchers to break new grounds in to the field of Autistic therapy using robotic solutions. There are various solutions present in the market which can be interfaced or programmed in order to interact with the Autistic children by dexterous movement, speech and mobility functions. These assist in enhancing learning for such children, enabling them to adjust in the society around them, have more focus and increase attention span etc. Keywords Autism Spectrum Disorder (ASD); Human Robot Interface (HRI); NAO; Robosapiens; Humonoid robots. I. INTRODUCTION Autism Spectrum Disorders (ASD) are neurodevelopmental disorders in which a person have abnormal social interaction, impaired communication and language abilities and imitational behavior [15]. Children who suffer from ASD prefer to have same unchangeable routine on daily basis, as change annoys them. They can learn through imitation and joint attention methodologies as well as similar tools to help them overcome the barriers created between them and the society around them. Prevalence rate of ASD in US is 1 out of every 88 newborn being diagnosed as ASD affected [15]. This prevalence rate is really high in gulf region, and continuously increasing, therefore it is one of the problem Qatar is investing in. The study of exact prevalence rate is undergoing, but still there no exact figure to define it. There are many different schools being established in Qatar to help children with ASD. Following is an account of various therapeutic methods used in these schools for children with ASD. A. Diagnosis and Therapies There is no specific cause of ASD, it can be said that ASD is caused due to genetic or environmental factors. There are many different set number of diagnosis methods, such as neuro imaging and behavior analysis can be used to indicate that a child is suffering from ASD. Therapies, on the other hand, are tailored to child s need, as ASD is not same from one child to the other. Behavioral therapies have been set following standards, but they are mostly effective at younger age [21]. There is a therapy for Early and Intensive Behavioral Intervention (EIBI) in which for two years 40 hours per week of educational therapy is focused upon. Most of the behavioral therapies have fixed durations, some have to be implemented by the family members of the autistic child. There are many different therapies presented in the book [21], from which it is clear that these therapies are affective at young age therefore the focus group in this paper is children of the ages between 5 and 11 years. B. Different technologies used for Autism Therapy There are many different technologies being used to help autistic children to become independent in the future or in the long run in their lives. Most of the technologies in use for aiding the autistic children fall into one of the following technological groups [16]: 1) Virtual Reality applications or environment It includes technologies that depict the environment, as 3D room etc. 2) Dedicated applications or Interactive environment Includes tablets, computers and mobile phones, dedicated technologies to autistic child 3) Telehealth systems or telerehablitation systems Includes devices through which the health condition of the autistic child can be monitored 4) Robots or avatars Devices through which different behavioral therapies or diagnosis techniques be implemented It has been seen that the use of robots with autistic children has shown great improvements in their behavioral and learning related therapies. C. Role of Robots Robots used for autistic children have been shown to have made tremendous amount of difference in 4 different areas of therapy. Specifically, social learning, imitation skills, communication and interaction with society. Robots for autism have been focus of study since late 20 th century but recently it

2 made a boom due to technological advancement [13]. In paper [13] in (table 4), it shows different therapies are being devised using robots in different countries and what are results of them. It is clearly seen that robots have made lives of autistic children easier, and helped them tremendously. It is necessary to see the ethical issues related to the robots, paper [12] shows that robots are ethically accepted specially robots which are implemented with supervision. It can also be seen that the robots which are being developed for autistic children are safe by following some specific standards, and a child is more comfortable in interaction with a robots as it is similar to a human but at the same time it is also a toy for them. II. BACKGROUND Robots have been developed and created for really long since the first machine came out in early 20 th century. Human robot interface (HRI) have been one of the topics being studied and made it possible nowadays, as the survey on HRI clearly points out in the paper, how HRI have changed lives and flourished with time [11]. To help and contribute to the society and technological advancement, machines are widely used, and robots have played a really important role to aid. Paper [9] presented many different robots that are being used to help autistic children. It is a brief survey on robots roles and their benefits. Robots paper [9] presented are Bobus, CHARLIE, CPAC, Diskcat, FACE, HOAP-2, Infanoid, IROMEC, Jumbo, KASPER, Keepon, Kismet, Labo-I, Lego Mindstorm NTX, Maestro, NAO, Paro, Pekee, Roball, Robota, Tito, TREVOR, Troy. There are many new robots that emerged, as paper was written in 2013, like Popchilla, Zeno, Lucy [14], Rubi, Romibo and Robosapiens. These robots are mostly designed as Humanoids or animal shaped robots, as both of them have different effects on the autistic child. It can be clearly seen from the paper [10] in which a cat robot was used and on contrary in paper [8] Zeno a humanoid robot being used, showed different results and improvements. The focus of the paper is on the robots that are Humanoid which is explained in next section. III. WHY HUMANOID Robots which are used to aid autistic children, they are being studied and have been a main focus nowadays especially, as not every robot showed same improvement as other. Each robot used to help autistic children differed from groups of children itself so it cannot be generalized or interpreted as being the best solution in terms of improvements in autistic children. The shape, size and aesthetics of the robot made a lot of difference. The humanoid robots, being similar to a human, showed better results than other robots [4]. Autistic children appeared more comfortable in interacting with a robot than human, as in paper [1], it can be clearly seen from the experiment that eye gaze, testing behaviors and tests per initiation showed higher results than in case of human interaction with autistic children. The two humanoid robots that we have worked on; NAO and Robosapiens, are briefly explained. Figure 1: NAO robot Figure 2: Robosapien Figure 3: Child with ASD interacting with Robosapien Figure 4: Match the patch game

3 Figure 5: Get me out game Figure 6: Where is my color? game A. NAO NAO is a humanoid, a robot with 58cm height and weighs 4.3 kg, looks like a little human (Figure 1). It is fully programmable, open, autonomous, and has features to adapt many different needed behaviors for autistic children [2]. Therefore, it is widely used nowadays to develop courses or therapies for autistic children. Perhaps the only shortcoming in NAO robot is the cost since it is really expensive compared to other types of robots. In [3, 6, 7] it is shown how NAO robot is being used in order to develop a Lego therapy session for autistic children and their families. Using NAO in Lego therapy showed that long term trainings do not exhaust the autistic children and they can easily follow as well as improve with time. Specifically in [3], it is clearly shown that using NAO in Lego therapy which is implementation of single behavior, have shown better interaction results of the autistic child with his or her family and they have positive response to surroundings. NAO robot is being used in a number of applications all around the world including interactive activities with children, especially children with special needs like the children with autism. In this context we have started using NAO robot with children suffering from autism or having some aspects of autism spectrum disorder (ASD) with respect to certain functional activities. One group of children are being focused on with speech related interactive and social skill development activities while other group is more into dexterous and mobility aspect of activities. In both cases the same robot is controlled with a remote command station which is essentially a laptop or a tablet PC with various behaviors in accordance with the requirements of the therapy are pre-programmed into a remotely operated program. As such NAO robot is in contact with the children along with their teachers for the activities requiring certain behavioral display by the robot. For instance, in one of the activities the robot acts as teacher s assistant. In this activity the teacher is trying to bring the students to do an activity such as putting a basket on one side of the room. In this activity, there are several baskets in the middle of the room and they are asked to put them in order on one side of the room. Obviously these are low functioning children who are not conversing very well with the teacher in this activity. Once, the teacher has tried two three times with the student and a good response is not received, the NAO robot is given the same instruction and the behavior is controlled remotely as a result the robot starts to work and start walking in the middle of the room, picks up the small basket and then walks to the other side of the room to place it where it is needed. As a result of this activity the students got interested in the activity or behavior and the next time when the teacher ask them, better response was received and they also tried to do the same thing what the robot did. In a different experiment, the robot is made a part of the activity along with the students. Hence, the robot is not just a facilitator in this case but also an active member of this activity which involves certain exercise-like routines. The teacher is asking the student to do these routines but only some of them follow the teacher in the beginning while others are not. At the same time, the teacher asked the robot to start doing the same thing and surprisingly it has been observed that the students who were not interested in the activity after seeing the robot doing the same functions began taking interest and started doing mimicry of the action the robot is doing. Although the teacher is doing the same thing, but it was observed that the children were more attracted to the activity of the robot! In terms of speech related activities, NAO robot at the moment has been programmed in few languages of the world but more predominantly it is still in English language. In our initial experiments, these were not useful with the children who were Arabic speaking. An alternate approach was used initially and pre-recoded waveforms of specific conversational phrases were played remotely, so the child would perceive that they were coming from the robot. This included certain questions and answers, and several reinforcement phrases. For instance, the robot would show a card with a picture on it asking the students what is it?, so that the actual linguistic file of what is it? is stored previously as a wave file and then played through the program. Then when the child sees this from the robot it tries to give an answer. If the answer is wrong the robot will be directed to play another file which says to try again, or please try again or you are close, etc. Sentences like these were programmed and one by one they were played if the answer was wrong. However, if the answer was correct the positive reinforcement was done by the robot by showing the gestures of high-five or happiness or excitement with audio file saying very good, excellent, keep it up, that was great, etc. Sentences like these can encourage the child to

4 continue the activity and, of course, very positive results were observed in terms of increased attention span and prolonged learning phase. The third activity with NAO robot is related to a personal contact or interaction with the child. This usually involves medium to high functioning children who have social interaction problems especially eye contact, following instructions and fear of being close to someone. For this scenario, the robot is programmed to ask the child to hold its hand. As the child holds the hand, the robot provides the positive reinforcement in terms of verbal encouragement as well as gestures of moving up and down as the gesture of happiness ( Hooray ) or something similar to that. Then the robot would ask the child if he or she could take it to a predefined destination inside the classroom. The robot waits for the child to try to give a little bit of push to its hand and senses it by the touch sensors located in the upper side of the palm and near the wrist of the robot. Once it senses the pressure from the child in a specific net direction, the robot starts moving in that direction until it reaches the location or the child starts to give a different type of push to the hand. By understanding the direction of the pressure from the child s hand, the robot would follow the child in the direction the child wants to take it. This activity has not been tested with the low functioning children because of the delicate nature of the robot and it has been observed that children would like to hold the hand with the fingers only because of the fear of the social contact. As such, sometimes they like to pull the fingers strongly and the fingers can detach from the socket of the robotic arm. Hence, to be careful with this scenario it has been tested with only high and medium functioning children. Of course the results have been surprisingly very positive and the child who is usually not very familiar with holding a finger of someone and walking with that person, was able to do it quite independently and quite freely with NAO robot. B. Robosapien Robosapiens are also humanoid robots (Figure 2). They are similar to NAO but smaller in size and lighter [22]. They are not easily programmable as in case of NAO, but at least 50 times cheaper. Therefore, Robosapien has been used in many of the projects being conducted at Qatar University, especially in KINDI research lab. There are four publications [17-20] which talks about the project done with Robosapien to help autistic children. First, Robosapien was used to do a specific behavior action which an autistic child can adapt to using a remote control. Then later it was developed in to Microsoft Kinect based system through which it can mimic the movement of the autistic children helping them to improve interactions [23]. Figure 3 shows an autistic child, who is suffering from severe challenges of behavior control, interacting with the Robosapien. The activity helped him to learn control over himself and to develop some level of focus and involvement in the activity. In publication [17], the Robosapien was developed to do behaviors using electromyogram (EMG), by movement of limbs. In publication [18, 20], different games were devised in order to improve the IQ and understanding of surroundings of the autistic children. These games are Match the patch game (Figure 4), Get me out (Figure 5) and where is my color? (Figure 6). These are further explained later in the section. Robosapien robot is a very low cost toy available commercially. Primarily, it is an infrared based remotely controlled robot which has a very complicated 30 plus keys keyboard which has lot of functionality in it and lots of dexterous as well as audio performances are available. Also, mobility in a variety of different patterns, such as walking or dancing etc., are also provided. However, the remote control is quite difficult object for children with autism to handle. They are more comfortable with joysticks that are more commonly used in games, e.g. PS joystick. As part of our ongoing research, first thing we did was to remove the dependencies on the remote control by changing it or converting the control to a friendlier input device. To do this first we had to map all the IR commands from the remote into a file and then any type of input device can be used to select one of these commands and then the output is given to the robot by using an IR transmitter. It is a USB IR Transmit and Receive device (USB IRT) that is quite successful in sending these big blocks of IR commands in in near real time mode. Initially, this joystick was given to medium to high functioning children and the activity that was placed in front of them was that they were asked first of all to select a color from a circle of color that has different colors in it. Usually, this is not a very simple activity for them due to lack of focusing capabilities. But when the teacher showed them how the robot will react to it, then suddenly their interest developed and they became very cooperative in selecting a color. Then, there were different color patterns (stickers) on the ground in the shape of maple leaves. The color the child chose is the color they want the robot to go to, e.g. if the child has selected the red color, the child is told to direct the robot using the joystick to the first red leaf and from there to second red leaf and then to third. Interestingly the child did not understand it initially but when he started moving the robot the first time he liked it so much that for some minutes he did not pay any attention to the instructions being given by the teacher. Instead, he was very busy playing by trying many different experiments with the robot; experimenting with different types of moves forward, backwards, right, left, making it move, swing it from right and left, etc. All of these were very easy for the child because of the user friendly joystick that he was using and the activity became a lot more fun for the child. Then instructions were repeated and the child came back to the activity and he found that the control he has it is easy for him to move the robot from one color to second and third color, the reinforcement that was given to him is basically by all of his classmates who were jumping and encouraging him for the way he did it. Then it was repeated with different children with different level of functionality and a very positive response from all of them was observed. They really liked playing with it and we were able to achieve some level of enhancement in the objectives of pattern recognition, eye and hand coordination that usually takes weeks by the teachers to help the students learn. The Robosapien activity made it happen in few hours. Second activity with Robosapien was planned with very low functioning child who was basically not able to distinguish between stationary and non-stationary move, so a simple ipad program was built for the child, again, using the same platform of IR commands. The ipad provided the child with a screen having a floor mat design on it with four colors to choose from. This was the replica of a small arena built on the ground with

5 the Robosapien standing in the middle on a circle with four colored rectangles on the corners of this area with different colors. The child is asked to pick one color and the robot will move automatically to that color. The activity involvement of the child was very little and most of the command and control was done by the software on the robot. However, initially the child was not cooperative as for him it was not interesting, until the robot was turned ON, the minute it was turned ON, the attention span increased and the focus was completely directed towards the robot. When the teacher asked the child to select the color he cooperated and selected the color. When he saw the robot moving to specific rectangle that he selected made him more interested, and the next color he selected without even being instructed and basically the robot started to follow him. Third activity was done with an addition of an IO device, which was a Kinect camera. This was a project called as I Copy You designed in such a way that Robosapien was copying the movements of anyone, who are moving hands or walking, in front of a Kinect camera connected to the same laptop. As such this activity was performed only with very high functioning children because for low functioning children it was difficult to hold them steady in one place in front of the camera. I Copy you project was designed to take the skeletal vectors from Kinect camera, and copying these vector movements to various motors and various limbs of the robots accordingly. Hence, a mimicry action can be produced by the robot for anyone who is acting in front of the camera. Different activities were designed with this interface. The most interesting was an activity called Get me out. In this activity a small maze was built on the floor and a child was standing in front of the camera and the robot is at the starting point of the maze and child was taught three different hand movements. The child was directed to use them in order to control the movement of the robot, basically forward, right and left, so that child knows how to steer the robot in the maze and ultimately take it out from the maze. It turned out to be really exciting activity although little bit difficult, as it not that easy to act and get an immediate response, there is some latency involved in it due to programming overheads. However, the child it was tested with found it to be very interesting and participated in activity with full heart and the performance was found to be positive and encouraging. IV. DICUSSION AND CONCLUSION In light of the above experiences, it can be seen that in general, children with autism suffer with the social isolation issues that implants in them short comings like not being able to have eye contact or converse properly, to be able to concentrate for long period of time with different types of activities, etc. Specifically, for the parents who are not aware of the issue for their children at very early age, they tend to isolate them more rather than exposing them to people as part of their own psychological barriers. As a result, autistic children become more and more isolated and hermitized. On the other hand, when they are doing this isolation and putting selfrestrain on themselves, weather voluntarily or in-voluntarily, they tend to soothe their playing instincts with toys. It turns out that the toys which mimic humans in shape, can make different types of noises, sounds and can also make lot of flashing lights etc., are really the kind of gadgets they are happy with and they would like to play with them a lot. Combining these ideas objectively with the shortcomings of such children, in order to enhance their missing skills by using what they really like, the toys in the form of a robot, really compliments the idea of what should be taught and how should it be taught to these children. We have seen from various interactions with these children that the robotic solutions are very strong, very assertive and the children would not understand that they are being taught as being part of their therapy rather they would be playing with them and subconsciously learning things that they are missing due to their disorder. Based on our experiences, it is really recommended that robots used should be robust in their activity as the child s reaction could be damaging to the body of the robot as well. The robots should not have a lot of delays while performing actions because this is a very critical aspect that unnecessary delays in the performance of the action would make these children to lose interest, and when they lose interest it is very difficult to bring them back to the same therapy. On the same note however, robot should also be made as attractive as possible because they like to play with such toys more than the dull shaped robotic So which robot to choose and for what activity, this has to be a careful selection based on the guidance of the care giver of the children, the teachers or the nurses or the parents. Sometimes these children can recommend things as well. For instance, in Robosapien study, we have seen that one of the child (seven years old girl) really liked the robot but said it looks so strange and asked us if the robot can wear a pink color frock? We did that and she got more attached to the robot then before. Hence, such recommendations would help making the robot more ergonomic and friendly with the child so the barrier of fear and strangeness is gone and as a result the child would interact better with the robot. In terms of activities it is very subjective because it depends on the nature of the deficiency in the child. However, from the above mentioned experiences, it has been seen that activities related to mimicking, conversations and dexterous action, putting things from one point to other point or handing things work really well with robots as main or facilitating players. Probably a fourth category where combination of all of this exists, is where facial recognition can be introduced so it usually occurs in more complex and probably in more expensive robots where face recognition can be done. The robot can understand where the child is and how he or she is moving. Accordingly, the robot changes the behavior which really helps in developing the interaction or the understanding of interactive behavior of the child and how it is developing it further. However having said this, it should be understood as well that this is not the single solution for all kinds of scenarios. One robot might suite well with one child and may not suite well with the other. So in our opinion the decision of which one to use or how to use should lay with the caregiver, the teacher or the parents, especially in an institution it should be done at the collective level. These children initially, when they are low functioning, they are not that responsive to most of the therapies so at that level an individual robot might help them a lot. However, when they start socializing with it, perhaps it would be best to come up with the activity at group level rather than individual level, such as mimicking exercises or combine group activities in

6 which we share responsibilities and do things collectively one at a time. Hence, the decision of how to use it is still very subjective but it has very good positive potential to work and it was expected that this would be next line of technological tool needed in the therapy with the children with autism ACKNOWLEDGMENT We would like to thank KINDI research center to give us the platform to gain practical experience and to develop innovative solutions to help the future of our country. We would also like to acknowledge the support of Step by Step School Qatar, Shafallah Institute Qatar, and Al-awsaj academy Qatar for their collaborations. REFERENCES [1] A. Peca, R. Simut, S. Pintea, and B. Vanderborght, Are Children with ASD more Prone to Test the Intentions of the Robonova Robot Compared to a Human?, Int. J. Soc. Robot., vol. 7, no. 5, pp , [2] A. robotics, NAO H25 datasheet. [3] B. Huskens, A. Palmen, and M. Van Der Werff, Improving Collaborative Play Between Children with Autism Spectrum Disorders and Their Siblings : The Effectiveness of a Robot- Mediated Intervention Based on Lego Ò Therapy, J. Autism Dev. Disord., vol. 45, no. 11, pp , [4] D. Hutchison and J. C. Mitchell, A Survey on Storytelling with Robots, in 6th International Conference on E-learning and Games, 2011, pp [5] D. P. Miller, I. R. Nourbakhsh, and R. Siegwart, Robots for Education, in Springer handbook of Robotics, 2008, pp [6] E. I. Barakova, J. C. C. Gillesen, B. E. B. M. Huskens, and T. Lourens, End-user programming architecture facilitates the uptake of robots in social therapies, Rob. Auton. Syst., vol. 61, no. 7, pp , [7] E. I. Barakova, P. Bajracharya, M. Willemsen, T. Lourens, and B. Huskens, Long-term LEGO therapy with humanoid robot for children, Expert Syst., vol. 00, no. 00, [8] I. Ranatunga, M. Beltran, N. A. Torres, N. Bugnariu, R. M. Patterson, C. Garver, and D. O. Popa, Human-Robot Upper Body Gesture Imitation Analysis for Autism Spectrum Disorders, in 5th International Conference, ICSR, 2013, pp [9] J. J. Cabibihan, H. Javed, and M. Ang Jr., Why Robots? A Survey on the Roles and Benefits of Social Robots in the Therapy of Children with Autism, Int. J. Soc. Robot., vol. 5, no. 4, pp , [10] K. H. Mun, J. Y. Kwon, B. H. Lee, and J. S. Jung, Design Developing an Early Model of Cat Robot for the Use of Early Treatment of Children with Autism Spectrum Disorder ( ASD ), Int. J. Control Autom., vol. 7, no. 11, pp , [11] M. A. Goodrich and A. C. Schultz, Human Robot Interaction: A Survey, Found. Trends Human Computer Interact., vol. 1, no. 3, pp , [12] M. Coeckelbergh, C. Pop, R. Simut, A. Peca, S. Pintea, and D. David, A Survey of Expectations About the Role of Robots in Robot-Assisted Therapy for Children with ASD: Ethical Acceptability, Trust, Sociability, Appearance, and Attachment, Sci. Eng. Ethics, vol. 21, no. 6, pp. 1 19, [13] N. Aresti-bartolome and B. Garcia-zapirain, Technologies as Support Tools for Persons with Autistic Spectrum Disorder: A Systematic Review, Int. J. Environ. Res. Public Health, vol. 11, no. 8, pp , [14] R. Khosla, K. Nguyen, and M.-T. Chu, Socially Assistive Robot Enabled Home-Based Care for Supporting People with Autism, in Pacific Asia Confrence on Information Systems, 2015, p. Paper 12. [15] R. Meresca and L. de Magistris, Autism: What Is It?, in Translational Approaches to Autism Spectrum Disorder, 2015, pp [16] S. Boucenna, A. Narzisi, E. Tilmont, F. Muratori, G. Pioggia, and D. Cohen, Interactive technologies for autistic children: A review, Cognit. Comput., vol. 6, no. 4, pp , [17] U. Qidwai and M. Shakir, Electromyography Controlled Robotic Toys for Autistic Children, Int. J. Biotechnol. Bioeng., vol. 2, no. 1, [18] U. Qidwai, Using Course Projects to Tackle Social Problems : Use of Robots for children with ASD, in International forum on Engineering Education, 2015, pp [19] U. Qidwai, M. Shakir, and O. B. Connor, Robotic toys for autistic children: Innovative tools for teaching and treatment, in th IEEE GCC Conference and Exhibition, GCC 2013, 2013, pp [20] U. Qidwai, M. Shakir, W. Hakouz, and N. Musa, Wirelessly Controlled Mimicing Humanoid Robot, in Computer Systems and Applications (AICCSA), 2014, pp [21] V. M. Durand, Behavioral Therapies, in The Molecular Basis of Autism, 2015, pp [22] Wowee, Robosapien X datasheet. [23] Z. Zheng, E. M. Young, A. R. Swanson, A. S. Weitlauf, E. Zachary, and N. Sarkar, Robot - mediated Imitation Skill Training for Children with Autism, IEEE Trans. Neural Syst. Rehabil. Eng., 2015.