An Ethological and Emotional Basis for Human-Robot Interaction

An Ethological and Emotional Basis fo Human-Robot Inteaction Ronald C. Akin*, Masahio Fujita**, Tsuyoshi Takagi**, Rika Hasegawa** Abstact This pape pesents the ole of ethological and emotional models as the basis fo an achitectue in suppot of entetainment obotic systems. Specific examples fo Sony s AIBO ae pesented as well as extensions elated to a new humanoid obot, SDR. I. INTRODUCTION Human-obot inteaction is of citical impotance in the entetainment obotics secto. In ode to poduce a desiable end poduct that can be enjoyed ove extended peiods of time, it is essential that an undestanding of not only obotics but also human psychology be bought to bea. In this pape we descibe two aspects of a softwae achitectue that addesses seveal of the fundamental needs posed by this domain: 1. Incopoation of high-fidelity ethological models of behavio as a basis fo poviding the ability fo people to elate in pedictable ways to a obotic atifact. 2. Geneation of motivational behavio (e.g., emotions) that suppots human conceptions of living ceatues, and thus encouages a natual bonding between the human and the obotic atifact. Figue 1 shows the ange of poducts that Sony cuently poduces fo the entetainment obotic secto. They include vaious vesions of dog-like obots (AIBOs) and the newe humanoid obot (SDR). Fotunately the entetainment obotics domain is highly toleant of outside-the-nom behavio and pefomance as it does not equie high pecision no epeatability as equied fo moe standad obotic applications [1]. Ethology efes to the study of animals in thei natual setting and was lagely founded in the ealy 1900s by Loenz [2] and Tinbegen [3]. Ou wok seeks to extact fom obsevational behavio (not neuoscientific models) suitable desciptions of animal activity that can be effectively mapped onto obotic systems to povide the appeaance of life-like activity. Studies of the manifestation of emotions in humans and thei simila occuence as motivational behavio in animals can also povide suppot fo effective inteactivity between a obot and a human [4,5]. By incopoating aspects of *College of Computing, Geogia Tech, Atlanta, GA, U.S.A. **Sony Digital Ceatues Laboatoy, Kitashinagawa, Tokyo, Japan emotional and instinctive behavio into a obotic achitectue we contend that a geate ability to elate to the end-use is povided. Figue 1. Sony's Entetainment Robots (Top and Lowe Left) AIBO Vaiants (Bottom Right) SDR II. ETHOLOGICAL BASIS The study of canine behavio has povided fetile gound fo the ceation of a novel achitectue fo AIBO. In paticula, the extensive body of eseach conducted by Scott [6] and Fox [7], among othes has povided a ich ethogam (categoization of behavioal pattens) that spans the ange of animal activities (See Table 1). The play and maladaptive subsystems ae teated as sepaate behavioal subsystems fo pagmatic easons within the achitectue. Investigative (seaching/seeking) Sexual Epimeletic (cae and attention giving) Eliminative (excetion and uination) Et-epimeletic (attention getting o cae soliciting) Ingestive (food and liquids) Allelomimetic (doing what othes in goup do) Comfot-seeking (shelte-seeking) Agonistic (associated with conflict) Miscellaneous Moto Play Maladaptive Table 1: Main Behavioal Subsystems of the Dog

Using Timbelake s behavioal systems appoach [8], dawn fom psychology, these canine behavios can be futhe oganized into vaious subsystems, modes, and modules, and then mapped onto a typical behavio-based achitectue [9]. Figues 2-5 illustate some epesentative oganizational examples within the design, focusing paticulaly on aspects of the agonistic subsystem. INVESTIGATIVE EPIMELETIC ET-EPIMELETIC ALLELOMIMETIC AGONISTIC SEXUAL ELIMINATIVE INGESTIVE COMFORT-SEEKING MISCELLANEOUS PLAY MALADAPTIVE Motivational Vaiables Coodinato Figue 2: Complete Set of Subsystems Fighting-Pedation Mode Fom a design pespective, a least-commitment stategy is taken egading the coodination mechanisms, with a pefeence towads MacFaland s motivational space methods [10], but fo computational easons using vaiations of the lateal inhibition methods descibed fist by Ludlow [11] and late by Blumbeg [12]. III. EMOTIONAL BASIS Although an ethological model povides a basis fo what kinds of behavio we should ealize within the obot, a paticula specific behavio must be selected in a given situation. The basic mechanism of action selection of ou ethological model is to evaluate both extenal stimuli and ongoing intenal dives. We employ the homeostasis egulation ule fo action selection [13]. Namely, intenal vaiables ae specified that must be egulated and maintained within pope anges. Behavioal actions and changes within the envionment poduce change in these intenal vaiables. The basic ule fo action selection is to use the egulation of the intenal vaiables as a motivational dive signal fo the behavio modules. (Fig. 6) Extenal Stimuli Regulation ange Intenal Vaiables Defense-Escape Mode Abite Dominant Attitude Mode Subodinate Attitude Mode Hai Raising Module Figue 3. Modes compising Agonistic Subsystem Behavio Envionment The selected Action causes the change of the intenal vaiables Sitting Couching Running away Yelping Tail between legs Defensive olling on back Move away fom theat Seek out human Coodinato Figue 4. Modules within Defense-Escape mode Stimulus = theat o dominant animal pesent + attack + escape oute/aea pesent + high fea (the escape aeas may include cones of ooms) Response = un(fast, towads escape oute/aea) + ea-position(both, back) Figue 5. Example: un-away module Figue 6. Role of Dives in Behavio Selection Anothe motivation to intoduce an intenal state model is to incopoate emotional expession behavios. Thee ae many poposals fo emotional models. Ekmann [14] poposed 6 basic emotional states: happiness, ange, sadness, fea, supise, and disgust. In addition, some eseaches popose the eduction of an emotional basis dimension into only 2 o 3 dimensions. We employ Takanishi s model [15], which is 3-dimensional: pleasant, aousal, and confidence. The 6 basic emotional states ae located within this 3-dimensional space. We futhe combine the intenal vaiables with pleasantness. Namely, if the obots vaiables ae within the egulated ange, the pleasantness is high. The aousal axis is contolled by both cicadian hythm and unexpected stimuli. Confidence is contolled by the confidence (cetainty) of ecognized extenal stimuli. As shown in Figue 7, the intenal state model geneates dive signals and the emotional signals to the behavios.

Evaluation Intenal Vaiables Dives Levels of Intenal Vaiables Mapping to [P, A, C] space 6 basic Emotions Behavios Figue 7: Relationship of Dives and Emotions to Behavios IV. AIBO ARCHITECTURAL IMPLEMENTATION In ode to veify the advantages of the ethological appoach, the model descibed in the pevious sections was implemented, focusing on checking if the following featues can be validated in an actual obot. (1) The fusion of intenal motivations and extenal stimuli. (2) The coodination of behavios via lateal inhibition. (3) Computational efficiency with a layeed achitectue. In ode to simplify and shoten development time, we implemented a subset of the oveall model with limited peception (ecognition tagets) as follows: Only 3 patial subsystems, as shown in Fig. 8, ae ealized. Only 3 envionmental objects, WATER, FOOD, and MASTER, can be discened using visual colo classification. Figue 8 shows the implemented softwae achitectue on the obot AIBO. As descibed in the pevious sections, oughly speaking, thee ae 3 pincipal components: Releasing Mechanism, Motivation Ceato, and the Action Selection Module. The Releasing Mechanism component computes its output RM[I] (Fig. 9) using envionmental peceptual esults, such as the distance to a ecognized object. As itemized above, we only use the colo camea signal fo this pupose and only 3 objects can cuently be detected. The Motivation Ceato computes its output Mot[I] (Fig. 9) using an Instinct and Emotional Model, which has 6 intenal vaiables: nouishment, moistue, bladde distension, tiedness, cuiosity, and affection. Futhemoe, anothe 6 vaiables act to keep the 6 intenal vaiables within some bounded values. These ae called instinct vaiables, which include hunge, thist, elimination, tiedness, cuiosity, and affection. The output of the Motivation Ceato Mot[I] is computed using these instinct vaiables. In the Action Selection Module, a behavio vaiable V[I] is computed using a function of RM[I] and Mot[I] as shown in the gaph of Figue 7. This computation is caied out fom behavios in a highe oganization level (e.g., subsystem, mode). Lateal inhibition is used to avoid behavioal ditheing (thashing between behavios) and is also caied out by the Action Selection module so that the system can select a single behavio fo execution. Fom the highest oganizational laye (subsystems) to the lowest laye (pimitive modules), the computations ae pefomed to select a pope action command, which is then sent to a Finite State Machine whee the specific sequences on how to achieve the behavio ae descibed. Thus, the action to be executed is selected based on the value V[I], which is affected by both Mot[I] elated to the intenal vaiables and RM[I] elated to the extenal stimuli. Fo example, even if the obot has high motivation fo ingestive behavio, without the elevant extenal stimuli (e.g., a food object), then the obot doesn t select the ingestive behavio, and vice vesa. Figue 10 shows a layeed and tee stuctued achitectue fo subsystems, modes, and pimitive modules. Figue 11 shows the implemented behavio tee, whee 3 subsystems, investigative, ingestive, and play, ae housed. Investigative efes to exploatoy behavios such as walking aound (locomotion), ingestive means consummatoy behavios such as eating o dinking, and play means inteactive behavios with a human such as giving/offeing its paw. Figue 8. Softwae achitectue Figue 9. State-space Diagam

Figue 10. Behavioal Tee (Whole) selected. Moeove, compaing Figues 14 and 15 with Figue 17, we obseve that the coesponding action is not selected (as expected) even when highe Motivation vaiable Mot[I] is pesent duing some time intevals. Compaing Figues 16 and 17, fo this same peiod, the Release Mechanism value RM[I] is small, so not enough extenal stimuli is pesented within that peiod to evoke the coesponding behavio. Duing such a peiod, the system selected investigative behavio. Thus, the motivation vaiables o the intenal vaiables combined with the extenal stimuli affect the action selection mechanism in this system, as anticipated. In the cuent implementation, we found one poblem, which occued within the ingestive banch of the behavioal tee. Since we integate an eating behavio and a dinking behavio as the possible outcomes of the ingestive behavio, both hunge and thisty ae the input signal to the motivation and both food and wate ae the input to the elease signals. Fo example, when the hunge motivation is lage, and WATER exists, then the highest laye selects ingestive behavio coectly. Because WATER doesn t poduce a lage Release Mechanism value fo the eating behavio, thee is no action that has both of lage RM[I] and Mot[I] in the lowest laye of the selected ingestive subsystem. This can be avoided by designing a pope tee stuctue. Figue 11. Behavioal Tee (Implemented) V. EXPERIMENTS AND RESULTS In ode to veify if the advantages of this appoach ae achieved, we built a test field as shown in Figue 12. Fo easy ecognition, ed, blue, and geen cicles with 12-cm diamete ae used, which coespond to FOOD, WATER, and MASTER espectively. The field is 120cm squae and is suounded by walls. The obot descibed in the pevious section is placed on the field and detemines the RM[I], Mot[I], V[I], selected behavio, duing a time couse of activity. Figues 13-17 show vaious time sequences fo some elevant measuements. Figue 13 shows the Time-Instinct vaiable gaph. Figue 14 and 15 show Time-Motivation vaiable gaphs coesponding to Mot[I] of the subsystems and modules. Figue 16 shows a Time-Release Mechanism (RM[I]) vaiable gaph, and Figue 17 shows the time sequence of selected behavios. Hee, the 6 intenal vaiables decease as time passes but incease when the coesponding behavio is executed. Compaing Figue 13 with Figue 17, we can obseve an incease in the value of the instinct vaiables as well as thei decease when the coesponding behavioal action is Figue 12. Field Figue 13. Instinct-Time gaph

Investigative Ingestive Play Figue 14. Motivation-Time gaph fo subsystem Figue 15. Motivation-Time gaph fo Module VI. EMOTIONALLY GROUNDED SYMBOLS Although ou goal is to implement dog-like behavio based on ethological studies, when we implemented symbol acquisition behavio, we need to lean the meanings of the acquied symbols in tems of the obot s needs [16,17]. Symbol gounding is a basic challenge in atificial intelligence, as discussed by Hanad [18] among othes. Fom a patten ecognition point of view, if we teat the classified categoies as symbols, we can say they ae physically gounded though the peceptual channel. Howeve, when we design behavios with objects that can be teated as physically gounded symbols, we ealize that we cannot assign pope behavioal esponses to all objects encounteed in advance. Fo example, using visual and audio patten classification technologies, a obot can ecognizes a new object with a ed colo and associate its name with the audio ponunciation as apple. Thus, the obot acquies the physically gounded symbol of apple (Fig. 18). Howeve, it doesn t know what to do with the apple (i.e., what is the coect behavioal esponse to an apple). This is because the obot doesn t lean the meaning of the apple. Physically Gounded Symbol Knows symbol epesentation by peceptual channels TOMATO Does NOT know what a TOMATO is. Play= sleep= TOMATO dink eat talk Figue 18: Behavioal Symbol Gounding Poblem Figue 16. Release Mechanism-Time gaph Figue 17. Behavio-Time gaph While evolution in natue pemits the leaned association of specific symbols with appopiate behavios, and indeed suitable design in obotic systems can also povide many of these associations, clealy new and unfoeseen objects must be dealt with, and thus pemitting use inteaction via teaching to occu. To solve the poblem, we poposed a concept of an emotionally gounded symbol, whee the physically gounded symbol is associated with the change of intenal vaiables when the obot applies a behavio in esponse to the object (Fig. 19). Then, when the obot sees o heas the symbol (apple), it knows which associated behavio causes the change of its intenal vaiables. Thus, we say the obot now knows the meaning of the symbol. (E.g. the apple is associated with an incease of the intenal vaiable nouishment and the obot knows the coect behavioal esponse when it sees o heas the symbol apple ).

Select Ty behavio Do some behavio fo TOMATO with pobability memoy, then the memoized VE-1 is output fom the associative memoy, which is the categoy indication of ed object. Thus, the symbol is gounded to both the visual and audio peceptual channels. Of couse if only the VE-1 ( ed object) is pesented, then the associative memoy can ecall AE_1 (phoneme sequence [ed]). TOMATO Ty to play with tomato! play dink eat sleep talk Visual Peception VE-1 AE-1 LTM VE-4 AE-4 VE-3 AE-3 VE-2 AE-2 VE-1 AE-1 If the change of the intenal vaiables is lage, Then the change is associated with TOMATO. Figue 19. TRY Behavio fo Expeimentation with new Envionmental Objects Audio Peception (a)leaning Visual Peception STM VE-1 Associative Memoy LTM VE-4 AE-4 VE-3 AE-3 VE-2 AE-2 VE-1 AE-1 Figue 20 shows the extended achitectue fo the emotionally gounded symbol system, which we call the Emotionally GOunded achitectue, o EGO achitectue. In this system, both physically gounded symbol acquisition and emotionally gounded acquisition ae achieved. Visual Peception Auditoy Peception Release Mechanism Behavio Dive LTM STM Emotion System Intenal Evaluation Vaiables Behavio Selection Figue 20. EGO Achitectue Mapping 6 Basic Emotions To Motion Geneato In Figue 21 the basis fo the physically gounded symbol acquisition is depicted. Assume that thee ae two peceptual channels, the visual peception channel and the auditoy peception channel. The visual peceptual channel outputs visual events (VEs), which ae categoy IDs of the visual peception module. The auditoy peceptual channel outputs auditoy events (AEs), which ae also categoy IDs of the auditoy peception module. These VEs and AEs can be consideed as gounding to the physical wold though the peceptual channels. Fo example, a paticula VE (VE- 1) is a colo segmentation event, which indicates a ed object in the visual input of the obot. An AE (AE-1) is a phoneme sequence [ed]. If these two events occu simultaneously, these two ae fist stoed in a Shot-Tem- Memoy (STM), and then memoized in an associative memoy o a Long-Tem-Memoy (LTM) (Figue 19(a)). The actual obot implementation includes dialogue with the human to lean the object name with human. Afte the leaning episode is ove, if only one event, e.g. the AE-1 (phoneme sequence [ed]), is input to associative STM (b)recalling fom Visual Stimuli Associative Memoy VE-1 AE-1 Figue 21. Physically Gounded Symbol Acquisition. (a) Associative leaning of visual event and audio event, (b) Recalling its name fom the visual event. In addition to the associative memoy capability of the visual and audio events, the EGO achitectue can memoize the emotional expeience, which is a basic concept of the emotionally gounded symbol acquisition. Fo example, afte the physically gounded symbol (e.g. apple) is acquied, the obot may ty to apply seveal behavios, such as eating and kicking. Then, the intenal vaiables elated to the applied behavios associated with the apple eceive a big change fo the intenal vaiables elated to eating, but not kicking. Now the symbol is associated with the change of the intenal vaiables, so that when the obot peceives the symbol, the change of intenal vaiables is also ecalled. This change of intenal vaiables can now be used to geneate the dive signals fo behavios so that the eating behavio is highly activated. The change of the intenal vaiables is also input to the emotional system and can vitually geneate the emotional state by the associated change of the intenal vaiables. Thus, the obot can ecall its pevious emotional expeience with the symbol. VII. SDR HUMANOID ARCHITECTURAL OVERVIEW We ae now in the pocess of extending ou eseach on the ethological achitectue fo use in the humanoid obot SDR-4X (Figue 25). The eseach hypothesis is that human behavio can also be effectively captued using ethological modeling. Unfotunately, the ethological liteatue fo humans is nowhee nea as ich as it is fo dogs, pincipally due to pivacy issues. Nonetheless, child behavio is

easonably well documented due to secuity concens and can seve as a basis fo ethological models of young childen. It is ecognized that a puely behavioal appoach cannot account fo all levels of human competence; so incopoating delibeation into eactivity also equies achitectual modification. In addition, speech-pocessing capabilities futhe inceases both the competency and the complexity of the system. Ou cuent achitectual thinking fo the humanoid is shown in Figue 22. n io is V io d u A ile t c a T y o m e M m e T t o h S ISM y o m e M m e T g n o L Envionments Delibeative Laye Situated Behavios Laye Configuation Dependent actions And eactions Figue 22: Peliminay design fo Humanoid achitectue The achitectue is based on the EGO achitectue shown ealie in Figue 18. It possesses peception, memoy, ISM (Intenal State Model), and behavio geneation components. The main diffeence fom the EGO achitectue is that thee is now a delibeative laye on top of situated behavio laye. The technical tagets of SDR-4X ae to implement a humanoid obot, which can walk on vaious floo conditions (a soft capet, a had wooden floo, and a slippey tatami floo), and can deal with obstacles without falling down. Even if it falls down by accident, it can ecove and esume its behavio. Then, it can also seach fo a human to inteact with via speech and motion. To achieve these goals, SDR-4X has the following featues: (1) Real-time adaptive motion contol (2) Real-time gait patten geneation (3) Real-time and eal wold space peception capability (4) Multimodal human obot inteaction Regading the eal-time and eal-wold space peception, a mico steeo-vision system with obstacle detection is implemented. On top of the detection system, we futhe implement a path planne so that SDR-4X can walk towad the taget place while avoiding obstacles. Figue 23 shows the obstacle avoidance behavio. Figue 23. Obstacle avoidance and path planning using a mico steeo-vision. Above the geneated occupancy gid, below the behavio duing execution. Anothe featue fo spatial peception involves sound localization with multiple micophones. SDR-4X has 7 micophones in its head to detect the sound diection in both the hoizontal and vetical diections. Regading the multimodal human inteaction technologies, we have implemented multi-face detection and multi-face identification (Fig. 24), a lage vocabulay continuous speech ecognize, a speake identification system, and unknown wod acquisition with unknown face leaning. In addition, a text-to-speech synthesize has been implemented. Figue 24. Multi-Face detection Using these technologies, a simple dialogue system with a tee stuctue has been implemented as descibed in the pevious section, to acquie and associate a new face with a new name. Duing the inteaction with a human, the EGO achitectue emembes the emotional expeience with that peson, so that the obot can have diffeent inteactions with diffeent people depending on the associated emotion with each individual.

Seveal technologies such as face detection, identification, and steeo-vision with obstacle avoidance behavio ae descibed. In the futue, we ae going to ealize even moe natual human inteaction with dialogue. The emotionally gounded concept is a key to undestanding the meaning of the use s utteed wods in elation to the obots peceptions, behavios, capabilities, and needs. Acknowledgments The authos thank D. Doi, the diecto of Digital Ceatues Laboatoy, Sony, fo his continuous suppot fo ou eseach activity. Figue 25. SDR-4X with emotional expession In addition, SDR-4X has two significant entetainment abilities, which ae dancing and singing. SDR-4X especially uses its speech synthesis technology fo changing the tone of its voice. Namely, with eithe a musical scoe o text data, SDR-4X can sing a song with emotional expession. Figue 26. Singing a song pefomance with dancing. In Mach 2002 in Japan, we pesented an exhibition at RoboDex, whee we gave demonstations of these pefomances in public. Pats of these demonstations wee conducted using the achitectue descibed in Figue 22. VIII. SUMMARY AND CONCLUSIONS An ethological model and emotional model fo autonomous dog-like behavio has been pesented. This is then extended into an emotionally gounded achitectue (EGO achitectue) fo leaning new objects by associating thei effect on intenal motivational and emotional vaiables that geneate how to behave in the pesence of these objects. The EGO achitectue is natually extended to the behavio contol achitectue fo a small humanoid, SDR-4X, which has eal-time and eal-wold peception and mobile capability with multimodal human inteaction capability. Refeences [1] Fujita, M., AIBO: Towads the Ea of Digital Ceatues, Intenational Jounal of Robotics Reseach, 20(10):781-794, Octobe 2001. [2] Loenz, K., The Foundations of Ethology, Spinge-Velag, New Yok, 1981. [3] Tinbegen, N., Social Behavio in Animals, Methuen, London, 1953. [4] Beazeal, C., Designing Socialable Robots, MIT Pess, 2002. [5] Dautenhahn, K. and Billad A., "Binging up Robots o Psychology of Socially Intelligent Robots: Fom Theoy to Implementation", Poc. 3d Intenational Confeence on Autonomous Agents, Seattle, WA, May 1999. [6] Scott, J.P. and Fulle, J.L., Genetics and the Social Behavio of the Dog, Univesity of Chicago Pess, Chicago, IL, 1965. [7] Fox, M., The Dog: Its Domestication and Behavio, Galand, New Yok, 1978. [8] Timbelake, W. and Lucas, G., "Behavio Systems and Leaning: Fom Misbehavio to Geneal Pinciples", in Contempoay Leaning Theoies: Instumental Conditioning Theoy and the Impacts of Biological Constaints on Leaning, S. Klein and R. Mowe (eds.), LEA Associates, Hillsdale, NJ, 1989. [9] Akin, R.C., Behavio-based Robotics, MIT Pess 1998. [10] McFaland, D. (ed.), Motivational Contol Systems Analysis, Academic Pess, London, 1974. [11] Ludlow, A., "The Behaviou of a Model Animal", Behaviou, LVIII, 1-2, pp. 131-172, 1976. [12] Blumbeg, B., "Action-Selection in Hamstedam: Lessons fom Ethology", Fom Animals to Animats 3, ed. Cliff et al, MIT Pess, 1994, pp. 108-117. [13] Akin, R.C., Homeostatic Contol fo a Mobile Robot, Dynamic Replanning in Hazadous Envionments, Poc. SPIE Confeence on Mobile Robots, Cambidge, MAA, pp. 240-249, 1988. [14] Ekman, P., and Davidson, R. J., The Natue of Emotion, Oxfod Univesity Pess, 1994 [15] Takanishi, A. An Anthopomophic Robot Head having Autonomous Facial Expession Function fo Natual Communication with Human, 9 th Intenational Symposium of Robotics Reseach (ISRR99), 1999, pp.197-304 [16] Fujita M. et al., An Autonomous Robot that eats infomation via inteaction with human and envionment, IEEE ROMAN-01, 2001, [17] Fujita M., et al., Physically and Emotionally gounded symbol acquisition fo autonomous obots, AAAI Fall Symposium: Emotional and Intelligent II, 2001, pp.43-46. [18] Hanad, S., The Symbol Gounding Poblem, Physica D 42, 1990, pp. 335-346.