Motion Control for Social Behaviours

Transcription

1 Motion Control for Social Behaviours Aryel Beck Supervisor: Nadia Magnenat-Thalmann Collaborators: Zhang Zhijun, Rubha Shri Narayanan, Neetha Das

2 INTRODUCTION In order for companion robots to be socially accepted they need to display appropriate social reactions and behaviors. They should use the same modalities as human do (i.e. voice, facial and body expression) > We need to build autonomous robots that have some social intelligence. Overall Research Question: Given a set of perceptions (enablers), how can a robot generate socially accepted behaviours? Our Research Question: Given a set of perceptions, how can a robot generate appropriate gaze behaviours? 2

3 What is Social Attention Social attention is the focus of cognitive processes on an individual or group in a mixed setting. We can divide attention in two overlapping systems: Top down attention and its role during social interactions Bottom-up attention and its role during social interactions

4 Gaze/Attention/Emotion Gaze longer and direct dominant positions Dovidio 85 Approach oriented/avoidance Oriented Adams 2005 Eyes look away for cognitive processing Mutlu 2013 and others Frequency of blink related to muscle tension Average blink rate to be used Blinking rate 11.6/min normally for schizophrenic Itti 2004 Harris 1996 Positive affect broaden and negative affect narrow the scope of attention Affective states low in motivational intensity broaden and Gable & Jones 2011 affective states high in motivational intensity narrow the scope of attention. There is an increase in the rate of gaze-aways over time. Bickmore 2012 Gaze direction systematically influenced the perceived Adams2005 emotion disposition conveyed by neutral faces. Fredickson 2004

5 Current Trend for Attention Systems Most of the systems in robotics use saliency maps. Common features used: color hue, sound localization (Ruesch 2008, Nakajima 2013). Data-driven methods (Mutlu 2013) and they shorcomings. These systems are not socially driven. Infotech 2011 For future work: Not much research on modeling the effect of emotion on attention.

6 Perception/Decision/Action Perception Decision Action

7 Example of Social Robots that can realize the Action

8 The Nadine Robot The Nadine robot uses pneumatic motors to display natural looking movements. It has 27 Degrees of Freedom: 7 for the face 3 for the neck 7 in each arm 3 in the waist

9 Main Classes of the controller: I2p Agent Control server: i2p Interface that receives instructions from the Network Nadine Controller: Execute the command, sync the output and send 1 frame to the hardware every 30ms. Text to Speech: Synthetizes the speech and produces the lip animation. Joint: Stores the trajectory and state of each joint. XML Library of Animations: Load and store the Pre-defined animations (XML). Online Movement Generation: Inverse Kinematics and Gaze

10 Perception: Kinect Skeleton Tracker

11 Facial Expression Tracking(Screenshot)

12 Audio Source -Screenshot

13 Decision Mechanism Behavior Based Architecture (bottom up) to drive the attention as well as others social reflexes behaviours. Behavior-based controllers consist of a collection of behaviors. Behaviors are processes or control laws that achieve and/or maintain goals We use a Winner takes all Coordinator

14 Behavior Based Architecture for Social Attention Advantages of the Proposed System: Bottom up decision mechanisms to model bottom up attention. Advantage of Modality Saliency in comparison to saliency maps. Robust to sensors defect. Behaviours can be combined and layered to construct social intelligence. Local fusion. Each behaviour has its own data structure and records. Other social reflexes can be added in the same architecture

15 Our System Our decision system for Attention is composed of 3 behaviours. Direct Attention towards speaker. Direct Attention towards user through Vision. Direct Attention towards interesting Gestures.

16 Direct Attention towards speaker: Speaker Detection (1) Compute the linear function given the direction of the sound. (2) For each user, compute the linear functions that passes through her. (3) Compare the slope of the linear function to find the speaker.

17 Direct Attention towards user through Vision: Features Extracted Agent should focus more on users closer to her. Users are placed within their social distance

18 Features Extracted Quantity of motion is related to the arousal dimension of emotion. We compute it as follow: (1) Change of coordinate so that the origin is the Pelvis. (2) For each upper body joint compute the distance travelled from 1 frame to the other. (3) Divide this value by the height of the user.

19 Features Extracted Summary Features used to drive the attention: Movement Detection Distance between user and agent (proxemics) Users Orientation (Kinect SDK)

20 Direct Attention towards user through Overview: Vision Each kinect frame: Records users positions, compute Quantity of Motion, Social Distance, Attention. Keep a short history of the recent events (Small buffer (100 frames)) Score each user using a weighted average of their features. Look at the user with the highest score.

21 System Testing and Integration Material for user testing: The attention system works with a higher cognitive level provided by a chatbot and google speech to text. Rather than putting a video, please try the system after lunch (No food inside the room!). Work is still in progress but feedback is useful

22 Future Work Test the system with users. Add more sound categories to the attention system. Add an emotion layer that affect the behaviors. A lot research in psychology points towards the effect of emotions on attention.

23 Thanks for your attention! Any questions?