Open-Domain Chatting Machine: Emotion and Personality Minlie Huang, Associate Professor Dept. of Computer Science, Tsinghua University aihuang@tsinghua.edu.cn http://aihuang.org/p 2017/12/15 1
Open-domain Chatting and Conservational AI 1 : 91: A : B 5, - : :5 0 1 A1 1:
Reshaping Human-Machine Interactions Microsoft: Conversation As a Platform (CAAP) Mouse + Keyboard GUI Screen Touch GUI Conversational UI
Virtual Conversational Agents 1966 1994 2010-2014 - 2016
Social Robot (with Physical Body) humanoid robot The first robot with emotions MIT Jibo: the world s first social robot for the home Buddy: protects your home, entertains the family, interfaces with devices, and assists the family
Challenges in Open-domain Chatting Machines Semantics Logics Consistency Interactiveness Content Quality Personality Identity Language Style Emotion Sentiment Dialogue Strategy Open-domain, open-topic conversational agents 2017/12/15 6
Challenges in Open-domain Chatting Machines One-to-many: one input, many many possible responses Knowledge & Reasoning: real understanding requires various knowledge or backgrounds Situational Context Who are you talking with? Stranger, or friend? Boss, or subordinate His mood and emotion? Unknown backgroundsthatare only shared by posterand responder 2017/12/15 7
Typical Solution I: Retrieval-based From MSRA Dr. Ming Zhou
Typical Solution II: Generation-based User: I am so happy to be here Machine: Glad with you.
Open-domain Chatting Machines Content quality: unknown words, long and diverse responses Beam Search (Li et al., 2015) Glimpse (Shao et al. 2017) Topics and keywords (Xing et al., 2017; Mou et al., 2016) Longer context (hierarchical models) (Serban et al., 2015/2016a/b) Personalization: considering user information (Li et al., 2016; Al-Rfou et al., 2016) Consider gender, age (Joshi et al. 2017) Consider social networks (Bhatia et al. 2017) 2017/12/15 10
Emotional Chatting Machine Emotion intelligence is a key human behavior for intelligence (Salovey and Mayer, 1990; Picard and Picard, 1997) Understanding emotion and affect is important for dialogue and conversation Enhance user performance Improve user satisfaction Less breakdowns Rule-based emotion adaptation Seen in early dialogue systems 2017/12/15 11
Emotional Chatting Machine -- - Social Interaction Data Post Response Post Response Emotion Classifier Emotion Tagged data Emotional Chatting Machine Post Response Post Response Our work is reported by MIT Technology Review, the Guardian, Cankao News, Xinhua News Agency etc. Prof Björn Schuller: an important step towards personal assistants that could read the emotional undercurrent of a conversation and respond with something akin to empathy.
Emotional Chatting Machine Emotion category embedding: High level abstraction of emotions Emotion internal state: Capturing the change of emotion state during decoding Emotion external memory: Treating emotion/generic words differentially 2017/12/15 13
Emotional Chatting Machine Internal emotion memory : emotional responses are relatively short lived and involve changes (Gross, 1998; Hochschild, 1979) Encoder Input emotion: Sad Emotion state GO y 1 (A) (lovely) (person) Decoder s state S 0 S 1 S T 1.0 0.9 Read Write 0.8 0.7 Write 0.8 Value decay Value decay 0.6 y T 0.0 0.0 0.0 Sad Sad Sad
Emotional Chatting Machine Internal emotion memory : emotional responses are relatively short lived and involve changes (Gross, 1998; Hochschild, 1979) y t-1 y t S t-1 S t Emotion state 1.0 0.9 0.8 Read Write 0.8 0.7 0.6 Sad Sad
Emotional Chatting Machine Internal emotion memory : emotional responses are relatively short lived and involve changes (Gross, 1998; Hochschild, 1979) 2017/12/15 16
Emotional Chatting Machine External emotion memory: generic words (person) and emotion words (lovely) y t-1 =lovely y t =person Emotional Generic Emotional Generic Type Selector Type Selector S t-1 Decoder s state 2017/12/15 17 S t
Emotional Chatting Machine External emotion memory: generic words (person) and emotion words (lovely) 2017/12/15 18
Emotional Chatting Machine Emotion Classification Dataset: the Emotion Classification Dataset of NLPCC 2013&2014 23,105 sentences collected from Weibo The STC dataset: a conversation dataset from (Shang et al., 2015) 219,905 posts and 4,308,211 responses Each post has about 20 responses 2017/12/15 19
Emotional Chatting Machine 2017/12/15 20 Hao Zhou, Minlie Huang, Xiaoyan Zhu, Bing Liu. Emotional Chatting Machine: Emotional Conversation Generation with Internal and External Memory. AAAI 2018.
More Chinese Examples post: other post: other post: other post: other 2017/12/15 21
Emotion Interaction Patterns LikeàLike (empathy) Sadness àsadness (empathy) Sadness àlike (comfort) Disgust à Disgust (empathy) Disgust à Like (comfort) Anger à Disgust HappinessàLike Hao Zhou, Minlie Huang, Xiaoyan Zhu, Bing Liu. Emotional Chatting Machine: Emotional Conversation Generation with Internal and External Memory. AAAI 2018. 22
Endowing a Chatting Machine with Personality Passing the Turning Test? Existing chatting machine lacks identity or personality Existing works Userembedding: learn implicitconversation style (Li et al., 2016; Al-Rfou et al., 2016) Require dialogue data from different users with userattributes tagged For chatbots: no such data available 2017/12/15 24
Endowing a Chatting Machine with Personality Generating coherent conversation w.r.t. identity/personality Generic Dialogue Data Pre-specified Chatbot Profile UserA: how old are you? UserB: I am six. UserA: do you like to play piano? UserB: I play violin. Identity-coherent Chatbot User: how old are you? Machine: I am three years old. User: do you like to play piano? Machine: Yes, I play piano. 2017/12/15 25
Endowing a Chatting Machine with Personality Encoder Profile detector Position detector Decoder 2017/12/15 26 Qiao Qian, Minlie Huang, Haizhou Zhao, Jingfang Xu, Xiaoyan Zhu. Assigning personality/identity to a chatting machine for coherent conversation generation. 2017, arxiv:1706.02861.
Endowing a Chatting Machine with Personality Profile detector Using profile or not Profile key selection Bidirectional decoder 2017/12/15 27
Endowing a Chatting Machine with Personality Loss function Loss on decoders: forward decoder and bidirectional decoder Loss on profile predictions: 2017/12/15 28
Endowing a Chatting Machine with Personality WD: 9,697,651 post-response pairs from Weibo 76,930 pairs from WD for 6 profile keys (name, gender, age, city, weight, constellation) with about 200 regular expression patterns, each annotated to positive or negative 42,193 positive pairs, each mapped to one of the keys Manual Dataset: real, human-written conversational posts Qiao Qian, Minlie Huang, Haizhou Zhao, Jingfang Xu, Xiaoyan Zhu. Assigning personality/identity to a chatting machine for coherent conversation generation. 2017, arxiv:1706.02861. 2017/12/15 29
Endowing a Chatting Machine with Personality Post-level evaluation Generated sample responses that exhibit session-level consistency Session-level evaluation 2017/12/15 30 Qiao Qian, Minlie Huang, Haizhou Zhao, Jingfang Xu, Xiaoyan Zhu. Assigning personality/identity to a chatting machine for coherent conversation generation. 2017, arxiv:1706.02861.
Endowing a Chatting Machine with Personality Generating responses that are coherent to robot s profile 2017/12/15 31 Qiao Qian, Minlie Huang, Haizhou Zhao, Jingfang Xu, Xiaoyan Zhu. Assigning personality/identity to a chatting machine for coherent conversation generation. 2017, arxiv:1706.02861.
Future Research Problems Multi-modality emotion perception and expression (voice, vision, text) Personality, identity, styleà human-like robot Introvert or extrovert Personalized (style, or profile) Learning to learn Grow up from interactions with human partners and environment
Summary Open-domain chatting machine is one of the most challenging AI tasks Requires the ability of understanding semantics, knowledge, and situational context Ability of making reasoning Still a long way to go: existing generation models are still far from the expectation of real-world applications 2017/12/15 33
Thanks for Attention Minlie Huang Email: aihuang@tsinghua.edu.cn Homepage: http://aihuang.org/p Acknowledgements: Prof. Xiaoyan Zhu, Hao Zhou, Zheng Zhang, Qiao Qian 2017/12/15 34