Emotional Design. D. Norman (Emotional Design, 2004) Model with three levels. Visceral (lowest level) Behavioral (middle level) Reflective (top level)

Transcription

1

2 Emotional Design D. Norman (Emotional Design, 2004) Model with three levels Visceral (lowest level) Behavioral (middle level) Reflective (top level)

3 Emotional Intelligence (EI) IQ is not the only indicator of intelligence (Emotional Intelligence book by Daniel Goleman, 1995) EI: Awareness and ability to manage one s emotions in a healthy manner. EI: Ability to sense, perceive, understand, and assess own and other people s emotions

4

5 Is Mr. Spock intelligent? Spock is only rational Descarte s Error (Damasio, 1994) Artificial intelligence searches unlimited search space to make a rational decision Missing somatic markers that associate feelings with decisions

6 Damasio s Somatic Marker Hypothesis Originated from the observation of individuals who had sustained damage to the ventromedial prefrontal cortex. Normal intellectual function Normal Neuropsychological function Normal on tests sensitive to frontal lobe function However, severe impairment in personal and social decision making and conduct. Difficulty with planning in the immediate, and future. No longer able to make personally advantageous decisions Often sustain social, personal, economic losses

7 The only deficit that could be detected was one in which these individuals failed to display emotion in situations in which emotion would be normatively expected. This led Damasio to posit that these individuals manifest a deficit in reasoning that is secondary to deficits in emotional processing.

8 What is Affect? The type and degree of emotion a person displays The experienced, subjective, and conscious aspect of feeling or emotion Positive Negative Neutral

9 Affect Theory Developed by Silvan S. Tomkins in 1962 Tomkins book Affect Imagery (3 vols.) Believed that the affect system is the motivating force in human life. Organized affect into 3 main categories: Positive, negative, and neutral Each has a low/high intensity label

10 Tomkins nine affects Positive: Enjoyment/Joy - smiling, lips wide and out Interest/Excitement - eyebrows down, eyes tracking, eyes looking, closer listening Neutral: Surprise/Startle - eyebrows up, eyes blinking Negative: Anger/Rage - frowning, a clenched jaw, a red face Disgust - the lower lip raised and protruded, head forward and down Dissmell (reaction to bad smell) - upper lip raised, head pulled back Distress/Anguish - crying, rhythmic sobbing, arched eyebrows, mouth lowered Fear/Terror - a frozen stare, a pale face, coldness, sweat, erect hair Shame/Humiliation - eyes lowered, the head down and averted, blushing

11 Affective Computing Computing that relates to, arises from, or deliberately influences emotions Coined by Rosalind Picard Founder and director of the Affective Computing Research Group at the MIT Media Lab. Her book, Affective Computing (1997) lays the groundwork for giving machines the skills of emotional intelligence.

12

13 Affective computing is related to other computing disciplines: Artificial Intelligence (AI), Virtual Reality (VR) and Human Computer interaction (HCI). Questions that need to be answered: What is an affective state (typically feelings, moods, sentiments etc.)? Which human communicative signals convey information about affective state? How various kinds of affective information can be combined to optimize inferences about affective states? How to apply affective information to designing systems?

14 Affective Computing Recognize emotions Express emotions Have emotions

15 Recognize Emotions Bio-signals (wearable sensors) Brain Signals, skin temperature, blood pressure, heart rate, respiration rate Facial Expressions Speech/Vocal expressions Gestures Limbic movements Text

16 Recognition we need an emotion model that allows us to differentiate between emotional states we need a classification scheme that uses specific features from an input signal to recognize the user s emotions

17 Text Sentiment Analyzing Discussion Board The Natural Language Toolkit, or more commonly NLTK, is a suite of libraries and programs for symbolic and statistical natural language processing (NLP) for the Python programming language.

18 Speech Paralinguistic Features of Speech how is it said? Prosodic features (e.g., pitch-related feature, energy-related features, and speech rate) Spectral features (e.g., MFCC - Mel-frequency cepstral coefficient and cepstral features) Spectral tilt, LFPC (Log Frequency Power Coefficients) F0 (fundamental frequency of speech), Long-term spectrum Studies show that pitch and energy contribute the most to affect recognition Speech disfluencies (e.g., filler and silence pauses) Context information (e.g., subject, gender, and turn-level features representing local and global aspects of the dialogue) Nonlinguistic vocalizations (e.g., laughs and cries, decode other affective signals such as stress, depression, boredom, and excitement)

19 Accuracy rates from speech are somewhat lower (35%) than facial expressions for the basic emotions. Speech Signal Sadness, anger, and fear are the emotions that are best recognized through voice, while disgust is the worst. Pre-processing Feature Extraction Classification Audio recordings collected in call centers and, meetings, Wizard of Oz scenarios interviews and other dialogue systems Classified Result ]M. Pantic, N. Sebe, J. F. Cohn, and T. Huang. Affective multimodal human-computer interaction. In ACM International Conference on Multimedia (MM), Rafael A. Calvo, Sidney D'Mello, "Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications

20

21 Facial Expressions Example: Active Appearance Model (AAM) (AAM) based system which uses AAMs to track the face and extract visual features. Support vector machines are used (SVMs) to classify the facial expressions and emotions. Lucey, P.; Cohn, J.F.; Kanade, T.; Saragih, J.; Ambadar, Z.; Matthews, I.;, "The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression," Computer Vision and Pattern Recognition Workshops (CVPRW), 2010 IEEE Computer Society Conference on, vol., no., pp , June 2010

22 Bio-signals 22 Physiological signals derived from Autonomic Nervous System (ANS) of human body. Fear for example increases heartbeat and respiration rates, causes palm sweating, etc. Psychological Metrics used are: GSR - Galvanic Skin Resistance RESSP - Respiration BVP - Blood Pressure Skin Temperature Electroencephalogram (EEG), Electrocardiogram (ECG), Electrodermal activity (EDA), Electromyogram (EMG) Skin conductivity sensors, blood volume sensors, and respiration sensors may be integrated with shoes, earrings or watches, and T-shirts Huaming Li and Jindong Tan Heartbeat driven medium access control for body sensor networks. In Proceedings of the 1st ACM SIGMOBILE international workshop on Systems and networking support for healthcare and assisted living environments (HealthNet '07). ACM, New York, NY, USA,

23 Gestures gesture and body motion information is an important modality for human affect recognition; combination of face and gesture is 35% more accurate than facial expression alone. Two categories of Body-Motion-based affect recognition Stylized The entirety of the movement encodes a particular emotion. Non-stylized More natural - knocking door, lifting hand, walking etc.

24 Fusion

25 Frequently used Detection and Estimation Techniques Neural Networks (NN) Hidden Markov Models (HMM) K-Nearest Neighbors (KNN) Linear Discriminant Analysis (LDA) Support Vector Machines (SVM) Gaussian Mixture Models (GMM) Discriminant Function Analysis (DFA) Sequential Forward Floating Search (SFFS)

26

27 Express emotions Kismet (Breazeal and Scassellati, 2002) Emotional expression for communication and social co-ordination Emotion for organisation of behaviour (action selection, attention and learning) Arbib and Fellous (2004)

28

29

30 More effective expression than humans: Human expression identified 50% of the time. Computer expression identified 70% of the time (Elliott, 1997).

31

32

33 Listening 2. Understand 3. Confused 4. Waving goodbye

34 EXPERIMENTS AND RESULTS

35 Basic Emotions Stress Level Complex Emotions Feature Estimator Alert the Driver Driving Aid Agent Speed, ABS, Traction Control Audio Linguistic / Non-linguistic Feature Detector Facial Expression Feature Detector Navigation Agent Route Selection Bio-signals Actions Steering Movement Interaction with Gas / Break Paddle... Safety Agent Inter agent communication to aid decision making Notify in case of Emergency Music, Climate Control Seat Pressure Feature Detector Affective Multimedia Agent 35 35

36 Have emotions Can machines feel? How would we know?

37 Criteria for having emotions System has behavior that appears to arise from emotions System has fast primary emotional responses to certain inputs System can cognitively generate emotions System can have emotional experience System s emotions interact with other processes (e.g. memory)

38