Social Context Based Emotion Expression

Transcription

1 Social Context Based Emotion Expression Radosław Niewiadomski (1), Catherine Pelachaud (2) (1) University of Perugia, Italy (2) University Paris VIII, France Social Context Based Emotion Expression 1

2 Overview Complex Facial Expressions of Emotions Expressions and Social Context of an interaction Perspectives and Future Work Social Context Based Emotion Expression 2

3 Beyond displays of basic emotions (1/2) Displays of basic emotions : Any basic emotion has a distinct facial expression Ekman 1975: Anger, fear, sadness, enjoyment, disgust, surprise (are universal) Different lists of candidates [Ekman et al.,, 1975, Plutchik 1980, Izard 1971] Seldomly observed Expressions in real-life : are often complex and involve several emotions People recognize expressions of two emotions [Ekman 1972, LaPlante et al.,, 2000, Nummemaa 1988] Expressions can be voluntarily displayed [Ekman et al.,, 1969, Gosselin et al.,, 1995] People recognize facial expressions of felt emotion [Wagner et al., 1986] and of fake emotion [Ekman 1982] and distinguish them [Gosselin et al.,, 1995, Frank et al.,, 1995] Social Context Based Emotion Expression 3

4 Beyond displays of basic emotions (2/2) Complex expressions in real-life may occur either as: superposition of emotions quick succession of different emotions masking of one emotion by another emotion suppression or overacting of an emotion inhibition of an expression of emotion voluntary expressions Complex expressions in ECAs: [Ekman et al., 1969] users differentiate displays of felt vs. fake emotion [Rhem et al.,, 2005] blends are perceived [Martin et al.,, 2006] Social Context Based Emotion Expression 4

5 Our approach to complex facial expressions Based on Ekman s work [Ekman et al.,, 1975, Ekman 2003] Decomposition of the face into areas Complex expressions are created by composing the face areas of two source expressions Different types of blends are implemented with different sets of fuzzy rules Rules extended to any facial expression using fuzzy similarity Neutral expression is a category [Young et al.,, 1997] Social Context Based Emotion Expression 5

6 Partitioning of the face More detailed partitioning Based on common language terms Facial areas: brows, upper eyelids, eyes, lower eyelids, nose, cheeks, lips (movement), lips (tension) Social Context Based Emotion Expression 6

7 Fuzzy similarity? six basic expressions Input expression X similarity: S(A,B) = M(A B)/M(A B) Social Context Based Emotion Expression 7

8 Fuzzy similarity: evaluation study On-line test Participants have to choose the degree of similarity between facial expressions 35 results at the moment Correlation is about 90% Social Context Based Emotion Expression 8

9 Complex facial expressions: masking (1/3) Masking of an elicited emotion by an expressed emotion masking = one felt emotion + one fake emotion Masking is asymmetric communicative function Deception cues implemented: Felt emotions as well as fake emotions are characterized by specific facial features [Ekman et al.,, 1975] Fake expressions are often asymmetric [Ekman 1989] Social Context Based Emotion Expression 9

10 Complex facial expressions: masking (2/3) Masked emotion may leak over the displayed emotion : Fake expressions Humans are not able to control all their facial muscles The upper area of the face is usually more difficult to control [Ekman et al.,, 1975] reliable features i.e. sadness brows lack in fake emotions as they are difficult to do voluntarily [Ekman et al.,, 1975] Felt expressions People are not able to fully inhibit felt emotions [Darwin 1872, Ekman et al.,, 1975] the inhibition hypothesis: elements of facial expressions that are hardly done voluntarily, are also hardly inhibited [Ekman 2003] Social Context Based Emotion Expression 10

11 Complex facial expressions: masking (3/3) + + Reliable features of sadness Fake joy Neutral expression = Sadness masked by joy Social Context Based Emotion Expression 11

12 Complex facial expressions: masking (3/3) + + Reliable features of sadness Fake joy Neutral expression = Sadness masked by joy Social Context Based Emotion Expression 12

13 Facial expressions in presence of others The social context in which interaction between two individuals take place constrains them to control their facial expressions Set of procedures for the management of emotional displays in presence of others Obeying social rules about expression of emotions Politeness some redressive actions in order to save the public self-image that every individual wants to claim for himself. [Brown and Levinson 1987] Social Context Based Emotion Expression 13

14 Theory of politeness Variables : D the social distance of S and H P the relative power of H on S R the absolute ranking Do the FTA Don't do the FTA on record off record no redressive action, bald with redressive action positive politeness negative politeness The strategy is chosen: o Wx = D (S,H)+P(H,S)+Rx [Brown and Levinson 1987] Social Context Based Emotion Expression 14

15 Politeness and non verbal behavior Matthias Rehm and Elisabeth André, 2005 : Informing the Design of Embodied Conversational Agents by Analyzing Multimodal Politeness Behaviors in Human-Human Communication The relations between some types of gestures and verbal politeness strategies in face threatening situations. All videos show students when criticizing of another student presentation On each video full body of the person is seen Politness stategy bald 6 positive 20 negative 55 off 23 Number of cases Social Context Based Emotion Expression 15

16 Annotation and rules (1/2) Annotation: negative masked, negative inhibited, negative expressed, fake negative, no expression, fake positive, positive masked, positive inhibited, and positive expressed Two Variables: Social power (P), social distance (D) o used in the theory of politeness o used in the many experiments about application of display rules The sum of power and social distance (threat) is used to choose the strategy Social Context Based Emotion Expression 16

17 Annotation and rules (2/2) Rules: Threat level very low quite low quite high very high positive emotion no emotion negative emotion express neutral (express) express express fake positive express inhibit neutral (express) Inhibit express fake positive Mask Consistent with results of some experiments [LaFrance et al.,, 2005, Deutsch 1990, Wagner et al.,, 1991, Buck et al.,, 1992] Social Context Based Emotion Expression 17

18 Evaluation study (1\4): procedure Five different animations presented in random order Each animation presents the same story, but with different non-verbal responses Three different social contexts, interaction with: friend, stranger, superior Social Context Based Emotion Expression 18

19 Evaluation study (2\4): procedure Participants can see five videos in arbitrary order Each video presents Greta playing cards Participants were asked to guess who is Greta s adversary For each situation: friend, stranger, superior they had to choose the most appropriate video from five videos Only a few results at the moment Results depend on: ability to modify facial behaviour in social context emotional intelligence of participants Social Context Based Emotion Expression 19

20 Evaluation study (3\4): examples Social Context Based Emotion Expression 20

21 Evaluation study (4\4): first observations Some participants confuse real emotional state with socially acceptable expression People use adjectives like: tense, relaxed, unpleasant to describe Greta Some animations are considered annoying : why she is always smiling.. Social Context Based Emotion Expression 21

22 Perspectives and Future Work Model of complex facial expressions Model of social context Future Work Finish evaluation studies More detailed consideration of emotions (i.e. dominance, activation) Inter-cultural variables, personality features Social Context Based Emotion Expression 22