Emotion as Events (and Cause as Pre-Event)

Transcription

1 Chinese temporal/ discourse annotiation workshop Emotion as Events (and Cause as Pre-Event) Chu-Ren Huang The Hong Kong Polytechnic University June 1, 2010

2 What is Emotion (Cannon, 1927) Emotion is the felt awareness of bodily reactions to something perceived or thought. As such, an emotion is a state event with a cause event as its pre-event. In addition, humans are known to act on their emotions. This can be interpreted as that post-events are instigated by emotion events. An annotated emotion corpus allows us to study the causeevent interaction centered upon an emotion event, as well as its implications for temporal mark-up of events involved.

3 Motivation: Emotion as Event Event and Temporal Annotation Two Approached to Temporal Annotation of Event Temporal Relation between event pairs Which pair to annotate: annotate two events only if one is an argument of another or modifies another. Sparse Annotation Time inseparable part of event, which are happenings arranged in temporal sequences Time does not define events, event types instead dictates how temporal sequences are arranged

4 Emotion as a state is more typically treated as a post-event, the result of external triggers. However, we treat emotion as a pivot: cognitively salient mental states linking natural pre- and post events. Identification of Emotion hence allows the possibility of automatic temporal annotation. Pre-event and post-event establish the temporal sequence of sub-events and implied temporal relation in event annotation. Instead of event pairs, we have a pivoted tri-tuple which can be decomposed into two pairs

5 What has been done (in terms of GLML) Not A Lot (to be honest) An annotated emotion corpus, which can be treated as a set of main events With cause event (as pre-event of emotion) annotated Study to automatically detect and identify different emotions And their causes

6 Background of Emotion Annotation (1/2) Advantages of Cognitive Emotion: Cognitive emotions consistently occur across domains due to their human psychological underpinning. A fine-grained emotion model will provide a more robust and versatile model for NLP. Emotion computing often requires a large and high-quality annotated data.

7 Our Emotion Taxonomy Primary emotions (Turner 2000) happiness, sadness, anger, fear, and surprise For each primary emotion, it is divided into 3 levels: high, moderate, and low. Complex emotion Complex emotion can be divided into first-order, secondorder complex emotions, and so on. For each complex emotion, the involved primary emotions have different weights. E.g. pride (happiness + fear)

8 Emotion Keywords (1/2) Emotion keywords need to be classified according to the given emotion taxonomy. Emotion Verbs (Chang et al. 1996)

9 Emotion Keywords (2/2) There are 188 English lemmas and 226 Chinese emotion words in our taxonomy. Emotion types Happiness (primary) Anger + Fear (complex) Keywords High: ecstatic, eager, joy, enthusiastic, happy Moderate: cheerful, satisfy, pleased, enjoy, interest Low: sanguine, serene, content, grateful Jealousy: jealous Suspicion: suspicion, distrustful Abhorrence: abhorrence Table 1: Examples of emotion taxonomy

10 The Emotion Corpus (1/3) Our large corpus includes an emotion corpus (the 8 kinds of emotion sentences) and a neutral sentence corpus. The working corpus are the Sinica Corpus (Chinese), the Chinese Gigaword corpus, and the British National Corpus (BNC, English). Sentences containing the given emotion keywords are initially extracted through pattern matching. Each extracted sentence is marked with its emotion label according to the emotion type to which the emotion word in that sentence belongs.

11 The Emotion Corpus (2/3) The emotion corpus contains 208,382 emotion sentences for the 8 emotion types. By the pattern-based approach, the accuracy is 82.17% for Gigaword corpus, 77.56% for Sinica Corpus, and 69.36% for BNC corpus Two kinds of contextual structures are handled with: the negation structure and the modal structure. S1 (Neg_Happiness): I am not happy about that. S2 (Netural): Though the palazzo is our family home, my father had never been very happy there. S3 (Pos_Happiness): I 've never been so happy. S4 (Netural): I can die happy if you will look after them when I have gone. S5 (Netural): Then you could move over there and we'd all be happy.

12 The Emotion Corpus (3/3) The corpus also contains 81, 990 neutral sentences A sentence is considered as neutral only when the sentence itself and its context (i.e. the previous sentence and the following sentence) do not contain any of the given emotion words. By the pattern-based approach, the accuracy is 98.16% for Gigaword corpus, 98.39% for Sinica Corpus, and 99.50% for BNC corpus.

13 The Emotion Annotation Scheme (1/2) It is a layer just beyond a sentence, and encodes different-level emotion information for a sentence. Elements <emotion> element <emotiontype> element <primaryemotion> element <neutral> element The presence of primary emotion can make our annotation scheme more robust. Our annotation scheme has the versatility to provide emotion data for different applications.

14 The Emotion Annotation Scheme (2/2) <emotion> <emotiontype name = "surprise" keyword ="surprised"> <primaryemotion order = "1" name = "surprise" intensity = "moderate"></primaryemotion> </emotiontype> <emotiontype name = "jealousy" keyword = jealousy > <primaryemotion order = "1" name = "anger" intensity = "moderate"></primaryemotion> <primaryemotion order = "2" name = "fear" intensity = "moderate"></primaryemotion> </emotiontype> <s n = "1"> Hari was surprised at the rush of pure jealousy that swept over her at the mention of Emily Grenfell.</s> </emotion> <neutral> <s n = "2"> By law no attempts may be made to hasten death or prolong the life of the sufferer. </s> </neutral> <emotion> <emotiontype> <primaryemotion name = "sadness"></primaryemotion> </emotiontype> <s n = "3">He looked hurt when she did n't join him, his emotions transparent as a child 's. </s> </emotion>

15 Emotion Detection and Classification Emotion Classification The enumerative representation vs. The compositional representation Single-label classification vs. Multi-label classification Single-label classification: MaxEnt Multi-label classification: Binary Relevance (BR): one-vs-rest single-label classification Label Powset (LP): Treat each possible combination of labels as a unique label, and hence convert multilabel classification to single-label classification. Hybrid Label Powset (HLP): a combination of BP and LP. (Godbole & Sarawagi, 2004)

16 Performances for Multi-label Classification All of these multi-label classification methods outperform single-label classification Accuracy Micro F1 Macro F1 BP LP HLP Our experiments show that the compositional representation allows better emotion classification The compositional way to represent an emotion permits emotion classification to detect different facets of an emotion.

17 Event-Emotion Interaction Emotion as a Special Types of Event States Emotions may be affected by many factors. Among them, the causal and caused events are the most important factors. Different emotion will have different pre- and postevents. We adopt the event ontology of Generative Lexicon theory (GL, Pustejovsky 1995) as well as the Generative Lexicon Markup Language (GLML) annotation scheme. Construction of event and emotion ontology as well as event-annotation on a large-scale corpus. Automatic analysis of opinion dynamics based on the ontologies and the annotated corpus.

18 Emotion Cause Events (1) Verbal events: events that involve verbs Proposition Mei2-not xiang3dao4-think ta1-3.sg.f shuo1-say de-poss dou1-all shi4-is zhen1-true hua4-word, rang4-lead ta1-3.sg.m zhen4jing1-shocked bu4yi3- very. He was shocked that what she said was the truth. Nominalization Zhe4-DET ci4-cl yan3chu1-performance de-poss jing1zhi4-exquisite dao4shi4-is ling4-cause wo3-1.sg shi2fen1-very jing1ya4-surprise. I was very surprised by this exquisite performance. (2) Nominal Events: events that are simply nouns Dui4yu2-for wei4lai2-future, lao3shi2shuo1-frankly wo3-1.sg hen3-very hai4pa4-scared. Frankly, I am very scared about the future.

19 Emotion cause corpus (Lee et al., 2010) Cause data No. of Instances Emotions Extracted Emotional With Causes Happiness 1,644 1,327 1,132 (85%) Sadness (76%) Fear (82%) Anger 1, (74%) Surprise 1, (85%) Total 5,958 4,260 (72%) 3,460 (81%)

20 Annotation Scheme 573 Y 0/shang1xin1/Sadness <PrefixSentence> ma1ma ye3 wen4 le ling2 ju1, dan4 shi4 mei2 you3 ren4 jian4 dao4 xiao3 bai2. </PrefixSentence> <FocusSentence>wei4 le [*01n] zhe4 jian4 shi4 [*02n], wo3 ceng2 <emotionword id=0>shang1 xin1</emotionword> le hou2 jiu3,dan4 ye3 wu2 ji3 yu4 shi4. </FocusSentence> <SuffixSentence>mei3 dang1 zai4 kan4 dao4 bai2 se4 de qi4 gou3, bu4 jin4 hui4 xiang3 qi3 xiao3 bai2 </SuffixSentence> 573 Y 0/to be sad/sadness <PrefixSentence> Mom also asked the neighbors, but no one ever saw Little White. </PrefixSentence> <FocusSentence> Because of [*01n] this [*02n], I have been feeling very <emotionword id=0> sad </emotionword> for a long time, but this did not help. </FocusSentence> <SuffixSentence> Whenever [I] see a white stray dog, [I] cannot help thinking of Little White. </SuffixSentence>

21 Corpus Analysis I Cause event types and position of each emotion Emotions Cause Type (%) Cause Position (%) Event Nominal Left Right Happiness Sadness Fear Anger Surprise Cause events tend to be expressed by verbal events than nominal events Cause events tend to occur at the position to the left of the emotion keyword

22 Corpus Analysis II Seven groups of linguistic cues Group Cue Words I causative verbs to cause : rang4, ling4, shi3 II reported verbs to think about : e.g. xiang3 dao4, xiang3 qi3, yi1 xiang3 to talk about : e.g. shuo1dao4, jiang3dao4, tan2dao4 III verbs of saying to say : e.g. shuo1, dao4 IV to see : e.g. kan4dao4, kan4jian4, jian4dao4 epistemic markers to hear : e.g. ting1dao4, ting1 shuo1 to know : e.g. zhi1dao4, de2zhi1, fa1xian4 to exist : you3 V prepositions for as in I will do this for you : wei4, wei4le for as in He is too old for the job : dui4, dui4yu2 VI conjunctions VII others because : yin1, yin1wei4, you2yu2 is : deshi4 at : yu2 can : neng2 Group I marks the end of the cause events and the other six groups mark the beginning of the cause events

23 Illustration Rule 1 i) C (B/F) + I(F) + E(F) + K(F) ii) E = the nearest Na/Nb/Nc/Nh after I in F iii) C = the nearest (N)+V+(N) before I in F/B [C yi1 la1 ke4 xi4 jun1 wu3 qi4 de bao4 guang1], [I shi3] [E lian2 he2 guo2 da4 wei2][k zhen4 jing1]. [C The revealing of Iraq s secret bacteriological weapons] [K shocked] [E the United Nations].

24 15 Linguistic Rules Two sets of rules: (i) specific rules (1-13) that apply to all emotional sentences; (ii) general rules (14-15) that apply to the sentences in which causes are not found using specific rules No. Rules 1 i) C(B/F) + I(F) + E(F) + K(F) ii) E = the nearest Na/Nb/Nc/Nh after I in F iii) C = the nearest (N)+(V)+(N) before I in F/B 2 i) E(B/F) + II/IV/V/VI(B/F) + C(B/F) + K(F) ii) E=the nearest Na/Nb/Nc/Nh before II/IV/V/VI in B/F iii) C = the nearest (N)+(V)+(N) before K in F 3 i) II/IV/V/VI (B) + C(B) + E(F) + K(F) ii) E = the nearest Na/Nb/Nc/Nh before K in F iii) C = the nearest (N)+(V)+(N) after II/IV/V/VI in B 4 i) E(B/F) + K(F) + IV/VII(F) + C(F/A) ii) E = a: the nearest Na/Nb/Nc/Nh before K in F; b: the first Na/Nb/Nc/Nh in B iii) C = the nearest (N)+(V)+(N) after IV/VII in F/A

25 Specific Rules I No. Rules 5 i) E(F)+K(F)+VI(A)+C(A) ii) E = the nearest Na/Nb/Nc/Nh before K in F iii) C = the nearest (N)+(V)+(N) after VI in A 6 i) I(F) + E(F) + K(F) + C(F/A) ii) E = the nearest Na/Nb/Nc/Nh after I in F iii) C = the nearest (N)+(V)+(N) after K in F or A 7 i) E(B/F) + yue4 C yue4 K the more C the more K (F) ii) E = the nearest Na/Nb/Nc/Nh before the first yue4 in B/F iii) C = the V in between the two yue4 s in F 8 i) E(F) + K(F) + C(F) ii) E = the nearest Na/Nb/Nc/Nh before K in F iii) C = the nearest (N)+(V)+(N) after K in F 9 i) E(F) + IV(F) + K(F) ii) E = the nearest Na/Nb/Nc/Nh before IV in F iii) C = IV+(an aspectual marker) in F

26 Specific Rules II No. Rules 10 i) K(F) + E(F) + de possession (F) + C(F) ii) E = the nearest Na/Nb/Nc/Nh after K in F iii) C = the nearest (N)+V+(N)+ +N after de in F i) C(F) + K(F) + E(F) ii) E = the nearest Na/Nb/Nc/Nh after K in F iii) C = the nearest (N)+(V)+(N) before K in F i) E(B) + K(B) + III (B) + C(F) ii) E = the nearest Na/Nb/Nc/Nh before K in F iii) C = the nearest (N)+(V)+(N) after III in F i) III(B) + C(B) + E(F) + K(F) ii) E = the nearest Na/Nb/Nc/Nh before K in F iii) C = the nearest (N)+(V)+(N) after III in B

27 General Rules General rules No Rules i) C(B) + E(F) + K(F) ii) E = the nearest Na/Nb/Nc/Nh before K in F iii) C = the nearest (N)+(V)+(N) before K in B i) E(B) +C(B) + K(F) ii) E = the first Na/Nb/Nc/Nh in B iii) C = the nearest (N)+(V)+(N) before K in B

28 Constraints Constraints are set to each rules to filter out incorrect causes E.g., Rule 1: the emotion keyword cannot be followed by the words de possession / deshi4 is that / shi4 is since it is very likely to have the cause event occurring after such words Rule 2: the cue word yuo3 to exist in III is excluded as it causes considerable noises Rule 4: only applies to instances containing keywords of happiness, fear, and surprise

29 Experiment 80% of the data as the development data 20% of the data as the test data Baseline find a verb to the left of the keyword in question, and consider the clause containing the verb as a cause

30 Evaluation Metrics Phase 1: assesses the recognition of an emotion cooccurrence with a cause Phrase 2: evaluates the detection of the cause texts for an emotion Overall Evaluation

31 Definition of Metrics

32 Table 6: The Overall Accuracy in Phase 1 Table Table 7: The 5: Detailed The Overall Performances in Phase 1 Results I Relaxed Match 1 Relaxed Match 2 Precision Recall F-score Precision Recall F-score Baseline O u r System Table 1: The overall performance Table 1: The overall performances Baseline Rule-based System Accuracy Table 2: The overall accuracy in Phase 1 Baseline Rule-based System Emotions Precision Recall F-score Precision Recall F-score With causes Without causes Table 3: The detailed performances in Phase 1

33 Table 6: The Overall Accuracy in Phase 1 Table Table 7: The 5: Detailed The Overall Performances in Phase 1 Results II Relaxed Match 1 Relaxed Match 2 Precision Recall F-score Precision Recall F-score Baseline Table 1: The overall performance O u r System Table 4: The detailed performances in Phase 2 Top three accurate rules: rules 7, 10, and 11 Top three contributive rules: rules 2, 15, and14

34 Conclusion Given an event type, it is possible to automatically identify the main event and its cause (pre-event), hence establishing their temporal relations. Sophia Yat Mei Lee, Ying Chen and Chu-Ren Huang A Text-driven Rule-based System for Emotion Cause Detection. NAACL-HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text. Chen, Ying, Sophia Lee, and Chu-Ren Huang A Cognitive-based System for Emotion Annotation LAW III, ACL-IJCNLP 2009.

35 Thank you