Visualizing the Affective Structure of a Text Document Hugo Liu, Ted Selker, Henry Lieberman MIT Media Laboratory {hugo, selker, lieber} @ media.mit.edu http://web.media.mit.edu/~hugo
Overview Motivation Strategies for Visualizing Affect Affective Classification of Text Using Real-World Knowledge Rendering Affective Structure Navigating Text Documents By Affect Future Directions 2
Motiv^tion Scenario! You re watching your Bambi DVD! You tell your DVD agent: take me to the really really sad part 3
Motiv^tion Thematic & Affective Indexing! Thematic Indexing Aids in Document Navigation! Table of Contents! DVD Scene Selection! Thematic units chapters, sections, and subsections important events! Affective Indexing Hypothesis:! can aid in document navigation 4
Str^tegies for Visu^liz^tion Affective Structure alone may be too abstract! Affective structure of stories is not as clearly delineated as thematic structure People may often agree, but not always! Affective structure alone is too abstract! Try this! Thread affective and thematic structures together along a story timeline The Fight Haunted by Ghosts The Confrontation Resuming Life Again 5
Str^tegies for Visu^liz^tion How can we show affect? (Assuming a timeline representation)! Plot a single emotion! PAD model (Mehrabian, 1995) Pro: comprehensive Con: dimensions are latent variables " harder to sense from text! Plot a composite of emotions Pro: get gestalt; Con: less information 6
Str^tegies for Visu^liz^tion Can we show multiple emotions? (Assuming a timeline representation)! Yes, but it could be confusing to visualize! COLORS! PROs! visually distinct! can codify a large number of categories! identifiable even in small slices in a timeline! Certain colors are inherently emotionally evocative (Valdez & Mehrabian, 1994) CON: affective colors may be culturally sensitive CON: still a learn-it component to any encoding scheme 7
@ffective Cl^ssific^tion of text Traditional Approaches to Textual Affect Sensing! Keyword Spotting The Affective Reasoner (Elliott, 1992) I had a really bad day at work. I got fired today.! Statistical NLP methods Requires very large input Hard to pick appropriate training corpus I had a really bad day at work. I got a pink slip and was shown the door. I can t believe my boss let me go so suddenly. And I got no severance pay. And I was the only person to be let go so I can t claim it was some mass firing. Maybe it was a conspiracy. To do away with me because I was too good for them. I got a pink slip and was shown the door. I can t believe my boss let me go so suddenly. And I got no severance pay. And I was the only person to be let go so I can t claim it was some mass firing. Maybe it was a conspiracy. To do away with me because I was too good for them. I had a bad day at work. (input too short)! Handcrafted models not comprehensive 8
@ffective Cl^ssific^tion of text Textual Affect Sensing from Real- World Knowledge (Liu et al., 2003)! We introduce a novel approach to sense the affective nature of a text s underlying meaning. I got fired today.! A consequence of getting fired is unemployment! People do not want to be unemployed.! Employment is used to make money.! Money buys food.! People cannot survive without food!! People like to survive.! We use a corpus of everyday world knowledge like shown above, called Open Mind Common Sense (Singh, 2002). 9
@ffective Cl^ssific^tion of text Open Mind Commonsense! A corpus of real-world knowledge containing 500,000 English sentences of common sense! Some directly affective (8%) Some people are scared of ghosts (affective)! But mostly non-affective knowledge. Ghosts are often found in haunted houses (non-affective)! PChaining: Propagate scary to haunted houses! Two plys of propagation give us 30% coverage 10
@ffective Cl^ssific^tion of text Emotus Ponens! Uses Open Mind to Affectively Classify Sentences into six basic Ekman Emotions: surprise, happiness, fear, anger, disgust, sadness (plus neutral).! Proposed by Ekman (1984) from research on universal facial expressions! In this work, we combine this with keyword-based sensing for more comprehensive sensing for stories 11
Rendering ^ffective structure Rendering Affective Structure! Input: Affectively annotated sentences [happy,sad,angry,fearful,disgusted,surprised]! Heuristic Smoothing Algorithm (Zooming Out) Naïve voting is not good enough: 49 sad/51 neutral There are natural dependencies between Ekman emotions! e.g. Sad, Angry, and Fearful are all high in anxiety Bayesian network used to model dependencies! Trained over 510 sentences across 15 story texts Aligning emotions with layout and thematic breaks Shifts of discourse, e.g. all of a sudden, or surprisingly.! Output: hyperlinked affective color bar 12
N^vig^ting text by ^ffective structure AffectiveColor! Encoding Scheme Happy Sad Angry Fearful Disgusted Surprised! Demos Little Red Riding Hood Fall of the House of Usher! Does AffectiveColor improve text navigation?! Indicative user study run over stories 13
N^vig^ting text by ^ffective structure User Study! 4 users! 4 story documents each (e.g. news article, novel)! Chosen individually for each user Two that each user characterized as familiar Two unfamiliar! Users performed two timed information access tasks per story, given thematic cues for important events e.g. navigate to where the wolf eats the grandmother.! Control: AffectiveColor were color is uniformly yellow! Results: All users improved with Affective Color 27% speed improvement for familiar story documents 36% for unfamiliar story documents Some confusion about color! Longer term study needed, after colors learned 14
Future Directions! Vary the visual representation! Extend work to non-story texts! Personalized affect sensing Commonsense models the typical person Personality Paintings: Model individuals 15
to contact us Hugo Liu Henry Lieberman Ted Selker {hugo, lieber, selker} @ media.mit.edu 16