FILMS, TEXTS, AND REAL-LIFE EXPERIENCES STEPHAN SCHWAN LEIBNIZ-INSTITUT FÜR WISSENSMEDIEN (IWM)
Why media? Media evolved to allow for relevant human experiences independent of space and time Substitutes for direct personal experiences Various forms: verbal storytelling, theater performance, novels, comics, films,...
Real World Situations and Events Commonalities / Differences? Text narration Film narration
Real World Situations and Events Commonalities / Differences? Film narration
Preservation of basic visual features in shots, including shapes, colours, dynamics Introduction of cinematographic techniques Rapid two-dimensional projection of static picture frames Abrupt shot transitions (cuts, fades etc.) Extradiegetic elements (e.g. music) Visual transformations (slow motion, fast motion) Ø Overlapping audiovisual experiences: Partly similar, partly different Real world <> Films
Are films sufficiently similar to real world experience to allow for immediate comprehension or do they require film literacy? Real world <> Films
Pictorial relation idenbty space Activity relation high experience consistent low experience no experience inconsistent Conceptual relation gaze inside/outside 0 10 20 30 40 50 60 70 80 90 100 Schwan & Ildirar, 2010; Ildirar & Schwan, 2014 Real world <> Films
Pictorial relation idenbty space Activity relation high experience consistent low experience no experience inconsistent Conceptual relation gaze inside/outside 0 Real world <> Films 10 20 30 40 50 60 70 80 90 100
How do films use cinematographic techniques to shape experience? Attention guidance Space: Viewpoint selection Time: Event segment selection Ø Cognitive offloading through simplification and optimization Real world <> Films
Eye movements Eye blinks Nakano et al., 2009 Real world <> Films
CorBcal acbvity Hasson et al. (2008)! CuRng et al. (2011) Real world <> Films
Optimal canonical viewpoints In real life, observer s viewpoint is often suboptimal Optimal viewpoint may change during event In films, camera position / cuts may change accordingly Spatial disorientation caused by cuts can be compensated by continuity editing GarsoWy, Huff & Schwan (2009), Huff & Schwan (2012) Real world <> Films
Real world <> Films Schwan & GarsoWy (2004)
Substantial perceptual overlap between real life conditions and film depictions, but also systematic differences Medium independent kind of narrative comprehension, but films require an additional film literacy Viewers delegate perceptual control of experience to a large extent filmmaker Filmmaker optimizes experience for viewers in terms of space, time, and attention Films are easier to process than real life? Real world <> Films
Text narration Commonalities / Differences? Film narration
Films <> Texts What do we know about commonalities / differences between processing/remembering (narrative) texts and films?
Showing vs. Telling / high vs. low overlap with real life perceptual conditions Translation of arbitrary symbols into a situation model / mental image Films <> Texts
Transportation: a state of cognitive and emotional immersion in a text. Transported readers are completely focused on the story world; they may lose track of time, fail to notice events going on around them, and experience vivid mental images of settings and characters. ü Higher vividness ü Less effort required ü Additional audio stimulation ü No need for imagining scenes - Less control - Imagination is limited by pictures Films <> Texts
Four conditions: Text-Text Text-Film Film-Text Film-Film First session: No differences in transportation More perceived effort for text Second session transportation: Text-Film > Film-Text/Text-Text > Film-Film Statistical interaction of medium with need for cognition at first session Films <> Texts
How to establish information equivalence between a film and a text? Films <> Texts
Films <> Texts
1 Segmentation of the text into discrete episodes, and episodes into exposition/complication/resolution by a sample of students 2 Segmentation of the movie by another student sample 3 Mapping of episodic components of text and movie 4 Rewriting of text to achieve better mapping 5 Repeat step 1 with revised text 6 Match of text fragments to movie 7 Match of movie stills to text 8 Conduct memory experiment: Text vs. Movie Films <> Texts
100 80 60 40 Movie Text 20 0 0 Days 7 Days Movie protocols were longer and contained more details More wrong intrusions in the text protocols Films <> Texts
Higher-level goal-directed activites are better remembered than lower level goal directed activities Goal-directed activities are better remembered than non-goal-directed activities Canonical orderings are almost perfectly remembered Activities at non-canonical positions are moved to canonical positions in memory Ø No substantial differences between texts and films Films <> Texts
More commonalities than differences between texts and films Texts require more cognitive effort than films Steeper gradient of forgetting for texts, particularly for details Films <> Texts
Real World Situation Text narration Commonalities / Differences? Film narration Real World Situation
Episodic memory: What where - when Vivid recollection (remember vs. know) Related to self (autobiographical memory) Full range of information both of outside world (visual, auditive percepts etc) and inner states (thoughts, bodily feelings, emotions) Simultaneous occurence Packed together as memory trace Activating one experiential element may reactivate whole package Episodic memory <> Film & Text
First Order Experience Second order experience Real life: I read a book / watch a movie TransportaBon Medium: I witness the activities / thoughts / feelings of fictional charcaters Simulataneous double episodic memory or mutually exclusive? The better second order memory, the worse first order memory (and vice versa)? Episodic memory <> Film & Text
First order experience: Conditions of viewing versus reading From specialized to multi-purpose environments From single shot viewing to multiple viewing episodes (feature films -> TV series) Films: Great variability in presentation formats Ø Film usage patterns become similar to book usage Ø More diversity within than across media? Episodic memory <> Film & Text
Films: Second order experience Audio-visual representation of outside world (situations, events) No direct representation of inner states Simultaneous presentation Texts: Symbolic representation of outside world Description of inner states Sequential presentation Individualized mental images Episodic memory <> Film & Text
Open questions: (How) do we integrate fictional surrogate experiences into our epsiodic/autobiographic memory? Or do we have a seperate episodic memory of fictional experiences? Do texts or films induce more elaborate episodic memory traces? Do other media-independent aspects (e.g. firstperson vs. Third-person perspective) play a more important role?...? Episodic memory <> Film & Text
Summary 1 Three instead of two components: Texts, Films AND Real World Experiences 2 More commonalities than differences between processing of films and texts 3 Main difference: Films require less effort than texts (even less than real world experiences?) 4 It s not only the differences texts and films, but also between reading and viewing 5 Individual differences may matter 6 Role of episodic memory for texts/films is complex 7 MUCH more empirical research is needed...