Professor Tom Troscianko

Transcription

1 What can films teach us about vision? Professor Tom Troscianko Department of Experimental Psychology University of Bristol

2 A general question What is a movie and why is it fun to watch?

3 Some scientific questions How do we perceive a changing world? Is the world as depicted in film markedly different from the one outside the cinema? How do any such differences affect perception when viewing a film?

4 A practical question Is it worth buying a large TV?

5 The traditional approach to vision Attention Models in the head

6

7

8 Fixation and saccades Our fovea is the only high-resolution area within the visual field Meaning that rapid shifts of fixation saccades are required to gain information about objects in different parts of the visual field

9

10

11 Stev

12 Stev X

13 Stev

14

15 Dynamic scene inspection Show Iain Gilchrist videos

16 Models in the head? So - we make eye movements and traditional accounts of vision assumed that we construct a detailed model of the world in our head It feels to us that this is what is happening But change blindness experiments suggested otherwise

17 If we have a detailed representation in our head, then we should have no difficulty in noticing changes in a scene which occur across interruptions of processing tasks

18

19

20

21 The trans-saccadic method Blackmore SJ, Brelstaff G, Nelson K, Troscianko T (1995) Is the richness of our visual world an illusion? - Trans-saccadic memory for complex scenes. Perception 24,

22

23

24 Basketball video Show video Monkey Business

25 More modern accounts of vision We pick up information about a scene from eye movements and this is processed in a manner which guides action Fixations pick up information about: Items with which we are about to interact The gross geometry of the scene Image locations which provide answers to questions that we are currently asking

26 Ilya Repin (ca. 1884) Unexpected visitor

27 Yarbus ( ).

28 Yarbus ( ).

29 Scanpaths from Yarbus (1967)

30 Scanpaths from Yarbus (1967) Free viewing Material circumstances Age of people Doing what before arrival? Clothes Position of people and objects How long has visitor been away?

31 Tatler et al (2010)

32 Tatler et al (2010)

33 Sensorimotor theory of vision O'Regan, J.K., & Noe, A. (2001). A sensorimotor account of vision and visual consciousness. Behavioral and Brain Sciences, 24(5), The central idea of our new approach is that vision is a mode of exploration of the world that is mediated by knowledge of what we call sensorimotor external memory that can be probed at will by the

34 Cinematic viewing The visual world of the cinema is quasistable It keeps jumps to a new shot with a mean frequency around 5 sec relevant perceptual processes can survive However, the basic signal is unstable What does vision science predict the effect of this global instability to be?

35 Vision science prediction If an instability occurs as a result of a saccade, we perceive stability in spite of most receptor signals being very different (Bridgeman 2010) visual instability will make the world appear unstable Such an unstable stimulus is likely to be aversive can cause migraine at the right temporal frequency (Arnold Wilkins)

36 Movie: continuity error Show video Office

37 Ceci pas la verité!

38 What are the timing statistics of movies? Cutting JE, DeLong J, Nothelfer CE (2010) Attention and the evolution of Hollywood film. Psychological Science doi: / Studies shot length statistics for 150 films from 1935 to 2005 Measured autocorrelation function for shot length setting the actual narrative aside, perhaps being engrossed in a film is, in part, to allow its 1/f temporal

39 Cutting paper

40 Film shots are getting shorter! Salt, B. (2009). The Shape of The New Review of Film and Television Studies, 7(4)

41 Unstable world of movies now with a story!

42 Perception of films The autocorrelation argument suggests be modified to allow for an unstable world whose temporal statistics conforms to a certain law We shall return to test this issue directly Meanwhile, we need a measure of

43 What can we measure? We shall points during the movie -

44 How to measure presence We measured subjective ratings of presence using a simple line-bisection task Each time a sound beeped or a light flashed, subjects made a mark on a line to indicate how much they X low presence high presence

45 What film to use? Hasson et al (2004) Science 209, United Artists, Sergio Leone director) fmri study Established three main types of scene giving reliably different profiles of response in the visual cortex: Faces (Fusiform gyrus, FFA) Landscapes/buildings (Colleteral sulcus, PPA) Hand-related scenes (Post-central sulcus) Therefore, if we use this movie and the points that give high activation in FFA and PPA, we will be studying the brain when it has a different pattern of response.

46 What film to use?

47 FFA scenes

48 FFA scenes

49 PPA scenes

50 PPA scenes

51 Screen size experiments Thanks to Lee De-Wit for the picture

52 Presence results Screen size p<0.001 Scene type p<0.001 NS interaction

53 Presence and memory The effect of screen size on presence and recognition memory for scenes

54 Presence results Screen size p<0.005 Scene type p<0.05 NS interaction

55 Memory results Screen size p<0.001 Scene type p<0.001 Size x type interaction p<0.001

56 Presence, reaction time, and arousal The effect of screen size on presence, reaction time, and arousal

57 What can we conclude? Presence ratings appear to reflect the degree of arousal of the viewer similar pattern of results to RT and pupil dilation Whereas recognition memory appears to operate on different principles Rated presence is a reliable measure of the degree of immersion in a movie

58

59 And now to something more interesting than screen size Nobody has properly studied the difference between colour and black & white movies Matched groups saw one or other movie

60 show installed a stereo projection system for 250k in the Watershed cinema In 3D and 2D

61 Avatar

62 Finally: what is the effect of visual instability on presence? The shoot-out scene experiment We used shoot-out scenes from obscure Western movies made in early 1960s Created a 26 minute showreel and tested presence on two groups of 21 subjects

63 Presence and mean shot length

64 Presence and mean shot length: replication with Good, Bad, Ugly

65 Presence and mean shot length in Avatar 2D r = -.626, p <.001 3D r =-.526, p =.003

66 Editing and presence The problem with the data up to now is that a movie editor can choose to put short shots in the more exciting parts of the movie What we need is a means of manipulating shot length independently of narrative Achieved this by making a video recording of 30 mins of a stage play with three static cameras Control condition: uncut video/sound from one camera Edit 2: like Edit 1, but in different time phase

67 Editing and presence: stage play

68 Editing and presence The presence in the edited conditions is significantly higher than in the control condition without edits These edits are not relevant to the narrative, being taken from an unrelated movie Suggesting that adding visual discontinuities

69 So: what does all this mean? The result that short shots give high presence is hard to explain with existing theories of vision such as the Stable World Assumption So what is going on Could it be that things which are unusual and tending towards being aversive/unpleasant get us excited?

70 Aversive stimuli studied by Arnold Wilkins

71 Wilkins asked people to classify modern art

72 A repetition

73

74

75

76

77

78 Design Debbie Ayles, 2004

79 A further repetition

80 A similar study with photographs from: Pretty J, Peacock J, Sellens M, Griffin M. The mental and physical health outcomes of green exercise. International Journal of Environmental Health Research 2005; 15:

81 Fernandez and Wilkins (2008)

82 So: what does all this mean? Wilkins and colleagues have identified that systematic deviations from normality in static images can produce an aversive response In a similar manner, movies have a temporal frequency spectrum which is unnatural Both types of stimulus (the Wilkins images and movies) produce high excitation in visual cortex This may be associated with high degrees of arousal, and therefore presence Like movies, the Wilkins images are compelling (Wilkins, private communication)

83 So: what does all this mean? So: A stimulus which violates the properties of natural scenes can be arousing The editing of movies therefore may make them more arousing The skill lies in making the visual story possible to follow in spite of these discontinuities

84 Op-art

85 Bridget Riley

86 Conclusions Films are immersive and arousing The degree of arousal is (partly) determined by the viewing situation The arousal in films is augmented by the temporal discontinuities (edits) in the film Films tend towards being aversive stimuli, but the (mild) levels of aversion appear to result in arousal/excitement we get excited by a world which seems slightly dangerous but we know we are safe!

87 A practical answer a large TV!

88 Flirting with danger: adult version

89 Flirting with danger: