This paper is in press (Psychological Science) Mona Lisa s Smile Perception or Deception? Isabel Bohrn 1, Claus-Christian Carbon 2, & Florian Hutzler 3,* 1 Department of Experimental and Neurocognitive Psychology, Freie Universität Berlin, Germany 2 Department of Psychology, University of Bamberg, Germany 3 Department of Psychology & Center for Neurocognitive Research, Paris-Lodron-Universität Salzburg, Austria * Corresponding author: Florian Hutzler Department of Psychology & Center for Neurocognitive Research Paris-Lodron-Universität Salzburg Hellbrunner Straße 34 A-5020 Salzburg Austria E-mail address: Florian.Hutzler@sbg.ac.at Phone: +43 662 8044 5114, Fax: +43 662 8044 5126
What turns Mona Lisa s smile into such a mysterious expression? Livingstone (2000) has suggested that Mona Lisa s portrait changes its expression depending on where you look at it. Her mouth as the essential feature of her remarkable expression (Kontsevich & Tyler, 2004) conveys the impression of a vague smile. Due to sfumato technique (Gombrich, 2005), this impression of a smile is more prominent in gradual luminance changes that we perceive mainly in the periphery of our vision (low spatial frequency ranges). It is less prominent in fine details that (high spatial frequency ranges) we perceive only at the centre of our gaze. This is a peculiarity of her portrait as demonstrated by Livingstone (2000), that would be possible to further investigate by experimental manipulation of exposure duration (see Schyns & Oliva, 1994). Consequently, the smile one vaguely perceives while looking at Mona Lisa s eyes (when her mouth appears blurred) vanishes, when one attempts to verify this impression by looking at the mouth with maximum visual acuity. Hence, the proposed basis for the elusive quality of Mona Lisa s smile is that [...] you can t catch her smile by looking at her mouth. She smiles until you look at her mouth (Livingstone, 2000). We simulated this phenomenon for the first time experimentally via a saccade-contingent display change technique that allowed us to subliminally alter the expression of faces dependent on the beholder s gaze position. Method To mimic Mona Lisa s elusive smile analogous to Livingstone s hypothesis, a Mona Lisa condition was implemented in which faces had a smiling mouth only as long as the participants gazed towards a region around the eyes. As soon as the mouth was looked at directly, the mouth showed a neutral expression (Figure 1A). The accompanying display changes occurred during the participant s saccadic eye movements when visual perception is suppressed, thereby keeping the participants unaware of the experimental manipulation and their ratings unaffected by rational considerations. To realize these saccade-contingent display changes, the beholders gaze was guided by a fixation cross that was (horizontally centered) superimposed on the faces either between the eyes or on the mouth, covering a visual angle of 5.5 (which is comparable to the extent of the facial region of the original artwork at a distance of 120 cm). The fixation cross was first positioned at the eyes, then at the mouth, this sequence was repeated twice during the 2,400 ms of a trial. Thereby, three large vertical saccades were elicited during which the display changes occurred. During each trial, participants performed two long fixations at the eyes and two at the mouth (450-600 ms each).
To prevent participants from accidentally observing a display change, a trial was instantly terminated after a failure to fixate the crosses properly. In addition to the Mona Lisa condition, two control conditions were implemented: Stable smiling and stable neutral faces. To guarantee a comparable exploration of the faces across all three conditions, the participant's gaze was guided across the faces in the same fashion, although only during the Mona Lisa condition did display changes actually occur. One hundred morphed female faces, each in a smiling and a neutral version (selected on the basis of pre-test ratings of facial expression) served as a stimulus pool for the stable smiling and stable neutral condition, respectively. To realize the elusive smile in the Mona Lisa condition, the mouths of smiling faces were transferred to the corresponding neutral faces: The resulting neutral faces with smiling mouths were then presented alternately with the neutral faces with neutral mouths as described above. Each participant was presented with 25 stimuli from each of the three conditions (stable smiling, stable neutral, and Mona Lisa ) in a randomized order, whereby the sampling of these 3x25 experimental stimuli from the pool of 3x100 stimuli was counterbalanced across participants. The stimuli were presented on a 200Hz CRT-monitor with a resolution of 640x320 pixels, placed at 37cm distance from the participants. Gaze position was monitored with a sampling-rate of 250Hz from the left eye by a SensoMotoric Instruments (SMI) High-Speed tracking column. At the end of each trial, the participants rated the corresponding face on a 5- point Likert-scale according to i.) facial expression and ii.) confidence in this judgment, iii.) attractiveness, iv.) trustworthiness, and v.) mysteriousness. Care was taken to ascertain that the participants were not aware of the display changes: In a post-test interview, 23 of 42 participants reported that they had not seen a single display change and insisted on this notion even when explicitly being informed about the experimental manipulation. Seven of these remaining participants were excluded because they did not reach the inclusion criteria of a minimum of 9 non-terminated trials per condition. The data of the remaining 16 participants (8 female) with a mean age of 23;4 (years;months) was submitted for analysis.
Results and Discussion The elusive smiles in the Mona Lisa condition (which vanished, when being looked at directly) affected the appreciation of faces as implicated by Livingstone (2000). While stable smiling faces were unsurprisingly rated most positive, the faces in the Mona Lisa condition were also evaluated as more attractive and trustworthy and as having a more positive expression than stable neutral faces (see Figure 1B, all ps<.005) even though in both conditions ( Mona Lisa and neutral) participants saw neutral expressions when looking directly at the mouths. The elusive smiles in the Mona Lisa condition furthermore reduced the participants confidence of their expression ratings (p<.05), although they were not aware of the display changes. Importantly, the faces in the Mona Lisa condition were not perceived as more mysterious despite their elusive smile. The present study s findings add weight to Ambadar et al. s (2005) concern that we should not only focus on the study of intense and sustained expression, as our appreciation of a face can also be affected by subtle and fleeting expression (e.g., microexpression, Porter & ten Brinke, 2008) that do not require a direct gaze in order to be perceived. In sum, although peripherally perceived facial expressions affected the appreciation of a face, Mona Lisa s smile seems to constitute only part of her enigma. Even though we might catch her smile, she keeps her mystery, if this is indeed her inherent virtue.
References Ambadar, Z., Schooler, J. W., & Cohn, J. F. (2005). Deciphering the enigmatic face - The importance of facial dynamics in interpreting subtle facial expressions. Psychological Science, 16(5), 403-410. Gombrich, E. H. (2005). The story of art (16th ed.). London: Phaidon. Kontsevich, L. L., & Tyler, C. W. (2004). What makes Mona Lisa smile? Vision Research, 44(13), 1493-1498. Livingstone, M. S. (2000). Is it warm? Is it real? Or just low spatial frequency? Science, 290(5495), 1299-1299. Porter, S., & ten Brinke, L. (2008). Reading between the lies: Identifying concealed and falsified emotions in universal facial expressions. Psychological Science, 19(5), 508-514. Schyns, P. G., & Oliva, A. (1994). From blobs to boundary edges - evidence for time-scaledependent and spatial-scale-dependent scene recognition. Psychological Science, 5(4), 195-200.
Page 6 of 8 Manuscript under review for Psychological Science 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 Figure Caption Figure 1. (A) In the Mona Lisa condition, faces comprised a smiling mouth while the beholder gazed at the eyes but the mouth subliminally changed to neutral if fixated directly. (B) Participants ratings (M, SE) for faces in the Mona Lisa condition, as well as the control conditions comprising stable smiling and neutral faces. Differences between conditions were significant for all but the mysteriousness scale. For Review Only
Manuscript under review for Psychological Science Page 7 of 8 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 For Review Only 211x157mm (96 x 96 DPI)
Page 8 of 8 Manuscript under review for Psychological Science 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 For Review Only 181x127mm (200 x 200 DPI)