New Social Signals in a New Interaction Context

Transcription

1 New Social Signals in a New Interaction Context Alessandro Vinciarelli University of Glasgow vincia@dcs.gla.ac.uk

2 Online Interactions 80% % 67% 71% 71% 40% 20% 20% 28% 28% 22% 21% 15% 13% 26% 17% 16% 18% 23% 0% Facebook LinkedIn Pinterest Instagram Twitter Duggan, Ellison, Lampe, Lenhart & Madden, Social Media Update 2014, Pew Research Center,

3 New Interactions The human brain evolved in a world in which only humans exhibited rich social behaviours, and a world in which all perceived objects were real physical objects. Anything that seemed to be a real person or place was real. Reeves & Nass, The Media Equation, CSLI,

4 New Interactions Old Brain The human brain evolved in a world in which only humans exhibited rich social behaviours, and a world in which all perceived objects were real physical objects. Anything that seemed to be a real person or place was real. Reeves & Nass, The Media Equation, CSLI,

5 New Social Signals Cues Attractiveness Clothes, etc. Expressions Gaze, etc. Vocalisations, Prosody, etc. Self-touching, Orientation Distances, Seating, etc. Codes Physical Appearance Face Behavior Voice Behavior Gestures and Postures Space and Environment Functions Forming Impressions Expression Emotion Relational Messages Deception Management Power Persuasion Richmond and McCroskey, Nonverbal Behaviors in Interpersonal Relations, Allyn and Bacon,

6 New Social Signals Cues Attractiveness Clothes, etc. Expressions Gaze, etc. Vocalisations, Prosody, etc. Self-touching, Orientation Distances, Seating, etc. Codes Physical Appearance Face Behavior Voice Behavior Gestures and Postures Space and Environment Functions Forming Impressions Expression Emotion Relational Messages Deception Management Power Persuasion Richmond and McCroskey, Nonverbal Behaviors in Interpersonal Relations, Allyn and Bacon,

7 New Social Signals Cues Codes Functions Forming Impressions Expression Emotion Relational Messages Deception Management Power Persuasion Vinciarelli & Pentland, New Social Signals in a New Interaction World: The Next Frontier of SSP, IEEE SMC Magazine (to appear),

8 Outline Personality and its Measurement Personality and Computing Voice and Face of Personality Online Personality Conclusions

9 Online Impressions (I) [...] the audience layer sits beyond the weak ties layer. It is made up of strangers [that] can play constructive roles when they are activated. Rainie & Wellmann, Networked, MIT Press,

10 Online Impressions (II) Photos and videos as social currency online [ ] Overall, 56% of Internet users do at least one of these [image] creating or curating activities. Rainie, Brenner and Purcell, Photos and Videos as Social Currency Online, Pew Research Center,

11 Pictures as Social Signals Cues Images Tagged as Favorite Codes Functions Forming Impressions Expression Emotion Relational Messages Deception Management Power Persuasion Segalin, Perina, Cristani & Vinciarelli, Unveiling the Multimedia Unconscious, Proceedings of ACM Multimedia, pp , 2013 (BNI paper) 11

12 Setting Control Experimental Setup Scenario Manipulation Naturalistic Field Observation Experiments Controlled Observation Laboratory Experiments Hecht, Guerrero, Perspectives on Nonverbal Research Methods, in The Nonverbal Communication Reader, Guerrero, De Vito, Hecht (Eds.), pp ,

13 Corpus Collection BFI-10 1st Person BFI-10 3rd Person BFI-10 1st Person Self-Assessment Flickr Assessment 13

14 The PsychoFlickr Corpus Number of Galleries Number of Pictures Number of Subjects 300 Gender Balance Category Balance % M / 28.7% F 100% Pro Users Subjects Distribution 100% = 300 Assessors 11 (UK) + 11 (Asia) Total Segalin, Items Perina, Cristani & Vinciarelli, Unveiling the Multimedia Unconscious, Proceedings of ACM Multimedia, pp , 2013 (BNI paper)

15 Aesthetic Preferences 17

16 Texture 18

17 Entropy Tamura Coarseness Tamura Contrast Tamura Directionality GLCM Contrast GLCM Correlation GLCM Energy GLCM Homogeneity Texture O C E A N 19

18 Composition 20

19 Color 21

20 Use of Light Average Saturation Valence Dominance Arousal Color Diversity Hue Circ. Variance Edge Pixels Level of Detail Low Depth of Field Color and Composition O C E A N 22

21 Black Blue Valence Dominance Arousal Color Diversity Hue Circ. Variance Edge Pixels Level of Detail Low Depth of Field Color Distribution O C E A N 23

22 Multiple Instance Regression Openness Conscientiousness Extraversion Agreeableness Neuroticism 0 Naive cit-knn clust-reg Topic-Sum Gen-Mog Gen-LDA CG Correlation (Spearman Coefficient) between actual and predicted traits. 24

23 Correlational Analysis 25

24 Outline Personality and its Measurement Personality and Computing Voice and Personality Face and Personality Conclusions

25 Further Reading A.Vinciarelli & G.Mohammadi, A Survey of Personality Computing, IEEE Transactions on Affective Computing, 5(3): , 2014 A.Wright, Current Directions in Personality Science and the Potential for Advances through Computing, IEEE Transactions on Affective Computing, 5(3): , 2014 A.Vinciarelli & G.Mohammadi, More Personality in Personality Computing, IEEE Transactions on Affective Computing, 5(3): ,

26 Conclusions Personality Computing is important for any technology dealing with people Personality Computing should move from the prediction of traits to the prediction of consequential outcomes Tighter integration with Personality Science is needed to improve both Computing and Psychology 28

27 Thank you! Noura Al Moubayed (University of Glasgow) Marco Cristani (University of Verona) Gelareh Mohammadi (Idiap Research Institute / EPFL) Maurizio Filippone (University of Glasgow) Alex McKay (University of Glasgow) Antonio Origlia (University of Napoli Federico II) Alessandro Perina (Microsoft Research) Cristina Segalin (University of Verona) Yolanda Vazquez Alvarez (University of Glasgow) Bjoern Schuller (Imperial College / TU Munich) 29