Personal Space Augmented Reality Tool

Transcription

1 Personal Space Augmented Reality Tool Julien Leroy - Matei Mancas - Bernard Gosselin University of Mons - UMONS Engineering Faculty (FPMs) - IT Research Unit - TCTS Lab Place du Parc 20, B-7000 Mons, Belgium { julien.leroy, matei.mancas, bernard.gosselin Abstract In this work, we present a new psychometric tool for modelling and analysing social interactions. We focus on the concept of interpersonal spaces which are the distances unconsciously established by people between them. The measure of these distances can give us a hint on how people know each other. Our main goal was to create a tool to simplify experimentation in this area of research within social sciences. The system we propose can help to analyse and model personal spaces using computer vision. It is an augmented reality viewer that can, in real time and in 3D, model social spaces around users in a scene using computer graphics. For the computer vision part, we base our work on the new Microsoft s 3D camera, Kinect. This new camera allows us to easily determine people position and reconstruct in 3D the scene for modelling purpose. Our tool gives an intuitive representation of the personal spaces and simplifies experimentation in natural setting. The results of these experiments could be use in the future to develop new forms of human computer interface attentive to human relationship and spatial behaviour. 1 Introduction Man communicates and interacts with the world around him, but this communication is not achieved just through spoken language. Most of the information transmitted when communicating is done through a wide range of non-verbal signs and social behaviours, including facial expressions, postures or gestures. The ability to express, detect and analyse these signals is called social intelligence. Social signal processing is an emerging area of research that aims to give a social intelligence to computers [1][2]. The goal is to enable them to collect, analyse and process the various social signals sent by humans to better interact with them. The main applications are to develop new forms of human-computer interfaces, more based on the human behaviour and to create multimodal tools to help research in cognitive and social sciences. These systems should improve the precision of observations and greatly accelerate experiments that are now often imprecise and time consuming. In our case, we were interested in extracting and understanding the social information that space could provide us. Indeed space speaks: even if people feel free to move, many rules unconsciously shape their evolution in space. Speed, direction or trajectory are controlled and comply with various patterns constrained by social mechanisms [3]. You will deviate more likely if you walk towards people looking like bad guys than if you see a nice little old lady, because being too close to them will cause you a feeling of discomfort and insecurity. In our work, we were interested in a particular type of social signal linked to space and people s environment: the interpersonal distances, the areas unconsciously established by people between them.

2 These spaces give an image of the relationship between individuals. People, at every moment, adjust and control their personal spaces. Any violation of these spaces causes a sensation of discomfort up to cause a feeling of aggression. Our main objective was to create a simple tool to visualize, measure and analyse these spaces, to help us in the future to get a better understanding of their core properties and to simplify research on the subject. The computer vision system we built is based on the recent technologies brought by Microsoft with their new camera, Kinect [4]. That lets us easily create a 3D representation of the world, detects people and realizes precise measures on social interactions. We have realized a validation step using the stop distance experiment (distance between two persons where they naturally stop before interacting) to try out our system in a natural setting and with unaware subjects. This paper is organized as follows: Section 2 reviews the related work, Section 3 introduces the concept of interpersonal distances, Section 4 discusses the model and factors of influences, Section 5 presents our proposed system, Section 6 exposes an experiment and results. We conclude in Section 7. 2 Related work Much research has been conducted in the field of psychology and social science related to the concept of interpersonal spaces. This concept was first introduced by E.T. Hall [5] in his research on human behaviour in public space and then discussed by R. Sommers [6]. Research in this domain has thus highlighted the main factors of influence on these areas and raised, sometimes measured, the impact they could have on the shape and size of these territories as well as the interpretation we could have of them. Most of the research was published before Since we have observed a drastic decline of the subject. As explained in [7], this decrease of interest could not be linked to the irrelevance of the concept. The study of social spaces can obviously lead to a better understanding of social mechanisms and context but also to developments in many areas such as improving the quality of life and work by optimizing the personal space available for individuals. One of the main reasons given to justify this interest decrease is the weakness of the methodology and measurement techniques. The experiments carried out to make measurements, used specially equipped rooms with for example a floor grid [8]. With a videotape of the scene then it is possible to identify the distance between the subjects by observing the pattern on the floor. To our knowledge, no system based on computer vision technologies has been developed for this purpose and to analyse in real time social spatial interaction. The only kind of development that we have noticed, finds its application in the field of human machine interfaces, where the use of personal spaces can play an essential role for example in simulating the behaviour of virtual avatar. Indeed, it was shown that users maintained a similar spatial behaviour in virtual worlds such as in the game Second Life [9]. It is therefore an important factor in making realistic behaviour for a virtual agent or a non playable character. However, these systems generally require complex installations, such as [10], which does not allow performing measurements in ecological conditions.

3 3 Spatial behavior clue: concept of interpersonal distances E.T.Hall, in his studies about the human behaviour in public space [5], introduced for the first time this concept of interpersonal distances. People create unconscious territories around them, which define and determine the interactions they can have with other people. Those territories are like some invisible bubbles surrounding them and keeping them far from each other, unless space has some physical constraints (small room, crowded environment...). Interpersonal distances are a form of non-verbal communication between two or more persons defined by the social relationship they have. In a way, the measure of these distances is a clue that can tell us how people know each other. E.T.Hall has proposed a first model to represent these different spaces; it divides the space around a person in four distinct regions as shown in Figure 1: 1. Intimate distance (0 to 45 cm): a really close space with high probability of physical contact, you can feel heat and odor from an other person. It s a distance for touching, whispering or embracing someone. It indicates a close relationship like with lovers or children. 2. Personal distance (45cm to 1.2m): distance for interacting with relatives like family members or good friends. The other person is at arm s length, only ritualized touch can happen like handshake. Unrequested penetration of this space will provoke discomfort, defensive postures and even avoidance behaviours. 3. Social distance (1.2m to 3.5m): distance for more formal or impersonal interactions. It s the distance you naturally pose when you meet stranger and establish a communication process with them. 4. Public distance (3.5 to infinity): distance for mass meeting, lecture hall or interactions with important personalities. Figure 1: Definition of Hall s personal space model. The space around people is divided in four territories: the intimate space, the personal space, the social space and the public space.

4 People update, control, and adjust their personal spaces continuously. Any unauthorised penetration of these spaces will cause a feeling of discomfort for people. This can lead to an aggressive response of the subject who may feel oppressed by the presence of the intruder. This concept is quite similar to the flight zone observed in animal behaviour: a space around them, where if someone goes into it will trigger an escape behaviour in response of the spatial aggression. Obtaining a better understanding of how these areas work could help to improve the communication between individuals. For example, to measure and express the influence of an individual s status on the form of personal spaces can have many applications in education science. A teacher could learn to control and optimise its spatial positioning; this can be critical to enable it to maintain a certain social distance with his pupil. To avoid being too close, which could cause a feeling of primary aversion, would change the perception of the student towards the teacher and therefore would reduce the performance of education. 4 Modeling and analysing personal space Many parameters are influencing the shape and size of our personal spaces. Hall proposed a first static model by experimenting on American middle-class people in the sixties but he doesn t take account of other variables. However, in his work, he also showed the cultural differences in the use of space and the impact this could have on the size of interpersonal distances. It classifies populations into two groups: contact and non-contact cultures. For instance, for Japanese people, a non-contact culture, the model should have larger separation distances, which would be the opposite for an Arab or Latin population. Obviously personal spaces are much more complex social mechanisms and depend on many parameters. They continuously evolve and adapt to people circumstances. They should be seen as dynamic and elastic territories varying with lot of parameters like: culture, sex, age, gender, size, social position, relationship or physical appearance. Social sciences have already realized a lot of experiments on this subject and showed the importance of this concept for explaining how people behave and interact together [11,12,13,14,15]. Many behavioural tests have shown the importance and impact that these areas could have on our actions. The observation of specific behaviours influenced by interpersonal distances shows that they can be correlated to the: sex, gender, detection of violent behaviour, diseases such as schizophrenia or autism, etc. Until recently, the measures were performed manually, using often special rooms, lab s scenarios and consuming a lot of time. The results are so often realized with unnatural setting and the results should be biased by the conditions of the experimentation. Like said in [7], these problems should be at the origin of a lost of research interest in interpersonal space. So we wanted to develop a precise tool using computer vision technologies to observe in real time Hall s model of personal spaces on ecological scene and in the same time realize measures on the social behaviour of the subjects. This way, we hope to get a better understanding of the basic shape of the personal space and actualize Hall s distances taking into account new variables. Since these spaces are culturally conditioned, we could expect an evolution since Hall s observation. This would be a first step to offer a precise methodology by using computer vision measurement system to realize research in this domain.

5 5 Proposed system for analysing social spaces The computer vision system we propose is an augmented reality viewer that can, in real time and in 3D, model Hall s social bubbles around users in a scene. It s based on the new Microsoft s 3D camera, Kinect. This new cheap 3D camera can compute accurate depth map and RGB image necessary to reconstruct the scene. The Kinect is essentially composed of two sensors: a classic RGB camera and a depth camera using an infrared camera and a laser based IR illuminator. The illuminator projects an irregular pattern of IR dots of varying intensities. The depth camera reconstructs a depth image by recognizing the distortion in this pattern. A calibration step lets us create a point cloud representation of the scene respecting real world coordinates. For the drawing, we use the OpenGL library [16], Figure 2. Figure 2: Left and center, Depth and RGB images obtained by the two cameras. Right, point cloud reconstruction obtained by calibrating the RGB and depth camera. With the recent OpenNI framework [17], that supports the Kinect camera, we could realize a user detection and segmentation. With these 3D data, we calculate the position of the user s centroid in the 3D space then compute their personal space with respect to Hall s model. Finally OpenNI has another interesting functionality for analysing non-verbal behaviour, it s the skeleton tracking possibilities, Figure 3. The body has his own language and we can assume that it influences the shape of the personal space during an interaction. With this tracking, we can study new clues and variables that interact with interpersonal distances. Body inclination is a good example, it s known like a sign of implication; we have observed that during a handshake people have tendency to lean forward. In the actual state, we only represent the skeleton, this simplify observations on the scene but in future works, we expect to automatically measure and determine body gestures then observe the impact on the personal space and the social interactions. Figure 3: Left, personal and intimate spaces 3D visualization on a single user. Right, handshake scenario: we represent skeleton tracking and the interpenetrating personal spaces while the interaction.

6 Our tool contains two parts, Figure 4: one for visualizing the 3D scene and another consisting in a form for notifying information like users sex, age,... The system, first works like an augmented reality viewer. He can reconstruct a scene in 3D, detects multiple human users than compute their personal space. This gives a simple tool to visualize how people interact spatially according to Hall s model. The second use of our system is to analyse spatial behaviour. If two or more people are present in the scene, we can automatically and precisely measure the distances between them and get variables like people s height. All the data can be input in the form and save in a text file that can easily be open in other program like Matlab or Excel. The objective is to easily realize statistics on the data and get a better understanding of the personal space. Figure 4: Interface of our system: the left part shows a 3D representation of the world and the right part is a form to save information on the subjects. 6 Experiments and results In psychology, to conduct experiments on variations of personal space, three types of methodologies are used: 1. The projective method that pushes subjects to imagine a scene and represent how themselves or others react spatially to the scenario. For that, we usually resort to the use of drawing or dolls. It was showed that this method could be regarded as unreliable, because it requires significant capacity for reflection that skews the observations that we could get [18]. 2. The laboratory method that aims to produce a spatial scenario in a controlled environment, such as a handshake. The subjects are often aware of participating in an experiment, which can cause variations in the perception of the environment and thus truncate the action. 3. The observation method of collecting data by studying the interaction between people in their natural environment, using measurement techniques causing as little interference as possible. It is obvious this method is the most difficult to achieve since it requires precise measurement systems that can operate independently of the environment.

7 The system we have developed thus tends to respond to this need for accurate measurement method, usable anywhere. The 3D camera makes it possible to perform measurements without any of the current problems associated with the enlightenment that often imposed by computer vision systems to operate in controlled environments. To test our system, we wanted to make an experiment to observe the spatial positioning of a subject toward an experimenter before he initiates a conversation. The main interest here was to test the system in conditions completely unforeseen. The scene took place in an empty classroom adjacent to a computer lab where students had a course. The scenario was to ask a student during a workout, to pass into the adjacent room where the experimenter was waiting him for a software demonstration. The experimenter was pretending to read, waiting until the subject comes closer before initiating a discussion. We wanted to see if using our system, we could easily put in evidence different types of approach behaviour by measuring the spatial positioning of the subjects. But also see if subjects follow the concept of personal space proposed by E.T. Hall. 21 undergraduate students participated: 17 men and 4 women. Some first results have appeared: subjects had a direct approach and stopped at the beginning of Hall s personal space. Men seem to stand out of the experimenter s personal space (mean=127 cm). Girls seem more intrusive and come into the personal space (mean=103 cm). 2. Two men had a two steps approach. First they stopped in the middle of Hall s social space, waiting the authorisation to come closer, then approached and positioned themselves like the first ones. 3. The three last had also a direct approach but come nearer than the others. They came really close in the experimenter s personal space (mean=75 cm). The first behaviour seems to follow Hall s model. The wait for an authorization, in social space, which was observed in the second case, could be explained by this concept of status, dominance. Expecting to be accepted into the personal territory of the experimenter, before moving, to avoid causing a situation of discomfort that could change the perception of the subject. The last case could be explained by the origin of the subjects. Two were Erasmus students from Africa. A contact culture, as defined by Hall, this would explain the search of proximity. The last one had an unexplained behaviour; he nearly came into the intimate space of the experimenter (48 cm). 7 Conclusion In this work, we proposed a new tool for experiments in social science on the understanding of spatial behaviour. First, we used the new camera from Microsoft, the Kinect to realize the 3D reconstruction of a scene, detecting and tracking people. Then based on these 3D data, we modelled subjects personal space by using the model proposed by E.T.Hall. Our tool also simplifies the measurement of distances between users on ecological conditions. This system will serve as a basis for achieving various experiments on the spatial behaviour of people and better understand the impact of several factors on personal spaces. A first attempt was performed; the primary objective was to try out our system and to develop scenarios. Some observations have already been made. Nevertheless, experiments at a larger scale are planned in the future to validate these observations. Tests on larger population are needed to better discern the different factors influencing the social spaces. Future works will focus on the impact of gaze and physical appearance on our social spaces.

8 References [1] A. Vinciarelli; M. Pantic; H. Bourlard, Social Signal Processing: Survey of an Emerging Domain, Image and Vision Computing, vol. 27, p , November [2] A. Pentland, Socially aware, computation and communication, Computer, vol. 38, p , [3] M. Cristani; V. Murino; A. Vinciarelli, Socially intelligent surveillance and monitoring: Analysing social dimensions of physical space, CVPRW, p , June [4] Microsoft, [5] E.T. Hall, The Hidden Dimension, Anchor Books, [6] R. Sommer, Personal Space : The behavioral Basis of Design, Prentice Hall Trade, June [7] D. Uzzell ; N. Horne, The influence of biological sex, sexuality and gender role on interpersonal distance, The British journal of social psychology, vol. 45, p , September [8] W. Ickinger ; S. Morris, Psychological characteristics and interpersonal distance, [9] D. Friedman; A. Steed; M. Slater, Spatial Social Behavior in Second Life, Intelligent Virtual Agents, vol. 4722,p , 2007 [10] T. Amaoka; H. Laga; S. Saito; M. Nakajima, Personal space modeling for humancomputer interaction, Entertainment Computing, p , [11] R.D. Middlemist; E.S. Knowles; C.F. Matter, Personal space invasions in the lavatory: suggestive evidence for arousal, Journal of personality and social psychology, vol. 33, p , May [12] D.E. Thompson; J.R. Aiello; Y.M. Epstein, Interpersonal distance preferences, Journal of Nonverbal Behavior, vol. 4, p , [13] R. Gifford, The experience of personal space: Perception of interpersonal distance, Journal of Nonverbal Behavior, vol. 7, p , [14] M. Kaitz; Y. Bar-Haim; M. Lehrer; E. Grossman, Adult attachment style and interpersonal distance, Attachment and human development, vol. 6, p , September [15] S. Iizuka; Y. Goto; K. Ogawa, A Study of Personal Space in Communicating, Health, 2007, p [16] OpenGL, [17] OpenNI, [18] L.Hayduk, Personal space: Where we now stand, Psychological Bulletin, vol. 94, p , 1983.