Technology meets Psychology: Psychological Background in Virtual Realities

Transcription

1 Technology meets Psychology: Psychological Background in Virtual Realities Paul Pürcher 1, and Margit Höfler 2 1 University of Graz, Austria 2 Danube University Krems, Austria Abstract In recent years, virtual realities (VRs) have become an important resource for various applications such as entertainment, architecture, therapy and learning. Previous research already investigated the effects of psychological processes such as immersion, flow and involvement in context of VRs. However, there are also more basic psychological processes such as perception and attention which both play a crucial role when creating and using VRs. Due to perceptual processes we perceive input from our environment and create a subjective reality in our mind which often does not correspond to the objective reality. Due to attentional processes, we are able to filter relevant information that is processed via our senses from all perceivable information. In the current study, we interviewed technicians about their degree of awareness and knowledge about such attentional and perceptual processes in general and within VRs. The findings showed that there is a lack of knowledge regarding these processes, suggesting that there is need for a broader discussion of the topic. This work aims to be the first step towards this direction. Keywords virtual realities; perception; attention; technicians. I. INTRODUCTION In recent years virtual realities (VRs) have not only become popular in fields such as entertainment [1] but also in learning [2-3], rehabilitation [4] and even therapy [5]. In general, VR is typically used to describe systems in which objects are created within a virtual environment and the user can interact with these objects to a certain extent. There are several ways how a virtual reality can be created [6]. For instance, in a windows system, a user can enter an interactive virtual world whereas in a vehicle-based system, the user is driving a vehicle and the visual environment is virtual (e.g. plane simulation). In a cave system, a room is provided, equipped with displays on each side that project a virtual reality. The advantage of such a system is that a user can move and interact with the system freely. In immersive VR systems, the experience of the VR is intensified by various additional input systems such that several senses (e.g. sight, auditory, etc.) are addressed. Finally, in augmented realities, computer-generated objects are superimposed on real objects. In the beginnings of VR, technical aspects were in the main focus (e.g., which hardware and software is needed in order to provide a realistic impression of the world). However, regardless of the technical approach used, to successfully create a VR it is also essential to be aware of which psychological factors contribute to such environments and also how they contribute. For example, social interactions, embodiment, engagement, immersion (i.e., the extent to which users are involved in the learning experience) and user presence are psychological processes that have already been investigated in previous research on VR [7-10]. However, other-more basic- psychological factors such as attentional or perceptual processes are often not considered, although these processes are relevant in order to use VRs not only in research but also in (clinical) practice [11]. Furthermore, it is still unclear if basic cognitive (and memory) processes are the same in virtual environments as in real-world settings [12]. To this end, a basic knowledge about attentional and perceptual processes is a key for the successful use of such technologies. A. Attention and perception Attention and perception have been investigated since the early days of Psychology. Due to our senses, we are able to receive and perceive input from our environment and to create a subjective reality in our mind. However, this subjective reality often does not correspond to the objective reality. One famous example is the Müller-Lyer Illusion [13] first demonstrated by Carl Franz Müller-Lyer in 1889 (see Fig. 1). Although the upper line in Fig. 1 seems to be longer than the lower one (subjective reality) both lines are indeed equal (objective reality). Furthermore, we are also not able to process all information that we receive from our environment because our physiological capacity is limited. For instance, if you look at the scene in Fig. 2, you might have the feeling of rich and detailed perception, but if you try to retrieve all these details afterwards, you might not be able to do so (e.g., you might have missed that there is a crane in the top right corner). Generally, research indicates that about four objects can be held in working MIPRO 2018/CE 709

2 memory [14]. Because of the limited physiological capacity, attention is necessary to filter relevant information from all perceivable information. However, even if we might have some idea what attention means, attention itself is not a single concept but rather subsumes various processes [15]. For instance, if you look at Fig. 2 again, some of the objects in the scene might attract and/or guide your attention more than others. Previous research has indicated that factors like color, orientation, size or movement guide our attention [16]. For instance, the palm tree in Fig. 2 is quite salient. Therefore, you might have looked at the palm first when starting to explore the scene before going to the next area of interest (e.g., the people in the scene). There are several models which aim to predict which regions of a scene might attract attention [17]. For instance, in Fig. 2b, a possible scan path (i.e., the order of regions one would fixate due to the saliency of the different regions) was computed using Saliency Toolbox ( According to this analysis, the legs of the man at the right side of the scene would be fixated first, followed by the green cover on the left. According to the model, the palm (or parts of it) would be fixated afterwards. A famous experiment that further supports the idea of limited and selective attentional processes is an experiment of Simons and Chabris [18] who had participants watch a scene with students wearing either white or black T-shirts playing basketball and count the number of passes of the white team. While doing so, a man wearing a gorilla costume passed through the scene. The results revealed that about half of the participants did not recognize the gorilla (although some of the participants that did not recognize it even looked at it [19]. This finding was also replicated within virtual environments: When Suma, Clark, Finkelstein and Wartell [20] had participants walk through an immersive virtual environment their results revealed that participants hardly noticed even large changes in the surrounding environment. In the current study we were interested in the prior knowledge of technicians (students) on these basic perceptual and attentional psychological concepts presented above. We were also interested in how they rate the importance of these processes with regard to VRs. The aim of this study was simply to get a first understanding of the attitude of future technicians towards the importance of these processes in their field of work. To answer these questions, we developed a structured interview. 2.A. 2.B. Figure 2. 2.A. Scene used for the interview (see also Table 1, Question 9). Participants task was to assess the sequence of inspections by writing a number on the areas (e.g. 1 = first inspection point, 2 = second inspection point, etc.) 2.B. Order of inspections of the same scene based on a saliency model ( II. METHODS A. Design, Material and Procedure In order to investigate the knowledge and the awareness of technicians (students) of perceptual and attentional processes we developed a qualitative structured interview that consisted of 11 questions (often followed by sub-questions; see Tab. 1). The questions ranged from very broad questions such how they define attention and perception in general and specific within VR. Furthermore, we asked specifically for some details, e.g. the study with the invisible gorilla [18] (Question 8 in Tab. 1). Figure 1..Müller-Lyer Illusion. See text for details. 710 MIPRO 2018/CE

3 Table 1. Questions of the structured interview (Note that the questions are slightly rephrased/shortened compared to the interview). 1. Previous experience on VR? (if so, please describe) 2. In what way is Psychology important in VR? 3. How would you define attention and in which way could it be important within VRs? 4. How would you define perception and in which way could it be important within VRs? 5. How many objects do you think can you perceive at the same time? 6. How could Psychology help improving a VR? 7. Do you know any perceptional phenomena? 8. Do you know the experiment with the invisible gorilla [18] and if so please explain? 9. "Palm Tree Picture" (see Fig. 2): How do you think people would inspect this picture? 10. Why do you think sometimes people remember objects of a scene, that weren t there? 11. Think back to the Picture: (Fig. 2): Which objects do you keep in mind easily/badly? Which different kinds of memory do you know? The oral interview was carried out at public (but quiet) places at the University of Graz or Graz University of Technology. For most of the questions, the interviewer (one of the authors) took written notes of the respective answers. Question 9 at Tab. 1 referred to the scene in Fig. 2a and asked the participants to set a sequence of inspection according to how they believe one would inspect the scene (1 = first inspection, 2 = second inspection, etc.). Participants pointed to the picture and the interviewer wrote the numbers on the picture, according to the participant s indications. In two further questions (Questions 10 and 11, respectively), we also asked participants about their previous knowledge of memory systems (long-term memory & short-term memory, [21]). All questions were asked in English; however, all participants were German speaking and preferred to answer the questions in German. All in all, the interview took about 15 min. B. Participants Four participants (2 females; M age: 26.3 years, SD age: 2.3) took part in the interview. All of them were students at Graz University of Technology. Participants gave informed consent prior participation. Moreover, the study was approved by the local ethics committee. III. RESULTS The questionnaire consisted of the 11 questions in Tab. 1. Two of four participants reported in question 1 (Q1) that they had worked with VR before, namely within university courses (e.g. programming). Two participants however stated that they did not have any knowledge in the field of VR, one of these two stated to have played a VR game before. When asked in what way Psychology might be important for VRs (Q2) one participant stated that Psychology is like a tool when it comes to creating VRs. Two participants also emphasized that Psychology is a component which should be considered in context of VRs and one claimed that further work has to be done. One person said that it would be difficult to think about Psychology in context of VR and that more knowledge would be needed. When asked about the definition of attention in general (Q3), two participants stated that attention is important to concentrate on a certain object; one further participant defined it as putting focus on a certain object. One participant said that the impact of attention is often underrated. However, all four participants thought that attentional processes are important when it comes to VR. With regard to the definition of perception (Q4), the answers were more widespread: Participants named factors such as certain things might be different for one person than for another person, impact of the sense and also just to be aware that an object is there. However, again all participants agreed that perception is important in context of VR. When participants were asked to estimate how many objects one can perceive at once (Q5), the estimates were 2, 5, 20 and objects, respectively. One of the participants also differed between conscious (2 objects) and subconscious perception (6 objects). Q6 showed that there is a bright variety of thoughts on how Psychology could help when creating a VR: The answers ranged from realizing experiments that would not be possible otherwise, evoking certain emotions, direct the attention on something in order to guide the behavior in the VR or also take Psychology into consideration when modelling the VR. Q7 showed that all interviewed students of a technical subject knew some perceptional phenomena, for instance, optical illusions were named often (e.g. moving circles, mirages). Two of the participants had knowledge of the experiment with the invisible gorilla (Q8, [18]). As a reason for the misperception of the gorilla during the basketball game, these two stated that attention plays a role here as the participants in that study concentrate on something else and this often leads to misperception. When asked about whether they think that they would recognize the gorilla themselves, one participant stated that he would most likely not perceive it, whereas the second participant stated it would depend on his concentration. Two participants were not aware of the experiment with the invisible MIPRO 2018/CE 711

4 gorilla [18]. In this case, the experiment was described to them after the end of the interview. In Q9 we had the participants mark the sequence of fixations in which they think the scene in Fig. 2 would be inspected. All four participants pointed to the stem of the palm tree and thought it would be an important feature, because it is very prominent in the picture. The other nominations varied between the different participants and concerned e.g. the sky and the people (see Fig. 3 for details). When asked about whether one would retrieve objects that were actually not there, two out of four participants stated that the picture might be linked to personal experiences or memory. One participant said that if there was, for instance, an umbrella, perceivers might automatically think of raindrops (and might incorrectly retrieve them afterwards). One participant did not believe that one could retrieve objects that were not in a previously presented scene. Finally, Q11 revealed very diverse thoughts of which objects are kept in mind more easily than others. All four participants agreed in the point that the palm tree would be remembered best. Asked for the different memory systems they knew, all four participants were able to state shortterm memory and long-term memory as memory systems. Two participants thought that the palm tree memory was in short-term memory at that moment and the next day it would be in long-term memory. facts when it comes to creating, applying and working with VRs. This seems to be of big interest, as future research is forecasted to merge fields together and a broader knowledge and proficiency of different fields of science will become more and more important. In sum, the current study revealed interesting insight in future technicians awareness of Psychology and sets a first direction, which is important to follow. 3.A. 3.B. IV. CONCLUSION In the current paper, we provide first insight into the knowledge and awareness of technicians on basic psychological processes in context of VRs. All in all, the interviews showed that knowledge about basic psychological processes was rather superficial, however we detected a strong interest in Psychology, as well as awareness that Psychology is an important component when it comes to create VRs (see also the answers to Q2). Psychological knowledge has been shown in some fields (e.g. memory systems such as long-term memory and short-term memory, partly also on the work of Simons and Chabris[18]). In other areas we detected a lack of knowledge (e.g. at the object-memory span asked in Q5 the answers stretched from 2-30 objects). Participants were however very interested in Psychology as a subject and very optimistic to learning more about it. We are aware that the interview and the questions asked in the interview were themselves rather superficial and that further and more detailed experiments are necessary, of course also involving a more substantial sample size. However, as mentioned above, the general aim of this study was to go a first step towards the important result of bringing the two fields of Technology and Psychology closer together. Future research should include more differentiated questions and a bigger sample size, as well as the extraction of a user-friendly guideline of psychological 3.C. 712 MIPRO 2018/CE

5 3.D. Figure 3. Answers for each participant (A, B, C, D) for Question 9 in the interview. Note that, for better readability, we re-labeled the scenes based on the answers of the participants. ACKNOWLEDGMENT We would like to thank Nikolaus Koren for his help during the preparation of this paper. Moreover, we would like to thank the participants for taking part without receiving any credit or monetary refund. REFERENCES [1] M. Zyda, From visual simulation to virtual reality to games, Computer, Volume 38, number 9, pp 25-32, [2] P. Pürcher, M. Höfler, C. Gütl, J. Pirker, L. Tomes, and A. Ischebeck, Individual vs. collaborative learning in a virtual world, Proceedings of MIPRO 2016, pp , [3] Z. Pan, A. D. Cheok, H. Yang, J. Zhu, and J. Shi, Virtual reality and mixed reality for virtual learning environments, Computers & Graphics, volume 30(1), pp 20-28, [4] S., Cao, Virtual Reality Applications in Rehabilitation, M. Kurosu (Eds.), Human-Computer Interaction. Theory, Design, Development and Practice, HCI Lecture Notes in Computer Science, Cham, Springer Verlag, volume [5] Freeman, S. Reeve, A. Robinson, A. Ehlers, D. Clark, B. Spanlang, and M. Slater, Virtual reality in the assessment, understanding, and treatment of mental health disorders, Psychological Medicine, volume 47, no.14, pp , [6] G. Bente, N. C. Krämer, and A. Peterson, (Eds.), Virtuelle Realitäten. Göttingen, Hogrefe Verlag, [7] M. Usoh, C. Alberto, and M. Slater, Presence: Experiments in the psychology of virtual environments, [8] A. Bowman, and R. P. McMahan, Virtual reality: how much immersion is enough?, Computer, volume 40, no. 7, [9] K. Kilteni, R. Groten, R., and M. Slater, The sense of embodiment in virtual reality, Presence: Teleoperators and Virtual Environments, volume 21(4), pp , [10] Argelaguet, L. Hoyet, M. Trico, and A. Lécuyer, The role of interaction in virtual embodiment: Effects of the virtual hand representation, IEEE Virtual Reality, pp. 3-10, April [11] J. Wilson, and A. Soranzo, The use of virtual reality in psychology: A case study in visual perception, Computational and mathematical methods in medicine, [12] S. Han, Y. Jiang, G. W. Humphreys, T. Zhou, and P. Cai, Distinct neural substrates for the perception of real and virtual visual worlds, NeuroImage, volume 24(3), pp , [13] Müller-Lyer Illusion. last retrieved on 20 Jan [14] S. J. Luck, and E. K. Vogel, The capacity of visual working memory for features and conjunctions, Nature, volume 390(6657), pp , [15] A. Styles, The psychology of attention (2nd ed.), New York, Psychology Press, [16] J. M. Wolfe, and T. S. Horowitz, Five factors that guide attention in visual search, Nature Human Behaviour, volume 1, no. 3, [17] L. Itti, and C. A. Koch, A saliency-based mechanism for overt and covert shifts of visual attention, Vision Research, volume 40, pp , [18] J. Simons, and C. F. Chabris, Gorillas in our midst: Sustained inattentional blindness for dynamic events, Perception, volume 28(9), pp , [19] Memmert, The effects of eye movements, age, and expertisebon inattentional blindness, Consciousness and Cognition, volume 15, pp , [20] A. Suma, S. Clark, S. L. Finkelstein, and Z. Wartell, Exploiting change blindness to expand walkable space in a virtual environment, IEEE Virtual Reality, pp , March [21] J. Bredenkamp, Gedächtnis, in M. A. Wirtz (Eds.), Dorsch Lexikon der Psychologie, Bern, Hogrefe Verlag, volume 18, p. 603, MIPRO 2018/CE 713