23 rd International Congress on Sound & Vibration Athens, Greece 10-14 July 2016 ICSV23 PERCEPTION OF AUDITORY-VISUAL SIMULTANEITY CHANGES BY ILLUMINANCE AT THE EYES Hiroshi Hasegawa and Shu Hatakeyama Graduate School of Engineering, Utsunomiya University, 7-1-2 Yoto, Utsunomiya-shi, Tochigi-ken, 321-8585, Japan, email: hasegawa@is.utsunomiya-u.ac.jp In this study, we measured the point of subjective simultaneity (PSS) between auditory and visual stimuli under various conditions of illuminance at the subject s eyes. The test stimuli were a pure tone of 1000 Hz at a sound pressure level of 50 db and a white light (LED). Both sound and light stimuli had a duration of 10 ms. The luminance of the LED was set at 21.2 cd/m 2, and the ambient illuminance around the LED was set at 0 lx. The illuminance at the subject s eyes was adjusted to 0, 500, or 1000 lx using a spotlight located above the subject s head. The test stimuli had a stimulus onset asynchrony (SOA) of 0, ±20, ±40, ±80, or ±160 ms, where positive values indicate that the visual stimulus was presented first. Four men in their early 20s participated in the experiments. We presented the test stimuli to the subject with the nine SOAs in random order. After each presentation of the test stimuli, we asked subjects to answer which of the stimuli sound or light they perceived first. Then we evaluated the PSS of the test stimuli and considered the influence of illuminance at the eyes on the PSS. As a result, the PSS shifted toward positive direction, i.e., the perception of light precedence as the illuminance at the eyes increased. 1. Introduction Many studies have been done on the perception of auditory and visual information [1] [15]. We have been studying about the percetion of auditory-visual simultaneity [11] [14]. We investigated the point of subjective simultaneity (PSS) between auditory and visual stimuli depending on the subject s attention, and showed that the PSS relatively shifted toward the perception of sound precedence when they paid attention to the light[11]. We also showed that the luminance of the light and the ambient illuminance of the room had an influence on the perception of the simultaneity between auditory and visual stimuli [12, 14]. In this study, we focused on the illuminance at the subject s eyes and investigated the perception of simultaneity between auditory and visual stimuli when the illuminance level was changed. 2. Experiment We investigated effects of the illuminance level at the subject s eyes on the perception of simultaneity between auditory and visual stimuli. 2.1 Stimuli We used a pure tone of 1000 Hz at 50 db SPL (sound pressure level) and a white light (LED; LP-10HW3B) with a luminance of 21.2 cd/m 2. Both stimuli had a duration of 10 ms. The auditory stimulus had a linear fade-in and fade-out of 2.5 ms to reduce transient responses. 1
2.2 Apparatus We carried out the experiments in a soundproof room (D-30; Fig. 1). We played the auditory stimuli on a CD player (XL-V1-N, Victor) with amplification through an amplifier (CR-D2, Onkyo), and presented them to each subject via headphones (SRH840, Shure). The luminance of the LED light was set to 21.2 cd/m 2 with a lighting controller (LMD01, Fitdesign). We adjusted the illuminance at the subject s eyes to 0, 500, or 1000 lx using a spotlight located above the subject s head as shown in Fig. 2). The timing of the light emission was controlled by a trigger signal sent from the CD player to the controller. The subject sat on a chair in front of the LED light. The testing space was closed off with a blackout curtain to eliminate all other light. Soundproof room (D-30) Subject Blackout curtain Amplifier (CR-D2, Onkyo) Experimenter Headphones (SRH840, Shure) LED Trigger signal CD player (XL-V1-N, Victor) Lighting controller (LMD01, Fitdesign) Light luminance control voltage signal DC power supply (PMC18-2, Kikusui) Figure 1: Experimental apparatus. The sound stimuli were presented via headphones to the subject. The visual stimuli were presented using a LED light controlled by a light controller. 2.3 Procedure We used a method of constant stimuli to determine the PSS between the sound and light stimuli. These stimuli were presented at offsets (stimulus onset asynchronies: SOAs) of 0, ±20, ±40, ±80, and ±160 ms, where positive values indicate that the visual stimulus was presented first. We presented the test stimuli with the nine SOAs in random order, every 5 s (Fig. 3). The PSS between the two stimuli was obtained by fitting a psychometric function to the results as the 50% point of the answer rate with respect to the SOA. After each presentation, the subject answered: The sound was perceived first, or The light was perceived first. 2.4 Subjects and trials Four men in their early 20s participated in the experiments. All had normal hearing acuity and normal or corrected-to-normal vision. We conducted 45 trials (9 SOAs 5 iterations) per session and performed 20 sessions in each illuminance level of 0, 500, and 1000 lx, in which the presentation order was different in every session. In total, we performed 10, 800 trials (45 trials 20 sessions 3 illuminance levels 4 subjects). 2 ICSV23, Athens (Greece), 10-14 July 2016
Spotlight (EL-S1000L/W, Mitsubishi Electric) LED (LP-10HW3B) Blackout curtain Illuminance at the eyes: 0, 500, or 1000 lx Headphones (SRH840, Shure) Subject Lighting controller (LMD01, Fitdesign) Soundproof room (D-30) Figure 2: Schematic figure of the experiment. Illuminance at the subject s eyes was adjusted to 0, 500, or 1000 lx using a spotlight located above the subject s head. 3. Results Figures 4 (a) (c) show the experimental results in the condtions that the illuminance level at the eyes were 0, 500, and 1000 lx, respectively. The vertical and horizontal axes denote the answer rate and the stimulus onset asynchrony (SOA) between the sound and light, respectively. The white and dark bars indicate the answer rates sound was perceived first and light was perceived first, respectively. In all the figures, the perception of sound precedence decreased and that of light precedence increased as the SOA increased. At an SOA of 0 ms, the perception of light precedence was greater in all the conditions. These results mean that the point of subjective simultaneity (PSS) shifted to sound precedence. Sound stimulus (50 db, 1 khz, 10 ms) Light stimulus (21.2 cd/m 2, 10 ms) 160 ~160 ms Lag 5 s Interstimulus interval 160 ~160 ms Lag Figure 3: Presentation timing of the sound and light stimuli. ICSV23, Athens (Greece), 10-14 July 2016 3
100 : Sound precedence : Light precedence 80 Answer rate (%) 60 40 20 0 160 80 40 20 0 20 40 80 160 Sound first Stimulus onset asynchrony (ms) Light first (a) Illuminance level: 0 lx 100 : Sound precedence : Light precedence 80 Answer rate (%) 60 40 20 0 160 80 40 20 0 20 40 80 160 Sound first Stimulus onset asynchrony (ms) Light first (b) Illuminance level: 500 lx 100 : Sound precedence : Light precedence 80 Answer rate (%) 60 40 20 0 160 80 40 20 0 20 40 80 160 Sound first Stimulus onset asynchrony (ms) Light first (c) Illuminance level: 1000 lx Figure 4: Experimental results in the conditions of the illuminance level of 0, 500, and 1000 lx at the eyes. The vertical and horizontal axes denote the answer rate and the stimulus onset asynchrony (SOA) between the sound and light, respectively. The white and dark bars indicate the perception of sound and light precedence, respectively. 4 ICSV23, Athens (Greece), 10-14 July 2016
4. Discussion We fitted a psychometric function [11] to the result of each subject in each condtion, and obtained the PSS as the 50% point of the answer rate with respect to the SOA. Table 1 shows the average values of PSS of all subjects in the conditions that the illuminance at the eyes was 0, 500, and 1000 lx. Table 1: Averaged PSS values in each illuminance level of 0, 500, and 1000 lx at the eyes. Illuminance level at the eyes (lx) 0 500 1000 PSS (ms) 33.5 30.0 26.7 In Table 1, all the PSS values were negative, i.e., the PSS shifted toward the perception of sound precedence. In comparing the PSS values among the illuminance levels, the PSS shifted toward positive direction, i.e., the perception of light precedence as the illuminance level increased. This tendency agreed with the our previous reports [14], that is, by raising the illuminance level at the eyes, relative attention to the LED light decreased, and thus attention to the sound relatively increased. Paying attention to the sound increased the speed of transmission of auditory information, so the sound and light were perceived as simultaneous when the light was presented earlier. The PSS then shifted toward the perception of light precedence. 5. Conclusion The point of subjective simultaneity (PSS) between the auditory and visual stimuli shifted toward the perception of light precedence as the illuminance level at the eyes increased. Further experiments would be required to ensure this result by increasing the number of subjects. References 1. Hershenson, M., Reaction time as a measure of intersensory facilitation, Journal of Experimental Psychology, 63 (3), 289 293 (1962). 2. Aschersleben, G. and Müsseler, J., Dissociations in the timing of stationary and moving stimuli, Journal of Experimental Psychology, 25 (6), 1709 1720, (1999). 3. Stone, J. V., Hunkin, N. M., Porrill, J., Wood, R., Keeler, V., Beanl, M., Port, M., and Porter, N. R., When is now? Perception of simultaneity, Proceedings of the Royal Society B Biological Science, 268, 31 38, (2001). 4. Sugita, Y. and Suzuki, Y., Implicit estimation of sound-arrival time, Nature, 421, No. 27, 911, (2003). 5. Neumann O. and Niepel M., Timing of perception and perception of time, Psychophysics beyond sensation: laws and invariants of human cognition, 245 269, (2003). 6. Fujisaki, W., Shimojo, S., Kashino, M., and Nishida, S., Recalibration of audiovisual simultaneity, Nature Neuroscience, 7, 773 778, (2004). 7. Lewald, J. and Guski, R., Auditory-visual temporal integration as a function of distance: no compensation for sound-transmission time in human perception, Neuroscience Letters, 357 (2), 119 122, (2004). ICSV23, Athens (Greece), 10-14 July 2016 5
8. Arnold, D. H., Johnston, A., and Nishida, S., Timing sight and sound, Vision Research, 45 (10), 1275 1284, (2005). 9. Zampini, M., Shore, D. I., and Spence, C., Audiovisual prior entry, Neuroscience Letters, 381 (3), 217 222, (2005). 10. Zampini, M., Guest, S., Shore, D. I., and Spence, C., Audio-visual simultaneity judgments, Perception & Psychophysics, 67 (3), 531 544, (2005). 11. Hasegawa, H., Yoshida, J., and Kasuga, M., Effect of attention to preceding stimuli on the simultaneity perception between an auditory and visual stimulus, Proceedings of 20th International Congress on Acoustics, 236, (2010). 12. Hasegawa, H., Kasuga, M., and Yoshida, J., Influence of light luminance on auditory-visual simultaneity perception, Proceedings of Forum Acusticum 2011, 000204, (2011). 13. Hasegawa, H., Yamamoto, Y., Saito, W., and Yoshida, J., Learning changes the perception of the equivalence of strengths between auditory and visual stimuli, Proceedings of 20th International Congress on Sound and Vibration, 341, (2013). 14. Hasegawa, H., Ishiwatari, K., and Yoshida, J., Ambient illuminance alters perception of auditoryvisual simultaneity, Proceedings of Forum Acusticum 2014, R17_12, (2014). 15. H. Hasegawa and W. Saito: Learning of the equality between an auditory and a visual stimulus alters the perception of auditory-visual simultaneity, Proceedings of 22nd International Congress on Sound and Vibration 2015, 676, (2015). 6 ICSV23, Athens (Greece), 10-14 July 2016