Audio-visual interaction of environmental noise

Transcription

1 Audio-visual interaction of environmental noise Anna Preis a), Honorata Hafke-Dys b), Malina Szychowska c),jędrzej Kociński d) and Jan Felcyn e) (Received: 9 June 2015; Revised: 3 January 2016; Accepted: 3 January 2016) Currently research into the psychological evaluation of noise in daily life is carried out without taking into account the sense of sight. The human senses interact with each other; thus some information coming from one sense can be skipped or ignored in favor of information coming from another sense, leading to completely different reactions or behavior. The aim of this paper is to verify, on the basis of psychophysical experiments, how a human being processes audio-visual information coming from the different environmental noises which can be encountered in daily life. The experiment was divided into three parts: auditory, visual, and audio-visual. In each part of the experiment, the ICBEN scale (0 10) was used to rate the presented stimuli. In the first part only audio stimuli were presented, and subjects were asked to rate their annoyance with the sound. In the second part of the experiment, the participants were asked to rate how pleasant the presented video clips were. Finally, in the last part of the experiment, participants were presented with a compatible and incompatible mix of audio and visual stimuli and asked to rate their annoyance. We found that several audio stimuli were assessed differently, to a significant extent, by listeners after video clips were added to them Institute of Noise Control Engineering. Primary subject classification: 63.7; Secondary subject classification: INTRODUCTION There is growing evidence showing the audio-visual interaction of noise in the environment. In daily life it may happen that a given sound source is heard but not seen, or the same sound source is perceived in a different visual setting. In the literature, the first problem is known as the influence of the visibility of the sound source on subjective noise annoyance assessment. Bangjun et al. 1 wanted to find out whether the visibility of the source of noise in similar acoustic environments can affect the level of noise annoyance. He proved that when the sound source cannot be seen, the noise annoyance is lower than when compared to the situation where both audio and visual information is available. In contrast, Maffei et al. 2 analyzed how the visual characteristics of a screen can influence the noise a) Institute of Acoustics, Faculty of Physics, Adam Mickiewicz University, Poznan, POLAND; apraton@amu.edu.pl. b) Institute of Acoustics, Faculty of Physics, Adam Mickiewicz University, Poznan, POLAND; honorata.hafke@ gmail.com. c) Institute of Acoustics, Faculty of Physics, Adam Mickiewicz University, Poznan, POLAND; mszychowska@ gmail.com. d) Institute of Acoustics, Faculty of Physics, Adam Mickiewicz University, Poznan, POLAND; jen@amu.edu.pl. e) Institute of Acoustics, Faculty of Physics, Adam Mickiewicz University, Poznan, POLAND; janaku@amu.edu.pl. perception of local residents. In the case of transparent barriers where sound sources were seen, perceived annoyance, as well as perceived loudness, were judged lower than when compared to the situations in which the sound sources were not seen (industrial, opaque or green barriers, see also Aylor and Marks 3 ). Sound source visibility is partly involved in studying the influence of recognition on noise annoyance caused by a given sound source. If we are unable to identify the sound source based on audio information alone, then adding visual information solves this problem. In their study, Van Renterghem et al. 4 concluded that when the subjects could not identify the sound source, they assessed it differently to when the picture of the sound source was shown and they could easily recognize it. There are also interesting findings regarding the environmental context of audio-visual interactions. Carles et al. 5 and Maffiolo et al. 6 showed that the sounds of nature influence our assessment of a landscape. On the other hand, being in green areas could result in better assessment of the acoustical nature of an evaluated area (Gidlöf-Gunnarsson and Öhrström 7 ). Also adding visual information to natural sounds, such as the sounds of birds or frogs, can enhance the subjective evaluation of the environment (Tsai and Lai 8 ). The second problem relates to the noise annoyance assessment of different audio-visual stimuli combinations. In these combinations, visual information was either associated or unassociated with sound (as in Cox's study 9 )or matched or unmatched with audio information (as in our 34 Noise Control Engr. J. 64 (1), January-February 2016 Published by INCE/USA in conjunction with KSNVE

2 previous study 10 ). The soundscape analyzed in our previous study 10 could be a combination of several sound sources in this paper we refer to one type of sound source in one audio clip. Cox pointed out that a picture of fingernails on a blackboard, or a picture of a dentist, made the associated sound more awful. On the other hand, in the conclusion of our previous study 10 it was stressed that if there is a need to change the audio-visual representations of an environment, audio information should be considered more important than visual information. In contrast, Viollon et al. 11 argued that the common statement the more urban visual setting, the more negative sound ratings depends on the type of sound source. The results of their study showed that in the case of human noises (speech, steps, etc.) the degree of urbanization does not have much influence on the sound ratings anymore. In audio-visual combinations, what matters is some visual characteristics, especially color. Fastl 12, for example, has shown that the loudness of a red train can be rated 15% higher than the loudness of a green train. A similar effect was observed in the case of car noise perception, where a nice image of a car drastically reduced the negative loudness assessment (Hashimoto and Hatano 13 ) or unpleasantness (Hatano et al. 14 ). In the studies cited above, visual information influences noise annoyance assessment. However, current studies 15 of noise annoyance neglect the sense of sight. The question arises: having all this evidence about the audio-visual interactions of environmental noise, can we still restrict our research in noise annoyance assessment to one modality, that is, only to the sense of hearing? The psychoacoustic experiments performed in this study should provide an answer to this question. Currently, the most popular approach defines noise annoyance in terms of a single physical variable, i.e. sound pressure level, which is easily identified and measured both in field or laboratory conditions (for recent reviews see Marquis-Favre et al. 15 ). Examples of such noise ratings are, among others, energy-based ratings such as L Aeq,T, L DEN, and SEL [see Eqns. (1) (3) 16,17 ]. Z 1 T L Aeq;T ¼ 10 log :1L pa ðþ t dt ð1þ T 1 L DEN ¼ 10 log :1L day þ :1 ð L eveningþ5þ 24 þ :1 ð L nightþ10þ ð2þ T SEL ¼ L Aeq;T þ 10 log 10 ; T 0 ¼ 1 ½Š: s ð3þ On the other hand, in the literature one can find a noise annoyance formula where annoyance is defined as a multicomponent concept depending on more than one acoustical variable. Psychoacoustic annoyance, PA 18 0 T 0 [au] belongs to such a multicomponent noise annoyance model. It is defined as follows: PA ¼ ðn 5 =sone q Þ 1 þ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi w 2 S þ w2 FR ; ð4þ with N 5 percentile loudness in sone and w S describing the effect of sharpness S in acum, S N 5 w S ¼ 1:75 0:251 g acum sone þ 10 ; ð5þ for S > 1.75 acum and w FR describing the influence of fluctuation strength, F, in vacil and roughness, R, in asper, 2:18 w FR ¼ ðn 5 =soneþ 0:4 0:4 F vacil þ 0:6 R : ð6þ asper Roughness (defined in aspers),sharpness(acums)and fluctuation strength (vacils) are related to the timbre of sound, and their definitions can be found in Fastl and Zwicker 18. Psychoacoustic annoyance calculated on the basis of this formula [Eqn. (4)] shows a better agreement with subjective noise annoyance assessment than single energy-based ratings 18. In this study the results of psychoacoustic experiments will be compared with objective noise characteristics such as L Aeq,T, PA, N 5, S, F, andr. We would like to test whether the correlation between noise characteristics and annoyance assessments is the same for the audio and audio-visual presentation of stimuli. 2 AIM The aim of this paper is to verify, on the basis of psychophysical experiments, how a human being processes audio-visual information coming from different environmental noises, which can be encountered in daily life. The experiment was divided into three parts: auditory experiment 1, visual experiment 2 and audiovisual experiment 3. In each part, the ICBEN scale (0 10) 19,20 was used to rate the presented stimuli. In experiment 1 only the audio stimuli were presented and subjects were asked to rate the sound annoyance. In experiment 2 participants were asked to rate the pleasantness of the presented visual clips. In experiment 3 participants were presented with compatible and incompatible mixes of audio and visual stimuli and asked to rate their annoyance. The results of the audio annoyance assessments were compared with objective noise characteristics. However, the main challenge in this study was to discover the hidden rules which people were probably using when combining audio and visual information in annoyance judgments of audio-visual stimuli. To attain this goal, the CART (classification and regression trees) method, Noise Control Engr. J. 64 (1), January-February 2016 Published by INCE/USA in conjunction with KSNVE 35

3 which uncovers the hidden structure in the data, was applied to the experimental data from experiment 3. 3 METHOD 3.1 Subjects Forty-four students (17 M, 27 F) with normal hearing (self-reported) and normal or corrected to normal vision (self-reported by each subject, correction was made with glasses or contact lenses) took part in the experiment. All the participants took part in all 3 experiments. Experiments 1 and 2 were conducted on the same day, and experiment 3 was conducted on another day. Experiment 1 was approximately 1.5 hours long, experiment 2 was 30 minutes long, and experiment 3 was 3 hours long. The participants always took breaks after each minutes (depending on how quick they were at rating the samples). Altogether with breaks, the whole experiment took around 7 hours for each participant. The experiments were always conducted in the same order. All the participants were provided with financial remuneration for taking part in this study. 3.2 Stimuli and Equipment The stimuli from five different environments were recorded at the same time as the video and audio clips and used in all 3 parts of the study. Table 1 presents the five different noises (20 s duration, 10 ms fade-in/out) that were recorded and used in experiments 1 and 3, and Fig. 1 presents screenshots of the video clips that were used in experiments 2 and 3. The audio stimuli were recorded in a 4-channel B-format with a first order ambisonics microphone ST450 MKII SoundField Portable and high quality recorder SQuadriga II HEAD Acoustics. The obtained audio recordings were converted to a 26-channel file using custom written software in C# language. The visual stimuli were recorded with a high definition camera Canon XF100. To calibrate the system, a sound meter Svan 912 AE with a G.R.A.S. 40AN microphone was placed in a position matching that of the participant's head. The auditory stimuli used in experiments 1 and 3 were set to the following sound levels: L Aeq,T = 45, 55, 65 db. In the analysis we refer to the auditory stimuli at different levels as: abbreviation_level, e.g. SEA_45 means the auditory recording of the sea presented at 45 dba, and RD_65 means the road presented at 65 dba. 3.3 Procedure All three experiments took place in an anechoic chamber at Adam Mickiewicz University in Poznań, Poland.It was appropriately adapted to present ambisonic recordings by installing a setup of speakers (Yamaha HS50m) arranged in a cubic form. It was also fitted with a high quality, quiet projector (NEC NP-PA500U), as well as a large, perforated (sound-permeable) screen (KAUBER X-Frame Standard ) for video presentation. The real picture of the whole setting is presented in Fig. 2. Throughout the whole duration of the study, the light in the anechoic chamber was turned off. The system used made it possible to mimic a real life experience in a laboratory setting. We tried to eliminate any equipment that is not used in real life situations, such as helmets, 3d glasses (see Maffei et al. 2 ) or headphones, to prevent the means of presentation influencing the results. In real life a person is surrounded by sounds; thus we decided that ambisony will be the best way to mimic the soundscape, by preserving the spatial context of the sound sources (see Parkman Carter and Braasch 21 ). In the visual part we decided to use a large screen and an HD projector to get the best quality and to suggest the visual context to the subjects. Of course the subjects were not surrounded by landscapes; however, the gathered results suggest that intermodal interactions exist in such a quasi-real paradigm, which was the goal of the experiment. To conclude, we must emphasize that in our experiment we compared audio, visual and audio-visual samples presented in the same way; thus the analysis gives a clear answer to the question concerning the existence of audio-visual interactions. We do know that the means of presentation is not strictly a real-life paradigm; however, we used real-life environmental signals (both audio and visual) presented in a fully-controlled way to mimic the spatial properties of soundscapes and the visual context of the environment. Table 1 Auditory stimuli used in the study. Stimulus Abbreviation Description Sea SEA Calm waves at the seaside Fountain FN Small fountain between buildings Airplane AP Take-off of an airplane close to the airport Road RD Busy street, fairly large volume of traffic Train TR Passing of a freight train 36 Noise Control Engr. J. 64 (1), January-February 2016 Published by INCE/USA in conjunction with KSNVE

4 SEA FN RD AP TR Fig. 1 Screenshots of the visual stimuli. In experiment 1 the participants were asked to assess the short-time annoyance caused by auditory stimuli on the standardized ICBEN scale (0 10), where 0 was not annoying at all and 10 was extremely annoying. Participants were presented with the 5 soundscapes at 3 sound levels (L Aeq,T = 45, 55, 65 db). Each stimulus was repeated 3 times. It was not suggested that the listeners imagine the sound sources. In experiment 2 the participants were asked to assess the pleasantness of visual stimuli on the standardized Fig. 2 Audio-visual setting in the anechoic chamber. ICBEN scale (0 10), where 0 was not pleasant at all and 10 was extremely pleasant. The results of the pleasantness assessments were converted to unpleasantness assessment (unpleasantness =10 pleasantness) in order to present the results from both senses on the same scale. Participants were presented with the 5 landscapes. Each stimulus was repeated 3 times. One can argue that we should have asked participants to assess the annoyance caused by videos, rather than the pleasantness (which then would be converted to unpleasantness). In our opinion, annoyance is not a term which seems to be natural in the case of sight. Intuitively, we rather assess our good-feeling in some landscapes i.e. pleasantness. That is why we decided to use this approach. We then converted pleasantness to unpleasantness on the basis of findings from a Master Thesis written in Polish Wpływ informacji wzrokowej na ocenę akustyczną przestrzeni miejskiej ( The influence of visual information on acoustic assessment of urban area ) by Marcin Praszkowski in 2012 at A. Mickiewicz University, in Poland. In experiment 3 the participants were asked to assess the short-term annoyance caused by the mixture of auditory and visual stimuli once again, rating the experience on the standardized ICBEN scale (0 10), where 0 was not annoying at all, and 10 was extremely annoying. Participants were presented with mixes of 5 different soundscapes at 3 sound levels (L Aeq,T = 45, 55, Noise Control Engr. J. 64 (1), January-February 2016 Published by INCE/USA in conjunction with KSNVE 37

5 65 db) and 5 landscapes. Only combinations which could be encountered in daily life were presented. Each stimulus presentation was repeated 3 times. In the figures, mixes are defined as audio stimulus abbreviation + video stimulus abbreviation, e.g.ap + FN is the mixture of the airplane audio clip and the fountain video clip. All the stimuli in each experiment were presented in random order. Every experiment was preceded by an oral instruction given by the researcher. Instructions were also presented on the screen at the beginning of the procedures. The experimental conditions are summarized in Table Objective Analysis An objective analysis of the auditory stimuli was conducted with dedicated software, ArtemiS 10. To obtain a close representation of what the participants really heard in the anechoic chamber, the auditory samples used in this analysis were recorded binaurally (Squadriga headset) from the position of the participant's head. The results of this analysis are shown in Table 3. 4 RESULTS 4.1 Experiments 1 and 2 The noise annoyance assessments of audio stimuli at three different noise levels and unpleasantness of visual stimuli, averaged across all subjects, are presented in Fig. 3. Because the assumptions of normality and sphericity for our data were not fulfilled, we performed bootstrap tests (with 10,000 iterations) for computing means and their confidence intervals 22. Bootstrapping is a statistical method for estimating the sampling distribution of an estimator by sampling with a replacement from the original sample. The method assigns measures of accuracy (defined in Table 2 Experimental conditions for all three experiments. No. of stimuli No. of levels No. of repeats Total Exp. 1 5: SEA, FN, AP, TR, RD Exp. 2 5: SEA, FN, AP, TR, RD Exp. 3 5 Compatible mixes: SEA + SEA, AP + AP, FN + FN, RD + RD, TR + TR 5 Incompatible mixes: AP + FN, AP + SEA, FN + SEA, RD + FN, RD + TR Table 3 Calculated noise characteristics of auditory stimuli. N 5 (sone) F (vacil ) R (asper ) S (acum) PA (au) SEA_ SEA_ SEA_ FN_ FN_ FN_ AP_ AP_ AP_ RD_ RD_ RD_ TR_ TR_ TR_ terms of confidence intervals) to sample estimates. When computing the statistical significance p of differences between two groups, we used permutation tests (again with 10,000 iterations). A permutation (exact) test was conducted to avoid the influence of skewness in the observed data 22. What was expected was that the relationship between noise level and annoyance ratings would be similar for all the investigated stimuli. The higher the sound level, the more annoying the noise. Out of all the audio stimuli, the least annoying is the SEA stimulus, while the most annoying are the RD and TR stimuli. A similar relationship occurs for the visual stimuli with the least unpleasant SEA and the most unpleasant RD and TR stimuli. Note that for the level of 45 db there is a significant difference of annoyance rating between TR and RD (p =.0229), while for level of 55 db there is no significant difference between these sources (p =.5028), and the biggest difference is for 65 db (p =.0002; TR is less annoying than RD). We assume this big difference in assessing annoyance for TR and RD comes from the effect called the railway bonus, when the same level of train provokes significantly smaller annoyance rating than for other noise sources 23. One can wonder why the AP audio stimulus is less annoying than the RD and, sometimes, TR in contrast to many papers which prove the AP noise to be more annoying 24. We think this is related to the height of the flight of the recorded airplane it was high enough to provoke only slight changes in sound level and that is why people found it more pleasant than the RD and TR noises, which were recorded close to the sources. 38 Noise Control Engr. J. 64 (1), January-February 2016 Published by INCE/USA in conjunction with KSNVE

6 Fig. 3 The averaged arithmetic means of noise annoyance assessments with 95% confidence intervals for audio stimuli: SEA (sea), FN (fountain), AP (airplane), RD (road), TR (train) presented at three sound levels: 45, 55 and 65 db and the averaged arithmetic means of annoyance (unpleasantness) ratings of the same visual stimuli. For the ICBEN scale, 0 means not annoying at all and 10 extremely annoying (a value not reached in our experiments; hence the y-axis scale is from 0 to 9). 4.2 Experiments 1 and 3 The averaged noise annoyance ratings of the investigated compatible and incompatible audio-visual stimuli (experiment 3), together with audio stimuli only (experiment 1), are presented in Fig. 4. Compatible means that the audio and video stimuli presented the same source (e.g. the sound of an airplane and a video of a takeoff) while incompatible means a mixture of different sources (e.g. airplane audio stimuli with a road traffic video). From the analysis of the data obtained in experiment 3 it is possible to answer two questions: (1) is there any significant difference between the annoyance judgments based on audio information only, and based on both types of information (i.e. audio and visual)?; and (2) are there any rules that people might apply when combining audio and visual information in the annoyance judgment of an audio-visual stimulus? The analysis of the results presented in Fig. 4 should answer the first question. The audio stimuli are marked in Fig. 4 as null (only audio, no video; annoyance ratings were taken from the Experiment 1 results), the compatible audio and visual information is marked as CO and the other five abbreviations represent the incompatible audio and visual information (the abbreviations should be interpreted as follows: audio stimulus abbreviation + video stimulus abbreviation, e.g. AP + SEA should be read as an audio sample of an airplane (AP) mixed with video samples of the sea (SEA)). For the sound level equal to 45 db, the compatible visual information added to the audio stimuli caused an increase in the noise annoyance assessments for RD (p <.0001) and TR (p <.0001) audio-visual stimuli. In the case of the RD audio stimulus, the same result was obtained also in the case of added incompatible TR visual information (p <.0001). For the SEA, FN and AP audio stimuli, neither the compatible nor the incompatible visual information caused a change in the noise annoyance assessments. A similar effect as the one observed for the sound level of 45 db occurs for the sound level equal to 55 db. However, the compatible visual information added to the audio Noise Control Engr. J. 64 (1), January-February 2016 Published by INCE/USA in conjunction with KSNVE 39

7 Fig. 4 The averaged arithmetic means of noise annoyance assessments with 95% confidence intervals for audio only (marked as NULL ), audio-visual compatible (both audio and video from the same sound source, marked as CO ) and incompatible (audio and video presented different sound sources) stimuli: SEA (sea), FN (fountain), AP (airplane), RD (road), TR (train) presented at three sound levels: 45, 55 and 65 db. The incompatible stimuli were: AP + FN, AP + SEA, FN + SEA, RD + FN, and RD + TR presented at each sound level. The meaning of the scale on the y-axis is the same as in Fig. 3. stimuli did not cause a statistically significant increase in the noise annoyance assessments for RD (p =.09), but did for TR (TR audio and TR video, p =.009). Adding incompatible TR visual information to RD audio also changes the annoyance rating (p =.0007). In addition, adding incompatible visual SEA information to the FN audio stimulus causes a decrease in annoyance rating (p =.047). For the sound level of 65 db, compatible visual information added to the FN audio stimuli caused a decrease in annoyance rating (p =.0011). In contrast, compatible visual information added to the TR audio stimulus increased the annoyance rating (p =.0041). To summarize, the answer to the first question (is there any significant difference between the annoyance judgments based on audio information only and those based on both types of information (i.e. audio and visual)?) is as follows: there is a significant difference in the noise annoyance assessment between the audio (plotted in Fig. 4. as x ) and the audio-visual stimulus (in Fig. 4 every other sign). However, it does not occur for all the stimuli. The difference depends on the type of sound source and sound level. Only for the audio TR does adding compatible visual information increase annoyance significantly for each sound level (45 db: p <.0001; 55 db: p =.009; 65 db: p =.0041). For the audio stimuli RD, all visual information (no matter whether compatible or not) changes the annoyance rating significantly, but only for the sound level of 45 db (RD + RD, p <.0001; RD + FN, p =.021; RD + TR, p <.0001). The only other significant change for the audio RD at the level of 55 db can be observed for the mix with the TR video (p =.0007). For the FN audio stimuli, significant differences occur only when adding compatible FN visual information at the level of 65 db (p =.0011) or adding incompatible SEA video at the level of 55 db (p =.0472). In contrast, for the SEA and AP audio stimuli no changes are observed with any combinations of audiovisual information independently of sound level. To provide an answer to the second question (are there any rules that people might apply when combining audio and visual information in the annoyance judgment of audio-visual stimulus?), the CART, a robust analysis method that uncovers the hidden structure in data was 40 Noise Control Engr. J. 64 (1), January-February 2016 Published by INCE/USA in conjunction with KSNVE

8 applied to the results obtained in experiment 3. The analysis was made in an R environment 25 using an rpart.rplot package 26. Because our discriminating factor had a quantitative character, the classification was based on the loglikelihood function. The splitting criteria SC [Eqn. (7)] take into account the sums of squares: SST [Eqn. (8)] is the sum of squares for the node. SSR and SSL are the sums of squares for right and left branch, respectively. SC ¼ SS T ðss L þ SS R Þ ð7þ SS T ¼ X ðy i y Þ 2 : ð8þ These criteria maximize the between-groups sum-ofsquares in a simple analysis of variance, producing a tree-like decision structure. We have chosen this method because it could explain the obtained results, and in addition it requires no assumption regarding the underlying distribution of the data 27. The decision tree obtained from the data of experiment 3 is presented in Fig. 5.Inthetopof the boxes, the average rating of annoyance is shown, computed for n observations (n could be found in the bottom of the boxes). The percentile values shown in the bottom right of the rectangles explain how many observations (in percent) are covered by the node relating to the number of all observations (which is equal to 1320). The text shown on the branches is the decision criterion for splitting observations into another two groups. According to this model, stimuli could be split into 7 different groups (Fig. 5). The lowest mean annoyance (2.4) was observed when the sound level was 45 or 55 db and the sound sources were SEA or FN. The AP audio stimuli were assessed differently, regarding the level of 45 or 55 db with annoyance ratings of 3.1 or 4.4 respectively. Other sound sources (RD or TR) caused higher annoyance (5.3), which depends on the sound level less than (4.8) for 45 db and more than (5.8) for 55 db. For the stimuli at the sound level equal to 65 db, the audio sound sources SEA or FN reduced annoyance to 5.2 points (the mean for all the stimuli at the sound level equal to 65 db was 6.5 points). Annoyance ratings increased by almost 2 points (1.9) in the case of other sound sources (AP, RD and TR). It might be interesting that the regression tree did not show any node depending on the video type. The analysis uncovers the order of importance of the variables taken into account during the decision process. The sound level was found to be the most important, then (depending on the sound level) the type of sound source. It is clear that the analysis uncovers two groups of different sound sources, with SEA and FN belonging to the first group and AP, RD and TR to the second group. It also confirmed how people proceed when judging the noise annoyance of the given sound source. At first, the sound level is the most important in the decision of which noise generates the highest annoyance. Next, for the sound levels of 45 and 55 db, the attention Fig. 5 The decision tree obtained from the data of experiment 3. Noise Control Engr. J. 64 (1), January-February 2016 Published by INCE/USA in conjunction with KSNVE 41

9 Table 4 The percentage of explained variance (o) calculated for sound characteristics in relation to the noise annoyance ratings of audio only or compatible audio-visual stimuli presentation (Pearson correlation test). Significant correlations (with p <.05) were marked with *. Sound characteristics versus ICBEN ratings Explained variance o Number of stimuli For all stimuli 15 For AP. RD. TR 9 For SEA and FN 6 ICBEN Audio Audio-video Audio Audio-video Audio Audio-video N * 0.3 * 0.88 * 0.86 * F R 0.45 * * 0.89 * 0.85 * 0.74 * S PA 0.56 * 0.29 * 0.85 * 0.86 * depends on the type of sound source. Sound sources which are a representation of natural sound sources (i.e. SEA, FN) generate the lowest noise annoyance ratings, in contrast to technological sound sources (i.e. AP, TR, RD). At this point we have to stress that the AP audio is quite ambiguous for listeners. It is rated low when the sound level is equal to 45 or 55 db, while its rating goes high when the sound level is 65 db. At higher sound levels, which in our study was 65 db, people turn their attention to the audio information (whether it is a natural sound source or a technological sound source ). 4.3 Comparison of Audio and Audio Visual Annoyance Assessments with Objective Noise Characteristics For each investigated stimulus, the following objective noise characteristics were calculated: percentile loudness measures N 5, fluctuation strength F, roughness R, sharpness S and psychoacoustic annoyance PA. The noise annoyance ratings obtained for audio stimuli were correlated with these objective noise characteristics (as we did before in Kaczmarek and Preis 28 ). The percentage of explained variance (presented in Table 4), being a measure of this correlation, was calculated for each noise characteristic. One can notice that N5 do not explain much of the variability in noise annoyance assessment of all the investigated stimuli. The different o value obtained for the whole set of stimuli compared to two subsets of the data confirms the results of the CART analysis uncovering two different groups of stimuli note that for technological sources N5 explains almost 90% of variance and is statistically significant, while for natural sources this relation does not occur. The same difference could be observed also for PA. 5 CONCLUSIONS The results of the experiments performed in this study enable the following conclusions to be drawn: 1. For three out of the five audio stimuli investigated in this study, adding visual information causes a significant change in the noise annoyance assessment of the audio-visual stimuli; 2. This change occurs for both compatible (the same sources in the audio and video clips) and incompatible visual information (one source in an audio clip and another, different source in a video clip); 3. The type of sound source influences the direction of the noise annoyance changes. For the two technological sound sources, i.e. RD and TR, the addition of visual information increases the noise annoyance assessment, while for the natural sound source, FN, the noise annoyance rating decreases. The decision tree obtained based on the experimental data revealed the existence of two groups of stimuli: one related to the natural sound sources and the other resembling technological sound sources. Different decision rules were applied to these sound sources, leading to the final estimate of their noise annoyance. 6 ACKNOWLEDGEMENTS This research was supported by a grant from National Science Centre: Project Number UMO-2011/ 03/B/HS6/ The authors would like to thank Agnieszka Klawiter for her help and advice. 7 REFERENCES 1. Z. Bangjun, S. Lili and D. Guoqing, The influence of the visibility of the source on the subjective annoyance due to its noise, Applied Acoustics, 64(12), , (2003). 42 Noise Control Engr. J. 64 (1), January-February 2016 Published by INCE/USA in conjunction with KSNVE

10 2. L. Maffei, M. Masullo, F. Aletta and M. Di Gabriele, The influence of visual characteristics of barriers on railway noise perception, Science of the Total Environment, , 41 47, (2013). 3. D.E. Aylor and L.E. Marks, Perception of noise transmitted through barriers, J. Acoust. Soc. Am., 59(2), , (1976). 4. T. Van Renterghem, A. Bockstael, V. DeWeirt and D. Botteldooren, Annoyance, detection and recognition of wind turbine noise, Science of the Total Environment, , (2013). 5. J.L. Carles, F. Bernáldez and J. de Lucioc, Audio-visual interactions and soundscape preferences, Landscape Research, 17(2), 52 56, (1992). 6. A.V. Maffiolo, M. Castellengo and D. Dubois, Qualitative judgments of urban soundscapes, InterNoise99, (1999). 7. A. Gidlöf-Gunnarsson and E. Öhrström, Noise and well-being in urban residential environments: The potential role of perceived availability to nearby green areas, Landscape and Urban Planning, 83(2 3), , (2007). 8. K.T. Tsai and P.R. Lai, The research of the interactions between the environmental sound and sight, International Congress on Acoustics, (2001). 9. T.J. Cox, The effect of visual stimuli on the horribleness of awful sounds, Applied Acoustics, 69, , (2008). 10. A. Preis, J. Kociński, H. Hafke-Dys and M. Wrzosek, Audiovisual interactions in environment assessment, Science of the Total Environment, 523, , (2015). 11. S. Viollon, C. Lavandier and C. Drake, Influence of visual setting on sound ratings in an urban environment, Applied Acoustics, 63(5), , (2002). 12. H. Fastl, Audio-visual interactions in loudness evaluation, International Congress on Acoustics, (2004). 13. T. Hashimoto and S. Hatano, Effect of factors other than sound to the perception of sound quality, International Congress on Acoustics, (2001). 14. S. Hatano, T. Hashimoto, Y. Kimura and T. Tanaka, Sound quality evaluation of construction machine, International Congress on Acoustics, (2001). 15. C. Marquis-Favre, E. Premat and D. Aubree, Noise and its effects A review on qualitative aspects of sounds. Part II. Noise and annoyance, Acta Acustica United Acustica, 91, , (2005). 16. B. Berglund, T. Lindvall, D.H. Schwela, (Eds), Guidelines for Community Noise, World Health Organization, (1999). 17. Directive 2002/49/EC of the European Parliament and of the Council, Official Journal of the European Communities, L189, 12 15, (2002). 18. H. Fastl and E. Zwicker, Psychoacoustics: Facts and Models, Springer Science & Business Media, (2007). 19. J.M. Fields, R.G. De Jong, T. Gjestland, I.H. Flindell, R.F.S. Job, S. Kurra, P.V. Lercher, T. Yano, R. Guski, U. Felscher-Suhr and R. Schuemer, Standardized noise reaction 11 questions for community noise surveys: Research and a recommendation, J. Sound Vibr., 242(4), , (2001). 20. International-Standards-Organization, ISO TS 15666: Acoustics Assessment of noise 11 annoyance by means of social and socio-acoustic surveys. Geneva (CH), (2003). 21. J. Parkman Carter and J. Braasch, Cross-modal soundscape mapping: Integrating ambisonic field recordings with high dynamic range spherical panoramic photography to produce interactive maps, J. Acoust. Soc. Am., 135, 2187, (2014). 22. Ch.H. Yu, Resampling methods: Concepts, applications, and justification, Practical Assessment, Research & Evaluation, 8(19), (2003). 23. H. Fastl, S. Kuwano and S. Namba, Psychoacoustics and rail bonus, InterNoise94, (1994). 24. S.A. Janssen, H. Vos, A.R. Eisses and E. Pedersen, A comparison between exposure response relationships for wind turbine annoyance and annoyance due to other noise sources, J. Acoust. Soc. Am., 130(6), (2011). 25. R. Core Team, A language and environment for statistical computing, Foundation for Statistical Computing, , (2015). 26. T. Therneau, B. Atkinson and B. Ripley, rpart: Recursive Partitioning and Regression Trees. R package version , L. Breiman, J.H. Friedman, R.A. Olshen and C.J. Stone, Classification and Regression Trees, Wadsworth, Belmont CA USA, (1984). 28. T. Kaczmarek and A. Preis, Annoyance of time-varying roadtraffic noise, Archives of Acoustics, 35(3), , (2010). Noise Control Engr. J. 64 (1), January-February 2016 Published by INCE/USA in conjunction with KSNVE 43