Proceedings of Meetings on Acoustics

Transcription

1 Proceedings of Meetings on Acoustics Volume 19, ICA 2013 Montreal Montreal, Canada 2-7 June 2013 Psychological and Physiological Acoustics Session 1aPP: In Memory of Bertram Scharf: Five Decades of Contributions to Auditory Perception 1aPP5. Connecting cues to signals in auditory attention Ervin R. Hafter* *Corresponding author's address: University of California, Berkeley, California 94720, Among his many fields of study, Bert Scharf played a major role in our understanding the role of auditory attention, especially at the level of basic psychophysics. Of special importance to this talk is the profound influence that he had on research in our lab (myself, Bert Schlauch, Joyce Tang, Kourosh Saberi and Poppy Crum) through his work on signal uncertainty in masking and its alleviation by specific informational cues. Scharf's resurrection of the probe-signal method led us to examine the effects of uncertainty on both the means and variances of effective bandwidths used by listeners in a detection task. Also shown will be cases where we used different kinds of cues to study detection at various levels of processing including judgments based on: specific spectral components, complex pitches derived from sets of harmonics and locations in frequency reliant on mentally tracking an FM stimulus through a period of occlusion. Published by the Acoustical Society of America through the American Institute of Physics 2013 Acoustical Society of America [DOI: / ] Received 22 Jan 2013; published 2 Jun 2013 Proceedings of Meetings on Acoustics, Vol. 19, (2013) Page 1

2 CONNECTING CUES TO SIGNALS IN AUDITORY ATTENTION One of the major achievements of Bertram Scharf s career in audition was the impact that he and his colleagues had on the study of auditory attention, both in its content and in the impetus that it gave to others. When listening for a tone in wideband noise, the effective masker is the portion of the noise falling within the so called critical band or auditory filter. Scharf viewed this task as an act of selective attention in which optimal performance requires the listener to base judgments entirely on the single filter centered on the signal s frequency. The idea is that attention may be directed to this filter through memory built up by experience in the task or by explicit cues presented before an experimental trial. In order to address this problem Scharf became an avid user of the probesignal method (1). Using this method, the effective internal filter is found by training the subject to expect one frequency but, on occasion, presenting probe signals whose frequencies differ from expectation. If performance is truly based only on energy in the expected filter, the off-center probes should be attenuated, strongly reducing their detection. He and his colleagues found that when the levels of tonal signals were set to produce 90% performance at the expected frequency when it was tested alone, performance fell to near chance for probes a half critical band away. Similarly, with qualitatively different signals such as a band of noise or a multi-tone complex, detection was best if their energy fell within the expected band (2). The task was somewhat changed in an experiment in which psychometric functions were obtained for a variety of probes (3). Then, the effects of attenuation on tones increasingly distant from the center seemed to bottom out, suggesting that the subject included some information from a filter closer in frequency to the probe. This is, perhaps, not surprising since attainment of a psychometric function requires use, at times, of higher levels, a stimulus that might be expected to change the subject s attention criterion to include more than a single band. This may also reflect a change in subjective strategy, one later called by Scarf, a heard but not heeded rule. It says that in the probe-signal method, the listener may reject any stimulus not exactly matched to the expected signal, even if it was loud and clear. Another case where this rule seems to hold is one where the probe signal method was used to test subjects who were asked to detect signals with a different duration than they expected (4). As probe durations deviated from expectation, performance fell to chance. However, when subjects were told that durations might be different, performance on probes showed almost no loss. Possibly the feature of Bert Scharf s work on attention that will be remembered longest is his speculation on the neural basis of auditory shared attention to frequency. Thinking that the mechanism for selection lie in the cochlear efferents, he worked with subjects about to have the olivocochlear bundle cut as part of treatment for an acoustic neuroma, giving them a battery of tests before and after the surgery (5). These are difficult experiments to do because of potential auditory disorders stemming from the mere presence of the neuroma, things that might affect pre and post operative performance in the behavioral tasks. Nevertheless, these were extremely clever ideas that have led others to think more deeply about the underpinnings of auditory attention. Research in Berkeley Scharf s work served as a beacon to our lab in Berkeley, where, the students, Poppy Crum, Ronald Hübner, David Johnson, Lyne Plamondon, Kourosh Saberi, Bert Schlauch and Joyce Tang, and I, were also studying attention and its control through subjects expectancies of specific signals, usually tones, in noise. With a primary goal being to examine the effects of signal uncertainty, we began with only two possible signals with tonal cues used to tell subjects where to attend (6). Deciding that we needed more uncertainty in order to get a good grasp on the problem, we moved toward the traditional solution (7) in which the signal presented in a wideband noise was selected at random from trial to trial from an extensive range of possibilities. With signal levels at each frequency set to produce the same detectability when tested alone, this task generally produces a 3-5 db loss in performance relative to listening for only one tone. We were also interested in an earlier speculation (8) that an ideal observer forced to monitor a wide range of frequencies might choose to pool information from adjacent critical bands into wider listening bands. The thought was that while this would reduce performance in the individual bands, reducing their number might provide some relief from the cognitive load of multiple calculations. Our approach (9) to testing this hypothesis required that the relation between listening bandwidths and uncertainty reflect good quantitative measures of both. The traditional measure of masked detection with no cost of presents a single signal throughout a block of trials. In search of a metric more amenable to a task with signals spread over a wide range of frequencies, signals would be selected at random on every trial, but they would be Proceedings of Meetings on Acoustics, Vol. 19, (2013) Page 2

3 preceded by an audible cue set to match their frequency. When performance turned out to be the same as in singletone testing, we declared this task to be the standard for performance with no uncertainty and used it to define an uncertainty index of 1. Measurement of the listening bands in this no-uncertainty condition was done with a simple modification of the classical probe-signal method, using relative frequencies set to a fixed percentage of the expected signal. To the extent that auditory filters exhibit a constant Q-factor across our range of possibilities, we felt comfortable in accumulating performance across trials in accord with the percentage difference of each probe from expectation. With concern for the heard-but-not-heeded rule discussed above, our subjects were told in advance that while the cue would generally match the signal in one of two intervals, it would sometimes be a bit off. If so, they should still say which interval had a tone. When one subject asked why we were doing this, we answered, There seems to be a bug in the computer program that runs the experiment. Since everyone is comfortable with the notion that computers make mistakes, this seemed to satisfy him. Further evidence in support of this technique as a measure of auditory filters bands without uncertainty is that with signals drawn from a range of more than three octaves, the log-bandwidths of the cumulative filters closely resemble those of auditory filters measured with notch-band noise (10). The traditional metrics used to quantify frequency uncertainty in a configuration such as this have been the frequency range or the number of independent critical bands within that range (7). However, these numbers have proven to be of little value for modeling uncertainty. Our approach was to control the number of listening bands monitored by the subject through control by the number of frequencies in the cue. So, the condition a singlefrequency cue was said to have an uncertainty of 1. When cues consisted of two tones, one matched to the signal and another chosen at random, it was deemed to have an uncertainty of 2; similarly, when the cue consisted of four tones, the uncertainty was 4. As uncertainty went from 1 to 4, performance declined in accord with the need to compute one, two or four likelihood ratios as described in Signal- Detection Theory (11). What is more, bandwidths of the effective filters increased, with the band for 4 being significantly wider than those from 1 or 2. These two factors together accounted for the entire loss. Wider effective bandwidths found under conditions of uncertainty is a reminder that while it is often assumed that representations of an auditory dimension are fixed at the cochlea, that information is likely to be modified in successive neural processing, and one cannot discount the possibility that psychoacoustic judgments are based on one of these higher levels. Indeed, one expects that the choice of the listening band is a privilege of the listener s brain that picks the information used to inform motor neurons to initiate a push of button 1 or button 2. From this perspective, we first proposed wider listening bands as a result of pooling data from neighboring cochlear filters; however, but another model might argue that effectively wider bands could also arise from errors in the placement of attention on frequencies in a cue. A test for this distinction required retesting performance across levels to obtain psychometric functions (12). In the model of filter misplacement, performance should fall to chance if the attended filter is misplaced to the point that it receive essentially no energy from the signal. Results supported the pooled bands over listening in the wrong place. Another question arising from this experiment asks whether tonal cues focus the listener on a specific place in memory like an auditory attentional searchlight, or if the audible cue somehow sensitizes a resonant region in memory that draws attention to itself without cognitive intervention. That question continues to the present, with results from quite different cuing paradigms supporting the sensitivity hypothesis (13). It is not clear why these two points of view cannot be fully reconciled but perhaps it stems from the distinction between energetic and informational masking, where the latter is strongly affected by variation in the masker. With this in mind, we (14) repeated the probe-signal task with a single frequency cue that was below the frequency of the signal by a factor of 3/2. This musical 5 th is a relation well known in western music and readily recognized by our subjects. In this condition, the peak of the probe-signal results fell at the frequency of the musical 5 th and responses to occasional probes at the frequency of the cue were at chance. The bandwidth of the listening band in this case was wider than that found in the no-uncertainty condition above, but the difference became smaller with repeated tests with the same subjects, suggesting that it was more an error of misplacement at the 5 th rather than a strategy pooling data across critical bands. Unlike Scharf (3 or 4), we (13) also found evidence of focused attention when the cue was a 5-tone chord for which the expected signal was the missing fundamental. While performance was less than with a matched tonal cue, these results still demonstrated alleviation of uncertainty in the frequency domain. In a final measure of the Proceedings of Meetings on Acoustics, Vol. 19, (2013) Page 3

4 ability to focus attention on a tonal signal, we (15) recruited subjects with absolute pitch and cued them to listen for tonal signals with visual displays showing the signal s corresponding musical notation. Results indicated no difference in performance relative to the case where these same subjects were cued with matching tones. Based on this work, one would say that Scharf s instincts were correct in the sense that appropriate sensory cues can focus the listener on the place in the frequency domain where a signal will appear. What we added was a demonstration that uncertainty could be manipulated and quantized through features of the cues and that direct measures with a probe-signal technique showed that the widths of the effective listening bands widen due to uncertainty. Moreover, at least partially effective cues can be quite different from the signal, such as tones related to it in relative pitch, chords related to it through their complex pitch and visual cues for listeners with absolute pitch. Having described the ability to enhance performance in tone detection with cue that were related to the signal, I will briefly described two experiments where a different kind cuing was used to enhance performance under uncertainty. Some of our pilot experiments had shown that presenting a pure-tone cue did not improve detection of a multi-tonal harmonic sequence for which the cue was the missing fundamental. From this we wondered if this might be explained in terms of the hypothesis offered above about signal detection based on monitoring stimulus domains at different levels of processing. Suppose that the multi-tonal harmonic signal was detected not on the basis of activity in multiple auditory filters in the frequency domain, but rather on its representation in a domain organized by the emergent property, complex pitch. If so, the appropriate cue for ameliorating uncertainty in this case would be a set of tones whose complex pitch matched that of the signal. In order to test this notion, we (16) measured detection of signals composed of three tones. In the first condition, the intention was for subjects to respond by monitoring internal filters attuned to frequency. For this, every trial presented three equally detectable tones drawn from a wide range but otherwise completely unrelated to one another. Thus, the subject would have to listen for the appearance of three bumps in the audio spectrum. In a second condition, while the subject could still just listen for the interval with three tones, the intention was to see if they could respond to putative internal bands attuned to complex pitch. For this, each trial selected a single low frequency called the fundamental, F(1). Then five of its harmonics, F(3), F(4), F(5), F(6,) and F(7) were calculated and three of these were drawn at random to be the signal. When performance in the second condition exceeded that in the first, the superiority was attributed to the higher detectability of an emergent property where false alarms would be fewer. Next, two kinds of cues were tested. When listening for a random selection of three unrelated frequencies, the audible cue has the same three tones as the signal. This is much like the matched frequency cuing for a single-tone signal discussed above. When listening for a randomly selected complex pitch, the three tones in the cue were also harmonics of the randomly chosen F(1) upon which the signal was based. But, they were the three other harmonics, i.e., the ones not used for the signal. Results showed that matched-frequency tones reduced uncertainty for detection of signals of the kind used in the first condition and, similarly, the matched harmonically related tones reduced uncertainty for detection of signals of the kind used in the second condition. Thus, cued detection allowed us to visualize performance of both cuing and detection at two very different functional dimensions in the auditory system. Finally, the probe-signal method was used to examine a case in which the listener must anticipate the frequency through analysis of the history of a dynamically changing tonal cue (17). For this, the cues were FM glides with randomly chosen velocities, slopes and starting points. After a time, the glide disappeared behind wideband occluding noise that totally masked it. When the occluder and cue (FM-glide) were suddenly turned off, what remained was a weaker noise and a brief single frequency tone to be detected. The signal was in the vicinity of the frequency that the glide would have achieved. In order to prevent from simply calculating that frequency from the slope and velocity of the cue, durations of the occluders were chosen at a random. Similarly, in order to prevent detection based on classic notions of continuity (18), the signals were a single frequency rather that an extension of the FM-glide. Correct detections of these frequency-varying probes showed a peak in performance near where the subject should have expected the glide, albeit slightly biased toward the last frequency heard before the glide was occluded. These results suggest that listeners were able to internally track the FM-glide as a way of attending to where it might reappear, essentially following the glide in their mind s ear. The Legacy of Bert Scharf In summary, the total story presented here argues for the use of signal uncertainty in the study of how we organize and respond to internal dimensions in auditory perception and for use of the probe signal-method for Proceedings of Meetings on Acoustics, Vol. 19, (2013) Page 4

5 analysis of the organization and resolution of those dimensions. We have tried to show how this approach can be applied to a number of putative dimension and have further argued that the sensory filters measured with one technique may not be the same as those measured with a different paradigm. Bert Sharf had a terrific impact on all of his colleagues, including us, and for that we owe him a debt of gratitude. I got to know Bert well during times spent working with him in Cassis in Provence in France where we enjoyed the beautiful hospitality of both Bert and Analiisa Scharf. I still picture him flying up and down those hills on his bicycle. The Scharfs included us in their weekly bridge group although I knew little of the sport, and I recall with great amusement driving to the lab in Marseilles the next day listening to his long critique of every hand played the evening before. As in his research, he was a perfectionist and he could remember a bad play he had made for weeks. He was a brilliant and compassionate man and I miss him. Acknowledgement Much gratitude to Kourosh Saberi, Bert Schlauch, Jing Xia and Kelly Whiteford for their gracious willingness to read and give useful comments to drafts of this paper. REFERENCES 1. G. Greenberg and W. Larkin, Frequency-response characteristic of auditory observers in detecting signals of a single frequency in noise: The probe-signal method. J. Acoust. Soc. Am., 44, (1968). 2. B. Scharf, S.Quigley, C. Aoki, C., N. Peachey. and A. Reeves, Focused auditory attention and frequency selectivity, Percept. Psychophys. 42, (1987). 3. H. Dai, B. Scharf and S. Buus, Effective attenuation of signals in noise under focused attention, J. Acoust.Soc. Am. 89, (1991). 4. H. Dai and B. Wright. Detecting signals of unexpected or uncertain durations, J. Acoust. Soc. Am. 98, (1995). 5. B. Scharf, J. Magnan, A. Chays, On the role of the olivocochlear bundle in hearing:16 case studies. Hearing Research, 103, (1997). 6. D. Johnson and, Uncertain-frequency detection- Cueing and condition of observation, Percept Psychophys. 28, (1980). 7. D. M. Green, Detection of auditory sinusoids of uncertainty frequency, J. Acoust. Soc. Am. 33, (1961). 8. and R. Kaplan, The interaction between motivation and uncertainty as a factor in detection, NASA project report, Ames Research Center, Moffit Field, CA, (1976) 9. R. Schlauch, E., Hafter, Listening bandwidths and frequency uncertainty in puretone signal detection, J. Acoust. Soc. Am. 90, (1991). 10. Patterson and I. Nimmo-Smith, Off-frequency listening and auditory-filter asymmetry, J. Acoust. Soc. Am., 67, (1980). 11. D. Green and J. Swets, Signal Detection Theory and Psychophysics. New York: John Wiley & Sons. (1966). 12. R. Hübner, and, "Cuing mechanisms in auditory signal detection," Perception & Psychophysics, 57, (1995). 13. L. Demany and C. Semal, The role of memory in auditory perception, in, Auditory Perception of Sound Sources, William Yost, Arthur Popper and Richard Fay (eds), Springer (2007). 14., R. Schlauch and J. Tang, Attending to auditory filters that were not stimulated directly, J Acoust Soc Am. 94, (1993). 15. L. Plamondon and, Selective attention in absolute pitch listeners, J. Acoust. Soc. Am. Suppl 1. 88, S49 (1990). 16. and K. Saberi, "A level of stimulus representation model for auditory detection and attention, J. Acoust. Soc. Am., 110, (2001). 17. P. Crum, and Predicting the path of a changing sound: Velocity tracking and auditory continuity J. Acoust. Soc. Am., 124, (2008). 18. A. Bregman, Auditory Scene Analysis: The Perceptual Organization of Sound. Cambridge, Massachusetts: MIT Press, (1990). Proceedings of Meetings on Acoustics, Vol. 19, (2013) Page 5