Binaurally-coherent jitter improves neural and perceptual ITD sensitivity in normal and electric hearing

Transcription

1 Binaurally-coherent jitter improves neural and perceptual ITD sensitivity in normal and electric hearing M. Goupell 1 (matt.goupell@gmail.com), K. Hancock 2 (ken_hancock@meei.harvard.edu), P. Majdak 1 (piotr.majdak@oeaw.ac.at), B. Laback 1 (bernhard.laback@oeaw.ac.at), B. Delgutte 2,3 (bertrand_delgutte@meei.harvard.edu) 1 Acoustics Research Institute, Austrian Academy of Sciences, Wohllebengasse 12-14, A-1040 Vienna, Austria 2 Eaton-Peabody Laboratory, Massachusetts Eye & Ear Infirmary, Boston, MA Research Laboratory of Electronics, Massachusetts Institute of Technology, Cambridge, MA Keywords: ITD, cochlear implants, binaurally-coherent jitter, inferior colliculus, temporal coding, binaural hearing 1 INTRODUCTION Most cochlear implants (CIs) encode information about sounds by amplitude modulating periodic electrical pulse trains whose rate are typically on the order of a 1000 pulses per second (pps). These high rates make it difficult to deliver binaural timing cues with bilateral CIs because the sensitivity of CI listeners to interaural time differences (ITDs) decreases with increasing pulse rate above 100 pps (Majdak et al., 2006; Laback et al., 2007; van Hoesel, 2007). The cause of this insensitivity to ITD may be a form of adaptation that prevents the binaural pathway from responding to high-rate stimulation (Hafter and Dye, 1983; Laback and Majdak, 2008). For example, neurons in the inferior colliculus (IC) of implanted cats best respond for pulse rates below about 100 pps (Snyder et al., 1995), and the decrease in firing rate at higher pulse rates impairs the ability to code ITD in bilaterally-implanted animals (Smith and Delgutte, 2007). The ability of CI users to discriminate ITD at high pulse rates is greatly improved by imposing binaurally-coherent jitter on the interpulse intervals (IPIs) (Laback and Majdak, 2008). The work described here combines psychophysics, neurophysiology, and modeling techniques to 1

2 further characterize the effects of temporal jitter on ITD coding and gain a better understanding of the underlying neural mechanisms. First, we used acoustic stimuli that closely mimic the electrical pulse trains used with CI listeners to show that ITD discrimination in normal-hearing (NH) subjects is also improved by jitter. Second, we studied neural correlates of the effect of jitter on ITD sensitivity by recording from IC neurons in implanted and NH cats. Lastly, we modeled neural responses to pulse trains to better understand the specific mechanism by which jitter improves ITD coding. Our results show clear parallels between the perceptual, neural, and modeling data. 2 PERCEPTUAL EXPERIMENT 2.1 Method Six NH human subjects participated in the experiment. They had normal hearing according to standard audiometric tests. Stimuli were 300-ms pulse trains with trapezoidal amplitude modulation (AM) (Fig. 1A). There were four trapezoids per stimulus, each with a rise-fall time of 20 ms, and a steady-state duration of 20 ms. The inter-trapezoid interval was 20 ms. Monophasic rectangular pulse trains were passed through a digital sixth-order bandpass Butterworth filter centered at 4.6 khz (1.5-kHz bandwidth). The A-weighted sound pressure level of the stimuli was 70 db. Binaurally-uncorrelated, low-pass filtered, white noise was used to mask low-frequency components that might contain useful binaural cues. The noise corner frequency was 3.5 khz, with a 24 db/oct roll-off, and the A-weighted sound pressure level of the noise was 60 db. The nominal IPI corresponded to the average IPI over the stimulus duration. To preserve the ITD information in the pulse timing, the jitter was coherent between the two ears (Fig. 1B). The 2

3 jitter was drawn from a rectangular distribution, where the parameter k defines the width of the distribution relative to the nominal IPI in percent. For each pulse in the train, the IPI was varied within the range of IPI (1 ± k/100). Thus, for k = 100%, the largest possible IPI was twice the nominal IPI and the smallest possible IPI was zero. The independent variables were pulse rate (400, 800, 938, 1182, and 1515 pps), ITD (200, 400, and 600 µs), and k (0, 25, 50, 75, and 90%). Stimulus conditions corresponding to combinations of the independent variables were presented in a balanced design. Each condition was repeated 100 times. Each trial represented a new random jitter manifestation. A two-interval, two-alternative forced-choice procedure was used in a lateralization discrimination test. The first interval contained a reference stimulus with zero ITD and no jitter evoking a centralized auditory image. The second interval contained the target stimulus with non-zero ITD. The interstimulus interval was 400 ms. The listeners indicated whether the second stimulus was perceived to the left or to the right of the first stimulus. Visual response feedback was provided after each trial. 2.2 Results The right side of Figure 1C shows the present experimental results and the left side shows a subset of the results with five CI listeners from Laback and Majdak (2008). For the periodic stimuli, ITD sensitivity decreases with increasing pulse rate. For both CI and NH listeners, jitter improves ITD performance, thereby compensating for much of the performance decrease with increasing pulse rate. The larger jitter (75 and 90%) improved ITD discrimination more than smaller jitter (25 and 50%), with the exception of the NH listeners at 1515 pps. An analysis of variance (ANOVA) (factors: listener type, rate, ITD, and k) on comparable conditions across CI and NH listeners showed a significant difference between the types of 3

4 listeners (p = 0.003). The NH listeners had an average percent correct of 80.7% while the CI listeners had an average percent correct of 76.6%. Therefore, two repeated-measures (RM) ANOVAs (factors: rate, ITD, and k) were performed, one for the CI listeners and one for the NH listeners. Both types of listeners showed mostly the same significant main effects and interactions. There was a significant effect of rate, ITD, and k (p < for all). There was a significant interaction of rate k (CI: p = 0.001; NH: p < ). For just the NH listeners, there was a significant interaction of ITD k (CI: p = 0.67; NH: p = 0.035). The data at 400 pps were examined with separate RM ANOVAs (factors: ITD and k) and subsequent Tukey HSD post-hoc tests since the jitter effect is much less pronounced at this rate. For the CI listeners, the effect of jitter was not significant. For the NH listeners, the performance for all jittered conditions was significantly higher than for periodic conditions at the 0.05 level. 2.3 Discussion The results show that binaurally-coherent jitter can improve ITD sensitivity at high pulse rates. The ITD sensitivity for periodic pulse trains decreased with increasing pulse rate, consistent with the idea of rate limitations. The larger the amount of irregularity in the pulse timing, the larger the improvement that occurred compared to the periodic pulse trains. For NH listeners, this even occurred for 400 pps, the lowest rate tested. In general, the trends were the same between the CI and NH listeners, but the NH listeners had better scores (4.1%) on average. 3 NEUROPHYSIOLOGY 3.1 Method Neural correlates of the effect of jitter on ITD discrimination were investigated by recording responses to pulse trains in the inferior colliculus (IC) of anesthetized cats. We used six deaf cats 4

5 bilaterally implanted with 6-contact intracochlear electrode arrays (Cochlear Corp.) and one NH cat. The recording methods were as described previously (Smith and Delgutte, 2007). In the CI experiments, stimuli were trains of 50-µs biphasic (anodic/cathodic) current pulses presented in wide bipolar configuration and presented 2--6 db above single-pulse threshold. In the NH experiment, 50-µs monophasic rectangular pulses were presented binaurally through closed acoustic systems at db above threshold. In both types of experiments, pulse trains were 300 ms in duration, and repeated once every 600 ms. Pulse rates were typically varied in half-octave steps from 20 to 640 pps in CI experiments and to 2560 pps in the NH experiment 1. Jittered pulse trains were created as described for the perceptual experiments, except that there was no trapezoidal AM, no bandpass filtering, and that the jitter was frozen : the same sequence of pulses was presented on every stimulus trial at a given pulse rate. Responses were obtained using 50% and 90% jitter; for clarity, the results for 50% jitter are omitted. 3.2 Jitter can restore ongoing neural firing at high pulse rates Figure 2A shows firing rate as a function of stimulus pulse rate for diotic stimulation, both with and without jitter, for one IC neuron from an implanted cat. The corresponding temporal discharge patterns are shown as dot rasters in Fig. 2C. Without jitter, firing rate initially increases linearly with pulse rate as the neuron fires one spike per pulse (indicated by regularly spaced firing patterns). For pulse rates above 112 pps the firing rate drops dramatically, and essentially goes to zero by 448 pps. This pattern of response to periodic pulse trains is typical of IC neurons (Snyder et al., 1995; Smith and Delgutte, 2007). When the pulse train is jittered, the response at low pulse rates is similar to that in the periodic condition, but the decline at high pulse rates is smaller and the neuron fires robustly for pulse rates up to 640 pps. Overall, jitter significantly increased the firing rate of about one-third of the 83 IC neurons tested in response to high-rate ( 1 The stimulus artifact with electrical stimulation made it difficult to obtain responses to pulse rates above 640 pps. 5

6 320 pps) pulse trains. Interestingly, the temporal firing patterns evoked by high-rate jittered pulse trains are characterized by sparse preferred times of firing rather than randomly-occurring spikes (Fig. 2C, top). Figure 2B shows a similar effect of jitter on the firing rate of a neuron from the IC of a NH cat (dot rasters not shown). Thus, in both implanted and NH cats, jitter can restore ongoing firing at high pulse rates. 3.3 Restoration of ongoing firing reveals ITD sensitivity A critical question is the extent to which jitter can improve ITD coding at high pulse rates. Figures 3A & 3B show rate-itd curves for the two neurons of Fig. 2, obtained using pulse rates of 640 (A) and 2560 pps (B), respectively. In both cases, periodic pulse trains evoke weak activity that is not obviously influenced by changes in ITD. Jittered pulse trains, in contrast, produce much larger firing rates that are clearly modulated by ITD. Neural ITD sensitivity was quantified using a signal-to-noise ratio (SNR) metric which normalizes the across-itd variance in spike rate by the total spike rate variance, including across-trial variance at each ITD. The SNR ranges between 0 and 1, meaning that ITD accounts for none and all of the total variance, respectively. ITD SNRs are shown to the right of the curves in Figs. 3A & B, and are much larger in response to jittered pulse trains in these two units. Figures 3C & 3D show the mean ITD SNR over the IC population as a function of pulse rate for the CI and NH experiments, respectively. ITD SNR is significantly greater for jittered pulse trains compared to periodic pulse trains (one-tailed paired t-test) for both the CI (p=0.02, n=84 rate-itd curves from 41 neurons) and NH data (p<0.001, n=18 rate-itd curves from 10 neurons). The effect of jitter on ITD sensitivity is clearly larger in the NH case (Fig. 3D), consistent with the psychophysical results, but the NH data are not definitive, because they 6

7 represent data from a single NH cat using a different range of pulse rates than in the implanted cats. For the CI experiments, the effect of jitter on ITD sensitivity depends on pulse rate. It is significant for pulse rates greater than or equal to 320 pps (p=0.02, paired t-test) but not for lower pulse rates (p=0.16). This is consistent with the notion that jitter improves ITD coding primarily through its effect on firing rate: low-rate periodic pulse trains evoke robust responses that jitter can do little to enhance. In summary, the physiological data are consistent with the hypothesis of Laback and Majdak (2008) regarding the mechanism by which jitter improves ITD discrimination. Specifically, jitter restores ongoing neural firing at high pulse rates in about one-third of IC neurons. Moreover, the responses evoked under such conditions are often ITD-sensitive. However, the overall effect on neural ITD sensitivity is relatively modest in CI experiments, and involves a minority of neurons. Detection-theoretic analyses are needed to quantitatively compare the effect of jitter on neural activity with the improvement in psychophysical performance (Hancock and Delgutte, 2004). 4 NEURAL MODELING To explore possible reasons why binaurally-coherent jitter improves ITD sensitivity, we examined the response of a model of the NH human auditory nerve (AN) (Meddis, 2006) to the jittered stimuli used in the perceptual experiments. Figure 4 shows the firing probability of a model AN fiber (4.6-kHz center frequency) for 400, 800, and 1515 pps pulse trains with either k = 0 or 90%. After a strong onset response, the firing probability is fairly flat throughout the periodic pulse train. In contrast, the jittered pulse train shows much greater fluctuations in firing probability after the onset. The instances of increased AN firing during the ongoing portion of the signal resulting from the jitter might be utilized by the binaural system to improve ITD 7

8 sensitivity at high pulse rates. In acoustic stimulation, the pronounced fluctuations with jitter may be partly due to the temporal overlap of the physical pulses at such a high rate and large jitter, which can introduce higher-order modulations. Additional filtering by the cochlea and the hair-cell synapse are also likely to contribute. These fluctuations are consistent with the neural data from the IC, where firings with jittered pulse trains tended to occur at precise sparse times that may correspond to the peaks in the firing probability for model AN fibers. We quantitatively measured the change in synchrony of the neural response when jitter was applied to a pulse train. We used the correlation index (CIn) derived from shuffled correlations (Joris et al., 2006), which provides a general measure of the tendency of a neuron to fire at specific times. Unlike the synchronization coefficient, this metric is applicable to both periodic and aperiodic stimuli. Because binaural processing of ITD in the medial superior olive (MSO) is based on a form of coincidence detection, CIn represents an appropriate measure of the strength of ITD coding. CIn is based on the counting of neural spike coincidences from multiple presentations of the same stimulus. We used a bin width of 10 µs and 100 stimulus presentations for each CIn calculation. CIn has a value of one for random uncorrelated spike trains, and a value greater than one for a correlated response. Figure 1D shows that, as expected, the synchrony (CIn) decreases with increasing pulse rate. Introducing jitter increases the synchronization over that for the periodic condition. The AN firing is more synchronized at the relatively low rate of 400 pps compared to the higher rates. Interestingly, the CIn mostly follow the overall trends in the NH psychophysical data. 5 GENERAL DISCUSSION We found that binaurally-coherent jitter improves ITD perception for high-rate pulse trains in implanted and NH humans, and can also improve ITD sensitivity in the IC of implanted and NH 8

9 cats by restoring ongoing firing at high pulse rates. We further found that jitter produces prominent fluctuations in the responses of a model of the AN to acoustic bandpass-filtered pulse trains, consistent with the sparse, precisely timed firings observed in IC neurons in response to jittered pulse trains. These response fluctuations produced by jitter were observed specifically in a model of normal hearing, and hence partly reflect AM produced by the overlap of filtered pulses in the physical stimulus as well as the overlap of cochlear filter impulse responses. Nevertheless, such fluctuations are likely robust to exact model assumptions because most filters will transform random frequency modulations of pulse timing into AM. Thus, similar AM could be created in the central auditory system of both NH and CI subjects by neural membrane time constants and synaptic transmission. Hafter and Dye (1983) proposed the concept of binaural adaptation to account for the observation that only the first (or first few) pulses in a high-rate periodic pulse train contribute to ITD perception. This concept is consistent with the low ITD sensitivity for high-rate periodic pulse trains in both CI and NH listeners, and with the onset-only response of most ITD-sensitive IC neurons to high-rate periodic pulse trains in NH and CI animals. Two key questions are: At what site(s) does binaural adaptation occur, and how does jitter overcome the effects of binaural adaptation? Although firing rate adaptation has been observed with high-rate pulse trains in AN fibers of both NH (Wickesberg and Stevens, 1998) and CI (Miller et al., 2008) animals, the effect appears too small and not sufficiently specific to account for binaural adaptation, so that the primary site of adaptation must be central (Wickesberg and Stevens, 1998). One possibility is that decreased phase locking at high pulse rates makes synaptic inputs too asynchronous to trigger spikes in the 9

10 coincidence detectors of the binaural pathway. Neurons in the MSO are sensitive to ITD via a binaural coincidence detection mechanism (Yin and Chan, 1990). There is also evidence that bushy cells in the ventral cochlear nucleus (VCN), which provide the excitatory inputs to MSO, act as monaural coincidence detectors (Carney, 1990). Consistent with this idea, the responses of model AN fibers show minimal phase locking to bandpass filtered pulse trains for rates above 400 pps. Moreover, both MSO principal neurons and VCN bushy cells contain low-threshold potassium (K + ) channels thought to play a role in coincidence detection (Manis and Marx, 1991; Smith, 1995). The relatively steady synaptic inputs to these neurons at high pulse rates may produce a sustained activation of the low-threshold K + channels, thereby suppressing subsequent repolarization and blocking firing (Colburn et al., 2008). In this view, jitter improves ITD sensitivity by increasing synchrony in the inputs to the coincidence detectors (Fig. 1D), thereby overcoming the depolarization block and allowing MSO neurons to fire more often. Another possible mechanism underlying binaural adaptation is inhibition at the level of the IC. Smith and Delgutte (2008) showed that a simple model in which IC neurons receive both a brief excitatory input and a delayed long-lasting inhibitory input can qualitatively account for both the onset responses observed in the IC at high pulse rates, and the restoration of sustained firing and ITD sensitivity for amplitude-modulated pulse trains. Presumably, the fluctuations in firing rate introduced by jitter (Fig. 4) could overcome the inhibition in the same way as externally-imposed AM. Interestingly, our perceptual stimuli had a low-rate trapezoidal AM, suggesting they may have induced substantial firing rates in IC neurons, even without jitter. Nevertheless, the improvement in ITD sensitivity suggests that the addition of jitter may have increased IC responses further. 10

11 The ITD discrimination performance is better for the NH subjects than for the CI subjects (who were all post-lingually deafened). This finding seems paradoxical because phase-locking to sinusoids in the AN is better with electric stimulation than with acoustic stimulation (Dynes and Delgutte, 1992), and additional degradation in phase locking is expected with pulse trains in the NH case due to basilar membrane filtering. Additionally, if the change in the precision of the neural timing is the important signal modification induced by binaurally-coherent jitter, again we would expect better CI results compared to NH results. Perhaps additional factors, like decreased neuronal survival cause the overall worse performance of the implanted subjects. Alternatively, deprivation of sound inputs prior to cochlear implantation may have altered the balance between excitation and inhibition, consistent with results from deafened animals (Vale and Sanes, 2002). In summary, we have shown that introducing binaurally-coherent jitter to high rate pulse trains improves both psychophysical and neural ITD sensitivity in CI and NH subjects. Although these improvements may be produced by multiple neural mechanisms operating at different processing stages, these mechanisms are likely triggered by the increased synchrony of neural firings produced by jitter in the AN. Regardless of the mechanism, the beneficial effect of jitter offers hope for more effective coding of binaural timing cues in bilateral cochlear implants. ACKNOWLEDGEMENTS Supported by NIH Grants RO1 DC and P30 DC005209, and FWF Project P18401-B15. KEH and BD thank Dr. David Ryugo for providing congenitally deaf cats from his colony. REFERENCES Carney LH (1990) Sensitivities of cells in anteroventral cochlear nucleus of cat to spatiotemporal discharge patterns across primary afferents. J Neurophysiol 64: Colburn HS, Chung Y, Zhou Y, Brughera A (2008) Models of brainstem responses to bilateral electrical stimulation. J Assoc Res Otolaryngol. Dynes SB, Delgutte B (1992) Phase-locking of auditory-nerve discharges to sinusoidal electric stimulation of the cochlea. Hear Res 58:

12 Hafter ER, Dye RH, Jr. (1983) Detection of interaural differences of time in trains of highfrequency clicks as a function of interclick interval and number. J Acoust Soc Am 73: Hancock KE, Delgutte B (2004) A physiologically based model of interaural time difference discrimination. J Neurosci 24: Joris PX, Louage DH, Cardoen L, van der Heijden M (2006) Correlation index: a new metric to quantify temporal coding. Hear Res : Laback B, Majdak P (2008) Binaural jitter improves interaural time-difference sensitivity of cochlear implantees at high pulse rates. Proc Natl Acad Sci USA 105: Laback B, Majdak P, Baumgartner WD (2007) Lateralization discrimination of interaural time delays in four-pulse sequences in electric and acoustic hearing. J Acoust Soc Am 121: Majdak P, Laback B, Baumgartner WD (2006) Effects of interaural time differences in fine structure and envelope on lateral discrimination in electric hearing. J Acoust Soc Am 120: Manis PB, Marx SO (1991) Outward currents in isolated ventral cochlear nucleus neurons. J Neurosci 11: Meddis R (2006) Auditory-nerve first-spike latency and auditory absolute threshold: a computer model. J Acoust Soc Am 119: Miller CA, Hu N, Zhang F, Robinson BK, Abbas PJ (2008) Changes across time in the temporal responses of auditory nerve fibers stimulated by electric pulse trains. J Assoc Res Otolaryngol 9: Smith PH (1995) Structural and functional differences distinguish principal from nonprincipal cells in the guinea pig MSO slice. J Neurophysiol 73: Smith ZM, Delgutte B (2007) Sensitivity to interaural time differences in the inferior colliculus with bilateral cochlear implants. J Neurosci 27: Smith ZM, Delgutte B (2008) Sensitivity of inferior colliculus neurons to interaural time differences in the envelope versus the fine structure with bilateral cochlear implants. J Neurophysiol 99: Snyder R, Leake P, Rebscher S, Beitel R (1995) Temporal resolution of neurons in cat inferior colliculus to intracochlear electrical stimulation: effects of neonatal deafening and chronic stimulation. J Neurophysiol 73: Vale C, Sanes DH (2002) The effect of bilateral deafness on excitatory and inhibitory synaptic strength in the inferior colliculus. Eur J Neurosci 16: van Hoesel RJ (2007) Sensitivity to binaural timing in bilateral cochlear implant users. J Acoust Soc Am 121: Wickesberg RE, Stevens HE (1998) Responses of auditory nerve fibers to trains of clicks. J Acoust Soc Am 103: Yin TC, Chan JC (1990) Interaural time sensitivity in medial superior olive of cat. J Neurophysiol 64:

13 A ms Interpulse-interval (IPI) B Left Periodic pulse train Right 80 ms Jittered pulse train Left Right C D Figure 1. Perceptual experiment and neural modeling. A: The envelope of the experimental stimuli. For clarity, only three of four trapezoids are shown and the pulses are shown in one trapezoid only. B: The stationary portion of a periodic pulse train (upper) and of a jittered pulse train (lower). Note that the binaurally-coherent jitter preserves the interaural time difference (marked with arrows). C: Percent correct scores for left/right discrimination as a function of the pulse rate. The data are averaged over all subjects, interaural time difference, and k (when needed). The error bars represent 95% confidence intervals. D: Correlation index (CIn) results. The data points show the mean ± standard deviation of the CIn measurements over five tokens. 13

14 Figure 2. Jitter increases firing rate of IC neurons in response to high-rate pulse trains. A: Response of one neuron in a bilaterally-implanted cat. B: Normal-hearing cat. C: Dot rasters illustrate the temporal response patterns corresponding to panel A. Each dot indicates a single spike, and alternating colors distinguish blocks of responses to different pulse rates. The neuron tends to entrain for pulse rates less than about 160 pps, with (top) or without jitter (bottom). In the jittered case, for pulse rates greater than about 320 pps, the neuron does not entrain, but exhibits sparse preferred times of firing (top). 14

15 Figure 3. Jitter restores neural ITD sensitivity in response to high-rate pulse trains. A, B: rate- ITD curves for the same neurons as in Fig. 2. Numbers to the right of each curve indicate ITD signal-to-noise ratio (SNR, see text). A: Bilaterally-implanted cat. B: Normal-hearing cat. C, D: Corresponding population data. ITD SNR as a function of pulse rate (mean ± standard error, data sets per pulse rate). 15

16 Figure 4. Firing probability of an auditory nerve fiber in response to 400-, 800-, and 1515-pps trapezoidally-modulated pulse trains, with either k = 0 or 90%. The bin width was 1 ms. For clarity, two trapezoids are shown for each pulse rate. 16