Improving Music Percep1on With Cochlear Implants David M. Landsberger

Transcription

1 Improving Music Percep1on With Cochlear Implants David M. Landsberger

2 Music Enjoyment With a Cochlear Implant is Low Tested music enjoyment with people with one Normal Hearing Ear and one ear with a cochlear implant Gave them a perceptual scale based on the normal ear so it could be compared across parbcipants and normal hearing listeners.

3 But combining the two is even Playing music to both the normal ear and electric ear sounds beder than the normal ear alone!

4 Even with an acous1c hearing loss, combining the two makes music more enjoyable We modified the acousbc signal so only low frequencies were presented (simulabng a typical hearing loss). Adding electric to acousbc makes music sound beder regardless of simulated hearing loss. (That is, green bars are taller than red bars)

5 One way to improve music with an implant is to combine it with some residual acous1c hearing

6 Musical Intervals are not provided accurately with a CI Cochlear Implant users hear higher notes as higher in pitch (i.e. they hear the ORDER of pitches) However, ORDER of pitch is not sufficient for music. CI users have great trouble with hearing the correct musical intervals. More in tune More in tune RaBng RaBng 220 Hz 440 Hz 880 Hz Semitone DistorBon NH Results NH listeners rated the song to be more out of tune when greater semitone compression (DistorBon < 1) or expansion (DistorBon > 1) was applied to the song. CI Results CI users showed minimal to no sensibvity to semitone distorbon through their clinical processors. Semitone DistorBon Landsberger et al. (in prep)

7 Why is music quality with an implant so poor? Cochlear Implants were designed for speech and not for music. Music requires much more precision, and is much more sensibve to distorbons. With a CI, a higher note is perceived as higher pitch (Ordinal Pitch) But with a CI, musical intervals are not maintained. e.g., doubling frequency (going from 440 Hz to 880 Hz) would be perceived as higher but not as an octave shi^. Results in harmonic structures, melodies, and chords to be distorted.

8 Why is pitch distorted? We do not truly understand how pitch is encoded in the normal auditory system and therefore do not know how to recreate it properly with an implant. However, all of the potenbal variables need to provide pitch are distorted with a cochlear implant.

9 Pitch is effected by where the electrode is (Place Pitch) Low Moderately Low Moderately High High

10 Pitch is effected by how fast the electrode s1mulates (Rate Pitch) Slow Low Pitch Fast High Pitch

11 Both 1ming and electrode choice effect pitch How High Is This Sound? Brighter colors indicate higher pitch Rate (pps) Horizontal axis represents where in the cochlea apical (lower pitch) electrodes on the le^ and basal (higher pitch) electrodes on the right. VerBcal axis represents how fast higher numbers indicate faster sbmulabon Electrode n=10 Both Bming and place cues effect pitch From Landsberger et al. (2016)

12 Both 1ming and electrode choice effect pitch Rate (pps) How High Is This Sound? Pitch presumably depends on a combinabon of Bming and place (electrode) cues. But both are distorted with implants Electrode n=10 We need to learn how to correct Bming and place cues as well as how to re-couple the two in order to improve pitch with implants. From Landsberger et al. (2016)

13 Place Pitch Music Demo

14 Place Pitch Cochlear implants distort the auditory input in two important ways: electrodes provide too coarse informabon Frequency Mismatch between what electrodes deliver and what biology expects. Red numbers: analysis filters of the cochlear implant s speech processor. Blue numbers: characterisbc frequency of the sbmulated neurons (i.e., what the auditory system expects)

15 Place Pitch To help this mismatch, we could: Use a longer electrode This would only be a parbal fix and could cause more damage Change the frequencies provided by the implant (red numbers) to match frequencies expected (blue numbers) This would eliminate lower frequencies that are important for music Would greatly reduce speech understanding

16 Rate Pitch Increasing rate increases pitch up to about 300 Hz. Supports about a 2 octave range (75 Hz 300 Hz) A change in rate (up to ~300 Hz) produces a change in pitch. (Figure from Clark, 1998) Supports frequencies up to approximately Middle C (261 Hz)

17 Problems with Rate Pitch Alone Data suggests rate alone may not be sufficient for musical intervals. A change in rate (up to ~300 Hz) produces a change in pitch. (Figure from Clark, 1998) Using only one electrode (using rate coding only) is not sufficient for good performance We stopped using single channel implants in the early 80s!

18 Solu1ons? Presumably we need to coding strategies that properly combine Bming and place informabon to provide pitch However, we need more research to determine what properly combining Bming and place informabon is! In the meanbme, adding some acousbc hearing seems to help with music percepbon.

19 Thank You! NYU: Monica Padilla Natalia Stupak Ann Todd Roozbeh Soleymani ear-lab.org Funding: 19