Springer Handbook of Auditory Research

Transcription

1

2 Springer Handbook of Auditory Research Series Editors: Richard R. Fay and Arthur N. Popper For other titles published in this series, go to

3 Ray Meddis Enrique A. Lopez-Poveda Richard R. Fay Arthur N. Popper Editors Computational Models of the Auditory System

4 Editors Ray Meddis University of Essex Colchester CO4 3SQ UK Richard R. Fay Loyola University of Chicago Chicago IL USA Enrique A. Lopez-Poveda Neuroscience Institute of Castilla y León University of Salamanca Salamanca, Spain ealopezpoveda@usal.es Arthur N. Popper University of Maryland College Park, MD USA apopper@umd.edu ISBN e-isbn DOI / Springer New York Dordrecht Heidelberg London Library of Congress Control Number: Springer Science+Business Media, LLC 2010 All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. Printed on acid-free paper Springer is part of Springer Science+Business Media (

5 Series Preface The Springer Handbook of Auditory Research presents a series of comprehensive and synthetic reviews of the fundamental topics in modern auditory research. The volumes are aimed at all individuals with interests in hearing research including advanced graduate students, post-doctoral researchers, and clinical investigators. The volumes are intended to introduce new investigators to important aspects of hearing science and to help established investigators to better understand the fundamental theories and data in fields of hearing that they may not normally follow closely. Each volume presents a particular topic comprehensively, and each serves as a synthetic overview and guide to the literature. As such, the chapters present neither exhaustive data reviews nor original research that has not yet appeared in peerreviewed journals. The volumes focus on topics that have developed a solid data and conceptual foundation rather than on those for which a literature is only beginning to develop. New research areas will be covered on a timely basis in the series as they begin to mature. Each volume in the series consists of a few substantial chapters on a particular topic. In some cases, the topics will be ones of traditional interest for which there is a substantial body of data and theory, such as auditory neuroanatomy (Vol. 1) and neurophysiology (Vol. 2). Other volumes in the series deal with topics that have begun to mature more recently, such as development, plasticity, and computational models of neural processing. In many cases, the series editors are joined by a co-editor having special expertise in the topic of the volume. Richard R. Fay, Chicago, IL Arthur N. Popper, College Park, MD v

6 Volume Preface Models have always been a special feature of hearing research. The particular models described in this book are special because they seek to bridge the gap between physiology and psychophysics and ask how the psychology of hearing can be understood in terms of what we already know about the anatomy and physiology of the auditory system. However, although we now have a great deal of detailed information about the outer, middle, and inner ear as well as an abundance of new facts concerning individual components of the auditory brainstem and cortex, models of individual anatomically defined components cannot, in themselves, explain hearing. Instead, it is necessary to model the system as a whole if we are to understand how man and animals extract useful information from the auditory environment. A general theory of hearing that integrates all relevant physiological and psychophysical knowledge is not yet available but it is the goal to which all of the authors of this volume are contributing. The volume starts with the auditory periphery by Meddis and Lopez-Poveda (Chapter 2) which is fundamental to the whole modeling exercise. The next level in the auditory system is the cochlear nucleus. In Chapter 3, Voigt and Zheng attempt to simulate accurately the responses of individual cell types and show how the connectivity among the different cell types determines the auditory processing that occurs in each subdivision. Output from the cochlear nucleus has two main targets, the superior olivary complex and the inferior colliculus. The superior olivary complex is considered first in Chapter 4 by Jennings and Colburn because its output also passes through the inferior colliculus, which is discussed in Chapter 6 by Davis, Hancock, and Delgutte, who draws explicit links between the modeling work and psychophysics. Much less is known about the thalamus and cortex, and Chapter 5 by Eggermont sets out what has been achieved so far in understanding these brain regions and what the possibilities are for the future. Four more chapters conclude this volume by looking at the potential of modeling to contribute to the solution of practical problems. Chapter 7 by Heinz addresses the issue of how hearing impairment can be understood in modeling terms. In Chapter 8, Brown considers hearing in connection with automatic speech recognition and reviews the problem from a biological perspective, including recent progress that has been made. In Chapter 9, Wilson, Lopez-Poveda, and Schatzer look more vii

7 viii Volume Preface closely at cochlear implants and consider whether models can help to improve the coding strategies that are used. Finally, in Chapter 10, van Schaik, Hamilton, and Jin address these issues and show how models can be incorporated into very large scale integrated devices known more popularly as silicon chips. As is the case with volumes in the Springer Handbook of Auditory Research, previous volumes have chapters relevant to the material in newer volumes. This is clearly the case in this volume. Most notably, the advances in the field can be easily seen when comparing the wealth of new and updated information since the publication of Vol. 6, Auditory Computation. As pointed out in this Preface, and throughout this volume, the models discussed rest upon a thorough understanding of the anatomy and physiology of the auditory periphery and the central nervous system. Auditory anatomy was the topic of first volume in the series (The Mammalian Auditory Pathway: Neuroanatomy) and physiology in the second (The Mammalian Auditory Pathway: Physiology). These topics were brought up to date and integrated in the more recent Vol. 15 (Integrative Functions in the Mammalian Auditory Pathway). There are also chapters in several other volumes that are germane to the topic in this one, including chapters in Cochlear Implants (Vol. 20), The Cochlea (Vol. 8), and Vertebrate Hair Cells (Vol. 27). Ray Meddis, Colchester, UK Enrique A. Lopez-Poveda, Salamanca, Spain Richard R. Fay, Chicago, IL Arthur N. Popper, College Park, MD

8 Contents 1 Overview... 1 Ray Meddis and Enrique A. Lopez-Poveda 2 Auditory Periphery: From Pinna to Auditory Nerve... 7 Ray Meddis and Enrique A. Lopez-Poveda 3 The Cochlear Nucleus: The New Frontier Herbert F. Voigt and Xiaohan Zheng 4 Models of the Superior Olivary Complex T.R. Jennings and H.S. Colburn 5 The Auditory Cortex: The Final Frontier Jos J. Eggermont 6 Computational Models of Inferior Colliculus Neurons Kevin A. Davis, Kenneth E. Hancock, and Bertrand Delgutte 7 Computational Modeling of Sensorineural Hearing Loss Michael G. Heinz 8 Physiological Models of Auditory Scene Analysis Guy J. Brown 9 Use of Auditory Models in Developing Coding Strategies for Cochlear Implants Blake S. Wilson, Enrique A. Lopez-Poveda, and Reinhold Schatzer 10 Silicon Models of the Auditory Pathway André van Schaik, Tara Julia Hamilton, and Craig Jin Index ix

9 Contributors Guy J. Brown Speech and Hearing Research Group, Department of Computer Science, University of Sheffield, Sheffield S1 4DP, UK, H. Steven Colburn Department of Biomedical Engineering, Boston University, Boston, MA 02215, USA, Kevin A. Davis Departments of Biomedical Engineering and Neurobiology and Anatomy, University of Rochester, Rochester, NY 14642, USA, Bertrand Delgutte Eaton-Peabody Laboratory, Massachusetts Eye and Ear Infirmary, Boston, MA 02114, USA, Jos J. Eggermont Department of Psychology, University of Calgary, Calgary, AB, Canada T2N 1N4, Tara Julia Hamilton School of Electrical Engineering and Telecommunications, The University of New South Wales, NSW 2052, Sydney, Australia, Kenneth E. Hancock Eaton-Peabody Laboratory, Massachusetts Eye and Ear Infirmary, Boston, MA 02114, USA, Michael G. Heinz Department of Speech, Language, and Hearing Sciences & Weldon School of Biomedical Engineering, Purdue University, West Lafayette, IN 47907, USA, xi

10 xii Contributors Todd R. Jennings Department of Biomedical Engineering, Boston University, Boston, MA 02215, USA, Craig Jin School of Electrical and Information Engineering, The University of Sydney, Sydney, NSW 2006, Australia, Enrique A. Lopez-Poveda Instituto de Neurociencias de Castilla y León, University of Salamanca, Salamanca, Spain, ealopezpoveda@usal.es Ray Meddis Hearing Research Laboratory, Department of Psychology, University of Essex, Colchester CO4 3SQ, UK, rmeddis@essex.ac.uk Reinhold Schatzer C. Doppler Laboratory for Active Implantable Systems, Institute of Ion Physics and Applied Physics, University of Innsbruck, 6020 Innsbruck, Austria, reinhold.schatzer@uibk.ac.at André van Schaik School of Electrical and Information Engineering, The University of Sydney, Sydney, NSW 2006, Australia, andre@ee.usyd.edu.au Herbert F. Voigt Department of Biomedical Engineering, Boston University, Boston, MA 02215, USA, hfv@enga.bu.edu Blake S. Wilson Duke Hearing Center, Duke University Medical Center, Durham, NC 27710, USA; Division of Otolaryngology, Head and Neck Surgery, Department of Surgery, Duke University Medical Center, Durham, NC 27710, USA; MED-EL Medical Electronics GmbH, 6020 Innsbruck, Austria, blake.wilson@duke.edu Xiaohan Zheng Biomedical Engineering Department, Boston University, Boston, MA 02215, USA, xhzheng@bu.edu

11 Chapter 1 Overview Ray Meddis and Enrique A. Lopez-Poveda Models have always been a special feature of hearing research. Von Helmholtz (1954) likened the ear to a piano, an array of resonances each tuned to a different frequency. In modern psychophysics, the dominant models are often drawn from radio or radar technology and feature filters, amplifiers, oscillators, detectors, integrators, etc. In physiology, there have been many models of the individual components along the auditory pathway such as the Davis (1965) battery theory of cochlear transduction and Hodgkin and Huxley (1952) models of the initiation of spike activity in nerve fibers. These models are attractive to researchers because they are explicit and quantitative. The particular models described in this book are special because they seek to bridge the gap between physiology and psychophysics. They ask how the psychology of hearing can be understood in terms of what we already know about the anatomy and physiology of the auditory system. Rapid recent progress in anatomy and physiology means that we now have a great deal of detailed information about the outer, middle, and inner ear as well as an abundance of new facts concerning individual components of the auditory brain stem and cortex. However, models of individual anatomically defined components cannot, in themselves, explain hearing. Instead, it is necessary to model the system as a whole if we are to understand how humans and animals extract useful information from the auditory environment. Although a general theory of hearing that integrates all relevant physiological and psychophysical knowledge is not yet available, it is the goal to which all of the authors of this volume are contributing. Despite the considerable complexity implied by a general theory of hearing, this goal looks to be achievable now that computers are available. Computers provide the ability to represent complexity by adopting a systems approach wherein models of individual components are combined R. Meddis (*) Hearing Research Laboratory, Department of Psychology, University of Essex, Colchester CO4 3SQ, UK rmeddis@essex.ac.uk R. Meddis et al. (eds.), Computational Models of the Auditory System, Springer Handbook of Auditory Research 35, DOI / _1, Springer Science+Business Media, LLC

12 2 R. Meddis and E.A. Lopez-Poveda to form larger collections of interacting elements. Each element of the model can be developed independently but with the prospect of integrating it into the larger whole at a later stage. Computational models are attractive because they are explicit, public, quantitative, and they work. They are explicit in that a computer program is a rigorous, internally consistent and unambiguous definition of a theory. They are public in that computer programs are portable and can be studied in anyone s laboratory so long as a generalpurpose computer is available. Anyone can check whether the model really does what the author says that it does by obtaining and running the program. A good modeler can, and should, make the computer code available for public scrutiny. Computer models are quantitative in that all parameters of the model must be specified before the program can be run. They also work in the sense that a good computational model should be a practical tool for those designing better hearing aids, cochlear implant coding strategies, automatic speech recognizers, and robotic vehicles. A computer model is a flexible working hypothesis shared among researchers. It is a research tool and will always be a work in progress. At this level of complexity, no modeler can hope to own the final model that is correct in every detail but he or she can hope to make a contribution to one or more components of the model. Perhaps he or she can design a better model of the basilar membrane response, or a primary-like unit in the cochlear nucleus. Even better, he or she might show how some puzzle in psychophysics is explained by how a particular structure processes acoustic information. In other words, auditory models can provide a shared framework within which research becomes a cooperative enterprise where individual contributions are more obviously seen to fit together to form a whole that is indeed greater than its parts. These general considerations have shaped the design of this book and produced four major requirements for individual authors to consider. First, a general model of hearing must be based on the anatomy and physiology of the auditory system from the pinna up to the cortex. Second, each location along the auditory processing pathway presents its own unique problems demanding its own solution. Each subcomponent is a model in itself and the design of connections within and between these components are all substantial challenges. Third, the models need to be made relevant, as far as possible, to the psychology of hearing and provide explanations for psychophysical phenomena. Finally, the practical requirements of clinicians and engineers must be acknowledged. They will use the models to conceptualize auditory processing and adapt them to solve practical design problems. Each chapter addresses these issues differently depending on the amount of relevant progress in different areas, but it is hoped that all of these issues have been addressed in the volume as a whole. The auditory periphery by Meddis and Lopez-Poveda (Chapter 2) is fundamental to the whole modeling exercise. It is the gateway to the system, and model mistakes at this level will propagate throughout. This stage has often been characterized as a bank of linear filters followed by half-wave rectification but, in reality, it is much more complex. Processing in the auditory periphery is nonlinear with respect to

13 1 Overview 3 level at almost every stage, including the basilar membrane response, the receptor potential, and the generation of action potential in auditory nerve (AN) fibers. Adaptation of firing rates during exposure to sound and the slow recovery from adaptation mean that the system is also nonlinear with respect to time. For example, the effect of an acoustic event depends on what other acoustic events have occurred in the recent past. Fortunately, the auditory periphery has received a great deal of attention from physiologists despite being the least accessible part of the auditory system because it is buried inside the temporal bone. As a result, there is a great deal of information concerning the properties of the individual peripheral components, including the stapes, outer hair cells, basilar membrane, inner hair cells, and the auditory nerve itself. Moreover, one element is very much like another along the cochlear partition. As a consequence, it is relatively easy to check whether models of the individual component processing stages are working as required. In humans, there are about 30,000 afferent AN fibers, each responding with up to 300 action potentials per second. The cochlear nucleus (CN, Chapter 3) receives all of the AN output and has the function of processing this information before passing it on to other nuclei for further processing. The anatomy of the cochlear nucleus is surprisingly complex, with a number of subdivisions each receiving its own copy of the AN input. Detailed analysis shows that each subdivision contains different types of nerve cells each with its own electrical properties. Some are inhibitory and some excitatory, and all respond in a unique way to acoustic stimulation. Patterns of interconnections between the cells are also complex. It is the job of the modeler to simulate accurately the responses of individual cell types and show how the connectivity among the different cell types determines the auditory processing that occurs in each subdivision. Models of the CN could occupy a whole volume by itself. Here we can only give a flavor of what has been achieved and what lies ahead. Output from the cochlear nucleus has two main targets, the superior olivary complex (SOC) and the inferior colliculus (IC). The SOC is considered first (Chapter 4) because its output also passes to the IC. Like the CN it is also complex and adds to the impression that a great deal of auditory processing is carried out at a very early stage in the passage of signals toward the cortex. As in the CN, there are different cell types and suggestive interconnections between them. However, we also see the beginning of a story that links the physiological with the psychophysical. The SOC has long been associated with the localization of sounds in the popular Jeffress (1948) model that uses interaural time differences (ITDs) to identify where a sound is coming from. The reader will find in Chapter 4 that the modern story is more subtle and differentiated than the Jeffress model would suggest. Recent modeling efforts are an excellent example of how the computational approach can deal simultaneously with the details of different cell types, their inhibitory or excitatory nature, and how they are interconnected. In so doing, they lay the foundation for an understanding of how sounds are localized.

14 4 R. Meddis and E.A. Lopez-Poveda All outputs from both the CN and the SOC find their way to the central nucleus of the IC, which is an obligatory relay station en route to the cortex. In comparison with the CN or the SOC, it is much less complex, with fewer cell types and a more homogeneous anatomical structure. The authors of Chapter 6 (Davis, Hancock, and Delgutte) use Aitkin s (1986) characterization of the IC as a shunting yard of acoustical information processing but then go on to show that its significance is much greater than this. For the first time, in this volume, explicit links are drawn between the modeling work and psychophysics. Localization of sounds, the precedence effect, sensitivity to amplitude modulation, and the extraction of pitch of harmonic tones are all dealt with here. The full potential of computer models to explain auditory processing is most evident in this chapter. The next stage is the thalamus, where information from the IC is collected and passed to the cortex. Unfortunately, relatively little is known about what this stage contributes to auditory processing. It does, however, have strong reciprocal links with the cortex, with information passing back and forth between them, and it may be best to view the thalamus and cortex as a joint system. Undoubtedly, the most sophisticated analyses of acoustic input occur in this region, and Chapter 5 (Eggermont) sets out what has been achieved so far and what the possibilities are for the future. Pitch, speech, language, music, and animal vocalization are all analyzed here and are affected when the cortex is damaged. Theories are beginning to emerge as to how this processing is structured and detailed computational models such as the spectrotemporal receptive fields are already being tested and subjected to critical analysis. Nevertheless, considerable effort will be required before it is possible to have detailed working models of the cortical processing of speech and music. Four more chapters conclude this volume by looking at the potential of modeling to contribute to the solution of practical problems. Chapter 7 by Heinz addresses the issue of how hearing impairment can be understood in modeling terms. Aging, genetic heritage, noise damage, accidents, and pharmaceuticals all affect hearing, but the underlying mechanisms remain unclear. Many of these questions need to be addressed by empirical studies but modeling has a role to play in understanding why damage to a particular part of the system has the particular effect that it does. Hearing loss is not simply a case of the world becoming a quieter place. Patients complain variously that it can be too noisy, that their problems occur only when two or more people are speaking simultaneously, that their hearing is distorted, or that they hear noises (tinnitus) that bear no relationship to events in the real world. Hearing loss is complex. Modeling has the potential to help make sense of the relationship between the underlying pathology and the psychological experience. It should also contribute to the design of better hearing prostheses. Computer scientists have a long-standing interest in hearing in connection with automatic speech recognition (ASR). Considerable progress has been made using the techniques of spectral and temporal analysis of speech signals in an engineering tradition. However, there has always been a minority interest in building models that mimic human hearing. This interest has become more pressing as the limitations of the engineering approach have become evident. One of these

15 1 Overview 5 limitations concerns how to separate speech from a noisy background before identification. However, this is only one aspect of the general problem of how to segregate sounds from different sources, a problem more generally known as auditory scene analysis. In Chapter 8, Brown reviews the problem from a biological perspective and reviews recent progress. This is the highest level of auditory modeling and the chapter addresses the very high-level issue of the focus of attention. These are all issues of interest to psychologists, computer scientists, and philosophers alike. In Chapter 9, Wilson, Lopez-Poveda, and Schatzer look more closely at cochlear implants and consider whether models can help to improve the coding strategies that they use. It is remarkable just how much progress has been made in the design and fitting of these devices and the enormous benefit that many patients have received. Nevertheless, the benefits vary considerably from patient to patient, and some types of acoustic stimulation benefit more than others. For example, implants work better with speech than with music. It is natural to want to push this technology to its limits, and one way forward is to explore the possibility of simulating natural hearing as closely as possible and incorporating these natural models into new coding strategies. Work has already begun but there is much more to be done. For some, the greatest justification of auditory modeling will come from the useful artefacts that will ultimately result from the modeling efforts. These are hearing devices that can be embedded in many applications in everyday life. Such devices will need to operate in real time, consume little power, and be inexpensive to manufacture. The final chapter in this book, by van Schaik, addresses these issues and shows how models can be incorporated into VLSI (very large scale integrated) devices known more popularly as silicon chips. It is in the nature of these efforts that they will need to wait until individual models have been produced and tested. Even then the technical challenges are formidable. Nevertheless, considerable progress has already been made and working devices have been designed and built. It is likely that they will be the medium by which auditory modeling has its greatest impact on the welfare of the general public. Taken together, these chapters reveal a mountain of achievement and show a field of intellectual endeavour on the verge of maturity. We do not yet have a complete working model of the auditory system and it is true that most modeling research projects are concentrated on small islands along the pathway between the periphery and the cortex. Nevertheless, it is increasingly clear that computer models will one day link up these islands to form a major theoretical causeway directing our understanding of how the auditory system does what it does for those fortunate enough to have normal hearing. Where hearing is imperfect as a result of genetics, damage, or simply aging, computer models of hearing offer the fascinating possibility of new explanations and new prostheses. While science atomizes hearing by focusing on ever smaller details, computer models have the power to resynthesize the hardwon findings of anatomists, physiologists, psychophysicists, and clinicians into a coherent and useful structure.