Cortical Encoding of Auditory Objects in the Cocktail Party Problem. Jonathan Z. Simon University of Maryland

Size: px
Start display at page:

Download "Cortical Encoding of Auditory Objects in the Cocktail Party Problem. Jonathan Z. Simon University of Maryland"

Transcription

1 Cortical Encoding of Auditory Objects in the Cocktail Party Problem Jonathan Z. Simon University of Maryland

2 Introduction Auditory Objects Magnetoencephalography (MEG) Decoding Neural Signals/Encoding Stimuli Cortical Representations of Auditory Objects I Cortical Representations of Auditory Objects II

3 Auditory Objects What is an Auditory Object? Perceptual/Psychophysical Construct Some Commonalities with Visual Objects I know it when I see it vs. Formal Definition

4 Auditory Object Definition E. g., Griffiths & Warren an object corresponds with something in the sensory world information related to the object is separate from information related to the rest of the sensory world abstracted so that object information can be generalized among particular sensory experiences in any one sensory domain

5 Auditory Objects at the Cocktail Party Alex Katz, The Cocktail Party

6 Auditory Objects at the Cocktail Party Alex Katz, The Cocktail Party

7 Auditory Objects at the Cocktail Party Alex Katz, The Cocktail Party

8 Auditory Objects at the Cocktail Party Alex Katz, The Cocktail Party

9 Auditory Objects at the Cocktail Party Alex Katz, The Cocktail Party

10 Auditory Objects at the Cocktail Party Alex Katz, The Cocktail Party

11 Auditory Objects at the Cocktail Party Alex Katz, The Cocktail Party

12 Auditory Objects at the Cocktail Party Alex Katz, The Cocktail Party

13 Auditory Objects at the Cocktail Party Alex Katz, The Cocktail Party

14 Auditory Objects at the Cocktail Party Ding & Simon, PNAS (2012)

15

16

17

18

19

20 Magnetoencephalography (MEG) scalp skull B MEG V EEG recording surface orientation of magnetic field Magnetic Dipolar Field Projection CSF tissue current flow Photo by Fritz Goro Direct electrophysiological measurement not hemodynamic real-time No unique solution for distributed source Measures spatially synchronized cortical activity Fine temporal resolution (~ 1 ms) Moderate spatial resolution (~ 1 cm)

21 MEG Phase Locking to Slow Temporal Modulations AM at 3 Hz 3 Hz phase-locked response MEG activity is precisely phase-locked to temporal modulations of sound response spectrum (subject R0747) 3 Hz 6 Hz 0 10 Ding & Simon, J Neurophysiol (2009) Wang et al., J Neurophysiol (2012) Frequency (Hz)

22 Modeling MEG Response to Speech Modulations Auditory Model

23 Modeling MEG Response using STRF model Ding & Simon, J Neurophysiol (2012) Spectro-Temporal Response Function (STRF) (up to ~ 10 Hz)

24 Neural Reconstruction of Speech Envelope MEG Responses Decoder Speech Envelope. (up to ~ 10 Hz) stimulus speech envelope reconstructed stimulus speech envelope 2 s Ding & Simon, J Neurophysiol (2012) Reconstruction accuracy comparable to single unit recordings & ECoG

25 Speech Stream as an Auditory Object a speech stream corresponds with something in the sensory world information related to a speech stream is separate from information related to the rest of the sensory world, e.g. other speech streams or noise a speech stream is abstracted: it is generalized among different sensory experiences, e.g. different sound mixtures

26 Neural Representation of an Auditory Object neural representation is of auditory object, something in sensory world when auditory object is with other sounds, the neural representation is of the auditory object, not the entire acoustic scene neural representation remains invariant under broad changes in acoustic representation of auditory object

27 Selective Neural Encoding

28 Unselective vs. Selective Neural Encoding

29 Auditory Object- Specific Representation Representative subject Grand average attentding to speaker 2 attending to speaker 1 stimulus reconstructed from MEG envelope of the attended speech

30 Auditory Object- Specific Representation Representative subject Grand average attentding to speaker 2 attending to speaker 1 stimulus reconstructed from MEG envelope of the attended speech

31 Single Trial Speech Reconstruction Attended Speech Reconstruction Speaker Two attentional focus speaker one speaker two Speaker One

32 Single Trial Speech Reconstruction Speaker Two Attended Speech Reconstruction attentional focus speaker one speaker two Speaker Two Background Speech Reconstruction Speaker One Speaker One

33 Overall Speech Reconstruction 0.2 correlation attended speech reconstruction attended speech background reconstruction background

34 Overall Speech Reconstruction correlation Distinct neural representations for different speech streams 0 attended speech reconstruction attended speech background reconstruction background

35 Reconstruction of Same-Sex Speech Correlation Attended Speech Reconstruction Speaker Two Single Trial Decoding Results Attended Speaker One Two attended speech background speech Speaker One

36 Invariance Under Acoustic Changes

37 Invariance Under Acoustic Changes

38 Invariance Under Acoustic Changes

39 Invariance Under Acoustic Changes

40 Invariance Under Acoustic Changes

41 Invariance Under Acoustic Changes

42 Object-Based Gain Gain-Control Models Control? Object-Based correlation.2.1 attended background Speaker Relative Intensity (db) Globally Based correlation.2.1 attended background Speaker Relative Intensity (db)

43 Object-Based Gain Gain-Control Models Control? Object-Based correlation.2.1 attended background Neural Results Speaker Relative Intensity (db) correlation.2.1 attended background Globally Based correlation.2.1 attended background Speaker Relative Intensity (db) Speaker Relative Intensity (db)

44 Object-Based Gain Gain-Control Models Control? Object-Based correlation.2.1 attended background Neural Results Speaker Relative Intensity (db) correlation.2.1 attended background Globally Based correlation.2.1 attended background Speaker Relative Intensity (db) Speaker Relative Intensity (db)

45 Object-Based Gain Gain-Control Models Control? Object-Based correlation.2.1 attended background Neural Results Speaker Relative Intensity (db) correlation.2.1 attended background Globally Based correlation.2.1 attended background Speaker Relative Intensity (db) Speaker Relative Intensity (db) Gain Control is Object-Based Neural representation is invariant to acoustic changes.

46 STRF model Spectro-Temporal Response Function (STRF)

47 STRF Results frequency (khz) Attended time (ms) Background time (ms) STRF separable (time, frequency) 300 Hz - 2 khz dominant carrier M50STRF positive peak M100STRF negative peak M100STRF strongly modulated, but not M50STRF 0 attended background time (ms)

48 Neural Sources M100STRF source near (same as?) M100 source: STG/PT M50STRF source is anterior and medial to M100 (same as M50?): HG anterior posterior Left M50STRF M100STRF M100 medial Right 5 mm

49 Cortical Object- Processing Hierarchy Attentional Modulation Influence of Relative Intensity 0 attended background time (ms) time (ms) 8 db 5 db 0 db -5 db -8 db clean M100STRF strongly modulated by attention, but not M50STRF. M100STRF invariant against acoustic changes (but not M50STRF?). Objects well-neurally represented at 100 ms, but not 50 ms.

50 Summary Cortical representations of speech show properties consistent with being neural representations of auditory objects Meet three formal criteria Object representation well-formed at 100 ms latency (STG, PT), but not at 50 ms (HG)

51 Acknowledgements Collaborators Catherine Carr Alain de Cheveigné Didier Depireux Mounya Elhilali Jonathan Fritz Cindy Moss David Poeppel Shihab Shamma Past Postdocs Dan Hertz Yadong Wang Current Grad Students Francisco Cervantes Marisel Villafane Delgado Kim Drnec Krishna Puvvada Past Grad Students Nayef Ahmar Claudia Bonin Maria Chait Nai Ding Victor Grau-Serrat Ling Ma Raul Rodriguez Juanjuan Xiang Kai Sum Li Jiachen Zhuo Collaborators Students Murat Aytekin Julian Jenkins David Klein Huan Luo Undergraduate Students Abdulaziz Al-Turki Nicholas Asendorf Sonja Bohr Elizabeth Camenga Corinne Cameron Julien Dagenais Katya Dombrowski Kevin Kahn Andrea Shome Ben Walsh Funding NIH R03 DC NIH R01 EB NIH R01 AG NIH R01 DC NIH R01 DC NIH R01 DC NIH R01 DC NIH F31 NS USDA

Cortical Encoding of Auditory Objects at the Cocktail Party. Jonathan Z. Simon University of Maryland

Cortical Encoding of Auditory Objects at the Cocktail Party. Jonathan Z. Simon University of Maryland Cortical Encoding of Auditory Objects at the Cocktail Party Jonathan Z. Simon University of Maryland ARO Presidential Symposium, February 2013 Introduction Auditory Objects Magnetoencephalography (MEG)

More information

Neural Representations of the Cocktail Party in Human Auditory Cortex

Neural Representations of the Cocktail Party in Human Auditory Cortex Neural Representations of the Cocktail Party in Human Auditory Cortex Jonathan Z. Simon Department of Electrical & Computer Engineering Department of Biology Institute for Systems Research University of

More information

Neural Representations of the Cocktail Party in Human Auditory Cortex

Neural Representations of the Cocktail Party in Human Auditory Cortex Neural Representations of the Cocktail Party in Human Auditory Cortex Jonathan Z. Simon Department of Biology Department of Electrical & Computer Engineering Institute for Systems Research University of

More information

Neural Representations of the Cocktail Party in Human Auditory Cortex

Neural Representations of the Cocktail Party in Human Auditory Cortex Neural Representations of the Cocktail Party in Human Auditory Cortex Jonathan Z. Simon Department of Biology Department of Electrical & Computer Engineering Institute for Systems Research University of

More information

Neural Representations of Speech, and Speech in Noise, in Human Auditory Cortex

Neural Representations of Speech, and Speech in Noise, in Human Auditory Cortex Neural Representations of Speech, and Speech in Noise, in Human Auditory Cortex Jonathan Z. Simon Department of Biology Department of Electrical & Computer Engineering Institute for Systems Research University

More information

Neural Representations of Speech at the Cocktail Party in Human Auditory Cortex

Neural Representations of Speech at the Cocktail Party in Human Auditory Cortex Neural Representations of Speech at the Cocktail Party in Human Auditory Cortex Jonathan Z. Simon Department of Electrical & Computer Engineering Department of Biology Institute for Systems Research University

More information

Neural Representations of Speech in Human Auditory Cortex

Neural Representations of Speech in Human Auditory Cortex Neural Representations of Speech in Human Auditory Cortex Jonathan Z. Simon Department of Electrical & Computer Engineering Department of Biology Institute for Systems Research University of Maryland http://www.isr.umd.edu/labs/cssl/simonlab

More information

Competing Streams at the Cocktail Party

Competing Streams at the Cocktail Party Competing Streams at the Cocktail Party A Neural and Behavioral Study of Auditory Attention Jonathan Z. Simon Neuroscience and Cognitive Sciences / Biology / Electrical & Computer Engineering University

More information

Robust Neural Encoding of Speech in Human Auditory Cortex

Robust Neural Encoding of Speech in Human Auditory Cortex Robust Neural Encoding of Speech in Human Auditory Cortex Nai Ding, Jonathan Z. Simon Electrical Engineering / Biology University of Maryland, College Park Auditory Processing in Natural Scenes How is

More information

Effects of aging on temporal synchronization of speech in noise investigated in the cortex by using MEG and in the midbrain by using EEG techniques

Effects of aging on temporal synchronization of speech in noise investigated in the cortex by using MEG and in the midbrain by using EEG techniques Hearing Brain Lab Computational Sensorimotor Systems Lab Effects of aging on temporal synchronization of speech in noise investigated in the cortex by using MEG and in the midbrain by using EEG techniques

More information

Over-representation of speech in older adults originates from early response in higher order auditory cortex

Over-representation of speech in older adults originates from early response in higher order auditory cortex Over-representation of speech in older adults originates from early response in higher order auditory cortex Christian Brodbeck, Alessandro Presacco, Samira Anderson & Jonathan Z. Simon Overview 2 Puzzle

More information

Chapter 5. Summary and Conclusions! 131

Chapter 5. Summary and Conclusions! 131 ! Chapter 5 Summary and Conclusions! 131 Chapter 5!!!! Summary of the main findings The present thesis investigated the sensory representation of natural sounds in the human auditory cortex. Specifically,

More information

Spectro-temporal response fields in the inferior colliculus of awake monkey

Spectro-temporal response fields in the inferior colliculus of awake monkey 3.6.QH Spectro-temporal response fields in the inferior colliculus of awake monkey Versnel, Huib; Zwiers, Marcel; Van Opstal, John Department of Biophysics University of Nijmegen Geert Grooteplein 655

More information

CURRICULUM VITAE Jonathan Z. Simon

CURRICULUM VITAE Jonathan Z. Simon CURRICULUM VITAE Jonathan Z. Simon Personal Information Mailing Address Email jzsimon@umd.edu Electrical & Computer Engineering Dept. Web http://www.isr.umd.edu/labs/cssl/simonlab/ University of Maryland

More information

ABSTRACT. Professor, Jonathan Z. Simon, Department of Electrical and Computer Engineering

ABSTRACT. Professor, Jonathan Z. Simon, Department of Electrical and Computer Engineering ABSTRACT Title of Document: TEMPORAL CODING OF SPEECH IN HUMAN AUDITORY CORTEX Nai Ding, Doctor of Philosophy, 2012 Directed By: Professor, Jonathan Z. Simon, Department of Electrical and Computer Engineering

More information

Neurobiology of Hearing (Salamanca, 2012) Auditory Cortex (2) Prof. Xiaoqin Wang

Neurobiology of Hearing (Salamanca, 2012) Auditory Cortex (2) Prof. Xiaoqin Wang Neurobiology of Hearing (Salamanca, 2012) Auditory Cortex (2) Prof. Xiaoqin Wang Laboratory of Auditory Neurophysiology Department of Biomedical Engineering Johns Hopkins University web1.johnshopkins.edu/xwang

More information

10/15/2016. Hearing loss. Aging. Cognition. Aging, Cognition, and Hearing Loss: Clinical Implications

10/15/2016. Hearing loss. Aging. Cognition. Aging, Cognition, and Hearing Loss: Clinical Implications Aging, Cognition, and Loss: Clinical Implications Samira Anderson, Au.D., Ph.D. MAC Conference 2016 1 Aging loss Cognition Frank Lin, M.D., Ph.D., Baltimore Longitudinal Study of Aging 2 Cognitive ability

More information

Analysis of in-vivo extracellular recordings. Ryan Morrill Bootcamp 9/10/2014

Analysis of in-vivo extracellular recordings. Ryan Morrill Bootcamp 9/10/2014 Analysis of in-vivo extracellular recordings Ryan Morrill Bootcamp 9/10/2014 Goals for the lecture Be able to: Conceptually understand some of the analysis and jargon encountered in a typical (sensory)

More information

Structure and Function of the Auditory and Vestibular Systems (Fall 2014) Auditory Cortex (3) Prof. Xiaoqin Wang

Structure and Function of the Auditory and Vestibular Systems (Fall 2014) Auditory Cortex (3) Prof. Xiaoqin Wang 580.626 Structure and Function of the Auditory and Vestibular Systems (Fall 2014) Auditory Cortex (3) Prof. Xiaoqin Wang Laboratory of Auditory Neurophysiology Department of Biomedical Engineering Johns

More information

USING AUDITORY SALIENCY TO UNDERSTAND COMPLEX AUDITORY SCENES

USING AUDITORY SALIENCY TO UNDERSTAND COMPLEX AUDITORY SCENES USING AUDITORY SALIENCY TO UNDERSTAND COMPLEX AUDITORY SCENES Varinthira Duangudom and David V Anderson School of Electrical and Computer Engineering, Georgia Institute of Technology Atlanta, GA 30332

More information

FINE-TUNING THE AUDITORY SUBCORTEX Measuring processing dynamics along the auditory hierarchy. Christopher Slugocki (Widex ORCA) WAS 5.3.

FINE-TUNING THE AUDITORY SUBCORTEX Measuring processing dynamics along the auditory hierarchy. Christopher Slugocki (Widex ORCA) WAS 5.3. FINE-TUNING THE AUDITORY SUBCORTEX Measuring processing dynamics along the auditory hierarchy. Christopher Slugocki (Widex ORCA) WAS 5.3.2017 AUDITORY DISCRIMINATION AUDITORY DISCRIMINATION /pi//k/ /pi//t/

More information

Representation of sound in the auditory nerve

Representation of sound in the auditory nerve Representation of sound in the auditory nerve Eric D. Young Department of Biomedical Engineering Johns Hopkins University Young, ED. Neural representation of spectral and temporal information in speech.

More information

The neurolinguistic toolbox Jonathan R. Brennan. Introduction to Neurolinguistics, LSA2017 1

The neurolinguistic toolbox Jonathan R. Brennan. Introduction to Neurolinguistics, LSA2017 1 The neurolinguistic toolbox Jonathan R. Brennan Introduction to Neurolinguistics, LSA2017 1 Psycholinguistics / Neurolinguistics Happy Hour!!! Tuesdays 7/11, 7/18, 7/25 5:30-6:30 PM @ the Boone Center

More information

Competing Streams at the Cocktail Party: Exploring the Mechanisms of Attention and Temporal Integration

Competing Streams at the Cocktail Party: Exploring the Mechanisms of Attention and Temporal Integration 12084 The Journal of Neuroscience, September 8, 2010 30(36):12084 12093 Behavioral/Systems/Cognitive Competing Streams at the Cocktail Party: Exploring the Mechanisms of Attention and Temporal Integration

More information

AUDL GS08/GAV1 Signals, systems, acoustics and the ear. Pitch & Binaural listening

AUDL GS08/GAV1 Signals, systems, acoustics and the ear. Pitch & Binaural listening AUDL GS08/GAV1 Signals, systems, acoustics and the ear Pitch & Binaural listening Review 25 20 15 10 5 0-5 100 1000 10000 25 20 15 10 5 0-5 100 1000 10000 Part I: Auditory frequency selectivity Tuning

More information

Computational Perception /785. Auditory Scene Analysis

Computational Perception /785. Auditory Scene Analysis Computational Perception 15-485/785 Auditory Scene Analysis A framework for auditory scene analysis Auditory scene analysis involves low and high level cues Low level acoustic cues are often result in

More information

Rhythm and Rate: Perception and Physiology HST November Jennifer Melcher

Rhythm and Rate: Perception and Physiology HST November Jennifer Melcher Rhythm and Rate: Perception and Physiology HST 722 - November 27 Jennifer Melcher Forward suppression of unit activity in auditory cortex Brosch and Schreiner (1997) J Neurophysiol 77: 923-943. Forward

More information

Chapter 40 Effects of Peripheral Tuning on the Auditory Nerve s Representation of Speech Envelope and Temporal Fine Structure Cues

Chapter 40 Effects of Peripheral Tuning on the Auditory Nerve s Representation of Speech Envelope and Temporal Fine Structure Cues Chapter 40 Effects of Peripheral Tuning on the Auditory Nerve s Representation of Speech Envelope and Temporal Fine Structure Cues Rasha A. Ibrahim and Ian C. Bruce Abstract A number of studies have explored

More information

Modulation and Top-Down Processing in Audition

Modulation and Top-Down Processing in Audition Modulation and Top-Down Processing in Audition Malcolm Slaney 1,2 and Greg Sell 2 1 Yahoo! Research 2 Stanford CCRMA Outline The Non-Linear Cochlea Correlogram Pitch Modulation and Demodulation Information

More information

Beyond Blind Averaging: Analyzing Event-Related Brain Dynamics. Scott Makeig. sccn.ucsd.edu

Beyond Blind Averaging: Analyzing Event-Related Brain Dynamics. Scott Makeig. sccn.ucsd.edu Beyond Blind Averaging: Analyzing Event-Related Brain Dynamics Scott Makeig Institute for Neural Computation University of California San Diego La Jolla CA sccn.ucsd.edu Talk given at the EEG/MEG course

More information

Effect of informational content of noise on speech representation in the aging midbrain and cortex

Effect of informational content of noise on speech representation in the aging midbrain and cortex J Neurophysiol 116: 2356 2367, 2016. First published September 7, 2016; doi:10.1152/jn.00373.2016. Effect of informational content of noise on speech representation in the aging midbrain and cortex Alessandro

More information

Precise Spike Timing and Reliability in Neural Encoding of Low-Level Sensory Stimuli and Sequences

Precise Spike Timing and Reliability in Neural Encoding of Low-Level Sensory Stimuli and Sequences Precise Spike Timing and Reliability in Neural Encoding of Low-Level Sensory Stimuli and Sequences Temporal Structure In the World Representation in the Brain Project 1.1.2 Feldman and Harris Labs Temporal

More information

How is the stimulus represented in the nervous system?

How is the stimulus represented in the nervous system? How is the stimulus represented in the nervous system? Eric Young F Rieke et al Spikes MIT Press (1997) Especially chapter 2 I Nelken et al Encoding stimulus information by spike numbers and mean response

More information

Binaural Hearing. Why two ears? Definitions

Binaural Hearing. Why two ears? Definitions Binaural Hearing Why two ears? Locating sounds in space: acuity is poorer than in vision by up to two orders of magnitude, but extends in all directions. Role in alerting and orienting? Separating sound

More information

Using MEG to map the auditory cortex. Jonathan Côté PhD Candidate in Etienne de Villers-Sidani s laboratory

Using MEG to map the auditory cortex. Jonathan Côté PhD Candidate in Etienne de Villers-Sidani s laboratory Using MEG to map the auditory cortex Jonathan Côté PhD Candidate in Etienne de Villers-Sidani s laboratory Sensory representations organization Sensory representations organization Organized map for most

More information

Decisions Have Consequences

Decisions Have Consequences Decisions Have Consequences Scott Makeig Swartz Center for Computational Neuroscience Institute for Neural Computation UCSD, La Jolla CA Precis of talk given at the recent Banbury Center workshop on decision

More information

Neural Correlates of Human Cognitive Function:

Neural Correlates of Human Cognitive Function: Neural Correlates of Human Cognitive Function: A Comparison of Electrophysiological and Other Neuroimaging Approaches Leun J. Otten Institute of Cognitive Neuroscience & Department of Psychology University

More information

Toward Extending Auditory Steady-State Response (ASSR) Testing to Longer-Latency Equivalent Potentials

Toward Extending Auditory Steady-State Response (ASSR) Testing to Longer-Latency Equivalent Potentials Department of History of Art and Architecture Toward Extending Auditory Steady-State Response (ASSR) Testing to Longer-Latency Equivalent Potentials John D. Durrant, PhD, CCC-A, FASHA Department of Communication

More information

TOWARD A BRAIN INTERFACE FOR TRACKING ATTENDED AUDITORY SOURCES

TOWARD A BRAIN INTERFACE FOR TRACKING ATTENDED AUDITORY SOURCES 216 IEEE INTERNATIONAL WORKSHOP ON MACHINE LEARNING FOR SIGNAL PROCESSING, SEPT. 13 16, 216, SALERNO, ITALY TOWARD A BRAIN INTERFACE FOR TRACKING ATTENDED AUDITORY SOURCES Marzieh Haghighi 1, Mohammad

More information

Sound localization psychophysics

Sound localization psychophysics Sound localization psychophysics Eric Young A good reference: B.C.J. Moore An Introduction to the Psychology of Hearing Chapter 7, Space Perception. Elsevier, Amsterdam, pp. 233-267 (2004). Sound localization:

More information

The Central Nervous System

The Central Nervous System The Central Nervous System Cellular Basis. Neural Communication. Major Structures. Principles & Methods. Principles of Neural Organization Big Question #1: Representation. How is the external world coded

More information

Cortical sound processing in children with autism disorder: an MEG investigation

Cortical sound processing in children with autism disorder: an MEG investigation AUDITORYAND VESTIBULAR SYSTEMS Cortical sound processing in children with autism disorder: an MEG investigation Nicole M. Gage CA, Bryna Siegel, 1 Melanie Callen 1 and Timothy P. L. Roberts Biomagnetic

More information

Atypical processing of prosodic changes in natural speech stimuli in school-age children with Asperger syndrome

Atypical processing of prosodic changes in natural speech stimuli in school-age children with Asperger syndrome Atypical processing of prosodic changes in natural speech stimuli in school-age children with Asperger syndrome Riikka Lindström, PhD student Cognitive Brain Research Unit University of Helsinki 31.8.2012

More information

Introduction to Computational Neuroscience

Introduction to Computational Neuroscience Introduction to Computational Neuroscience Lecture 5: Data analysis II Lesson Title 1 Introduction 2 Structure and Function of the NS 3 Windows to the Brain 4 Data analysis 5 Data analysis II 6 Single

More information

Practicalities of EEG Measurement and Experiment Design YATING LIU

Practicalities of EEG Measurement and Experiment Design YATING LIU Practicalities of EEG Measurement and Experiment Design YATING LIU 2014.02.04 Content Designing Experiments: Discuss & Pilot Event Markers Intra- and Intertrial Timing How Many Trials You Will Need How

More information

Neuromagnetic recordings reveal the temporal dynamics of auditory spatial processing in the human cortex

Neuromagnetic recordings reveal the temporal dynamics of auditory spatial processing in the human cortex Neuroscience Letters 396 (2006) 17 22 Neuromagnetic recordings reveal the temporal dynamics of auditory spatial processing in the human cortex Hannu Tiitinen a,b,, Nelli H. Salminen a, Kalle J. Palomäki

More information

Dynamic, adaptive task-related processing of sound in the auditory and prefrontal cortex

Dynamic, adaptive task-related processing of sound in the auditory and prefrontal cortex Dynamic, adaptive task-related processing of sound in the auditory and prefrontal cortex Jonathan Fritz 1, Stephen David 1,3, Bernhard Englitz 1,4, Serin Atiani 1,5, Diego Elgueda 1, Michael Locastro 1,

More information

But, what about ASSR in AN?? Is it a reliable tool to estimate the auditory thresholds in those category of patients??

But, what about ASSR in AN?? Is it a reliable tool to estimate the auditory thresholds in those category of patients?? 1 Auditory Steady State Response (ASSR) thresholds have been shown to be highly correlated to bh behavioral thresholds h in adults and older children with normal hearing or those with sensorineural hearing

More information

Brain Imaging of auditory function

Brain Imaging of auditory function Who am I? Brain Imaging of auditory function Maria Chait I do (mostly) MEG functional brain imaging of auditory processing. Now, I am at the EAR Institute, University College London & Equipe Audition,

More information

The role of amplitude, phase, and rhythmicity of neural oscillations in top-down control of cognition

The role of amplitude, phase, and rhythmicity of neural oscillations in top-down control of cognition The role of amplitude, phase, and rhythmicity of neural oscillations in top-down control of cognition Chair: Jason Samaha, University of Wisconsin-Madison Co-Chair: Ali Mazaheri, University of Birmingham

More information

Decoding the attended speech stream with multichannel EEG: implications for online, daily-life applications

Decoding the attended speech stream with multichannel EEG: implications for online, daily-life applications Journal of Neural Engineering PAPER OPEN ACCESS Decoding the attended speech stream with multichannel EEG: implications for online, daily-life applications To cite this article: Bojana Mirkovic et al 2015

More information

Robustness, Separation & Pitch

Robustness, Separation & Pitch Robustness, Separation & Pitch or Morgan, Me & Pitch Dan Ellis Columbia / ICSI dpwe@ee.columbia.edu http://labrosa.ee.columbia.edu/ 1. Robustness and Separation 2. An Academic Journey 3. Future COLUMBIA

More information

Dynamics of Precise Spike Timing in Primary Auditory Cortex

Dynamics of Precise Spike Timing in Primary Auditory Cortex The Journal of Neuroscience, February 4, 2004 24(5):1159 1172 1159 Behavioral/Systems/Cognitive Dynamics of Precise Spike Timing in Primary Auditory Cortex Mounya Elhilali, Jonathan B. Fritz, David J.

More information

Task Difficulty and Performance Induce Diverse Adaptive Patterns in Gain and Shape of Primary Auditory Cortical Receptive Fields

Task Difficulty and Performance Induce Diverse Adaptive Patterns in Gain and Shape of Primary Auditory Cortical Receptive Fields Article Task Difficulty and Performance Induce Diverse Adaptive Patterns in Gain and Shape of Primary Auditory Cortical Receptive Fields Serin Atiani, 1 Mounya Elhilali, 3 Stephen V. David, 2 Jonathan

More information

Report. Direct Recordings of Pitch Responses from Human Auditory Cortex

Report. Direct Recordings of Pitch Responses from Human Auditory Cortex Current Biology 0,, June, 00 ª00 Elsevier Ltd. Open access under CC BY license. DOI 0.0/j.cub.00.0.0 Direct Recordings of Pitch Responses from Human Auditory Cortex Report Timothy D. Griffiths,, * Sukhbinder

More information

Interaction between Attention and Bottom-Up Saliency Mediates the Representation of Foreground and Background in an Auditory Scene

Interaction between Attention and Bottom-Up Saliency Mediates the Representation of Foreground and Background in an Auditory Scene Interaction between Attention and Bottom-Up Saliency Mediates the Representation of Foreground and Background in an Auditory Scene Mounya Elhilali 1., Juanjuan Xiang 2., Shihab A. Shamma 3,4, Jonathan

More information

AccuScreen ABR Screener

AccuScreen ABR Screener AccuScreen ABR Screener Test Methods Doc no. 7-50-1015-EN/02 0459 Copyright notice No part of this Manual or program may be reproduced, stored in a retrieval system, or transmitted, in any form or by any

More information

Early Occurrence Of Auditory Change Detection In The Human Brain

Early Occurrence Of Auditory Change Detection In The Human Brain Early Occurrence Of Auditory Change Detection In The Human Brain SABINE GRIMM University of Leipzig, Germany (formerly University of Barcelona, Spain) ICON, Brisbane, July 28 th, 2014 Auditory deviance

More information

Encoding of natural sounds by variance of the cortical local field potential

Encoding of natural sounds by variance of the cortical local field potential J Neurophysiol 115: 2389 2398, 2016. First published February 24, 2016; doi:10.1152/jn.00652.2015. Encoding of natural sounds by variance of the cortical local field potential Nai Ding, 1 Jonathan Z. Simon,

More information

An Auditory-Model-Based Electrical Stimulation Strategy Incorporating Tonal Information for Cochlear Implant

An Auditory-Model-Based Electrical Stimulation Strategy Incorporating Tonal Information for Cochlear Implant Annual Progress Report An Auditory-Model-Based Electrical Stimulation Strategy Incorporating Tonal Information for Cochlear Implant Joint Research Centre for Biomedical Engineering Mar.7, 26 Types of Hearing

More information

Entrainment of neuronal oscillations as a mechanism of attentional selection: intracranial human recordings

Entrainment of neuronal oscillations as a mechanism of attentional selection: intracranial human recordings Entrainment of neuronal oscillations as a mechanism of attentional selection: intracranial human recordings J. Besle, P. Lakatos, C.A. Schevon, R.R. Goodman, G.M. McKhann, A. Mehta, R.G. Emerson, C.E.

More information

The dynamics of the construction of auditory perceptual representations: MEG studies in humans

The dynamics of the construction of auditory perceptual representations: MEG studies in humans The dynamics of the construction of auditory perceptual representations: MEG studies in humans Maria Chait 1, 3 and Jonathan Z. Simon 2 1 Neuroscience and Cognitive Science Program, University of Maryland,

More information

Processing and Plasticity in Rodent Somatosensory Cortex

Processing and Plasticity in Rodent Somatosensory Cortex Processing and Plasticity in Rodent Somatosensory Cortex Dan Feldman Dept. of Molecular & Cell Biology Helen Wills Neuroscience Institute UC Berkeley Primary sensory neocortex Secondary and higher areas

More information

MULTI-CHANNEL COMMUNICATION

MULTI-CHANNEL COMMUNICATION INTRODUCTION Research on the Deaf Brain is beginning to provide a new evidence base for policy and practice in relation to intervention with deaf children. This talk outlines the multi-channel nature of

More information

Introduction to Computational Neuroscience

Introduction to Computational Neuroscience Introduction to Computational Neuroscience Lecture 10: Brain-Computer Interfaces Ilya Kuzovkin So Far Stimulus So Far So Far Stimulus What are the neuroimaging techniques you know about? Stimulus So Far

More information

Left-hemisphere dominance for processing of vowels: a whole-scalp neuromagnetic study

Left-hemisphere dominance for processing of vowels: a whole-scalp neuromagnetic study Auditory and Vestibular Systems 10, 2987±2991 (1999) BRAIN activation of 11 healthy right-handed subjects was studied with magnetoencephalography to estimate individual hemispheric dominance for speech

More information

Sensory Physiology Bi353 Fall Term 2016

Sensory Physiology Bi353 Fall Term 2016 Lectures 2-3p MWF 111 Lillis (CRN 11012) Lab/Discussion Section 1 (CRN 11013) Friday 10-110:50a Hue 129 Lab/Discussion Section 2 (CRN 11014) Friday 11a-11:50a Hue 129 Lab/Discussion Section 3 (CRN 16400)

More information

Encoding of Temporal Information by Timing, Rate, and Place in Cat Auditory Cortex

Encoding of Temporal Information by Timing, Rate, and Place in Cat Auditory Cortex Encoding of Temporal Information by Timing, Rate, and Place in Cat Auditory Cortex Kazuo Imaizumi 1,2 *, Nicholas J. Priebe 3, Tatyana O. Sharpee 4,5, Steven W. Cheung 1, Christoph E. Schreiner 1,4 * 1

More information

Mechanisms Underlying Selective Neuronal Tracking of Attended Speech at a Cocktail Party

Mechanisms Underlying Selective Neuronal Tracking of Attended Speech at a Cocktail Party Article Mechanisms Underlying Selective Neuronal Tracking of Attended Speech at a Cocktail Party Elana M. Zion Golumbic, 1,4 Nai Ding, 5,6 Stephan Bickel, 7,8 Peter Lakatos, 4 Catherine A. Schevon, 2 Guy

More information

EEG reveals divergent paths for speech envelopes during selective attention

EEG reveals divergent paths for speech envelopes during selective attention EEG reveals divergent paths for speech envelopes during selective attention Cort Horton a, Michael D Zmura a, and Ramesh Srinivasan a,b a Dept. of Cognitive Sciences, University of California, Irvine,

More information

HCS 7367 Speech Perception

HCS 7367 Speech Perception Long-term spectrum of speech HCS 7367 Speech Perception Connected speech Absolute threshold Males Dr. Peter Assmann Fall 212 Females Long-term spectrum of speech Vowels Males Females 2) Absolute threshold

More information

This is a repository copy of Involvement of right STS in audio-visual integration for affective speech demonstrated using MEG.

This is a repository copy of Involvement of right STS in audio-visual integration for affective speech demonstrated using MEG. This is a repository copy of Involvement of right STS in audio-visual integration for affective speech demonstrated using MEG. White Rose Research Online URL for this paper: http://eprints.whiterose.ac.uk/76243/

More information

Sound Production. Phonotaxis in crickets. What is sound? Recognition and Localization. What happens over time at a single point in space?

Sound Production. Phonotaxis in crickets. What is sound? Recognition and Localization. What happens over time at a single point in space? Behaviour Sound Production Phonotaxis in crickets Recognition and Localization scraper close scraper file open Males open and close wings rhythmically. On each closing stroke, scraper contacts file causing

More information

Auditory Physiology PSY 310 Greg Francis. Lecture 30. Organ of Corti

Auditory Physiology PSY 310 Greg Francis. Lecture 30. Organ of Corti Auditory Physiology PSY 310 Greg Francis Lecture 30 Waves, waves, waves. Organ of Corti Tectorial membrane Sits on top Inner hair cells Outer hair cells The microphone for the brain 1 Hearing Perceptually,

More information

21/01/2013. Binaural Phenomena. Aim. To understand binaural hearing Objectives. Understand the cues used to determine the location of a sound source

21/01/2013. Binaural Phenomena. Aim. To understand binaural hearing Objectives. Understand the cues used to determine the location of a sound source Binaural Phenomena Aim To understand binaural hearing Objectives Understand the cues used to determine the location of a sound source Understand sensitivity to binaural spatial cues, including interaural

More information

Discrimination of temporal fine structure by birds and mammals

Discrimination of temporal fine structure by birds and mammals Auditory Signal Processing: Physiology, Psychoacoustics, and Models. Pressnitzer, D., de Cheveigné, A., McAdams, S.,and Collet, L. (Eds). Springer Verlag, 24. Discrimination of temporal fine structure

More information

Infant Hearing Development: Translating Research Findings into Clinical Practice. Auditory Development. Overview

Infant Hearing Development: Translating Research Findings into Clinical Practice. Auditory Development. Overview Infant Hearing Development: Translating Research Findings into Clinical Practice Lori J. Leibold Department of Allied Health Sciences The University of North Carolina at Chapel Hill Auditory Development

More information

A Consumer-friendly Recap of the HLAA 2018 Research Symposium: Listening in Noise Webinar

A Consumer-friendly Recap of the HLAA 2018 Research Symposium: Listening in Noise Webinar A Consumer-friendly Recap of the HLAA 2018 Research Symposium: Listening in Noise Webinar Perry C. Hanavan, AuD Augustana University Sioux Falls, SD August 15, 2018 Listening in Noise Cocktail Party Problem

More information

The Evolution of Neuroprosthetics

The Evolution of Neuroprosthetics The Evolution of Neuroprosthetics National Academy of Engineering, Google, 2011 Eric C. Leuthardt MD Assistant Professor of Neurological Surgery & Biomedical Engineering Director, Center for Innovation

More information

Auditory fmri correlates of loudness perception for monaural and diotic stimulation

Auditory fmri correlates of loudness perception for monaural and diotic stimulation PROCEEDINGS of the 22 nd International Congress on Acoustics Psychological and Physiological Acoustics (others): Paper ICA2016-435 Auditory fmri correlates of loudness perception for monaural and diotic

More information

ABSTRACT. Auditory streaming: behavior, physiology, and modeling

ABSTRACT. Auditory streaming: behavior, physiology, and modeling ABSTRACT Title of Document: Auditory streaming: behavior, physiology, and modeling Ling Ma, Doctor of Philosophy, 2011 Directed By: Professor Shihab A. Shamma, Department of Electrical and computer Engineering,

More information

Auditory temporal edge detection in human auditory cortex

Auditory temporal edge detection in human auditory cortex available at www.sciencedirect.com www.elsevier.com/locate/brainres Research Report Auditory temporal edge detection in human auditory cortex Maria Chait a,, David Poeppel b,c, Jonathan Z. Simon d,c a

More information

Prelude Envelope and temporal fine. What's all the fuss? Modulating a wave. Decomposing waveforms. The psychophysics of cochlear

Prelude Envelope and temporal fine. What's all the fuss? Modulating a wave. Decomposing waveforms. The psychophysics of cochlear The psychophysics of cochlear implants Stuart Rosen Professor of Speech and Hearing Science Speech, Hearing and Phonetic Sciences Division of Psychology & Language Sciences Prelude Envelope and temporal

More information

An Overview of BMIs. Luca Rossini. Workshop on Brain Machine Interfaces for Space Applications

An Overview of BMIs. Luca Rossini. Workshop on Brain Machine Interfaces for Space Applications An Overview of BMIs Luca Rossini Workshop on Brain Machine Interfaces for Space Applications European Space Research and Technology Centre, European Space Agency Noordvijk, 30 th November 2009 Definition

More information

Power and phase properties of oscillatory neural responses in the presence of background activity

Power and phase properties of oscillatory neural responses in the presence of background activity J Comput Neurosci (2013) 34:337 343 DOI 10.1007/s10827-012-0424-6 Power and phase properties of oscillatory neural responses in the presence of background activity Nai Ding & Jonathan Z. Simon Received:

More information

Role of F0 differences in source segregation

Role of F0 differences in source segregation Role of F0 differences in source segregation Andrew J. Oxenham Research Laboratory of Electronics, MIT and Harvard-MIT Speech and Hearing Bioscience and Technology Program Rationale Many aspects of segregation

More information

Adaptive Changes in Cortical Receptive Fields Induced by Attention to Complex Sounds

Adaptive Changes in Cortical Receptive Fields Induced by Attention to Complex Sounds J Neurophysiol 9: 337 36, 7. First published August 5, 7; doi:/jn.55.7. Adaptive Changes in Cortical Receptive Fields Induced by Attention to Complex Sounds Jonathan B. Fritz, Mounya Elhilali, and Shihab

More information

Hearing the Universal Language: Music and Cochlear Implants

Hearing the Universal Language: Music and Cochlear Implants Hearing the Universal Language: Music and Cochlear Implants Professor Hugh McDermott Deputy Director (Research) The Bionics Institute of Australia, Professorial Fellow The University of Melbourne Overview?

More information

Restoring Communication and Mobility

Restoring Communication and Mobility Restoring Communication and Mobility What are they? Artificial devices connected to the body that substitute, restore or supplement a sensory, cognitive, or motive function of the nervous system that has

More information

Frequency refers to how often something happens. Period refers to the time it takes something to happen.

Frequency refers to how often something happens. Period refers to the time it takes something to happen. Lecture 2 Properties of Waves Frequency and period are distinctly different, yet related, quantities. Frequency refers to how often something happens. Period refers to the time it takes something to happen.

More information

Introduction to sensory pathways. Gatsby / SWC induction week 25 September 2017

Introduction to sensory pathways. Gatsby / SWC induction week 25 September 2017 Introduction to sensory pathways Gatsby / SWC induction week 25 September 2017 Studying sensory systems: inputs and needs Stimulus Modality Robots Sensors Biological Sensors Outputs Light Vision Photodiodes

More information

Chapter 59 Temporal Coherence and the Streaming of Complex Sounds

Chapter 59 Temporal Coherence and the Streaming of Complex Sounds Chapter 59 Temporal Coherence and the Streaming of Complex Sounds Shihab Shamma, Mounya Elhilali, Ling Ma, Christophe Micheyl, ndrew J. Oxenham, Daniel Pressnitzer, Pingbo Yin, and Yanbo Xu bstract Humans

More information

Lecture Outline. The GIN test and some clinical applications. Introduction. Temporal processing. Gap detection. Temporal resolution and discrimination

Lecture Outline. The GIN test and some clinical applications. Introduction. Temporal processing. Gap detection. Temporal resolution and discrimination Lecture Outline The GIN test and some clinical applications Dr. Doris-Eva Bamiou National Hospital for Neurology Neurosurgery and Institute of Child Health (UCL)/Great Ormond Street Children s Hospital

More information

Effects of Cochlear Hearing Loss on the Benefits of Ideal Binary Masking

Effects of Cochlear Hearing Loss on the Benefits of Ideal Binary Masking INTERSPEECH 2016 September 8 12, 2016, San Francisco, USA Effects of Cochlear Hearing Loss on the Benefits of Ideal Binary Masking Vahid Montazeri, Shaikat Hossain, Peter F. Assmann University of Texas

More information

Est-ce que l'eeg a toujours sa place en 2019?

Est-ce que l'eeg a toujours sa place en 2019? Est-ce que l'eeg a toujours sa place en 2019? Thomas Bast Epilepsy Center Kork, Germany Does EEG still play a role in 2019? What a question 7T-MRI, fmri, DTI, MEG, SISCOM, Of ieeg course! /HFO, Genetics

More information

From Tones to Speech: Magnetoencephalographic Studies

From Tones to Speech: Magnetoencephalographic Studies Chapter 28 From Tones to Speech: Magnetoencephalographic Studies Bernd Lütkenhöner and David Poeppel Abbreviations AEP auditory evoked magnetic field AM amplitude modulation CV consonant-vowel DC direct

More information

Multiscale Evidence of Multiscale Brain Communication

Multiscale Evidence of Multiscale Brain Communication Multiscale Evidence of Multiscale Brain Communication Scott Makeig Swartz Center for Computational Neuroscience Institute for Neural Computation University of California San Diego La Jolla CA Talk given

More information

Nikos Laskaris ENTEP

Nikos Laskaris ENTEP Nikos Laskaris ENTEP Reflections of learning at the University of Patras and opportunities to discover at BSI/RIKEN Understanding begins by elucidating basic brain mechanisms. This area of research is

More information

SUPPLEMENTARY INFORMATION. Table 1 Patient characteristics Preoperative. language testing

SUPPLEMENTARY INFORMATION. Table 1 Patient characteristics Preoperative. language testing Categorical Speech Representation in the Human Superior Temporal Gyrus Edward F. Chang, Jochem W. Rieger, Keith D. Johnson, Mitchel S. Berger, Nicholas M. Barbaro, Robert T. Knight SUPPLEMENTARY INFORMATION

More information