Cortical Encoding of Auditory Objects at the Cocktail Party. Jonathan Z. Simon University of Maryland

Size: px
Start display at page:

Download "Cortical Encoding of Auditory Objects at the Cocktail Party. Jonathan Z. Simon University of Maryland"

Transcription

1 Cortical Encoding of Auditory Objects at the Cocktail Party Jonathan Z. Simon University of Maryland ARO Presidential Symposium, February 2013

2 Introduction Auditory Objects Magnetoencephalography (MEG) Neural Representations of Auditory Objects in Cortex: Decoding Neural Representations of Auditory Objects in Cortex: Encoding

3 Auditory Objects What is an auditory object? perceptual (not neural) construct commonalities with visual objects several potential formal definitions

4 Auditory Object Definition Griffiths & Warren definition: corresponds with something in the sensory world object information separate from information of rest of sensory world abstracted: object information generalized over particular sensory experiences

5 Auditory Objects a few more details punctate or streaming, serial or parallel not quite what in what vs. where

6 Auditory Objects at the Cocktail Party Alex Katz, The Cocktail Party

7 Auditory Objects at the Cocktail Party Alex Katz, The Cocktail Party

8 Auditory Objects at the Cocktail Party Alex Katz, The Cocktail Party

9 Auditory Objects at the Cocktail Party Alex Katz, The Cocktail Party

10 Auditory Objects at the Cocktail Party Alex Katz, The Cocktail Party

11 Auditory Objects at the Cocktail Party Alex Katz, The Cocktail Party

12 Auditory Objects at the Cocktail Party Alex Katz, The Cocktail Party

13 Auditory Objects at the Cocktail Party Ding & Simon, PNAS (2012)

14 Experiments

15 Magnetoencephalography (MEG) scalp skull B MEG V EEG recording surface orientation of magnetic field Magnetic Dipolar Field Projection CSF tissue current flow Photo by Fritz Goro Direct electrophysiological measurement not hemodynamic real-time No unique solution for distributed source Measures spatially synchronized cortical activity Fine temporal resolution (~ 1 ms) Moderate spatial resolution (~ 1 cm)

16 Magnetoencephalography (MEG) scalp skull B MEG V EEG recording surface orientation of magnetic field Magnetic Dipolar Field Projection CSF tissue current flow Photo by Fritz Goro Direct electrophysiological measurement not hemodynamic real-time No unique solution for distributed source Measures spatially synchronized cortical activity Fine temporal resolution (~ 1 ms) Moderate spatial resolution (~ 1 cm)

17 Phase-Locking in MEG to Slow Temporal Modulations AM at 3 Hz 3 Hz phase-locked response MEG activity is precisely phase-locked to temporal modulations of sound response spectrum (subject R0747) 3 Hz 6 Hz 0 10 Ding & Simon, J Neurophysiol (2009) Wang et al., J Neurophysiol (2012) Frequency (Hz)

18 Phase-Locking in MEG to Slow Temporal Modulations AM at 3 Hz 3 Hz phase-locked response MEG activity is precisely phase-locked to temporal modulations of sound response spectrum (subject R0747) 3 Hz 6 Hz 0 10 Ding & Simon, J Neurophysiol (2009) Wang et al., J Neurophysiol (2012) Frequency (Hz)

19 MEG Responses to Speech Modulations Auditory Model

20 MEG Responses Predicted by STRF Model Ding & Simon, J Neurophysiol (2012) (up to ~10 Hz) Spectro-Temporal Response Function (STRF)

21 Neural Reconstruction of Speech Envelope Speech Envelope Decoder MEG Responses. (up to ~ 10 Hz) stimulus speech envelope reconstructed stimulus speech envelope 2 s Ding & Simon, J Neurophysiol (2012) Zion-Golumbic et al., Neuron (in press) Reconstruction accuracy comparable to single unit & ECoG recordings

22 Neural Reconstruction of Speech Envelope Speech Envelope Decoder MEG Responses. (up to ~ 10 Hz) stimulus speech envelope reconstructed stimulus speech envelope 2 s Ding & Simon, J Neurophysiol (2012) Zion-Golumbic et al., Neuron (in press) Reconstruction accuracy comparable to single unit & ECoG recordings

23 Neural Reconstruction of Speech Envelope Speech Envelope Decoder MEG Responses. (up to ~ 10 Hz) stimulus speech envelope reconstructed stimulus speech envelope 2 s Ding & Simon, J Neurophysiol (2012) Zion-Golumbic et al., Neuron (in press) Reconstruction accuracy comparable to single unit & ECoG recordings

24 Speech Stream as an Auditory Object corresponds with something in the sensory world information separate from information of rest of sensory world e.g. other speech streams or noise abstracted: object information generalized over particular sensory experiences e.g. different sound mixtures

25 Neural Representation of an Auditory Object neural representation is of something in sensory world when other sounds mixed in, neural representation is of auditory object, not entire acoustic scene neural representation invariant under broad changes in specific acoustics

26 Selective Neural Encoding

27 Selective Neural Encoding

28 Selective Neural Encoding

29 Unselective vs. Selective Neural Encoding

30 Stream-Specific Representation representative subject attending to speaker 1 attending to speaker 2 grand average over subjects reconstructed from MEG attended speech envelopes reconstructed from MEG Identical Stimuli!

31 Stream-Specific Representation representative grand average attending to subject over subjects reconstructed from MEG speaker 1 attended speech envelopes attending to speaker 2 reconstructed from MEG Identical Stimuli!

32 Single Trial Speech Reconstruction

33 Single Trial Speech Reconstruction

34 Overall Speech Reconstruction correlation Distinct neural representations for different speech streams 0 attended speech reconstruction attended speech background reconstruction background

35 Invariance Under Acoustic Changes

36 Invariance Under Acoustic Changes

37 Invariance Under Acoustic Changes

38 Invariance Under Acoustic Changes

39 Invariance Under Acoustic Changes

40 Gain-Control Models Stream-Based Gain Control? Object-Based correlation.2.1 attended background Speaker Relative Intensity (db) Stimulus- Based correlation.2.1 attended background Speaker Relative Intensity (db)

41 Gain-Control Models Stream-Based Gain Control? Object-Based correlation.2.1 attended background Neural Results Speaker Relative Intensity (db) correlation.2.1 attended background Stimulus- Based correlation.2.1 attended background Speaker Relative Intensity (db) Speaker Relative Intensity (db)

42 Gain-Control Models Stream-Based Gain Control? Object-Based correlation.2.1 attended background Neural Results Speaker Relative Intensity (db) correlation.2.1 attended background Stimulus- Based correlation.2.1 attended background Speaker Relative Intensity (db) Speaker Relative Intensity (db)

43 Gain-Control Models Stream-Based Gain Control? Object-Based correlation.2.1 attended background Neural Results Speaker Relative Intensity (db) correlation.2.1 attended background Stimulus- Based correlation.2.1 attended background Speaker Relative Intensity (db) Speaker Relative Intensity (db) Stream-based not stimulus-based Neural representation is invariant to acoustic changes.

44 Neural Representation of an Auditory Object neural representation is of something in sensory world when other sounds mixed in, neural representation is of auditory object, not entire acoustic scene neural representation invariant under broad changes in specific acoustics

45 Forward STRF Model Spectro-Temporal Response Function (STRF)

46 Forward STRF Model Spectro-Temporal Response Function (STRF)

47 STRF Results frequency (khz) Attended time (ms) Background time (ms) TRF STRF separable (time, frequency) 300 Hz - 2 khz dominant carriers M50STRF positive peak M100STRF negative peak M100STRF strongly modulated by attention, but not M50STRF unattended attended

48 STRF Results frequency (khz) Attended time (ms) Background time (ms) TRF STRF separable (time, frequency) 300 Hz - 2 khz dominant carriers M50STRF positive peak M100STRF negative peak M100STRF strongly modulated by attention, but not M50STRF unattended attended

49 STRF Results frequency (khz) Attended time (ms) Background time (ms) TRF STRF separable (time, frequency) 300 Hz - 2 khz dominant carriers M50STRF positive peak M100STRF negative peak M100STRF strongly modulated by attention, but not M50STRF unattended attended

50 Neural Sources M100STRF source near (same as?) M100 source: PT anterior Left M50STRF M100STRF Right M50STRF source is anterior and medial to M100 (same as M50?): HG posterior M100 medial 5 mm

51 Cortical Object- Processing Hierarchy Attentional Modulation Influence of Relative Intensity 0 attended background time (ms) time (ms) 8 db 5 db 0 db -5 db -8 db clean M100STRF strongly modulated by attention, but not M50STRF. M100STRF invariant against acoustic changes. Objects well-neurally represented at 100 ms, but not 50 ms.

52 Not Just Speech Competing Tone Streams Tone Stream in Masker Cloud Frequency Frequency Time Time Xiang et al., J Neuroscience (2010) Elhilali et al., PLoS Biology (2009)

53 Tone Stream in Masker Cloud Frequency Time

54 Tone Stream in Masker Cloud Frequency Time

55 Tone Stream in Masker Cloud Frequency Time

56 Tone Stream in Masker Cloud Frequency Time

57 Competing Tone Streams Frequency Time

58 Competing Tone Streams Frequency Time

59 Competing Tone Streams Frequency Time

60 Summary Cortical representations of speech found here: consistent with being neural representations of auditory (perceptual) objects meet 3 formal criteria for auditory objects Object representation fully formed by 100 ms latency (PT), but not by 50 ms (HG) Not special to speech

61 Acknowledgements Grad Students Francisco Cervantes Marisel Villafane Delgado Kim Drnec Krishna Puvvada Past Grad Students Nayef Ahmar Claudia Bonin Maria Chait Nai Ding Victor Grau-Serrat Ling Ma Raul Rodriguez Juanjuan Xiang Kai Sum Li Jiachen Zhuo Undergraduate Students Abdulaziz Al-Turki Nicholas Asendorf Sonja Bohr Elizabeth Camenga Corinne Cameron Julien Dagenais Katya Dombrowski Kevin Kahn Andrea Shome Ben Walsh Collaborators Students Murat Aytekin Julian Jenkins David Klein Huan Luo Collaborators Catherine Carr Alain de Cheveigné Didier Depireux Mounya Elhilali Jonathan Fritz Cindy Moss David Poeppel Shihab Shamma Past Postdocs Dan Hertz Yadong Wang Funding NIH R01 DC #373 Cortical Representations of Music in Human Listeners #889 Cortical Encoding of Speech in Challenging Listening Environments #961 Neural Entrainment to Speech, a Matter of Time or Frequency?

62 Thank You

63 Reconstruction of Same-Sex Speech Correlation Attended Speech Reconstruction Speaker Two Single Trial Decoding Results Attended Speaker One Two attended speech background speech Speaker One

Cortical Encoding of Auditory Objects in the Cocktail Party Problem. Jonathan Z. Simon University of Maryland

Cortical Encoding of Auditory Objects in the Cocktail Party Problem. Jonathan Z. Simon University of Maryland Cortical Encoding of Auditory Objects in the Cocktail Party Problem Jonathan Z. Simon University of Maryland Introduction Auditory Objects Magnetoencephalography (MEG) Decoding Neural Signals/Encoding

More information

Neural Representations of the Cocktail Party in Human Auditory Cortex

Neural Representations of the Cocktail Party in Human Auditory Cortex Neural Representations of the Cocktail Party in Human Auditory Cortex Jonathan Z. Simon Department of Electrical & Computer Engineering Department of Biology Institute for Systems Research University of

More information

Neural Representations of the Cocktail Party in Human Auditory Cortex

Neural Representations of the Cocktail Party in Human Auditory Cortex Neural Representations of the Cocktail Party in Human Auditory Cortex Jonathan Z. Simon Department of Biology Department of Electrical & Computer Engineering Institute for Systems Research University of

More information

Neural Representations of the Cocktail Party in Human Auditory Cortex

Neural Representations of the Cocktail Party in Human Auditory Cortex Neural Representations of the Cocktail Party in Human Auditory Cortex Jonathan Z. Simon Department of Biology Department of Electrical & Computer Engineering Institute for Systems Research University of

More information

Neural Representations of Speech, and Speech in Noise, in Human Auditory Cortex

Neural Representations of Speech, and Speech in Noise, in Human Auditory Cortex Neural Representations of Speech, and Speech in Noise, in Human Auditory Cortex Jonathan Z. Simon Department of Biology Department of Electrical & Computer Engineering Institute for Systems Research University

More information

Neural Representations of Speech at the Cocktail Party in Human Auditory Cortex

Neural Representations of Speech at the Cocktail Party in Human Auditory Cortex Neural Representations of Speech at the Cocktail Party in Human Auditory Cortex Jonathan Z. Simon Department of Electrical & Computer Engineering Department of Biology Institute for Systems Research University

More information

Neural Representations of Speech in Human Auditory Cortex

Neural Representations of Speech in Human Auditory Cortex Neural Representations of Speech in Human Auditory Cortex Jonathan Z. Simon Department of Electrical & Computer Engineering Department of Biology Institute for Systems Research University of Maryland http://www.isr.umd.edu/labs/cssl/simonlab

More information

Competing Streams at the Cocktail Party

Competing Streams at the Cocktail Party Competing Streams at the Cocktail Party A Neural and Behavioral Study of Auditory Attention Jonathan Z. Simon Neuroscience and Cognitive Sciences / Biology / Electrical & Computer Engineering University

More information

Robust Neural Encoding of Speech in Human Auditory Cortex

Robust Neural Encoding of Speech in Human Auditory Cortex Robust Neural Encoding of Speech in Human Auditory Cortex Nai Ding, Jonathan Z. Simon Electrical Engineering / Biology University of Maryland, College Park Auditory Processing in Natural Scenes How is

More information

Effects of aging on temporal synchronization of speech in noise investigated in the cortex by using MEG and in the midbrain by using EEG techniques

Effects of aging on temporal synchronization of speech in noise investigated in the cortex by using MEG and in the midbrain by using EEG techniques Hearing Brain Lab Computational Sensorimotor Systems Lab Effects of aging on temporal synchronization of speech in noise investigated in the cortex by using MEG and in the midbrain by using EEG techniques

More information

Over-representation of speech in older adults originates from early response in higher order auditory cortex

Over-representation of speech in older adults originates from early response in higher order auditory cortex Over-representation of speech in older adults originates from early response in higher order auditory cortex Christian Brodbeck, Alessandro Presacco, Samira Anderson & Jonathan Z. Simon Overview 2 Puzzle

More information

Chapter 5. Summary and Conclusions! 131

Chapter 5. Summary and Conclusions! 131 ! Chapter 5 Summary and Conclusions! 131 Chapter 5!!!! Summary of the main findings The present thesis investigated the sensory representation of natural sounds in the human auditory cortex. Specifically,

More information

Spectro-temporal response fields in the inferior colliculus of awake monkey

Spectro-temporal response fields in the inferior colliculus of awake monkey 3.6.QH Spectro-temporal response fields in the inferior colliculus of awake monkey Versnel, Huib; Zwiers, Marcel; Van Opstal, John Department of Biophysics University of Nijmegen Geert Grooteplein 655

More information

Neurobiology of Hearing (Salamanca, 2012) Auditory Cortex (2) Prof. Xiaoqin Wang

Neurobiology of Hearing (Salamanca, 2012) Auditory Cortex (2) Prof. Xiaoqin Wang Neurobiology of Hearing (Salamanca, 2012) Auditory Cortex (2) Prof. Xiaoqin Wang Laboratory of Auditory Neurophysiology Department of Biomedical Engineering Johns Hopkins University web1.johnshopkins.edu/xwang

More information

USING AUDITORY SALIENCY TO UNDERSTAND COMPLEX AUDITORY SCENES

USING AUDITORY SALIENCY TO UNDERSTAND COMPLEX AUDITORY SCENES USING AUDITORY SALIENCY TO UNDERSTAND COMPLEX AUDITORY SCENES Varinthira Duangudom and David V Anderson School of Electrical and Computer Engineering, Georgia Institute of Technology Atlanta, GA 30332

More information

Analysis of in-vivo extracellular recordings. Ryan Morrill Bootcamp 9/10/2014

Analysis of in-vivo extracellular recordings. Ryan Morrill Bootcamp 9/10/2014 Analysis of in-vivo extracellular recordings Ryan Morrill Bootcamp 9/10/2014 Goals for the lecture Be able to: Conceptually understand some of the analysis and jargon encountered in a typical (sensory)

More information

ABSTRACT. Professor, Jonathan Z. Simon, Department of Electrical and Computer Engineering

ABSTRACT. Professor, Jonathan Z. Simon, Department of Electrical and Computer Engineering ABSTRACT Title of Document: TEMPORAL CODING OF SPEECH IN HUMAN AUDITORY CORTEX Nai Ding, Doctor of Philosophy, 2012 Directed By: Professor, Jonathan Z. Simon, Department of Electrical and Computer Engineering

More information

Structure and Function of the Auditory and Vestibular Systems (Fall 2014) Auditory Cortex (3) Prof. Xiaoqin Wang

Structure and Function of the Auditory and Vestibular Systems (Fall 2014) Auditory Cortex (3) Prof. Xiaoqin Wang 580.626 Structure and Function of the Auditory and Vestibular Systems (Fall 2014) Auditory Cortex (3) Prof. Xiaoqin Wang Laboratory of Auditory Neurophysiology Department of Biomedical Engineering Johns

More information

CURRICULUM VITAE Jonathan Z. Simon

CURRICULUM VITAE Jonathan Z. Simon CURRICULUM VITAE Jonathan Z. Simon Personal Information Mailing Address Email jzsimon@umd.edu Electrical & Computer Engineering Dept. Web http://www.isr.umd.edu/labs/cssl/simonlab/ University of Maryland

More information

Competing Streams at the Cocktail Party: Exploring the Mechanisms of Attention and Temporal Integration

Competing Streams at the Cocktail Party: Exploring the Mechanisms of Attention and Temporal Integration 12084 The Journal of Neuroscience, September 8, 2010 30(36):12084 12093 Behavioral/Systems/Cognitive Competing Streams at the Cocktail Party: Exploring the Mechanisms of Attention and Temporal Integration

More information

FINE-TUNING THE AUDITORY SUBCORTEX Measuring processing dynamics along the auditory hierarchy. Christopher Slugocki (Widex ORCA) WAS 5.3.

FINE-TUNING THE AUDITORY SUBCORTEX Measuring processing dynamics along the auditory hierarchy. Christopher Slugocki (Widex ORCA) WAS 5.3. FINE-TUNING THE AUDITORY SUBCORTEX Measuring processing dynamics along the auditory hierarchy. Christopher Slugocki (Widex ORCA) WAS 5.3.2017 AUDITORY DISCRIMINATION AUDITORY DISCRIMINATION /pi//k/ /pi//t/

More information

10/15/2016. Hearing loss. Aging. Cognition. Aging, Cognition, and Hearing Loss: Clinical Implications

10/15/2016. Hearing loss. Aging. Cognition. Aging, Cognition, and Hearing Loss: Clinical Implications Aging, Cognition, and Loss: Clinical Implications Samira Anderson, Au.D., Ph.D. MAC Conference 2016 1 Aging loss Cognition Frank Lin, M.D., Ph.D., Baltimore Longitudinal Study of Aging 2 Cognitive ability

More information

Representation of sound in the auditory nerve

Representation of sound in the auditory nerve Representation of sound in the auditory nerve Eric D. Young Department of Biomedical Engineering Johns Hopkins University Young, ED. Neural representation of spectral and temporal information in speech.

More information

Rhythm and Rate: Perception and Physiology HST November Jennifer Melcher

Rhythm and Rate: Perception and Physiology HST November Jennifer Melcher Rhythm and Rate: Perception and Physiology HST 722 - November 27 Jennifer Melcher Forward suppression of unit activity in auditory cortex Brosch and Schreiner (1997) J Neurophysiol 77: 923-943. Forward

More information

How is the stimulus represented in the nervous system?

How is the stimulus represented in the nervous system? How is the stimulus represented in the nervous system? Eric Young F Rieke et al Spikes MIT Press (1997) Especially chapter 2 I Nelken et al Encoding stimulus information by spike numbers and mean response

More information

Sound localization psychophysics

Sound localization psychophysics Sound localization psychophysics Eric Young A good reference: B.C.J. Moore An Introduction to the Psychology of Hearing Chapter 7, Space Perception. Elsevier, Amsterdam, pp. 233-267 (2004). Sound localization:

More information

Interaction between Attention and Bottom-Up Saliency Mediates the Representation of Foreground and Background in an Auditory Scene

Interaction between Attention and Bottom-Up Saliency Mediates the Representation of Foreground and Background in an Auditory Scene Interaction between Attention and Bottom-Up Saliency Mediates the Representation of Foreground and Background in an Auditory Scene Mounya Elhilali 1., Juanjuan Xiang 2., Shihab A. Shamma 3,4, Jonathan

More information

The neurolinguistic toolbox Jonathan R. Brennan. Introduction to Neurolinguistics, LSA2017 1

The neurolinguistic toolbox Jonathan R. Brennan. Introduction to Neurolinguistics, LSA2017 1 The neurolinguistic toolbox Jonathan R. Brennan Introduction to Neurolinguistics, LSA2017 1 Psycholinguistics / Neurolinguistics Happy Hour!!! Tuesdays 7/11, 7/18, 7/25 5:30-6:30 PM @ the Boone Center

More information

AUDL GS08/GAV1 Signals, systems, acoustics and the ear. Pitch & Binaural listening

AUDL GS08/GAV1 Signals, systems, acoustics and the ear. Pitch & Binaural listening AUDL GS08/GAV1 Signals, systems, acoustics and the ear Pitch & Binaural listening Review 25 20 15 10 5 0-5 100 1000 10000 25 20 15 10 5 0-5 100 1000 10000 Part I: Auditory frequency selectivity Tuning

More information

Task Difficulty and Performance Induce Diverse Adaptive Patterns in Gain and Shape of Primary Auditory Cortical Receptive Fields

Task Difficulty and Performance Induce Diverse Adaptive Patterns in Gain and Shape of Primary Auditory Cortical Receptive Fields Article Task Difficulty and Performance Induce Diverse Adaptive Patterns in Gain and Shape of Primary Auditory Cortical Receptive Fields Serin Atiani, 1 Mounya Elhilali, 3 Stephen V. David, 2 Jonathan

More information

Neural Correlates of Human Cognitive Function:

Neural Correlates of Human Cognitive Function: Neural Correlates of Human Cognitive Function: A Comparison of Electrophysiological and Other Neuroimaging Approaches Leun J. Otten Institute of Cognitive Neuroscience & Department of Psychology University

More information

Computational Perception /785. Auditory Scene Analysis

Computational Perception /785. Auditory Scene Analysis Computational Perception 15-485/785 Auditory Scene Analysis A framework for auditory scene analysis Auditory scene analysis involves low and high level cues Low level acoustic cues are often result in

More information

The Central Nervous System

The Central Nervous System The Central Nervous System Cellular Basis. Neural Communication. Major Structures. Principles & Methods. Principles of Neural Organization Big Question #1: Representation. How is the external world coded

More information

Precise Spike Timing and Reliability in Neural Encoding of Low-Level Sensory Stimuli and Sequences

Precise Spike Timing and Reliability in Neural Encoding of Low-Level Sensory Stimuli and Sequences Precise Spike Timing and Reliability in Neural Encoding of Low-Level Sensory Stimuli and Sequences Temporal Structure In the World Representation in the Brain Project 1.1.2 Feldman and Harris Labs Temporal

More information

TOWARD A BRAIN INTERFACE FOR TRACKING ATTENDED AUDITORY SOURCES

TOWARD A BRAIN INTERFACE FOR TRACKING ATTENDED AUDITORY SOURCES 216 IEEE INTERNATIONAL WORKSHOP ON MACHINE LEARNING FOR SIGNAL PROCESSING, SEPT. 13 16, 216, SALERNO, ITALY TOWARD A BRAIN INTERFACE FOR TRACKING ATTENDED AUDITORY SOURCES Marzieh Haghighi 1, Mohammad

More information

Infant Hearing Development: Translating Research Findings into Clinical Practice. Auditory Development. Overview

Infant Hearing Development: Translating Research Findings into Clinical Practice. Auditory Development. Overview Infant Hearing Development: Translating Research Findings into Clinical Practice Lori J. Leibold Department of Allied Health Sciences The University of North Carolina at Chapel Hill Auditory Development

More information

Decoding the attended speech stream with multichannel EEG: implications for online, daily-life applications

Decoding the attended speech stream with multichannel EEG: implications for online, daily-life applications Journal of Neural Engineering PAPER OPEN ACCESS Decoding the attended speech stream with multichannel EEG: implications for online, daily-life applications To cite this article: Bojana Mirkovic et al 2015

More information

Early Occurrence Of Auditory Change Detection In The Human Brain

Early Occurrence Of Auditory Change Detection In The Human Brain Early Occurrence Of Auditory Change Detection In The Human Brain SABINE GRIMM University of Leipzig, Germany (formerly University of Barcelona, Spain) ICON, Brisbane, July 28 th, 2014 Auditory deviance

More information

Dynamic, adaptive task-related processing of sound in the auditory and prefrontal cortex

Dynamic, adaptive task-related processing of sound in the auditory and prefrontal cortex Dynamic, adaptive task-related processing of sound in the auditory and prefrontal cortex Jonathan Fritz 1, Stephen David 1,3, Bernhard Englitz 1,4, Serin Atiani 1,5, Diego Elgueda 1, Michael Locastro 1,

More information

Encoding of natural sounds by variance of the cortical local field potential

Encoding of natural sounds by variance of the cortical local field potential J Neurophysiol 115: 2389 2398, 2016. First published February 24, 2016; doi:10.1152/jn.00652.2015. Encoding of natural sounds by variance of the cortical local field potential Nai Ding, 1 Jonathan Z. Simon,

More information

Chapter 40 Effects of Peripheral Tuning on the Auditory Nerve s Representation of Speech Envelope and Temporal Fine Structure Cues

Chapter 40 Effects of Peripheral Tuning on the Auditory Nerve s Representation of Speech Envelope and Temporal Fine Structure Cues Chapter 40 Effects of Peripheral Tuning on the Auditory Nerve s Representation of Speech Envelope and Temporal Fine Structure Cues Rasha A. Ibrahim and Ian C. Bruce Abstract A number of studies have explored

More information

Decisions Have Consequences

Decisions Have Consequences Decisions Have Consequences Scott Makeig Swartz Center for Computational Neuroscience Institute for Neural Computation UCSD, La Jolla CA Precis of talk given at the recent Banbury Center workshop on decision

More information

Nikos Laskaris ENTEP

Nikos Laskaris ENTEP Nikos Laskaris ENTEP Reflections of learning at the University of Patras and opportunities to discover at BSI/RIKEN Understanding begins by elucidating basic brain mechanisms. This area of research is

More information

Neuromagnetic recordings reveal the temporal dynamics of auditory spatial processing in the human cortex

Neuromagnetic recordings reveal the temporal dynamics of auditory spatial processing in the human cortex Neuroscience Letters 396 (2006) 17 22 Neuromagnetic recordings reveal the temporal dynamics of auditory spatial processing in the human cortex Hannu Tiitinen a,b,, Nelli H. Salminen a, Kalle J. Palomäki

More information

Toward Extending Auditory Steady-State Response (ASSR) Testing to Longer-Latency Equivalent Potentials

Toward Extending Auditory Steady-State Response (ASSR) Testing to Longer-Latency Equivalent Potentials Department of History of Art and Architecture Toward Extending Auditory Steady-State Response (ASSR) Testing to Longer-Latency Equivalent Potentials John D. Durrant, PhD, CCC-A, FASHA Department of Communication

More information

Beyond Blind Averaging: Analyzing Event-Related Brain Dynamics. Scott Makeig. sccn.ucsd.edu

Beyond Blind Averaging: Analyzing Event-Related Brain Dynamics. Scott Makeig. sccn.ucsd.edu Beyond Blind Averaging: Analyzing Event-Related Brain Dynamics Scott Makeig Institute for Neural Computation University of California San Diego La Jolla CA sccn.ucsd.edu Talk given at the EEG/MEG course

More information

Introduction to Computational Neuroscience

Introduction to Computational Neuroscience Introduction to Computational Neuroscience Lecture 5: Data analysis II Lesson Title 1 Introduction 2 Structure and Function of the NS 3 Windows to the Brain 4 Data analysis 5 Data analysis II 6 Single

More information

Integration of Visual Information in Auditory Cortex Promotes Auditory Scene Analysis through Multisensory Binding

Integration of Visual Information in Auditory Cortex Promotes Auditory Scene Analysis through Multisensory Binding Article Integration of Visual Information in Auditory Cortex Promotes Auditory Scene Analysis through Multisensory Binding Graphical Abstract Authors Huriye Atilgan, Stephen M. Town, Katherine C. Wood,

More information

Entrainment of neuronal oscillations as a mechanism of attentional selection: intracranial human recordings

Entrainment of neuronal oscillations as a mechanism of attentional selection: intracranial human recordings Entrainment of neuronal oscillations as a mechanism of attentional selection: intracranial human recordings J. Besle, P. Lakatos, C.A. Schevon, R.R. Goodman, G.M. McKhann, A. Mehta, R.G. Emerson, C.E.

More information

Processing and Plasticity in Rodent Somatosensory Cortex

Processing and Plasticity in Rodent Somatosensory Cortex Processing and Plasticity in Rodent Somatosensory Cortex Dan Feldman Dept. of Molecular & Cell Biology Helen Wills Neuroscience Institute UC Berkeley Primary sensory neocortex Secondary and higher areas

More information

Role of F0 differences in source segregation

Role of F0 differences in source segregation Role of F0 differences in source segregation Andrew J. Oxenham Research Laboratory of Electronics, MIT and Harvard-MIT Speech and Hearing Bioscience and Technology Program Rationale Many aspects of segregation

More information

Introduction to Computational Neuroscience

Introduction to Computational Neuroscience Introduction to Computational Neuroscience Lecture 10: Brain-Computer Interfaces Ilya Kuzovkin So Far Stimulus So Far So Far Stimulus What are the neuroimaging techniques you know about? Stimulus So Far

More information

Binaural Hearing. Why two ears? Definitions

Binaural Hearing. Why two ears? Definitions Binaural Hearing Why two ears? Locating sounds in space: acuity is poorer than in vision by up to two orders of magnitude, but extends in all directions. Role in alerting and orienting? Separating sound

More information

Using MEG to map the auditory cortex. Jonathan Côté PhD Candidate in Etienne de Villers-Sidani s laboratory

Using MEG to map the auditory cortex. Jonathan Côté PhD Candidate in Etienne de Villers-Sidani s laboratory Using MEG to map the auditory cortex Jonathan Côté PhD Candidate in Etienne de Villers-Sidani s laboratory Sensory representations organization Sensory representations organization Organized map for most

More information

Auditory Physiology PSY 310 Greg Francis. Lecture 30. Organ of Corti

Auditory Physiology PSY 310 Greg Francis. Lecture 30. Organ of Corti Auditory Physiology PSY 310 Greg Francis Lecture 30 Waves, waves, waves. Organ of Corti Tectorial membrane Sits on top Inner hair cells Outer hair cells The microphone for the brain 1 Hearing Perceptually,

More information

ANALYSIS OF PATTERNS OF NEURAL ACTIVITY IN RESPONSE TO AUDITORY STIMULI DILLON A. HAMBROOK. Bachelor of Science, University of Lethbridge, 2011

ANALYSIS OF PATTERNS OF NEURAL ACTIVITY IN RESPONSE TO AUDITORY STIMULI DILLON A. HAMBROOK. Bachelor of Science, University of Lethbridge, 2011 ANALYSIS OF PATTERNS OF NEURAL ACTIVITY IN RESPONSE TO AUDITORY STIMULI DILLON A. HAMBROOK Bachelor of Science, University of Lethbridge, 2011 A Thesis Submitted to the School of Graduate Studies of the

More information

Modulation and Top-Down Processing in Audition

Modulation and Top-Down Processing in Audition Modulation and Top-Down Processing in Audition Malcolm Slaney 1,2 and Greg Sell 2 1 Yahoo! Research 2 Stanford CCRMA Outline The Non-Linear Cochlea Correlogram Pitch Modulation and Demodulation Information

More information

Effect of informational content of noise on speech representation in the aging midbrain and cortex

Effect of informational content of noise on speech representation in the aging midbrain and cortex J Neurophysiol 116: 2356 2367, 2016. First published September 7, 2016; doi:10.1152/jn.00373.2016. Effect of informational content of noise on speech representation in the aging midbrain and cortex Alessandro

More information

Mechanisms Underlying Selective Neuronal Tracking of Attended Speech at a Cocktail Party

Mechanisms Underlying Selective Neuronal Tracking of Attended Speech at a Cocktail Party Article Mechanisms Underlying Selective Neuronal Tracking of Attended Speech at a Cocktail Party Elana M. Zion Golumbic, 1,4 Nai Ding, 5,6 Stephan Bickel, 7,8 Peter Lakatos, 4 Catherine A. Schevon, 2 Guy

More information

The role of amplitude, phase, and rhythmicity of neural oscillations in top-down control of cognition

The role of amplitude, phase, and rhythmicity of neural oscillations in top-down control of cognition The role of amplitude, phase, and rhythmicity of neural oscillations in top-down control of cognition Chair: Jason Samaha, University of Wisconsin-Madison Co-Chair: Ali Mazaheri, University of Birmingham

More information

Chapter 59 Temporal Coherence and the Streaming of Complex Sounds

Chapter 59 Temporal Coherence and the Streaming of Complex Sounds Chapter 59 Temporal Coherence and the Streaming of Complex Sounds Shihab Shamma, Mounya Elhilali, Ling Ma, Christophe Micheyl, ndrew J. Oxenham, Daniel Pressnitzer, Pingbo Yin, and Yanbo Xu bstract Humans

More information

Report. Direct Recordings of Pitch Responses from Human Auditory Cortex

Report. Direct Recordings of Pitch Responses from Human Auditory Cortex Current Biology 0,, June, 00 ª00 Elsevier Ltd. Open access under CC BY license. DOI 0.0/j.cub.00.0.0 Direct Recordings of Pitch Responses from Human Auditory Cortex Report Timothy D. Griffiths,, * Sukhbinder

More information

Dynamics of Precise Spike Timing in Primary Auditory Cortex

Dynamics of Precise Spike Timing in Primary Auditory Cortex The Journal of Neuroscience, February 4, 2004 24(5):1159 1172 1159 Behavioral/Systems/Cognitive Dynamics of Precise Spike Timing in Primary Auditory Cortex Mounya Elhilali, Jonathan B. Fritz, David J.

More information

EEG reveals divergent paths for speech envelopes during selective attention

EEG reveals divergent paths for speech envelopes during selective attention EEG reveals divergent paths for speech envelopes during selective attention Cort Horton a, Michael D Zmura a, and Ramesh Srinivasan a,b a Dept. of Cognitive Sciences, University of California, Irvine,

More information

EDGE DETECTION. Edge Detectors. ICS 280: Visual Perception

EDGE DETECTION. Edge Detectors. ICS 280: Visual Perception EDGE DETECTION Edge Detectors Slide 2 Convolution & Feature Detection Slide 3 Finds the slope First derivative Direction dependent Need many edge detectors for all orientation Second order derivatives

More information

Hearing the Universal Language: Music and Cochlear Implants

Hearing the Universal Language: Music and Cochlear Implants Hearing the Universal Language: Music and Cochlear Implants Professor Hugh McDermott Deputy Director (Research) The Bionics Institute of Australia, Professorial Fellow The University of Melbourne Overview?

More information

The dynamics of the construction of auditory perceptual representations: MEG studies in humans

The dynamics of the construction of auditory perceptual representations: MEG studies in humans The dynamics of the construction of auditory perceptual representations: MEG studies in humans Maria Chait 1, 3 and Jonathan Z. Simon 2 1 Neuroscience and Cognitive Science Program, University of Maryland,

More information

HCS 7367 Speech Perception

HCS 7367 Speech Perception Long-term spectrum of speech HCS 7367 Speech Perception Connected speech Absolute threshold Males Dr. Peter Assmann Fall 212 Females Long-term spectrum of speech Vowels Males Females 2) Absolute threshold

More information

Left-hemisphere dominance for processing of vowels: a whole-scalp neuromagnetic study

Left-hemisphere dominance for processing of vowels: a whole-scalp neuromagnetic study Auditory and Vestibular Systems 10, 2987±2991 (1999) BRAIN activation of 11 healthy right-handed subjects was studied with magnetoencephalography to estimate individual hemispheric dominance for speech

More information

Power and phase properties of oscillatory neural responses in the presence of background activity

Power and phase properties of oscillatory neural responses in the presence of background activity J Comput Neurosci (2013) 34:337 343 DOI 10.1007/s10827-012-0424-6 Power and phase properties of oscillatory neural responses in the presence of background activity Nai Ding & Jonathan Z. Simon Received:

More information

Hearing II Perceptual Aspects

Hearing II Perceptual Aspects Hearing II Perceptual Aspects Overview of Topics Chapter 6 in Chaudhuri Intensity & Loudness Frequency & Pitch Auditory Space Perception 1 2 Intensity & Loudness Loudness is the subjective perceptual quality

More information

A Consumer-friendly Recap of the HLAA 2018 Research Symposium: Listening in Noise Webinar

A Consumer-friendly Recap of the HLAA 2018 Research Symposium: Listening in Noise Webinar A Consumer-friendly Recap of the HLAA 2018 Research Symposium: Listening in Noise Webinar Perry C. Hanavan, AuD Augustana University Sioux Falls, SD August 15, 2018 Listening in Noise Cocktail Party Problem

More information

The role of periodicity in the perception of masked speech with simulated and real cochlear implants

The role of periodicity in the perception of masked speech with simulated and real cochlear implants The role of periodicity in the perception of masked speech with simulated and real cochlear implants Kurt Steinmetzger and Stuart Rosen UCL Speech, Hearing and Phonetic Sciences Heidelberg, 09. November

More information

Discrimination of temporal fine structure by birds and mammals

Discrimination of temporal fine structure by birds and mammals Auditory Signal Processing: Physiology, Psychoacoustics, and Models. Pressnitzer, D., de Cheveigné, A., McAdams, S.,and Collet, L. (Eds). Springer Verlag, 24. Discrimination of temporal fine structure

More information

CS/NEUR125 Brains, Minds, and Machines. Due: Friday, April 14

CS/NEUR125 Brains, Minds, and Machines. Due: Friday, April 14 CS/NEUR125 Brains, Minds, and Machines Assignment 5: Neural mechanisms of object-based attention Due: Friday, April 14 This Assignment is a guided reading of the 2014 paper, Neural Mechanisms of Object-Based

More information

Cortical sound processing in children with autism disorder: an MEG investigation

Cortical sound processing in children with autism disorder: an MEG investigation AUDITORYAND VESTIBULAR SYSTEMS Cortical sound processing in children with autism disorder: an MEG investigation Nicole M. Gage CA, Bryna Siegel, 1 Melanie Callen 1 and Timothy P. L. Roberts Biomagnetic

More information

Information and neural computations

Information and neural computations Information and neural computations Why quantify information? We may want to know which feature of a spike train is most informative about a particular stimulus feature. We may want to know which feature

More information

Sensory Physiology Bi353 Fall Term 2016

Sensory Physiology Bi353 Fall Term 2016 Lectures 2-3p MWF 111 Lillis (CRN 11012) Lab/Discussion Section 1 (CRN 11013) Friday 10-110:50a Hue 129 Lab/Discussion Section 2 (CRN 11014) Friday 11a-11:50a Hue 129 Lab/Discussion Section 3 (CRN 16400)

More information

A Precluding But Not Ensuring Role of Entrained Low-Frequency Oscillations for Auditory Perception

A Precluding But Not Ensuring Role of Entrained Low-Frequency Oscillations for Auditory Perception 12268 The Journal of Neuroscience, August 29, 2012 32(35):12268 12276 Behavioral/Systems/Cognitive A Precluding But Not Ensuring Role of Entrained Low-Frequency Oscillations for Auditory Perception Benedict

More information

Report. Neural Response Phase Tracks How Listeners Learn New Acoustic Representations

Report. Neural Response Phase Tracks How Listeners Learn New Acoustic Representations Current Biology 23, 1 7, June 3, 2013 ª2013 Elsevier Ltd All rights reserved http://dx.doi.org/10.1016/j.cub.2013.04.031 Neural Response Phase Tracks How Listeners Learn New Acoustic Representations Report

More information

The neural code for interaural time difference in human auditory cortex

The neural code for interaural time difference in human auditory cortex The neural code for interaural time difference in human auditory cortex Nelli H. Salminen and Hannu Tiitinen Department of Biomedical Engineering and Computational Science, Helsinki University of Technology,

More information

EEG Analysis on Brain.fm (Focus)

EEG Analysis on Brain.fm (Focus) EEG Analysis on Brain.fm (Focus) Introduction 17 subjects were tested to measure effects of a Brain.fm focus session on cognition. With 4 additional subjects, we recorded EEG data during baseline and while

More information

Frequency refers to how often something happens. Period refers to the time it takes something to happen.

Frequency refers to how often something happens. Period refers to the time it takes something to happen. Lecture 2 Properties of Waves Frequency and period are distinctly different, yet related, quantities. Frequency refers to how often something happens. Period refers to the time it takes something to happen.

More information

Brain Imaging of auditory function

Brain Imaging of auditory function Who am I? Brain Imaging of auditory function Maria Chait I do (mostly) MEG functional brain imaging of auditory processing. Now, I am at the EAR Institute, University College London & Equipe Audition,

More information

ABSTRACT. Auditory streaming: behavior, physiology, and modeling

ABSTRACT. Auditory streaming: behavior, physiology, and modeling ABSTRACT Title of Document: Auditory streaming: behavior, physiology, and modeling Ling Ma, Doctor of Philosophy, 2011 Directed By: Professor Shihab A. Shamma, Department of Electrical and computer Engineering,

More information

Note: This is a working paper dated 6/20/13, and is not yet peer-reviewed or published

Note: This is a working paper dated 6/20/13, and is not yet peer-reviewed or published Title: Envelope responses in single-trial EEG indicate attended speaker in a cocktail party Abbreviated Title: Envelope responses indicate attended speaker Authors: Cort Horton 1, Ramesh Srinivasan 1,2,

More information

Effects of Cochlear Hearing Loss on the Benefits of Ideal Binary Masking

Effects of Cochlear Hearing Loss on the Benefits of Ideal Binary Masking INTERSPEECH 2016 September 8 12, 2016, San Francisco, USA Effects of Cochlear Hearing Loss on the Benefits of Ideal Binary Masking Vahid Montazeri, Shaikat Hossain, Peter F. Assmann University of Texas

More information

21/01/2013. Binaural Phenomena. Aim. To understand binaural hearing Objectives. Understand the cues used to determine the location of a sound source

21/01/2013. Binaural Phenomena. Aim. To understand binaural hearing Objectives. Understand the cues used to determine the location of a sound source Binaural Phenomena Aim To understand binaural hearing Objectives Understand the cues used to determine the location of a sound source Understand sensitivity to binaural spatial cues, including interaural

More information

But, what about ASSR in AN?? Is it a reliable tool to estimate the auditory thresholds in those category of patients??

But, what about ASSR in AN?? Is it a reliable tool to estimate the auditory thresholds in those category of patients?? 1 Auditory Steady State Response (ASSR) thresholds have been shown to be highly correlated to bh behavioral thresholds h in adults and older children with normal hearing or those with sensorineural hearing

More information

Chapter 11: Sound, The Auditory System, and Pitch Perception

Chapter 11: Sound, The Auditory System, and Pitch Perception Chapter 11: Sound, The Auditory System, and Pitch Perception Overview of Questions What is it that makes sounds high pitched or low pitched? How do sound vibrations inside the ear lead to the perception

More information

The Science. This work was performed by Dr. Psyche Loui at Wesleyan University under NSF-STTR # Please direct inquiries to

The Science. This work was performed by Dr. Psyche Loui at Wesleyan University under NSF-STTR # Please direct inquiries to The Science This work was performed by Dr. Psyche Loui at Wesleyan University under NSF-STTR #1720698. Please direct inquiries to research@brain.fm Summary Brain.fm designs music to help you do things.

More information

Acoustics Research Institute

Acoustics Research Institute Austrian Academy of Sciences Acoustics Research Institute Modeling Modelingof ofauditory AuditoryPerception Perception Bernhard BernhardLaback Labackand andpiotr PiotrMajdak Majdak http://www.kfs.oeaw.ac.at

More information

HCS 7367 Speech Perception

HCS 7367 Speech Perception Babies 'cry in mother's tongue' HCS 7367 Speech Perception Dr. Peter Assmann Fall 212 Babies' cries imitate their mother tongue as early as three days old German researchers say babies begin to pick up

More information

Synaptopathy Research Uwe Andreas Hermann

Synaptopathy Research Uwe Andreas Hermann Potential diagnose and basic understanding of hidden hearing loss Synaptopathy Research Uwe Andreas Hermann Motivation Synaptopathy is a current hot topic in the research field because it focuses on a

More information

The Integration of Features in Visual Awareness : The Binding Problem. By Andrew Laguna, S.J.

The Integration of Features in Visual Awareness : The Binding Problem. By Andrew Laguna, S.J. The Integration of Features in Visual Awareness : The Binding Problem By Andrew Laguna, S.J. Outline I. Introduction II. The Visual System III. What is the Binding Problem? IV. Possible Theoretical Solutions

More information

Atypical processing of prosodic changes in natural speech stimuli in school-age children with Asperger syndrome

Atypical processing of prosodic changes in natural speech stimuli in school-age children with Asperger syndrome Atypical processing of prosodic changes in natural speech stimuli in school-age children with Asperger syndrome Riikka Lindström, PhD student Cognitive Brain Research Unit University of Helsinki 31.8.2012

More information

Auditory gist perception and attention

Auditory gist perception and attention Auditory gist perception and attention Sue Harding Speech and Hearing Research Group University of Sheffield POP Perception On Purpose Since the Sheffield POP meeting: Paper: Auditory gist perception:

More information

Encoding of Temporal Information by Timing, Rate, and Place in Cat Auditory Cortex

Encoding of Temporal Information by Timing, Rate, and Place in Cat Auditory Cortex Encoding of Temporal Information by Timing, Rate, and Place in Cat Auditory Cortex Kazuo Imaizumi 1,2 *, Nicholas J. Priebe 3, Tatyana O. Sharpee 4,5, Steven W. Cheung 1, Christoph E. Schreiner 1,4 * 1

More information

Systems Neuroscience Oct. 16, Auditory system. http:

Systems Neuroscience Oct. 16, Auditory system. http: Systems Neuroscience Oct. 16, 2018 Auditory system http: www.ini.unizh.ch/~kiper/system_neurosci.html The physics of sound Measuring sound intensity We are sensitive to an enormous range of intensities,

More information