NATURAL LANGUAGE PROCESSING Prof. L. Sbattella SPEECH ANALYSIS 3/3 PERCEPTION Dott. Ing. Sonia Cenceschi Sonia.cenceschi@polimi.it
Psychoacoustics and perception Philip G. Zimbardo, Richard J. Gerrig, Psychology and Life, HarperCollins Your perception of the world relies on more than just the information arriving at your sensory receptors. Your ability to transform and interpret sensory information your ability to have what you know interact with what you see allows you to recognize Madonna, Oprah Winfrey, and Bill Clinton Sensation is what gets the show started, but something more is needed to make a stimulus meaningful and interesting and, most important, to make it possible for you to respond to it effectively The processes of perception provide the extra layers of interpretation that enable you to navigate successfully through your environment
Psychoacoustics and perception The role of perception A percept Is to make sense of sensation Is what is perceived (the phenomenological, or experienced outcome of the process of perception) Distal and proximal stimulus (what we perceive and previous used informations)
psychoacoustics and language perception 1. Sensing The conversion of physical energy into neural codes recognized by the brain 2. Organizing An internal representation of an object is formed and a percept of the external stimulus is developed 3. Identifying and recognizing Assign meaning to percepts To identify and recognize involves higher level cognitive processes: theories, memories, values, beliefs, and attitudes concerning the objects
Approaches to the study of perception Nature or Nurture (learn)? Most modern theories agree that your experience of the world consists of a combination of nature and nurture. But these theories disagree on the size of the portions that make up this combination Helmholts Classical Theory (1866) The Gestalt Approach: Koffka-1935, Köhler-1947, Wertheimer-1923 The Gibson s Ecological Optics (1966, 1079) Your focus of attention determines the types of information that will be most readily available to your perceptual processes What types of environmental stimuli require your attention How attention functions to selectively highlight objects and events
A Unified Theory of Perception What are the physiological mechanism involved in perception? Stimulus-driven, or bottom-up processing, works its way up the brain, while expectation-driven, or top-down processing, complements it. What is the process of perceiving? Central role of Gestalt Theories and new approaches and conceptual problem solving What are the properties of the physical world that allow you to perceive? Gibson s theory the environment makes available certain types of information and your perceptual apparatus is innately prepared to recover that information
psychoacoustics and language perception SELECTIVE ATTENTION First your internal representations of the stimuli on which you have focused attention become highlighted in memory Second your internal representations of the unattended stimuli are somewhat suppressed PREATTENTIVE PROCESSING It operates on sensory inputs before you attend to them, as they first come into the brain from the sensory receptors. Guided Search serial / parallel search putting features together
Language perception Sound qualities of the voice are extremely complex because they are related to physical perception and psychoacoustics phenomena BASILAR MEMBRANE (memory effect) audio vibrations start nerve impulses fibers react to different frequencies in different areas, transmitting the oscillation to the Cortis organ Cortis organ A series of ciliated cells and nerve fibers transmit the sound potential to the brain (electricity) 8
Language perception The nerve pulses go from the Cortis organ to the auditory cortex through the acoustic nerve LEFT EMISPHERE LANGUAGE ELABORATION The Broca Area interprets the nerve impulses The Wernicke Area is more directly connected to the understanding of meanings RIGHT EMISPHERE MUSIC ELABORATION But it s important in tonal languages (like chinese) where intonation is phonologically relevant 9
psychoacoustics and language perception THE SHEPARD TONE at the end of Echoes : Pink Floyd The Shepard tone is an auditory illusion consisting of a superposition of sine waves separated by octaves.
Language perception à EXAMPLE OF FORENSICS TRANSCRIPTIONS The brain completes the meaning basing on previous knowledge Cognitive theories Top-down speech processing Expectation and linguistic knowledge set the frame Incoming words are compared to hypotheses Bottom-up processing Acoustic signal is transferred to words Message formed from words you do not always have to believe what you feel
Links and contacts MUSIC INFORMATION RETRIEVAL http://recherche.ircam.fr/equipes/analysesynthese/peeters/articles/peeters_2003_cuidadoaudiofeat ures.pdf http://oldsite.clsp.jhu.edu/ws2000/presentations/preliminary/victor_z ue/zue-lecture2.pdf PRAAT http://www.fon.hum.uva.nl/praat/ izotope RX https://www.izotope.com/en/products/audio-repair/rx/... SONIA CENCESCHI sonia.cenceschi@polimi.it 12
References Beckam M. N., Edwards J., Articulatory evidence for differentiating stress categories, Phonological Structure and Phonetic Form Papers in Laboratory Phonology III, Cambridge University Press, 1994, 7-33. Pierrehumbert, J.M., The phonology and phonetics of English intonation. Ph.D. dissertation, MIT, Indiana University Linguistic Club, 1987. Veilleux, N., Shaeuck-Hufnagel, S., and A. Brugos. Transcribing Prosodic Structure of Spoken UHerances with ToBI, IAP 2006. (MIT OpenCourseWare), hep://ocw.mit.edu (Accessed). License: ISICS, 4/9/2012Crea.veCommons BY-NC-SA hhp://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-911- transcribingprosodic-structure-of-spoken-uherances-with-tobi-january-iap-2006/ Steinhauer, K., Alter, K., & Friederici, A. D. (1999). Brain potentials indicate immediate use of prosodic cues in natural speech processing. Nature neuroscience, 2(2), 191-196. Hruska, C., Alter, K. (2004). Prosody in dialogues and single sentences: How prosody can influence speech perception. In A. Steube (Ed.), Information structure: Theoretical and empirical aspects (pp. 221-226). Berlin: De Gruyter. Baumann, S., & Schumacher, P.B. (2012), (De-)Accentuation and the Processing of Information Status: Evidence from Event-Related Brain Potentials. Language and Speech, 55(3), 361-381. Bo gels, S., Schriefers, H., Vonk, W., & Chwilla, D. J. (2011a). Pitch accents in context: How listeners process accentuation in referential communication. Neuropsychologia, 49(7), 2022-2036.... SPEECH PERCEPTION, PROSODY AND INTONATION POLITECNICO DI MILANO