A new path to understanding vision
|
|
- Corey Fisher
- 5 years ago
- Views:
Transcription
1 A new path to understanding vision from the perspective of the primary visual cortex Frontal brain areas Visual cortices Primary visual cortex (V1) Li Zhaoping Retina
2 A new path to understanding vision Traditional paths to understanding vision (1) Low level vision, mid-level vision, high-level vision (2) David Marr: primal sketch, 2.5 d sketch, 3-d model. Li Zhaoping July 16, 2018, presented at APCV 2018, HangZhou, China
3 Talk outline (1) The functional role of the primary visual cortex (V1) (2) In light of V1 s role a new plan to understanding vision (3) A first example study in this new plan
4 1953, Stephen Kuffler Center-surround feature detectors in retina
5 1960s-70s, Hubel and Wiesel Bar/edge feature detectors in V1 1953, Stephen Kuffler Center-surround feature detectors in retina
6 Some standard models of neural encoding in retina/v1: From Fig. 1 of Carandini et al 2005 Explaining: Retinal receptive fields V1 properties: Orientation tuning, Color tuning Motion direction tuning Spatial frequency tuning Ocular dominance (eye-of-origin tuning) Simple/Complex cells Explaining 80% of the response variance Explaining only 35% of the response variance!! Plus: models of gain control, response normalization by population activities
7 Do we know early vision? What is V1 doing? Do we really know what the early visual system does? Carandini, Demb, Mante, Tolhurst, Dan, Olshausen, Gallant, Rust, 2005 Standard models of neurons in the retina account for 80% of the response variance. However, standard models of V1 capture only 35% of the V1 variances. How close are we to understand V1? Olshausen and Field Standard models combining linear filtering, rectification and squaring, and response Is V1 normalization doing sparse still miss coding 86% of the (more variances efficient in V1 responses. encoding than in retina)? Olshausen and Field 1996: Emergence of simple-cell receptive field properties by learning a sparse coding for natural images Bell and Sejnowski 1997: The independent components of natural scenes are edge filters. Simoncelli and Olshausen 2001: Natural image statistics and neural representation.
8 Do we know early vision? What is V1 doing? Do we really know what the early visual system does? Carandini, Demb, Mante, Tolhurst, Dan, Olshausen, Gallant, Rust, 2005 Standard models of neurons in the retina account for 80% of the response variance. However, standard models of V1 capture only 35% of the V1 variances. How close are we to understand V1? Olshausen and Field Standard models combining linear filtering, rectification and squaring, and response Is V1 normalization doing sparse still miss coding 86% of the (more variances efficient in V1 responses. encoding than in retina)? Other than stereo vision, why not make encoding more efficient in retina already? V1 has ~100 as many cells as retina (Barlow 1981), quite inefficient/redundant! What about contextual influences to neural responses??!!!!
9 The primary visual cortex (V1) 1953, Stephen Kuffler, retina, Hubel and Wiesel, V1 Then Experimentally: V1 and beyond Theoretical/modelling, Reichardt, Marr, etc. 2005: How close are we to understand V1? Olshausen and Field 2005 Do we really know what the early visual system does? Carandini, Demb, Mante, Tolhurst, Dan, Olshausen, Gallant, Rust, 2005 Standard models of V1 neural receptive field combining linear filtering, rectification and squaring, and response normalization captures only 15-35% of the variances in V1 responses. 2012, David Hubel, in answer to What Do You Feel Are the Next Big Questions in the Field? We have some idea for the retina, the lateral geniculate body, and the primary visual cortex, but that s about it. (Hubel & Wiesel 2012, Neuron)
10 The primary visual cortex (V1) 1953, Stephen Kuffler, retina, Hubel and Wiesel, V1 Then Experimentally: V1 and beyond Theoretical/modelling, Reichardt, Marr, etc. 2005: How close are we to understand V1? Olshausen and Field 2005 Do we really know what the early visual system does? Carandini, Demb, Mante, Tolhurst, Dan, Olshausen, Gallant, Rust, 2005 Standard models of V1 neural receptive field combining linear filtering, rectification and squaring, and response normalization captures only 15-35% of the variances in V1 responses. 2012, David Hubel, in answer to What Do You Feel Are the Next Big Questions in the Field? We have some idea for the retina, the lateral geniculate body, and the primary visual cortex, but that s about it. (Hubel & Wiesel 2012, Neuron) Questions: Is a lack of understanding of V1 hindering our progress beyond V1? Physiologically Functionally (in behaviour)?
11 Information bottlenecks in the visual pathway: 10 7 bits/second ~ 10 6 neurons, ~10 spikes/neuron ~1 bit/spike To be or not to be, This is the question.. 40 bits/second (Sziklai, 1956) 10 9 bits/second (Kelly 1962) ~ 25 frames/second, 2000x2000 pixels, 1 byte/pixel
12 Information bottlenecks in the visual pathway: We are nearly blind! Vision ~ Looking (selecting ) + Seeing top-down vs. bottom-up selection 10 7 bits/second ~ 10 6 neurons, ~10 spikes/neuron ~1 bit/spike To be or not to be, This is the question.. 40 bits/second (Sziklai, 1956) Task: find a uniquely oriented bar 10 9 bits/second (Kelly 1962) ~ 25 frames/second, 2000x2000 pixels, 1 byte/pixel
13 Information bottlenecks in the visual pathway: Questions: which brain areas are doing the selection? Frontal? Parietal? top-down vs. bottom-up selection Task: find a uniquely oriented bar 10 9 bits/second (Kelly 1962) ~ 25 frames/second, 2000x2000 pixels, 1 byte/pixel 10 7 bits/second ~ 10 6 neurons, ~10 spikes/neuron ~1 bit/spike To be or not to be, This is the question.. 40 bits/second (Sziklai, 1956)
14 Information bottlenecks in the visual pathway: Questions: which brain areas are doing the selection? Saliency regardless of visual features Frontal? Parietal? Koch & Ullman 1985, Itti & Koch 2001, etc top-down vs. bottom-up selection 10 7 bits/second ~ 10 6 neurons, ~10 spikes/neuron ~1 bit/spike To be or not to be, This is the question.. 40 bits/second (Sziklai, 1956) Task: find a uniquely oriented bar 10 9 bits/second (Kelly 1962) ~ 25 frames/second, 2000x2000 pixels, 1 byte/pixel
15 Eye Extrastriate cortices V 1 LGN Parietal cortex Thalamus Superior colliculus Brain stem, etc to control eye muscle Frontal eye field In lower mammals, a smaller percentage of the total cortex is for prefrontal areas (Fuster 2002) Non-mammals do not have a neocortex Forebrain of fish is more for olfaction than vision (Butler & Hodos 2005) 11.5% 17% 3.5 % 12.5% 29%
16 The V1 Saliency Hypothesis: A bottom-up saliency map in the primary visual cortex (Li 1999, 2002) Retina inputs Saliency map V1 firing rates (highest at each location) = Winner-take-all Superior colliculus
17 The V1 Saliency Hypothesis: A bottom-up saliency map in the primary visual cortex (Li 1999, 2002) Retina inputs V1 firing rates (highest at each location) Winner-take-all Superior colliculus
18 The V1 Saliency Hypothesis: Neural activities as A bottom-up saliency map in the primary visual cortex (Li universal 1999, 2002) currency to bid for visual selection. Retina inputs V1 firing rates (highest at each location) Winner-take-all Superior colliculus
19 The V1 Saliency Hypothesis: A bottom-up saliency map in the primary visual cortex (Li 1999, 2002) Retina inputs V1 V1 firing rates (highest at each location) Winner-take-all Superior colliculus Bosking et al 1997 iso-feature suppression (Blakemore & Tobin 1972, Gilbert & Wiesel 1983, Rockland & Lund 1983, Allman et al 1985, Hirsch & Gilbert 1991, Li & Li 1994, etc)
20 The V1 Saliency Hypothesis: A bottom-up saliency map in the primary visual cortex (Li 1999, 2002) Retina inputs V1 V1 firing rates (highest at each location) Winner-take-all Iso-orientation suppression Iso-color suppression Bosking et al 1997 Bosking et al 1997 iso-feature suppression (Blakemore & Tobin 1972, Gilbert & Wiesel 1983, Rockland & Lund 1983, Allman et al 1985, Hirsch & Gilbert 1991, Li & Li 1994, etc Wachtler et al 2003 Jones et al 2001 ) Superior colliculus Iso-motion (direction) suppression
21 The V1 Saliency Hypothesis: A bottom-up saliency map in the primary visual cortex (Li 1999, 2002) Retina inputs V1 V1 firing rates (highest at each location) Winner-take-all Superior colliculus Bosking et al 1997 Left eye image Right eye image A surprising prediction: an invisible feature attract attention! (Zhaoping 2008, 2012) Escapes iso-orientation suppression Fused perception Escapes iso-eye-of-origin suppression Looking without seeing! Replicated by multiple research groups since! A fingerprint of V1
22 Look for saliency signals in the brain without the awareness/top-down confound Collaboration with Fang Fang (Beijing University), and his student Xilin Zhang (Neuron, 2012) Salient orientation pop-out Quickly masked to prevent awareness (verified) Vernier task probe, may be at cued location or the other hemi-field
23 Look for saliency signals in the brain without the awareness/top-down confound Collaboration with Fang Fang (Beijing University), and his student Xilin Zhang (VSS 2011) Cueing effect Now measure which brain areas are activated by this invisible cue? By fmri, ERP V1, but not IPS, FEF!! i.e.,saliency effect without awareness Orientation contrast at cue
24 Testing the V1 theory on behaving monkeys --- Yan, Zhaoping, & Li, Submitted. Saccade to an uniquely oriented bar ASAP V1 neural responses to input stimulus (spikes/sec) for trials with faster saccades Receptive field for trials with slower or failed saccades Time since visual input onset (second) Quantitative, zero-parameter, predictions from theory Imply that higher visual areas are not involved Solid curve ---Planck s law Squares: --- data points Koene & Zhaoping 2007 Zhaoping & Zhe 2015,
25 Talk outline (1) The functional role of the primary visual cortex (V1) Attentional selection (2) In light of V1 s role a new plan to understanding vision (3) A first example study in this new plan
26 Information bottlenecks in the visual pathway: 10 7 bits/second ~ 10 6 neurons, ~10 spikes/neuron ~1 bit/spike Bottom-up selection To be or not to be, This is the question.. 40 bits/second (Sziklai, 1956) 10 9 bits/second (Kelly 1962) ~ 25 frames/second, 2000x2000 pixels, 1 byte/pixel
27 A new path to understanding vision 20 frames, 20 megabytes/second 40 bits/second Traditional paths to understanding vision (1) Low level vision, mid-level vision, high-level vision To be or not to be, This is the question.. 40 bits/second (Sziklai, 1956) (2) David Marr: primal sketch, 2.5 d sketch, 3-d model bits/second (Kelly 1962) ~ 25 frames/second, 2000x2000 pixels, 1 byte/pixel Bottom-up selection
28 Talk outline (1) The functional role of the primary visual cortex (V1) Attentional selection (2) In light of V1 s role a new plan to understanding vision (3) A first example study in this new plan
29 A new path to understanding vision 20 frames, 20 megabytes/second Looking periphery 40 bits/second Seeing center
30 A new path to understanding vision 20 frames, 20 megabytes/second Looking periphery 40 bits/second Seeing center
31 Looking and seeing --- peripheral and central Two separate processes Demo: (Zhaoping & Guyader 2007) V1: tuned to primitive bars IT & higher areas: X shape recognition, rotationally invariant Add a horizontal bar or a vertical bar to each oblique bar
32 Looking and seeing --- peripheral and central Two separate processes Demo: (Zhaoping & Guyader 2007) Add a horizontal bar or a vertical bar to each oblique bar Add a horizontal bar or a vertical bar to each oblique bar
33 Display span 46x32 degrees in visual angle --- condition A
34 Gaze arrives at target after a few saccades Looking --- mainly by the bottom up saliency of the unique orientation. Then...
35 Gaze dawdled around the target, then abandoned and returned. Seeing
36 A new path to understanding vision 20 frames, 20 megabytes/second Looking 40 bits/second Seeing Peripheral vision Must qualitatively differ in Seeing Central vision Computational algorithms? Focus on feedforwardfeedback / V1 A demo of crowding in the periphery
37 Summation percept dominant Perception =? In V1, signals are efficiently encoded by these two de-correlated channels (Li & Atick 1994) Ocular sum Ocular diff Subject task: report the perceived tilt. C = C Left eye Right eye Zhaoping 2017
38 Why does perception prefer ocular summation? (Zhaoping 2017) Higher areas for analysis-by-synthesis (c.f. predictive coding) Feedforward, feedback, verify, and re-weight (FFVW) If I perceive it, it is likely (prior) shown to both my left and my right eyes, so it should resemble the input in the sum channel!!! The Bayesian(?) monster V1 Feedback to verify Ocular sum Ocular diff Retina Left eye Right eye
39 Why does perception prefer ocular summation? (Zhaoping 2017) Higher areas for analysis-by-synthesis (c.f. predictive coding) Feedforward, feedback, verify, and re-weight (FFVW) If I perceive it, it is likely (prior) shown to both my left and my right eyes, so it should resemble the input in the sum channel!!! The Bayesian(?) monster V1 Feedback to verify Ocular sum Ocular diff Not because periphery cannot see tilt! Retina Left eye Right eye Proposal: Top-down feedback to V1 is weaker or absent in peripheral vision for analysis-by-synthesis!
40 Why does perception prefer ocular summation? (Zhaoping 2017) Higher areas for analysis-by-synthesis (c.f. predictive coding) Feedforward, feedback, verify, and re-weight (FFVW) If I perceive it, it is likely (prior) shown to both my left and my right eyes, so it should resemble the input in the sum channel!!! The Bayesian(?) monster V1 Feedback to verify Ocular sum Ocular diff Retina Left eye Right eye Proposal: Top-down feedback to V1 is weaker or absent in peripheral vision for analysis-by-synthesis!
41 A central disk (non-zero disparity) and a surrounding ring (zero disparity) Testing it in depth perception Zhaoping & Ackermann, 2018 Dots for the central disk correlated A V1 neuron s response to disparity in random dot stereograms Anticorrelated Correlated Dots for the central disk Anti-correlated Disparity (degrees) (Cumming and Parker 1997 ) Proposal: Top-down feedback to V1 is weaker or absent in peripheral vision for analysis-by-synthesis!
42 Proposal: Top-down feedback to V1 is weaker or absent in peripheral vision for analysis-by-synthesis! Dots for the central disk correlated A central disk (non-zero disparity) and a surrounding ring (zero disparity) Testing it in depth perception Zhaoping & Ackermann, 2018 No reversed depth percept V1 s report vetoed prior expects correlated inputs between the eyes Dots for the central disk Anti-correlated Top-down feedback to verify V1 s report V1 feeds forward reverse depth to higher brain areas!
43 Proposal: Top-down feedback to V1 is weaker or absent in peripheral vision for analysis-by-synthesis! Dots for the central disk correlated A central disk (non-zero disparity) and a surrounding ring (zero disparity) Testing it in depth perception Zhaoping & Ackermann, 2018 No reversed depth percept If peripheral vision has no feedback Dots for the central disk Anti-correlated V1 feeds forward reverse depth to higher brain areas!
44 Proposal: Top-down feedback to V1 is weaker or absent in peripheral vision for analysis-by-synthesis! A central disk (non-zero disparity) and a surrounding ring (zero disparity) Testing it in depth perception Zhaoping & Ackermann, 2018 Dots for the central disk correlated Accuracy reporting disparity-defined depth Dots for the central disk Anti-correlated
45 Thanks to Funding source: Gatsby Foundation, EPSRC, BBSRC. Collaborators: Wu Li, Yin Yan, Ansgar Koene, Li Zhe, Joealle Ackermann, etc A new path to understanding vision Falsifiable 20 frames, 20 megabytes/second 40 bits/second Opening the window to our brain Theory motivate inspire Experiments
Pre-Attentive Visual Selection
Pre-Attentive Visual Selection Li Zhaoping a, Peter Dayan b a University College London, Dept. of Psychology, UK b University College London, Gatsby Computational Neuroscience Unit, UK Correspondence to
More informationA bottom up visual saliency map in the primary visual cortex --- theory and its experimental tests.
A bottom up visual saliency map in the primary visual cortex --- theory and its experimental tests. Li Zhaoping University College London Adapted and updated from the invited presentation at COSYNE (computational
More informationLateral Geniculate Nucleus (LGN)
Lateral Geniculate Nucleus (LGN) What happens beyond the retina? What happens in Lateral Geniculate Nucleus (LGN)- 90% flow Visual cortex Information Flow Superior colliculus 10% flow Slide 2 Information
More information2/3/17. Visual System I. I. Eye, color space, adaptation II. Receptive fields and lateral inhibition III. Thalamus and primary visual cortex
1 Visual System I I. Eye, color space, adaptation II. Receptive fields and lateral inhibition III. Thalamus and primary visual cortex 2 1 2/3/17 Window of the Soul 3 Information Flow: From Photoreceptors
More informationM Cells. Why parallel pathways? P Cells. Where from the retina? Cortical visual processing. Announcements. Main visual pathway from retina to V1
Announcements exam 1 this Thursday! review session: Wednesday, 5:00-6:30pm, Meliora 203 Bryce s office hours: Wednesday, 3:30-5:30pm, Gleason https://www.youtube.com/watch?v=zdw7pvgz0um M Cells M cells
More informationProperties of V1 neurons tuned to conjunctions of visual features: application of the V1 saliency hypothesis to visual search behavior
Properties of 1 neurons tuned to conjunctions of visual features: application of the 1 saliency hypothesis to visual search behavior Li Zhaoping 1,2, Li Zhe 2 Published 2012 in PLoS ONE 7(6): e36223. doi:10.1371/journal.pone.0036223
More informationReading Assignments: Lecture 5: Introduction to Vision. None. Brain Theory and Artificial Intelligence
Brain Theory and Artificial Intelligence Lecture 5:. Reading Assignments: None 1 Projection 2 Projection 3 Convention: Visual Angle Rather than reporting two numbers (size of object and distance to observer),
More informationEarly Stages of Vision Might Explain Data to Information Transformation
Early Stages of Vision Might Explain Data to Information Transformation Baran Çürüklü Department of Computer Science and Engineering Mälardalen University Västerås S-721 23, Sweden Abstract. In this paper
More informationComputational Models of Visual Attention: Bottom-Up and Top-Down. By: Soheil Borhani
Computational Models of Visual Attention: Bottom-Up and Top-Down By: Soheil Borhani Neural Mechanisms for Visual Attention 1. Visual information enter the primary visual cortex via lateral geniculate nucleus
More informationCompeting Frameworks in Perception
Competing Frameworks in Perception Lesson II: Perception module 08 Perception.08. 1 Views on perception Perception as a cascade of information processing stages From sensation to percept Template vs. feature
More informationCompeting Frameworks in Perception
Competing Frameworks in Perception Lesson II: Perception module 08 Perception.08. 1 Views on perception Perception as a cascade of information processing stages From sensation to percept Template vs. feature
More informationTHE ENCODING OF PARTS AND WHOLES
THE ENCODING OF PARTS AND WHOLES IN THE VISUAL CORTICAL HIERARCHY JOHAN WAGEMANS LABORATORY OF EXPERIMENTAL PSYCHOLOGY UNIVERSITY OF LEUVEN, BELGIUM DIPARTIMENTO DI PSICOLOGIA, UNIVERSITÀ DI MILANO-BICOCCA,
More informationEDGE DETECTION. Edge Detectors. ICS 280: Visual Perception
EDGE DETECTION Edge Detectors Slide 2 Convolution & Feature Detection Slide 3 Finds the slope First derivative Direction dependent Need many edge detectors for all orientation Second order derivatives
More information(Visual) Attention. October 3, PSY Visual Attention 1
(Visual) Attention Perception and awareness of a visual object seems to involve attending to the object. Do we have to attend to an object to perceive it? Some tasks seem to proceed with little or no attention
More informationComputational Cognitive Science
Computational Cognitive Science Lecture 15: Visual Attention Chris Lucas (Slides adapted from Frank Keller s) School of Informatics University of Edinburgh clucas2@inf.ed.ac.uk 14 November 2017 1 / 28
More informationComputational Cognitive Science. The Visual Processing Pipeline. The Visual Processing Pipeline. Lecture 15: Visual Attention.
Lecture 15: Visual Attention School of Informatics University of Edinburgh keller@inf.ed.ac.uk November 11, 2016 1 2 3 Reading: Itti et al. (1998). 1 2 When we view an image, we actually see this: The
More informationV1 (Chap 3, part II) Lecture 8. Jonathan Pillow Sensation & Perception (PSY 345 / NEU 325) Princeton University, Fall 2017
V1 (Chap 3, part II) Lecture 8 Jonathan Pillow Sensation & Perception (PSY 345 / NEU 325) Princeton University, Fall 2017 Topography: mapping of objects in space onto the visual cortex contralateral representation
More informationPsychophysical tests of the hypothesis of a bottom-up saliency map in primary visual cortex
Psychophysical tests of the hypothesis of a bottom-up saliency map in primary visual cortex In press for Public Library of Science, Computational Biology, (2007). Citation: Zhaoping L. May KA (2007) Psychophysical
More informationProf. Greg Francis 7/31/15
s PSY 200 Greg Francis Lecture 06 How do you recognize your grandmother? Action potential With enough excitatory input, a cell produces an action potential that sends a signal down its axon to other cells
More informationModels of Attention. Models of Attention
Models of Models of predictive: can we predict eye movements (bottom up attention)? [L. Itti and coll] pop out and saliency? [Z. Li] Readings: Maunsell & Cook, the role of attention in visual processing,
More informationVisual Selection and Attention
Visual Selection and Attention Retrieve Information Select what to observe No time to focus on every object Overt Selections Performed by eye movements Covert Selections Performed by visual attention 2
More informationMorton-Style Factorial Coding of Color in Primary Visual Cortex
Morton-Style Factorial Coding of Color in Primary Visual Cortex Javier R. Movellan Institute for Neural Computation University of California San Diego La Jolla, CA 92093-0515 movellan@inc.ucsd.edu Thomas
More informationCognitive Modelling Themes in Neural Computation. Tom Hartley
Cognitive Modelling Themes in Neural Computation Tom Hartley t.hartley@psychology.york.ac.uk Typical Model Neuron x i w ij x j =f(σw ij x j ) w jk x k McCulloch & Pitts (1943), Rosenblatt (1957) Net input:
More informationVision II. Steven McLoon Department of Neuroscience University of Minnesota
Vision II Steven McLoon Department of Neuroscience University of Minnesota 1 Ganglion Cells The axons of the retinal ganglion cells form the optic nerve and carry visual information into the brain. 2 Optic
More informationNonlinear processing in LGN neurons
Nonlinear processing in LGN neurons Vincent Bonin *, Valerio Mante and Matteo Carandini Smith-Kettlewell Eye Research Institute 2318 Fillmore Street San Francisco, CA 94115, USA Institute of Neuroinformatics
More information196 IEEE TRANSACTIONS ON AUTONOMOUS MENTAL DEVELOPMENT, VOL. 2, NO. 3, SEPTEMBER 2010
196 IEEE TRANSACTIONS ON AUTONOMOUS MENTAL DEVELOPMENT, VOL. 2, NO. 3, SEPTEMBER 2010 Top Down Gaze Movement Control in Target Search Using Population Cell Coding of Visual Context Jun Miao, Member, IEEE,
More informationNeuroscience Tutorial
Neuroscience Tutorial Brain Organization : cortex, basal ganglia, limbic lobe : thalamus, hypothal., pituitary gland : medulla oblongata, midbrain, pons, cerebellum Cortical Organization Cortical Organization
More informationPrimary Visual Pathways (I)
Primary Visual Pathways (I) Introduction to Computational and Biological Vision CS 202-1-5261 Computer Science Department, BGU Ohad Ben-Shahar Where does visual information go from the eye? Where does
More informationVision Seeing is in the mind
1 Vision Seeing is in the mind Stimulus: Light 2 Light Characteristics 1. Wavelength (hue) 2. Intensity (brightness) 3. Saturation (purity) 3 4 Hue (color): dimension of color determined by wavelength
More informationKey questions about attention
Key questions about attention How does attention affect behavioral performance? Can attention affect the appearance of things? How does spatial and feature-based attention affect neuronal responses in
More informationAttention: Neural Mechanisms and Attentional Control Networks Attention 2
Attention: Neural Mechanisms and Attentional Control Networks Attention 2 Hillyard(1973) Dichotic Listening Task N1 component enhanced for attended stimuli Supports early selection Effects of Voluntary
More informationOPTO 5320 VISION SCIENCE I
OPTO 5320 VISION SCIENCE I Monocular Sensory Processes of Vision: Color Vision Mechanisms of Color Processing . Neural Mechanisms of Color Processing A. Parallel processing - M- & P- pathways B. Second
More informationMeasuring the attentional effect of the bottom-up saliency map of natural images
Measuring the attentional effect of the bottom-up saliency map of natural images Cheng Chen 1,3, Xilin Zhang 2,3, Yizhou Wang 1,3, and Fang Fang 2,3,4,5 1 National Engineering Lab for Video Technology
More informationContextual Influences in Visual Processing
C Contextual Influences in Visual Processing TAI SING LEE Computer Science Department and Center for Neural Basis of Cognition, Carnegie Mellon University, Pittsburgh, PA, USA Synonyms Surround influence;
More informationPhotoreceptors Rods. Cones
Photoreceptors Rods Cones 120 000 000 Dim light Prefer wavelength of 505 nm Monochromatic Mainly in periphery of the eye 6 000 000 More light Different spectral sensitivities!long-wave receptors (558 nm)
More informationPlasticity of Cerebral Cortex in Development
Plasticity of Cerebral Cortex in Development Jessica R. Newton and Mriganka Sur Department of Brain & Cognitive Sciences Picower Center for Learning & Memory Massachusetts Institute of Technology Cambridge,
More informationObject recognition and hierarchical computation
Object recognition and hierarchical computation Challenges in object recognition. Fukushima s Neocognitron View-based representations of objects Poggio s HMAX Forward and Feedback in visual hierarchy Hierarchical
More informationLISC-322 Neuroscience Cortical Organization
LISC-322 Neuroscience Cortical Organization THE VISUAL SYSTEM Higher Visual Processing Martin Paré Assistant Professor Physiology & Psychology Most of the cortex that covers the cerebral hemispheres is
More informationParallel streams of visual processing
Parallel streams of visual processing RETINAL GANGLION CELL AXONS: OPTIC TRACT Optic nerve Optic tract Optic chiasm Lateral geniculate nucleus Hypothalamus: regulation of circadian rhythms Pretectum: reflex
More informationObject Perception Perceiving and Recognizing Objects
Object Perception Perceiving and Recognizing Objects Extrastriate cortex Outside V1 in occipital lobe Dorsal pathway Ventral pathway Modular organization of visual areas associated with object recognition
More informationCan Saliency Map Models Predict Human Egocentric Visual Attention?
Can Saliency Map Models Predict Human Egocentric Visual Attention? Kentaro Yamada 1, Yusuke Sugano 1, Takahiro Okabe 1 Yoichi Sato 1, Akihiro Sugimoto 2, and Kazuo Hiraki 3 1 The University of Tokyo, Tokyo,
More informationA theory of a saliency map in primary visual cortex (V1) tested by psychophysics of color-orientation interference in texture segmentation
A theory of a saliency map in primary visual cortex (V1) tested by psychophysics of color-orientation interference in texture segmentation Published in Visual Cognition, 2006, 14 (4/5/6/7/8), 911-933.
More informationSenses are transducers. Change one form of energy into another Light, sound, pressure, etc. into What?
1 Vision 2 TRANSDUCTION Senses are transducers Change one form of energy into another Light, sound, pressure, etc. into What? Action potentials! Sensory codes Frequency code encodes information about intensity
More informationAssociative Decorrelation Dynamics: A Theory of Self-Organization and Optimization in Feedback Networks
Associative Decorrelation Dynamics: A Theory of Self-Organization and Optimization in Feedback Networks Dawei W. Dong* Lawrence Berkeley Laboratory University of California Berkeley, CA 94720 Abstract
More informationJust One View: Invariances in Inferotemporal Cell Tuning
Just One View: Invariances in Inferotemporal Cell Tuning Maximilian Riesenhuber Tomaso Poggio Center for Biological and Computational Learning and Department of Brain and Cognitive Sciences Massachusetts
More informationIncreasing Spatial Competition Enhances Visual Prediction Learning
(2011). In A. Cangelosi, J. Triesch, I. Fasel, K. Rohlfing, F. Nori, P.-Y. Oudeyer, M. Schlesinger, Y. and Nagai (Eds.), Proceedings of the First Joint IEEE Conference on Development and Learning and on
More informationIntroduction to Computational Neuroscience
Introduction to Computational Neuroscience Lecture 11: Attention & Decision making Lesson Title 1 Introduction 2 Structure and Function of the NS 3 Windows to the Brain 4 Data analysis 5 Data analysis
More informationusing deep learning models to understand visual cortex
using deep learning models to understand visual cortex 11-785 Introduction to Deep Learning Fall 2017 Michael Tarr Department of Psychology Center for the Neural Basis of Cognition this lecture A bit out
More informationSelective Attention. Inattentional blindness [demo] Cocktail party phenomenon William James definition
Selective Attention Inattentional blindness [demo] Cocktail party phenomenon William James definition Everyone knows what attention is. It is the taking possession of the mind, in clear and vivid form,
More informationAdventures into terra incognita
BEWARE: These are preliminary notes. In the future, they will become part of a textbook on Visual Object Recognition. Chapter VI. Adventures into terra incognita In primary visual cortex there are neurons
More informationGoals. Visual behavior jobs of vision. The visual system: overview of a large-scale neural architecture. kersten.org
The visual system: overview of a large-scale neural architecture Daniel Kersten Computational Vision Lab Psychology Department, U. Minnesota Goals Provide an overview of a major brain subsystem to help
More informationHow does the ventral pathway contribute to spatial attention and the planning of eye movements?
R. P. Würtz and M. Lappe (Eds.) Dynamic Perception, Infix Verlag, St. Augustin, 83-88, 2002. How does the ventral pathway contribute to spatial attention and the planning of eye movements? Fred H. Hamker
More informationNatural Scene Statistics and Perception. W.S. Geisler
Natural Scene Statistics and Perception W.S. Geisler Some Important Visual Tasks Identification of objects and materials Navigation through the environment Estimation of motion trajectories and speeds
More informationThe Visual System. Cortical Architecture Casagrande February 23, 2004
The Visual System Cortical Architecture Casagrande February 23, 2004 Phone: 343-4538 Email: vivien.casagrande@mcmail.vanderbilt.edu Office: T2302 MCN Required Reading Adler s Physiology of the Eye Chapters
More informationCOGS 101A: Sensation and Perception
COGS 101A: Sensation and Perception 1 Virginia R. de Sa Department of Cognitive Science UCSD Lecture 5: LGN and V1: Magno and Parvo streams Chapter 3 Course Information 2 Class web page: http://cogsci.ucsd.edu/
More informationA saliency map in primary visual cortex
Opinion A saliency map in primary visual cortex Zhaoping Li I propose that pre-attentive computational mechanisms in primary visual cortex create a saliency map. This map awards higher responses to more
More informationOn the implementation of Visual Attention Architectures
On the implementation of Visual Attention Architectures KONSTANTINOS RAPANTZIKOS AND NICOLAS TSAPATSOULIS DEPARTMENT OF ELECTRICAL AND COMPUTER ENGINEERING NATIONAL TECHNICAL UNIVERSITY OF ATHENS 9, IROON
More informationlateral organization: maps
lateral organization Lateral organization & computation cont d Why the organization? The level of abstraction? Keep similar features together for feedforward integration. Lateral computations to group
More informationSignificance of Natural Scene Statistics in Understanding the Anisotropies of Perceptual Filling-in at the Blind Spot
Significance of Natural Scene Statistics in Understanding the Anisotropies of Perceptual Filling-in at the Blind Spot Rajani Raman* and Sandip Sarkar Saha Institute of Nuclear Physics, HBNI, 1/AF, Bidhannagar,
More informationAttention Response Functions: Characterizing Brain Areas Using fmri Activation during Parametric Variations of Attentional Load
Attention Response Functions: Characterizing Brain Areas Using fmri Activation during Parametric Variations of Attentional Load Intro Examine attention response functions Compare an attention-demanding
More information- 1 - Published in Journal of Experimental Psychology: human perception and performance, Vol 37(4), Aug 2011, doi: 10.
- 1 - Published in Journal of Experimental Psychology: human perception and performance, Vol 37(4), Aug 2011, 997-1006. doi: 10.1037/a0023099 A clash of bottom-up and top-down processes in visual search:
More informationA Neurally-Inspired Model for Detecting and Localizing Simple Motion Patterns in Image Sequences
A Neurally-Inspired Model for Detecting and Localizing Simple Motion Patterns in Image Sequences Marc Pomplun 1, Yueju Liu 2, Julio Martinez-Trujillo 2, Evgueni Simine 2, and John K. Tsotsos 2 1 Department
More informationPerception & Attention. Perception is the best understood cognitive function, esp in terms of underlying biological substrates.
Perception & Attention Perception is the best understood cognitive function, esp in terms of underlying biological substrates. Perception & Attention Perception is the best understood cognitive function,
More informationIntroduction to sensory pathways. Gatsby / SWC induction week 25 September 2017
Introduction to sensory pathways Gatsby / SWC induction week 25 September 2017 Studying sensory systems: inputs and needs Stimulus Modality Robots Sensors Biological Sensors Outputs Light Vision Photodiodes
More informationCS294-6 (Fall 2004) Recognizing People, Objects and Actions Lecture: January 27, 2004 Human Visual System
CS294-6 (Fall 2004) Recognizing People, Objects and Actions Lecture: January 27, 2004 Human Visual System Lecturer: Jitendra Malik Scribe: Ryan White (Slide: layout of the brain) Facts about the brain:
More informationFundamentals of Cognitive Psychology, 3e by Ronald T. Kellogg Chapter 2. Multiple Choice
Multiple Choice 1. Which structure is not part of the visual pathway in the brain? a. occipital lobe b. optic chiasm c. lateral geniculate nucleus *d. frontal lobe Answer location: Visual Pathways 2. Which
More informationPHY3111 Mid-Semester Test Study. Lecture 2: The hierarchical organisation of vision
PHY3111 Mid-Semester Test Study Lecture 2: The hierarchical organisation of vision 1. Explain what a hierarchically organised neural system is, in terms of physiological response properties of its neurones.
More information1. The responses of on-center and off-center retinal ganglion cells
1. The responses of on-center and off-center retinal ganglion cells 2. Responses of an on-center ganglion cell to different light conditions 3. Responses of an on-center ganglion cells to different light
More information9.14 Classes #21-23: Visual systems
9.14 Classes #21-23: Visual systems Questions based on Schneider chapter 20 and classes: 1) What was in all likelihood the first functional role of the visual sense? Describe the nature of the most primitive
More informationFundamentals of psychophysics
Fundamentals of psychophysics Steven Dakin UCL Institute of Ophthalmology Introduction To understand a system as complex as the brain, one must understand not only its components (i.e. physiology) and
More informationObject detection in natural scenes by feedback
In: H.H. Būlthoff et al. (eds.), Biologically Motivated Computer Vision. Lecture Notes in Computer Science. Berlin, Heidelberg, New York: Springer Verlag, 398-407, 2002. c Springer-Verlag Object detection
More informationNeural circuits PSY 310 Greg Francis. Lecture 05. Rods and cones
Neural circuits PSY 310 Greg Francis Lecture 05 Why do you need bright light to read? Rods and cones Photoreceptors are not evenly distributed across the retina 1 Rods and cones Cones are most dense in
More informationVisual system invades the endbrain: pathways to striatum and cortex (continued) Why this happened in evolution
Visual system invades the endbrain: pathways to striatum and cortex (continued) Why this happened in evolution What were the adaptive advantages? Visual information reaching the striatum directly: Advantages
More informationDepartment of Computer Science, University College London, London WC1E 6BT, UK;
vision Article Ocularity Feature Contrast Attracts Attention Exogenously Li Zhaoping Department of Computer Science, University College London, London WCE 6BT, UK; z.li@ucl.ac.uk Received: 7 December 27;
More informationCompound Effects of Top-down and Bottom-up Influences on Visual Attention During Action Recognition
Compound Effects of Top-down and Bottom-up Influences on Visual Attention During Action Recognition Bassam Khadhouri and Yiannis Demiris Department of Electrical and Electronic Engineering Imperial College
More informationConsciousness The final frontier!
Consciousness The final frontier! How to Define it??? awareness perception - automatic and controlled memory - implicit and explicit ability to tell us about experiencing it attention. And the bottleneck
More informationWhat is mid level vision? Mid Level Vision. What is mid level vision? Lightness perception as revealed by lightness illusions
What is mid level vision? Mid Level Vision March 18, 2004 Josh McDermott Perception involves inferring the structure of the world from measurements of energy generated by the world (in vision, this is
More informationVIDEO SALIENCY INCORPORATING SPATIOTEMPORAL CUES AND UNCERTAINTY WEIGHTING
VIDEO SALIENCY INCORPORATING SPATIOTEMPORAL CUES AND UNCERTAINTY WEIGHTING Yuming Fang, Zhou Wang 2, Weisi Lin School of Computer Engineering, Nanyang Technological University, Singapore 2 Department of
More informationSensation & Perception The Visual System. Subjectivity of Perception. Sensation vs. Perception 1/9/11
Sensation & Perception The Visual System Subjectivity of Perception We do not perceive the world directly Perception depends on brain stimulation Distinction between sensation and perception Sensation
More informationPerception & Attention
Perception & Attention Perception is effortless but its underlying mechanisms are incredibly sophisticated. Biology of the visual system Representations in primary visual cortex and Hebbian learning Object
More informationModeling the Deployment of Spatial Attention
17 Chapter 3 Modeling the Deployment of Spatial Attention 3.1 Introduction When looking at a complex scene, our visual system is confronted with a large amount of visual information that needs to be broken
More informationObject vision (Chapter 4)
Object vision (Chapter 4) Lecture 8 Jonathan Pillow Sensation & Perception (PSY 345 / NEU 325) Princeton University, Spring 2015 1 Outline for today: Chap 3: adaptation Chap 4: intro to object vision gestalt
More informationPsychophysical Tests of the Hypothesis of a Bottom-Up Saliency Map in Primary Visual Cortex
Psychophysical Tests of the Hypothesis of a Bottom-Up Saliency Map in Primary Visual Cortex Li Zhaoping *, Keith A. May Department of Psychology, University College London, London, United Kingdom A unique
More informationNetworks and Hierarchical Processing: Object Recognition in Human and Computer Vision
Networks and Hierarchical Processing: Object Recognition in Human and Computer Vision Guest&Lecture:&Marius&Cătălin&Iordan&& CS&131&8&Computer&Vision:&Foundations&and&Applications& 01&December&2014 1.
More informationExtrastriate Visual Areas February 27, 2003 A. Roe
Extrastriate Visual Areas February 27, 2003 A. Roe How many extrastriate areas are there? LOTS!!! Macaque monkey flattened cortex Why? How do we know this? Topography Functional properties Connections
More informationPerceptual Grouping in a Self-Organizing Map of Spiking Neurons
Perceptual Grouping in a Self-Organizing Map of Spiking Neurons Yoonsuck Choe Department of Computer Sciences The University of Texas at Austin August 13, 2001 Perceptual Grouping Group Two! Longest Contour?
More informationCognitive Neuroscience Attention
Cognitive Neuroscience Attention There are many aspects to attention. It can be controlled. It can be focused on a particular sensory modality or item. It can be divided. It can set a perceptual system.
More informationDynamic Visual Attention: Searching for coding length increments
Dynamic Visual Attention: Searching for coding length increments Xiaodi Hou 1,2 and Liqing Zhang 1 1 Department of Computer Science and Engineering, Shanghai Jiao Tong University No. 8 Dongchuan Road,
More informationPERCEPTION AND ACTION
PERCEPTION AND ACTION Visual Perception Ecological Approach to Perception J. J. Gibson in 1929 Traditional experiments too constrained Subjects cannot move their heads Study of snapshot vision Perception
More informationSensation and Perception. A. Sensation: awareness of simple characteristics B. Perception: making complex interpretations
I. Overview Sensation and Perception A. Sensation: awareness of simple characteristics B. Perception: making complex interpretations C. Top-Down vs Bottom-up Processing D. Psychophysics -- thresholds 1.
More informationSpatial Attention: Unilateral Neglect
Spatial Attention: Unilateral Neglect attended location (70-90 ms after target onset). Self portrait, copying, line bisection tasks: In all cases, patients with parietal/temporal lesions seem to forget
More informationPsych 333, Winter 2008, Instructor Boynton, Exam 2
Name: ID # ID: A Psych 333, Winter 2008, Instructor Boynton, Exam 2 Multiple Choice (38 questions, 1 point each) Identify the letter of the choice that best completes the statement or answers the question.
More informationID# Exam 1 PS 325, Fall 2003
ID# Exam 1 PS 325, Fall 2003 Read each question carefully and answer it completely. Pay careful attention to the point value of questions so that you allocate your time appropriately (1 point = 1 minute).
More informationLighta part of the spectrum of Electromagnetic Energy. (the part that s visible to us!)
Introduction to Physiological Psychology Vision ksweeney@cogsci.ucsd.edu cogsci.ucsd.edu/~ /~ksweeney/psy260.html Lighta part of the spectrum of Electromagnetic Energy (the part that s visible to us!)
More informationOverview of the visual cortex. Ventral pathway. Overview of the visual cortex
Overview of the visual cortex Two streams: Ventral What : V1,V2, V4, IT, form recognition and object representation Dorsal Where : V1,V2, MT, MST, LIP, VIP, 7a: motion, location, control of eyes and arms
More informationContextual influences in V1 as a basis for pop out and asymmetry in visual search
Proc. Natl. Acad. Sci. USA Vol. 96, pp. 10530 10535, August 1999 Psychology Contextual influences in V1 as a basis for pop out and asymmetry in visual search ZHAOPING LI* Gatsby Computational Neuroscience
More informationSpectrograms (revisited)
Spectrograms (revisited) We begin the lecture by reviewing the units of spectrograms, which I had only glossed over when I covered spectrograms at the end of lecture 19. We then relate the blocks of a
More informationCN530, Spring 2004 Final Report Independent Component Analysis and Receptive Fields. Madhusudana Shashanka
CN530, Spring 2004 Final Report Independent Component Analysis and Receptive Fields Madhusudana Shashanka shashanka@cns.bu.edu 1 Abstract This report surveys the literature that uses Independent Component
More informationThe Integration of Features in Visual Awareness : The Binding Problem. By Andrew Laguna, S.J.
The Integration of Features in Visual Awareness : The Binding Problem By Andrew Laguna, S.J. Outline I. Introduction II. The Visual System III. What is the Binding Problem? IV. Possible Theoretical Solutions
More informationLocal Disparity Not Perceived Depth Is Signaled by Binocular Neurons in Cortical Area V1 of the Macaque
The Journal of Neuroscience, June 15, 2000, 20(12):4758 4767 Local Disparity Not Perceived Depth Is Signaled by Binocular Neurons in Cortical Area V1 of the Macaque Bruce G. Cumming and Andrew J. Parker
More information